Stream Worker Basics

Stream workers are complex event processors that allow you to process streams of events and query them in real time.

If you are new to stream workers, this section explains in basic terms what stream workers are, how they work, and what the required and optional parts of a stream worker are.

If you haven't already, check out Getting Started with Stream Workers for a quick introduction.

Unique Name Requirements

All the elements such as streams, tables, triggers, functions, window, sink, sources, query workers, and indexes must be defined with unique names.

For example, If you want to create two separate indexes with the same name in for different tables, the stream workers will not allow it, informing you that there’s another index element with the same name.

Best Practices

Best practice is to keep stream worker functionality limited to one business use case per stream worker. Additionally, stream workers can use shared sinks and sources to reduce code duplication and improve maintainability.

📄️ How Stream Workers Work

Stream workers are complex event processors. They consume events, process them, and then send the data somewhere. This page explains that process more thoroughly, but still at a high level.

📄️ Stream Worker Syntax

This page explains the most basic syntax of stream workers and provides examples.

📄️ Stream Worker Elements

The page below provides a brief description of all the potential parts of a stream worker. Items marked as required are necessary components for every stream worker. For example, every stream worker needs a source, but not all stream workers need an aggregation.

📄️ Stream Worker Order

Stream worker components must be listed in a certain order in order to be valid.

📄️ Stream Processors

Typically the first step in a stream processing flow is to consume the data to be cleaned, enriched, transformed or summarized to produce the required output.