Introduction

Data Streams is a Catalyst service that enables you to stream real-time, event-driven data from a publisher to its subscribers through a channel. It follows a publish-subscribe (pub/sub) model, where publishers send data to a channel and subscribers receive the data from that channel in real time over WebSocket connections.

Unlike traditional HTTP requests, WebSocket connections remain persistently open once established rather than creating a new connection for every request-response cycle. This enables data to be pushed automatically to all connected subscribers as new events occur, eliminating the need for repeated server polling.

Developers can utilize Data Streams to build real-time features such as live dashboards, chat applications, notification systems, and more without implementing custom messaging infrastructure.

You can create and manage multiple channels from the Catalyst console based on your application needs. Once a channel is created, publishers can send data to the channel using the Catalyst SDK (Java, JavaScript, Python), REST API, or Signals, and subscribers can connect to the same channel over a WebSocket connection to receive the published data in real time.

Components of Data Streams

Publisher

A Publisher, also referred to as a data producer, is any application, service, or component that sends data to a Data Streams channel. Publishers can publish data to a channel using the SDK or API offerings. Multiple publishers can send data to the same channel.

Catalyst services, such as Serverless Functions and AppSail, or any other third-party services that produce live data can be considered as Publishers. Signals, a Catalyst service, can also be a data producer because it can collect events from Zoho and other external services and stream them to a channel.

Learn more about different publishing methods.

Channel

Channel are the core communication layer in Data Streams. They receive data from publishers and distribute it to all subscribers connected to that channel. Channels can be created in the console, and each channel is identified by its channel name and unique channel ID. These attributes are used by both publishers and subscribers to publish or receive streamed data.

Learn more about Channels and their configurations.

Subscriber

A subscriber is a service that receives streamed data from a Data Streams channel through a WebSocket connection. There can be multiple subscribers subscribed to a single channel, and each subscriber will receive the published data in real-time.

To establish a WebSocket connection, a subscriber requires a token pair for authentication. Once connected, you need to choose a subscriber type to determine the data it needs to receive, such as only live events, all available events from the earliest point, or a resumption from where the subscriber previously left off. Each message received by a subscriber must be acknowledged before the next message is delivered.

Subscribers will continue to receive data as long as they remain connected to the channel. If a subscriber disconnects, it can reconnect and resume receiving data from its last acknowledged position by using the appropriate subscriber type.

Data Streams Architecture

Data Streams is made up of Publishers (data producers) that send data to a channel, and subscribers that receive the data from that channel in real time. The channel acts as a persistent communication layer that receives the published data and distributes it to all subscribers connected to it.

Publishers can be applications, back-end services, or external systems that generate events such as:

• CRM updates on Leads or Deals
• Changes made to the Contact record
• Live chat messages
• Notifications regarding payment statuses
• Updates to order statuses
• Updates to sports scores

In the Catalyst console under Test Data Flow, you will be able to view a preview of the published data.

Subscribers are clients or servers that consume this streamed data. These can include:

• Mobile or web clients
• Backend services
• Notification systems
• Analytics dashboards
• IoT devices
• Microservices

The following diagram illustrates Data Streams’ underlying architecture:

This architecture enables low-latency event streaming without requiring clients to constantly poll the server for updates.

Catalyst offers REST APIs support, and SDKs for Data Streams in the following runtimes:

• Java SDK
• JavaScript SDK
• Python SDK