Domain Event Stream

Overview

The KaireonAI Domain Event Stream publishes lightweight notifications whenever the platform records an interaction or makes a recommendation. Subscribers (analytics workers, ML feature pipelines, external sinks) can consume these events without polling the OLTP database. This stream is separate from the POST /api/v1/events ingestion endpoint, which accepts inbound customer behavior events.

Topics

Topic	When fired	Carries
`interaction.recorded.v1`	Every successful interaction history insert (impressions, clicks, conversions, complaints, unsubscribes)	tenantId, customerId, offerId, creativeId, channelId, interactionType, outcomeTypeKey, campaignId, recommendationId, timestamp
`decision.made.v1`	Every successful `POST /api/v1/recommend` decision	tenantId, customerId, recommendationId, offersReturned, topOfferId, topPropensity, decisionFlowKey, timestamp

Event envelope

Every event is published as JSON with this shape:

{
  "type": "interaction.recorded.v1",
  "timestamp": "2026-04-28T12:00:00.000Z",
  "payload": {
    "tenantId": "...",
    "customerId": "...",
    "offerId": "...",
    "creativeId": "...",
    "channelId": "...",
    "interactionType": "click",
    "outcomeTypeKey": "click",
    "campaignId": null,
    "recommendationId": "...",
    "timestamp": "2026-04-28T12:00:00.000Z"
  }
}

Choosing a backend

The publisher backend is selected via the EVENT_PUBLISHER environment variable. Pick the trade-off that fits your operational story — durability, cost, ops weight all vary.

`EVENT_PUBLISHER`	Backend	Durability	When to use
`redis` (default)	Redis pub/sub	None — fire-and-forget; consumers must be online	Local dev with Upstash or `docker compose up redis`; playground; small self-hosted setups where the OLTP interaction history table is enough
`kafka` / `redpanda`	Kafka API (kafkajs)	Replayable with consumer groups + offsets	Local dev with `docker compose --profile kafka up redpanda`; self-hosted Kafka or Redpanda; Confluent Cloud; Aiven
`msk` / `aws_msk`	Amazon MSK (IAM-auth Kafka)	Replayable	AWS-native deployments using Managed Streaming for Apache Kafka
`eventbridge`	AWS EventBridge	Pay-per-event, durable, fan-out to many AWS targets	When downstream consumers are AWS Lambda / Step Functions / SQS
`kinesis`	AWS Kinesis Data Streams	Replayable, 24-hr to 365-day retention	When downstream consumers want streaming-shard semantics

Switch backends without code changes — set EVENT_PUBLISHER and the per-backend env vars below. Topic names + event payloads stay identical, so subscribers don’t need to be rewritten.

Local dev with Redpanda (free Kafka API)

Redpanda is a Kafka-compatible broker that runs as a single binary, no Zookeeper, no JVM. It is included as an optional Docker Compose profile:

# Start the broker (~5 seconds)
docker compose --profile kafka up -d redpanda

# Point the platform at it
export EVENT_PUBLISHER=kafka
export KAFKA_BROKERS=localhost:19092   # host port (in-cluster clients use redpanda:9092)
npm run dev

For Redpanda Cloud Serverless (free tier survives indefinitely for low-traffic):

export EVENT_PUBLISHER=kafka
export KAFKA_BROKERS=your-cluster.redpanda.com:9092
export KAFKA_TLS_ENABLED=true
export KAFKA_SASL_MECHANISM=scram-sha-256
export KAFKA_SASL_USERNAME=<from-redpanda-cloud-console>
export KAFKA_SASL_PASSWORD=<from-redpanda-cloud-console>

Durability — what each backend gives you

redis: none. Consumers must be online when an event fires. If no subscriber is connected to a topic, the event is silently dropped.
kafka / redpanda / msk: events are persisted to the broker. Consumers can replay from any offset within the configured retention window (default Kafka retention is 7 days). Multiple consumer groups read independently.
eventbridge: durable per-event delivery to subscribed targets. No replay (use SQS DLQ for failures).
kinesis: durable shards with 24-hour to 365-day replay window.

In all backends, the publish call is fail-open: if the broker is unreachable or required env vars are missing, the platform logs a warning and continues. The OLTP source of truth remains the interaction history table — query it directly if you need a replayable record older than your stream’s retention window.

Configuration reference

Common

Env var	Required	Notes
`EVENT_PUBLISHER`	No	One of `redis` (default) / `kafka` / `redpanda` / `msk` / `aws_msk` / `eventbridge` / `kinesis`

Redis backend (`EVENT_PUBLISHER=redis`)

Env var	Required	Notes
`REDIS_URL`	When `EVENT_PUBLISHER=redis`	If unset on the redis backend, event stream is disabled; platform keeps functioning

Kafka / Redpanda backend (`EVENT_PUBLISHER=kafka` or `redpanda`)

Env var	Required	Notes
`KAFKA_BROKERS`	Yes	Comma-separated `host:port` list
`KAFKA_CLIENT_ID`	No	Defaults to `kaireon-platform`
`KAFKA_TLS_ENABLED`	No	`true` to enable TLS
`KAFKA_SASL_MECHANISM`	No	`plain`, `scram-sha-256`, or `scram-sha-512`
`KAFKA_SASL_USERNAME` / `KAFKA_SASL_PASSWORD`	When SASL enabled	Auth credentials
`KAFKA_CONSUMER_GROUP_ID`	No	Defaults to `kaireon-consumers`

MSK backend (`EVENT_PUBLISHER=msk`)

Env var	Required	Notes
`MSK_BROKERS`	Yes	Comma-separated bootstrap brokers
`MSK_REGION`	No	Defaults to `us-east-1`
`MSK_AUTH_MODE`	No	`iam_role` (default) or `sasl_scram`
`MSK_ROLE_ARN`	When IAM mode	IAM role for sigv4
`MSK_SASL_USERNAME` / `MSK_SASL_PASSWORD`	When SASL mode	—
`MSK_CONSUMER_GROUP_ID`	No	—
`MSK_TOPIC_PREFIX`	No	Optional prefix for all topics

(Equivalent env-var sets exist for eventbridge and kinesis — see the platform infrastructure container module for the full list.)

Sample subscriber (ioredis)

import Redis from "ioredis";

const sub = new Redis(process.env.REDIS_URL!);

await sub.subscribe("interaction.recorded.v1");
await sub.subscribe("decision.made.v1");

sub.on("message", (channel, raw) => {
  const event = JSON.parse(raw);
  switch (channel) {
    case "interaction.recorded.v1":
      console.log("interaction:", event.payload);
      break;
    case "decision.made.v1":
      console.log("decision:", event.payload);
      break;
  }
});

Where it’s wired

Surface	What it does
Domain-event publisher	Singleton wired via the platform infrastructure container; selects a backend from `EVENT_PUBLISHER`.
Redis backend	Publish over Redis pub/sub. No broker-side retention — see durability below.
Interaction store	Inserts the row, then publishes `interaction.recorded.v1` inside the same transaction.
`/api/v1/respond` route	Hot-path entrypoint that drives the interaction store on every outcome.

What happens when a publish fails

Every backend rejects on a failed publish — it logs the transport error and rethrows. Failure tolerance is the caller’s decision, not the transport’s, and the two callers make opposite choices:

Outbox events (outbox_events, drained by the outbox publisher) are the durable path. A rejection drives the retry/backoff ladder and, once maxRetries is exhausted, moves the event to dead_letter_events. Nothing is marked published unless the transport accepted it.
Stream events (interaction.recorded.v1, decision.made.v1) and the workers’ *.completed notifications are best-effort: they are emitted after the durable write has already committed, so a failure is logged and swallowed at the call site. Letting it propagate would fail a job that already succeeded and cause the work to be re-run.

Before 2026-08-15 every backend swallowed publish failures internally. The outbox therefore marked undelivered events published and dropped them, and its retry/DLQ branch was unreachable on all five backends. If you are running an older build, treat kaireon_outbox_processed_total{status="published"} from that period as “handed to the transport”, not “delivered”.

Rejection on publish is about handing the event to the broker. It is separate from durability once accepted: Redis pub/sub still retains nothing, so a consumer that is offline still misses events. See the backend table above.

Get Started

Tutorials

Use Cases

Decisioning

Studio

Data Pipelines

AI & ML

Operations & Reporting

Governance & Security

Integrations

Reference

Domain Event Stream

Overview

Topics

Event envelope

Choosing a backend

Local dev with Redpanda (free Kafka API)

Durability — what each backend gives you

Configuration reference

Common

Redis backend (`EVENT_PUBLISHER=redis`)

Kafka / Redpanda backend (`EVENT_PUBLISHER=kafka` or `redpanda`)

MSK backend (`EVENT_PUBLISHER=msk`)

Sample subscriber (ioredis)

Where it’s wired

What happens when a publish fails

​Overview

​Topics

​Event envelope

​Choosing a backend

​Local dev with Redpanda (free Kafka API)

​Durability — what each backend gives you

​Configuration reference

​Common

​Redis backend (EVENT_PUBLISHER=redis)

​Kafka / Redpanda backend (EVENT_PUBLISHER=kafka or redpanda)

​MSK backend (EVENT_PUBLISHER=msk)

​Sample subscriber (ioredis)

​Where it’s wired

​What happens when a publish fails

Overview

Topics

Event envelope

Choosing a backend

Local dev with Redpanda (free Kafka API)

Durability — what each backend gives you

Configuration reference

Common

Redis backend (`EVENT_PUBLISHER=redis`)

Kafka / Redpanda backend (`EVENT_PUBLISHER=kafka` or `redpanda`)

MSK backend (`EVENT_PUBLISHER=msk`)

Sample subscriber (ioredis)

Where it’s wired

What happens when a publish fails