Internals

Documentation Index

Fetch the complete documentation index at: https://docs.kaireonai.com/llms.txt

Use this file to discover all available pages before exploring further.

The platform is a single Next.js application whose business logic is grouped by responsibility: infrastructure backends, domain logic, runtime executors, and types. This page is the index for contributors and operators — read it first to find the right subsystem, then dive into the source.

​

What it indexes

The internal library code is organised across roughly 50 subdirectories plus a handful of root-level utilities. The sections below describe what lives in each subsystem, the public surface other code depends on, and the supported extension point for adding new functionality. This page is scoped to the 14 subsystems that did not have an architecture page before — five priority ones (infrastructure, GitOps, connector executors, AI, Flow) and nine secondaries (audit forwarding, governance, scoring, arbitration, fairness, explanations, ML governance, negotiation, qualification). Subsystems already covered by their own page (e.g., the user-facing slice of Flow, or the dashboards layer) are linked from Related at the bottom.

​

Quick start

Three navigation patterns cover most contributor questions:

1. “Which backend handles X?” — The infrastructure container is the single source of truth. Every infrastructure choice (cache, event publisher, interaction store, search index) is selected from one environment variable per concern. Adding a new Kafka or DynamoDB backend means adding one switch case.
2. “What’s the public API of module Y?” — Each module exposes a single barrel export per subsystem, listed in the Reference section below.
3. “How does runtime X load executors / handlers?” — Modules with a runtime keep their per-node executors in a registry that the interpreter walks. Today only Flow follows this pattern.

For root-level utility files, consult the generated codemap at the repository root — it lists every file with its top-line purpose.

​

How it works

The codebase follows a consistent layering pattern: API routes never call backends directly — they always go through the infrastructure container’s getters. This keeps the per-module isolation that makes parallel development possible (see the Module Isolation table in the repository’s contributor guide).

​

Reference

​

Infrastructure

The dependency-injection container. Every infrastructure backend (cache, event publisher, interaction store, search index) is selected behind a single getter that reads one environment variable per concern to choose the implementation. Business logic imports the getter, never the concrete class.

• Public surface: getters for the cache, event publisher, interaction store, and search index, plus a process-shutdown hook that the worker process calls on SIGTERM.
• Backends shipped: Redis (cache plus event pub/sub), PostgreSQL (interactions plus full-text search), Kafka / Redpanda / MSK / EventBridge / Kinesis (events), ScyllaDB / DynamoDB / Keyspaces (interactions), and OpenSearch (search). Each backend is one self-contained module.
• Public interfaces: the cache, event publisher, interaction store, search index, and their domain-event payload shapes are exported as TypeScript interfaces. Business logic imports these only.
• Extension point: add a new event-publisher backend by writing a module that implements the event-publisher interface, then add a switch branch in the container so a single environment variable selects it. Set that variable in deployment and the rest of the platform picks it up with no other code changes. The same pattern applies to the interaction store and search index.

​

GitOps

Apply-and-diff for tenant configuration as YAML. A bundle (a set of Categories, Channels, Offers, Decision Flows, Qualification Rules, Contact Policies, Arbitration Profiles) is parsed from disk, diffed against the tenant’s database, and either dry-run-reported or written. The reconciliation key is the resource name, not the database id, so the same bundle applies cleanly across dev / stage / prod with different ids per environment.

• Public surface: apply, diff, parse-from-YAML, parse-from-JSON, and tenant-export operations, plus the resource-bundle and result types.
• Resource model: generic resource and bundle wrappers, plus a per-kind diff and apply-result shape. The API version is kaireonai.com/v1. Per-kind validation is Zod-based and pinned per resource kind.
• Drift and three-way merge: a periodic reconciler compares live database state against a baseline snapshot, then either reports drift or performs a three-way merge between the baseline, the desired bundle, and the current state.
• Extension point: add a new resource kind by adding the literal to the resource-kind union, registering a Zod spec schema for it, and adding the corresponding apply branch. Default behavior for unrecognized kinds is “validated but reported as unchanged”.

​

Connector executors

The output side of the connector layer. Every supported destination (Salesforce, Iterable, Klaviyo, Shopify, Snowpipe, Slack, Twilio, and so on) ships an executor here. This layer is distinct from the connector registry under the domain layer — the registry declares the connector types and their auth fields; the executors are the runtime that calls those endpoints. Routes look up an executor by connector type to get a per-type send / pull / test handle.

• Public surface: a lookup function (connector type → executor) and a set of “implemented connector types” exposed for the UI to gray out coming-soon entries.
• Categories shipped: Commerce (Shopify, Stripe, Mailchimp), CDP (Intercom, Zendesk, Iterable, Klaviyo, Segment, Braze), Messaging (Slack, Teams, WhatsApp, PagerDuty, Twilio SMS), Analytics (Amplitude, Mixpanel, PostHog), Marketing automation (Customer.io, MoEngage, CleverTap, HubSpot Marketing, ActiveCampaign), Reverse ETL (Snowpipe, Fivetran, Hightouch, Census, Databricks, Kinesis).
• Contract: every executor declares its connector type and exposes a send operation, an optional pull operation, and a test-connection operation. Standard helpers (HTTP fetch with retries, basic-auth, bearer-auth) are provided.
• Coming-soon stub: any registered-but-unimplemented connector type returns a structured 503 at first dispatch — the loud-fail replacement for the previous silent-undefined behavior. Add the executor before promoting the connector in the registry.
• Extension point: add a new connector by exporting an executor from the appropriate category file (or a new file under the connector-executors layer), then adding it to the implemented-types set so the UI surfaces it. Also add the type to the registry under the domain layer so the UI renders the auth form.

​

AI

The AI authoring and analysis layer. Three responsibilities:

1. Provider routing — pick a language-model client from environment configuration.
2. Tool registries — the per-module tool catalogues that the in-app AI panel and the MCP server expose.
3. Document import — the V1 PDF / PPTX → typed-proposals pipeline.

• Public surface: a configuration loader (sync and async), a language-model factory, the AI-provider enum, the AI-config Zod schema, and the environment-variable name map.
• Provider routing: the language-model factory returns a client for one of google | anthropic | openai | ollama | lm_studio | bedrock. The local-dev default is Ollama at 127.0.0.1:11434 with model qwen2.5:3b.
• Tool registries: eleven module-scoped tool sets are re-exported from a single barrel — Data, Studio, Algorithms, Dashboards, Content, CMS, Metrics, Mutations, Docs, Intelligence, and Import. Each tool is a typed definition with a Zod-validated input schema and a handler.
• ML routing: an internal router decides whether an analysis request goes to the LLM or to a heuristic fallback. This is used by the analyzers (segmentation, policy, content, rule parser, transform parser).
• Document import: the V1 PDF / PPTX → typed-proposals pipeline lives under the AI layer’s import sub-area: an extraction orchestrator, a hybrid extractor, an apply / revert orchestrator, and a token-budget cost guard. Storage is pluggable — local filesystem or S3 — gated by a single environment variable read at startup.
• Extension point: add a new AI tool by adding an entry to the corresponding module’s tool array. Add a new LLM provider by adding a case to the factory and to the provider Zod schema. Add a new attachment-storage backend by implementing the blob-store interface and registering it in the storage factory.

​

Flow

The data and decisioning fabric — a typed Pipeline IR, an in-process batch interpreter, an AI authoring layer, an MCP server, atomic file ingestion, six target load modes, hook execution, dataset and row-level validators, YAML connectors, row-level lineage, and configurable UI pages. The user-facing surface is documented at Flow Overview; this section covers the internal layout.

• Sub-areas:
  • IR. A pipeline parser that returns a typed parse result, plus the canonical Pipeline Zod schema and a barrel that re-exports node, transform, source, and target types.
  • Runtime. The in-process batch interpreter walks a registry of per-node executors. The default registry is the canonical example.
  • Connectors. A singleton connector registry plus a plugin SDK for third-party connector authors.
  • Repo. Pipeline-IR persistence with versioning (save, get-latest, get-version, list-versions) plus a runtime feature-flag store.
  • AI authoring. Coordinator, patch apply, patch schema, system prompt, plus PII-redaction and audit guardrails.
  • Observability. Row-level lineage and run-level metrics.
  • Schedule. Cron / interval / rrule resolution plus a “due pipelines” selector for the cron tick.
• Runtime sub-areas: cloud-store source backends (S3 / GCS / Azure / SFTP / HTTP-pull), per-node executors registered into the default registry, the typed transform op set, six target write strategies (including CDC-mirror), dataset validators, the file handler, hooks, SQL helpers, and the streaming runtime gated by FLOW_STREAMING_ENABLED.
• Extension point: add a new IR node kind by adding its type to the IR types barrel, then registering its executor in the default registry. Add a new YAML connector by writing a spec under the connector layer (the registry singleton auto-loads it). Add a custom plugin connector via the plugin SDK. Add a new streaming runtime by implementing the streaming-runtime contract under the runtime layer and gating with FLOW_STREAMING_ENABLED=true.

​

Other modules

The nine remaining undocumented subsystems. Each gets one paragraph plus public surface and extension hook.

​

Audit forwarding

A single-file SIEM forwarder. Ships audit-log batches to a configured SIEM backend (Splunk, Datadog, or Elastic). Called by the SIEM-ship cron route. Distinct from the per-tenant hash-chained audit-log writer, which lives at the root of the library layer.

• Public surface: the SIEM-backend literal union, the audit-event and config shapes, an environment-variable loader, and the ship operation.
• Extension point: add a new SIEM backend by extending the backend union and adding a transport branch inside the ship operation.

​

Governance

Four-eyes approval gate plus multi-stage approval policy. The four-eyes module enforces that the requester and approver of a sensitive action are different users; the workflow module defines the policy plus stage state machine; the stages module is the per-stage decider used by approval-route handlers.

• Public surface: four-eyes enablement check, four-eyes evaluation, four-eyes-violation thrower, the policy and stage shapes, the request-state shape, the four-eyes-violation error, the policy resolver, the request initializer, the per-stage decider, the stage-insert builder, and the stage-decision result shape.
• Extension point: add a new approval stage by extending the stage union and adding the per-stage rule to the per-stage decider.

​

Scoring

The scoring algorithms — every model used by the decision-flow runtime and the pipeline runtime to score candidate offers. Includes scorecard, logistic regression, gradient-boosted trees (with TreeSHAP), Bayesian, neural collaborative filtering, ONNX runner, Thompson bandit, epsilon-greedy, online learner, and the auto-learn / auto-upgrade orchestrators.

• Public surface: a single dispatcher (score with model), a trainer, a “score offer set” entry point, an external-endpoint variant, plus a typed scoring-result shape and options.
• Per-algorithm modules: scorecard, logistic regression, Bayesian, gradient-boosted (real LightGBM-trained, walked in-process), exact TreeSHAP, neural collaborative filtering plus its SHAP variant, Thompson bandit, epsilon-greedy, online learner, external endpoint, ONNX runner plus blob store, auto-learn, and auto-upgrade.
• Extension point: add a new scoring algorithm by writing a module that exports a score… function returning the scoring-result shape, then adding a dispatch case to the main dispatcher.

​

Arbitration

Multi-objective arbitration — selects which candidate(s) to surface among scored offers under budget, fairness, and pacing constraints. Includes the realtime apply path (called from the recommend route), a Lagrangian solver, an EXP3-IX online bandit, goal-seek, budget pacing, and an online-tuning weight updater.

• Public surface: a realtime-arbitration entry point, feature-flag readers (EXP3-IX, budget pacing, goal-seek), a bandit-arm selector and config shape, a bandit-state reader, a Lagrangian solver and its result shape, plus apply, budget-pacing, cross-offer, goal-seek, constraints, online-weights, and load-cross-offer modules.
• Extension point: add a new arbitration objective by adding a field to the weights and objective-scores shapes in the root arbitration module. Add a new bandit algorithm by writing a sibling module and gating it with a tenant-settings flag analogous to the EXP3-IX gate.

​

Fairness

Group-fairness, counterfactual-fairness, individual-fairness, and intersectional metrics. Used by the model-governance reports and the fairness dashboard. Distinct from the scoring layer: scoring produces scores; fairness audits the scores’ distribution across groups.

• Public surface: an evaluation entry point plus per-sample, per-group-stats, and report shapes; counterfactual, individual, and intersectional evaluators; plus report-export and scorer-resolver helpers.
• Extension point: add a new fairness metric by adding a function at the appropriate level (group-level or counterfactual / individual / intersectional) and extending the report shape used by the consumer.

​

Explanations

Decision explanations for regulator, agent, and customer audiences. Includes LIME, counterfactual search, SHAP-narrative LLM generation, multi-language directives, PII redaction, prompt templates, quality scoring, and a result cache.

• Public surface: a narrative generator with input, result, not-found, and generation-error shapes; a LIME computer plus a global feature-importance helper; a counterfactual finder with counterfactual and counterfactual-scorer shapes; an advanced-explanation runner plus an explanation-type union; a supported-languages constant, a language-directive applicator, and a leak scanner; a quality scorer.
• Extension point: add a new explanation type by extending the explanation-type union and adding a branch in the advanced-explanation runner. Add a new audience mode by extending the narrative-mode shape and the prompt templates.

​

ML governance

ML model governance and lifecycle utilities. Distinct from the scoring layer (which is the algorithms): this layer handles registry status (draft / challenger / champion / archived), drift detection (PSI plus KS), auto-rollback, counterfactual training, target encoding, auto-binning, and preprocessing.

• Public surface: a registry-status union, a registry-metadata shape, a registry error, a model-promotion operation, and a metadata reader; PSI and KS computers plus their drift-samples and result shapes; auto-binning, auto-rollback, counterfactual-trainer, preprocessing, and target-encoding modules.
• Extension point: add a new registry status by extending the status union and adding the transition rule to the promotion operation. Add a new drift metric by writing a sibling to the PSI / KS computers.

​

Negotiation

Customer ↔ offer negotiation. Includes a multi-turn agent loop, single-turn apply-mode, an evaluation harness, the realtime-apply route helper called from the recommend route, and Zod-validated guardrails (discount bands, term bands).

• Public surface: discount-band, term-band, guardrails, and proposal Zod schemas plus their inferred types; an agent runner plus disabled, guardrail-violation, and agent errors; realtime-apply tenant-settings, offer, and decision shapes; plus apply-mode, eval-harness, multi-turn, realtime-apply-route, and guardrails modules.
• Extension point: add a new negotiation guardrail by extending the guardrails Zod schema and adding the corresponding check inside the agent runner. Add a new negotiation strategy by writing a sibling to the agent runner and wiring it through apply-mode or multi-turn.

​

Qualification

The decisioning-gates classifier and the rule-inheritance walker. Classifies every qualification rule into one of three stages (eligibility | fit | match), resolves effective rules at the four scope levels (global → category → subcategory → offer), detects conflicts, and applies time-aware windows. Distinct from the root qualification engine, which is the runtime evaluator.

• Public surface: a stages constant, a stage union, a stage-descriptions map, a classifiable-rule shape, and a rule classifier; a scoped-rule shape, a context-scopes shape, an effective-rules resolver, a rule-conflict shape, and a conflict detector; plus a deprecated re-export shim, skip-reason tally, stage-backfill, and time-aware modules.
• Extension point: add a new decisioning stage by extending the stages constant and the rule classifier. Add a new rule scope by extending the context-scopes shape and the resolver.

​

Configuration

Most subsystems on this page read environment variables via the infrastructure container. Direct environment-variable dependencies are listed below; cross-link to Environment Variables for the full reference.

​

Honest limits

• The audit-forwarding subsystem has no dedicated test directory — it is exercised only via the SIEM-ship cron route.
• Caching is intentionally not surfaced as a separate subsystem here; all caching goes through the cache getter on the infrastructure container.
• The legacy pipeline subsystem is vestigial after the legacy ETL deletion (Apr 28). It now contains only DAG, formula-to-SQL, and SQL-safety helpers retained for the new Flow IR runtime. Do not extend it; new pipeline work belongs under Flow.
• The state subsystem is browser-only Zustand stores, not server-side library code. Listed for completeness but out of scope for this internals page.
• The queue subsystem is a single-file BullMQ wrapper. Worker concurrency is set by WORKER_CONCURRENCY in the worker process; queue topology is not currently documented per-queue.
• The database subsystem ships a Prisma 7 plus pg-adapter pattern plus DDL helpers. The DDL helpers create real PostgreSQL tables for tenant data schemas — this is intentional, see the “Real DDL” note in the contributor guide.
• A registered-but-unimplemented connector returns a structured 503 at first dispatch. This is the loud-fail replacement for the previous silent-undefined behavior — but it does mean a tenant can configure a connector in the UI that will fail at first dispatch. Add the executor before promoting the connector in the registry.
• The AI subsystem has a fast-changing surface — the tools registry gains entries with most product changes. The eleven-tool count cited above is accurate at time of writing but rotates.
• Flow is the largest and youngest subsystem on this page (Phases 6.0 → 6.6 shipped Apr 27). The streaming runtime is gated by FLOW_STREAMING_ENABLED and is not wired to any production broker — it is a feature-flag-only scaffold for self-hosters.
• GitOps V1 scope is explicit: only Categories, Channels, Arbitration Profiles, Offers, and Decision Flows reconcile fully. Other recognized kinds report as unchanged.

​

Related

• Flow Overview — user-facing slice of the Flow subsystem.
• Infrastructure — operator-facing slice of the infrastructure subsystem (per-backend toggles plus health).
• Environment Variables — full reference for every environment variable read by this layer.
• Architecture Overview — system-level layering above this page.
• Architecture Engine — decision-flow runtime above the per-module internals here.
• Security Model — multi-tenant isolation that gates every call into these modules.
• GitOps API — HTTP surface that calls the GitOps subsystem.
• Repository: platform/src/lib/ — open the source for any subsystem above.