> ## Documentation Index
> Fetch the complete documentation index at: https://docs.kaireonai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Flow Editor UI

> The 2-pane Flow editor at /data/flow-pipelines/[id]/edit — visual canvas, JSON IR editor, SQL preview, lineage, and schedule.

The Flow editor is the single surface for designing, inspecting, and
operating pipelines. It lives at
`/data/flow-pipelines/[id]/edit`. The legacy single-pane
`pipeline-flow-editor` (PipelineNode/Edge model) was deleted on
2026-04-28 — every pipeline is now IR-native (`irVersion: "1.0"`).

## Layout

| Pane       | Width             | Contents                                                                                                                                          |
| ---------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Center** | flex (\~72%)      | Tabbed editor: **Visual / JSON IR / SQL Preview / Lineage / Schedule**                                                                            |
| **Right**  | \~26% (resizable) | **Node Config** — the per-kind form for the node you click on the canvas; when nothing is selected it shows a tip pointing at the docked AI panel |

The AI Panel (Pipeline mode) is **not** in the right pane — it is a shell-level
panel docked at the top-right of the page that auto-activates whenever the URL
is under `/data/flow-pipelines`.

A top bar shows the pipeline name, status pill, IR version pill, Publish
button, and a Run-now button. A bottom strip shows the last run, next
scheduled fire, DLQ count, and quick links. Run history lives on the
dedicated [Pipeline Runs](/operations-reporting/runs) page reached from the
sidebar — the editor's previous "Recent runs" left pane was redundant
and was removed on 2026-05-02.

## Tab reference

### Visual

React Flow canvas, kind-coloured nodes (source / transform / validate /
target / branch / join / enrich / outcomes), deterministic vertical
waterfall layout. Each node card shows a one-line config summary inline
(source: path · pattern · format; target: schema · loadMode; transform:
op types; validate: rule count; outcomes: resolution mode · attribution
window). Click a node → right pane swaps to
**Node Config**, an editable per-kind form (the `join` kind is the one
exception — it shows a read-only JSON view with a jump to the JSON IR tab).

Above the canvas there is an **Add Node** toolbar with six buttons:
\*\*+ Transform / + Validate / + Enrich / + Branch / + Target /

* Outcomes\*\* (the addable kinds, from `ADDABLE_KINDS` in
  `node-defaults.ts`). Click a chainable button (Transform / Validate /
  Enrich / Branch) and kaireon inserts a default-configured node into the
  IR between the last upstream and your first sink, repoints the sinks
  that consumed the prior upstream so they now flow through the new node,
  and selects it so you can edit. **+ Target** and **+ Outcomes** are
  terminal **sinks** — they append a new node that consumes the same
  upstream as the existing chain rather than being spliced into it (see
  the Target and Outcomes sections below). Defaults intentionally use
  placeholder values (e.g. a transform with a `rename: from_field →
  to_field` op, or an outcomes node mapping the `_kaireon_rid` token) so
  you can tell at a glance which fields to customize.

Source and Join are not in the toolbar — every starter pipeline already
has one source, multi-source pipelines are advanced, and Join requires
picking two upstream node refs. Add those via the JSON IR tab or the AI
panel.

File-level archival is no longer a separate node kind — the source
node's `atomicity` config (`successFolder` / `failureFolder`) moves
the source file to a date-templated archive folder on parse success
and to a failure folder on parse error. See
[file ingestion](/data/connectors/file-ingestion) for the full atomic
staging flow.

### JSON IR

Read/write IR editor with format-on-blur. Server-side validation goes
through `parsePipelineIR` — invalid IR is rejected with the structured
`errors` array rendered verbatim. Save creates a new IR version (POST
`/api/v1/pipelines/:id/ir`) and bumps the version pill.

### SQL Preview

Renders the **exact** `INSERT … SELECT` each target node would execute,
using the same builders the runtime uses (`buildLoadSql` +
`buildSelectExpr`). `blue_green`, `incremental_watermark`, and
`cdc_mirror` targets render an explicit "preview not available — runtime
load-mode helper handles it" note rather than fabricated SQL.

GET `/api/v1/pipelines/:id/sql-preview` powers this tab.

### Lineage

See [Flow Lineage](/data/observability/flow-lineage).

### Schedule

See [Flow Schedule](/data/pipelines/flow-schedule).

## Run history

Run history is a dedicated page at **Sidebar → Data → Pipeline Runs**
(`/data/flow-runs`) — the editor's previous in-pane "Recent runs"
preview was redundant once the standalone page landed and was removed
on 2026-05-02. It is a run-level table (Run id / Status / Started /
Completed / Rows In / Rows Out / Processed / Error), one row per run.

## Node config — what each form does

The right pane swaps to a per-kind form when you click a node:

* **Source** — Path / file mask / format / wait policy. If the parent
  connector already supplies the bucket (S3 / GCS / Azure), the form
  hides the bucket input and only asks for the prefix.
* **Transform** — Add op rows in order; each op has its own subform
  (cast type, formula, rename pair, etc.). A collapsible **Sample row
  preview** at the bottom takes one JSON row and shows the before /
  after diff for each op (added / removed / changed fields highlighted).
  Complex ops (`aggregate`, `lookup_join`, `vector_embed`, `geo_resolve`,
  `sentiment_score`, `language_detect`) preview as "not previewable
  client-side — run the pipeline" because they need runtime context.
  Note that `summarize`, `vector_embed`, `geo_resolve`, and
  `sentiment_score` in `llm` mode are **passthrough** at runtime (they
  return rows unchanged until a provider is wired); `aggregate`,
  `lookup_join`, `sentiment_score` in `rule` mode, and `language_detect`
  do run. See [Transforms](/data/transforms/transforms).
* **Validate** — Row-level rules (`notNull`, `regex`, `range`,
  `fieldType`, `maxLength`), optional dataset-level row-count check,
  optional quarantine table for failed rows. The same Sample row
  preview at the bottom shows pass/fail per rule.
* **Enrich** (Beta — not yet runnable) — Provider picker (`llm_tag` /
  `geocode` / `ml_score`) with a per-provider config block. The
  output-field input checks the destination DataSchema in real time: if
  the column doesn't exist, it shows a red "missing column" badge with an
  inline **+ Add as \<dataType>** button that POSTs to
  `/api/v1/schemas/fields` with a sensible default type (`string` for
  `llm_tag`, `float` for `geocode` lat/lon and `ml_score`). The form
  itself carries a **BETA** banner because the enrich executor **throws
  at runtime** — the providers are not wired in this build. The node
  validates and saves, but a pipeline that contains it will fail when
  run; remove the enrich node (or implement the provider call) before
  running.
* **Branch** — `when` predicate + `then` route per case, plus a default
  route. The route inputs are **dropdowns** populated from the current
  IR's other nodes (excluding the branch itself and source nodes).
  Stale references (a `then` value that no longer matches any node id)
  render with a red border so they're easy to spot and fix.
* **Target** — Substantial form. Includes:
  * **Reads from** Select listing every other node in the IR — so
    multi-target IRs (one source fan-out to multiple destinations)
    are editable in the visual surface.
  * Schema name (`schema.table`, defaults to `public.`).
  * Load mode (`append` / `truncate` / `upsert` / `blue_green` /
    `incremental_watermark` / `cdc_mirror`). `cdc_mirror` applies an
    op-column change feed — configure the op column and CDC key. See
    [Loading Modes & Validation](/data/pipelines/loading-modes-validation).
  * **Full-refresh hard warning** — picking `truncate` on a
    one-source-one-target shape surfaces a recommend-blue\_green
    banner with a one-click switch.
  * **`failOnEmptySource` Switch** (default on) for destructive
    modes — aborts before TRUNCATE / blue\_green when upstream is
    empty.
  * **`backupBeforeLoad` Switch** for destructive modes — opt-in
    pre-load snapshot to `<table>_backup_<runId>` with retain-N
    pruning.
  * Mode-specific config blocks: `upsertKey`, `watermarkColumn`, and
    for `cdc_mirror` the op column (`cdcOpColumn`) + CDC key
    (`upsertKey`).
  * **Inbound-FK probe** when `blue_green` is picked — fires
    `GET /api/v1/schemas/:id/incoming-fks` and surfaces an amber
    warning listing referencing tables that would block the swap's
    final DROP.
  * **Inline mode-switch validation** — empty `upsertKey`, missing
    watermark column, all-keys upsert, etc. all surface as inline
    errors that block save.
* **Outcomes** — Inbound response-file ingestion (opens / clicks /
  bounces / conversions / unsubscribes) into interaction history and
  model learning; there is no output table. Includes:
  * **Reads from** Select — the upstream node carrying the vendor rows
    (sources / transforms, not other sinks).
  * **Column mapping** — a **Resolve the offer by** switch (Creative
    token `_kaireon_rid` **or** Offer id/name) that shows exactly one
    offer-resolution input, plus required `customerId` + `outcomeKey`
    and optional `channelId` / `timestamp` / `conversionValue` /
    `eventId`. Cleared optional columns store `undefined` (not `""`)
    so the IR's `.min(1)` optionals don't reject on save. A destructive
    inline error appears if neither offer-resolution column is set
    (mirrors the IR `superRefine`).
  * **Attribution window (days)** — number input, `1`–`365`, default 7.
  * **Suppression rules** — an enable Switch (default on), the trigger
    `outcomeKeys`, a mechanism select (`consent_revoke` /
    `do_not_contact`), and — for `consent_revoke` — the
    `revokePurposes` list. The column mapping references the **upstream
    vendor file** columns, so these inputs deliberately do **not**
    autocomplete from the destination schema (an outcomes node has no
    destination schema). See [Pipeline IR → Outcomes
    node](/data/transforms/pipeline-ir#outcomes-node) for the full
    behaviour.

The toolbar above the canvas has **+ Target** and **+ Outcomes**
alongside + Transform / + Validate / + Enrich / + Branch. Both sink
buttons append a new node consuming the same upstream as the existing
chain, without re-pointing it — pick a destination schema (target) or
map the response columns (outcomes) in the right pane.

## Honest residuals

Documented gaps, deferred to follow-up plans:

* Drag-to-create-edge / right-click "Delete" on the visual canvas.
* Pane collapse buttons (react-resizable-panels v4 typing settles).
* Run-over-run trend chart (hardening pass).
* Visual DAG highlight when clicking a lineage row (Wave 4 polish).
* Selective replay-from-DLQ runtime (`replayDlqOnly` flag).
* Sample row preview's `expression` op shows the formula text rather
  than evaluating client-side — the server-side formula engine is the
  source of truth and replicating it in the browser would risk drift.
* The 1500-line `node-config-panel.tsx` is partially extracted
  (constants + field-input primitives moved to
  `components/flow-editor/forms/`); the per-kind subforms still live
  in the panel. Full per-form file split is a follow-up refactor.

These aren't broken — they aren't built yet, and the UI calls them
out by name where they would otherwise appear.