Decision Traces - KaireonAI

What a decision trace is

Every time /api/v1/recommend runs, the engine takes a list of candidate offers and pushes them through four stages — qualification (which offers is this customer eligible for?), contact policy (which of those are we allowed to send right now?), scoring (how good is each remaining offer for this customer?), and ranking (given placements and channel coupling, which N do we return?). A decision trace is the forensic record of that journey: how many candidates entered each stage, which rules fired, which offers were filtered out and why, what scores the models produced, and which offers made the final cut. Traces are written asynchronously after the decision is delivered, so they never slow down /recommend. They are sampled per tenant (decisionTraceEnabled + decisionTraceSampleRate); at 10% sampling a high-traffic tenant still gets enough forensic coverage to investigate any specific customer or incident.

What you see when you open a trace in Studio

Open Studio → Decision Traces and click any row to expand it. The expanded view has three layers, each answering a different question:

1. Stage Timeline — which stage filtered offers, and by how much?

A vertical timeline of the four pipeline stages with the candidate count entering and exiting each one (12 → 4, 4 → 4, 4 → 4, 4 → 4). A green dot means the stage ran cleanly; an amber dot means it ran in degraded mode (a scoring model threw, fallback scores were used). This is the “where did the candidates go?” view. Use it when you suspect a stage is filtering too aggressively or running slow.

2. Per-Offer Journey — for THIS customer and THIS request, why this offer and not that one?

A table with one row per offer ID that appeared anywhere in the trace, showing its trajectory across all four stages on a single line:

Qualification — pass (✓) or fail (✗) with the exact reason text and the ruleId of the rule that fired. Example: Missing attribute "tier" (d0cc9358-…).
Contact Policy — pass (✓) or blocked (○) with the policy reason and policyId. Example: frequency_cap (pol_email_daily_cap).
Score — the numerical score, the rank within the scoring stage, the modelType that produced it (gradient_boosted, thompson_bandit, etc.), and the top-3 signed feature contributions rendered as horizontal bars (green = positive, red = negative).
Final — a 🏆 trophy with the rank in the delivered response, or em-dash if the offer didn’t make it.

A header strip above the table shows the experiment variant (when present), ranking weights, and the first 8 chars of inputsHash / policyVersionHash so you can cross-reference against drift logs and policy snapshots. Selected offers are highlighted green. This is the “Why didn’t Customer X get Offer Y?” view. Compliance reviewers, support escalations, and ops engineers debugging weird-looking results live here.

3. Explain button — narrative prose for a human reader

The Explain button at the right of each trace row opens a dialog that calls the LLM-narrated explanation endpoint. The dialog has three tabs — one per audience:

Tab	What it produces	When to use it
Regulator	~400-word formal prose with full factor detail. Writes an audit-log entry so the narrative is discoverable during a DSAR or regulator review.	When you need a defensible, compliance-grade explanation for an auditor.
Agent	Structured JSON describing the selected offer, top factors, alternatives considered, and policies fired. Machine-readable.	Driving a call-center console, a support troubleshooting UI, or any internal tool that needs to consume the explanation programmatically.
Customer	One or two plain-language sentences, suitable to show to the end customer in-product.	Embedding a “why are you seeing this?” caption next to a recommended offer.

A Regenerate button bypasses the 7-day cache (noCache: true); the dialog footer shows which model produced the narrative, whether the result was cached, and the token counts. The Explain button is gated by a per-tenant opt-in (tenantSettings.aiAnalyzerSettings.llmExplanationsEnabled); when it’s off, the button shows a banner that links to settings. See LLM Explanations for the full lifecycle, PII redaction, audit log, and rate limits.

When to reach for which layer

“Latency is up, where?” → Stage Timeline.
“Customer X is asking why they didn’t get the Premium Card.” → Per-Offer Journey for that customer’s trace.
“Our compliance officer wants a written record for the Brown v. Tenant audit.” → Explain → Regulator tab.
“The support agent UI needs to render the explanation.” → Explain → Agent tab (JSON).
“Embed a one-liner under the offer card in the customer app.” → Explain → Customer tab.

GET /api/v1/decision-traces

List decision traces with offset-based pagination. Traces are only recorded when tracing is enabled in tenant settings and the request passes the sample rate check.

Query Parameters

Parameter	Type	Default	Description
`customerId`	string	—	Filter traces for a specific customer
`requestId`	string	—	Filter by request/interaction ID
`limit`	integer	`50`	Max results (max 200)
`offset`	integer	`0`	Pagination offset

Response

{
  "traces": [
    {
      "id": "trace_001",
      "customerId": "CUST001",
      "requestId": "550e8400-e29b-41d4-a716-446655440000",
      "decisionFlowKey": "default-flow",
      "pipeline": [
        { "stage": "qualification", "candidates": 47, "passed": 32, "durationMs": 12 },
        { "stage": "contact_policy", "candidates": 32, "passed": 28, "durationMs": 8 },
        { "stage": "scoring", "candidates": 28, "durationMs": 15 },
        { "stage": "ranking", "topScore": 0.872, "method": "priority_weighted", "durationMs": 5 }
      ],
      "result": {
        "count": 5,
        "topOffer": { "offerId": "offer_001", "score": 0.872 }
      },
      "totalDurationMs": 42,
      "createdAt": "2026-03-16T14:30:00.000Z"
    }
  ],
  "total": 1250,
  "limit": 50,
  "offset": 0
}

GET /api/v1/decision-traces/

Get a single decision trace with full pipeline detail.

Response

Returns the complete trace object with all pipeline stages, scoring details, and result.

Per-stage timeline (UI)

The studio detail view at /studio/decision-traces/{id} renders a per-stage timeline alongside the JSON trace. Each pipeline stage (Enrich, Compute, Eligibility, Fit, Match, Ranking, Negotiation) is shown as a horizontal bar with the stage label, elapsed milliseconds, and the candidate count before/after. Hovering any bar reveals the rule firings, scoring inputs, and reasons recorded for that stage. Use this to diagnose which stage is slow or filtering more aggressively than expected without scrolling through the raw trace payload.

Provenance deep-dive — “Why this customer got these offers”

Expanding any row at /studio/decision-traces opens a per-offer journey table directly inline. For each offer ID that appeared anywhere in the persisted JSON arrays, the row shows the offer’s trajectory across all four stages on one line: qualification (pass / fail with the exact rule reason and ruleId), contact policy (pass / blocked with the policy reason and policyId), score (the model’s score, rank, and modelType, plus the top-3 signed feature contributions from scoringResults[].explanations[] rendered as horizontal bars), and final (selected with rank, or not selected). Rows that were selected by the decision flow are highlighted in green. The header strip above the table surfaces, when present:

experiment.variant — which experiment arm this decision was assigned to.
rankingWeights — the ranking-profile weights (PRIE coefficients, diversity / emphasis multipliers).
The first 8 chars of inputsHash and policyVersionHash — for cross-referencing against drift logs or policy snapshots.
A degraded badge when degradedScoring = true. This is set whenever a scoring model ran in fail-soft fallback mode rather than producing a real score — a model key that couldn’t be resolved, a scorer that threw, or an imported ONNX model whose runtime is unavailable or errored (the score falls back to 0.5). The per-decision badge complements the aggregate scoring_model_failures metric, so a single trace shows the degradation instead of hiding it behind a fleet-wide counter.

This deep-dive reads only what the trace already persists — there is no new API call. Older traces persisted before the JSON-array enrichment landed render an empty-state caption; the count-based timeline continues to work for them.

Enabling Decision Traces

Decision traces are controlled by two tenant settings:

Setting	Description
`decisionTraceEnabled`	Master toggle for trace recording
`decisionTraceSampleRate`	Sampling rate (0.0 - 1.0). Set to `1.0` to trace every request

Configure these via the Settings API or the platform UI under Settings.

At high traffic volumes, set the sample rate below 1.0 to avoid excessive storage. A 10% sample rate (0.1) typically provides sufficient forensic coverage.

JSON Field Shapes

Each trace persists five JSON arrays that capture the forensic detail of the decision. Downstream aggregators (the selection_frequency, anomaly_candidates, and why_not_ranked endpoints in Dashboard Data) expect the following shapes on newly-written rows. Older rows written before a field was added are still readable — fields default to null in the aggregation queries.

`scoringResults`

Best-first list of offers that reached the scoring stage. Each entry also carries model attribution — which model actually scored that candidate: modelId (the AlgorithmModel.id), modelType (e.g. scorecard, bayesian), and propensitySource (how the propensity was obtained — model, adaptation, or cold_start). With scoped model routing, different candidates in the same decision can be attributed to different models. Each entry also carries an explanations[] array of per-feature contributions — { field, contribution } for every predictor the scorer considered. The contribution is normalized across model types: linear/Bayesian scorers report their signed log-odds weight, and a scorecard reports the rule’s awarded points (previously these surfaced as 0). A fail-soft entry (e.g. a degraded ONNX score) additionally carries degraded: true and a reason, so the explanation says the score is a fallback rather than a real attribution.

[
  { "offerId": "off_premium_card", "score": 0.89, "rank": 1,
    "modelId": "mdl_scorecard_v1", "modelType": "scorecard", "propensitySource": "model" },
  { "offerId": "off_gold_plus", "score": 0.71, "rank": 2,
    "modelId": "mdl_bayes_v1", "modelType": "bayesian", "propensitySource": "adaptation" }
]

`selectedOffers`

The final ranked result returned to the caller, in delivery order.

[
  { "offerId": "off_premium_card", "score": 0.89, "rank": 1 },
  { "offerId": "off_gold_plus", "score": 0.71, "rank": 2 }
]

`qualificationResults`

One row per offer that the qualification stage evaluated. passed=false indicates a rejection; ruleId identifies the rule that evaluated. When a rule could not be evaluated because its required data was missing from the decision context and the rule’s onMissing behavior is "skip" (the default), the row carries skipped: true plus a warning describing what was missing and, when applicable, an attribute field naming the missing key — the fail-open is visible in the trace, not a silent pass. See Missing data: onMissing.

[
  { "offerId": "off_premium_card", "passed": true, "reason": "passed" },
  { "offerId": "off_restricted", "passed": false, "reason": "segment_mismatch", "ruleId": "rule_42" },
  {
    "offerId": "off_new_card",
    "passed": true,
    "reason": "Skipped: attribute \"customer.credit_score\" not present (not enriched)",
    "ruleId": "rule_17",
    "skipped": true,
    "warning": "attribute \"customer.credit_score\" not present in decision context",
    "attribute": "customer.credit_score"
  }
]

`contactPolicyResults`

One row per offer that the contact-policy stage evaluated. blocked=true indicates the policy suppressed the offer for this customer.

[
  { "offerId": "off_premium_card", "blocked": false, "reason": "passed" },
  { "offerId": "off_weekly_promo", "blocked": true, "reason": "frequency_cap", "policyId": "pol_email_daily_cap" }
]

`rejectedOffers`

The merged per-stage rejection list — one entry per candidate any stage removed, in the same shape as the recommend response’s rejectedOffers[] (see Recommend). This list is complete: every candidate-dropping stage contributes a structured reason — decisioning gates (eligibility), contact-policy suppressions (contact_policy), consent, channel/creative/placement filters (channel), suppression rules, tenant-wide frequency caps (frequency_cap), the maturity-ramp cold-start floor (maturity_ramp), hard guardrails, the atomic channel-coupling cascade (coupling), and the rank node’s topN cut (rank). Entry shape: { offerId, offerName?, stage, reason, detail? } where stage ∈

eligibility | contact_policy | consent | channel | suppression | frequency_cap | maturity_ramp | guardrail | coupling | rank

, reason is a self-contained human-readable sentence, and detail is an optional structured object (exposureProbability, capKey, scope, …). Rows written before the column landed default to [].

[
  {
    "offerId": "off_new_card",
    "offerName": "New Card Launch",
    "stage": "maturity_ramp",
    "reason": "Below cold-start exposure floor — maturity ramp withheld this offer for this customer today (exposure probability 25%, mode bayesian_ci)",
    "detail": { "exposureProbability": 0.25, "mode": "bayesian_ci" }
  },
  {
    "offerId": "off_gold_plus",
    "offerName": "Gold Plus",
    "stage": "rank",
    "reason": "Ranked below the cut — rank node (topN) kept top 2 of 3 candidates (this candidate scored 0.4100)",
    "detail": { "score": 0.41, "kept": 2, "considered": 3, "method": "topN" }
  }
]

POST /api/v1/decisions//narrative

Generate an LLM-written natural-language explanation of a single decision trace. The feature is opt-in per tenant — see LLM Explanations.

Request Body

{
  "mode": "regulator",
  "noCache": false
}

Field	Type	Default	Description
`mode`	`"regulator" \| "agent" \| "customer"`	`"regulator"`	Audience for the explanation. Changes tone, length, and format.
`noCache`	boolean	`false`	When `true`, bypass the cache and force a fresh LLM call.

Response

{
  "narrative": "The system recommended offer_premium_card for customer CUST001 because ...",
  "mode": "regulator",
  "model": "claude-sonnet-4-7",
  "cached": false,
  "tokens": { "input": 1420, "output": 310 },
  "createdAt": "2026-04-18T21:04:18.000Z"
}

Field	Type	Description
`narrative`	string	The generated explanation. Prose for regulator and customer modes, structured JSON for agent mode.
`mode`	string	Echoes the requested mode.
`model`	string	Identifier of the LLM used.
`cached`	boolean	`true` if the result was served from Redis.
`tokens.input` / `tokens.output`	number	LLM token counts when reported by the provider.
`createdAt`	ISO-8601 string	Creation timestamp of the narrative (cached or fresh).

Auth, Limits, and Tenancy

Requires tenant authentication (session or API key).
Rate limited to 20 requests / minute / tenant. Exceeding returns 429.
Tenant opt-in required: tenantSettings.aiAnalyzerSettings.llmExplanationsEnabled = true. Otherwise returns 403 with "LLM explanations are not enabled for this tenant.".
The decision trace must belong to the authenticated tenant or the response is 404.

Audit Log (regulator mode)

When mode = "regulator", a row is appended to the audit log:

{
  "tenantId": "tenant_abc",
  "action": "generate_narrative",
  "entityType": "decision_trace",
  "entityId": "trace_001",
  "entityName": "regulator narrative",
  "changes": {
    "mode": "regulator",
    "model": "claude-sonnet-4-7",
    "cached": false,
    "narrativePreview": "..."
  }
}

Caching

Results are cached in Redis for 7 days under the composite key (tenantId × decisionTraceId × mode × model × inputsHash). Repeating the same request returns cached: true instantly.

POST /api/v1/decisions//shap

Return exact per-feature TreeSHAP contributions for a recorded decision’s gradient_boosted scoring step. Designed for EU AI Act Article 13/22 audit workflows: the response is a mathematically-grounded, additivity-verified breakdown of every feature’s effect on the raw margin (logit space). Unlike the LLM narrative endpoint, this returns raw numbers — deterministic, cacheable, and auditable independent of any LLM availability. Pair the two when you need both prose and numerics in a regulator export.

Request Body

{
  "modelId": "mdl_premium_card_gbm",
  "offerId": "off_premium_card",
  "attributes": {
    "recent_transaction_count": 12,
    "tenure_months": 36,
    "average_balance": 15400,
    "segment_index": 2
  },
  "round": 6
}

Field	Type	Default	Description
`modelId`	string	required	The `AlgorithmModel.id` used to score the trace. Must be tenant-owned and `modelType = "gradient_boosted"`.
`offerId`	string	optional	The candidate offer the SHAP is being computed for. Echoed in the audit log.
`attributes`	object	required	The customer/offer/context attributes passed to the scorer at decision time. Required because raw attributes are not persisted on the trace (PII minimization).
`round`	integer	`6`	Decimal places for output (`0` = raw doubles).

Response

{
  "decisionTraceId": "trace_001",
  "modelId": "mdl_premium_card_gbm",
  "modelName": "Premium Card GBM",
  "offerId": "off_premium_card",
  "shapValues": {
    "recent_transaction_count": 1.314,
    "tenure_months": 0.428,
    "average_balance": -0.211,
    "segment_index": 0.083
  },
  "baseline": -0.602,
  "rawMargin": 1.012,
  "additivityResidual": 0.0,
  "featureCount": 4
}

Field	Type	Description
`shapValues`	object	Per-feature Shapley contributions in raw-margin (logit) space.
`baseline`	number	Cover-weighted expected raw-margin across the ensemble.
`rawMargin`	number	The chosen leaf-value sum for these attributes. `sigmoid(rawMargin) = score`.
`additivityResidual`	number	`	rawMargin − baseline − Σ shapValues	`. Should be ≈ 0; non-zero indicates a malformed model.
`featureCount`	number	Number of features that received a non-zero contribution.

Additivity guarantee

For any input x:

sum(shapValues) + baseline = rawMargin
score(x)                   = sigmoid(rawMargin)

This identity is built into the algorithm (Lundberg, Erion, Lee 2018, Algorithm 2 — path-dependent variant). The endpoint reports additivityResidual so consumers can verify the invariant on-wire.

Probability-space approximation

SHAP values are reported in raw-margin space — the only space where the additivity identity holds. To approximate a feature’s effect on the probability, use:

delta_p_i ≈ sigmoid(baseline + shap_i) − sigmoid(baseline)

This is monotone with shap_i but not strictly additive in probability space. UI surfaces typically show both: the raw φ for compliance, and a sigmoid-mapped delta for human-readable display.

Auth, Limits, and Tenancy

Requires tenant authentication (session or API key).
Rate limited to 30 requests / minute / tenant.
Tenant opt-in required: tenantSettings.aiAnalyzerSettings.llmExplanationsEnabled = true (same flag as the narrative endpoint).
The decision trace, the model, and the request must all belong to the same tenant. Cross-tenant lookup returns 404.
Returns 400 if the model’s modelType is not gradient_boosted or if it has no trained trees.

Audit Log

Every successful call writes:

{
  "tenantId": "tenant_abc",
  "action": "compute_shap",
  "entityType": "decision_trace",
  "entityId": "trace_001",
  "entityName": "shap for Premium Card GBM / off_premium_card",
  "changes": {
    "modelId": "mdl_premium_card_gbm",
    "modelName": "Premium Card GBM",
    "offerId": "off_premium_card",
    "featureCount": 4,
    "baseline": -0.602,
    "rawMargin": 1.012,
    "additivityResidual": 0.0
  }
}

This ensures DSAR exports and regulator queries can trace which decisions have had SHAP attributions computed and when.

Roles

Endpoint	Allowed Roles
`GET /decision-traces`	admin, editor, viewer
`GET /decision-traces/{id}`	admin, editor, viewer
`POST /decisions/{id}/narrative`	admin, editor, viewer (tenant-scoped)
`POST /decisions/{id}/shap`	admin, editor, viewer (tenant-scoped)

​What a decision trace is

​What you see when you open a trace in Studio

​1. Stage Timeline — which stage filtered offers, and by how much?

​2. Per-Offer Journey — for THIS customer and THIS request, why this offer and not that one?

​3. Explain button — narrative prose for a human reader

​When to reach for which layer

​GET /api/v1/decision-traces

​Query Parameters

​Response

​GET /api/v1/decision-traces/

​Response

​Per-stage timeline (UI)

​Provenance deep-dive — “Why this customer got these offers”

​Enabling Decision Traces

​JSON Field Shapes

​scoringResults

​selectedOffers

​qualificationResults

​contactPolicyResults

​rejectedOffers

​POST /api/v1/decisions//narrative

​Request Body

​Response

​Auth, Limits, and Tenancy

​Audit Log (regulator mode)

​Caching

​POST /api/v1/decisions//shap

​Request Body

​Response

​Additivity guarantee

​Probability-space approximation

​Auth, Limits, and Tenancy

​Audit Log

​Roles

What a decision trace is

What you see when you open a trace in Studio

1. Stage Timeline — which stage filtered offers, and by how much?

2. Per-Offer Journey — for THIS customer and THIS request, why this offer and not that one?

3. Explain button — narrative prose for a human reader

When to reach for which layer

GET /api/v1/decision-traces

Query Parameters

Response

GET /api/v1/decision-traces/

Response

Per-stage timeline (UI)

Provenance deep-dive — “Why this customer got these offers”

Enabling Decision Traces

JSON Field Shapes

`scoringResults`

`selectedOffers`

`qualificationResults`

`contactPolicyResults`

`rejectedOffers`

POST /api/v1/decisions//narrative

Request Body

Response

Auth, Limits, and Tenancy

Audit Log (regulator mode)

Caching

POST /api/v1/decisions//shap

Request Body

Response

Additivity guarantee

Probability-space approximation

Auth, Limits, and Tenancy

Audit Log

Roles