Fairness + Drift Monitoring

KaireonAI ships two production-grade model-health surfaces:

Fairness — standard audit-recognized bias metrics (demographic parity, disparate impact / four-fifths, equal opportunity, equalized odds, and Gini concentration) evaluated over any slice of decision outcomes.
Drift — Population Stability Index + two-sample Kolmogorov– Smirnov tests over feature distributions, with a multi-feature rollup verdict (none / monitor / alert).

Both are purpose-built for the EU AI Act Article 10 § 2(f) examination-for-biases requirement and for Reg B / ECOA disparate- impact demonstrations under the four-fifths rule (29 CFR § 1607.4D).

Fairness evaluation

POST /api/v1/fairness/evaluate

Two input modes: Inline — CI / regression workflows supply samples directly:

curl -X POST https://your-host/api/v1/fairness/evaluate \
  -H "Content-Type: application/json" \
  -H "X-Api-Key: $KAIREON_API_KEY" \
  -d '{
    "mode": "inline",
    "samples": [
      { "group": "A", "prediction": true,  "label": true  },
      { "group": "A", "prediction": false, "label": true  },
      { "group": "B", "prediction": true,  "label": false },
      ...
    ]
  }'

Trace-join — server pulls a decision-trace window and joins to a caller-supplied groupLookup map:

curl -X POST https://your-host/api/v1/fairness/evaluate \
  -H "Content-Type: application/json" \
  -H "X-Api-Key: $KAIREON_API_KEY" \
  -d '{
    "mode": "trace-join",
    "from": "2026-04-01T00:00:00Z",
    "to": "2026-04-22T00:00:00Z",
    "offerId": "off_premium_card",
    "groupLookup": {
      "cust_1": "female", "cust_2": "male", ...
    },
    "labelLookup": { "cust_1": true, ... }
  }'

Protected attributes live only in the request — this route never persists them beyond the audit log’s aggregate counts.

Response

{
  "sampleSize": 1000,
  "groups": [
    { "group": "female", "count": 520, "positiveDecisions": 180, "positiveRate": 0.346 },
    { "group": "male",   "count": 480, "positiveDecisions": 240, "positiveRate": 0.500 }
  ],
  "demographicParityGap": 0.154,
  "disparateImpactRatio": 0.692,
  "fourFifthsRuleViolated": true,
  "equalOpportunityGap": 0.08,
  "equalizedOddsGap": 0.12,
  "giniCoefficient": 0.071,
  "lorenzCurve": [
    { "cumulativeShare": 0, "cumulativePositiveShare": 0 },
    { "cumulativeShare": 0.48, "cumulativePositiveShare": 0.39 },
    { "cumulativeShare": 1, "cumulativePositiveShare": 1 }
  ],
  "warnings": [
    "disparate impact ratio 0.692 is below 0.80 four-fifths-rule threshold (29 CFR § 1607.4D)"
  ]
}

Metrics explained:

demographicParityGap — max − min positive-decision rate.
disparateImpactRatio — min / max. Below 0.80 triggers the four-fifths-rule flag when every group has ≥ 30 samples.
equalOpportunityGap — TPR gap across groups (requires ground- truth label on each sample).
equalizedOddsGap — the larger of the TPR-gap and FPR-gap across groups.
giniCoefficient — concentration of positive decisions across groups, computed via a size-weighted Lorenz curve (0 = every group sees the same rate, 1 = one group captures every decision). Returned alongside the lorenzCurve points.

Continuous fairness recheck

The publish-time fairness hard-gate can also run after a flow goes live. GET /api/v1/cron/fairness-recheck (Bearer CRON_SECRET) sweeps every tenant with fairnessPolicy.enabled = true and fairnessPolicy.continuousRecheck = true, re-runs the gate over the last 7 days of traces, and auto-pauses any still-active decision flow whose thresholds are breached (audit-logged as update / auto_pause with reason fairness_recheck_drift).

Drift detection

POST /api/v1/models/:id/drift

Submit two feature-value snapshots (reference + current) and the endpoint returns PSI + KS per feature plus an aggregate severity:

curl -X POST https://your-host/api/v1/models/mdl_gbm/drift \
  -H "Content-Type: application/json" \
  -H "X-Api-Key: $KAIREON_API_KEY" \
  -d '{
    "reference": {
      "recent_transaction_count": [ ... training-window values ... ],
      "tenure_months":            [ ... ]
    },
    "current": {
      "recent_transaction_count": [ ... last 24h values ... ],
      "tenure_months":            [ ... ]
    }
  }'

Response:

{
  "modelId": "mdl_gbm",
  "features": [
    {
      "feature": "recent_transaction_count",
      "psi": { "psi": 0.284, "severity": "alert", "bins": [...] },
      "ks":  { "d": 0.22, "pValue": 1e-5, "significant": true }
    }
  ],
  "overallSeverity": "alert",
  "alertFeatures": ["recent_transaction_count"]
}

PSI thresholds (Siddiqi 2005, Basel-recognized)

PSI	Severity	Action
`< 0.1`	none	no action
`0.1 – 0.25`	monitor	watch next week
`≥ 0.25`	alert	investigate / retrain

KS significance

Two-sided p-value via the Kolmogorov distribution Q(λ) with regime switching (alternating series for λ ≥ 0.3, Jacobi-theta alternative for smaller λ). significant: true when p < 0.05.

Overall severity

alert fires if any feature has PSI ≥ 0.25 OR a significant KS with D > 0.1. monitor fires if any feature crosses 0.1 PSI without alerting. Otherwise none.

Rate limits + audit

POST /fairness/evaluate — 20/min/tenant, audit-logged as action=fairness_evaluate.
POST /models/:id/drift — 30/min/tenant, audit-logged as action=drift_evaluate.

Both are tenant-scoped; cross-tenant lookup returns 404.

Get Started

Tutorials

Use Cases

Decisioning

Studio

Data Pipelines

AI & ML

Operations & Reporting

Governance & Security

Integrations

Reference

Fairness + Drift Monitoring

Fairness evaluation

POST /api/v1/fairness/evaluate

Response

Continuous fairness recheck

Drift detection

POST /api/v1/models/:id/drift

PSI thresholds (Siddiqi 2005, Basel-recognized)

KS significance

Overall severity

Rate limits + audit

​Fairness evaluation

​POST /api/v1/fairness/evaluate

​Response

​Continuous fairness recheck

​Drift detection

​POST /api/v1/models/:id/drift

​PSI thresholds (Siddiqi 2005, Basel-recognized)

​KS significance

​Overall severity

​Rate limits + audit

Fairness evaluation

POST /api/v1/fairness/evaluate

Response

Continuous fairness recheck

Drift detection

POST /api/v1/models/:id/drift

PSI thresholds (Siddiqi 2005, Basel-recognized)

KS significance

Overall severity

Rate limits + audit