> ## Documentation Index
> Fetch the complete documentation index at: https://docs.kaireonai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Negotiation eval harness — nightly CI

> W7.2 — runs lib/negotiation/eval-harness.ts against a deterministic synthetic dataset every night and fails the workflow on regressions.

## What it does

A nightly script — `tools/scripts/run-negotiation-eval.ts`, run from the
`platform/` working directory — drives the negotiation eval harness
(`runDefaultNegotiationEval`) with a fixed seed (default 7), prints a
JSON summary, and writes the full report to
`platform/perf/baselines/<date>-negotiation-eval.json`.

The script's exit code is the regression signal:

| Exit | Condition                                                                                                 |
| ---- | --------------------------------------------------------------------------------------------------------- |
| 0    | `report.zeroViolationClearance === true` — every row passed AND every violation code fired at least once. |
| 1    | Any row failed OR uncovered violation codes exist.                                                        |

## When it runs

`.github/workflows/nightly-eval.yml` schedules the script at
**06:17 UTC daily**. The cron offset (17 minutes past) deconflicts
with other cron-driven jobs in the deployment.

`workflow_dispatch:` is also enabled so operators can re-run on
demand.

## What gets archived

Each run uploads `platform/perf/baselines/*-negotiation-eval.json`
as the workflow artifact `negotiation-eval-<run_id>` with 30-day
retention. Operators can compare today's report against a 7-day-prior
artifact to spot drift even when the daily exit code stays green.

## Honest limits

* Synthetic-only dataset. Production negotiation transcripts are not
  fed into this harness — they would expose customer PII and require
  red-team review before use.
* On failure the workflow posts to Slack via its `Notify Slack on failure`
  step **when the `SLACK_WEBHOOK_URL` repo secret is set**; when the secret
  is absent that step is skipped and operators fall back to the standard
  GitHub Actions failure email. The eval script itself never POSTs anywhere —
  notification lives entirely in the workflow.
* Default seed is 7, set as the `NEGOTIATION_EVAL_SEED` env on the workflow's
  run step. It is **not** exposed as a `workflow_dispatch` input — to
  investigate a flake, run the script locally with a different value
  (`NEGOTIATION_EVAL_SEED=42 npx tsx ../tools/scripts/run-negotiation-eval.ts`)
  or edit that env in the workflow.

## Cross-references

* Negotiation guardrail engine: `lib/negotiation/guardrails.ts`.
* Apply-mode + multi-turn (`lib/negotiation/apply-mode.ts`,
  `lib/negotiation/multi-turn.ts`) are implemented and wired into the
  recommend hot-path (`lib/negotiation/realtime-apply.ts`) plus a dedicated
  `POST /api/v1/decisions/:id/negotiate/apply` route — but stay behind
  default-off gates: the tenant apply-mode flag, offer-level `negotiable`,
  guardrail validation, a two-sided kill switch, a per-tenant/offer
  apply-rate budget, and optional regulator-review sign-off.