Skip to main content

​
What it does

The ranking engine multiplies offer scores by a weight vector {propensity, relevance, impact, emphasis, diversity}. Operators historically set those weights manually and rarely changed them — EXP3-IX online tuning treats the weight vector as arms of a contextual bandit and updates per recorded outcome. Reference: Neu (2015), Explore no more: Improved high-probability regret bounds for non-stochastic bandits. EXP3-IX is the variance-bounded variant of EXP3 that handles adversarial feedback within high-probability regret bounds of O(√(K·T·log K)), where K is the number of arms and T the round count.

​
Why use it

  • Optimal weight mix shifts with time-of-day, segment, channel, campaign pressure. Manually chasing optimality is lost margin.
  • The bandit converges in O(√T log K) regret to the best-fixed weight mix among the configured arms.
  • All math is online / incremental. No retraining job.

​
Configuration

Two settings live under tenantSettings.aiAnalyzerSettings.ranking:
The bandit’s running log-weights are persisted at ranking.banditState after every outcome update. That row IS the state — there is no separate table.

​
Honest limits

  • No auto-bootstrap. When banditConfig.arms is missing or empty, the bandit-state reader returns nothing and the wire is a structured no-op. Operators must explicitly configure arms.
  • Arm-index thread-through. The realtime /api/v1/recommend hot path samples an arm (selectBanditArmForRecommend) and persists banditArmIndex on the auto-recorded impression’s response payload, so /respond knows which arm to credit on outcome. Batch pipelines do NOT sample the bandit.
  • Sampled weights are now applied to ranking (fixed 2026-07-15, silent-gap audit M-03). The recommend route threads the sampled arm’s weight mix through DecisionFlowContext.banditWeights into the flow’s Score node’s formula-mode scoring, where it overrides any inline formula or resolved strategyProfileId/strategyOverrides weights for that decision — the whole point of sampling is to actually explore the alternative mix, not just log that it was picked. banditArmIndex / banditArmId are still echoed in the response and on the auto-recorded impression for /respond to credit on outcome, same as before. Before this fix, arm selection drove learning and telemetry only — the sampled mix never reached scoring, so ranking stayed on the static formula/profile regardless of which arm was picked. Only Score nodes using method: "formula" are affected; non-formula scoring methods are untouched by the bandit.
  • Reward derivation. V1 maps outcome classification to reward: positive → 1, negative/neutral → 0. More nuanced reward shaping (revenue-aware, time-discounted) is a follow-up.

​
What gets logged

Per outcome that lands in respond/route.ts with the flag on:
When the helper returns null (no arms configured) or the response payload lacks banditArmIndex, the wire silently no-ops without emitting a log line. That’s the contract — silence is healthy “no work to do.” Failures emit an ERROR bandit update failed line with the underlying message and never block the respond response.

​
Operational checklist

  • Configure 2–5 arm presets covering the operating envelope you care about. Three is a good starting point.
  • Choose gamma based on how much exploration you can tolerate. The code default (when hyperparams is omitted in initBandit) is 0.05; setting gamma = 0.1 reserves 10 % of traffic for uniform exploration.
  • Watch the bandit.update rate vs the outcome.recorded rate: they should match when the flag is on. A divergence means arm-index threading is broken upstream.
  • To rollback: flip exp3IxEnabled to false. The wire stops on the next request. The persisted banditState is not deleted — on re-enable the bandit picks up where it left off.

​
Cross-references