Skip to main content
The Algorithms module is the machine learning layer that predicts how likely a customer is to engage with each offer. Every scoring model takes a customer-offer pair and produces a propensity score between 0 and 1. That score becomes the P (Propensity) factor in the PRIE ranking formula, which combines it with Relevance, Impact, and Emphasis to determine the final ranking. KaireonAI ships with 9 scoring engines — from a transparent scorecard you can configure in minutes (no training data needed) to gradient-boosted trees (LightGBM) and neural collaborative filtering that learns latent user-item embeddings from interaction data. You can start simple and upgrade later without changing your Decision Flows; the engine is a configuration detail, not a structural one. The module also includes a full experimentation framework with champion/challenger testing, holdout groups, and uplift measurement so you can measure real-world impact before rolling out changes.

When to Use Which Engine

Start with a Scorecard during initial setup. Once you have 100+ interactions, add a Bayesian model as a challenger. At 1,000+ interactions, test Logistic Regression via champion/challenger experiments to see if accuracy improves. At 5,000+ interactions — and with the ML Worker deployed — try Gradient Boosted for the best accuracy on non-linear patterns.

Scoring Engines

Scorecard

A weighted point system where you define rules that match customer or offer fields against conditions and award points. The raw total is normalized to 0—1 using sigmoid or linear normalization. Example (retail rewards): Award 20 points if reward_tier = "gold", 15 points if visit_frequency >= 3, 10 points if age >= 25. Supported operators: eq, neq, gt, gte, lt, lte, in, not_in, contains, starts_with. Every rule evaluation is returned in the explanations array — fully auditable. Pros: Transparent, instant setup, no training data, easy to audit. Cons: Cannot capture non-linear feature interactions, manual maintenance.

Bayesian (Naive Bayes)

An adaptive probability model that starts with a uniform prior and updates as it observes real outcomes. Each predictor contributes a log-likelihood ratio, and the posterior probability becomes the score. Laplace smoothing prevents zero-probability issues. Key features:
  • Cold-start handling: Untrained model returns 0.5 for all customers (uniform prior). Score spread increases as data arrives.
  • Incremental learning: With autoLearn: true and learnMode: "per_outcome", each recorded outcome updates the model. A RETRAIN_EVERY_N threshold (default 100) controls full recomputation frequency.
  • Training enrichment: Training enriches customer attributes from schema tables (ds_* tables) so the model learns from real features like age, income, tenure — not empty context objects.
Predictor field names are resolved against your schema-table columns in two accepted forms: the bare column name (credit_score) or the schema-qualified name (banking_customers.credit_score). Both resolve during training — use whichever your schema editor produced. What still must match is the column name itself: if your ds_customers table has household_income, reference household_income (or customers.household_income), not a generic income. A predictor whose column doesn’t exist contributes nothing and shows importance: 0.
Pros: Learns from data, handles cold-start, interpretable per-field contributions. Cons: Assumes feature independence, less accurate than tree-based models on complex data.

Logistic Regression

A linear classifier with sigmoid activation. Trained on customer features, offer features, and interaction history via full-batch gradient descent with optional L2 regularization. Fast, interpretable per-feature coefficients, and works well with 1,000+ samples. Real weights are fit only with ≥ 20 usable labeled rows spanning both classes; below that, training falls back to a metrics-only pass (weights unchanged). See the Logistic Regression algorithm page for the fit details and a caveat on how the L2 config is wired. Pros: Fast training, fully interpretable coefficients, small model footprint, in-process scoring. Cons: Can only capture linear relationships — combine with feature engineering for non-linear effects.

Gradient Boosted

A LightGBM tree ensemble — the highest-accuracy engine in the platform. Training runs in the Python ML Worker using LightGBM; the trained ensemble is serialized as portable tree JSON and scored in-process in Node, so the /recommend hot path never calls the Python service. Architecture:
  • Training: Python ml-worker receives a compact JSON payload (feature_names, training_data, hyperparams) and fits a LightGBM booster. The booster is dumped as portable tree JSON with split_feature, threshold, default_left, and leaf_value at each node.
  • Scoring: The Node scorer walks every tree per record, summing leaf values into a raw margin, then applies sigmoid. Typical latency is 5—50µs for a 100-tree ensemble. Missing values are routed via LightGBM’s native default_left convention.
  • Zero hops in decision path: The /recommend API never calls the ML Worker. Training is the only phase that does.
Requirements:
  • 5,000+ labeled interactions for meaningful accuracy (the engine will cold-start to 0.5 for everything until enough trees exist).
  • The ML Worker must be reachable at training time. See ML Worker Setup to deploy it. If ML_WORKER_URL is unset, GBM training fails with an “ML worker unavailable” error and the model stays in its previous state.
Troubleshooting:
  • ML Worker unreachable: Verify ML_WORKER_URL points to a running worker and that curl $ML_WORKER_URL/health returns status: ok. The platform exposes GET /api/v1/ml-worker/health as a probe.
  • All scores are 0.5: The model has no trained trees yet. Run Train to kick off LightGBM training.
  • Low AUC on training: Increase nEstimators, decrease learningRate, or collect more labeled interactions.
Pros: Best accuracy on non-linear structured data, captures feature interactions automatically, handles missing values natively, in-process scoring. Cons: Requires the ML Worker for training, needs 5,000+ interactions, per-tree path contributions are a lightweight SHAP-inspired approximation rather than true SHAP. Optional preprocessing (WoE binning + target encoding). Set config.preprocessing.enabled = true and the platform fits Weight-of-Evidence bins and categorical target encodings on the labeled training set (fitPreprocessing), ships them to the ML Worker alongside the training payload, and the Python trainer applies the same encoders before LightGBM sees the features (echoed back in the response’s preprocessing_used list). It is off by default so existing trains are unchanged — opt in per model when categorical features or monotonic binning help.

Thompson Bandit

A Thompson Sampling multi-armed bandit using Beta-distributed arms. Each offer is an arm with alpha (successes + 1) and beta (failures + 1). At scoring time, the engine draws from each arm’s Beta distribution — arms with higher expected reward win more often, but uncertain arms still get explored. Pros: Automatic explore/exploit balance, no feature engineering, Bayesian uncertainty. Cons: Stochastic scores (different each request), does not use customer features directly.

Epsilon-Greedy

A simpler bandit that exploits the best-known arm with probability 1 - epsilon and explores randomly with probability epsilon. Epsilon decays over time. Unpulled arms receive an optimistic score of 1.0 to encourage initial exploration. Pros: Simple, deterministic during exploitation, easy to tune. Cons: Less sample-efficient than Thompson, no uncertainty modeling.

Neural Collaborative Filtering

A two-tower embedding model with an MLP head. Customer and offer each get a learned embedding vector. At scoring time, embeddings are concatenated and passed through a hidden layer (ReLU) and output layer (sigmoid). Architecture: user_embedding + item_embedding -> hidden (ReLU) -> output (sigmoid) Training uses mini-batch SGD with binary cross-entropy loss and Xavier/Glorot initialization. Pros: Captures latent factors, handles sparse interaction matrices. Cons: Requires significant interaction data, cold-start for new users/items (falls back to zero embedding).

Online Learner

A streaming SGD logistic regression model that learns from one example at a time. Each outcome is fed back to the online-learning routine after delivery — no batch training required. Effective learning rate: lr_t = learningRate / (1 + decayRate * step). Pros: True real-time learning, no batch jobs, low memory. Cons: Linear model only, sensitive to learning rate, can be noisy.

External Model Endpoint

Call external HTTP prediction endpoints — SageMaker, Vertex AI, Azure ML, MLflow, BentoML, or any HTTP endpoint that returns scores. Example SageMaker config:
External calls add 50—200ms of network latency per recommendation. For latency-critical use cases (under 50ms), use built-in models. Enable response caching to reduce repeated calls for the same customer.

Imported ONNX Model (BYO)

Beyond the nine configurable engines, you can bring your own model by uploading an ONNX file to POST /api/v1/models/import (admin-only, multipart). The importer persists the model with modelType: "onnx_imported" and its ordered featureNames, then scores it in-process at decision time via a lazily-loaded onnxruntime-node session. This type is injected at import time — it is not one of the nine configurable modelType enum values and cannot be created through the model wizard.
  • Single ONNX file, 100 MB cap. Larger models offload to a blob store when one is configured; otherwise they store inline in modelState.
  • Fail-soft: if onnxruntime-node isn’t installed, or a scoring call throws, the engine returns 0.5 with a degraded explanation rather than breaking /recommend, and the decision sets degradedScoring = true on its trace (plus a scoring_model_failures metric increment) so the fallback is visible per-decision, not just in aggregate. (A malformed model state is the exception — it surfaces loudly.)
  • Not trained in-platform — the imported bytes are the model. Re-import to update it.

Engine Comparison


Explanation Details by Engine

When explain=true is passed to the Recommend API, each decision includes a modelExplanation object with engine-specific details. The structure of the details array varies by engine type:
Scorecard explanations are the most detailed — every rule evaluation is returned with matched/unmatched status and point contribution. This makes scorecards ideal for regulated industries that require a full audit trail of scoring decisions.

Cold Start and Propensity Smoothing

New offers have no interaction data, so ML scores are unreliable. Propensity smoothing blends the model score with a prior estimate until sufficient evidence accumulates:
(Example with priority = 70, propensitySmoothingWeight = 25)
Set higher (50—100) for extended exploration of new offers. Set lower (5—10) if you have fast feedback loops and trust the model quickly.

Model Maturity Ramp

While propensity smoothing adjusts the score, the maturity ramp adjusts the exposure. New offers start at just 2% exposure and ramp linearly to 100% as interactions accumulate:
The ramp uses a deterministic hash of customerId + offerId + date so the same customer sees consistent results within a day.
Smoothing and the maturity ramp work together. Smoothing ensures a new offer’s score is reasonable; the ramp ensures it is not shown to everyone until the model has evidence. Together they provide fair but cautious treatment for new offers.

Model Resolution Hierarchy

When a Score node executes, the engine resolves which model to use for each candidate via an override priority chain:
First match wins. Override resolution happens independently per candidate — two offers in the same request can be scored by different models.
If the resolved model is unavailable or the circuit breaker is open, the engine falls back to pre-computed propensity scores, then to priority-based scoring as a last resort.

Champion/Challenger Testing

Run a new model against your production model using live traffic with deterministic customer assignment.

Model Registry

Every model carries an indexed registryStatus column that pins it to one of four lifecycle states: Promotions run through the model-registry promotion workflow, which enforces legal transitions (draft → challenger → champion → archived → draft) and the “one champion per family” invariant in a single transaction. Every promotion writes an audit-log row keyed by entityType=algorithm_model, action=registry_promote, recoverable via DSAR or compliance review.
Self-hosters upgrading from a 2026-04-22-or-earlier deployment must add the new columns and the DB-level CHECK constraint:
Existing rows continue to work without the backfill — the read path falls back to config.registry JSON when the columns are null. Backfill is a one-time cleanup, not a hard prerequisite.

How It Works

  1. Configure the split — Set weights (e.g., 80/20 champion/challenger)
  2. Deterministic assignmentFNV-1a(customerId + ":cc") produces a stable hash. Same customer always gets the same model.
  3. Override bypass — When enabled, champion/challenger takes precedence over the normal override hierarchy.

Example: After 14 Days

The challenger shows promise but has not reached statistical significance at 95% confidence. The experiment needs more traffic.

Experiments

Experiments wrap champion/challenger testing with holdout groups, traffic management, and statistical analysis.

Creating an Experiment

Traffic split must sum to exactly 100%. The API validates championPct + sum(challenger trafficPct) = 100 and rejects the request otherwise.

Uplift Calculation

KaireonAI uses a two-proportion z-test:

Power Calculator

Before launching, estimate required sample size and duration given your baseline conversion rate, minimum detectable effect, and daily traffic volume. Returns required sample size per variant (80% power, 95% confidence) and estimated duration in days.

Auto-Promotion

Auto-promote is disabled by default. The system provides uplift magnitude, p-value, and confidence intervals, but the final promotion decision is left to the operator. Enable autoPromote: true only when you have guardrail checks and are comfortable with automated rollouts.
When enabled, auto-promotion triggers when: (1) experiment has run for at least promoteAfterDays, (2) challenger exceeds champion by at least promoteThreshold, and (3) result is statistically significant at 95% confidence.

Model Lifecycle

1

Create

Define model with key, name, engine type, and engine-specific config. Starts in draft status.
2

Configure

Set up predictors (feature fields), target field, and engine settings.
3

Train

Kick off training (requires 50+ interaction records for data-driven engines).
4

Evaluate

Review accuracy, precision, recall, F1, AUC. Compare against previous versions.
5

Promote

Set to active to make available for Decision Flows.
6

Monitor

Track in Model Health Dashboard. Scheduled drift checks auto-enqueue retraining when performance degrades.

Auto-Learning Modes

Auto-Upgrade Recommendations


Field Reference

Algorithm Model

Experiment


API Quick Reference

Score Request Example

Returns per-offer scores sorted by propensity.
For full API reference with request/response schemas and error codes, see the Algorithm Models API Reference.

Worked Example: Three Engines Score the Same Offer

A retail rewards member (income: 75000, reward_tier: gold, visit_frequency: 4/week) is evaluated for a BOGO beverage offer: All four produce a 0—1 score, but arrive at it differently. The scorecard is transparent and manual. The Bayesian adapts from data while remaining interpretable. Logistic regression adds learned linear weights. The gradient boosted ensemble captures non-linear interactions between features — at the cost of needing the ML Worker for training.

Decision Flows

See how models plug into the Score stage.

Composable Pipeline

The score node uses the same scoring resolver.

Dashboards

Monitor model health, drift, and experiment results.