> ## Documentation Index
> Fetch the complete documentation index at: https://docs.kaireonai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Gradient Boosted Trees

> Tree ensemble with sigmoid output. Captures non-linear feature interactions. Best raw accuracy on tabular data when you have enough outcomes. SHAP-explainable.

`modelType: "gradient_boosted"` — a LightGBM sum-of-trees ensemble (gradient boosting machine). Each tree contributes a small additive margin; the final sigmoid converts the cumulative margin to a probability. Training runs in the Python [ML Worker](/self-host/deploy/ml-worker); scoring walks the trees in-process in Node. The accuracy ceiling on tabular data is hard to beat with anything that's not also a tree ensemble.

## When to use

* **You have several thousand labeled outcomes** (5,000+ recommended for meaningful accuracy; the hard training floor is 50), especially with many feature columns.
* **You suspect non-linear interactions** — "income matters more for high-credit-score customers" — that logistic\_regression can't capture without manual feature engineering.
* **You want SHAP-explainable predictions** — TreeSHAP runs in polynomial time on tree ensembles and gives exact per-feature attributions.

**Skip it when** the dataset is small (\< 1k outcomes) — trees overfit unless you have enough samples per leaf. Use `logistic_regression` or `bayesian` first.

## The math

```
For each tree t = 1..T:
  margin_t = walkTree(root, featureVector)   # leaf value at the path's end

rawMargin = Σ_t margin_t
score     = sigmoid(rawMargin)
```

Each tree is grown by gradient descent on the loss function (binary cross-entropy by default), where the gradient at each step is the negative residual of the previous ensemble. Each leaf stores a single numeric value (the contribution to the margin).

## Fixture config

A minimal 2-tree ensemble (the proof script uses this exact fixture):

```json theme={null}
{
  "modelType": "gradient_boosted",
  "modelState": {
    "feature_names": ["credit_score", "income", "age"],
    "trees": [
      {
        "tree_structure": {
          "split_feature": 0,
          "threshold": 740,
          "left_child":  { "leaf_value": -0.8 },
          "right_child": {
            "split_feature": 1,
            "threshold": 75000,
            "left_child":  { "leaf_value": 0.2 },
            "right_child": { "leaf_value": 0.9 }
          }
        }
      },
      {
        "tree_structure": {
          "split_feature": 2,
          "threshold": 30,
          "left_child":  { "leaf_value": -0.3 },
          "right_child": {
            "split_feature": 1,
            "threshold": 60000,
            "left_child":  { "leaf_value": -0.1 },
            "right_child": { "leaf_value": 0.6 }
          }
        }
      }
    ]
  }
}
```

Produces `score=0.8176` with `rawMargin=1.50` for the standard test customer. The path-contribution explanation correctly identifies `income` as the dominant feature (it appears in both trees as a deep split).

## Hyperparameters

These live on `model.config` and are passed straight through to LightGBM at training time (field names mirror LightGBM's Python API):

| Config key        | Default | Meaning                                                   |
| ----------------- | ------- | --------------------------------------------------------- |
| `numLeaves`       | `31`    | Max leaves per tree — the main capacity/overfitting knob. |
| `maxDepth`        | `-1`    | Depth cap; `-1` lets LightGBM decide from `numLeaves`.    |
| `learningRate`    | `0.05`  | Shrinkage per tree; lower needs more estimators.          |
| `nEstimators`     | `100`   | Number of boosting trees.                                 |
| `minChildSamples` | `20`    | Minimum data points per leaf — higher regularizes.        |
| `regAlpha`        | `0`     | L1 regularization on leaf weights.                        |
| `regLambda`       | `0`     | L2 regularization on leaf weights.                        |
| `subsample`       | `1.0`   | Row sampling ratio per tree.                              |
| `colsampleBytree` | `1.0`   | Feature sampling ratio per tree.                          |

## Training

Unlike the other built-in engines, gradient boosting trains **out-of-process** in the Python [ML Worker](/self-host/deploy/ml-worker) — LightGBM tree-building is too heavy for the Node request path. The platform orchestrates it for you:

1. `POST /api/v1/algorithm-models/{id}/train` gathers labeled interactions (deduplicated, neutral outcomes skipped), enriches them with customer attributes from your schema tables, and builds a training matrix from the model's selected predictors.
2. It calls the ML Worker's `/train/gbm` endpoint with `feature_names`, `training_data`, and the `hyperparams` above.
3. The worker fits a LightGBM booster and returns portable tree JSON (`booster.dump_model()` shape). The platform persists it — plus metrics and `feature_importance` — into `modelState.trees`.
4. At decision time the Node scorer walks those trees **in-process**. The ML Worker is never on the `/recommend` hot path.

**Requirements & failure modes:**

* **`ML_WORKER_URL` must be set** and the worker reachable at training time. If it's unset or unreachable, training fails with an `MLWorkerUnavailableError` (surfaced as `"ml-worker unavailable…"`), and **the model stays in its previous state** — the last trained ensemble keeps scoring.
* **≥ 50 labeled interactions** (`MIN_TRAINING_SAMPLES`) are required, otherwise training returns `insufficient_data`.
* **At least one selected predictor** is required — with none, training errors with *"No predictors selected."*

You can also set `config.preprocessing.enabled = true` to fit Weight-of-Evidence bins + categorical target encodings on the TypeScript side and have the worker apply them before LightGBM sees the features (off by default).

<Note>
  Manually swapping a pre-built LightGBM export into `modelState.trees` via `PUT /api/v1/algorithm-models/{id}` also works — the tree JSON shape in the fixture above matches `booster.dump_model()`. But the managed `POST …/train` flow above is the supported path.
</Note>

## Score interpretation

* `score` ∈ `[0, 1]` — calibrated probability (well-calibrated when the ensemble has enough trees and isotonic post-calibration was applied during training).
* `rawMargin` — the pre-sigmoid log-odds. Operators reading the trace can see how many trees voted positively vs negatively.
* `explanations[]` — the top \~20 contributing features along the **chosen path** through each tree, a lightweight SHAP-inspired approximation (not exact). For regulator-grade attributions that sum exactly to the margin, request `shapValues` instead.
* `shapValues` — full TreeSHAP attributions when `computeShap: true`. More expensive but exact. See [SHAP](/ai-ml/shap).

## Pitfalls

* **Overfitting on small data** — trees memorize. If you have \< 1k outcomes, the test-set accuracy will be much worse than the training-set accuracy. Use early stopping during offline training.
* **Categorical encoding** — GBT libraries handle categoricals natively if told, but the engine's tree format assumes numeric inputs. Pre-encode categoricals as ordinal (and let the GBT pick split points) or one-hot.
* **Drift over time** — tree splits are brittle to feature distribution shifts. Retrain monthly at minimum, weekly if conversion rate or customer mix is moving.
* **Calibration drift after retraining** — without isotonic post-calibration, the raw GBT score is a margin, not a probability. Run isotonic on a held-out set to keep `sigmoid(margin)` calibrated.
* **Large model JSON** — a 500-tree ensemble can be 5–50 MB. Watch `modelState` size; the engine reads it on every score call. Consider downsampling trees or using leaf quantization.

## Lifecycle & cadence

Gradient-boosted is **offline-retrain only** and is the most expensive model type to retrain (multi-second to minute-scale wall time depending on tree count and data volume). Default `autoLearn: false` means a freshly-created GBM model is frozen at whatever ensemble was in `modelState.trees` at creation — typically nothing, which is why a newly-created GBM scores `0.5` for every candidate.

To enable: `PUT { "autoLearn": true, "learnMode": "scheduled", "learnSchedule": "24h" }`. Weekly (`"7d"`) is acceptable for stable workloads; daily is the default. **Pair with drift checks** — GBT splits are brittle to feature-distribution shifts, and the drift-triggered retrain path catches sudden changes between scheduled runs. Always run a one-off `POST /algorithm-models/{id}/train` before flipping `status: "active"`, otherwise the model goes live with no trees. See [Learning cadence](/ai-ml/learning-cadence) and [Model lifecycle](/ai-ml/model-lifecycle).

## Cross-reference

* [Algorithm Selection Guide](/decisioning/algorithm-selection-guide).
* [SHAP](/ai-ml/shap) — TreeSHAP is exact and fast for this algorithm.
* [Logistic Regression](/ai-ml/algorithms/logistic-regression) — try this first; promote to GBT only if it materially beats logistic on hold-out.