Gradient Boosted Trees

​

When to use

• You have several thousand labeled outcomes (5,000+ recommended for meaningful accuracy; the hard training floor is 50), especially with many feature columns.
• You suspect non-linear interactions — “income matters more for high-credit-score customers” — that logistic_regression can’t capture without manual feature engineering.
• You want SHAP-explainable predictions — TreeSHAP runs in polynomial time on tree ensembles and gives exact per-feature attributions.

​

The math

For each tree t = 1..T:
  margin_t = walkTree(root, featureVector)   # leaf value at the path's end

rawMargin = Σ_t margin_t
score     = sigmoid(rawMargin)

​

Fixture config

{
  "modelType": "gradient_boosted",
  "modelState": {
    "feature_names": ["credit_score", "income", "age"],
    "trees": [
      {
        "tree_structure": {
          "split_feature": 0,
          "threshold": 740,
          "left_child":  { "leaf_value": -0.8 },
          "right_child": {
            "split_feature": 1,
            "threshold": 75000,
            "left_child":  { "leaf_value": 0.2 },
            "right_child": { "leaf_value": 0.9 }
          }
        }
      },
      {
        "tree_structure": {
          "split_feature": 2,
          "threshold": 30,
          "left_child":  { "leaf_value": -0.3 },
          "right_child": {
            "split_feature": 1,
            "threshold": 60000,
            "left_child":  { "leaf_value": -0.1 },
            "right_child": { "leaf_value": 0.6 }
          }
        }
      }
    ]
  }
}

​

Hyperparameters

​

Training

1. POST /api/v1/algorithm-models/{id}/train gathers labeled interactions (deduplicated, neutral outcomes skipped), enriches them with customer attributes from your schema tables, and builds a training matrix from the model’s selected predictors.
2. It calls the ML Worker’s /train/gbm endpoint with feature_names, training_data, and the hyperparams above.
3. The worker fits a LightGBM booster and returns portable tree JSON (booster.dump_model() shape). The platform persists it — plus metrics and feature_importance — into modelState.trees.
4. At decision time the Node scorer walks those trees in-process. The ML Worker is never on the /recommend hot path.

• ML_WORKER_URL must be set and the worker reachable at training time. If it’s unset or unreachable, training fails with an MLWorkerUnavailableError (surfaced as "ml-worker unavailable…"), and the model stays in its previous state — the last trained ensemble keeps scoring.
• ≥ 50 labeled interactions (MIN_TRAINING_SAMPLES) are required, otherwise training returns insufficient_data.
• At least one selected predictor is required — with none, training errors with “No predictors selected.”

​

Score interpretation

• score ∈ [0, 1] — calibrated probability (well-calibrated when the ensemble has enough trees and isotonic post-calibration was applied during training).
• rawMargin — the pre-sigmoid log-odds. Operators reading the trace can see how many trees voted positively vs negatively.
• explanations[] — the top ~20 contributing features along the chosen path through each tree, a lightweight SHAP-inspired approximation (not exact). For regulator-grade attributions that sum exactly to the margin, request shapValues instead.
• shapValues — full TreeSHAP attributions when computeShap: true. More expensive but exact. See SHAP.

​

Pitfalls

• Overfitting on small data — trees memorize. If you have < 1k outcomes, the test-set accuracy will be much worse than the training-set accuracy. Use early stopping during offline training.
• Categorical encoding — GBT libraries handle categoricals natively if told, but the engine’s tree format assumes numeric inputs. Pre-encode categoricals as ordinal (and let the GBT pick split points) or one-hot.
• Drift over time — tree splits are brittle to feature distribution shifts. Retrain monthly at minimum, weekly if conversion rate or customer mix is moving.
• Calibration drift after retraining — without isotonic post-calibration, the raw GBT score is a margin, not a probability. Run isotonic on a held-out set to keep sigmoid(margin) calibrated.
• Large model JSON — a 500-tree ensemble can be 5–50 MB. Watch modelState size; the engine reads it on every score call. Consider downsampling trees or using leaf quantization.

​

Lifecycle & cadence

​

Cross-reference

• Algorithm Selection Guide.
• SHAP — TreeSHAP is exact and fast for this algorithm.
• Logistic Regression — try this first; promote to GBT only if it materially beats logistic on hold-out.