Skip to main content
Status as of 2026-05-03 PM. Import endpoint, runtime, integrity check, audit trail, multi-input + out-of-band blob, AND scoring dispatch are now all shipped. The pipeline runner branches on modelType === "onnx_imported" and awaits the async ONNX scoring path, fail-soft to score 0.5 with a degraded-explanation marker when onnxruntime-node is not installed in the deployment image. Install the dep explicitly when a tenant enables ONNX BYO: npm install onnxruntime-node@^1.20.

​
Import endpoint

POST /api/v1/models/import — multipart form with these parts: Auth: tenant + admin role. The bytes are persisted base64-encoded inside the model’s modelState.onnxBytesBase64 slot, along with a sha256 digest at bytesHashSha256 so the runner can verify integrity at scoring time.

​
Scoring

The ONNX scoring runner exposes a per-instance score function that loads the ONNX session lazily through Node’s dynamic-require shim. The runtime dep is not in the platform’s package.json by default — install it explicitly when a tenant enables ONNX BYO:
The scoring barrel exposes async wrappers for per-instance and batch ONNX scoring. The pipeline runner branches on modelType === "onnx_imported" at both per-candidate scoring sites (the formula PRIE path and the non-formula model fallback) and awaits the async path instead of the sync default scorer. When the runtime dep is missing, the ONNX scorer is fail-soft to score 0.5 and adds a degraded-explanation marker ({ engineType: "onnx_imported", degraded: true, reason: "onnx_runtime_missing" }) to the trace so the operator can detect the missing-install state via audit-log analytics. A malformed-model-state error still bubbles — that’s a configuration bug that should surface in the import audit.

​
Honest limits

  • Multi-input + out-of-band blob store supported by the scoring runner (was single-input only in earlier releases). Float32 + int64 + bool tensor dtypes only — float64 / fp16 / string tensors are not supported. Note: the /models/import endpoint persists a single-input model via featureNames; the multi-input inputs form is a runner capability consumed from modelState.inputs when present.
  • 100 MB hard cap per uploaded file — larger uploads are rejected (400). Files above the inline threshold (ONNX_INLINE_BYTES_LIMIT, default 10 MB) are offloaded to the out-of-band blob store instead of the modelState JSON column, but only when ONNX_BLOB_STORE_URL is configured (file:// or s3://); otherwise they are stored inline.
  • In-memory session cache; cache eviction relies on process restart. Keep V1 deployments to <50 ONNX models per tenant.
  • CPU-only inference. No GPU support.
  • The onnxruntime-node dep is opt-in. When absent, scoring degrades to 0.5 + degraded-explanation marker rather than throwing — the fail-soft path keeps /recommend responsive for tenants that don’t use ONNX BYO.

​
Roadmap

  • Streaming-input models (multi-call session reuse) for transformer use cases.
  • GPU runtime via onnxruntime-node GPU build.

​
Audit trail

Every import writes an audit log row with action: "model_import_onnx", entityType: "algorithm_model", and changes: { family, bytesHashSha256, size, featureCount }. DSAR exports cite the import audit row, the model’s bytesHashSha256, and (when the runtime is installed at scoring time) each scoring decision back to the trace’s scoringResults[].modelType: "onnx_imported" entry.