Skip to main content

​
Deployment Tiers

Choose a tier based on your traffic volume, latency requirements, and budget.
Monthly cost, decisions/sec, and P99 latency in this table are planning estimates to size a tier — not benchmarked guarantees. Actual figures depend on cloud pricing, pipeline complexity, candidate-offer count, and enrichment/model configuration. Benchmark on your own workload before committing to a tier.
The Hobby tier is perfect for evaluation and proof-of-concept work. You can scale up to Startup with minimal configuration changes when you are ready for production traffic.

​
Background Jobs (BullMQ)

Several features run asynchronously through BullMQ queues backed by Redis: DSAR exports, model retraining, journey wait-step advancement, drift detection, batch decisioning, and sample-data seeding. Each job needs two halves to work:
  1. A producer — the API enqueues jobs into Redis. ✅ Always on, regardless of tier.
  2. A consumer — a Node.js process running BullMQ Worker instances that read jobs back out and execute the handlers. Required, otherwise queued jobs silently pile up forever in Redis.
KaireonAI ships two ways to run the consumer:
Hobby tier guarantee: the $6-8/mo Hobby figure assumes WORKER_INPROCESS=1. The single App Runner service hosts both producer and consumer. No second container, no second bill.
If you run a dedicated kaireon-worker container, you must set WORKER_INPROCESS=0 on the API service. Otherwise both processes will compete for the same jobs (BullMQ prevents double-execution via job locks, but it’s wasteful).

​
Redis backend portability

Any Redis-protocol-compatible service works. The codebase has zero Upstash-specific dependencies — it uses plain ioredis. To switch providers, change REDIS_URL and restart: For BullMQ Cluster or Sentinel HA setups, a small wrapper around new Redis(url) is needed — see Self-Hosted Deployment for the Helm value.

​
Optional Services

​
ML Worker (Python)

The ML Worker is a separate Python/FastAPI service required for training gradient_boosted models and for higher-accuracy scikit-learn analysis on datasets > 5K rows. Scoring trained models works without it — the tree ensemble runs in-process in Node.js, so the /recommend hot path never calls Python. Without the ML Worker:
  • ✅ /recommend works for all model types (scoring is in-process)
  • ✅ Scorecard, Bayesian, linear, and logistic models train in-process
  • ❌ gradient_boosted model training returns a 503 (ML_WORKER_URL is not configured)
  • ⚠️ Auto-Segmentation, Policy Recommender, and Content Intelligence fall back to LLM-based analysis (still functional, lower accuracy on > 5K rows)
See ML Worker Setup for deployment instructions.

​
Response Time Breakdown

What happens during a /api/v1/recommend call:
These per-stage durations are approximate estimates for a typical pipeline with 50-100 candidate offers — not benchmarked figures. Latency scales roughly linearly with candidate count and varies with enrichment and model configuration.

​
Scaling Levers

Add more App Runner or ECS instances. Each instance handles approximately 100-500 req/s depending on pipeline complexity. App Runner auto-scales based on concurrency — tune the auto-scaling configuration’s max-concurrency-per-instance value to control when new instances spin up.
A larger database instance reduces query time. This is most impactful when qualification rules or enrichment queries are complex. Moving from t3.micro to r6g.large can cut DB-bound latency by 60-70%.
Enrichment caching reduces database load by 80%+. A cache hit resolves in ~1-2ms versus ~10-30ms for a cache miss. Set the REDIS_URL environment variable to enable caching in production. This is the single highest-impact optimization.
Use read replicas for dashboard and analytics queries. Keep the primary instance dedicated to decision writes and real-time reads. RDS and Aurora both support up to 15 read replicas.
PgBouncer reduces PostgreSQL connection overhead and is essential at more than 50 concurrent connections. The Helm chart deploys PgBouncer by default (pgbouncer.enabled: true, transaction pooling, pool size 25). For App Runner, use Supabase’s built-in connection pooler or deploy PgBouncer separately.
For non-real-time use cases, use the /api/v1/recommend/batch endpoint with batch customer lists. Batch mode amortizes connection and parsing overhead across many decisions, achieving higher throughput at the cost of individual response latency.

​
Cost Optimization Tips

Start small

Begin with the Hobby tier for evaluation. Scale up only when you have real traffic that demands it.

Free database tiers

Supabase and Neon free tiers provide sufficient PostgreSQL capacity for development and small production workloads.

Enable Redis early

Redis caching is the single biggest performance win. Enable it before scaling compute.

Pay-per-request pricing

App Runner charges per request-second, making it cost-effective for unpredictable or bursty traffic patterns.

Graduate to containers

Move to ECS or EKS only when you need sustained throughput above 100 req/s. Container orchestration adds operational overhead.

Spot instances for workers

Use Spot instances for worker pods handling non-latency-sensitive batch processing. Spot pricing can reduce compute costs by 60-90%.