VPC & on-prem deployment

​

Why on-prem

• Private VPC deployments (AWS / GCP / Azure) with no public ingress from the platform’s side
• Air-gapped clusters (no outbound egress at all)
• BYOK (bring-your-own-key) encryption and secrets

​

Prerequisites

​

Architecture options

​

Option A — Private VPC, managed add-ons

• Platform (API / worker / ml-worker): Helm chart → your K8s cluster
• Database: managed Postgres (RDS / Cloud SQL / Neon VPC)
• Cache / queue: managed Redis (ElastiCache / Memorystore / Upstash VPC)
• Secrets: AWS Secrets Manager / GCP Secret Manager / Azure Key Vault, mounted via External Secrets Operator
• Outbound traffic: optionally allowed to your LLM provider of choice; otherwise disabled

​

Option B — Fully air-gapped

• All three images mirrored to an internal registry
• Internal Postgres StatefulSet + internal Redis StatefulSet (the chart ships both)
• LLM explanations feature either disabled or pointed at an in-VPC LLM endpoint (vLLM, Ollama, self-hosted Claude via Bedrock PrivateLink, on-prem GPU box, etc.)
• All ML training happens in-cluster via the bundled kaireon-ml-worker pod
• Zero outbound egress required

​

Step-by-step (Option A)

​

1. Mirror images to your registry

# Source images (public ECR)
API=422500312304.dkr.ecr.us-east-1.amazonaws.com/kaireon-api:latest
WORKER=422500312304.dkr.ecr.eu-west-2.amazonaws.com/kaireon/worker:latest
ML=422500312304.dkr.ecr.us-east-1.amazonaws.com/kaireon-ml:latest

# Pull, retag, push to your private registry
for IMG in $API $WORKER $ML; do
  docker pull $IMG
  docker tag $IMG your-registry.internal/$(basename ${IMG%:*}):latest
  docker push your-registry.internal/$(basename ${IMG%:*}):latest
done

​

2. Verify image signatures (optional but recommended)

# Requires cosign. Images are signed keyless (GitHub OIDC) by the release
# workflow, which runs on version tags. Verify before allow-listing in your
# admission policy:
cosign verify your-registry.internal/kaireon-api:latest \
  --certificate-identity-regexp='^https://github.com/kaireonai/platform/\.github/workflows/release\.yml@refs/tags/.*$' \
  --certificate-oidc-issuer=https://token.actions.githubusercontent.com

​

3. Create a values override

namespace: kaireon-prod

api:
  image:
    repository: your-registry.internal/kaireon-api
    tag: "v1.0.0"
  replicas: 6
  hpa:
    enabled: true
    minReplicas: 6
    maxReplicas: 30
  readOnlyRootFilesystem: false   # Next.js writes cache to /app/.next/cache
  podSecurityContext:
    runAsUser: 1001
    runAsGroup: 1001
    fsGroup: 1001
  topologySpreadConstraints:
    - maxSkew: 1
      topologyKey: topology.kubernetes.io/zone
      whenUnsatisfiable: DoNotSchedule
      labelSelector:
        matchLabels:
          app.kubernetes.io/component: api
  priorityClassName: kaireon-critical

worker:
  image:
    repository: your-registry.internal/kaireon-worker
    tag: "v1.0.0"
  replicas: 3
  keda:
    enabled: true
    minReplicas: 3
    maxReplicas: 20

mlWorker:
  enabled: true                    # required for gradient_boosted training
  image:
    repository: your-registry.internal/kaireon-ml
    tag: "v1.0.0"

database:
  mode: external
  external:
    host: kaireon-db.xxx.eu-west-1.rds.amazonaws.com
    port: 5432
    name: kaireon
    username: kaireon
    existingSecret: kaireon-db-credentials
    secretKey: password
    sslMode: require

redis:
  mode: external
  external:
    host: kaireon-cache.xxx.0001.euw1.cache.amazonaws.com
    port: 6379
    tls: true
    existingSecret: kaireon-redis-credentials

ingress:
  enabled: true
  className: alb
  host: kaireon.internal.example.com
  aws:
    scheme: internal        # internet-facing | internal — use internal for a private VPC
    targetType: ip
    certificateArn: ""      # ACM certificate ARN for HTTPS (optional)

​

4. Apply the chart

helm upgrade --install kaireon ./helm \
  -n kaireon-prod --create-namespace \
  -f values-prod.yaml \
  --wait --timeout 10m

# Verify
kubectl -n kaireon-prod get pods
kubectl -n kaireon-prod rollout status deployment/kaireon-api
kubectl -n kaireon-prod exec deploy/kaireon-api -- curl -sf localhost:3000/api/health

​

5. Schema sync

​

Hardening checklist

• readOnlyRootFilesystem: true where workload permits (requires adding an emptyDir volume for /app/.next/cache on the API pod)
• runAsNonRoot: true on every pod (enabled by default in the chart)
• allowPrivilegeEscalation: false + all capabilities dropped (enabled by default)
• topologySpreadConstraints with whenUnsatisfiable: DoNotSchedule across 3+ zones
• A pod-disruption-budget preserves N-1 availability during drains (shipped in templates/pdb.yaml)
• A network policy blocks all pod-to-pod traffic except allow-listed flows (shipped in templates/networkpolicy.yaml; review for your CNI)
• Secrets sourced from AWS Secrets Manager via the External Secrets Operator, not plain kubernetes.io/Secret objects (set secrets.provider: aws-secrets-manager in values.yaml)
• Database SSL mode require or higher
• Redis TLS enabled with rediss:// and password
• Admission controller enforcing signed images (cosign + Kyverno / OPA Gatekeeper)
• Log sink configured (Fluent Bit → CloudWatch / Loki / Splunk)
• Metrics scrape configured (Prometheus ServiceMonitor — the chart ships Grafana dashboards in helm/dashboards/)
• Backup policy on the Postgres instance (PITR ≥ 7 days)
• CONNECTOR_ENCRYPTION_KEY is a 32-byte random, rotated every 90 days, and stored in your secrets backend

​

Air-gapped (Option B) additions

• Set database.mode: internal and redis.mode: internal — the chart provisions StatefulSets with local storage
• Set config.EVENT_PUBLISHER: redis and config.INTERACTION_STORE: pg (no cloud-backed stores)
• Disable llmExplanationsEnabled at the tenant level, OR deploy an in-VPC LLM and configure its endpoint via the AI provider settings (Ollama / vLLM / Bedrock PrivateLink)
• Mirror the ml-worker Python dependencies to an internal PyPI proxy (the Dockerfile bakes them into the image, so this is only needed if you rebuild)

​

Upgrades

# Preview what will change
helm diff upgrade kaireon ./helm -n kaireon-prod -f values-prod.yaml

# Apply
helm upgrade kaireon ./helm -n kaireon-prod -f values-prod.yaml --wait

​

Observability

• API overview — request rate, p50/p95/p99 latency, 4xx/5xx split
• Decision engine — recommend latency breakdown by stage (enrich / compute / filter / score / rank)
• Decision performance — offer CTR, ranking weight drift, experiment uplift
• Model health — AUC trend per model, drift PSI, training sample count
• Worker queues — BullMQ depth per queue, retry count, DLQ depth
• Infrastructure — CPU / memory / disk / network per pod

​

Support

• Docs: docs.kaireonai.com
• Enterprise licensees get a dedicated support channel and access to the release repository as part of their agreement.
• For Enterprise support contracts and on-prem licensing, email sales@kaireonai.com.