Security Hardening

This document provides a comprehensive security checklist and implementation guidance for hardening KaireonAI in production environments.

1. Checklist Summary

Use this checklist to track your hardening progress. Each area links to the detailed section below.

Area	Priority	Section
TLS / HTTPS	P0	Details
Security Headers	P0	Details
Rate Limiting	P0	Details
Secret Management	P0	Details
Network Policies	P1	Details
Pod Security Standards	P1	Details
Database SSL	P0	Details
Audit Log Retention	P1	Details
RBAC	P1	Details
Dependency Scanning	P2	Details
Provenance Signing (cosign)	P0	Provenance signing install guide — install via AWS Secrets Manager (cloud) or local key file (self-host). Without it, every `/api/v1/decisions/:id/provenance` response is `unsigned`.

2. TLS / HTTPS Everywhere

All traffic to and within the KaireonAI platform must be encrypted in transit.

2.1 Ingress TLS Termination

Use an AWS ALB or nginx-ingress controller with TLS certificates managed by cert-manager. nginx-ingress with cert-manager:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kaireon-ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - app.kaireon.example.com
        - api.kaireon.example.com
      secretName: kaireon-tls
  rules:
    - host: app.kaireon.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: kaireon-app
                port:
                  number: 3000

AWS ALB Ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kaireon-ingress
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:ACCOUNT:certificate/CERT-ID
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    alb.ingress.kubernetes.io/target-type: ip
spec:
  ingressClassName: alb
  rules:
    - host: app.kaireon.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: kaireon-app
                port:
                  number: 3000

2.2 Internal Service TLS

For service-to-service communication within the cluster, consider a service mesh (Istio or Linkerd) for automatic mTLS. At minimum, ensure no sensitive data traverses unencrypted paths.

3. Security Headers

Configure the following HTTP security headers on all responses. These can be set at the ingress level or in the Next.js application middleware.

3.1 Ingress-Level Configuration (nginx)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      more_set_headers "Strict-Transport-Security: max-age=63072000; includeSubDomains; preload";
      more_set_headers "X-Frame-Options: DENY";
      more_set_headers "X-Content-Type-Options: nosniff";
      more_set_headers "X-XSS-Protection: 0";
      more_set_headers "Referrer-Policy: strict-origin-when-cross-origin";
      more_set_headers "Permissions-Policy: camera=(), microphone=(), geolocation=()";

3.2 Content Security Policy

Apply CSP to prevent XSS and data injection attacks. Adjust script-src and style-src directives based on your CDN and third-party integrations.

Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; img-src 'self' data: https:; font-src 'self' data:; connect-src 'self' https://api.kaireon.example.com; frame-ancestors 'none'; base-uri 'self'; form-action 'self';

Note: unsafe-inline and unsafe-eval in script-src are necessary for Next.js in production. Use nonce-based CSP if your deployment supports it.

3.3 Header Reference

Header	Value	Purpose
Strict-Transport-Security	max-age=63072000; includeSubDomains; preload	Enforce HTTPS for 2 years
X-Frame-Options	DENY	Prevent clickjacking
X-Content-Type-Options	nosniff	Prevent MIME-type sniffing
X-XSS-Protection	0	Disable legacy XSS filter (use CSP)
Referrer-Policy	strict-origin-when-cross-origin	Limit referrer information leakage
Permissions-Policy	camera=(), microphone=(), geolocation=()	Disable unnecessary browser features
Content-Security-Policy	(see above)	Prevent XSS and injection attacks

4. Rate Limiting

Protect API endpoints from abuse and denial-of-service attacks.

4.1 nginx-ingress Rate Limiting

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/limit-rps: "20"
    nginx.ingress.kubernetes.io/limit-burst-multiplier: "5"
    nginx.ingress.kubernetes.io/limit-connections: "10"
    nginx.ingress.kubernetes.io/limit-whitelist: "10.0.0.0/8"

4.2 Per-Endpoint Rate Limits

For granular control, implement rate limiting in the application layer using the Redis-backed rate limiter.

Endpoint Pattern	Rate Limit	Window	Notes
`/api/v1/decisions`	100 req/sec	Per IP	Decision scoring
`/api/v1/auth/*`	10 req/min	Per IP	Authentication endpoints
`/api/v1/connectors`	30 req/min	Per user	Connector CRUD
`/api/v1/pipelines/*/run`	5 req/min	Per user	Pipeline execution
All other `/api/v1/*`	60 req/min	Per user	General API

4.3 Response Headers

Include rate limit information in responses to help clients self-regulate:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 87
X-RateLimit-Reset: 1708700000
Retry-After: 30

5. Secret Management

5.1 AWS Secrets Manager + External Secrets Operator (ESO)

Never store secrets in Kubernetes manifests, Helm values, or environment variable files in source control. ExternalSecret resource:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: kaireon-secrets
  namespace: kaireon
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: ClusterSecretStore
  target:
    name: kaireon-secrets
    creationPolicy: Owner
  data:
    - secretKey: DATABASE_URL
      remoteRef:
        key: kaireon/production/database
        property: url
    - secretKey: NEXTAUTH_SECRET
      remoteRef:
        key: kaireon/production/auth
        property: nextauth_secret
    - secretKey: JWT_SIGNING_SECRET
      remoteRef:
        key: kaireon/production/auth
        property: jwt_signing_secret
    - secretKey: CONNECTOR_ENCRYPTION_KEY
      remoteRef:
        key: kaireon/production/encryption
        property: connector_key

5.2 Secret Rotation Policy

Secret	Rotation Frequency	Method
DATABASE_URL (password)	90 days	Secrets Manager automatic rotation
NEXTAUTH_SECRET	180 days	Manual rotation with rolling deploy
JWT_SIGNING_SECRET	180 days	Dual-key validation during rotation
CONNECTOR_ENCRYPTION_KEY	365 days	Re-encrypt connectors during rotation
TLS certificates	60 days	cert-manager auto-renewal

5.3 Rotation Procedure

Create the new secret value in AWS Secrets Manager.
Update the ESO ExternalSecret if the remote ref path changed.
Wait for ESO to sync (up to refreshInterval), or force sync: kubectl annotate externalsecret kaireon-secrets force-sync=$(date +%s) --overwrite.
Perform a rolling restart: kubectl rollout restart deployment/kaireon-app -n kaireon.
Verify application health after restart.
Remove the old secret version from Secrets Manager after confirming stability.

6. Network Policies

Restrict pod-to-pod communication to only what is necessary.

6.1 Default Deny All

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: kaireon
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

6.2 Allow Application Traffic

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-app-traffic
  namespace: kaireon
spec:
  podSelector:
    matchLabels:
      app: kaireon-app
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              name: ingress-nginx
      ports:
        - port: 3000
          protocol: TCP
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: postgresql
      ports:
        - port: 5432
          protocol: TCP
    - to:
        - podSelector:
            matchLabels:
              app: redis
      ports:
        - port: 6379
          protocol: TCP
    - to:  # DNS resolution
        - namespaceSelector: {}
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - port: 53
          protocol: UDP
        - port: 53
          protocol: TCP
    - to:  # External HTTPS (connectors, OIDC providers)
        - ipBlock:
            cidr: 0.0.0.0/0
            except:
              - 10.0.0.0/8
              - 172.16.0.0/12
              - 192.168.0.0/16
      ports:
        - port: 443
          protocol: TCP

7. Pod Security Standards

Apply Kubernetes Pod Security Standards to prevent privilege escalation.

7.1 Namespace Label (Restricted)

kubectl label namespace kaireon \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/audit=restricted \
  pod-security.kubernetes.io/warn=restricted

7.2 Pod Security Context

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kaireon-app
spec:
  template:
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        fsGroup: 1000
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: app
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop:
                - ALL
          volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: nextjs-cache
              mountPath: /app/.next/cache
      volumes:
        - name: tmp
          emptyDir: {}
        - name: nextjs-cache
          emptyDir: {}

8. Database SSL

8.1 RDS SSL Configuration

Ensure all database connections use SSL by appending sslmode=require (or sslmode=verify-full) to the connection string.

DATABASE_URL=postgresql://$DB_USER:$DB_PASSWORD@kaireon-db.abc123.us-east-1.rds.amazonaws.com:5432/kaireon?sslmode=require&sslrootcert=/app/certs/rds-ca.pem

8.2 Certificate Bundle

Download the RDS CA certificate bundle and mount it into application pods:

curl -o rds-ca.pem https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem
kubectl create configmap rds-ca-cert --from-file=rds-ca.pem -n kaireon

8.3 In-Cluster PostgreSQL

For the Startup tier running in-cluster PostgreSQL, enable SSL in the Helm values:

postgresql:
  tls:
    enabled: true
    autoGenerated: true

9. Audit Log Retention

9.1 Application Audit Logs

KaireonAI emits structured audit log entries for all state-changing API operations. These logs include:

Timestamp (ISO 8601)
User identity (user ID, email, IP address)
Action performed (CREATE, UPDATE, DELETE)
Resource type and identifier
Request body hash (not the body itself, for sensitive data protection)
Response status code

9.2 Retention Policy

Log Type	Retention Period	Storage
Application audit logs	1 year	CloudWatch Logs / S3 lifecycle
API access logs	90 days	CloudWatch Logs
Database query logs	30 days	RDS CloudWatch integration
Kubernetes audit logs	90 days	CloudWatch Logs
Security event logs	2 years	S3 Glacier after 90 days

9.3 Log Forwarding

Use Fluent Bit as a DaemonSet to ship logs from pods to CloudWatch Logs or an external SIEM:

[OUTPUT]
    Name              cloudwatch_logs
    Match             kube.kaireon.*
    region            us-east-1
    log_group_name    /kaireon/production/app
    log_stream_prefix pod-
    auto_create_group true

9.4 Tamper Protection

Enable CloudWatch Logs log group data protection for PII detection.
Use S3 Object Lock (compliance mode) for long-term audit log archives.
Enable CloudTrail for AWS API audit trail.

10. RBAC and Least-Privilege Access

10.1 Kubernetes RBAC

Create dedicated ServiceAccounts for each component. Do not use the default ServiceAccount.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: kaireon-app
  namespace: kaireon
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT:role/kaireon-app-role
automountServiceAccountToken: false
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: kaireon-app-role
  namespace: kaireon
rules:
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get", "list"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get"]
    resourceNames: ["kaireon-secrets"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: kaireon-app-binding
  namespace: kaireon
subjects:
  - kind: ServiceAccount
    name: kaireon-app
    namespace: kaireon
roleRef:
  kind: Role
  name: kaireon-app-role
  apiGroup: rbac.authorization.k8s.io

10.2 IAM Roles for Service Accounts (IRSA)

Map Kubernetes ServiceAccounts to IAM roles with minimal permissions:

ServiceAccount	IAM Role	Permissions
kaireon-app	kaireon-app-role	Secrets Manager read, S3 read/write (uploads)
kaireon-worker	kaireon-worker-role	S3 read/write, SQS send/receive
external-secrets	kaireon-eso-role	Secrets Manager read

10.3 Database Access Control

Create separate database roles for the application and workers with the minimum required privileges.
The application role should have SELECT, INSERT, UPDATE, DELETE on application tables.
The worker role should have SELECT, INSERT, UPDATE on pipeline-related tables only.
No role should have DROP, CREATE, or ALTER privileges in production. Schema migrations run as a separate CI/CD step with a privileged role.

11. Additional Hardening

11.1 Dependency Scanning

Run npm audit in CI pipelines. Fail builds on critical or high severity vulnerabilities.
Use Snyk or Trivy for container image scanning before pushing to ECR.
Enable Amazon ECR image scanning on push.

11.2 Image Provenance

Use specific image tags, never latest.
Sign container images with cosign and verify signatures in admission controllers.
Pull images only from trusted registries (ECR, Docker Hub official images).

11.3 Runtime Protection

Deploy Falco for runtime anomaly detection (unexpected process execution, file access).
Enable AWS GuardDuty for EKS runtime threat detection.
Set readOnlyRootFilesystem: true on all containers (writable paths via emptyDir mounts).

12. API Key Authentication

KaireonAI supports API key authentication as an alternative to JWT Bearer tokens.

How It Works

API keys are passed via the X-API-Key header
Keys are stored as SHA-256 hashes in the ApiKey database table (never in plaintext)
Each key has: name, hashedKey, prefix (first 8 chars for identification), tenantId, role, expiresAt, revokedAt
Keys use the krn_ prefix for easy identification in logs

Production Checklist

Rotate API keys every 90 days
Set expiresAt on all keys — never issue non-expiring keys in production
Monitor kaireon_http_request_duration_seconds filtered by API key usage patterns
Revoke keys immediately when team members leave (set revokedAt)
Use separate keys per integration (never share keys across services)

Key Management API

POST   /api/v1/api-keys          — Create new key (returns plaintext once)
GET    /api/v1/api-keys          — List keys (hashed, no plaintext)
DELETE /api/v1/api-keys/:id      — Revoke a key

13. Connector Credential Encryption

All connector authentication configurations (authConfig field) are encrypted at rest using AES-256-GCM.

Configuration

Set the CONNECTOR_ENCRYPTION_KEY environment variable to a 32-byte hex string:

# Generate a key
openssl rand -hex 32

# Set in environment or Helm values
CONNECTOR_ENCRYPTION_KEY=<64-char-hex-string>

How It Works

On POST /api/v1/connectors and PUT /api/v1/connectors/:id, the authConfig JSON is encrypted before storage
On GET /api/v1/connectors, credentials are decrypted for display (masked in list view, full in detail for admins)
Each encrypted value includes a random 12-byte IV and 16-byte auth tag
If CONNECTOR_ENCRYPTION_KEY is not set, credentials are stored in plaintext (development only)

Production Checklist

Set CONNECTOR_ENCRYPTION_KEY in all environments
Store the key in AWS Secrets Manager (see Section 5)
Rotate the key annually — re-encrypt all connectors after rotation
Never log or expose the encryption key in application logs

Get Started

Deploy & Operate

Runbooks

Data Platform

Decisioning Studio

Execute & Optimize

Intelligence

Platform & Security

Integrations

Reports

Release Notes

Documentation Index

​1. Checklist Summary

​2. TLS / HTTPS Everywhere

​2.1 Ingress TLS Termination

​2.2 Internal Service TLS

​3. Security Headers

​3.1 Ingress-Level Configuration (nginx)

​3.2 Content Security Policy

​3.3 Header Reference

​4. Rate Limiting

​4.1 nginx-ingress Rate Limiting

​4.2 Per-Endpoint Rate Limits

​4.3 Response Headers

​5. Secret Management

​5.1 AWS Secrets Manager + External Secrets Operator (ESO)

​5.2 Secret Rotation Policy

​5.3 Rotation Procedure

​6. Network Policies

​6.1 Default Deny All

​6.2 Allow Application Traffic

​7. Pod Security Standards

​7.1 Namespace Label (Restricted)

​7.2 Pod Security Context

​8. Database SSL

​8.1 RDS SSL Configuration

​8.2 Certificate Bundle

​8.3 In-Cluster PostgreSQL

​9. Audit Log Retention

​9.1 Application Audit Logs

​9.2 Retention Policy

​9.3 Log Forwarding

​9.4 Tamper Protection

​10. RBAC and Least-Privilege Access

​10.1 Kubernetes RBAC

​10.2 IAM Roles for Service Accounts (IRSA)

​10.3 Database Access Control

​11. Additional Hardening

​11.1 Dependency Scanning

​11.2 Image Provenance

​11.3 Runtime Protection

​12. API Key Authentication

​How It Works

​Production Checklist

​Key Management API

​13. Connector Credential Encryption

​Configuration

​How It Works

​Production Checklist

1. Checklist Summary

2. TLS / HTTPS Everywhere

2.1 Ingress TLS Termination

2.2 Internal Service TLS

3. Security Headers

3.1 Ingress-Level Configuration (nginx)

3.2 Content Security Policy

3.3 Header Reference

4. Rate Limiting

4.1 nginx-ingress Rate Limiting

4.2 Per-Endpoint Rate Limits

4.3 Response Headers

5. Secret Management

5.1 AWS Secrets Manager + External Secrets Operator (ESO)

5.2 Secret Rotation Policy

5.3 Rotation Procedure

6. Network Policies

6.1 Default Deny All

6.2 Allow Application Traffic

7. Pod Security Standards

7.1 Namespace Label (Restricted)

7.2 Pod Security Context

8. Database SSL

8.1 RDS SSL Configuration

8.2 Certificate Bundle

8.3 In-Cluster PostgreSQL

9. Audit Log Retention

9.1 Application Audit Logs

9.2 Retention Policy

9.3 Log Forwarding

9.4 Tamper Protection

10. RBAC and Least-Privilege Access

10.1 Kubernetes RBAC

10.2 IAM Roles for Service Accounts (IRSA)

10.3 Database Access Control

11. Additional Hardening

11.1 Dependency Scanning

11.2 Image Provenance

11.3 Runtime Protection

12. API Key Authentication

How It Works

Production Checklist

Key Management API

13. Connector Credential Encryption

Configuration

How It Works

Production Checklist