POST /api/v1/experiments/power-calculator
Calculates the required sample size and estimated experiment duration for an A/B test. Uses a two-proportion z-test formula with traffic estimates derived from the tenant’s recent interaction history.
Request Body
| Field | Type | Required | Description |
|---|
baselineConversionRate | number | Yes | Current conversion rate (0.001-1.0) |
minimumDetectableEffect | number | Yes | Smallest effect size to detect (0.001-1.0) |
significanceLevel | number | No | Alpha level for the test (0.01-0.20, default: 0.05) |
power | number | No | Statistical power (0.5-0.99, default: 0.8) |
trafficSplit | number | No | Proportion of traffic in the treatment group (0.1-0.9, default: 0.5) |
Example
curl -X POST https://playground.kaireonai.com/api/v1/experiments/power-calculator \
-H "Content-Type: application/json" \
-H "X-Tenant-Id: my-tenant" \
-d '{
"baselineConversionRate": 0.05,
"minimumDetectableEffect": 0.02,
"significanceLevel": 0.05,
"power": 0.8,
"trafficSplit": 0.5
}'
Response
{
"requiredSampleSize": 7850,
"perVariant": 3925,
"estimatedDuration": {
"days": 16,
"basedOnDailyTraffic": 500
},
"recommendation": "With 500 daily decisions, you need ~16 days to reach statistical significance.",
"sensitivityTable": [
{ "effect": 0.01, "sampleSize": 31400, "days": 63 },
{ "effect": 0.02, "sampleSize": 7850, "days": 16 },
{ "effect": 0.05, "sampleSize": 1260, "days": 3 },
{ "effect": 0.10, "sampleSize": 320, "days": 1 }
]
}
Response Fields
| Field | Type | Description |
|---|
requiredSampleSize | integer | Total sample size needed across both variants |
perVariant | integer | Sample size per variant |
estimatedDuration.days | integer or null | Estimated days to reach the required sample size |
estimatedDuration.basedOnDailyTraffic | integer | Average daily decisions from the last 14 days |
recommendation | string | Human-readable guidance |
sensitivityTable | array | Sample sizes for different effect sizes |
Sensitivity Table
The sensitivity table shows how the required sample size changes with different minimum detectable effects. This helps teams understand the trade-off between experiment duration and the ability to detect small effects.
If estimatedDuration.basedOnDailyTraffic is 0, the tenant has no recent decision traffic. Start sending decisions through the Recommend API to get accurate duration estimates.
Roles
admin, editor, viewer
See also: Experiments | Algorithm Models
