Calculate the sample size needed per variant to detect a given lift.
Sample size scales with the inverse square of the absolute lift, so detecting small lifts on low baseline rates requires enormous samples. Halving the detectable lift quadruples the requirement. Most A/B tests are stopped far too early to detect the lifts they claim, which is why fixing sample size before starting is essential.
A/B Test Sample Size
n per variant = (z_α√(2p̄q̄) + z_β√(p₁q₁ + p₂q₂))² ÷ (p₂ − p₁)²
n per variant = (z_α√(2p̄q̄) + z_β√(p₁q₁ + p₂q₂))² ÷ (p₂ − p₁)² Sample size scales with the inverse square of the absolute lift, so detecting small lifts on low baseline rates requires enormous samples. Halving the detectable lift quadruples the requirement.
Most A/B tests are stopped far too early to detect the lifts they claim, which is why fixing sample size before starting is essential.
This calculator takes 4 inputs: Baseline conversion rate, Minimum detectable relative lift, Significance level, Statistical power. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.