Work out statistical confidence for test instantly with clear inputs, formula shown and shareable results.
A two-proportion z-test compares conversion rates using the pooled standard error. The confidence level is the probability that a difference this large would not arise by chance if the variants were truly identical.
Two-proportion z-test
z = (p_b - p_a) / √[p̄(1-p̄)(1/n_a + 1/n_b)]
Figures are estimates based on the inputs given. Marketing performance, platform fees and conversion behaviour vary by audience, channel and season. Use this as a planning guide, not a forecast.
No. It means a one in twenty chance of a false positive, which is why peeking at results repeatedly inflates errors.
A statistically significant 0.2% lift may not be worth implementing. Significance and materiality are different questions.