Confidence Interval Calculator
Calculate a confidence interval around a model metric estimate or sample mean.
About Confidence Interval Calculator
A confidence interval expresses the uncertainty around a sample estimate (such as a model's average accuracy across evaluation runs): CI = mean ± z × (std / √n), where z is the critical value for the chosen confidence level (1.96 for 95%). A 95% confidence interval means that if the sampling process were repeated many times, about 95% of the resulting intervals would contain the true population value. In ML, confidence intervals are commonly reported around cross-validation metrics to communicate how much a reported accuracy or score might vary due to sampling.
Formula
CI = mean +/- z × (std / sqrt(n))
- mean
- Sample mean
- z
- Critical value for the confidence level
- std
- Sample standard deviation
- n
- Sample size
Worked example
With the default values below, Confidence Interval Calculator returns Lower Bound of 0.8321. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Sample Mean
- 0.85
- Sample Standard Deviationminimum 0
- 0.05
- Sample Size (n)accepts 1–10,000,000
- 30
- Confidence Levelone of 80%, 85%, 90%, 95%, 98%, 99%, 99.9%
- 95%
Results
- Lower Bound
- 0.8321
- Upper Bound
- 0.8679
- Margin of Error
- 0.017892
- Standard Error
- 0.009129
- Z-value Used
- 1.960
Step by step
- 1Standard error: σ ÷ √n= 0.009129
- 2Margin of error: z(95%) × SE= 0.017892
- 3Confidence interval: mean ± margin= [0.8321, 0.8679]
Assumptions and limits
- The result is derived only from the 4 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to 6 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
Why does a larger sample size narrow the confidence interval?
The standard error (σ/√n) shrinks as n grows, since more observations give a more precise estimate of the true mean — this is why evaluating a model on more test samples or more cross-validation folds produces a tighter, more trustworthy confidence interval.
What does '95% confidence' actually mean?
It does not mean there's a 95% chance the true value lies in this specific interval — it means that if you repeated the sampling process many times and built an interval each time using the same method, about 95% of those intervals would contain the true population parameter.
This calculator uses a z-value — when should I use a t-value instead?
Use the z-distribution when the sample size is large (n ≥ 30) or the population standard deviation is known; for small samples with an estimated standard deviation, a t-distribution (with n−1 degrees of freedom) gives a more accurate, slightly wider interval.
How is this used to compare two ML models?
If two models' confidence intervals for a metric (e.g. accuracy) don't overlap, that's suggestive evidence one model outperforms the other; if they do overlap substantially, the observed difference may not be statistically significant — a proper hypothesis test (see the P-value Calculator) is more rigorous for this comparison.