Hallucination Rate Estimate Calculator

Work out hallucination rate estimate instantly with clear inputs, formula shown and shareable results.

About Hallucination Rate Estimate Calculator

A hallucination rate measured on a sample is an estimate with uncertainty, and reporting it without a confidence interval invites false conclusions. The normal approximation gives a margin of error of z x sqrt(p(1-p)/n), so 26 flags in 400 reviews is about 6.5 percent plus or minus 2.4 points at 95 percent confidence — which cannot be distinguished from a 5 percent target.

Formula

Proportion confidence interval

p = flagged / sampled; margin = z x sqrt(p(1-p)/n); interval = p +/- margin

Worked example

With the default values below, Hallucination Rate Estimate Calculator returns Observed hallucination rate of 6.50%. Change any input above and every figure here is recalculated in your browser.

Inputs used

Responses reviewedminimum 1: 400
Responses with unsupported claimsminimum 0: 26
Confidence levelaccepts 80–99.9 %: 95 %

Results

Observed hallucination rate: 6.50%
Margin of error: 2.42%
Upper confidence bound: 8.92%
Lower confidence bound: 4.08%

Step by step

1Values used
2Proportion confidence interval
3Observed hallucination rate= 6.50
4Margin of error= 2.42
5Upper confidence bound= 8.92
6Lower confidence bound= 4.08

Assumptions and limits

The result is derived only from the 3 inputs listed above. Anything not among them is not part of the calculation.
In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

How many reviews do I need to detect a 1 point change?

Roughly 2,000 to 4,000 at these rates. Detecting small differences in low-rate events needs large samples, which is why teams track trends rather than single measurements.

What counts as a hallucination?

Define it before measuring: unsupported by the retrieved context, contradicted by it, or factually wrong about the world are three different metrics with very different rates.