Skip to content
ENExplore Calculators

Annotation QA Sample Size Calculator

Work out annotation qa sample size instantly with clear inputs, formula shown and shareable results.

About Annotation QA Sample Size Calculator

Zero-acceptance sampling asks how many items you must inspect to be confident of catching at least one defect if the batch is at or above a given error rate. The probability of missing every defect in n draws is (1-p)^n, so n = ln(1 - confidence) / ln(1 - p). Detecting a 2 percent error rate with 95 percent confidence needs about 149 items, regardless of whether the batch is 5,000 or 500,000.

Formula

Zero-acceptance sampling

n = ln(1 - confidence) / ln(1 - p); detection probability = 1 - (1 - p)^n; reject the batch on any defect found

Worked example

With the default values below, Annotation QA Sample Size Calculator returns Items to spot-check of 149. Change any input above and every figure here is recalculated in your browser.

Inputs used

Items in the batchminimum 1
5,000
Error rate you must be able to detectaccepts 0.01–50 %
2 %
Detection confidenceaccepts 50–99.99 %
95 %

Results

Items to spot-check
149
Share of the batch reviewed
2.98%
Probability of catching at least one error
95.07%
Errors allowed before rejecting the batch
0

Step by step

  1. 1Values used
  2. 2Zero-acceptance sampling
  3. 3Items to spot-check= 149
  4. 4Share of the batch reviewed= 2.98
  5. 5Probability of catching at least one error= 95.07
  6. 6Errors allowed before rejecting the batch= 0

Assumptions and limits

  • The result is derived only from the 3 inputs listed above. Anything not among them is not part of the calculation.
  • In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
  • Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

Why is the acceptance number zero?

This plan is designed to reject on any defect, which is what gives the small sample size. Allowing defects requires a larger sample from a binomial or AQL table.

Should the sample be random?

Yes, and drawn across annotators and time within the batch. Reviewing the first hundred items measures the warm-up period rather than the batch.

Related Calculators

Annotation QA Sample Size Calculator | Calcrivo