Distillation Speedup Calculator

Work out distillation speedup instantly with clear inputs, formula shown and shareable results.

About Distillation Speedup Calculator

Knowledge distillation trains a small student to match a large teacher's output distribution rather than just the hard labels. Because decoding cost is roughly proportional to parameter count, the speedup tracks the parameter ratio, and the interesting number is what accuracy that buys: retaining 96 percent of quality at a fifth of the cost is usually a better trade than serving the teacher.

Formula

Distillation trade-off

speedup ~ teacher params / student params; student latency = teacher latency / speedup; accuracy cost = 100% - retained

Worked example

With the default values below, Distillation Speedup Calculator returns Expected speedup of 5.39. Change any input above and every figure here is recalculated in your browser.

Inputs used

Teacher parametersminimum 1: 7,000,000,000
Student parametersminimum 1: 1,300,000,000
Teacher latency per requestminimum 0.1 ms: 120 ms
Accuracy retained by the studentaccepts 1–100 %: 96 %

Results

Expected speedup: 5.39 x
Student latency per request: 22.29 ms
Parameters removed: 81.43%
Accuracy given up: 4.00%

Step by step

1Values used
2Distillation trade-off
3Expected speedup= 5.39 x
4Student latency per request= 22.29 ms
5Parameters removed= 81.43
6Accuracy given up= 4.00

Assumptions and limits

The result is derived only from the 4 inputs listed above. Anything not among them is not part of the calculation.
In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

Why does soft-target training help?

The teacher's full probability distribution carries information about how classes relate, which is far richer supervision than a one-hot label and lets a small student learn from less data.

Is the speedup really linear in parameters?

Approximately, for models of similar shape on the same hardware. Depth versus width matters too, since deep narrow students parallelise worse than shallow wide ones.