Work out distillation speedup instantly with clear inputs, formula shown and shareable results.
Knowledge distillation trains a small student to match a large teacher's output distribution rather than just the hard labels. Because decoding cost is roughly proportional to parameter count, the speedup tracks the parameter ratio, and the interesting number is what accuracy that buys: retaining 96 percent of quality at a fifth of the cost is usually a better trade than serving the teacher.
Distillation trade-off
speedup ~ teacher params / student params; student latency = teacher latency / speedup; accuracy cost = 100% - retained
The teacher's full probability distribution carries information about how classes relate, which is far richer supervision than a one-hot label and lets a small student learn from less data.
Approximately, for models of similar shape on the same hardware. Depth versus width matters too, since deep narrow students parallelise worse than shallow wide ones.