Work out quantisation size savings instantly with clear inputs, formula shown and shareable results.
Quantisation scales weight memory linearly with bits per weight, so fp16 to int4 is a four-fold reduction and fp32 to int8 is four-fold as well. The saving is not only memory: smaller weights mean fewer bytes moved per token, and since decoding is memory-bandwidth bound, throughput usually improves by a similar factor.
Quantisation saving
saved = parameters x (bytes before - bytes after); reduction = saved / original
For single-stream decoding, yes, because it is bandwidth bound. For large-batch prefill, which is compute bound, gains are smaller unless the hardware has native low-precision matrix units.
int8 is usually within noise. Modern 4-bit methods lose a small amount on knowledge-heavy and reasoning tasks; below 4 bits, degradation becomes clearly measurable.