Skip to content
ENExplore Calculators

Quantisation Size Savings Calculator

Work out quantisation size savings instantly with clear inputs, formula shown and shareable results.

About Quantisation Size Savings Calculator

Quantisation scales weight memory linearly with bits per weight, so fp16 to int4 is a four-fold reduction and fp32 to int8 is four-fold as well. The saving is not only memory: smaller weights mean fewer bytes moved per token, and since decoding is memory-bandwidth bound, throughput usually improves by a similar factor.

Formula

Quantisation saving

saved = parameters x (bytes before - bytes after); reduction = saved / original

Worked example

With the default values below, Quantisation Size Savings Calculator returns Memory saved of 9.78 GiB. Change any input above and every figure here is recalculated in your browser.

Inputs used

Parametersminimum 1
7,000,000,000
Bytes per weight beforeaccepts 0.25–8
2
Bytes per weight afteraccepts 0.25–8
0.5

Results

Memory saved
9.78 GiB
Original footprint
13.04 GiB
Quantised footprint
3.26 GiB
Reduction
75.00%

Step by step

  1. 1Values used
  2. 2Quantisation saving
  3. 3Memory saved= 10,500,000,000
  4. 4Original footprint= 14,000,000,000
  5. 5Quantised footprint= 3,500,000,000
  6. 6Reduction= 75.00

Assumptions and limits

  • The result is derived only from the 3 inputs listed above. Anything not among them is not part of the calculation.
  • In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
  • Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

Does quantisation always speed up inference?

For single-stream decoding, yes, because it is bandwidth bound. For large-batch prefill, which is compute bound, gains are smaller unless the hardware has native low-precision matrix units.

What accuracy loss should I expect?

int8 is usually within noise. Modern 4-bit methods lose a small amount on knowledge-heavy and reasoning tasks; below 4 bits, degradation becomes clearly measurable.

Related Calculators

Quantisation Size Savings Calculator | Calcrivo