GPU Memory Calculator
Estimate the GPU memory required to train or run a given model architecture.
About GPU Memory Calculator
GPU memory for inference is dominated by model weights: memory_GB = parameters_billions × bytes_per_param × overhead_multiplier. FP16/BF16 halves memory versus FP32, while INT8 and INT4 quantization shrink it further at some accuracy cost. The overhead multiplier (commonly ~20%) accounts for activations, the KV cache, and framework/runtime overhead that sit on top of raw weight storage. Training memory is substantially higher because it also needs gradients and optimizer states.
Formula
total_memory_GB = parameters_billions * bytes_per_param * (1 + overhead_percent / 100)
- parameters_billions
- Model parameter count in billions
- bytes_per_param
- Bytes per parameter (4 for FP32, 2 for FP16, 1 for INT8, 0.5 for INT4)
- overhead_percent
- Percentage overhead for activations and KV cache
Worked example
With the default values below, GPU Memory Calculator returns Total VRAM Needed of 16.80. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Model Parametersaccepts 0.01–2,000 billion
- 7 billion
- Precisionone of FP32 (4 bytes), FP16 / BF16 (2 bytes), INT8 (1 byte), INT4 (0.5 bytes)
- FP16 / BF16 (2 bytes)
- Activation Overheadaccepts 0–200 %
- 20 %
Results
- Total VRAM Needed
- 16.80 GB
- Base Weight Memory
- 14.00 GB
- Bytes per Parameter
- 2.00 bytes
Step by step
- 1Base weight memory: parameters × bytes per parameter= 14.00 GB
- 2Total memory with overhead: base memory × (1 + overhead%)= 16.80 GB
Assumptions and limits
- The result is derived only from the 3 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, fees or charges or compounding frequency into account.
- Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
Does this include training memory (gradients, optimizer states)?
No, this estimates inference-only memory. Training requires roughly 4x more memory per parameter to also store gradients and optimizer states (e.g. Adam's two moment buffers).
Why does precision matter so much?
Lower-precision formats store each parameter in fewer bytes, directly reducing memory. INT4 quantization can shrink a model to 1/8th the size of FP32 with some quality tradeoff.
What's a safe overhead percentage to use?
20% is a reasonable default for short-context inference. Long-context workloads with a large KV cache may need 40-100%+ overhead — use the KV Cache Memory Calculator for a more precise figure.
Does batch size affect this estimate?
Batch size mainly affects activation and KV cache memory, not the base weight memory calculated here. Larger batches increase the effective overhead needed.