Batch Size Calculator
Estimate an optimal training batch size based on dataset size and memory limits.
About Batch Size Calculator
The maximum training batch size that fits in GPU memory is: max_batch = (available_vram − model_size − optimizer_states) / per_sample_memory. Model weights and optimizer states (like Adam's momentum and variance buffers) occupy a fixed amount of VRAM regardless of batch size, so subtracting them first leaves the memory actually available for activations, which scale linearly with batch size. Batch sizes are conventionally rounded down to a power of 2 for better GPU kernel efficiency and predictable memory alignment.
Formula
max_batch_size = floor((available_vram - model_size - optimizer_states) / per_sample_memory)
- available_vram
- Total available GPU VRAM in GB
- model_size
- Memory occupied by model weights in GB
- optimizer_states
- Memory occupied by optimizer buffers in GB
- per_sample_memory
- Activation memory per sample in GB
Worked example
With the default values below, Batch Size Calculator returns Max Batch Size of 240. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Available VRAMaccepts 0.1–1,000 GB
- 24 GB
- Model Size (Weights)accepts 0–1,000 GB
- 4 GB
- Optimizer States Sizeaccepts 0–1,000 GB
- 8 GB
- Per-Sample Memoryaccepts 0.01–100,000 MB
- 50 MB
Results
- Max Batch Size
- 240 samples
- Recommended Batch Size (Power of 2)
- 128 samples
- Free Memory for Activations
- 12.00 GB
Step by step
- 1Free memory for activations: available VRAM − model size − optimizer states= 12.00 GB
- 2Max batch size: free memory ÷ per-sample memory= 240
- 3Recommended batch size: largest power of 2 ≤ max batch size= 128
Assumptions and limits
- The result is derived only from the 4 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, fees or charges or compounding frequency into account.
- Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
Why round to a power of 2?
GPU kernels and memory allocators are often optimized for power-of-2 sizes, which can improve throughput and reduce memory fragmentation compared to arbitrary batch sizes.
What if optimizer states take more memory than expected?
Adam-style optimizers store two additional buffers (first and second moment estimates) per parameter, typically 2x the model size in the same precision, plus a FP32 master copy if using mixed precision — this can add up to 3-4x model size total.
How do I measure per-sample memory accurately?
The most reliable way is to run a single training step with a small batch size and observe actual GPU memory usage via nvidia-smi or a profiler, then divide the activation memory by the batch size used.
Can I exceed this maximum with gradient accumulation?
Gradient accumulation lets you simulate a larger effective batch size by accumulating gradients over multiple smaller forward/backward passes before updating weights, without needing the full batch in memory at once.