Skip to content
ENExplore Calculators

Batch Size Selection Calculator

Work out batch size selection instantly with clear inputs, formula shown and shareable results.

About Batch Size Selection Calculator

GPU memory splits into a fixed part — weights, gradients and optimiser state — and a part that scales with batch size, which is dominated by stored activations for the backward pass. The largest micro-batch is therefore the free memory divided by activation memory per sample. Gradient accumulation then reaches any target effective batch by summing gradients over several micro-batches before stepping the optimiser.

Formula

Batch and accumulation

free = total - fixed model memory; micro-batch = floor(free / activation per sample); accumulation steps = ceil(target / micro-batch)

Worked example

With the default values below, Batch Size Selection Calculator returns Largest micro-batch that fits of 159. Change any input above and every figure here is recalculated in your browser.

Inputs used

GPU memory availableminimum 1 GB
40 GB
Memory held by weights, gradients and optimiserminimum 0.1 GB
12 GB
Activation memory per sampleminimum 0.1 MB
180 MB
Target effective batch sizeminimum 1
256

Results

Largest micro-batch that fits
159
Gradient accumulation steps needed
2
Effective batch size achieved
318
Peak memory used
39.95 GB

Step by step

  1. 1Values used
  2. 2Batch and accumulation
  3. 3Largest micro-batch that fits= 159
  4. 4Gradient accumulation steps needed= 2
  5. 5Effective batch size achieved= 318
  6. 6Peak memory used= 39.95 GB

Assumptions and limits

  • The result is derived only from the 4 inputs listed above. Anything not among them is not part of the calculation.
  • In particular, this calculator does not take inflation, fees or charges or compounding frequency into account.
  • Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

Does gradient accumulation give identical results to a large batch?

Almost. The gradient is mathematically the same, but batch-normalisation statistics are computed per micro-batch, so networks using BatchNorm behave slightly differently.

How can activation memory be reduced?

Gradient checkpointing recomputes activations in the backward pass, cutting activation memory by roughly the square root of the layer count at about 30 percent extra compute.

Related Calculators

Batch Size Selection Calculator | Calcrivo