Training Time Calculator
Estimate total model training time from dataset size, batch size, and hardware speed.
About Training Time Calculator
Training time can be estimated from the total compute (FLOPs) required divided by the effective throughput of your GPU cluster: time = (dataset_samples × epochs × 6 × model_params) / (gpu_count × gpu_tflops × 1e12 × utilization). The factor of 6 approximates the combined multiply-add FLOPs of a forward pass (≈2×params) and backward pass (≈4×params) per training sample. Utilization accounts for the fact that real training rarely reaches 100% of a GPU's theoretical peak throughput due to communication overhead, data loading, and memory-bound operations.
Formula
time_hours = (samples * epochs * 6 * params) / (gpu_count * gpu_tflops * 1e12 * utilization * 3600)
- samples
- Total dataset samples
- epochs
- Number of training epochs
- params
- Model parameter count
- gpu_count
- Number of GPUs
- gpu_tflops
- Peak TFLOPs per GPU
- utilization
- Sustained utilization fraction (typically 0.3-0.5)
Worked example
With the default values below, Training Time Calculator returns Estimated Training Time of 0.00. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Dataset Samplesaccepts 1–1,000,000,000,000 samples
- 1,000,000 samples
- Epochsaccepts 1–1,000 epochs
- 3 epochs
- Model Parametersaccepts 0.001–2,000 billion
- 7 billion
- GPU Countaccepts 1–100,000 GPUs
- 8 GPUs
- GPU Throughputaccepts 0.1–5,000 TFLOPs
- 312 TFLOPs
- Utilizationaccepts 1–100 %
- 40 %
Results
- Estimated Training Time
- 0.00 days
- Estimated Training Time
- 0.0 hours
- Total Training FLOPs
- 1.260 × 10¹⁷
Step by step
- 1Total training FLOPs: samples × epochs × 6 × parameters= 1.260e+17 FLOPs
- 2Effective cluster throughput: GPUs × TFLOPs × 1e12 × utilization= 9.984e+14 FLOPs/sec
- 3Training time: total FLOPs ÷ effective throughput= 0.0 hours
Assumptions and limits
- The result is derived only from the 6 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to 3 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
Where does the factor of 6 come from?
A forward pass costs roughly 2 FLOPs per parameter per token (one multiply, one add), and the backward pass costs about twice that (roughly 4 FLOPs per parameter), for a combined ~6 FLOPs per parameter per training sample.
What utilization percentage should I use?
Large multi-GPU training runs commonly achieve 30-50% of theoretical peak FLOPs due to communication, memory bandwidth limits, and pipeline bubbles. Well-optimized single-GPU runs can reach higher, 50-70%.
What TFLOPs value should I use for my GPU?
Use the GPU's advertised peak FLOPs at the precision you're training in (e.g. BF16/FP16), such as ~312 TFLOPs for an A100 or ~989 TFLOPs for an H100 at BF16 with sparsity disabled.
Does this account for data loading or checkpointing overhead?
No, this is a pure compute-bound estimate. Real wall-clock time is often longer due to I/O, checkpointing, and evaluation passes.