TFLOPS & MFU Calculator
Calculate achieved TFLOPS from operation count and time, and Model FLOPs Utilization (MFU) against hardware peak.
Inputs
Total floating-point operations performed, e.g. for one training step.
Achieved TFLOPS
416.67TFLOPS
Model FLOPs Utilization (MFU)
133.55%
Theoretical Peak TFLOPS
312.00TFLOPS
Step by step
Achieved TFLOPS: operations ÷ (time × 1e12)
5.000e+14 ÷ (1.2 × 1e12)
= 416.67 TFLOPS
Theoretical peak: hardware TFLOPs × GPU count
312 × 1
= 312.00 TFLOPS
MFU: achieved ÷ theoretical peak × 100
416.67 ÷ 312.00 × 100
= 133.55%
How it works
Achieved TFLOPS measures actual sustained compute throughput: TFLOPS = operations / (time_seconds × 1e12). Comparing this to hardware's advertised theoretical peak gives Model FLOPs Utilization (MFU) = achieved / theoretical × 100 — a key efficiency metric in large-scale training. Well-optimized large language model training runs typically achieve 30-55% MFU; the gap from 100% comes from memory bandwidth limits, kernel launch overhead, communication between GPUs, and imperfect operator fusion, all of which prevent hardware from sustaining its theoretical peak FLOPs in real workloads.
Formulas
Achieved TFLOPS
achieved_tflops = operations / (time_seconds × 1e12)
- operations
- Total floating-point operations performed
- time_seconds
- Wall-clock time taken
Model FLOPs Utilization
MFU = (achieved_tflops / (hardware_peak_tflops × gpu_count)) × 100
- achieved_tflops
- Measured throughput in TFLOPS
- hardware_peak_tflops
- Advertised peak TFLOPS for the GPU
- gpu_count
- Number of GPUs
Frequently Asked Questions
What is a good MFU percentage for LLM training?
Well-tuned large-scale training runs (e.g. GPT-3/PaLM-scale models on modern GPU clusters) typically report MFU in the 30-55% range; achieving above 50% is considered excellent, since perfect 100% utilization is essentially unreachable due to memory bandwidth and communication overhead.
Why can't real training runs reach 100% MFU?
The theoretical peak TFLOPs figure assumes every compute unit is perfectly fed with data every cycle, but real training involves memory bandwidth limits, inter-GPU communication for gradient synchronization, kernel launch overhead, and non-matmul operations (normalization, activation functions) that don't run at peak matmul throughput.
Why does hardware selection matter for the MFU calculation?
MFU is only meaningful relative to a specific hardware's theoretical peak — the same achieved TFLOPS represents a much higher MFU on slower hardware (e.g. an RTX 4090) than on faster hardware (e.g. an H100), since the denominator (theoretical peak) differs substantially.