Calculate GPU utilization percentage from actual versus theoretical compute throughput.
GPU utilization measures what fraction of elapsed wall-clock time the GPU spent actively computing versus sitting idle: utilization% = active_time / total_time × 100. Idle gaps typically come from data loading bottlenecks (the GPU waits for the next batch), synchronization barriers in distributed training, checkpointing pauses, or CPU-bound preprocessing — all of which represent wasted, paid-for GPU time. Identifying low utilization is the first step toward optimizing a training pipeline, since even the fastest GPU delivers no value while idle; tools like Nsight Systems or PyTorch's profiler can pinpoint exactly where idle gaps occur.
utilization_percent = (active_time / total_time) × 100
Well-optimized training pipelines typically sustain 85-95%+ utilization; anything consistently below 70% usually indicates a bottleneck worth investigating, such as slow data loading, insufficient data pipeline parallelism, or excessive CPU-side preprocessing between GPU steps.
The most common causes are a data loader that can't keep pace with the GPU's consumption rate (fix: more data loader workers or pre-fetching), synchronous multi-GPU communication overhead, frequent checkpointing pauses, or CPU-bound preprocessing/augmentation happening in the training loop instead of overlapped with GPU compute.
Not entirely — utilization measures whether the GPU is 'busy,' but it doesn't measure whether that busy time is spent efficiently (e.g. using large enough batch sizes and effective kernels to approach peak FLOPs) — for that, see the TFLOPS/MFU calculator, which measures achieved compute relative to theoretical peak.