Calculate training throughput in samples processed per second.
Training throughput in samples per second is the most direct measure of how fast a model trains: samples/s = batch_size / step_time_seconds. This figure directly determines how long a full epoch takes (steps_per_epoch × step_time), and is the metric most sensitive to hardware, batch size, model architecture, and data pipeline efficiency. Comparing samples/sec across different batch sizes or hardware configurations is the most practical way to evaluate real-world training speed improvements from optimizations like mixed precision, gradient checkpointing, or hardware upgrades.
samples_per_second = batch_size / step_time_seconds
Larger batches improve GPU utilization up to a point (better parallelism per step), increasing samples/sec, but beyond the point where the GPU is already compute-saturated, step time grows roughly proportionally with batch size, causing samples/sec to plateau rather than keep increasing.
Divide your total training samples (dataset size × number of epochs) by samples/sec to get total training time in seconds — this is the same calculation used by the Training Time Calculator, just expressed through throughput instead of step count.
If step_time_seconds is measured for the full multi-GPU setup processing one global batch, then samples/sec already reflects the combined throughput of all GPUs — just ensure batch_size refers to the full global batch, not the per-GPU micro-batch, to avoid underestimating throughput.