Work out epoch time estimate instantly with clear inputs, formula shown and shareable results.
One epoch is ceil(samples / batch size) optimisation steps, so epoch time is that count multiplied by measured step time. Step time is the honest way to model this because it already includes forward, backward, optimiser and any data-loading stall. Throughput in samples per second is the figure to compare when changing batch size, precision or hardware.
Epoch time
steps = ceil(samples / batch size); epoch time = steps x step time
Step time grows with batch size once the GPU is saturated. The gain comes from better utilisation at small batches and from fewer optimiser and communication overheads, not from free parallelism.
Then step time is dominated by waiting for input and no GPU change helps. Measure GPU utilisation; below about 80 percent, fix prefetching, worker count and decoding first.