Training Time Estimate Calculator

Work out training time estimate instantly with clear inputs, formula shown and shareable results.

About Training Time Estimate Calculator

Single-device work is total steps times step time. Adding devices divides that, but not perfectly: gradient synchronisation grows with device count, so an efficiency factor of roughly 1/(1 + 0.05 log2 devices) captures the observed sub-linear scaling. Device-hours, not wall-clock hours, are what the invoice is based on, and they rise as you add devices.

Formula

Distributed training time

single-device hours = steps x step time; efficiency = 1 / (1 + 0.05 log2 devices); wall clock = single-device hours / (devices x efficiency)

Worked example

With the default values below, Training Time Estimate Calculator returns Wall-clock training time of 0.86. Change any input above and every figure here is recalculated in your browser.

Inputs used

Epochsminimum 1: 30
Steps per epochminimum 1: 4,000
Step time on one deviceminimum 0.1 ms: 180 ms
Devices training in parallelminimum 1: 8

Results

Wall-clock training time: 0.86 h
Total optimisation steps: 120,000
Billed device-hours: 6.90 device-h
Wall-clock time in days: 0.04 days

Step by step

1Values used
2Distributed training time
3Wall-clock training time= 0.86 h
4Total optimisation steps= 120,000
5Billed device-hours= 6.90 device-h
6Wall-clock time in days= 0.04 days

Assumptions and limits

The result is derived only from the 4 inputs listed above. Anything not among them is not part of the calculation.
In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

Why do device-hours increase with more GPUs?

Communication overhead is pure extra work. You buy shorter wall-clock time with more total compute, which is usually worth it for iteration speed but costs more money.

How can scaling efficiency be improved?

Larger per-device batches, gradient compression or bucketing, faster interconnect such as NVLink or InfiniBand, and overlapping communication with backward computation.