Work out compute budget (chinchilla) instantly with clear inputs, formula shown and shareable results.
The Chinchilla scaling study found that for a fixed compute budget, parameters and training tokens should grow together at roughly 20 tokens per parameter — far more data than earlier models used. Multiplying by the 6ND rule gives the FLOPs, and dividing by device throughput times model FLOPs utilisation converts that into device time you can actually schedule.
Chinchilla-optimal budget
tokens = 20 x parameters; FLOPs = 6 x parameters x tokens; device time = FLOPs / (peak throughput x utilisation)
Because inference cost matters. Overtraining a smaller model past the compute-optimal point costs more to train but is permanently cheaper to serve, which is why ratios of 100 or more are now common.
Large well-tuned runs reach 40 to 55 percent model FLOPs utilisation. Below 30 percent, look at data loading, sequence packing and communication overlap.