Skip to content
ENExplore Calculators

FLOPs Estimate Calculator

Work out flops estimate instantly with clear inputs, formula shown and shareable results.

About FLOPs Estimate Calculator

A forward pass costs about two floating-point operations per parameter per token, because each weight participates in one multiply and one add. The backward pass costs roughly twice the forward pass, giving the widely used 6ND estimate for training and 2ND for inference. These figures ignore attention's quadratic term, which is small until context length approaches the hidden dimension.

Formula

Compute estimate

training FLOPs = 6 x N x D; inference FLOPs = 2 x N x D

N
Parameters
D
Tokens processed

Worked example

With the default values below, FLOPs Estimate Calculator returns Total floating-point operations of 4.200e+19. Change any input above and every figure here is recalculated in your browser.

Inputs used

Model parametersminimum 1
7,000,000,000
Tokens processedminimum 1
1,000,000,000
Workloadone of Training (6 x N x D), Inference / forward only (2 x N x D)
Training (6 x N x D)

Results

Total floating-point operations
4.200e+19
FLOPs per token
42,000,000,000
Petaflop-days
0.4861
GPU-days at 300 TFLOP/s sustained
1.620 GPU-days

Step by step

  1. 1Values used
  2. 2Compute estimate
  3. 3Total floating-point operations= 4.200e+19
  4. 4FLOPs per token= 42,000,000,000
  5. 5Petaflop-days= 0.4861
  6. 6GPU-days at 300 TFLOP/s sustained= 1.620 GPU-days

Assumptions and limits

  • The result is derived only from the 3 inputs listed above. Anything not among them is not part of the calculation.
  • In particular, this calculator does not take inflation, tax or compounding frequency into account.
  • Figures are rounded for display to 4 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

When does the 6ND rule break down?

At very long contexts. Attention adds roughly 12 x layers x d x seq^2 per sequence, which starts to dominate once sequence length exceeds the hidden dimension.

Why is sustained throughput far below peak TFLOPs?

Memory bandwidth, kernel launch overhead and communication mean real utilisation is typically 35 to 55 percent of peak on large training runs.

Related Calculators

FLOPs Estimate Calculator | Calcrivo