Perplexity Calculator
Calculate the perplexity of a language model from its cross-entropy loss.
About Perplexity Calculator
Perplexity is the standard evaluation metric for language models, derived directly from cross-entropy loss: Perplexity = e^H (when H is measured in nats, natural log) or Perplexity = 2^H (when H is measured in bits, log base 2). Intuitively, perplexity represents the model's average 'branching factor' — the effective number of equally likely next-token choices it's weighing at each step. Lower perplexity means the model assigns higher probability to the actual next tokens, indicating a better fit to the text distribution; a perplexity of 1 would mean perfect, deterministic prediction.
Formula
perplexity = e^H (nats) or 2^H (bits)
- H
- Average per-token cross-entropy loss
Worked example
With the default values below, Perplexity Calculator returns Perplexity of 12.1825. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Cross-Entropy Lossaccepts 0–700
- 2.5
- Log Base Usedone of Natural log (nats) — Perplexity = e^H, Log base 2 (bits) — Perplexity = 2^H
- Natural log (nats) — Perplexity = e^H
Results
- Perplexity
- 12.1825
- Cross-Entropy Loss
- 2.5000
Step by step
- 1Perplexity: e^H= 12.1825
Assumptions and limits
- The result is derived only from the 2 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to 4 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
What does a perplexity of 20 mean intuitively?
A perplexity of 20 suggests the model is, on average, as uncertain about the next token as if it were choosing uniformly at random among about 20 equally likely options — lower perplexity indicates more confident, accurate predictions.
Why do I need to know which log base the cross-entropy uses?
Cross-entropy computed with natural log (nats) converts to perplexity via e^H, while cross-entropy computed with log base 2 (bits) converts via 2^H — using the wrong exponentiation base gives a numerically different (incorrect) perplexity value.
Can perplexity be used to compare different language models directly?
Only if they were evaluated on the same test set with the same tokenizer — perplexity values are not directly comparable across models with different vocabularies or tokenization schemes, since the 'branching factor' interpretation depends on the token granularity.