Perplexity Calculator
Calculate the perplexity of a language model from its cross-entropy loss.
Inputs
Average per-token cross-entropy loss, typically in nats (natural log base).
Perplexity
12.1825
Cross-Entropy Loss
2.5000
Step by step
Perplexity: e^H
e^2.5
= 12.1825
How it works
Perplexity is the standard evaluation metric for language models, derived directly from cross-entropy loss: Perplexity = e^H (when H is measured in nats, natural log) or Perplexity = 2^H (when H is measured in bits, log base 2). Intuitively, perplexity represents the model's average 'branching factor' — the effective number of equally likely next-token choices it's weighing at each step. Lower perplexity means the model assigns higher probability to the actual next tokens, indicating a better fit to the text distribution; a perplexity of 1 would mean perfect, deterministic prediction.
Formula
perplexity = e^H (nats) or 2^H (bits)
- H
- Average per-token cross-entropy loss
Frequently Asked Questions
What does a perplexity of 20 mean intuitively?
A perplexity of 20 suggests the model is, on average, as uncertain about the next token as if it were choosing uniformly at random among about 20 equally likely options — lower perplexity indicates more confident, accurate predictions.
Why do I need to know which log base the cross-entropy uses?
Cross-entropy computed with natural log (nats) converts to perplexity via e^H, while cross-entropy computed with log base 2 (bits) converts via 2^H — using the wrong exponentiation base gives a numerically different (incorrect) perplexity value.
Can perplexity be used to compare different language models directly?
Only if they were evaluated on the same test set with the same tokenizer — perplexity values are not directly comparable across models with different vocabularies or tokenization schemes, since the 'branching factor' interpretation depends on the token granularity.