METEOR Score Calculator
Calculate the METEOR score to evaluate translation quality with synonym matching.
About METEOR Score Calculator
METEOR improves on BLEU by incorporating recall, synonym matching, and stemming. It computes a weighted F-measure heavily favoring recall (α=0.9), then applies a fragmentation penalty that penalizes translations where matches are scattered rather than contiguous.
Formula
METEOR
METEOR = F_mean * (1 - 0.5*(chunks/matches)^3)
- F_mean
- Weighted harmonic mean: 10*P*R / (9P + R)
- chunks
- Number of contiguous matched segments
Worked example
With the default values below, METEOR Score Calculator returns METEOR Score of 0.6989. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Unigram Matchesminimum 0
- 20
- Candidate Length (words)minimum 1
- 25
- Reference Length (words)minimum 1
- 28
- Chunks (contiguous matches)minimum 1
- 8
Results
- METEOR Score
- 0.6989
- F-mean
- 0.7220
- Fragmentation Penalty
- 0.0320
Step by step
- 1Precision: matches ÷ candidate length= 0.8000
- 2Recall: matches ÷ reference length= 0.7143
- 3F-mean (α=0.9): 10PR/(9P+R)= 0.7220
- 4Fragmentation penalty: 0.5×(chunks/matches)³= 0.032000
- 5METEOR = F-mean × (1 − penalty)= 0.698917
Assumptions and limits
- The result is derived only from the 4 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to 4 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
How does METEOR handle synonyms?
METEOR uses WordNet synonym databases and stemming to match words that are semantically equivalent but lexically different (e.g., 'buy' and 'purchase'), giving it better correlation with human judgment than pure n-gram overlap methods.
Why does METEOR correlate better with human judgments than BLEU?
METEOR's recall emphasis, synonym matching, stemming, and fragmentation penalty all capture aspects of translation quality that humans care about but simple n-gram precision misses.