METEOR Score Calculator
Calculate the METEOR score to evaluate translation quality with synonym matching.
Inputs
METEOR Score
0.6989
F-mean
0.7220
Fragmentation Penalty
0.0320
Step by step
Precision: matches ÷ candidate length
20 ÷ 25
= 0.8000
Recall: matches ÷ reference length
20 ÷ 28
= 0.7143
F-mean (α=0.9): 10PR/(9P+R)
10×0.800×0.714/(9×0.800+0.714)
= 0.7220
Fragmentation penalty: 0.5×(chunks/matches)³
0.5×(8/20)³
= 0.032000
METEOR = F-mean × (1 − penalty)
0.7220 × (1 − 0.0320)
= 0.698917
How it works
METEOR improves on BLEU by incorporating recall, synonym matching, and stemming. It computes a weighted F-measure heavily favoring recall (α=0.9), then applies a fragmentation penalty that penalizes translations where matches are scattered rather than contiguous.
Formula
METEOR
METEOR = F_mean * (1 - 0.5*(chunks/matches)^3)
- F_mean
- Weighted harmonic mean: 10*P*R / (9P + R)
- chunks
- Number of contiguous matched segments
Frequently Asked Questions
How does METEOR handle synonyms?
METEOR uses WordNet synonym databases and stemming to match words that are semantically equivalent but lexically different (e.g., 'buy' and 'purchase'), giving it better correlation with human judgment than pure n-gram overlap methods.
Why does METEOR correlate better with human judgments than BLEU?
METEOR's recall emphasis, synonym matching, stemming, and fragmentation penalty all capture aspects of translation quality that humans care about but simple n-gram precision misses.