Calculate the METEOR score to evaluate translation quality with synonym matching.
METEOR improves on BLEU by incorporating recall, synonym matching, and stemming. It computes a weighted F-measure heavily favoring recall (α=0.9), then applies a fragmentation penalty that penalizes translations where matches are scattered rather than contiguous.
METEOR
METEOR = F_mean * (1 - 0.5*(chunks/matches)^3)
METEOR uses WordNet synonym databases and stemming to match words that are semantically equivalent but lexically different (e.g., 'buy' and 'purchase'), giving it better correlation with human judgment than pure n-gram overlap methods.
METEOR's recall emphasis, synonym matching, stemming, and fragmentation penalty all capture aspects of translation quality that humans care about but simple n-gram precision misses.