BERTScore Calculator
Calculate BERTScore, a contextual embedding-based metric for text similarity evaluation.
Inputs
BERTScore F1
0.8996
Precision
0.9200
Recall
0.8800
Step by step
BERTScore F1: harmonic mean of P and R
2 × 0.9200 × 0.8800 / (0.9200 + 0.8800)
= 0.899556
How it works
BERTScore evaluates text generation quality using contextual embeddings from BERT. It computes token-level cosine similarity between candidate and reference embeddings, then aggregates using greedy matching to produce precision, recall, and F1 scores that capture semantic similarity beyond surface-level n-gram matching.
Formula
BERTScore F1
F1 = 2 * P_BERT * R_BERT / (P_BERT + R_BERT)
- P_BERT
- Precision from greedy cosine-similarity matching of candidate to reference tokens
- R_BERT
- Recall from greedy matching of reference to candidate tokens
Frequently Asked Questions
Why is BERTScore better than BLEU for some tasks?
BERTScore captures semantic equivalence (paraphrases, synonyms) that surface-level n-gram metrics miss entirely. 'The cat sat on the mat' and 'A feline rested on the rug' have low BLEU but high BERTScore.
Which BERT model is used for scoring?
The default is often RoBERTa-large, but any contextual embedding model can be used. The choice affects absolute scores but relative rankings tend to be consistent across model choices.