BERTScore Calculator
Calculate BERTScore, a contextual embedding-based metric for text similarity evaluation.
About BERTScore Calculator
BERTScore evaluates text generation quality using contextual embeddings from BERT. It computes token-level cosine similarity between candidate and reference embeddings, then aggregates using greedy matching to produce precision, recall, and F1 scores that capture semantic similarity beyond surface-level n-gram matching.
Formula
BERTScore F1
F1 = 2 * P_BERT * R_BERT / (P_BERT + R_BERT)
- P_BERT
- Precision from greedy cosine-similarity matching of candidate to reference tokens
- R_BERT
- Recall from greedy matching of reference to candidate tokens
Worked example
With the default values below, BERTScore Calculator returns BERTScore F1 of 0.8996. Change any input above and every figure here is recalculated in your browser.
Inputs used
- BERTScore Precisionaccepts 0–1
- 0.92
- BERTScore Recallaccepts 0–1
- 0.88
Results
- BERTScore F1
- 0.8996
- Precision
- 0.9200
- Recall
- 0.8800
Step by step
- 1BERTScore F1: harmonic mean of P and R= 0.899556
Assumptions and limits
- The result is derived only from the 2 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to 4 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
Why is BERTScore better than BLEU for some tasks?
BERTScore captures semantic equivalence (paraphrases, synonyms) that surface-level n-gram metrics miss entirely. 'The cat sat on the mat' and 'A feline rested on the rug' have low BLEU but high BERTScore.
Which BERT model is used for scoring?
The default is often RoBERTa-large, but any contextual embedding model can be used. The choice affects absolute scores but relative rankings tend to be consistent across model choices.