Cohen Kappa Calculator
Calculate Cohen's Kappa to measure inter-rater agreement corrected for chance.
About Cohen Kappa Calculator
Cohen's Kappa measures agreement between two raters (or a model's predictions and true labels) while correcting for the agreement expected by chance alone: κ = (p_o − p_e) / (1 − p_e), where p_o is observed agreement and p_e is the agreement expected under random labeling given the class distributions. Kappa = 1 means perfect agreement, 0 means agreement no better than chance, and negative values mean agreement worse than chance. It's especially useful over raw accuracy when classes are imbalanced, since chance agreement is high in that case.
Formula
kappa = (p_o - p_e) / (1 - p_e)
- p_o
- Observed agreement fraction
- p_e
- Expected agreement by chance
Worked example
With the default values below, Cohen Kappa Calculator returns Cohen's Kappa of 0.7000. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Observed Agreement (p_o)accepts 0–1
- 0.85
- Expected Agreement by Chance (p_e)accepts 0–1
- 0.5
Results
- Cohen's Kappa
- 0.7000
- Observed Agreement
- 0.85%
- Expected Agreement (Chance)
- 0.50%
Step by step
- 1Kappa: (p_o − p_e) / (1 − p_e)= 0.7000
Assumptions and limits
- The result is derived only from the 2 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax or fees or charges into account.
- Figures are rounded for display to 4 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
How is expected agreement (p_e) calculated?
p_e is computed from the marginal probabilities of each class for both raters: for each class, multiply its frequency under rater A by its frequency under rater B, then sum across all classes — this gives the agreement rate expected if labels were assigned independently at random.
What kappa value indicates good agreement?
A common rule of thumb (Landis & Koch) treats 0.61-0.80 as 'substantial' agreement and above 0.80 as 'almost perfect', though acceptable thresholds vary by field and application.
Why not just use raw percent agreement?
Raw percent agreement doesn't account for the fact that some agreement would happen purely by chance, especially when one class is much more common — Kappa corrects for this baseline, giving a fairer measure of true agreement.