Work out inter rater reliability instantly with clear inputs, formula shown and shareable results.
Cohen's kappa corrects raw agreement for the agreement expected by chance: (po - pe)/(1 - pe). With 45 and 30 agreements against 25 disagreements, observed agreement is 75% but chance agreement is 51%, giving kappa 0.490 - only moderate. Raw agreement always flatters, which is why kappa is the reporting standard.
Cohen's kappa
kappa = (observed - expected) / (1 - expected)
Expected agreement
pe = sum of the products of the marginal proportions
Because when one category dominates, chance agreement is already high. 90% agreement on a 95% prevalent category can give a kappa near zero.
Above 0.61 is usually considered substantial and 0.81 almost perfect, though clinical decisions often demand 0.80 or better.