Work out coding reliability (qualitative) instantly with clear inputs, formula shown and shareable results.
Percent agreement between coders always flatters, because agreement happens by chance too. With six equally likely categories, chance agreement is 16.7%, so 84% raw agreement corrects to a kappa of 0.808 - strong. Qualitative research conventionally requires kappa above 0.80 for published coding and 0.67 for exploratory work.
Percent agreement
po = agreed decisions / total decisions
Chance-corrected kappa
kappa = (po - pe) / (1 - pe), with pe = 1 / categories
Because two coders assigning codes at random would still agree sometimes. With two categories chance agreement is 50%, so raw agreement is nearly meaningless.
Commonly 10-25% of the material is double-coded to establish reliability, then one coder completes the rest once agreement is demonstrated.