Calculate the cross-entropy loss between predicted and true probability distributions.
Cross-entropy measures how well a predicted probability distribution q matches a true distribution p: H(p,q) = -Σ p(x) × log(q(x)). It's the standard loss function for classification tasks, penalizing confident-but-wrong predictions heavily due to the logarithm. Cross-entropy decomposes into H(p,q) = H(p) + KL(p‖q), where H(p) is the true distribution's own entropy and KL(p‖q) is the extra 'inefficiency' from using q instead of p — since H(p) is fixed for a given label, minimizing cross-entropy during training is equivalent to minimizing KL divergence.
H(p, q) = -sum(p(x) * ln(q(x)))
It directly measures the distance between predicted and true probability distributions in a way that's differentiable and penalizes confidently wrong predictions much more heavily than uncertain ones, providing strong gradient signal during training.
That happens only when the predicted distribution q exactly matches the true distribution p, meaning KL divergence is zero and the predictions are perfect.
Perplexity is simply e raised to the cross-entropy loss (when using natural log); it's commonly used to evaluate language models because it has a more intuitive interpretation as an 'effective vocabulary size' of uncertainty.
log(0) is undefined (negative infinity), so in practice predictions are clipped to a small epsilon value to avoid numerical errors — this calculator applies that safeguard automatically.