Categorical Cross-Entropy Calculator
Calculate categorical cross-entropy loss between a one-hot true label and predicted class probabilities.
Inputs
Exactly one entry should be 1; the rest 0, e.g. "0, 1, 0, 0".
Model's predicted probability for each class; should sum to ~1.
Categorical Cross-Entropy Loss
0.356675
True Class Index
1
Sum of Predicted Probabilities
1.0000
Should be ≈1.0 for a valid probability distribution.
Step by step
Per-class terms: −y_i × ln(p_i) (only the true class contributes)
−Σ y_i × ln(p_i)
= 0.0000, 0.3567, 0.0000, 0.0000
CCE: sum of per-class terms
Σ [0.0000, 0.3567, 0.0000, 0.0000]
= 0.356675
Sanity check: predicted probabilities should sum to ≈1
Σ p_i
= 1.0000
How it works
Categorical cross-entropy generalizes binary cross-entropy to multi-class classification: CCE = −Σ y_i·ln(p_i), summed over all classes. Because the true label is one-hot (exactly one class has y_i = 1 and the rest are 0), every term except the true class's vanishes, so CCE reduces to −ln(p_true) — the loss is entirely determined by how much probability mass the model assigned to the correct class. This is the standard loss paired with a softmax output layer for multi-class neural network classifiers.
Formula
CCE = -sum(y_i × ln(p_i))
- y_i
- True label for class i (one-hot: 1 for correct class, 0 otherwise)
- p_i
- Predicted probability for class i
Frequently Asked Questions
Why do all terms except the true class disappear?
Because the true label is one-hot encoded, y_i = 0 for every incorrect class, making y_i × ln(p_i) = 0 for those terms; only the true class (where y_i = 1) contributes −ln(p_true) to the total loss.
What predicted probability gives the lowest possible loss?
A predicted probability of 1.0 for the true class gives a loss of exactly 0 (since −ln(1) = 0), which is the theoretical minimum; in practice predictions never reach exactly 1.0 due to the softmax function's asymptotic behavior.
How does this relate to softmax cross-entropy loss in frameworks like PyTorch?
PyTorch's CrossEntropyLoss and TensorFlow's CategoricalCrossentropy both implement this exact formula, typically combining it with the softmax computation internally (from raw logits) for better numerical stability than computing softmax and cross-entropy as separate steps.