Calculate categorical cross-entropy loss for multi-class classification tasks.
Categorical cross-entropy generalizes binary cross-entropy to multi-class classification: CCE = −Σ y_i·ln(p_i), summed over all classes. Because the true label is one-hot (exactly one class has y_i = 1 and the rest are 0), every term except the true class's vanishes, so CCE reduces to −ln(p_true) — the loss is entirely determined by how much probability mass the model assigned to the correct class. This is the standard loss paired with a softmax output layer for multi-class neural network classifiers.
CCE = -sum(y_i × ln(p_i))
Because the true label is one-hot encoded, y_i = 0 for every incorrect class, making y_i × ln(p_i) = 0 for those terms; only the true class (where y_i = 1) contributes −ln(p_true) to the total loss.
A predicted probability of 1.0 for the true class gives a loss of exactly 0 (since −ln(1) = 0), which is the theoretical minimum; in practice predictions never reach exactly 1.0 due to the softmax function's asymptotic behavior.
PyTorch's CrossEntropyLoss and TensorFlow's CategoricalCrossentropy both implement this exact formula, typically combining it with the softmax computation internally (from raw logits) for better numerical stability than computing softmax and cross-entropy as separate steps.