Softmax Calculator
Convert a vector of logits into a normalized probability distribution using the softmax function.
Inputs
Comma-separated raw model output values, e.g. 2.0, 1.0, 0.1
Divides logits before exponentiation. Lower values sharpen the distribution; higher values flatten it.
Softmax Probabilities
0
Max Probability
0.6590
Distribution Entropy
0.8467nats
Step by step
Scale logits by temperature: x_i / T
[2, 1, 0.1] ÷ 1
= [2.000, 1.000, 0.100]
Exponentiate (shifted by max for stability): e^(x_i − max)
e^(x_i − 2.000)
= [1.0000, 0.3679, 0.1496]
Normalize: e^(x_i) / Σ e^(x_j)
each value ÷ 1.5174
= [0.6590, 0.2424, 0.0986]
How it works
Softmax converts a vector of arbitrary real-valued logits into a probability distribution that sums to 1: softmax(x_i) = e^(x_i) / Σ e^(x_j). It's the standard final activation for multi-class classification and the mechanism behind attention weights in transformers. The temperature parameter scales logits before exponentiation — lower temperature sharpens the distribution toward the largest logit (more confident/deterministic), while higher temperature flattens it toward uniform (more random), which is exactly how temperature sampling works in LLM text generation.
Formula
softmax(x_i) = e^(x_i / T) / sum(e^(x_j / T))
- x_i
- Logit value for class i
- T
- Temperature parameter (default 1)
Frequently Asked Questions
Why subtract the maximum value before exponentiating?
This is a standard numerical stability trick: subtracting the max logit before exponentiating prevents overflow from very large exponentials, and it doesn't change the final result because it cancels out in the normalization.
How does temperature affect the output?
Temperature below 1 makes the distribution more peaked (closer to a one-hot/argmax choice), while temperature above 1 flattens it toward uniform, increasing randomness — this is the same temperature parameter used in LLM sampling.
What is entropy telling me here?
Entropy measures how uncertain or spread out the resulting probability distribution is; low entropy means the model is confident in one class, while high entropy means probabilities are more evenly spread across classes.
How is softmax related to cross-entropy loss?
Softmax converts logits to probabilities, and cross-entropy loss then measures how far those predicted probabilities are from the true label distribution — together they form the standard classification loss (softmax + cross-entropy).