Huber Loss Calculator
Calculate Huber loss for a single residual, a robust loss that blends MSE and MAE behavior.
Inputs
Threshold where the loss transitions from quadratic to linear.
Huber Loss
2.000000
Residual (ŷ − y)
2.5000
Quadratic Region?
false
Step by step
Residual: a = ŷ − y
12.5 − 10
= 2.5000
Linear region (|a| > δ): δ × (|a| − 0.5 × δ)
1 × (2.5000 − 0.5 × 1)
= 2.000000
How it works
Huber loss combines the best properties of MSE and MAE: for small residuals (|a| ≤ δ), it behaves like squared error, Huber(a) = 0.5 × a²; for large residuals (|a| > δ), it switches to a linear penalty, Huber(a) = δ × (|a| − 0.5 × δ). This makes it quadratic (and smoothly differentiable) near zero for stable gradients, while remaining linear — and therefore robust to outliers — for large errors, unlike pure MSE which lets outliers dominate the loss. The delta parameter controls where this transition happens: smaller delta makes the loss behave more like MAE, larger delta makes it behave more like MSE.
Formulas
Quadratic region (|a| ≤ δ)
L = 0.5 * a^2
- a
- Residual (predicted - actual)
- \delta
- Threshold between quadratic and linear regions
Linear region (|a| > δ)
L = delta * (|a| - 0.5 * delta)
- a
- Residual (predicted - actual)
- \delta
- Threshold between quadratic and linear regions
Frequently Asked Questions
How do I choose the delta parameter?
Delta should reflect the scale of 'normal' residuals you expect — errors smaller than delta are treated as ordinary noise (quadratic penalty), while errors larger than delta are treated as likely outliers (linear penalty); it's often tuned via cross-validation or set based on domain knowledge of acceptable error magnitude.
Why is Huber loss preferred over MSE for noisy data?
MSE squares every error, so a few extreme outliers can dominate the total loss and distort model training; Huber loss caps the penalty growth to linear beyond delta, making the model less sensitive to a small number of extreme residuals.
What happens as delta approaches infinity or zero?
As delta → ∞, Huber loss behaves exactly like MSE everywhere (always quadratic); as delta → 0, it behaves increasingly like MAE (nearly always linear), so delta effectively interpolates between the two loss functions.