ELU Calculator
Compute the Exponential Linear Unit activation function for a given input.
About ELU Calculator
ELU behaves like the identity for positive inputs but smoothly saturates toward −α for large negative inputs instead of flattening to a fixed slope: ELU(x) = x for x > 0, and α·(e^x − 1) for x ≤ 0. Unlike ReLU and Leaky ReLU, ELU is smooth (continuously differentiable) at x=0, and its negative saturation value pushes mean activations closer to zero across a layer, which can speed up training convergence compared to ReLU-family functions that only allow non-negative (or unboundedly negative, for Leaky ReLU) outputs.
Formula
ELU(x) = x if x > 0, else alpha × (e^x - 1)
- x
- Input value
- alpha
- Saturation magnitude for negative inputs (default 1.0)
Worked example
With the default values below, ELU Calculator returns ELU(x) of -0.632121. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Input (x)accepts -100–100
- -1
- Alphaaccepts 0.01–10
- 1
Results
- ELU(x)
- -0.632121
- ELU'(x)
- 0.367879
Step by step
- 1ELU(x) = α × (e^x − 1) (negative branch)= -0.632121
- 2Derivative: 1 if x > 0, else ELU(x) + α= 0.367879
Assumptions and limits
- The result is derived only from the 2 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to 6 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
Why does ELU saturate at −α instead of continuing linearly like Leaky ReLU?
The exponential term e^x approaches 0 as x becomes very negative, so α×(e^x − 1) approaches −α — this bounded negative saturation (rather than an unbounded linear decrease) helps push a layer's mean pre-activation closer to zero, which some research suggests improves training dynamics.
Is ELU differentiable everywhere, including at x = 0?
Yes — unlike ReLU and Leaky ReLU which have a sharp corner at x=0, ELU's positive and negative branches meet with matching slopes at x=0 when derivative continuity is checked, making it a smooth (C¹ continuous) function throughout.
What's a typical value for alpha in ELU?
1.0 is the standard default from the original ELU paper and is used in the vast majority of implementations; larger alpha values increase how negative the saturation point can go, but 1.0 works well across most architectures without additional tuning.