Swish Calculator
Compute the Swish (SiLU) activation function output for a given input value.
About Swish Calculator
Swish is a smooth, self-gated activation function: Swish(x) = x·σ(β·x), where the input gates itself through a sigmoid rather than being gated by a separate learned mechanism. With β=1, Swish is identical to SiLU (Sigmoid Linear Unit). Unlike ReLU, Swish is smooth and non-monotonic (it dips slightly below zero for small negative inputs before returning to near-zero for large negative inputs), and unlike ReLU it has no hard zero cutoff, which empirically improves optimization and final accuracy in deep networks like EfficientNet, where it was introduced.
Formula
Swish(x) = x × sigma(beta × x)
- x
- Input value
- sigma
- Sigmoid function
- beta
- Shape parameter (1.0 = SiLU)
Worked example
With the default values below, Swish Calculator returns Swish(x) of 1.761594. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Input (x)accepts -100–100
- 2
- Betaaccepts 0–10
- 1
Results
- Swish(x)
- 1.761594
- σ(β × x)
- 0.880797
Step by step
- 1σ(β × x)= 0.880797
- 2Swish(x) = x × σ(β × x)= 1.761594
Assumptions and limits
- The result is derived only from the 2 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to 6 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
What is the difference between Swish and SiLU?
SiLU is simply Swish with β fixed at exactly 1.0 — the two names are often used interchangeably in practice, since most implementations that reference 'Swish' use the β=1 variant rather than treating β as a learnable or tunable parameter.
How does beta affect the shape of Swish?
As β → 0, Swish approaches a scaled linear function (x/2); as β → ∞, Swish approaches ReLU's shape (a sharp cutoff at zero) — β=1 (SiLU) sits between these extremes as a smooth, gentle curve, and it can be made a learnable parameter to let the network adapt the activation shape during training.
Why does Swish sometimes outperform ReLU in practice?
Swish's smoothness (no sharp corner at zero) and slight non-monotonicity (allowing small negative outputs near zero) appear to improve gradient flow and optimization landscape in deep networks, which is why it was found to outperform ReLU on image classification benchmarks like ImageNet in the original EfficientNet research.