Mish Activation Calculator
Calculate the Mish activation output: x times tanh(softplus(x)).
Inputs
Mish(x)
1.943959
softplus(x)
2.126928
tanh(softplus(x))
0.971979
Step by step
softplus(x) = ln(1 + e^x)
ln(1 + e^2)
= 2.126928
tanh(softplus(x))
tanh(2.126928)
= 0.971979
Mish(x) = x × tanh(softplus(x))
2 × 0.971979
= 1.943959
How it works
Mish is a smooth, self-gated activation function similar in spirit to Swish: Mish(x) = x·tanh(softplus(x)) = x·tanh(ln(1+e^x)). Like Swish, it is smooth everywhere, non-monotonic (dipping slightly below zero for small negative inputs), and unbounded above for positive inputs, but its softplus-then-tanh construction gives it a slightly different curvature. Mish was introduced as an activation for computer vision architectures (notably used in YOLOv4) and has empirically shown modest accuracy improvements over both ReLU and Swish on several benchmark tasks, though at a higher computational cost due to the combination of exponential, logarithm, and tanh operations.
Formula
Mish(x) = x × tanh(ln(1 + e^x))
- x
- Input value
- tanh
- Hyperbolic tangent function
- ln
- Natural logarithm
Frequently Asked Questions
How is Mish different from Swish?
Both are smooth, self-gated activations of the form x × (gating function), but Swish gates with a sigmoid (x × σ(βx)) while Mish gates with tanh(softplus(x)) — the softplus-tanh combination gives Mish a slightly smoother, more gradual transition near zero and marginally different curvature for negative inputs.
Is Mish more expensive to compute than ReLU or Swish?
Yes — computing Mish requires an exponential, a logarithm (for softplus), and a tanh, making it noticeably more expensive per-element than ReLU's single comparison or even Swish's single sigmoid, which is a practical tradeoff against its reported accuracy benefits.
Where has Mish been used successfully?
Mish gained popularity in computer vision, most notably as the activation function used in YOLOv4's backbone network, where it contributed to measurable accuracy improvements over the ReLU-family activations used in earlier YOLO versions.