Compute the Mish activation function output, a smooth self-regularized non-monotonic function.
Mish is a smooth, self-gated activation function similar in spirit to Swish: Mish(x) = x·tanh(softplus(x)) = x·tanh(ln(1+e^x)). Like Swish, it is smooth everywhere, non-monotonic (dipping slightly below zero for small negative inputs), and unbounded above for positive inputs, but its softplus-then-tanh construction gives it a slightly different curvature. Mish was introduced as an activation for computer vision architectures (notably used in YOLOv4) and has empirically shown modest accuracy improvements over both ReLU and Swish on several benchmark tasks, though at a higher computational cost due to the combination of exponential, logarithm, and tanh operations.
Mish(x) = x × tanh(ln(1 + e^x))
Both are smooth, self-gated activations of the form x × (gating function), but Swish gates with a sigmoid (x × σ(βx)) while Mish gates with tanh(softplus(x)) — the softplus-tanh combination gives Mish a slightly smoother, more gradual transition near zero and marginally different curvature for negative inputs.
Yes — computing Mish requires an exponential, a logarithm (for softplus), and a tanh, making it noticeably more expensive per-element than ReLU's single comparison or even Swish's single sigmoid, which is a practical tradeoff against its reported accuracy benefits.
Mish gained popularity in computer vision, most notably as the activation function used in YOLOv4's backbone network, where it contributed to measurable accuracy improvements over the ReLU-family activations used in earlier YOLO versions.