Compute the sigmoid activation function output for a given input value.
The sigmoid function squashes any real-valued input into the range (0, 1): σ(x) = 1/(1+e^(−x)), making it a natural choice for representing probabilities and for binary classification output layers. Its derivative, σ'(x) = σ(x)·(1−σ(x)), is conveniently expressed in terms of the function's own output, which made it historically popular for hand-derived backpropagation. However, the derivative is at most 0.25 (at x=0) and shrinks toward 0 for large |x|, causing the 'vanishing gradient' problem in deep networks — this is why ReLU and its variants have largely replaced sigmoid in hidden layers, though sigmoid remains standard for binary output layers and gates (e.g. LSTM gates).
Sigmoid function
sigma(x) = 1 / (1 + e^(-x))
Sigmoid derivative
sigma'(x) = sigma(x) × (1 - sigma(x))
The sigmoid derivative σ(x)×(1−σ(x)) has a maximum value of only 0.25 (at x=0) and approaches 0 as |x| grows large in either direction, so in deep networks, gradients get multiplied by these small values at every layer during backpropagation, shrinking exponentially and making early layers learn very slowly.
Sigmoid remains the standard choice for binary classification output layers (interpreting the output as a probability) and for gate mechanisms inside LSTM and GRU cells, where its bounded (0,1) output naturally represents a 'how much to let through' gating signal.
Sigmoid outputs values strictly between 0 and 1 (never reaching either endpoint), with σ(0) = 0.5 exactly — this midpoint is why 0.5 is the conventional decision threshold for binary classifiers using a sigmoid output.