Calculate the expected number of active neurons given a dropout probability.
Dropout randomly zeroes each neuron's activation with probability p during training, so the expected number of active neurons in a layer of size N is N × (1 − p). At inference time, standard ('inverted') dropout implementations instead keep all neurons active and scale outputs by 1 / (1 − p) during training, so the effective magnitude of activations matches what the network will see at inference (or apply no scaling at inference and scale by (1-p) then instead — frameworks like PyTorch and TensorFlow use inverted dropout by default, applying the 1/(1−p) scaling during training only).
Expected active neurons
active_neurons = total_neurons × (1 - p)
Inverted-dropout scale factor
scale = 1 / (1 - p)
0.5 (50%) is the classic default for fully connected layers from the original dropout paper, though modern architectures often use lower rates (0.1–0.3) for convolutional layers and even lower or no dropout in transformer feed-forward blocks when other regularization is present.
During training, only (1−p) fraction of neurons contribute to each forward pass, so their outputs are scaled up by 1/(1−p) to keep the expected activation magnitude constant; at inference all neurons are active without dropout, so no scaling is needed if this 'inverted dropout' convention was used during training.
Standard dropout zeroes entire neuron activations (not individual weights), which is why it's described in terms of 'active neurons' — variants like DropConnect instead drop individual weight connections.
It's common practice to use lower dropout near the input (or none at all) and higher dropout in deeper, wider layers with more redundant capacity, since early layers often extract lower-level features that are more sensitive to being randomly zeroed.