Dropout Probability Calculator
Calculate the expected number of active neurons given a dropout probability.
About Dropout Probability Calculator
Dropout randomly zeroes each neuron's activation with probability p during training, so the expected number of active neurons in a layer of size N is N × (1 − p). At inference time, standard ('inverted') dropout implementations instead keep all neurons active and scale outputs by 1 / (1 − p) during training, so the effective magnitude of activations matches what the network will see at inference (or apply no scaling at inference and scale by (1-p) then instead — frameworks like PyTorch and TensorFlow use inverted dropout by default, applying the 1/(1−p) scaling during training only).
Formulas
Expected active neurons
active_neurons = total_neurons × (1 - p)
- total_neurons
- Total neurons in the layer
- p
- Dropout probability
Inverted-dropout scale factor
scale = 1 / (1 - p)
- p
- Dropout probability
Worked example
With the default values below, Dropout Probability Calculator returns Expected Active Neurons of 819.2. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Total Neurons in Layeraccepts 1–10,000,000
- 1,024
- Dropout Probability (p)accepts 0–95
- 20
Results
- Expected Active Neurons
- 819.2
- Expected Dropped Neurons
- 204.8
- Retained Fraction
- 80.0%
- Inverted-Dropout Scale Factor
- 1.2500
- Guidance
- Typical hidden layer size — dropout of 20–50% is common for fully connected layers.
Step by step
- 1Retained fraction: 1 − p= 0.800
- 2Active neurons (expected): total × (1 − p)= 819.2
- 3Inverted-dropout inference scale: 1 ÷ (1 − p)= 1.2500
Assumptions and limits
- The result is derived only from the 2 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to 4 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
What's a typical dropout rate?
0.5 (50%) is the classic default for fully connected layers from the original dropout paper, though modern architectures often use lower rates (0.1–0.3) for convolutional layers and even lower or no dropout in transformer feed-forward blocks when other regularization is present.
Why does inference use a scale factor?
During training, only (1−p) fraction of neurons contribute to each forward pass, so their outputs are scaled up by 1/(1−p) to keep the expected activation magnitude constant; at inference all neurons are active without dropout, so no scaling is needed if this 'inverted dropout' convention was used during training.
Does dropout apply per-neuron or per-weight?
Standard dropout zeroes entire neuron activations (not individual weights), which is why it's described in terms of 'active neurons' — variants like DropConnect instead drop individual weight connections.
Should dropout rate differ by layer depth?
It's common practice to use lower dropout near the input (or none at all) and higher dropout in deeper, wider layers with more redundant capacity, since early layers often extract lower-level features that are more sensitive to being randomly zeroed.