Dense Layer Parameters Calculator
Calculate the number of weights and biases in a fully connected dense layer.
Inputs
Total Parameters
393,728
Weight Parameters
393,216
Bias Parameters
512
Step by step
Weights: input × output
768 × 512
= 393,216
Biases
512
= 512
Total parameters
393,216 + 512
= 393,728
How it works
A dense (fully connected) layer has input_size × output_size weights plus output_size biases. This is the fundamental building block of neural networks, connecting every input neuron to every output neuron.
Formula
Dense Layer
params = input_size × output_size + output_size (if bias)
- input_size
- Number of input features
- output_size
- Number of output neurons
Frequently Asked Questions
Why are dense layers expensive?
Parameters grow quadratically (O(n²)) with layer width. A 4096→4096 dense layer has 16.7M parameters, which is why transformers use attention (O(n×d)) instead of fully connected layers for sequence modeling.
When should I omit bias?
Bias is often omitted before batch normalization (which has its own bias term) or in certain architectural choices like transformer attention projections.
You might also need
- CNN Parameter Count CalculatorCommonly used together
- Trainable Parameters CalculatorCommonly used together
- Parameter Count CalculatorCommonly used together
- Multi-head Attention Parameters CalculatorAlso in Neural Networks
- GRU Parameters CalculatorAlso in Neural Networks
- RNN Parameters CalculatorAlso in Neural Networks