Calculate the number of weights and biases in a fully connected dense layer.
A dense (fully connected) layer has input_size × output_size weights plus output_size biases. This is the fundamental building block of neural networks, connecting every input neuron to every output neuron.
Dense Layer
params = input_size × output_size + output_size (if bias)
Parameters grow quadratically (O(n²)) with layer width. A 4096→4096 dense layer has 16.7M parameters, which is why transformers use attention (O(n×d)) instead of fully connected layers for sequence modeling.
Bias is often omitted before batch normalization (which has its own bias term) or in certain architectural choices like transformer attention projections.