Calculate the total number of parameters in a neural network architecture.
Parameter count varies by layer type. A dense (fully connected) layer has params = input × output + output, where the extra output term is the bias vector. A Conv2D layer has params = kernel_h × kernel_w × in_channels × out_channels + out_channels, since the same kernel is shared across all spatial positions. An LSTM layer has four gates (input, forget, output, cell), each acting like a dense layer over the concatenated input and hidden state, giving params = 4 × [hidden × (input + hidden)] + 4 × hidden.
Dense layer
params = input_size * output_size + output_size
Conv2D layer
params = kernel_h * kernel_w * in_channels * out_channels + out_channels
LSTM layer
params = 4 * (hidden * (input + hidden)) + 4 * hidden
Convolutional layers share the same small kernel across every spatial location, so the parameter count depends only on kernel size and channel counts, not on the spatial dimensions of the input.
An LSTM cell contains four internal gates (input, forget, output, and cell/candidate), each with its own weight matrix and bias, so its parameter count is roughly four times that of one dense transformation of the same size.
No, this calculates only the specified layer type. BatchNorm/LayerNorm add a small number of additional parameters (typically 2 × channels) not included here.
Sum the parameter counts of every layer in the architecture. For transformer models specifically, see the Transformer Parameters Calculator for a purpose-built formula.