LSTM Parameters Calculator
Calculate the total trainable parameter count of an LSTM layer from its input and hidden sizes.
Inputs
Dimensionality of the input vector at each timestep.
Stacked LSTM layers; layers after the first take the previous hidden size as input.
Total LSTM Parameters
1,665,024
First Layer Parameters
1,665,024
Approx. Parameters per Gate
416,256
Step by step
Per-layer LSTM params: 4 × [(input + hidden) × hidden + hidden]
4 × [(300 + 512) × 512 + 512]
= 1,665,024
Stacked layers: first layer + (num_layers − 1) × [4 × ((hidden + hidden) × hidden + hidden)]
1 layer(s) total
= 1,665,024
Approx. parameters per gate (forget/input/cell/output)
1,665,024 ÷ 4 ÷ 1
= 416,256
How it works
An LSTM cell has four internal gates — forget, input, cell (candidate), and output — each of which is a dense transformation over the concatenated input and previous hidden state, plus a bias vector. This gives params = 4 × [(input_size + hidden_size) × hidden_size + hidden_size] for a single layer. When LSTM layers are stacked, every layer after the first takes the previous layer's hidden size as its input size, since the previous layer's hidden state becomes the next layer's input.
Formula
params = 4 * ((input_size + hidden_size) * hidden_size + hidden_size)
- n_i
- Input size (first layer) or hidden size (subsequent layers)
- n_h
- Hidden size (LSTM cell units)
Frequently Asked Questions
Why is there a factor of 4 in the formula?
An LSTM cell contains four gates (forget, input, cell/candidate, and output), and each gate has its own independent weight matrix and bias, so the total parameter count is four times that of a single dense transformation.
Why does layer 2+ use hidden_size as its input size?
In a stacked (multi-layer) LSTM, each layer after the first receives the hidden state output of the previous layer as its input, so its effective input dimension equals the previous layer's hidden size rather than the original input size.
How does this compare to a GRU's parameter count?
A GRU has only three gates (reset, update, and the candidate activation) instead of four, so a GRU with the same input/hidden size has roughly 75% of an equivalent LSTM's parameters.
Does this include parameters for a final output/projection layer?
No, this counts only the recurrent LSTM cell(s). Any subsequent dense/output projection layer's parameters should be added separately, e.g. using the Parameter Count Calculator.