LSTM Parameters Calculator
Calculate the number of trainable parameters in an LSTM layer.
About LSTM Parameters Calculator
An LSTM cell has four internal gates — forget, input, cell (candidate), and output — each of which is a dense transformation over the concatenated input and previous hidden state, plus a bias vector. This gives params = 4 × [(input_size + hidden_size) × hidden_size + hidden_size] for a single layer. When LSTM layers are stacked, every layer after the first takes the previous layer's hidden size as its input size, since the previous layer's hidden state becomes the next layer's input.
Formula
params = 4 * ((input_size + hidden_size) * hidden_size + hidden_size)
- n_i
- Input size (first layer) or hidden size (subsequent layers)
- n_h
- Hidden size (LSTM cell units)
Worked example
With the default values below, LSTM Parameters Calculator returns Total LSTM Parameters of 1,665,024. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Input Sizeaccepts 1–1,000,000
- 300
- Hidden Sizeaccepts 1–1,000,000
- 512
- Number of Stacked Layersaccepts 1–1,000
- 1
Results
- Total LSTM Parameters
- 1,665,024
- First Layer Parameters
- 1,665,024
- Approx. Parameters per Gate
- 416,256
Step by step
- 1Per-layer LSTM params: 4 × [(input + hidden) × hidden + hidden]= 1,665,024
- 2Stacked layers: first layer + (num_layers − 1) × [4 × ((hidden + hidden) × hidden + hidden)]= 1,665,024
- 3Approx. parameters per gate (forget/input/cell/output)= 416,256
Assumptions and limits
- The result is derived only from the 3 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to whole numbers, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
Why is there a factor of 4 in the formula?
An LSTM cell contains four gates (forget, input, cell/candidate, and output), and each gate has its own independent weight matrix and bias, so the total parameter count is four times that of a single dense transformation.
Why does layer 2+ use hidden_size as its input size?
In a stacked (multi-layer) LSTM, each layer after the first receives the hidden state output of the previous layer as its input, so its effective input dimension equals the previous layer's hidden size rather than the original input size.
How does this compare to a GRU's parameter count?
A GRU has only three gates (reset, update, and the candidate activation) instead of four, so a GRU with the same input/hidden size has roughly 75% of an equivalent LSTM's parameters.
Does this include parameters for a final output/projection layer?
No, this counts only the recurrent LSTM cell(s). Any subsequent dense/output projection layer's parameters should be added separately, e.g. using the Parameter Count Calculator.