Calculate the number of trainable parameters in an LSTM layer.
An LSTM cell has four internal gates — forget, input, cell (candidate), and output — each of which is a dense transformation over the concatenated input and previous hidden state, plus a bias vector. This gives params = 4 × [(input_size + hidden_size) × hidden_size + hidden_size] for a single layer. When LSTM layers are stacked, every layer after the first takes the previous layer's hidden size as its input size, since the previous layer's hidden state becomes the next layer's input.
params = 4 * ((input_size + hidden_size) * hidden_size + hidden_size)
An LSTM cell contains four gates (forget, input, cell/candidate, and output), and each gate has its own independent weight matrix and bias, so the total parameter count is four times that of a single dense transformation.
In a stacked (multi-layer) LSTM, each layer after the first receives the hidden state output of the previous layer as its input, so its effective input dimension equals the previous layer's hidden size rather than the original input size.
A GRU has only three gates (reset, update, and the candidate activation) instead of four, so a GRU with the same input/hidden size has roughly 75% of an equivalent LSTM's parameters.
No, this counts only the recurrent LSTM cell(s). Any subsequent dense/output projection layer's parameters should be added separately, e.g. using the Parameter Count Calculator.