Calculate the number of trainable parameters in a GRU layer.
A GRU has 3 gates (reset, update, new), each with input-to-hidden and hidden-to-hidden weight matrices. Total parameters = 3 × (input_size × hidden_size + hidden_size²) + biases. GRUs have 75% of LSTM parameters (3 gates vs 4).
GRU Parameters
params = 3 × (input_size × hidden_size + hidden_size^2) + 3 × 2 × hidden_size
GRUs have 3 gates vs LSTMs' 4 gates, so GRUs have 75% the parameters of an equivalently-sized LSTM. GRUs often match LSTM performance with fewer parameters.
GRUs are preferred when you want fewer parameters and faster training. LSTMs may perform slightly better on very long sequences due to their separate cell state mechanism.