RNN Parameters Calculator
Calculate the number of trainable parameters in a recurrent neural network layer.
Inputs
Total Parameters
394,240
Input Weights
131,072
Hidden Weights
262,144
Step by step
Input→Hidden weights: input × hidden
256 × 512
= 131,072
Hidden→Hidden weights: hidden × hidden
512 × 512
= 262,144
Biases (input + hidden)
2 × 512
= 1,024
Total
131072 + 262144 + 1024
= 394,240
How it works
A vanilla RNN layer has two weight matrices: W_ih (input to hidden, size input×hidden) and W_hh (hidden to hidden, size hidden×hidden), plus two bias vectors of size hidden. Total = input×hidden + hidden² + 2×hidden.
Formula
RNN Parameters
params = input_size × hidden_size + hidden_size^2 + 2 × hidden_size
- input_size
- Dimensionality of input features
- hidden_size
- Dimensionality of hidden state
Frequently Asked Questions
How do LSTM parameters compare to vanilla RNN?
LSTMs have 4× the parameters of a vanilla RNN because they have four gate matrices (input, forget, cell, output) instead of one.
Why does hidden size dominate parameter count?
The hidden-to-hidden matrix is h×h, which grows quadratically. For large hidden sizes (512+), this dominates total parameters.
You might also need
- GRU Parameters CalculatorCommonly used together
- Parameter Count CalculatorCommonly used together
- LSTM Parameters CalculatorCommonly used together
- Multi-head Attention Parameters CalculatorAlso in Neural Networks
- CNN Parameter Count CalculatorAlso in Neural Networks
- Trainable Parameters CalculatorAlso in Neural Networks