Trainable Parameters Calculator
Calculate the number of trainable parameters in a model with frozen layers.
Inputs
Trainable Parameters
350,000,000
Frozen Parameters
6,650,000,000
Trainable %
5.00%
Step by step
Frozen params
7,000,000,000 × 95%
= 6,650,000,000
Trainable params
7,000,000,000 − 6,650,000,000
= 350,000,000
How it works
When fine-tuning, you often freeze most layers and only train a small subset. LoRA typically trains <1% of total parameters. This calculator shows how many parameters are actually updated during training based on the frozen percentage.
Formula
Trainable Params
trainable = total_params × (1 - frozen_pct/100)
- frozen_pct
- Percentage of parameters that are frozen (not updated)
Frequently Asked Questions
Why freeze layers?
Freezing pretrained layers reduces compute, memory, and risk of catastrophic forgetting. Only training top layers or adapter modules (LoRA) achieves good results with far less resources.
How does LoRA reduce trainable parameters?
LoRA adds small low-rank matrices alongside frozen weight matrices. With rank=8 on a 7B model, you might only train 0.1% of total parameters while achieving near-full fine-tuning quality.