Skip to content
ENExplore Calculators

LoRA Parameter Count Calculator

Work out lora parameter count instantly with clear inputs, formula shown and shareable results.

About LoRA Parameter Count Calculator

LoRA replaces a full weight update with the product of two thin matrices: for a d-by-d projection it trains a d-by-r and an r-by-d pair, so 2dr parameters instead of d^2. At rank 16 on a 4096-dimensional model that is 131,072 parameters per matrix versus 16.7 million, which is why adapters are a few tens of megabytes and can be swapped per tenant at serving time.

Formula

LoRA parameter count

per module = 2 x d x r; total = per module x modules per layer x layers; base per layer = 12 d^2

Worked example

With the default values below, LoRA Parameter Count Calculator returns Trainable LoRA parameters of 16,777,216. Change any input above and every figure here is recalculated in your browser.

Inputs used

Hidden dimensionminimum 8
4,096
LoRA rank (r)accepts 1–512
16
Layers adaptedminimum 1
32
Adapted matrices per layeraccepts 1–12
4

Results

Trainable LoRA parameters
16,777,216
Parameters per adapted matrix
131,072
Share of the base model retrained
0.2604%
Adapter size in fp16
32.00 MiB

Step by step

  1. 1Values used
  2. 2LoRA parameter count
  3. 3Trainable LoRA parameters= 16,777,216
  4. 4Parameters per adapted matrix= 131,072
  5. 5Share of the base model retrained= 0.2604
  6. 6Adapter size in fp16= 33,554,432

Assumptions and limits

  • The result is derived only from the 4 inputs listed above. Anything not among them is not part of the calculation.
  • In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
  • Figures are rounded for display to 4 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

Which matrices should be adapted?

Query and value projections are the classic minimum; adapting all four attention projections plus the FFN matrices gives more capacity at proportionally more parameters.

How high should rank go?

Rank 8 to 16 handles style and format adaptation. New knowledge or a new language usually needs 64 or more, and beyond about 256 full fine-tuning becomes competitive.

Related Calculators

LoRA Parameter Count Calculator | Calcrivo