Work out model parameter count instantly with clear inputs, formula shown and shareable results.
A transformer block holds four d-by-d attention projections (query, key, value and output), giving 4d^2, plus a feed-forward network of two matrices d-by-md and md-by-d, giving 2md^2. With the usual expansion factor of 4 that is 12d^2 per layer. Adding the vocab-by-d embedding table reproduces published parameter counts closely, since layer norms and biases contribute well under one percent.
Transformer parameter count
per layer = 4d^2 (attention) + 2 m d^2 (FFN); total = layers x per layer + vocab x d
Models with an untied output head have a second vocab-by-d matrix. Weight tying reuses the input embedding and saves those parameters.
Yes. Sharing key and value projections across query heads shrinks two of the four projections, so attention parameters fall below 4d^2.