Positional Encoding Calculator

Compute sinusoidal positional encoding values for a given position and dimension.

About Positional Encoding Calculator

Sinusoidal positional encoding injects position information into transformer inputs without learned parameters. Even dimensions use sin, odd dimensions use cos, with wavelengths forming a geometric progression from 2π to 10000×2π, allowing the model to attend to relative positions.

Formula

Positional Encoding

PE(pos, 2i) = sin(pos / 10000^(2i/d_model)); PE(pos, 2i+1) = cos(pos / 10000^(2i/d_model))

pos: Token position in the sequence
i: Dimension index

Worked example

With the default values below, Positional Encoding Calculator returns PE sin (even dim) of -0.544021. Change any input above and every figure here is recalculated in your browser.

Inputs used

Positionminimum 0: 10
Model Dimensionminimum 2: 512
Dimension Index (i)minimum 0: 0

Results

PE sin (even dim): -0.544021
PE cos (odd dim): -0.839072
Angle: 10.000000

Step by step

1Angle = pos / 10000^(2i/d_model)= 10.000000
2PE(pos, 2i) = sin(angle)= -0.544021
3PE(pos, 2i+1) = cos(angle)= -0.839072

Assumptions and limits

The result is derived only from the 3 inputs listed above. Anything not among them is not part of the calculation.
In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
Figures are rounded for display to 6 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

Why use sin and cos instead of learned embeddings?

Sinusoidal encodings generalize to longer sequences than seen during training and don't add trainable parameters. However, modern models (GPT, BERT) often use learned positional embeddings or RoPE instead.

Why the 10000 base?

10000 creates a range of wavelengths so that nearby positions have similar encodings while distant positions are easily distinguishable. It was chosen empirically in the original Transformer paper.