Top K Retrieval Sizing Calculator

Work out top k retrieval sizing instantly with clear inputs, formula shown and shareable results.

About Top K Retrieval Sizing Calculator

Retrieved context competes with the system prompt and the generated answer for the same window. Used tokens are k times chunk size plus prompt overhead, and whatever is left caps the response length. A negative remainder means the request will be rejected or silently truncated, which is the most common cause of RAG systems ignoring the last retrieved document.

Formula

Context budget

used = k x chunk tokens + prompt overhead; remaining = context window - used; max k = floor((window - overhead) / chunk)

Worked example

With the default values below, Top K Retrieval Sizing Calculator returns Context tokens consumed by the prompt of 4,496. Change any input above and every figure here is recalculated in your browser.

Inputs used

Chunks retrieved (k)minimum 1: 8
Tokens per chunkminimum 16: 512
System prompt and question tokensminimum 1: 400
Model context windowminimum 128: 8,192

Results

Context tokens consumed by the prompt: 4,496
Tokens left for the response: 3,696
Maximum chunks that fit: 15
Context window utilisation: 54.88%

Step by step

1Values used
2Context budget
3Context tokens consumed by the prompt= 4,496
4Tokens left for the response= 3,696
5Maximum chunks that fit= 15
6Context window utilisation= 54.88

Assumptions and limits

The result is derived only from the 4 inputs listed above. Anything not among them is not part of the calculation.
In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

Does a bigger k always improve answers?

No. Beyond about five to ten chunks, irrelevant context dilutes attention and accuracy often falls. Reranking a larger candidate set down to a small k works better than raising k.

Where should the most relevant chunk go?

Models attend most reliably to the beginning and end of a long context, so place the top-ranked chunk at one of those positions rather than in the middle.