Work out top k retrieval sizing instantly with clear inputs, formula shown and shareable results.
Retrieved context competes with the system prompt and the generated answer for the same window. Used tokens are k times chunk size plus prompt overhead, and whatever is left caps the response length. A negative remainder means the request will be rejected or silently truncated, which is the most common cause of RAG systems ignoring the last retrieved document.
Context budget
used = k x chunk tokens + prompt overhead; remaining = context window - used; max k = floor((window - overhead) / chunk)
No. Beyond about five to ten chunks, irrelevant context dilutes attention and accuracy often falls. Reranking a larger candidate set down to a small k works better than raising k.
Models attend most reliably to the beginning and end of a long context, so place the top-ranked chunk at one of those positions rather than in the middle.