Work out reranking cost instantly with clear inputs, formula shown and shareable results.
A cross-encoder reranker scores every candidate against the query, so cost scales with candidates multiplied by tokens per candidate — not with the number of chunks finally used. Retrieving 50 candidates and reranking to 5 costs ten times as much scoring as no reranking at all, which is why candidate depth is the main cost lever in a RAG pipeline.
Reranking cost
tokens per query = candidates x tokens per candidate; cost = queries x tokens per query x price / 1,000,000
Usually yes. Cross-encoders typically lift retrieval quality substantially over pure vector similarity because they see the query and document together rather than comparing independent embeddings.
Start at 25 to 50. Going deeper increases cost linearly while recall gains flatten, so measure recall@k on your own data to find the knee.