Skip to content
ENExplore Calculators

Cache Hit Savings For LLM Calculator

Work out cache hit savings for llm instantly with clear inputs, formula shown and shareable results.

About Cache Hit Savings For LLM Calculator

A cache turns the blended cost per request into a weighted average of hit and miss costs. Savings scale linearly with hit rate and with the gap between the two costs, so both prompt caching, where a cached prefix is billed at roughly a tenth of the input rate, and full response caching, where a hit costs almost nothing, pay off quickly at high volume.

Formula

Blended cost with cache

cost = requests x (hit rate x cost per hit + (1 - hit rate) x cost per miss); savings = uncached cost - cached cost

Worked example

With the default values below, Cache Hit Savings For LLM Calculator returns Monthly savings from caching of $1,260.00. Change any input above and every figure here is recalculated in your browser.

Inputs used

Requests per monthminimum 1
1,000,000
Cache hit rateaccepts 0–100 %
35 %
Cost per uncached requestminimum 0
0.004
Cost per cached requestminimum 0
0.0004

Results

Monthly savings from caching
$1,260.00
Monthly cost with cache
$2,740.00
Monthly cost without cache
$4,000.00
Effective cost per request
$0.002740

Step by step

  1. 1Values used
  2. 2Blended cost with cache
  3. 3Monthly savings from caching= 1,260.00
  4. 4Monthly cost with cache= 2,740.00
  5. 5Monthly cost without cache= 4,000.00
  6. 6Effective cost per request= 0.002740

Assumptions and limits

  • The result is derived only from the 4 inputs listed above. Anything not among them is not part of the calculation.
  • In particular, this calculator does not take inflation, tax or compounding frequency into account.
  • Figures are rounded for display to 6 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

What hit rate is realistic?

Exact-match response caching on open-ended chat rarely exceeds 10 to 20 percent, but prompt-prefix caching of a long system prompt or shared document context routinely exceeds 80 percent.

What are the risks of caching responses?

Stale answers when underlying data changes, and cross-tenant leakage if the cache key omits the user or permission scope. Always include authorisation context in the key.

Related Calculators

Cache Hit Savings For LLM Calculator | Calcrivo