Vertex AI Cost Calculator
Estimate Vertex AI spend across generative token usage, custom training node-hours and always-on prediction endpoints.
About Vertex AI Cost Calculator
Vertex AI bills three unrelated things: tokens for generative models, node and accelerator hours for training, and deployed replica hours for prediction endpoints. Because output tokens cost several times input tokens, response length is usually the strongest lever on a generative bill, while a deployed endpoint with a minimum replica count is a fixed monthly floor regardless of traffic. Teams estimate inference cost from request counts and are then surprised by verbose outputs, or leave a training endpoint deployed after an experiment ends and pay for it all month. Confirm current per-token and per-node-hour rates for your model and region in the Google Cloud pricing calculator, since model prices change with every release.
Formula
Vertex AI Cost
token cost = (fresh input tokens + cached input × (1 − cache discount)) ÷ 1M × input price + output tokens ÷ 1M × output price; training = hours × (node price + accelerators × accelerator price); prediction = nodes × hours × node price.
- output tokens
- Generated tokens, priced several times higher than input tokens
- context cache
- Reused prompt prefix billed at a reduced rate
- prediction node-hour
- Charge for a deployed endpoint replica, billed while deployed rather than per request
Worked example
With the default values below, Vertex AI Cost Calculator returns Monthly Cost of $974.70. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Model requests per monthaccepts 0–10,000,000 thousands
- 500 thousands
- Average input tokens per requestaccepts 0–2,000,000 tokens
- 900 tokens
- Average output tokens per requestaccepts 0–100,000 tokens
- 250 tokens
- Input price per million tokensaccepts 0–1,000 USD/M tokens
- 0.3 USD/M tokens
- Output price per million tokensaccepts 0–1,000 USD/M tokens
- 2.5 USD/M tokens
- Input tokens served from context cacheaccepts 0–100 %
- 0 %
- Discount on cached input tokensaccepts 0–100 %
- 75 %
- Custom training hoursaccepts 0–100,000 hours
- 40 hours
- Training price per node-houraccepts 0–1,000 USD/node-hour
- 0.19 USD/node-hour
- Accelerators attached during trainingaccepts 0–1,000 accelerators
- 2 accelerators
- Accelerator price per houraccepts 0–1,000 USD/hour
- 2.48 USD/hour
- Prediction endpoint nodesaccepts 0–10,000 nodes
- 2 nodes
- Hours the endpoint is deployedaccepts 0–744 hours
- 730 hours
- Prediction price per node-houraccepts 0–1,000 USD/node-hour
- 0.22 USD/node-hour
Results
- Monthly Cost
- $974.70
- Generative Token Charge
- $447.50
- Custom Training Charge
- $206.00
- Prediction Endpoint Charge
- $321.20
- Tokens Processed
- 575.00 millions
- Output Share of the Token Charge
- 69.8%
- Cost per Thousand Requests
- $1.9494
- Largest Cost Line
- Generative token usage — shorten responses before anything else
Step by step
- 1Values used
- 2Vertex AI Cost
- 3Monthly Cost= 974.70
- 4Generative Token Charge= 447.50
- 5Custom Training Charge= 206.00
- 6Prediction Endpoint Charge= 321.20
- 7Tokens Processed= 575.00 millions
- 8Output Share of the Token Charge= 69.8
Assumptions and limits
- The result is derived only from the 14 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax or compounding frequency into account.
- Figures are rounded for display to 4 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
How is Vertex AI Cost calculated?
token cost = (fresh input tokens + cached input × (1 − cache discount)) ÷ 1M × input price + output tokens ÷ 1M × output price; training = hours × (node price + accelerators × accelerator price); prediction = nodes × hours × node price. Vertex AI bills three unrelated things: tokens for generative models, node and accelerator hours for training, and deployed replica hours for prediction endpoints. Because output tokens cost several times input tokens, response length is usually the strongest lever on a generative bill, while a deployed endpoint with a minimum replica count is a fixed monthly floor regardless of traffic.
Why does Vertex AI Cost matter?
Teams estimate inference cost from request counts and are then surprised by verbose outputs, or leave a training endpoint deployed after an experiment ends and pay for it all month. Confirm current per-token and per-node-hour rates for your model and region in the Google Cloud pricing calculator, since model prices change with every release.
What values do I need to enter?
This calculator takes 14 inputs: Model requests per month, Average input tokens per request, Average output tokens per request, Input price per million tokens, Output price per million tokens, Input tokens served from context cache, Discount on cached input tokens, Custom training hours, Training price per node-hour, Accelerators attached during training, Accelerator price per hour, Prediction endpoint nodes, Hours the endpoint is deployed, Prediction price per node-hour. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.
Is an always-on endpoint cheaper than per-token billing?
Only at sustained high volume. A deployed endpoint bills replica hours whether or not requests arrive, so it wins when utilisation is high and steady, while token-billed managed models suit spiky or low-volume traffic because idle time is free.