Estimate Vertex AI spend across generative token usage, custom training node-hours and always-on prediction endpoints.
Vertex AI bills three unrelated things: tokens for generative models, node and accelerator hours for training, and deployed replica hours for prediction endpoints. Because output tokens cost several times input tokens, response length is usually the strongest lever on a generative bill, while a deployed endpoint with a minimum replica count is a fixed monthly floor regardless of traffic. Teams estimate inference cost from request counts and are then surprised by verbose outputs, or leave a training endpoint deployed after an experiment ends and pay for it all month. Confirm current per-token and per-node-hour rates for your model and region in the Google Cloud pricing calculator, since model prices change with every release.
Vertex AI Cost
token cost = (fresh input tokens + cached input × (1 − cache discount)) ÷ 1M × input price + output tokens ÷ 1M × output price; training = hours × (node price + accelerators × accelerator price); prediction = nodes × hours × node price.
token cost = (fresh input tokens + cached input × (1 − cache discount)) ÷ 1M × input price + output tokens ÷ 1M × output price; training = hours × (node price + accelerators × accelerator price); prediction = nodes × hours × node price. Vertex AI bills three unrelated things: tokens for generative models, node and accelerator hours for training, and deployed replica hours for prediction endpoints. Because output tokens cost several times input tokens, response length is usually the strongest lever on a generative bill, while a deployed endpoint with a minimum replica count is a fixed monthly floor regardless of traffic.
Teams estimate inference cost from request counts and are then surprised by verbose outputs, or leave a training endpoint deployed after an experiment ends and pay for it all month. Confirm current per-token and per-node-hour rates for your model and region in the Google Cloud pricing calculator, since model prices change with every release.
This calculator takes 14 inputs: Model requests per month, Average input tokens per request, Average output tokens per request, Input price per million tokens, Output price per million tokens, Input tokens served from context cache, Discount on cached input tokens, Custom training hours, Training price per node-hour, Accelerators attached during training, Accelerator price per hour, Prediction endpoint nodes, Hours the endpoint is deployed, Prediction price per node-hour. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.
Only at sustained high volume. A deployed endpoint bills replica hours whether or not requests arrive, so it wins when utilisation is high and steady, while token-billed managed models suit spiky or low-volume traffic because idle time is free.