Combine prompt and completion costs to estimate the total cost of an LLM API call.
The total cost of running an LLM in production combines both input (prompt) and output (completion) token pricing, multiplied by request volume: total = (prompt_tokens × input_price + completion_tokens × output_price) × requests. Output tokens are typically priced 2-5x higher than input tokens because generation is more compute-intensive than reading context. Use this to budget monthly or annual LLM spend for a given workload.
total_cost = (prompt_tokens * input_price + completion_tokens * output_price) * requests
Generating tokens (autoregressive decoding) requires a full forward pass per token, while processing prompt tokens can be batched and parallelized, making input tokens cheaper to serve.
Use the Token Calculator to estimate tokens from sample prompts and expected response lengths, then plug those averages in here.
No, this covers only standard chat/completion API calls. Embeddings, fine-tuning, and image/audio modalities have separate pricing.
Shorten prompts, cache repeated context, use a smaller/cheaper model where acceptable, and cap completion length with a max_tokens setting.