Work out batch inference cost instantly with clear inputs, formula shown and shareable results.
Offline scoring cost is simply total tokens times the blended price, but batch endpoints typically halve that in exchange for a delivery window measured in hours. Cost per item is the figure to compare against the business value of each scored record — at fractions of a cent per item, the bottleneck is usually engineering time rather than inference spend.
Batch cost
total tokens = items x tokens per item; cost = tokens x price / 1,000,000 x (1 - batch discount)
Anything user-facing. Batch jobs have completion windows of up to 24 hours and no latency guarantee, so they suit enrichment, backfills and evaluation runs.
Trim boilerplate from the prompt, cap the output with a schema or a max-token limit, and truncate retrieved context to what the task genuinely needs.