Dataflow Cost Calculator
Price a Dataflow pipeline across worker vCPU, memory, shuffle data processed and Streaming Engine.
Inputs
Autoscaling means you rarely pay for the maximum — use the average worker count from the job graph.
Use 730 for an always-on streaming pipeline, or the sum of batch run times.
Streaming Engine moves state off the workers, letting them be smaller, and bills data processed instead.
Monthly Cost
$722.74
Worker Compute Charge
$619.52
Data Processed Charge
$55.00
Worker Disk Charge
$48.22
Billed vCPU-Hours
8,800vCPU-hours
Average Workers Billed
11.0workers
Cost per GB Processed
$0.1445
Cost per Pipeline Hour
$3.61
Step by step
Values used
Maximum workers = 20 workers; Average share of maximum workers used = 55 %; vCPUs per worker = 4 vCPU; Memory per worker = 16 GB; Pipeline hours per month = 200 hours; Price per vCPU-hour = 0.0560 USD/vCPU-hour; Price per GB-hour = 0.0036 USD/GB-hour; Data processed by shuffle = 5,000 GB; Batch shuffle price per GB = 0.0110 USD/GB; Use Streaming Engine = No; Streaming Engine price per GB = 0.0180 USD/GB; Persistent Disk per worker = 400 GB; Disk price per GB-month = 0.0400 USD/GB-month
Dataflow Cost
cost = average workers × (vCPU × vCPU-hour price + GB × GB-hour price) × hours + GB processed × shuffle or Streaming Engine price + worker disk.
Monthly Cost
= 722.74
Worker Compute Charge
= 619.52
Data Processed Charge
= 55.00
Worker Disk Charge
= 48.22
Billed vCPU-Hours
= 8,800 vCPU-hours
Average Workers Billed
= 11.0 workers
How it works
Dataflow bills worker resources by the hour and shuffle volume by the gigabyte, so a pipeline with a wide GroupByKey can spend more on data processed than on workers. Enabling Streaming Engine shifts state off the workers, which lets each worker use a 30 GB disk instead of hundreds of gigabytes — a real saving that the disk line here reflects. Autoscaling means the maximum worker count is a safety limit rather than a cost, so estimating from peak workers overstates the bill badly while ignoring shuffle understates it. Confirm the per-vCPU-hour and per-GB data-processed rates for your region in the Google Cloud pricing calculator.
Formula
Dataflow Cost
cost = average workers × (vCPU × vCPU-hour price + GB × GB-hour price) × hours + GB processed × shuffle or Streaming Engine price + worker disk.
- average workers
- Maximum workers multiplied by the share autoscaling actually used
- data processed
- Bytes moved through shuffle or Streaming Engine, billed separately from compute
- Streaming Engine
- Offloads pipeline state from workers, allowing 30 GB boot disks instead of large ones
Frequently Asked Questions
How is Dataflow Cost calculated?
cost = average workers × (vCPU × vCPU-hour price + GB × GB-hour price) × hours + GB processed × shuffle or Streaming Engine price + worker disk. Dataflow bills worker resources by the hour and shuffle volume by the gigabyte, so a pipeline with a wide GroupByKey can spend more on data processed than on workers. Enabling Streaming Engine shifts state off the workers, which lets each worker use a 30 GB disk instead of hundreds of gigabytes — a real saving that the disk line here reflects.
Why does Dataflow Cost matter?
Autoscaling means the maximum worker count is a safety limit rather than a cost, so estimating from peak workers overstates the bill badly while ignoring shuffle understates it. Confirm the per-vCPU-hour and per-GB data-processed rates for your region in the Google Cloud pricing calculator.
What values do I need to enter?
This calculator takes 13 inputs: Maximum workers, Average share of maximum workers used, vCPUs per worker, Memory per worker, Pipeline hours per month, Price per vCPU-hour, Price per GB-hour, Data processed by shuffle, Batch shuffle price per GB, Use Streaming Engine, Streaming Engine price per GB, Persistent Disk per worker, Disk price per GB-month. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.
Why is my shuffle charge larger than my compute charge?
Because shuffle is billed per GB moved and a single GroupByKey or join can move many times your input volume. Reducing key cardinality, filtering earlier in the graph and using combiners to pre-aggregate all cut shuffle bytes directly.