Cost a Dataproc cluster including the per-vCPU premium, Spot secondary workers and disk, with idle time exposed.
Dataproc bills the underlying Compute Engine resources plus a premium on every vCPU, so the cluster shape drives both lines together. Secondary workers are the lever that matters: because they hold no HDFS data, running them on Spot capacity cuts the compute charge sharply while a job loss costs only the tasks in flight. Long-lived Dataproc clusters spend a large share of their hours idle, and the honest comparison is an ephemeral cluster created per job against a persistent one you keep warm for convenience. Confirm the Dataproc premium and Compute Engine rates for your region in the Google Cloud pricing calculator.
Dataproc Cluster Cost
cost = (on-demand vCPU + Spot vCPU × (1 − Spot discount)) × (vCPU price + GB per vCPU × GB price) × hours + total vCPU × Dataproc premium × hours + disk.
cost = (on-demand vCPU + Spot vCPU × (1 − Spot discount)) × (vCPU price + GB per vCPU × GB price) × hours + total vCPU × Dataproc premium × hours + disk. Dataproc bills the underlying Compute Engine resources plus a premium on every vCPU, so the cluster shape drives both lines together. Secondary workers are the lever that matters: because they hold no HDFS data, running them on Spot capacity cuts the compute charge sharply while a job loss costs only the tasks in flight.
Long-lived Dataproc clusters spend a large share of their hours idle, and the honest comparison is an ephemeral cluster created per job against a persistent one you keep warm for convenience. Confirm the Dataproc premium and Compute Engine rates for your region in the Google Cloud pricing calculator.
This calculator takes 14 inputs: Master nodes, vCPUs per master, Primary workers, vCPUs per worker, Secondary (Spot) workers, Compute Engine price per vCPU-hour, Memory per vCPU, Price per GB-hour, Dataproc premium per vCPU-hour, Spot discount on secondary workers, Cluster hours per month, Cluster hours with no job running, Persistent Disk per node, Disk price per GB-month. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.
No. Primary workers hold HDFS data, so losing one can force recomputation or leave the cluster under-replicated. Keep primaries on standard capacity for stability and put the elastic portion of the cluster into Spot secondary workers.