Thread Pool Size Calculator
Calculate optimal thread pool size for a service based on request rate and latency.
About Thread Pool Size Calculator
The classic thread pool sizing formula — threads = cores × (1 + wait_time/service_time) — captures a simple insight: while a thread is blocked waiting (on I/O, a lock, or a network call), its CPU core sits idle unless another thread is available to run, so the higher the ratio of wait time to actual CPU service time, the more threads are needed to keep cores busy. Purely CPU-bound work (near-zero wait) needs a pool close to core count, since extra threads there mostly add context-switching overhead without more useful parallelism; heavily I/O-bound work justifies a much larger pool (or, at scale, switching to an async/event-driven model that doesn't need one OS thread per in-flight task at all).
Formula
Optimal thread pool size
optimal_threads = cores × (1 + wait_time / service_time) × target_utilization%
- c
- CPU cores
- W
- average wait time per task
- S
- average service (CPU) time per task
- u
- target utilization fraction
Worked example
With the default values below, Thread Pool Size Calculator returns Optimal Thread Pool Size of 40. Change any input above and every figure here is recalculated in your browser.
Inputs used
- CPU Cores Availableminimum 1
- 8
- Average Wait Time per Task (ms, e.g. I/O blocking)minimum 0
- 40
- Average Service Time per Task (ms, actual CPU work)minimum 0.01
- 10
- Target CPU Utilization (%)accepts 1–100
- 100
Results
- Optimal Thread Pool Size
- 40
- Workload Profile
- I/O-bound workload — wait time dominates; a much larger thread pool (or an async/non-blocking model) lets many tasks overlap I/O wait, since blocked threads use minimal CPU.
- Wait-to-Service Time Ratio
- 4.00
- Unrounded Optimal Size
- 40.00
Step by step
- 1Values used
- 2Optimal thread pool size
- 3Optimal Thread Pool Size= 40
- 4Workload Profile= I/O-bound workload — wait time dominates; a much larger thread pool (or an async/non-blocking model) lets many tasks overlap I/O wait, since blocked threads use minimal CPU.
- 5Wait-to-Service Time Ratio= 4.00
- 6Unrounded Optimal Size= 40.00
Assumptions and limits
- The result is derived only from the 4 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
Why would adding more threads than this formula suggests hurt performance?
Beyond the point where cores are kept fully busy, additional threads mostly add context-switching overhead (cache thrashing, scheduler bookkeeping) and increased memory usage (each thread needs its own stack) without contributing more useful parallel work — for CPU-bound workloads especially, pool sizes far beyond core count often reduce throughput rather than increase it.
How do I estimate the wait-to-service ratio for a real workload?
Profile a representative task and measure time spent blocked (in syscalls like read()/recv(), or waiting on a lock/database query) versus time spent actively executing CPU instructions — tools like `strace -T` (per-syscall timing) or application-level tracing/APM tools can break this down directly.
Does this formula apply to async/event-loop-based servers too?
Not directly — this sizing model assumes a traditional blocking-thread-per-task model where a blocked thread consumes a pool slot doing nothing useful. Async/event-driven architectures (Node.js, async Rust/Python) sidestep the need for large thread pools by handling many in-flight I/O operations on far fewer threads, since a single thread can juggle many non-blocking operations concurrently.