Estimate total test suite execution time given test count and parallel workers.
Test suite execution time scales down roughly linearly with parallelism — splitting tests across more workers divides the base execution time — but flaky tests that fail and get retried effectively run twice, adding overhead proportional to the flaky rate. Modeling that overhead separately makes clear that flaky tests cost real pipeline time even when they eventually pass, which is often an underappreciated tax on CI throughput distinct from genuine test failures.
totalTime = (tests / parallelism × avgDuration) + (tests × flakyRate / parallelism × avgDuration)
Yes — retries of flaky tests are typically distributed across the same parallel workers as the initial run, so the flaky overhead is also divided by the parallelism factor in this model, though in practice retry scheduling can add extra coordination delay not captured here.
Best-in-class suites keep flaky rates under 1%; rates above 3–5% meaningfully inflate CI time and erode trust in the suite, often signalling shared test state, timing assumptions, or external service dependencies that need fixing rather than just retrying around.
Not necessarily — some CI systems retry failed tests multiple times before marking the suite as failed; each additional retry attempt compounds the overhead this calculator estimates, so suites with aggressive multi-retry policies should scale the flaky overhead accordingly.