Skip to content
ENExplore Calculators

Concurrent Request Capacity Calculator

Work out concurrent request capacity instantly with clear inputs, formula shown and shareable results.

About Concurrent Request Capacity Calculator

Little's law states that the average number of items in a stable system equals the arrival rate multiplied by the average time each item spends there: L = lambda x W. For an inference service that means 40 requests per second at 2.5 seconds each keeps 100 requests in flight, regardless of how the work is distributed. Dividing by the per-replica concurrency limit gives replica count directly.

Formula

Little's law

L = lambda x W; replicas = ceil(L / concurrency per replica)

L
Requests in flight
lambda
Arrival rate
W
Average time in system

Worked example

With the default values below, Concurrent Request Capacity Calculator returns Requests in flight (Little's law) of 100.00. Change any input above and every figure here is recalculated in your browser.

Inputs used

Arrival rateminimum 0.01 req/s
40 req/s
Average time in systemminimum 0.001 s
2.5 s
Concurrent requests per replicaminimum 1
8

Results

Requests in flight (Little's law)
100.00
Replicas required
13
Concurrency utilisation
96.15%
Spare concurrency
4.00

Step by step

  1. 1Values used
  2. 2Little's law
  3. 3Requests in flight (Little's law)= 100.00
  4. 4Replicas required= 13
  5. 5Concurrency utilisation= 96.15
  6. 6Spare concurrency= 4.00

Assumptions and limits

  • The result is derived only from the 3 inputs listed above. Anything not among them is not part of the calculation.
  • In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
  • Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

Does Little's law need any distribution assumptions?

No, which is what makes it so useful. It holds for any stable queueing system regardless of arrival or service-time distribution.

Should I size to 100 percent utilisation?

No. Queueing delay grows sharply above about 70 to 80 percent utilisation, so provision headroom for arrival bursts and slow requests.

Related Calculators

Concurrent Request Capacity Calculator | Calcrivo