Skip to content
Calcrivo

API Gateway Throughput Calculator

Calculate the maximum sustainable request throughput an API gateway can push through a fixed backend connection pool.

Inputs

ms

Average time the backend takes to respond to one request.

connections

Number of concurrent connections the gateway maintains to the backend.

ms

Gateway timeout for a backend request; used to flag if latency risks timeouts.

%

Processing overhead the gateway itself adds (routing, auth, transformation) as a percent reduction in achievable throughput.

Effective Max Throughput

3,600.0req/s

Theoretical Max Throughput

4,000.0req/s

Timeout Headroom

4,950ms

High Timeout Risk

false

Step by step

  1. Theoretical max RPS: pool size ÷ (latency / 1000)

    200 ÷ (50/1000)

    = 4000.0 req/s

  2. Effective max RPS after gateway overhead

    4000.0 × (1 − 10/100)

    = 3600.0 req/s

  3. Timeout headroom

    5000 − 50

    = 4950ms

How it works

With a fixed backend connection pool, each connection can serve only one request at a time, so the gateway's maximum throughput is bound by how many connections it has and how long each request occupies one. Formula: max_rps = connection_pool_size / (backend_latency_ms / 1000), reduced by the gateway's own processing overhead (auth, routing, transformation) to get an effective, realistic ceiling.

Formula

Effective max throughput

effective_max_rps = (connection_pool_size / (backend_latency_ms / 1000)) × (1 - overhead_percent / 100)

connection_pool_size
Number of concurrent backend connections
backend_latency_ms
Average backend response time in ms
overhead_percent
Gateway processing overhead as a percentage

Frequently Asked Questions

How do I increase max throughput without adding backend capacity?

Increase the connection pool size (if the backend can handle more concurrent connections) or reduce backend latency (caching, query optimization) — both directly raise the theoretical ceiling in the formula.

What if my actual RPS is lower than the calculated max?

Check for connection pool exhaustion/queueing, keep-alive settings, DNS resolution overhead, or backend latency spikes under load — real-world throughput is often below the theoretical figure due to tail latency, not just averages.

Why does gateway overhead matter?

API gateways add real processing time for authentication, rate limiting, request/response transformation and logging — this can be a meaningful percentage of the total request time, especially for fast backends where gateway overhead is a bigger share of total latency.

Is this the same as HTTP/2 or connection multiplexing throughput?

No — this model assumes one request occupies one pooled connection for its full duration (like a typical HTTP/1.1 keep-alive pool). Multiplexed protocols (HTTP/2, gRPC) can serve multiple concurrent requests per connection, which would require a different concurrency model.

You might also need

Related Calculators

Istio Sidecar CPU CalculatorIstio Sidecar Memory CalculatorSLI Calculator