Skip to content
ENExplore Calculators

API Gateway Throughput Calculator

Calculate the maximum request throughput an API gateway instance can sustain.

About API Gateway Throughput Calculator

With a fixed backend connection pool, each connection can serve only one request at a time, so the gateway's maximum throughput is bound by how many connections it has and how long each request occupies one. Formula: max_rps = connection_pool_size / (backend_latency_ms / 1000), reduced by the gateway's own processing overhead (auth, routing, transformation) to get an effective, realistic ceiling.

Formula

Effective max throughput

effective_max_rps = (connection_pool_size / (backend_latency_ms / 1000)) × (1 - overhead_percent / 100)

connection_pool_size
Number of concurrent backend connections
backend_latency_ms
Average backend response time in ms
overhead_percent
Gateway processing overhead as a percentage

Worked example

With the default values below, API Gateway Throughput Calculator returns Effective Max Throughput of 3,600.0. Change any input above and every figure here is recalculated in your browser.

Inputs used

Backend Latencyaccepts 0.1–60,000 ms
50 ms
Connection Pool Sizeaccepts 1–1,000,000 connections
200 connections
Request Timeoutaccepts 1–600,000 ms
5,000 ms
Gateway Overheadaccepts 0–100 %
10 %

Results

Effective Max Throughput
3,600.0 req/s
Theoretical Max Throughput
4,000.0 req/s
Timeout Headroom
4,950 ms
High Timeout Risk
false

Step by step

  1. 1Theoretical max RPS: pool size ÷ (latency / 1000)= 4000.0 req/s
  2. 2Effective max RPS after gateway overhead= 3600.0 req/s
  3. 3Timeout headroom= 4950ms

Assumptions and limits

  • The result is derived only from the 4 inputs listed above. Anything not among them is not part of the calculation.
  • In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
  • Figures are rounded for display to 1 decimal place, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

How do I increase max throughput without adding backend capacity?

Increase the connection pool size (if the backend can handle more concurrent connections) or reduce backend latency (caching, query optimization) — both directly raise the theoretical ceiling in the formula.

What if my actual RPS is lower than the calculated max?

Check for connection pool exhaustion/queueing, keep-alive settings, DNS resolution overhead, or backend latency spikes under load — real-world throughput is often below the theoretical figure due to tail latency, not just averages.

Why does gateway overhead matter?

API gateways add real processing time for authentication, rate limiting, request/response transformation and logging — this can be a meaningful percentage of the total request time, especially for fast backends where gateway overhead is a bigger share of total latency.

Is this the same as HTTP/2 or connection multiplexing throughput?

No — this model assumes one request occupies one pooled connection for its full duration (like a typical HTTP/1.1 keep-alive pool). Multiplexed protocols (HTTP/2, gRPC) can serve multiple concurrent requests per connection, which would require a different concurrency model.

Related Calculators

API Gateway Throughput Calculator | Calcrivo