RoCE Network Calculator

Size a lossless RoCEv2 fabric for a GPU or storage cluster, including header efficiency and port count.

About RoCE Network Calculator

RoCEv2 puts InfiniBand transport inside UDP, so each packet pays Ethernet, IP, UDP, BTH, ICRC, FCS and inter-frame gap — 82 bytes on the wire. With a 4 KB payload that is only about 2% overhead, which is why RDMA fabrics run large MTUs. Ports are then sized from raw demand and the oversubscription target, which for AI fabrics is normally 1:1. RoCE assumes a lossless fabric, so undersizing ports or accepting oversubscription converts congestion into PFC pause storms and collapses collective operations.

Formula

RoCE Network

RoCEv2 wire overhead = 14 Ethernet + 20 IP + 8 UDP + 12 BTH + 4 ICRC + 4 FCS + 20 preamble/IFG = 82 bytes; goodput = demand × payload ÷ (payload + 82).

BTH: 12-byte InfiniBand Base Transport Header carried inside UDP port 4791
ICRC: 4-byte invariant CRC that protects the RDMA payload end to end

Worked example

With the default values below, RoCE Network Calculator returns Usable RDMA Goodput of 12,548.8. Change any input above and every figure here is recalculated in your browser.

Inputs used

RDMA endpoints (GPUs or NICs)accepts 1–100,000 endpoints: 64 endpoints
Rate per endpointaccepts 1–800 Gbps: 200 Gbps
RoCE payload per packetaccepts 256–9,216 bytes: 4,096 bytes
Fabric port speedaccepts 10–800 Gbps: 400 Gbps
Target oversubscription (x:1)accepts 1–8 x: 1 x

Results

Usable RDMA Goodput: 12,548.8 Gbps
Raw Cluster Demand: 12,800 Gbps
Wire Efficiency: 98.04%
Fabric Access Ports Needed: 32 ports
Uplink Ports for the Target Ratio: 32 ports

Step by step

1Values used
2RoCE Network
3Usable RDMA Goodput= 12,548.8 Gbps
4Raw Cluster Demand= 12,800 Gbps
5Wire Efficiency= 98.04
6Fabric Access Ports Needed= 32 ports
7Uplink Ports for the Target Ratio= 32 ports

Assumptions and limits

The result is derived only from the 5 inputs listed above. Anything not among them is not part of the calculation.
In particular, this calculator does not take inflation, tax or compounding frequency into account.
Figures are rounded for display to 2 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

How is RoCE Network calculated?

RoCEv2 wire overhead = 14 Ethernet + 20 IP + 8 UDP + 12 BTH + 4 ICRC + 4 FCS + 20 preamble/IFG = 82 bytes; goodput = demand × payload ÷ (payload + 82). RoCEv2 puts InfiniBand transport inside UDP, so each packet pays Ethernet, IP, UDP, BTH, ICRC, FCS and inter-frame gap — 82 bytes on the wire. With a 4 KB payload that is only about 2% overhead, which is why RDMA fabrics run large MTUs. Ports are then sized from raw demand and the oversubscription target, which for AI fabrics is normally 1:1.

Why does RoCE Network matter?

RoCE assumes a lossless fabric, so undersizing ports or accepting oversubscription converts congestion into PFC pause storms and collapses collective operations.

What values do I need to enter?

This calculator takes 5 inputs: RDMA endpoints (GPUs or NICs), Rate per endpoint, RoCE payload per packet, Fabric port speed, Target oversubscription (x:1). The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.