RoCE Network Calculator

Size a lossless RoCEv2 fabric for a GPU or storage cluster, including header efficiency and port count.

Inputs

RDMA endpoints (GPUs or NICs)endpoints

Rate per endpointGbps

RoCE payload per packetbytes

Fabric port speedGbps

Target oversubscription (x:1)x

Usable RDMA Goodput

12,548.8Gbps

Raw Cluster Demand

12,800Gbps

Wire Efficiency

98.04%

Fabric Access Ports Needed

32ports

Uplink Ports for the Target Ratio

32ports

Step by step

Values used
RDMA endpoints (GPUs or NICs) = 64 endpoints; Rate per endpoint = 200 Gbps; RoCE payload per packet = 4,096 bytes; Fabric port speed = 400 Gbps; Target oversubscription (x:1) = 1 x
RoCE Network
RoCEv2 wire overhead = 14 Ethernet + 20 IP + 8 UDP + 12 BTH + 4 ICRC + 4 FCS + 20 preamble/IFG = 82 bytes; goodput = demand × payload ÷ (payload + 82).
Usable RDMA Goodput
= 12,548.8 Gbps
Raw Cluster Demand
= 12,800 Gbps
Wire Efficiency
= 98.04
Fabric Access Ports Needed
= 32 ports
Uplink Ports for the Target Ratio
= 32 ports

How it works

RoCEv2 puts InfiniBand transport inside UDP, so each packet pays Ethernet, IP, UDP, BTH, ICRC, FCS and inter-frame gap — 82 bytes on the wire. With a 4 KB payload that is only about 2% overhead, which is why RDMA fabrics run large MTUs. Ports are then sized from raw demand and the oversubscription target, which for AI fabrics is normally 1:1. RoCE assumes a lossless fabric, so undersizing ports or accepting oversubscription converts congestion into PFC pause storms and collapses collective operations.

Formula

RoCE Network

RoCEv2 wire overhead = 14 Ethernet + 20 IP + 8 UDP + 12 BTH + 4 ICRC + 4 FCS + 20 preamble/IFG = 82 bytes; goodput = demand × payload ÷ (payload + 82).

BTH: 12-byte InfiniBand Base Transport Header carried inside UDP port 4791
ICRC: 4-byte invariant CRC that protects the RDMA payload end to end

Frequently Asked Questions

How is RoCE Network calculated?

RoCEv2 wire overhead = 14 Ethernet + 20 IP + 8 UDP + 12 BTH + 4 ICRC + 4 FCS + 20 preamble/IFG = 82 bytes; goodput = demand × payload ÷ (payload + 82). RoCEv2 puts InfiniBand transport inside UDP, so each packet pays Ethernet, IP, UDP, BTH, ICRC, FCS and inter-frame gap — 82 bytes on the wire. With a 4 KB payload that is only about 2% overhead, which is why RDMA fabrics run large MTUs. Ports are then sized from raw demand and the oversubscription target, which for AI fabrics is normally 1:1.

Why does RoCE Network matter?

RoCE assumes a lossless fabric, so undersizing ports or accepting oversubscription converts congestion into PFC pause storms and collapses collective operations.

What values do I need to enter?

This calculator takes 5 inputs: RDMA endpoints (GPUs or NICs), Rate per endpoint, RoCE payload per packet, Fabric port speed, Target oversubscription (x:1). The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.

You might also need

PFC Buffer Calculator Network Fabric Latency Calculator Storage Network Throughput Calculator