Size a lossless RoCEv2 fabric for a GPU or storage cluster, including header efficiency and port count.
RoCEv2 puts InfiniBand transport inside UDP, so each packet pays Ethernet, IP, UDP, BTH, ICRC, FCS and inter-frame gap — 82 bytes on the wire. With a 4 KB payload that is only about 2% overhead, which is why RDMA fabrics run large MTUs. Ports are then sized from raw demand and the oversubscription target, which for AI fabrics is normally 1:1. RoCE assumes a lossless fabric, so undersizing ports or accepting oversubscription converts congestion into PFC pause storms and collapses collective operations.
RoCE Network
RoCEv2 wire overhead = 14 Ethernet + 20 IP + 8 UDP + 12 BTH + 4 ICRC + 4 FCS + 20 preamble/IFG = 82 bytes; goodput = demand × payload ÷ (payload + 82).
RoCEv2 wire overhead = 14 Ethernet + 20 IP + 8 UDP + 12 BTH + 4 ICRC + 4 FCS + 20 preamble/IFG = 82 bytes; goodput = demand × payload ÷ (payload + 82). RoCEv2 puts InfiniBand transport inside UDP, so each packet pays Ethernet, IP, UDP, BTH, ICRC, FCS and inter-frame gap — 82 bytes on the wire. With a 4 KB payload that is only about 2% overhead, which is why RDMA fabrics run large MTUs. Ports are then sized from raw demand and the oversubscription target, which for AI fabrics is normally 1:1.
RoCE assumes a lossless fabric, so undersizing ports or accepting oversubscription converts congestion into PFC pause storms and collapses collective operations.
This calculator takes 5 inputs: RDMA endpoints (GPUs or NICs), Rate per endpoint, RoCE payload per packet, Fabric port speed, Target oversubscription (x:1). The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.