Break down ESP tunnel overhead byte by byte, find the encapsulated packet size and the largest inner MTU that fits.
ESP in tunnel mode adds a fresh 20-byte outer IPv4 header, an 8-byte ESP header, the cipher's IV, padding to the cipher block size, two trailer bytes and the integrity check value — plus 8 more bytes of UDP if NAT traversal is in play. Because padding aligns the plaintext to a 16-byte boundary, the total varies with payload size but stays inside the familiar 56–73 byte band. Get the overhead wrong and encapsulated packets exceed the path MTU, which either fragments every large flow or blackholes it entirely when ICMP needed-fragmentation messages are filtered.
IPSec Overhead
overhead = outer IP (20) + UDP NAT-T (8) + ESP header (8) + IV + pad + pad length/next header (2) + ICV, which lands at 56–73 bytes for tunnel mode.
overhead = outer IP (20) + UDP NAT-T (8) + ESP header (8) + IV + pad + pad length/next header (2) + ICV, which lands at 56–73 bytes for tunnel mode. ESP in tunnel mode adds a fresh 20-byte outer IPv4 header, an 8-byte ESP header, the cipher's IV, padding to the cipher block size, two trailer bytes and the integrity check value — plus 8 more bytes of UDP if NAT traversal is in play. Because padding aligns the plaintext to a 16-byte boundary, the total varies with payload size but stays inside the familiar 56–73 byte band.
Get the overhead wrong and encapsulated packets exceed the path MTU, which either fragments every large flow or blackholes it entirely when ICMP needed-fragmentation messages are filtered.
This calculator takes 5 inputs: Inner packet size, Cipher and integrity suite, Tunnel mode (adds a new outer IPv4 header), NAT traversal (UDP 4500 encapsulation), Path MTU. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.
It is a round number comfortably below every realistic worst case: 1500 minus about 73 bytes of ESP overhead leaves 1427, and 1400 leaves slack for an extra VLAN tag, GRE header or a 1492-byte PPPoE path. Clamping TCP MSS to 1360 alongside it is what stops fragmentation in practice.