MTU 1500: fragment packets after 1472 bytes

Question

I have couple of VMs running on Openstack. I have one data network gateway assigned into the switch with MTU=1500, where VMs has to running.

Also in Openstack, I have set the default MTU=1500 of the network.

I am trying to ping from the VM to gateway (OR any ip on same network) of the switch with 1500 packetsize, it's de-fragmenting the packet.

localhost:~# ping -M do -s 1500 10.4.14.18 
PING 10.4.14.18 (10.4.14.18) 1500(1528) bytes of data.
From 10.4.14.17 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 10.4.14.17 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 10.4.14.17 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 10.4.14.17 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 10.4.14.17 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 10.4.14.17 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 10.4.14.17 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 10.4.14.17 icmp_seq=1 Frag needed and DF set (mtu = 1500

Instead, If I am sending packets with smaller mtu (1472) it works,

localhost:~# ping -M do -s 1472 10.4.14.18
PING 10.4.14.18 (10.4.14.18) 1472(1500) bytes of data.
1480 bytes from 10.4.14.18: icmp_seq=1 ttl=64 time=0.965 ms
1480 bytes from 10.4.14.18: icmp_seq=2 ttl=64 time=0.515 ms
^C
--- 10.4.14.18 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1875ms
rtt min/avg/max/mdev = 0.515/0.740/0.965/0.225 ms

I can see 28 bytes of difference, i don't understand, where this 28 bytes are being utilised ?

Solution

The 1500 byte MTU holds for standard Ethernet at the network layer. In other words: 1500 byte can be transmitted in Ethernet frames without being fragmented. However, further protocols on top of Ethernet of course cut into the maximum payload.

In your case (the case of ping) the additional protocols are IP and ICMP. They both have headers with lengths 20 byte and 8 byte, respectively.

This means that you can transmit at most

1500 bytes (Ethernet mtu) - 20 byte (IP header) - 8 byte (ICMP header) 
= 1472 byte

See also this question regarding MTU for UDP.