Can't ping gateway after upgrading to Ubuntu 16.04 | arp issue?

Question

After upgrading to Ubuntu 16.04 from 14.04 I can't ping gateway, at first interface eth0 wasn't starting up, and I read that refreshing new mac address would fix that, so I decided to remove the network interface from VM and added a new one instead, then it turned out that eth0 had to be renamed to something else (ens###) as shown in "ifconfig -a" to start the interface. But now I can't ping the gateway, routes are all fine.

root@Hostname:~# arp -a
? (192.168.1.1) at <incomplete> on ens192
? (192.168.1.82) at 00:5:56:ab:bb:cc [ether] on ens192

Could the old mac address stuck around somewhere? Or why does it say incomplete? when I ping gateway its Destination unreachable, it was working fine before the upgrade.

You can see its requesting the gateway mac-address in tcpdump

tcpdump output Image

Update: Solved

Okay figured it out, looks like when I added the network adapter I chose vmxnet3 but the old adapter was E1000. I just removed the interface again and added E1000 adapter and renamed the interface name again in /etc/network/interface file and restarted. Its working fine now. Thanks Daniele, checking the hypersivsor again and the vm helped

Solution

A little recap

Recently Ubuntu adopted systemd, which handles the naming of network interfaces when the system boots. The naming will be associated to something physical in the hardware (like the slot the card is inserted in), so you can add/remove/replace network hardware and the names don't suddenly change on you (as it was before, with the ethX scheme).

Now, you did the upgrade and the network interface was renamed, so your configuration in /etc/network/interfaces needed to be fixed. But, being confused, you first tried removing the virtual NIC and then added a new one. That gave you a different mac address AND did not solve your connectivity problem. Then you realized the renaming has happened, you fixed your configuration and the connectivity was back again.

The problem (to be solved):

Your VM now can talk to another machine (192.168.1.82) in the same LAN, but cannot talk to the gateway anymore.

The fact the VM can't receive an arp reply from the gateway means they can't even communicate at the ethernet level.

Possible causes (to be checked):

• check if the new virtual NIC is correctly attached to the same virtual network the gateway is attached to.
• check if there is any filter (like iptables or something that adds iptables rules, probably on the hypervisor, not the VM itself) applied to the gateway NIC or the VM NIC, which might allow traffic with the old mac address but not the new one.
• check if the new virtual NIC type you chose is supported by the guest OS.

Comments

[alari]

It is set to right gateway vlan and there are no rules under iptables on the vm, not sure if hypersivsor would have any as it was working fine just an hour before the upgrade, will check it

[Daniele Barresi]

Just to narrow down a little: what virtualization platform are you using? VMware, kvm, xen...

[alari]

its Vmware platform

[alari]

Okay figured it out, looks like when I added the network adapter I chose vmxnet3 but the old adapter was E1000. I just removed the interface again and added E1000 adapter and renamed the interface name again in /etc/network/interface file and restarted. Its working fine now. Any idea how to add vmxnet3 if needed and where to configure it? Should i install vmware tools for that?

[Daniele Barresi]

Yes, you must install VMware Tools to have a driver for the vmxnet network adapter.

[Daniele Barresi]

Edited my answer to include the NIC adapter compatibility issue as a possible cause of problems.