How to configure macvtap to let it pass multicast packet correctly?

Question

I'm trying to create VMs which can see each other and host. They are hosted with qemu/kvm and managed via libvirt. The network adapter of VMs is created with macvtap(VEPA mode) for performance.

After enabling hairpin over switch, the unicast stream is all OK, no matter among VMs or between VM and Host.

When it comes to multicast, however, I'm faced with problem. I find that host cannot ping6 each VM. With tcpdump, I observe that the Neighbor Solicitation packet to multicast address ff02::1:ff00:212 from host is not received by my VM, whose ipv6 address is 2001:da8:a0:600::212/64. Obviously this multicast packet should have been passed to VM by macvtap.

As a result of the problem of multicast, all ipv6 packet is lost for Neighbor Discovery cannot work properly.

I'm sure that there is nothing with the switch, because when I run tcpdump over the physical network adapter, I can see Neighbor Solicitation twice per second, one out, one in.

After I set the macvtap interface over the host to promiscuous mode, the VM get the Neighbor Solicitation packet, along with some other multicast packet which should be filtered by macvtap, but no unicast packet to other VM, even when I ping6 other VMs on the host at the same time.

So I think enabling promiscuous mode over all macvtap interface is an acceptable workaround but not graceful.

All my host and VM is CentOS 7.0. I've tried to install kernel-ml(linux 4.1.3) from elrepo on my host but it make not difference.

So:

1. how can I set all macvtap interface to promiscuous mode everytime it's started by libvirt?
2. I'm not familiar with net driver. But according to http://www.makelinux.net/ldd3/chp-17-sect-14, I'm suspicious that there is some bugs in the driver of macvlan so that the kernel cannot set the multicast list of the interface correctly. However, there is no set_multicast_list in linux/driver/net/{macvlan.c,macvtap.c}. Where is the correct place to search for help?

See also: https://bugzilla.redhat.com/show_bug.cgi?id=1035253

Solution 1

libvirt's macvlan has gained support for multicast. Unfortunately it is disabled by the default setting trustGuestRxFilters="no", and the documentation is not explicit that this breaks multicast. As you observed, breaking multicast also breaks IPv6.

https://bugzilla.redhat.com/show_bug.cgi?id=1035253#c15

You may be able to work around this by manually setting trustGuestRxFilters="yes". There is a limitation: "support depends on the guest network device model, as well as the type of connection on the host". "Currently it is only supported for the virtio device model, and for macvtap connections on the host".

https://libvirt.org/formatdomain.html#elementsNICS

IMO the natural model would be to allow multicast by default. Blocking multicast reception, on a network you think you're directly connected to, is an unpleasant surprise. Especially since macvtap still seems to allow sending multicast packets (as well as spoofed MAC source addresses!).

Solution 2

The answer by sourcejedi contains the solution, but maybe not explicitly enough. Using "virsh edit", set the trustGuestRxFilters attribute on the network device:

<interface type='direct' trustGuestRxFilters='yes'>

See libvirt docs. The same can be done for all interfaces on a libvirt network.

Solution 3

faced the same issue with macvtap. I found a way to fix it but I don't know how to automate it inside virsh. sudo ip link set dev macvtap0 allmulticast on

Comments

[Given92]

It's more graceful than promisc, but this is still a workaround. The only way I could figure out to configure it automatically is crontab.

[Given92]

this solution is not perfect for large amount of packets in ipv6 network.

[Given92]

I have just figured out this solution occasionally. Thanks very much. This feature requires additionally documented.

[3ronco]

Thanks, that caused a lot of headaches. I think you're right it would be natural to configure IPv6 (intending it's use) and expecting things to simply work and multicast is such an essential part of IPv6 that adding some confusing libvirt config kungfu to make it work isn't obvious and helpful here or at least giving a word of warning that you need promiscous mode on your macvtap interface would mitigate the unclear situation.

[ws6079]

Maybe also worth highlighting here (although stated in the documentation) that trustGuestRxFilterscurrently only works with the virtio driver.

[mu1988]

@ws6079 Thanks! I've added a direct link to the document, and highlighted this limitation.