How to bridge networking in this nested virtualization case?

Question

This is a follow-up for my earlier question. Based on feedback from Daniel B and Deltic, I switched from VirtualBox to QEMU, which can handle 64-bit guests on a VM as a host (Hyper-V, in my case) - a scenario referred to as nested virtualization.

After installing QEMU and reading some documentation, I successfully converted the given OVA image into (a native to QEMU) QCOW2 format. I noticed that the resulting image is much larger than the original one (6.4G vs. 1.3G), but, perhaps, this is normal. Then I learned how to launch a VM, using QEMU, and tried to do just that:

qemu -nographic -hda <IMAGE_NAME>.qcow2 &

Then I realized (and later confirmed by further reading of QEMU documentation) that the default networking scheme implies lack of bridging between the host and the guest OSes. My attempt to enable the bridging, as follows, failed (based on this blog post):

qemu -nographic -net bridge:br0 -hda <IMAGE_NAME>.qcow2 &

I'm hesitant to use the method, based on using bridge-utils, described here, as it might mess up networking for the host OS. Also, based on this document, I tried to redirect ports (see below), but it didn't work for me either. If TAP interfaces is the solution, then how should I apply it to my situation.

My situation is that the guest VM starts a Web application, which can be accessed in a local mode by navigating to the URL http://192.168.xx.yy/<APPLICATION_NAME>. Basically, I need to expose the application (running in nested virtualization mode) to the world. So, my question is twofold:

1) what is the optimal (and, hopefully, the easiest) way to enable network bridging in my situation;

2) should I and, if Yes, how exactly can I redirect ports to allow users to access the application at https://<FQDN of the host VM>/<APPLICATION_NAME>, preferably, via non-standard ports?

Solution

Your host itself is already a virtual machine, and it sounds like you don't have access to the physical server or network. Depending on your hosting provider, bridging may or may not work.

---

Bridging Prerequisites

A virtual machine connected to a bridge needs to be able to obtain its own IP address as well as use its own unique MAC address. You would need to check with your hosting provider to see what MAC address and IP address restrictions your Hyper-V virtual machine has.

MAC addresses

Particularly, if your hosting provider prevents MAC address spoofing (often done to prevent hijacking other virtual machines' networks), then you may not be able to use bridging without special support from your hosting provider.

IP addresses

If you can get over that hurdle, your nested virtual machine needs to be able to obtain an IP address. You can do this if your hosting provider assigns you more than one IP address or an IP address range.

Alternative: NAT

If you cannot meet both of the prerequisites for bridging, you can set up NAT and then port forward the ports you need from the nested virtual machine through the host's only networking route.

It's a little bit of a mess to port forward but definitely doable. I've successfully used libvirt's instructions before.

---

Building the Bridge

There are multiple ways to build a bridge, but since you're on Debian, I'll give you a sample /etc/network/interfaces configuration that implements bridging on boot:

auto eth0
iface eth0 inet manual

auto br0
iface br0 inet static
        address 192.168.0.111
        netmask 255.255.255.0
        gateway 192.168.0.1
        metric 0
        bridge_ports eth0
        bridge_stp on
        bridge_fd 0
        bridge_maxwait 0
        dns-nameservers 8.8.8.8 8.8.4.4
iface br0 inet6 dhcp

The configuration above assumes that the enslaved interface is eth0 and your host is 192.168.0.111 on the 192.168.0.0/24 subnet.

What's ideal is if you can configure a bridge like this and your nested virtual machine has free rein to take another address on that subnet.

I assume that you have a private/fixed IP network because in your question, you wrote "192.168.xx.yy", so all that's left to do to make your nested virtual machine accessible to the outside world is to attach a public/floating IP address to that private/fixed IP address.

Again, if you only have one IP address and/or one MAC address to work with, you may have to settle for NAT and port forwarding. Otherwise, you should be able to mould this networking configuration to fit your network and configure bridging as you wanted.

Comments

[Aleksandr Blekh]

Excellent answer. I very much appreciate your help. Your assumption about me not having access to the physical server / network is correct. However, I can inquire about needed information on the Hyper-V virtual machine and then try relevant steps from your advice. Will work on all that and share (hopefully, positive) results.

[Aleksandr Blekh]

This phrase is a bit unclear to me: "attach a public/floating IP address to that private/fixed IP address". How exactly this can be done in my case? Could you clarify or refer to the relevant section in documentation?

[Deltik]

@AleksandrBlekh: It depends on the hosting provider's networking. Do you get one public IP address? Multiple? And are you on a private IP network? Does the public IP attach to the private IP like in a home network? There are a lot possible configurations, so I can only guess unless you provide more details about your network.