Routing networking on kvm

Question

There are a lot of configuration used for Kvm-networking. But i'm not able to get to the guest from the host or outside. I'm working on Ubuntu 11.04. On the guest i've a WindowsXp with dhcp.

I want the guest to be in the same network of the host. i've tried to use ip aliases

i've set up bridged networking in /etc/network/interfaces

auto eth0
iface eth0 inet manual

auto eth0:1
iface eth0:1 inet static
address 192.168.0.11
netmask 255.255.255.0

auto br0
iface br0 inet static
         address 192.168.0.10
         netmask 255.255.255.0
         gateway 192.168.0.1
         bridge_ports eth0
         bridge_stp off
         bridge_fd 0
         bridge_maxwait 0

then changed default network in /etc/libvirtd/qemu/network/default.xml

<network>
<name>default</name>
<uuid>831a93e1-0b84-0b0e-9ca2-23c407983968</uuid>
<forward mode='route'/>
<bridge name='virbr0' stp='on' delay='0' />
 <ip address='192.168.122.1' netmask='255.255.255.0'>
  <dhcp>
    <range start='192.168.122.100' end='192.168.122.254' />
    <host mac='52:54:00:7c:df:88' name='vm' ip='192.168.122.99' />
  </dhcp>
</ip>
</network>

the network in /etc/libvirt/qemu/vm.xml

<interface type='network'>
  <mac address='52:54:00:7c:df:88'/>
  <source network='default'/>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>

the guest from dhcp gets the correct ip. finally direct traffic from external interfaces to internal and back

sudo iptables -t nat -A PREROUTING -d 192.168.0.11 -j DNAT --to-destination 192.168.122.99

sudo iptables -t nat -A POSTROUTING -d 192.168.122.99 -j SNAT --to-source 192.168.0.11

so in the end the configuration is this:

$> brctl show
bridge name          bridge id      STP enabled interfaces
br0             8000.0026b902076d   no      eth0
virbr0          8000.fe54007cdf88   yes     vnet0

$> route
Tabella di routing IP del kernel
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.0.0     *               255.255.255.0   U     0      0        0 br0
192.168.0.0     *               255.255.255.0   U     0      0        0 eth0
192.168.122.0   *               255.255.255.0   U     0      0        0 virbr0
link-local      *               255.255.0.0     U     1000   0        0 br0
default         192.168.0.1     0.0.0.0         UG    100    0        0 br0

$> ifconfig

br0       Link encap:Ethernet  HWaddr 00:26:b9:02:07:6d  
      indirizzo inet:192.168.0.10  Bcast:192.168.0.255  Maschera:255.255.255.0

eth0      Link encap:Ethernet  HWaddr 00:26:b9:02:07:6d  
      indirizzo inet6: fe80::226:b9ff:fe02:76d/64 Scope:Link

eth0:1    Link encap:Ethernet  HWaddr 00:26:b9:02:07:6d  
      indirizzo inet:192.168.0.11  Bcast:192.168.0.255  Maschera:255.255.255.0

virbr0    Link encap:Ethernet  HWaddr fe:54:00:7c:df:88  
      indirizzo inet:192.168.122.1  Bcast:192.168.122.255  Maschera:255.255.255.0

vnet0     Link encap:Ethernet  HWaddr fe:54:00:7c:df:88  
      indirizzo inet6: fe80::fc54:ff:fe7c:df88/64 Scope:Link

what is wrong? Or how i can set up a guest visible outside the host?

Solution 1

I ran into this one a time ago. But there seem to be no way to setup the bridge on an alias interface, like eth0:1. Use the real interface eth0.

auto eth0
iface eth0 inet static

auto br0
iface br0 inet static
     bridge_ports eth0
     address 192.168.0.10
     netmask 255.255.255.0
     gateway 192.168.0.1
     broadcast 192.168.0.255
     bridge_stp off
     bridge_fd 0
     bridge_maxwait 0

Also, the address, netmask, gateway and broadcast values are the mininal configuration an interface should have. It may work with less values, but may cause weird network behavior.

As I further remember, you don't have to edit default.xml at all. You just have to make shure, that every KVM Guest has it's network interface setup to your needs.

<interface type='bridge'>
  <mac address='00:01:b4:02:00:db'/> # change per guest
  <source bridge='br0'/>             # the name of your source bridge
  <target dev='vnet0'/>              # the name, the network interface has for the guest
</interface>

The setup is to complex to cover all varieties here, like DHCP vs static setup. Did you check out the Ubuntu documentation on KVM? Helped me a lot to get into this.

Solution 2

Why not simply set the guest to use br0? Instead of configuring it with a "network", you can configure it with a "bridge" and attach the VM to br0. example:

<interface type='bridge'>
<source bridge='br0'/>
<mac address='00:16:3e:1a:b3:4a'/>
</interface>

This way the guest will get an IP from the external net and can contact the work and host through it.

NOTE: However, the downside of using a bridge is that you won't see the client ip address of connections to your VM. Instead, the IP address of the bridge host will be logged in apache, auth.log, etc on a linux VM and similarly in a windows VM.

Solution 3

Please consider my answer just a slim addition to @ansi_lumen's answer.

If you want your VM to see the network exactly as KVM host sees, you have to configure your networking first without thinking about KVM. The bridge mode should be used and it shouldn't have any negative effect on your server comparing with standard usage.

So if you have a network interface called enp1s0 on your host, you should reset its settings. Ubuntu example in /etc/netplan/01-netcfg.yaml:

  ethernets:
    enp1s0:
      dhcp4: no 
      dhcp6: no 

Now you move your settings to bridges block and point to your real interface:

  bridges:
    br0:
      dhcp4: no
      dhcp6: no
      interfaces: [enp1s0]
      addresses: [192.168.1.239/24]
      gateway4: 192.168.2.1
      nameservers:
        addresses: [8.8.8.8]
      parameters:
        stp: true
        forward-delay: 4

Expected is a short network interruption if no mistakes made:

sudo netplan generate 
sudo netplan --debug apply; sleep 30; echo 'press CTRL+C, network is working'; sleep 300; sudo ifconfig enp1s0 192.168.1.239 up; sudo ip route add default via 192.168.1.2 dev enp1s0

Now you apply this network configuration in KVM:

<network>
  <name>host-bridge</name>
  <forward mode="bridge"/>
  <bridge name="br0"/>
</network>

Or if you are using terraform-provider-libvirt, then in libvirt_domain resource:

  network_interface {
    bridge = "br0"
    mac    = "00:00:00:00:00:01"
  }

You don't need to touch anything else in KVM.

The reference:

https://fabianlee.org/2019/04/01/kvm-creating-a-bridged-network-with-netplan-on-ubuntu-bionic/

Comments

[lucasart]

I'm facing the same issue with alias NICs as apparently this cannot be done. However, when bridging directly the primary NIC (here, eth0), won't the aliases be brought down afterwards?

[laimison]

What kind of interface VM will have in the end? Is it still vnet0? Is it broader bridging issue when original IP is not visible on VM? Or is it specific to this approach and how solution should be changed to solve original IP appearance?

[laimison]

Am I right that using this approach host acts as a switch? So VM directly communicates with network exactly as host sees it. For instance IP will be preserved in logs, etc. (between VM-IP <-> IPs in host's network)

[moestly]

@laimison yes, the VMs could get ips from an external DHCP (router for example) and are visible to the network as normal clients.