Running Vagrant/libvirt: trying to establish public/private networks

Question

I'm running Vagrant 1.7 (from their official 1.7 packages) on Ubuntu 14.04 LTS and I'm trying to provision several private VMs and one public one. My Ansible provisioning scripts change all the vagrant/root passwords and replace the Vagrant stock SSH keys for the security aspect.

All my internal IPs are 10.10.20.0/24. I have one VM that needs the IP 10.42.7.226 (which has a NAT going to it from our real external IP for ports 80/443).

The hypervisor is at 10.42.7.227 and I created a br0 with that IP connected to eth0. My network configuration looks like the following:

auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
    address 10.42.7.227
    netmask 255.255.255.0
    network 10.42.7.0
    broadcast 10.42.7.255
    gateway 10.42.7.1
    dns-nameservers 8.8.8.8


auto br0
iface br0 inet static
    address 10.42.7.227
    netmask 255.255.255.0
    gateway 10.42.7.1
    bridge_ports eth0
    bridge_stp off
    bridge_maxwait 0
    bridge_fd 0

I'm working on this remotely, which is why I was afraid to remove the IP from eth0. I realize that br0 should supersede it (br* have the real IPs and the eth* don't).

In my Vagrent config, I have a block like the following:

  config.vm.define "haproxy" do |haproxy|
    haproxy.vm.network :private_network, ip: 10.10.20.12
    haproxy.vm.network :public_network, ip: 10.42.7.226
    haproxy.vm.hostname = vars.hostname('haproxy')
    haproxy.hostmanager.aliases = vars.aliases('haproxy')
    haproxy.vm.provision "ansible" do |ansible|
      ansible.playbook = "ansible/haproxy.yml"
    end
  end

However in that VM, I only see the following adapters:

eth0      inet addr:192.168.121.189     
eth1      inet addr:10.10.20.12    
eth2      inet addr:10.42.7.169  Bcast:10.42.7.255  Mask:255.255.255.0

and on the hypervisor I see the following on br0:

brctl show
bridge name bridge id       STP enabled interfaces
br0     8000.0022192df5a7   no      eth1
virbr0      8000.000000000000   yes     
virbr1      8000.525400eda938   yes     virbr1-nic
                            vnet0
                            vnet2
                            vnet4
                            vnet6
virbr2      8000.525400c3098d   yes     virbr2-nic
                            vnet1
                            vnet3
                            vnet5
                            vnet7

and the hypervisor's ifconfig for the virbr* devices:

virbr0    inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
virbr1    inet addr:192.168.121.1  Bcast:192.168.121.255  Mask:255.255.255.0        
virbr2    inet addr:10.10.20.1  Bcast:10.10.20.255  Mask:255.255.255.0

It looks like libvirt/kvm are establishing some 192 networks for some reason. I'm not too concerned about those, but I'm guessing my 10.42.7.226 got converted into 10.42.7.169 somehow within the Vagrant/libvirt provisioning.

I'm guessing I need another network in libvirt? All I seem to have is the default:

<network>
  <name>default</name>
  <uuid>baa4b92a-b8ee-4e2f-a31a-bb3112b51dc0</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:9d:3b:a9'/>
  <ip address='10.10.20.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='10.10.20.2' end='10.10.20.254'/>
    </dhcp>
  </ip>
</network>

I'm a little confused on the documentation for libvirt. I basically need my haproxy VM to have an adapter with the IP address 10.42.7.226 accessible on the physical network connected to eth0 of the host machine. What do I need to do in this configuration to make this happen?

Solution 1

The current version of vagrant-libvirt plugin appears to support using a conventional bridge with macvtap.

Create a bridge on your host, and use that as the public_network device in your Vagrantfile.

Host having primary adapter em1, dhcp:

auto em1
iface em1 inet manual
auto br0
iface br0 inet dhcp
  bridge_ports em1
  bridge_stp off
  bridge_fd 0
  bridge_maxwait 0

Vagrantfile:

config.vm.network :public_network,
    :dev => "br0",
    :mode => "bridge",
    :type => "bridge"

This results in the guest having two adapters - one that is bound to the management network, another that used your bridge to retrieve an IP address via DHCP.

Solution 2

For the configuration you want, you need to have the virtual machine's NIC use your existing bridge br0 on the host. Unfortunately vagrant-libvirt doesn't seem to support this configuration (it only uses macvtap, which is meant to take over a physical interface completely and doesn't help you here because the host cannot use the interface).

I would contact the author of vagrant-libvirt and ask for this functionality to be added.

Comments

[djsumdog]

ahh. I should have read the rest of that README. Actually this could work. I should be able to connect eth1 to the same physical network and let my VM bridge to eth1, then assign it an IP on that network within the VM.