Setting up bridged LXC containers with static IPs

Question

I'm trying to setup multiple LXC containers on a host, each with public static IPs of their own.

My host is running the latest Ubuntu. It has a single network interface named eth0. The static IPs are pingable from the internet and are named eth0:210, eth0:211... The numbers after the colon are the least significant byte of the addresses. In addition to these interfaces I have br0 setup on the host's public IP. There are also the lo, veth2LPP9A, and lxcbr0 interfaces. The lxcbr0 has the address of a private IP.

The host /etc/network/interfaces looks like:

auto br0
iface br0 inet static
        bridge_ports eth0
        bridge_fd 0
        [...]

So far I've used various online sources, including Bridging LXC containers to host eth0 so they can have a public IP to help me set this up.

The container's config file has:

lxc.network.type = veth
lxc.network.link = br0

I've removed the static lxc.network.ipv4 configuration from this file since it caused problems. When I ran lxc-ls --fancy with this configuration, I'd see the same public IP twice in the output. Additionally, it would mess with the subnet configuration of the container's /etc/network/interfaces.

Speaking of the container's interfaces file, it looks a bit like:

auto eth0
iface eth0 inet static
        address [...]
        netmask 255.255.255.255
        #gateway [...]
        dns-nameservers 8.8.8.8

        post-up route add [...] dev eth0
        post-up route add default gw [...]
        post-down route del [...] dev eth0
        post-down route del default gw [...]

I had to comment out the gateway and add the route add commands to this file. Otherwise, the container would take minutes to boot.

The /proc/sys/net/bridge/bridge-nf-* files on the host are all set to 0. The /proc/sys/net/ipv4/ip_forward value is 1.

The problem is, even though the container's "route -n" looks like it should, I can't ping out of the container. SSHing to what should be the container's IP, connects me to the host.

EDIT: Removing the container's static IP from the host did help, but now I'm getting a new error. Trying to ping the container from the host results in Redirect HostFrom , New nexthop. The packets are just going from the gateway to the host, repeatedly. Running a traceroute from the host shows that the first stop is at the gateway. Then all of the other routes are * * *. I get the same problem regardless if the container is online or not.

Solution 1

In fact you can set the address and gateway from within the host and configure the container not to touch the interface at all using the keyword manual.

Place this within the guests /etc/network/interfaces:

auto eth0
iface eth0 inet manual

Also leave it up to the container's config file to set up the interface:

lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = lxc-bridge-nat
lxc.network.ipv4 = 192.168.100.16/24
lxc.network.ipv4.gateway = auto

The guest will behave like it's BIOS already set up the interface and just use it.

Especially explore lxc.network.ipv4.gateway.

Solution 2

The best way I find and the quickest is to use the lxc profiles

lxc profile list - command the list all the profiles you have Then

lxc profile copy default minecraft(this is the name of your new profile)

Then lxc profile edit minecraft

This will come up

devices:
  eth0:
    ipv4.address: 192.168.1.114/24  - add this line and enter any ip address you like
    name: eth0
    nictype: macvlan - this is my setting
    parent: enp0s25
    type: nic
  root:
    path: /
    pool: lxc_zfs
    type: disk
name: mine
used_by:

then save it

next assign the profile to your LXC container like this

lxc profile assign YOUR_CONTAINER_NAME YOUR_NEW_PROFILE_NAME

then just restart the container and your new ip address is set to that container

Solution 3

Since you are bridging, you need to set the IP addresses in the container only, and not on the host. The host should only have its own IP address(es).

Solution 4

I just did it the other day with Ubuntu 14.04. It is simple. You just have to edit the /etc/network/interfaces file inside your container, and set this:

auto eth0
iface eth0 inet static
 address $IP
 netmask $NETMASK
 gateway $GW
 dns-nameservers $DNS

Replace every variable with the desired value.

You DON'T have to do anything else!

PS: Notice the space before some lines. It is mandatory.

Solution 5

I could set up correctly my lxc containers after following @Enrique Moreno Tent's answer, so I will explain what to do in more details in case you don't know how to set up the other items.

1. Access your LXC container via lxc-attach command

Command:

$ lxc-attach -n YOUR-CONTAINER-NAME

2. See your current configurations with ifconfig

Command

root@mycontainer:~# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.3.160  netmask 255.255.255.0  broadcast 10.0.3.255
        inet6 fe80::216:3eff:fec9:2fa0  prefixlen 64  scopeid 0x20<link>
        ether 00:16:3e:c9:2f:a0  txqueuelen 1000  (Ethernet)
        RX packets 62  bytes 7142 (7.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 60  bytes 9788 (9.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 20285  bytes 175021803 (175.0 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 20285  bytes 175021803 (175.0 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

```

3. Getting address, netmask and gateway

Seeing the result of the ifconfig command above, we then see:

Address is the address you want, you can change it to 10.0.3.166.

Netmask is there: 255.255.255.0

Gateway: For the gateway you use the broadcast address with here is 10.0.3.255

As you can see above, you currently have all information you need to fill your guest (container) /etc/network/interfaces.

4. Getting dns-nameservers value.

Issue the command:

cat /etc/resolv.conf

But maybe is best using Google DNS, which asre 8.8.8.8 and 8.8.4.4

5. Editing /etc/network/interfaces INSIDE THE CONTAINER

auto eth0 iface eth0 inet static address 10.0.3.166 netmask 255.255.255.0 gateway 10.0.3.255 dns-nameservers 8.8.8.8

View more solutions

Comments

[Dominik Dorn]

Thx! This saved my life... my container would always try to get a new dhcp address... setting it to a fixed value inside the container (static) resulted in the container not even booting anymore!

[Dominik Dorn]

this resulted in my container not booting anymore (ubuntu 14.04 server).. the answer from sebastianwagner works for me.

[HRJ]

Note that this doesn't work well if the host is suspended and resumed. The lxc continues with its allocated ip-address but the host thinks it doesn't have an ip-address anymore. I have found it is better to configure the dnsmasq for lxc-net and handout a fixed address there.

[Ian Macintosh]

@HRJ - You should include the 'how' in your comment or link to the ref rather than just saying 'do it'. This is a technical site and you should say something actually useable.

[Ian Macintosh]

It seems like the correct place to do this is both in /etc/default/lxc where you can set your range (see LXC_DHCP_RANGE) and in /etc/dnsmasq.d-available/lxc as per the dnsmasq docs or manpage

[Ian Macintosh]

See How to let built-in DHCP assign a static IP to LXC container based on name, not MAC address

[rvazquezglez]

ifconfig has been replaced with ip a in more recent debian/ubuntu versions linuxconfig.org/…

[am70]

very nice answer but should be updated for LXC 3

[Chux Uzoeto]

I was looking for a solution for macvlan, so thanks for your suggestion. However, this seems so wrong because a profile is shared across all containers using that profile. So, for it to work you will have to make multiple copies of a shared profile, and use one profile per container. Now, imagine that you have tens of containers ... LOL. Oh well, I will have to live with dhcp assigned addresses for now ...