Correct way to route between 2 interfaces with netplan in Ubuntu 18.04

Question

Problem and Network Configuration

I am currently trying to allow communication between two interfaces, each with their own subnet, in Ubuntu 18.04 server using netplan but I am having a difficult time getting the configuration correct. Here is a graphical representation of what the network looks like:

---

Network Explanation

In the image the center yellow device is a DHCP server for the clients on the left using the enp8s0 interface with static ip 192.168.254.254 and subnet mask of 255.255.255.240. The clients (orange boxes) get their ip from the DHCP server. Each client is also hosting a webpage via Nginx. All these devices are running Ubuntu 18.04 server. The addresses possibly changing on each client machine is not a concern.

On the right side, the yellow "server" has interface enp7s0 configured with a static ip of 172.16.0.1 and subnet mask of 255.255.255.252. This interface is then connected to my laptop that has its interface set to 172.16.0.2 with the same subnet mask.

---

Overall Goal

What I am trying to do is be able to view the website at any one of the clients from my laptop. There is no need for connection to the internet on any of these machines and all connections are done over ethernet cables.

---

Current Configuration

Netplan:

Yellow "server" netplan config file:

network:
  version: 2
  renderer: networkd
  ethernets:
    enp7s0:
      addresses: [172.16.0.1/30]
      gateway4: 172.16.0.1
      routes:
        - to: 192.168.254.240/28
          via: 172.16.0.1
          on-link: true
    enp8s0:
      addresses: [192.168.254.254/28]
      gateway4: 192.168.254.254
      routes:
        - to: 172.16.0.0/30
          via: 192.168.254.254
          on-link: true

IP Forwarding:

The line net.ipv4.ip_forward=1 is uncommented in the /etc/sysctl.conf file.

Running cat /proc/sys/net/ipv4/ip_forward returns 1.

DHCP Server:

I have set INTERFACESv4 equal to enp8s0 in /etc/default/isc-dhcp-server.

Lastly, my /etc/dhcp/dhcpd.conf is configured as follows:

# option definitions common to all supported networks...

default-lease-time 600;
max-lease-time 7200;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
authoritative;

# So DHCP server knows of other subnet
subnet 172.16.0.0 netmask 255.255.255.252 {
}

# DHCP server subnet
subnet 192.168.254.240 netmask 255.255.255.240 {
  range 192.168.254.241 192.168.254.253;
  option subnet-mask 255.255.255.240;
  option routers 192.168.254.254;
  option broadcast-address 192.168.254.255;
  default-lease-time 600;
  max-lease-time 7200;
}

Solution

I would not be using routes to try and route traffic properly between the two subnets. You might end up in a routing loop that'll break things.

What you should probably consider doing is actually making your system behave as a router and do all the forwarding with NAT. A quick and simple way to do this is to have NAT MASQUERADE on each of the interfaces. (But you'll also need to yank out the route rules you have in place, since they won't work properly.)

You need to add rules to the firewall that would handle the following cases:

• 192.168.254.240/28 -> 172.16.0.0/30
• 172.16.0.0/30 -> 192.168.254.240/28

With iptables you'd need to set it up like so (the commented lines with # at the beginning just explain what each rule does):

# Allow traffic to be forwarded from enp7s0 to enp8s0
iptables -A FORWARD -i enp7s0 -j ACCEPT
# Allow traffic to be forwarded from enp8s0 to enp7s0
iptables -A FORWARD -i enp8s0 -j ACCEPT

You also need to set up NAT rules, and this is not going to be very nice, but we have to let the 'router' be the one that we masquerade sources as.

iptables -t nat -A POSTROUTING -o enp7s0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o enp8s0 -j MASQUERADE

This should permit bidirectional NAT traversal between the subnets. Make sure you save these rules though for the future.

Let me know if this doesn't work, I'll go and dig into the issue more deeply if that happens.

Comments

[Thomas Ward]

Is the DHCP server also the router? Or does it only serve DHCP? To route between the networks you need a router that can handle the NATing of the traffic between networks

[John T]

It should also be acting as the router between the subnets but that is where I am having difficulty. Many of the previous questions I see on this topic are using the /etc/network/interfaces file and don't have a similar setup to what I have. Some guidance on configuring the router correctly would help tremendously. Also, thank you for putting the picture in the question!

[Thomas Ward]

You don't have NAT configured it seems. I'm in the middle of an answer that might help. I presume, though, you want free bidirectional communication between the two subnets without any restrictions on which system on which subnet can communicate to which system on the other, so there's no real Access Control Lists between the two subnets?

[John T]

Correct, ideally I would like to be able to communicate back and forth between the subnets to any system I like. Most often it will be like the scenario above where I only need my laptop that is on the 172.16.0.0/30 subnet to be able to communicate with any machine on the 192.168.254.240/28 subnet. Having the ability to go the other way as well may be beneficial in the future though.

[John T]

If I try to ping one of the clients, lets say 192.168.254.244, from my laptop with the command ping 192.168.254.244 -I 172.16.0.2 it will ping but just doing ping 192.168.254.244 will not. If I also open my browser and try to navigate to 192.168.254.244 I do not get the webpage that is being hosted on that client.

[John T]

I have also seen ufw used instead of iptables, which is a solution I had tried earlier but still didn't work. Is there an advantage to using one over the other?

[Thomas Ward]

UFW is just an 'uncomplicated' way of maintaining rules, it uses iptables/netfilter under the hood.

[Thomas Ward]

ping not working sounds like a gateway problem, with a misconfigured gateway or such. Let me dig into this further on a test network. (Also, the very small subnet IP masks you're using can make gateway configuration a PITA, so I may instead to testing with a larger subnet range such as a /24 for each segment of the network...)

[Thomas Ward]

@JohnT I actually think the problem is the routes you've set on the router. It doesn't know how to route traffic around. I set the two LAN interfaces without routes on my 18.04 router box in a lab environment, and then used iptables and NAT to determine how to route traffic around. Does your laptop you mentioned sit on both subnets? And if not, I think you've failed basic routing 101 because the left side of your network is all /28s talking to each other in differing IP ranges which means you have no choice but to have the MASQUERADE ruleset in place...

[Thomas Ward]

... as well as ALLOW rules to accept traffic from every other system to reach back inside the single-subnet on the right. It sounds like what you want is a basic router and not much else...

[John T]

This is my first venture really delving into routing and networking so yes, I probably would have failed basic routing 101 which is why I am asking questions to learn. Please don't be rude about it. The laptop is only on the 172.16.0.0/30 subnet. From what I understand based on this Subnet Mask Cheat Sheet my clients on the left side should all be in the same IP range on the /28 subnet. My DHCP server only hands out addresses between 192.168.254.241 and 192.168.254.253.

[John T]

I will remove the routes from my netplan configuration file and try your answer again.

[John T]

In my stupidity, my laptop was still connected to my normal router and was trying to route through there. After disabling my WiFi and trying again your answer worked. Thank you for the help!