Port forwarding between bridged interfaces

Question

So I have a bunch of bridge interfaces bound with my main ethernet device (em1, blame HP). These serve various LXC containers I have running on my server and easily allows me to access them from other physical devices on the network.

name    id                  STP   interfaces    IP
br0     8000.989096db8b8a   no    em1           10.10.0.2
                                  veth236T4V    10.10.0.15
                                  veth269GNR    10.10.0.16
                                  vethBYBC0Y    10.10.0.17

These all get their IPs from the main network DHCP (which assigns static leases).

I want to move a service that has been running on the main host (em1, 10.10.0.2, ports 9000, 9001) to the first LXC container. I have done this and can now access things through 10.10.0.15:9000-9001, but everything else on the network expects to see it on 10.10.0.2:9000-9001.

Traditional port forwarding through iptables doesn't seem to work. I've tried:

-A PREROUTING -i em1 -p tcp --dport 9000 -j DNAT --to 10.10.0.15:9000
-A PREROUTING -i em1 -p tcp --dport 9001 -j DNAT --to 10.10.0.15:9001

And I've tried br0 instead of em1 but neither work.

In a hail of 3am research I found a load of stuff suggesting I need ebtables but I'd never even heard of that before. Half of the problem seems to be that most people use lxcbrN devices with LXC but I needed the external IP. I'm not sure what I need. This isn't helped by the ebtables documentation seemingly defining the word "port" as something else.

I'm out of my depth. I can't feel the floor any more and I'm starting to tread water. Can anyone throw me a line and say for certain what I need to redirect a couple of ports between bridged interfaces?

Solution

You can use iptables. Below is a script version of a proposed solution. I do not know what iptables rules you might already have, so some merging work might be required.

#!/bin/sh
FWVER=0.02
#
# test-oli rule set 2016.01.14 Ver:0.02
#     Having tested this on my test server using port 80,
#     convert for what Oli actually wants (which I can not test).
#
# test-oli rule set 2016.01.14 Ver:0.01
#     Port forward when this computer has one nic and
#     is not a router / gateway.
#     In this case the destination is a guest VM on this
#     host but, with bridged networking and all IP addresses
#     from the main LAN, that should not be relevant.
#
#     This script may conflict with other iptables rules on the
#     host, I don't know. On my test server, clobbering the existing
#     iptables rules is O.K. because I do not use the virbr0 stuff,
#     nor the default virtual network,  anyhow.
#
#     References:
#     http://askubuntu.com/questions/720207/port-forwarding-between-bridged-interfaces
#     http://ubuntuforums.org/showthread.php?t=1855192
#     http://www.linuxquestions.org/questions/linux-networking-3/iptables-forwarding-with-one-nic-80009/
#
#     run as sudo
#
echo "test-oli rule set version $FWVER..\n"

# The location of the iptables program
#
IPTABLES=/sbin/iptables

# Setting the EXTERNAL and INTERNAL interfaces and addresses for the network
# Use br0 instead of eth0. While using eth0 seems to work fine, the packet counters
# don't work, so debugging information is better and more complete using br0.
#
#
INTIF="br0"
INTIP="10.10.0.2"
FORIP="10.10.0.15"
UNIVERSE="0.0.0.0/0"

echo " Internal Interface: $INTIF  Internal IP: $INTIP  Forward IP $FORIP"

# CRITICAL:  Enable IP forwarding since it is disabled by default
#
echo Enabling forwarding...
echo "1" > /proc/sys/net/ipv4/ip_forward

# Clearing any previous configuration
#
echo " Clearing any existing rules and setting default policy to ACCEPT.."
$IPTABLES -P INPUT ACCEPT
$IPTABLES -F INPUT
$IPTABLES -P OUTPUT ACCEPT
$IPTABLES -F OUTPUT
$IPTABLES -P FORWARD ACCEPT
$IPTABLES -F FORWARD
$IPTABLES -t nat -F
# Delete user defined chains
$IPTABLES -X
# Reset all IPTABLES counters
$IPTABLES -Z
# While my references do not have it, I think this is needed.
$IPTABLES -t nat -Z

# First we change the destination of any incoming port 80 traffic
#
$IPTABLES -t nat -A PREROUTING -p tcp -i $INTIF --dport 9000 -j DNAT --to-destination $FORIP:9000
$IPTABLES -t nat -A PREROUTING -p tcp -i $INTIF --dport 9001 -j DNAT --to-destination $FORIP:9001

# And then we do the actual forward
# FORWARD rules would only be needed if the default policy is not ACCEPT
# (Shown here for completeness)
#
$IPTABLES -A FORWARD -p tcp -i $INTIF -d $FORIP --dport 9000 -j ACCEPT
$IPTABLES -A FORWARD -p tcp -i $INTIF -d $FORIP --dport 9001 -j ACCEPT

# Now, we need to change the source address, otherwise the reply packets
# would be sent directly to the client, causing confusion.
$IPTABLES -t nat -A POSTROUTING -o $INTIF -j SNAT --to-source $INTIP

echo "test-oli rule set version $FWVER done."

Comments

[Oli]

Well I thought this looked suspiciously like what I was already trying through ufw but I stripped it back to the (PREROUTING/FORWARD/POSTROUTING) rules, ran them and it only bloody works. Many thanks. Adding a bounty as a reward for such a thorough answer (which I will award in 24 hours when I can).

[naisanza]

@DougSmythies does your solution have iptables Filtering for Bridge devices enabled or disabled? (net.bridge.bridge-nf-call-ip6tables, net.bridge.bridge-nf-call-iptables, net.bridge.bridge-nf-call-arptables)

[Doug Smythies]

@naisanza : As far as I know/recall, disabled. I don't even have those files (/proc/sys/net/bridge does not exist).

[naisanza]

@DougSmythies you're right, it appears that /proc/sys/net/bridge seems to appear in the Desktop version of Ubuntu, and not the Server version (I noticed this when I realized the Desktop version had been installed)