port forwarding to application in network namespace with vpn

Question

I was able to set up a network namespace, establish a tunnel with openvpn and start an application that uses this tunnel inside the namespace. So far so good, but this application can be accessed via a web interface and I cant't figure out how to route requests to the web interface inside my LAN.

I followed a guide from @schnouki explaining how to set up a network namespace and run OpenVPN inside of it

ip netns add myvpn
ip netns exec myvpn ip addr add 127.0.0.1/8 dev lo
ip netns exec myvpn ip link set lo up
ip link add vpn0 type veth peer name vpn1
ip link set vpn0 up
ip link set vpn1 netns myvpn up
ip addr add 10.200.200.1/24 dev vpn0
ip netns exec myvpn ip addr add 10.200.200.2/24 dev vpn1
ip netns exec myvpn ip route add default via 10.200.200.1 dev vpn1
iptables -A INPUT \! -i vpn0 -s 10.200.200.0/24 -j DROP
iptables -t nat -A POSTROUTING -s 10.200.200.0/24 -o en+ -j MASQUERADE
sysctl -q net.ipv4.ip_forward=1
mkdir -p /etc/netns/myvpn
echo 'nameserver 8.8.8.8' > /etc/netns/myvpn/resolv.conf

After that, I can check my external ip and get different results inside and outside of the namespace, just as intended:

curl -s ipv4.icanhazip.com
<my-isp-ip>
ip netns exec myvpn curl -s ipv4.icanhazip.com
<my-vpn-ip>

The application is started, I'm using deluge for this example. I tried several applications with a web interface to make sure it's not a deluge specific problem.

ip netns exec myvpn sudo -u <my-user> /usr/bin/deluged
ip netns exec myvpn sudo -u <my-user> /usr/bin/deluge-web -f
ps $(ip netns pids myvpn)
 PID TTY      STAT   TIME COMMAND
1468 ?        Ss     0:13 openvpn --config /etc/openvpn/myvpn/myvpn.conf
9302 ?        Sl    10:10 /usr/bin/python /usr/bin/deluged
9707 ?        S      0:37 /usr/bin/python /usr/bin/deluge-web -f

I'm able to access the web interface on port 8112 from within the namespace and from outside if I specify the ip of veth vpn1.

ip netns exec myvpn curl -Is localhost:8112 | head -1
HTTP/1.1 200 OK
ip netns exec myvpn curl -Is 10.200.200.2:8112 | head -1
HTTP/1.1 200 OK
curl -Is 10.200.200.2:8112 | head -1
HTTP/1.1 200 OK

But I do want to redirect port 8112 from my server to the application in the namespace. The goal is to open a browser on a computer inside my LAN and get the web interface with http://my-server-ip:8112 (my-server-ip being the static ip of the server that instantiated the network interface)

EDIT: I removed my attempts to create iptables rules. What I'm trying to do is explained above and the following commands should output a HTTP 200:

curl -I localhost:8112
curl: (7) Failed to connect to localhost port 8112: Connection refused
curl -I <my-server-ip>:8112
curl: (7) Failed to connect to <my-server-ip> port 8112: Connection refused

I tried DNAT and SNAT rules and threw in a MASQUERADE for good measure, but since I don't know what I'm doing, my attempts are futile. Perhaps someone can help me put together this construct.

EDIT: The tcpdump output of tcpdump -nn -q tcp port 8112. Unsurprisingly, the first command returns a HTTP 200 and the second command terminates with a refused connection.

curl -Is 10.200.200.2:8112 | head -1
listening on vpn0, link-type EN10MB (Ethernet), capture size 262144 bytes
IP 10.200.200.1.36208 > 10.200.200.2.8112: tcp 82
IP 10.200.200.2.8112 > 10.200.200.1.36208: tcp 145

curl -Is <my-server-ip>:8112 | head -1
listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
IP <my-server-ip>.58228 > <my-server-ip>.8112: tcp 0
IP <my-server-ip>.8112 > <my-server-ip>.58228: tcp 0

EDIT: @schnouki himself pointed me to a Debian Administration article explaining a generic iptables TCP proxy. Applied to the problem at hand, their script would look like this:

YourIP=<my-server-ip>
YourPort=8112
TargetIP=10.200.200.2
TargetPort=8112

iptables -t nat -A PREROUTING --dst $YourIP -p tcp --dport $YourPort -j DNAT \
--to-destination $TargetIP:$TargetPort
iptables -t nat -A POSTROUTING -p tcp --dst $TargetIP --dport $TargetPort -j SNAT \
--to-source $YourIP
iptables -t nat -A OUTPUT --dst $YourIP -p tcp --dport $YourPort -j DNAT \
--to-destination $TargetIP:$TargetPort

Unfortunately, traffic between the veth interfaces seized and nothing else happened. However, @schnouki also suggested the use of socat as a TCP proxy and this is working perfectly.

curl -Is <my-server-ip>:8112 | head -1
IP 10.200.200.1.43384 > 10.200.200.2.8112: tcp 913
IP 10.200.200.2.8112 > 10.200.200.1.43384: tcp 1495

I have yet to understand the strange port shuffling while traffic is traversing through the veth interfaces, but my problem is solved now.

Solution 1

I've always had issues with iptables redirections (probably my fault, I'm pretty sure it's doable). But for a case like yours, it's IMO easier to do it in user-land without iptables.

Basically, you need to have a daemon in your "default" workspace listening on TCP port 8112 and redirecting all traffic to 10.200.200.2 port 8112. So it's a simple TCP proxy.

Here's how to do it with socat:

socat tcp-listen:8112,reuseaddr,fork tcp-connect:10.200.200.2:8112

(The fork option is needed to avoid socat from stopping after the first proxied connection is closed).

EDIT: added reuseaddr as suggested in the comments.

If you absolutely want to do it with iptables, there's a guide on the Debian Administration site. But I still prefer socat for more advanced stuff -- like proxying IPv4 to IPv6, or stripping SSL to allow old Java programs to connect to secure services...

Beware however that all connections in Deluge will be from your server IP instead of the real client IP. If you want to avoid that, you will need to use a real HTTP reverse proxy that adds the original client IP to the proxied request in a HTTP header.

Solution 2

Interconnecting network namespace with main namespace always bothers me. The reason I usually create a namespace is because I want it isolated. Depending on what it is you are trying to achieve with namespaces creating interconnects can defeat that purpose.

But even isolated I still want to poke it over the network, for convenience.

This solution lets you keep isolation and forward some connections to it anyway. You don't need to create all that network between the two network namespaces just to forward one port. Run this in the namespace where you want to accept connections. Must be run as root for ip netns exec to work.

socat tcp-listen:8112,fork,reuseaddr \
  exec:'ip netns exec myvpn socat STDIO "tcp-connect:127.0.0.1:8112"',nofork

It listens for connections in one network namespace where you run it, on port 8112, then connected client gets exec to run ip netns exec myvpn ... to execute the rest inside the myvpn network namespace, then once inside the myvpn network namespace it creates second connection again with another socat.

Or run it as systemd service

This also uses socat.

Create service config file /etc/systemd/system/deluge-web-netns.service with content:

[Unit]
Description=Forwarder to deluge-web in netns
After=network-online.target
#Requires=deluge-web.service
#After=deluge-web.service

[Service]
Type=simple

ExecStart=/usr/bin/socat tcp-listen:8112,fork,reuseaddr exec:'ip netns exec vpn-netns socat STDIO "tcp-connect:127.0.0.1:8112"',nofork
#User=deluge
#Group=deluge
SyslogIdentifier=deluge-web-fwd

Restart=on-failure

# Time to wait before forcefully stopped.
TimeoutStopSec=300

[Install]
WantedBy=multi-user.target

Check/Fix the value in ExecStart=/path/to/socat to "be an absolute path to an executable" as required by man systemd.service.

Consider setting correct values and enabling the directives Requires and After.

Then enable and start with commands:

systemctl daemon-reload
systemctl enable deluge-web-netns
systemctl start  deluge-web-netns

Or run it under xinetd

This also uses socat.

create conf file, e.g. /etc/xinetd.d/deluge-web-fwd with content

service deluge-web-vpn-netns
{
    type            = UNLISTED
    socket_type     = stream
    protocol        = tcp
    port            = 8112
    flags           = IPv4 KEEPALIVE
    wait            = no
    user            = root
    server          = /sbin/ip
    server_args     = netns exec vpn-netns socat STDIO tcp-connect:127.0.0.1:8112
}

Restart xinetd

service xinetd restart

Program ncat can also be used similar to socat.

Solution 3

For deluge here is my solution. No need for iptables. Here are the steps:

1. Start your openvpn tunnel
2. Create namespace and bring your openvpn tunnel there:

ip netns add $NS
# Wait for the TUN to come up
while [[ $(ip route|grep $TUN|wc -l) == 0 ]]; do sleep 1; done
MY_IP=$(ip addr show $TUN|grep inet|cut -d' ' -f6|cut -d'/' -f1)
# The way you extract gateway IP might be different for your openvpn connection
GATEWAY_IP=$MY_IP
# jail my $TUN (VPN interface) into the namespace
ip link set $TUN netns $NS
# Bring the interface up with a subnet (equivalent to the one given to me by VPN server)
ip netns exec $NS ifconfig $TUN $MY_IP/24 up
# Bring loopback up
ip netns exec $NS ifconfig lo 127.0.0.1/8 up
# Set up remote gateway (your pointtopoint VPN IP address)
ip netns exec $NS route add default gw $GATEWAY_IP

3. Establish veth connection between your default namespace and the one you've created:

# Set up veth interfaces for communication between namespaces
ip link add veth0 type veth peer name veth1
# Move the second veth to your namespace
ip link set veth1 netns $NS
# give an IP from unused IP range to first veth
ifconfig veth0 10.1.1.1/24 up
# And the second one
ip netns exec $NS ifconfig veth1 10.1.1.2/24 up
# TODO: set up a bridge between veth1 and eth interface to let it communicate with LAN
# Set up DNS client. ip netns will emulate /etc/resolv.conf using this file:
mkdir -p /etc/netns/$NS
echo "nameserver 8.8.4.4" >/etc/netns/$NS/resolv.conf

4. Run your deluged in the $NS and your deluge-web in your default namespace. Point deluge-web to the 10.1.1.2 veth IP address, where deluged will be listening for its connection.

Voila! You've got deluged secured behind the VPN while your deluge-web is freely accessible on your home network

Solution 4

@AndrDevEK's answer is useful. To expand upon that, you may not want to install socat. In which case you can achieve the same thing with a slightly convoluted SSH port-forward setup. In particular the feature of port-forwarding to/from a unix-domain socket is useful here, because unix-domain sockets operate independently of network namespaces:

sudo ip netns exec myvpn su -c "ssh -N -L /tmp/myunixsock:localhost:8112 localhost" $USER &
ssh_pid1=$!
ssh -N -L localhost:8112:/tmp/myunixsock localhost &
ssh_pid2=$!

Cleanup:

sudo kill $ssh_pid1
kill $ssh_pid2
rm /tmp/myunixsock

The first ssh -N -L is started within the myvpn namespace. This creates a unix-domain socket /tmp/myunixsock and listens on it. Incoming connections are forwarded to localhost:8112 (inside the myvpn namespace). The second ssh -N -L is started in the default namespace. This creates a listening TCP port and forwards incoming connections to the unix-domain socket.

It should be noted that in order for this to work, ssh inside your network namespace will need to be working if it is not already (and passwordless pubkey operation is helpful):

sudo ip netns exec myvpn ip link set up dev lo
sudo ip netns exec myvpn /usr/sbin/sshd -o PidFile=/run/sshd-myvpn.pid
ssh-copy-id localhost

View more solutions

Comments

[Hauke Laging]

Disclaimer: I have no experience with veth devices at all (find this very interesting, though... ;-) ). Have you used tcpdump for checking how far the incoming packets get? If tcpdump -i veth0 doesn't show anything then tcpdumo -i lo may be necessary.

[pskiebe]

I added the non-verbose output of tcpdump

[pskiebe]

You just made my day! I never came across socat and it accomplishes exactly what I was trying to do with iptables for quite some time now. I tested several applications and they are all working flawlessly, connecting to the outside world through tun0, while still providing access to their web interface through veth1.

[pskiebe]

After doing some testing, I added the reuseaddr flag. This prevents port already in use errors when starting and stopping socat in rapid succession: socat -4 TCP-LISTEN:8112,reuseaddr,fork TCP:10.200.200.2:8112

[approximatenumber]

working like a charm

[Igor]

Since I am not too experienced with linux administration and it did cost me some time to find out: make sure to escape both : characters inside the single-quotes or else you might run into an error saying ... wrong number of parameters (2 instead of 1) (2 or 3). Otherwise: works great! Huge thank you!

[Felipe]

This should be the accepted answer. It's the best way to deal with forwarding ports rather than creating additional interfaces or dealing with iptables.

[Vincent Olivert Riera]

If systemd complains, try putting that long socat command into a script file, and just call that script file in the ExecStart.

[AnyDev]

@Igor Thanks. I suspect you got quotes wrong. What you describe sounds familiar. For example if you omit inner double quotes you will get similar error as you described. First socat ends up handling colons as parameters to exec which is not the intention. Then your method of escaping '\:' fixes it. The example as I wrote it works for me in bash-4.2 + socat-1.7.3.2.