route traffic on certain port through certain interface?

Question

I am currently running a system in which I need to be able to send and recieve HTTP and HTTPS traffic directly, however all other traffic must go via a VPN.

My setup is :

x2 interfaces : eth0 (local network, behind a router) and tun0 (openvpn client)

Currently all traffic is routed through my vpn, i was wondering if it were possible to not route http and https traffic (80, 443) through the VPN.

Here is the routing table when the system and openvpn client have started :

0.0.0.0/1 via 10.8.13.5 dev tun0 
default via 192.168.1.1 dev eth0 
10.8.13.1 via 10.8.13.5 dev tun0 
10.8.13.5 dev tun0 proto kernel scope link src 10.8.13.6 
128.0.0.0/1 via 10.8.13.5 dev tun0 
176.67.168.144 via 192.168.1.1 dev eth0 
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.56 
192.168.1.1 dev eth0 scope link 

I have read through other similar questions however either the answers don't reroute the traffic or just block it.

Thanks in advance :)

Solution

Routing is at IP layer 3. TCP is at layer 4, so routing alone isn't enough to deal with this.

In short: the interesting traffic has to be tagged with iptables, and tagged packets selected with ip rule's fwmark to use a separate routing table. Then two more fixes have to be applied for the locally initiated/receiving traffic case, which is more difficult than the routed case. All the settings are of course done on the local system.

Routing table 80 (a matching symbolic name can be added in /etc/iproute2/rt_tables but that's not mandatory) and mark 0x80 were "arbitrarily" chosen.

ip route add table 80 192.168.1.0/24 dev eth0 scope link src 192.168.1.56
ip route add table 80 default dev eth0 via 192.168.1.1

Using -I to ensure that the iptables rules aren't appended too late. You should check with your current rules how to reorder that if needed:

iptables -t mangle -N markports
iptables -t mangle -I PREROUTING 1 -j CONNMARK --restore-mark
iptables -t mangle -I OUTPUT 1 -m mark --mark 0 -j markports
iptables -t mangle -I OUTPUT 2 -j CONNMARK --save-mark
iptables -t mangle -A markports -p tcp --dport 80 -j MARK --set-mark 0x80
iptables -t mangle -A markports -p tcp --dport 443 -j MARK --set-mark 0x80

ip rule add fwmark 0x80 lookup 80

This blog: Netfilter Connmark » To Linux and beyond ! gives good informations on CONNMARK.

This should have been working, but actually the wrong default outgoing IP will have been selected at the first routing decision, because the route was about to be through tun0. At the reroute check made because of the mangle/OUTPUT's mark (see this Packet flow in Netfilter and General Networking schematic for clarification), this IP won't change. If the traffic handled was routed instead of being locally initiated, this problem wouldn't happen (using a separate net namespace to ensure this is a solution for services, probably not for a Desktop). So this requires also a layer of MASQUERADE (or SNAT for more complex cases) on top of it:

iptables -t nat -I POSTROUTING 1 -m mark --mark 0x80 -j MASQUERADE

Now that the outgoing source IP is correct, it's still not working: the reverse path filter triggers in the return path for about the same reason: the routing decision made before PREROUTING doesn't know the fwmark for now (despite the previous schematic placing mangle/PREROUTING before the routing decision, that's apparently not the case), thus considers the return traffic packets to be spoofed and drops them early. The eth0 interface's rp_filter has to be put in loose mode for this to be allowed. This might have some (very minor behind NAT) security issues but I found it unavoidable for this non-routed case:

echo 2 > /proc/sys/net/ipv4/conf/eth0/rp_filter

You'll have to find how to set it permanently (eg echo net.ipv4.conf.eth0.rp_filter=2 > /etc/sysctl.d/90-local-loose-mode.conf if nothing else alters it later).

Tested ok using namespaces with similar settings to OP's.

NOTE: DNS requests will still go through the tunnel. Some geolocalized web services might not work as expected.

Comments

[A.B]

newer kernels ( >= 4.17) can have ip rules with tcp or udp selectors (policy routing has now a leg in layer 4). So this could be done without any use of iptables now, but I'm not sure that would give a better result (except maybe with regard to MTU/MSS)