Route IP Traffic Based on Process to Different Default Routes / Interfaces

Question

I'm trying to determine if it is possible to selectively route IP packets from a process or process group through a specific interface while all other packets are routed through another interface. That is, I want all traffic from /usr/bin/testapp to be routed through eth1 while all other packets go through eth0. Packets in this case can be TCP, UDP, ICMP, etc. and can be configured by the end users to use various port(s).

Because I'm not able to easily force the process in question to bind to a specific interface, I am trying to achieve the same outcome via routing. Is this possible?

--- edit ---

Through a helpful suggestion here, and in many other places, is to mark packets based on UID; that is not really the goal. The goal is to mark/filter/route based on process regardless of user. That is to say, if alice, bob and charlie all run their own instance of /usr/bin/testapp; all packets from all three instances should go through eth1 while all other packets from the system should go through eth0.

Note that marking by source/destination port, user name/UID, etc. is not sufficient since various users may run testapp and they may setup different ports in their own ~/.config/testapp.conf or whatever. The question is about filtering by process.

One option that is available, though I do not know how helpful it is, is to use a /bin/(ba|z)?sh-based wrapper around the native binary.

--- edit ---

I'm referring to routing on a system running a modern Linux kernel, say 4.0 or better. If there are software dependencies beyond iproute2, nftables, conntrack and similar tools, I'm willing to explore open source solutions, though basic tools are preferable.

Solution

Your routing the packets through either eth1 or eth0. tables mangle should address this. To do so, I had to mark packets and set up rules for handling it. First, add a rule that make the kernel route packets marked with 2 through table

ip rule add fwmark 2 table 3

Add a route for redirecting traffic over a different interface, assuming the gateway being 10.0.0.1:

ip route add default via 10.0.0.1 table 3

Flush your routing Cache.

ip route flush cache

Now, set a firewall rule for marking designated packets:

iptables -t mangle -A OUTPUT -p tcp --dport 465 -j MARK --set-mark 2

Finally, relax the reverse path source validation. Some suggest you to set it to 0, but 2 seems a better choice according to https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt. If you skip this, you will receive packets (this can be confirmed using tcpdump -i tap0 -n), but packets do not get accepted. The command to change the setting so packets get accepted:

sysctl -w net.ipv4.conf.tap0.rp_filter=2

Reference : http://serverfault.com/questions/345111/iptables-target-to-route-packet-to-specific-interface

Comments

[Massimo]

This might be able to help: stackoverflow.com/questions/4314163/….

[dmourati]

Can you control the source/destination ports and/or IP? If so, you can easily do this with iproute2 and fwmark. Also, consider running testapp inside a vm/container and doing it that way.

[Nehal Dattani]

Users will launch other processes (that binds to network) as well or only testapp ?

[justinzane]

Thanks, however, unless I'm not understanding, you example seems to only manage packets with a dest. port of 465. While that might work for something like an MTA, it would not be very helpful for anything that used an a priori undefined set of destination ports.

[CIA]

@justinzane Couldn't you just change from port 465 to whatever port your processes are using? Otherwise, there's no way to differentiate down to the process level, because network routing generally happens at the network layer, not the application layer.