How do I force ssh to use a second interface with higher metric?

Question

I have a Crunchbang VM with two interfaces, eth0 and eth1, each of which connects to an OpenWRT VM (eth0 being 10.232.64.20 and eth1 being 10.232.65.20). I'm using Network Manager and DHCP. My overall goal is having multiple ssh connections, and bonding them with ifenslave.

By default, eth1 (for some reason) is the default gateway:

user@crunchbang:~$ ip ro
default via 10.232.65.1 dev eth1  proto static
10.232.64.0/24 dev eth0  proto kernel  scope link  src 10.232.64.20
10.232.65.0/24 dev eth1  proto kernel  scope link  src 10.232.65.20

I added a route for eth0:

user@crunchbang:~$ sudo ip route add default via 10.232.64.1 dev eth0  proto static metric 1

Then I have two routes:

user@crunchbang:~$ ip ro
default via 10.232.65.1 dev eth1  proto static
default via 10.232.64.1 dev eth0  proto static  metric 1
10.232.64.0/24 dev eth0  proto kernel  scope link  src 10.232.64.20
10.232.65.0/24 dev eth1  proto kernel  scope link  src 10.232.65.20

However, ssh only gets out via eth1:

user@crunchbang:~$ ssh -b 10.232.64.20 [email protected]
ssh: connect to host 1.2.3.4 port 22: Connection timed out

user@crunchbang:~$ ssh -b 10.232.65.20 [email protected]
Enter passphrase for key '/home/user/.ssh/id_rsa': 

After changing the eth0 metric I have:

user@crunchbang:~$ ip ro
default via 10.232.64.1 dev eth0  proto static  metric 1
default via 10.232.65.1 dev eth1  proto static  metric 2
10.232.64.0/24 dev eth0  proto kernel  scope link  src 10.232.64.20
10.232.65.0/24 dev eth1  proto kernel  scope link  src 10.232.65.20

And now ssh only gets out via eth0:

user@crunchbang:~$ ssh -b 10.232.64.20 [email protected]
Enter passphrase for key '/home/user/.ssh/id_rsa': 

user@crunchbang:~$ ssh -b 10.232.65.20 [email protected]
ssh: connect to host 1.2.3.4 port 22: Connection timed out

How do I force ssh to use an interface with a higher metric?

Edit

I have implemented and tested the configuration in the 4.2. Routing for multiple uplinks/providers section of the Linux Advanced Routing & Traffic Control HOWTO. Given that the configuration is simple, and that I didn't encounter errors, I'll just show code and results, with minimal explanation.

root@crunchbang:~# ip route add 10.232.64.0/24 dev eth0 src 10.232.64.20 table T0
root@crunchbang:~# ip route add default via 10.232.64.1 table T0
root@crunchbang:~# ip route add 10.232.65.0/24 dev eth1 src 10.232.65.20 table T1
root@crunchbang:~# ip route add default via 10.232.65.1 table T1
root@crunchbang:~# ip route flush table main
root@crunchbang:~# ip route add 10.232.64.0/24 dev eth0 src 10.232.64.20
root@crunchbang:~# ip route add 10.232.65.0/24 dev eth1 src 10.232.65.20
root@crunchbang:~# ip rule add from 10.232.64.20 table T0
root@crunchbang:~# ip rule add from 10.232.65.20 table T1
root@crunchbang:~# ip route add default scope global nexthop via 10.232.64.1 dev eth0 weight 1 nexthop via 10.232.65.1 dev eth1 weight 1

Here are the routing tables generated:

root@crunchbang:~# ip route show table T0
default via 10.232.64.1 dev eth0 
10.232.64.0/24 dev eth0  scope link  src 10.232.64.20 

root@crunchbang:~# ip route show table T1
default via 10.232.65.1 dev eth1 
10.232.65.0/24 dev eth1  scope link  src 10.232.65.20 

root@crunchbang:~# ip ro
default 
    nexthop via 10.232.64.1  dev eth0 weight 1
    nexthop via 10.232.65.1  dev eth1 weight 1
10.232.64.0/24 dev eth0  scope link  src 10.232.64.20 
10.232.65.0/24 dev eth1  scope link  src 10.232.65.20 

With that configuration, ssh connects via both interfaces:

user@crunchbang:~$ ssh -b 10.232.64.20 [email protected]
Enter passphrase for key '/home/user/.ssh/id_rsa': 

user@crunchbang:~$ ssh -b 10.232.65.20 [email protected]
Enter passphrase for key '/home/user/.ssh/id_rsa': 

However, it does appear that I need to lose Network Manager. If anyone could explain why that's a bad idea, or warn of pitfalls, I would appreciate it.

Edit2

Removing Network Manager went well. I have just one last question. What is the current standard way to load the configuration at boot?

Solution

First, your solution for the problem is good. Second, it depends on the OS. Crunchbag is debian based so this solutions could do the job:

https://serverfault.com/questions/487939/permanently-adding-source-policy-routing-rules

On RHEL based systems there is also the possibility to add <ifname>-rule and <ifname>-route.

Comments

[mirimir]

I just discovered 4.2. Routing for multiple uplinks/providers. I'll update the question pending implementation.

[mirimir]

It worked, so I'll update my question.