Linux Networking routing to virtual ip addresses from a different subnet

Question

New to linux networking, had a question about routing to virtual ip addresses from a different subnet.

I have three servers (they can all ping each other):

A 10.10.51.182 (connected to router Y)
B 10.10.51.183 (connected to router Y)
C 10.10.60.10 (connected to router Z)

On server A, I created 1000 virtual interfaces:

# ifconfig eth0:0 192.168.1.1 netmask 255.255.0.0  
# ifconfig eth0:1 192.168.1.2 netmask 255.255.0.0  
....  
# ifconfig eth0:999 192.168.5.200 netmask 255.255.0.0  

Each virtual interface is simulating a camera device, with camera simulation software.

On server B, I added the following route:

# route add -net 192.168.0.0 netmask 255.255.0.0 eth0

And voila, B can access/ping the virtual interfaces of machine A.

Server C that is on a different subnet (10.10.60.0), added this route, but even I had super low hope for it to work:

# route add -net 192.168.0.0 netmask 255.255.0.0 eth0

As expected, I wasn't able to reach the virtual interfaces of server A, so I removed the route and added the following route using gateway ip of router Y:

# route add -net 192.168.0.0 netmask 255.255.0.0 gw 10.10.51.1 eth0
SIOCADDRT: Network is unreachable

The gateway 10.10.51.1 is pingable by server C, however I think the error message implies that the gw is not directly on the same subnet (10.10.60).

What configuration/route can be configured on server A and/or C for C to ping/access the virtual interfaces of server A. I don't have access to change the configuration of the routers.

Though, if it's absolutely not possible using changing just the configuration of the servers, could someone please explain in networking terms why B can access virtual interfaces of A, but C cannot access them. Does the router Y see packets bound to '192.168.0.0' from C and automatically drop them, whereas when packets originating from the same subnet (B) get allowed?

Solution 1

Simple solution create a tunnel between the two servers, e.g:

On server A:

ip tunnel add tunnel mode ipip remote 10.10.60.10
ip addr add 10.1.1.1/24 dev tunnel
sysctl -w net.ipv4.ip_forward=1

The last command is to forward packets from your newly created tunnel device to your virtual ethernet devices.

On Server C

ip tunnel add tunnel mode ipip remote 10.10.51.182
ip addr add 10.1.1.2/24 dev tunnel
ip route add 192.168.0.0/16 via 10.1.1.1

Depending on your firewalls between the servers you may have to adjust some rules.

Explenation: Server A and Server B are on a shared network segment, e.g. they can send packets to each other without the need to send the packets to their gateway. This means Server B just tries directly to resolve the address 192.168.1.1 via ARP and Server A replies to them.

Server A and Server C are on different network segments, e.g. if Server C just asks for 192.168.1.1 (this would be your route command for Server C) it will receive no answer. To solve this problem you generally can specify how you can reach a specific system via routing tables but you can only specify the next hop. As router Z seems to not know about 192.168.0.0/24 you have to create a tunnel between the two systems.

One small additional hint, you don't need to create virtual ethernet devices, you can add an arbitrary number of ip addresses to one network device, e.g:

for first in {1..4} ; do
   for second in {1..255} ; do
     ip addr add 192.168.$first.$second/16 dev eth0
    done
done

Solution 2

You have a few options:

• Computer C needs a link to the same physical net as A and B are on.
• Router Y needs to have an address in 192.168.0.0/16 and router Z needs a route for 192.168.0.0/16 via router Y.
• Create an ip-ip (or VPN) tunnel between computer C and computer A; route traffic for 192.168.0.0/16 via the tunnel interface.

Creating the tunnel is probably your best bet for modifying only computers A and C. See the IP-IP Howto for steps.

Comments

[Saad Malik]

Thanks, this was exactly what I needed. Following your instructions, almost verbatim, and I'm able to connect to all my virtual cameras.

[Saad Malik]

The only addition to make it all work: ip link set dev tunnel up

[Saad Malik]

I've been reading a ton about ipip, gre, and sit; some very interesting protocols. Out of curiosity, if I had 20 servers that need to connect to server A using tunneling, would I be required to create 20 separate tunnels on A or can I specify 'remote any' for the tunnel configuration on server A. Before I get down vote, I already tried and failed, but maybe there is a way or is this not possible?

[sunnysideup]

@SimFox3 You should starting talking to your network people so they can propagate the necessary routes to your routers. But yes you need 20 different tunnels (each one with a different IP on A) or as an alternative if the 20 servers are on the same network segment create one tunnel and use this server as an router for the 192.168.0.0/16 network.

[cocobear]

@UlrichDangel If I have 20 servers on the same network segment. I create one tunnel on server C, and I can connect 192.168.x.x on server C. But how can I connect 192.168.x.x on another servers. I already add route on other servers, using ip route add 192.168.0.0/16 via 10.10.60.10