Keepalived router on virtual ip is not reachable

Question

I've setup keepalived to manage a virtual ip between two hosts.

My setup is the following

Server #1: Hostname folmer, ip 192.168.0.1/22 dev p2p1
Server #2: Hostname flemming, ip 192.168.0.2/22 dev p2p1
VIP: 192.168.0.3/22

Keepalived is working and the VIP is switched between the servers when one goes down.

Problem: On the local network I can ping 192.168.0.3, but when I set my default route to 192.168.0.3 instead of 192.168.0.1, I can no longer reach the internet through server #1.

Keepalived conf on server #1:

global_defs {
    notification_email {
        [SNIP]
        [SNIP]
    }
    notification_email_from [SNIP]
    smtp_server 127.0.0.1
    smtp_connect_timeout 30
    router_id folmer
}
vrrp_instance VI_1 {
    state MASTER
    interface p2p1
    virtual_router_id 52
    priority 150
    advert_int 1
    garp_master_delay 2
    smtp_alert
    authentication {
        auth_type PASS
        auth_pass [SNIP]
    }
    virtual_ipaddress {
        192.168.0.3 label p2p1:0
    }
}

server #1 ip's:

p2p1      Link encap:Ethernet  HWaddr 00:0a:f7:40:d7:5f  
          inet addr:192.168.0.1  Bcast:192.168.3.255  Mask:255.255.252.0
          inet6 addr: fe80::20a:f7ff:fe40:d75f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:11446972 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11382043 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:5461610409 (5.4 GB)  TX bytes:9274459351 (9.2 GB)
          Interrupt:16 

p2p1:0    Link encap:Ethernet  HWaddr 00:0a:f7:40:d7:5f  
          inet addr:192.168.0.3  Bcast:0.0.0.0  Mask:255.255.255.255
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:16 

Edit

Still not wokring as needed. Routing is configured and it is working, as long as the clients use the IP 192.168.0.1 as the gateway instead of 192.168.0.1. Obviously I want it to work with 192.168.0.3.

Solution

I guess that

ip addr show | grep global

will show that your virtual address is

192.168.0.3/32

/32 is usually not the desired result, therefore you should add e.g. /24:

virtual_ipaddress {
  192.168.0.3/24 label p2p1:0
}

Comments

[c4f4t0r]

but can you ping 192.168.0.3 and where changed the default router? maybe in 3 pc?

[Daniel]

Yes I can ping 192.168.0.3 from a separate computer on the same network. But I can not access the internet through 192.168.0.3 as gateway.

[c4f4t0r]

have you enabled ip forward and iptables masquerade in the two nodes?

[Daniel]

Yes, and it works through 192.168.0.1 instead of .3

[c4f4t0r]

echo 1 > /proc/sys/net/ipv4/ip_forward ; iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o p2p1:0 -j MASQUERADE or try iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o p2p1 -j MASQUERADE

[Daniel]

It is not a problem with my iptables setup and ip_forward is enabled. Traffic on aliased devices, p2p1:0 e.g. match the p2p1 device in iptables. And SNAT'ing happens as the traffic leave the WAN interface (em1 in my case) and I have -t nat -A POSTROUTING -o em1 -j SNAT --to <my-ip>. All traffic from the p2p1 devices going out on em1 is allowed through -A FORWARD -i p2p1 -o em1 -j ACCEPT - and a matching rule for -i p2p1:0 does not change anything, I have tried.

[David Houde]

I assume it doesn't work when failed over as well?

[Daniel]

No, the issue appers to be the same on the duplicated system.

[Daniel]

Good catch, here, 4 years later. I guess you are correct, but I cannot try it out since the system is no longer in use. The line inet addr:192.168.0.3 Bcast:0.0.0.0 Mask:255.255.255.255 confirms your assumption.