rhel 6 default gateway not working

Question

I have six similar RHEL 6 servers in the lab and the settings are same in all the servers. The settings are,

static IP address: 129.XXX.XXX.XX
Subnet Mask: 255.255.254.0
Default Gateway: 129.XXX.XXX.1

My default gateway has suddenly stopped working in one of the servers (server1) and am investigating what caused this issue.

Now, if I do route -n in server1, I get the output as,

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
129.XXX.XXX.0   0.0.0.0         255.255.254.0   U     1      0        0 eth0

I suspected it could be firewall issue but I checked with the school network and the firewall is configured correctly. My /etc/hosts file is as below.

127.0.0.1   server1.xxx.edu        localhost

My /etc/resolv.conf is as below.

# Generated by NetworkManager
search xxx.edu. xxx.edu
nameserver 129.xxx.xx.xx

In server2 which belongs to the same network as server1, route -n gives the output as below.

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
129.xxx.xxx.0   0.0.0.0         255.255.254.0   U     1      0        0 eth2
0.0.0.0         129.xxx.xxx.1   0.0.0.0         UG    0      0        0 eth2

I am trying to figure out if am missing something else.

EDIT

The contents of /etc/sysconfig/network on server1,

NETWORKING=yes
HOSTNAME=server1.xxx.edu
GATEWAY=255.255.254.0

Edit 2:

After I changed the default gateway, if I execute route -n command the output is as below.

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
129.xxx.xxx.0   0.0.0.0         255.255.254.0   U     0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1002   0        0 eth0

I also tried removing the default gateway entry in the /etc/sysconfig/networks file as well. But I get the same output for route -n as above.

cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE="eth0"
NM_CONTROLLED="yes"
ONBOOT=yes
TYPE=Ethernet
BOOTPROTO=none
IPADDR=129.xxx.xxx.xx
PREFIX=23
GATEWAY=129.xxx.xxx.1
DNS1=129.xxx.xx.xx
DOMAIN=xxx.edu.
DEFROUTE=no
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME="System eth0"
LAST_CONNECT=1406927310

Solution

It looks like you have an invalid GATEWAY value in /etc/sysconfig/network for server1. So, change it to your gateway ip address:

GATEWAY=129.XXX.XXX.1

Save file, restart networking service to make the change.

Comments

[cuonglm]

On server1, what is content of /etc/sysconfig/network?

[Ramesh]

@Gnouc, please see the edit.

[Ramesh]

thanks. As soon as posted the edit I tried out that option. But unfortunately it seems to not work.

[cuonglm]

@Ramesh: Did you restart networking service?

[Ramesh]

no luck. I tried rebooting the machine as well.

[cuonglm]

How route table look like after you restart? And did you check config file of your network card? /etc/sysconfig/network-scripts/ifcfg-*

[Ramesh]

thanks for providing more info. I have updated my question with more details.

[cuonglm]

@Ramesh: What happen if you try adding gateway manually?

[Ramesh]

it works. However, if I restart the network services, it disappears. I already tried to make this setting permenant in /etc/sysconfig/network but unfortunately it is messing up.

[cuonglm]

Strange. Do you have any other network script for other network card that set default gateway?

[cuonglm]

@Ramesh: And try add this GATEWAYDEV=eth0 in /etc/sysconfig/network.