Server not accessible on eth1 (additional network interface) CentOS 7 on AWS EC2

Question

I have created a CentOS 7 instance on EC2, installed my required application-ware, assigned 1 Elastic IP to default network interface (eth0) and it is all accessible.

Now I created another Network Interface, assigned an Elastic IP to that and then attach the ENI afterwards to the instance. Now, the instance isn't reachable on the eth1 (new additional network interface).

I have tried various approaches found on google creating ifcfg-eth1 in /etc/sysconfig/network-scripts but with no luck.

For help, this is the output from ifconfig -a (IPs masked):

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9001
        inet 1XX.3XX.2XX.4XX  netmask 255.255.240.0  broadcast 1XX.3XX.2XX.4XX
        inet6 fe80::2e:a1ff:fe01:c763  prefixlen 64  scopeid 0x20<link>
        ether 02:2e:a1:01:c7:63  txqueuelen 1000  (Ethernet)
        RX packets 219  bytes 28188 (27.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 284  bytes 31055 (30.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        ether 02:78:73:34:66:35  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 23  bytes 4968 (4.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 23  bytes 4968 (4.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

And this is from ip addr:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:2e:a1:01:c7:63 brd ff:ff:ff:ff:ff:ff
    inet 1XX.3XX.2XX.4XX/20 brd 1XX.3XX.2XX.4XX scope global dynamic eth0
       valid_lft 3404sec preferred_lft 3404sec
    inet6 fe80::2e:a1ff:fe01:c763/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 02:78:73:34:66:35 brd ff:ff:ff:ff:ff:ff

My /etc/sysconfig/network-scripts/ifcfg-eth0 file has these contents:

DEVICE="eth0"
BOOTPROTO="dhcp"
ONBOOT="yes"
TYPE="Ethernet"
USERCTL="yes"
PEERDNS="yes"
IPV6INIT="no"
PERSISTENT_DHCLIENT="1"

Solution

Finally, no one but this guy at: https://www.internetstaff.com/multiple-ec2-network-interfaces-on-red-hat-centos-7/ helped me.

For preserving information if the link becomes invalid in future, here is the fruitful content from the post:

1. Force your default gateway to be eth0:

   Edit /etc/sysconfig/network and add:

   GATEWAYDEV=eth0
   

   Not doing this left the default gateway of the main routing table set to the last interface to be configured, which caused some strange behavior.

2. Configure each additional interface you've added:

   In /etc/sysconfig/network-scripts, create an ifcfg-ethX for each new interface.

   Modify:

   1. The DEVICE name to match the ENI.

      DEVICE="eth1"
      BOOTPROTO="dhcp"
      ONBOOT="yes"
      TYPE="Ethernet"
      USERCTL="yes"
      PEERDNS="yes"
      IPV6INIT="no"
      PERSISTENT_DHCLIENT="1"
      

3. Add a custom route for each additional interface.

   Again in /etc/sysconfig/network-scripts, create a route-ethX file for each interface.

   Modify:

   1. The device name.
   2. Increment the table number.
   3. The gateway to your VPC subnet's gateway.

   4. Change the source IP to the assigned internal network address of the ENI.

      default via 10.0.0.1 dev eth0 table 1
      10.0.0.0/24 dev eth0 src 10.0.0.10 table 1
      

4. Also in /etc/sysconfig/network-scripts, create a rule-ethX for each interface.

   Modify:

   1. Increment the table number to match route-ethX.

   2. Change the IP to the assigned internal network address of the ENI.

      from 10.0.0.10/32 table 1
      

Restart the network service and you should be up and running. You can confirm with ip rule:

# ip rule
0:  from all lookup local 
32764:  from 10.0.0.10 lookup 3 
32765:  from 10.0.0.11 lookup 2 
32766:  from all lookup main 
32767:  from all lookup default 

Note that Amazon suggested a custom route and rule for eth0, but we found allowing eth0 to use the default main routing table not only worked but was more flexible.

Comments

[Admin]

I'm not an expert in this area, but it looks to me like you don't have IP addresses assigned to eth1, and you probably need to set up routing properly. Asymmetric routing is a possible problem you may have once you get the basics going. Have you read widely on this issue, such as Amazon documentation? I found an interesting article which might be of assistance : blog.bluemalkin.net/multiple-ips-and-enis-on-ec2-in-a-vpc

[mangolier]

Thanks, @VPZ this helped me a lot. However I also had to delete NetworkManager package to make it work properly. I'm using CentOS 7