Assign multiple IPs to 1 Entry in hosts file

Question

I have a web sever that connects to an internal database through a VPN. There are 2 IPs (primary and secondary) to the database from the web server. How can I setup my /etc/hosts file so that if the primary IP is not available then the secondary IP will be used?

Would this work for my hosts file?

141.131.286.1   abc.efg.datastore.com   #primary

141.131.286.237 abc.efg.datastore.com   #secondary

Solution 1

The hosts file does not provide such mechanism. If you list two IPs for the same name, only the first one will be used. So, there is no such thing as primary and secondary IPs.

Also, the hosts file does not handle URLs. It just handles names like the ones provided in the question. A URL contains complete path and protocol such as http://host/path/to/resource.

Solution 2

You can't provide resilience or round robin load balancing via the /etc/hosts file - it is not designed for that purpose.

Instead, your options are ... (in no particular order)

1. Configure your network properly, so that routes change when a link is dropped
2. Use DNS round-robin load balancing (not A Good Idea ^TM) using a managed service (eg. loaddns.com or dnsmadeeasy.com etc.)
3. Use a local L3 load balancer for the outbound traffic (HAProxy?) with the back-ends defined as necessary
4. Build the resilience into your web application itself

Solution 3

/etc/hosts doesn't support round robin but you can write a simple bash script to sed replace an entry tagged with a #RoundRobin comment (or any other tag you wish to use, just reflect it in the grep line in the script).

#!/bin/bash
fqdnips=( $(nslookup sub.domain.com|grep Address:|awk -F\  '{ print $2 }'|grep -v '#') )

new=`printf "${fqdnips[@]}"`
old=`grep "#RoundRobin" /etc/hosts|awk -F\  '{ print $1 }'`
sed -i "s/$old/$new/g" /etc/hosts

The above script grabs the output of nslookup for sub.domain.com and stores it in an array. It then prints the top most value to $new and grabs the existing value for tag #RoundRobin assigned in /etc/hosts ... lastly, it performs a sed replace

/etc/hosts file entry would look like this

127.0.0.1        localhost
::1              localhost
11.12.13.14      sub.domain.com      #RoundRobin

Lastly, place this script in the root's crontab to run every hour or so and you'll now have an /etc/host round-robin.

This is particularly useful if you have a coded page that is pulling some data from an API and the DNS lookup for the API server is causing a lot of hang time in the page's script execution... resulting in high cpu consumption for what would otherwise appear to be a simple page. To avoid the costly DNS lookup (particularly if your site is doing hundreds of them per minute do to heavy traffic), you should use /etc/hosts to resolve the FQDN of the remote API server. This will dramatically reduce the CPU usage for pulling the API data and generating the page.

Solution 4

Easy to setup, please follow the instruction:

1. install dnsmasq
2. edit /etc/resolv.conf and set "nameserver 127.0.0.1" as a first DNS
3. add normal DNS as alternative (google one for example) "nameserver 8.8.8.8" as a second line
4. make sure two required records are in your /etc/hosts file
5. now check with a command host abc.efg.datastore.com
   
   that should respond two records with RR-DNS so if one node from the list is down - your application will be connected to another one

Solution 5

Note that at least on macOS, in contradiction to what the other answers say, the system resolver will return all entries associated with a host name in /etc/hosts instead of stopping at the first.

View more solutions

Comments

[dmourati]

linuxmisc.com/4-linux/391d5fa7270c26bc.htm

[Massimo]

As an alternative and corret solution, install a local dns server. For example the easy-peasy to configure dnsmasq thekelleys.org.uk/dnsmasq/doc.html

[Antti Rytsölä]

DNS round-robin load balancing isn't usually resilient. One is selected and others are not tried.

[Antti Rytsölä]

Another option could be to use netcat or another software to forward the connection to an IP. Then change the forward if one IP is lost.

[Ben Lessani]

@anttiR DNS RR alone has no resilience, but used via a managed DNS service provider it does. I've edited my answers and given some examples to be clearer.

[Antti Rytsölä]

I doubt it would work great with database. They have the tendency to fetch one IP and stick to it. An internet website on the other hand would work great.

[Ben Lessani]

That would depend on the resolver of the host machine. If the DNS resolver is set to be a non-caching service - or polls the DNS registrars DB directly, then it would work. But like I said, its not a good idea, its just an idea.

[dmourati]

Edited title and naming in OPs question to eliminate the confusion about URLs/hosts.

[dmourati]

I think I blew up your network point because there are none in his real question. /etc/hosts maps IPs to hostnames, not URLs or "network paths"

[Jeter-work]

IP stack automatically checks hosts file before consulting DNS. I believe it is a part of the IP definition, but I know that Windows and Linux both follow this pattern. He does not need dnsmasq, and if he changes his /etc/resolv.conf in this way, he will lose connection to his real DNS server.

[Jeter-work]

What I see tells me that he's got one database server with two addresses. No round robin needed. It looks like the single server is setup for failover (why put two IP addresses on the database otherwise?). So, two DNS entries weighted differently provides failover, no? BTW, I suspect that his DNS admin already has done this. Though one of those addresses is a .1 which worries me.

[Jeter-work]

No, it won't work. Second occurrence of a value in /etc/hosts would never be used. And if he renames the second IP address with a different host name, now he has to do something in his app to do load balancing. That's going to be expensive and a kludge, when DNS or DNS + Load Balancer is the right answer.

[Jeter-work]

Wouldn't dns caching alleviate your case of hundreds of lookups per minute?

[bortran]

Browsers will be able to handle this, if 10 isn't available it will fallback on 20. It surely depends on the app and there should also be the point where you check for availability - if you want to get around setting up something more complicated and expensive with load balancing.

[Ihor Kolodyuk]

not true, he can put multiple lines to resolv.conf so next line can be "nameserver 8.8.8.8", for example, + nscd for caching existing records, this approach exactly solve problem that was described

[Ihor Kolodyuk]

That is exactly I told to him, dnsmasq is acting as a nameserver proxy. This approach works.

[Satalink]

I don't control the DNS server, is that something I can set up on the CentOS web server?

[Satalink]

ref: stackoverflow.com/questions/11020027/dns-caching-in-linux

[Matt Nordhoff]

A records don't have a priority field. Some authoritative DNS services have record weighting, but that's done by probabilistically including or excluding the records.

[Clon]

I have tried Ihor's solution and the round robin works for records with the same name in the /etc/host file. In Ubuntu 18.04.3 "bionic" I had to disable systemctl-resolved and enable dnsmasq, and added my previous DNS servers to /etc/resolv.conf, and also the line "dns=dnsmasq" to /etc/NetworkManager/NetworkManager.conf, and restarted Networkmanager service. Round robin works great, so it's the best answer for the second question- Would this work for my hosts file?. +1 point and thanks for sharing.

[Clon]

@Xalorous: Ihor is right- round robin works well with dnsmasq. It does not with, for example, systemd-resolvd, which is the default service for the latest Ubuntu versions.

[dave_thompson_085]

This is not true, at least not blanketly. If the hosts file is used and has multiple entries, every resolver I have seen returns all of them to the program, just as it does when a DNS server returns multiple A or AAAA records. What the program does varies -- some use only the first entry, some do more complicated things that might or might not match what the user would like.

[dave_thompson_085]

I don't think the standards address a hosts file at all, but all the systems I have looked at -- Linux (with glibc), FreeBSD, Solaris and Windows -- do this. Whether and how a program calling the resolver uses multiple addresses is up to that program and varies wildly.

[zneak]

@dave_thompson_085, per other answers, that at least seems to be unusual, unless every platform changed how they handle this in the last 8 years and we’re just the first people to notice.

[dave_thompson_085]

I doubt it's changed; I know I used this 'feature' (for testing, at a prior job) by around 2012 at the latest. And gethostbyname was designed to return multiple addresses from its creation in the 1980s. I think people were misled by the fact that many programs use only the first result from the resolver -- either from hosts file or from DNS. (But DNS servers can easily vary the order on each request, and some do, so that the selected first entry changes, whereas doing that with hosts file is very clumsy.)

[Massimo]

I tried on one server, I confirm you: it doesn't work. Yours is a specific case, not valid as a generic answer, if you can't say how you solved it.

[user1133275]

@Massimo please provide a specific counter example if you are having trouble making this work.

[Remember Monica]

I verified that glibc on a 2021 debian system only supports one ip per host in /etc/hosts, and looking at changelogs and documentation, it is unlikely that this ever was different. What is the source for the claim that this ever was supported? The fact that gethostbyname can return multiple IPs has no relöevance on whether /etc/hosts supports this.

[Remember Monica]

dig does not use the hosts file at all - either those entries come from the dns server or this answetr was made up

[user1133275]

@RememberMonica The man page said "Unless it is told to query a specific name server, dig will try each of the servers listed in /etc/" . I just re-tested this and it works. Can you show a counter test/example?

[Remember Monica]

It says "/etc/resolv.conf" not "/etc". Which entry in /etc/resolv.conf do you think corresponds to your hosts file?

[user1133275]

@RememberMonica I don't know the internals but I assume it's some systemd thing that is going on in Ubuntu 20.04. Just try it yourself already it only takes 2 seconds.

[Gareth]

The systemd-resolverd stub resolver on my Ubuntu 20 system appears to support returning more than one IP per host. Running dig <host-with-2-ips> shows both IPs.