/etc/hosts file refer to another configuration file

Question

How can I get the /etc/hosts file to refer to another configuration file for it's list of hosts?

Example /etc/hosts:

 ## My Hosts
 127.0.0.1   localhost
 255.255.255.255 broadcasthost

 #Other Configurations
 <Link to /myPath/to/MyConfig/ConfigFile.txt>

 #Other Addresses
 3.3.3.3 MyAwesomeDomain.com
 4.4.4.4 SomeplaceIWantToGoTo.com

ConfigFile.txt

##My additional Hosts
1.1.1.1 SomeLocation.com
2.2.2.2 AnotherLocation.com

How do I add a link/Reference to /etc/hosts file such that ConfigFile.txt will be loaded?

Solution 1

You can't. The format for /etc/hosts is quite simple, and doesn't support including extra files.

There are a couple approaches you could use instead:

• Set up a (possibly local-only) DNS server. Some of these give a lot of flexibility, and you can definitely spread your host files over multiple files, or even machines. Others (such as dnsmasq) offer less (but still sufficient) flexibility, but are easy to set up. If you're trying to include the same list of hosts on a bunch of machines, then DNS is probably the right answer.

• Set up some other name service (NIS, LDAP, etc.). Check the glibc NSS docs for what is supported. Personally, I think you should use DNS in most all cases.

• Make yourself an /etc/hosts.d directory or similar, and write some scripts to concatenate them all together (most trivial: cat /etc/hosts.d/*.conf > /etc/hosts, though you'll probably want a little better to e.g., override the default sort by current locale), and run that script at boot, or from cron, or manually whenever you update the files.

Personally, at both home and work, to have machine names resolvable from every device, I run BIND 9. That does involve a few hours to learn, though.

Solution 2

It's not possible, as the format is usually deeply coded into platforms' libc. However it is imaginable that an OS adds this feature, making it a non cross-platform solution.

Alternatively, you can automatically update a certain block in your hosts file. This is particularly helpful if you have a script that dynamically outputs host entries for a certain project or so (possibly with changing IPs).

Here's an example: you want to create hosts from Terraform state via terraform-inventory.

Relevant inventory output (for instance, mapping an EC2 "Name" tag to groups of exactly one host each):

$ terraform-inventory --list | jq 'with_entries(select(.key | match("^name_")))'
{
  "name_myhost-a": [
    "10.101.118.131"
  ],
  "name_myhost-b": [
    "10.101.111.189"
  ]
}

print-updated-hosts-entries.sh

#!/bin/sh
exec terraform-inventory --list | \
    jq -r 'to_entries |
           map(select(.key | match("^name_"))) |
           map(.value[0] + " " + .key[5:]) |
           join("\n")'

Script output:

./print-updated-hosts-entries.sh
10.101.118.131 myhost-a
10.101.111.189 myhost-b

And the command line to update a marked block in /etc/hosts with the script output:

sudo cp /etc/hosts "/etc/hosts.bak.$(date +%Y%m%d%H%M%S)" && \
    (
        sed -n '1,/^# MYMARKER BEGIN/{/^# MYMARKER BEGIN/!p;}' /etc/hosts; \
        echo "# MYMARKER BEGIN"; \
        ./print-updated-hosts-entries.sh; \
        echo "# MYMARKER END"; \
        sed -n '/^# MYMARKER END/,${/^# MYMARKER END/!p;}' /etc/hosts; \
    ) | \
    sudo tee /etc/hosts.new | \
    sed -n '/^# MYMARKER BEGIN/,/^# MYMARKER END/p' && \
        sudo mv /etc/hosts.new /etc/hosts

Explanation:

• The first line obviously creates a backup
• The subshell in parentheses has two sed calls to print all lines before and after the marker begin/end, respectively. We insert the markers in any case, putting the script output in between those lines. Even if the script fails, we still have to surrounding content of /etc/hosts (and the backup in a catastrophic scenario).
• sudo tee /etc/hosts.new writes the piped content into a new file
• sed -n '/^# MYMARKER BEGIN/,/^# MYMARKER END/p' prints the updated block for convenience
• sudo mv /etc/hosts.new /etc/hosts moves the new file into place. This must be done in a separate step because if the pipe buffer runs out of space, tee /etc/hosts would start writing the file while the existing content is still being read.

Solution 3

I’ve had a similar requirement for me to work seamlessly from different locations with dynamic IPs, especially since I use virtual machines. My simple solution to this is as follows:

1. Create a script to update host entries for each entity. E.g., if I have a host called ‘myhost.net’, with sub-domains ‘app.myhost.net’, I would invoke that script and supply an IP address that will be written for each domain and saved in a host file called ‘my host’. The same script has a flag to delete or update a single host file.

2. Create another script to merge those files into /etc/hosts. This script will find all the host files you’ve created and merge them into the single hosts file at /etc/hosts.

This has worked for me perfectly. I work remotely and sometimes need to move between locations for any reason or another, and would be burdened by the demand to edit that one file so many times. Instead I just run ONE script. Yes, one script. The first script automatically runs the other one to do the merging automatically.

Comments

[DogEatDog]

Thanks! I was wondering why each attempt was failing. It would be so much easier if it were possible, but I'll use the /etc/hosts.d

[Gilles 'SO- stop being evil']

@DogEatDog I recommend running a local DNS server. Dnsmasq is very easy to set up; on user-friendly distributions, you install the package and it just works.

[moodboom]

@Gilles has it right - dnsmasq likely does exactly what you want on a home network. It lets you assign static IPs to all your LAN resources by MAC address, grant them hostnames, and resolve requests for them - all while letting unknown hosts pass through to an upstream DNS server.

[derobert]

@moodboom & Gilles Thank you for suggesting that, I've noted it in the answer. Not sure if I was using dnsmasq back in 2013, but I certainly am now.

[l0b0]

Glob expansions are sorted (at least in Bash, according to LC_COLLATE), so the caveat in your answer does not apply.

[derobert]

@l0b0 that's a good point, though you probably don't want to sort by LC_COLLATE, at least not without setting it to C or POSIX. Having your config file suddenly in a different order because you changed the system locale would be surprising. Edited the answer to incorporate.

[l0b0]

@derobert The point is that's what happens, not whether you'd want that or not.

[derobert]

@l0b0 did you look at my edit to the answer? I think that edit makes it clear that * will sort the files in locale order (what happens, both in bash and per POSIX).