script / command to get IPs from list of hostnames and combine into a single file

Question

This is beyond my current skills it seems as I've been trying for a while and not making much headway.

I've been asked to get a list of hosts and IPs for security to run a scan against those servers. There is a hostlist named hosts.linux on the server with all the hostnames, just no IPs. I'm trying to come up with a script that will take those names from that file and then run a command such as the host command to get the IP.

This command works for instance:

host csx-svc-spls-06 | awk '{ print $3 }'

and it returns just the IP of that server. Is it possible to read from the file, have it run the command, and export the name of the server and then the IP address on one line to a new file?

Solution 1

I'm not sure of the implications of using nslookup over dig, but I think this might work:

for i in `cat linux.hosts`; do nslookup $i | grep ^Name -A1| awk '{print $2}';echo;done > outputfile

Solution 2

dig has a +short option that makes it produce only the resulting IP addresses and nothing else.

---

This code:

for h in google.com yahoo.com notfound.nosuch; do
  printf '%s = ' "$h"
  printf '%s' "$(dig +short "$h")" | tr '\n' ,
  echo
done

Produces the output:

google.com = 108.177.112.102,108.177.112.139,108.177.112.101,108.177.112.138,108.177.112.100,108.177.112.113
yahoo.com = 98.138.252.38,98.139.180.180,206.190.39.42
notfound.nosuch = 

---

By contrast, this code:

for h in google.com yahoo.com notfound.nosuch; do
  printf "$h = %s\\n" $(dig +short "$h")
done

Produces the output:

google.com = 108.177.112.138
google.com = 108.177.112.100
google.com = 108.177.112.101
google.com = 108.177.112.113
google.com = 108.177.112.139
google.com = 108.177.112.102
yahoo.com = 98.139.180.180
yahoo.com = 206.190.39.42
yahoo.com = 98.138.252.38
notfound.nosuch = 

---

To read the hostnames from a file, replace the for lines in either of the above code snippets with for h in $(cat hostlist.txt); do — although you may want to add some input validation. Up to you.

---

Take your pick. If you want another format, please comment.

---

Note: if all you want is the IP addresses and you don't care about having the hostnames included in the output, then all you need is:

dig +short $(cat hostlist.txt)

Bear in mind, though, that any hostnames that don't appear in DNS will be silently ignored with this command.

Solution 3

To just iterate over the list with the command that you used on a single host (slightly modified to also include the hostname itself):

while read h; do
    host "$h" | awk '{ print $1, $3 }'
done <linux.hosts >outputfile.txt

This reads from linux.hosts, applies the host and awk commands on each entry in there, and saves the result to outputfile.txt.

Or, if you will, on one line:

while read h; do host "$h" | awk '{ print $1, $3 }'; done <linux.hosts >outputfile.txt

---

The following is a much more generic approach that takes hostname aliases and multiple IP addresses into account and produces one line of output for each line of input in the supplied file:

Given a list of hosts in hostnames.txt, such as

www.uu.se
www.kth.se
www.su.se

Then, given a short awk script (script.awk),

/^;/ || NF == 0 { next }

$4 == "CNAME"   { cn[$5] = (cn[$5] ? cn[$5] "," $1 : $1) }
$4 == "A"       { ip[$1] = (ip[$1] ? ip[$1] "," $5 : $5) }

END {
    for (i in ip) {
        if (cn[i])
            printf("%s (%s) = ", i, cn[i])
        else
            printf("%s = ", i)

        print ip[i]
    }
}

The bash command line

dig +noall +answer -f hostnames.txt | awk -f script.awk

would produce

su.se. (www.su.se.) = 193.11.30.171
www.kth.se. = 130.237.28.40
live.webb.uu.se. (www.uu.se.) = 130.238.7.135,130.238.7.133,130.238.7.134

This takes host name aliases and hosts with multiple IP addresses into account. The host name aliases are the names in parentheses.

The dig command by itself would produce the output

; <<>> DiG 9.4.2-P2 <<>> www.uu.se +noall +answer
;; global options:  printcmd
www.uu.se.              300     IN      CNAME   live.webb.uu.se.
live.webb.uu.se.        300     IN      A       130.238.7.134
live.webb.uu.se.        300     IN      A       130.238.7.135
live.webb.uu.se.        300     IN      A       130.238.7.133
www.kth.se.             600     IN      A       130.237.28.40
www.su.se.              300     IN      CNAME   su.se.
su.se.                  300     IN      A       193.11.30.171

and it does to through a batch query (-f) on the list of hostnames.

This is what the awk code parses and summarizes.

The awk script detects the CNAME entries for host name aliases and saves a comma-separated list of aliases for each real hostname in the cn array, and does a similar operation to save the IP addresses in the ip array for each hostname.

The END block goes through the hostnames in the ip array and checks to see whether it has any aliases or not. If it has, these are printed in parenthesis. Then the list of IP numbers are added to the end of the line.

Comments

[Jeff Schaller]

How is the input file formatted? One name per line?

[RomanPerekhrest]

post a testable input fragment

[Ivan Windon]

Yes, it's one hostname per file, such as this:

[Ivan Windon]

abs-tzg-nbu-20 abx-bcn-app-d1 abx-bcn-app-t1 abx-bcn-app-t2 abx-bcn-app-t3 abx-bri-ora-t2 abx-bri-ora-t3 abx-bri-ora-t4 abx-bri-ora-t5 abx-cmc-dbr-t01 abx-cmc-dbr-t02 abx-cmc-dbt-t01 abx-cmc-dbt-t02 abx-cvs-ora-t5 abx-cvs-was-d01

[Jeff Schaller]

Please edit your question with the updates, as you can see that the formatting in comments is limited.

[Ivan Windon]

The command you provided worked. Shows the hostname, with the IP on the next line. Thanks for the help.

[Mrdeep]

Glad to help! Please, consider "accepting" this answer if it has answered your question.