awk in ssh command line complains 'unexpected newline or end of string' How to resolve?

Question

While running the below script had an issue, ^ unexpected newline or end of string How could I resolve this ?

[root@emrbldbgdapd2 ~]# ./collectdata.sh
collect the data of 10.209.61.124
awk: cmd. line:1: {print
awk: cmd. line:1:       ^ unexpected newline or end of string
collect the data of 10.209.61.125
awk: cmd. line:1: {print
awk: cmd. line:1:       ^ unexpected newline or end of string
[root@emrbldbgdapd2 ~]#
==========
[root@emrbldbgdapd2 ~]# more collectdata.sh
for i in `cat test`
do echo "collect the data of $i"
ssh -o LogLevel=error -o ConnectTimeout=5 $i 'hostname;
echo "############################";
free -g;
echo "######################################";
echo "######################################";
netstat -nr | grep [0-9] | tr [a-z] [A-Z];
echo "######################################";
echo "######################################";
mount|awk '{print $1,$3,$5}'|sort;
echo "######################################";
echo "######################################";
ip a s | grep -i eth*;
echo "######################################"'>output/$i-`date "+%d%b%Y"`
done
[root@emrbldbgdapd2 ~]#

Solution 1

You are trying to use single quotes within a singly-quoted string. The first internal single quote, on the same line as the mount | awk | sort pipeline, will terminate the single quote that is started on the ssh line.

The actual error comes from awk. It gets handed a command line that looks something like {print ,,} (due to $1, $3, $5 being expanded to empty strings by the calling shell and the single quotes being absent at this point). This is interpreted as an awk program with a syntax error (missing }) to be run on a file called ,,}.

I'd recommend to put the script in an actual script file and that you run this on the remote host instead, to avoid quoting issues and to allow you to create a more maintainable procedure.

Furthermore,

1. Use read in a while-loop rather than doing a for-loop over the result of cat.
2. The internal script in particular lacks necessary quoting in several places. This is needed so that e.g. grep [0-9] won't trigger filename globbing.
3. grep eth* looks odd. Did you mean grep -e 'eth.'? Apart from acting as a filename globbing pattern that would pick up any name in the current directory that starts with eth, the pattern (if interpreted as a regular expression) would match et followed by zero or more h.
4. See Have backticks (i.e. `cmd`) in *sh shells been deprecated? regarding your use of backticks.

Modified script:

#!/bin/sh

now=$( date +%d%b%Y )

if [ -d output ]; then
    echo 'Output directory missing' >&2
    exit 1
fi

while read host; do
    printf 'Collecting data from "%s"...\n' "$host"

    ssh -n -o LogLevel=error -o ConnectTimeout=5 "$host" \
      sh <<'END_SCRIPT' >output/"$host-$now"
        echo '## HOSTNAME:'
        hostname
        echo '## FREE:'
        free -g
        echo '## NETSTAT:'
        netstat -nr | grep '[0-9]' | tr 'a-z' 'A-Z'
        echo '## MOUNT:'
        mount | awk '{ print $1, $3, $5 }' | sort
        echo '## IP:'
        ip a s | grep -i 'eth.*'
END_SCRIPT
done <hosts.txt

Solution 2

As mentioned in other answers, the issues is with using single-quotes in the command you're running on the remote. As the syntax-coloring here on SE shows, the {print $1,$3,$5} is not quoted, and the single-quotes around it don't reach the remote shell.

One way to work around that would be to send the script to the remote through stdin with a here-doc:

ssh somehost /bin/sh > outputfile <<"EOF"
hostname
free -g
netstat -nr | grep "[0-9]" | tr "[a-z]" "[A-Z]"
mount | awk '{print $1,$3,$5}' | sort
# and so on
EOF

Put the here-doc separator "EOF" in quotes, so that the $1 variables aren't expanded by the local shell. That should take care of the quoting issues with the local shell, but you still need to quote stuff for the remote, like anything that looks like a glob ([0-9 and foo*).

tr doesn't need the brackets for the a-z arguments (they don't do any harm, they just tell it to change a [ to a [ and the same with ]). Though I think the output of that netstat contains the interface names, and uppercasing them will make the names different.

Comments

[ilkkachu]

That should fix the problem of the awk script getting split, but then the remote shell would expand the $1 and others before awk sees them. So you need ' ... | awk "{print \$1,\$3,\$5}" | ...' or ' ... | awk '\''{print $1,$3,$5}'\'' | ...' or such

[Cbhihe]

+1 for yr advice to use read in a while rather than cat in a for.

[Stéphane Chazelas]

If you use ssh within a while-read loop, you should use -n otherwise the first ssh will consume all the input.