sort: write failed | broken pipe

Question

Good evening,

The following is a piece of the code I'm using in a script. Launching from a SSH sesssion works fine, however, when it runs via cron, it displays broken pipe errors on screen.

I can't reproduce it via SSH.

Code:

IP=$(sort --random-sort /root/ips.csv | head -n 1); nc -zv -w 2 $IP 443 2>&1 | grep succeeded >> outfile

Error in screen:

sort: write failed: standard output; Broken pipe
sort: write error

Any tips/pointers?

Thank you!

Solution 1

When head finishes after handling the first line, it exits, closing the other end of the pipe. sort may still be trying to write more, and writing to a closed pipe or socket returns the EPIPE error. But it also raises the SIGPIPE signal, killing the process, unless the signal is ignored or handled. With the signal ignored, sort sees the error, complains, and exits. If the signal is not ignored, sort just dies.

We can use the trap builtin to ignore a signal from the shell, getting the error:

$ trap "" PIPE
$ sort bigfile | head -1 > /dev/null 
sort: write failed: standard output: Broken pipe
sort: write error

But sadly, we can't use trap to un-ignore the signal and get the desired behaviour, as POSIX requires that it's not allowed to do that in non-interactive shells (scripts). It does allow it for interactive shells, but Bash's trap doesn't do it in that case either.

To test:

sh$ trap '' PIPE                     # ignore the signal    
sh$ PS1='another$ ' bash             # run another shell
another$ trap - PIPE                 # try to reset the signal
                                     # it doesn't work
another$ sort bigfile |head -1 > /dev/null
sort: write failed: 'standard output': Broken pipe
sort: write error

Instead, we could use an external tool, like a Perl one-liner to run a script or command with the signal un-ignored (sort exits silently here):

another$ perl -e '$SIG{PIPE}="DEFAULT"; exec "@ARGV"' \
         'sort bigfile |head -1' > /dev/null 
another$ 

---

As for your situation with cron, the reason could be that systemd apparently makes SIGPIPE ignored by default, mentioning:

[SIGPIPE is] not really useful for normal daemons though, and as we try to provide a good, useful execution environment for daemons, we turn this off. Of course, shells and suchlike should turn this on again.

Of course, this is also mentioned in the documentation (systemd.exec):

IgnoreSIGPIPE=
Takes a boolean argument. If true, causes SIGPIPE to be ignored in the executed process. Defaults to true because SIGPIPE generally is useful only in shell pipelines.

On my Debian system, /lib/systemd/system/cron.service explicitly sets IgnoreSIGPIPE=false, undoing the systemd default for cron. You may want to check if that would help in your case.

Solution 2

Since --random-sort is a GNU extension anyway, you might as well use GNU shuf which comes from the same collection of GNU utilties (GNU coreutils):

ip=$(shuf -n 1 ips.tsv)

Beside avoiding the SIGPIPE issue, that is also more efficient as it doesn't need to shuffle the whole file as GNU sort --random-sort does (shuf uses some Reservoir Sampling algorithm on large inputs to do that).

Comments

[Admin]

Chances are that cron's current working directory is not the location of ips.csv, so the file is not there to read, breaking the pipe. Try IP=$(sort --random-sort /path/to/ips.csv [...].

[Admin]

Good point. Actually I have the full path to the file, but I forgot to include it when editing the code to post online.

[Admin]

Since I can't seem to figure out how to make this code work, are there any alternatives to this code? Objective is to randomize a list and connect to the first IP (occurance) of that list?