How can I fix a Broken Pipe error?

bash ruby .bash-profile rvm

160,152

Solution 1

Seeing "Broken pipe" in this situation is rare, but normal.

When you run type rvm | head -1, bash executes type rvm in one process, head -1 in another.¹ The stdout of type is connected to the "write" end of a pipe, the stdin of head to the "read" end. Both processes run at the same time.

The head -1 process reads data from stdin (usually in chunks of 8 kB), prints out a single line (according to the -1 option), and exits, causing the "read" end of the pipe to be closed. Since the rvm function is quite long (around 11 kB after being parsed and reconstructed by bash), this means that head exits while type still has a few kB of data to write out.

At this point, since type is trying to write to a pipe whose other end has been closed – a broken pipe – the write() function it caled will return an EPIPE error, translated as "Broken pipe". In addition to this error, the kernel also sends the SIGPIPE signal to type, which by default kills the process immediately.

(The signal is very useful in interactive shells, since most users do not want the first process to keep running and trying to write to nowhere. Meanwhile, non-interactive services ignore SIGPIPE – it would not be good for a long-running daemon to die on such a simple error – so they find the error code very useful.)

However, signal delivery is not 100% immediate, and there may be cases where write() returns EPIPE and the process continues to run for a short while before receiving the signal. In this case, type gets enough time to notice the failed write, translate the error code and even print an error message to stderr before being killed by SIGPIPE. (The error message says "-bash: type:" since type is a built-in command of bash itself.)

This seems to be more common on multi-CPU systems, since the type process and the kernel's signal delivery code can run on different cores, literally at the same time.

It would be possible to remove this message by patching the type builtin (in bash's source code) to immediately exit when it receives an EPIPE from the write() function.

However, it's nothing to be concerned about, and it is not related to your rvm installation in any way.

Solution 2

You can fix a broken pipe at the expense of another process by inserting tail -n +1 in your pipe, like this:

type rvm | tail -n +1 | head -1

The +1 tells tail to print the first line of input and everything that follows. Output will be exactly the same as if tail -n +1 wasn't there, but the program is smart enough to check standard output and closes the pipe down cleanly. No more broken pipes.

Solution 3

Let's try with yes, an endless process printing yes...

Before, the yes process was killed by SIGPIPE when reached limit.

➜ set -o pipefail
➜ yes | head -n 1
y
➜ echo $?        
141

My solution

➜ yes | (head -n 1;dd status=none of=/dev/null)
y


# the process will still running and output to null

You can replace yes with your program.

Solution 4

The write error: Broken pipe message refers to a writing process that tries to write to a pipe with no readers left on the reading end of that pipe and the special circumstance that the SIGPIPE signal is set to be ignored either by the current or the parent process. If it was the parent process that set SIGPIPE to be ignored, it is not possible for the child process to undo that again in a non-interacitive shell.

However, it is possible to kill type rvm when head -1 terminates by using explicit subshells. This way we can background type rvm, send typepid to the head -1 subshell and then implement a trap on EXIT there to kill type rvm explicitly.

trap "" PIPE        # parent process sets SIGPIPE to be ignored
bash                # start child process
export LANG=C
# create a fake rvm function
eval "
rvm() {
$(printf 'echo line of rvm code %s\n' {1..10000})
}
"

# rvm is a function
# bash: type: write error: Broken pipe
type rvm | head -1

# kill type rvm when head -1 terminates
# sleep 0: do nothing but with external command
( (sleep 0; type rvm) & echo ${!} ; wait ${!} ) | 
    (trap 'trap - EXIT; kill "$typepid"; exit' EXIT; typepid="$(head -1)"; head -1)

View more solutions

160,152

Jason Shultz

Updated on September 18, 2022

Comments

Jason Shultz over 1 year
I recently reinstalled RVM (following the instructions at http://rvm.io) after a fresh install of Ubuntu 12.10 when I got an SSD Drive.

Now, when I type: type rvm | head -1

I receive the following error:
```
rvm is a function
-bash: type: write error: Broken pipe
```
But if I immediately repeat the command then I only receive:
```
rvm is a function
```
And it appears everything is ok? What's happening? What can I do to fix it? It doesn't happen always. It appears to be more sporadic. I've tried to find some kind of pattern to it but haven't yet.
Somebody still uses you MS-DOS over 9 years

Nice trick. I've used in a different situation than the one provided here. Thanks!
Benubird about 9 years

Note: you may need to use use tail -n +1, otherwise tail thinks the "+1" is meant to be a filename.
Roger Dueck over 8 years

Looked like a great solution, but on Ubuntu 14.04.2 with tail 8.21 I get "tail: write error: Broken pipe", which is no improvement.
Huuu over 8 years

Version 8.21 seems to require the -n flag. see edit
Andrew Beals over 8 years

@RogerDueck is correct. I also see this on a Mandriva system for a similar sort of problem find /var/lib/mysql -xdev -type f -daystart -mmin +5 -print0 | xargs -0 ls -ldt | tail -n +1 | head reliably yields xargs: ls: terminated by signal 13. As we know, the problem is one of input exhaustion and there's really only one command that deals with buffering: dd. Adding | dd obs=1M to the pipeline fixes the SIGPIPE for my use case.
Marco Marsala about 8 years

It the output of the first command is binary, you can use tail -c +1, I was sometimes getting the broken pipe error when piping the binary output of ImageMagick convert (PDF to TIFF conversion) to tesseract (an OCR app supporting only TIFF files)
Andrew Beals almost 8 years

I will further amend my suggestion, although I will note that I don't believe that xargs or type should be kvetching about SIGPIPE, to this: type rvm | (head -1 ; dd of=/dev/null) This, of course, is similar to other suggestions as it's causing all of the input to be processed, but dd should be the most efficient program to handle such things.
Andrew Beals almost 8 years

Commentary on SIGPIPE violators here: mail-index.netbsd.org/tech-userlevel/2013/01/07/msg007110.ht‌ml
Piotr Dobrogost over 7 years

@AndrewBeals (...) there's really only one command that deals with buffering: dd Reference needed.
Piotr Dobrogost over 7 years

From grawity's answer: type gets enough time to notice the failed write, translate the error code and even print an error message to stderr before being killed by SIGPIPE. I think your solution doesn't prevent the producer process (type here) from reacting to the failed write (due to closed pipe), does it?
jarno about 4 years

@AndrewBeals It does not help, if the source part of pipe is slow.
jarno about 4 years

IIRC there may be some race condition in this code at least if you use greater n for head -n. Maybe it is better to read typeid by read -r typeid than by typepid="$(head -1)".
jarno about 4 years

To be more general you should run the kill command like kill "$typepid" 2>/dev/null || : to avoid error, if the leaf side of the pipe finishes first.
jarno almost 4 years

See my answer to a related question.
eel ghEEz over 2 years

cat > /dev/null works, too. Indeed, like @user1686 said, the timing between erroring out with EPIPE vs the asynchronous SIGPIPE affects the error code. In my WSL2 system, echoing 162 or more empty lines exhibited the transition from 0 (ignored EPIPE?) to 141 (SIGPIPE, bash exit code 128+13) as the exit code. Then cat appears to consume the rest of the pipe (just as dd). $ { for((i=0;i<200;i++)); do echo; done } | { head -n 1; cat > /dev/null; }; echo $? 0