Cat to named pipe causes hang

Question

I'm attempting some streaming with FFMpeg, and I think I have finally found the solution I need to screen blanking. This requires writing PNG's to a named pipe and then having FFMpeg read them as an overlay. However, I seem to be having a weird issue.

#!/bin/bash

mkfifo /tmp/stream_pipe
cat /path/to/transparent/or/splash/screen.png > stream_pipe
echo "done!"

In this example, echo "done!" never actually runs.

So, I moved to the terminal and tried the same thing, and sure enough the cat command hangs indefinitely.

I don't recall ever having issues with catting to named pipes and, indeed, I saw a lot of named pipe references that use just this. What could possibly be going wrong here? Shouldn't cat exit once it has finished writing the file to the pipe? The file being written is just a normal PNG file, so this has really got my puzzled.

Or is it that cat attempts to close the pipe upon writing perhaps? This would make sense why it would hang since it's not a legitimate file, but I would imagine that cat would know the difference or wouldn't care.

EDIT: cat $file | echo -n > /tmp/stream_pipe exhibits the same behavior

Solution 1

cat $file >/tmp/stream_pipe didn't work, but I was able to make it work (to some degree) by assigning the pipe to a file descriptor like so:

#!/bin/bash

exec 3<>/tmp/stream_pipe
cat /path/to/file.png >&3

I said it works to some degree, because running it stand-alone clogs the pipe and the script appears to hang. This is due to the fact that pipes in Ubuntu have a 1M buffer limit as seen via the command systcl fs.pipe-max-size. Unfortunately, settings this value via sysctl doesn't seem to change it. However, so long as the pipe data is being consumed by another process, there shouldn't be an issue. Any writes to a full pipe will block (it looks like a hang, but it's not) until data is consumed at which point the write will continue.

Solution 2

You need to have something reading from the FIFO.

Try this:

#!/bin/bash 

mkfifo /tmp/stream_pipe
cat /path/to/transparent/or/splash/screen.png > stream_pipe &
cat stream_pipe
echo "done!"

You should get the contents of the file printed to the screen (same as if you do cat /path/to/transparent/or/splash/screen.png).

Comments

[Rmano]

Can't check now, but you probably need to read the other side of the pipe... Try to do a cat /tmp/stream_pipe > /dev/null in another terminal. I seems to remember that the pipe will cache around 4k of data and then block, waiting to be emptied.

[Rmano]

I don't think, but shouldn't be a problem. Put your cat >pipe in background and let the multitasking do the magic...

[Chuck R]

That's the thing, this file wasn't enough to clog or saturate the pipe. So, it should have executed without issue. The file was only 300k or something, but the pipe buffer is 1M.

[Chuck R]

ffmpeg is reading from the FIFO.

[Sam Brightman]

As I understand it there are 3 sizes: PIPE_BUF (maximum atomic write - default 4K), pipe size (total size, default 64K) and max pipe size (maximum that pipe size can be set to, default 1MB). I have no problem setting the maximum size with sudo sysctl -w fs.pipe-max-size=2097152. The size of a given pipe can be set with a system call up to that limit. I don't think PIPE_BUF is settable (maybe at compile time). See man7.org/linux/man-pages/man7/pipe.7.html