Why can't an input file be an output file?

Question

Specifically in the case of cat x y > y and cat x >> x where x and y are files.

What would happen if I didn't get a "input file is output file" error? Basically, why am I not allowed to run those commands?

Solution

cat x y > y would be the same as cp x y (because y would have been truncated by the time cat started) followed by cat y >> y.

cat x >> x would be an infinite loop until your disk was full or you hit the filesystem's file-size limit (because it keeps adding to the end of the file and then reading it back in to add again).

Note that this "error" doesn't mean that cat didn't do anything - it's just a notice that this particular file was skipped (in GNU cat, at least). You're "allowed" to run it, it just probably didn't do what you wanted and it's letting you know.

The test in GNU cat for generating that message is that:

1. the output (fd 0) is to a regular file; and
2. the device (st_dev field) and inode (st_ino field) are the same for both the output fd and the input file under examination; and
3. the offset in the input file is less than the size of the file (so it's not empty; I can't think of a case where the offset wouldn't be zero at this point, but presumably there is one).

If all those apply, it prints the diagnostic and moves on to the next input file.

Some versions of the cat command don't make this check and you can make these things happen. I can imagine a system where they do make the check and it's not able to tell they're the same file as well, though I couldn't off-hand tell you of one. In either case, it probably ends badly. It's possible that, rather than a loop, you just end up with the file so far doubled, if the cat implementation only makes a single write of the entire memory-mapped file, but that's just a lesser kind of badly.

Comments

[John Militer]

poor cat­

[Kusalananda]

What implementation of cat is able to give these warnings? The redirection is not part of the command line, so cat would have no idea where its output goes. Is it bash's "noclobber" message that this refers to maybe?

[Michael Homer]

@Kusalananda That's what fstat is for. You just have to check if the device & inode are the same as one of the inputs. Here it is in GNU cat, for example.

[Kusalananda]

Would you have to use some special flags with cat for this to take effect? The GNU cat that I have installed does not behave like this...

[Michael Homer]

cd /tmp ; echo 1 > a ; echo 2 > b ; cat a b > b -> cat: b: input file is output file works for me. The code's been there forever too.

[Kusalananda]

Ah, ok, I see now. I was testing on empty files.

[Michael Homer]

Yes, it specifically tests that the offset is less than the file size. I can't think of a case where the offset wouldn't just be zero at this point, but I suppose it must be possible.