Ctrl+D for ending terminal line input

Question

If I do

$ cat > file.txt

textCtrl-DCtrl-D

Question 1: If I don't press enter, why do I have to press Ctrl-D twice?

If I do

$ cat > file.txt

pa bam pshhhCtrl-Z

[2]+  Stopped         cat > file.txt
$ cat file.txt
$ cat > file.txt

pa bam pshhh

Ctrl-Z

[2]+  Stopped         cat > file.txt
$ cat file.txt
pa bam pshhh

Why is the second time the file with 1 line?

Solution 1

In Unix, most objects you can read and write - ordinary files, pipes, terminals, raw disk drives - are all made to resemble files.

A program like cat reads from its standard input like this:

n = read(0, buffer, 512);

which asks for 512 bytes. n is the number of bytes actually read, or -1 if there's an error.

If you did this repeatedly with an ordinary file, you'd get a bunch of 512-byte reads, then a somewhat shorter read at the tail end of the file, then 0 if you tried to read past the end of the file. So, cat will run until n is <= 0.

Reading from a terminal is slightly different. After you type in a line, terminated by the Enter key, read returns just that line.

There are a few special characters you can type. One is Ctrl-D. When you type this, the operating system sends all of the current line that you've typed (but not the Ctrl-D itself) to the program doing the read. And here's the serendipitous thing: if Ctrl-D is the first character on the line, the program is sent a line of length 0 - just like the program would see if it just got to the end of an ordinary file. cat doesn't need to do anything differently, whether it's reading from an ordinary file or a terminal.

Another special character is Ctrl-Z. When you type it, anywhere in a line, the operating system discards whatever you've typed up until that point and sends a SIGTSTP signal to the program, which normally stops (pauses) it and returns control to the shell.

So in your example

$ cat > file.txt
pa bam pshhh<Ctrl+Z>
[2]+  Stopped         cat > file.txt

you typed some characters that were discarded, then cat was stopped without having written anything to its output file.

$ cat > file.txt
pa bam pshhh
<Ctrl+Z>
[2]+  Stopped         cat > file.txt

you typed in one line, which cat read and wrote to its output file, and then the Ctrl-Z stopped cat.

Solution 2

That's because Ctrl+D is a hack.

Deep down, Ctrl+D (despite being called the eof character) doesn't actually mean end-of-file: it means “send the pending input to the application now”. This is actually close to the meaning of Ctrl+M (eol), which sends the pending input plus a newline.

When you press Ctrl+D immediately after a Ctrl+M (i.e. at the beginning of a line) or after another Ctrl+D, the pending input is empty. Thus the application receives 0 bytes of input. In a read call, reading 0 bytes signals the end of the file.

---

When you press Ctrl+Z, the pending input is discarded. Thus only what had already been sent to the application (which is cat) by entering a newline or Ctrl+D before pressing Ctrl+Z is processed.

Comments

[Admin]

This thread under the stackexchange sites umbrella have the answer you are looking at: stackoverflow.com/questions/7369170/…. Hope it helps.

[Admin]

Yeah, even though not python related, my question is a duplicate.

[Admin]

When you type ctrl-z, are you really getting just a new shell prompt, or are you also getting a message about cat being stopped?

[Admin]

I'll update the question

[Ramesh]

More information regarding ctrl + D from one of Gille's answer could be found here.

[mikeserv]

these things are only true of canonical mode terminals. And even in that case they can be changed.

[Mark Plotnick]

@mikeserv That's true. Here, I wanted to explain just what the OP was seeing. I considered also describing raw/-icanon terminal mode, other special characters, how they can be customized, how they differ by OS, etc. but didn't want to make the answer overly long.

[supercat]

Does the above imply that if input weren't going through cat, that a program which read data from the keyboard and didn't stop the first time read yielded a zero could keep on going, and the number of control-D's required would be determined by the number of consecutive zeroes the program required to decide it was done?

[Mark Plotnick]

@supercat A program can keep on reading if it wants. In the ex editor, if you type a control-D as the first character of a line, the editor shows you a few lines of the program rather than exiting. (In ex and vi, Control-D is a mnemonic for "down"). And with many shells, if you type Control-D but have jobs running in the background, the shell will inform you of this rather than exiting, but if you type Control-D again, the shell decides that you really want to exit anyway and will do so.

[supercat]

@MarkPlotnick: Is there any way by which these zero-byte hiccups can be sent via pipe?

[Mark Plotnick]

@supercat Nope. A write of zero length to a pipe is just like not having written anything to it at all.

[user2859406]

As you said, Ctrl-D does not mean end-of-file. In fact reason it does not mean that is because Ctrl-D is EOT (end-of-text).