Will read() ever block after select()?

Question

I'm reading a stream of data through TCP/IP socket. The stream load is very uneven. Sometimes large bulks of data arrive every second, sometimes no data come for an hour. In the case of long inactivity period (no data from remote server, but connection is still online) my program should take some actions.

I'm implementing a timeout using a select(). It tells me if there are data ready, but I don't know exactly how much can I read without causing read() to block. Blocking is unacceptable as it may last far longer than the timeout I need.

For the sake of efficiency, stream is read into large buffer and read() call is provided with that buffer size.

Will read() block after select() if the buffer to be filled is greater than amount of data available right now in the socket?

Solution 1

Actually it should not block (that is what select() is for!), but in fact, it might, exceptionally. Normally, read() should return up to the maximum number of bytes that you've specified, which possibly includes zero bytes (this is actually a valid thing to happen!), but it should never block after previously having reported readiness.

Nevertheless, see the Linux select man page:

Under Linux, select() may report a socket file descriptor as "ready for reading", while nevertheless a subsequent read blocks. This could for example happen when data has arrived but upon examination has wrong checksum and is discarded. There may be other circumstances in which a file descriptor is spuriously reported as ready. Thus it may be safer to use O_NONBLOCK on sockets that should not block.

Solution 2

There is O_NONBLOCK which can be set by fcntl/F_SETFL and should result in non-blocking read.

Solution 3

A blocking file descriptor will block on read() until there is something to read - could be one byte or your entire request. A non-blocking descriptor won't block on read() if there is nothing to read. Select() is not read(). It basically puts the process to sleep and monitors the file descriptor(s), including non-blocking descriptors. When there is activity on one of the descriptors (or the timeout period expires) select returns and you can read your data, or do something else in the case of the timeout.

So you have two separate issues. (1) You want to "take some actions" when there is no data. That's the select timeout. (2) Once you have data (notified by select) you don't want to block on a read. That's the non-blocking mode. When you get EAGAIN on the non-blocking read you loop back to the select and/or "take some actions" and loop back to select.

Comments

[Basilevs]

How do you implement efficient timeout with this? Cycle is not an option. Sleeping cycle is bad for its response time.

[Basilevs]

Oh, I get it. You mean a combination of select and nonblocking read. Not so convenient, but will do if there is no better option out there.

[Basilevs]

Will select block on nonblocking descriptor? Manual says it will return if the next read call will not block. Nonblocking descriptor won't block.

[Basilevs]

I'd like the code to be as much compatible with unnamed pipes as possible and I'm no sure if recv will work with them.

[JeremyP]

@Basilevs: I'm pretty sure that select() will wait for a non blocking descriptor to have data on it (or end of file).

[ony]

@Basilevs, my manual says "Those listed in readfds will be watched to see if characters become available for reading (more precisely, to see if a read will not block; in particular, a file descriptor is also ready on end-of-file)". I guess that's way to fix original sentence for situation when read will return immediately even when no characters are available. BTW, don't forget that select may be interrupted by signal earlier than any event on descriptors or timeout will happen.

[ony]

read will block if there is no bytes available at all and it's not the end-of-file yet if descriptor is in blocking mode, as I remember.

[Basilevs]

-1 Manual says: If no messages are available at the socket, the receive calls wait for a message to arrive, unless the socket is nonblocking (see fcntl(2)) in which case the value -1 is returned and the external variable errno set to EAGAIN.

[ninjalj]

@Basilevs: MKo surely meant MSG_DONTWAIT

[caf]

Exactly right. Keep using select(), but set the socket non-blocking and when select() says its readable, call read() until it returns EAGAIN.

[caf]

@Basilevs: Yes, select() blocks on non-blocking file descriptors - after all, that's the only sensible way to use non-blocking file descriptors.

[Basilevs]

@caf well, sleeping cycle is an another one

[caf]

@Basilevs: I wouldn't call that "sensible" ;)

[Nemo]

No, zero bytes is not a valid thing to happen (except at EOF); see the POSIX spec for recv() under "RETURN VALUE". The behavior documented by that Linux select() man page is also a violation of POSIX. This is how every Unix has worked since at least 1985; if your Unix does not, then your Unix is broken.

[Damon]

@Nemo: What are you trying to say? (1) One may assume that EOF and orderly shutdown are valid things to happen, and a condition that a program should expect and be prepared to handle. (2) The question is specific to Linux, insofar it is adequate for an answer to point out Linux special behaviour. (3) Although irrelevant, such a behaviour is not strictly a violation of POSIX as you put it, since POSIX allows conditions (arrival of OOB data is an example) which mark a socket as ready even though a subsequent recv will block. This is admittedly a rare incident, but possible and legitimate.

[Nemo]

I am saying that recv() returning zero means orderly shutdown (or EOF) and nothing else, ever. Yes, this select() behavior is a violation of POSIX; non-inline OOB data does not mark the socket ready for reading by select(). (Search for "OOB" in the POSIX spec for select.) POSIX simply does not permit either of the things you mention, period. Linux violates POSIX for select(). You are mistaken about when read()/recv() can return 0, even on Linux.

[Nemo]

Incidentally, I asked about Linux select() behavior on the linux-kernel list and got a good response from Alan Cox.

[David Schwartz]

@Nemo POSIX does not say that a select hit guarantees anything at all about any future operation, nor would that make any sense.

[Nemo]

@DavidSchwartz: Actually, that is exactly what it says. "A descriptor shall be considered ready for reading when a call to an input function with O_NONBLOCK clear would not block, whether or not the function would transfer data successfully." Obviously that (readable) state can change after the select, but only if someone performs an operation that changes it. Linux violates POSIX here, period. (I am old enough to remember when there was no Linux, and select+read on blocking sockets was standard procedure. Linux changed that; POSIX never did.)

[David Schwartz]

@Nemo The "only if someone performs an operation that changes it" is not found in POSIX. And even if that was true, Linux kernel code could perform operations that change it. POSIX does not prohibit that. The notion that this sentence in POSIX makes select anything more than a status reporting function with no future guarantees is entirely false. The same logic could argue that a write after checking disk space couldn't fail for insufficient disk space, which is also false. You only have a guarantee if the standard states so, you can't infer it.

[Nemo]

@DavidSchwartz: It is implicit in the concept of "state" and "O/S". If you write a file and then read it back, you get the same bytes you wrote. Of course, if someone else overwrites the file in the interim, that is not true... But it is true otherwise. The O/S is no more free to magically make your socket un-readable than it is to magically clobber your files. I am kind of amazed anyone would try to argue otherwise. (Even the Linux devs agree this violates POSIX; they just say it is a deliberate violation for speed.) And once again: No Unix ever behaved this way before Linux.

[David Schwartz]

@Nemo This is an attempt to "synthesize" a guarantee you don't have by the process of being unable to imagine any way the guarantee could be violated. This is simply incorrect. You can't get a guarantee that way. And you wouldn't have that guarantee about reading/writing files either -- a system with an anti-virus program that modifies files that it thinks are viruses wouldn't be violating POSIX. POSIX defines select the same way it defines all status-reporting functions, and those don't have future guarantees.

[Nemo]

@DavidSchwartz: An anti-virus program that "modifies files" would be an example of "someone else overwrites the file in the interim", so you are not even contradicting me here. The O/S kernel does not have license to randomly mutate visible system state. It cannot unilaterally un-allocate memory returned by malloc (and yes Linux w/ overcommit also violates POSIX), it cannot trash your files, it cannot fill your disk, and it cannot make a ready-for-read socket unready. All of this is fundamental to the very concept of "system state".

[David Schwartz]

@Nemo So an anti-virus built into the kernel violates POSIX? You're being absolutely ridiculous. POSIX knows how to give a guarantee and also how not to.

[Nemo]

@DavidSchwartz: In fact I am absolutely citing the spec. pubs.opengroup.org/onlinepubs/9699919799/functions/write.htm‌​l "After a write() to a regular file has successfully returned: Any successful read() from each byte position in the file that was modified by that write shall return the data specified by the write() for that position until such byte positions are again modified."

[David Schwartz]

@Nemo Notice it says, "until such byte positions are again modified", which does not constrain who or what can modify them. I would also add that by using words like "shall return", POSIX shows that they know how to provide a guarantee when they intend to do that.

[Damon]

I don't know, I think "cannot un-ready" is a harsh limitation not in accordance with reality. select must work on terminals, and modern computers have stuff like hotplug (for disks, and even GPUs) What if I disconnect the terminal, should the descriptor stay ready? What if I unplug a disk? Surely, it is possible for a descriptor to become un-ready whether the OS wants it or not. The terminal (disk) may be ready to receive (write), only there may not be a terminal at all any more. What about TCP? You cannot even know whether your NIC will be up when the datagrams are to be sent out, even...

[Damon]

... even after a "successful" send. It is quite possible that a "successful" send (i.e. the descriptor was ready, and stayed ready, the kernel accepted your write request and copied data to kernel memory, and you got a success return code) it still fails simply because there is no way to complete the operation. Or, maybe the receiving computer at the other end of the world crashed... You may only know a second or two later (or 30 secs). Things are not so 100% clear in the real world, unluckily.

[Fra]

@caf: You write "Keep using select(), but set the socket non-blocking and when select() says its readable, call read() until it returns EAGAIN" - programatically, does this mean entering a loop calling read() until EAGAIN is set at which point you break out?

[Damon]

That's what one usually does, yes. It is the only 100% safe thing. Plus, that's also what you do e.g. when using edge-triggered epoll. Depending on your situation, you may, however, do something different to prevent starvation of slow senders: Use select on a non-blocking socket, and when ready read once but remember that the socket is ready. Then read all other ready sockets round robin, and come back to this one eventually... until you get EAGAIN. This prevents a fast sender from completely starving a slow one. Not always needed, but sometimes (that's much more complicated, obviously).