Async connect and disconnect with epoll (Linux)

Question

I need async connect and disconnect for tcp client using epoll for Linux. There are ext. functions in Windows, such as ConnectEx, DisconnectEx, AcceptEx, etc... In tcp server standard accept function is working, but in tcp client doesn't working connect and disconnect... All sockets are nonblocking.

How can I do this?

Thanks!

Solution 1

To do a non-blocking connect(), assuming the socket has already been made non-blocking:

int res = connect(fd, ...);
if (res < 0 && errno != EINPROGRESS) {
    // error, fail somehow, close socket
    return;
}

if (res == 0) {
    // connection has succeeded immediately
} else {
    // connection attempt is in progress
}

For the second case, where connect() failed with EINPROGRESS (and only in this case), you have to wait for the socket to be writable, e.g. for epoll specify that you're waiting for EPOLLOUT on this socket. Once you get notified that it's writable (with epoll, also expect to get an EPOLLERR or EPOLLHUP event), check the result of the connection attempt:

int result;
socklen_t result_len = sizeof(result);
if (getsockopt(fd, SOL_SOCKET, SO_ERROR, &result, &result_len) < 0) {
    // error, fail somehow, close socket
    return;
}

if (result != 0) {
    // connection failed; error code is in 'result'
    return;
}

// socket is ready for read()/write()

In my experience, on Linux, connect() never immediately succeeds and you always have to wait for writability. However, for example, on FreeBSD, I've seen non-blocking connect() to localhost succeeding right away.

Solution 2

From experience, when detect non-blocking connection , epoll is a little different from select and poll.

with epoll:

After connect() call is made, check return code.

If the connection can not be completed immediately, then register EPOLLOUT event with epoll.

Call epoll_wait().

if the connection failed, your events will be fill with EPOLLERR or EPOLLHUP, otherwise EPOLLOUT will be triggered.

Solution 3

I have tried the Sonny's solution and the epoll_ctl will return invalid argument. So i think maybe the right way to do this is as follow:

1.create socketfd and epollfd

2.use epoll_ctl to associate the socketfd and epollfd with epoll event.

3.do connect(socketfd,...)

4.check the return value or errno

5.if errno == EINPROGRESS, do epoll_wait

Solution 4

I have a "complete" answer here in case anyone else is looking for this:

#include <sys/epoll.h>
#include <errno.h>
....
....
int retVal = -1;
socklen_t retValLen = sizeof (retVal);

int status = connect(socketFD, ...);
if (status == 0)
 {
   // OK -- socket is ready for IO
 }
else if (errno == EINPROGRESS)
 {
    struct epoll_event newPeerConnectionEvent;
    int epollFD = -1;
    struct epoll_event processableEvents;
    unsigned int numEvents = -1;

    if ((epollFD = epoll_create (1)) == -1)
    {
       printf ("Could not create the epoll FD list. Aborting!");
       exit (2);
    }     

    newPeerConnectionEvent.data.fd = socketFD;
    newPeerConnectionEvent.events = EPOLLOUT | EPOLLIN | EPOLLERR;

    if (epoll_ctl (epollFD, EPOLL_CTL_ADD, socketFD, &newPeerConnectionEvent) == -1)
    {
       printf ("Could not add the socket FD to the epoll FD list. Aborting!");
       exit (2);
    }

    numEvents = epoll_wait (epollFD, &processableEvents, 1, -1);

    if (numEvents < 0)
    {
       printf ("Serious error in epoll setup: epoll_wait () returned < 0 status!");
       exit (2);
    }

    if (getsockopt (socketFD, SOL_SOCKET, SO_ERROR, &retVal, &retValLen) < 0)
    {
       // ERROR, fail somehow, close socket
    }

    if (retVal != 0) 
    {
       // ERROR: connect did not "go through"
    }   
}
else
{
   // ERROR: connect did not "go through" for other non-recoverable reasons.
   switch (errno)
   {
     ...
   }
}

View more solutions

Comments

[Ambroz Bizjak]

I believe your error check after the epoll_wait() is incorrect - you should always check the result of the connection attempt via getsockopt(SO_ERROR), even if you didn't get EPOLLERR. See EINPROGRESS in the man page linux.die.net/man/2/connect Also, assert() is the wrong way to handle critical errors - it would mean that you have proven that it can never happen. Use exit() instead, which will terminate the program even when NDEBUG is defined.

[Sonny]

Just added the edits suggested. The un-edited version seems to work for me.

[Ambroz Bizjak]

@Matt I know it does, you're probably doing something wrong here. What exactly are you trying, where is it failing? Have you put the socket into non-blocking mode using fcntl?

[hookenz]

Oops, never mind. A bug on my part. Yeah I had non-blocking sockets. I had a bug when checking the result of connect! classic c mistake.

[Ambroz Bizjak]

Yeah, I forgot to mention in my answer that epoll can return EPOLLERR or EPOLLHUP in addition to EPOLLOUT. Thanks for mentioning, it's corrected.

[zapstar]

Just as an FYI: If non-blocking connect fails, Windows will notify via an exceptional condition rather than socket writability. msdn.microsoft.com/en-us/library/windows/desktop/ms740141.as‌​px

[wonder]

Adding -1 as the timeout, shouldn't the epoll_wait block indefinitely in the above program?