What is the size of a socket send buffer in Windows?

Question

Based on my understanding, each socket is associated with two buffers, a send buffer and a receive buffer, so when I call the send() function, what happens is that the data to send will be placed into the send buffer, and it is the responsibility of Windows now to send the content of this send buffer to the other end.

In a blocking socket, the send() function does not return until the entire data supplied to it has been placed into the send buffer.

So what is the size of the send buffer?

I performed the following test (sending 1 GB worth of data):

#include <stdio.h>

#include <WinSock2.h>
#pragma comment(lib, "ws2_32.lib")

#include <Windows.h>

int main()
{
    // Initialize Winsock
    WSADATA wsa;
    WSAStartup(MAKEWORD(2, 2), &wsa);

    // Create socket
    SOCKET s = socket(AF_INET, SOCK_STREAM, 0);

    //----------------------

    // Connect to 192.168.1.7:12345
    sockaddr_in address;
    address.sin_family = AF_INET;
    address.sin_addr.s_addr = inet_addr("192.168.1.7");
    address.sin_port = htons(12345);
    connect(s, (sockaddr*)&address, sizeof(address));

    //----------------------

    // Create 1 GB buffer ("AAAAAA...A")
    char *buffer = new char[1073741824];
    memset(buffer, 0x41, 1073741824);

    // Send buffer
    int i = send(s, buffer, 1073741824, 0);

    printf("send() has returned\nReturn value: %d\nWSAGetLastError(): %d\n", i, WSAGetLastError());

    //----------------------

    getchar();
    return 0;
}

Output:

send() has returned
Return value: 1073741824
WSAGetLastError(): 0

send() has returned immediately, does this means that the send buffer has a size of at least 1 GB?

This is some information about the test:

• I am using a TCP blocking socket.
• I have connected to a LAN machine.
• Client Windows version: Windows 7 Ultimate 64-bit.
• Server Windows version: Windows XP SP2 32-bit (installed on Virtual Box).

---

Edit: I have also attempted to connect to Google (173.194.116.18:80) and I got the same results.

Edit 2: I have discovered something strange, setting the send buffer to a value between 64 KB and 130 KB will make send() work as expected!

int send_buffer = 64 * 1024;    // 64 KB
int send_buffer_sizeof = sizeof(int);
setsockopt(s, SOL_SOCKET, SO_SNDBUF, (char*)send_buffer, send_buffer_sizeof);

Edit 3: It turned out (thanks to Harry Johnston) that I have used setsockopt() in an incorrect way, this is how it is used:

setsockopt(s, SOL_SOCKET, SO_SNDBUF, (char*)&send_buffer, send_buffer_sizeof);

Setting the send buffer to a value between 64 KB and 130 KB does not make send() work as expected, but rather setting the send buffer to 0 makes it block (this is what I noticed anyway, I don't have any documentation for this behavior).

So my question now is: where can I find a documentation on how send() (and maybe other socket operations) work under Windows?

Solution 1

After investigating on this subject. This is what I believe to be the correct answer:

When calling send(), there are two things that could happen:

• If there are pending data which are below SO_SNDBUF, then send() would return immediately (and it does not matter whether you are sending 5 KB or you are sending 500 MB).

• If there are pending data which are above or equal SO_SNDBUF, then send() would block until enough data has been sent to restore the pending data to below SO_SNDBUF.

Note that this behavior is only applicable to Windows sockets, and not to POSIX sockets. I think that POSIX sockets only use one fixed sized send buffer (correct me if I'm wrong).

---

Now back to your main question "What is the size of a socket send buffer in Windows?". I guess if you have enough memory it could grow beyond 1 GB if necessary (not sure what is the maximum limit though).

Solution 2

I can reproduce this behaviour, and using Resource Monitor it is easy to see that Windows does indeed allocate 1GB of buffer space when the send() occurs.

An interesting feature is that if you do a second send immediately after the first one, that call does not return until both sends have completed. The buffer space from the first send is released once that send has completed, but the second send() continues to block until all the data has been transferred.

I suspect the difference in behaviour is because the second call to send() was already blocking when the first send completed. The third call to send() returns immediately (and 1GB of buffer space is allocated) just as the first one did, and so on, alternating.

So I conclude that the answer to the question ("how large are the send buffers?") is "as large as Windows sees fit". The upshot is that, in order to avoid exhausting the system memory, you should probably restrict blocking sends to no more than a few hundred megabytes.

Your call to setsockopt() is incorrect; the fourth argument is supposed to be a pointer to an integer, not an integer converted to a pointer. Once this is corrected, it turns out that setting the buffer size to zero causes send() to always block.

To summarize, the observed behaviour is that send() will return immediately provided:

• there is enough memory to buffer all the provided data
• there is not a send already in progress
• the buffer size is not set to zero

Otherwise, it will return once the data has been sent.

KB214397 describes some of this - thanks Hans! In particular it describes that setting the buffer size to zero disables Winsock buffering, and comments that "If necessary, Winsock can buffer significantly more than the SO_SNDBUF buffer size."

(The completion notification described does not quite match up to the observed behaviour, depending I guess on how you interpret "previously buffered send". But it's close.)

Note that apart from the risk of inadvertently exhausting the system memory, none of this should matter. If you really need to know whether the code at the other end has received all your data yet, the only reliable way to do that is to get it to tell you.