boost::asio::buffer: Getting the buffer size and preventing buffer overflow?

Question

I have the two following functions for sending and receiving packets.

void send(std::string protocol)
{
    char *request=new char[protocol.size()+1];
    request[protocol.size()] = 0;
    memcpy(request,protocol.c_str(),protocol.size());

    request_length = std::strlen(request);
    boost::asio::write(s, boost::asio::buffer(request, request_length));
}
void receive()
{
    char reply[max_length];
    size_t reply_length = boost::asio::read(s, boost::asio::buffer(reply, request_length));
    std::cout << "Reply is: ";
    std::cout.write(reply, reply_length);
    std::cout << "\n";
}

The questions pertain to this part boost::asio::buffer(reply, request_length) where the request length is the length of a string which was initially setup when the packet was sent. How do I check the size of the buffer without knowing request_length? Another question is how do I prevent buffer overflow?

Solution 1

To get the size of a buffer, the boost::asio::buffer_size() function can be used. However, in your example, this will most likely be of little use to you.

As explained in the buffer overview, Boost.Asio use buffer classes to represent buffers. These classes provide an abstraction and protect Boost.Asio operations against buffer overruns. Although the result of boost::asio::buffer() is passed to operations, the meta-data, such as the size of the buffer or its underlying type, is not transmitted. Also, these buffers do not own the memory, so it is the applications responsibility to ensure the underlying memory remains valid throughout the duration of the buffer abstraction's lifetime.

The boost::asio::buffer() function provides a convenient way to create the buffer classes, where the size of the buffer is deduced from the type possible. When Boost.Asio is able to deduce the buffer length, then Boost.Asio operations will not invoke a buffer overflow when using the resulting buffer type. However, if the application code specifies the size of the buffer to boost::asio::buffer(), then it is the applications responsibility to ensure that the size is not larger than the underlying memory.

When reading data, a buffer is required. The fundamental question becomes how does one know how much memory to allocate, if Boost.Asio does not transmit the size. There are a few solutions to this problem:

• Query the socket for how much data is available via socket::available(), then allocate the buffer accordingly.

  std::vector<char> data(socket_.available());
  boost::asio::read(socket_, boost::asio::buffer(data));
  

• Use a class that Boost.Asio can grow in memory, such as boost::asio::streambuf. Some operations, such as boost::asio::read() accept streambuf objects as their buffer and will allocate memory as is required for the operation. However, a completion condition should be provided; otherwise, the operation will continue until the buffer is full.

  boost::asio::streambuf data;
  boost::asio::read(socket_, data,
                    boost::asio::transfer_at_least(socket_.available()));
  

• As Öö Tiib suggests, incorporate length as part of the communication protocol. Check the Boost.Asio examples for examples of communication protocols. Focus on the protocol, not necessarily on the Boost.Asio API.

  • In a fixed size protocol, both the data producer and consumer use the same size message. As the reader knows the size of the message, the reader can allocate a buffer in advance.

  • In a variable length protocol, the messages are often divided into two parts: a header and a body. The header is normally fixed size, and can contain various meta-information, such as the length of the body. This allows a reader to read a header into a fixed size buffer, extract the body length, allocate a buffer for the body, then read the body.

    // Read fixed header.
    std::vector<char> data(fixed_header_size);
    boost::asio::read(socket_, boost::asio::buffer(data));
    
    protocol::header header(data);
    network_to_local(header); // Handle endianess.
    
    // Read body.
    data.resize(header.body_length());
    boost::asio::read(socket_, boost::asio::buffer(data));  
    
    protocol::body body(data);
    network_to_local(body); // Handle endianess.    
    

Solution 2

Typically a communication protocol either uses fixed length messages or messages that contain header that tells the length of message.

Boost.Asio online documentation contains large set of examples and tutorials so you should perhaps start from there. Wikipedia is good source for explaining data transmission terminology, boost asio documentation does not do it.

Solution 3

I think your question is confusing, but this might help:

void receive() {
  enum { max_length = 1024 };
  char reply[max_length];
  size_t reply_length;
  std::cout << "Reply is: ";
  while ( (reply_length = ba::read(s, basio::buffer(reply, max_length))) > 0) {
    std::cout.write(reply, reply_length);
  }
  std::cout << "\n";
}

Comments

[pandoragami]

The boost::asio::buffer(,) returns what exactly. I see that it returns something into the ba::read(s,buffer(,)) but what is it?

[pandoragami]

I've already looked through the boost asio tutorials but those just gloss over the examples and don't explain in great detail what the function calls do, what they return, etc. I know I could look into the hpp files also but thats like a needle in the haystack and even more useless than the examples themselves. PLus I'm sure there are lots of things not said in the boost examples/tutorials but only found in the hpp files which could take years of practice to consume.

[Tanner Sansbury]

@lost_with_coding: A type that represents memory, but does not own the memory. The function helps adapt commonly used types so that they meet the type requirements for many of the Boost.Asio operations. This overview provides some information.

[Öö Tiib]

@lost_with_coding perhaps the boost asio examples expect people to be familiar with concept of digital communication and meaning of related terms like protcols, clients, servers, requests etc. used in examples. I suggest against reverse engineering such knowledge from code, there are easier sources to learn all that.

[Tanner Sansbury]

@lost_with_coding: The examples do not explain what the API does because that is documented in the reference section. The Boost.Asio library is complex, not complicated. There is a large amount of information in the documentation, and it may be worth taking the time to familiarize oneself with the various parts of the documentation.

[pandoragami]

@twsansbury thanks that reference link is useful. Never knew it existed.

[Matt Munson]

Great stuff. boost::asio::streambuff with boost::asio::read_until() worked well for me in the case where messages end with a delimiter.

[Torkel Bjørnson-Langen]

You don't have to pass max_length to asio::buffer() in this example. (Since asio can get it using sizeof.)