boost::asio::streambuf - how to reuse buffer?

Question

I'm implementing TCP server that uses both asio socket.async_read() and boost::asio::async_read_until() methods for asynchronous reading data from socket. Both use the same handler for reading data from boost::asio::streambuf.

The handler that perfectly works invoked via async_read() :

void handle_read(const boost::system::error_code& ec, std::size_t ytes_transferred) )
{
    m_request_buffer.commit(bytes_transferred);
    boost::asio::streambuf::const_buffers_type rq_buf_data = m_request_buffer.data();
    std::vector<uint8_t> dataBytes(boost::asio::buffers_begin(rq_buf_data), boost::asio::buffers_begin(rq_buf_data) + bytes_transferred);

    //process data here

    m_request_buffer.consume(bytes_transferred);
    bytes_transferred = 0;
}

My server depending on processing of data may shutdown connection or continue reading via the same socket.

But, if handle_read() is called from the 2-nd boost::asi::async_read_until() call, I'm getting a number of zero's in dataBytes and then valid data goes.

I tried a simple test-case and found out that after writing data to streambuf, and commit() + consume() the data in streambuf still keeps previous buffer.

So, is there any way to clear data in boost::asio::streambuf and reuse it in boost::asio::async_read_until() ?

Live Coliru

If compiled with USE_STREAM=1, the live example works fine. But what std::istream does different comparing with buffer consume() ?

Solution

When using Boost.Asio operations that operate on streambuf or stream objects that use a streambuf, such as std::ostream and std::istream, the underlying input and output sequences will be properly managed. If a buffer is provided to an operation instead, such as passing passing prepare() to a read operation or data() to a write operation, then one must explicitly handle the commit() and consume().

The issue in the example is that it violates the API contract, causing uninitialized memory to be committed to the input sequence. The commit() documentation states:

Requires a preceding call prepare(x) where x >= n, and no intervening operations that modify the input or output sequence.

The use of the std::ostream between the prepare() and commit() violates this contract, as it will modify the input sequence:

// Prepare 1024 bytes for the output sequence.  The input sequence is
// empty.
boost::asio::streambuf streambuf;
streambuf.prepare(1024);

// prepare() and write to the output sequence, then commit the written
// data to the input sequence.  The API contract has been violated.
std::ostream ostream(&streambuf);
ostream << "1234567890";

// Commit 10 unspecified bytes to the input sequence.  Undefined
// behavior is invoked.
streambuf.commit(10);

---

Here is a complete example demonstrating using streambuf with annotated comments:

#include <iostream>
#include <vector>
#include <boost/asio.hpp>

int main()
{
  std::cout << "with streams:" << std::endl;
  {
    boost::asio::streambuf streambuf;

    // prepare() and write to the output sequence, then commit the written
    // data to the input sequence.  The output sequence is empty and
    // input sequence contains "1234567890".
    std::ostream ostream(&streambuf);
    ostream << "1234567890";

    // Read from the input sequence and consume the read data.  The string
    // 'str' contains "1234567890".  The input sequence is empty, the output
    // sequence remains unchanged.
    std::istream istream(&streambuf);
    std::string str;
    istream >> str;
    std::cout << "str = " << str << std::endl;

    // Clear EOF bit.
    istream.clear();

    // prepare() and write to the output sequence, then commit the written
    // data to the input sequence.  The output sequence is empty and
    // input sequence contains "0987654321".
    ostream << "0987654321";

    // Read from the input sequence and consume the read data.  The string
    // 'str' contains "0987654321".  The input sequence is empty, the output
    // sequence remains unchanged.
    istream >> str;
    std::cout << "str = " << str << std::endl;
  }

  std::cout << "with streams and manual operations:" << std::endl;
  {
    boost::asio::streambuf streambuf;

    // prepare() and write to the output sequence, then commit the written
    // data to the input sequence.  The output sequence is empty and
    // input sequence contains "1234567890".
    std::ostream ostream(&streambuf);
    ostream << "1234567890";

    // Copy 10 bytes from the input sequence.  The string `str` contains
    // "1234567890".  The output sequence is empty and the input
    // sequence contains "1234567890".
    auto data = streambuf.data();
    std::string str(boost::asio::buffers_begin(data),
                    boost::asio::buffers_begin(data) + 10);
    std::cout << "str = " << str << std::endl;

    // Consume 10 bytes from the input sequence.  The input sequence is
    // now empty.
    streambuf.consume(10);

    // prepare() and write to the output sequence, then commit the written
    // data to the input sequence.  The output sequence is empty and
    // input sequence contains "0987654321".
    ostream << "0987654321";

    // Copy 10 bytes from the input sequence.  The string `str` contains
    // "0987654321.  The output sequence is empty and the input
    // sequence contains "0987654321".    
    data = streambuf.data();
    str.assign(boost::asio::buffers_begin(data),
               boost::asio::buffers_begin(data) + 10);
    std::cout << "str = " << str << std::endl;

    // Consume 10 bytes from the input sequence.  The input sequence is
    // now empty.
    streambuf.consume(10);
  }
}

Output:

with streams:
str = 1234567890
str = 0987654321
with streams and manual operations:
str = 1234567890
str = 0987654321

For more information on streambuf usage, consider reading this answer.