Copy data from fstream to stringstream with no buffer?

Question

Is there anyway I can transfer data from an fstream (a file) to a stringstream (a stream in the memory)?

Currently, I'm using a buffer, but this requires double the memory, because you need to copy the data to a buffer, then copy the buffer to the stringstream, and until you delete the buffer, the data is duplicated in the memory.

std::fstream fWrite(fName,std::ios::binary | std::ios::in | std::ios::out);  
    fWrite.seekg(0,std::ios::end); //Seek to the end  
    int fLen = fWrite.tellg(); //Get length of file  
    fWrite.seekg(0,std::ios::beg); //Seek back to beginning  
    char* fileBuffer = new char[fLen];  
    fWrite.read(fileBuffer,fLen);  
    Write(fileBuffer,fLen); //This writes the buffer to the stringstream  
    delete fileBuffer;`

Does anyone know how I can write a whole file to a stringstream without using an inbetween buffer?

Solution 1

// need to include <algorithm> and <iterator>, and of course <fstream> and <sstream>
ifstream fin("input.txt");
ostringstream sout;
copy(istreambuf_iterator<char>(fin),
     istreambuf_iterator<char>(),
     ostreambuf_iterator<char>(sout));

Solution 2

 ifstream f(fName);
 stringstream s;
 if (f) {
     s << f.rdbuf();    
     f.close();
 }

Solution 3

In the documentation for ostream, there are several overloads for operator<<. One of them takes a streambuf* and reads all of the streambuffer's contents.

Here is a sample use (compiled and tested):

#include <exception>
#include <iostream>
#include <fstream>
#include <sstream>

int main ( int, char ** )
try
{
        // Will hold file contents.
    std::stringstream contents;

        // Open the file for the shortest time possible.
    { std::ifstream file("/path/to/file", std::ios::binary);

            // Make sure we have something to read.
        if ( !file.is_open() ) {
            throw (std::exception("Could not open file."));
        }

            // Copy contents "as efficiently as possible".
        contents << file.rdbuf();
    }

        // Do something "useful" with the file contents.
    std::cout << contents.rdbuf();
}
catch ( const std::exception& error )
{
    std::cerr << error.what() << std::endl;
    return (EXIT_FAILURE);
}

Solution 4

The only way using the C++ standard library is to use a ostrstream instead of stringstream.

You can construct a ostrstream object with your own char buffer, and it will take ownership of the buffer then (so no more copying is needed).

Note however, that the strstream header is deprecated (though its still part of C++03, and most likely, it will always be available on most standard library implementations), and you will get into big troubles if you forget to null-terminate the data supplied to the ostrstream.This also applies to the stream operators, e.g: ostrstreamobject << some_data << std::ends; (std::ends nullterminates the data).

View more solutions

Comments

[Ben Voigt]

What's the point? Are you trying to improve throughput? You're likely going to need to ditch fstream in that case, iostreams are SLOW. Are you trying to decrease your memory footprint? Reading the file in chunks instead of all at once could help with that.

[Benjamin Lindley]

@Charles -- Nonetheless, I think this is what he intended. He didn't want to allocate a new char array. He wanted to read directly from the fstream object to the stringstream object.

[Charles Salvia]

@BenVoigt, you're right - I misread the OP. I thought they were asking how to read directly into a stringstream.

[Brad]

This seems to be what I was looking for! @PigBen you are right for me not wanting to use a char array. I have a question: does the copy method account for the current writing position of the stringstream? If I use tellp(8), then use copy() will it still work, or will it write from the beginning of the stringstream? EDIT: I tried it out formyself and it seems to work great. Thank you!

[Ben Voigt]

Still, I like pinkfloydx's solution (made into a complete example by Andre) better. std::copy is going to have to move one element at a time, while operator <<(istream&) can potentially be much faster.

[Benjamin Lindley]

@Ben Voigt: So do I. I actually didn't know you could do that so easily. My testing shows a slight difference. On a 50k file, copied 10 times each with both methods, his averaged 1.45 seconds per copy, and mine averaged 1.62 seconds.

[Ben Voigt]

Are you serious? Disk-to-memory speed of 35kB/sec? That's so 1990. Coincidentally, earlier today I started on a rewrite of some of my code from ifstream to MapViewOfFile since the profiling I did on Friday showed a huge amount of time spent in fstream. Fully 10% of my runtime was ofstream taking a lock (this when the file is used from a single thread). Simply ludicrous.