ReadAllLines for a Stream object?

Question

There exists a File.ReadAllLines but not a Stream.ReadAllLines.

using (Stream stream = Assembly.GetExecutingAssembly().GetManifestResourceStream("Test_Resources.Resources.Accounts.txt"))
using (StreamReader reader = new StreamReader(stream))
{
    // Would prefer string[] result = reader.ReadAllLines();
    string result = reader.ReadToEnd();
}

Does there exist a way to do this or do I have to manually loop through the file line by line?

Solution 1

You can write a method which reads line by line, like this:

public IEnumerable<string> ReadLines(Func<Stream> streamProvider,
                                     Encoding encoding)
{
    using (var stream = streamProvider())
    using (var reader = new StreamReader(stream, encoding))
    {
        string line;
        while ((line = reader.ReadLine()) != null)
        {
            yield return line;
        }
    }
}

Then call it as:

var lines = ReadLines(() => Assembly.GetExecutingAssembly()
                                    .GetManifestResourceStream(resourceName),
                      Encoding.UTF8)
                .ToList();

The Func<> part is to cope when reading more than once, and to avoid leaving streams open unnecessarily. You could easily wrap that code up in a method, of course.

If you don't need it all in memory at once, you don't even need the ToList...

Solution 2

The .EndOfStream property can be used in the loop instead of checking if the next line is not null.

List<string> lines = new List<string>();

using (StreamReader reader = new StreamReader("example.txt"))
{
    while(!reader.EndOfStream)
    {
        lines.Add(reader.ReadLine());
    }
}

Solution 3

using (Stream stream = Assembly.GetExecutingAssembly().GetManifestResourceStream("Test_Resources.Resources.Accounts.txt"))
using (StreamReader reader = new StreamReader(stream))
{
    // Would prefer string[] result = reader.ReadAllLines();
    string[] result = reader.ReadToEnd().Split(Environment.NewLine.ToCharArray(), StringSplitOptions.RemoveEmptyEntries);
}

Solution 4

The use of Split here:

reader
   .ReadToEnd()
   .Split(Environment.NewLine.ToCharArray(), StringSplitOptions.RemoveEmptyEntries);

is not equivalent to ReadLine. If you look at the source for ReadLine, StreamReader.cs, you will see that it handles line terminators: \r, \n, and \r\n correctly. ReadLine does not return an extra empty string when the line terminator is \r\n, which is typical in DOS/Windows. Split "sees" (parses) \r followed by \n as 2 separate delimiters and returns an empty string.

'StringSplitOptions.RemoveEmptyEntries' in the above code does remove these empty strings, but it will also remove any empty lines which appear in the input as well.

Thus for the input line1\r \r line3\r ReadLine returns 3 lines. The 2nd is empty. Split creates 4 strings. (There is an additional string after the last \r.) It then removes the 2nd and the 4th.

Note that Split is not well suited to parsing text lines which are "post-fix" delimited. That is the delimiter appears after the token. While Split is suitable for infix, where the delimiters appear between the tokens. It is the difference between a,b,c and line1\r,line2,line3\r. For these inputs, Split returns 3 strings or 4 strings respectively.

Solution 5

Using the following extension method:

public static class Extensions
{
    public static IEnumerable<string> ReadAllLines(this StreamReader reader)
    {
        string line;
        while ((line = reader.ReadLine()) != null)
        {
            yield return line;
        }
    }
}

It's possible to get to your desired code:

using (Stream stream = Assembly.GetExecutingAssembly().GetManifestResourceStream("Test_Resources.Resources.Accounts.txt"))
using (StreamReader reader = new StreamReader(stream))
{
    string[] result = reader.ReadAllLines().ToArray();
}

View more solutions

Comments

[Coops]

Do you need to check !reader.EndOfStream as well or is that not necessary?

[Jon Skeet]

@CodeBlend: No, ReadLine returns null when it reaches the end of the stream.

[Brondahl]

@JonSkeet Is there any advantage to checking .ReadLine() != null, rather than !reader.EndOfStream? It seems like the latter makes the while loop much neater and easier to read (see Bryan Johnson's formulation below)? (especially when combined with yield return, since you then no longer need the line variable at all?)

[Brondahl]

I would propose an edit, but editing a @JonSkeet answer seems ... idk ... sacrilegious ;)

[Jon Skeet]

@Brondahl: Yes, there's a benefit: you may not know whether the stream has ended until you try to read from it. There may be no more data to come, which is what you actually care about. (An edit here would have been inappropriate - edits shouldn't change the intention, which this one certainly would have done.) (I suspect there may be cases where StreamReader.EndOfStream will basically give the wrong result here, but I'm not going to spend time trying to prove it right now.)

[Brondahl]

@JonSkeet I think this is getting to the part of Streams that I've never really needed to understand since I've only ever used them with Files or WebRequests. If I've understood you correctly, then you're saying that there are ways of loading data into a Stream such that .EndOfStream could return false, but calling .ReadLine() would still return null. Is that right? Is the opposite also possible? That .EndOfStream could return true, but calling .ReadLine() would return actual data?

[Jon Skeet]

@Brondahl: Yes, you've understood me correctly. The reverse should not be true, unless the stream is misbehaving.

[Brondahl]

See discussion on Jon Skeet's answer. Using .EndOfStream doesn't necessarily work for all Streams. I suspect it'll be fine for "simple" streams like Files and WebRequests, though.