How read a file into a seq of lines in F#

Question

This is C# version:

public static IEnumerable<string> ReadLinesEnumerable(string path) {
  using ( var reader = new StreamReader(path) ) {
    var line = reader.ReadLine();
    while ( line != null ) {
      yield return line;
      line = reader.ReadLine();
    }
  }
}

But directly translating needs a mutable variable.

Solution 1

open System.IO

let readLines (filePath:string) = seq {
    use sr = new StreamReader (filePath)
    while not sr.EndOfStream do
        yield sr.ReadLine ()
}

Solution 2

If you're using .NET 4.0, you can just use File.ReadLines.

> let readLines filePath = System.IO.File.ReadLines(filePath);;

val readLines : string -> seq<string>

Solution 3

To answer the question whether there is a library function for encapsulating this pattern - there isn't a function exactly for this, but there is a function that allows you to generate sequence from some state called Seq.unfold. You can use it to implement the functionality above like this:

new StreamReader(filePath) |> Seq.unfold (fun sr -> 
  match sr.ReadLine() with
  | null -> sr.Dispose(); None 
  | str -> Some(str, sr))

The sr value represents the stream reader and is passed as the state. As long as it gives you non-null values, you can return Some containing an element to generate and the state (which could change if you wanted). When it reads null, we dispose it and return None to end the sequence. This isn't a direct equivalent, because it doesn't properly dispose StreamReader when an exception is thrown.

In this case, I would definitely use sequence expression (which is more elegant and more readable in most of the cases), but it's useful to know that it could be also written using a higher-order function.

Solution 4

    let lines = File.ReadLines(path)                

    // To check
    lines |> Seq.iter(fun x -> printfn  "%s" x) 

Solution 5

On .NET 2/3 you can do:

let readLines filePath = File.ReadAllLines(filePath) |> Seq.cast<string>

and on .NET 4:

let readLines filePath = File.ReadLines(filePath);;

View more solutions

Comments

[Yin Zhu]

Thanks! Btw, is there a library function for that?

[ChaosPandion]

@David - There certainly should be. I believe the .NET libraries are slowly moving towards more IEnumerable interfaces.

[AruniRC]

on using this i am getting the following exception: {"Cannot read from a closed TextReader."} at the match sr.ReadLine() with line. any help please as to why?

[Nick Heiner]

Does that require holding the entire file in memory at once, or can it process line by line?

[Joel Mueller]

"The ReadLines and ReadAllLines methods differ as follows: When you use ReadLines, you can start enumerating the collection of strings before the whole collection is returned; when you use ReadAllLines, you must wait for the whole array of strings be returned before you can access the array. Therefore, when you are working with very large files, ReadLines can be more efficient."

[Tomas Petricek]

@AruniRC I think the solution by @ChaosPandion is much nicer than the one using unfold, so I would go with that :-)

[User]

I needed to read a file already opened by another process so I modified as: use fs = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite); use sr = new StreamReader(fs)

[Mr. Curious]

@AruniRC, the Seq is lazy--by the time you evaluate it later in the code, the reader might already be closed, hence the Cannot read from a closed TextReader. You will have to force evaluation of the sequence immediately, for example by converting to list with Seq.toList, or some other trick.

[Botond Balázs]

The first one of these is not lazy (ReadAllLines eagerly reads all lines into an array).

[Mikeb]

probably obvious to most but also needs an open System.IO directive