Powershell - How do I extract the first line of all text files in a directory into a single output file?

Question

I have a directory with about 10'000 text files of varying lengths. All over 1GB in size.

I need to extract the first line of each file and insert it into a new text file in the same directory.

I've tried the usual MS-DOS batch file method, and it crashes due to the files being too large.

Is there a way of doing this in Powershell using Streamreader?

Solution 1

In order to read only one line, you can also use :

$file = new-object System.IO.StreamReader($filename)
$file.ReadLine()
$file.close()

Using OutVariable you can write it in one line :

$text = (new-object System.IO.StreamReader($filename) -OutVariable $file).ReadLine();$file.Close()

Solution 2

EDIT: Of course there in an inbuilt way:

$firstLine = Get-Content -Path $fileName -TotalCount 1

[Ack Raf's comment]

---

Original:

I would suggest looking at File.ReadLines: this method reads the contents of the file lazily – only reading content with each iteration over the returned enumerator.

I'm not sure if Select-Object -first 1 will pro-actively halt the pipeline after one line, if it does then that is the easiest way to get the first line:

$firstLine = [IO.File]::ReadLines($filename, [text.encoding]::UTF8) | Select-Object -first 1

Otherwise something like:

$lines = [IO.File]::ReadLines($filename, [text.encoding]::UTF8); # adjust to correct encoding
$lineEnum = $lines.GetEncumerator();
if ($lineEnum.MoveNext()) {
  $firstLine = $lineEnum.Current;
} else {
  # No lines in file
}

NB. this assumes at least PowerShell V3 to use .NET V4.

Solution 3

Short and sweet:

cd c:\path\to\my\text\files\
Get-Content *.txt -First 1 > output.txt

Edit Nov 2018: According to the docs, "The TotalCount parameter limits the retrieval to the first n lines." This appears to minimize resource usage. Test it yourself and add your comments.

cd c:\path\to\my\text\files\
Get-Content *.txt -TotalCount 1 > output.txt