Print a file, skipping the first X lines, in Bash

Question

I have a very long file which I want to print, skipping the first 1,000,000 lines, for example.

I looked into the cat man page, but I did not see any option to do this. I am looking for a command to do this or a simple Bash program.

Solution 1

You'll need tail. Some examples:

$ tail great-big-file.log
< Last 10 lines of great-big-file.log >

If you really need to SKIP a particular number of "first" lines, use

$ tail -n +<N+1> <filename>
< filename, excluding first N lines. >

That is, if you want to skip N lines, you start printing line N+1. Example:

$ tail -n +11 /tmp/myfile
< /tmp/myfile, starting at line 11, or skipping the first 10 lines. >

If you want to just see the last so many lines, omit the "+":

$ tail -n <N> <filename>
< last N lines of file. >

Solution 2

Easiest way I found to remove the first ten lines of a file:

$ sed 1,10d file.txt

In the general case where X is the number of initial lines to delete, credit to commenters and editors for this:

$ sed 1,Xd file.txt

Solution 3

If you have GNU tail available on your system, you can do the following:

tail -n +1000001 huge-file.log

It's the + character that does what you want. To quote from the man page:

If the first character of K (the number of bytes or lines) is a `+', print beginning with the Kth item from the start of each file.

Thus, as noted in the comment, putting +1000001 starts printing with the first item after the first 1,000,000 lines.

Solution 4

If you want to skip first two line:

tail -n +3 <filename>

If you want to skip first x line:

tail -n +$((x+1)) <filename>

Solution 5

A less verbose version with AWK:

awk 'NR > 1e6' myfile.txt

But I would recommend using integer numbers.

View more solutions

Comments

[paxdiablo]

Or "tail --lines=+<LinesToSkip> ..." for the readable-commands crowd :-)

[paxdiablo]

Not the most efficient answer since you'd need to do a "wc -l" on the file to get a line count, followed by an addition to add the million :-). You can do it with just "tail".

[Dana the Sane]

I'm not sure, my understanding was that 1e6 would be known at the time of calling. Counting backwards isn't the fastest though.

[guns]

++ for using awk, which is oh so marginally more portable than tail

[NickSoft]

in centos 5.6 tail -n +1 shows the whole file and tail -n +2 skips first line. strange. The same for tail -c +<num>.

[Joel Clark]

Nick you may be running up against windows style line endings.

[Andres F.]

@JoelClark No, @NickSoft is right. On Ubuntu, it's tail -n +<start number>, I just tested it. So tail -n +1 won't skip anything, but start from the first line instead.

[Asclepius]

In the more general case, you'd have to use sed 1,Xd where X is the number of initial lines to delete, with X greater than 1.

[Asclepius]

This is somewhat misleading because someone may interpret (x+1) literally. For example, for x=2, they may type either (2+1) or even (3), neither of which would work. A better way to write it might be: To skip the first X lines, with Y=X+1, use tail -n +Y <filename>

[Asclepius]

@Marlon, sorry but that's wrong. That only works for 1d. If, for example, you use it on 2d, you'll delete only line 2. It doesn't delete the range of lines.

[Marlon]

@A-B-B sorry, meant to say that this was the easiest solution by far which is why I +1 it not trying to correct the author.

[morgant]

I can confirm that tail -n +2 is required to skip the first line on Darwin/Mac OS X as well.

[osirisgothra]

this must be outdated, but, tail -n+2 OR tail -n +2 works, as with all short commands using getopt, you can run the parameter right next to it's switch, providing that the switch is the last in the group, obviously a command like tail -nv+2 would not work, it would have to be tail -vn+2. if you dont believe me try it yourself.

[arekolek]

useful if you need to skip some lines in the middle of the file, e.g., awk '!(5 < NR && NR < 10)'

[Lloeki]

Works for BSD tail too (OS X)

[G-Man Says 'Reinstate Monica']

This is a syntax error in bash — in what shell does it work?

[aamadeo]

I run this in bash. The < and > are not part of the command, the name of the file should replace "< File >"

[Andrew]

On Solaris, you need to use: /usr/xpg4/bin/tail (found this in the man page).

[cerd]

Most readily usable answer for both cli and scripting.

[springloaded]

This makes more sense if you don't know how long the file is and don't want to tell tail to print the last 100000000 lines.

[CSTobey]

awk 'NR > 6 {print}' is sufficient... no need for the if or the $0.

[Lon Kaut]

Think I discovered this by accident but both head and tail allow you to omit the -n as if it's implied. You can simply <command> | tail +2

[Tom]

better than tail imo, since we don't have to know the number of lines to be tail-ed. we just remove the 1st line and that's all

[Mike Pennington]

@springloaded if you need to know the number of lines in the file, ‘wc -l’ will easily give it to you

[CervEd]

@Tom you don't need to know the number tailed, to skip the first line use tail +2

[Tom]

good point indeed

[Joel Mellon]

@Lloeki Awesome! BSD head doesn't support negative numbers like GNU does, so I assumed tail didn't accept positives (with +) since that's sort of the opposite. Anyway, thanks.

[Joel Mellon]

Also, to clarify this answer: tail -n +2 huge-file.log would skip first line, and pick up on line 2. So to skip the first line, use +2. @saipraneeth's answer does a good job of exaplaining this.

[gabrielf]

Actually awk 'NR>6' is sufficient as print is the default action block :-) See linuxhandbook.com/awk-command-tutorial for a really good awk tutorial which explains this well.

[Drew Noakes]

This version works in the Cygwin tools that come with Git for Windows, whereas tail and sed do not. For example git -c color.status=always status -sb | awk 'NR > 1' gives a nice minimal status report without any branch information, which is useful when your shell already shows branch info in your prompt. I assign that command to alias gs which is really easy to type.