unix - head AND tail of file

Question

Say you have a txt file, what is the command to view the top 10 lines and bottom 10 lines of file simultaneously?

i.e. if the file is 200 lines long, then view lines 1-10 and 190-200 in one go.

Solution 1

You can simply:

(head; tail) < file.txt

And if you need to uses pipes for some reason then like this:

cat file.txt | (head; tail)

Note: will print duplicated lines if number of lines in file.txt is smaller than default lines of head + default lines of tail.

Solution 2

ed is the standard text editor

$ echo -e '1+10,$-10d\n%p' | ed -s file.txt

Solution 3

For a pure stream (e.g. output from a command), you can use 'tee' to fork the stream and send one stream to head and one to tail. This requires using either the '>( list )' feature of bash (+ /dev/fd/N):

( COMMAND | tee /dev/fd/3 | head ) 3> >( tail )

or using /dev/fd/N (or /dev/stderr) plus subshells with complicated redirection:

( ( seq 1 100 | tee /dev/fd/2 | head 1>&3 ) 2>&1 | tail ) 3>&1
( ( seq 1 100 | tee /dev/stderr | head 1>&3 ) 2>&1 | tail ) 3>&1

(Neither of these will work in csh or tcsh.)

For something with a little better control, you can use this perl command:

COMMAND | perl -e 'my $size = 10; my @buf = (); while (<>) { print if $. <= $size; push(@buf, $_); if ( @buf > $size ) { shift(@buf); } } print "------\n"; print @buf;'

Solution 4

(sed -u 10q; echo ...; tail) < file.txt

Just another variation on the (head;tail) theme, but avoiding the initial buffer fill issue for small files.

Solution 5

It took make a lot of time to end-up with this solution which, seems to be the only one that covered all use cases (so far):

command | tee full.log | stdbuf -i0 -o0 -e0 awk -v offset=${MAX_LINES:-200} \
          '{
               if (NR <= offset) print;
               else {
                   a[NR] = $0;
                   delete a[NR-offset];
                   printf "." > "/dev/stderr"
                   }
           }
           END {
             print "" > "/dev/stderr";
             for(i=NR-offset+1 > offset ? NR-offset+1: offset+1 ;i<=NR;i++)
             { print a[i]}
           }'

Feature list:

• live output for head (obviously that for tail is not possible)
• no use of external files
• progressbar one dot for each line after the MAX_LINES, very useful for long running tasks.
• progressbar on stderr, assuring that the progress dots are separated from the head+tail (very handy if you want to pipe stdout)
• avoids possible incorrect logging order due to buffering (stdbuf)
• avoid duplicating output when total number of lines is smaller than head + tail.

View more solutions

Comments

[cnicutar]

What do you mean "in one go" ?

[toop]

@cnicutar ie. not going head -10 file looking at the data and then separately going tail -10 file and looking at the data

[sorin]

@toop If you want a real working example, see stackoverflow.com/a/44849814/99834

[Paul]

What if the file has more or less than 200 lines? And you don't know the number of lines ab initio?

[Paul]

Nice, I have always used cat and head or tail piped, good to know that I can use them individually!

[toop]

How can I then pipe these first 10+last 10 into another command?

[Paul]

head file.txt; tail file.txt | your_program

[toop]

@Paul - with 'your_program' as wc -l it returns 10 instead of 20

[Paul]

(head .vimrc ; tail .vimrc) | wc -l

[kev]

@Paul I've changed sed to ed

[Samus_]

it needs more work in order to avoid issues when offset is larger than the file

[jstarek]

Why use cat when you can just call head -10 file.txt?

[glenn jackman]

or, without having to spawn a subshell: { head file; tail file; } | prog (spacing inside the braces, and the trailing semicolon are required)

[chepner]

Strictly speaking, this doesn't give you the tail of the original file, but the tail of the stream after head has consumed the first 10 lines of the file. (Compare this with head < file.txt; tail < file.txt on a file with fewer than 20 lines). Just a very minor point to keep in mind. (But still +1.)

[Simon Hibbs]

Nice. If you want a gap between the head and tail parts: (head;echo;tail) < file.txt

[ricardo]

Can you make the number of lines variable, so the call is something like: head_ tail(foo, m,n) - returning the first m snd last n lines of text?

[Paul]

@ricardo that would involve writing a bash script that takes 3 args and passes them to tail and head or a function by alias-ing it.

[Kevin]

This works for files of known length, but not files whose length is unknown.

[zellyn]

Curious about why/how this works. Asked it as a new question: stackoverflow.com/questions/13718242

[Camille Goudeseune]

Yay, this works with piped output, not just files: a.out | awk -v ...

[Samus_]

indeed :) but that's awk's normal behavior, most commandline programs work on stdin when invoked without arguments.

[mah]

Wow... a down-vote for having an answer quite similar to others (yet timestamped before them) after almost two years, from someone who chose not to post why they down-voted. Nice!

[jfs]

+1 for stream support. You could reuse stderr: COMMAND | { tee >(head >&2) | tail; } |& other_commands

[jfs]

@chepner: { tee >(head >&2) | tail; } 2>&1 < file.txt instead of { head; tail; } < file.txt supports overlapping lines if you need it. {} instead of () are to avoid creating a subshell.

[jfs]

btw, it breaks for files larger than the buffer size (8K on my system). cat >/dev/null fixes it: COMMAND | { tee >(head >&2; cat >/dev/null) | tail; } |& other_commands

[fedorqui]

Would be best to post the relevant parts of the code.

[nametal]

addition to @chepner 's comment: (head;less) < file.txt will show the remaining lines (line 11+) after head

[chepner]

@nametal Actually, you might not even get that much. While head only displays the first 10 lines of its input, there is no guaranteed that it didn't consume more of it in order to find the 10th line ending, leaving less of the input for less to display.

[modular]

Sorry to say, but the answer only works in some cases. seq 100 | (head; tail) gives me only first 10 numbers. Only on much larger input size (like seq 2000) the tail gets some input.

[where23]

I use it to check time from log file. (head -n1; tail -n1) < file.txt

[plhn]

Maybe this is possible : cat file.txt | (head; tail)

[Josiah Yoder]

@alkar @J.F.Sebastian seq 100 | { tee >(head >&2) | tail; } 2>&1; also works to avoid the problem that we don't know how much head consumes from the stream

[sorin]

Very close to the desired behaviour but it seems that for <10 lines it does add extra new lines.

[sorin]

If you want to avoid duplication of lines see stackoverflow.com/a/44849814/99834

[Alexej Magura]

@chepner I get the first and last line with (head -n1; tail -1): cat FILE | (head -n1; tail -1). What system are you running these on/are these the GNU versions of the above commands?

[Alexej Magura]

@chepner Note: I still get the actual first and last line even with (head; tail) < FILE. I'm using GNU coreutils version 8.4

[bennythejudge]

this is not working for me on a 89 lines text file on macos and I have tried both the native head/tail and also ghead and gtail. Only head is printing some text.

[Jan]

I loved the solution, but after playing for a a while I noticed that in some cases the tail was running before the head ... there are no guaranteed ordering between head and tail commands :\ ...

[haui]

Note: The buffering issue (head swallows to much from the input stream on small files so that no tail is printed) is fixed by an answer below: cat file.txt | (sed -u 10q ; echo ; tail)

[Mat M]

Basically, it uses collections.deque to simulate tail

[jamesbtate]

This is the best answer here. I added it to my .bashrc in a function so I can just pipe to headtail.

[Admin]

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

[steveb]

This solution seems to sometimes work and sometimes not. So far the sed -u solution mentioned in the comments here and in one of the answers, appears to work for me.