Best tool in unix for viewing large files

Question

I am a novice in unix. I am facing a problem in viewing big log files in unix using vi. Could you please suggest the best tool for fast viewing of big files on unix? Additionally, could you to post your own ways of viewing the big files on unix?

Solution 1

less doesn't need to keep the whole file in memory so it is good for viewing giant files. But for log files, the line wrapping is a nuisance.

Solution 2

It depends on what you are looking for in this big log file.

• If you just want to be impressed by its size, cat is enough (you can also roughly locate some unexpected visual patterns).
• If you just want to take a look at it, you can use more or less.
• If you want to monitor it while it is growing, you may be interested by tail -f.
• If you are looking for specific patterns, take a look at grep.
• If you want to extract some useful information from your big data, perl or python are your friends.

Solution 3

Actually vi (at least vim) is very performant on large files. I regularly use it to edit files in the dozens of MB range without problems.

You just need to be aware that a few operations will be slow on large files: big visual selects, global searching, and syntax highlighting. For large files, always turn off syntax highlighting (if you have it on by default): :syn off . Then you should be fine.

Solution 4

Use less as they have already told you, or most which is an extended version of less with more options and cool stuff.

It is usually included in the repository of any linux distro.

View more solutions

Comments

[fresskoma]

Which you can disable with "-S" ;)

[mathStudent001]

+1 for less. And for viewing log files that keep changing, you can press shift-f, which will emulate a 'tail -f' on the file. And as a bonus, if you had highlighted something with a search, the new incoming text will also be highlighted.

[SourceSeeker]

+1 for most and its multiple windows (although I mostly use less).

[dalloliogm]

thanks :) I don't understand why the negative vote, maybe someone tought I was pulling the leg :-)

[warren]

I prefer less <filename> then Shift-F to follow, personally, over tail -f :)