How do I open a large file in a terminal (using vim causes terminal to hang)?
Solution 1
When dealing with large files, it's better to minimize the amount of information you're going to deal with, and there are tools for that. One tool that has been mentioned is grep
to search for specific lines that you need. Another two tools that I'd recommend using are tail
and head
that can display specific number of lines or bytes from top of the file ( head
) or from the end ( tail
). This is useful if you only need to read those specific parts of the file, but nothing in between.
Alternative way is to use split
command. It can break down a file based on number of bytes or lines. For example,
$ mkdir SPLIT
$ cd SPLIT/
$ split -l 5 /etc/passwd
$ ls
xaa xab xac xad xae xaf xag xah xai xaj
Now, my /etc/password
is broken down into smaller files, 5 lines each, which you can open with vim or other text editor
Solution 2
Use the command less
. less
uses a more efficient way of reading a file into memory. There are 2 equivalent commands: more
and most
(that last one you need to install). Those put the single page into memory.
But your BEST option for searching would be to use grep
. Example:
grep {searchstring} {file}
And have a look at logrotate
so you span that log over multiple files.
And split
can split your current file into equal parts so you have an easier life looking at that log.
Solution 3
It depends on what your goal is.
If you want to scroll through the entire logfile, you can use the less
or more
commands to get a page-by-page view of the file at hand.
Otherwise, if you know where in the file you're looking for, you can use the head
or tail
commands to grab a section/excerpt of the file. See man head
and man tail
for more information as to how to search for exact parts.
If you're looking for a specific string/entry, you can use grep
to search the file for your string. You can either use cat file | grep something
or grep something file
depending on what you want to do.
Alternatively, if you want to do some more advanced things, you could look at using sed
or awk
for more advanced operations, if need be.
Related videos on Youtube
Dave
Updated on September 18, 2022Comments
-
Dave over 1 year
I'm using Ubuntu 14.04. I have a large log file:
-rw-r--r-- 1 rails root 1792390124 Jan 10 14:54 /var/log/unicorn/unicorn.log
Nonetheless, I need to open it and find some things in it. I tried vim, but running
vim /var/log/unicorn/unicorn.log
just opens a blank screen and even when I use Ctrl+C, nothing happens. The terminal is hung. Without upgrading any of the hardware on my machine (but I'm open to installing software), how can I open the file and look around (I don't need to edit it, I just need to read some lines from it).
-
pLumo about 7 yearsFinding things in log files is best done using grep. -A and -B might help here.
-
Eliah Kagan about 7 yearsWorking with huge files in linux (on Stack Overflow) may help, too.
-
-
Dave about 7 yearsI am looking for a place in teh file but the reason (I think) grep alone isn't good enough is because I'm looking for a string sequence (beginning with "Http") that occurs before another string sequence (beginning with "120"). I'm not clear on how to do that with grep alone.
-
Sergiy Kolodyazhnyy about 7 years@Dave well, if you need to find a specific string in file, post another question, explaining exactly what the pattern is , and I'm sure people can provide solutions to that. This question is in different context , I'd say, so new post would be better
-
Dave about 7 yearsHey you were the one who was asking about why I wanted to open the large text file. I'm happy to keep this question in its original context.
-
Eliah Kagan about 7 yearsLike
less
(but unlikemore
),most
supports scrolling back as well as forward. It just uses different keys to scroll back one page:less
uses B (for back), whilemost
uses U (for up). In most terminals, the Page Up key fine in both, though. (Inmost
, B scrolls all the way to the bottom; and inless
, U scrolls up... but only by half a screen.) Fortunately, bothless
andmost
accept/
for searching forward,?
for searching backward, and H for help, so users accustomed to one can usually use the other with ease. -
Serrano Pereira over 3 yearsOnce done editing, you can join them again using
cat xa* > out
.