How do I open a large file in a terminal (using vim causes terminal to hang)?

11,642

Solution 1

When dealing with large files, it's better to minimize the amount of information you're going to deal with, and there are tools for that. One tool that has been mentioned is grep to search for specific lines that you need. Another two tools that I'd recommend using are tail and head that can display specific number of lines or bytes from top of the file ( head ) or from the end ( tail ). This is useful if you only need to read those specific parts of the file, but nothing in between.

Alternative way is to use split command. It can break down a file based on number of bytes or lines. For example,

$ mkdir SPLIT

$ cd SPLIT/                                                                                                       

$ split -l 5  /etc/passwd

$ ls
xaa  xab  xac  xad  xae  xaf  xag  xah  xai  xaj

Now, my /etc/password is broken down into smaller files, 5 lines each, which you can open with vim or other text editor

Solution 2

Use the command less. less uses a more efficient way of reading a file into memory. There are 2 equivalent commands: more and most (that last one you need to install). Those put the single page into memory.

But your BEST option for searching would be to use grep. Example:

grep {searchstring} {file}

And have a look at logrotate so you span that log over multiple files.

And split can split your current file into equal parts so you have an easier life looking at that log.

Solution 3

It depends on what your goal is.

If you want to scroll through the entire logfile, you can use the less or more commands to get a page-by-page view of the file at hand.

Otherwise, if you know where in the file you're looking for, you can use the head or tail commands to grab a section/excerpt of the file. See man head and man tail for more information as to how to search for exact parts.

If you're looking for a specific string/entry, you can use grep to search the file for your string. You can either use cat file | grep something or grep something file depending on what you want to do.

Alternatively, if you want to do some more advanced things, you could look at using sed or awk for more advanced operations, if need be.

Share:
11,642

Related videos on Youtube

Dave
Author by

Dave

Updated on September 18, 2022

Comments

  • Dave
    Dave over 1 year

    I'm using Ubuntu 14.04. I have a large log file:

    -rw-r--r-- 1 rails root 1792390124 Jan 10 14:54 /var/log/unicorn/unicorn.log
    

    Nonetheless, I need to open it and find some things in it. I tried vim, but running

    vim /var/log/unicorn/unicorn.log
    

    just opens a blank screen and even when I use Ctrl+C, nothing happens. The terminal is hung. Without upgrading any of the hardware on my machine (but I'm open to installing software), how can I open the file and look around (I don't need to edit it, I just need to read some lines from it).

    • pLumo
      pLumo about 7 years
      Finding things in log files is best done using grep. -A and -B might help here.
    • Eliah Kagan
      Eliah Kagan about 7 years
      Working with huge files in linux (on Stack Overflow) may help, too.
  • Dave
    Dave about 7 years
    I am looking for a place in teh file but the reason (I think) grep alone isn't good enough is because I'm looking for a string sequence (beginning with "Http") that occurs before another string sequence (beginning with "120"). I'm not clear on how to do that with grep alone.
  • Sergiy Kolodyazhnyy
    Sergiy Kolodyazhnyy about 7 years
    @Dave well, if you need to find a specific string in file, post another question, explaining exactly what the pattern is , and I'm sure people can provide solutions to that. This question is in different context , I'd say, so new post would be better
  • Dave
    Dave about 7 years
    Hey you were the one who was asking about why I wanted to open the large text file. I'm happy to keep this question in its original context.
  • Eliah Kagan
    Eliah Kagan about 7 years
    Like less (but unlike more), most supports scrolling back as well as forward. It just uses different keys to scroll back one page: less uses B (for back), while most uses U (for up). In most terminals, the Page Up key fine in both, though. (In most, B scrolls all the way to the bottom; and in less, U scrolls up... but only by half a screen.) Fortunately, both less and most accept / for searching forward, ? for searching backward, and H for help, so users accustomed to one can usually use the other with ease.
  • Serrano Pereira
    Serrano Pereira over 3 years
    Once done editing, you can join them again using cat xa* > out.