Reading the last n lines of a file in Ruby?
Solution 1
If on a *nix system with tail
, you can cheat like this:
last_25_lines = `tail -n 25 whatever.txt`
Solution 2
Is the file large enough that you need to avoid reading the whole thing? If not, you could just do
IO.readlines("file.log")[-25..-1]
If it is to big, you may need to use IO#seek
to read from near the end of the file, and continue seeking toward the beginning until you've seen 25 lines.
Solution 3
There is a library for Ruby called File::Tail. This can get you the last N lines of a file just like the UNIX tail utility.
I assume there is some seek optimization in place in the UNIX version of tail with benchmarks like these (tested on a text file just over 11M):
[john@awesome]$du -sh 11M.txt
11M 11M.txt
[john@awesome]$time tail -n 25 11M.txt
/sbin/ypbind
/sbin/arptables
/sbin/arptables-save
/sbin/change_console
/sbin/mount.vmhgfs
/misc
/csait
/csait/course
/.autofsck
/~
/usb
/cdrom
/homebk
/staff
/staff/faculty
/staff/faculty/darlinr
/staff/csadm
/staff/csadm/service_monitor.sh
/staff/csadm/.bash_history
/staff/csadm/mysql5
/staff/csadm/mysql5/MySQL-server-community-5.0.45-0.rhel5.i386.rpm
/staff/csadm/glibc-common-2.3.4-2.39.i386.rpm
/staff/csadm/glibc-2.3.4-2.39.i386.rpm
/staff/csadm/csunixdb.tgz
/staff/csadm/glibc-headers-2.3.4-2.39.i386.rpm
real 0m0.012s
user 0m0.000s
sys 0m0.010s
I can only imagine the Ruby library uses a similar method.
Edit:
for Pax's curiosity:
[john@awesome]$time cat 11M.txt | tail -n 25
/sbin/ypbind
/sbin/arptables
/sbin/arptables-save
/sbin/change_console
/sbin/mount.vmhgfs
/misc
/csait
/csait/course
/.autofsck
/~
/usb
/cdrom
/homebk
/staff
/staff/faculty
/staff/faculty/darlinr
/staff/csadm
/staff/csadm/service_monitor.sh
/staff/csadm/.bash_history
/staff/csadm/mysql5
/staff/csadm/mysql5/MySQL-server-community-5.0.45-0.rhel5.i386.rpm
/staff/csadm/glibc-common-2.3.4-2.39.i386.rpm
/staff/csadm/glibc-2.3.4-2.39.i386.rpm
/staff/csadm/csunixdb.tgz
/staff/csadm/glibc-headers-2.3.4-2.39.i386.rpm
real 0m0.350s
user 0m0.000s
sys 0m0.130s
still under a second, but if there is a lot of file operations this makes a big difference.
Solution 4
Improved version of manveru's excellent seek-based solution. This one returns exactly n lines.
class File
def tail(n)
buffer = 1024
idx = [size - buffer, 0].min
chunks = []
lines = 0
begin
seek(idx)
chunk = read(buffer)
lines += chunk.count("\n")
chunks.unshift chunk
idx -= buffer
end while lines < ( n + 1 ) && pos != 0
tail_of_file = chunks.join('')
ary = tail_of_file.split(/\n/)
lines_to_return = ary[ ary.size - n, ary.size - 1 ]
end
end
Solution 5
I just wrote a quick implemenation with #seek
:
class File
def tail(n)
buffer = 1024
idx = (size - buffer).abs
chunks = []
lines = 0
begin
seek(idx)
chunk = read(buffer)
lines += chunk.count("\n")
chunks.unshift chunk
idx -= buffer
end while lines < n && pos != 0
chunks.join.lines.reverse_each.take(n).reverse.join
end
end
File.open('rpn-calculator.rb') do |f|
p f.tail(10)
end
Comments
-
thedeepfield over 3 years
I need to read the last 25 lines from a file (for displaying the most recent log entries). Is there anyway in Ruby to start at the end of a file and read it backwards?