Re: read huge text file from end
"Eric Sosman" <Eric.Sosman@sun.com> wrote in message
news:1162329515.169868@news1nwk...
quickcur@yahoo.com wrote On 10/31/06 15:45,:
Hi,
I have very large text files and I am only interested in the last 200
lines in each file. How can I read a huge text file line by line from
the end, something line the "tail" command in Unix?
Do as "tail" does: Get the size of the file, seek to
a position (200 * average_line_length + safety_margin) bytes
before the end, and start reading. Be prepared for some
glitches if you land in the middle of a multi-byte sequence;
you may need to be tolerant of a malformed line and/or
character decoding errors when you start reading.
Of course, this simply isn't going to work for files
that contain statefully-encoded regions, or that have been
progressively compressed or encrypted. For "very large"
files, compression is distinctly likely -- even if you're
not using it now, you might want to ponder before committing
to a strategy that would prevent using it in the future.
Hopefully, the compression would be handled by the underlying OS, and it
would all work "transparently" to your application.
Otherwise, you're no longer dealing with text files (in the traditional
sense), and if you've got custom file formats, you could do tricks like
actually encode the offset of the 200th line from the end into the header.
- Oliver