Learn, Share, Build

260
October 05, 2017, at 4:17 PM

During for line in f:, my code save the lines who contain a specific data. Unfortunately, I have to read the entire file to be sure than it is the most resent data. In a second time, I have to check the entire file (between 5000-8000 lines) until I got the correct line several time (for each data).

So, my question is, it is possible to open a file and go to a specific line, read it and do it again. I saw different answer about it, but I can't save all the file in a str because I don't have so much RAM on my device ... That's why I want to search directly in the file.

Answer 1

Do it with iterators and generators, files xreadlines (python 2) it is lazily evaluated so the file is not been loaded into memory until you consume it:

def drop_and_get(skiping, it):
    for _ in xrange(skiping):
        next(it)
    return next(it)
f = xrange(10000)#lets say your file is this generator
drop_and_get(500, iter(f))
500

So you can actualy do something like:

with open(yourfile, "r") as f:
    your_line = drop_and_get(5000, f.xreadlines())
    print your_line

You can actually even skip xreadlines since the file object is an iterator itself

with open(yourfile, "r") as f:
    your_line = drop_and_get(5000, f)
    print your_line
Answer 2

Daniel solution is very good. A simpler alternative is to loop on the file handle and break when the required line is reached. Then you can resume the loop to actually process those lines.

Note that there's no miracle unless the size of the lines don't change (in which case you could memorize file position and seek to it afterwards): you have to read all file data from the start. You just don't need to store it to memory using readlines(). Never use readlines()

Here's my naive approach, doesn't use generator or complex stuff, but is as efficient, and simple:

# skip first 5000 lines
for i,line in enumerate(f):
    if i == 5000:
       break
# process the rest of the file
for line in f:
    print(line.rstrip())
Rent Charter Buses Company
READ ALSO
Learn, Share, Build

Learn, Share, Build

I have a function that I need to return back with the original question if there is text input rather than the number in the range given

232
Learn, Share, Build

Learn, Share, Build

I need to add string after a word in a specific line of a text fileThe original file is:

266
Learn, Share, Build

Learn, Share, Build

I'm trying to traverse an ftp directory using FTPwalker, however I cannot connect

347
Learn, Share, Build

Learn, Share, Build

I am trying to use the BeautifulSoup library in Python to extract the jpg image names from a html scriptIn the url wherever you find srcset it is always proceeded by a jpg file name

270