Reading lines with file iterators
In older versions of Python, the traditional way to read a file line by line in a for loop
was to read the file into a list that could be stepped through as usual:
>>> file = open('data.txt')
>>> for line in file.readlines(): # DON'T DO THIS ANYMORE!
... print(line, end='')If you’ve already studied the core language using a first book like Learning Python, you
may already know that this coding pattern is actually more work than is needed today—
both for you and your computer’s memory. In recent Pythons, the file object includes
an iterator which is smart enough to grab just one line per request in all iteration con-
texts, including for loops and list comprehensions. The practical benefit of this exten-
sion is that you no longer need to call readlines in a for loop to scan line by line—the
iterator reads lines on request automatically:
>>> file = open('data.txt')
>>> for line in file: # no need to call readlines
... print(line, end='') # iterator reads next line each time
...
Hello file world!
Bye file world.Better still, you can open the file in the loop statement itself, as a temporary which will
be automatically closed on garbage collection when the loop ends (that’s normally the
file’s sole reference):
>>> for line in open('data.txt'): # even shorter: temporary file object
... print(line, end='') # auto-closed when garbage collected
...
Hello file world!
Bye file world.Moreover, this file line-iterator form does not load the entire file into a line’s list all at
once, so it will be more space efficient for large text files. Because of that, this is the
prescribed way to read line by line today. If you want to see what really happens inside
the for loop, you can use the iterator manually; it’s just a next method (run by the
next built-in function), which is similar to calling the readline method each time
through, except that read methods return an empty string at end-of-file (EOF) and the
iterator raises an exception to end the iteration:
>>> file = open('data.txt') # read methods: empty at EOF
>>> file.readline()
'Hello file world!\n'
>>> file.readline()
'Bye file world.\n'
>>> file.readline()
''>>> file = open('data.txt') # iterators: exception at EOF
>>> file.__next__() # no need to call iter(file) first,
'Hello file world!\n' # since files are their own iteratorFile Tools | 143