The scanner could similarly be improved. As a rule of thumb, we can also usually speed
things up by shifting processing from Python code to built-in tools. For instance, if
we’re concerned with speed, we can probably make our file scanner faster by using the
file’s line iterator to step through the file instead of the manual readline loop in Ex-
ample 4-1 (though you’d have to time this with your Python to be sure):
def scanner(name, function):
for line in open(name, 'r'): # scan line by line
function(line) # call a function objectAnd we can work more magic in Example 4-1 with the iteration tools like the map built-
in function, the list comprehension expression, and the generator expression. Here are
three minimalist’s versions; the for loop is replaced by map or a comprehension, and
we let Python close the file for us when it is garbage collected or the script exits (these
all build a temporary list of results along the way to run through their iterations, but
this overhead is likely trivial for all but the largest of files):
def scanner(name, function):
list(map(function, open(name, 'r')))def scanner(name, function):
[function(line) for line in open(name, 'r')]def scanner(name, function):
list(function(line) for line in open(name, 'r'))File filters
The preceding works as planned, but what if we also want to change a file while scanning
it? Example 4-3 shows two approaches: one uses explicit files, and the other uses the
standard input/output streams to allow for redirection on the command line.
Example 4-3. PP4E\System\Filetools\filters.py
import sys
def filter_files(name, function): # filter file through function
input = open(name, 'r') # create file objects
output = open(name + '.out', 'w') # explicit output file too
for line in input:
output.write(function(line)) # write the modified line
input.close()
output.close() # output has a '.out' suffix
def filter_stream(function): # no explicit files
while True: # use standard streams
line = sys.stdin.readline() # or: input()
if not line: break
print(function(line), end='') # or: sys.stdout.write()
if name == 'main':
filter_stream(lambda line: line) # copy stdin to stdout if run
162 | Chapter 4: File and Directory Tools