putting the pieces back together to re-create the original file (cat). Because we had all
sorts of different machines in the house, though, we needed a more portable solution.†
Splitting Files Portably
Since all the computers in my house ran Python, a simple portable Python script came
to the rescue. The Python program in Example 6-5 distributes a single file’s contents
among a set of part files and stores those part files in a directory.
Example 6-5. PP4E\System\Filetools\split.py
#!/usr/bin/python
"""
################################################################################
split a file into a set of parts; join.py puts them back together;
this is a customizable version of the standard Unix split command-line
utility; because it is written in Python, it also works on Windows and
can be easily modified; because it exports a function, its logic can
also be imported and reused in other applications;
################################################################################
"""
import sys, os
kilobytes = 1024
megabytes = kilobytes 1000
chunksize = int(1.4 megabytes) # default: roughly a floppy
def split(fromfile, todir, chunksize=chunksize):
if not os.path.exists(todir): # caller handles errors
os.mkdir(todir) # make dir, read/write parts
else:
for fname in os.listdir(todir): # delete any existing files
os.remove(os.path.join(todir, fname))
partnum = 0
input = open(fromfile, 'rb') # binary: no decode, endline
while True: # eof=empty string from read
chunk = input.read(chunksize) # get next part <= chunksize
if not chunk: break
partnum += 1
filename = os.path.join(todir, ('part%04d' % partnum))
fileobj = open(filename, 'wb')
fileobj.write(chunk)
fileobj.close() # or simply open().write()
input.close()
assert partnum <= 9999 # join sort fails if 5 digits
return partnum
† I should note that this background story stems from the second edition of this book, written in 2000. Some
ten years later, floppies have largely gone the way of the parallel port and the dinosaur. Moreover, burning
a CD or DVD is no longer as painful as it once was; there are new options today such as large flash memory
cards, wireless home networks, and simple email; and naturally, my home computers configuration isn’t
what it once was. For that matter, some of my kids are no longer kids (though they’ve retained some backward
compatibility with their former selves).
Splitting and Joining Files | 283