And Much More
Finally, multiprocessing provides many more tools than these examples deploy, in-
cluding condition, event, and semaphore synchronization tools, and local and remote
managers that implement servers for shared object. For instance, Example 5-34 dem-
onstrates its support for pools—spawned children that work in concert on a given task.
Example 5-34. PP4E\System\Processes\multi6.py
"Plus much more: process pools, managers, locks, condition,..."
import os
from multiprocessing import Pool
def powers(x):
#print(os.getpid()) # enable to watch children
return 2 ** x
if name == 'main':
workers = Pool(processes=5)
results = workers.map(powers, [2]*100)
print(results[:16])
print(results[-2:])
results = workers.map(powers, range(100))
print(results[:16])
print(results[-2:])
When run, Python arranges to delegate portions of the task to workers run in parallel:
C:\...\PP4E\System\Processes> multi6.py
[4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4]
[4, 4]
[1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768]
[316912650057057350374175801344, 633825300114114700748351602688]And a little less...
To be fair, besides such additional features and tools, multiprocessing also comes with
additional constraints beyond those we’ve already covered (pickleability, mutable state,
and so on). For example, consider the following sort of code:
def action(arg1, arg2):
print(arg1, arg2)if __name__ == '__main__':
Process(target=action, args=('spam', 'eggs')).start() # shell waits for childThis works as expected, but if we change the last line to the following it fails on Win-
dows because lambdas are not pickleable (really, not importable):
Process(target=(lambda: action('spam', 'eggs'))).start() # fails!-not pickleable256 | Chapter 5: Parallel System Tools