of a mutable message cache copied by pickling it to pass to a new process, for example,
has no effect on the original.
The pickleability constraints for process arguments on Windows can limit
multiprocessing’s scope in other contexts as well. For instance, in Chapter 12, we’ll
find that this module doesn’t directly solve the lack of portability for the os.fork call
for traditionally coded socket servers on Windows, because connected sockets are not
pickled correctly when passed into a new process created by this module to converse
with a client. In this context, threads provide a more portable and likely more efficient
solution.
Applications that pass simpler types of messages, of course, may fare better. Message
constraints are easier to accommodate when they are part of an initial process-based
design. Moreover, other tools in this module, such as its managers and shared memory
API, while narrowly focused and not as general as shared thread state, offer additional
mutable state options for some programs.
Fundamentally, though, because multiprocessing is based on separate processes, it
may be best geared for tasks which are relatively independent, do not share mutable
object state freely, and can make do with the message passing and shared memory tools
provided by this module. This includes many applications, but this module is not nec-
essarily a direct replacement for every threaded program, and it is not an alternative to
process forks in all contexts.
To truly understand both this module package’s benefits, as well as its tradeoffs, let’s
turn to a first example and explore this package’s implementation along the way.
The Basics: Processes and Locks
We don’t have space to do full justice to this sophisticated module in this book; see its
coverage in the Python library manual for the full story. But as a brief introduction, by
design most of this module’s interfaces mirror the threading and queue modules we’ve
already met, so they should already seem familiar. For example, the multiprocessing
module’s Process class is intended to mimic the threading module’s Thread class we
met earlier—it allows us to launch a function call in parallel with the calling script;
with this module, though, the function runs in a process instead of a thread. Exam-
ple 5-29 illustrates these basics in action:
Example 5-29. PP4E\System\Processes\multi1.py
"""
multiprocess basics: Process works like threading.Thread, but
runs function call in parallel in a process instead of a thread;
locks can be used to synchronize, e.g. prints on some platforms;
starts new interpreter on windows, forks a new process on unix;
"""
import os
from multiprocessing import Process, Lock
The multiprocessing Module| 245