Chapter 8Multithreading and multiprocessing
Python has APIs that allow us to write both multithreading and multiprocessing
applications. The principle behind multithreading and multiprocessing is simply
to take copies of our code and run them in additional threads or processes. The
operating system automatically schedules the threads and processes across available
CPU cores to provide fair processing time allocation to all the threads and processes.
This effectively allows a program to simultaneously run multiple operations. In
addition, when a thread or process blocks, for example, when waiting for IO, the
thread or process can be de-prioritized by the OS, and the CPU cores can be allocated
to other threads or processes that have actual computation to do.
Here is an overview of how threads and processes relate to each other:
Threads exist within processes. A process can contain multiple threads but it always
contains at least one thread, sometimes called the main thread. Threads within
the same process share memory, so data transfer between threads is just a case of
referencing the shared objects. Processes do not share memory, so other interfaces,
such as files, sockets, or specially allocated areas of shared memory, must be used for
transferring data between processes.