Chapter 7 ■ Server arChiteCture

123

There are two interesting technical details worth commenting on. One is the fact that the first recv() returns
immediately—it is only the second and third recv() calls that show a delay before returning data, as does the final
recv() before learning that the socket has been closed. This is because the operating system’s network stacks are
cleverly going ahead and including the text of the first request in the same three-way handshake that sets up the TCP
connection. Thus, by the time the connection officially exists and accept() can return, there is already data waiting
that can be returned immediately from recv()!
The other detail is that send() causes no delay. This is because its semantics on a POSIX system are that it returns
as soon as the outgoing data has been enrolled in the operating system network stack’s outgoing buffers. There is
never a guarantee that the system has really sent any data just because send() has returned! Only by turning around
and listening for more client data can the program force the operating system to block its progress and wait to see the
result of sending.
Let’s get back to the topic at hand. How can these limitations of a single-threaded server be overcome? The rest
of this chapter explores two competing techniques for preventing a single client from monopolizing a server. Both
techniques allow the server to talk to several clients at once. First, I will cover the use of threads (processes work too),
giving the operating system the job of switching the server’s attention between different clients. Then I will turn to
asynchronous server design, where I show how to handle the switches of attention yourself to converse with several
clients at once in a single thread of control.

Threaded and Multiprocess Servers

If you want your server to converse with several clients simultaneously, a popular solution is to leverage your
operating system’s built-in support for allowing several threads of control to proceed independently through the same
section of code, either by creating threads that share the same memory footprint or by creating processes that run
independently of one another.
The advantage of this approach is its simplicity: take the same code that runs your single-threaded server and
launch several copies of it.
Its disadvantage is that the number of clients to which you can talk is limited by how your operating system
concurrency mechanisms scale. Even an idle or slow client will occupy the attention of an entire thread or process,
which even if blocked in recv() will both occupy system RAM and a slot in the process table. Operating systems rarely
scale well to thousands or more threads running simultaneously, and the context switches required as the system’s
attention turns from one client to the next will begin to bog down your service as it becomes busy.
You might expect that a multithreaded or multiprocess server would need to be composed of a master thread
of control that runs a tight accept() loop that then hands off the new client sockets to some sort of waiting queue of
workers. Happily, the operating system makes things much easier on you: it is perfectly permissible for every thread
to have a copy of the listening server socket and to run its own accept() statement. The operating system will hand
each new client connection to whichever thread is waiting for its accept() to complete, or else keep the connection
queued if all the threads are currently busy until one of them is ready. Listing 7-4 shows an example.

Listing 7-4. Multithreaded Server

#!/usr/bin/env python3

Foundations of Python Network Programming, Third Edition

https://github.com/brandon-rhodes/fopnp/blob/m/py3/chapter07/srv_threaded.py

Using multiple threads to serve several clients in parallel.

import zen_utils
from threading import Thread