Chapter 7 ■ Server arChiteCture
128
- When the client socket itself is then presented to you with a POLLIN event, you recv() up to
4KB of data. If the request is not yet framed with a concluding question mark, then you save
the data to the bytes_received dictionary and continue back to the top of the loop to poll()
further. Otherwise, you have a complete question, and you can act on the client’s request by
looking up the corresponding reply and putting it in your bytes_to_send dictionary. This
involves a crucial pivot: switching the socket from POLLIN mode, where you want to know
when more data arrives, to POLLOUT mode, where you want to be notified as soon as the
outgoing buffers are free because you are now using the socket to send instead of receive. - The poll() call now notifies you immediately with POLLOUT whenever the outgoing buffers
on the client socket can accept at least one byte, and you respond by attempting a send()
of everything you have left to transmit and by keeping only the bytes that send() could not
squeeze into the outgoing buffers. - Finally, a POLLOUT arrives whose send() lets you complete the transmission of all
remaining outward-bound data. At this point, a request-response cycle is complete,
and you pivot the socket back into POLLIN mode for another request. - When a client socket finally gives you an error or closing status, you dispose of it and any
outgoing or incoming buffers. Of all the simultaneous conversations you may be having,
that one, at least, is now complete.
The key to the asynchronous approach is that this single thread of control can handle hundreds, or eventually
thousands, of client conversations. As each client socket becomes ready for its next event, the code steps forward into
that socket’s next operation, receives or sends what data it can, and then immediately returns to poll() to watch for
more activity. Without requiring a single operating system context switch (aside from the privilege-mode escalations
and de-escalations involved in entering the operating system itself for the poll(), recv(), send(), and close()
system calls), this single thread of control can handle a large number of clients by keeping all client-conversation
states in one set of dictionaries, indexed by client socket. Essentially, you substitute the key lookup supported by
Python dictionaries for the full-fledged operating system context-switch that a multithreaded or multiprocess server
would require to switch its attention from one client to another.
Technically, the previous code can run correctly even without setting every new client socket to nonblocking
mode with sock.setblocking(False). Why? Because Listing 7-6 never calls recv() unless there is waiting data, and
recv() never blocks if at least one byte of input is ready; and it never calls send() unless data can be transmitted, and
send() never blocks if at least one byte can be written to the operating system’s outgoing network buffers. But the
setblocking() call is prudent anyway in case you make an error. In its absence, a misplaced call to send() or recv()
would block and make you unresponsive to all but the one client on which you were blocked. With the setblocking()
call in place, a mix-up on your part will raise socket.timeout and alert you to the fact that you have somehow
managed to make a call that cannot be immediately acted upon by the operating system.
If you unleash several clients against this server, you will see that its single thread juggles all of the simultaneous
conversations with great aplomb. But you had to dive into quite a few operating system internals with Listing 7-6. What if
you want to focus on your client code and let someone else worry about the details of select(), poll(), or epoll()?
Callback-Style asyncio
Python 3.4 introduced the new asyncio framework to the Standard Library, designed in part by Python inventor
Guido van Rossum. It provides a standard interface for event loops based on select(), epoll(), and similar
mechanisms in an attempt to unify a field that had become fragmented in the era of Python 2.
After considering Listing 7-6 and noticing how little of its code is specific to the sample question-and-answer
protocol that you are studying in this chapter, you can probably already imagine the responsibilities that such a
framework undertakes. It maintains a central select-style loop. It keeps a table of sockets on which I/O activity is
expected, and it adds or removes them from the attention of the select loop as necessary. It cleans up and abandons the
sockets once they closed. Finally, when actual data has arrived, it defers to user code to determine the correct response.