115Chapter 7
Server Architecture
There are two challenges facing the author of a network service. The first is the core challenge of writing code that
will correctly respond to incoming requests and craft the appropriate responses. The second is the task of installing
this network code inside a Windows service or Unix daemon that starts automatically when the system boots, logs
its activity to a persistent store, raises an alert if it cannot connect to its database or back-end data store, and either
protects itself completely against all possible failure modes or can be quickly restarted should it fail.
This book focuses on the first of these two challenges. Not only is the second challenge, that of keeping a process
up and running on your operating system of choice, a topic to which an entire book could be dedicated, but it is one
that would take this book far afield from its central topic of network programming. This chapter, therefore, will spend
only one section introducing the topic of deployment before moving on to its real topic of how network servers can be
crafted as pieces of software.
Our treatment of network servers will then fall naturally into three topics. I will first cover a simple single-threaded
server, similar to the UDP servers (covered in Chapter 2) and TCP servers (covered in Chapter 3), and focus on its
limitations: it can serve only one client at a time, making any other clients wait, and even when talking to that client,
it will probably keep the system CPU almost entirely idle. Once you understand this challenge, you will proceed to
study the two competing solutions: either duplicating the single-threaded server in multiple threads or processes or
taking the duty of multiplexing away from the operating system and doing it in your own code by using asynchronous
network operations.
While studying threaded versus asynchronous network code, you will first implement each pattern from the
ground up, and then you will look at frameworks that implement each pattern on your behalf. All of the frameworks
that I illustrate will be from the Python Standard Library, but the text will also point out major third-party competitors
to the Standard Library where they exist.
Most of the scripts in this chapter can also run under Python 2, but the most advanced framework introduced—the
new asyncio module—is specific to Python 3, and it is a big advance in standardization that can be enjoyed only by
programmers ready to make the upgrade.
A Few Words About Deployment
You will deploy a network service either to a single machine or to several machines. Clients can use a service that
lives on a single machine by simply connecting to its IP address. A service running on several machines requires a
more complicated approach. You could give each client the address or hostname of a single instance of the service,
for example the instance that is running in the same machine room as a particular client, but you will gain no
redundancy. If that instance of the service goes down, then the clients that were hardwired to its hostname or IP
address will fail to connect.
A more robust approach is to have your DNS server return every IP address at which the service lives when its
name is accessed and write clients that fall back to the second or third IP address that they are given if the first one fails.
The approach that scales best in the industry today is to place your services behind a load balancer to which clients
connect directly and that then forwards each incoming connection to an actual server sitting behind it. If a server fails,