Chapter 9Event-driven servers
Web client numbers continued to grow though, and the need arose for servers to
be able to handle very large numbers of simultaneous client connections, numbers
that proved problematic using the multiprocessing approaches. This spurred the
development of event-driven web servers, such as nginx and lighttpd, which can
handle many thousands of simultaneous connections in a single process. These
servers also leverage preforking, maintaining a number of event-driven processes
in line with the number of CPU cores in a machine, and hence making sure the
server's resources are fully utilized while also receiving the benefits of the
event-driven architecture.
WSGI
Python web applications were originally written against these early integration
protocols: CGI, FastCGI, and a now mostly defunct mod_python Apache module.
This proved troublesome though since Python web applications were tied to the
protocol or server they had been written for. Moving them to a different server or
protocol required some reworking of the application code.
This problem was solved with PEP 333, which defined the Web Services Gateway
Interface (WSGI) protocol. This established a common calling convention for web
servers to invoke web application code, similar to CGI. When web servers and web
applications both support WSGI, servers and applications can be exchanged with
ease. WSGI support has been added to many modern web servers and is nowadays
the main method of hosting Python applications on the Web. It was updated for
Python 3 in PEP 3333.
Many of the web frameworks we discussed earlier support WSGI behind the scenes
to communicate with their hosting web servers, Flask and Django included. This is
another big benefit to using such a framework— you get full WSGI compatibility
for free.
There are two ways a web server can use WSGI to host a web application. Firstly
it can directly support hosting WSGI applications. Pure Python servers such as
Gunicorn follow this approach, and they make serving Python web applications
very easy. This is becoming a very popular way to host Python web applications.