Chapter 3 ■ tCp53But how, then, are network clients and servers supposed to process large amounts of data without entering
deadlock? There are, in fact, two possible answers. First, they can use socket options to turn off blocking so that calls
like send() and recv() return immediately if they find that they cannot send any data yet. You will learn more about
this option in Chapter 7, where you will look in earnest at the possible ways to architect network server programs.
Or the programs can use one of several techniques to process data from several inputs at a time, either by splitting
into separate threads or processes (one tasked with sending data into a socket, perhaps, and another tasked with reading
data back out) or by running operating system calls such as select() or poll() that let them wait on busy outgoing and
incoming sockets at the same time and respond to whichever is ready. These are also explored in Chapter 7.
Finally, note carefully that the foregoing scenario cannot ever happen when you are using UDP. This is because
UDP does not implement flow control. If more datagrams are arriving than can be processed, then UDP can simply
discard some of them and leave it up to the application to discover that they went missing.
Closed Connections, Half-Open Connections
There are two more points that should be made, on a different subject, from the foregoing example.
First, Listing 3-2 shows you how a Python socket object behaves when an end-of-file is reached. Just as a Python
file object returns an empty string upon a read() when there is no more data left, a socket simply returns an empty
string when the socket is closed.
I never worried about this in Listing 3-1, because in that case I had imposed a strict enough structure on the
protocol—exchanging a pair of messages of exactly 16 bytes—that I did not need to close the socket to signal when
communication was done. The client and server could send a message while lazily leaving the socket open and close
their sockets later without worrying that anyone was hanging waiting on them to close.
But in Listing 3-2, the client sends—and thus the server also processes and sends back—an arbitrary amount
of data whose length is decided only by the number the user enters on the command line. And so you can see in the
code, twice, the same pattern: a while loop that runs until it finally sees an empty string returned from recv(). Note
that this normal Pythonic pattern will not work once you reach Chapter 7 and explore nonblocking sockets, where
recv() might raise an exception simply because no data is available at the moment. In that case, other techniques
are used to determine whether the socket has closed.
Second, you will see that the client makes a shutdown() call on the socket after it finishes sending its
transmission. This solves an important problem. If the server is going to read forever until it sees end-of-file, then how
will the client avoid having to do a full close() on the socket and thus forbid itself from running the many recv() calls
that it still needs to make to receive the server’s response? The solution is to “half-close” the socket—that is, to shut
down communication permanently in one direction without destroying the socket itself. In this state, the server can
no longer read any data, but it can still send any remaining reply back in the other direction, which will still be open.
The shutdown() call can be used to end either direction of communication in a two-way socket as shown in
Listing 3-2. Its argument can be one of three symbols.
• SHUT_WR: This is the most common value used, since in most cases a program knows when its
own output is done but not necessarily when its conversation partner will be finished. This
value says that the caller will be writing no more data into the socket and that reads from its
other end should respond that there is no more data and indicate end-of-file.• SHUT_RD: This is used to turn off the incoming socket stream so that an end-of-file error is
encountered if your peer tries to send any more data to you on the socket.• SHUT_RDWR: This closes communication in both directions on the socket. It might not, at first,
seem useful because you can also just perform a close() on the socket, and communication
is similarly ended in both directions. The difference between closing a socket and shutting it
down in both directions is a rather advanced one. If several programs on your operating system
are allowed to share a single socket, then close() merely ends your process’s relationship with
the socket but keeps it open as long as another process is still using it. The shutdown() method,
on the other hand, will always immediately disable the socket for everyone using it.