Chapter 3 ■ tCp
52
client grinds to a halt soon afterward. The amount of data processed before they seize up varies on the Ubuntu laptop
on which I am writing this chapter, but on the test run that I just completed here on my laptop, the Python script
stopped with the server saying this:
$ python tcp_deadlock.py server ""
Listening at ('0.0.0.0', 1060)
Processing up to 1024 bytes at a time from ('127.0.0.1', 60482)
4452624 bytes processed so far
And the client is frozen about 350,000 bytes farther ahead in writing its outgoing data stream.$ python tcp_deadlock.py client "" 16000000
Sending 16000000 bytes of data, in chunks of 16 bytes
8020912 bytes sent
Why have both client and server been brought to a halt?
The answer is that the server’s output buffer and the client’s input buffer have both finally filled, and TCP has
used its window adjustment protocol to signal this fact and stop the socket from sending additional data that would
have to be discarded and later resent.
Why has this resulted in deadlock? Consider what happens as each block of data travels. The client sends it
with sendall(). Then the server accepts it with recv(), processes it, and transmits its capitalized version back
out with another sendall() call. And then what? Well, nothing! The client is never running any recv() calls—not
while it still has data to send—so more and more data backs up until the operating system buffers are not willing to
accept any more.
During the run shown previous, about 4MB were buffered by the operating system in the client’s incoming
queue before the network stack decided that it was full. At that point, the server blocks in its sendall() call, and
its process is paused by the operating system until the logjam clears and it can send more data. With the server no
longer processing data or running any more recv() calls, it is now the client’s turn to have data start backing up. The
operating system seems to have placed a limit of around 3.5MB on the amount of data it is willing to queue up in that
direction because the client got roughly that far into producing data before finally being brought to a halt as well.
On your own system, you will probably find that different limits are reached; the foregoing numbers are arbitrary
and based on the mood of my laptop at the moment. They are not at all inherent in the way TCP works.
The point of this example is to teach you two things—besides, of course, showing that recv(1024) indeed returns
fewer bytes than 1,024 if a smaller number are immediately available!
First, this example should make much more concrete the idea that there are buffers sitting inside the TCP
stacks on each end of a network connection. These buffers can hold data temporarily so that packets do not have
to be dropped and eventually resent if they arrive at a moment that their reader does not happen to be inside of
a recv() call. But the buffers are not limitless. Eventually, a TCP routine trying to write data that is never being
received or processed is going to find itself no longer able to write, until some of the data is finally read and the
buffer starts to empty.
Second, this example makes clear the dangers involved in protocols that do not alternate lock step with the client
requesting a limited amount of data and then waiting for the server to answer or acknowledge. If a protocol is not
strict about making the server read a complete request until the client is done sending and then sending a complete
response in the other direction, then a situation like the one created here can cause both of them to freeze without any
recourse other than killing the program manually and then rewriting it to improve its design.