Chapter 5 ■ Network Data aND Network errors

84

Finally, what if you want the simplicity and efficiency of this fifth pattern but you do not know ahead of time the
length of each message—perhaps because the sender is reading data from a source whose length they cannot predict?
In such cases, do you have to abandon elegance and slog through the data looking for delimiters?
Unknown lengths are no problem if you use the sixth, and final, pattern. Instead of sending just one, try sending
several blocks of data that are each prefixed with their length. This means that as each chunk of new information
becomes available to the sender, it can be labeled with its length and placed on the outgoing stream. When the end
finally arrives, the sender can emit an agreed-upon signal—perhaps a length field giving the number zero—that tells
the receiver that the series of blocks is complete.
A simple example of this idea is shown in Listing 5-2. Like the previous listing, this sends data in only one
direction—from the client to the server—but the data structure is much more interesting than in the previous listing.
Each message is prefixed with a 4-byte length contained in a struct. Since 'I' means a 32-bit unsigned integer, each
frame can be up to 4GB in length. This sample code sends a series of three blocks to the server followed by a zero-length
message, which is just a length field with zeros inside and then no message data after it, to signal that the series of
blocks is over.

Listing 5-2. Framing Each Block of Data by Preceding It with Its Length

#!/usr/bin/env python3

Foundations of Python Network Programming, Third Edition

https://github.com/brandon-rhodes/fopnp/blob/m/py3/chapter05/blocks.py

Sending data over a stream but delimited as length-prefixed blocks.

import socket, struct
from argparse import ArgumentParser

header_struct = struct.Struct('!I') # messages up to 2**32 - 1 in length

def recvall(sock, length):
blocks = []
while length:
block = sock.recv(length)
if not block:
raise EOFError('socket closed with %d bytes left'
' in this block'.format(length))
length -= len(block)
blocks.append(block)
return b''.join(blocks)

def get_block(sock):
data = recvall(sock, header_struct.size)
(block_length,) = header_struct.unpack(data)
return recvall(sock, block_length)

def put_block(sock, message):
block_length = len(message)
sock.send(header_struct.pack(block_length))
sock.send(message)

def server(address):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(address)