The Multiprocessing and Threading Modules
Parsing the user agent is challenging. There are many variations; we've chosen
a common one for the parse_agent() function. If the user agent matches the
given regular expression, we'll have the attributes of an AgentDetails namedtuple.
If the user agent information doesn't match the regular expression, we'll simply use
the None value instead.
We'll use these three parsers to build AccessDetails instances from the given
Access objects. The main body of the access_detail_iter() function looks
as follows:
for access in iterable:
try:
meth, uri, protocol = parse_request(access.request)
yield AccessDetails(
access= access,
time= parse_time(access.time),
method= meth,
url= urllib.parse.urlparse(uri),
protocol= protocol,
referrer = urllib.parse.urlparse(access.referer),
agent= parse_agent(access.user_agent)
)
except ValueError as e:
print(e, repr(access))
We've used a similar design pattern to the previous access_iter() function. A new
object is built from the results of parsing some input object. The new AccessDetails
object will wrap the previous Access object. This technique allows us to use
immutable objects, yet still contain more refined information.
This function is essentially a mapping from an Access object to an AccessDetails
object. We can imagine changing the design to use map() as follows:
def access_detail_iter2(iterable):
def access_detail_builder(access):
try:
meth, uri, protocol = parse_request(access.request)
return AccessDetails(
access= access,
time= parse_time(access.time),
method= meth,
url= urllib.parse.urlparse(uri),
protocol= protocol,
referrer = urllib.parse.urlparse(access.referer),
agent= parse_agent(access.user_agent)
)