HTTP and Working with the Web
Summary
We looked at the principles of the HTTP protocol. We saw how to perform
numerous fundamental tasks with the standard library urllib and the
third-party Requests packages.
We looked at the structure of HTTP messages, HTTP status codes, the different
headers that we may encounter in requests and responses, and how to interpret them
and use them for customizing our requests. We looked at how URLs are formed, and
how to manipulate and construct them.
We saw how to handle cookies and redirects, how to handle errors that might occur,
and how to use secure HTTP connections.
We also covered how to submit data to websites in the manner of submitting a
form on a web page, and how to extract the parameters that we need from a page's
source code.
Finally, we looked at the third-party Requests package. We saw that as compared
to the urllib package, Requests, automates and simplifies many of the tasks that
we may routinely need to carry out with HTTP. This makes it a great choice for
day-to-day HTTP work.
In the next chapter, we'll be employing what we've learned here to carry out detailed
interactions with different web services, querying APIs for data, and uploading our
own objects to the web.