Chapter 2The standard method for encrypting HTTP traffic is called HTTP Secure, or HTTPS.
It uses an encryption mechanism called TLS/SSL, and it is applied to the TCP
connection on which the HTTP traffic travels. HTTPS typically uses TCP port 443,
as opposed to the default HTTP port 80.
To most users, this process is almost transparent. In principle, we only need to
change the http in a URL to an https. Since urllib supports HTTPS, the same is
true for our Python clients.
Note that not all servers support HTTPS, so simply changing the URL scheme to
https: isn't guaranteed to work for all sites. If this is the case, then the connection
attempt may fail in a number of ways, including a socket timeout, a connection
reset error, or possibly even an HTTP error, such as a 400 range error or a 500 range
error. An increasing number of sites are enabling HTTPS however. Many others
are switching to it and using it as their default protocol, so it's worth investigating
whether it's available so you can give your application's users extra security.
The Requests library
So that's it for the urllib package. As you can see, access to the standard library
is more than adequate for most HTTP tasks. We haven't touched upon all of its
capabilities. There are numerous handler classes which we haven't discussed, plus
the opener interface is extensible.
However, the API isn't the most elegant, and there have been several attempts made
to improve it. One of these is the very popular third-party library called Requests.
It's available as the requests package on PyPi. It can either be installed through
Pip or be downloaded from http://docs.python-requests.org, which hosts
the documentation.
The Requests library automates and simplifies many of the tasks that we've been
looking at. The quickest way of illustrating this is by trying some examples.
The commands for retrieving a URL with Requests are similar to retrieving a URL
with the urllib package, as shown here:
import requests
response = requests.get('http://www.debian.org')