Learning Python Network Programming

(Sean Pound) #1

HTTP and Working with the Web


The source code will pop up in a section of the window. In the preceding screenshot,
it's at the bottom left corner of the screen. Here, you will see some lines of code that
looks like the following example:


<form action="http://search.debian.org/cgi-bin/omega"
method="get" name="P">
<p>
<input type="hidden" value="en" name="DB"></input>
<input size="27" value="" name="P"></input>
<input type="submit" value="Search"></input>
</p>
</form>

You should see the second highlighted. This is the tag that corresponds to
the search text box. The value of the name attribute on the highlighted tag
is the key that we use in our data_dict, which in this case is P. The value in our
data_dict is the term that we want to search for.


To get the URL, we need to look above the highlighted for the enclosing


tag. Here, our URL will be of the value of the action attribute, [http://](http://)
search.debian.org/cgi-bin/omega. The source code for this web page is included
in the source code download for this book, in case Debian changes their website
before you read this.

This process can be applied to most HTML pages. To do this, look for the
corresponding to the input text box, then find the URL from the enclosing
tag. If you're not familiar with HTML, then this can be a bit of a trial and error
process. We'll be looking at some more methods of parsing HTML in the
next chapter.


Once we have our input name and URL, we can construct and submit the POST
request, as shown in the previous section.


HTTPS


Unless otherwise protected, all HTTP requests and responses are sent in clear text.
Anyone with access to the network that the messages travel over can potentially
intercept our traffic and read it without hindrance.


Since the web is used for transferring quite a lot of sensitive data, solutions have
been created for preventing eavesdroppers from reading the traffic, even if they
are able to intercept it. These solutions, for the most part, employ some form
of encryption.

Free download pdf