Chapter 3To reduce memory use—and increase performance—we prefer to use generator
expressions and functions as much as possible. These iterate through collections in a
lazy (or non-strict) manner, computing values only when required. Since iterators can
only be used once, we're sometimes forced to materialize a collection as a tuple (or
list) object. Materializing a collection costs memory and time, so we do it reluctantly.
Programmers familiar with Clojure can match Python's lazy generators with the
lazy-seq and lazy-cat functions. The idea is that we can specify a potentially
infinite sequence, but only take values from it as needed.
Using stateful mappings
Python offers several stateful collections; the various mappings include the dict class
and a number of related mappings defined in the collections module. We need to
emphasize the stateful nature of these mappings and use them carefully.
For our purposes in learning functional programming techniques in Python, there
are two use cases for mapping: a stateful dictionary that accumulates a mapping
and a frozen dictionary. In the first example of this chapter, we showed a frozen
dictionary that was used by the ElementTree.findall() method. Python doesn't
provide an easy-to-use definition of an immutable mapping. The collections.abc.
Mapping abstract class is immutable but it's not something we can use trivially.
We'll dive into details in Chapter 6, Recursions and Reductions.
Instead of the formality of using the collections.abc.Mapping abstract class, we
can fall back on confirming that the variable ns_map appears exactly once on the left
side of an assignment statement, methods such as ns_map.update() or ns_map.
pop() are never used, and the del statement isn't used with map items.
The stateful dictionary can be further decomposed into two typical use cases; they
are as follows:
- A dictionary built once and never updated. In this case, we will exploit
the hashed keys feature of the dict class to optimize performance. We can
create a dictionary from any iterable sequence of (key, value) two tuples
via dict( sequence ). - A dictionary built incrementally. This is an optimization we can use to
avoid materializing and sorting a list object. We'll look at this in Chapter 6,
Recursions and Reductions. We'll look at the collections.Counter class
as a sophisticated reduction. Incremental building is particularly helpful
for memoization. We'll defer memoization until Chapter 16, Optimizations
and Improvements.