Linux Kernel Architecture

Chapter3:MemoryManagement

Figure 3-1 illustrates the difference between the two approaches.

Memory

CPU 0 CPU 1 CPU 3

Memory

CPU 0 CPU 1 CPU 3

Memory Memory

UMA NUMA

Figure 3-1: UMA and NUMA systems.

A mix of both machine types with discontiguous memory is also possible. Such a mix would then
represent a UMA system whose RAM is not contiguous but has large holes. Here it is often helpful
to apply the principles of NUMA organization to make memory access simpler for the kernel. In
fact, the kernel distinguishes three configuration options —FLATMEM,DISCONTIGMEM,andSPARSEMEM.
SPARSEMEMandDISCONTIGMEMserve practically the same purpose, but in the view of developers, differ in
the quality of their code —SPARSEMEMis regarded as more experimental and less stable but does feature
performance optimizations. Discontiguous memoryis presumed to be more stable, but is not prepared
for new features like memory hotplugging.

In the following sections, we restrict ourselves largely toFLATMEMbecause this memory organization type
is used on most configurations and is also usually the kernel default. The fact that we do not discuss
the other options is no great loss because all memory models make use of practically the same data
structures.

Real NUMA systems will set the configuration optionCONFIG_NUMA, and the memory management
codes will differ between the two variants. Since the flat memory model will not make sense on NUMA
machines, only discontiguous and sparse memory will be available. Notice that the configuration option
NUMA_EMUallows AMD64 systems with a flat memory to enjoy the full complexities of NUMA systems by
splitting the memory into fake NUMA zones. This can be useful for development when no real NUMA
machine is available — for some reason, these tend to be rather costly.

This book focuses on the UMA case, and does not considerCONFIG_NUMA.Thisdoesnotmean
that the NUMA data structures can be completely neglected. Since UMA systems can choose
the configuration optionCONFIG_DISCONTIGMEMif their address space contains large holes, then more
than one memory node can also be available on systems that do not employ NUMA techniques
otherwise.

Figure 3-2 summarizes the various possible choices for the configuration options related to memory
layout.

Notice that we will come across the termallocation orderquite often in the following discussion. It denotes
the binary logarithm of the number of pages that are contained in a memory region. An order 0 allocation
consists of one page, an order two allocation of 2^1 =2 pages, an order three allocation of 2^2 =4pages,
and so on.