Linux Kernel Architecture

Chapter3:MemoryManagement

Sparse memory

NUMA

UMA

Discontiguous

Flat memory memory

NUMA

UMA

Address space without holes

Figure 3-2: Overview of possible memory setups for flat, sparse, and

discontiguous memory on UMA and NUMA machines.

3.2 Organization in the (N)UMA Model

The various architectures supported differ greatly in terms of how they manage memory. Owing to the

intelligent design of the kernel and, in some cases, intervening compatibility layers, these differences are

so well concealed that generic code can usually ignore them. As discussed in Chapter 1, a major issue is

the varying number of indirection levels for page tables. A second key aspect is the division into NUMA

and UMA systems.

The kernel uses identical data structures for machines with uniform and non-uniform memory access so

that the individual algorithms need make little or no distinction between the various forms of memory

arrangement. On UMA systems, a single NUMA nodeis introduced to help manage the entire system

memory. The other parts of memory management are led to believe that they are working with a pseudo-

NUMA system.

3.2.1 Overview

Before we look at the data structures used to organize memory in the kernel, we need to define a few

concepts because the terminology is not always easyto understand. Let’s first consider NUMA systems.

This will enable us to show that it is very easy to reduce them to UMA systems.

Figure 3-3 is a graphic illustration of the memory partitioning described below (the situation is somewhat

simplified, as you will see when we examine the data structures closely).

First, RAM memory is divided intonodes. A node is associated with each processor of the system and is

represented in the kernel by an instance ofpg_data_t(these data structures are defined shortly).

Each node is split intozonesas further subdivisions of memory. For example, there are restrictions as

to the memory area that can be used for DMA operations (with ISA devices); only the first 16 MiB are

suitable. There is also a highmem area that cannot be mapped directly. Between these is the ‘‘normal‘‘

memory area for universal use. A node therefore comprises up to three zones. The kernel introduces the

following constants to distinguish between them.