All modern processors use cache memory. Whenever data is read from memory, an entire "line"
(typically 16 or 32 bytes) is brought into the cache. If the program exhibits good locality of reference
(e.g., it's striding down a character string), future references to adjacent data can be retrieved from the
fast cache rather than the slow main memory. Cache operates at the same speed as the cycle time of
the system, so for a 50 MHz processor, the cache runs at 20 nanoseconds. Main memory might
typically be four times slower than this! Cache is more expensive, and needs more space and power
than regular memory, so we use it as an adjunct rather than as the exclusive form of memory on a
system.
The cache contains a list of addresses and their contents. Its list of addresses is constantly changing, as
the processor references new locations. Both reads and writes go through the cache. When the
processor wants to retrieve data from a particular address, the request goes first to the cache. If the
data is already present in the cache, it can be handed over immediately. Otherwise, the cache passes
the request on, and a slower access to main memory takes place. A new line is retrieved, and it takes
its place in the cache.
If your program has somewhat perverse behavior and just misses cache every time, you end up with
worse performance than if there was no cache at all. This is because all the extra logic of figuring out
what is where doesn't come free.
Sun currently uses two types of cache:
- Write-through cache— This always initiates a write to main memory at the same time it
writes to the cache. - Write-back cache— In the first instance, this writes only to cache. The data is transferred to
main memory when the cache line is about to be written again and a save hasn't taken place
yet. It will also be transferred on a context switch to a different process or the kernel.