Cache Memory, Associative Memory:
Fig. Memory hierarchies with cache memory
The word cache is pronounced as cash. The cache memory is placed in between CPU and
main memory. The processor is connected to the cache memory through a cache controller.
It is a semiconductor memory. It consists of static RAMs. Its access time is about 10 ns
(nano second). 1 nanosecond = 10–9 second which is much less than that of the main
memory.
The access time of the main memory is about 50 ns. The capacity of the cache memory is
2 to 3 percent of that of the main memory. It stores instruction codes and data, which are
to be currently executed by the CPU. It is used to reduce the average access time for
instructions and data, which are normally stored in the main memory. A cache memory also
needs a cache controller. Cache controller ICs are available.
The modern 32-bit and 64-bit microprocessors operate at very high speed. Their clock
rates are usually in the range of 1GHz–3.8GHz. The memory matching with high-speed
microprocessor must be very fast. But very fast memory is very expensive. If a fast
microprocessor operates with conventional main memory, it has to operate with several
wait states. This will reduce the speed of the computer. A compromise is made, and a high-
speed cache memory is used to supply currently needed instructions and data to CPU. The
main memory stores program and data, which is to be processed by the CPU. The currently
needed instructions and data of the program are loaded into the cache from the main
memory.
There are two types of cache schemes: write-through and write-back. In a write through
cache the main memory is updated each time the CPU writes into the cache. The advantage
of the write-through cache is that the main memory always contains the same data as the
cache contains. This characteristic is desirable in a system which uses direct memory
access (DMA) scheme of data transfer. The I/O devices communicating through DMA
receive the most recent data. In a write-back cache, only the cache memory is updated
during a write operation. The updated locations in the cache are marked by flags so that
later on when the word is removed from the cache, it is copied into the main memory. The
words are removed from the cache time to time to make room for a new block of words.
The contents of updated cache memory locations are copied into the main memory when
there are free processor cycles. This requires additional hardware support, but improves
performance, since the exchanges between cache and the main memory are fewer and
better timed. The write-back scheme increases performance by reducing the utilization of
buses and preventing unnecessary bottlenecks in the system. This scheme is faster and
hence it is preferred.
The technique of accessing a cache memory differs from that of the main memory. To
access main memory the CPU sends an address to it. In response of this the main memory
sends data contained at the specified memory address. On the other hand cache memory
usesparallel searching of the required data. It first compares the incoming address to the ad
dresses present in the cache. If the address matches, it is said that a ‘hit’ has occurred. Then
the corresponding data is read by the CPU. If the address does not match, it is said that a
‘miss’ has occurred. When a miss occurs, the data is read from the main memory. The data
read from the main memory is also written to the cache memory, so that when this specific
address is accessed next time a hit occurs. Usually, hit ratio is above 90 per cent. The hit
ratio is defined as the ratio of the number of hits to the total number of read requests sent
to the cache memory by the CPU. Usually, set associative cache memory is used in a
computer.
The memory in which parallel search of the required data is done, is known as associative
memory. Set associative memory is a kind of associative memory.
Cache memory is placed at two or three levels. They are called first-level cache (L1),
second-level cache (L2), and third-level cache (L3). Some microprocessors contain L1 and L2
within the microprocessors. Some microprocessors contain L1, L2 and L3 within the
microprocessor. Cache within the microprocessor is called internal cache. The cache outside
the processor is called external cache.