Scribd
Upload
7 views17 pages

Introduction to Cache Memory Concepts

The document provides an introduction to cache memory in computer architecture, detailing its role as a fast buffer between the processor and main memory. It explains key concepts such as the memory hierarchy, cache fundamentals, and the principles of locality that govern cache performance. Additionally, it covers cache organization, addressing, and provides an example of calculating cache block addresses.

Uploaded by

subhamyadav1921
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
7 views17 pages

Introduction to Cache Memory Concepts

The document provides an introduction to cache memory in computer architecture, detailing its role as a fast buffer between the processor and main memory. It explains key concepts such as the memory hierarchy, cache fundamentals, and the principles of locality that govern cache performance. Additionally, it covers cache organization, addressing, and provides an example of calculating cache block addresses.

Uploaded by

subhamyadav1921
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

CS223 Computer Architecture & Organization

Introduction to Cache Memory

John Jose
Associate Professor
Department of Computer Science & Engineering
Indian Institute of Technology Guwahati
Multi-cycle Operations
MIPS Pipeline

Instruction Instr. Decode Execute Memory Write


Fetch Reg. Fetch Addr. Calc Access Back
Next PC

Adder
Adder

0?
4 RS1

MEM/WB
EX/MEM

Memory
RS2

ID/EX
Address

ALU
IF/ID
Memory

Data
File
Reg
Ins

WB Data
E
Imm x

RD RD RD
Processor Memory Performance Gap
Relationship of Caches and Pipeline
IF ID EX MEM WB
Memory

Next PC

Adder
Adder

0?
4 RS1

MEM/WB
EX/MEM

Memory
RS2

ID/EX
Address

Data
ALU
IF/ID
Memory

File
Reg
Ins

WB Data
E
Imm x

RD RD RD
Role of memory
❖ Programmers want unlimited amount of fast memory
❖ Create the illusion of a very large and fast memory
❖ Implement the memory of a computer as a hierarchy
❖ Multiple levels of memory with different speeds and sizes
❖ Keep the smaller and faster memories close to the processor and
the slower, large memory below that.
Memory Hierarchy
Cache Memory - Introduction
❖ Cache is a small, fast buffer between processor and memory
❖ Old values will be removed from cache to make space for new
values
❖ Principle of Locality : Programs access a relatively small portion
of their address space at any instant of time
❖ Temporal Locality : If an item is referenced, it will tend to be
referenced again soon
❖ Spatial Locality : If an item is referenced, items whose addresses
are close by will tend to be referenced soon
Access Patterns

Ref: MISSISSIPPI

Ref: ABCDEFAGHI
Cache Fundamentals
❖ Block/Line : Unit of storage in cache. One block is multiple words
❖ Hit : An access where the data requested by the processor is present in
the cache
❖ Miss : An access where the data requested by the processor is not
present in the cache
❖ Hit Time : Time to access the cache memory block and return the data to
the processor.
❖ Hit Rate / Miss Rate: Fraction of memory access found (not found) in the
cache
❖ Miss Penalty : Time to replace a block in the cache with the
corresponding block from the next level.
CPU – Cache Interaction
❖The transfer unit ❖The tiny, very fast CPU
between the CPU register file has room for four 4-
register file and the byte words
cache is a 4-byte word
line 0
line 1 ❖The small fast L1 cache has
❖The transfer unit room for two 4-word blocks
between the cache and
block 10 abcd
main memory is a .. ❖The big slow main memory
4-word block (16B) block 21 pqrs has room for many 4-word
...
. blocks
block 30 wxyz
...
General Organization of a Cache
t tag bits B = 2b bytes
❖Cache is an array
per line per cache block
of sets
❖Each set contains v tag 0 1 ••• B–1

one or more lines set 0:


••• E lines
v tag 0 1 ••• B–1
❖Each line holds a per set
block of data v tag 0 1 ••• B–1
•••
S= 2s sets set 1:
v tag 0 1 ••• B–1

•••
v tag 0 1 ••• B–1
•••
set S-1:
v tag 0 1 ••• B–1

Cache size: C = B x E x S data bytes


Addressing Caches
CPU wants → Address A: t bits s bits b bits
m-1 0
v tag 0 1 ••• B–1

set 0:
••• <tag> <set index><block offset>
v tag 0 1 ••• B–1

v tag 0 1 ••• B–1

•••
set 1: v tag 0 1 ••• B–1
❖The word at address A is in the cache if
•••
the tag bits in one of the <valid> lines in
v tag 0 1 ••• B–1

set S-1:
••• set <set index> match <tag>
v tag 0 1 ••• B–1

❖The word contents begin at offset


<block offset> bytes from the beginning of
the block
Addressing Caches
CPU wants → Address A: t bits s bits b bits
m-1 0
v tag 0 1 ••• B–1

set 0:
••• <tag> <set index><block offset>
v tag 0 1 ••• B–1

v tag 0 1 ••• B–1

•••
set 1:
❖ Locate the set based on <set index>
v tag 0 1 ••• B–1

•••

v tag 0 1 ••• B–1


❖ Locate the line in the set based on
•••
set S-1: v tag 0 1 ••• B–1
<tag>
❖ Check that the line is valid
❖ Locate the data in the line based on
<block offset>
Cache Block Concepts
For a 32KB direct mapped cache with 64 byte cache block, give the
address of the starting byte of the first word in the block that contains
the address 0X 7245E824.
Cache Block Concepts
32KB direct mapped cache with 64 byte cache block
address 0X 7245E824.

# sets = CS/(BSxA) = 215/(26x1) = 29 = 512 sets


Tag Index =9 Offset=6

Address: 0x 7245E824
-----------0010 0100 offset (6 bits)
Block address range from ----- 0000 0000 to ----- 0011 1111
0x 7245E800 to 0x 7245E83F

Hence starting address is 0x 7245E800


johnjose@[Link]
[Link]

Common questions

Powered by AI

Increasing cache size generally improves hit rate by holding more data, though at higher costs and potential increases in access time. Larger block sizes enhance spatial locality but also risk unnecessary data loading, increasing miss penalties. Higher associativity improves cache hit rates and reduces conflict misses but complicates design and slightly increases access time due to more complex indexing and search operations. Optimizing these factors requires balancing cost, speed, and design complexity.

The CPU-cache data transfer unit, which is smaller (4-byte words) than the cache-main memory transfer unit (16-byte blocks), ensures frequent, rapid CPU access, maintaining high-speed processing. However, larger cache-main memory transfers maximize throughput and reduce the overhead of individual cache misses, thus preventing frequent costly accesses to slower main memory. This balance ensures high overall system performance.

The tag helps in identifying the specific cache block among multiple blocks in a set by comparison. The set index specifies which cache set to look into, and the block offset provides the exact location of data within a block. Altogether, these components ensure precise data retrieval from cache by narrowing down to the exact line and block location.

In a 32KB direct-mapped cache with 64-byte blocks, each block corresponds to a unique cache set determined by the block's index bits, resulting in 512 sets. This organization is straightforward, providing fast determination of where a block resides within the cache. This simplicity allows for efficient cache line replacement but may lead to higher conflict misses compared to associative caches.

Underinvestment in cache size can lead to frequent cache misses due to insufficient space to keep recently or frequently accessed data, leading to more main memory accesses. This increases miss penalties and reduces overall system efficiency and speed, counteracting the advantages of temporal and spatial locality. Hence, adequate cache sizing is vital for maintaining high performance.

Cache hit time, being the time taken to access the cache, directly affects CPU speed. A lower hit time can significantly enhance system performance. Conversely, miss penalty, which is the time taken to fetch data from lower memory levels upon a cache miss, can drastically slow down performance if too high. Balancing between hit time and miss penalty is crucial for optimal system efficiency.

To resolve address 0X 7245E824 in a 32KB direct-mapped cache with 64-byte cache blocks, first determine the set by using the set index derived from address bits, resulting in 512 sets. The address binary representation points the block starting at 0x7245E800. The cache uses these determinations to retrieve the block containing the desired address quickly.

The Principle of Locality underpins effective cache memory design by exploiting both spatial and temporal locality. It ensures that future fetches will likely access nearby data or recently used data, thus justifying the caching mechanism, optimizing performance by reducing unnecessary memory fetches from lower levels.

Memory hierarchy leverages both temporal locality and spatial locality principles, providing a multi-level memory system where smaller, faster cache levels are closer to the processor, thereby masking the speed and size limitations of lower levels. By frequently accessing smaller portions of the address space efficiently, the architecture suggests an illusion of a large fast memory space.

Multi-level cache architectures offer several benefits, such as increased hit rates by different cache levels storing varying data scopes, and reduced latency by providing a hierarchical data retrieval path. However, they also present challenges like increased complexity in data management between levels, higher costs, and design intricacies ensuring efficient data synchronization and coherency. Efficient design and management are crucial to leverage multi-level caches effectively.

You might also like