Cache Memory Design

for RISC-V Processors

Introductory Computer Architecture
Priyesh Shukla (IPA – Spring 2026)
IIIT Hyderabad
Overview

Lecture Outline
Why Cache? The Processor-Memory Gap ~10 min

Cache Basics & Terminology ~15 min

Cache Associativity (Direct, Set, Fully) ~20 min

Replacement Policies (LRU, FIFO, Random) ~10 min

Write Policies (Write-Through vs Write-Back) ~10 min

Cache Coherence & MESI Protocol ~15 min

Performance Analysis & RISC-V Design ~10 min

2 / 30
Part 1: The Processor-Memory Gap

Why Do We Need Cache?

Real-World Analogy
Imagine you're writing an essay at your desk. Your desk (cache) holds the books you're actively using.
The bookshelf across the room (main memory) has more books, but takes time to walk to. The library
downtown (disk) has everything — but it takes even longer to get there.

CPU Register L1 Cache Main Memory

~0.25 ns ~1 ns ~100 ns
Fastest, smallest Fast, small (32–64 KB) Slow, large (GBs)

3 / 30
 Part 1: The Processor-Memory Gap

The Memory Hierarchy

Exploiting locality to bridge the speed gap between CPU and memory
~1 KB | <1
Registers
ns
Faster
Smaller
Costlier L1 Cache 32–64 KB | ~1 ns

L2 Cache 256 KB–1 MB | ~5 ns

L3 Cache 2–32 MB | ~20 ns

Slower
Larger
Cheaper Main Memory (DRAM) 4–64 GB | ~100 ns

Disk / SSD TBs | ~ms

4 / 30
Part 1: The Processor-Memory Gap

Principles of Locality
Why caches work — patterns in how programs access data

Temporal Locality Spatial Locality

If a data item is accessed, it is likely to be If a data item is accessed, nearby items are
accessed again soon. likely to be accessed soon.

RISC-V Example: loop: lw x5, 0(x10)

RISC-V Example: Accessing array[0], array[1],
addi x10, x10, 4
array[2]... sequentially
bne x5, x0, loop

Key Insight: Caches exploit both types of locality. Temporal locality keeps recently used data in cache.
Spatial locality is exploited by fetching entire cache lines (blocks) at a time.

5 / 30
Part 2: Cache Basics

How Cache Works

A small, fast memory that stores copies of frequently accessed data

→ → →
CPU issues Check cache Hit? Miss?
memory request for data Return data Fetch from RAM

Cache Hit Cache Miss

Data NOT found in cache. The CPU must
Data found in cache. The CPU gets the
wait while the data is fetched from main
data quickly without accessing main
memory — much slower. The block is then
memory. This is the fast path.
placed into cache.

6 / 30
Part 2: Cache Basics

Cache Terminology
The smallest unit of data transferred between cache and main
Cache Line (Block)
memory (typically 32 or 64 bytes).

Bits stored alongside each cache line to identify which memory

Tag
address the line holds.

Index Bits from the address used to select which cache set to look in.

Offset Bits that identify the specific byte within a cache line.

A flag indicating whether the cache line contains valid data (1 =

Valid Bit
valid, 0 = invalid).

A flag indicating whether the cached data has been modified and
Dirty Bit
differs from main memory.
7 / 30
Part 2: Cache Basics

Address Breakdown for Cache

How a RISC-V 32-bit address maps to cache lookup fields

32-bit RISC-V Memory Address

Tag (20 bits) Index (7 bits) Offset (5 bits)

Identifies which memory block Selects the cache set Byte within the block

Example: 128-entry, direct-mapped cache with 32-byte blocks

Cache size: 128 entries × 32 bytes = 4 KB
Offset bits:
log₂(32) = 5 bits → selects byte within 32-byte block
Index bits:
log₂(128) = 7 bits → selects one of 128 cache lines
Tag bits:
32 − 7 − 5 = 20 bits → identifies which block from memory
8 / 30
Part 2: Cache Basics

RISC-V Cache Hit Example

Instruction: lw x5, 0(x10) // Load word from address in x10 into x5

1 CPU computes effective address: x10 = 0x0000_4A60

2 Extract index bits [11:5] → index = 83 (0x53)

3 Go to cache line 83, compare stored tag with address tag bits [31:12]

4 Tags match AND valid bit = 1 → CACHE HIT!

5 Use offset bits [4:0] to select correct bytes → return data to x5

Result: Data delivered to CPU in ~1 cycle (L1 hit). No main memory access needed! 9 / 30
Part 3: Cache Associativity

Direct-Mapped Cache
Each memory block maps to exactly one cache line
Memory Blocks Cache (4 lines)
Key Points
Block 0 → Line 0
Block 1 →
• Each block has exactly ONE
Block 2 → Line 1
place it can go
• Line = Block mod N
Block 3 → • Block 0 & Block 4 compete for
Line 2 Line 0
Block 4 →line = block mod
Mapping: • Simple hardware
4 • Can cause conflict misses
Block 5 Line 3

Block 6

Block 7

10 / 30
Part 3: Cache Associativity

Set-Associative Cache
Each memory block maps to a set, and can go in any way within that set
2-Way Set-Associative Cache (4 lines total, 2 sets)

Way 0 Way 1
Set 0 V | Tag | Data V | Tag | Data

Way 0 Way 1
Set 1 V | Tag | Data V | Tag | Data

set = (block address) mod (number of sets)

A block can be placed in ANY way within its assigned set. Both ways are checked in parallel
during lookup. 11 / 30
Part 3: Cache Associativity

Fully Associative Cache

A block can be placed in ANY cache line — maximum flexibility
Any block → Any line

Line 0
Block A

→ any Line 1
Block B

Line 2
Block C

Line 3

Advantages Disadvantages
Must compare tag of EVERY line in
No conflict misses. Optimal placement of parallel. Very expensive hardware.
blocks. Lowest miss rate. Practical only for small caches (e.g.,
TLBs).
12 / 30
Part 3: Cache Associativity

Associativity Comparison
Property Direct-Mapped N-Way Set Fully Associative

Ways per set 1 N (2, 4, 8…) All lines

Number of sets = cache lines lines / N 1

Tag comparisons 1 N All lines

Conflict misses High Medium None

Hardware cost Low Medium High

Hit time Fastest Moderate Slowest

In Practice: Most modern RISC-V processors use 2-way or 4-way set-associative L1 caches as a good
tradeoff between miss rate and hardware complexity. L2/L3 caches tend to have higher associativity (8-
way or 16-way).
13 / 30
Part 3: Cache Associativity

Worked Example: Direct-Mapped Cache

4-line cache, block size = 1 word, address sequence: 0, 8, 0, 6, 8

Access Block Line (mod 4) Hit/Miss Action

0 0 0 Miss Load block 0 → Line 0

8 8 0 Miss Evict block 0, load 8 → Line 0

0 0 0 Miss Evict block 8, load 0 → Line 0

6 6 2 Miss Load block 6 → Line 2

8 8 0 Miss Evict block 0, load 8 → Line 0

Conflict Miss Problem (Thrashing)

Blocks 0 and 8 both map to Line 0. They keep evicting each other! This is 0% hit rate — the worst case
for direct-mapped cache. A 2-way set-associative cache would fix this by allowing both blocks to coexist in
the same set.
14 / 30
Part 4: Replacement Policies

Replacement Policies
When the cache is full, which block do we evict? (Only applies to associative caches)

LRU (Least Recently Used)

✓ Best hit rate in practice
Evict the block that hasn't been accessed for the longest time. Based on
temporal locality — if you haven't used it recently, you probably won't need ✗ Expensive to track for high
it soon. associativity

FIFO (First In, First Out)

✓ Simple hardware
Evict the block that was loaded earliest, regardless of recent access
✗ Ignores recent usage
patterns. Simple circular pointer implementation. patterns

Random ✓ Simplest, no tracking

needed
Pick a random block to evict. Surprisingly competitive! Uses a simple
✗ Unpredictable, no worst
LFSR (pseudo-random number generator). case is better

15 / 30
Part 4: Replacement Policies

LRU Replacement: Worked Example

2-way set-associative, Set 0. Access sequence: A, B, C, A, D
Access Way 0 Way 1 Result LRU → evict?

A A — Miss —

B A B Miss LRU = A

C C B Miss → evict A LRU = B

A C A Miss → evict B LRU = C

D D A Miss → evict C LRU = A

How LRU Tracking Works (for 2-way)

For a 2-way cache, we only need 1 bit per set: it points to the least recently used way. On every access, flip this
bit to point to the OTHER way. For 4-way and above, tracking LRU precisely requires more bits and becomes
expensive — pseudo-LRU approximations are often used.
16 / 30
Part 5: Write Policies

Write Policies: What Happens on a Store?

Write-Through Write-Back
Writes update ONLY the cache. Main
Every write updates BOTH the cache AND
memory is updated later when the block is
main memory simultaneously.
✓ Simple, memory always consistent evicted.
✓ Fast — writes stay in cache
✓ Easy recovery on crash ✓ Reduces memory traffic
✗ Slow — every store goes to memory ✗ Complex — needs dirty bit
✗ Uses write buffer to hide latency ✗ Memory can be stale

RISC-V Store Example: sw x5, 0(x10)

Write-Through: Write x5 to cache line AND send write to memory bus (buffered).
Write-Back: Write x5 to cache line only. Set dirty bit = 1. Memory updated when this line is evicted.

Most modern RISC-V implementations (SiFive U74, etc.) use write-back for L1 data cache.
17 / 30
Part 5: Write Policies

Write-Miss Policies
What happens when we write to an address NOT in the cache?

Write-Allocate No-Write-Allocate
On a write miss, fetch the block from On a write miss, write directly to main
memory into the cache, then perform the memory without bringing the block into the
write in the cache. cache.

Usually paired with write-back. Usually paired with write-through.

Typical Pairings in Practice

Combination Hit Policy Miss Policy Used In

Common Write-Back Write-Allocate Most L1 D-caches

18 / 30
Alternative Write-Through No-Write-Allocate Some GPU caches
Part 6: Cache Coherence

The Cache Coherence Problem

What happens when multiple cores cache the same memory location?

Core 0 Core 1
Which
value?
Cache: X = 5 Cache: X = 7

Memory: X = 5

1 Both cores read X = 5 from memory 2 Core 1 writes X = 7 (only updates its 3 Core 0 reads X and gets stale value 5
into their caches own cache with write-back) — inconsistency!

19 / 30
Part 6: Cache Coherence

MESI Protocol Overview

The most common snooping-based cache coherence protocol

M Modified Only copy, dirty. Must write back before sharing.

E Exclusive Only copy, clean. Can transition to Modified on write.

S Shared Multiple copies exist, all clean. Read-only.

I Invalid Not valid. Must fetch from memory or another cache.

Key Idea: Each cache line has a 2-bit state field. All caches "snoop" on the shared bus and update their states
when other cores read/write. 20 / 30
Part 6: Cache Coherence

MESI: A Step-by-Step Example

Two RISC-V cores accessing address X
Step Action Core 0 State Core 1 State Explanation

1 Core 0 reads X E (Exclusive) I (Invalid) Only copy in system

2 Core 1 reads X S (Shared) S (Shared) Core 0 snoops, both go to Shared

3 Core 1 writes X I (Invalid) M (Modified) Core 1 invalidates Core 0's copy

4 Core 0 reads X S (Shared) S (Shared) Core 1 writes back, both Shared

RISC-V Perspective
The RISC-V ISA uses the FENCE instruction to enforce memory ordering. In multicore RISC-V systems, the
hardware coherence protocol (like MESI) handles cache-to-cache transfers transparently. The programmer uses
atomic instructions (AMO) and FENCE for synchronization.
21 / 30
Part 7: Performance & Design

The Three C's of Cache Misses

Compulsory (Cold) Miss
First-ever access to a block. Unavoidable — the data was never in cache.
Example: First iteration of a loop reading a new array.

Capacity Miss
The cache is too small to hold all active blocks. Blocks are evicted and later needed again.
Example: Working set exceeds cache size.

Conflict Miss
Multiple blocks compete for the same cache set (in direct-mapped or set-associative caches).
Example: Two arrays whose addresses map to the same set.

Increasing associativity reduces conflict misses. Increasing cache size reduces capacity misses. Prefetching helps with
compulsory misses. 22 / 30
Part 7: Performance & Design

Average Memory Access Time (AMAT)

The key metric for evaluating cache performance

AMAT = Hit Time + Miss Rate × Miss Penalty

Hit Time: Time to access the cache and determine hit/miss (typically 1–3 cycles)

Miss Rate: Fraction of accesses that are misses (e.g., 5% = 0.05)

Miss Penalty: Additional time to fetch data from the next level (e.g., 100 cycles for DRAM)

Worked Example
Given: L1 hit time = 1 cycle, miss rate = 5%, miss penalty = 100 cycles
AMAT = 1 + 0.05 × 100 = 1 + 5 = 6 cycles

Even a 5% miss rate makes average access 6× slower than a pure hit! 23 / 30
Part 7: Performance & Design

Multi-Level Cache Design

Modern processors use 2–3 levels of cache to balance speed and capacity
L1 Cache
L1 Size: 32–64 KB | 2–4 way | 1–3 cycles
Split: I-cache + D-cache. Per-core. Optimized for speed.

L2 Cache
L2 Size: 256 KB – 1 MB | 4–8 way | ~10 cycles
Unified (data + instructions). Per-core or shared. Larger, slower.

L3 Cache
L3 Size: 2–32 MB | 8–16 way | ~30 cycles
Shared across all cores. Last level before DRAM. Inclusive design.

AMAT = Hit_L1 + MissRate_L1 × (Hit_L2 + MissRate_L2 × (Hit_L3 + MissRate_L3 × Penalty_Mem))

24 / 30
Part 7: Performance & Design

Cache Design in RISC-V Processors

Typical cache parameters in real RISC-V implementations
SiFive U74 Typical Textbook
Parameter
(Linux-capable) Design

L1 I-Cache 32 KB, 4-way 4–64 KB, 2/4-way

L1 D-Cache 32 KB, 8-way 4–64 KB, 2/4-way

Cache Line Size 64 bytes 32 or 64 bytes

Write Policy Write-back, write-allocate Either

Replacement Pseudo-LRU LRU / Random

L2 Cache 2 MB shared, 16-way Optional

Coherence MOESI-based MESI (simplified)

Note: RISC-V is an open ISA — cache design is implementation-specific, not mandated by the ISA. Different
vendors make different tradeoffs. 25 / 30
Part 7: Performance & Design

Cache and the RISC-V Pipeline

How cache integrates with the 5-stage RISC-V pipeline

IF ID EX MEM WB
→ → → →
Fetch Decode Execute Memory Write Back

I-Cache D-Cache
Access Access

Cache Access in the Pipeline

IF Stage (Instruction Fetch): Accesses the I-cache to fetch the next instruction. On a miss, the pipeline stalls.
MEM Stage (Memory Access): Accesses the D-cache for load/store instructions (lw, sw, lb, sb, etc.). On a
miss, the pipeline stalls.

Cache miss penalty = pipeline stall cycles. This is why cache hit rate directly impacts CPI (Cycles Per
Instruction). 26 / 30
Part 7: Performance & Design

Cache Optimization Techniques

Increase Block Size
Exploits spatial locality better, but increases miss penalty and can increase conflict misses.

Increase Associativity
Reduces conflict misses, but increases hit time and hardware cost. Diminishing returns past 8-way.

Add More Cache Levels

L2 and L3 caches reduce the effective miss penalty seen by L1.

Hardware Prefetching
Predict future accesses and fetch blocks before they're needed. Reduces compulsory misses.

Software Optimization
Loop tiling, array padding, data layout changes improve spatial/temporal locality in programs.

27 / 30
Part 7: Performance & Design

RISC-V Memory & Cache Instructions

Instruction Category Purpose

lw / lh / lb Load Read word/half/byte from D-cache

sw / sh / sb Store Write word/half/byte to D-cache

FENCE Ordering Ensure all prior memory ops complete before subsequent ones

FENCE.I I-cache sync Synchronize I-cache with D-cache (after code modification)

AMO (lr/sc) Atomic Load-reserved / store-conditional for lock-free sync

Zicbom ext. Cache mgmt Cache block operations: clean, flush, invalidate

Key Insight for RISC-V

RISC-V separates I-cache and D-cache by design. If you write self-modifying code (e.g., JIT compilation), you
MUST use FENCE.I to make the I-cache see the new instructions. This is a common source of bugs in RISC-V
OS development! 28 / 30
Practice

Practice Problem: Design a Cache

Problem

Design a cache for a RISC-V processor with these constraints:

32-bit address space • 16 KB total • 64-byte lines • 4-way set-associative

Solution
Total lines = 16 KB / 64 B = 256 lines
Sets = 256 / 4 = 64 sets
Offset bits = log₂(64) = 6 bits
Index bits = log₂(64) = 6 bits

Tag bits = 32 − 6 − 6 = 20 bits

Each line: 1 valid + 1 dirty + 20 tag + 512 data = 534 bits

Total SRAM = 256 × 534 bits ≈ 16.8 KB (overhead ≈ 5%)

29 / 30
 Key Takeaways
1 Caches bridge the processor-memory speed gap by exploiting locality

2 Direct-mapped is simple but has conflict misses; set-associative is the sweet spot

3 LRU replacement performs best; hardware cost grows with associativity

4 Write-back + write-allocate is the most common policy in modern processors

5 MESI protocol maintains coherence in multicore systems via snooping

6 AMAT = Hit Time + Miss Rate × Miss Penalty — the fundamental equation

7 RISC-V cache design is implementation-specific; the ISA provides FENCE and AMO for ordering

Thank You — Questions? 30 / 30