Module -2

Memory System Design

Syllabus
 Semiconductor memory technology

• A device for storing digital information that is fabricated by

using integrated circuit technology.
• Memories are integrated circuits (ICs) designed to
store data in a digital form (0s and 1s). They consist of
a series of storage cells, each capable of holding a
single bit. These cells are organized into a grid or
matrix, allowing for efficient access using addresses.
 Semiconductor memory technology:
Two types

RAM ROM
Whose content can be changed in a short time Whose content cannot be changed like RAM
and unlimited number of times.
The data on the ROM’s are retained even when
The data on the RAM’s is retained only when powered off and are termed as non-volatile
they are powered and RAM’s loses its data when memories.
powered off and are usually termed volatile
memories.
 • Bidirectional data bus provides read/write facility and is
RAM Chip constructed with three state buffers (1, 0 & high
impedance state).
• Chip select are control inputs to enable the chip.
Function Table for • When CS1 = 1 & 𝐶𝑆2 = 0, the memory can be placed in read
or write mode. Other wise memory unit is inhibited
RAM
 DRAM
DRAM (Dynamic RAM) is a type of volatile memory

Each bit of data is stored in a tiny capacitor inside a memory

cell.

A MOS transistor and a capacitor together store one bit (0 or

1) based on whether the capacitor is charged (1) or
discharged (0).

The charge leaks over time, so the memory must be

refreshed regularly to keep the data.

The word "dynamic" means that constant refreshing is

needed, or else the data will be lost
DRAM
Advantages of DRAM

• Very dense
• Low cost per bit
• Simple memory cell structure

Disadvantages of DRAM

• Complex manufacturing process

• Data requires refreshing
• More complex external circuitry required (read and refresh periodically)
• Volatile memory
• Relatively slow operational speed
Static RAM
• SRAM is a type of volatile memory that stores data
as long as power is supplied without needing to
refresh it frequently.
• Each SRAM cell is made up of 6 transistors (instead
of a capacitor like DRAM).
• Two cross-coupled invertors (T1, T2, T3, T4) forming a
flip-flop (used to store data).
• Two access transistors (T5, T6) controlled by the address
line for reading and writing data.
SRAM
1. Advantages

• SRAM is the fastest type of memory that consumes less power than DRAM.
• SRAM holds data without external periodic refresh, for as long as the circuit is powered. SRAM is most reliable
type of memory as well.

2. Disadvantages

• SRAM is less dense than DRAM

• SRAM takes up much more space than DRAM as it takes six transistors to store a single bit and hence is
expensive.

3. Applications

• SRAM is used in cache memory due to its fast access. In fact SRAM is the fastest memory of all memories
available .
ROM Chip

Since ROM can only read, the data bus is in output mode.

The address lines are 9 for 512 bytes of ROM.

 ROM Types

Masked ROM PROM EPROM EEPROM

MASKED ROM PROM
• Pre-programmed during manufacturing; data • Blank memory that can be programmed once
is permanently written. using a special device called a PROM
• Cannot be modified or erased after programmer.
production. • Data is stored using fuses that are
permanently burned during programming.
• Used for fixed firmware like in calculators,
older gaming consoles, and embedded • Once written, it cannot be changed (One-Time
systems. Programmable - OTP).
• Fast and cheap for mass production, but not • Used in early microcontrollers, BIOS chips, and
flexible for updates security applications.
EPROM EEPROM
• Can be erased and reprogrammed • Can be erased and rewritten
multiple times using UV light. electrically, no need for UV light.
• Has a quartz window on top for UV • Supports byte-by-byte modification,
light exposure, which resets all data. unlike EPROM which erases everything
• Requires special EPROM programmers at once.
to write new data.
• Slower than RAM but more flexible
• Used in old firmware storage, than EPROM.
industrial applications, and
microcontrollers. • Used in BIOS chips, embedded
• Slower and less flexible compared to systems, smart cards, and USB
modern memory. drives.
Memory Interleaving
Memory interleaving is a technique used to make memory access faster by dividing memory into
multiple small blocks (banks) and accessing them in an alternating manner.

Instead of storing data in a single continuous memory block, it is split into multiple memory banks.

When the CPU requests data, different parts of the data can be fetched from different banks at the
same time.

This reduces the waiting time (latency) and speeds up memory access.
Memory Interleaving: example
• Suppose a 64 MB memory made up of the 4 MB chips as shown in the below:

• The memory is organized into banks (groups of chips).

• Each bank has holds 4 MB.
• The total number of banks = 16, meaning we have 16 separate memory blocks,
each 4 MB each.
Memory interleaving example with 4 banks.
Red banks are refreshing and can't be used.
 High order memory
interleaving

Two Types
Low order memory
interleaving
High order memory
interleaving

• Memory is divided into multiple

modules (or banks) instead of
being one big block
• Uses high bits to choose memory
bank and low bits for word
selection.
• Consecutive words are stored in
the same module, which may
slow things down.
• It is also known as Memory
Banking.
High order memory
interleaving
• Problem with Consecutive Data: Since all
consecutive words are stored in one module, the
other memory banks are not used, reducing
efficiency.
• Slower Data Transfer: The maximum speed of data
transfer is limited by how fast one memory
module can respond.
• No Parallel Access: Even with multiple banks, we
can’t access multiple words at once if they are all
in the same bank.
Example: To get 90 (Data), 1000 will be provided
by the processor.
In this 10 will indicate data is in Module 10 and 00
is the address of 90 (data)
Low Order Memory
Interleaving
• The least significant bits (rightmost bits)
of the memory address decide the bank
number.
• The higher order bits (leftmost bits)
decide the word location inside the
bank.
• This ensures that consecutive
addresses are stored in different banks
instead of the same one.
 Low Order Memory
Interleaving
• Faster memory access – Multiple banks work
at the same time.
• Efficient data transfer – No waiting for
one bank to finish before accessing the
next.
• Better CPU performance – Reduces memory
bottlenecks

In this case, to get 90 (Data), 0010 will

be given;
1000 will give = 30
 • Faster data access due to parallel memory
usage
• Higher data transfer rates
• Minimizes CPU waiting time
Advantages of • Reduces bottlenecks and improves performance
Memory Interleaving
• Efficient use of multiple memory modules
• Useful in High-Performance Systems:
Interleaving is widely used in supercomputers,
gaming PCs, and high speed data processing
systems to ensure smooth performance.
 • Memory hierarchy is the
organization of different types of
memory based on speed, cost, and
capacity to ensure efficient data
storage and access.
• Faster memory is smaller and
expensive (Registers, Cache).
• Slower memory is larger and
cheaper (HDD, Cloud Storage).
• Data moves between levels based
on need (frequently used data stays
in faster memory).
• Improves overall system
performance by balancing speed
and cost.

Memory Hierarchy
 Memory Management System (MMS)

A program with its data normally resides in auxiliary memory.

When a program or segment of program is to be executed , it is transferred to

main memory by the CPU with use of I/O processor.

MMS: Part of computer system that supervises the flow of information

between auxiliary memory and main memory
 Problem: Slow Main Memory Solution: Cache Memory
Cache • Modern processors are very fast, but
main memory (RAM) is much slower
• Cache is a small, fast memory
placed between the processor
Memory in comparison.
• If the processor has to wait too long
and RAM.
• It stores copies of frequently
for memory access, performance accessed data from main
decreases. memory.
• A solution is needed to reduce • If data is found in cache (cache
memory access time and speed up hit), access is very fast.
execution.
• If data is not in cache (cache
miss), it is fetched from main
memory, which is slower.
Cache
Memory
Cache Memory
• How does the system know which data and
instructions are needed in advance? Locality of
Reference: Programs tend to access nearby
instructions and data frequently.
• If one instruction is executed, nearby instructions are
likely to be used soon. The same principle applies to
data references, not just instructions.
• Cache stores a copy of frequently used main memory
content.
• When the CPU needs data, it first checks if it's in the
cache. If found, the data is delivered quickly from the
cache. If not found, a block of main memory
(including the required data) is loaded into the cache.
• Since nearby data is also fetched, future memory
requests are likely to be fulfilled from the cache. This
reduces the need to access slower main memory,
improving processing speed.
 Cache Memory: Locality of Reference
Locality of Reference is a pattern in how programs access memory. It has two main types:
1. Temporal Locality – If a piece of data is accessed once, it is likely to be accessed again
soon.
1. Example: A loop running multiple times uses the same instructions repeatedly.
2. Spatial Locality – If a piece of data is accessed, nearby data is also likely to be
accessed soon.
1. Example: Instructions stored close together (like a function) are used repeatedly.

Temporal and spatial locality help store frequently used data in smaller, faster memory. This makes the
processor feel like it has access to a large, fast memory, even though the actual large memory is slower.
Hit Ratio
Cache memory performance is measured using the Hit Ratio.

If the CPU finds the required data in the cache, it's a hit.

If the data is missing and found in main memory instead, it's a miss.

Hit Ratio = (Number of Hits) / (Total Memory Accesses).

A higher hit ratio means the cache is working well, following the locality of reference
(frequently used data stays closer to the CPU).
 Mapping Functions
• When the CPU requests data, a block of memory
is moved into the cache.
• If the CPU later needs any data from this block, it
can read it directly from the cache (faster access).
• Cache memory can store only a limited number of
blocks compared to main memory.
• A mapping function decides which blocks from
main memory are placed in the cache.
• If the cache is full and new data is needed, the
system must remove an old block to make space.
• A replacement algorithm decides which block to
remove.
 Mapping Functions
• Data from main memory is transferred to cache
memory using a process called mapping.
• There are three types of mapping techniques used
in cache organization:
• Associative Mapping
• Direct Mapping
• Set-Associative Mapping
 • Main memory has 32K words, each containing 12 bits.
• Cache memory can store 512 words at a time.
CACHE MEMORY: • Every word in cache has a duplicate copy in main memory
EXAMPLE • The CPU first sends a 15-bit address to cache.
• If the data is found in cache (hit) → CPU takes the 12-bit data
directly.
• If the data is not found in cache (miss) → CPU retrieves it from main
memory and stores it in cache for future use.
 • Associative memory is the fastest and most flexible way
to organize cache.
• It stores both the address and data of memory words.
Associative • Any location in cache can store any word from main
memory.
mapping • It allows a block of main memory to be stored anywhere
in the cache, instead of being restricted to a specific
location. This improves cache utilization and reduces
conflicts in memory access.
Associative mapping
• The CPU sends a 15-bit address to the argument register to
search for data in the cache. If the address exists in the cache,
the corresponding data is retrieved instantly.
• If the address is found (cache hit) → The data is sent to the
CPU.
• If the address is not found (cache miss) → The CPU fetches
data from main memory and stores it in the cache.
This method is efficient but expensive since it requires extra
hardware for searching all addresses quickly.
 • Direct Mapping is the simplest cache memory
Direct Mapping mapping technique.
• In this method, each block of main memory
maps to only one specific cache block.
• It is fast and easy to implement but may cause
frequent cache misses if multiple blocks map to
the same location.
 The CPU address (15 bits) is divided into two
parts:
• Index field (9 bits) → Helps locate a specific
Direct Mapping cache block.
• Tag field (6 bits) → Identifies the actual memory
block stored in the cache.
• The index field is used to find the location in
the cache.
• The tag field is checked to confirm if the right
data is stored in that cache location.
 Direct mapping
cache organization

Example 1: CPU wants to access

data from Address = 00 000
 Direct Mapping with block size of 8 words
• The index field is divided between block and
word field
• If block field is of 6 bits and word filed is of 3
bits, then cache is divided into 64 = 26 blocks
with each block having 8= 23 words.
• Every time a miss occurs, entire block of 8
words is transferred from main memory to Same block
cache memory. address with 8
different words in
• Advantage: Larger blocks improve hit ratio it
due to sequential nature of programs
 • If two different memory
addresses have the same
index but different tags,
they will keep replacing
Drawback of Direct each other in the cache.
• This can reduce
Mapping performance because the
CPU repeatedly fetches
data from main memory
instead of the cache.
 • Set-associative mapping is a middle-ground
approach between direct mapping and fully
associative mapping.
• It balances speed, flexibility, and efficiency in
cache memory.
Set associative • Each index in the cache can store multiple
mapping words (blocks) instead of just one.
• These multiple storage locations form a set.
• Example: A 2-way set-associative cache means
each set can hold two blocks of memory.
 • The CPU address is divided into three parts:
• Index: Identifies the set in the cache.
• Tag: Helps verify if the data is present.
• Offset: Specifies the exact location within a block.
• When accessing data:
Set associative • The index is used to locate the set.
mapping • Both tag values in the set are compared with the CPU
address.
• If a match is found (hit), the data is retrieved from the
cache.
• If no match (miss), the required word is fetched from
main memory and stored in the cache.
Set associative mapping Set associative cache organization for
set size of 2
• A 2-way set-associative cache means each set
stores 2 words of data.
• Each data word has 12 bits, and each tag has 6
bits.
• So, each cache entry requires 2 × (6 + 12) = 36
bits.
• The cache can store 512 words, and since each
entry holds 2 words of main memory, the total
storage capacity is 1024 words.
 • Advantage:
• Increase in set size improves the hit
ratio as more words with different
Set associative tags but same index address can
reside inside cache.
mapping • Drawback
• Increase in set size requires more
complex comparison logic
Handling Cache
Overload: Cache
Replacement
 In a cache system, the same data can be stored in both the
cache and the main memory at the same time.

This means two

If the cache version is modified different values exist for
Write but the main memory version is
not, inconsistencies may occur.
the same memory
address.
Policies in
Cache Memory To prevent mismatched data,
write policies are used:
Write-Through
Write-Back

These updating systems make sure that the cache and

memory stay synchronized and avoid data conflicts.
 • Only the cache is updated first.
• The main memory is updated later when the data
is removed from the cache.
• A flag (dirty bit) is used to mark modified data.
Write Back • Advantage: Reduces unnecessary writes to the
main memory, improving efficiency.
• Disadvantage: The main memory may have
outdated data until it is updated.
Example
 A write miss happens when the CPU tries
to write data, but the required memory
Handling Write location is not found in the cache. In such
Miss in Cache cases, there are two ways to handle the
situation:
Memory
1. Write Allocation (Fetch-on-Write)
2. Write Around
 • The missing data is first loaded into the cache from
main memory.
• Then, the CPU writes the new data into the cache.
• Later, the updated data may be written back to main
memory (depending on the write policy).
Write Allocation • Advantage:
(Fetch
- on-Write) • If the CPU needs the same data again, it is
already in the cache, improving performance.
• Disadvantage:
• Requires extra memory fetch operations, which
may slow down the process.
 • The data is written directly to main
memory without loading it into the cache.
• The cache remains unchanged.
• Advantage:
Write Around • Prevents cache pollution (useful when the
data is not likely to be needed again soon).
• Disadvantage:
• If the CPU needs the data again, it will cause
a cache miss and require fetching it later.
Write
Around
References
Chapter 12
Computer System Architecture-
Morris Mano (12.1, 12.2 and 12.5)