Next Generation Computer Architecture
-By Dr Kushal Dinkar Badgujar,
COEP Technological University
Module 2
Advanced Computer architecture
concepts
[Link]
The basic computer architecture illustrated below consists of the following main components:
I. Memory: Stores both instructions and data in a unified space.
II. Control Unit: Fetches instructions from memory, decodes them, and directs execution.
III. Arithmetic Logic Unit (ALU): Performs arithmetic operations and logical comparisons.
IV. Accumulator: A special register within the ALU that temporarily holds intermediate results.
V. Input Unit: Provides data and instructions from external sources.
VI. Output Unit: Sends processed results to external devices.
Figure 1: Basic Computer Architecture
1
Data Flow
The execution cycle in this basic architecture is as follows:
I. Fetch: The Control Unit retrieves an instruction from memory.
II. Decode: The instruction is interpreted by the Control Unit.
III. Execute: The ALU carries out the operation, using the Accumulator for temporary storage.
IV. Store/Output: The result is either stored back into memory or sent to the Output Unit.
Unlike earlier mechanical calculators, a computer built on this model stores both its program instructions
and its data in the same Main Memory.
Figure 2: I/O interfacing
The computing world has produced a wide variety of Instruction Set Architectures (ISAs), each tailored
to different needs. Among them RISC and CISC are broad design [Link] (Reduced Instruction
Set Computing) and CISC (Complex Instruction Set Computing) centers are two fundamentally different
strategies for designing processor hardware and instruction sets.
Table 1: Comparison of RISC and CISC Architectural Paradigms
Design Attribute RISC CISC
Primary Goal Executing basic commands with ex- Reducing the total instruction count
treme speed. per task.
Instruction Format Consistent, fixed-width encoding. Varied, flexible instruction lengths.
Cycle Requirements Aimed at single-cycle completion. Often requires multiple cycles per in-
struction.
Data Interaction Explicit Load/Store segregation. Allows direct memory-to-processor
operations.
Register Strategy Extensive general-purpose register Compact register usage with higher
file. memory dependency.
Control Logic Hardwired, simplified instruction de- Sophisticated, microcode-based
coding. translation.
Common Architectures ARM, RISC-V, MIPS. x86 (Intel/AMD), Motorola 68k.
2
The pentium was the first CISC architecture based processor to move beyond the traditional single-pipeline
execution model, introducing superscalar architecture to consumers. (Fig.3).
Figure 3: Pentium Architecture
Key Architectural Features
The Intel Pentium architecture, significant advancement in processor design by utilizing a superscalar approach.
The design highlights several critical components defined in this era of processing:
I. Dual-Pipeline Execution (u-Pipeline and v-Pipeline): The hallmark of the Pentium. It features
two parallel execution pipelines, allowing the CPU to execute two instructions simultaneously per clock
cycle, provided those instructions do not conflict.
II. Instruction Decode & Prefetch Buffer: The CPU uses a prefetch buffer to grab chunks of code from
memory before they are needed, which are then passed to the decoders to prepare them for the dual
pipelines.
III. Split L1 Cache (Code & Data): The Pentium features separate caches for code and data (8KB each).
By splitting these, the CPU can fetch instructions and data at the same time, preventing the "memory
bottleneck" that limited older chips.
IV. Floating Point Pipeline: Unlike the 486 (where a math co-processor was often a separate chip), the
Pentium integrated a dedicated, high-performance floating-point unit directly into the silicon to accelerate
scientific and graphical calculations.
V. Branch Target Buffer (BTB): This component predicts whether a program "jump" (branch) will
occur. If the guess is correct, the pipeline remains full and efficient; if it fails, the pipeline must be cleared,
which creates a performance penalty. BTB helps by predicting the branch target address before execution
finishes.
VI. Bus Interface Unit: This manages communication with the motherboard, ensuring that the high-speed
internal work of the chip stays synchronized with the slower system RAM.
VII. Translation Lookaside Buffer (TLB) TLB is a specialized high-speed hardware cache located within
the processor’s Memory Management Unit (MMU). Its primary function is to cache recent linear-to-
3
physical address translations, eliminating the latency of walking through main-memory page tables on
every memory reference.
The Pentium architecture is classified as a CISC (Complex Instruction Set Computer) architecture.
While it implemented performance-enhancing features from RISC philosophy—such as pipelining and super-
scalar execution—it maintained full backwards compatibility with the x86 instruction set by using internal
hardware to decode complex instructions into simpler, RISC-like micro-operations.
2.2 Understanding RISC architecture with its implementation
We want to compute:
9
F = · C + 32
5
with C = [Link] this design, the constant 32 is supplied directly by the Immediate and Branch Unit during
decode, so the conversion is performed in a single instruction [Link] us understand the instruction flow
using RISC architecture.
Figure 4: RISC pipeline
Pipeline Behavior
- The Immediate and Branch Unit injects the constant 32 directly during decode, eliminating the need for a
separate load instruction.
- The ALU combines multiplication and addition in the execute stage, producing the final result in one pass.
- This demonstrates how RISC pipelines efficiently handle arithmetic with immediates, keeping instruction
flow simple and predictable.
Final Result
25◦ C −→ 77◦ F
Consider another example involving dependency between instructions as follows:
4
Table 2: Celsius → Fahrenheit Example
Stage What Happens Action
Instruction Fetch CPU fetches the instruction from instruc- Fetch instruction: F = (9/5) ∗ C + 32.
(IF) tion memory (i-cache).
Operand Fetch & Decode opcode, identify operands, read Read C = 25 from register file. Immediate
Decode (OF) registers, and load immediate constants. and Branch Unit supplies constant 32 and
multiplier 95 .
Execute (EX) ALU performs arithmetic operations. Compute (9/5) · 25 = 45, then add imme-
diate 32 → 77.
Memory Access If instruction is load/store, access data Not required here (all operands are in reg-
(MA) memory (d-cache). isters or immediate).
Register Write- Result written back into register file. Write 77 into destination register F .
Back (RW)
[Link] r1 , r2 , r3
[Link] r4 , r1 , r3
Figure 5: RISC Sequence flow
Figure 6: Stall of instructions
5
The data hazard caused by dependencies between instructions is of 3 types:
[Link] (Read after write)
[Link] (write after read)
[Link] (write after write)
WAR and WAW dependences are there because we have a finite number of registers.
In-Order Implementation Problems :
1. Structural Hazard: Not possible for 5-stages pipeline.
2. Data Hazard: Wrong data is read.
3. Control Hazard:Instructions are fetched using wrong path.
The question is how to tackle these problems?
2.2.1 Data forwarding
Forwarding is used resolve pipeline data dependencies by routing results from one stage directly to a subsequent
stage before the official write-back occurs.
Figure 7: Forwarfing unit
We have following options to forward the data from later to earlier stages:
1. RW → OF,
2. RW → EX,
3. RW → MA,
4. MA → EX.
Multiplexers are used to combine the data from later stage to earlier stage as follows:
Figure 8: Architecture by S. Sarangi
6
2.3 Performances
I. Performance Equation:
Programs Programs Instructions Cycles
P = = × ×
Seconds Instructions Cycles Seconds
This equation consists of three components: compiler efficiency (programs/instructions), architectural
efficiency (instructions/cycles) also called IPC, and clock frequency (cycles/seconds). It helps engineers
identify exactly which part of the system is the primary performance bottleneck.
II. CPU Time:
CPU time = Instruction count × CPI × Clock cycle time
This calculates the total time a processor takes to run a program. By breaking it down into instruction
count, average cycles per instruction (CPI), and clock period, designers can optimize specific areas,
such as using better algorithms to lower instruction counts or pipelining to reduce [Link] is reciprocal
of CPI.
III. Relative Performance:
Execution timeY PerformanceX
n= =
Execution timeX PerformanceY
This metric quantifies how much faster one system (X) is compared to another (Y). If the resulting
factor n is greater than 1, system X is faster, providing a standard, objective way to measure the success
of a new hardware design or optimization.
IV. Amdahl’s Law:
1
Speedupoverall =
(1 − Fractionenhanced ) + Fractionenhanced
Speedupenhanced
This law identifies the limits of performance improvement. It states that the total speedup is constrained
by the portion of the task that cannot be enhanced; therefore, spending excessive resources optimizing
a tiny fraction of a program yields diminishing returns.
V. Dynamic Energy:
1
Energydynamic ∝ × Capacitive load × Voltage2
2
Dynamic energy represents the energy consumed by charging and discharging the capacitive load of
transistors whenever they switch states. Because voltage is squared, reducing the operating voltage is
the most effective strategy for increasing energy efficiency in portable devices.
VI. Dynamic Power:
1
Powerdynamic ∝ × Capacitive load × Voltage2 × Frequency
2
Dynamic power builds upon dynamic energy by factoring in the switching frequency. High-performance
processors must balance high clock speeds (frequency) with voltage levels to prevent the chip from
exceeding thermal design power (TDP) limits.
VII. Static Power:
Powerstatic ∝ Currentstatic × Voltage
Static power, or leakage power, is consumed by the transistor even when it is not switching. As
transistors shrink in modern manufacturing, static power has become a significant portion of total
power consumption, posing a major challenge for idle power management.
7
VIII. Availability:
MTTF
Availability =
MTTF + MTTR
Availability measures the percentage of time a system is functional. It is defined by the Mean Time To
Fail (MTTF) and Mean Time To Repair (MTTR). Increasing availability involves either increasing the
reliability of components or improving diagnostic and repair speeds.
IX. Die Yield:
1
Die yield = Wafer yield ×
(1 + Defects/Area × Die area)N
This estimates the proportion of functional chips produced on a silicon wafer. Since manufacturing
defects are statistically distributed, smaller chips have higher yields, while larger, more complex chips
(N ) face higher risks of containing fatal defects.
X. Averaging Methods:
v
n n u n
1X X uY
AM = Timei ; WAM = Weighti × Timei ; GM = t
n
Timei
n i=1 i=1 i=1
These methods summarize performance benchmarks. The Arithmetic Mean (AM) is a simple average;
the Weighted Arithmetic Mean (WAM) accounts for the significance of specific workloads; and the
Geometric Mean (GM) provides a consistent ratio-based comparison across different system architectures.
XI. Average Memory-Access Time (AMAT):
AMAT = Hit time + Miss rate × Miss penalty
Hit time is the time required to access data from the cache when the data is found there (a "hit").AMAT
calculates the average cost of memory accesses. It demonstrates that performance is impacted not just
by cache speed (hit time), but also by the frequency of misses and the latency of fetching data from the
main memory (miss penalty).
XII. Misses per Instruction:
Misses per instruction = Miss rate × Memory access per instruction
This metric provides a direct way to see how cache performance affects the overall instruction stream. By
understanding how many memory misses occur for every executed instruction, architects can determine
the impact of the cache hierarchy on the total CPU time.
XIII. Cache Index Size:
Cache size
2index =
Block size × Set associativity
This formula determines how many index bits are required to map memory addresses into cache sets. It
defines the structural relationship between the total capacity of the cache and how that memory is
organized and partitioned.
XIV. Power Utilization Effectiveness (PUE):
Total Facility Power
PUE =
IT Equipment Power
PUE is the industry-standard metric for data center efficiency. It compares the total energy consumed
by the facility to the energy used solely for IT computing equipment. A PUE closer to 1.0 indicates
less energy is wasted on overhead, such as cooling and lighting.
8
Numerical Example
Imagine you have a program that takes 100 seconds to run on a single-core processor.
Identify the Fractions=
[Link] analyze the code and find that 80 seconds of the task can be parallelized (Fractionenhanced = 0.8, or
80%). [Link] remaining 20 seconds must run sequentially and cannot be optimized (1 − 0.8 = 0.2, or 20%).
Apply an Extreme Optimization (Infinite Hardware)
Suppose you upgrade your hardware so that the parallelizable portion runs infinitely fast (Speedupenhanced =
∞).
Calculate Overall Speedup
1
Speedupoverall =
(1 − 0.8) + 0.8
∞
1 1
Speedupoverall = = = 5x
0.2 + 0 0.2
2.4 Instruction-Level Parallelism (ILP)
Pipelining, Superscalar architecture, and Out-of-Order (OoO) processing are all specific techniques used to
exploit and maximize Instruction-Level Parallelism (ILP).
Instruction-Level Parallelism (ILP) is the inherent measure of how many instructions in a program can
be executed simultaneously without affecting the final result. If a program has high ILP, it means many
instructions are independent of one another.
Pipelining exploits ILP by overlapping the different stages of multiple instructions (Fetch, Decode, Execute,
etc.), ensuring the CPU hardware is always working on different parts of multiple instructions at the same
time.
Superscalar Architecture exploits ILP by providing the physical hardware redundancy (multiple execution
units) necessary to actually run those independent instructions in parallel.
Out-of-Order Processing acts as the "smart scheduler" that finds hidden ILP. It looks ahead in the code
to identify independent instructions that can be executed now, even if they appear later in the program,
preventing the CPU from stalling while waiting for a dependent instruction to finish.
2.5 Superscalar architecture
A superscalar processor contains multiple functional units (like ALUs, Load/Store units, and Floating-Point
units) that can operate simultaneously. This means the CPU can fetch, decode, and execute several instructions
at once, provided they are independent.
9
Figure 9: Superscalar architecture
Superscalar Architecture
I. Instruction Fetch Unit
Working Principle: Fetches multiple instructions per clock cycle using wide buses and branch prediction.
Working Explained: Retrieves instruction bundles (2–4 at once) to keep the pipeline full and ready for
parallel decoding.
II. Instruction Decode Unit
Working Principle: Decodes several instructions simultaneously, identifying their types and dependencies.
Working Explained: Splits the fetched bundle into individual operations, checks for hazards, and assigns
them to suitable execution units.
III. Operand Fetch Unit
Working Principle: Supplies operands to execution units using register renaming and dependency
tracking.
Working Explained: Retrieves data from registers or memory, resolves conflicts, and prepares operands
for concurrent execution.
IV. Execution Units
Working Principle: Multiple functional units (ALUs, Load/Store, Floating-Point) operate in parallel.
Working Explained:
• ALUs handle integer arithmetic and logic.
• Load/Store units manage memory access.
• Floating-Point unit executes complex mathematical operations.
Independent instructions are dispatched to these units simultaneously.
V. Write-Back Unit
Working Principle: Collects results from all execution units and updates registers or memory in correct
order.
Working Explained: Synchronizes outputs, ensuring program correctness even when instructions finish
at different times.
10
Overall Working Flow
• Fetch: Multiple instructions are fetched together.
• Decode: Instructions are decoded in parallel, dependencies checked.
• Dispatch: Independent instructions are sent to different execution units.
• Execute: Several instructions run simultaneously.
• Write-Back: Results are stored back in registers or memory.
2.6 Out-of-order pipilines
Consider instruction as follows:
1:add r3,r1,r2
2:add r4,r3,5
3:sub r3,r1,7
Instructions 1 and 2 have RAW depedency, 2 and 3 have WAR while 1 and 3 have WAW [Link]
can make them partially independent.
Figure 10: Out of order pipelines
2.7 Parallel fetch
Figure 11: Fetch and decode stages for advanced processor
In a standard processor, instructions are fetched sequentially based on the Program Counter (PC). To
improve this, parallel fetch logic attempts to retrieve an entire "instruction packet" or block of instructions in
a single [Link] components of this operation include:
11
Branch Prediction: Since a block of instructions might contain conditional jumps, the processor must
predict the outcome of these branches before the instructions are even executed to ensure it fetches the
correct subsequent code.
Instruction Cache (i-cache): The i-cache is organized to support wide reads, allowing the hardware to pull
multiple instructions at once instead of individual words.
Decoding: Once a packet is fetched, the decoder must be capable of processing multiple instructions in
parallel to feed them into the execution units.
This approach helps prevent the fetch stage from becoming a bottleneck, especially in high-performance
Out-of-Order (OOO) processors that rely on a constant supply of instructions to keep their execution units
busy. By keeping the pipeline full, the processor can achieve higher instructions-per-cycle (IPC) rates.
2.8 Supercomputer for scientific studies
Figure 12: Indian Institute of Tropical Meteorology, Pune has a supercomputer called Prithvi (Earth weather
monitoring device), used for climate research and operational forecasting.
This architecture is designed to balance high-speed computational power with massive data storage and
robust management layers, typical of modern supercomputing facilities.
• Computational Infrastructure
– Login Nodes (8 nodes): Entry point for users to access the HPC cluster.
– Compute Nodes (384 nodes): Standard processing units for general-purpose parallel computing.
– High Memory Nodes (36 nodes): Optimized for memory-intensive tasks.
– GPU Nodes (22 nodes): Equipped with NVIDIA Tesla V100s for AI and accelerated computing.
• Storage Systems
– Primary Storage (1.5 PiB): High-speed capacity for active projects (PiB is 250 bytes).
– Archival Storage (500 TiB): Long-term data retention and backup.
• Management and Orchestration
12
– Service/Management Nodes (8 nodes): Administrative overhead and cluster service orchestra-
tion.
– Master Nodes (2 nodes): Overall cluster control and job scheduling.
• Interconnect and Communication
– Networking Fabric: Utilizes InfiniBand EDR (100 Gbps) for high-speed, low-latency inter-node
communication via L1 and L2 switches.
– Network Protocols: Supports 1G (Secondary), 10G, IPMI (out-of-band management), and
InfiniBand.
– Security: Firewall positioned at the entry point to secure user access.
Figure 13: Indian Institute of Tropical Meteorology, Pune has a supercomputer called Prithvi (Earth weather
monitoring device), used for climate research and operational forecasting.
13