Computer Organization

and Architecture
ITT 303
Introduction and Basics

Dr. Janibul Bashir

NIT Srinagar
Autumn 2023

[Link]
MISSION
Build fundamentally better computers that are:
1. Secure/Reliable/Safe
2. Energy Efficient - Memory Centric instead of Compute
Centric
3. Low Latency - High Performance
4. Architectures for ML, Medicine, Genomics etc.

2
GOALS
– Understand the basics
– Understand the principles (of design)

Based on such understanding:

– learn how a modern computer works underneath
– evaluate tradeoffs of different designs and ideas
– implement a principled design (a simple microprocessor)

3
WHAT TO STUDY?

4
WHY TO STUDY?
– design better hardware
– design better software
– design better systems
– make better tradeoffs in design
– understand why computers behave the way they do
– solve problems better
– think “in parallel”
– think critically

5
WHY COMPUTING?

SOLVE PROBLEMS

GAIN INSIGHTS

“The purpose of computing is [to gain] insight” (Richard Hamming)

6
HOW IS COMPUTING DONE?

ORCHESTRATING ELECTRONS

7
Where are we at present?
“C” as a model of computation
Computer Prg
Programmer’s view of a
computer system works

■ How does an assembly

program end up executing as Architect/microarchitect’s view:
digital logic? How to design a computer that
meets system design goals.
■ What happens in-between?
Choices critically affect both
■ How is a computer designed the SW programmer and
using logic gates and wires the HW designer
to satisfy specific goals?
HW designer’s view of a
computer system works
Digital Logic Digital logic as a
model of computation
8
Meeting point?
Machine Instructions:
High level program • A binary stream.
• Symbolic notations to machine instructions: Assembly
instruction.
Convert High level • Assembler: Converts symbolic to machine instruction.
System instructions to machine • Compiler: High level to assembly.
software’s instructions
Compiler
Operating system

Allocate memory
Handling input output
Digital Logic And many other functions

9
 Levels of Transformation
Problem
Algorithm
Algorithm Program/Languag
Step-by-step procedure that is e
Runtime System
guaranteed to terminate where
(VM, OS, MM)
each step is precisely stated
and can be carried out by a ISA (Architecture) ISA (Instruction Set Architecture)
computer Interface/contract between
Microarchitecture SW and HW.
Microarchitecture Logic
An implementation of the ISA Circuits Digital logic circuits
Electrons Building blocks of micro-arch (e.g., gates)

10
Why Study Computer Architecture?

❑ All the major software companies are now developing their own
chips: Microsoft Pluton Processor, Google (building for pixels),
Facebook.
❑ Enable better systems: Make computers fast and reliable.
❑ Design application specific chips.
❑ The industrial shift has resulted in many motivating problems such
as addressing memory bottlenecks, security concerns, reliability
problems.
❑ It makes you a better programmer.

11
Architecture Today
New Demands of
the users at the Problem
higher level:
Algorithm
e.g., new
objectives Program/Language
Up
ok
Lo

• Co-Design software and hardware

Runtime System • Optimize across top and bottom
(VM, OS, MM)

Loo ISA (Architecture)

kD Microarchitecture
ow
n
Logic
Many new issues Circuits
are from the
Electrons
bottom: thermal
and reliability
12
The Power of Abstraction
■ Levels of transformation create abstractions
❑ Abstraction: A higher level only needs to know about the
interface to the lower level, not how the lower level is
implemented
❑ E.g., high-level language programmer does not really need to
know what the ISA is and how a computer executes instructions

■ Abstraction improves productivity

❑ No need to worry about decisions made in underlying levels
❑ E.g., programming in Java vs. C vs. assembly vs. binary vs. by
specifying control signals of each transistor every cycle

■ Then, why would you want to know what goes on

underneath or above?

13
Crossing the Abstraction Layers
■ As long as everything goes well, not knowing what happens
in the underlying level (or above) is not a problem.

■ What if
❑ The program you wrote is running slow?
❑ The program you wrote does not run correctly?
❑ The program you wrote consumes too much energy?

■ What if
❑ The hardware you designed is too hard to program?
❑ The hardware you designed is too slow because it does not provide the
right primitives to the software?

■ One goal of this course is to understand how a processor

works underneath the software layer and how decisions
made in hardware affect the software/programmer
14
 Problem: Information Loss Across Layers
Higher level information is not visible to HW
Access Patterns Code Optimizations Data Types

These get lost while translation

Software Solution: More expressive interfaces

Hardware Express Memory - Transfer

higher level info to HW
100001111…
10101011...

Low Level Instructions

15
WHY IS ARCHITECTURE EXCITING TODAY?
MANY exciting things are happening today:

– Make systems reliable and secure (RowHammer and Side channel attacks).
– Make performance efficient systems (more demanding workloads)
– Energy efficiency

16
Challenges and Future opportunities
– Training a single AI model can emit as much carbon as five cars in their lifetime
(MIT Review) – develop energy efficient systems
– Communication dominates arithmetic operations - a memory access consumes
100-1000X more power than a complex arithmetic operation. (move from compute
centric to memory centric systems, computing near (or using) memory)
– Non Volatile Memories (Intel optane and phase change memories)
– Application specific accelerators.
– Security challenges - RowHammer (induce bit flips in DRAM chips)

17
Novel concepts in Architecture Today
■ New computing paradigms
❑ Processing in Memory
❑ Neuromorphic computing

■ Software-Hardware Co-Design
Exciting time to understand and dwell in
❑ Machine learning architectures. the field of computer architectures
❑ Application specific architectures.
❑ Neural Network accelerators.

■ Novel Memories.
❑ Phase Change Memory
❑ Non-volatile Main Memory

18
An Example: Multi-Core Systems
Multi-Core
Chip

L2 CACHE 1
L2 CACHE 0
SHARED L3 CACHE

DRAM INTERFACE

DRAM BANKS
CORE 0 CORE 1

DRAM MEMORY
CONTROLLER

L2 CACHE 2

L2 CACHE 3
CORE 2 CORE 3

*Die photo credit: AMD Barcelona

19
Example: Apple M1 Max system on chip

20
Tesla Chip

21
Unexpected Slowdowns in Multi-Core
High priority

No change even by making matlab low priority

Reason: Memory performance hog

Low priority

(Core 0) (Core 1)
Moscibroda and Mutlu, “Memory performance attacks: Denial of memory service
in multi-core systems,” USENIX Security 2007.
22
A Question or Two
■ Can you figure out why there is a disparity in slowdowns if
you do not know how the processor executes the
programs?

■ Can you fix the problem without knowing what is

happening “underneath”?

Reason For Slowdown: Because DRAM controller is giving high priority to matlab requests.

Homework 1:Read the paper

"Memory Performance Attacks: Denial of Memory Service in Multi-Core Systems"

23
Now That We Know What Happens Underneath
■ How would you solve the problem?

■ What is the right place to solve the problem?

❑ Programmer? Problem
❑ System software? Algorithm
❑ Compiler? Program/Languag
❑ Hardware (Memory controller)? e
Runtime System
❑ Hardware (DRAM)? (VM, OS, MM)
❑ Circuits? ISA (Architecture)
Microarchitecture
Logic
Circuits
Electrons

24
Course Info?
■ Instructor: Dr. Janibul Bashir
❑ janibbashir@[Link]
❑ [Link] [Link]/
❑ PhD from IIT Delhi, worked at Samsung Research India.

■ Lectures:
❑ MWTF 9:00am-10:40am

■ Assignment
■ 3 Assignments with fixed deadlines.
■ 2 quizzes, 1 Midterm, 1 Major, 1 Viva
■ No form of cheating will be tolerated.

25
What Will You Learn
■ Computer Architecture: The science and art of designing, selecting, and
interconnecting hardware components and designing the hardware/software
interface to create a computing system that meets functional, performance,
energy consumption, cost, and other specific goals.

■ Traditional definition: “The term architecture is used here to describe the

attributes of a system as seen by the programmer, i.e., the conceptual
structure and functional behavior as distinct from the organization of the
dataflow and controls, the logic design, and the physical implementation.”
Gene Amdahl, IBM Journal of R&D, April 1964

26
What Will You Learn?
■ Fundamental principles and tradeoffs in designing the hardware/software
interface and major components of a modern programmable microprocessor
❑ Focus on state-of-the-art (and some recent research and trends)
❑ Trade-offs and how to make them

■ How to design, implement, and evaluate a functional modern processor

❑ Semester-long assignments.
❑ Using simulators.

■ How to dig out information, think critically and broadly

■ How to work even harder!

27
Takeaway

It all starts from the basic building blocks and design principles

And, knowledge of how to use, apply, enhance them

28
Basic Building blocks
– Electrons
– Transistors
– Logic Gates
– Combinational Logic Circuits
– Sequential Logic Circuits
– Storage Elements and Memory
…
– Cores
– Caches
– Interconnect
– Memories
...

29
Course Structure
System design
Software interface Processor Design
7 8 Memory
6
Processor system
Pipelining
2 design
3
Language
Assembly 4

Instruction set architecture

5

Design of a simple processor

of bits Building blocks: Computer
language
gates, registers, arithmetic
11
and memories
Multiprocessors

Software Hardware Hardware

engineer designer designer

30
Recommended Books

31
Thank You

32