CSE309 - Computer

Architecture and Organization

Performance Issues
 Outline
• Basic measures of
• Designing for computer performance
performance
• Clock speed
• Microprocessor speed
• Performance balance
• Instruction execution
rate
• Improvements in chip
organization and • Calculating the mean
architecture
• Arithmetic mean
• Multicore • Harmonic mean
• MICs • Geometric mean
• GPGPUs
• Benchmark principles
• Amdahl’s Law
• Little’s Law • SPEC benchmarks

2
Desing for Performance
While cost of computers ↓ → performance and capacity ↑

Processors are so cheap.

Some applications that require the great power of today’s microprocessor-based systems.

■ Image processing
■ Three-dimensional rendering
■ Speech recognition
■ Videoconferencing
■ Multimedia authoring the driving factors behind the need to design
■ Voice and video annotation for performance
of files • Microprocessor speed
■ Simulation modeling • Performance balance
• Improvements in chip organization and
architecture
3
Desing for Performance – Microprocessor speed
Microprocessors → the performance is increased 4-5 times in every three years

Main memory → the capacity is increased almost 4 times in every 3 three years
dynamic random-access memory (DRAM)

No matter how fast microprocessors are, their full potential cannot be realized
unless they are given a constant and smooth stream of instructions to execute.

chip manufacturers processor designers

Try to produce more Try to develop more advanced

powerful processors techniques to run these powerful
processors efficiently
4
Desing for Performance – Microprocessor speed

1. Pipelining
These techniques;
1. Fetch, 2. Decode, 3. Operand fetch, 4. Execute, 5. Write-back
1. Pipelining
2. Branch prediction In pipelining, these steps done simultaneously.
3. Superscalar execution i.e. While instruction 1 is executed, instruction 2 is decoded,
4. Data flow analysis instruction 3 is fetched.
5. Speculative execution

Different componets of the processor is simulatneously used,

so the efficiency and speed is increased.
5
Desing for Performance – Microprocessor speed
2. Branch prediction
Running the program line-by-line (instruction-by-instruction)
These techniques; But, Conditions and If Statements?
1. Pipelining
2. Branch prediction if (x<5) {
do this It will go through if block or else block.
3. Superscalar execution } else {
4. Data flow analysis do this Then, it will skip the other.
5. Speculative execution }
Branch prediction is the processor’s ability to guess which
path (branch) will be taken next. If the processor makes a
correct prediction:
• It prefetches and buffers the necessary instructions in
advance
• This allows the processor to continue working without
6
any delay
Desing for Performance – Microprocessor speed
These techniques;
1. Pipelining 3. Superscalar execution
2. Branch prediction Normally, a processor executes only one instruction per
3. Superscalar execution clock cycle.
4. Data flow analysis This is the ability to issue more than one instruction in every
5. Speculative execution processor clock cycle.
In effect, multiple parallel pipelines are used.

4. Data flow analysis

If an instruction is independent from the results of other instruction, it can be executed instantly.
This prevents unnecessary delay.

5. Speculative execution
Using branch prediction and data flow analysis, processor try to predict instructions that are likely to be
executed later in the program flow.
Then it executes the predicted instructions in advance (before they are actually nedded).
So, the processor is always busy. 7
Desing for Performance – Performance balance
The performance of the processors has been grown so fast.
What about the other components?

Critical problem → data flow between processor and main memory

imbalance here causes to waste of time and performance

Ways to faster main memory;

1. wider bus data paths in DRAM
2. including a cache or buffer
3. including complex and efficient cache structures
4. including higher-speed and hierarchial buses between processor and memory
8
Desing for Performance – Performance balance
Ethernet modem
The problem of getting that (max speed)

data moved between processor Graphics display

and peripheral Wi-Fi modem

(max speed)

Hard disk
Strategies here also include;
• caching and buffering Optical disc

schemes Laser printer

• the use of higher-speed

Scanner
interconnection buses
• the use of more elaborate Mouse

interconnection structures Keyboard

• the use of multiple- 101 102 103 104 105 106 107 108 109 1010 1011
processor configurations Data Rate (bps)

Typical I/O Device Data Rates (Fig 2.1)

Figure 2.1 Typical I/O Device Data Rates 9
 Desing for Performance – Improvements in Chip
Improvements in Chip Organization and Architecture

Increase hardware speed of processor → shrinking logic

gate size

Increase size and speed of caches

More data The transfer

is stored is fast

The balance between size and speed is important !!

Change processor organization and architecture

→ Parallelism
10
[Link]
Desing for Performance – Improvements in Chip

But the problems !! A↓ ⇒ R↑

Symbol Meaning
R Resistance (measured in ohms, Ω)

1. Power more energy is required ρ

Resistivity of the material (a
constant)
more heat is generated
L Length of the wire (in meters)
Cross-sectional area of the wire
A
(in m²)
2. RC delay thinner interconnection = higher resistance
closer wires = higher capacitance d↓ ⇒ C↑
Symbol Meaning
3. Memory processors are so fast C Capacitance (measured in farads, F)
latency data access speed is slower Dielectric constant (permittivity) of
ε
the medium
data transfer speed is slower
Surface area of overlap between the
A
two conductors
d Distance between the two conductors 11
Multicore
Two-core chips, four-core chips,
The use of multiple processors on the same chip 8-core chips ….
more than 50 cores per chip

The use of multiple

processors on the Strategy is to use
same chip provides two simpler
the potential to processors on the
increase chip rather than
performance one more complex
without increasing processor
the clock rate

As caches became
larger it made With two
performance
processors larger
sense to create
two and then caches are
three levels of justified
cache on a chip
It is now common for the second-level
cache to also be private to each core.

12
Multicore Computer Structure (Fig 1.2)
MIC – GPU – GPGPU
many integrated core (MIC)
• Dozens to hundreds cores on a chip
(the number of cores much much higher than the multicore)
• Each cheap is a simple processor
• High pararllelism thanks to high number of cores

graphics processing units (GPUs)

• a core designed to perform parallel operations on graphics data
• used to encode and render 2D and 3D graphics as well as process video

general-purpose computing on GPUs (GPGPU)

• Since GPUs perform parallel operations on multiple sets of data, they are
increasingly being used as vector processors for a variety of applications that
require repetitive computations.
• This blurs the line between the GPU and the CPU 13
Two Laws

Two laws that are associated with performance

Amdahl’s Law and Little’s Law

14
Amdahl`s Law Note: the others; a memory cache
hierarchy, and speedup in memory
access time and I/O transfer rate
The use of parallel processors is one of the best way to improve the performance

Amdahl's Law is a fundamental principle used to estimate the potential performance improvement
when transitioning to parallel processing in computer systems.

f : fraction of the program that is parallelizable (0 ≤ f ≤ 1)

N : number of cores
1-f : fraction of the program that must be executed serially

No matter how many cores you add, the overall speedup is limited by the portion of the
task that remains serial.
Let`s say you have a program
1 1 1 1
• The parallelized part is 0.8, the rest is serial. SpeedUp = = = = = 2.5
𝑓 0.8 0.2 + 0.2 0.4
• The number of cores is 4. (1 − 𝑓) +
𝑁
(0.2) +
4
So, f=0.8, 1-f=0.2, and N=4 15
Amdahl`s Law
For workloads that are not fully parallelizable, even an infinite number of cores cannot deliver infinite speedup.

1
lim 𝑆𝑝𝑒𝑒𝑑𝑈𝑝 = f = 0 .9 5
N→∞ 1−𝑓

Spe dup
If f = 0.9, max speedup = 10 f = 0 .9 0
If f = 0.99, max speedup = 100

f = 0 .7 5

f = 0 .5

N um be r of Pr oce ssor s

The effect of number of cores in SpeedUp(Fig 2.4)

Figur e 2 .4 Am da hl’s La w for M ult ipr oce ssor s
16
 Little`s Law
The average number of items in a system is equal to the arrival rate multiplied
by the average time each item spends in the system.

L = λ⋅W
L : Average number of items (e.g., tasks, customers) in the system
Λ : Average arrival rate (e.g., tasks per second)
W : Average time an item spends in the system

•The system must be stable (input rate equals output rate over time).
•The averages must be taken over a sufficiently long time period.
•The law applies to a wide variety of systems, regardless of the distribution of arrivals or service times.

Little’s Law is a simple, yet powerful mathematical principle used in queueing theory and system
performance analysis. It helps to understand the relationship between the number of items in a system,
the arrival rate, and the average time items spend in the system.

17
[Link]
Measure of processor speed – Clock speed

All operations are governed by a system clock

All operations begin with the pulse of the clock

Speed of processor → measured in cycles per second

(Hertz, Hz) → it measures pulse frequency produced
by the clock → each pulse is a cycle
Measure of processor speed – Clock speed
Clock rate = clock speed = the rate of pulses
Clock cycle = clock tick = one pulse
Cycle time = time between two pulse

1 GHz processor received 1 billion pulses per second

(!) The clock rate design → the physical layout of the processor

(!) Most instructions on most

processors require multiple Simultaneously
clock cycles to complete. pipelining multiple
Some instructions may take instructions
only a few cycles, while others
require dozens.
Clock speed alone is not enough. 19
Measure of processor speed – Instruction execution rate
T: the processor time needed to execute a given program
Ic: instruction count, the number of machine instructions executed
for that program execution
CPI: cycles per instruction, average cycle of an instruction
𝜏: cycle time (𝜏 = 1/f) (Hz)

p: the number of processor cycles needed to decode and execute

the instruction
m: the number of memory references needed
k: the ratio between memory cycle time and processor cycle time
Table 2.1: Performance Factors and System Attributes
Instruction set
architecture
affects Ic and p

20
Measure of processor speed – Instruction execution rate

Ic: instruction count, the number of

machine instructions executed for that
program execution
CPI: cycles per instruction, average cycle of
an instruction
f: clock rate (Hz)
MIPS: millions of instructions per second

(1 * 45000) + (2 * 32000)
+ (2 * 15000) + (2 * 8000)
CPI = = 1.55 cyc.
100000 intsr.

(40 * 106)
MIPS = ≈ 25.8 sec
(1.55 * 106)

T = 100000 * 1.55 / 40 ≈ 3.87 ms 21

Measure of processor speed – Instruction execution rate

Consider the execution of a program that results in the execution of 2 million

instructions on a 400-MHz processor. The program consists of four major types
of instructions. The instruction mix and the CPI for each instruction type are
given below, based on the result of a program trace experiment:

22
Measure of processor speed – Calculation of mean
the use of benchmarks to compare systems involves calculating the mean
value of a set of data points related to execution time.

Harmonic mean has a

tendency to produce a
misleading result when the
data is skewed to larger
values or when there is a
small-value outlier

23
Measure of processor speed – Calculation of mean
the use of benchmarks to compare systems involves calculating the mean
value of a set of data points related to execution time.

f(x): a continuous monotonic function

24
Measure of processor speed – Calculation of mean

Type When to Use Example

– When values can be added directly Average execution time, avg memory
– When values are similar in scale access time, avg CPU burst time
Arithmetic
– When there are no extreme (outlier) values

– When dealing with percentages, ratios, or Performance benchmarking, relative

Geometric growth rates improvements, composite power efficiency
– When values combine multiplicatively scores

– When working with rates (speed, Average throughput (i.e. number of avg task
efficiency, etc.) a processor completes per time), execution
Harmonic
– When the average of reciprocals is more time on different processors, effective
meaningful bandwidth

25
Benchmarks and SPEC – Benchmarks

Benchmarks provide guidance to customers trying to decide which system to buy, and can be
useful to vendors and designers in determining how to design systems to meet benchmark goals.

Desirable characteristics of a benchmark program:

[Link] is written in a high-level language, making it portable across different

machines
[Link] is representative of a particular kind of programming domain or paradigm,
such as systems programming, numerical programming, or commercial
programming
[Link] can be measured easily
[Link] has wide distribution

26
Benchmarks and SPEC – SPEC benchmarks

computer performance measurements has led to the development of standardized

benchmark suites

A benchmark suite
● a collection of programs
● defined in a high-level language
● provide a representative test of a computer in a particular
application or system programming area

Standard Performance Evaluation Corporation (SPEC) is an industry consortium. It defines

and maintains the best known such collection of benchmark suites.

27
Benchmarks and SPEC – SPEC benchmarks

SPEC CPU2006
● Best known SPEC benchmark suite
● Industry standard suite for processor
■ SPECviewperf
intensive applications
■ SPECwpc ● Appropriate for measuring performance for
■ SPECjvm2008 applications that spend most of their time
■ SPECjbb2013 (Java doing computation rather than I/O
Business Benchmark) ● Consists of 17 floating point programs written
■ SPECsfs2008 in C, C++, and Fortran and 12 integer
■ SPECvirt_sc2013 programs written in C and C++
● Suite contains over 3 million lines of code
● Fifth generation of processor intensive suites
from SPEC
28
Benchmarks and SPEC – SPEC benchmarks
Terms Used in SPEC Documentation
Benchmark
Peak metric
A program written in a high-level
This enables users to attempt to optimize system
language that can be compiled and
performance by optimizing the compiler output
executed on any computer that
Speed metric
implements the compiler
This is simply a measurement of the time it takes
System under test
to execute a compiled benchmark
This is the system to be evaluated
Used for comparing the ability of a computer to
Reference machine
complete single tasks
This is a system used by SPEC to
Rate metric
establish a baseline performance for all
This is a measurement of how many tasks a
benchmarks
computer can accomplish in a certain amount of
Each benchmark is run and measured
time
on this machine to establish a reference
This is called a throughput, capacity, or rate
time for that benchmark
measure
Base metric
Allows the system under test to execute
These are required for all reported
simultaneous tasks to take advantage of multiple
results and have strict guidelines for
processors 29
compilation
1
2
3
4
Table 2.5
5
SPEC
6 CPU2006
7 Integer Benchmarks
8
9
10
11
12
30
1
2
3
4
5
6
7 Table 2.6
8
9 SPEC
10 CPU2006
11 Floating-Point
12
Benchmarks
13
14
15
16
31
17
Benchmarks and SPEC – SPEC benchmarks
SPEC Benchmark steps
(SPEC CPU2006 integer benchmark example)
1. Compile and run each program on the system under test for 3 times.
12 programs to eliminate the bias
caused by variations
Then, find the median of 3 recorded time results for each program

2. Calculate run time ratio for each program

i → the program
Tref → reference time
Tsut → recorded time for system under test

3. Calculate the overall benchmarking metric using geometric mean of 12 runtime ratios

32
 33
Step 1 Step 2
Step 1 Step 2
34
3. Calculate the overall benchmarking metric using geometric mean of 12 runtime ratios

Step 3 for Sun Blade 1000

rG = 3.12

Step 3 for Sun Blade X6250

rG = 71.59

35