Isheanesu Mazivisa N02420944X

Innocent Dube N02421013R

Elshammah T Mpofu N02422475Q

PARALLEL COMPUTING LAWS Nkosilathi Moyo N02427417V

Prince Moyo N02421142Y

Bokang L Maphosa N02426535T

Shelton Tavarwira N02427070Y

James Dirori N02419913X

Naledi Ncube N02425117K

Munashe Chivazve N02422141W

This presentation is focused on the 3 fundamental laws that govern parallel and
distributed processing.

Amdahl’s Law Moore’s Law Gustavo’s Law

 • Proposed by Gene Amdahl in 1967
• It states that overall performance improvement is limited by the
unparallelizable (sequential) portion of a task, even with infinite
processors.
• Basically, what Amdahl says is that imagine you run a restaurant. Even
if you hire 50 more prep chefs, the restaurant can only go as fast as the
head chef can taste and the cashier can ring up orders.
Amdahl’s Law
• Basic Speedup Formula:

1
𝑆 𝑁 = 𝑃
1−𝑃 +𝑁

Where:
• 𝑆(𝑁) = Theoretical speedup with N processors
• 𝑃 = Parallelizable fraction of the program (0 ≤ 𝑃 ≤ 1)
• 𝑁 = Number of processors/cores
• 1 𝑃 = Sequential fraction of the program
 Example 1: Moderate Parallelization
Given: 𝑃 = 0.85 (85% parallelizable), 𝑁 = 8 processors

1 1 1
𝑆(8) = = = ≈ 3.90
0.85 0.15 + 0.10625 0.25625
(1 − 0.85) + 8

Amdahl’s Law 1 1
Maximum possible speedup: 1−0.85 = 0.15 ≈ 6.67
Detailed Calculations and
Examples
Example 2: Highly Parallel Application
Given: 𝑃 = 0.95, 𝑁 = 64 processors

1 1 1
𝑆(64) = = = ≈ 15.42
0.95 0.05 + 0.01484 0.06484
0.05 + 64
1
Maximum possible speedup: 0.05 = 20
 Implications for Parallel and Distributed Processing
• : Each additional processor yields progressively smaller performance
gains
• One slow step can ruin your speedup
• Emphasizes the importance of minimizing sequential components
Amdahl’s Law Practical Applications
Implications and Practical • Estimating if buying more processors is worth it
Applications • Finding and fixing the slow sequential parts
• Designing new parallel algorithms
• Predicting maximum possible speedup
 • Proposed by Gordon Moore in 1965
• It states that "Computer chips(transistors) get twice as powerful every two
years."
• The formula for Transistor Count Projection is :

𝑡−𝑡0

Moore’s Law 𝑇(𝑡) = 𝑇0 × 2 𝜏

Where:
• 𝑇(𝑡) = Transistor count at time t
• 𝑇0 = Initial transistor count at time 𝑡0
• 𝜏 = Doubling period (historically ~2 years)
• 𝑡, 𝑡0 = Time in years
 Example: Processor Evolution
1971: Intel 4004 - 2,300 transistors
2023: Apple M2 Ultra - 134 billion transistors

Using Moore's Law projection:

Moore’s Law
Detailed Calculations and 134 × 109
ln( 2300 )
Examples Doublings = ≈ 26 doublings
ln(2)

Years = 26 × 2 = 52 years(Matches 1971−2023)

 Implications for Parallel and Distributed Processing
• Helped transition from frequency scaling to core scaling
• Necessitated parallel programming models
• Led to heterogenous architectures ( CPU+GPU+Accelerators)

Moore’s Law
Implications
 • Proposed by John Gustaffson in 1988
• It states that “when you have more computers, don't just do the same job
faster—do a MUCH bigger job in the same time.”
• It is an optimistic version of Amdahl’s Law since it states that rather than
solving fixed problems faster, parallel computing enables solving larger,
more complex problems in the same time frame.
Gustavo’s Law • Basic Mathematical Formula:

Primary Equation:
𝑆(𝑁) = 𝑁 − 𝛼(𝑁 − 1)
Where:
• 𝑆(𝑁) = Scaled speedup with N processors
• 𝑁 = Number of processors
• 𝛼 = Sequential fraction of the scaled workload
 • Proposed by John Gustaffson in 1988
• It states that “when you have more computers, don't just do the same job
faster—do a MUCH bigger job in the same time.”
• It is an optimistic version of Amdahl’s Law since it states that rather than
solving fixed problems faster, parallel computing enables solving larger,
more complex problems in the same time frame.
Gustavo’s Law • Basic Mathematical Formula:

Primary Equation:
𝑆(𝑁) = 𝑁 − 𝛼(𝑁 − 1)
Where:
• 𝑆(𝑁) = Scaled speedup with N processors
• 𝑁 = Number of processors
• 𝛼 = Sequential fraction of the scaled workload
 Example 1: Scientific Computing Application
Given: 𝛼 = 0.05, 𝑁 = 100 processors
𝑆(100) = 100 − 0.05(100 − 1) = 100 − 4.95 = 95.05

Example 2: Data-Intensive Processing

Gustavo’s Law Given: 𝛼 = 0.01, 𝑁 = 1000 processors
Detailed Calculations and 𝑆(1000) = 1000 − 0.01(1000 − 1) = 1000 − 9.99 = 990.01
Examples
 Implications for Parallel and Distributed Processing
• Makes near linear speedup possible with scaled workloads
• Better reflects modern big data and scientific computing
• Encourages algorithm design for weak scaling

Gustavo’s Law Practical Applications

Implications and Practical • Scientific Computing: Climate modeling, molecular dynamics
Applications • Big Data Analytics: Large-scale data processing
• Machine Learning: Training on massive datasets
• Computer Graphics: Rendering complex scenes
 COMPARATIVE ANALYSIS AND INTERRELATIONSHIPS

Fundamental Assumptions Comparison

Aspect Amdahl's Law Gustafson's Law Moore's Law

Problem Size Fixed Scales with processors Not specified

Primary Constraint Sequential portion Memory/communication Physical limits

Time Consideration Minimize for fixed work Fixed time, maximize work Technology evolution

Philosophy Pessimistic/bounded Optimistic/scalable Observational/predictive

Practical Focus Parallelizing existing code Designing parallel algorithms Hardware development
 The trinity of Amdahl's Law, Moore's Law, and Gustafson's Law provides a
comprehensive framework for understanding parallel and distributed
computing. While Amdahl reveals the fundamental limitations, Gustafson
illuminates the practical opportunities, and Moore's Law charts the
technological evolution that makes both perspectives relevant. Together, they
Conclusion guide the design, implementation, and optimization of parallel systems,
remaining profoundly relevant in an era of heterogeneous, distributed, and
specialized computing architectures.