IFB 206

KOMPUTASI PARALEL DAN

SISTEM TERDISTRIBUSI
(parallel computing and
distributed system)

Parallel Computing

Lisa Kristiana PhD

Informatics – Fall 2026
Course Content

Why Use Types of

Parallel How It Works? Parallel
Computing? Computing
Learning outcomes
(Capaian pembelajaran)
• Memahami fungsi komputasi dengan cara parallel
• Memahami cara kerja komputasi parallel
• Mampu mendefinisikan tipe-tipe arsitektur komputasi paralel
Why Use
Parallel
Computing?
• Some problems are too
big or complex for a
single processor to
handle efficiently.
• By splitting the work,
tasks can be
completed much
faster than if done one
after the other.
Real-
World
Analogy:
Cooking!
How It Looks
Inside a
Computer?
• A modern CPU has multiple
cores
• Each core works on a different
part of the same problem
• Results are combined at the
end
Sequential Program – Serial Computing
• Runs on a single core.
• Processes one number at a time.
Pseudocode: SERIAL PROCESS

Sum = a + b + c + d Step 1: total = 1

(one after another) Step 2: total = 3
Step 3: total = 6
Step 4: total = 10
n = 5 Step 5: total = 15
total = 0 Final Sum = 15
print("Serial Computation")
for i in range(1, n + 1):
total += i
print(f"step {i}: total = {total} ")

print("Final Serial sum is", total)

 Parallel Program – Parallel Computing
from multiprocessing import Process, Queue • Uses multiple cores (e.g., 4 cores).
def partial_sum(start, end, q, pid): • Splits work across cores (each core
s = sum(range(start, end + 1)) checks 1 number simultaneously).
print(f"Process {pid}: sum({start} to {end}) = {s}")
[Link](s) Pseudocode:
if __name__ == "__main__":
Sum1 = a + b
q = Queue() Sum2 = c + d
Final Sum = Sum1 + Sum2
p1 = Process(target=partial_sum, args=(1, 3, q, 1))
p2 = Process(target=partial_sum, args=(4, 5, q, 2))
Parallel Computation:
[Link]()
[Link]()
Process 2: sum(4 to 5) = 9
[Link]()
Process 1: sum(1 to 3) = 6
[Link]() Final Parallel Sum = 15

total = [Link]() + [Link]()

print("Final Parallel Sum =", total)
 Klasifikasi ini dikenal sebagai Flynn’s Taxonomy,
berdasarkan
Tipe Instruction stream (alur instruksi) dan Data
stream (alur data), yaitu:
Arsitektur • SISD (Single Instruction, Single Data)
• SIMD (Single Instruction, Multiple Data)
Komputasi • MISD (Multiple Instruction, Single Data)

Paralel • MIMD (Multiple Instruction, Multiple Data)

SISD (Single Instruction,
Single Data) -1
SISD represents the simplest and most traditional form of computing
• Single Instruction: One instruction is executed at a time by a single
control unit.
• Single Data: Each instruction operates on a single data stream (one
piece of data at a time).
Examples
• Intel 8086 (early x86 CPU)
• ARM Cortex-M (microcontrollers)
• Modern single-core processors (e.g., in IoT devices)
SISD (Single Instruction, Single Data) -2
Advantages:
• Simple to design and program.
• Deterministic behavior (no race conditions or concurrency issues).
• Efficient for linear tasks (e.g., basic arithmetic, small-scale data
processing).
Drawbacks:
• Performance bottleneck: Cannot speed up tasks via parallel
processing.
• Unsuitable for modern workloads requiring parallelism (e.g., AI, big
data).
• Limited by clock speed scaling (physical constraints on Moore's Law).
PU = Processing Unit
SIMD (Single Instruction, Multiple Data)-1
• A single instruction
• Is executed simultaneously
• On multiple data elements

Key Characteristics:

• Single control unit

• Synchronous execution
• High efficiency for large data sets
• Best suited for data-level parallelism
 SIMD (Single Instruction, Multiple Data)-2
Implementation of SIMD:
1. GPUs (Graphics Processing Units)
• Thousands of lightweight cores
• Use SIMD/SIMT execution models
• Widely used in: Machine learning, Image and
video processing, Scientific computing
2. Vector Processors
• Operate on vector registers
• One instruction processes an entire vector of
data
• Used in high-performance computing systems

Instruction: C = A + B
1 A1 + B1 → C1
2 A2 + B2 → C2
3 A3 + B3 → C3
4 A4 + B4 → C4
(all executed in parallel)
 MISD (Multiple Instruction, Single Data)-1
• Multiple different instructions
• Are executed simultaneously
• On the same single data stream

Key Characteristics:
• Very rare in practice
• Mostly theoretical
• Used to increase: Reliability and Fault tolerance
• Not designed for speedup like SIMD or MIMD

Why MISD Is Rare?

• High Cost: Developing multiple versions requires significant resources (time, money, personnel).
• Complexity: Managing and synchronizing multiple versions adds overhead.
• Performance: Running multiple versions in parallel increases computational load.
• No Guarantee: Independent implementations may still share common flaws (e.g., design errors).
 MISD (Multiple Instruction, Single Data)-2
Applications N-version programming is a fault-tolerance Pseudocode
• Aerospace: technique where:
Input: D #same input data
Flight control • N independent versions of a program
systems, • Are developed separately #Run multiple programs
spacecraft • Run in parallel result1 ← AlgorithmA(D)
software. • Using the same input data result2 ← AlgorithmB(D)
• Medical: Critical • And their outputs are compared (voted) to result3 ← AlgorithmC(D)
medical devices detect errors
(e.g., #Majority voting
pacemakers, if result1 == result2 or
infusion pumps). result1 == result3 then
final_result ← result1
• Nuclear: Control elif result2 == result3 then
systems for final_result ← result2
nuclear power else
plants. final_result ← ERROR #no
consensus
• Automotive: end if
Autonomous
vehicle software. Output: final_result
MIMD (Multiple Instruction, Multiple Data)-1
• Multiple processors
• Execute different instructions
• On different data
• Independently and asynchronously

Key Characteristics:
Pseudocode
• Asynchronous
execution # Conceptual parallel execution (MIMD)
• Highly flexible
• Most common process_1(data_A): run_algorithm_A(data_A)
process_2(data_B): run_algorithm_B(data_B)
architecture today process_3(data_C): run_algorithm_C(data_C)
• Supports both: process_4(data_D): run_algorithm_D(data_D)
Shared memory and
Distributed memory run_in_parallel(process_1, process_2, process_3, process_4)
MIMD (Multiple Instruction,
Multiple Data)-2
Examples
• Multicore CPUs
• Server clusters
• Cloud computing Distributed systems
systems
• Distributed systems
Quick Compare
Architecture Instruction Data Common?
SISD Single Single Yes
SIMD Single Multiple Yes
MISD Multiple Single Rare
MIMD Multiple Multiple Yes
 Lisa Kristiana

Thank you lisa@[Link]

Dc: lisa_krist_35002