Parallel & Distributed Systems
Multiprocessor | Multicomputer | Multi-Core
Exam Notes (5-7 Marks)
1. Multiprocessor System
A multiprocessor system contains two or more CPUs that share a common main
memory and are under the control of a single operating system.
Key Characteristics
• Shared Memory: All processors access a single global memory address space.
• Tightly Coupled: Processors are closely connected and communicate through
shared memory.
• Single OS: One operating system manages all processors.
• Bus/Switch Interconnect: Processors communicate via a shared bus or
crossbar switch.
• Uniform Memory Access (UMA): Each processor has equal access time to all
memory locations.
Types of Multiprocessor Systems
• UMA (Uniform Memory Access): All processors share memory uniformly (e.g.,
SMP - Symmetric Multiprocessing).
• NUMA (Non-Uniform Memory Access): Memory access time depends on the
memory location relative to the processor.
• COMA (Cache Only Memory Access): All main memory is converted to caches.
Advantages
• High Performance: Tasks divided among processors improve speed.
• Resource Sharing: Processors share memory and I/O devices, reducing cost.
• Reliability: If one processor fails, others can continue.
• Scalability: More processors can be added (within limits).
Disadvantages
• Memory Contention: Multiple processors competing for same memory can
cause bottlenecks.
• Synchronization Overhead: Shared data access requires careful
locking/synchronization.
• Limited Scalability: Shared bus becomes a bottleneck with many processors.
Applications
• Database servers, scientific computing, real-time simulations, enterprise servers.
2. Multicomputer System
A multicomputer system consists of multiple independent computers (nodes), each with
its own CPU, memory, and OS, connected through a network. They communicate via
message passing.
Key Characteristics
• Distributed Memory: Each node has its own private memory — no shared
memory.
• Loosely Coupled: Nodes are independent computers connected by a network.
• Message Passing: Communication between nodes happens via MPI (Message
Passing Interface) or similar protocols.
• Multiple OS: Each node can run its own operating system.
• High Scalability: Easily scalable by adding more nodes.
Types of Multicomputer Systems
• Cluster Computing: Group of commodity computers connected via LAN (e.g.,
Beowulf cluster).
• Grid Computing: Geographically distributed computers working together (e.g.,
SETI@Home).
• MPP (Massively Parallel Processing): Thousands of processors working in
parallel (e.g., IBM BlueGene).
Advantages
• High Scalability: Can scale to thousands of nodes easily.
• No Memory Bottleneck: Each node has independent memory.
• Fault Tolerance: Failure of one node does not affect others.
• Cost-Effective: Built using commodity hardware.
Disadvantages
• Message Passing Overhead: Communication via network is slower than shared
memory access.
• Complex Programming: Requires explicit message passing, harder to program
than shared memory.
• Network Latency: Performance depends heavily on network speed.
Applications
• Weather forecasting, big data processing (Hadoop clusters), scientific
simulations, cloud computing.
3. Multi-Core System
A multi-core processor integrates two or more independent processing units (cores)
onto a single chip. Each core can execute instructions independently and in parallel.
Key Characteristics
• Multiple Cores on One Chip: Cores share the same die (silicon chip) but
operate independently.
• Shared Cache (L2/L3): Cores typically share L2 or L3 cache for fast data
exchange.
• Single OS: One operating system manages all cores.
• Low Power Consumption: More efficient than having multiple separate
processors.
• On-Chip Communication: Cores communicate via on-chip buses, extremely
fast.
Types of Multi-Core Processors
• Dual-Core: 2 cores (e.g., Intel Core 2 Duo).
• Quad-Core: 4 cores (e.g., Intel Core i5).
• Hexa/Octa-Core: 6 or 8 cores (e.g., Intel Core i7, AMD Ryzen 7).
• Many-Core: Dozens to thousands of cores (e.g., GPU, Intel Xeon Phi).
Advantages
• Parallel Processing: Multiple threads execute simultaneously on different cores.
• Energy Efficient: Less power than multiple separate chips for same
performance.
• Low Latency: On-chip communication is much faster than external bus.
• Cost Effective: Single chip packaging reduces manufacturing cost.
• Improved Throughput: Multiple tasks can run simultaneously without context
switching.
Disadvantages
• Cache Coherence Problem: Maintaining consistent data across all cores'
caches is complex.
• Heat Dissipation: Multiple cores generate more heat on the same chip.
• Software Must Be Parallel: Single-threaded programs don't benefit from
multiple cores.
• Limited Physical Scalability: Cannot add cores indefinitely due to chip size
constraints.
Applications
• Personal computers, laptops, smartphones, gaming consoles, embedded
systems, servers.
Comparison Table: Multiprocessor vs. Multicomputer vs. Multi-Core
Feature Multiprocessor Multicomputer Multi-Core
Memory Shared Distributed Shared (on-chip)
Communication Shared Bus / Message Passing On-chip interconnect
Memory
OS Single OS Multiple OS Single OS
Cost Moderate High Low
Scalability Limited Highly Scalable Moderate
Example SMP Servers Cluster / Grid Intel i7, AMD Ryzen
Quick Recall — One-Line Definitions
• Multiprocessor: Multiple CPUs sharing one memory under one OS — tightly
coupled.
• Multicomputer: Multiple independent computers connected via network —
loosely coupled with message passing.
• Multi-Core: Multiple cores on a single chip, sharing cache, under one OS —
most energy efficient.