Computer Organization
And Architecture
UNIT 5
PARALLEL PROCESSING AND MULTICORE
COMPUTERS
ASSIGNMENT
M BALA VISWANATHAN
1117242107023
B.E CSD A
3RD SEM
1. What is parallel processing?
The simultaneous use of multiple compute resources
(processors/cores) to solve a computational problem.
2. State two advantages of parallel processing.
1. Reduced execution time for large problems (higher speed).
2. Ability to solve larger, more complex problems (higher
throughput).
3. Define throughput and speedup.
Throughput: The number of tasks completed per unit of time.
Speedup: The ratio of the time taken to solve a problem on a
single processor to the time taken on a parallel system.
4. What is the main objective of parallel processor design?
To maximize performance by dividing a task among multiple
processors to be solved concurrently.
5. Write any two applications of parallel computers.
Scientific simulations (e.g., weather forecasting) and Big Data
analytics.
6. Differentiate between uniprocessor and multiprocessor systems.
A uniprocessor has a single CPU that executes one instruction
stream. A multiprocessor has multiple CPUs that can execute
multiple instruction streams concurrently.
7. What is granularity in parallel computing?
The size or amount of computation done between communication or
synchronization events. (Fine-grained: small tasks, frequent
communication; Coarse-grained: large tasks, infrequent
communication).
8. Define Amdahl’s Law and its significance.
It states that the maximum speedup of a program is limited by the
fraction of the program that must be executed sequentially. It
highlights the diminishing returns of adding more parallel processors.
9. Mention two limitations of sequential processing.
1. Performance is limited by processor clock speed and
instruction-level parallelism.
2. Inability to solve problems whose size or real-time
requirements exceed the capacity of a single processor.
10. What is concurrency in computing?
The ability of a system to deal with multiple tasks in progress at the
same time. It does not necessarily imply they are executing
simultaneously.
Flynn's Taxonomy & Architectures
11. Who proposed Flynn’s taxonomy?
Michael J. Flynn.
12. List the four categories of Flynn’s classification.
SISD, SIMD, MISD, MIMD.
13. Define SISD with an example.
Single Instruction, Single Data stream. A classical von Neumann
architecture with a single processor. Example: A simple, single-core
personal computer.
14. Define SIMD and give an example of SIMD system.
Single Instruction, Multiple Data streams. All processing units execute
the same instruction on different data elements simultaneously.
Example: A GPU (Graphics Processing Unit).
15. What is MISD architecture?
Multiple Instruction, Single Data stream. Multiple processors operate
on the same data stream with different instructions. Rarely used;
theoretical.
16. Define MIMD and mention one example.
Multiple Instruction, Multiple Data streams. Multiple processors
execute different instruction sequences on different data sets
independently. Example: A modern multicore CPU or a computer
cluster.
17. Differentiate between SIMD and MIMD.
SIMD has a single control unit broadcasting the same instruction to
all processors. MIMD has multiple independent control units, each
fetching its own instructions.
18. Mention two advantages of SIMD processors.
1. High efficiency for data-parallel, regular computations (e.g.,
vector/matrix operations).
2. Simpler hardware and control logic compared to MIMD.
19. Define vector processor.
A processor that can perform operations on entire arrays (vectors) of
data in a single instruction, a type of SIMD architecture.
20. What are array processors?
A parallel computer with multiple ALUs (Arithmetic Logic Units)
synchronized to perform the same operation on multiple data points
simultaneously, a type of SIMD architecture.
Symmetric Multiprocessors (SMP)
21. Define SMP.
Symmetric Multiprocessor; a multiprocessor system where two or
more identical processors are connected to a single, shared main
memory and are controlled by a single OS instance.
22. Mention any two characteristics of SMP systems.
1. Uniform memory access (UMA) time for all processors.
2. Processors share I/O devices and are managed by one operating
system.
23. What is bus organization in SMP?
An SMP architecture where processors, memory, and I/O are
connected via a common system bus.
24. State two advantages of bus organization.
1. Simplicity and low cost.
2. Ease of implementation and expansion.
25. Mention two drawbacks of bus-based multiprocessors.
1. The system bus becomes a performance bottleneck as the
number of processors increases.
2. Limited scalability due to bus bandwidth constraints.
26. What is cache coherence?
The problem that arises when multiple processors have caches and
may hold different values for the same memory location. A protocol
is needed to maintain a consistent view of data.
27. Define availability in SMP context.
If one processor fails, the system can continue to operate (with
reduced performance), increasing system reliability.
28. What is incremental growth in SMP systems?
The ability to improve system performance by adding more
processors to an existing system.
29. Write any two functions of the operating system in an SMP
environment.
1. Schedule threads or processes across available processors.
2. Manage shared resources and enforce synchronization.
30. Mention any two issues faced in multiprocessor operating
systems.
1. Scheduling and load balancing across processors.
2. Synchronization and deadlock prevention in a concurrent
environment.
NUMA and Clusters
31. Expand NUMA and define it.
Non-Uniform Memory Access. A multiprocessing architecture where
each processor has its own local memory, but can also access the
memory of other processors. Access time depends on the memory
location.
32. How does NUMA differ from SMP?
In SMP (UMA), memory access time is uniform. In NUMA, access to
remote memory is slower than access to local memory.
33. What is meant by non-uniform memory access time?
The time for a processor to access memory is not constant; it is faster
to access its local memory and slower to access memory attached to
another processor.
34. Define Cluster Computing.
A type of parallel system consisting of a collection of interconnected,
independent computers (nodes) that work together as a single,
integrated computing resource.
35. Mention two advantages of cluster systems.
1. High scalability and cost-effectiveness using commodity
hardware.
2. High availability and fault tolerance.
36. What is the communication method used in clusters?
High-speed networks (like InfiniBand, Myrinet) and message-passing
libraries (like MPI).
37. What are interconnection networks?
The hardware and software infrastructure that connects processors
to memory or processors to each other in a parallel system.
38. Define message passing.
A parallel programming paradigm where processes exchange data by
explicitly sending and receiving messages.
39. Give any two design challenges of cluster computing.
1. Achieving low-latency, high-bandwidth communication
between nodes.
2. Managing the cluster as a single system and providing a single
system image.
40. What is fault tolerance in clusters?
The ability of the cluster to continue operating correctly even if one
or more of its nodes fail.
Multithreading and Multicore
41. Define multithreading.
A technique that allows a single CPU core to execute multiple threads
of execution concurrently, by rapidly switching between them.
42. Distinguish between process and thread.
A process is an instance of a running program with its own full
address space. A thread is a lightweight unit of execution within a
process that shares the process's address space.
43. What is thread-level parallelism?
Parallelism achieved by executing different threads concurrently on
multiple processors or cores.
44. Define thread context.
The minimal state information (program counter, register values,
stack pointer) that must be saved and restored to switch between
threads.
45. What is thread switching?
The act of saving the context of one thread and loading the context
of another so it can execute on the processor.
46. Differentiate between fine-grained and coarse-grained
multithreading.
Fine-grained: The processor switches threads after every
instruction cycle.
Coarse-grained: The processor switches threads only on a long-
latency event, like a cache miss.
47. What is simultaneous multithreading (SMT)?
A technique that allows multiple independent threads to issue
instructions to a superscalar processor's functional units in a single
clock cycle. (e.g., Intel's Hyper-Threading).
48. Define chip multiprocessor (CMP).
A processor that contains multiple independent execution cores on a
single chip. Also known as a multicore processor.
49. What is multicore architecture?
An architecture that integrates two or more complete computational
cores (CPUs) onto a single integrated circuit die.
50. Write two advantages of multicore processors.
1. Higher performance within a similar power and thermal
envelope as a single-core chip.
2. Improved ability to run multiple applications simultaneously.
51. Mention any two challenges of multicore systems.
1. Difficulty of writing correct and efficient parallel software.
2. The "memory wall" and contention for shared resources (e.g.,
last-level cache, memory bandwidth).
Advanced Concepts and Performance
52. What is instruction-level parallelism (ILP)?
The ability of a processor to execute multiple instructions from a
single thread simultaneously in a single clock cycle.
53. How does multithreading improve CPU utilization?
By executing instructions from another thread when one thread is
stalled (e.g., waiting for memory), it hides latency and keeps the
processor busy.
54. Write two differences between superscalar and multithreaded
processors.
1. Superscalar exploits ILP from a single
thread; Multithreaded exploits thread-level parallelism (TLP)
from multiple threads.
2. Superscalar has high hardware complexity for dynamic
scheduling; Multithreaded requires duplicating thread contexts
(registers).
55. Define hyper-threading technology.
Intel's proprietary implementation of Simultaneous Multithreading
(SMT), which makes a single physical core appear as two logical cores
to the operating system.
56. What is the difference between hardware multithreading and
software multithreading?
Hardware Multithreading: Supported directly by the CPU
hardware (duplicate register sets, etc.).
Software Multithreading: Managed by the operating system
and a runtime library, where the hardware sees only a single
thread context.
57. Mention any two types of explicit multithreading.
Blocked (Coarse-grained) multithreading and Interleaved (Fine-
grained) multithreading.
58. What is blocked multithreading?
The processor executes one thread until it encounters a long-latency
event, then it switches to another thread.
59. What is interleaved multithreading?
The processor switches to a different thread at each instruction cycle,
regardless of stalls.
60. Define super-scalar architecture.
A processor architecture that can issue multiple instructions from a
single instruction stream in a single clock cycle by having multiple
parallel functional units.
61. Define speedup.
Speedup = (Execution Time on a Single Processor) / (Execution Time
on a Parallel System with N processors).
62. Define efficiency of a parallel system.
Efficiency = (Speedup) / (Number of Processors). It measures how
well the processors are utilized.
63. Write the formula for speedup.
S(p) = T(1) / T(p), where T(1) is time on one processor and T(p) is time
on p processors.
64. What is scalability in parallel systems?
The ability of a parallel system to demonstrate a proportionate
increase in performance as the number of processors is increased.
65. Define latency.
The time delay between the initiation of an operation and its
completion (e.g., time to fetch a word from memory).
66. What is bandwidth?
The rate at which data can be transferred, measured in data per unit
time (e.g., GB/s).
67. Mention two factors affecting parallel performance.
1. Communication and synchronization overhead between
processors.
2. Load imbalance among processors.
68. Define load balancing.
The practice of distributing work evenly across all processors to
ensure no processor is idle while others are still working.
69. What is parallel overhead?
The extra work in a parallel system not present in the sequential
version, including communication, synchronization, and thread
management costs.
70. Write two advantages of using multithreading for performance
improvement.
1. Hides latency by keeping the processor busy with other threads.
2. Enables better utilization of multiple functional units within a
core (as in SMT).