Parallel Programming module 4
SHARED-MEMORY PROGRAMMING WITH
OPENMP
In modern computing, the demand for high-performance applications has increased
significantly, requiring efficient utilization of multicore processors. Shared-memory
programming is one approach to achieve parallelism by allowing multiple threads to
access a common memory space. In this model, all processors share a single address
space, and data can be communicated between threads directly through shared variables.
OpenMP (Open Multi-Processing) is a widely used API for shared-memory parallel
programming in C, C++, and Fortran. It simplifies the development of parallel
applications by providing compiler directives, runtime library routines, and environment
variables to manage thread-based parallelism. OpenMP uses a fork-join model: the master
thread forks a team of threads to execute parallel regions and joins them back after
completion.
Key Concepts and Formulas
Speedup and Efficiency
The effectiveness of parallel programs can be measured using speedup and efficiency:
Speedup (S):
S=TsTpS = \frac{T_s}{T_p}S=TpTs
Where:
TsT_sTs = execution time of the sequential program
TpT_pTp = execution time of the parallel program
Efficiency (E):
E=SN×100%E = \frac{S}{N} \times 100\%E=NS×100%
Where:
NNN = number of threads or cores
Amdahl’s Law
Amdahl’s Law predicts the theoretical maximum speedup of a parallel program:
Smax=1(1−P)+PNS_{\text{max}} = \frac{1}{(1 - P) + \frac{P}{N}}Smax=(1−P)+NP1
Where:
PPP = fraction of the program that can be parallelized
NNN = number of processors
Observation: Even if NNN is very large, speedup is limited by the sequential portion
(1−P)(1-P)(1−P) of the program.
Dept Of. CSE (DS), AIET 18
Parallel Programming module 4
OpenMP Programming Basics OpenMP provides compiler directives (pragmas) to
easily parallelize loops and sections of code.
Figure 1 shared memory architecture
Parallel For Loop
#include <stdio.h>
#include <omp.h>
int main() {
int i;
int sum = 0;
#pragma omp parallel for reduction(+:sum)
for(i = 1; i <= 10; i++) {
sum += i;
printf("Thread %d processing i = %d\n", omp_get_thread_num(), i);
}
printf("Sum = %d\n", sum);
return 0;
}
Output
Thread 0 processing i = 1
Thread 1 processing i = 2
Thread 2 processing i = 3
...
Sum = 55
Dept Of. CSE (DS), AIET 19
Parallel Programming module 4
Explanation: The reduction clause ensures correct accumulation of sum across threads.
Each thread handles a part of the loop simultaneously, improving performance.
Real-World Example
Consider a weather simulation application where temperature, pressure, and humidity
must be calculated at multiple grid points. Using OpenMP, each thread can compute
values for a portion of the grid simultaneously. This reduces simulation time dramatically
compared to sequential computation.
Another example is image processing: applying a filter to a large image can be
parallelized by dividing the image into blocks, each processed by a separate thread.
Applications
OpenMP is suitable for many domains:
Scientific Computing: Simulations, fluid dynamics, molecular modeling.
Data Analytics: Parallel processing of large datasets in memory.
Image and Signal Processing: Parallel filtering, transformations, and analysis.
Financial Modeling: Monte Carlo simulations, risk analysis.
Real-Time Systems: Games, multimedia processing where shared memory
reduces latency.
Advantages
Simple and easy to use via compiler directives.
Scales well on multi-core CPUs.
Supports incremental parallelization; existing sequential code can be parallelized
gradually.
Shared-memory model avoids explicit message passing.
Limitations
Limited to a single shared-memory system; does not scale across multiple
machines.
Risk of data races if shared variables are not handled properly.
Memory bandwidth can become a bottleneck as the number of threads increases.
Debugging parallel programs is more challenging.
Real-Time Programming Example
Task: Compute the sum of squares of first 100 numbers using 4 threads.
#include <stdio.h>
#include <omp.h>
int main() {
Dept Of. CSE (DS), AIET 20
Parallel Programming module 4
int i, sum = 0;
#pragma omp parallel for reduction(+:sum) num_threads(4)
for(i = 1; i <= 100; i++) {
sum += i * i;
}
printf("Sum of squares = %d\n", sum);
return 0;
}
Output:
Sum of squares = 338350
Explanation: Each thread computes a portion of the sum independently, and the
reduction clause combines results safely. This reduces computation time by utilizing all
cores efficiently.
CONCLUSION
OpenMP provides a simple, practical, and efficient way to exploit shared-memory
parallelism. It allows developers to enhance performance of CPU-bound applications
without drastically changing existing code. While limited to shared-memory architectures
and prone to data race issues, OpenMP remains a popular choice for scientific computing,
real-time systems, and parallel processing tasks. With careful design, developers can
achieve significant speedup and improved resource utilization.
Dept Of. CSE (DS), AIET 21