CSCI 551
Numerical and Parallel
Programming
OpenMP Patterns Tutorial
May 14, 2024 Sam Siewert
OpenMP Patterns
Basic OpenMP Examples
– csci551/code/hello_openmp/
– csci551/code/ipp2-source/ch5/
Advanced OpenMP Examples
– csci551/code/functiongen/timeprofiles_omp.c
– csci551/code/openmp-sharpen/
– csci551/code/openmp_dct3/
– csci551/code/PP-Quinn-Source/openmp/
Well-Structured “{…}” Block with defaults, locals, & reduction
– E.g., csci551/code/hello_openmp/piseries.c
Well-Structured Block with no defaults
– E.g., csci551/code/hello_openmp/omppi.c
Function (well-structured) Block with explicit thread map
– E.g., csci551/code/hello_openmp/piseriesompfunct.c
Sam Siewert 2
Quick Build Notes
Always build with –fopenmp
– Sometimes OpenMP is ignored (silent failure)
– Not all compilers have OpenMP support or latest
– Some compilers give warnings and errors that others do not
Sam Siewert 3
Well-Structed Block with Defaults
csci551/code/hello_openmp/piseries.c
#pragma omp parallel // all defaults
#pragma omp parallel for // default for
#pragma omp parallel for threads(thread_count) // scalable
#pragma omp parallel for threads(thread_count) reduction(+:sum)
Notes:
• idx is locally declared in block
• iterations is read only
• num is locally declared
• sum is global but reduced
Sam Siewert 4
Why use reduction on sum?
Most often OK, but not always on every machine, for
every compiler, in all scenarios
Issue is “+=“ used with global sum
Reduction indicates that summing should be done when
all threads are done
Alternative is “omp critical” or “omp atomic”
Otherwise, default for pragma is OK
Sam Siewert 5
Use of #pragma omp critical
OpenMP omp critical solves race condition corruption
– BUT … Slows down the parallel block considerably
– Often can be avoided using reduction, atomic, or simply re-
designing to use a better pattern
≈ 3x faster!
Time = 0.023 seconds, accurate Time = 0.067 seconds, accurate
Sam Siewert 6
Use of #pragma omp atomic
OpenMP omp atomic solves race condition for one value
– BUT … Slows down the parallel block (smaller scope)
– Often can be lower impact than omp critical
≈ same here
Time = 0.067 seconds, accurate Time = 0.058 seconds, accurate
Sam Siewert 7
Best Pattern for Loop - Reduction
Reduction often used in loops (series, integration, sums,
averages, worst, best, etc.)
– Allows compiler to optimize OpenMP
– Takes advantage of order independence of operators that can
be reduced (addition can be done in any order)
– csci551/code/hello_openmp/incdec.c
Increment
• Local
• Protected global
Decrement
• Local
Reduction for a global sum in any block • Protected global
ZERO for common COUNT
Sam Siewert 8
Avoiding Critical or Atomic
Example - csci551/code/hello_openmp/omppi.c
No defaults
Reduce necessary global sum
All indexes must be private to each thread
“n” is Read-Only and can be shared globally
Avoids Critical or Atomic use
Thread safety Summary
1. Use MUTEX Critical Section or Atomic Update
2. Shared Only Read Only Data
3. Use Thread Indexed Global Data Unique to Each Thread
4. Use Stack Only and Function Entry Point for Each Thread
Sam Siewert 9
Functional OpenMP
Write OpenMP like Pthreads but Simpler!!
Threaded function call
Thread specific divide & conquer
• Total # of threads
• Which thread is this out of #?
• Divide up work based on thread rank
Loop Range based on Rank
• Thread does subrange of series
• Returns subrange summation
Sam Siewert 10