CSCI 551
Numerical and Parallel
Programming
Pthread, OpenMP, MPI Tool Tutorial
October 24, 2023 Sam Siewert
Tools for ECC-Linux - Coding
Editor Options
1. “vi” or “vim” – copy my .vimrc or edit for
colors and line numbers (industry, on)
Learn vi using cheat sheet - here
Key concept: command vs. edit mode (esc)
2. “nano” – simple editor to get you started
3. VS Code remote – learn, get, get extension
Debugger Options
1. GDB – simple command line, works with
threads and MPI with some extensions
2. Compile “-O0” and “-g” – no optimization,
debug symbols on
3. DDD debugger -
[Link]
4. VS-code debugger – C++ extension, WSL
Get your favorite distro from MS Store
Sam Siewert 2
VS-Code
Compiling C/C++ with WSL on
Windows
– Install WSL from MS Store (enable first in
Control Panel, Apps, Features)
Using remote extension to build on
ECC-Linux
Using VS-Code on ECC-Linux can be
done, but can be tricky
Sam Siewert 3
ECC-Linux Building Code
If you are new to Linux command line development
– csci551/documents/Linux/[Link]
f
– csci551/documents/Linux/[Link]
– csci551/documents/Linux/[Link]
f
– csci551/documents/Linux/GDB%20Cheat%[Link]
– csci551/documents/Linux/[Link]
Command line – good practice for industry HPC
Makefiles (best option)
VS Code (alternative and fine if available)
Sam Siewert 4
Intel PS XE Tools
/opt/intel/compilers_and_libraries_2020.0.166/linux/mpi/
intel64/bin
– C compiler – mpicc
– C++ compiler – mpicxx
– CPU info - cpuinfo
Linux CPU info
1. lscpu
2. cat /proc/cpuinfo
3. htop
Generic info compared to
Intel tools
Sam Siewert 5
Pthread GDB
Usual GDB methods, but be aware of thread commands
Threads are shared memory
This makes debugging
Pthread code very similar to
single thread code
GNU GDB – Threads
Issues are which thread is
executing code at breakpoint
Each thread has it’s own
stack (and registers)
Threads may have thread
indexed memory or shared
Sam Siewert 6
Notes on MPI Scaling and Debug
Cluster parallel use (check for saturation of nodes used)
– Monitor by having htop window up for each node used
Manually bring up htop on each node used in terminal
Script for this -
[Link]
ces/
– [Link] (cluster use at high level)
– U. of Oregon TAU
– You can also just measure speed-up and compute P%
MPI debugging
– TotalView (used at LLNL) - [Link]
– LLNL tutorials on TotalView – Part-1, Part-2, Part-3
– Simple method with GDB – Fogal on Parallel MPI Debug
– Use MPI _Barrier and printf of syslog
Scheduling beyond mpiexec (mpirun) – Flux & Slurm
Sam Siewert 7
Debugging OpenMP
Use GDB, but learn thread commands (like Pthreads)
Tutorials with more Examples -
[Link]
TotalView - [Link]
DDT -
[Link]
hpc/forge/ddt
Sam Siewert 8
Parallel GDB
GDB with MPI programs
[Link]
– [Link]
– [Link]
ggers
Fogal on Debugging MPI with GDB
– [Link]
apers/[Link]
– Google Scholar Link
Sam Siewert 9
Advanced Tools for Home Systems
Intel Vtune – profiler download, License options
– On ECC-Linux (component of PS XE – tools)
Sysprof - [Link]
Ftrace – built into Linux – overview, Rostedt tutorial,
[Link]
Perf – Linux Wiki
CUDA – nvtop, jtop, nsight -
[Link]
Sam Siewert 10
CUDA: GP-GPU Use – “jtop”
[Link]
Clone and install jtop as documented
1. sudo apt-get install python-pip
CUDA Samples – Video
2. sudo -H pip install -U jetson-stats make –j 4
3. Restart Jetson
Similar tool - nvtop
Sam Siewert 11
CUDA Examples with jtop
Transpose with –dimX=4096 dimY=4096
Matrix transpose test
Must be multiple of 128
Sam Siewert 12
GPU Loading Over Time
Transpose with –dimX=4096 dimY=4096
Harder to make fully parallel
Sam Siewert 13