Outline

Elements in MPI Paradigm

MPI Basic Routines
MPI Basic Programs

SCSP 3133: High Performance Data

Processing
Module 5: Introduction to MPI Programming

November 25, 2020

SCSP 3133: High Performance Data Processing

 Outline
Elements in MPI Paradigm
MPI Basic Routines
MPI Basic Programs

Outline

1 Elements in MPI Paradigm

2 MPI Basic Routines
3 Basic MPI Program
4 Compiling MPI Program
5 Executing MPI Program
6 Basic MPI Program Practice

SCSP 3133: High Performance Data Processing

 Outline
Elements in MPI Paradigm MPI Process
MPI Basic Routines MPI Communicator
MPI Basic Programs

Elements in MPI Paradigm

Message

Communicator

Rank 0 Rank 1
Process

Rank 2
Rank 3

Rank 4

- program
- local/private data
example of sending message instruction
"send message to process with Rank X"

SCSP 3133: High Performance Data Processing

 Outline
Elements in MPI Paradigm MPI Process
MPI Basic Routines MPI Communicator
MPI Basic Programs

Process
A process consists of:
1 Instruction set (Program)
2 Private variables
The variables of each process are private.
A process is identified by its Rank value.
MPI implements SPMD computational model,
in which, a single program is written, and each
process will executes its personal copy of the
program.

SCSP 3133: High Performance Data Processing

 Outline
Elements in MPI Paradigm MPI Process
MPI Basic Routines MPI Communicator
MPI Basic Programs

Communicator
Communicator is a communication domain of
processes.
A processes are only allowed to communicate
with other processes in the same
communicator.
However, a process may belong to more than
one communicator.
By using a different communicator for each of
process group, no message will ever interfere
with message destined to any other group.
SCSP 3133: High Performance Data Processing
 Outline
Elements in MPI Paradigm MPI Process
MPI Basic Routines MPI Communicator
MPI Basic Programs

MPI COMM WORLD

MPI COMM WORLD is a default

communicator, which is defined to include ALL
processes involved in the parallel execution.

SCSP 3133: High Performance Data Processing

 Outline
Elements in MPI Paradigm
MPI Basic Routines
MPI Basic Programs

MPI Basic Routines

To initiate the parallel computation.
int MPI Init( int *argc, char ***argv )
To determine the number of process in a
communicator
int MPI Comm size( MPI Comm comm,
int *size)
To determine the rank of a process
int MPI Comm rank( MPI Comm comm,
int *rank)
To shutdown the parallel computation
int MPI Finalize( )
SCSP 3133: High Performance Data Processing
 Hello World 1
Outline
Compiling and Executing MPI Program
Elements in MPI Paradigm
Executing MPI Program on single machine
MPI Basic Routines
Hello World 2
MPI Basic Programs
etc: Executing MPI Program on multiple machines

Source code

1 #i n c l u d e < s t d i o . h>
2 #i n c l u d e <mpi . h>
3 main ( i n t a r g c , c h a r ∗ a r g v [ ] ) {
4 i n t m p i e r r o r , npes , myrank ; /∗ g l o b a l v a r i a b l e s ∗/
5 m p i e r r o r = M P I I n i t (& a r g c , &a r g v ) ;
6 m p i e r r o r = MPI Comm size (MPI COMM WORLD, &n p e s ) ;
7 m p i e r r o r = MPI Comm rank (MPI COMM WORLD, &myrank ) ;
8
9 p r i n t f ( ” From p r o c e s s %d o ut o f %d , H e l l o
World ! ! \ n” , myrank , n p e s ) ; /∗ g l o b a l i n s t r u c t i o n ∗/
10
11 mpierror = MPI Finalize () ;
12 }

SCSP 3133: High Performance Data Processing

 Hello World 1
Outline
Compiling and Executing MPI Program
Elements in MPI Paradigm
Executing MPI Program on single machine
MPI Basic Routines
Hello World 2
MPI Basic Programs
etc: Executing MPI Program on multiple machines

mpicc

mpicc - to build the MPI program

Syntax:
mpicc -o [executable file] [source code file]
Example:
mpicc -o helloworld helloworld.c

SCSP 3133: High Performance Data Processing

 Hello World 1
Outline
Compiling and Executing MPI Program
Elements in MPI Paradigm
Executing MPI Program on single machine
MPI Basic Routines
Hello World 2
MPI Basic Programs
etc: Executing MPI Program on multiple machines

mpirun

Syntax:
mpirun -np [number of process] [executable file]
Example:
mpirun -np 4 helloworld

SCSP 3133: High Performance Data Processing

 Hello World 1
Outline
Compiling and Executing MPI Program
Elements in MPI Paradigm
Executing MPI Program on single machine
MPI Basic Routines
Hello World 2
MPI Basic Programs
etc: Executing MPI Program on multiple machines

Message Passing Model Diagram

myrank,
npes,
mpierror MPI_COMM_WORLD
myrank,
npes, Rank 0 Rank 1
mpierror

myrank, myrank,
Rank 3 Rank 2 npes,
npes,
mpierror mpierror

Expected output:
From process 0 out of 4, Hello World!!
From process 1 out of 4, Hello World!!
From process 2 out of 4, Hello World!!
From process 3 out of 4, Hello World!!
Welcome!!, a greeting from process 2.

SCSP 3133: High Performance Data Processing

 Hello World 1
Outline
Compiling and Executing MPI Program
Elements in MPI Paradigm
Executing MPI Program on single machine
MPI Basic Routines
Hello World 2
MPI Basic Programs
etc: Executing MPI Program on multiple machines

MPI Basic Exercise 1

Compile and build the program.

Execute the program using mpirun command
with:
1 process
2 processes
4 processes
6 processes

SCSP 3133: High Performance Data Processing

 Hello World 1
Outline
Compiling and Executing MPI Program
Elements in MPI Paradigm
Executing MPI Program on single machine
MPI Basic Routines
Hello World 2
MPI Basic Programs
etc: Executing MPI Program on multiple machines

Assigning specific operation to a process

Programmer can specify specific instructions to

a process by manipulating the rank of
processes.

SCSP 3133: High Performance Data Processing

 Hello World 1
Outline
Compiling and Executing MPI Program
Elements in MPI Paradigm
Executing MPI Program on single machine
MPI Basic Routines
Hello World 2
MPI Basic Programs
etc: Executing MPI Program on multiple machines

Source code
1 #i n c l u d e < s t d i o . h>
2 #i n c l u d e <mpi . h>
3 main ( i n t a r g c , c h a r ∗ a r g v [ ] ) {
4 i n t m p i e r r o r , npes , myrank ; /∗ g l o b a l v a r i a b l e ∗/
5 m p i e r r o r = M P I I n i t (& a r g c , &a r g v ) ;
6 m p i e r r o r = MPI Comm size (MPI COMM WORLD, &n p e s ) ;
7 m p i e r r o r = MPI Comm rank (MPI COMM WORLD, &myrank ) ;
8
9 p r i n t f ( ” From p r o c e s s %d o ut o f %d , H e l l o
World ! ! \ n” , myrank , n p e s ) ; /∗ g l o b a l i n s t r u c t i o n ∗/
10
11 i f ( myrank == 2 ) {
12 int x , y ; /∗ l o c a l v a r i a b l e ∗/
13 p r i n t f ( ” Welcome ! ! , a g r e e t i n g from p r o c e s s
%d . \ n” , myrank ) ; /∗ l o c a l i n s t r u c t i o n ∗/
14 }
15
16 mpierror = MPI Finalize () ;
17 }
SCSP 3133: High Performance Data Processing
 Hello World 1
Outline
Compiling and Executing MPI Program
Elements in MPI Paradigm
Executing MPI Program on single machine
MPI Basic Routines
Hello World 2
MPI Basic Programs
etc: Executing MPI Program on multiple machines

Message Passing Model Diagram

myrank,
npes,
mpierror MPI_COMM_WORLD
myrank,
npes, Rank 0 Rank 1
mpierror

myrank, myrank,
Rank 3 Rank 2 npes,
npes,
mpierror mpierror
x, y

Expected output:
From process 0 out of 4, Hello World!!
From process 1 out of 4, Hello World!!
From process 2 out of 4, Hello World!!
From process 3 out of 4, Hello World!!
Welcome!!, a greeting from process 2.

SCSP 3133: High Performance Data Processing

 Hello World 1
Outline
Compiling and Executing MPI Program
Elements in MPI Paradigm
Executing MPI Program on single machine
MPI Basic Routines
Hello World 2
MPI Basic Programs
etc: Executing MPI Program on multiple machines

mpirun on cluster computer

Syntax:
mpirun -np [number of process] -hostfile [hostfile]
[executable file]
Example:
mpirun -np 4 -hostfile hostfile helloworld

SCSP 3133: High Performance Data Processing

 Hello World 1
Outline
Compiling and Executing MPI Program
Elements in MPI Paradigm
Executing MPI Program on single machine
MPI Basic Routines
Hello World 2
MPI Basic Programs
etc: Executing MPI Program on multiple machines

End

SCSP 3133: High Performance Data Processing