Programming Lab Assignment
Topic: MPI Point-to-Point & Non-Blocking Communications
Assignment Overview
You will implement, test, analyze, and benchmark multiple MPI communication patterns using
blocking, non-blocking, synchronous, buffered, and ready sends. The assignment requires deep
understanding of MPI behavior, message ordering, deadlocks, and performance implications.
Coding Tasks
Task 1 — Fully Asynchronous Pipeline Processing
Write an MPI program where:
P0 generates N chunks of data.
P1 performs CPU-expensive transformation.
P2 performs summarization.
P3 stores the final result.
Requirements:
1. All communication must be non-blocking (MPI_Isend, MPI_Irecv).
2. Processing stages must overlap.
3. You must use MPI_Testany or MPI_Testsome for progress control.
4. Program must handle pipeline stalling and buffer exhaustion.
5. Use MPI_Status to track chunk provenance.
Task 2 — Buffered vs Ready Send Benchmark
You must write a benchmark comparing:
MPI_Bsend
MPI_Rsend
Using arrays of size from 1 KB up to 50 MB.
Your results must include:
1. Throughput vs message size
2. Latency vs message size
3. CPU overlap ratio
4. Analysis of when ready send is unsafe
Task 3 — Multi-Source Dynamic Aggregation
Create a system where:
5 worker processes send partial data to master
Workers send messages with unknown timing
Master must accept messages using wildcards (MPI_ANY_TAG, MPI_ANY_SOURCE)
Master must dynamically detect completion using MPI_Probe followed by MPI_Recv
Results must be stored based on sender rank and tag
Task 4 — Race-Free Distributed Summation with Optional Out-of-Order Responses
Recreate a distributed summation similar to the exercise in the PDF, but with the following hard
constraints:
1. Multiple workers compute partial sums.
2. Workers return results in unpredictable order.
3. Master must store results in exact worker order (0→N−1), even if messages arrive out of
order.
4. Only non-blocking receives allowed (MPI_Irecv).
5. You must use:
o MPI_Waitsome or
o MPI_Test loops