Explain the concept of dealing with I/O
In parallel and distributed computing, Input/Output (I/O) refers to how data is read
from and written to files or devices by multiple processes.
Efficient handling of I/O is important to avoid conflicts and performance bottlenecks.
There are two main approaches:
a) Serial (Centralized) I/O
• Only one process (usually process 0) performs all I/O operations.
• Other processes send/receive data to/from this master process using MPI
communication.
Example:
if (rank == 0) {
read_input_data();
MPI_Bcast(&data, count, MPI_INT, 0, MPI_COMM_WORLD);
Here, process 0 reads the data once and broadcasts it to all other processes.
b) Parallel (Distributed) I/O
• Each process performs its own I/O operations on separate parts of the data.
• Supported by MPI-IO, an advanced part of MPI for high-performance I/O.
Example (MPI-IO):
MPI_File_open(MPI_COMM_WORLD, "[Link]", MPI_MODE_RDONLY, MPI_INFO_NULL,
&fh);
MPI_File_read_at(fh, offset, buffer, count, MPI_INT, &status);
MPI_File_close(&fh);
Each process reads a different portion of the file simultaneously.
Write a note on MPI derived datatypes?
An MPI derived datatype allows you to describe a collection of data items in memory
by specifying their types and their relative locations (displacements).
The idea is that instead of sending individual pieces of data one by one, you define a
single "blueprint" for a complex data structure.
When you use this derived datatype in a communication function (like MPI_Send or
MPI_Bcast):
• Sender: The MPI implementation uses the blueprint to gather the specified data
items from their scattered locations in memory into a contiguous buffer (packing)
before transmitting them as a single message.
• Receiver: The MPI implementation uses the same blueprint to distribute the
received data items into their correct, possibly scattered, destinations in the
receiver's memory (unpacking).
Example: In the trapezoidal rule program, we had to call MPI_Bcast three times to send
a (double), b (double), and n (int). With a derived datatype, we can define a single type
representing "two doubles and one int" and use just one MPI_Bcast call.
Write a pseudo code for parallel program?
Pseudocode for Parallel Program:
Get a, b, n;
h = (b - a) / n;
local_n = n / comm_sz; // Number of trapezoids for this process
local_a = a + my_rank * local_n * h; // Starting point for this process
local_b = local_a + local_n * h; // Ending point for this process
local_integral = Trap(local_a, local_b, local_n, h); // Calculate local integral
if (my_rank != 0)
Send local_integral to process 0;
else /* my_rank == 0 */
total_integral = local_integral; // Process 0 starts with its own integral
for (proc = 1; proc < comm_sz; proc++) {
Receive local_integral from proc;
total_integral += local_integral; // Accumulate integrals from other processes
if (my_rank == 0)
Explain the working of 8 processes in tree structure?
Tree-Structured Communication (Manual Approach)
To improve this, we can design more efficient communication patterns. A common
approach is a tree- structured global sum (like a binary reduction tree
How it works (example with 8 processes):
Phase 1: Processes 1, 3, 5, 7 send their values to processes 0, 2, 4, 6, respectively. The
receivers add the values to their own. (4 sends, 4 adds, done concurrently).
Phase 2: Processes 2 and 6 send their new sums to processes 0 and 4, respectively. The
receivers add the values. (2 sends, 2 adds, done concurrently).
Phase 3: Process 4 sends its sum to process 0. Process 0 adds the value. (1 send, 1
add).
Benefits: This dramatically reduces the work on process 0. In an 8-process example,
process 0 performs only 3 receives and 3 additions (compared to 7 in the original
scheme). More importantly, many of these operations occur concurrently across
different processes.
Scalability: For comm_sz processes, the original scheme requires comm_sz - 1
receives/adds on process 0. A tree-structured sum requires only log_2(textcomm_sz)
receives/adds on process 0. This is a significant improvement, especially for large
comm_sz (e.g., reducing 1023 operations to 10 for 1024 processes).
Write a note on MPI_reduce function?
MPI_Reduce Function
The MPI_Reduce function in MPI is a collective communication operation used to
combine values from all processes in a communicator using a specified operation
(like sum, max, min, etc.) and return the result to a single process (called the root
process).
Syntax :
int MPI_Reduce(
const void *sendbuf, // Pointer to data to send
void *recvbuf, // Pointer to receive buffer (valid at root)
int count, // Number of elements
MPI_Datatype datatype, // Data type of elements
MPI_Op op, // Reduction operation (MPI_SUM, MPI_MAX, etc.)
int root, // Rank of root process
MPI_Comm comm // Communicator
);
Parameters
• sendbuf: Data from the calling process to be reduced.
• recvbuf: Location where the result is stored at the root process.
• count: Number of elements to be reduced.
• datatype: Type of data (e.g., MPI_INT, MPI_FLOAT).
• op: Reduction operation, e.g.,
o MPI_SUM – sum of values
o MPI_MAX – maximum value
o MPI_MIN – minimum value
o MPI_PROD – product of values
• root: Rank of the process where the final result is stored.
• comm: Communicator (usually MPI_COMM_WORLD)
Example
int rank, size, value, result;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
value = rank + 1; // Each process has a value
// Sum values from all processes and store result at root (rank 0)
MPI_Reduce(&value, &result, 1, MPI_INT, MPI_SUM, 0, MPI_COMM_WORLD);
if (rank == 0) {
printf("Sum of values = %d\n", result);