![Page 1: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/1.jpg)
Message Passing Interface
Using resources fromhttp://www3.cs.stonybrook.edu/~rezaul/Spring-2012/CSE613/CSE613-lecture-14.pdf
![Page 2: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/2.jpg)
IntroductionOne of the oldest and most widely used approaches for programming parallel computersTwo key attributes Assumes a partitioned address space Supports only explicit parallelismTwo immediate implications of partitioned address space Data must be explicitly partitioned and placed to appropriate partitions Each interaction (read-only and read/write) requires cooperation between two processes:
process that has the data, and the one that wants to access the data
CoreMemory
CoreCore Core
CoreMemory
CoreCore Core
CoreMemory
CoreCore Core
CoreMemory
CoreCore Core
Network
A Cluster of Multicores
![Page 3: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/3.jpg)
The Minimal Set of MPI Routines The MPI library contains over 125 routines But fully functional message-passing programs can be written using only the
following 6 MPI routines
All 6 functions return MPI_SUCCESS upon successful completion, otherwise return an implementation-defined error code
All MPI routines, data-types and constants are prefixed by MPI_ All of them are defined in mpi.h ( for C/C++ )
![Page 4: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/4.jpg)
Starting and Terminating the MPI Library
Both MPI_Init and MPI_Finalize must be called by all processes Command line should be processed only after MPI_Init No MPI function may be called after MPI_Finalize
![Page 5: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/5.jpg)
Communicators A communicator defines the scope of a communication operation Each process included in the communicator has a rank associated with the
communicator By default, all processes are included in a communicator called MPI_COMM_WORLD, and each process is given a unique rank between 0 and p–
1, where p is the number of processes Additional communicator can be created for groups of processes To get the size of a communicator: Int MPI_Comm_size(MPI_Commcomm,
int*size) To get the rank of a process associated with a communicator: Int MPI_Comm_rank(MPI_Commcomm,int*rank)
![Page 6: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/6.jpg)
MPI hello world Compile the program with
mpicc –O3 –o hello_mpi hello_mpi.cc Run with
mpirun –np 2 ./hello_mpinp means number of processes
![Page 7: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/7.jpg)
MPI Standard Blocking Send Format
![Page 8: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/8.jpg)
MPI Standard Blocking Receive Format
![Page 9: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/9.jpg)
MPI Data types
![Page 10: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/10.jpg)
Blocking send and receive Compile the program with
mpicc –O3 –o hello_mpi hello_mpi.cc Run with
mpirun –np 2 ./hello_mpinp means number of processes
![Page 11: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/11.jpg)
Non-blocking send and receiveInt MPI_Isend(void* buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm, MPI_Request* req)
int MPI_Irecv( void* buf, int count, MPI_Datatype datatype, int src,int tag, MPI_Comm comm, MPI_Request* req)
The MPI_Request object is used as an argument to the following two functions to identify the operation whose status we want to query or to wait for its completion.
Int MPI_Test (MPI_Request*req,int*flag,MPI_Status*status)intMPI_Wait(MPI_Request*req,MPI_Status*status)
Returns *flag = 1, if the operation associated with *req has completed, otherwise returns *flag = 0 Waits until the operation associated with *req completes
![Page 12: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/12.jpg)
Non-blocking send and receive
![Page 13: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/13.jpg)
Non-blocking send and receive
![Page 14: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/14.jpg)
MPI Collective Communication & ComputationOperations
Synchronization― Barrier
Data Movement― Broadcast― Scatter― Gather― All-to-all
Global Computation― Reduce― Scan
* These functions must be called by all MPI processes
![Page 15: Message Passing Interface Using resources from](https://reader035.vdocument.in/reader035/viewer/2022062911/5a4d1bee7f8b9ab0599e5482/html5/thumbnails/15.jpg)
Broadcast Example#include <mpi.h> #include <stdio.h> int main(int argc, char** argv) { int rank;
int buf; const int root=0; MPI_Init(&argc, &argv);MPI_Comm_rank(MPI_COMM_WORLD, &rank); if(rank == root) { buf = 777; } printf("[%d]: Before Bcast, buf is %d\n", rank, buf); /* everyone calls bcast, data is taken from root and ends up in everyone's buf */ MPI_Bcast(&buf, 1, MPI_INT, root, MPI_COMM_WORLD); printf("[%d]: After Bcast, buf is %d\n", rank, buf); MPI_Finalize(); return 0;
}