![Page 1: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/1.jpg)
Experts in numerical software andHigh Performance Computing
Message Passing Interfacea brief introduction
University of Birmingham
30th January 2017
Ning Li - Senior Technical Consultant
Sally Bridgwater – HPC Application Analyst
![Page 2: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/2.jpg)
2High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 3: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/3.jpg)
3High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 4: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/4.jpg)
4High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 5: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/5.jpg)
5High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 6: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/6.jpg)
6High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 7: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/7.jpg)
7High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 8: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/8.jpg)
8High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 9: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/9.jpg)
9High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 10: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/10.jpg)
10High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 11: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/11.jpg)
11High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 12: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/12.jpg)
12High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 13: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/13.jpg)
13High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 14: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/14.jpg)
14High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 15: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/15.jpg)
15High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 16: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/16.jpg)
16High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 17: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/17.jpg)
17High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 18: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/18.jpg)
18High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Best domain decomposition is dependent on algorithm.
![Page 19: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/19.jpg)
19High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
3rd party partitioning software often required
![Page 20: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/20.jpg)
20High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
A subset of processes responsible for each model
Atmospheric Model
Hydrology
Model
Ocean
Model
Land Surface Model
![Page 21: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/21.jpg)
21High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
From: source rankLetter # 1234 (tag)
To: destination rank
The message has:
A type - integer, real, etc.
A count
A location – memory address
Both source and destination
ranks need to provide these
Please send count number data of this type, located at memory address a on the source rank, to the destination rank and write to memory address b there.
![Page 22: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/22.jpg)
22High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 23: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/23.jpg)
23High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 24: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/24.jpg)
24High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 25: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/25.jpg)
25High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 26: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/26.jpg)
26High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 27: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/27.jpg)
27High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
MPI_Recv()
MPI_Send()
Process 0
MPI_Recv()
MPI_Send()
Process 1
Tim
e
![Page 28: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/28.jpg)
28High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 29: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/29.jpg)
29High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 30: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/30.jpg)
30High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
MPI_Barrier
Tim
eT
![Page 31: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/31.jpg)
31High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Root
10010110
0
10010110
2
10010110
1
10010110
Size-1
10010110
![Page 32: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/32.jpg)
32High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Proc 0
01101
Proc 1
10101
Proc 2
11001
Proc 3
01011
Proc 4
11101
01101 10101 11001 01011 11101
![Page 33: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/33.jpg)
33High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 34: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/34.jpg)
34High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 35: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/35.jpg)
35High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 36: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/36.jpg)
36High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 37: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/37.jpg)
37High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 38: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/38.jpg)
38High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Basic datatype
col_type: a derived datatype that is strided
row_type: a derived datatype that is contiguous
![Page 39: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/39.jpg)
39High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 40: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/40.jpg)
40High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 41: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/41.jpg)
41High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Rank=0
Coord=(0,0)
Rank=1
Coord=(0,1)
Rank=2
Coord=(0,2)
Rank=3
Coord=(1,0)
Rank=4
Coord=(1,1)
Rank=5
Coord=(1,2)
![Page 42: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/42.jpg)
42High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
![Page 43: Message Passing Interface a brief introduction](https://reader034.vdocument.in/reader034/viewer/2022050714/623144eb869a015a9f1f281a/html5/thumbnails/43.jpg)
43High Performance Computing Consulting | Numerical Algorithms | Software Engineering Services | www.nag.com
Experts in High Performance Computing, Algorithms and Numerical Software Engineering
www.nag.com | blog.nag.com | @NAGtalk