Message Passing Computing 1 iCSC2015,Helvi Hartmann, FIAS Message Passing Computing Lecture 2 Message Passing Helvi Hartmann FIAS Inverted CERN School of Computing, February 2015 Message Passing Computing 2 iCSC2015,Helvi Hartmann, FIAS Cluster Computing parallel computing on systems with distributed memory Message Passing Computing 3 iCSC2015,Helvi Hartmann, FIAS ZMQ, PVM, MPI, HPX, OpenMP Any questions? Message Passing Computing 4 iCSC2015,Helvi Hartmann, FIAS What Methods of Message Passing do you know? Message Passing Computing 5 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Message Passing Computing 6 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Message Passing Computing 7 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Message Passing Computing 8 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Message Passing Computing 9 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ Message Passing Computing 10 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ Message Passing Computing 11 iCSC2015,Helvi Hartmann, FIAS OpenMP Message Passing Computing 12 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ Message Passing Computing 13 iCSC2015,Helvi Hartmann, FIAS Socket Programming Message Passing Computing 14 iCSC2015,Helvi Hartmann, FIAS ZMQ automatically reconnects to peers queues messages at sender and receiver routes messages handles sockets errors does all I/O in background threads lock free techniques for communication Message Passing Computing 15 iCSC2015,Helvi Hartmann, FIAS ZMQ Socket Types Request/Reply blocking Message Passing Computing 16 iCSC2015,Helvi Hartmann, FIAS ZMQ Socket Types Push/Pull nonblocking Message Passing Computing 17 iCSC2015,Helvi Hartmann, FIAS ZMQ Socket Types Publish/Subscribe Message Passing Computing 18 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ Message Passing Computing 19 iCSC2015,Helvi Hartmann, FIAS Message Passing Message Passing Computing 20 iCSC2015,Helvi Hartmann, FIAS PVM Master Slave Process 1Process 0 Process 2 Process 0Process 1Process 2 Message Passing Computing 21 iCSC2015,Helvi Hartmann, FIAS MPI Processes node0 node1 Process 0Process 1 ressource manager Message Passing Computing 22 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Processes send receive Processes send receive Message Passing Computing 23 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Synchronous Send (MPI_Ssend) Message Passing Computing 24 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Buffered Send (MPI_Bsend) Message Passing Computing 25 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Message Passing Computing 26 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Synchronous Send (MPI_Ssend) Buffered Send (MPI_Bsend) Message Passing Computing 27 iCSC2015,Helvi Hartmann, FIAS MPI - Blocking Processes send receive Processes send receive function returns immediately Message Passing Computing 28 iCSC2015,Helvi Hartmann, FIAS MPI - Nonblocking Message Passing Computing 29 iCSC2015,Helvi Hartmann, FIAS MPI - Nonblocking Message Passing Computing 30 iCSC2015,Helvi Hartmann, FIAS MPI Collectives Broadcast Message Passing Computing 31 iCSC2015,Helvi Hartmann, FIAS MPI Collectives ScatterGather Message Passing Computing 32 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ Message Passing Computing 33 iCSC2015,Helvi Hartmann, FIAS Futures ? locality 1 future locality 1 Time ? locality 2 Message Passing Computing 34 iCSC2015,Helvi Hartmann, FIAS Futures ? locality 1 2 Time ? locality 2 ? locality 3 ? locality 4 34 Message Passing Computing 35 iCSC2015,Helvi Hartmann, FIAS Inter Process Communication From every process accessible information File Asynchrounous notification of an event Signal output of each process streams directly as input for another process Pipe Processes access same data in memory Shared MemoryOpenMP Data is send to specific sockets SocketZMQ Data is send to specific sockets Message Passing PVM MPI Processes outsource work to subroutines Remote Procedure CallZMQ Message Passing Computing 36 iCSC2015,Helvi Hartmann, FIAS Message Passing Computing 37 iCSC2015,Helvi Hartmann, FIAS :far-far-more-advanced-than-any-software-weve-ever-created-bill-gates/Llano html itter_globus_ html