![Page 1: Programming Models for Exascale Systems: What, When and How?nowlab.cse.ohio-state.edu/static/media/workshops/presentations/es… · Programming Models for Exascale Systems: What,](https://reader034.vdocument.in/reader034/viewer/2022050410/5f8768f493f18a16db08333f/html5/thumbnails/1.jpg)
Programming Models for Exascale Systems: What, When and How?!
!Kathy Yelick!
Associate Laboratory Director for Computing Sciences!Lawrence Berkeley National Laboratory !
and !Professor of Electrical Engineering and Computer Science!
UC Berkeley
![Page 2: Programming Models for Exascale Systems: What, When and How?nowlab.cse.ohio-state.edu/static/media/workshops/presentations/es… · Programming Models for Exascale Systems: What,](https://reader034.vdocument.in/reader034/viewer/2022050410/5f8768f493f18a16db08333f/html5/thumbnails/2.jpg)
1. Lightweight cores will have all/most of the system performance – Need fine-‐grained parallelism; avoid unnecessary synchroniza8on – Cores not powerful enough for complex communica8on protocols ?
2. On-‐chip interconnect offers opportuni=es for performance – New models of communica8on may be essen8al
3. Hardware is heterogeneous: no single ISA – Portability and performance portability are challenging
4. New levels of memory hierarchy, possibly soIware-‐controlled – Locality and communica8on-‐avoidance paramount
5. Performance variability may increase – SoCware or hardware control clock speeds
6. Resilience will be paramount at scale – Failures grow with the number of components and connec8ons
Why is Change Necessary?
2 Programing Models and Environments
![Page 3: Programming Models for Exascale Systems: What, When and How?nowlab.cse.ohio-state.edu/static/media/workshops/presentations/es… · Programming Models for Exascale Systems: What,](https://reader034.vdocument.in/reader034/viewer/2022050410/5f8768f493f18a16db08333f/html5/thumbnails/3.jpg)
Don’t Overstake Claims – Prove them!
Programing Models and Environments 3
![Page 4: Programming Models for Exascale Systems: What, When and How?nowlab.cse.ohio-state.edu/static/media/workshops/presentations/es… · Programming Models for Exascale Systems: What,](https://reader034.vdocument.in/reader034/viewer/2022050410/5f8768f493f18a16db08333f/html5/thumbnails/4.jpg)
The Trade-Off Is With Us
Programing Models and Environments 4
Locality Load Balance
![Page 5: Programming Models for Exascale Systems: What, When and How?nowlab.cse.ohio-state.edu/static/media/workshops/presentations/es… · Programming Models for Exascale Systems: What,](https://reader034.vdocument.in/reader034/viewer/2022050410/5f8768f493f18a16db08333f/html5/thumbnails/5.jpg)
• Target new applica=ons, not rewri=ng old ones – The mo8va8on is higher (all that code to be wriIen) – The barrier to entry is lower (no code to throw away)
• Target parts of applica=ons, not full rewrites – Support for incremental adop8on is key – Need to leverage exis8ng libraries and parts of exis8ng codes
• Target new programming paOerns – Irregular-‐in-‐space communica8on (random access to large data) – Irregular-‐in-‐8me (one-‐sided access to shared state) – Irregular workload (dynamic load balancing)
• Look for low hanging performance fruit – Do you rooflining homework – What’s the threshold? 15% -‐ no, 15x -‐ yes
Approach to Adoption
5 Programing Models and Environments
![Page 6: Programming Models for Exascale Systems: What, When and How?nowlab.cse.ohio-state.edu/static/media/workshops/presentations/es… · Programming Models for Exascale Systems: What,](https://reader034.vdocument.in/reader034/viewer/2022050410/5f8768f493f18a16db08333f/html5/thumbnails/6.jpg)
• Target new applica=ons, not rewri=ng old ones – The mo8va8on is higher (all that code to be wriIen) – The barrier to entry is lower (no code to throw away)
• Target parts of applica=ons, not full rewrites – Support for incremental adop8on is key – Need to leverage exis8ng libraries and parts of exis8ng codes
• Target new programming paOerns – Irregular-‐in-‐space communica8on (random access to large data) – Irregular-‐in-‐8me (one-‐sided access to shared state) – Irregular workload (dynamic load balancing)
• Look for low hanging performance fruit – Do you rooflining homework – What’s the threshold? 15% -‐ no, 15x -‐ yes
Approach to Adoption
6 Programing Models and Environments
Work with applica8ons people to understand
what they need!
![Page 7: Programming Models for Exascale Systems: What, When and How?nowlab.cse.ohio-state.edu/static/media/workshops/presentations/es… · Programming Models for Exascale Systems: What,](https://reader034.vdocument.in/reader034/viewer/2022050410/5f8768f493f18a16db08333f/html5/thumbnails/7.jpg)
Random Access to large state
Dynamic load
balancing
Fusion of observa8on
into simula8on
Unstruct-‐ured DAG algorithms
Application Challenges (Opportunities?)
7 Programing Models and Environments
Infinite 15-‐20% Infinite 2x Speedup
Programming mode role: encourage a way of thinking
![Page 8: Programming Models for Exascale Systems: What, When and How?nowlab.cse.ohio-state.edu/static/media/workshops/presentations/es… · Programming Models for Exascale Systems: What,](https://reader034.vdocument.in/reader034/viewer/2022050410/5f8768f493f18a16db08333f/html5/thumbnails/8.jpg)
Lightweight mechanisms, flexible policies • Atomic remote memory opera=ons (compare&swap,…) • 1-‐way put + signal (2 addresses, data + increment) • SoIware-‐managed memory as an op#on • Fault detec=on and resilience job schedulers, etc. • Transparent performance informa=on
Hardware changes desired/required?
8 Programing Models and Environments
![Page 9: Programming Models for Exascale Systems: What, When and How?nowlab.cse.ohio-state.edu/static/media/workshops/presentations/es… · Programming Models for Exascale Systems: What,](https://reader034.vdocument.in/reader034/viewer/2022050410/5f8768f493f18a16db08333f/html5/thumbnails/9.jpg)
Programing Models and Environments 9
Give an applica=on some op=miza=ons and you feed it for a
machine.
Teach a computer to generate op=mized code for an applica=on
and you feed it for a life=me
![Page 10: Programming Models for Exascale Systems: What, When and How?nowlab.cse.ohio-state.edu/static/media/workshops/presentations/es… · Programming Models for Exascale Systems: What,](https://reader034.vdocument.in/reader034/viewer/2022050410/5f8768f493f18a16db08333f/html5/thumbnails/10.jpg)
Thank you!
![Page 11: Programming Models for Exascale Systems: What, When and How?nowlab.cse.ohio-state.edu/static/media/workshops/presentations/es… · Programming Models for Exascale Systems: What,](https://reader034.vdocument.in/reader034/viewer/2022050410/5f8768f493f18a16db08333f/html5/thumbnails/11.jpg)
• What kind of programming models will be good for Exascale systems? Will it be evolu=onary or revolu=onary?
• Will MPI+X (together with PGAS and Task-‐based models) will be best suited for Exascale systems?
• Will newer programming models like HPC++, Legion, HPX, etc. gain more trac=on?
• How will the upcoming hardware trend (accelerators, coprocessors, high bandwidth memory, on-‐chip networking, etc.) have impact on the suitability of programming models?
11 Programing Models and Environments