Research@SOIC

Panel: Beyond Exascale ComputingINTERNATIONAL ADVANCED RESEARCH WORKSHOP

ON HIGH PERFORMANCE COMPUTINGFrom Clouds and Big Data to Exascale and Beyond

Cetraro (Italy)July 9 2014

Geoffrey Fox gcf@indiana.edu

http://www.infomall.orgSchool of Informatics and Computing

Digital Science CenterIndiana University Bloomington

Research@SOIC 2

Comments I• Main comment: Machines are changing but not so clear

problems are. Do we know what post exascale age problems are?

• What are Big Data Problems: There is big data but current data analytics (machine learning) algorithms are classic parallel algorithms. adjective current maybe important -- may change

• Note two interesting aspects of Google or Amazon or Facebook parallelisms

• Back end data and users. both in billions or higher• Science is dominated by back end data• So I do expect differences between post exascale architectures

for extreme scale science and commercial

Research@SOIC 3

Comments II• Famous analytics libraries are R and Mahout. Not optimized for

scalable processing. Need to change this• Surely this will develop a new set of algorithms and architecture

challenges. Claim is algorithm change gave as much performance increase as hardware in simulations. Will this happen in analytics. what will be the needs of these complex new algorithms

• Current analytics are pretty homogeneous e.g. "deep learning" is extreme. 11 billion highly uniform parameters which are weights of artificial neural network like the initial finite difference algorithms we started with in parallel simulation area. Simulations have introduced heterogeneous algorithms with adaptive meshes etc; this likely to occur in analytics and lead to change

Research@SOIC 4

Comments III• Abstractions. Not so clear problems have structure

proposed for post exascale machine architectures. We can reformulate them but is that because they want to be reformulated or because efficient use of hardware demands reformulation?

• When Mother Nature evolves a blade of grass or a star, it is not put on a dataflow stack. it is updated in place

• I personally find it painful to map problems into multiple programming models -- cuda mpi threads ....