general assembly 2019

All-members- Call

05 March 2020 14h00 – 16h30

Agenda

05/03/2020ETP4HPC All-members telephone conference 3

14h00 – 14h10 Welcome and context Jean-Pierre Panziera

14h10 – 14h45 News on ETP4HPC and EuroHPC Jean-Pierre Panziera

14h45 – 16h30

SRA 4: what is new? Michael Malms

Presentation of 4 of the SRA subtopics

- Application-Codesign- Trustworthy Computing- Mathematical Methods and Algorithms- Data Everywhere

Andreas Wierse (Sicos BW)Jens Krüger (Fraunhofer)Dirk Pleiter (JSC) Sai Narasimhamurthy (Seagate)

Extremes' prediction in the Digital Continuum Peter Bauer (ECMWF)

What's next: upcoming topics Michael Malms

GA - 2

GA – 2: current planning


Situation● From a legal side, need to hold the elections and the votes within the next 3

months (leading us into May/ June)● Currently, even events planned for March/ April 2020 are being cancelled, and

many postponed events will probably be held in early summer 2020

Suggestion (tbc)1. All-members-telco to explain the details of the process for vote on the

Steering Board and for vote on the resolutions2. Vote on the SB remotely (either by mail or electronically)3. All-members-telco to share the outcome of the SB election and to present

the financial report and the resolutions in details, with request to the members to vote on this (to be detailed later)

4. Have a networking event on the Sunday of ISC (21/06/2020) in Frankfurt

ETP4HPC News

Members


Overall, 96 members as of today

7 new members in 2019 ● 3 SMEs ● 3 Research institutes ● 1 Association

Agenium

SABRI

PLLANA

As of today: Members by statue


Steering Board as of today


New election upcoming● Research organisations

BSC, CEA, CINECA, Fraunhofer, FZ Jülich● SMEs

E4, Megware, ParTec● International Organisations

Fujitsu, Intel● European Organisations

Atos-Bull, ESI Group, Infineon, Seagate

Elected by the Steering Board ● ATOS-Bull, represented by Jean-Pierre Panziera, as Chairman● ParTec, represented by Hugo Falter as Secretary ● ClusterVision, represented by Frank van der Hout, as Treasurer from 10/04/2018 until 12/02/2019 ● Fraunhofer, represented by Franz-Josef Pfreund, as Treasurer as from 17/06/2019● SEAGATE, represented by Sai Narasimhamurthy Vice-chair for Industry● CEA, represented by Jean Gonnord, as Vice-chair for Research

ETP4HPC Office 2019 ● Pascale Bernier-Bruna (Atos): communication,

website, social media● Carolien Dekker:

accounting● Chris Ebell (Partec):

ad hoc support● Maike Gilliot (Teratec):

office coordination, EXDCI-2 coordination, workshops

● Michael Malms (IBM): SRA and related activites

● Jean-Philippe Nominé (CEA): office coordination, support to Chairman

● Marcin Ostasz: roadmap, industrial relations, member relations, back office


We are looking for additional support Join us !

2019 Activities

Activity report Detailed and illustrated report available

● In print Order your copies via [email protected]

● Online on www.etp4hpc.euLook for it in the latest News or on the Resources page


EuroHPC and RIAG Activities

EuroHPC


● Formal existence started in Oct. 2018

● 32 Participating states– 26/27 EU countries

– Associated membersIceland, Montenegro, North Macedonia, Norway, Switzerland & Turkey

● 2019-20 budget from EU 486M€

● 3 pre-Exascale + 5 Petascalesystems

● 180M€ for Research and Innovation

EuroHPC/ RIAG


● Governing Board– Members appointed– First meetings took place

● Infrastructure Advisory Group (InfraGA)– Members appointed – Hosting entities for peta-scale and pre-exa-scale

machines selected; procurements ongoing

● Research and Innovation Advisory Group (RIAG)– Members appointed – First meeting on 01/03/2019

RIAG 2019


MEMBERS• Jean-Pierre Panziera (Atos)• Axel Auweter (Megware)• Dr Carlo Cavazzoni (Cineca)• Jean Gonnord (CEA)

OBSERVERS• Hugo Falter (Partec)• Dr Katerina Slaninová (IT4Innovations)• Isabella Weger (ECMWF)

RIAG 2019 – SRIA / WP 2019-20 (1/2)● 04/05/2019 – RIAG issues SRIA 2019● 25/07/2019 – 5 calls EuroHPC_0[1-5] published 25/07

95M€ from EuroHPC budget + matching funds from PS

● EuroHPC_0[4-5] closed 14/11– EuroHPC_04-2019; RIA+CSA: “HPC Competence Centres” (28+2M€)– EuroHPC_05-2019; RIA: “Stimulating the innovation potential of SMEs”(10M€)

– One inadmissible proposal – one eligible proposal per topic– Evaluation implemented according to H2020 rules and standards– Pending GB decision to start GA preparations


EuroHPC-0[123]-2019 calls


EuroHPC-0[123]-2019 statistics

Note: “ * 11M EUR flexibility fund reserved to allocate across topics”


RIAG 2019 – SRIA 2020● Topics proposed by RIAG:

– EPI phase 2– 2x Pilot systems using European technology– Education & Training – to be further defined

+ 1 pilot for Quantum Computing


RIAG 2020● Published together with INFRAG group a common vision paper

● Prepare long term Strategic Research & Innovation Agenda for next Multi-annual Financial Framework “Horizon Europe” (2021-27)– ETP4HPC SRA-4 will be a strong basis for the proposal– Expect final draft June 2020

● Prepare SRIA for Work Programme 2021-22


Workshop in Luxemburg Jan 10, 2020

EuroHPC/DG-CNECT requested discussion session together with ETP4HPC and BDVA:● the amendment of the EuroHPC Regulation for the next MFF● industry access to the EuroHPC supercomputers● direct co-investment, in particular in computing capabilities● etc…


New RIAG-member form within ETP4HPC ● Jean Gonnord’s mission for CEA has ended. New

ETP4HPC representative for CEA is Jean-Philippe Nomine.

● For replacement, follow the same process used for the RIAG-Observer nominations as presented at the GA in March 2019


Timeline and next steps


Date Action

04/05/2020 SB confirmed and validated the process

05/03 –10/03 (EoD)

Formal call for participations of candidates (please notify by e-mail to [email protected])

11/03 –15/03 (EoD) SB votes on the 4th RIAG members by e-mail to the office

16/03 or 17/03

Short extraordinary SB meeting to ratify and approve the outcome of the vote (tied votes shall be resolved by the SB during this meeting)

17/03/2020 Publishing of the results to all members ETP4HPC Chairman informs EuroHPC JU on the change

23/03/2020 Next RIAG Meeting, including the newly elected RIAG member (participation in person if possible)

Other activities

SRA 4 online ● In print:

in 2 weeks time order your copies at ETP4HPC Office

● Online on www.etp4hpc.eu


Industrial Users Working Group (IU WG) 1/2Position Paper (in the process of Completion)● Goal:

– Articulate ETP4HPC IU WG to other organizations/initiatives involved in this activity

– Summarize our current value proposition to HPC community in general ( based on all works & discussions we have had until now)

● Highlights our readiness for openness and collaboration● Aim is to follow through with more discussions (incl. during the EuroHPC summit)

– EuroHPC Industry Session (Managed by INFRAG – involvement from the Industrial User Task Force )

– Special Interest Group (Various Orgs and EU funded projects) on industry engagement

04 mars 2020ETP4HPC SB meeting - Telephone Conference 29

Industrial Users Working Group (IU WG) 2/2

IU WG Reach out● Evangelos Flores (who is liaising on EuroHPC Industry Session) ● Lee Margetts (PRACE Industrial Advisory Committee)● Industrial Users Task Force, led by Teratec/HLRS (In progress)

Next Major Step (New IU WG Roadmap): This needs to be formalized based on the evolving developments (IUTF, PRACE SIG, Competence centers, etc)

04 mars 2020ETP4HPC SB meeting - Telephone Conference 30

Ongoing projects EXDCI-2 and HPC-GIG


EXDI-2 admin Amendment ongoing and interim reporting • To be finalised

EXDCI-2 activities

WP2• SRA• Collaboration with “upstream technologies”• Work on the KPIs WP4 • project spin-off InitiativeWP6 • EuroHPC Summit Week, with sessions organised by

ETP4HPC, and PBB being active part of the Organisation Committee

• Deliverable submitted

• Ongoing

• Sessions organised by ETP4HPC• PBB being active part of the

Organisation Committee

HPC-GIG • Support activity for RIAG and INFRAG : for the time smaller role of ETP4HPC

ETP4HPC Handbook – 2019 edition● A catalogue of the 48 on-going

HPC projects – not just FETHPC projects, but HPC-related projects

● Online on ww.etp4hpc.eu

● In cooperation with EXDCI


Main events in 2019

Main Events in 2019


● General Assembly in Sassenheim on 19 March 2019

● EuroHPC Summit Week ● Forum Teratec 2019● ISC 2019● ETP4HPC BoF at SC19● ITC Event

Coming events


EuroHPC Summit Week 2020 ● Porto, Portugal● From 23 to 27 March 2020 ● ETP4HPC co-organiser

● ETP4HPC will again be involved in many private or public sessions:– EuroHPC RIAG meeting– Jean-Pierre Panziera chairs the second

part of the Opening Plenary session– EuroHPC BoF– EXDCI-2 workshops (some cancelled)– Transcontinuum work session (a follow

up of the SRA work, organized by Michael Malms with AIOTI / BDVA / EOCS)


ISC20● Large booth again to host some of

our SME members– Current plan:– 4 co-exhibitors– Each will have some wall space– More information soon

● Submission of a BoF-proposaljointly with our member Appentraon Quality Assurance and Coding Standards for Parallel Software


Teratec Forum 2020● 16-17 June 2020● Near Paris● Same size booth as in previous

years● In the new “Europa Village” set up

within the exhibition area


THANKS!

You can find us at:@[email protected]

The EXDCI-2 project has received funding from the European Unions Horizon 2020 research and innovation programmed under grant agreement no. 800957.

EXDCI-2 General Presentation

SRA -4 : a very short snapshotET4HPC All members - call

March 5th 2020

Andreas Wierse, SICOS

Jens Krueger, Fraunhofer

Sai Narasimhamurthy, Seagate

Dirk Pleiter, JSC

Michael Malms, ETP4HPC

2

Thank you !

Gabriel Antoniu

André Brinkman

Laurent Cargemel

Paul Carpenter

Herbert Cornelius

Marc Duranton

Hans-Christian Hoppe

Benny Koren

Jens Krueger

Erwin Laure

Guy Lonsdale

Manolis Marazakis

Sai Narasimhamurthy

María Pérez-Hernández

Dirk Pleiter

Pascale Rosse-Laurent

Estela Suarez

Adrian Tate

Andreas Wierse

SRA – 4 work group leaders

3

• System Architecture

• Laurent Cargemel, ATOS

• Estela Suarez, JSC

• Herbert Cornelius, MEGWARE

• System Hardware Components

• Marc Duranton, CEA (HiPEAC)

• Benny Koren, MELLANOX

• System Software and Management

• Pascale Rosse-Laurent, ATOS

• Maria Perez, UPM (BDVA)

• Manolis Marazakis (FORTH)

• Programming Environment

• Guy Lonsdale, SCAPOS

• Paul Carpenter, BSC

• Gabriel Antoniu, INRIA (BDVA)

• I/O & Storage

• Sai Narasimhamurthy, SEAGATE

• Andre Brinkmann, JGU

• Mathematics & Algorithms

• Dirk Pleiter, JSC

• Adrian Tate, NAG

• Application co-design

• Erwin Laure, KTH

• Andreas Wierse, SICOS

• Centre-to-edge-framework

• Jens Krueger, FRAUNHOFER

• Hans-Christian Hoppe, INTEL

A real team work: working groups and leads – more than 80 experts

4

Digital continuum

4

HPC in the loop

Smart sensors

Internet of Things

Physical systems

HPC / Cloud

Data analytics / Cognitive computing Big Data

Processing, abstracting,

understandingas soon as possible

Collaboration between edge devices and the HPC/cloud ensuring:• Data security and Privacy• Lower bandwidth• Better use of HPC/Cloud→ creating a continuous flow

Enabling Intelligent data processing at the edge:• Fog computing• Edge computing• Stream analytics

Transforming data into information as soon as possible

5

21/01/2020

Structure of technical chapters: “Research clusters” and “Research Domains”

System Architecture

For each research cluster define:

• relevance & impact (why chosen?)

• maturity (time to market)

• Hurdles to overcome

• Driving competence in Europe

• Cost of research to gain sign. uptake

System Hardware Components

System Software & Management

Programming Environment

IO & Storage

Mathematical methods & Algorithms

Application co-design

Centre-to-edge framework

Research - domains

Data

everywhere

AI

everywhere

Energy

efficiencyResilience

Developm. methods

and standardsHPC and the

digital continuum

Trustworthy

computing

6

Today : A short snapshot on:

System Architecture

For each research cluster define:

• relevance & impact (why chosen?)

• maturity (time to market)

• Hurdles to overcome

• Driving competence in Europe

• Cost of research to gain sign. uptake

System Hardware Components

System Software & Management

Programming Environment

IO & Storage

Mathematical methods & Algorithms

Application co-design

Centre-to-edge framework

Research - domains

Data

everywhere

AI

everywhere

Energy

efficiencyResilience

Developm. methods

and standardsHPC and the

digital continuum

Trustworthy

computing

7

• Application priorities and influences on HPC paradigm change:

• E.g. fundamental sciences; climate, weather, and earth sciences; life

sciences & health; energy; engineering & manufacturing; chemistry &

materials sciences; etc.

• Increasing uptake from industry

• Although at lower level than academia

• AI plays an increasing role in established and new HPC fields

• E.g. humanities, social sciences, finance, etc.

• Increased focus on data

• Also increased need for cyber security

Application co-design (Andreas Wierse) - 1

8

• Main Challenges:

• Porting, adapting, optimizing for new architectures

• Requires co-design, stable programming environments

• New opportunities through convergence of HPC, HPDA, and AI

• More complex workflows, data handling

• Memory bandwidth and communication latency becomes more

important than FLOPS

• Long term maintenance of applications and codes

• Continued funding and stable (standardized) environments


9


Intersections – the “heat map”

10

• Key: Critical interplay between modern HPC architectures

and (mathematical) methods for their efficient exploitation

• Increasing parallelism → scalable algorithms

• Reduced precision arithmetics and non-standard floating-point

number formats → robust algorithms

• Algorithms for unconventional architectures

• Data-flow architectures

• In-memory/-network processing

• Application-specific accelerators

Mathematical methods and algorithms (Dirk Pleiter) - 1

11

• Research priorities for 2021-2024:

• Robust methods and algorithms enabling extreme scalability

• Aim for new levels or degrees of concurrency

• Global communication-hiding/avoiding algorithms

• Fault-resilient algorithms

• Methods for (scalable) data analytics and artificial intelligence

• Algorithms for data discovery

• Algorithms reducing energy-to-solution

• Characterisation of algorithms in terms of energy-to-solution

• Reduction of data movement, exploitation of reduced precision and special hw

• Vertical integration and validation of mathematical methods and algorithms

• Vertical integration and validation

• Tuning of algorithmic parameters for exascale


12

• Intersections with Research Clusters:

• “AI everywhere” and “AI everywhere”

• Overlap with the research priority “Methods for (scalable) data analytics and

artificial intelligence”

• “Energy efficiency”

• Established opportunities for reducing energy-to-solution for solving given

numerical problem through use of different/new algorithms

• “Vertical integration and validation of mathematical methods and

algorithms” necessary to realize energy savings in practice

• “Resilience”

• Fault-resilient algorithms

• Algorithms addressing reproducibility of numerical results in the presence of

less reliable systems


13

• Main influences on HPC in this cluster:

• From model-driven and simulation-driven science to data science

• We are now in an era of “Data Centric” computing

• The data sphere: 175 ZB by 2025 (Source: IDC)

• Sensors and IoT becoming a major part of science and innovation

• Societal trends such as driverless cars and large experiments such as SKA

• Need to process data at the “edge” as well as at the data center

• HPC will increasingly start to play a role here

• Workflows across the whole digital continuum (edge - “fog” cloud data centers

– HPC systems) will exhibit specific HPC needs at each level

• Embedded HPC (fog-level)

• HPC clouds

• Supercomputers

Data everywhere – research cluster (Sai Narasimhamurthy) - 1

14

• Relevance and impact - why chosen:

• Scientific simulations and workflows will also generate and work with

enormous volumes of data

• Data needed by scientific workflows is now highly distributed

geographically

• Edge + Distributed “Fog” nodes + Clouds + HPC Centers

• Accumulated historical data + real-time data

• Various data processing techniques needed

• HPC data processing techniques: in-transit/in situ processing

• Big Data Processing Techniques (batch + real time/streaming)

• Trend: support decentralization of (AI-based) analytics towards the edge

• Cloud based data access & associated APIs (S3, Swift, etc)


15

• Hurdles to overcome:

• Requirement for high bandwidth between the various infrastructure entities

(Edge, Fog, Cloud/HPC system)

• Still lacking – inspite of in-transit and in-situ processing in HPC systems

• 5G networks may play a role here

• However data generation continues to outstrip the evolution of N/W bandwidths

• Data logistics and data life-cycle management

• When to move data, how long to retain data, etc

• Collecting provenance metadata becomes important

• Making data infrastructures lot more resilient/performant at scale

• Usage of new tech.s like NVRAM still not very clear

• Data consistency across these infras still a big problem

• Data federation: lack of unified APIs, need to cope with sensitive data


16

• Main influences on HPC in this cluster:

• Security is becoming an important factor

• HPC data centres usage model are changing; data science, deep learning

• HPC in the continuum

• Trustworthy computing for the complete HW / SW stack

• chip design + firmware

• operating systems

• runtimes

• applications design

• interfaces, etc.

• establishing processes and accountability within data centres

Trustworthy Computing – research cluster (Jens Krueger) - 1

17

• Relevance and impact - why chosen:

• Strategic investment

• trustworthiness of vendor products

• handling of critical data

• withstand new threat scenarios

• new customers

• GDPR compliance

• handling of personal data

• HPC technologies in critical environments

• HPC “in-the-loop”

• embedded HPC

Trustworthy Computing - research cluster (Jens Krueger) - 2

18

• Hurdles to overcome:

• Establishing security within the mindset of HPC

• implement security as obligatory part of every (EU-) call

• Cross-sectorial / cross-disciplinary effort

• a chain is only as strong as the weakest link (HPC as part of a workflow)

• Security vs. Energy Efficiency vs. Performance

• security as a class A member

Trustworthy Computing - research cluster (Jens Krueger) - 3

The EXDCI-2 project has received funding from the European Unions Horizon 2020 research and innovation programmed under grant agreement no. 800957.

EXDCI-2 General Presentation

Thank you !

October 29, 2014 PETER BAUER 2020

Extremes predictionin the digital continuum

Peter Bauer, ECMWF


European Commission’s Green Deal: Digital technologies

“Accessible and interoperable data are at the heart of data-driven innovation. This data, combined with digital infrastructure (e.g. supercomputers, cloud, ultra-fast networks) and artificial intelligence solutions, facilitate evidence-based decisions and expand the capacity to understand and tackle environmental challenges. The Commission will support work to unlock the full benefits of the digital transformation to support the ecological transition. An immediate priority will be to boost the EU’s ability to predict and manage environmental disasters. To do this, the Commission will bring together European scientific and industrial excellence to develop a very high precision digital model of the Earth."


The Digital Twin of the Earth from models and observations

September 2019 tropical cyclone Lorenzofrom simulations and satellite observations

Step 1: Turing test

Step 2: Digital Twin


The Digital Continuum

https://www.etp4hpc.eu/pujades/files/Blueprint%20document_20190904.pdf Scientific Research Agenda-4 (in preparation)

Processing: observational data and simulation models

Abstracting: impact-domain specific (food, water,…) information extraction

Understanding: realistic representation of Earth system

Important: fast access to information for, e.g., effective disaster response

https://www.etp4hpc.eu/pujades/files/Blueprint%20document_20190904.pdf


Digital Twin of the Earth

Let’s assume the digital twin needs to resolve Earth (= 3D: atmosphere, ocean; 2D: land, cryosphere, biosphere):• 1 km in horizontal space, 500 m in vertical space, • 6 hours in time for (near term = seasons/year), 1 day in time (long term = centuries)• 100 variables in 3D, 100 variables in 2D• 100 ensemble members for uncertainty

Near-term:• 3D: 500,000,000 x 200 x 4 x 365 x 100 x 100 = 1.46.1018

• 2D: 500,000,000 x 4 x 365 x 100 x 100 = 7.3.1015

O (1018-1019)Long term:

• 3D: 500,000,000 x 200 x 100 x 365 x 100 x 100 = 3.65.1019

• 2D: 500,000,000 x 100 x 365 x 100 x 100 = 1.825.1017

→ Needs observations for past+present and simulations for past+present+future

Past Present FutureObservations:Simulations:


The Digital Continuum: Smart sensorsSmart sensors in mobile phones, cars, wind farms, agriculture farms etc. for weather and air quality

1-hourly near-surface ozone observational coverage

(~1,000 samples; Copernicus)

• Available public station networks rather well exploited (WMO, EC Copernicus)

• Smart sensors offer huge potential:• Large gaps globally (tropics), regionally (less developed

countries), locally (cities)• Limited set of parameters• But, data not necessarily open (public availability vs business

model dilemma)• But, fast data access and quality control very important• But, edge computing needs new effort


The Digital Continuum: Internet of ThingsIoT for accessing relevant data & IoT for disseminating user specific information

• Combining weather data from operational providers + IoT devices + privately owned networks (stations, drones, aircraft, balloons)

• Providing personalised/localised forecast and associated services


The Digital Continuum: Big dataExtreme-scale data collection-processing-dissemination on tight schedules

Data acquisition

Data assimilation Forecast Product generation

Dissemination

RMDCN

Internet

Web services Internet

Archive Data Handling System

IoT

Workflows:

Hardware:

50 member ensemble forecast = compressed GRIB2 data @ 16bit & 24bit:

• @ 1.45km = TCo7999: 1,102 TiB

• Required to read 70%: x1.7 = 1,874 TiB

• Time to solution is 1 hour: 1,874 TiB / 3,600s = 533 GiB/s

• Performance 34 nodes: 140 GiB/s

• Required nodes: 533 GiB/s / 140 GiB/s = x3.8 = 122 nodes

100s billions within hours

PB within hours

Millions of requests


[Reichstein et al. 2019, Nature]

The Digital Continuum: Data analytics / cognitive computingMachine learning for information extraction, product tailoring, surrogate modelling

Observational data processing (edge & cloud &HPC):• Quality control and bias correction• Data selection Example:• Inversion (=retrieval) intelligent use of IoT data in• Data fusion (combining observations) combination with satellites• …

Prediction models (cloud & HPC):• Data assimilation (combining models

w/ observations) Example:• Surrogate model components accelerate forecast models• Prediction itself with ML-twin of chemistry• Model error statistics• …

Service output data processing (cloud &HPC): Example:• Product generation and dissemination identify pre-cursors of • Product feature extraction (data mining) extremes from• Product error statistics PB of data• Interactive visualisation and selection• Data handling (access prediction)• …


The Digital Continuum: HPC / Cloud

… 5% sustained!

Data acquisition

Data assimilation Forecast Product generation

Dissemination

RMDCN

Internet

Web services Internet

Archive Data Handling System

Extreme-scale computing to generate digital twin


The Digital Continuum: Physical systemsCyber-physical entanglement of real-time, digital twin of Earth system, optimal information of user-specific information on demand and physical systems in impact areas: energy, water, food, health, law

SMART FARMING


NovelEmergingEstablished

The Digital Continuum & Weather and Climate (extremes)

Cyber-physical entanglement of real-time, digital twin of Earth system, optimal information of user-specific information on demand and physical systems in impact areas: energy, water, food, health, law

Extreme-scale computing to generate digital twin

Machine learning for information extraction, product tailoring, surrogate modelling

Extreme-scale data collection-processing-dissemination on tight schedules

IoT for accessing relevant data & IoT for disseminating user specific information

Smart sensors in mobile phones, cars, wind farms, agriculture farms, cities

etc. for weather and air quality


The Digital Continuum will be essential for Destination EarthCommunications 19/02/2020: Strategy:

Action:

general assembly 2019

Documents