pl 40-007 katowice, bankowa 14; …paad: platforms for scientific data analysis and hpc joachim j....
TRANSCRIPT
![Page 1: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/1.jpg)
PAAD: Platforms for Scientific Data Analysis and HPC
Joachim J. WłodarzUniversity of Silesia, Faculty of Mathematics, Physics
and Chemistry, Theoretical Chemistry Dept. PL 40-007 Katowice, Bankowa 14; <[email protected]>
KU KDM’16, Zakopane, 16-18.03.2016
![Page 2: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/2.jpg)
PAAD: rationale
● research is de facto data processing● data⇒information⇒knowledge (we hope :-)● Big Data are commonplace● 4V rule: Variety, Velocity, Veracity, Volume● HPC needed for many tasks● interactive scientific computing welcome● scientific data archiving required (Nature, ...)
![Page 3: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/3.jpg)
PAAD: for biosciences
● high-speed genome sequencing● morphogenesis modeling● biological processes modeling● image processing● Galaxy bioinformatics interactive toolkit● scientific data archival storage
![Page 4: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/4.jpg)
PAAD: for chemistry & physics ● laboratory data acquisition and processing● quantum chemical calculations● particle physics Monte Carlo calculations● algebraic calculations and tests on CAS ● interactive computing: Sage/Jupyter/IPython● calculations for the LHCPhenoNet project● scientific data archival storage
![Page 5: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/5.jpg)
PAAD: for earth sciences
● GIS data processing● geo-referenced data processing● meteorological data processing● geomorphological and explorational data● simulations of geological processes● scientific data archival storage
![Page 6: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/6.jpg)
PAAD: for any research
● Linux-based environment● other environments in virtual machines● universal HPC/HA setup, batch & interactive● Python-based interactive computing ● numerical and symbolic calculations● open source computing software (!)● archival data storage
![Page 7: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/7.jpg)
PAAD: for education
● Sage/Jupyter/IPython CAS environments● “living documents” paradigm (“notebook”)● ready-to-use materials from iCSE project● http://icse.us.edu.pl/materialy-dydaktyczne/● from linear algebra to molecular modeling● multiuser setup, browser-based access● any decent browser supported
![Page 8: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/8.jpg)
PAAD: outline of the infrastructure
HA nodes HPC nodes
storage
interconnects
frontend/master nodes
storage
![Page 9: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/9.jpg)
PAAD: hardware dimensioning
● storage: 80 TB/yr * 5 yr = 400 TB ⇒ 700 TB● memory: 8-32 GB/job ⇒ ~ 16 GB/job● CPU: 4-16 C/job ⇒ ~ 8 C/job● 3-4 grp * 5-10 job/grp ⇒ 40 jobs (?)● HPC nodes: ⇒ ~ 40: 16 C, 128 GB ⇒ 44● HA nodes: ⇒ 4: 16 C, 256 GB ⇒ 4● CI: 56 Gbps IB, storage: IB or 10 Gbps Eth
![Page 10: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/10.jpg)
PAAD: computing accelerators
● performance ⇒ ~100 TFLOPS (DP)● energy consumption ⇒ ~ 40-50 kW● support for paralell processing● support of GPU-accelerated software● cost effective hardware● ⇒12 nodes: +2x Xeon Phi 7120P (~Intel64)● ⇒12 nodes: +2x Nvidia Tesla 40M
![Page 11: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/11.jpg)
PAAD: the machinery
![Page 12: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/12.jpg)
Acknowledgements
![Page 13: PL 40-007 Katowice, Bankowa 14; …PAAD: Platforms for Scientific Data Analysis and HPC Joachim J. Włodarz University of Silesia, Faculty of Mathematics, Physics](https://reader030.vdocument.in/reader030/viewer/2022040405/5e977e63b90f7c29da22d6b1/html5/thumbnails/13.jpg)
Thank you for your attention