openhpc introduction

OpenHPC Introduction

Adrian ReberCentOS Dojo CERN 2018October 19

What is OpenHPC?

2 CENTOS DOJO CERN 2018

Software Repository

yum and zypper

CentOS 7 and SLES 12

x86_64 and aarch64

Why OpenHPC?

Multiple Compilers

Multiple MPIs

Three compilersTwo versions each

Already six permutations

Six compiler permutationsThree MPIs

Two versions eachAlready 36 permutations

Common setup formany HPC sites

What is OpenHPC?

Community effort toreduce duplication

OpenHPC: Mission and Vision

Vision: OpenHPC components and best practices will enable and accelerateinnovation and discoveries by broadening access to state-of-the-art, open-sourceHPC methods and tools in a consistent environment, supported by a collaborative,worldwide community of HPC users, developers, researchers, administrators, andvendors.

OpenHPC: Mission and Vision

Mission: to provide a reference collection of open-source HPC softwarecomponents and best practices, lowering barriers to deployment, advancement,and use of modern HPC methods and tools.

OpenHPC: Current Project Members

OpenHPC: Current TSC Members• Reese Baird, Intel (Maintainer)• Brayford, LRZ (Maintainer)• Coulter, Indiana University (End-User/Site Representative)• Chris Downing, Red Oak Consulting (Maintainer)• Craig Gardner, SUSE (Maintainer)• Renato Golin, Linaro (Testing Coordinator)• Michael Karo, Altair (Maintainer)• Janet Lebens, Cray (Maintainer)• Thomas Moschny, ParTec (Maintainer)• Takayuki Okamoto, Fujitsu (Maintainer)• Kevin Pedretti, Sandia National Laboratory (Maintainer)• Paul Peltz, Los Alamos National Laboratory (Maintainer)• Nam Pho, Harvard Medical School (Maintainer)• Cyrus Proctor, Texas Advanced Computing Center (Maintainer)• Adrian Reber, Red Hat (Maintainer)• Karl W. Schulz, UT Austin (Project Lead, Testing Coordinator)• Jeff Schutkoske, Cray (Component Development Representative)• Derek Simmel, Pittsburgh Supercomputing Center (End-User/Site Representative)• Chris Simmons, UT Dallas (Maintainer)• Nirmala Sundararajan, Dell (Maintainer)

Building Blocks: Pick and Choose

OpenHPC: Project History

OpenHPC: Number of Components

v1.0.1

v1.1.1

v1.2.1

v1.3v1.3.1

v1.3.2

v1.3.3 v1.3.4

v1.3.5

Jul-15 Jan-16 Aug-16 Mar-17 Sep-17 Apr-18 Oct-18

OpenHPC: Components Changed

41.6%43.8%

v1.0.1 v1.1 v1.1.1 v1.2 v1.2.1 v1.3 v1.3.1 v1.3.2 v1.3.3 v1.3.4 v1.3.5

What is OpenHPC?

Software Repository

Including:Lmod

Including:Provisioning: Warewulf

Including:Provisioning: xCAT

Including:Monitoring: Nagios

Including:Monitoring: Ganglia

Including:Resource Manager: PBS Pro

Including:Resource Manager: Slurm

Including:Compiler: GCC

Including:Compiler: LLVM

Including:Compiler: Intel

Including:Compiler: ARM

Including:MPI: Open MPI

Including:MPI: MVAPICH2

Including:MPI: MPICH

Including:MPI: Intel

Including:EasyBuild

Including:Spack

Including:Charliecloud

Including:Singularity

Including:Lustre Client

Including:BeeGFS Client

Including:Lot’s of different libraries

But not just aSoftware Repository

Excellent DocumentationIncluding Recipes

Each Release iscompletely tested

OpenHPC: Same Interface Everywhere

Some Technical Details

Based on RPM

Tries to avoid Conflicts

# repoquery --whatprovides "libopenblas.so.0()(64bit)”openblas-0:0.2.20-3.el7.x86_64openblas-gnu7-ohpc-0:0.2.20-1.ohpc.x86_64

# repoquery --whatprovides "libltdl.so.7()(64bit)"libtool-ltdl-0:2.4.2-22.el7_3.x86_64libtool-ohpc-0:2.4.6-1.x86_64

RPM ColoringOpenHPC RPMs add ”(ohpc)” color string to shared objects (.so) for ”Provides:”and ”Requires:”:

# cat /usr/lib/rpm/fileattrs/ohpc.attr%__ohpc_provides /usr/lib/rpm/ohpc-find-provides%__ohpc_requires /usr/lib/rpm/ohpc-find-requires %{buildroot} /opt/ohpc

%__ohpc_path ^/opt/ohpc%__elf_exclude_path ^/opt/ohpc

%__ohpc_magic ^ELF (32|64)-bit.*$%__ohpc_flags magic_and_path

RPM Coloring# rpm -q --requires gnu7-compilers-ohpc | egrep “libc.so|libgc”

Without coloring With coloringlibgcc_s.so.1()(64bit) libgcc_s.so.1()(64bit)(ohpc)libgcc_s.so.1(GCC_3.0)(64bit) libgcc_s.so.1(GCC_3.0)(64bit)(ohpc)libgcc_s.so.1(GCC_3.3)(64bit) libgcc_s.so.1(GCC_3.3)(64bit)(ohpc)libgcc_s.so.1(GCC_4.2.0)(64bit) libgcc_s.so.1(GCC_4.2.0)(64bit)(ohpc)

libc.so.6()(64bit) libc.so.6()(64bit)libc.so.6(GLIBC_2.10)(64bit) libc.so.6(GLIBC_2.10)(64bit)libc.so.6(GLIBC_2.11)(64bit) libc.so.6(GLIBC_2.11)(64bit)libc.so.6(GLIBC_2.14)(64bit) libc.so.6(GLIBC_2.14)(64bit)

OpenHPC Home: https://openhpc.community/Primary GitHub Site: https://github.com/openhpc/ohpcBuildsystem: https://build.openhpc.community/Package Repositories: http://build.openhpc.community/OpenHPC:/Component Submission: https://github.com/openhpc/submissionsSystem Registry: System Registration FormCI Infrastructure: http://test.openhpc.community:8080/OpenHPC Wiki: https://github.com/openhpc/ohpc/wikiMailing Lists: https://openhpc.community/support/mail-lists/

THANK YOU

openhpc introduction

Documents

cluster computing with openhpc - louisiana state...

meeting of the technical steering committee (tsc) board ›...

optimisation des requêtes. introduction introduction

introduction introduction to brittle deformation

an introduction to nsan introduction to ns2an introduction...

introduction & strategy module introduction &...

introduction and thesis introduction - …...1 introduction...

crowpiwithminecraftpiedition · 2...

openhpc (v1.2) cluster building recipes 2017-04-04 ·...

introduction - daks2k3a4ib2z.cloudfront.net€¦ ·...

introduction to elasticsearch and logstash. - kit · pdf...

introductions (leads) academic introduction creative...

introduction | einleitung | introduction | introducción ·...

the sacrament of baptism introduction video introduction...

[introduction] an introduction to wavelets

openhpc (v1.3.3) cluster building recipes - groups.io ·...

class 1 introduction 1 introduction

introduction to transducers, introduction to transducers...

industry introduction industry introduction company...

openhpc: community project...