singularity community and singularitypro on hpe …...singularity community and singularitypro on...

Singularity Community and SingularityPRO on HPE high-performance serversThe power of open source for enterprise performance computing

Whitepaper

Singularity Community and SingularityPRO on HPE high-performance servers

2

Table of Contents

2 Executive summary

3 Introduction

4 Use cases

5 Mobility of compute

7 Drop-in replacement for standalone processes

8 Architectural differences between Singularity and other containers

8 SingularityPRO on HPE infrastructure, ready for the enterprise

10 SingularityPRO add-on services

11 Container Library

12 Key-signingandverificationservices

12 Conclusions

Executive summary

Singularity has become an attractive container technology for running batch-style jobs

becauseitwasdesignedspecificallytoencapsulatereproducibleapplicationstacksintoa

singlefile.ItssimplicityallowsforseamlessintegrationwithGPUsandinterconnectsthatare

commontohigh-performancecomputing(HPC)environments.RunningSingularityPRO™

onHewlettPackardEnterprise(HPE)HPCserverplatformsexpandsuponSingularity

Community’s open source capabilities and includes commercial support and access to a

growingvalue-addedcontainerecosystem.

This white paper illustrates the business drivers for adopting container-based software

modelsandthecapabilitiesbuiltintotheSingularityPROcommercialoffering.Itwillalso

address how Singularity container technology running on HPE platforms solve the challenges

ofparallelizedAI,deep-learning/machine-learning,anddataanalyticsworkloadsonlarge

clusters—allwithoutcompromisingsecurityorprivacy.

Threekeytakeawaysfromthiswhitepaperinclude:n Typical use cases for Singularity Community/SingularityPRO running on HPE server

platformsn The unique value of SingularityPRO for today’s enterprisesn ThebenefitsofSingularityPROcomparedtoothercontainerofferings


3

Introduction

Containersareahottopicineveryfacetofhigh-performancecomputing.Applieduse

casesareseeninavarietyofindustries,includingacademia,finance,enterprise,and

pharmaceuticals.451Researchexpectsmorethan250%growthinthecontainermarketfrom

2016to2020[1].Containerscombinespeedanddensitywiththemobilityoftraditionalvirtual

machines(VMs)whilerequiringfarfewercomponentstoremainportableandrunanywhere.

ContainersaremadepossiblebyasetoffacilitiesintheLinux®kernelthatallowlightweight

partitioning of a host operating system into isolated spaces where applications can safely

run.Usingcontainerspresentsloweroverheadintermsofasmallermemoryfootprintand

higherefficiencybecausetheysharethekernelwiththehostoperatingsystem—whichmeans

containerscanachievehigherdensity.Inshort,containersenablemoreproductivity.

Not only are containers orders of magnitude faster in provisioning, and lighter weight, they

alsoenableapplicationstoworkinthesamewayondevelopers’workstations,on-premises

servers,andanypublicorprivatecloud.

Proven open source container solution

Released in 2016, Singularity Community is an open source-based container platform de-

signedforscientificandHPCenvironments.ForHPC,Singularitymakeswhatwaspreviously

impossible,possible.

WithSingularity,theentireexecutionenvironmentiscontainedwithinasinglefilethatstarts

with a base Linux distribution, augmented by applications, libraries, data, and scripts—all

foracontainerizedapplicationworkflow.Singularitycontainerseasilyintegrateintostandard

HPCworkflowsandcanbedeployedandstartedontensofthousandsofnodeswith

minimaleffort.

By moving away from the microservices architecture embraced by other container platforms,

Singularity’s unique design meets HPC users’ needs for a container solution that not only

offers high performance, but also supports mobility, reproducibility, and seamless integration

withhost-providedresources.Inadditiontoenablinggreatercontrolovertheapplication

environments, Singularity also supports a bring-your-own-environment (BYOE) model—

transportingaconfigurationfromascientist’sworkstationtothedatacenter.


4

High-performance enterprise-class container platform

SingularityPRO builds on the success of the open source Singularity Community version,

leveraging the open source code base to provide a container platform designed for

Enterprise Performance Computing (EPC), including deep learning, IoT, and predictive

analyticsworkloads.

SingularityPRO includes all of the functionality of the open source version, plus enterprise-

gradeenhancementsthatmaketheplatformstronger,highlysecure,andmorefeature-rich

(describedbelow).WheretheopensourceversionofSingularityissubjecttorollingcode

changes from the open source community at large, SingularityPRO is curated and supported

bySylabs,thecompanybehindSingularity.

Use cases

SingularityPROrunningonHPEplatforms(includingHPESuperdomeFlex,HPEIntegrity

Superdome X, HPE Integrity MC990 X, and HPE Apollo systems) delivers high-performance

computingtoenterprises.Thisisdonebyprovidingasecureandrepeatablemethodto

packageapplicationsandtheirdependenciesintoasinglefilethatiscryptographically

verifiabletoensurereproducibility.Thesefeaturesarecriticallyimportantinthefollowing

enterpriseusecases.

A Major milestone in Memory-Driven Computing

To help enterprises embrace the possibilities of a world transformed by exponentialdatagrowth,HPEoffersSuperdomeFlex—theindustry’sonly in-memory computing solution with a unique modular design that scaleseasilyandeconomicallyforbusinessesofanysize.Asignificantmilestone in the Memory-Driven Computing innovation roadmap, this platform will help enterprises stay ahead of the competition by turning criticaldataintoreal-timebusinessinsights.Builttohandlethemostdemandingapplications,HPESuperdomeFlexdeliversanunprece-dentedcombinationofscale,modularity,flexibility,andreliabilitysothatenterprises can turn these insights into action, and action into success—knowingthatthebusinesswillremainalwayson.


5

Cluster Multi-tenancy

In an HPC environment, users are not allowed full, unrestricted administrative/root access to

sharedproductionsystems[2].Instead,usersoftenreceivecredentialswithlimitedaccessto

reducethethreatsurfaceareas.Whilelimited-usercredentialssatisfysecurity,compliance,

and audit requirements, users must be able to have enough environment privilege to

develop,modify,andtesttheirapplicationcontainers.

Figure 1: Singularity adds a new layer of isolation

Unlikeotherplatforms,Singularitydoesnotrequireausertohaverootprivilegeswithin

a container, and it does not require users to be added to a special group with advanced

privilegestostartthecontainerruntime.Singularity’suniquesecuritymodelensures

that untrusted users can run untrusted containers without impacting the security of the

underlyinghostsystem.

EnablinguserstodeploySingularitycontainersonaclusterprovidestheflexibilitytheyneed,

whilealsomaintainingthesecuritypostureofthecluster.

Mobility of compute

Enterpriseworkloadsareevolving.Jobsnowconsistofartificialintelligence(AI),machine

learning(ML),anddeeplearning(DL)workloadsthatweresolelywithinthedomainofthe

scientificresearchcommunity.SupportingthedemandingEPCusecasesfoundintoday’slife

sciences,defense,financialtechnology,oilandgas,manufacturing,andmanyothertypesof

workloadsrequireacontainerplatformthatdelivershighlevelsofperformance,portability,

andsecurity.


6

Mission-critical innovations

Forenterprisesrunningmission-criticalapplicationsoncostlyproprietarysystems, HPE Integrity Superdome X sets new high standards for x86 avail-ability,scalability,andperformance.TheidealplatformforcriticalLinuxandWindows®workloads,HPESuperdomeXblendsx86efficiencieswithprovenHPE mission-critical innovations for a superior uptime experience and ground-breakingperformance.Breakthroughscalabilityofupto16socketsand48TBofmemoryhandlein-memorydatabasesandlargescale-upx86workloads.Through the unique HPE nPars technology, Superdome X adds agility and de-livers20xgreaterreliabilitythanplatformsrelyingonsoftpartitionsalone.Formaximizing application uptime, standardizing, or consolidating, HPE Integrity SuperdomeXhelpstransformtoday’smission-criticalenvironments.

Singularity running on HPE server platforms delivers such a platform—enabling users to

createanapplicationenvironmentforrunningHPCworkloadsandapplicationswithout

theperformancepenaltiesorcomplexitiesofaccessingGPUandnetworkinterconnects.

SingularityPROsimplifiesthedeploymentofapplicationsacrossdifferentclustersand

supercomputers(HPESuperdomeFlex,HPEIntegritySuperdomeX,andHPEIntegrity

MC990 X systems) by avoiding the laborious process of re-hosting the applications for each

distinctenvironment—withoutrequiringavirtualizedhardwarelayer.Singularitycontainers

arejustsinglefiles.Ifyoucanmoveafilefromonehosttoanother,youcandeploya

Singularitycontainer.

The SingularityImageFormat(SIF)isaconduitfortransportingentireapplicationenvironments,

aswellasprovidingusersandadministratorswithameansofprotection.WithSingularity

single-filecontainers,usersbenefitfromextrememobility,enhancedreproducibility,and

compliancecontrol.

SingularityPROandassociatedSingleImageFormat(SIF)containerscanhave

cryptographicallysignedandevolvableoverlaystoenableacontrols-compliantworkflow,

whichcreatestrustedcontainers.Unlikeothercontainerplatforms,SingularityPROhas

a mechanism to validate a runtime image and all data regions through a self-signing

mechanism.Bysigningandverifyingcontainers,distributorsandusersestablishalevelof

trustunavailabletoothercontainerformats.

Figure 2: Singularity Image Format file structure and usage

Immutable RuntimeContainer Image

Glob

al H

eade

r

Reci

pe D

efin

ition

Labe

ls

Envi

ronm

ent

Writable Overlay

Sign

atur

e Bl

ock

CRYPTOGRAPHICALLY SIGNED

EVOLVABLE

Desc

ripto

rs


7

Drop-in replacement for standalone processes

Singularityintegrateswithallbatchresourcemanagers—withzeromodifications—bycalling

theSingularitycommanddirectly.

OneofSingularity’sarchitecturallydefinedfeaturesistheabilitytoexecutecontainersas

iftheywerenativeprogramsorscriptsonahostcomputer.Asaresult,integrationwith

schedulerssuchasUnivaGridEngine,Torque,SLURM,SGE,andmanyothersisassimple

asrunninganyothercommand.Allstandardinput,output,errors,pipes,IPC,andother

communication pathways used by locally running programs are synchronized with the

applicationsrunninglocallywithinthecontainer.

Figure 3: Positioning of Singularity in a Linux system

High-performanceinterconnectssuchasInfiniBandandIntel®Omni-PathArchitecture

(IntelOPA)areprevalentintheHPC/enterpriseperformancecomputing(EPC)space.

Deep-learningworkloads/applicationsalsobenefitfromthehigh-bandwidthandlow-latency

characteristicsoftheseinterconnecttechnologies.

Singularity offers native support for OpenMPI by utilizing a hybrid MPI container approach,

whereOpenMPIexistsbothinsideandoutsidethecontainer.Similartothesupportfor

InfiniBandandIntelOPAdevices,SingularitynativelysupportsanyPCIe-attacheddevice

withinthecomputenode,suchasaccelerators(GPUs).

Appl

icat

ion

VIRTUALIZED

File System

Kernel

Virtualized Hardware

Appl

icat

ion

Appl

icat

ion

NATIVE

Appl

icat

ion

Appl

icat

ion

File System

Host Kernel

Physical Hardware

C

B

A

File System

Appl

icat

ion

Appl

icat

ion

Appl

icat

ion

Appl

icat

ion

Appl

icat

ion

Appl

icat

ion

Container

Host Kernel

Physical Hardware

NATIVE CONTAINED

Virtual Machine Architecture

SingularityArchitecture


8

Architectural differences between Singularity and other container platforms

Security is a common concern for enterprises considering the adoption of containers in

asharedcomputingenvironment.Thisisdueinlargeparttoothercontainerplatforms

requiringelevatedprivilegedaemonsorconfigurationswherethelockingcapabilities

arelimitedandchallengingtoimplement[3].Anotherfundamentaldifferencebetween

Singularityandothercontainersistheimageformatitself.ASingularitycontainerisa

singlefilethatcanbemovedaround,thesameasanyotherfile.Othercontainerruntimes

contain layers, which are assembled during runtime and do not offer the same mobility

andreproducibilityasaSingularitycontainer.

Andfinally,unlikeothercontainerplatforms,Singularityfavorsintegrationoverisolation,

allowingittoworkwithcommonHPCtechnologiessuchashigh-speedinterconnects,batch

schedulers,resourcemanagers,MPIs,andGPUswithlittleornoadditionalconfiguration.

SingularityPRO on HPE infrastructure, ready for the enterprise

SingularityPROisacertifiedbinaryreleaseofSingularitybuiltentirelyfromtheopensource

code base—augmented with the licensing, support, and expert professional services

requestedbyleadingorganizations,universities,andlaboratories.

Unparalleled scale for data-intensive workloads

HPE Integrity MC990 X Server delivers in-memory computing performance for Linux-based applications at an unparalleled scale with mission-critical reliabilityandmodularflexibility.Anadvancedsymmetricmultiprocessing(SMP)systemdesignedfordata-intensiveworkloads,theHPEMC990XServer features enterprise-class Intel Xeon® E7-8800/4800 v4 processors androbustreliability,availability,andserviceability.The5Umodularchassiscontains4socketswithupto192threads.Byaddingchassisandleveraginghigh-bandwidthNUMAlinktechnology,theHPEMC990XServercanscaleasasinglesystemfrom4to32socketsandfrom1to48TBofcache-coherentsharedmemory.

Enabling the data-driven organization

TheHPEApolloFamilyisdesignedtodeliverefficientrack-scalesolutionsfor Big Data, analytics, object storage, and high-performance computing workloads.Withrack-scaleefficiency,theHPEApolloSystemsFamily:

n Deliversjusttherightamountofperformanceandefficiencywithsystemsoptimizedforspecificworkloads

n Accelerates time to value by reducing implementation timen Providesarchitecturalflexibilitywithbothscale-upandscale-outsolutionsn Helps reduce capital and operating expenditures (CAPEX and OPEX)


9

Stronger platform, better support

While many components of an enterprise computing environment (local or cloud) consist

of essential open software components, administration and support of the software need to

comefromsomewhere.Inshort,“free”softwareisnotreallyfree.

Building on the success of Singularity

Community—an open source container

development platform used by over

25,000 top academic, government and

enterprise users, that’s installed on over

3 million cores and running over a million

jobs per day—SingularityPRO includes

numerous enterprise-grade support features:

n Long-term support, where security

patchesandbug-fixesarebackported

intoSingularityPROversions.Thisway,

administrators are released from the

burden of continually updating the

Singularity code base to the latest

open-sourceversion.n Early releases of security patches,

delivered to SingularityPRO customers

before propagation into the source

communityrelease.n Stability, by providing long-term

support, along with bug and security

fixes.n Customized service/support options,

enabling SingularityPRO users to choose

the tiered service/support option that

bestmeetstheirneeds.n Access to a vast ecosystem of resources,

including a container Remote Builder,

Container Library, and Key-signing

service(describedonright).

Figure 4: Subscription provides access to SingularityPRO and a vast ecosystem of ser-vices. Compare features and choose the right version for your organization.

SIF:SingleFileContainerFormat

CryptographicallyVerifiable

NoPersistentGlobalDaemonProcess

Support for Non-root Users

Running Containers

BlockingPrivilegeEscalationwithin a Container

“BringYourOwnEnvironment”Usage Model

SupportforAI/HPCWorkflowsand Architectures

SupportforGPUsNatively

Code Curation

Streamlined Security Updates

SylabsCloudFeatures

SignedPackagesandRepositories

Additional Self-Service Help

Container Build Services

Cryptographic Key Service

Container Library

Sing

ular

ityPR

O

Sing

ular

ity C

omm

unity

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

Features


10

SingularityPRO add-on services

In2018,Sylabsismakingavailablemultiplevalue-addservicesforSingularityPRO.Access

to these services will be available for demonstration purposes to open source Singularity

Communityusers.Theserviceswillalsobeofferedundervarioustiers(trial,SMBplan,and

Enterpriseplan)toSingularityPROcustomers.

Remote Builder

Buildingacontainerrequireselevatedprivilege.InmanyHPCandEPCenvironments,

however,elevatedprivilegesarenotpossiblebecause:

n Regular users cannot have administrative access to any cluster resourcen Usinganexternalworkstationtobuildacontainerbreaksthechainoftrust

The Remote Builder addresses these challenges by moving the build process to a secure,

controlledenvironment.

Duringthebuildprocess,outputstreamsbacktotherequester,sotheusercanmonitor

thebuild’sprogress.Uponcompletion,theSIFimageistransferredbacktotheuser’s

workstation,fromwhichpointitcanbeexecutedwithSingularity,orsenttotheContainer

Library—withnoelevatedadministrativeprivilegesrequired.Inaddition,noworkflow

modificationsarenecessary.Addingasingleflagenablesthebuildtobecompleted

remotelywithoutelevatedprivilege.

The Remote Builder implements appropriate levels of isolation between the components

performingthebuildswithelevatedprivileges,isolatingthemfromasharedcluster.System

administratorsreceiveaturnkeysolutionthatempowersuserstobuildSingularityimages,as

wellasprovidesacentralizedauditingandmonitoringconsoleforSingularitybuilds.These

servicesareavailableinthecloudandon-premises.


11

Container Library

TheContainerLibrarywascreatedanddesignedforhostingSingularityPROcontainers.The

full-featuredLibrarycanbehostedon-premisesinyourdatacenterortheSylabscloud.Users

can upload, download, search, and browse for containers in public and private areas, as well

asshareprivatecontainerswithotherusersorviaageneratedlink.Securityandprivacyinthe

Container Library are based on a user-owner of library objects, and the concept of public or

privatecollections.


Key-signing and verification services

WithSingularity3.0,thenewSingularityImageFormat(SIF)willdelivercontainersigningand

validationservicestoSingularityandtheContainerLibrary.Thesekey-signingandverification

serviceseliminatetheriskofunknowinglydownloadingandrunningcompromisedorrogue

containers.

Theabilitytoquicklyidentifycontainerssignedbytrustedsources—bothinternaland

external—enhances an organization’s auditing capabilities and its ability to enforce policies

forrestrictingthetypesofcontainersallowedtorunonacluster.

Conclusions

Containers promise to seamlessly move applications between environments—from

developmenttoQAtoa10,000-nodecluster.Containersensurethateachapplicationwill

runthesamewayandwillproducethesameresultinanyenvironment—onlyfaster.

SingularityrunningonHPEHPCplatformssimplifiestheprocessofmovingcontainersacross

asingleinfrastructureoracrosshybridenvironments.Thissolutionalsopreservesprivilege

separation to satisfy the security, privacy, and auditing requirements found in all supercomputer

andenterpriseenvironments.

Raising the bar for container platforms, SingularityPRO running on HPE HPC platforms leverages

the power of AI, machine learning, and deep learning to deliver unique enterprise-level

services.SingularityPRO’sadvancedecosystemofresourcesnotonlyextendstheoverall

valueoftheplatformbutalsoextendsitseaseofuseandsecurity.

Thiswhitepaperisforinformationalpurposesonly.SYLABSMAKESNOWARRANTIES,EXPRESSORIMPLIED,INTHISWHITEPAPER.Sylabscannotberesponsibleforerrorsintypographyorphotography.

SingularityPROisatrademarkofSylabsInc.

Othertrademarksandtradenamesmaybeusedinthisdocumenttorefertoeithertheentitiesclaimingthemarksandnamesortheirproducts.Sylabsdisclaimsproprietaryinterestinthemarksandnamesofothers.

©Copyright2018SylabsInc.Allrightsreserved.

Informationinthisdocumentissubjecttochangewithoutnotice.

[1] https://451research.com/images/Marketing/press_releases/Application-container-market-will-reach-2-7bn-in-2020_final_graphic

[2]Eventhoughusershavelimitedaccesstoproductionsystems,theycanhavefulladministrativeaccesstotheirowndevelopmentvirtualmachine.

[3]Dockerdaemonattacksurface,https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface

[email protected]/contactwww.sylabs.io

©2018Sylabs.io.Allrightreserved.

https://451research.com/images/Marketing/press_releases/Application-container-market-will-reach-2-7bn-in-2020_final_graphic

https://451research.com/images/Marketing/press_releases/Application-container-market-will-reach-2-7bn-in-2020_final_graphic

https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface

singularity community and singularitypro on hpe …...singularity community and singularitypro on...

Documents