deep-hybrid-datacloud · deep-hybriddatacloud 10/22/18 3/22 deep project objectives focus on...

DEEP-HybridDataCloud has received funding from the European Union’s Horizon 2020research and innovation programme under grant agreement No 777435.

DEEP-Hybrid-DataCloud

Álvaro López Garcí[email protected]

Spanish National Research Council

Calipso+ JRA2 blueprint hands onALBA SyncrhotronOctober 22, 2018

DEEP-HybridDataCloud 10/22/18 2/22

DEEP-Hybrid-DataCloud: context● H2020 project, EINFRA-21 call

– Topic: Platform-driven e-Infrastructure towards the European Open Science Cloud

– Scope: Computing e-infrastructure with extreme large datasets ● (…) prototypes to cope with very large data resources (...)● (…) software layers supporting applications such as modelling, simulation, pattern recognition, visualisation, etc. (...)● (…) supported by robust mathematical methods and tools (...)● (…) Clean slate approaches to high-performance computing and data management (…)

● DEEP-Hybrid-DataCloud: Designing and Enabling E-Infrastructures for intensive data Processing in a Hybrid DataCloud– Started as a spin-off project (together with eXtreme-DataCloud) from INDIGO-DataCloud technologies

– Submitted in March 2017

– Started November 1st 2017

– Grant agreement number 777435

● Global objective: Promote the use of intensive computing services by different research communities and areas, an the support by the corresponding e-Infrastructure providers and open source projects


DEEP project objectives● Focus on intensive computing techniques for the analysis of very large datasets considering demanding use cases

– Pilot applications from different research communities

– Three techniques of wide interest: deep learning, post processing and on-line analysis of data streams

– Improved list of requirements for e-Infrastructures → future generation

● Evolve up to production level intensive computing services exploiting specialized hardware– New solutions to better interact with bare metal resources in the cloud

– Use of hardware accelerators such as GPUs and low-latency interconnects

● Integrate intensive computing services under a hybrid cloud approach– Assuring interoperability with existing EOSC

– Expanding over multiple IaaS using high level networking technologies

– Enrich orchestration tools for supporting multiple services and providers

● Define a “DEEP as a Service” solution to offer an adequate integration path to developers of final applications– Implement a catalog of the most useful services and applications as well defined building blocks

– Offer a DevOps approach for the application development

● Analyse the complementarity with other ongoing projects targeting added value services for the cloud– In particular those related to the management of extremely large datasets

– Explore different e-Infrastructures and complementary services

– Identification, integration and/or co-development of missing functionalities


DEEP consortium● Balanced set of partners

– Strong technological background on development, implementation, deployment and operation of federated e-Infrastructures

● 9 academic partners– CSIC, LIP, INFN, PSNC, KIT, UPV, CESNET, IISAS, HMGU

● 1 industrial partner– Atos

● 6 countries– Spain, Italy, Poland, Germany, Czech Republic, Slovakia


DEEP pilot use cases● Three techniques of wide interest, involving

– Large, heterogeneous data sets

– Intensive computing demands that would benefit from using hardware accelerators (GPUs, low latency interconnects)

Deep learning applications● Pilot cases: stem cells, biodiversity applications,

medical images● Objective: Provide a general, distributed architecture

and pipeline to train, retrain and use deep learning (and other machine learning) models


DEEP pilot use cases

Massive Online Data Streams: Online analysis of data streams

● Pilot case: intrusion detection systems

● Provide an architecture able to analyze massiveon-line data streams, also with historical records

Post-processing● Pilot cases: post-processing of HPC simulations● Flexible pipeline for the analysis of simulation data

generated at HPC resources


Our vision

● We need to build added value and advanced services on top of bare IaaS and PaaS infrastructures

● Ease and lower the entry barrier for non-skilled scientists– Transparent execution on e-Infrastructures

– Build ready to use modules and offer them through a catalog or marketplace

– Implement common software development techniques also for scientist’s applications (DevOps)

● Build and promote the use of intensive computing services by different research communities and areas, an the support by the corresponding e-Infrastructure providers and open source projects


Who is the user?

dom

ain

expe

rtis

e

machine learning expertise

technological expertise

level of knowledge beingrequired depends on thespecific use case and the user profile


Deep learning use cases

● Category 1: Deploy a readily trained network for somebody else to use it on his/her data set

● Domain knowledge

● Category 2: Retrain (parts of) a trained network to make use of its inherent knowledge and to solve a new learning task

● Domain + machine learning knowledge

● Category 3: Completely work through the deep learning cycle with data selection, model architecture, training and testing

● Domain + machine + technological knowledge


Previously...● Scientists create a deep learning

application on their personal computers

● The deep learning model is trained in a GPU node (maybe also locally)– What happens if they do not have

access to one?

● The work is published (or not)– Model architecture, configuration,

scientific publication, etc.

● But:– How can a scientist easily offer it to a

broader audience?

– What about dependencies?


Offering developed models as a service

● Development of APIs and web applications

● Scientists need to know what an API is– REST, GET, POST, PUT...

● Lack of API consistency → hard for external developers to consume them

● Provide users with a generic API (OpenAPI) component where they application can be plugged


Service composition

● An application may consist on several components that need to be deployed, configured, etc → service composition

● Service composition, if done properly, provide a way to re-deploy the same topology over different infrastructures → catalog of components

● Scientists should not need to deal with technologies and infrastructures they do not care at all

● We need therefore different roles, to perform different tasks– Comparison with laboratory technician

● In INDIGO-DataCloud we started with this approach, but this needs to be generalized (and the roles recognized)

DEEP-HybridDataCloud 15/22

Different roles for different tasks

Resource Provider

Enabler

Solver

ResourceProvider

User

IaaS

PaaS

e-Infrastructures (Federations)

SaaS

AAI

Site A Site BVM

Docker (utils)

Docker (app)

ComputeCompute Network

Orchestrator

Storage

Repositories

Access Portal

UtilsApp

APIs

Output

User Input

1:N


DEEP Open Catalog● Collection of ready-to-use modules

– Comprising machine learning, deep learning, big data analytics tools

– ML Marketplace

https://marketplace.deep-hybrid-datacloud.eu

– GitHub

https://github.com/indigo-dc?utf8= &q=DEEP-OC✓&q=DEEP-OC

– DockerHub

https://hub.docker.com/u/deephdc/

● Based on DEEPaaS API component– Expose underlying model functionality with a common API

– Based on OpenAPI specifications

– Minimal modifications to user applications.

● Goal: execute the same module on any infrastructure:


DEEP high level Architecture


DEEP plan and statusM

1 N

ov 2

01

7 M8 Jun 2018M4 Feb 2018 M12 Oct 2018

M14 Feb 2019M18 Jun 2019

M24 Oct 2019M28 Feb 2020

M30

Apr

20

20

Projectstart

KoM Bologna

DI4R2nd release

1st release

Pilottestbed

1st review

EOSC-Hubdays

Plan and requirements

Design

First Prototype

Design Full pilot testbed

(at least) Two pilot applications for the 1st prototype

Santander course

EOSCdays

First Prototype EOSC exploitationM

ilest

on

e: b

atch

-lik

e su

pp

ort

Mile

sto

ne:

ap

p a

s a

serv

ice

https://github.com/indigo-dc?utf8

https://hub.docker.com/u/deephdc/


DEEP 1st release: componentsSoftware component Functionalities

PaaS Orchestrator● Hybrid deployments on multiple sites● Support to specifying specialized computing hardware● Improved support for deployment failures

Infrastructure Manager (IM)● Improved support for hybrid deployments● Support for additional TOSCA types

DEEPaaS API

● Support for training a machine learning application● Support for performing inferences/analisys/predictions● Support only for synchronous requests● OpenID Connect support● Support for standalone service & OpenWhisk action

Alien4Cloud● Suppor for visual composition of TOSCA templates● PaaS orchestrator support

Virtual Router● Improvements to reach production level● Virtualized routing over distributed infrastructures

cloud-info-provider ● Support for GPU and Infibinand resources

uDocker ● Improved support for GPUs and Infiniband


DEEP 1st release: servicesService Functionalities Preview endpoint

Visual application topology composition and deployment

● Graphical composition of complex application topologies

● Deployment through PaaS orchestratorhttps://a4c.ncg.ingrid.pt

ML/DL training facility as a service

● Provide continuous training and retraining of developed models

https://a4c.ncg.ingrid.pt

DEEP as a Service● Deployment of DEEP Open Catalog

components as server-less functionshttps://vm028.pub.cloud.ifca.es/

DEEP Open Catalog

● Ready-to-use machine learning and deep learning applications, including:

➢ Machine learning frameworks + JupyterLab➢ ML/DL ready to use models➢ BigData analytic tools

https://marketplace.deep-hybrid-datacloud.eu

● All services are OIDC-ready, following AARC/AARC2 blueprint recommendations

● Also work on:– TOSCA templates and TOSCA types

– Documentation and configuration recipes for GPU supoort

– Patches to upstream projects (Apache Libcloud, Apache OpenWhisk, OpenStack)


DEEP demos● Scientists prepares their application

– Jupyter Notebook with GPU

– https://www.youtube.com/watch?v=Xy9KJuakd5o

● Scientists trains the model– Execution through the orchestrator (video with too low level details, don’t be scared)

– https://www.youtube.com/watch?v=5x90Qjytj8Y

● Scientists (or anybody) uses the model– Directly on their laptop

– Directly on top of a IaaS or PaaS component

– On an HPC or supercomputer node (through uDocker)

– Using DEEPaaS

– https://www.youtube.com/watch?v=aM9-VMgrYKY


Conclusions and remarks

● Different roles for different are needed and should be recognized– Scientists should not focus on the technology (unless they want to do so)

● We provide added value services that span various sites (hybrid) to scientists to ease access to computing power without technology knowledge

● We develop catalogs or marketplaces with ready-to-use solutions– Ready to use applications, ready to use APIs, ready to use clusters

● We provide ways to re-deploy a given topology on a different site at zero-cost (to an extent)– Possibility to redeploy the whole stack and architecture used for a research

https://a4c.ncg.ingrid.pt/

https://a4c.ncg.ingrid.pt/

https://vm028.pub.cloud.ifca.es/

https://marketplace.deep-hybrid-datacloud.eu/


Contacts

Web page:

https://deep-hybrid-datacloud.eu

Email:

[email protected]

https://twitter.com/DEEP_eu

https://www.youtube.com/watch?v=Xy9KJuakd5o

https://www.youtube.com/watch?v=5x90Qjytj8Y

https://www.youtube.com/watch?v=aM9-VMgrYKY

DEEP-HybridDataCloud

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 777435.

Thank youAny Questions?

deep-hybrid-datacloud · deep-hybriddatacloud 10/22/18 3/22 deep project objectives focus on...

Documents