community clouds from scratch
DESCRIPTION
Community Clouds from Scratch Cloud Computing represents a radical change in the way we organize and use computing resources and storage. The scientific and academic communities face the challenge of not only adapting their procedures to this new paradigm, but also contributing Cloud Computing development and leading its evolution towards open, secure and interoperable computing infrastructures, which will playing a key role in the community clouds paradigm. The Spanish MEGHA initiative promotes and coordinates contributions to cloud computing R&D, education and management made by institutions affiliated with RedIRIS [7] in Spain. In the first phase (2010–2012), MEGHA validated federated cloud platforms using Opennebula and OCCI [10] to streamline the use of cloud technologies among R&E service centers. Representative infrastructure providers (CESCA, CESGA, PIC), middleware providers (OpenNebula, RedIRIS, OSAmI-Commons) and users (UAB, UOC, UM) together with intermediate/identity/brokers resources (RedIRIS) joined efforts to demonstrate the viability of this approach. The results stimulated the development of use cases including e-learning platforms on demand (Learning Apps project), a distributed HPC platform (e-Science), and Virtual Labs (VDI) in a hybrid scenario (Academic services). Next Steps? As next goal, the Spanish research and academic community is working to assess the possibilities of creating a productive Infrastructure Cloud Computing service within member institutions. With this new approach new challenges appear: Federated user authentication and authorization mechanisms. Brokering architecture scenario. Secure VM image distribution and validation. A federated cloud accounting system integrating the accounting records of multiple cloud managers and supporting federated cloud governance. Monitoring and notification of unpredictable changes in availability and readability status. Security Policies and Service Level Agreements (SLA’s).TRANSCRIPT
![Page 1: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/1.jpg)
![Page 2: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/2.jpg)
Community Community Clouds from Scratch Clouds from Scratch A Real World Introduction
[email protected] – Berlin 2013 OpenNebula Conf 2013
Jordi Guijarro - [email protected]@cloudadms
![Page 3: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/3.jpg)
![Page 4: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/4.jpg)
About
![Page 5: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/5.jpg)
What Cloud What Cloud Computing really Computing really means for R&E means for R&E A strategy, a tactical solution...
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 6: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/6.jpg)
A Research and Education Network is a non-profit Internet service provider dedicated to supporting the needs of the research and education communities.
It has a high-speed backbone network.
Research and Education Networks are usually the places where new Internet protocols and architectures are introduced before their public deployment:
R&E Networks
Protocols: IPv6 and multicast.
Architecture: Client/Server and Cloud Computing
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 8: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/8.jpg)
JUST A BUZZWORD?What Cloud Computing really means for R&E Community ?
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 9: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/9.jpg)
What Cloud Computing really means
SELFSERVICE(s)?
jguijarro@cescajguijarro@cesca,.cat – Reykjavik 2012,.cat – Reykjavik 2012 TNC 2012TNC [email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 10: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/10.jpg)
What Cloud Computing really mean for R&E Community?
RESPONSIBILITIES...
jguijarro@cescajguijarro@cesca,.cat – Reykjavik 2012,.cat – Reykjavik 2012 TNC 2012TNC [email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 11: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/11.jpg)
MEGHA Working MEGHA Working
GroupGroup Let's pull all together...
jguijarro@cescajguijarro@cesca,.cat – Reykjavik 2012,.cat – Reykjavik 2012 TNC 2012TNC [email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 12: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/12.jpg)
MISSIONMEGHA Working Group
the development of innovation in Cloud Computing technologies within the scope of the institutions affiliated to RedIRIS
for the application of open and interoperable Cloud technologies in other areas in which it can influence, such as business in government.
TO PROMOTE
TO ACT AS A CATALYST
INTEROPERABLE
OPEN
PROMOTION
USE CASES
TESTBED
IDENTITY
http://wiki.rediris.es/megha
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 13: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/13.jpg)
Deploy a federated Cloud infrastructure as a platform for innovation and evaluation.
Explore the possibilities for the application of Cloud computing has in research, academic and management environments.Facilitate the creation of an ecosystem of open-source middleware components to deploy interoperable Clouds and coordinate training and education activities for its deployment.
MAIN OBJECTIVESMEGHA Working Group
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 14: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/14.jpg)
MEGHA IntercloudMEGHA Intercloud
projectproject From myth to reality...
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 16: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/16.jpg)
CESCA Cloud InfrastructureCESCA Cloud Infrastructure
![Page 17: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/17.jpg)
CESGA Cloud InfrastructureCESGA Cloud Infrastructure
![Page 18: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/18.jpg)
INTEROPERABILITYMEGHA Intercloud Project
A “Virtual Poka Yoke” ?A “Virtual Poka Yoke” ?
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 19: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/19.jpg)
OPEN PLEASE! MEGHA Intercloud Project
Director,...
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 22: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/22.jpg)
FEDERATED MEGHA Intercloud Project
PKIRISGRIDX.509 Cert
UI
OpenN
ebula
3.8
r
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 23: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/23.jpg)
Public high speed connectivity.
Images, instances and networks management.
SSH access (root).User manual & Service Provider Howtos (ONE 3.x)“Terms of Service”.
MATURING TOGETHER
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 25: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/25.jpg)
Some Real World Some Real World
User CasesUser Cases Cloudified...
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 26: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/26.jpg)
UOC: Learning Apps
REAL WORLD USER CASES
A store on the internet inspired by Apple's App Store which focuses on educational applications
The provision of the educational environments is automated (OCCI 0.8) and the amount of infr. depends on parameters such as a preconfigured concurrency.
RESTFUL API
http://www.learningappsstore.org/
ON DEMAND
ACADEMICACADEMIC
EDUCATION
AUTOMATION
CONTEXTUALIZATION
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 27: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/27.jpg)
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 28: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/28.jpg)
PIC: Grid & CloudREAL WORLD USER CASES
DIRAC (Interware) allows homogeneous access to heterogeneous computing resources.
Providing transparent access and interoperability between Grids and Clouds OCCI 1.1 interface.
SCIENTIFICINTEGRATION
CONTEXTUALIZATION
DIRACINTERWARE
Víctor Méndez (www.pic.es)
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 29: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/29.jpg)
PIC: Grid & Cloud
REAL WORLD USER CASES
DIRAC WMS
TaskQueue
VM Scheduler
VM Director
MeghaCloudInfrastructure
Command-line
DIRAC API
Web Portal
VirtualMachine
UserAdmin
Image Repo
Cloud Manager
DIRAC CS
Cloud Driver
Integration Design: VM Job Running
Víctor Méndez (www.pic.es)[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 30: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/30.jpg)
PIC: Grid & CloudREAL WORLD USER CASES
Víctor Méndez (www.pic.es)
DIRAC WMS
TaskQueue
VM Scheduler
OCCI Director(VM Director)
MeghaCloudInfrastructure
VirtualMachine
Image Repo
OCCI / OpenNebula
?
OCCI Image
OCCI RESTful Client
Integration Design: Occi VM Submission
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 31: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/31.jpg)
PIC: Grid & Cloud
REAL WORLD USER CASES
jguijarro@cescajguijarro@cesca,.cat – Reykjavik 2012,.cat – Reykjavik 2012 TNC 2012TNC 2012
Virtual Machine
VM Monitor
Job Agent
DIRAC WMS
TaskQueue
VM Manager
Job
Cloud Manager
VirtualMachine
MeghaCloud Infrastructure
VMVM
VM
Job Run
CPU statistics
Job State Update
Job Matcher
Víctor Méndez (www.pic.es)
Integration Design: VM Job Running
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 32: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/32.jpg)
Next Steps... Next Steps... This is only the tip of the iceberg
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 33: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/33.jpg)
DETECTED PROBLEMSNEXT STEPS...
[email protected] – Berlin 2013 OpenNebula Conf 2013
Small and medium Scientific Research centersdon ́t have the manpower or know-how tobuild a 24/365 data center
Server deployment takes a long time
Can t́ solve peak problems
Usually can ́t have more than one data center
![Page 34: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/34.jpg)
TEST PLATFORMNEXT STEPS FOR...
VO's ? SUNSTONE FEDERATION ?
COMPATIBILITY AND FEATURES
TO BUILD A MARKETPLACE ?
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 36: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/36.jpg)
Some Some
conclusions... conclusions... Options to cater to all tastes!
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 37: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/37.jpg)
NEW CHALLENGESNEXT STEPS...
[email protected] – Berlin 2013 OpenNebula Conf 2013
Federated user authentication and authorization mechanisms and user management between different cloud managers.
Secure VM image distribution and validation.among heterogeneous cloud managers.
Federated Cloud accounting system integrating the accounting records of multiple cloud managers and supporting the Federated Cloud governance.
Monitoring and notification of unpredictable change of availability.
![Page 38: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/38.jpg)
Distributed scenario
FUTURE COMMUNITY FEDERATED CLOUD MODELFUTURE COMMUNITY FEDERATED CLOUD MODEL
Scalability (Commercial prov. included)
Resource sharingHigh Availability
Efficient use of infrastructure
[email protected] – Berlin 2013 OpenNebula Conf 2013
![Page 39: Community clouds from scratch](https://reader033.vdocument.in/reader033/viewer/2022060108/55510c0bb4c9057b478b5614/html5/thumbnails/39.jpg)
Strategy, strategy, strategy, ...
Dedicated to those who made MEGHA possible.
Gràcies! Gràcies! Gracias! Gracias! ThanksThanks! !
Jordi Guijarro [email protected]
@cloudadms