cern openstack status rackspace analyst event tim bell [email protected] april 2014
TRANSCRIPT
Responsible for IT Infrastructure Group at CERN• Operating Systems
• Clouds
• Security Infrastructure
Member of the OpenStack Management Board and User Committee
Prior to joining CERN• Ran the Deutsche Bank European Private
Banking Infrastructure team
• Consultant and Developer at IBM
2
About Tim
Tim Bell - CERN OpenStack
Tim Bell - CERN OpenStack 3
Tim Bell - CERN OpenStack 4
Tim Bell - CERN OpenStack 5
>100 PB archive
>30 PB/year data to record and analyse
>10,000 servers
>75,000 disk drives
>45,000 tapes
>Next run of LHC expected to double data rates
>But many limits…Tim Bell - CERN OpenStack 6
CERN Geneva Computing Facilities
Production since July 2013• Started with Grizzly version of OpenStack
• Upgrade online to Havana in February 2014
Team of 5 cloud engineers Currently
• ~3,000 hypervisors in two data centres
• ~60,000 cores
• ~3PB disk storage pool for volumes
• Adding around 100 hypervisors per week
Outlook• ~150,000 cores by 2015 with same staff levels
7
Private OpenStack Cloud Status
Tim Bell - CERN OpenStack
CERN data is currently sent to ~150 of sites around the world using the Worldwide LHC Computing Grid
Many sites are working with clouds• Local private clouds
• Bursting to public clouds for peak load
Some opportunistic clouds emerging• 2 clouds of around 20,000 cores in ATLAS and
CMS experimental areas
• Grants for research
8
Using Clouds
Tim Bell - CERN OpenStack
9
CERN Openlab in a nutshell
Tim Bell - CERN OpenStack
• A science – industry partnership to drive R&D and innovation with over a decade of success
• Evaluate state-of-the-art technologies in a challenging environment and improve them
Test in a research environment today what will be used in many business sectors tomorrow
• Train next generation of engineers/employees
• Disseminate results and outreach to new audiences
See http://openlab.web.cern.ch
10
Goals for a year of joint research
Tim Bell - CERN OpenStack
A reference architecture for federation of OpenStack clouds
Blueprints and code contributions to the open source communities
Presentations and white papers to allow others to build on our findings
Rackspace Public Cloud
CERN Private Cloud
Rackspace Private Cloud @
CERN
Identity federation now in the latest open source release• Developed by Openlab fellow
• Will be included in core OpenStack code base for Icehouse
Rackspace cloud testing progressing well• CERN workloads run in Rackspace London public cloud
• Installed a Rackspace private cloud on CERN premises
11
Status – Completed Work
Tim Bell - CERN OpenStack
Enhance federation of resources• Images
• Service Catalog
Demonstrate at the next Openlab summit• Paris in November 2014
Test at scale and deploy
12
Next Steps
Tim Bell - CERN OpenStack
CERN is running large scale OpenStack clouds in production• Small team
• Growing and stable
Openlab provides a framework for sharing challenges with industry• Rackspace collaboration already producing
enhancements to the open source cloud software
Rackspace clouds have been successfully tested using CERN physics workloads• Similar performance and stability with existing
solutions
13
Conclusions
Tim Bell - CERN OpenStack
Q & A
14
Questions
Tim Bell - CERN OpenStack
CERN
15Tim Bell - CERN OpenStack
CERN was founded 1954: 12 European States “Science for Peace”
Today: 21 Member States
Member States: Austria, Belgium, Bulgaria, the Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Israel, Italy, the Netherlands, Norway, Poland, Portugal, Slovakia, Spain, Sweden, Switzerland and the United Kingdom
Candidate for Accession: Romania
Associate Members in Pre-Stage to Membership: Serbia
Applicant States for Membership or Associate Membership:Brazil, Cyprus (awaiting ratification), Pakistan, Russia, Slovenia, Turkey, Ukraine
Observers to Council: India, Japan, Russia, Turkey, United States of America; European Commission and UNESCO
~ 2,300 staff ~ 1,000 other paid personnel > 11,000 users Budget (2013) ~1,000 MCHF
Constraints• Staff levels fixed
• Budget flat
• Geneva computer centre at its limits
• Infrastructure tools were brittle and fragile
Approach• Expand to a second data centre in Hungary
• 200 Gbit/s network from Geneva to Budapest
• Adopt common open source tools
• Move to a cloud model
• Project started in 2012
16
Move To The Clouds
Tim Bell - CERN OpenStack
SAML2 specification• http://
docs.oasis-open.org/security/saml/v2.0/saml-core-2.0-os.pdf
Shibboleth• http://shibboleth.net/
Extenal Identity Providers blueprint• https://blueprints.launchpad.net/keystone/+
spec/identity-providers
Identity Providers and Mapping rules CRUD specification• https://review.openstack.org/#/c/59848/
17
If you are interested...
Tim Bell - CERN OpenStack