cloudstack-development-story
Post on 19-Oct-2014
8.943 views
DESCRIPTION
Citrix Cloud Platform Group CTO, Sheng Liang expalined the CloudStack Development Story at Hokkaido UnivTRANSCRIPT
![Page 1: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/1.jpg)
The CloudStack development story and future vision
Sheng Liang, CTO Cloud Platforms, Citrix Systems
August 29, 2012
![Page 2: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/2.jpg)
![Page 3: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/3.jpg)
AWS is setting the standard…as measured by capacity…
Every day through 2011, AWS added the same amount of server processing capacity, on average, that it took to run the Amazon online retailing operation in 2000, when it was a $2.76bn company.
Prickett-Morgan. “AWS Cloud Double Fluffs in 2011.” The Register, 6 Jan 2012.
Q4 2006 Q4 2007 Q4 2008 Q4 2009 Q4 2010 Q4 2011
2.9B 14B 40B102B
262B
762BTotal Number of Objects
Stored in Amazon S3
Peak Requests:
500,000+ per second
Source: UBS
![Page 4: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/4.jpg)
…data center footprint and geographic distribution…
…the company said that with the opening of its AWS data center in São Paulo, Brazil in mid-December, the company has doubled its AWS data-center footprint.
Prickett-Morgan. “AWS Cloud Double Fluffs in 2011.” The Register, 6 Jan 2012.
AWS RegionsAmazon Edge Locations (CloudFront & Route 53)
![Page 5: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/5.jpg)
2006e 2007e 2008e 2009e 2010e 2011e$0
$200
$400
$600
$800
$1,000
$1,200
$1,400
All Other
…and, most importantly, revenue…
It has been estimated that AWS could be a $1 billion business for the online retailer come next year…could hit $2.5B in 2014.
Hickey, Andrew. “Amazon Q3 Cloud Revenue Skyrockets” CRN. 26 Oct 2011.
Amazon Web Services Revenue Model$M
Source: UBS
![Page 6: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/6.jpg)
How did Amazon build its Cloud?
Servers StorageNetworking
Open Source Xen Hypervisor
Amazon Proprietary Orchestration Software
AWS API (EC2, S3, …)
Amazon eCommerce Platform
![Page 7: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/7.jpg)
How can we build a cloud using CloudStack?
“Cloud OS or Data Center OS”
Servers StorageNetworking
Open Source Xen Hypervisor
Amazon Proprietary Orchestration Software
AWS API (EC2, S3, …)
Amazon eCommerce Platform
XenServer
Apache CloudStack
User Portal
Cloud API
VMware KVM OVM Hyper-V Bare-Metal
![Page 8: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/8.jpg)
The Virtual Datacenter OS allows businesses to efficiently pool all types of hardware resources - servers, storage and network – into an aggregated on-premise cloud
VMware press release Sept 15, 2008
![Page 9: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/9.jpg)
Eucalyptus is the only cloud architecture to support the same application programming interfaces (APIs) as public clouds, and today Eucalyptus is fully compatible with the Amazon AWS public cloud infrastructure.
Eucalyptus Systems Press Release April 2009
![Page 10: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/10.jpg)
OpenNebula
Zimory
Cassatt
3Tera
Enomaly
Joyent
Nimbula
AbiquoYunteq
OnAppEucalyptus VMware
![Page 11: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/11.jpg)
Cloud.com June 2009
![Page 12: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/12.jpg)
Prototype 1.0 GA2.0
RefactorAWS
Compatibility2.2
Refactor3.0 Quality
Improvements
2008
Sept 2008: VMOpsFounded
2009
Nov 2009: CloudStack 1.0 GA
2010
May 2010: Cloud.comLaunch &CloudStack 2.0 GA
2011
July 2011: Citrix Acquires Cloud.com
2012
April 2012: Apache CloudStack
4.XRefactor
![Page 13: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/13.jpg)
• Initial target: hosting companies like Rackspace and Savvis
• 3 engineers built a fully functional prototype in 5 months
• Use the demo to sell to early customers (ReliaCloud, CloudCentral, 1800hosting.com, Go Daddy, etc.)
Prototype 1.0 GA2.0
RefactorAWS
Compatibility2.2
Refactor3.0 Quality
Improvements4.X
Refactor
2008 2009 2010 2011 2012
![Page 14: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/14.jpg)
• Took 6 more months to make 1.0 software production ready
• Deployed on 5 production customers
2.0 Refactor
AWS Compatibility
2.2 Refactor
3.0 Quality Improvements
4.XRefactor
2008 2009 2010 2011 2012
Prototype 1.0 GA
![Page 15: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/15.jpg)
• Product first, architecture second
• From web hosting to enterprise workload
• Multi-hypervisor, SAN, and VLAN support
• Learn needs of enterprise workload from: Tata Communications, Korea Telecom, Macquarie Telecom
• Competition: vCloud Express
2.0 Refactor
AWS Compatibility
2.2 Refactor
3.0 Quality Improvements
4.XRefactor
2008 2009 2010 2011 2012
Prototype 1.0 GA
![Page 16: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/16.jpg)
• Private cloud demand picked up
• Zynga wanted private cloud
• Support Amazon-style flat networking and security groups
• Competition: Eucalyptus
2.0 Refactor
AWS Compatibility
2.2 Refactor
3.0 Quality Improvements
4.XRefactor
2008 2009 2010 2011 2012
Prototype 1.0 GA
![Page 17: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/17.jpg)
• Second major refactoring of CloudStack code
• Network-as-a-service combing both Amazon and traditional style networking
• More flexible orchestration engine
2.0 Refactor
2.2 Refactor
3.0 Quality Improvements
4.XRefactor
2008 2009 2010 2011 2012
Prototype 1.0 GAAWS
Compatibility
![Page 18: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/18.jpg)
• Citrix acquisition
• Rapid growth of CloudStack user base
• Quality is more important than new features
2.0 Refactor
3.0 Quality Improvements
4.XRefactor
2008 2009 2010 2011 2012
Prototype 1.0 GAAWS
Compatibility2.2
Refactor
![Page 19: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/19.jpg)
• Third major refactoring of CloudStack code
• Apache contribution drive rapid growth of CloudStack developer base
• Apache license compliance
• Services framework
• Hadoop integration
2.0 Refactor
4.XRefactor
2008 2009 2010 2011 2012
Prototype 1.0 GAAWS
Compatibility2.2
Refactor3.0 Quality
Improvements
![Page 20: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/20.jpg)
• Optimize Hadoop on cloud infrastructure• Use HDFS as object store
+
![Page 21: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/21.jpg)
How is cloud different from legacy infrastructure?
![Page 22: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/22.jpg)
How to handle failures
![Page 23: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/23.jpg)
23#CitrixSynergy
8%Kashi Venkatesh Vishwanath and Nachiappan Nagappan, Characterizing Cloud Computing Hardware Reliability, SoCC’10
Annual Failure Rate of servers
• Server failure comes from:ᵒ 70% - hard diskᵒ 6% - RAID controllerᵒ 5% - memoryᵒ 18% - other factors
• Application can still fail for other reasons:ᵒ Network failureᵒ Software bugsᵒ Human admin error
![Page 24: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/24.jpg)
Internet
Core Routers
Access Routers
Aggregation Switches
Load Balancers
Top of Rack Switches
…
…
Servers
![Page 25: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/25.jpg)
25#CitrixSynergy
40%Phillipa Gill, Navendu Jain & Nachiappan Nagappan, Understanding Network Failures in Data Centers: Measurement, Analysis and Implications, SIGCOMM 2011
Effectiveness of network redundancy in reducing failures
•Bugs in failover mechanism
• Incorrect configuration•Protocol issues such as TCP back-off, timeouts, and spanning tree reconfiguration
![Page 26: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/26.jpg)
Cloud workloads
Traditional-Style
Reliable hardware, backup entire cloud, and restore for users when failure happens
Amazon-Style
Tell users to expect failure. Users to build apps that can withstand infrastructure failure
Link Aggregation
Storage Multi-pathing
VM HA, Fault Tolerance
VM Live Migration
VM Backup/Snapshots
Multi-site Redundancy
Chaos Monkey
Ephemeral Resources
![Page 27: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/27.jpg)
Designing a zone for a traditional workload
vCenter/XenCenter
Hypervisor Cluster
Hypervisor Cluster
Hypervisor Cluster
Enterprise Networking (e.g., VLAN)
Enterprise Storage (e.g., SAN)
Hypervisor
Storage
SAN
Networking
L2 VLANs
Network Services
Load Balancing VPN
Multi-tier Apps
Multi-tier VLANs OVF
vSphere or XenServer EnterpriseTraditional-Style Availability Zone
![Page 28: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/28.jpg)
Designing a zone for an Amazon-style workload
Hypervisor
Storage
Local EBS
Networking
L3 SDN based L2 Elastic IP
Network Services
Security Groups ELB
Multi-tier Apps
3rd Party Tools (e.g., RightScale, enStratus)
XenServer Advanced
Object store
GSLB
CloudFormation
Software Defined Networks (e.g., Security Groups, EIP, ELB,...)
Amazon-Style Availability Zone
Server Racks
Server Racks
Server Racks
Server Racks
Server Racks
Server Racks
Server Racks
Server Racks
Server Racks
Server Racks
Server Racks
Server Racks
Elastic Block Storage
![Page 29: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/29.jpg)
AWS-style Availability
Zone
AWS-style Availability
Zone
AWS-style Availability
Zone
CloudStack can Support Both Styles
Traditional Style
Availability Zone
Apache CloudStack
Traditional Style
Availability Zone
![Page 30: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/30.jpg)
CloudStack Future
![Page 31: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/31.jpg)
146 Companies 238 Developers
100’s of Production Clouds
32,000 Community Members
Global User Groups
Service Providers
Enterprises Universities
![Page 32: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/32.jpg)
Apache CloudStack community projects
• SDNᵒ Niciraᵒ Midokuraᵒ Big Switch Networksᵒ Stratosphere
• Backup/DRᵒ Sungard
• Networkingᵒ Cisco (VXLAN, Nexus)ᵒ Brocade (ADX)
• Smart Storageᵒ Hadoop + S3 API for object storeᵒ NetApp (FlexPod, object store)ᵒ Basho RIAK CSᵒ Caringo object storeᵒ Cloudian S3
• PaaSᵒ CloudFoundry implementation through
IronFoundry and Stackato teamsᵒ Engine Yardᵒ Cumulogicᵒ GigaSpaces
![Page 33: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/33.jpg)
“The Apache Way”
• Collaborative software development
• Commercial-friendly standard license
• Consistently high quality software
• Respectful, honest, technical-based interaction
• Faithful implementation of standards
• Security as a mandatory feature
![Page 34: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/34.jpg)
…
Innovative Cloud Applications and Services
Servers StorageNetworking
Innovative Cloud Infrastructure
![Page 35: CloudStack-Development-Story](https://reader034.vdocument.in/reader034/viewer/2022051608/54441baab1af9f640a8b47a8/html5/thumbnails/35.jpg)
More information: http://cloudstack.orghttp://cloudstack.jp