Apache CloudStack

Sebastien Goasguen @sebgoaBuild a Cloud Day, Feb 1st 2013

Ghent, Belgium

Big Thanks !

SaaS

PaaS…

IaaS…

IaaS challenges

• Not out of the box by itself• Need a farm of hypervisors

– Xen, KVM, VMware…

• Need storage– For image catalogue– For volume/snapshot management

• Need flexible network that can be configured on-demand– VLANS, no VLANS, existing net infra, SDN…

IaaS is really:

• A Data Center Orchestrator– Data storage– Data movement– Data processing

• That can:– Handle failures– Support large scale– Be programmed

DevOps

• Bring your developers closer to the operations• Make your sys admins develop the apps• Move to Software Defined Data Center,

Automate everythingDev:

JavaIDE

Ops: Shell CLI

What is CloudStack ?• Open source Infrastructure as a Service (IaaS)

solution.• “Programmable” Data Center orchestrator• Hypervisor agnostic (with addition of bare metal

provisioning)• Support scalable storage (Ceph, RIAK CS…)• Support complex enterprise networking (e.g

Firewall, load balancer, VPN, VPC…)• Multi-tenant

ASF

Apache Way

• “Transparency, consensus, non-affiliation,respect for fellow developers, and meritocracy, in no specific order “

Apache Process

• 100% community driven• New ideas, decisions only taken on mailing lists.

Votes taken by community• Project led by Project Management Committee

(PMC):– http://www.apache.org/dev/pmc.html– http://incubator.apache.org/guides/committer.html

• Non committers get invited as committers:– http://community.apache.org/newcommitter.html

Apache Processes

What the ASF did to CloudStack ?

ContributionsExamples:

Sungard: Announced that 6 developers were joining the Apache projectSchuberg Philis: Big contribution in building/packaging and Nicira supportPCextreme: RBD/Ceph supportCaringo: Support for own object storeBasho: Devcloud packaging, S3 integration

• “A little bit about the internals”

A Very Flexible IaaS Platform

Compute Hypervisor

Storage Block & Object

Network Network & Network Services

Primary Storage Secondary Storage

http://www.slideshare.net/cloudstack/cloudstack-architecture

Architecture / Language

• Java application• Tomcat6, Axis2, Maven build + ant

– Ant going away in 4.1

• Moving towards a plugin architecture

Releases

• Apache CloudStack 4.0 released in November

• Time based releases– Rather slip features but guarantee release

• Apache CloudStack 4.1 planned for March 22nd. Feature Freeze Yesterday...

• 4.2 most likely in July

Pod 1

….

Cluster N

Access LayerAccess Layer

Host 2Host 2

Cluster 1

CloudStack Cloud Architecture

Host 1Host 1

Hypervisor is the basic unit of scale.

Cluster consists of one ore more hosts of same hypervisor

All hosts in cluster have access to shared (primary) storage

Pod is one or more clusters, usually with L2 switches.

Availability Zone has one or more pods, has access to secondary storage.

One or more zones represent cloud

PrimaryStorage

Zone 1

….

L3 coreL3 core

SecondaryStorage

Pod N

CloudStack Management

Server

InternetInternet

Cloud Interactions

CloudStackCloudStack

Cloud user{API client (Fog/etc)}

End User UI

End User UI

Admin UI

Admin UI

MySQLMySQL

CloudStackCloudStackClustered

CloudStackManagement

Server

ClusteredCloudStack

ManagementServer

Domain Admin

UI

Domain Admin

UI

CS Admin & End-user API

Cloud user{ec2 API client }

ec2 API

Monitoring CS API vSphere ClusterPrimaryStorage

vcentervcenter

Cluster Mgmt

XS ClusterPrimaryStorage

vCenter API

XAPI

KVM ClusterPrimaryStorageJSON

OVM Cluster PrimaryStorage

XenApi

NetConf

Nitro APIJuniper SRX

Netscaler

Console Proxy VMConsole

Proxy VMConsole Proxy VMConsole

Proxy VM

JSON

Cloud user

HTTPSAjax Console

Ajax Console

VNC

Sec. StorageVM

Sec. StorageVM

NFS Server

NFSSec. Storage

VMSec. Storage

VM

HTTP (Template Download)

HTTP (Template Copy)

HTTP (Swift)

NFS

Router VMRouter VMRouter VMRouter VM

Router VMRouter VM

JSON

{Proxied} SSH

http://www.slideshare.net/cloudstack/cloudstack-architecture

Networking

• Extremely flexible to:– Provide isolation with VLANs– Provide isolation at L3 with shared L2 (scalability)– Support hardware devices that exposes API– Deployed on existing networking infrastructure– Support new networking paradigm (SDN)

• Support for Nicira Virtual P• Extensive use of Open VSwitch

Layer-2 Guest Virtual Network

Public Network/Internet

Guest Virtual Network 10.1.1.1/8VLAN 100

Gateway address 10.1.1.1

DHCP, DNSNATLoad BalancingVPN

Public IP 65.37.141.11

10.1.1.1Guest VM 1Guest VM 1

10.1.1.3Guest VM 2Guest VM 2

10.1.1.4Guest VM 3Guest VM 3

10.1.1.5Guest VM 4Guest VM 4

CSVirtual Router

CSVirtual Router

Public Network/Internet

Guest Virtual Network 10.1.1.1/8VLAN 100

Private IP10.1.1.112

DHCP, DNS

Public IP 65.37.141.112

10.1.1.1Guest VM 1Guest VM 1

10.1.1.3Guest VM 2Guest VM 2

10.1.1.4Guest VM 3Guest VM 3

10.1.1.5Guest VM 4Guest VM 4

NetScalerLoad

Blancer

NetScalerLoad

Blancer

Private IP10.1.1.111

Public IP 65.37.141.111

Juniper SRX

Firewall

Juniper SRX

Firewall

CS Virtual Router provides Network Services External Devices provide Network ServicesNetwork Hardware exposing API can be controlled

CSVirtual Router

CSVirtual Router

Slide from Chiradeep Vittal, http://www.slideshare.net/cloudstack/cloudstack-networking

New Isolation types ?

Hugo’s talk

Storage• Primary Storage:

– Anything that can be mounted on the node of a cluster. – Cluster LVM…iSCSI…– Holds disk images of running VMs– Support for CEPH with KVM hypervisors

• Secondary Storage:– Available across the zone– Holds snapshots and templates (image repo)– Can use Openstack swift or any object store (Gluster FS…)– New support for Caringo

• Can use NFS for both to start• Storage Abstraction refactoring underway

Wido’s talk

“Once deployed, how do you use it ?”

A powerful UI

An extensive API

API• Not REST but a Query API• A set of methods available over http(s)• Unauthenticated on integration port• Authenticated on 8080 using Access and Secret Key• Python/Ruby clients available• Internal Marvin client• Cloudmonkey CLI• Other clouds client support the API

“Not a Standard, But…”

Enabling EC2 and S3

Oved’s talk

“Lessons learned from Deploy and Use :”

Noa’s talk

“Build an Ecosystem:a PaaSa SaaS …”

Charles’s talk

Brian’s talk

“To get your feet wet…”

DevCloud

• A Virtual box appliance packaged to provide a working CloudStack environment.

• Aimed at developers but has other use cases:– Xen PV hosts gives nested virtualization– Local EC2/S3 Cloud on your laptop– Networking experiments ?

Self-Contained

DevCloud: as HostRun CloudStack on local machineUse DevCloud to setup hosts

“Time for a Demo ?”

Build A Cloud Events

• http://www.buildacloud.org• @build_cloud or @sebgoa on twitter• Upcoming events in Europe in 2013:

– BACD Ghent Belgium February 1st

– FOSDEM February 2-3, Brussels– In the works, Cloudcamp, Edinburgh March 27th– Probable LinuxTag, Berlin, May– Probable Summer: RMLL 6-11 July, Brussels– Probable in Fall: Linuxcon Europe, OWF Paris

Info• Apache incubator project• http://incubator.apache.org/cloudstack• #cloudstack and #cloudstack-dev on irc.freenode.net• @CloudStack on Twitter• http://www.slideshare.net/cloudstack• http://incubator.apache.org/cloudstack/mailing-lists.html• Collaboration Conference, Nov 30th -Dec 2nd

– http://collab12.cloudstack.org/– http://www.youtube.com/diycloudcomputing

Welcoming contributions and feedback, Join the fun !