cloud foundry elastic architecture and deploy based on openstack

CloudFoundry: Elastic Architecture Design and Deploy Based on OpenStack

郭峰 (Golfen Guo)@ 教授也是浮云golfen.guo@emc.com

Cloud Platform and Application Lab, EMC Labs China

EMC Labs China

Advanced Technology Research and Development

Big Data Lab

Cloud Infrastructure and System Lab

Cloud Platform and Applications Lab

University Collaboration

Industry Standards Office

IP Portfolio Development

VisionBecome an elite research and

advanced technology institute in China

-Become the model

for future EMC Labs worldwide

Agenda

• What is PaaS

• CloudFoundry Overview

• Elastic Architecture in CloudFoundry

• Introduce to BOSH

• CPI and OpenStack

• Deploy PaaS with BOSH

What is PaaS? (from Search Cloud Computing)

• Platform as a Service (PaaS) is a way to rent hardware, operating systems, storage and network capacity over the Internet.

• The service delivery model allows the customer to rent virtualized servers and associated services for running existing applications or developing and testing new ones.

RENT, not BUILD one!

Virtual Server and Associated Service

How PaaS Benefit Us?

Before: Now:

How PaaS Benefit Us? (Cont.)

Traditional IT

Storage

Servers

Networking

Middleware

Virtualization

Applications

Runtime

Storage

Servers

Networking

Middleware

Virtualization

Applications

Runtime

Agility and Cost

Savings

Storage

Servers

Networking

Middleware

Virtualization

Applications

RuntimeY

From Accelerating your Journey to Application Transformation, EMC World 2012

PaaS powered

Cloud Foundry – The Open PaaS

PaaS in Action…

$ cat hw.c#include <stdio.h>main() { printf(“Hello World\n”);}

$ cc hw.c; ./a.out

Another analogy from Mark Lucovsky,

Every one knows, “Hello World”:

PaaS in Action... (Cont. )

$ cat hw.rb require 'rubygems' require 'sinatra'

$hits = 0 get '/' do $hits = $hits + 1 "Hello World - #{$hits}" end

$ vmc push hw

“Hello World” in the Cloud:

Scale App On Demand

Step 1: Scale instance to 10:

$ vmc instances hw 10

Step 2: Change the codes:

get '/' do $hits = $redis.incr(‘hits’) "Hello World - #{$hits}" end

Step 3: bind Redis service to App:

$ vmc create-service redis –bind hw

Implement a Scalability “Hello World” in Cloud:

Scale App On Demand(Cont.)

From Cloud Foundry Launch Event, April, 2011 PaaS powered

Simplify Deployment [mysqld]user = foobarport = 3306basedir = /usrbind-address = 172.58.77.101key_buffer = 16Mthread_stack = 128Kthread_cache_size = 8…

[nginx]http.include mime.types;default_type: application/octet-stream;log_format: main ‘$remote_addr - $remote_user []…’keepalive_timeout 65;

[tomcat]<Connector redirectPort=“8443” emptySessionPath…/><bean id=“sessionFactory” class=“org.springframework…/> [frontend]dependencies:- mysqlclient- rubyfiles:- core/app/fe/**/*- core/common/**/*

[blah]- blah blah blah

Web App

Before:

Simplify Deployment (Cont.)

Web App

# to target and login to cloud foundryvmc target http://api.cloudfoundry.comvmc login

# to create and boot the app for the first timevmc push myapp –instances 2 –mem 64M –path ../code

# to create the database and bind it to the appvmc create-service mysql –name mydb –bind myapp

# update live app with new codevmc update myapp –path ../code

First Touch CloudFoundry

prompt> gem install vmcprompt> vmc target api.cloudfoundry.comprompt> vmc loginprompt> vmc push

Would you like to deploy from the current directory? [Yn] Yes Application Name: hello Application Deployed URL: 'hello.cloudfoundry.com'? hello-bob.cloudfoundry.com Detected a Sinatra Application, is this correct? [Yn] Yes Memory Reservation [Default:128M] (64M, 128M, 256M, 512M or 1G) (Press Enter to take default) Would you like to bind any services to 'hello'? [yN]: No Uploading Application: Checking for available resources: OK Packing application: OK Uploading (0K): OK Push Status: OK Staging Application: OK Starting Application: OK

First Touch CloudFoundry(Cont.)Upload your app

Your apps deployed

Your all created services

Memory of each instance

Change served instances

Start, stop, update, restart

Services of current app

Information of app

PaaS Architecture Pattern

RabbitMQ

MongoDB

PaaS Architecture Pattern (Cont. )

Conclusion

Abstract to three layers:

• Routers for finding right endpoint of Apps

• Runtime for Apps

• Services provided by platform, consumed by Apps

The Keys of Design…

• Elastic

• Failover/System Robust

• Resource Recycling

Elastic Architecture in CloudFoundry

Design principals:1. Each components can be run standalone;2. Each components can be scale-out, and notify the peers with

message;3. The components communicate only with message or RESTful

Self-government and Loose Couples :• Easy to add new components. eg Stager, UAA, ACM…;• Easy to evolve each component. eg. CC_ng, Router v2…

Elastic Architecture in CloudFoundryA request comes…

Pluggable Runtime Support

Refers to https://github.com/cloudfoundry/vcap-staging

• stager -> vcap-staging

Stager StagingPlugin

PHP Rails3Java_We

bPlay …

klass = StagingPlugin.load_plugin_for(plugin_name)plugin = klass.from_file(config_path)plugin.stage_application

extends StagingPlugin

Start Script

Stop Scrip

tServe

Pluggable Runtime Support (Cont.)

So what we need to do is…• Extends Class StagingPlugin in

Common.rb

Line3 ~ Line 62, 59 lines of codes to support PHP.

Pluggable Runtime Support (Cont.)

Key methods to rewrite:• stage_application

• start_command

• startup_script

• stop_command

• stop_script

Pluggable Services SupportRefers to a nice presentation by Nicholas

Kushmerick

Cloud Foundry Services in last forum:• Service advertisement

• Service Gateway -> Cloud Controller

• POST /services/v1/offerings

• DELETE /services/v1/offerings/:label

• Instance management

• Cloud Controller -> Service Gateway

• Provision: POST /gateway/v1/configurations

• Bind: POST /gateway/v1/configurations/:id/handles

• Unbind: DELETE /gateway/v1/configurations/:id/handles/:handle

• Unprovision: DELETE /gateway/v1/configurations/:id

But …

• Modular Design

• Version Tolerance

• Flexible Runtime/Service

• Elastic Architecture

• Many kinds of nodes

• Many nodes each kind

Complex deployment process like other distribution system

Trade Off

Practical problem at CloudFoundry.com

40+ unique node types

75+ unique software packages

500-5,000 VMs

2x/week cf.com updates

24*7*365 non-stop operation

No-downtime deployments

Reliable ,robust ,repeatable deployments ,updates , capacity adjustments

Small teams manage many instances

How to deploy CloudFoundry?

CloudFoundry BOSH CloudFoundry BOSH is an open source tool-chain for release

engineering, deployment, and lifecycle management of large scale distributed services

– Prescriptive way of creating releases and managing systems and services

– It is not a collection of shell scripts, not a pile of Perl

Built to deploy and manage production-class, large scale clusters– Production grade Cloud Foundry clusters: 500+ VMs, 40+ jobs, 75+

packages– Multi-node, multi-tier, complex clusters

Built for DevOps usage and scale by a crack team of veterans– A project, not a product: command line interface, YAML, etc.– Continuous improvement, iterative development, rough edges

https://github.com/cloudfoundry/bosh

CloudFoundry BOSH (Cont.)

Built from the need to operate cloudfoundry.com– Over 40 different types of VMs– Hundreds of running instances of VMs– Pushing at least 2 changes per week

End-to-end management– Release engineering– Deployment– Lifecycle management

Generic solution– Any service– Any cloud– Any operating system

BOSH Architecture

1. Upload Stemcell1. CLI -> Director2. Director -> Blobstore

2. Upload Release1. CLI -> Director2. Director -> Blobstore

3. Deploy1. CLI -> Director2. Director -> A3. Agent -> B4. Agent -> C

BOSH Concepts Stemcell

– VM template– BOSH Agent– IaaS Plugin

Release– Jobs

Job– Packages– Templates (scripts, confs)– Monitoring

Package– Source/blobs– Dependencies– Packaging (scripts)

IaaS Neutral

Cloud Foundry BOSH

Cloud Provider Interface(CPI)

vSphere: battle tested implement AWS: code complete

https://github.com/piston/openstack-bosh-cpi

OpenStack: testable release

Cloud Provider Interface

Stemcell– create_stemcell (image, cloud_properties)– delete_stemcell (stemcell)

VM– create_vm (agent_id, stemcell, resource_pool, networks,

disk_locality, env)– delete_vm (vm)– reboot_vm (vm)– configure_networks (vm, networks)

Disk– create_disk (size, vm_locality)– delete_disk (disk)– attach_disk (vm, disk)– detach_disk (vm, disk)

Cloud Provider Interface (Cont.)For OpenStack

Stemcell– create_stemcell– delete_stemcell

VM– create_vm– delete_vm– reboot_vm– configure_networks

Disk– create_disk– delete_disk– attach_disk– detach_disk

Cloud Provider Interface

OpenStack Compute

OpenStackImage Service

Deploy PaaS with BOSH

Deployments

Release

Network

Resource pools

Properties

Update concurrency

Compilation workers

Cloud properties

Deployments for CloudFoundyCloudfoundry.ymlname: cloudfoundry

release: name: cloudfoundry version: 89.1-dev

compilation: workers: 4 network: default cloud_properties: ram: 1024 disk: 2048 cpu: 2

update: canaries: 1 canary_watch_time: 3000-90000 update_watch_time: 3000-90000 max_in_flight: 2 max_errors: 1

Deployments for CloudFoundy (Cont.)Cloudfoundry.yml

networks: - name: default subnets: - static: - 192.168.2.50 - 192.168.2.89 range: 192.168.2.0/24 gateway: 192.168.2.1 dns: - 10.254.174.10 cloud_properties: name: PrivateNetwork- name: lb subnets: - static: - 192.168.2.90 - 192.168.2.99 range: 192.168.2.0/24 gateway: 192.168.2.1 dns: - 10.254.174.10 cloud_properties: name: PrivateNetwork

Deployments for CloudFoundy (Cont.)Cloudfoundry.ymlresource_pools:

- name: infrastructure network: default size: 29 stemcell: name: bosh-stemcell version: 0.6.2 cloud_properties: ram: 256 disk: 2048 cpu: 1 env: bosh: password:

Deployments for CloudFoundy (Cont.)Cloudfoundry.ymljobs:

- name: cloud_controller template: cloud_controller instances: 1 resource_pool: infrastructure networks: - name: default static_ips: - 192.168.2.60

- name: nats template: nats instances: 1 resource_pool: infrastructure networks: - name: default static_ips: - 192.168.2.52

Deployments(CloudFoundy)Cloudfoundry.yml

properties: domain: cflocal.com

env: {}

networks: apps: default management: default

nats: user: nats password: aaa3ij3122 address: 192.168.2.52 port: 4222

Use Case

bosh deployment cf.yml

git pull

bosh upload releaseRun

bosh deployDone!

bosh target cf

Bug report to Dev

Acknowledgments

VMware China R&D Center Network & Information Center, Shanghai Jiao Tong Univ.

CloudFoundry Communityhttp://www.cloudfoundry.org

Piston Communityhttps://github.com/piston/openstack-bosh-cpi

Backup Slides

Management Path– vmc info

health manager

service

cloud controller

app app

router

Deployment Path– vmc push

health manager

service

cloud controller

app app

router

Data Path– Client Request

health manager

service

cloud controller

app app

router

Evolving Cloud Foundry - Kernel

stager

Package cache

caldecott

cloud controller

router

nats health manager

app appservice

warden

How to write a custom service?

• Public API for advertising services and managing service instances

• Endpoints:– Cloud Controller– Service Gateway

(one per service)• REST / JSON / HTTP(S)

Cloud Controll

MySQL Gatewa

yMongoD

BGatewa

How to write a custom service? (Cont.)

Step 1: Service advertisement

Service Gateway -> Cloud Controller, two methods: POST /services/v1/offerings DELETE /services/v1/offerings/:label

:label => 'mysql-5.1':url => 'http://12.34.56.78:4321',:plans => ['free', 'bronze', 'silver', 'gold']:tags => ['mysql', 'mysql-5.1', 'database']:active => true,:description => 'MySQL database service':acls => { :wildcards => '*@vmware.com', :plans => { :bronze => { :users => ['joe@vmware.com', 'jane@vmware.com'] } }}

How to write a custom service? (Cont.)

Step 2: Instance management

Cloud Controller -> Service Gateway, four methods:

Provision: POST /gateway/v1/configs Bind: POST /gateway/v1/configs/:id/handles Unbind: DELETE /gateway/v1/configs/:id/handles/:handle Unprovision: DELETE /gateway/v1/configs/:id

Private API App

Service Gateway

Service Node(eg, PostgreSQL)

Cloud Controller

vmc, STS

Developer

End Users

Avmc create-service postgresql

BPOST /services/v1/configurations

ECREATE DATABASE abc123CREATE ROLE pqr456

CPOST /gateway/v1/configurations

G{host: 12.34.56.78, port: 3456, db: abc123, user: pqr456}

HVCAP_SERVICES = [postgres: {host: 12.34.56.78, port: 3456, db: abc123, user: pqr456}]

I GET /foo/bar

SELECT *FROM FOO

host 12.34.56.78Service

(eg, PostgreSQL)

DNATS: provision

F{host: 12.34.56.78, port: 3456, db: abc123, user: pqr456}

Reference implementation

For legacy service – Service Broker

• Cloud Foundry users may already have existing services• Legacy database that hasn’t / won’t / can’t be moved

CloudFoundry.com• Services not yet supported

• Service Broker exposes an arbitrary legacy service instance to Cloud Foundry so that it can be used just like any other instance

For legacy service – Service Broker (Cont.)

AppService Gateway

Core Service(eg, PostgreSQL)

Cloud Controllervmc, STS

Service Broker

Brokered Service(eg, Oracle DB)

Service Broker Admin Tools

Bregister hrdb {host: 87.65.43.21, port: 6543, db: def123, user: stu456}

CPOST /broker/v1/configuration

EPOST /services/v1/configurations

Dvmc create-service hrdb

POST /gateway/v1/configurationsG{host: 87.65.43.21, port: 6543, db: def123, user: stu456}

HVCAP_SERVICES = [hrdb: {host: 87.65.43.21, port:6543, db: def123, user: stu456}]

IGET /foo/bar

SELECT *FROM FOO

host 12.34.56.78

ACREATE DATABASE def123CREATE USER stu456

Connection

Details

cloud foundry elastic architecture and deploy based on openstack

emc corporation

emc world2012 paas

emc labs china copyright

paas powered copyright

open paas copyright

code copyright

app vmc

ok copyright

Technology

how to deploy openstack on tianhe-2 supercomputer · how to...

deploy openstack with saltstackqiniuppt.qiniudn.com/shi...

things you must know before you deploy openstack

deploy tosca network functions virtualization (nfv)...

intel cloud foundry and openstack

docker openstack cloud foundry

how to deploy your application to heroku and cloud foundry

altoros case study: software-defined drug delivery w/ cloud...

suse openstack cloud + stackato = cloud foundry private ......

ho15982 deploy openstack - susecon · ho15982 deploy...

cloud foundry bosh cpi for openstack

as a service: cloud foundry on openstack - lessons learnt

openstack deployment and operations guide...

rapidly deploy an openstack cloud with mirantis + solidfire

openstack deployment manual - bright...

openstack deployment and operations guide technologies....

openstack and cloud foundry for cloud native apps

openstack & cloud foundry (openstack fall 2012 summit)

openstack + vmware: deploy, upgrade, & operate a powerful...

cloud foundry and openstack