cloudify intro

CLOUDIFY INTRO

July 2013Dotan Horovits, Director, Customer Services

2

The Cloud Environment High level architecture App provisioning Self-healing Scale up/down On-Demand DR Cloudify Player – PaaS in a box

AGENDA

The Basics…

5

ENTERPRISE IT - TODAY

http://www.slideshare.net/pizak/understanding-platform-as-a-service-8393601

6

No Agility Can’t provision resources in minutes. Takes

weeks or months No optimized utilization

Most data centers are at 10%-20% average CPU utilization (Gartner)

No Consistent Management Everybody looking for easier management and

availability through automation and accountability

– Each BU pays for its own use and manages its own resources – IT ops focus on IT enablement rather than on going IT

management

WHAT’S WRONG?

7

8

ENTERPRISE IT - TOMORROW

http://www.slideshare.net/pizak/understanding-platform-as-a-service-8393601

How?

GIGASPACES CLOUDIFY ANY APP, ON ANY CLOUD, YOUR WAY

February 2012

CLOUDIFY POSITIONING

App

Infra.

Convenience

Control

Public clouds(AWS, GCS Rackspace,..)

Private clouds(OpenStack, CloudStack..)

PaaS(Heroku, Google AppEngine, Amazon BeansTalk, CloudFoundry)

DevOps Automation

Amazon OpsWorks, RightScale

• Open• Any Cloud• Enterprise-Grade

11

12

CLOUDIFY & AMAZON OPSWORK

Cloudify provide the equivalent of Amazon OpsWork on other clouds

CLOUDIFY – FULL APPLICATION LIFE CYCLE AUTOMATION

Cloudify

Continuous Integration

Monitoring & Alarming

Configuration Management

Infrastructure (IaaS /

Traditional)

API

13

14

Managing Big Data on the Cloud

• No Code Change• Plug-in to the current way

of running enterprise Apps• Cloud Properties injected• Baby Step approach• Cloud Portability• Bare-Metal Cloud

Guiding Principles

Application Infrastructure as Code

Configuration & Deployment Automation

Consistent Monitoring & Management

Failover & Scaling

CLOUDIFY FOUNDATIONS

15

THE CLOUDIFY RECIPE – YOUR APPLICATION BLUEPRINT Middleware services to run Dependencies between services How to install & configure services Where to get application & service binaries When to add or remove instances How to monitor each of the services.

16

17

RECIPE DOMAIN MODEL

Application

• Name• Properties

(config)• Services• Service

Dependencies

Service

• Name• Properties

(config)• SLA• Infra templates• Lifecycle events• Custom

commands• Monitoring

1..*

RECIPE - APPLICATION STRUCTURE & DEPENDENCIESapplication {

name="petclinic"service { name = "mongod" }service { name = "mongoConfig" }service { name = "apacheLB" }service { name = "mongos" dependsOn = ["mongoConfig", "mongod"]}service { name = "tomcat" dependsOn = ["mongos","apacheLB"]

}}

18

HOW IT WORKS

19

INSTALL APPLICATION

20

POST DEPLOYMENT

21

SELF HEALING

22

AUTO SCALE

23

GET 100’S OF RECIPES OUT OF THE BOX

Leverage Puppet Templates

24

27

CLOUDIFY ARCHITECTURE

The USM allows you to deploy & manage any middleware service in any tier using an extensible recipe to describe it

Using the Cloud Driver, Cloudify can trigger host provisioning in any virtual environment and abstract it from your application

The brains of the system. Orchestrates the deployment of the application, continuously monitors it and triggers alerts and scaling rules based on real-time metrics and load

Local Cloud Basic development environment testing the app in a local mode. Good for

unit testing. BYON Cloud

Development and Deployment environment - running the app in a distributed mode leveraging existing pool of VMs/physical machines.

Private Cloud Deployment environment. Launching VMs on demand.

Public Cloud Deployment environment. Launching VMs on demand. Supporting EC2 ,

HP Cloud , Rackspace , Terramark ,Azure , openstack …

28

MAIN CLOUD DRIVERS

Management Tools

29 29

COMPLETE VISIBILITY

30

CUSTOMIZED AVAILABILITY AND PERFORMANCE MONITORING TO TRACK

YOUR APPLICATIONS

EASY TROUBLESHOOTING

31

BROWSE ALL YOUR LOGS WITH A SINGLE CLICK

MONITORING THE APPLICATION

32

Verifying application deployment vs. planned

Custom metrics at the service (cluster) level

MONITORING THE APPLICATION

33

Verifying Infrastructure usage and resource

utilization Correlating VM metrics for comparison

Recipe Deep Dive

34

35

APPLICATION DESCRIPTION THROUGH RECIPES

Recipe DSLLifecycle scriptsAvailability & Monitoring

ProbesCustom plug-ins(optional)

application {name="petclinic"service { name = "mongod" }service { name = "mongoConfig" }service { name = "apacheLB" }service { name = "mongos" dependsOn = ["mongoConfig", "mongod"]}service { name = "tomcat" dependsOn = ["mongos","apacheLB"]

}

36

APPLICATION DESCRIPTION THROUGH RECIPES

Recipe DSLLifecycle scriptsAvailability & Monitoring

ProbesCustom plug-ins(optional)

service { name "mysql" icon "mysql.png" type "DATABASE"

...}

37

APPLICATION DESCRIPTION THROUGH RECIPES

Recipe DSLLifecycle scriptsAvailability & Monitoring

ProbesCustom plug-ins(optional)

Lifecycle { install "mysql_install.groovy"  start "mysql_start.groovy"  startDetectionTimeoutSecs 900  startDetection "mysql_startDetection.groovy"  stopDetection {    !ServiceUtils.isPortOccupied(jdbcPort)  }  preStop ([         "Win.*":"killAllMysql.bat",                     "Linux.*":"mysql_stop.groovy” ])  shutdown ([                           "Linux.*":"mysql_uninstall.groovy"  ])}

ENVIRONMENT BINDING

38

compute { template "SMALL_LINUX"}

SMALL_LINUX : computeTemplate { imageId "us-east-1/ami-76f0061f“ remoteDirectory "/home/ec2-user/gs-files“ machineMemoryMB 1600 hardwareId "m1.small" locationId "us-east-1" localDirectory "upload" keyFile "myKeyFile.pem"

options ([ "securityGroups" : ["default"]as String[], "keyPair" : "myKeyFile” ]) overrides (["jclouds.ec2.ami-query":"", "jclouds.ec2.cc-ami-query":""]) privileged true}

SMALL_LINUX : computeTemplate { machineMemoryMB 5850 remoteDirectory "/tmp/gs-files” username username password password custom ( "nodesList" : ([ ([

"id" : "byon-pc-lab{0}","host-list" : "0.0.0.0”

]) ]) ]) // enable sudo. privileged true}

MONITORING PROBES

39

monitors {

def ctxPath = ("default" == context.applicationName)?"":"${context.applicationName}“

def metricNamesToMBeansNames = [ "Current Http Threads Busy": ["Catalina:type=ThreadPool,name=\"http-bio-${currHttpPort}\"", "currentThreadsBusy"], "Current Http Thread Count": ["Catalina:type=ThreadPool,name=\"http-bio- ${currHttpPort}\"", "currentThreadCount"],

return getJmxMetrics("127.0.0.1",currJmxPort,metricNamesToMBeansNames) }

AUTO SCALING

40

scalingRules ([ scalingRule { serviceStatistics { metric "Total Requests Count" statistics Statistics.maximumThroughput movingTimeRangeInSeconds 20 } highThreshold { value 1 instancesIncrease 1 } lowThreshold { value 0.2 instancesDecrease 1 } }])

SUMMARY – CLOUDIFY VALUE

Any App, Any StackDeploy any middleware stack using a recipe based deployment mechanism

Automatic Self-HealingCrashed nodes and machines are automatically replaced by new ones, following recipe instructions

Auto-Scale, Your Way Automatic scaling of your application services based on out-of-the-box or custom metrics. Scale Up or Out

Automation of the Entire Lifecycle Deploy, manage, and update your application using a single platform

Any Cloud Support all major cloud and virtualization platforms. Your Application is completely decoupled from the Cloud API

Cluster-Aware Availability & Performance Monitoring Pluggable monitoring, collects tier and application KPIs

Fully Testable on Your Laptop Easily start, debug & test on your laptop with a fully functional cloud emulator. No VMs, no hassles.

50

Thank you!