It’s the App, Stupid!Orchestration, Automation, Scaling
& What’s in Between
Uri Cohen GigaSpaces @uri1803#CCCEU13 #ItsTheAppStupid
Meet Petsy, Selling Pet Art and Supporting Pet Artists Since 2013
So Their Services Are Deployed on CloudStack
Gunicorn
PostgreSQL
Hadoop
ActiveMQ
Nginx
MongoDB
Logstash
Graphite
Jenkins
Nagios
The Business Is Doing
Great, Thx (esp. Cat
paper crafts), But…
Rolling out new code is Painful
http://www.interestingtopics.net/storage/5762c0bec0add88fae0c3958d61effe1.jpg
The Business Is Doing
Great, Thx (esp. Cat
paper crafts), But…
MTTR is… mehhh
Mehhh
http://fineartamerica.com/featured/wooly-sheep-ramona-johnston.html
They Need to Automate!!
Let’s have a closer look at DevOpsProcesses
It’s about Workflows
and Triggers
Automated Deployment
• Triggers: Manual, CI Server• Flow: – Provision Cloud
Resources: Compute, Storage, Network
– Configure servers – Push code– Start components
http://www.flickr.com/photos/smilemark/4611091236/sizes/m/in/photostream/
InfraUpgrade
• Trigger: Critical updates, patches, etc.
• Flow: Take down relevant processes, one by one, apply patch, and reconnect to cluster. In some cases restarts server
Code Push (or more generally,
CD)
• Trigger: CI server, Manual • Flow: Canary,
Red/Black, A/B…– In general – Partial
deploy, verify, rollback / complete deploy
Node Failure • Trigger: #$%@
happens… • Flow: Detect, provision
VM, attach storage, add to network, reconfigure app (load balancer in case of web serve, DB connection in case of database, etc.)
Scaling • Trigger: System
SLAs, Schedule• Flow: Add new
nodes, push code, reconfigure dependent nodes, reconfigure LB
Automation Encompasses All Layers
Proxy Config WAR Schema
Infrastructure
Middleware Apache Tomcat MySQL
Application
Host Networks Storage
Automation Building Blocks
Environment Creation
SW Infra. Setup & Config
Code Push Monitoring & Alarming
Repairing Scaling
Let’s Look at Some Tools
Orchestration Tools
Environment Creation
SW Infra. Setup & Config
Code Push Monitoring & Alarming
Repairing Scaling
CM Tools
Environment Creation
SW Infra. Setup & Config
Code Push Monitoring & Alarming
Repairing Scaling
Automation Tools
Environment Creation
SW Infra. Setup & Config
Code Push Monitoring & Alarming
Repairing Scaling
Metric & Log Collection Tools
Environment Creation
SW Infra. Setup & Config
Code Push Monitoring & Alarming
Repairing Scaling
Still, when you need to
automate & orchestrate,
that’s what is looks like…
Where You Want to Be
Environment Creation
SW Infra. Setup & Config
Code Push Monitoring & Alarming
Repairing Scaling
How It’s Done in AWS
How It’s Done in AWS
He Calls It DevOps Automation
http://www.allthingsdistributed.com/2013/02/aws-opsworks.html
Still a Bit Too Rigid
OpsWorks – We Can Do
Better!
• Integrated with CloudStack (and other clouds)• Custom
Workflows• Not limited to
Chef• Open Monitoring
and Policies
Our TOSCA-Inspired Building Blocks
Application Topologies
Workflows
Policies
Application Topologies
• Nodes – The layers in the cake -
e.g. VM, web server, app module
– Actions (can be grouped into Interfaces)
• Relationships– Actions– Requirements <->
capabilities
Workflows
• Create and interact with nodes using interfaces
Policies• Trigger workflows
based on a certain system state
Where We Are Today
http://cloudifysource.orghttp://github.com/cloudifysource/cloudify
Next Steps • 2nd Gen (AKA Project Cosmo) is underway github.com/cloudifysource/cosmo-manager github.com/cloudifysource/cosmo-mezzanine-example
DSL
GUI
Second Gen ArchitectureBlueprint + RuntimeModel
Metrics
Plugins
WorkflowEngine
TaskBroker
PolicyEngine
AgentMetrics
Collector
Plugin
RESTAPI
Agent
Plugin
Thank You!
