intro to cloudify
DESCRIPTION
Ron Zavner's presentation at the Infracoders London Meetup July 9th, 2013.TRANSCRIPT
GigaSpaces Cloudify Any App, On Any Cloud, Your Way
February 2012
2
Intro to Big data world & Cloud apps Challenges of big data applications
Outages & disaster recovery Complexity of multi tiers application Lock-in to specific cloud Monitoring & Scaling
PaaS make our life easier …
® Copyright 2013 GigaSpaces Ltd. All Rights Reserved
AGENDA
WE’RE LIVING IN A REAL TIME WORLD…
Homeland Security
Real Time Search
Social
eCommerce
User Tracking & Engagement
Financial Services
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved3
BIG DATA PREDICTIONS
“Over the next few years we'll see the adoption of scalable frameworks and platforms for handling streaming, or near real-time, analysis and processing. In the same way that Hadoop has been borne out of large-scale web applications, these platforms will be driven by the needs of large-scale location-aware mobile, social and sensor use.”
Edd Dumbill, O’REILLY
4® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
5
THE BIG GUYS…
Managing Big Data on the Cloud
• AWS – around 0.5M servers• Facebook – less than 0.1M servers• Google – around 1M servers
6
STORE 135+ BILLION MESSAGES A MONTH
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
7
THE OUTAGE PROBLEM
A TYPICAL BIG DATA SYSTEM…
9
LOCKED IN TO SPECIFIC CLOUD?
Managing Big Data on the Cloud
10
DISASTER RECOVERY
Realization:
What You Really Care about Is App Portability
OS is the same on any cloud
Most clouds have compute & storage
Elasticity & scaling have same effects on the app, regardless of the cloud
Cloud Portability Facts
Zynga moved ~80% of their workload from Amazon to their private zCloud
“own the base, rent the spike”
http://code.zynga.com/2012/02/the-evolution-of-zcloud/
13
You must use an AUTOMATION layer
14
Managing Big Data on the Cloud
• No Code Change• No lock-in• Full Monitor & Control
Moving Apps At Massive Scale – The Right Way
CLOUDIFY POSITIONING IN THE CLOUD STACK
15
PaaS
IaaS
DevOps(Automation)
Productivity
Control
ChefPuppet
CloudFoundryHeroku
GAEOpenShift
Rightscale
Public clouds(AWS, Rackspace,..) Private clouds
(Vmware, OpenStack..)
High productivity with full control
Enstratus
HOW DOES IT WORK?1 Upload your recipes
2 Cloudify creates VMs & installs agents
3 Agents install and manage your app
4 Cloudify automates the scaling
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved16
® Copyright 2012 GigaSpaces. All Rights Reserved17
EXTENSIVE PLATFORM SUPPORT
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved18
BOOTSTRAPPING
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved19
INSTALL APPLICATION
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved20
POST DEPLOYMENT
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved21
SELF HEALING
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved22
GETTING COMPUTE RESOURCES IN A PORTABLE WAY
compute { template "SMALL_LINUX"}
SMALL_LINUX : template imageId "us-east-1/ami-76f0061f“ remoteDirectory "/home/ec2-user/gs-files“ machineMemoryMB 1600 hardwareId "m1.small" locationId "us-east-1" localDirectory "upload" keyFile "myKeyFile.pem"
options ([ "securityGroups" : ["default"]as
String[], "keyPair" : "myKeyFile"])
overrides (["jclouds.ec2.ami-query":"",
"jclouds.ec2.cc-ami-query":""])privileged true
}
SMALL_LINUX : template{ imageId "1234" machineMemoryMB 3200 hardwareId "103" remoteDirectory "/root/gs-files" localDirectory "upload" keyFile "gigaPGHP.pem" options ([ "openstack.securityGroup" : "default", "openstack.keyPair" : "gigaPGHP"
])privileged true
}
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved23
UNDERSTAND APPLICATION STRUCTURE & DEPENDENCIESapplication {
name="petclinic"service { name = "mongod" }service { name = "mongoConfig" }service { name = "apacheLB" }service { name = "mongos" dependsOn = ["mongoConfig",
"mongod"]}service { name = "tomcat" dependsOn = ["mongos","apacheLB"]
}}
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved24
DYNAMICALLY CONFIGURE APPLICATION TIERS
1
23
serviceContext.attributes.thisInstance["port"] = config.port
dbServiceInstances = ServiceContext.attributes[“db”].instances
dbServicePort = dbServiceInstances[1].port
customCommands ([
"addNode" : "apacheLB_addNode.groovy",
"removeNode" : "apacheLB_removeNode.groovy",
"load" : "apacheLB-load.groovy“ ])
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved25
AUTO SCALE YOUR WAY
scalingRules ([ scalingRule { serviceStatistics { metric "Total Requests Count" statistics Statistics.maximumThroughput movingTimeRangeInSeconds 20 } highThreshold { value 1 instancesIncrease 1 } lowThreshold { value 0.2 instancesDecrease 1 } }])
® Copyright 2012 GigaSpaces Ltd. All Rights Reserved26
5 MINUTES SETUP
/* Credentials - You must enter your * cloud provider account credentials */
user="ENTER_USER_HERE"apiKey="ENTER_API_KEY_HERE"keyFile="ENTER_KEY_FILE_HERE"keyPair="ENTER_KEY_PAIR_HERE"
// Advanced usage
hardwareId="m1.small"locationId="us-east-1"linuxImageId="us-east-1/ami-1624987f"ubuntuImageId="us-east-1/ami-82fa58eb"
27
Intro to Big data world & Cloud apps Challenges of big data applications
Outages & disaster recovery Complexity of multi tiers application Lock-in to specific cloud Monitoring & Scaling
PaaS make our life easier - Cloudify
® Copyright 2013 GigaSpaces Ltd. All Rights Reserved
SUMMARY
