Next Generation Software Development:Continuous Cloud Deployment
@SachaLaboureyCEO, CloudBees, Inc.
April 18, 2013 – PhillyJUGPhoto credit: @romainguy
©2013 CloudBees, Inc. All Rights Reserved
2
• Born in 1975 in Switzerland
• JBoss– Clustering lead – 2001– GM Europe - 2003– CTO - 2005
• Red Hat– JBoss acquisition in June 2006– co-GM of Red Hat’s middleware division– Left Red Hat in April 2009
• CloudBees– Started in April 2010– About 40 bees in 7 countries
Introduction – Sacha Labourey
©2013 CloudBees, Inc. All Rights Reserved
X
3
• What’s the life of a developer today?• Challenges ahead…
– Ideal solution!• Cloud 101• Read my lips: SERVICE
– Software vs. Service– Continuous Cloud Delivery
• Reality Check• Quick demos
Agenda
©2013 CloudBees, Inc. All Rights Reserved
Life of a Developer Today
©2013 CloudBees, Inc. All Rights Reserved
5
• Release v1.0
• Gather requirements for v2.0 • Work on v2.0 • Release v2.0don’t screw it up: once it is out, it is out.
• Have your customer upgrade– Support v1.0 for years
• Support and patch dozen of “branches” in parallel
• Gather requirements for v3.0 • Work on v3.0
Typical Software Development
Mo
nth
s/Y
ears
Packaged Software
©2013 CloudBees, Inc. All Rights Reserved
6
Developers vs. Sys Admins (aka Love Story)
Photo Credit: Joseph Viles/ The CW Photo Credit: http://www.sysadminday.com/people.html
©2013 CloudBees, Inc. All Rights Reserved
7
M. Developer
Interests & Hobbies:• Give a try to the last framework• Put in the production application
the last shiny object seen at a conference
• Participate to the Scala vs. Ceylon debate, static vs. dynamic typing, etc.
• Dream of meeting with James Gosling
M. Sys Admin/ops
Interests and Hobbies:• Not to be beeped at 2am in the
morning• Not to be beeped at 4am in the
morning• Participate to the Debian vs. Fedora
vs. Ubuntu debate• Dream of meeting with Linus
Torvald
<sarcastic mode=“on” />
©2013 CloudBees, Inc. All Rights Reserved
8
Development? Friction!
SourceCode
Managemnt
JenkinsContinuousIntegration
UnitTest
FunctionalTest
LoadTest
IntegrationTest
Developer
“Cloud you please provision a server for me?”
“Could I get X, Y and Z installed on that server?”
“Could you please patch my server?”
“Could I please get a database dump?”
Could I get a new Git server?
Where could I run my Jenkins integration tests?
IT
©2013 CloudBees, Inc. All Rights Reserved
9
Deployment? Friction!
Configure OS Secure OS Setup JVM
Install AS on OS
Configure AS Secure AS
provision nodes
Install load-balancer (LB)
Configure LB
Secure LBSetup BC for cluster
QA cluster
Configure DNS
Provision node:
Provision cluster:
Provision node
Install DB Configure DB
Secure DBSetup BC for DB
Provision DB:
Provision cluster
Provision DBDeploy schema
Deploy App to each node
Deploy App:
(not a transparent update process)
Analyse sec. bulletins
Analyse QA bulletins
Analyse AS/OS logs
maintain OS maintain JVM maintain AS
maintain DB
Manage environment (recurring)
maintain FWValidate/QA stack
Measure App performance
Acquire Hardware
Provision node
Update Cluster Update LB
Metering/Scaling (recurring)
Update Security
©2013 CloudBees, Inc. All Rights Reserved
10
Friction, friction, friction
provision nodes
Install load-balancer (LB) Configure LB
Secure LBSetup BC for cluster QA cluster
Configure DNS
Provision cluster:
Provision cluster
Provision DB Deploy schema
Deploy App to each node
Deploy App:
(not a transparent update process)
Configure OS Secure OS Setup JVM
Install AS on OS
Configure AS Secure AS
provision nodes
Install load-balancer (LB)
Configure LB
Secure LBSetup BC for cluster
QA cluster
Configure DNS
Provision node:
Provision cluster:
Provision node
Install DB Configure DB
Secure DBSetup BC for DB
Provision DB:
Provision cluster
Provision DBDeploy schema
Deploy App to each node
Deploy App:
(not a transparent update process)
Analyse sec. bulletins
Analyse QA bulletins
Analyse AS/OS logs
maintain OS maintain JVM maintain AS
maintain DB
Manage environment (recurring)
maintain FWValidate/QA stack
Measure App performance
Acquire Hardware
Provision node
Update Cluster Update LB
Metering/Scaling (recurring)
Update Security
©2013 CloudBees, Inc. All Rights Reserved
11
Why do we do software development again?
©2013 CloudBees, Inc. All Rights Reserved
12
To serve the BUSINESS obviously!
Business Needs
SpecificationImplementa-tion
Deployment
©2013 CloudBees, Inc. All Rights Reserved
13
IT serves the BUSINESS?
Increase Revenues
Implement e-commerce site
JVM? Ruby? PHP? Java? Scala?Play? Groovy? Lift? Jruby? CFML? Grails? Spring? EE6?JBoss? Tomcat? Glassfish? Jetty? MySQL? MongoDB? CouchDB? Cassandra? JMS? AMQP? Memcache? Ehcache? Quartz? Lucene? Solr? Jquery? GWT? iPhone? Android? JIRA? Redmine? SVN? Git? Maven? Ant/Ivy? Jenkins? Artifactory/Nexus? etc.
AMD/Intel? HP? DELL? SPARC? UX? AWS? Linux? Windows? Solaris? RHEL? Ubuntu? Debian? Arch? EXT2/3? NFS/CIFS? Backup? Chef? Puppet? Continuous Deployment? Apache HTTPD? Mod_jk? Mod_proxy? Nginx? Haproxy? Firewall? SSL? Server provisioing? Load-balancing? Failover? Monitoring? Management? Alerting? Escalation procedure? etc.
15 seconds
2 weeks
18 months
©2013 CloudBees, Inc. All Rights Reserved
14
14
Didn’t we simply ask him to build an e-commerce site?
15
• Lots of FRICTION• Less time spent on
DELIVERING VALUE
And that’s about to get worse…
Software Development got Difficult
©2013 CloudBees, Inc. All Rights Reserved
Challenges ahead!
©2013 CloudBees, Inc. All Rights Reserved
17
• In parallel, demand for new needs are growing fast!– Mobile– Big Data– Social– Local– …
At the same time…
©2013 CloudBees, Inc. All Rights Reserved
18
Entering the post-PC era
Thanks to Mary Meeker and Matt Murphy, From David Skok presentation @ MassTLC
©2013 CloudBees, Inc. All Rights Reserved
19
19
Source: Mary Meeker , 2011, www.kpcb.com/internettrends2011
©2013 CloudBees, Inc. All Rights Reserved
20
Small but big!
©2013 CloudBees, Inc. All Rights Reserved
Source: Mary Meeker , 2012, http://www.kpcb.com/insights/2012-internet-trends
21
India just made the switch!
©2013 CloudBees, Inc. All Rights Reserved
Source: Mary Meeker , 2012, http://www.kpcb.com/insights/2012-internet-trends
22
• Drivers are accelerating the need for NEW apps– Mobile, Mobile, Mobile, Social, Big Data, Etc.
• Time-to-market is more important than ever!– “I need it for yesterday!”– Can’t afford 18m cycles
Plus, you must be in DISCOVERY MODE(admit it: you don’t really know what’s needed!)
Drivers & Acceleration
©2013 CloudBees, Inc. All Rights Reserved
+
+Jav
a.lang
.Stack
Overfl
owErro
r
23
• You are (t)asked– To do many more new applications– To deliver them much earlier– To deliver incremental changes very quickly– To measure whether what you are doing works– To scale quickly the few apps that will be successful– To have them interact with an increasing number of
different devices
… Yet, nothing else should change?
Impact
©2013 CloudBees, Inc. All Rights Reserved
Java.l
ang.St
ackOve
rflowE
rror
Looking For a Solution?
©2013 CloudBees, Inc. All Rights Reserved
25
• You should be able to EXPERIMENT• That means
– Get a first release out FAST (i.e. TTM)– TRY lots of different things (i.e. DISCOVERY)
• KEEP what works• KILL what doesn’t
– ITERATE FAST (i.e. GO GO G0)
Ideally…
©2013 CloudBees, Inc. All Rights Reserved
26
• In theory, fast-iteration is great! But…
FAST_ITERATION + FRICTION => HEAT!
Not feasible in your current environment You need a no-friction environment
(hint: this is what the cloud is about)
“ Yes, but… ”
©2013 CloudBees, Inc. All Rights Reserved
Cloud 101
XaaS
©2013 CloudBees, Inc. All Rights Reserved
28
Traditional software stackWe have done this for 20 years!
©2013 CloudBees, Inc. All Rights Reserved
Server
Hypervisor
VM
AS
JVM
LB
Application
You
29©2013 CloudBees, Inc. All Rights Reserved
Servers
OS
VM
AS
JVM
LB
ApplicationsCloud Computing: How to do it? Who does what?
Cloud Provider?
30
Easy: « Just use a XaaS! »
©2013 CloudBees, Inc. All Rights Reserved
31
• SalesForce.com, Zendesk, NetSuite, etc.• Focuses on recurring IT needs
– CRM, e-mail, ERP, etc.– Very quick setup, but very limited
customization– Not for developers
• THINK– Standard applications
SaaS
©2013 CloudBees, Inc. All Rights Reserved
32
• AWS – the most popular example• Server Lego blocks
– servers, storage blocks, DNS, IP addresses, etc.• Development environment:
– You rebuild your stacks “in the cloud”– Flexible but cOmpLi CaTed
• THINK– Custom-built systems
IaaS
©2013 CloudBees, Inc. All Rights Reserved
33
ü Directly managing your IaaS resources provides you with resource elasticity (CAPEX-free), but…
O The amount of soft-IT typically required to do so is … higher!– All of the traditional IT activities remain
(maintain/patch/monitor OS+JVM+AS+DB++, etc.)– + cloud-specific items: elasticity/security/automation
• I can read your mind: this is where you will start• My advice? Do it and then move on…
IaaS – Consequences
©2013 CloudBees, Inc. All Rights Reserved
34
• Cloud concepts are applied to Applications– On-demand, pay-as-you-go, elasticity, etc.– No need to handle updates, patches, scalability, failover, etc.
• This is about APPLICATIONS– NOT about systems– NOT about OS, middleware, etc.– You focus on your app, the PaaS
orchestrates the underlyinginfrastructure so “it happens”
• THINK– CUSTOM APPLICATIONS
PaaS
©2013 CloudBees, Inc. All Rights Reserved
And how using the Cloud will help me?
©2013 CloudBees, Inc. All Rights Reserved
36
• We focus a lot on the P/I/S of XaaS• But not enough on the … S!
• We are entering a SERVICE-ERA– IaaS - You are CONSUMING IT resources– SaaS - You are CONSUMING an online Service– PaaS - You are CONSUMING development services
This is not about software anymore!
The Cloud is refocusing IT roles
©2013 CloudBees, Inc. All Rights Reserved
37
• What’s the big difference?– They’re just a long series of 1 and 0 at the end of the day,
right?
• A Services is not just “hosted software”– i.e. take your current software, host it, and keep doing what
you’ve been doing to date
Software vs. Service
©2013 CloudBees, Inc. All Rights Reserved
38
Software != Service
©2013 CloudBees, Inc. All Rights Reserved
Not the same …
[ requirement gathering | development |
QA |success measurement |
release | support ]
… processes
39
Packaged Software vs Cloud Software Delivery
Development Methodology VariesBonus: “We use CI”
Unachievable without Continuous Integration and Continuous Delivery
©2013 CloudBees, Inc. All Rights Reserved
40
• Release v1.0
• Gather requirements for v2.0 • Work on v2.0 • Release v2.0don’t screw it up: once it is out, it is out.
• Have your customer upgrade– Support v1.0 for years
• Support and patch dozen of “branches” in parallel
• Gather requirements for v3.0 • Work on v3.0
• Release v0.1• Implement micro-feature, deploy,
measure success, keep or kill • Implement micro-feature, deploy,
measure success, keep or kill • Patch a bug• Implement micro-feature, deploy,
measure success, keep or kill • Implement micro-feature, deploy,
measure success, keep or kill • Implement micro-feature, deploy,
measure success, keep or kill
To Be Or Not To Be…M
on
ths/
Yea
rs
Day
s/W
eeks
Packaged Software As-A-Service
©2013 CloudBees, Inc. All Rights Reserved
41
Time
v1.0 Release
v2.0 Release
Time
v1.0.x ContinuousReleases
Risk/Cost
Risk/Cost
Packaged Software As-A-Service
Lower Risk and Cheap Experimentation
Same basic value prop as agile, but code is being released continuously
©2013 CloudBees, Inc. All Rights Reserved
42
• Integrated & automated “pipeline”• Always be in a “release-ready” state
Continuous Cloud Delivery
©2013 CloudBees, Inc. All Rights Reserved
Code Build Test Stage Deploy
#Fail#Fail #Fail#Fail
43
Complex Scenarios are possible!
43
SQE Job
Database Tests
Compliance Tests
Stress Tests
Developer Job
Developer Tests
Functional Tests
X-Team Tests
Security Tests
Access Tests
Production Job
Backup Production
Deploy
Monitor
Developer Org
QA Org
Production Org
Promote
Promote
©2013 CloudBees, Inc. All Rights Reserved
44
• Your Code Your Service• Always be in a “deploy-ready state”
– This requires a NO-friction environment• Enables high-frequency iterations
– Code, test, deploy, measure, keep-or-kill
Continuous Cloud Delivery
©2013 CloudBees, Inc. All Rights Reserved
Why am I telling you all of this?
Enter into the world of CloudBees!
©2012 Cloud Bees, Inc. All Rights Reserved
46
The CloudBees Platform
cloudbees.com
Test
Stage
Build
Code
JenkinsExecutor
Fo
rg
e
Re
po
si
to
ri
es
Git
SVN
mvn
CloudBees API
Grand Central Web Console
SDK
SessionClustering
MySQL
Router
Application
PartnerEcosystem
©2013 CloudBees, Inc. All Rights Reserved
Code Test Stage Deploy ManageBuild
Develop Deploy
Iterate
47
What You Interact With When You Use CloudBees
47
CLI/SDKGrandCentral Web
Console IDE
©2013 CloudBees, Inc. All Rights Reserved
48©2013 CloudBees, Inc. All Rights Reserved
Integrated « add-ons » for DEV and RUN
49©2013 CloudBees, Inc. All Rights Reserved
Flexible Deployments
cloudbees.com
Amazon - US
Amazon - EUR
AmazonVirtual Private CloudUS or EUR
HP Cloud ServicesOpenStack
• Fully integrated with Amazon AWS• Provided as a SERVICE, nothing for you to maintain/patch
SDK
Developers
EnterpriseDatacenter
50
ClickStartsGet started in a snap!
©2013 CloudBees, Inc. All Rights Reserved
51
ClickStacksReuse or create your own software stack
©2013 CloudBees, Inc. All Rights Reserved
CustomerManaged
CommunityManaged
CloudBeesCurated
Fork
Push Push Clone
52
Development? No friction!
DeveloperCloudBees
PaaS
©2013 CloudBees, Inc. All Rights Reserved
53
Deployment? No Friction!
Configure OS Secure OS Setup JVM
Install AS on OS Configure AS Secure AS
provision nodesInstall load-balancer (LB)
Configure LB
Secure LB Setup BC for cluster QA cluster
Configure DNS
Provision node:
Provision cluster:
Provision node Install DB Configure DB
Secure DB Setup BC for DB
Provision DB:
Provision cluster Provision DB Deploy schema
Deploy App to each node
Deploy App:
(not a transparent update process)
Analyse sec. bulletins
Analyse QA bulletins Analyse AS/OS logs
maintain OS maintain JVM maintain AS
maintain DB
Manage environment (recurring)
maintain FW Validate/QA stack
Measure App performance
Acquire Hardware Provision node
Update Cluster Update LB
Metering/Scaling (recurring)
Update Security
(transparent, multi-version,HA, scalable)
• No need to care about servers, load-balancers, firewalls, backups, etc.
• The environment is constantly managed and monitored
• Scalability happens in realtime
• Integrated failover/HA
• We do « Ops », you do « Dev »
Deploy app to traditional Java platform Deploy app to CloudBees
DEPLOY
©2013 CloudBees, Inc. All Rights Reserved
54
Value à la carte – NO FRICTION
OURYOUR TEAMYOUR PROJECT
©2013 CloudBees, Inc. All Rights Reserved
55
Typical Mobile App with Java back-end
???CustomMobile
Back-end
Fast-IterationDiscovery
StableNO-discovery
©2013 CloudBees, Inc. All Rights Reserved
56
Continuous Cloud Delivery at CloudBees
Jenkins Core
Jenkins Plugins
Staging
Production
Web Console
SDK
API
Identity
Provisioning
Monitoring
Messaging
Services Platform
Generic App Container
Database
Router
Curated Stack
Git
SVN
Forge
Unit Tests
StageCross Service Tests UAT Prod
As needed, multiple
times per day
©2013 CloudBees, Inc. All Rights Reserved
Is this real?!?
©2013 Cloud Bees, Inc. All Rights Reserved
58
• Lose it!– Mobile application– >> millions of users– > 30,000 transactions per minute, at peak time
• And… only 4 employees and …2 developers!– No IT, no servers, no DevOps– Complete focus on SOFTWARE DEVELOPMENT– Anything else is handled by CloudBees
• Unmatched productivity level!– This is possible TODAY– Will you wait for your competitors to shoot first?
Example: Lose it!
©2013 CloudBees, Inc. All Rights Reserved
59
Yeah, but the cloud is just for startups!
0
2
4
6
8
10
12
Estimated Amazon Web Services Revenue ($B)
Source: Macquarie Capital; Morgan Stanley
©2013 CloudBees, Inc. All Rights Reserved
60
… Or not!
0
2
4
6
8
10
12
Estimated Amazon Web Services Revenue ($B)
Source: Macquarie Capital; Morgan Stanley
88% CAGR(2009-2012)
FY12: VMW: $4.3B, RHT: $1.1B Dell Server, Storage, Networking: $11B
©2013 CloudBees, Inc. All Rights Reserved
61
• This is « the industrialization of IT »• Went there, done that (many times):
« I am against the cloud »
Specialisation, standardisation, regulation, etc.
economy of scale, higher availability, etc.
… unstoppable!
©2013 CloudBees, Inc. All Rights Reserved
First Demo• Complete Java Server Application…
• (Backbone.js front-end, REST back-end, DB)
• … with full lifecycle automation …• (Code, build, test, deploy)
• … and do it in a single click!
ClickStart
63
cloudbees.com
What’s up with this new framework!
Get started in a snap! “ClickStart”
CloudBeesGit
Repo
Database
Build/Testdb:create
App
PartnerServicePartnerService
CloudBees API
Grand Central Web
Console
app:create
gi t cl one Jenkins job create
Bind
Bind
Trigger
Deploy
Wow! At yourservice
©2013 CloudBees, Inc. All rights reserved
64
cloudbees.com
Continous Delivery at work
CloudBeesGit
Repo
Database
Build/Test
App
PartnerServicePartnerService
Bind
Bind
TriggerDeployI want to make some
changes!
Local GitRepo gi t cl one
gi
t
co
mm
it
gi t push
Ch
an
ge
s
Wow! At yourservice
©2013 CloudBees, Inc. All Rights Reserved
#fail #fail #fail
Mobile!
Code Build App Test Beta
DeployGADeploy
• Get started fast!• Team Work• Automatically
push new code to test
#fail
• Build on several targets: iOS, Android, etc.
• If it fails, stop
• Test is costly!• Easy-frst, then
increase!• Basic test• Multi-device• Gesture• Etc.
• If it fails, stop
• Deploy in the feld on real users
• Gather input• Retrieve dumps• Etc.• If it fails, stop
• D-Day! • Push apps to
real-life users!• If it fails, stop
Build Matrix• Testing mobile application requires testing on lots
of different devices, form factors, etc.
• Create one build job per form factor/generation?
• No!
• Jenkins Build Matrix!
Jenkins Build Matrix
http://wiki.cloudbees.com/bin/view/DEV/Matrixed_Jenkins_Android_Builds
So much more!
Getting started• Lots of new things to learn!
• (delegation of trust, access to legacy data, latency, elasticity, iterativity, etc.)
• Identify an application
• Start small & Non-critical
• Focus on « Learning »
• And extend from there
• (and enjoy!)
70
• We are entering a SERVICE-era• The Cloud == the commoditization of IT
– Read: it is inevitable• Packaged software is dead
– Too long cycles, no ability to measure, etc.• How can you remain competitive?
I’m ready to deploy! Are you?
TIME TO WAKE-UP!
©2013 CloudBees, Inc. All Rights Reserved
Thank you
@SachaLabourey
“It is easier to ask forgiveness than it is to ask permission”-- Grace Hopper