puppet camp tokyo 2014: fireballs, ice bats and 1,000,000 plugins: a story of continuous delivery
DESCRIPTION
Presented at Puppet Camp Tokyo 2014: "Fireballs, ice bats and 1,000,000 plugins: a story of continuous delivery" by Otto Jongerius, AtlassianTRANSCRIPT
Otto Jongerius Build Engineering @ Atlassian
@xzu
Fireballs, Icebats and 1,000,000 Plugins
PuppetCamp Tokyo - May 2014
Continuous Delivery (a refresher)
Develop => Build => Test => Release!
Continuous Delivery (a refresher)
Develop => Build => Test => Release!
smaller change delta
more feedbackfaster feedback
automated upgrade smaller magnitude of failure
Our happy path
0
200
400
600
800
2011 2012 2013 2014 2015
Deployments/year
quarterly fortnightly weekly
daily
many/day
A little bit of history…
A little bit of history…
0
12500
25000
37500
50000
October 2011 June 2012 January 2013 June 2013 April 2014
50.000
32.000
24.000
15.000
500
Approximate number of OnDemand instances
A little bit of history…
Atlassian Application
Application Core
Application Bundled Plugins
plugin1 plugin2
plugin3 plugin4
pluginN
A little bit of history…
Atlassian Application
Application Core
Application Bundled Plugins
plugin1 plugin2
plugin3 plugin4
pluginN
Alice
Bob
Charlie
DaveEd
FrankGerry
A little bit of history…bash$ cat jira/pom.xml … <dependency> <groupId>com.atlassian.jira</groupId> <artifactId>jira-api</artifactId> <version>${project.version}</version> </dependency> …
A little bit of history…
Atlassian Application
Application Core
Application Bundled Plugins
plugin1 plugin2
plugin3 plugin4
pluginN
Atlassian Application
Application Core
Application Bundled Plugins
plugin1 plugin2
plugin3 plugin4
pluginN
Atlassian Application
Application Core
Application Bundled Plugins
plugin1 plugin2
plugin3 plugin4
pluginN
Enter the fireball
Enter the fireballbash$ cat fireball/pom.xml … <dependency> <GroupId>${jira.group.id}</GroupId> <artifactId>${jira.war.artifact.id}</artifactId> <version>${jira.war.version}</version> … </dependency> …
Launching the fireballOpenVZ Container
Postgres
ApacheConfluence
JIRA
Bamboo
CrowdCus
tom
er In
stan
ce
NFS
Launching the fireball
Datacenter
Rack Rack Rack RackRack
Rack Rack Rack RackRack
Rack Rack Rack RackRack
Rack Rack Rack RackRack
Rack
Launching the fireball
bash$ sudo /path/to/new.version/install.sh
INSTALLED_SERVICES=“postgres jira bamboo confluence crowd apache” for service in ${INSTALLED_SERVICES}; do /path/to/new.version/$service/install.sh done
…
OR
http://www.flickr.com/photos/64577732@N00/1414883135http://www.flickr.com/photos/44124473769@N01/292642699
Launching the fireball
Right?
A little bit of history…
jira/install.sh
postgres/install.sh
confluence/install.sh
bamboo/install.sh
crowd/install.sh
* (over) simplified model
A little bit of history…
jira/install.sh
postgres/install.sh
confluence/install.sh
bamboo/install.sh
crowd/install.sh
* (over) simplified model
A little bit of history…
jira/install.sh
postgres/install.sh
confluence/install.sh
bamboo/install.sh
crowd/install.sh
* (over) simplified modelhttp://www.flickr.com/photos/64577732@N00/1414883135
A little bit of history…
http://www.flickr.com/photos/7337467@N04/3051244623
Our happy path
0
200
400
600
800
2011 2012 2013 2014 2015
Deployments/year Progress
quarterly fortnightly weekly
daily
many/day
You are here
A little bit of history…
Atlassian Application
Application Core
Application Bundled Plugins
plugin1 plugin2
plugin3 plugin4
pluginN
Alice
Bob
Charlie
DaveEd
FrankGerry
Enter the icebat
Fire + Ball
Ice + Bat
Enter the icebat
j2ee-appjira
postgres
crowd
confluence
bamboo
apache
j2ee-plugin
vendor plugin
Enter the icebat
j2ee-appjira
postgres
crowd
confluence
bamboo
apache
j2ee-plugin
vendor plugin
Enter the icebat
j2ee-appjira
postgres
crowd
confluence
bamboo
apache
j2ee-plugin
vendor plugin
Enter the icebat
j2ee-appjira
postgres
crowd
confluence
bamboo
apache
j2ee-plugin
vendor plugin
Enter the icebat
j2ee-appjira
postgres
crowd
confluence
bamboo
apache
j2ee-plugin
vendor plugin
postgres
Enter the icebat
j2ee-appjira
postgres
crowd
confluence
bamboo
apache
j2ee-plugin
vendor plugin
postgres5.4
6.1
5.0
2.7.5
1.1
hier
a - v
ersi
ons.
yam
l
Enter the icebat
Development5.4m3
6.2-snapshot
5.1-m1
2.7.5-m6
1.2.1-snapshot
hier
a - v
ersi
ons.
yam
lDogfooding Production
5.4m2.1
6.1m7
5.1
2.7.5-m5
1.2hi
era
- ver
sion
s.ya
ml 5.4m2
6.1m6
5.0-m1
2.7.5-m2
1.1-m1
hier
a - v
ersi
ons.
yam
l
Local overridesglobal configuration
datacenter configurationrack configurationlocal configuration
http://www.flickr.com/photos/streamishmc/4936051/
Hard Labour
Jason ‘Mechanical Turk’ Birch
The Great Leap Forward
The Great Leap Forward
The Great Leap Forward
Applications
The Great Leap Forward
Plugins
The Great Leap Forward
Environments
The Great Leap ForwardVersions
The great leap forward
The great leap forward
“client”
The great leap forward
“client”
facterenvironment
licensed products
The great leap forward
“client”
facterenvironment
licensed products
manifesto
The great leap forward
“client”
facterenvironment
licensed products
puppet apply
manifesto
Our happy path
0
200
400
600
800
2011 2012 2013 2014-01 2014-12 2015
Deployments/year Progress
quarterly fortnightly weekly
many/week
many/day
You are here
twice daily
Next Steps
• Onboard everything into Manifesto, such that development teams control their own destinies • Automated acceptance testing • Fault tolerance for manifesto • Dark feature (feature flags) control within Manifesto/icebat
Special thanks to…
Ryan ThomasJason BirchMatt Moor
Thanks!
Dev OpsDevs who can OpOps who can Dev
We’re hiring! atlassian.com/jobs (or speak to me)