modelling devops & seeing the system
Post on 15-Jan-2015
14.214 Views
Preview:
DESCRIPTION
TRANSCRIPT
Modelling
Devops
http://farm6.staticflickr.com/5291/5540608712_a041894564_z.jpg
@patrickdeboishttp://jedi.be/blog
This presentation๏Devops ‘Models’ in the wild
๏Seeing the system๏‘Monitoring Stream Mapping Technique’๏Observations
๏Focus on Flow๏Identifying 4 areas of improvement๏Example practices
models to explain devops
DEV OPS
Paul Peisner
http://dev2ops.org/2012/09/use-devops-to-turn-it-into-a-strategic-weapon/
Damon Edwards
https://cacoo.com/diagrams/uapwdcN6SDfwClDY-351A0.png
Mathias Marschall
An emerging set of practices
http://www.slideshare.net/dev2ops/you-cant-change-culture-but-you-can-change-behavior-and-behavior-becomes-culture
GeneKim
1. “Seeing”the System
Value Stream Mapping
“Monitoring”Stream Mapping
aka Reverse Value Stream Mapping
• Cross-Silo Group exercise
• (both dev, test, ops ...)
• Use a big wall
• preferred near their officespace
• preferred permanent
• Whiteboard/Long Paper for drawing
• Post-it with different colors
Exercise Preparation
ProductionDraw 4 quadrantsStart upper right
ProductionPut all production
components on the board
ProductionEach component =
1 post-it
Production
Components (architecture)
Put all production
components on the board
“I don’t know”can be a great conversation
starter
Production
Components (architecture)
External
Add components outside your control your team depends
outside your company (like cloud services)
Production
Components (architecture)
External
Add components outside your control
your team depends on inside your company(like firewall, dns, mail,
ldap)
Production
Components (architecture)
Supportive
Add the supportive components
(backup, monitoring...)
Production
Components (architecture)
Explain how these components interact
(architecture)
Production
Components (architecture)
Explainhow you know they are working correctly
“monitoring”
Production
EndUserComponents (architecture)
Add ‘core’ functionalities you’re providing to the
end-user
Production
EndUserComponents (architecture)
Map the functionalities to your components
(is your monitoring covering all components?)
Production
People (process)
EndUserComponents (architecture)
Now repeat the processfor people/groupsinvolved in support
Production
People (process)
EndUserComponents (architecture)
Add all groups/people involved for support
Production
People (process)
EndUserComponents (architecture)
External
Add all externalgroups/people involved
for support
Production
People (process)
EndUserComponents (architecture)
Supportive
Add all supportivegroups/people involved
for support
Production
People (process)
EndUserComponents (architecture)
Explain how these people interact
(process)
Production
People (process)
EndUserComponents (architecture)
How do you the process is working well?
(KPI’s)
Production
People (process)
EndUserComponents (architecture)
Relate the people/groups
to each of the components(who get alerts)
Production
Components (architecture)People (process)
EndUser
Add the enduser support
services you offer
Production
Components (architecture)People (process)
EndUser
Relate enduser supportto the process and
to the components to the functionalities
“To fix something we know we have to
go through development
process”
Production
Components (architecture)People (process)
EndUserBusiness
Enter Project Mode
Production
Components (architecture)People (process)
EndUserBusiness
Repeat mappingfor people/process
initiate improvements in production
Production
Components (architecture)People (process)
EndUserBusiness
Repeat mappingfor people/process
initiate improvements in production
Production
Components (architecture)People (process)
EndUserBusiness
Repeat mappingfor people/process
initiate improvements in production
Production
Components (architecture)People (process)
EndUserBusiness
Repeat mappingfor people/process
initiate improvements in production
Production
Components (architecture)People (process)
EndUserBusiness
Repeat mappingfor people/process
initiate improvements in production
Production
Components (architecture)People (process)
EndUserBusiness
Repeat mappingfor people/process
initiate improvements in production
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
Repeat mapping for
dev/test components
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
Repeat mapping for
dev/test components
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
Repeat mapping for
dev/test components
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
Repeat mapping for
dev/test components
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
Repeat mapping for
dev/test components
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
“Seeing the devops system”
a first step ofevolving
“commonground”
Some Observations
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
More maturepeople consider
dev/test/qa as part of production
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
More maturedev/test/qa/prod
components are the same
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
More mature= Less ????
about end-user functionalities
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
More mature= > monitoring/
support for external/cloud
services
2. Focus on Flow
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
Remove friction
http://philippe.kruchten.com/2013/11/24/friction/
“Friction: the resistance that one surface or object encounters when moving over another.” [Merriam-Webster dict.]
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
Find your bottleneck(s) = friction
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
Technical Debt
Social Debt
OPSDEV
http://www.jedi.be/blog/2012/05/12/codifying-devops-area-practices/
4 areas of improvement
OPSDEV
Area 1: Extend delivery to production
http://www.jedi.be/blog/2012/05/12/codifying-devops-area-practices/
OPSDEV
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
http://www.jedi.be/blog/2012/05/12/codifying-devops-area-practices/
OPSDEV
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
http://www.jedi.be/blog/2012/05/12/codifying-devops-area-practices/
OPSDEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
http://www.jedi.be/blog/2012/05/12/codifying-devops-area-practices/
Tools Can you ‘technically’ do it
Process Should you do it
People(culture)
Will you do it
“Layers per Area”
Think ‘tags’ of things you do in an area
Area X
“Area Maturity Level”a way to quantify
your progress
http://groups.google.com/group/devops/browse_frm/thread/f3de603a4cea493e?scoring=d&
Initial Unpredictable poorly controlled and reactive
Managed Focused on project, often and reactive
Defined Focused on organization and proactive
Quantitatively Managed Measured and controlled
Optimizing Focus on Improvement
CMMI - Maturity Levels (Process centric)
Intro Using Source Control ...
Novice Builds Triggered by Commit ...
Intermediate Automated Deployment to Testing ...
Advanced Automated Functional Testing ...
Insane Continuous Deployment to Prod ...
Alternative Maturity Levels(cfr. Continuous Integration Model)
http://blogs.urbancode.com/continuous-integration/continuous-integration-maturity-model/
Name AreaAreaAreaArea
Provision dev/test and prod from the same src
DEVDEV delivery to ProdEmbed Project
knowledgeProvision dev/test and prod from the same src Embed Operations
knowledgeEmbed Operations
knowledgefeedback from Prod OPS
Tools IntroLayer Level
Practice: Use a configuration mangement system like chef/puppet to provision dev,test and prod from the same source
Pattern: Automation, reuse of code
Principles: By reusing the code it gets tested more && often more frequent/earlier feedback
Production
Components (architecture)People (process)
Dev, Test, QA
EndUserBusiness
Different places where we can improve
http://devopsdays.org/blog/2010/05/16/the-panel-experiment-and-ignite-devops/
Andrew Schaefer
Some examplepractices
always understand the context before applying
http://www.jedi.be/blog/2013/05/24/Infrastructure%20as%20Code/
cfengine, puppetchef, ansible, salt, ...
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
Infrastructure as code
http://www.jedi.be/blog/2013/05/24/Infrastructure%20as%20Code/
cfengine, puppetchef, ansible, salt, ...
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
Infrastructure as code
Using same technologyversioning/testing/...
http://www.jedi.be/blog/2013/05/24/Infrastructure%20as%20Code/
cfengine, puppetchef, ansible, salt, ...
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
Infrastructure as code
Using same technologyversioning/testing/...
Executable ‘documentation’
http://www.jedi.be/blog/2013/05/24/Infrastructure%20as%20Code/
cfengine, puppetchef, ansible, salt, ...
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
Infrastructure as code
Using same technologyversioning/testing/...
Executable ‘documentation’
Common language
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
http://www.hashicorp.com/
Reproduceableenvironments
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
http://www.hashicorp.com/
Reproduceableenvironments
Dev/Test machines are using the same setup as
Production
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
http://www.hashicorp.com/
Reproduceableenvironments
Dev/Test machines are using the same setup as
Production
When your app requires changes on the system you can try them and change them ourself
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
http://www.hashicorp.com/
Reproduceableenvironments
Dev/Test machines are using the same setup as
Production
When your app requires changes on the system you can try them and change them ourself
Ops can propagateproduction changes to
dev easily
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
Monitoring & Metrics
#monitoring{love,sucks}
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
Monitoring & Metrics
#monitoring{love,sucks}
Enable devs to see the production usage/
problems(graphite)
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
Monitoring & Metrics
#monitoring{love,sucks}
Enable devs to see the production usage/
problems(graphite)
Devs can add metrics in their code(Statsd)
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
Monitoring & Metrics
#monitoring{love,sucks}
Enable devs to see the production usage/
problems(graphite)
Devs can add metrics in their code(Statsd)
Make non-subjectivedecisions at the start of
the project
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into OperationsDevelopers
wearing pagers
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into OperationsDevelopers
wearing pagers
Devs you have run operations learn and
take it back to project
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into OperationsDevelopers
wearing pagers
Devs you have run operations learn and
take it back to project
Devs know the app better and can teach/
train operations people
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into OperationsDevelopers
wearing pagers
Devs you have run operations learn and
take it back to project
Devs know the app better and can teach/
train operations people
Better Tests to preventproduction problems/
pain
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
https://github.com/Netflix/SimianArmy
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
https://github.com/Netflix/SimianArmy
Experience problemsearlier in the pipeline
(less costly)
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
Visualize frequency of production errors and enable extraction of
‘context’
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
https://speakerdeck.com/jnewland/chatops-at-github
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
https://speakerdeck.com/jnewland/chatops-at-github
Shared view on the system events
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
(Public)Post-Mortems
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
Objective viewon problems
(think retrospective)
(Public)Post-Mortems
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
Game Days
http://www.youtube.com/watch?v=LCZT_Q3z520&noredirect=1
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
Find bottlenecks in procedures/technology
Game Days
http://www.youtube.com/watch?v=LCZT_Q3z520&noredirect=1
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
SadOps
http://github.com/railsmachine/sadops
Training through simulation
OPS DEV
Area 4: Embed Operations knowledge into Project
Area 2: Extend operations feedback to project
Area 1: Extend delivery to production
Area 3: Embed Projectknowledge into Operations
SadOps
http://github.com/railsmachine/sadops
“You build ityou run it”
Amazon
DEVOPS
3. Find Feedback loops
Caveat: DifferentiateSymptoms vs real cause
Your bottleneck will move
is it even in Dev & Ops ?
MGMT FIN SALESHR
Business CustomerOPSDEV
Beware of the DEVOPS WIZARDSthat don’t know your context
Devops is not just about tools
or automation
Understanding the systemneeds to be done together
Takeways
Everbody should care about the whole system
4. Look forContinuous
Improvement
Places to look for ideas
Places to look for ideas (2)
?
top related