ops happens: devops beyond deployment - damon edwards
Post on 21-Feb-2017
103 Views
Preview:
TRANSCRIPT
The systemic force behind most DevOps problems? Silos.
Dev Release OperatePlanning
Handoff
!Handoff
!Handoff
!
The systemic force behind most DevOps problems? Silos.
Dev Release OperatePlanning
Application Knowledge
Handoff
!Handoff
!Handoff
!
The systemic force behind most DevOps problems? Silos.
Dev Release OperatePlanning
Application Knowledge
Operational Knowledge
Handoff
!Handoff
!Handoff
!
The systemic force behind most DevOps problems? Silos.
Dev Release OperatePlanning
Application Knowledge
Operational Knowledge
Business Intent
Handoff
!Handoff
!Handoff
!
The systemic force behind most DevOps problems? Silos.
Dev Release OperatePlanning
Application Knowledge
Operational Knowledge
Business Intent
Handoff
!Handoff
!Handoff
!
Ownership but limited Accountability
The systemic force behind most DevOps problems? Silos.
Dev Release OperatePlanning
Application Knowledge
Operational Knowledge
Business Intent
Handoff
!Handoff
!Handoff
!
Ownership but limited Accountability
Accountability but no Ownership
Dev/Test Release Operate Planning
Cross-Functional Product TeamsCross-Functional Product TeamsCross-Functional Product Teams
Common perception of Enterprise DevOps Transformations
Dev/Test Release Operate Planning
Cross-Functional Product TeamsCross-Functional Product TeamsCross-Functional Product Teams
?
Reality of most Enterprise DevOps Transformations
Reinforces siloed divide at most critical point in lifecycle
Dev/Test Release Operate Planning
Cross-Functional Product TeamsCross-Functional Product TeamsCross-Functional Product Teams
Handoff
!
Delays critical feedback loops
Dev/Test Release Operate Planning
Cross-Functional Product TeamsCross-Functional Product TeamsCross-Functional Product Teams
Prevents meaningful organizational improvement
Dev/Test Release Operate Planning
Cross-Functional Product TeamsCross-Functional Product TeamsCross-Functional Product Teams
Time to Market
Cost
Quality
Handoff
!
Enterprise Operations is under tremendous pressure
“Go faster!” “Open it up!”
“Be more secure!” “Be more reliable!”
Organizational complexity compounds the system complexity
Ticket System SysAdmin
NetEng
CDN
SysEng Ticket System
SecOps
Ticket System
DNS
PM
DBA
?I just need an environment! Something isn’t working!
Silos + Time = Islands of Automation
Puppet Chef
Shell ScriptsData ETL
PowershellScriptsNetwork
Management
Monitoring
Ansible
LegacyDatacenterAutomation
Container Management
SQL Tools
New
Tools
New
Tools
What has to happen to achieve end-to-end transformation?
Dev/Test Release Operate Planning
Cross-Functional Product TeamsCross-Functional Product TeamsCross-Functional Product Teams
2. Integrate3.Enable
Standard Platforms
Ops APIs
1. Simplify
Internal Service Providers
Dev/Test Release Operate Planning
Cross-Functional Product TeamsCross-Functional Product TeamsCross-Functional Product Teams
2. Integrate3.Enable
Standard Platforms
Ops APIs
1. Simplify
Internal Service Providers
What has to happen to achieve end-to-end transformation?
“Shift Left” as much Ops activity as possible
Writing / Running Automated Tests Writing / Exercising Deploy Automation Running Security Scanning Tools
“Deploy.”
“Shift Left” as much Ops activity as possible
Writing / Running Automated Tests Writing / Exercising Deploy Automation Running Security Scanning Tools
Writing / Exercising Automated Runbooks Writing / Exercising Monitoring/Metrics Operational Control (safely!)
“Deploy.”
“Operate.”
HOW?
“Shift Left” as much Ops activity as possible
Writing / Running Automated Tests Writing / Exercising Deploy Automation Running Security Scanning Tools
Writing / Exercising Automated Runbooks Writing / Exercising Monitoring/Metrics Operational Control (safely!)
“Deploy.”
“Operate.”
How do you “shift left” ops procedures and incident response?
Those who build something define the procedures to fix it Those who build something fix it when it breaks
1
2
But... How do you safely and securely give out access? How do you enable the experts to contribute remediations? How do you give the experts visibility into operations? How do you do postmortems days/weeks/months later?
Those who build something define the procedures to fix it Those who build something fix it when it breaks
1
2
How do you “shift left” ops procedures and incident response?
Who created the procedure? Who reviewed it? Who? When? Where? Approval trail?
Step 4: Make Compliance Really Happy
Example of these principles at work
Mark Maun
Jody Mulkey
Justin Dean
Sources: https://www.youtube.com/watch?v=_hr4KiB19bQ http://rundeck.org/stories/mark_maun.html
Ticketmaster’s “Support at the Edge” model
Example of these principles at work
Mark Maun
Jody Mulkey
Justin Dean
Sources: https://www.youtube.com/watch?v=_hr4KiB19bQ http://rundeck.org/stories/mark_maun.html
Ticketmaster’s “Support at the Edge” model• Automated Ops procedures written/vetted by the delivery teams
Example of these principles at work
Mark Maun
Jody Mulkey
Justin Dean
Sources: https://www.youtube.com/watch?v=_hr4KiB19bQ http://rundeck.org/stories/mark_maun.html
Ticketmaster’s “Support at the Edge” model• Automated Ops procedures written/vetted by the delivery teams• Ops remained in full control of what can run and security policy
Example of these principles at work
Mark Maun
Jody Mulkey
Justin Dean
Sources: https://www.youtube.com/watch?v=_hr4KiB19bQ http://rundeck.org/stories/mark_maun.html
Ticketmaster’s “Support at the Edge” model• Automated Ops procedures written/vetted by the delivery teams• Ops remained in full control of what can run and security policy• Empowered support teams with self-service ops tasks
Example of these principles at work
Mark Maun
Jody Mulkey
Justin Dean
Sources: https://www.youtube.com/watch?v=_hr4KiB19bQ http://rundeck.org/stories/mark_maun.html
Ticketmaster’s “Support at the Edge” model• Automated Ops procedures written/vetted by the delivery teams• Ops remained in full control of what can run and security policy• Empowered support teams with self-service ops tasks• Empowered developers with limited self-service operations
Example of these principles at work
Mark Maun
Jody Mulkey
Justin Dean
Sources: https://www.youtube.com/watch?v=_hr4KiB19bQ http://rundeck.org/stories/mark_maun.html
Ticketmaster’s “Support at the Edge” model• Automated Ops procedures written/vetted by the delivery teams• Ops remained in full control of what can run and security policy• Empowered support teams with self-service ops tasks• Empowered developers with limited self-service operations• Combined with new incident response model
Better for the business and better way to work
90% Reduction in MTTR 50% Reduction in escalations 55% Reduction of overall support costs
Recap
Automated Tests Deployment Automation Running Security Scanning Tools Automated Runbooks Monitoring/Metrics Operational Control
“Shift-Left” operational concernsTake the transformation
all the way to Operations
Explicit investment in process and tooling
top related