lessons learned from new relic's devops journey
TRANSCRIPT
Lessons Learned from New Relic’s DevOps Journey
2
Housekeeping
Confidential ©2008-15 New Relic, Inc. All rights reserved.
Today’s webinar is being recorded. A link to the
recording will be emailed to all registrants shortly after.
Please be sure to ask any questions via the “questions panel”
There will be a brief survey after the webinar for any additional questions or
feedback.
©2008-15 New Relic, Inc. All rights reserved.
Devops IS NOT:▪ A job or team title▪ A particular process▪ An end to itself▪ A technology or set of tools▪ About getting to "devops"
Devops IS:▪ A methodology▪ A mindset▪ An organizational attitude▪ About getting to "yes"▪ A way of organizing teams
What even is a devops?
5
©2008-15 New Relic, Inc. All rights reserved.
● Devops IS:o A word for a collection of org chart and cultural changes that
address the unique demands of building large consumer software projects today.
What even is a devops?
6
©2008-15 New Relic, Inc. All rights reserved.
Developers: seek change to ship releases and achieve product goals
Natural Conflict
Operations: seeks stability to improve uptime and sleep
7
©2008-15 New Relic, Inc. All rights reserved.
Natural Conflict
Operations, Test, and Support: Seek stability and risk reduction
Product, Business, and Developers: Seek growth and change
8
©2008-15 New Relic, Inc. All rights reserved. 9
©2008-15 New Relic, Inc. All rights reserved. 10
©2008-15 New Relic, Inc. All rights reserved. 11
©2008-15 New Relic, Inc. All rights reserved.
Stage 1: The Beginning (20-100 engineers)
Pre-history:▪ Traditional IT Operations group primarily concerned with servers,
telephones, and networking.▪ Operations was on call for everything, but didn’t do deployments
or get too concerned with the needs of the applications.
Site Engineering:▪ No more phones or offices, just focused on application delivery and
reliability.▪ If we were “devops”, it was because we were programmers doing
operations.
12
Set the right pattern early
Infrastructure is a Force Multiplier
©2008-15 New Relic, Inc. All rights reserved.
Stage 2: Growth (100-300 engineers)
Steady increase in scale and tempo.▪ More frequent deployments to reduce engineer time cost (if it’s
hard, do it more often)▪ Teams placed on call for their own services, with easy escalation
back into Site Engineering
15
Oncall IS Operations Ownership
©2008-15 New Relic, Inc. All rights reserved.
Protect Your Sleep!
Track your incidents and ruthlessly squelch noisehttps://github.com/newrelic/insights-about-pagerduty
Manage your incidents with a process everyone can follow:
http://www.ocadotechnology.com/our-blog/articles/how-we-set-up-chatops-within-slack
Leaven your Product Teams
©2008-15 New Relic, Inc. All rights reserved.
Distributed Expertise
Not just more data and services, too many more▪ Too many to have in one Site Engineering group ▪ Matrix-managed a team of SREs into Forward SRE positions within
product groups
19
©2008-15 New Relic, Inc. All rights reserved.
Stage 3: Shuffling the Deck (500+ engineers)
Site Reliability Engineer is now a peer title to Software Engineer▪ Teams can hire and allocate SRE positions as they like▪ SREs naturally bunch up in operations-focused groups, but many
teams have them▪ SRE alignment driven by interest group activities rather than
reporting
20
Total Ownership with Contract Points
©2008-15 New Relic, Inc. All rights reserved.
Biological Complexity
Each team should be a small, full-ownership group whose operational engagement extends as far down the stack as necessary to make their product work. It tries to capture the energy and rapid-response stance of a startup team.
22
©2008-15 New Relic, Inc. All rights reserved.
Q&A
Please use the questions panel in your webinar client.
©2008-15 New Relic, Inc. All rights reserved.
Continue the conversation!
Visit our Online Technical Community at
https://discuss.newrelic.com/c/nerdvana/devops-chat
For more information, see our DevOps hub:
https://newrelic.com/devops