Is your Gatekeeper Locking out Quality?
Amy Phillips
Songkick
Songkick - Track your favorite artists and never miss them live
Amy Phillips - Songkick - @itjustbroke
Amy Phillips - Songkick - @itjustbroke
More than 500,000 people across the UK and Europe signed up
235,000 tickets sold
53,000 scalpers identified
A story about moving to frequent releases
Amy Phillips - Songkick - @itjustbroke
The problem with releases
• Pressure to deliver
• Tight timeframes
• Often infrequent
Amy Phillips - Songkick - @itjustbroke
A typical release
Code Release candidate
Bug ReportBug
ReportBug
Report
Developer Build Manager Tester Ops Engineer
Release to Production
Sign off
Business Rep
Amy Phillips - Songkick - @itjustbroke
But is this going to deliver quality?
Amy Phillips - Songkick - @itjustbroke
What is Quality?
Do you consider LinkedIn to be a quality website?
Amy Phillips - Songkick - @itjustbroke
Some comments on LinkedIn’s quality
Hardly any problems with it, no glitches. Nice UX and usability.
Bad UI, too many crashes and bugs. Overloaded with features.
It meets my needs but can be a hassle to use at times.
I have found it very buggy lately
Amy Phillips - Songkick - @itjustbroke
What is Quality?
“Quality is value to some person who matters“
- Jerry Weinberg
Amy Phillips - Songkick - @itjustbroke
What is Testing?
Tested = Checked + Explored
“You’re not done testing until you’ve checked that the software meets expectations and you’ve explored whether there are additional risks.”
- Elisabeth Hendrickson
Amy Phillips - Songkick - @itjustbroke
The Gatekeeper process
Can I commit? Can I deploy? Can I release?
Automated integration test and build
Regression test Monitor
Release Commit
Code
Amy Phillips - Songkick - @itjustbroke
The Gatekeeper process
Can I commit?Can I deploy? Can I release?
Automated integration test and build
Regression test Monitor
Release Commit
Code
Who broke the
build?
When will testing be
finished?
How important is this
bug?
When will this be released?
It doesn’t
work
Amy Phillips - Songkick - @itjustbroke
What are we trying to achieve with releases?
Releases when we want them.
Deliver quality products to users.
Without the unexpected surprises of downtime, bugs, unhappy users.
Amy Phillips - Songkick - @itjustbroke
A common problem
Releases require people from different teams to come together to make them happen.
We release something that doesn’t meet the requirements.
Somebody wants reassurance that the release will be successful.
Technical limitations make releases more difficult.
Amy Phillips - Songkick - @itjustbroke
The Songkick Technology Team
Developers
Testers
Tech Ops
Designers
Product Managers
Cross functional teams, Agile approach
Service Oriented Architecture
Ruby on rails, Go, PHP, JavaScript, Java, C#, Swift
Amy Phillips - Songkick - @itjustbroke
The Songkick Gatekeeper process
Automated integration test and build
Manual testing of feature and release Release Commit Code
CI Server Test Environment
~1 hour ~2 hours
Amy Phillips - Songkick - @itjustbroke
Fix the right problem
Step 1 - Identify the single biggest (perceived) bottleneck. Remove it.
Step 2 - Identify the next biggest (perceived) bottleneck. Remove it.
Repeat until you run out of bottlenecks.
Amy Phillips - Songkick - @itjustbroke
A DevOps Approach
Automated integration test and build
Automated release testing
Release Commit
Code
CI Server Test Environment
~20 minutes
Test Test Monitor
Developer Environment Production
Amy Phillips - Songkick - @itjustbroke
The result: Releases month on month
0.00
28.0056.00
84.00
112.00140.00
Category Title
Mar Jul Nov Mar JulAmy Phillips - Songkick - @itjustbroke
In Summary
Locking down a release process and introducing sign offs doesn’t guarantee quality.
Good quality doesn’t just mean bug-free.
Collaborative nature of DevOps can remove the need for gatekeepers AND improve quality.
Amy Phillips - Songkick - @itjustbroke
Songkick is hiring
songkick.com/jobs
Amy Phillips - Songkick - @itjustbroke
Thanks!
Amy Phillips - Songkick - @itjustbroke