Встреча quality built in @spotify от Андрея Дзыни : как стать...
TRANSCRIPT
December 6, 2014
Andrii Dzynia Road Manager to Continuous Delivery
@adzynia
Quality Built In
2
Spotify brings you the right music for
every moment!
Started in 2006 (in Sweden) Now 1500+ employees, 500+ engineers Over 30 million songs available Over 20,000 songs added every day 5 development centres across the globe
What is Quality?
3
4
Not just software defects, but stakeholders expectations as well
when expectations matches the reality.
Quality is a “state” ….
Why Quality is Important?
5
Cost of Bug/Change
6
What is a way to build high quality product?
7
8Do not release until it’s perfect
9
No one is perfect.
Implementing internal back office system
Example
11
Step 1 - Prototyping phase
Technical spike investigation Hacking different Back-End solutions
Ad-hoc design discussions Stub implementation of Front End
Getting early user’s feedback on Front End
12
WebApp with
stubs
Whatever works to get end-user understanding
Sketches on
whiteboard
13
Step 2 - Setting delivery pipeline
Add more unit tests Add more integration tests Add more functional tests Automate deployment configuration Add dashboards and system monitoring
Clarifying End User Needs
14
http://blog.crisp.se/2013/02/05/yassalsundman/continuous-delivery-vs-continuous-deployment
15
Step 3 - Connecting the dots
Connect back-end and front-end parts into
End-2-End system
Get user’s feedback and iterate
Keep releasing small, usable, incremental pieces16
17
What is under the hood ?
18
20
Test Automation Pyramid
21
How to avoid Ice-Cream effect?
22
Be strict with this rule.
Testable micro-services architecture 23
Unit Tests
Component Tests
Functional Tests
System/End-2-End Tests
Integration Tests
Unit Tests
Component Tests
Functional Tests
JS Unit Tests
JS Component Tests
UI Functional
Tests
24
Unit Tests
Check logic of minimal code snippet with no or mocked dependencies
https://github.com/mockito/mockitohttps://github.com/junit-team/junit
25
https://github.com/jsevellec/cassandra-unit
Very much like unit tests but emulating external components
e.g. cassandra database
Component Tests
26
Client App
Fake Back-End Server
Spotify Back End
Fake back-end for client apps
http://jsonstub.com
https://github.com/azagniotov/stubby4jhttps://github.com/dreamhead/moco
27
Integration Tests
Check an integration between components
You do not need whole system to check the integrationhttps://github.com/spotify/helios/blob/master/docs/testing_framework.md
28
@docker containers
Create Test Environment ‘on the fly’
https://www.docker.com
29
Check services API behaviour or
End user use cases
Functional Tests
http://seleniumhq.orghttp://hc.apache.org
30
http://graphwalker.org
Model Based Testing
32
Code Review &
Human Testing on
Pull Request
Delivery Pipeline33
34
Was not covered during this talk …
Load Testing Gradual Rollouts A/B Testing Feature Toggles Monitoring/Alerting Information Radiators
… but stay tuned
35
Technical Test Engineer Test Engineer
! = Manual Tester ! = Test Automator
~ Software Engineer in Test
Test Engineering Roles and Responsibilities
~ Context Driven Tester
36
Technical Test Engineer
Work as a software developer Advocate testability of the product
Argue on software design with software engineers Help with building new/injecting existed development tools
37
https://github.com/jsevellec/cassandra-unit
Cassandra Unithttps://github.com/mikaellanger/job-dsl-plugin
Jenkins job-dsl-plugin
https://github.com/spotify/heliosDocker orchestration
Spoticloude.g. cli control over amazon cloud http://dashing.io
Dashboards
Development Productivity Tools
38
Knows business domain Focused on exploring the product
Free to use any programming language to test specific use case Free to use any programming language to automate his work
Test Engineer
39
Mind Map as a Tool
https://www.mindmup.com
Product tree Scenarios Playbooks Checklists
Session notes
Quality Engineers40
http://www.satisfice.com/blog/archives/1372
TEST JUMPERS: ONE VISION OF AGILE TESTING
http://www.satisfice.com/blog/archives/1364“RESPONSIBLE TESTER”
http://www.satisfice.com/articles/omega_tester.pdfOMEGA TESTER
@jamesmarcusbach calls us Test Jumpers
41
Any other ways to improve
quality?
42
test ideas during healthy discussions
test requirements via end users collaboration
discuss system design and conner cases earlier in the planing/design meetings
define definition of done
Break Uncertainty
43
But have responsible person to figure out what consensus means in each case
Use Google Docs collaboration for finding group consensus
Social Programming
44
write “checks” during implementation shape your thoughts via pair discussions peer review before merging to master
45
User Stories Planning meeting Standup meeting Open workspace TDD Refactoring etc……
Not mentioned Engineering Practices
http://www.extremeprogramming.org/rules.html
Employee Early Builds Testing
46
47
One week to do whatever you want
Do it alone or with a team
Crazy ideas comes out
Hackathons
48
QA engineer at Spotify
http://continuousdelivery.com/2014/02/visualizations-of-continuous-delivery/
How to check you are on a right path ?
49
That is easy to spot 50
51
Ready for a journey?Make Quality explicit Find Quality promoters Define your way of Quality
improvements Set right Quality constraints Share results as early as possible
Keep looking further quality improvements Probably you will never end :)
and …
Stop saying QA when you mean Testing
Check out http://www.spotify.com/jobs or @Spotifyjobs for more information
Want to join the band?