how do you implement continuous delivery? part 4: automated testing
TRANSCRIPT
HOW DO YOU IMPLEMENT CONTINUOUS DELIVERY?
Part 4: Automated testing
Share this ebook.
PERSPECTIVES
IMPLEMENTING AUTOMATED TESTING
2
Agile Testing QuadrantsImplementing
Automated Testing support programming critique the product
tech
nolo
gy f
aci
ng
busi
ness
faci
ng
http://www.exampler.com/old-blog/2003/08/22/#agile-testing-project-2 http://lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/
Agile Testing QuadrantsImplementing
Automated Testing support programming critique the product
tech
nolo
gy f
aci
ng
busi
ness
faci
ng
http://www.exampler.com/old-blog/2003/08/22/#agile-testing-project-2 http://lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/
functional tests prototypessimulation
showcasesexploratory
testingusability testing
performance testsload tests
security tests
unit testscomponent tests
system tests
Balance your tests
Agile Testing QuadrantsImplementing
Automated Testing
Software Testing Pyramid Implementing
Automated Testing
Unit tests
Service tests (API,
Integration, Component)
http://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid
End-to-endBusiness facing
LocalizedTechnology
facing
UI tests
Manual
Tests
Software Testing Pyramid Implementing
Automated Testing
Tests at the bottom of the pyramid
focus on smaller sections of code, e.g.
unit tests.
These tests are the foundation of a
good test automation strategy, they
are quick to run and there should be
many of them.
They run at the earlier stages of the
pipeline.
Unit tests
Service Tests
Tests in the middle of the pyramid cover
larger aggregation of code -
components, services, etc.
Service tests provide many advantages
of end-to-end tests while avoiding UI
complexities.
They run only after the build has passed
unit level tests.
Implementing
Automated Testing
Software Testing Pyramid
Tests at the top cover the "full stack”
and are the slowest to run.
Don’t write a test for every acceptance
criteria (antipattern), instead use a few
journeys to cover main areas of the
code.
They run only after the build has passed
both the unit level and service level
tests.
UI tests
Implementing
Automated Testing
Software Testing Pyramid
Testers and developers should collaborate to write,
run and maintain tests.
Siloed testing where development hands over tests to
QA not only creates long feedback loops, but also
leads to testers duplicating automated tests with
manual tests.
Expensive automated testing tools tend to make the
feedback loop worse. Developers should be able to
run all tests, including performance tests, to help
them reproduce and diagnose any issue reported by
QA.
Implementing
Automated Testing
Working Practices
Anti-Pattern: Ice-cream cone
http://watirmelon.com/2012/01/31/introducing-the-software-testing-ice-cream-cone/
Unit tests
Service tests (API, Integration,
Component)
UI tests
Manual Tests
Implementing
Automated Testing
Anti-Pattern: Ice-cream cone
x Avoid inverting your test pyramid
x Testing like this through the UI is slow and leads to brittle
tests.
x Avoid using only a UI-oriented testing tool, as that
focuses effort on writing UI-level automated tests.
x If a bug is found by users, manual testing or high level
testing, push a test to catch that lower down the
pyramid.
x The only tests at a given level should be to test
something that can't be caught at a lower level, i.e.
when testing multiple components together, your tests
should only check component integration, not each
component. That should have be done by lower-level
tests.
Implementing
Automated Testing
Flaky tests
Tests take too much work to maintain
Too much effort to add tests for legacy
codebases
ChallengesImplementing
Automated Testing
Test Data
Types of test data
Test-specific data: This is the data that drives
the behaviour under test. It represents the
specifics of the case under test.
Test reference data: Data that needs to be there
but actually has little bearing upon the
behaviour under test.
Application reference data: Irrelevant to the
behaviour under test, but needs to be there to
allow the application to start up.
Test Data
Rather than using database dumps, use the application’s API to set up the correct state.
Don’t use production data.
Avoid dependencies between tests.
Best practicesTest Data
DEPLOYMENT PATTERNS
17
Stay tuned for part 5
LEARN MORE
Deploy a great product faster.Agile teams deliver working software early and often.Go automates and streamlines the build-test-release cycle for worry-free, continuous delivery of your product.
Share this ebook.
Visit our Continuous Delivery Channel for more posts like this.