automated testing - back to the roots
TRANSCRIPT
goodies
levels
crosstab
lessons
✓ Product quality
Frequency and severity of defects
✓ Documentation
Business requirements
Technical specification
The authorization page
✓ should be at the correct url (3531ms)
✓ should display client_not_specified if client_id is missing as a parameter (2818ms)
✓ should display invalid_redirect_uri if wrong redirect_uri is sent as a parameter (2796ms)
✓ should display invalid_redirect_uri if client does not have a redirect_uri configured (2789ms)
✓ should redirect to the login page if the user is not logged in (2434ms)
✓ should redirect back to client with an access token on successful login (6149ms)
✓ should go back to client, when clicking on browser back button after successful login (5674ms) When the user is already logged in
✓ should redirect back to client with an access token (1975ms)
✓ should redirect back to client with an id_token if the requested scope includes openid (2036ms)
✓ should redirect back to client with an id_token that contains the provided nonce (2150ms)
✓ should redirect back to client with a code as a query parameter (2100ms)
✓ should redirect back to client with a code and id_token if response_type includes both (2232ms)
✓ should go back to client, when clicking on browser back button (2448ms)
✓ should redirect to the login page if the login state has expired (4210ms)
✓ Continuous integration and deployment
Rapid cycles = early feedback
Smaller scope = smaller risk
Effort per feature
Time
✓ Refactoring
Gerard Meszaros (2003)
✓ Efficiency
Category Business value
Product quality Frequency and severity of defects
Continuous integration and deployment Rapid cycles
DocumentationBusiness requirements Efficiency of team communication
Technical specification Effort for understanding the system
Development
Enables refactoring Cost of new features and bugfixes
Modular design Cost of new features and bugfixes
Reduce interactive debugging Cost of new features and bugfixes
Component2 Component3
C3
moc
k
Component1
Unit1 Unit2
component test
unit testU
2 m
ock
unit test
manual exploratory test
end-to-end test
acceptance test
minutes / hours
~10 s / ~ 10 min
~10 ms / ~ 10 s
~30 s / ~ 2 min
✓ goodies
✓ levels
crosstab
lessons
Production bug
prevention
Continuous integration
and deployment
Documentation Development support
Business requirements
Technicalspecification
Enables refactoring
Flexible design
Reduceinteractive debugging
Exploratory ■
Prod-Smoke ▪ ■
Acceptance ▪ ■ ■
End-to-end ▪ ■ ▪
Component ▪ ■ ■ ■
Unit ▫ ■ ■ ▪ ■ ■
Unit tests - focus on describing the intention
Coverage 100%
Atomic
Independent
Aggressive maintenance
Critical code review
Production bug
prevention
Continuous integration
and deployment
Documentation Development support
Business requirements
Technicalspecification
Enables refactoring
Flexible design
Reduceinteractive debugging
Unit ▫ ■ ■ ▪ ■ ■
Component tests - focus on enabling refactoring
Cover 100% interface
Avoid referring internals
Mock dependencies
Production bug
prevention
Continuous integration
and deployment
Documentation Development support
Business requirements
Technicalspecification
Enables refactoring
Flexible design
Reduceinteractive debugging
Component ▪ ■ ■ ■
Recap
- Code review is your only chance for testing the readability of the specification
- Don’t let unit tests pile up and rot
- Acceptance tests and contract tests for business requirements tracking
- Unit tests for technical specification
- Maintain component tests interface-coverage
Why not?
References
Meszaros G. (2007). xUnit Test Patterns: Refactoring Test Code. Addison Wesley
http://xunitpatterns.com/~gerard/xpau2003-test-automation-manifesto-paper.pdf
http://xunitpatterns.com/~gerard/xunitTutorialSlides-V1.pdf
Clemson T. (2014). Testing Strategies in a Microservice Architecturehttp://martinfowler.com/articles/microservice-testing/