intro to agile testing
TRANSCRIPT
Introduction to Agile TestingTristan McCarthy - Lead ConsultantOpenCredo
@trismccarthy
Agenda
9:00 - 10:30 Learning
10:30 - 10:50 Break
10:50 - 12:30 Learning
12:30 - 13:30 Lunch
13:30 - 15:00 Learning
15:00 - 15:20 Break
15:20 - 17:00 Learning
2
Focus areas
●Requirements
●Defining acceptance tests
●Implementing automation
●Integrating into the development team
3
Why test
4
●Does the system do what we expect?
● Have changes affected previous behaviour?
●How does the system work?
●Is this system ready to be used?
Common understanding
Regression Testing
Test Driven Developme
nt
Acceptance Testing
Test Automatio
nBehaviour
Driven Development
Functional Testing
Continuous
Integration
UI Testing
Agile Testing
5
Requirements
6
Gathering Requirements
7
●Good testing starts with good requirements
●Capture intent and not solution
●Think about non functional requirements
●Make sure that requirements are clear
8
Requirement
Secure expensive equipment
Acceptance Criteria
All equipment must have a kensington lock connected
Test Passes
Requirement
Secure expensive equipment
Acceptance Criteria
All equipment must be physically secured to desks
Test Fails
Example: Simple Requirement
Add a new LinkedIn contact
As a userIn order to keep in touch with a previous colleagueI want to be able to add a new contact
9
Add a new LinkedIn contact
Users who wish to keep in touch with previous colleagues should be able to add another user as a contact so that they can benefit from updates and posts from that user.
Some assumptions….User account creation, authentication, updates and search functionality already
exists.Connecting with former colleagues is a unique journey with it’s own flow
compared with connecting with friends or strangers.
Practical: Create a requirement
●Choose a product owner
●Pick a well understood web based system as a basis
●Identify a feature (new or existing) to work against
●Describe the intent of the feature with a simple narrative
10
Example: Acceptance Criteria
11
Add a new LinkedIn contact
Users who wish to keep in touch with previous colleagues should be able to add another user as a contact so that they can benefit from updates and posts from that user.
Acceptance Criteria
● User should be able to ask to connect with another user when viewing a profile
● Connection should be made only after the other user has agreed
● User should see the connection in their contact list if successful
Practical: Add acceptance criteria
●Capture high level statements describing expected outcome
●Add acceptance criteria around restrictions / limitations
12
Defining acceptance tests
13
Identifying scenarios
14
●Start by defining the use case
●Identify additional scenarios by considering different entry points
●Keep test scenarios small
●Don’t try to draw 1-1 relationships between acceptance tests and
criteria
●Focus on posing questions rather than answering them
Example: Scenario list
Scenario: Connection request is approved
Scenario: Connection request is rejected
Scenario: User attempts to request again after rejection
15
Add a new LinkedIn contactUsers who wish to keep in touch with previous colleagues should be able to add another user as a contact so that they can benefit from updates and posts from that user.
Acceptance Criteria● User should be able to ask to connect with another user when viewing a profile● Connection should be made only after the other user has agreed● User should see the connection in their contact list if successful
Practical: Defining your scenarios
●Identify the “happy path” scenarios at a high level
●Extend the scenarios with negative test cases and “what if’s”
16
Test structure
Preconditions
Describe expected state at test start
Actions
Describe interactions with the system
Assertions
Describe expected outcomes
17
Introducing Behaviour Driven Development
Introduced by Dan North in 2006
Devised as a way of getting the benefits of TDD earlier in the process
Natural language layer on top of testing
Links directly into test automation
Helps when working with non technical stakeholders
Can produce “Living Documentation”
Maps standard test structure to “keywords”Preconditions = “Given”
Actions = “When”
Assertions = “Then”
18
Using Behaviour Driven Development
Preconditions, actions and assertions are all expressed as sentences with a keyword
Each sentence in a test is a “step”
Multiple steps come together to form a “scenario”
Multiple scenarios are grouped into a “feature”
Feature is not a story, multiple stories can go into a feature
19
Feature: Introducing BDD
Scenario: Explain BDDGiven a room full of peopleWhen I explain BDDThen attendees should be able to write their own BDD scenario
Example: Test StepsScenario: Connection request is approved
Given the following new users:|Bob||Jeff|And I am authenticated as user ‘Bob’When I request to connect with ‘Jeff’And ‘Jeff’ accepts my connection
requestThen I should see ‘Jeff’ in my contact
list
20
Scenario: User attempts to request after rejection
Given the following new users:|Bob||Jeff|And I am authenticated as user ‘Bob’And ‘Jeff’ has previously rejected a
connection request from meWhen I request to connect with ‘Jeff’Then I should be informed that I am
not permitted to send another request to ‘Jeff’
Practical: Capturing test steps
Choose a “happy path” scenario to expand
Describe the steps using BDD structure
Describe the preconditions as briefly as possible
Describe the actions a user will perform
Describe the assertions used to validate the functionality 21
Acceptance method
Sometimes useful to define how a story will be verified
Some functional outcomes may not be easily verifiable in the
standard testing approach
Gives clarity when team responsibilities are unclear
Enables a pragmatic approach to quality assurance 22
Test automation
23
Positives Negatives
vs
Faster feedback
Consistent results
Supports CI / CD
Integrates into Agile development
Can be easily reused and extended
Standardised reporting
Complexity
Skills gap
Lead time
Cost
Implementing automation
24
Levels of testing
●Low level unit tests
●Service / integration tests
●User interface tests
○ Note, not necessarily “Graphical User
Interface”
●Speed and value implications at each
layer
25
www.martinfowler.com
Building a test framework
26
●Collection of tools to support a number of needs○ Way of defining test cases
○ Tools for interacting with the system under test
○ Tools for making assertions
○ Mechanism for reporting of results
●Structured for re-use and maintenance
●Should be held to similar standards as development code
●Ideally uses the same development language as the system
under test
Example: An acceptance test framework
27
Growing over time
Start with just the basics
A test framework should evolve in line with the codebase
Capabilities are added to the framework only when required
Don’t be scared to refactor
28
Example: Growth over time
29
Authenticate as user
Authenticate as admin
Add a new contact
Log In User Dashboard
Admin Dashboard
User Search
Profile View
Request Connection
Managing state - why
30
All tests should be consistently repeatable
Tests should be able to run in parallel
It should not be possible for tests to be affected by other test runs
or manual testing
Tests should be focused on just the area relevant to the scenario
Managing state - how
31
Place the system in an expected state
Maintain test data per scenario
Destroy test data
Expose all capabilities in a programmatic way
Integrating into the team
32
Single team
●Testing should always be an aspect of the development team
●Automated tests should be written at the same time the code is
implemented
●Quality control must become part of the team definition of done
●All team members should be encouraged to engage with tests 33
Verifiable requirements
●Testing should be involved at every stage of the development
process
●Easier communication with demonstrable requirements
●Capturing test scenarios helps stakeholders streamline what they
want
●Scope clarity enables better estimation
34
Execution
35
Integrate automation into the continuous integration pipeline
Run against production like test environments
Use tests as deployment gates
Publish reports and make available
Example: CI /CD pipeline
36
Build Code& Run Unit Tests
Run Acceptance Tests
Deploy to QA Environment
Deploy to UAT Environment
Deploy to Production Environment
Deploy to Perf Test Environment
Commit code change
Run Performance Tests
Manual acceptance / look and feel testing
New Requirement
Developer
Tests fail
Change rejected
Tests FailTests pass
Change accepted
Defect management
●Review test reports and raise defects
●Ensure that defects make their way back into the workstream
●Manage tests based on expected outcome
●Use automation to speed up defect resolution
●Use defects to improve automation coverage 37
Further reading
Print ResourcesAgile Testing Lisa Crispin, Janet GregoryTest focused agile development, testing in a development team
The Cucumber BookMatt Wynne, Aslak HellesoyCucumber BDD tool and the ‘Gherkin’ DSL
38
Web Resourceshttps://dannorth.net/Dan North
Origins of BDD, team processhttps://gojko.net/ Gojko Adzik
Requirements gatheringhttp://sauceio.com/ Sauce LabsTesting process, automation tools
Questions
39
40
www.opencredo.com
@trismccarthy
https://opencredo.com/author/tristan/
https://github.com/tristanmccarthy