nltestdag_20161118-b
TRANSCRIPT
November 18th, 2016
Testing without Requirements
Sr. Test ConsultantALTEN Technology
André VERSCHELLING
Testing without Requirements
• Over the past few years in teaching and consulting with testers and test managers worldwide, I have noticed something interesting. On one hand, testers complain they hardly ever get user requirements adequate for testing. On the other hand, when discussing what to base tests upon, the main response is "user requirements.“
• Randall W. Rice, CSTE, CSQA, CTFL
• http://www.riceconsulting.com/articles/testing-without-defined-requirements.htm
2
Why do you need requirements?
• To fulfill your test strategy
3
Risk analysis
Test Strategy
Test Approach
Test Design
Test Execution
Selection of thoroughness of test
Selection of coverage to achieve thoroughness
Selection of test design technique
Test cases
You’ll need a test basis (a/o
requirements)
When you have no requirements
• Ask questions about the system / software under test
• Anticipate on common risks / issues
• Let the user perform acceptance tests
• Random testing
• Evaluate against common checklists
• Define user scenarios
• Look for applicable standards
Others will tell you the same
4
When you have no DEFINED requirements
• Ask questions
• Use training material
• Use service manuals
• Read through mails, sticky notes, whiteboard captures, etc.
• Compare with the previous version
You all do, don’t you?
5
What questions do you ask …
… when you have to test something like these?
6
What level …
• … do your questions address?
7
System
Sub system
Component
Unit
•Do we fulfil the intended use?
•Did we built the right system?
User needs
•Did we capture all stated and implied needs?
•Did we built the system right?Quality
•Is the system / product functioning conform the requirements?
Functionality
•Is the system product robust and reliable?
Integrity
≠
Let’s grow a tree
8
System Function Tree
System
Sub-system 1
Main function 1
Main function 2
Sub-system 2
Main function 3
Sub-function 3.1
Sub-function 3.2
Main function 4
9
System Function Tree
• Not exactly rocket sciences
• So prepare:
1 Participant per discipline (Sys.Arch., SW Arch., El.Eng., Mech.Eng.,Test,
Norm Compliance, Service, etc.)
Create initial vision on paper
Only describe systems, functions, features, not implementation
Leveling does not define or determine format
Prepare 5 minute pitch for meeting
10
System Function Tree
• Meet
Present vision
Elect formatRemember: Leveling does not define or determine format
Decide on type and level of detail required
11
System Function Tree
• Follow-up
Unambiguous description of system, function, sub-function, etc.
What is the aim? (customer, user perspective)
What inputs are required?
How is this function controlled?
What factors can disturb it?
Use description understandable by all disciplines
12
System
Noise factors•
•
Control factors•
•
Inputs
Error states•
•
Ideal function
e.g. P-diagram
13
System Function Tree example
Next steps
14
Risk analysisTest
StrategyTest
ApproachTest
DesignTest
Execution
Selection of thoroughness of test
Selection of coverage to achieve thoroughness
Selection of test design technique
Test cases
A/o based upon noise factors and
error states
A/o using inputs and control
factors as test basis
Grow your tree
• And continue test business as usual
Assess risks
Define a strategy and approach
Use Test Design Techniques!
Use Test Types!
and all the other stuff
15
