more important than techniques - selalegacy17.sela.co.il/s/sigist12/_allpresentations/19july/...the...
TRANSCRIPT
![Page 1: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/1.jpg)
More Important than Techniques
Michael Stahl, Intel
SIGiST 2012
![Page 2: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/2.jpg)
(c) Michael Stahl, 2012, All Rights Reserved 2
Disclaimers
Names and brands referenced herein may be claimed as the
property of third parties
The views expressed in this presentation are solely my own, and do not in any manner represent the views of my employer
Information in this presentation is provided “AS IS” without any warranties or representations of any kind
![Page 3: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/3.jpg)
![Page 4: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/4.jpg)
![Page 5: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/5.jpg)
Seriously…
How to design a Test?
(c) Michael Stahl, 2012, All Rights Reserved
![Page 6: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/6.jpg)
IEEE 829-1998
(c) Michael Stahl, 2012, All Rights Reserved
![Page 7: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/7.jpg)
IEEE 829-2008
(c) Michael Stahl, 2012, All Rights Reserved
![Page 8: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/8.jpg)
System
Under Test
Setup Output
Input
(c) Michael Stahl, 2012, All Rights Reserved
![Page 9: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/9.jpg)
The ISTQB Test Techniques
Equivalence Partitioning
Boundary Values Analysis
Decision Table Testing
State Transition Testing
Use Case Testing
Experience-based Testing
Etc. etc.
Source: ISTQB, Foundation level syllabus
(c) Michael Stahl, 2012, All Rights Reserved
![Page 10: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/10.jpg)
The “standard” Test Design
Setup
Apply Input
Expected
Results
Teardown
(c) Michael Stahl, 2012, All Rights Reserved
![Page 11: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/11.jpg)
The “standard” Test Design
Setup
Apply Input
Expected
Results
Teardown
Test Techniques Equivalence Partitioning
Boundary Values Analysis
Decision Table Testing
State Transition Testing
Use Case Testing
Experience-based Testing
Etc. etc.
(c) Michael Stahl, 2012, All Rights Reserved
![Page 12: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/12.jpg)
Outline
Input and Output basics
Preconditions, Postconditions
Foreground, Background
Test flow
Test your test
Testing non-functional attributes
(c) Michael Stahl, 2012, All Rights Reserved
![Page 13: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/13.jpg)
Input
(c) Michael Stahl, 2012, All Rights Reserved
![Page 14: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/14.jpg)
Input basics: Valid
Input
Range
Enumeration
Combinations (if applicable)
(c) Michael Stahl, 2012, All Rights Reserved
![Page 15: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/15.jpg)
Input basics: Invalid
Range
Enumeration
Combinations (in rare cases!)
Input
(c) Michael Stahl, 2012, All Rights Reserved
![Page 16: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/16.jpg)
Input basics: Invalid
Invalid Combinations (in rare cases!)
(c) Michael Stahl, 2012, All Rights Reserved
![Page 17: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/17.jpg)
Input basics: Invalid
Invalid Combinations (in rare cases!)
(c) Michael Stahl, 2012, All Rights Reserved
![Page 18: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/18.jpg)
Output
(c) Michael Stahl, 2012, All Rights Reserved
![Page 19: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/19.jpg)
Output basics: Valid
Output
Range
Enumeration
Error codes or messages (c) Michael Stahl, 2012, All Rights Reserved
![Page 20: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/20.jpg)
Output basics: Invalid
Output
(c) Michael Stahl, 2012, All Rights Reserved
![Page 21: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/21.jpg)
A “Wider” look at Input:
Preconditions
(c) Michael Stahl, 2012, All Rights Reserved
![Page 22: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/22.jpg)
Input is one part of Preconditions
Environment
System under test
Input Configuration
Input
(c) Michael Stahl, 2012, All Rights Reserved
![Page 23: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/23.jpg)
Preconditions are Assumed or
Validated
Input Assumed Validated
(c) Michael Stahl, 2012, All Rights Reserved
![Page 24: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/24.jpg)
Assumed Preconditions
The system has a
printer
The user can read There IS a user
The system has the
needed fonts
(c) Michael Stahl, 2012, All Rights Reserved
![Page 25: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/25.jpg)
The Environment:
Control, Set, Cause or Monitor?
System under test
Input Configuration
Input
Environment
Control Cause Set Monitor
(c) Michael Stahl, 2012, All Rights Reserved
![Page 26: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/26.jpg)
A “Wider” look at Output:
Postconditions
(c) Michael Stahl, 2012, All Rights Reserved
![Page 27: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/27.jpg)
Output is one part of Postconditions
Output
Output
Environment
System under test
Configuration
(c) Michael Stahl, 2012, All Rights Reserved
![Page 28: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/28.jpg)
Postconditions have Variants and
Invariants
Output Invariants
![Page 29: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/29.jpg)
Post Conditions: Variants and
Invariants
(c) Michael Stahl, 2012, All Rights Reserved
![Page 30: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/30.jpg)
Creating Preconditions
Obvious:
User inputs
CLI arguments
Less obvious:
Configuration
Files, Windows registry
API calls
Sensors
The environment
OS, HW, installed stuff, interrupts
(c) Michael Stahl, 2012, All Rights Reserved
![Page 31: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/31.jpg)
Checking Postconditions
Variants:
“Expected Results”
Configurations; Environment
Invariants:
Configurations; Environment
System resources
Memory
File handles
Pay special attention after Exceptions!
(c) Michael Stahl, 2012, All Rights Reserved
![Page 32: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/32.jpg)
Checking Postconditions
Background
Known state – before the test
Foreground
Stuff that changes
(c) Michael Stahl, 2012, All Rights Reserved
![Page 33: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/33.jpg)
Checking Postconditions
After: Before:
(c) Michael Stahl, 2012, All Rights Reserved
![Page 34: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/34.jpg)
Checking Postconditions
After: Before:
(c) Michael Stahl, 2012, All Rights Reserved
![Page 35: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/35.jpg)
Checking Postconditions
Rule#1:
Set background value to
highlight change
Rule#2:
2+2 = 2*2
(c) Michael Stahl, 2012, All Rights Reserved
![Page 36: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/36.jpg)
Output: Background and Foreground
Int Foo(int x) {
int status = SUCCESS;
if (x > 1000) retVal = ERROR_OUT_OF_RANGE
else process(x, &status);
return status;
}
What if process() does nothing ?
(c) Michael Stahl, 2012, All Rights Reserved
![Page 37: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/37.jpg)
Output: Background and Foreground
Int Foo(int x) {
int status = UNUSED_VALUE;
if (x > 1000) retVal = ERROR_OUT_OF_RANGE
else process(x, &status);
return status;
}
(c) Michael Stahl, 2012, All Rights Reserved
![Page 38: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/38.jpg)
Array testing:
char src[16]={0}, dst[16]={0};
// src gets value from user
strncpy(src, input, sizeof(input));
// src is copied to dst
strncpy(dst, src, sizeof(dst));
Output: Background and Foreground
(c) Michael Stahl, 2012, All Rights Reserved
![Page 39: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/39.jpg)
Array testing:
char src[16]={0}, dst[16]={0};
// src gets value from user
strncpy(src, input, sizeof(input));
// src is copied to dst
strncpy(dst, src, sizeof(dst)); //bug!
Output: Background and Foreground
(c) Michael Stahl, 2012, All Rights Reserved
![Page 40: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/40.jpg)
Array testing:
Incorrect test:
1. Enter “SIGIST” in input field
2. Verify dst = “SIGIST”
dst before:
dst after:
Test Result = PASS //
Output: Background and Foreground
(c) Michael Stahl, 2012, All Rights Reserved
![Page 41: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/41.jpg)
Array testing:
Correct test:
1. Set dst to {A}
2. Enter “SIGIST” in input field
3. Verify dst = “SIGISTAAAAAAAAAA”
dst before:
dst after:
Test Result = FAIL //
Output: Background and Foreground
(c) Michael Stahl, 2012, All Rights Reserved
![Page 42: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/42.jpg)
Linked list - Add
Everything together…
Head
Null Invariant,
background
Variant,
foreground
Variant Variant
(c) Michael Stahl, 2012, All Rights Reserved
![Page 43: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/43.jpg)
Setup
(c) Michael Stahl, 2012, All Rights Reserved
![Page 44: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/44.jpg)
Test flow (naïve)
Did “Set” work?
Must be validated!
Set execution
conditions Apply Input Check Output
(c) Michael Stahl, 2012, All Rights Reserved
![Page 45: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/45.jpg)
Validate “Set” first!
Get execution
Conditions
(record the
Background)
Set to different
Values
(change the
foreground)
Get execution
Conditions
(validate the
foreground)
Set-able variables are Input!
- Apply the “Basics”
- Do you allow setting invalid values? (c) Michael Stahl, 2012, All Rights Reserved
![Page 46: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/46.jpg)
Test flow
Set execution
conditions
Check Output
Verify execution
conditions Blocked!
Teardown Apply Input Set
background
Fail?
DUT
(c) Michael Stahl, 2012, All Rights Reserved
![Page 47: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/47.jpg)
Teardown
(c) Michael Stahl, 2012, All Rights Reserved
![Page 48: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/48.jpg)
Teardown
Reset conditions
to default Clear
background Clean the
environment
(c) Michael Stahl, 2012, All Rights Reserved
![Page 49: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/49.jpg)
Teardown & Test efficiency
State-aware Setup
Combine tests
(c) Michael Stahl, 2012, All Rights Reserved
![Page 50: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/50.jpg)
Validate your test!
(c) Michael Stahl, 2012, All Rights Reserved
![Page 51: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/51.jpg)
Set/Verify
execution
conditions
Check Output Apply Input
Set
background
DUT
Validate your test!
(c) Michael Stahl, 2012, All Rights Reserved
![Page 52: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/52.jpg)
Set/Verify
execution
conditions
Check Output Apply Input
Set
background
Validate your test!
(c) Michael Stahl, 2012, All Rights Reserved
![Page 53: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/53.jpg)
Check Output Apply Input
Set
background
DUT
Validate your test!
(c) Michael Stahl, 2012, All Rights Reserved
![Page 54: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/54.jpg)
Set/Verify
execution
conditions
Check Output Apply Input
DUT
Validate your test!
(c) Michael Stahl, 2012, All Rights Reserved
![Page 55: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/55.jpg)
Set/Verify
execution
conditions
Check Output Apply Input
Set
background
DUT
Validate your test!
(c) Michael Stahl, 2012, All Rights Reserved
![Page 56: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/56.jpg)
Set/Verify
execution
conditions
Check Output Apply Input
Set
background
DUT
Validate your test!
(c) Michael Stahl, 2012, All Rights Reserved
![Page 57: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/57.jpg)
Set/Verify
execution
conditions
Check Output
Apply Input
Set
background
DUT
Validate your test!
(c) Michael Stahl, 2012, All Rights Reserved
![Page 58: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/58.jpg)
(c) Michael Stahl, 2012, All Rights Reserved
![Page 59: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/59.jpg)
Testing non-functional attributes
(c) Michael Stahl, 2012, All Rights Reserved
![Page 60: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/60.jpg)
Check Output
Set/Verify
execution
conditions
Apply Input
Set
background
DUT
(c) Michael Stahl, 2012, All Rights Reserved
![Page 61: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/61.jpg)
Check Output
Set/Verify
execution
conditions
Apply Input
Set
background
DUT
What’s the limit?
(c) Michael Stahl, 2012, All Rights Reserved
![Page 62: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/62.jpg)
Spec: Standby Current < 15.0
What’s the limit?
(c) Michael Stahl, 2012, All Rights Reserved
![Page 63: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/63.jpg)
Spec: < 15.0
What’s the limit?
(c) Michael Stahl, 2012, All Rights Reserved
![Page 64: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/64.jpg)
Test the Capability
Expected Results:
Based on capability – not on Spec*
Rationale:
Increase the test sensitivity
Find the real limit
* Unless the capabilities are worse than Spec…
(c) Michael Stahl, 2012, All Rights Reserved
![Page 65: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/65.jpg)
Fail if > 5.5?
6?
6.5?
Risk: False alarms
Increasing the test sensitivity
(c) Michael Stahl, 2012, All Rights Reserved
![Page 66: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/66.jpg)
Increasing the test sensitivity
Spec: < 15.0
Where to put the test limit?
Mean+1 = 4; Mean+2 = 5; Mean+3 = 6
(c) Michael Stahl, 2012, All Rights Reserved
![Page 67: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/67.jpg)
What’s a Fail then?
Use Process Control methods and rules
Example: Fail if:
• One measurement above +3
• Two consecutive points falling
above the +2σ limit
• Three consecutive points
falling above the +1σ limit
• Seven consecutive points
falling above the centerline
• Ten consecutive points falling
below the centerline
• …
Source: Wikipedia - Western Electric rules
(c) Michael Stahl, 2012, All Rights Reserved
![Page 68: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/68.jpg)
“Tighten the Screws”
Spec: < 15.0
How much regression do you allow?
(c) Michael Stahl, 2012, All Rights Reserved
![Page 69: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/69.jpg)
Example – real case
CPU Utilization measurements
The spec: PASS if below 50%
Capability (760 measurements):
(c) Michael Stahl, 2012, All Rights Reserved
![Page 70: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/70.jpg)
Where would YOU put the limit?
Mean+1 = 39.5
Mean+2 = 52.25
Mean+3 = 64 $#^%&#@!!!
?
(c) Michael Stahl, 2012, All Rights Reserved
![Page 71: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/71.jpg)
Additional info
= 760
Different results due to
“system capability”
Different results due to
different scenario
x
(c) Michael Stahl, 2012, All Rights Reserved
![Page 72: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/72.jpg)
Per scenario distribution
(c) Michael Stahl, 2012, All Rights Reserved
![Page 73: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/73.jpg)
Per scenario distribution
Mean+1 = 27.5
Mean+2 = 29.1
Mean+3 = 30.6 !!!
(c) Michael Stahl, 2012, All Rights Reserved
![Page 74: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/74.jpg)
Per scenario distribution
Mean+1 = 35.4
Mean+2 = 37.6
Mean+3 = 39.8 !!!
(c) Michael Stahl, 2012, All Rights Reserved
![Page 75: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/75.jpg)
Mean+1 = 22.6
Mean+2 = 24.9
Mean+3 = 27.2
Per group-of-scenarios distribution
(c) Michael Stahl, 2012, All Rights Reserved
![Page 76: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/76.jpg)
Summary
(c) Michael Stahl, 2012, All Rights Reserved
![Page 77: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/77.jpg)
Summary
(c) Michael Stahl, 2012, All Rights Reserved
![Page 78: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/78.jpg)
Summary
(c) Michael Stahl, 2012, All Rights Reserved
![Page 79: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/79.jpg)
Summary
(c) Michael Stahl, 2012, All Rights Reserved
![Page 80: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/80.jpg)
Summary
Quis custodiet ipsos custodes?
(c) Michael Stahl, 2012, All Rights Reserved
![Page 81: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/81.jpg)
Summary
(c) Michael Stahl, 2012, All Rights Reserved
![Page 82: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/82.jpg)
Acknowledgements / Resources
Resource:
The Craft of Software Testing – Brian
Marick
Thanks!
Intel colleagues Shmuel Gershon, Dan
Alloun , Bet-Sheva Honigstein-Magen,
Issi Hazan – for suffering through the initial
version of this presentation, and making it
much better
(c) Michael Stahl, 2012, All Rights Reserved
![Page 83: More Important than Techniques - SELAlegacy17.sela.co.il/s/sigist12/_AllPresentations/19July/...The ISTQB Test Techniques Equivalence Partitioning Boundary Values Analysis Decision](https://reader031.vdocument.in/reader031/viewer/2022041022/5ed393f78faf431f4563e4ef/html5/thumbnails/83.jpg)
Thank You!