test case design methods
TRANSCRIPT
-
8/6/2019 Test Case Design Methods
1/51
TEST CASE DESIGNTECHNIQUES
-
8/6/2019 Test Case Design Methods
2/51
OUTLINETest Case
Definition
Test case criteria Test case essentials
Test case syntax
Equivalent Partitioning
Boundary Analysis
Constraint Analysis
State Transition Condition Combination
2
-
8/6/2019 Test Case Design Methods
3/51
TEST CASE
Definition:A test that (ideally) executes a single
well defined test objective (Testing ComputerSoftware Kaner, Faulk, Nguyen)
Definition: A specific set of test data and
objective (IEEE 729-1983)
3
-
8/6/2019 Test Case Design Methods
4/51
WHY WRITE TEST CASE?
Accountability
Reproducibility Tracking
Automation
o n ugs
To verify that tests are being executed correctly
For compliance
To measure test coverage
4
-
8/6/2019 Test Case Design Methods
5/51
TEST CASE CRITERIAAn excellent test case satisfies the following
criteria:
Reasonable probability of catching error
Neither too simple nor too complex
Best of its breed
Makes program failures obvious
5
-
8/6/2019 Test Case Design Methods
6/51
How is a good test ?
Refers and maintains tight control over specific testdata
Has detailed enough Test Design Steps.
Is aware of the Testers ex erience
TEST CASE CRITERIA
Has clear criteria for Pass or Fail
6
-
8/6/2019 Test Case Design Methods
7/51
A Not-so-Good Test ?
Leaves it up to the user to find test data Gives very high level instructions that leave too much
room for artistic interpretation
Does not consider the Testers ex erience
TEST CASE CRITERIA
Leaves out follow-up verification steps which make itdifficult to determine Pass or Fail criteria.
7
-
8/6/2019 Test Case Design Methods
8/51
Checklist to review test cases
Is the test procedure accurate and explicit enough for anon-expert user?
Is any supporting documentation required (i.e.
TEST CASE CRITERIA
, ,
Do any test cases test multiple conditions?
Are test cases duplicated?
Are the scripts in a logical testing order?
8
-
8/6/2019 Test Case Design Methods
9/51
TEST CASE ESSENTIALS
Things usually to include in a test case: Tracking Information
Test Case ID
Test Description
Purpose/Objective/Title
Procedures/Steps
Script
Parameters/Data
Input
Output Default
Call to system (printer, system clock, available RAM, APIs)
Expected Result
Observed Result
Pass/Fail/Blocked/Skipped Bug ID
Notes/Comments
Environment 9
-
8/6/2019 Test Case Design Methods
10/51
Test case objective
The most important part of a test case is
the 1-line title describing the objective ofthe test
That 1-line title can be called
TEST CASE ESSENTIALS
Test Name
Test Case
Test Objective
Test Goal/Purpose
10
-
8/6/2019 Test Case Design Methods
11/51
Test case objective
It is most important because
It gives the reader a description and idea ofthe test
A good test name makes review easier
TEST CASE ESSENTIALS
Easier to pass to another person Easier to pass to automation team
In many cases, may be the only part of a test
case documented
11
-
8/6/2019 Test Case Design Methods
12/51
TEST CASE OBJECTIVE SYNTAX
Action + Function + Operating Condition
In which: Function may be function, feature, validation point Operating Condition may be data
VerifyTestValidateProveExecutePrintCalculateRunany action verb
12
-
8/6/2019 Test Case Design Methods
13/51
TEST CASE EXAMPLES
Action Function Operating ConditionRun annual report from standard data (file location)
Run annual report on Day 1 of fiscal year
Run annual report from empty spreadsheetRun annual report on last day of fiscal year
13
-
8/6/2019 Test Case Design Methods
14/51
VALIDATION POINTS
Tests need validation points
This is the expected result.Write it.
Define clearly state: what behavior, result or point
you are attempting to validate.
14
-
8/6/2019 Test Case Design Methods
15/51
TEST CASE TEMPLATE
TC ID Description StepsExpected
Result
Observed
resultStatus
TC00
1
/*Test
objective*/
/*Very clear and
specific steps*/
Pre-condition:
/*you need to
pre determine
what your
/* write down
the result that
you get when
Passed/Failed
15
Steps:
1. Action 1
2. Action 2
..
program s
supposed to
do*/
excecute t e
test case*/
-
8/6/2019 Test Case Design Methods
16/51
Test Case
Test case criteria Test case essentials
Test case syntax
OUTLINE
16
Equivalent Class Partitioning
Boundary Analysis
Constraint Analysis
State Transition Condition Combination
-
8/6/2019 Test Case Design Methods
17/51
Test Case
Test case criteria Test case essentials
Test case syntax
OUTLINE
17
Equivalent Class Partitioning
Boundary Analysis
Constraint Analysis
State Transition Condition Combination
-
8/6/2019 Test Case Design Methods
18/51
EXAMPLE
Specification:
-This program isdesigned to add two
numbers, which you
18
will enter.-Each number
should be one or
two digits
-
8/6/2019 Test Case Design Methods
19/51
There are 199 x 199 = 39,601 test cases for
valid values: definitely valid: 0 to 99
might be valid: -99 to -1
EQUIVALENCE CLASS AND
BOUNDARY ANALYSIS
19
There are infinitely many invalid cases: 100 and above
-100 and below
anything non-numeric
-
8/6/2019 Test Case Design Methods
20/51
EQUIVALENCE CLASS AND
BOUNDARY ANALYSIS
Equivalence Class: Two tests belong to the
same equivalence class if the expected result ofeach is the same..
Boundaries: Mark the point or zone of
another. Generally, each class is partitioned by the
boundary values.
It is important to recognize that two tests are equivalentonly with respect to a specific risk.
20
-
8/6/2019 Test Case Design Methods
21/51
-
8/6/2019 Test Case Design Methods
22/51
EQUIVALENCE CLASS AND
BOUNDARY ANALYSIS
General Steps
Identify the classes Identify the boundaries
Identify the expected output(s) for valid
nput s Identify the expected error-handling (EH)
for invalid inputs
Generate a table of test cases (maximum, 9test cases for each partition of each class).
22
-
8/6/2019 Test Case Design Methods
23/51
EQUIVALENCE CLASS AND
BOUNDARY ANALYSIS
Other Equivalence Classes and Special Cases
Negative valuesNumber of digit or characters
0
Non-printable characters or Upper ASCII (128-254)
Space
Nothing
Etc.
23
-
8/6/2019 Test Case Design Methods
24/51
Specification: text
box with valid
values from 1 to9999 or 1 to 4-digit
value
EXAMPLE 2
24
-
8/6/2019 Test Case Design Methods
25/51
Test Case
Test case criteria Test case essentials
Test case syntax
OUTLINE
25
Equivalent Class Partitioning
Boundary Analysis
Constraint Analysis
State Transition Condition Combination
-
8/6/2019 Test Case Design Methods
26/51
CONSTRAINT ANALYSIS
General Steps
Identify the dependent variables and studytheir respective constraints.
Identify all possible input and output
List the variables in a tabular format withtheir input/output interfaces and theirrespective constraints.
26
-
8/6/2019 Test Case Design Methods
27/51
CONSTRAINT ANALYSIS
DATA RELATIONSHIPS
27
Result???
-
8/6/2019 Test Case Design Methods
28/51
This table shows how the variables are related.
CONSTRAINT ANALYSIS
DATA RELATIONSHIPS
Here are the columns: Field: End Date, and Start Date are examples of
fields.
Entry Source: What dialog boxes can you use to
enter data into this field? Can you import data intothis field? Can data be calculated into this field? Listevery way to fill the field -- every screen, etc.
28
-
8/6/2019 Test Case Design Methods
29/51
Display: List every dialog box, error messagewindow, etc., that can display the value ofthis field.
Print: List all the reports that print thevalue of this field (and any other functions
CONSTRAINT ANALYSIS
DATA RELATIONSHIPS
that print the value).Constrained By: List every variable that
constrains the value of this variable.
Constraint: List every variable that this one
constrains.
29
-
8/6/2019 Test Case Design Methods
30/51
CONSTRAINT ANALYSIS
FUNCTIONALITY RELATIONSHIPS
The Let Windows
manage my virtualmemory settings
radio buttonInactive
Features
availability of thefeatures in the Let
me specify my own
virtual memory
settings group.
Active
Features30
-
8/6/2019 Test Case Design Methods
31/51
Test Case
Test case criteria Test case essentials
Test case syntax
OUTLINE
31
Equivalent Class Partitioning
Boundary Analysis
Constraint Analysis
State Transition Condition Combination
-
8/6/2019 Test Case Design Methods
32/51
STATE TRANSITIONDefinition
Involves an analysis of the relationship among the
states and the events or actions that cause thetransitions from one state to another.
General Steps Identify all supported states
For each test case, define:The starting state
The input events that cause the transitions
The output results of each transitionThe end state
32
-
8/6/2019 Test Case Design Methods
33/51
EXAMPLE 1Navigation Command
Current View Mode
-
8/6/2019 Test Case Design Methods
34/51
Navigation CommandCurrent View Mode
34
-
8/6/2019 Test Case Design Methods
35/51
TRANSITIONING EXAMPLE
35
-
8/6/2019 Test Case Design Methods
36/51
Test Case
Test case criteria Test case essentials
Test case syntax
OUTLINE
36
Equivalent Class Partitioning Boundary Analysis
Constraint Analysis
State Transition Condition Combination
-
8/6/2019 Test Case Design Methods
37/51
CONDITION COMBINATION
Condition Combination Involves an analysis of the combination relationship
of the variables.
Each combination represents a condition to be tested
Identify the variables. Identify the number of possible unique values of each
variable.
Create a table illustrating the complete uniquecombination of conditions.
37
-
8/6/2019 Test Case Design Methods
38/51
COMBINATION
TEST CASE DESIGN
-
8/6/2019 Test Case Design Methods
39/51
39
-
8/6/2019 Test Case Design Methods
40/51
TEST METHODS
Some Common Test Methods and Test
Types Requirement Testing
Exploratory Testing
Regression Testing
40
-
8/6/2019 Test Case Design Methods
41/51
TTESTEST MMETHODETHOD
DefinitionsDefinitions: A definitive procedure for the identification,
measurement, and evaluation of a material, product,
system, or service that produces a test result.
(ASTM American Society for Testing and Materials)
41
-
8/6/2019 Test Case Design Methods
42/51
CCOMMONOMMON TTESTEST MMETHODSETHODS
ANDAND TTESTEST TTYPESYPES
Test approaches: Requirement-based Testing (Passive Testing)
Exploratory Testing (Ad Hoc Testing) .
The best practice is a combination of a few methods tosatisfy the project needs
Test Types Regression Testing
Smoke Testing
Stress Testing
Load Testing Performance Testing
42
-
8/6/2019 Test Case Design Methods
43/51
RREQUIREMENTEQUIREMENT--BASEDBASED TTESTINGESTING
Requirement-based Testing is:
Testing the requirement
Testing the specifications
requirement and spec documentationThe primary goal of Requirement-based
Testing is RequirementValidation.
43
-
8/6/2019 Test Case Design Methods
44/51
RREQUIREMENTEQUIREMENT--BASEDBASED TTESTINGESTING
The Mechanical Approach for Test development:
Analyze requirements
Make at least a test case for every requirement
Execute the test cases
44
-
8/6/2019 Test Case Design Methods
45/51
EEXPLORATORYXPLORATORY TTESTINGESTING
Exploratory Testing can have many names: Ad hoc,
Discovery, Error Guessing
The primary goal of exploratory testing is to uncover
new defects in the product on-the-fly.
The task is finding ways to creatively design
experiments, exploring, experimenting, going from a
known place to an unknown place.
45
-
8/6/2019 Test Case Design Methods
46/51
EEXPLORATORYXPLORATORY TTESTINGESTING
Goal:probe for weak areas of the program. Some
various error conditions should be checked are: Out of boundary
Null input and overflow (storage of the sum)
Non-numeric
Hardware and memory error handling
Exploratory Testing is the testing approach which
can produce the highest number of bugs
46
-
8/6/2019 Test Case Design Methods
47/51
RREGRESSIONEGRESSION TTESTINGESTING
Most common type of work we do!Re-run
-
ExpensiveFirst to automate
47
-
8/6/2019 Test Case Design Methods
48/51
RREGRESSIONEGRESSION TTESTINGESTING
Regression Testing: performed after making a
functional improvement or repair to the program.
Regression Purpose:
- eri y t at t e mo i ications ave not cause any
unintended effects
- Make sure that the bugs marked as fixed by
developers are really fixed.
Re-test all test cases can be done using automatedtesting tools.
48
-
8/6/2019 Test Case Design Methods
49/51
RREGRESSIONEGRESSION TTESTINGESTING
Some of the most popular strategies for selecting
regression test suites:
Retest all: Rerun all test cases. Simple but impossible in the
time that we have in our everyday practice.
Retest Risk Use Cases: Choose aseline tests to re-run risk
heuristics.
Retest by Profile: Choose baseline tests to re-run by allocating
time in proportion to operational profile.
Retest Changed Segment: Choose baseline tests to re-run by
comparing code changes.
49
-
8/6/2019 Test Case Design Methods
50/51
SSMOKEMOKE TTESTINGESTING
Definition: Asmoke test is a collection of written teststhat are performed on a system prior to being accepted
for further testing, also known as build verificationtest.
s s a s a ow an w e approac o e
application.
These written tests can either be performed manuallyor using an automated tool.
50
-
8/6/2019 Test Case Design Methods
51/51
SUMMARYSUMMARY
Different test methods have different goals:
Projects with lots of documentation tend of userequirements based testing as the test
development methods.
ro ec s w e ocumen a on re y on a
hoc/exploratory testing. Projects in later phases of release (iteration,
functional increments) rely more on regression
testing.
51