test design

Limited Business Circulation

Page:1Version 1.1/Testing Essentials/001QSIT Copyright

Software Testing Essentials 1 www.qsitglobal.com

Testing Essentials Unit Test Design specification

QSIT- Quality Solutions for Information Technology Pvt. Ltd.,

A 806 & 807, Mittal Tower,BANGALORE- 560 001,INDIAURL: www.qsitglobal.comPH: +91 80 41134334Fax: +91 80 25597445

Software Testing Essentials 2www.qsitglobal.com

Exercise: Which Group is Better?

Found pre-release

Function A 100

Function B 0

Function C 0

Function D 0

Function E 0

Total 100

Function A 50

Function B 6

Function C 6

Function D 6

Function E 6

Total 74

Two groups test the

same program.

• The functions are

equally important

• The bugs are

equally significant

Testing Group 1

Testing Group 2

From Marick, Classic

Testing

Mistakes

Exercise: Which Group is Better?

Found pre-release

Found later

Function A 100 0 100

Function B 0 12 12

Function C 0 12 12

Function D 0 12 12

Function E 0 12 12

Total 100 48 148

Function A 50 50 100

Function B 6 6 12

Function C 6 6 12

Function D 6 6 12

Function E 6 6 12

Total 74 74 148

Inputs and Outputs of Test Design Specification

Business

Requirements

Design

Specification

Testability

Types of

Testing

Coverage

High Level

Test Design

Testing

Requirements

Detailed Test

Design Specification

Adequecy

Criteria

Define

Process

Define

Techniques

Completion

Criteria

Test Data Adequacy Criteria

Code Based Testing

Requirement Based Testing

•Have I

•Tested

•Exercised

•Forced

•Found

•Have I

•Thought

•Applied all inputs

•Completely explored

•Run all the Scenarios

Approach

� Study Business Requirements

� Arrive at Environmental Requirements

� Identify test related Risks

� Decide Automation Requirements

� Prepare Test Documents

� Plan for Test Completion

� Analyze Track changes

� Review Test design effectiveness

Testing Techniques

� Code Based (White Box testing/Structured)

– Statement Coverage

– Edge Coverage

– Condition Coverage

– Path Coverage

� Code Complexity

– Cyclomatic Complexity

� Specification Based (Black Box Testing/ Functional)

– Boundary Value Analysis

– Equivalence Partitioning

– Decision table

– Cause Effect Graphing

– Control Flow Based Criteria

– Data Flow based criteria

Testing Techniques (Contd…)

� Domain Specific Technique

– Object Oriented Testing

– Embedded Testing

– Client Server

– Web Applications

� Usage Based

– Statistical testing (Musa’s)SRET

– Special Testing

� Fault Based

– Error Guessing

– Mutation

– Fault Seeding

Test Exit/Completion Criteria

� Executed at least once?

� Requirements been tested or verified?

� Test Documentation

� Documents updated and submitted to Configuration Manager

� Test Summary Report

Review: Defining the Test Approach

� A good test approach is:

– Diversified

– Risk-focused

– Product-specific

– Practical

– Defensible

� The techniques you apply should follow your test approach

Dimensions of Test Techniques

� Think of the testing you do in terms of five dimensions:

– Testers: who does the testing.

– Coverage: what gets tested.

– Potential problems: why you're testing (what risk you're testing for).

– Activities: how you test.

– Evaluation: how to tell whether the test passed or failed.

� Test techniques often focus on one or two of these, leaving the rest to the skill and imagination of the tester.

Test Techniques – Discussion

� There are as many as 200 published testing techniques. Many of the ideas are overlapping, but there are common themes.

� Similar sounding terms often mean different things, e.g.:

– User testing

– Usability testing

– User interface testing

� What are the differences among these techniques?

Test Techniques—Dominant Test Approaches

� Of the 200+ published Functional Testing techniques, there are ten basic themes.

� They capture the techniques in actual practice.

� In this course, we call them:– Function testing– Equivalence analysis– Specification-based testing– Risk-based testing– Stress testing– Regression testing– Exploratory testing– User testing– Scenario testing– Stochastic or Random testing

“So Which Technique Is the Best?”

Testers

Coverage

Potential problems

Activities

Evaluation

Technique A

Technique B

Technique C

Technique E

Technique F

Technique G

Technique H

• Each has strengths and weaknesses

• Think in terms of complement

• There is no “one true way”

• Mixing techniques can improve coverage

Technique D

What is black box testing?

– Knows only what the software is supposed to do

– Doesn’t know how it operates inside

Why named as black box testing?

– You cannot see inside the box

Black Box Techniques

Who does black box testing?

– Testers

– Business Analysis

– Clients

When you do?

– Once the functionality is developed

Why not by developers?

– Very difficult to break one’s own code

– To test the software from end-user perspective

– Fresh eyes always help

Requirements Based - Black Box Techniques

� Low Level Techniques– Boundary value analysis

– Equivalence partitioning

� High Level techniques– Express requirements in simple formal notations like

� Decision table

� State machine

� Cause-effect Graph

� Use case

The notation allows generation of scenarios

– Different test cases for every scenario

– Makes requirements verifiable

Boundary Value Analysis

� A Black Box Testing Method

� Complements to Equivalence partition

� BVA leads to a selection of test cases that exercise bounding values

� Design test cases test

– Min values of an input

– Max values of an input

– Just above and below input range

Less than 6 More than 15

� Two types of BVA:

– Range

� Above and below Range

– Value

� Above and below min and max number

Boundary Value Analysis:

– Large number of errors tend to occur at boundaries of the input domain.

– BVA leads to selection of test cases that exercise boundary values.

– BVA complements equivalence partitioning.

– Rather than select any element in an equivalence class, select those at the ''edge' of the class.

� Examples:

– For a range of values bounded by a and b, test (a-1), a,

(a+1), (b-1), b, (b+1).

– If input conditions specify a number of values n, test with (n-

1), n and (n+1) input values.

– Apply 1 and 2 to output conditions (e.g., generate table of

minimum and maximum size).

– If internal program data structures have boundaries (e.g.,

buffer size, table limits), use input data to exercise structures

on boundaries.

Equivalence Partitioning

� Divide the input domain into classes of data for which test cases can be generated.

� Attempting to uncover classes of errors.

� Divides the input domain of a program into classes of data

� Derives test cases based on these partitions

� An equivalence class is a set of valid or invalid states of input

� Test case design is based on equivalence classes for an input domain.

Invalid Inputs

Valid Inputs

SYSTEM`

Output

� Useful in reducing the number of Test Cases required

� It is very useful when the input/output domain is amenable to partitioning

Invalid Valid Range Invalid

Less than 6 Between 6 and 15 More than 15

� Here test cases are written to uncover classes of errors for every input condition.

� Equivalence classes are:

RangeUpper bound + 1Lower bound – 1Within bound

ValueMaximum length + 1Minimum length – 1Valid value and Valid lengthInvalid valueSet

In-setOut-of-set

BooleanTrueFalse

� Equivalence Partitioning partitions the data to partition of a set

� Partition refers to collection of mutually disjoint subsets whose union is the entire set

� Chose one data element from each partitioned set

� The KEY is the choice of equivalence relation!

� EC based testing allows

– to have a sense of complete testing

– Helps avoid redundancy

Exercise: BVA, EP

A Life Insurance company in USA has base premium of$0.50 for all ages. Anyone under the age of 25 has topay an additional premium of $1.65 and anyonebetween the age of 25 to 65 has to pay an additional$2.87 and anyone who is over 65 has to pay anadditional premium of $ 6.0 per month. Derive Testcases to test the logic using the low level black boxtechnique. Note: The Normal Life expectancy in US isconsidered to be 105 Years.

Exercise: BVA, EP Solution

EP for the scenario are: 0-25, 26-65, 66-105

BVA for the scenario are:{0-25} - {-1,0, 1, 24, 25, 26}{26-65} {25, 26,27, 64,65,66}{66-105} {65,66,67, 104,105,106}

High Level Technique – Decision Table

� Identify the decision variable and conditions.

� Identify resultant actions to be selected or controlled

� Identify which actions should be produced in response to

particular combinations of conditions.

� Derive the logic function for the model to validate its

completeness and consistency

� Exposes errors in specification

� Columns in decision table are converted to test cases.

� Similar to Condition Coverage used in White Box Testing

Value 1 Value 2 Value 3

Login √ √ X

Password X √ X

Successful Login X √ X

Unsuccessful Login √ X √

Warning Message √(W) NA √(W)

CONDITION

ACTION

High Level Technique – State Transition Diagram

� Events are caused by input

� Actions are likely to cause output

� Limited to their ability to detect some types of faults

� Detect the most obvious incorrect transitions and outputs

� Will not detect more subtle faults that are only detectable through exercising sequence of transitions.

� State Transition Diagram

State 1

State 2

Transition

InputEvent

Action

Output

Example

Manage_display_changes specifications

The software responds to input requests to change display mode for a time display device.

The display mode can be set to one of four values:

Two corresponding to displaying either the time or date.The other two when altering either the time or the date.

Four possible input requestsChange modeResetTime setDate set

State Transition Diagram - Example

Change Mode (CM) : Activation of this shall cause the display mode to move between “display time (T)” and “display date (D).

Reset (R) : If the display mode is set to T or D, then a “reset” shall cause the display mode to be set to “alter time (AT)” or “alter date “AD” modes.

Time Set (TS) : Activation of this shall cause the display mode to return to T from AT.

Date Set (DS) : Activation of this shall cause the display mode to return to T from AD

State Transition Diagram - Example

STD for Manage_display_changes

Displaying Time (S1)

Displaying Date (S2)

Changing Time (S3)

Changing Date (S4)

State Transition Diagrams

Start Sate S1 S2 S1 S3 S2 S4 S1 S1 S2 S2 S3 S3 S4 S4 …

Input - - - - - - -

Event CM CM R TS R DS TS DS TS DS CM R CM R

Action Date Mode Time ModeRest Time

Rest time

Rest Date

Rest Date Error Error Error Error Error Error Error Error

OutputDate Display

Time Display

New Value for time

New Time

New value for date

New Date

Error Message

End State S2 S1 S3 S1 S4 S2 S1 S1 S2 S2 S3 S3 S4 S4 …

OAT – Orthogonal Array Technique.

Consider this:

� 11 Check Boxes in Font dialog of MS Word for various effects such as superscript, strikethrough, emboss, etc.

� Obviously these effects have impact on how the characters in a particular font are displayed

� Can be used in multiple combinations such as Strikethrough + Subscript + Emboss.

Example Cont.

� The total number of combinations of effects in this case would 211 or 2048 combinations

� This doesn't include different font types, styles, etc.

� Number of combinations increase rapidly as additional dependent parameters are included resulting in potential explosion in the number of tests

Testing Challenges…

� Industry has a lot of evidence to suggest that most software defects occur from simple interactions between the variables of 2 parameters

� How to choose the combinations out of all the possibilities from a risk based perspective where it may not be feasible to test all possible combinations?

Orthogonal Array Technique

� OATS is a systematic, Statistical way of testing pair-wise interactions.

� Useful for integration testing of software components in OO systems.

� Useful for testing of combinations of configurable options such as Web page

� Orthogonal arrays are two dimensional arrays of numbers which possesses the interesting quality that by choosing any two columns in the array, we receive an array of all the pair-wise combinations of values in the array.

High Level Technique –Orthogonal Array Testing

� Orthogonal arrays are named in the pattern

– LRuns (Levels Factors)

� Runs: Number of rows in the array (No.of test Cases)

� Factors: Number of columns in an array (No. of variables)

� Levels: Maximum number of values that can be taken by any single factor.

How to use this techniqueThe OATS technique is simple and straightforward. The steps are

outlined below.

� 1. Decide how many independent variables will be tested for interaction. This will map to the Factors of the array.

� 2. Decide the maximum number of values that each independent variable will take on. This will map to the Levels of the array

� 3. Find a suitable orthogonal array with the smallest number of Runs. A suitable array is one that has at least as many Factors as needed from Step 1 and has at least as many levels for each of those factors as decided in Step 2.

� 4. Map the Factors and values onto the array

� 5. Choose values for any "left over" Levels

� 6. Transcribe the Runs into test cases, adding any particularly suspicious combinations that aren't

A Simple ExampleConsider a web page with three distinct sections

� Top,

� Middle, and

� Bottom

that can be individually shown or hidden by the user. You wish to test the interactions of the different sections.

Following the instructions laid out previously, let's create a test set for the system.

A Simple Example� There are three independent variables (the sections of the page).

� 2. Each variable can take on two values (hidden or visible).

� 3. An L4(23) orthogonal array will do the trick -- two levels for the values and three factors for the variables. Note that the number of runs is not necessary to pick an appropriate array.

� 4. Mapping the values onto the array would look like Shown in the Figure where Hidden=0 and Visible=1:

A Simple Example

OA before mapping factorsFactor 1 Factor 2 Factor 3

� Run 1 0 0 0

� Run 2 0 1 1

� Run 3 1 0 1

� Run 4 1 1 0

OA after mapping factorsTop Middle Bottom

� Test 1 Hidden Hidden Hidden

� Test 2 Hidden Visible Visible

� Test 3 Visible Hidden Visible

� Test 4 Visible Visible Hidden

A Simple Example� 5. There are no "left over" Levels. In other words, there is a value

mapped to every level in the array.

� 6. Taking the test case values from each run, you end up with four test cases. That is all that is needed to test all of the pair-wise interactions amongst the three variables. The test cases might transcribe to:

A. Display the home page and hide all sections.

B. Display the home page and show all but the Top section.

C. Display the home page and show all but the Middle section.

D. Display the home page and show all but the Bottom section.

“Note”

Note that not all of the possible combinations are tested.

It would take eight test cases to test all of the combinations.

You can add test cases that are particularly suspicious, but don't show up in the pair-wise list.

For example,

You might want to add a test case to display all sections or only the middle section to the list above if you felt there was a strong

possibility of error with those particular combinations.

An Example That Doesn't Fit the ArrayLet's look at an example that doesn't fit perfectly into any available array. This example takes into account an object-oriented system that contains • A client class (C1) with two subclasses (C2 and C3).These client classes interact with a server class hierarchy consisting of class S1 with subclasses S2 and S3.

The server class contains a method foo() which takes an instance of class M1 as a parameter. M1 has two subclasses, M2 and M3.

Figure below depicts the classes involved.

To test all combinations of the classes involved, it would take 27 test cases (three clients that can each send three messages to three servers -- 3 x 3 x 3 = 27).

That doesn't seem outrageous, but this assumes that the method foo() can be tested with only a single test case. In most circumstances, it will take many test cases to test a particular method.

Also, this interaction is probably a very small portion of the entire

system being tested.

Using the OATS technique can significantly reduce the number of test cases.

An Example That Doesn't Fit the Array

1. There are three independent variables

(the client, the server, and the message class)

2. Each variable can take on three values.

3. Ideally, we would us an array that contains three levels and three factors (an L?(33) OA). However, no such published array exists. Therefore, we need to look for the smallest array that will handle our problem. An L9(34) orthogonal will work. It has the three levels for the values and four factors is more than enough for the three variables.

4. Mapping the values onto the array would look like Figure 4 where:

A. For Client, C1=0; C2=1; C3=2.

B. For Server, S1=0; S2=1; S3=2.

C. For Message, M1=0; M2=1; M3=2.

OA before mapping factors

Factor 1 Factor 2 Factor 3 Factor 4

Run 1 0 0 0 0

Run 2 0 1 1 2

Run 3 0 2 2 1

Run 4 1 0 1 1

Run 5 1 1 2 0

Run 6 1 2 0 2

Run 7 2 0 2 2

Run 8 2 1 0 1

Run 9 2 2 1 0

OA after mapping factors

Client Server Message

� Test 1 C1 S1 M1

� Test 2 C1 S2 M2

� Test 3 C1 S3 M3

� Test 4 C2 S1 M2

� Test 5 C2 S2 M3

� Test 6 C2 S3 M1

� Test 7 C3 S1 M3

� Test 8 C3 S2 M1

� Test 9 C3 S3 M2

5. There are no "left over" Levels. However, you'll notice that there was an extra Factor in the original array.

This factor can simply be ignored; it does not change the properties of the test set generated from the array.

You still get an even distribution of the pair-wise combinations.

6. Taking the test case values from each run, you end up with nine test cases.

As mentioned before, these nine combinations might map to a larger set of test cases that must be executed against each of the nine combinations.

test design

total function

function d

function c

function b

function e

test completion

test documents plan

test documentation documents

Documents

a test design poster for smarter testing · a test design...

test design techniques -...

test planning test design test analysis test design...

gravitron design test

test design and documentation. test design test design is to...

test design technics

test design - ojp

national institute of design design aptitude test -...

test album design

using a test design tool to become a digital organization -...

software test architecture design focusing on test...

software test design

essential test design

test design best practices by gabriel rodriguez. copyright...

well test design

design test report

test-case design

joy design test

design for test low power test -...

system test design systtcon 1 the location of test design...