blackbox
TRANSCRIPT
Software Quality AssuranceBlack Box
Seminar: Oana FEIDIQuality Manager – Continental Automotive
Previewthe problem …
very large or infinite number of test scenarios+
finite amount of time=
impossible to test everything
the solution …Software test techniques exist to reduce
the number of tests to be run whilst still providing sufficient coverage of the system under test
OverviewBlack-box testing:
Test cases derived from specifications The focus is not the design, nor the
implementation
positive testing → testing the implementation against specified conditions
negative testing → testing the implementation against unspecified conditions (unspecified inputs)→ stability and robustness of specifications
Input Output
Equivalence partition
create partitions of the input and output values of the component
each partition shall contain a set or range of values, chosen such that all values can reasonably expected to be treated by the component in the same way
both valid and invalid values are partitioned in this way
For each test case specify: Input to the component Partition exercised The expected outcome of the test case
Test completeness criteria: test at least one input/output pair for each equivalence partition
Indicates when to stop testing
Equivalence partition
Example: f(int month) – want to test this function
…, -1, 0, 1, 2, 3, …., 9, 10, 11, 12, 13, 14, ….______||____________________||__________ invalid1 valid partition invalid2
f(int month, int nrdays) – want to test this function1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑31 28/29 31 30 31 30 31 31 30 31 30 31
Valid partition: [(1, 3, 5, 7, 8, 10, 12; 31), (4, 6, 9, 11; 30), …] Invalid partition: [(1, 2, 3, 5, 7, 8, 10, 12; 30), …]
Boundary Analysis
refinement of equivalence partitioning for which each edge of an equivalence class is a representative element of the class
invalid-input elements are found just beyond the ends
For each test case specify: the input(s) to the component the partition boundaries exercised The expected outcome of the test case
Test completeness criteria: test at least one input/output pair for each equivalence partition and the “borders” between the equivalence partitions
Boundary Analysis
Example: f(int month) – want to test this function
if (month > 0 && month < 13) or
if (month >= 0 && month < 13)
…, -1, 0, | 1, 2, 3, …., 9, 10, 11, 12 |, 13, 14, ….
|_________________________|valid partition
Test: 0, 1, 12, 13
Exercise 1 Use equivalence partition and boundary analysis
techniques to identify the values that have to be tested for the following parameters
State Transition Testing use a model of the states the component may
occupy, transitions between those states, the events which cause those transitions, and the actions which may result from those transitions
the model shall comprise states, transitions, events, actions and their relationships
For each test case specify: the starting state of the component the input(s) to the component the expected outputs from the component the expected final state
Test completeness criteria: 100% of the state transition diagram
State Transition Testing
Exercise 2
State 1
Dim
State 2
ON
State 3
OFF
State 0
Voltage
1) E1/A1 or2) E5/A1
E3E7
E8
1) E4(C1) or2) E10(C1) or3) E2(C1) or
4) E9(C1)
1) E1/A1 or2) E5(C1)/A1
E6/A2
(C2)
/A
3
default
Identify the paths using the state transition techniques
Condition DescriptionC1 All doors are closedC2 Duty cycle>51
Action DescriptionA1 Turn IL onA2 Turn IL offA3 Decrement duty cycle by
DIMM_STEP_INTERIOR_LIGHT_TIME every 20 ms
Event DescriptionE1 At least one of the driver door, assistant
door or trunk is openE2 A lock request is issuedE3 Duty cycle from PWM = 51E4 SHORT_INTERIOR_LIGHT_TIME expiredE5 An unlock request is issuedE6 CONTIN_INTERIOR_LIGHT_TIME expiredE7 Voltage failure flag is setE8 Voltage failure flag is not setE9 IGN is onE10 UNLOCK_INTERIOR_LIGHT_TIME expired
State DescriptionState 0: Voltage Voltage failure flag is set, IL is OFFState 1: Dim IL is dimmingState 2: ON IL is onState 3: OFF IL is off
Black Box - summary Equivalence partition
create partitions of the input and output values of the component each partition shall contain a set or range of values, chosen such
that all values can reasonably expected to be treated by the component in the same way
both valid and invalid values are partitioned in this way
Boundary Analysis refinement of equivalence partitioning for which each edge of an
equivalence class is a representative element of the class invalid-input elements are found just beyond the ends
State Transition Testing use a model of the states the component may occupy, transitions
between those states, the events which cause those transitions, and the actions which may result from those transitions
the model shall comprise states, transitions, events, actions and their relationships
Decision table Decision Table
A decision table is a tabular form that presents a set of conditions and their corresponding actions.
Condition Stubs Condition stubs describe the conditions or factors that will affect the
decision or policy. They are listed in the upper section of the decision table.
Action Stubs Action stubs describe, in the form of statements, the possible policy
actions or decisions. They are listed in the lower section of the decision table.
Rules Rules describe which actions are to be taken under a specific
combination of conditions. They are specified by first inserting different combinations of condition attribute values and then putting X's in the appropriate columns of the action section of the table.
Decision Table Methodology1. Identify Conditions & Values
Find the data attribute each condition tests and all of the attribute's values.
2. Compute Max Number of Rules
Multiply the number of values for each condition data attribute by each other.
3. Identify Possible Actions
Determine each independent action to be taken for the decision or policy.
4. Enter All Possible Rules
Fill in the values of the condition data attributes in each numbered rule column.
5. Define Actions for each Rule
For each rule, mark the appropriate actions with an X in the decision table.
6. Verify the Policy Review completed decision table with end-users.
7. Simplify the Table Eliminate and/or consolidate rules to reduce the number of columns.
Example
A marketing company wishes to construct a decision table to decide how to treat clients according to three characteristics: Gender, City Dweller, and age group: A (under 30), B (between 30 and 60), C (over 60). The company has four products (W, X, Y and Z) to test market.
Product W will appeal to female city dwellers. Product X will appeal to young females. Product Y will appeal to Male middle aged shoppers
who do not live in cities. Product Z will appeal to all but older females.
Example 1. Identify Conditions & Values
gender: M, F city dweller: Y, N age group: A, B, C
2. Compute Maximum Number of Rules: 2 x 2 x 3 = 12
3. Identify Possible Actions: market product W market product X market product Y market product Z
Example
4. Enter All Possible
Rules 1 2 3 4 5 6 7 8 9 10 11 12
Gender F M F M F M F M F M F M
City Y Y N N Y Y N N Y Y N N
Age A A A A B B B B C C C C
1 2 3 4 5 6 7 8 9 10 11 12
W X X X
X X X
Y X
Z X X X X X X X X X X
• 5. Define Actions for each Rule
Example Simplify the Table
◦ rules 2, 4, 6, 7, 10, 12 have the same action pattern ◦ rules 2, 6 and 10 have two of the three condition values (gender
and city dweller) identical and all three of the values of the non- identical value (age) are covered, so they can be condensed into a single column 2
◦ The rules 4 and 12 have identical action pattern, but they cannot be combined because the indifferent attribute "Age" does not have all its values covered in these two columns. Age group B is missing
Conditions
Gender F M F M F F M F F M
City Y Y N N Y N N Y N N
Age A - A A B B B C C C
Actions
W X X X
X X X
Y X
Z X X X X X X X X
1 2 3 4 5 6 7 8 9 10
Conclusions
Advantages of Black Box Testing more effective on larger units of code
than glass box testing tester needs no knowledge of
implementation, including specific programming languages
tester and programmer are independent of each other
tests are done from a user's point of view
will help to expose any ambiguities or inconsistencies in the specifications
test cases can be designed as soon as the specifications are complete
Conclusions
Disadvantages of Black Box Testing only a small number of possible inputs can
actually be tested, to test every possible input stream would take nearly forever
without clear and concise specifications, test cases are hard to design
there may be unnecessary repetition of test inputs if the tester is not informed of test cases the programmer has already tried
may leave many program paths untested cannot be directed toward specific segments of
code which may be very complex (and therefore more error prone)
most testing related research has been directed toward glass box testing
THANK YOU!