experimental investigation of effectiveness of logical expressions using t-way testing

RANJAN BHAMBROO

Experimental Investigation of Effectiveness of Logical Expressions Using t-way

Testing

Introduction

Experimental Evaluation of logical expressions.

Same idea but continued on new path.

T – way testing instead of pair-wise testing.

New tool for generating logical expressions.

New tool from NIST to construct t-way test sets.

T-way Testing

T-way

Pair – wise = 2-way

T-way where t=2,3,4,5,6.

T- way more effective than pairwise.

But test cases not enough need more test cases.

T-way Testing

Example:For 15 variables total number of

combinations = 215

215 = 32768 combinations

T-way values Test Cases

2-way 10

3-way 24

4-way 58

5-way 131

6-way 296

Tools Used

Boolean Expression Generator – ECU( Developed by Ranjan Bhambroo)

Advanced Combinatorial Testing System(ACTS) for constructing t-way combinatorial test sets – NIST

Fault Evaluator – ECU

Boolean Expression Generator(New Tool)

Generates Logical Expressions Automatically.Simple and user friendly interface. Different Modes to generate Expressions: a)Random Number of variables. b)Fixed Number of variables. c)Interval of variablesCan Control complexity of expressions.Allows user to save expressions in a text file.

Experimental Results Of T-way Testing

Fault Evaluator

Boolean Expression Generator

Advanced Combinatorial

Testing System

Logical

Expressions

T-way

Tes

t

Sets

Effectiveness

Experimental Results Of T-way Testing

Note Using Pair Wise Test Cases

10 expressions/set VNF ORF VRF (ENF) ASF

All Fault Types (weighted avg.)

Set1 36.25 45.58 24.96 60.27 74.51 30.34

Set2 24.67 39.57 17.9 46.55 54.84 22.32

Set3 28.57 43.18 19.54 45.78 75 24.54

Set4 28.06 46.99 21.69 55.95 62.79 26.96

Set5 30.67 39.86 20.38 62.07 60.98 25.3

Average 29.64443.03

620.89

4 54.124 65.624 25.892

Std. Deviation 3.823 2.974 2.376 6.801 7.909 2.678

Experimental Results Of T-way Testing

Note Using 3-Wise Test Cases10 expressions/set VNF ORF VRF (ENF) ASF

All Fault Types (weighted avg.)

Set1 40.88 48.03 27.61 63.93 80 33.27

Set2 26.39 42.54 20.62 48.21 72.41 25.22

Set3 28.97 45.19 22.04 53.23 71.88 26.88

Set4 32.24 52.11 26.02 59.7 76.47 31.52

Set5 36.22 53.85 29.12 68.75 86.67 34.47

Average 32.9448.34

425.08

258.76

477.48

6 30.272

Std. Deviation 5.157 4.201 3.248 7.339 5.457 3.611

Experimental Results Of T-way Testing

Note Using 4-Wise Test Cases10 expressions/set VNF ORF VRF (ENF) ASF

All Fault Types (weighted avg.)

Set1 43.07 53.54 36.11 63.93 95 40.79

Set2 33.33 51.49 27.32 55.36 68.97 31.96

Set3 39.31 54.81 30.16 58.06 93.75 35.49

Set4 34.87 52.82 30.46 62.69 88.24 35.55

Set5 50.39 66.67 41.66 79.17 96.67 47.07

Average40.19

455.86

6 33.142 63.842 88.526 38.172

Std. Deviation 6.14 5.508 5.125 8.266 10.179 5.267

Experimental Results Of T-way Testing

Note Using 5-Wise Test Cases10 expressions/set VNF ORF VRF (ENF) ASF

All Fault Types (weighted avg.)

Set1 47.45 55.91 39.25 67.21 97.5 43.97

Set2 38.19 59.7 33.84 62.5 89.66 38.77

Set3 39.73 57.04 31.57 58.06 96.88 36.69

Set4 42.76 69.86 36.35 64.18 94.12 41.55

Set5 50.39 70.94 45.05 81.25 96.67 50.11

Average43.70

4 62.69 37.212 66.64 94.966 42.218

Std. Deviation 4.601 6.423 4.68 7.881 2.893 4.653

Experimental Results Of T-way Testing

Note Using 6-Wise Test Cases10 expressions/set VNF ORF VRF (ENF) ASF

All Fault Types (weighted avg.)

Set1 48.18 55.91 42.4 67.21 97.5 46.44

Set2 41.67 60.45 39.38 62.6 100 43.56

Set3 42.07 60.74 36.79 59.68 96.88 41.43

Set4 40.79 60.56 37.29 65.67 100 42.31

Set5 53.54 70.94 48.73 81.25 100 53.29

Average 45.25 61.7240.91

8 67.282 98.876 45.406

Std. Deviation 4.902 4.953 4.377 7.449 1.391 4.29

Experimental Results Of T-way Testing

2-way 3-way

Experimental Results Of T-way Testing

4-way 5-way

Experimental Results Of T-way Testing

THANK YOU