A Test Generation Tool

for Specifications in the

Form of State Machine

Zheng-Wen Shen

2004/11/23

References

A Test Generation Tool for Specifications in the

Form of State Machines

Q. M. Tan, A. Petrenko and G. v. Bochmann

To appear in IEEE International Conference on

Communications -- 96, Dallas, USA, 1996.

Outline

1. Introduction

2. Using the tool 1. An Example

2. Functions provided by the Tool

3. The FSM specification

3. Test derivation methods 1. Preambles

2. Postambles

3. Transition Cover

4. State Identification Sequences

4. Experiments and Applications

5. Conclusion

1. Introduction

2. Using the tool 1. An Example

2. Functions provided by the Tool

3. The FSM specification

3. Test derivation methods 1. Preambles

2. Postambles

3. Transition Cover

4. State Identification Sequences

4. Experiments and Applications

5. Conclusion

1. Introduction

TAG tool - Test Automatic Generation

Automatically generates test case for an FSM

specification.

Transition identification approach for test derivation

Output test cases in the form of an SDL skeleton.

FSM has been extensively used in the testing

phases of system developments.

E.g. conformance testing of communication

protocols.

Communication protocols are the rules that

govern the communication between the

different components within a distributes system.

7-layer OSI model

Protocol conformance testing

A protocol specification generally can lead to

several implementations in software and/or

hardware.

Assure the compatibility with other

implementations of the same protocol.

The existing test derivation methods

Check each transition in an FSM specification at

least once (branch coverage criteria.)

Verify the tail state of the transition to obtain

high fault coverage and to guarantee

conformance in the context of a more general

fault model.

Using state identification techniques

Defect the fault that the FSM enters a different state than

specified.

Most of the existing protocols are not

completely specified.

Not all the sequences of interactions are foreseen.

The existing test derivation methods for

protocols are limited to completely specified

specifications.

TAG working directly for deterministic, partially

specified FSM specifications.

1. Introduction

2. Using the tool 1. An Example

2. Functions provided by the Tool

3. The FSM specification

3. Test derivation methods 1. Preambles

2. Postambles

3. Transition Cover

4. State Identification Sequences

4. Experiments and Applications

5. Conclusion

2. Using the tool

A complete test suite that guarantees full fault

coverage may be derived

A set of test cases that cover a given test

purpose may be derived.

Output format

Mnemonic format

SDL skeleton

2.1 An example

To achieve a particular test purpose which is a certain

transition to be tested:

1. Bring the FSM from its initial state to the starting state of

the transition under test using the shortest input sequence

possible. – a preamble of the test case

2. Execute the transition and check the observed output

3. Check a tail state of the transition by observing its reaction

to a pre-selected set of state identification sequences.

4. Apply an input sequence to return to the initial state of the

FSM. – a postamble of the test case

State cover

The set of all preambles.

Transition cover

The set of sequences used to execute all specified

transitions.

State identification sequences

Distinguish states by their output reactions.

TAG implements the HSI method

HSI method

Similar to the widely used W-method

A characterization set is used for state identification.

A tuple of subsets of a W set.

Can be applied to partially specified FSMs.

The INRES responder

Input alphabet

1- CR; 2- IDISreq; 3- ICONrsp; 4- DT0; 5- DT1

Output alphabet

0- NULL; 1- ICONind; 2- DR; 3- CC; 4- ACK0;

5- ACK0,IDATind; 6- ACK1; 7- ACK1,IDATind

States

S0- Closed (the initial state); S1- Opening; S2-

Waiting_DT0; S3- Waiting_DT1

Test suite and intermediate results

State Identifier = {{41}, {41}, {4}, {4}}

Preamble = {ε, 1, 13, 135}

Transition Cover = {1, 4, 5, 11, 12, 13, 14, 15,

131, 132, 134, 135, 1351, 1352, 1354, 1355}

Postamble = {ε, 2, 2, 2}

Test Suite = {41, 441, 541, 1241, 13241, 135241,

141, 1141, 1441, 1541, 134, 1314, 13514, 13514,

13554}

2.2 Functions provided by the tool

A text file containing the FSM specification with suffix “.fsm”

Symbol table (.tbl)

FSM structure (.cpl)

Complete test derivation

A complete test suite is generated.

Selective test derivation

Give the test purpose, which specifies one transition in the FSM.

The test output is written in a text file.

Mnemonic format (.mnc)

SDL skeleton (.sdl)

2.3 the FSM specification

1. the state definitions

2. The input definitions

3. The output definitions

4. The transition definitions

5. The variable declaration (optional)

6. The homing sequence definition (optional)

7. “end;”

The state definitions are a list of state names.

The input definition are a list of input names.

The output definitions are list of output names.

The transition definitions define the FSM state table

itself by a list of transition specifications.

Current state name, input name, output name and next state

name.

May be followed by a set of the comments.

The variable definitions define the variables in

parameters.

Integer, Charstring, Octetstring, Boolean

Using Keyword “homing” to give a sequence of

input names as a homing sequence.

1. Introduction

2. Using the tool 1. An Example

2. Functions provided by the Tool

3. The FSM specification

3. Test derivation methods 1. Preambles

2. Postambles

3. Transition Cover

4. State Identification Sequences

4. Experiments and Applications

5. Conclusion

3.1 Preambles

A tree with the initial state as its root is constructed such that the tail states of the outgoing transitions from the state corresponding to a current node.

All nodes in this tree must become a current node once and only once in the order that they enter this tree.

The path from the root to a given node is preamble for the corresponding state.

S0

S1

S2 S3

1

3

5

Preamble = {ε, 1, 13, 135 }

3.2 Postambles

A tree with given state as its root is constructed.

Once the initial state has been added to the tree,

the procedure stops.

The path from the root to the last added node is

a postamble from the given state.

Maybe no postamble from the given state.

S0

S1

2

Preamble = {ε, 2, 2, 2 }

S0

S2

2

S0

S3

2

S0

3.3 Transition Cover

The transition cover can be obtained by appending

each outgoing transition from this state to its preamble.

Preamble = {ε, 1, 13, 135 }

S0:ε::1, ε::4, ε::5

S1: 1::1, 1::2, 1::3, 1::4, 1::5

S2: 13::1, 13::2, 13::4, 13::5

S3: 135::1, 135::2, 135::4, 135::5

Transition Cover = {1, 4, 5, 11, 12, 13, 14, 15, 131, 132,

134, 135, 1351, 1352, 1354, 1355}

3.4 State Identification Sequences.

The characterization set W is a set of input

sequence.

Harmonized state identification sets (HSI sets),

subsets of W.

{D0, D1, …, Dn-1}

Di is a set of prefixes of sequences in W

n is the number of states of S.

For any two distinguishable states Si and Sj of S, there

exists a sequence σ that is a prefix of both σi ∈ Di and

σj ∈ Dj such that σ can be accepted by these two states

and produces different outputs.

How is Optimal ?

1. The number of sequences in W is minimal.

2. The sum of their lengths is minimal.

To obtain a minimal characterization set for a

give FSM maybe an NP-Hard problem.

A heuristic solution

A1: A set P0 of all the state pairs that are distinguishable for the given FSM is produced.

A2: With P0, a characterization set W is obtained by forming a search tree from the input alphabet to distinguish the state pairs in P0.

A3: From this W, some HSI set Di is selected for each state i, such that the number of distinguishable state pairs and the length of a sequence are traded off.

The result of INRES responder

The result P0 of the algorithm A1 is the set of all

possible state pairs.

The characterization set W from the algorithm

A2 is {41}

The HSI sets {{41}, {41}, {4}, {4}}

The algorithm A1

The algorithm A1 is obtained by adapting the

FSM minimization algorithm give in

[ G. J. Holzmann. Design and Validation of

Computer Protocols, 1991 ]

The algorithm A2

Forms the root as the current node to probe

For each input word, a son is built if it can lead

to a better solution than other build nodes.

If it is estimated that a subtree with a minimal

number of branches and with a minimal average

length that distinguishes a maximal number of

state pairs could be formed by probing an

unprobed son of ti.

The search tree in A2

The algorithm A3

A sequence σk in W is selected such that weighted sum of its length and the number of the state pairs that it can not distinguish in the left state pairs is minimal.

For each remaining state pair (l, m) that can be distinguished by σk, find a prefix of σk such that (l, m) is distinguished.

Then the prefix is put into the HSI sets Dl and Dm.

W = {σ0, σ1,… σn-1}

Remain state pairs

(l, m)

σk :

σ :

HSI Sets: {D0, D1, D2, .. Dl, …, Dm, …Dn-1}

distinguish ?

1. Introduction

2. Using the tool 1. An Example

2. Functions provided by the Tool

3. The FSM specification

3. Test derivation methods 1. Preambles

2. Postambles

3. Transition Cover

4. State Identification Sequences

4. Experiments and Applications

5. Conclusion

4. Experiments and Applications

State num. 10 50 100 150 200

Transition num. 100 2500 10000 23500 40000

CPU time (sec.) 0.11 2.96 40.87 197.06 760.89

1. Introduction

2. Using the tool 1. An Example

2. Functions provided by the Tool

3. The FSM specification

3. Test derivation methods 1. Preambles

2. Postambles

3. Transition Cover

4. State Identification Sequences

4. Experiments and Applications

5. Conclusion

5. Conclusion

Proposed a heuristic solution to derive near-

optimal harmonized state identification sets.