towards interoperability test generation of time dependent protocols: a case study zhiliang wang,...
TRANSCRIPT
Towards Interoperability Test Generation of Time Dependent Protocols:
a Case Study
Zhiliang Wang, Jianping Wu, Xia Yin Department of Computer Science
Tsinghua Universityhttp://netlab.cs.tsinghua.edu.cn
(Globecom2004)
2
Outline• Background
• Motivation
• Our proposed method
• Conclusion and Future work
3
Background: Protocol Testing
• Conformance testing– Basic method of protocol testing
– Test whether an implementation conforms to its protocol specification
– Test a single protocol entity
• Interoperability testing– Complement of conformance testing
– Test whether two or more protocol implementations can communicate with each other correctly and inter-operate as a whole system to perform functions specified in protocol specifications
– Test two or more protocol entities
4
Background: Protocol Testing
Protocol Specification
Test Generation
Abstract Test Suite
Test Implementation
Implementation Processes
Protocol Impl.SUT
Test Execution
Test Verdict
5
Background: Time-Dependent Protocols
• Real-life network protocols – Behaviors of input and output– Their time of occurrence– Time dependent!
• Conformance testing of time-dependent protocols – Protocol modeling: Timed Automaton and its variants (e.g. TIOA)– Many researches have been done
• Interoperability testing of time-dependent protocols – No work considered time constraints– Existing methods are difficult to be applied in the real-life interoperability testing
• This paper studied interoperability test generation for time dependent protocols
6
Background: Neighbor Discovery Protocol
• One of the basic protocols in IPv6 protocol set
• Should be supported in all IPv6 implementations
• Corresponding functions in IPv4– ARP (Address Resolution Protocol)
– ICMP router discovery
– ICMP redirect function
• Using 5 ICMP packet types:– RS: Router Solicitation
– RA: Router Advertisement
– NS: Neighbor Solicitation
– NA: Neighbor Advertisement
– Redirect
7
Outline• Background
• Motivation
• Our proposed method
• Conclusion and Future work
8
Motivation• System (Protocol) Modeling
– Single protocol entity: TIOA (Timed Input Output Automata)
– Two or more protocol entities: CTIOA (Communicating TIOA)• An TIOA set
• Channels between TIOAs
• Test Generation Method
Generate grid automaton for each TIOA
Calculate a composite product of grid automata
Apply existing non-timed interoperability test generation methods
Problem: state space explosion!!
An improved method
calculate the composite automaton of TIOAs using
reachability analysis method
Generate interoperability test sequences
Analyze the executability of derived test sequences
9
Outline• Background
• Motivation
• Our proposed method– Formal model and protocol specification
– Test architecture
– Test generation
– Executability analysis
• Conclusion and Future work
10
Formal Model• Communicating Timed Input Output Automata (CTIOA)
– a set of TIOAs M: Mi =
• SMi: set of states;
• LMi: set of input and output actions: (“!a”, “?b”);
• sMi0: initial state;
• CMi: set of clocks;
• TMi: set of transitions:
– a set of channels C: • Cij represents the communicating channel from TIOA Mi to Mj.
• Semantic of channels: the output of TIOA Mi can be transferred to the input of Mj via channel Cij
• An assumption: time taken for an IO action
via a channel is approximately equal to 0
(Normal network load)
)T,C,s,L,(S MiMi0MiMiMi
ss RPl /][
MjMiCij
a!a ?a
Predicate of clock values
Clock set to be reset
IO action
11
Protocol Specification• Router discovery function in ND Protocol
– Two types of nodes: Router and Host
– Protocol specification
SUT
Router
1 2
3
?RS [0<t11<3]
!S_RA
!uS_RA [T1<t11<T2]/
{t11}
? R S
!uS_RA/{t11}
Host
1(0)
2
t21:=0
!RS
[t21=1]
3
?S_RA
?uS_RA
?uS_RA
?uS_RA
1(1) 1(2) 1(3)!RS
[t22=4]/{t22}!RS
[t22=4]/{t21}
?S_RA ?S_RA
?uS_RA
?uS_RA ?uS_RA
interface enabled
RS
uS_RA, S_RA
?RS3][t11?RS/{t12}
0.5]t12[0 !S_RA/{t11}
1]t21[0
1]t21[0 /{t22}
3.5]t11[3 /{t11}
uS_RA: Unsolicited Router Advertisements
S_RA: Solicited Router Advertisements
RS: Router Solicitations
Clock set:Router node: {t11, t12}Host node: {t21, t22}
Channel set: {CRH, CHR}
1 2Transition Examples:
?RS[t11>=3]/{t12}1 3
?RS[0<t11<3]
12
Test Architecture
SUT
Router Host
TestSystem
PO
Control line Control line
PO: Point of ObservationControl line: e.g. Telnet
13
Test Cases Generation (overview)
Calculate the composite automaton of TIOAs using
reachability analysis method
Generate interoperability test sequences
Analyze the executability of derived test sequences
14
Reachability Analysis• Global Reachability Graph
• Synchronizing Transitions
HostRouter
1
2
1(0)
1(1)
1,1(0)
2,1(1)
?RS 3][t11/{t12}
RS1]t2103[t11
/{t12, t22}
!RS1]t21[0
/{t22}
M111111 T)s,R,Pa,!,(str M222222 T)s,R,P?a,,(str and________________________________________________________
M2 | |1M2121 T))s,s(R,P,a,),s,((str2||tr1 and 21 PPP and 21 RRR
15
Reachability Analysis• An example of Global Reachability Graph (Tree)
– DFS-based method1,1(0)
1,1(0)
R!uS_RA [T1<t11<T2]/t11:=0
H?uS_RA[0<t21<=1]
2,1(1) 3,1(1)
H!RS [0<t21<=1]/t22:=0R?RS [t11>=3]/t12:=0
H!RS [0<t21<=1]/t22:=0R?RS [0<t11<T3]
2,1(2)1,2
H!RS [t22=4]/t22:=0R?RS
R!S_RA [0<t12<=0.5]/t11:=0H?S_RA
2,1(3) 1,2
H!RS [t22=4]/t21:=0R?RS
R!S_RA [0<t12<=0.5]/t11:=0H?S_RA
1,2
R!S_RA [0<t12<=0.5]/t11:=0H?S_RA
1,2
R!S_RA [3<t11<=3.5]/t11:=0H?S_RA
1,2
H?uS_RA
R!uS_RA [T1<t11<T2]/t11:=0
3,1(2)
H!RS [t22=4]/t22:=0R?RS
3,1(3) 1,2
H!RS [t22=4]/t21:=0R?RS
R!S_RA [3<t11<=3.5]/t11:=0H?S_RA
1,2
R!S_RA [3<t11<=3.5]/t11:=0H?S_RA
Model
16
Test Cases Generation(overview)
Calculate the composite automaton of TIOAs using
reachability analysis method
Generate interoperability test sequences
Analyze the executability of derived test sequences
17
Test Generation• An example
1,1(0)
1,1(0)
R!uS_RA [T1<t11<T2]/t11:=0
H?uS_RA[0<t21<=1]
2,1(1) 3,1(1)
H!RS [0<t21<=1]/t22:=0R?RS [t11>=3]/t12:=0
H!RS [0<t21<=1]/t22:=0R?RS [0<t11<T3]
2,1(2)1,2
H!RS [t22=4]/t22:=0R?RS
R!S_RA [0<t12<=0.5]/t11:=0H?S_RA
2,1(3) 1,2
H!RS [t22=4]/t21:=0R?RS
R!S_RA [0<t12<=0.5]/t11:=0H?S_RA
1,2
R!S_RA [0<t12<=0.5]/t11:=0H?S_RA
1,2
R!S_RA [3<t11<=3.5]/t11:=0H?S_RA
1,2
H?uS_RA
R!uS_RA [T1<t11<T2]/t11:=0
3,1(2)
H!RS [t22=4]/t22:=0R?RS
3,1(3) 1,2
H!RS [t22=4]/t21:=0R?RS
R!S_RA [3<t11<=3.5]/t11:=0H?S_RA
1,2
R!S_RA [3<t11<=3.5]/t11:=0H?S_RA(b)
(a)
18
Test Cases Generation(overview)
Calculate the composite automaton of TIOAs using
reachability analysis method
Generate interoperability test sequences
Analyze the executability of derived test sequences
19
Executability Analysis• Executability: satisfies all time constrains
• IOTS = ST1·ST2·...·STn
[Theorem 1] tijk must be represented by a linear combination
of tts(s = 0,1,…,k-1) and dij (initial local clock values).
T0 T1 TnT2 Tn-1
tt0 tt1
ST1 ST2
......
......
ttn-1
STn-1 STn
tij1
tij2 tij
(n-1)tij
nGlobal ClocksLocal Clocks
C1({dij|i, j}, tt0) C∧ 2({dij|i, j}, tt0, tt1) ... C∧ ∧ n({dij|i, j}, tt0, tt1, ..., ttn-1)
C1({tij1|i, j}) C∧ 2({tij
2|i, j}) ... C∧ ∧ n({tijn|i, j}) Local Clocks
C({dij|i, j}, {ttk|k}) Global Clocks
20
Executability Analysis (cont)• Problem 1: Executability Determination
Given (1) IOTS = ST1·ST2·...·STn;(2) each initial local clock values dij. Determine if IOTS is executable.
C1({dij|i, j}, tt0) C∧ 2({dij|i, j}, tt0, tt1) ... C∧ ∧ n({dij|i, j}, tt0, tt1, ..., ttn-1)
C1(tt0) C∧ 2(tt0, tt1) ... C∧ ∧ n(tt0, tt1, ..., ttn-1)
21
Executability Analysis (cont)• Algorithm:
for k = 1 to n do
Solute the linear programming problem below: Say a set of linear constraints: C1(tt0) ∧ C2(tt0, tt1) ∧ ... ∧ Ck(tt0, tt1, ..., ttk-1) ∧ (tt0 ≥0) ∧ ... ∧ (ttk-1 ≥ 0) Evaluate Max x = ttk-1. if no solution, then return FALSE; else according to constraints Ck(tt0, tt1, ..., ttk-1) ∧ (ttk-1 ≥0),
evaluate the lower and upper bound of ttk-1: ),,,(, 21011 kLk
Uk ttttttPtttt ;
end.
return TRUE.
22
Executability Analysis (cont)• Example:
Init Clock Value Test Seq. d11 d12 d21 d22
Executable? k=?
when stop
Value bound of ttk
0 0 0 0 No 1 -- (a) 3 0 0 0 No 4 -- 0 0 0 0 No 1 --
(b) 3 0 0 0 Yes -- tt0 ∈ [0,1]
tt1∈ [0,0.5] tt2 ∈ (T1,T2)
1,1(0)
1,1(0)
R!uS_RA [T1<t11<T2]/t11:=0
H?uS_RA[0<t21<=1]
2,1(1) 3,1(1)
H!RS [0<t21<=1]/t22:=0R?RS [t11>=3]/t12:=0
H!RS [0<t21<=1]/t22:=0R?RS [0<t11<T3]
2,1(2)1,2
H!RS [t22=4]/t22:=0R?RS
R!S_RA [0<t12<=0.5]/t11:=0H?S_RA
2,1(3) 1,2
H!RS [t22=4]/t21:=0R?RS
R!S_RA [0<t12<=0.5]/t11:=0H?S_RA
1,2
R!S_RA [0<t12<=0.5]/t11:=0H?S_RA
1,2
R!S_RA [3<t11<=3.5]/t11:=0H?S_RA
1,2
H?uS_RA
R!uS_RA [T1<t11<T2]/t11:=0
3,1(2)
H!RS [t22=4]/t22:=0R?RS
3,1(3) 1,2
H!RS [t22=4]/t21:=0R?RS
R!S_RA [3<t11<=3.5]/t11:=0H?S_RA
1,2
R!S_RA [3<t11<=3.5]/t11:=0H?S_RA(b)
(a)
23
Executability Analysis (cont)• Problem 2: Generating Executable Test Sequence
– Basic idea: select a set of appropriate initial clock values
– assume that (1) a set of linear constraints C({dij|i, j}, {ttk|k}); (2) the value bounds of all initial clocks (dij [0, D∈ ij]) are known, determine bounds of dij(i=1,2; j=1,2,…,ni) to make time constraints satisfied.
• Method: Solve a linear programming problem– evaluate the value of Max(dij|i, j) and Min(dij|i,j)
• No solution: must-inexecutable
• Has a solution: may-executable
the value bound of dij: [Min(dij|i,j), Max(dij|i, j)]
• Problem: – More than one local clocks
– Difficult to determine initial values of each local clock
24
Executability Analysis (cont)• Problem 3: Adaptive Executable Test Sequence
– A heuristic method: set part of initial clock values in advance according to the test scenario
– determine whether the test sequence is must-inexecutable or may-executable
• If must-inexecutable then prune it from reachability tree;
• Else evaluate value bounds of unset initial clocks.
– divide the valid clock region into several sub-regions, so that only a unique test sequence tree exists in one sub-region
25
• Example: set d21=01,1(0)
2,1(1) 3,1(1)
1,2 1,2
RS
S_RA
RS
S_RA
1,2 1,2
uS_RA uS_RA
1] tt0d11 -[3 d11}]-3 min{1,tt0[0
0.5]tt1[0 tt0]-d11-3.5 tt1 tt0-d11-[3
T2]tt2[T1 T2]tt2[T1
T2 d11 2 0 d21 3 d110 0 d21
d110 2 3 T2
d110 2 3 T2
1,1(0)
2,1(1) 3,1(1)
1,2 1,2
RS
S_RA S_RA
1,2 1,2
uS_RA uS_RA
1]tt0 d11 -[3 RS
d11]-3tt0[0
0.5] tt1[0 tt0]-d11-3.5 tt1 tt0-d11-[3
T2]tt2[T1 T2]tt2[T1
1,1(0)
2,1(1)
1,2
RS
S_RA
1,2
uS_RA
1] tt0d11 -[3
0.5]tt1[0
T2]tt2[T1
1,1(0)
3,1(1)
1,2
RS
S_RA
1,2
uS_RA
d11}]-3 min{1,tt0[0
tt0]-d11-3.5 tt1 tt0-d11-[3
T2]tt2[T1
2 d110 0 d21 T2 d113 0 d21 3 d112 0 d21
Adaptive test
sequence
26
Conclusion and Future work• Present a formal method to generate timed interoperability test
sequence for time dependent protocols
• Executability analysis can be reduced to a Linear Programming problem.
• Alleviate state space explosion problem
• Future work: – Analyze fault coverage of this method
– Extend this method to the more common situations of interoperability testing
– Apply it to the real-life protocol testing activities
