model based test case generation
TRANSCRIPT
-
8/6/2019 Model Based Test Case Generation
1/24
MODEL BASED TEST
GENERATION
-
8/6/2019 Model Based Test Case Generation
2/24
WHATS A MODEL?
A model:
Is an abstraction or simplified representation ofthe
system from a particular perspective
Supportsinvestigation, discovery, explanation, prediction,or construction
May be expressed as a description, table, graphical
diagram, or quantitative mathematical model
Is not necessarilycomprehensive
2
-
8/6/2019 Model Based Test Case Generation
3/24
MODELS IN EVERYDAY LIFE
3
-
8/6/2019 Model Based Test Case Generation
4/24
EXAMPLES OF MODELS
Flow Charts
Data Flow Diagrams
Entity- Relationship Diagrams
State Diagrams
Deployment Diagrams
ClassDiagrams
Use Ca
se
s ActivityDiagrams
State Transition Tables
4
-
8/6/2019 Model Based Test Case Generation
5/24
MODEL BASED TEST GENERATION
5
set of
Test Cases
sequence of stimuli
expected responses
Formal Model
Test GenerationDirectives
Generator
-
8/6/2019 Model Based Test Case Generation
6/24
MODELING
6
Modeling Validation
Refinement Verification
SpecificationAsmL
ImplementationC/C++/C#
Product Idea/ Specification
Are you buildingthe product right ?
Are you
building theright product?
What productare you
building?
-
8/6/2019 Model Based Test Case Generation
7/24
FSM: START STATE & ACTION
7
Start state (Entry Point)
Files Shown: 0
File Select: Not Selected
Exit Point: False
-
8/6/2019 Model Based Test Case Generation
8/24
OTHER
STATES
8
start
-
8/6/2019 Model Based Test Case Generation
9/24
SIMPLE FSM MODEL (STATE MODEL)
9
-
8/6/2019 Model Based Test Case Generation
10/24
WALK EVERY STATE-CHANGE
10
Create
Invert Selection
Invert Selection
Delete
-
8/6/2019 Model Based Test Case Generation
11/24
TEST EVERY WALK < 5 STEPS
11
A: Invert Selection
B: Create; Delete
C: Create; Invert Selection; Invert Selection; Delete
-
8/6/2019 Model Based Test Case Generation
12/24
MODEL BASED TEST GENERATION
TOOLS
T-VEC(Test Vector Generation System)
TestMaster
UniTesk (Unified Testingand spec toolkit)
GOCHA Techbeans
AS
MLEtc..
12
-
8/6/2019 Model Based Test Case Generation
13/24
THE ASML MODELING TOOLKIT
Abstract state machine Language is a powerfulmodelinglanguage
AsmL provides Authoringvia MS Word and XML
Simulation and interoperabilityvia COM and .NET
Automatic test oracle
Semi-automatic test case generation
Integration with Microsofts test platform
13
-
8/6/2019 Model Based Test Case Generation
14/24
ASML: GETTING STARTED, SORTEXAMPLE
14
var A as Seq ofInteger = []
Swap()choose i in indices(A),
j in indices(A)where i < j and A(i) > A(j)
A(i) := A(j)A(j) := A(i)
sort()step until fixpoint
swap()
A = [2,3,1]
A = [1,2,3]
A = [2,1,3]
Non-determinism
A = [1,3,2]
Parallelism
-
8/6/2019 Model Based Test Case Generation
15/24
ASML: PREDEFINED DATA
TYPES
Sets
menu = {"ham", "cheese","blt"}
15
SequencesSequencesrandomrandom == [[3,3, 6,6, 7,7, 1,1, 55]] ////asas SeqSeq ofof IntegerInteger
MapsMapsdirectorydirectoryas Map ofas Map of StringString toto Integer =Integer =
{{policepolice -->>112, info112, info -->>118118}}
TuplesTuplesnameAndAgenameAndAge == ((John,39John,39)) ////((StringString,,IntegerInteger))
-
8/6/2019 Model Based Test Case Generation
16/24
USING MODELS FOR TEST
16
Model
Implementation
Test OracleTest Cases
Are run byProvides
actual results for
Pass
No pass
Provides expected
results forGenerates
User
Info
-
8/6/2019 Model Based Test Case Generation
17/24
ASML-BASED TEST HARNESS
Side-by-side runs
Discrepancies
automaticallyflagged
17
AsmLmodel
Test harness
Implementationunder test
Anyclient
-
8/6/2019 Model Based Test Case Generation
18/24
ASML TEST CASE GENERATION - 1
Solves two problems
How to find interesting parameter values
and combinations How to find interestingcall sequences
Approach Use information derived from the model to
optimize the number oftest cases
18
-
8/6/2019 Model Based Test Case Generation
19/24
ASML TEST CASE GENERATION -
2
Divide state space accordingto state predicates
System covers the state predicates Driven byinput generator (parameter selection) or
Driven by enabledness (call sequences)
Tester providesinputs, givesinterestingstate
predicates, directs the search
19
-
8/6/2019 Model Based Test Case Generation
20/24
THE DUAL ROLE OF ASM MODELS
20
AsmL
ModelM
Test cases
Guided
generation(Param
Select
withFeedback)
Reduce
FSM
tool
IUT
Call next action
Get state
AsmL
ModelM
Testing harness
Is S
valid according
to M?
Fail: witness
Pass S
view
view
view
FSM
-
8/6/2019 Model Based Test Case Generation
21/24
ASML TEST CASE GENERATION - 3
Generate only as many test cases as needed to havefull coverage ofthe specification
21
Generator ModelInput
Coverage criteriaCoverage criteria
Branch: Every branch of the spec must beBranch: Every branch of the spec must betakentaken
Modified condition: independent atomicModified condition: independent atomicconditionsconditions
OffOff--byby--One: offOne: off--byby--one errorsone errors
HyperHyperstate coverage: Everystate coverage: Every abstractabstractstatestatemust be visitedmust be visited
Feedback
-
8/6/2019 Model Based Test Case Generation
22/24
ASML TEST (ASMLT)
asmlT can be used to:
1. Find interestingsequences ofmethod calls finds the shortest route in a generated FSM that uses every edge to
traverse the states
2. Find interestingparametersfor each method call
Access Driven Filtering(ADF) you write a predicate (boolean function) to decide ifa given
parameter isinteresting
3. Perform a conformance test
Compare Model with Implementation
22
-
8/6/2019 Model Based Test Case Generation
23/24
REFERENCES
Model-Based Test Generation Tools, Alan Hartman,AGEDIS CONSORTIUMhttp://www.agedis.de/documents/ModelBasedTestGenerationTools_cs.pdf
Finite State Model-Based Testingon a Shoestring, HarryRobinsonhttp://www.model-based-testing.org/shoestring.htm
Model Based Testing, L. Apfelbaum, et al, Software Quality Week Conference inMay, 1997.http://www.model-based-testing.org/sqw97.pdf
Abstract State Machine Language, Microsoft Researchhttp://research.microsoft.com/fse/asml/
The AsmL Test Generator tool, Microsoft Researchhttp://research.microsoft.com/fse/AsmL/doc/AsmLTester.html
Intelligent Test Automation, HarryRobinsonSoftware Testing& QualityEngineering(STQE) magazine Sept/Oct 2000http://www.model-based-testing.org/robinson.pdf
23
-
8/6/2019 Model Based Test Case Generation
24/24
24
Queries..??