erlang code evolution control - unich.it
TRANSCRIPT
Erlang Code EvolutionControl
1Universitat Politècnica de València2Universidad Politécnica de Madrid
David Insa1, Sergio Pérez1, Josep Silva1, Salvador Tamarit2
LOPSTR 2017 12/10/2017
Content
• Introduction• The technique indetail• Type Analysis Phase• TestCaseGeneration Phase• Comparison Phase
• Recording the trace• SecEr tool• Conclusions&FutureWork
Introduction• Indebugging programmers usebreakpoints toobservethe values ofanexpression during an execution
• Is this feature available intesting?
• It would beuseful tofocus the testcaseson an specific point without codemodifications
OURPROPOSALØ Introducethe ability tospecify Points OfInterest (POI) inthe context of
testing
Ø Atechnique tocomparetwo equivalent POIs indifferent versionsofthe same program for Erlang
Old Version
main(X,Y) ->A = X + Y,
D = X – Y,A * D.
New Version
main(X,Y) ->A = add(X,Y),
D = sub(X,Y),A * D.
add(X,Y)->
X + Y.
sub(X,Y) ->
X – Y.
1.- Identify a POI and a set of input functions2.- A test suite is automatically generated
Each test case contains:• A call to an input function with specific arguments• The sequence of values the POI is evaluated to (trace)
3.- Each test case is passed against the newversion and both traces are compared
4.- A report of the success or failure of the testcases is provided
main(5,4)OldVersionTrace:1NewVersionTrace:1
Success
Introduction
We have implemented our approach for Erlanginatool named SecEr
Old Version
main(X,Y) ->A = X + Y,
D = X – Y,A * D.
New Version
main(X,Y) ->A = add(X,Y),
D = sub(X,Y),A * D.
add(X,Y)->
X + Y.
X – Y.
1.- Identify a POI and a set of input functions2.- A test suite is automatically generated
Each test case contains:• A call to an input function with specific arguments• The sequence of values the POI is evaluated to (trace)
3.- Each test case is passed against the newversion and both traces are compared
4.- A report of the success or failure of the testcases is provided
main(5,4)OldVersionTrace:1NewVersionTrace:1
Success
Introduction
We have implemented our approach for Erlanginatool named SecEr
sub(Y,X) ->
main(5,4)OldVersionTrace:1NewVersionTrace:-1
Failure
Content
• Introduction• The technique indetail• Type Analysis Phase• TestCaseGeneration Phase• Comparison Phase
• Recording the trace• SecEr tool• Conclusions&FutureWork
TypeAnalysisPhase
TestCaseGeneration
Phase
ComparisonPhase
Input TestCase
FinalReport
The technique in detail
TypeAnalysisPhase
TestCaseGeneration
Phase
ComparisonPhase
Input TestCase
FinalReport
The technique in detail
TypEr PropEr CutErInput InputTypesModule.erl
TypEr PropEr CutErFunction
Type analysis phase
TypEr.erl
Fun
2.- The length ofthe list is unkwnown
3.- The repeated-variablerestriction is ignored
f(1,[1,2,5,6])
f(1,[2,5])
Solution: Consider each clause independently andrefinethe result totypes perclause
TypEr+newcode PropEr CutEr InputsTypes〈Clause
,Types〉Input
f(A,[A,B]) “-spec f(1|2,[1|2|5|6])->…”
1.- The infered types refer tothe whole function
g(0,0)->…g(1,1)->… “-spec g(0|1, 0|1)->…”
g(0,1)
Type analysis phase
.erl
FunTypEr
+newcode PropEr
Propertytest
prop_identity() ->?FORALL(X, any(), id(X) = X).
id(X) -> X.
genPropErTypes Value
CutEr InputsInput〈Clause,Types〉
gen
PropEr
true|false
Type analysis phase
... ...
PropErgen
PropErtypes Input
.erl
FunTypEr
+ newcode PropErPropEr+newcode CutEr InputsInput〈Clause
,Types〉
TypErtypes
TypEr toPropEr
ClauseParameters
2.- The length ofthe list is unknown
3.- Repeated variablerelation is lostSolution: Traverse the list parameters ofthe clause element by element
Solution: Storethe values ofalready treated variables
f(A,[A,B])
Type analysis phase
〈Clause,Types〉
PropErtypes
TypErtypes ≠
.erl
FunTypEr
+newcodePropEr
+newcodeInput
concolicgen
Input
RuntimeErrors
〈Clause,Types〉
CutEr InputsCutEr
+newcode
CutEr
Inputs...
...
Inputs
Type analysis phase
TypeAnalysisPhase
TestCaseGeneration
Phase
ComparisonPhase
Input TestCase
FinalReport
The technique in detail
.erl1
POI1
codeinstrumentator
.erl1’
Input Input execute Trace1inputselector
randomgen
mutatinggenNew? New
InputT
NewInput
F
add
Continue until alimit is reached
Storeall the generated testcases(Input+Trace)
TestCase
Test case generation phase
TypeAnalysisPhase
TestCaseGeneration
Phase
ComparisonPhase
Input TestCase
FinalReport
The technique in detail
.erl2
POI2
codeinstrumentator
.erl2’
TestCase
testcaseselector
Test case
Trace1
Input
execute Trace2 tracecomparator
ComparisonResult
Result
FinalReport
Resultanalyzer
add
Comparison phase
Content
• Introduction• The technique indetail• Type Analysis Phase• TestCaseGeneration Phase• Comparison Phase
• Recording the trace• SecEr tool• Conclusions&FutureWork
Recording the trace
• There areseveral tools for tracing executions inErlang• None ofthem allows us tocollect the traceofpatterns• Debuggers will not provide avalue for aPOIif it is inside anexpression whose evaluation failsOURPROPOSAL
Ø Collect the tracesasaside effect when executing the codeØ Approach based on message passing toatracing server
Ø The code needs tobeinstrumented (4STEPS)
{1,B,3} = {1,2,4}
Recording the trace (1 & 2)
1. Obtain andannotate the Abstract Syntax Tree ofthe program.Annotate each node with two lists ofvariables:
2. Find the selected POIinthe ASTwith atop-down traversal:
§ Variablesbeingbound inits subtree§ Variablesthatwere alreadyboundwhen reaching the node
§ Storethe current traversed pathwith tuples oftheform (Node,ChildIndex)
§ The result is apath that yields directly tothe POI
A
B C
D
POI
GF
(POI,1)
(F,1)
(E,2)
(B,1)
E
3. Analyze the location ofthe POI• Expressions: Add asend command toinform the tracing server• Patterns:Need special treatment
Recording the trace (3)
{1,B,3} = {1,2,4}
Targetexpressions• Pattern-matching• List comprehension• Expressions with clauses:
• if• case• functions• etc.
2
Recording the trace (3)
Dividethe ASTpath into two sub-paths:• PathBefore:Root ->deepest targetexpression• PathAfter:First child ofthe targetexpression ->POI
A
B C
D TE
POI
GF
(POI,1)
(F,1)
(TE,2)
(B,1)PathBefore(B,1),(TE,2)
PathAfter(F,1),(POI,1)
B
TE
F
POI
Recording the trace (4)
4. Perform the actualinstrumentation• Traverse the PathBefore• Transform the code following aruleaccording toPathAfter• TraversePathBefore backwards toupdate the AST
Five exclusiverulestoinstrument expressions• LEFT_PM(pattern-matching)• PAT_GEN_LC (list comprehensions)• CLAUSE_PAT (pattern inexpressions with clauses)• CLAUSE_GUARD (guard inexpressions with clauses)• EXPR(expressions)
A
B C
D TE
GF
B
TE
F
POI
NewSubtree
p = e → p = begin np = e, tracer!{add, npoi}, np end(p = e, _ ) = last(PathBefore)⋀ (_, pos(p)) = hd(PathAfter)
(_, npoi, np) = pfv(p,PathAfter)
(LEFT_PM)if
where
Recording the trace (4)
{1,B,3} = {1,2,4}
{1,B,3} = begin{1,POI,FV} = {1,2,4},tracer ! {add,POI},{1,POI,FV}
end
p = e → p = begin np = e, tracer!{add, npoi}, np end(p = e, _ ) = last(PathBefore)⋀ (_, pos(p)) = hd(PathAfter)
(_, npoi, np) = pfv(p,PathAfter)
p = e
p = begin
np = etracer!{add, npoi}
npend
Content
• Introduction• The technique indetail• Type Analysis Phase• TestCaseGeneration Phase• Comparison Phase
• Recording the trace• SecEr tool• Conclusions&FutureWork
SecEr command:$ ./secer -f FILE –li LINE – var VARIABLE [-oc OCCURRENCE]
-f FILE –li LINE – var VARIABLE [-oc OCCURRENCE][-funs INPUT_FUNCTIONS] –to TIMEOUT
SecEr tool
Old_Version_POI
New_Version_POI
SecEr toolhappy0.erl happy1.erl
$ ./secer -f happy0.erl –li 9 – var Happy -oc 1
-f happy1.erl –li 18 – var Happy -oc 1
-funs [main/2] –to 15
SecEr tool
SecEr tool
SecEr tool
Content
• Introduction• The technique indetail• Type Analysis Phase• TestCaseGeneration Phase• Comparison Phase
• Recording the trace• SecEr tool• Conclusions&FutureWork
ConclusionsConclusions
• Combination ofErlang existing tools andmutation toimprove the result• Newapproach toautomatically check the behaviour preservation betweenversions
• Newtracing process that allows for placing aPOIinpatterns,guards andexpressions
Type AnalysisPhase
TestCaseGeneration
Phase
ComparisonPhaseInput Test
Case
FinalReport
{1,B,3} = {1,2,4}
{1,B,3} = begin{1,POI,FV} = {1,2,4},tracer ! POI,{1,POI,FV}
end
FutureWork• Adapt the approach todeal with indeterminism• Increase the information stored intracestoreport non-functional properties such asefficiency
• Allow for the specification ofalist ofPOIs instead ofasinglePOI• Make the tool compatible with tests previously defined by theuser
Future Work
Thank you for your attention!!
Any question?