explaining inconsistent codeswt.informatik.uni-freiburg.de/teaching/ss2015/... · muhammad numair...

Explaining Inconsistent CodeMuhammad Numair Mansur

Introduction● 50% of the time in debugging● Fault localization.● Becomes more tedious as the program size increase.● Automatically explaining and localizing inconsistent code.

2

Code Inconsistency● A code fragment is inconsistent if it is not a part of any normally terminating

execution. ● Not necessarily always a bug !● But sometimes inconsistent code results in an error.

3

Examples (Unreachability) Generated using Bixie

4

Examples (Unreachability) Generated using Bixie

5

Examples (conflicting assumptions) Generated using Bixie

6

Examples (conflicting assumptions) Generated using Bixie

7

Our Goal

Automatically explain inconsistent code.

8

Our Goal

Automatically explain inconsistent code.

9

AlgorithmInconsistentprogram Automata

Error Invariant Automaton

Pre

Post

Our Goal

Automatically explain inconsistent code.

10

AlgorithmInconsistentprogram Automata

Error Invariant Automaton

Pre

Post

Finite automataA F.A is a 5 tuple: (Q, Σ, δ, qo,F)

Q: A finite set of states.

Σ: A finite set of input symbols called an alphabet.

δ: A transition function ( δ: Q x Σ → Q ).

qo : initial state.

F: A finite set of final states.

11

Finite automataExample:

12

S1

S4

S2 S3

Finite automataExample:

13

S1

S4

S2 S3

States

Finite automataExample:

14

S1

S4

S2 S3

States

Transitions

Finite automata

15

F.Ainput(A sequence from the input alphabet)

Output(accept or reject)

● Transitions through the states based on the input● True, if ends in an accepting state

Finite automata

16

S1

S4

S2 S3

Σ = {a,b,c}Input: abca

Example:

Finite automataExample:

17

S1

S4

S2 S3

aΣ = {a,b,c}Input: abca

Finite automataExample:

18

S1

S4

S2 S3

a

b

Σ = {a,b,c}Input: abca

Finite automataExample:

19

S1

S2 S3

a

b

c

Σ = {a,b,c}Input: abca

S4

Finite automataExample:

20

S1

S4

S2 S3

a

b

c

a

accept !

Σ = {a,b,c}Input: abca

Program automataA simple and an abstract model of a program.

21

Program automataA simple and an abstract model of a program.Defined in terms of a finite automata. State (Q) = Program Location (Loc)Transition (δ) = Program Statement (δp)Alphabet(Σ) = A set of program statementsInitial State (q0) = Initial program Location ( )Final State ( F ) = Final program Location ( )

22

Program automata

23

Program automata

24

assume( !b ) means that the branch of if () is taken where b is “not true”

assume( b ) means that the branch of if () is taken where b is “true”

Program automata

25

An assertion on the program state that x != null

Program automata

● A run ρ is a finite sequence of locations and statements.lo sto l1 …..stn-1 ln

● A path(ρ) sto st1 ….. stn-1 is the path associated with a run.● A run ρ is accepting if its final state is le ● A word π ∈ * is a path if π = path(ρ) for some accepting run ρ.

26

Our Goal

To automatically explain inconsistent code.

27

AlgorithmInconsistentprogram Automata

Error Invariant Automaton

Pre

Post

AlgorithmInput: : precondition state formula

: program automata : Postcondition state formula

output: : error invariant automata.

requires: is inconsistent subject to and .

ensures: explains .28

AlgorithmStep 1: Translate the program automata into a single path of statements π .

29

AlgorithmStep 1: Translate the program automata into a single path of statements π .

30

AlgorithmStep 1: Translate the program automata into a single path of statements π .

31

It can be composed of many atomic statements.

AlgorithmExample:

This was the first step in getting the final result, an error invariant automata.32

1

2

3 4

5 6 7

8

9

1

2

2,9

9

Error Invariant Automaton● An abstraction of the program, that only mentions the statements and facts

that are relevant for understanding the cause of the inconsistency. ● The irrelevant statements are first summarized as first order logical

formulas and then eliminated. ● These formulas are called error invariants. ● An error invariant captures the reason of abnormal program termination.● So, at a high level, an Error Invariant Automaton replaces code which does

not contribute to the inconsistency with a suitably chosen invariant.

Lets see this in practice on a fragment of code.

33

Error Invariant Automaton 1: public TaskDialog(Tast task)

~: . . . . . . 6: txtDescription.setTask(task.getDescription());~: . . . . . .16: if (notification)

{. . . . .

}~: . . . . 27: chbRegular.setEnabled(task == null);

~: . . . .

}

34

Error Invariant Automaton 1: public TaskDialog(Tast task)

~: . . . . . . 6: txtDescription.setTask(task.getDescription());~: . . . . . .16: if (notification)

{. . . . .

}~: . . . . 27: chbRegular.setEnabled(task == null);

~: . . . .

}

line 1 - 5

line 6

line 7 - 26

line 27

line 28 - end

No Effect on inconsistency

assert ( task != null )

Arbitrary codeNo effect on task == null

An assertion that task might be null

No Effect on inconsistency

35

Error Invariant Automaton 1: public TaskDialog(Tast task)

~: . . . . . . 6: txtDescription.setTask(task.getDescription());~: . . . . . .16: if (notification)

{. . . . .

}~: . . . . 27: chbRegular.setEnabled(task == null);

~: . . . .

}

line 1 - 5

line 6

line 7 - 26

line 27

line 28 - end

No Effect on inconsistency

assert ( task != null )

Arbitrary codeNo effect on task == null

An assertion that task might be null

No Effect on inconsistency

36

Error Invariant Automaton 1: public TaskDialog(Tast task)

~: . . . . . . 6: txtDescription.setTask(task.getDescription());~: . . . . . .16: if (notification)

{. . . . .

}~: . . . . 27: chbRegular.setEnabled(task == null);

~: . . . .

}

line 1 - 5

line 6

line 7 - 26

line 27

line 28 - end

No Effect on inconsistency

assert ( task != null )

Arbitrary codeNo effect on task == null

An assertion that task might be null

No Effect on inconsistency

37

Error TraceAn error trace is a sequence of statements π = st0st1... stn, together with and

. describes the initial state and is an assertion that is violated.

That means, in an error trace Λ PF( π ) Λ

is unsatisfiable.

38

Error TraceAn error trace is a sequence of statements π = st0st1... stn, together with and

. describes the initial state and is an assertion that is violated.

That means, in an error trace Λ PF( π ) Λ

is unsatisfiable.Example:

Λ task null Λ task null Λ

39

Error InvariantAn error invariant for a position ∈ [ ] in an error trace is a first order logical formula such that.● The conjunction of the first order logical formulas for each statement

implies Ii.● Ii and the conjunction of the remaining formulas is unsatisfiable.

40

ErrInv( )In the previous work, the authors introduced a function ⊼ which given an error trace, computes:

I0,sti1,I1,sti2. . . stik,IkSuch that,sti1, sti2 …..stik is a subsequence of ⊼ and Ij is an inductive invariant for the position ij and ij+1 .

41

Inductive error invariantWe say that an error invariant is inductive for position i < j if :

42

Inductive error invariantWe say that an error invariant is inductive for position i < j if :

43

Inductive error invariantWe say that an error invariant is inductive for position i < j if :

is called an inductive error invariant.

44

Error Invariant AutomatonAn Error Invariant Automaton is an inconsistent program automaton with a mapping from locations of to state formulas, such that for all locations , is an error invariant for .

45

AlgorithmNow, after applying step 1 we got a single path π A.

Step 2: Apply ErrInv( π A )ErrInv(π A) = ErrInv( π )

= I0st(li1). . . . . . st(lik)Ik.

Algorithm

47

Now, after applying step 1 we got a single path π A.

Step 2: Apply ErrInv( π A )ErrInv(π A) = ErrInv( π )

= I0st(li1). . . . . . st(lik)Ik.

I2

I5

I3

I4

I1

I0

Algorithm

48

Now, after applying step 1 we got a single path π A.

Step 2: Apply ErrInv( π A )ErrInv(π A) = ErrInv( π )

= I0st(li1). . . . . . st(lik)Ik.

I2

I5

I3

I4

I1

I0

errorinvariants

Algorithm

49

Example:

assume(task !=null)

assume(task =null)

Algorithm

50

Example:

assume(task !=null)

assume(task =null)

assume(task !=null)

assume(task =null)

true

task != null

false

ErrInv()

AlgorithmStep 3:The locations covered with an inductive error invariant can be collapsed into a single location.

51

I0I1

I5

I1

I4

I0

I0

I1

I4

I5

AlgorithmStep 4: For each remaining non-atomic statement, apply the algorithm recursively to all these smaller automata.

52

I0

I5

I2

I1

AlgorithmStep 4: For each remaining non-atomic statement, apply the algorithm recursively to all these smaller automata.

53

I0

I5

Apply the algo recursively to these non-atomic statements

I2

I1

AlgorithmStep 4: For each remaining non-atomic statement, apply the algorithm recursively to all these smaller automata.

54

I0

I5

In case of this location

Pre

PostI2

I1

AlgorithmStep 4: For each remaining non-atomic statement, apply the algorithm recursively to all these smaller automata.

55

I0

I5

In case of the location with invariant I2

Inconsistent w.r.t pre and post

PostI2

I1 Pre

AlgorithmStep 4: For each remaining non-atomic statement, apply the algorithm recursively to all these smaller automata.

56

I0

I5

In case of the location with invariant I2

I2

I1

AlgorithmStep 4: For each remaining non-atomic statement, apply the algorithm recursively to all these smaller automata.

57

I0

I5

In case of the location with invariant I2

pre

post

I2

recursively

I1

AlgorithmStep 4: For each remaining non-atomic statement, apply the algorithm recursively to all these smaller automata.

58

I0

I5

I2

I

I1

Algorithm1: public TaskDialog(Tast task)~: . . . . . . 6: txtDescription.setTask(task.getDescription());~: . . . . . .16: if (notification)

{. . . . .

}~: . . . . 27: chbRegular.setEnabled(task == null);

~: . . . .

}

59

EvaluationsThe approach was tested on 6 real world examples.For each of these examples, Error invariant automatas were generated using the technique introduced.All the generated error invariant automatas represented real world inconsistencies with no false alarms.Running time to prove inconsistency using unsat ranged from 0.008 seconds in one of the experiments to 0.019 seconds.

60

EvaluationsUsability testing was also conducted on 11 programmers.Half of the test subjects were shown the full programs, while the other half were just shown the error invariant automata.All candidates took 1 hour and 6 minutes to identify the bug.For the full programs without error invariant automata : 51 minutesWith error invariant automata : 17 minutes

61

ConclusionThe experiments indicate that EIA provide useful visual assistance to spot inconsistencies.EIA can also be used for fault localization on a single trace and thus provide a general tool to assist programmers in debugging.

62

References

63

Martin Schaf, Daniel Schawrtz, Thomas Wies Book Title: Joint meeting of the European Software Engineering conference and the Symposium on the Foundations of Software Engineering, ESEC/FSE’13, Saint Petersburg, Russian Federation, August18-26,2013 url: http://dl.acm.org/citation.cfm?id=2491411 Title: Explaining Inconsistent Code, pages: 521 - 531

Bixie - FInd inconsistencies in Java code. http://martinschaef.github.io/bixie/

Jurgen Christ, Evren Ermis, Martin Schaf, Thomas Weis Book Title: Verification, Model Checking, and Abstract Interpretation,14th International Conference, VMCAI 2013, Rome, Italy, January 20-22, 2013. Proceedingsurl: http://link.springer.com/book/10.1007/978-3-642-35873-9 Title: Flow-Sensitive Fault Localization, pages : 189 - 208

.