Using First-Order Theorem Provers in Data Structure Verification

Charles Bouillaguet

Ecole Normale Supérieure, Cachan, France

Viktor KuncakMartin Rinard

MIT CSAIL

Implementing Data Structures is Hard

Often small, but complex code Lots of pointers Unbounded, dynamic allocation Complex shape invariants

Dag Properties involving arithmetic (ordering…)

Need strong invariants to guarantee correctness e.g. lookup in ordered tree needs sortedness

How to obtain reliable data structure implementations?

Approach Prove that the program is correct For all program executions (sound)

Verified properties: Data structure operations do not crash Data structure invariants are preserved Data structure content is correctly updated

Infrastructure

Jahob system for verifying data structure implementation Kuncak, Wies, Zee, Rinard, Nguyen,

Bouillaguet, Schmitt, Marnette, Bugrara

Analyzed programs: subset of Java

Specification : subset of Isabelle’s language

Summary of Verified Data Structures

Implementations of relations Add a binding Remove all bindings for a given key Test key membership Retrieve data bound to a key Test emptiness

Verified implementations Linked list Ordered tree Hash table

An Example : Ordered Trees

Implementation of a finite map Operations

insert lookup remove

Representation invariants: tree shaped (acyclicity, unique

parent) ordering constraints

leftright

keyvalue

public static FuncTree update(int k, Object v, FuncTree t) { FuncTree new_left, new_right; Object new_data; int new_key; if (t==null) { new_data = v; new_key = k; new_left = null; new_right = null; } else { if (k < t.key) { new_left = update(k, v, t.left); new_right = t.right; new_key = t.key; new_data = t.data; } else if (t.key < k) { … } else { new_data = v; new_key = k; new_left = t.left; new_right = t.right; } } FuncTree r = new FuncTree(); r.left = new_left; r.right = new_right; r.data = new_data; r.key = new_key; return r;}

Sample code

public static FuncTree update(int k, Object v, FuncTree t)/*: requires "v ~= null” ensures "result..content = t..content - {(x,y). x=k} + {(k,v)} */{ FuncTree new_left, new_right; Object new_data; int new_key; if (t==null) { new_data = v; new_key = k; new_left = null; new_right = null; } else { if (k < t.key) { new_left = update(k, v, t.left); new_right = t.right; new_key = t.key; new_data = t.data; } else if (t.key < k) { … } else { new_data = v; new_key = k; new_left = t.left; new_right = t.right; } } FuncTree r = new FuncTree(); r.left = new_left; r.right = new_right; r.data = new_data; r.key = new_key; //: "r..content" := "t..content - {(x,y). x=k} + {(k,v)}";

return r;

}

3 lines spec30 lines code

no null dereferences

Sample code

postcondition holds and invariants preserved

Ordered tree: interfacepublic ghost specvar content :: "(int * obj) set" = "{}";

public static FuncTree empty_set()ensures "result..content = {}"

public static FuncTree add(int k, Object v, FuncTree t)requires "v ~= null & (ALL y. (k,y) ~: t..content)”ensures "result..content = t..content Un {(k,v)}”

public static FuncTree update(int k, Object v, FuncTree t)requires "v ~= null”ensures "result..content = t..content - {(x,y). x=k} + {(k,v)}”

public static Object lookup(int k, FuncTree t) ensures "((k, result) : t..content) | (result = null & (ALL v. (k,v) ~: t..content))”

public static FuncTree remove(int k, FuncTree t)ensures "result..content = t..content - {(x,y). x=k}”

Representation Invariantspublic final class FuncTree{

private int key;private Object data;private FuncTree left, right;

/*: public ghost specvar content :: "(int * obj) set"; invariant ("content definition") "this ~= null --> content = {(key, data)} Un left..content Un right..content"

invariant ("null implies empty") "this = null --> content = {}"

invariant ("left children are smaller") "ALL k v. (k,v) : left..content --> k < key”

invariant ("right children are bigger") "ALL k v. (k,v) : right..content --> k > key"

*/

equality between sets

implicit universal quantification over this

explicit quantification

arithmetic

tuples

abstract set-valued field

How could these properties be verified?

Standard Approach

Transform program into a logic formula Using weakest precondition The program is correct iff the formula is valid

Prove the formula Very difficult formulas: interactively (Coq, Isabelle) Decidable classes: automated (MONA, CVCL, Omega) This talk: difficult formulas in automated way :)

eauto ; intros . intuition ; subst . apply Extensionality_Ensembles. unfold Same_set. unfold Included. unfold In. unfold In in H1.intuition. destruct H0. destruct (eq_nat_dec x1 ArraySet_size).subst. rewrite arraywrite_match in H0 ; auto. intuition. subst. apply Union_intror. auto with sets. assert (x1 < ArraySet_size). omega. clear n. apply Union_introl. rewrite arraywrite_not_same_i in H0.unfold In. exists x1. intuition.omega.

inversion H0 ; subst ; clear H0. unfold In in H3. destruct H3. exists x1. intuition. rewrite arraywrite_not_same_i. intuition ; omega. omega. exists ArraySet_size. intuition. inversion H3. subst. rewrite arraywrite_match ; trivial.

low efficiency 1 line per grad student-minute parallelization looks non-trivial

Formulas in Jahob

Very expressive specification language Higher-Order features

How to prove formulas automatically?

Convert them to something simpler Decidable classes First-Order Logic

Automated reasoning in Jahob

Why FOL?

Existing theorem provers: SPASS, E, Vampire, Theo, Prover9, … continuously improving (yearly competition)

Effective on formulas with short proofs

Handle nicely formulas with quantifiers

HOL FOL Ideas :

avoid axiomatizing rich theories Translate what can naturally be expressed in FOL soundly approximate the rest Sound, incomplete approach

Full details in long version of the paper (x,y) z.content ∈ Content(x,y,z)⇢ w.f := y ⇢(x=y ⋀ w=v) ⋁ (x ≠ y ⋀ w=f(y) )

λx.E = λx.F⇢ x. E=F∀ …

Arithmetic

Numbers are uninterpreted constants in FOL Provers do not know that 1+1=2 ! Still need to reason about arithmetic…

Our Solution Provide partial, incomplete axiomatization

• Still cannot deduce 1+1=2 !

• comparison between constants in formula

Satisfactory results in practice• ordering of elements in tree• array bound checks

Observation

Most formulas are easy to prove ie in no measurable time have very short proofs (in # of resolution step)

Problem often concentrated in a small number that take very long to prove

We applied two existing techniques to make them easier1. Eliminating type/sort information2. Filtering unnecessary assumptions

Sort Information

Specification language has sorts Integers Objects Boolean

Translate to unsorted FOL

∀(x : Obj). P(x)

⇣∀x. Obj(x) P(x)⇒

Sort Information

Encoding sort information bigger formulas longer proofs

Formulas become harder to prove

Temptation to omit sort information…

Effect on hard formulas Formulas that take more than 1s to prove, from

the Tree implementation (SPASS)

BenchmarkTime (s) Proof length

Generated clauses

with w/o with w/o with w/o

Tree.remove

5.3 1.1 799 155 18 376 9 425

3.6 0.3 1 781 309 19 601 1 917

9.8 4.9 1 781 174 33 868 27 108

8.1 0.5 1 611 301 31 892 3 922

8.1 4.7 1 773 371 37 244 28 170

7.9 0.3 1 391 308 41 354 3 394

Tree.remove_max

+∞ 0.22 97 1 075

78.9 6.8 2 655 1 159 177 755 19 527

34.8 0.8 4 062 597 115 713 5 305

Omitting Sorts (cont’d)

Great speed-up (more than x10 sometimes) ! However:

∀ (x y:S). x = y

∃ (x y:T). x ≠ y

Satisfiable with sorts (S={a}, T={b,c})…

Unsatisfiable without!

Omitting sort guards breaks soundness!!!

Possible workaround: type-check generated proof

When it is possible to skip type-checking ?

Omitting Sorts Result

We proved the following

Theorem. Suppose thati. Sorts are pair-wise disjoint (no sub-sorting)

ii. Sorts have the same cardinality

Then omitting sort guards is

sound and complete

This justify this useful optimization

Assumption Filtering

Provers get confused by too many assumptions

Lots of useless assumptions Hardest shown benchmark needs 12 out of 56 Big benchmark: on average 33% necessary

Assumption filtering Try to eliminate irrelevant assumptions automatically Give a score to assumption based on relevance

Experimental results

Benchmarklines of

codelines of

specification# of

methodsverif. time

Sets as imperative list

60 24 9 18s

Relation as functional Linked list

76 26 9 12s

Relation as functional Ordered trees

186 38 10 178s

Relation as hash table (using f.list)

69 53 10 119s

Verification effort

Decreased as we improved the system functional list was easy a few days for trees two hours for simple hash table

FOL : Currently most usable method for these kind of data structures

Related work

Interactive Provers – Isabelle, Coq, HOL, PVS, ACL2 First-Order ATP

Vampire – Voronkov [04] SPASS – Weidenbach [01] E – Shultz [IJCAR04]

Program Checking ESC/Java2 – Kiniry, Chalin, Hurlin Krakatoa – Marche, Paulin-Mohring, Urbain [03] Spec# – Barnett, DeLine, Jacobs, Fähndrich, Leino, Schulte, Venter

[05] Hob system: verify set implementations (we verify relations)

Shape analysis PALE - Møller and Schwartzbach [PLDI01] TVLA - Sagiv, Reps, and Wilheim [TOPLAS02] Roles - Kuncak, Lam, and Rinard [POPL02]

Multiple Provers - Screenshot

Conclusion

Jahob verification system Automation by translation HOLFOL

omitting sorts theorem gives speedup filtering automates selection of assumptions

Promising experimental results strong properties: correct implementation

• Do not crash• operations correctly update the content, clarifies behavior in

case of duplicate keys, …• representation invariants preserved (ordering, treeness, each

element is in appropriate bucket) relatively fast verification effort much smaller than using interactive

provers

Thank you

Formal Methods are the Future of computer Science.

Always have been…

Always will be.

Questions ?

Converting to GCL

Conditionnal statement: easy [| if cond then tbranch else fbranch |] =

(Assume cond; [| tbranch|] ) (Assume !cond; [| fbranch|] )

Procedure calls: Could inline (potentially exponential blowup) Desugaring (modularity) :

• [| r = CALL m(x, y, z) |] = Assert (m’s precondition);

Havoc r;

Havoc {vars modified by m} ;

Assume (m’s postcondition)

Loops: invariant required [| while /*: invariant */ (condition) {lbody} |] =

assert invariant;

havoc vars(lbody);

assume invariant;

((assume condition;

[| lbody |];

assert invariant;

assume false)

(assume !condition))

Converting to GCL (cont’d)

invariant hold initially

no assumptions on variables except that invariant hold

condition hold

or condition do not hold and execution continues

invariant is preserved

no need to verify anything more

Verification condition for remove ((((fieldRead Pair_data null) = null) & ((fieldRead FuncTree_data null) = null) & ((fieldRead

FuncTree_left null) = null) & ((fieldRead FuncTree_right null) = null) & (ALL (xObj::obj). (xObj : Object)) & ((Pair Int FuncTree) = {null}) & ((Array Int FuncTree) = {null}) & ((Array Int Pair) = {null}) & (null : Object_alloc) & (pointsto Pair Pair_data Object) & (pointsto FuncTree FuncTree_data Object) & (pointsto FuncTree FuncTree_left FuncTree) & (pointsto FuncTree FuncTree_right FuncTree) & comment ''unalloc_lonely'' (ALL (x::obj). ((x ~: Object_alloc) --> ((ALL (y::obj). ((fieldRead Pair_data y) ~= x)) & (ALL (y::obj). ((fieldRead FuncTree_data y) ~= x)) & (ALL (y::obj). ((fieldRead FuncTree_left y) ~= x)) & (ALL (y::obj). ((fieldRead FuncTree_right y) ~= x)) & ((fieldRead Pair_data x) = null) & ((fieldRead FuncTree_data x) = null) & ((fieldRead FuncTree_left x) = null) & ((fieldRead FuncTree_right x) = null)))) & comment ''ProcedurePrecondition'' (True & comment ''FuncTree_PrivateInv content definition'' (ALL (this::obj). (((this : Object_alloc) & (this : FuncTree) & ((this :: obj) ~= null)) --> ((fieldRead (FuncTree_content :: (obj => ((int * obj)) set)) (this :: obj)) = (({((fieldRead (FuncTree_key :: (obj => int)) (this :: obj)), (fieldRead (FuncTree_data :: (obj => obj)) (this :: obj)))} Un (fieldRead (FuncTree_content :: (obj => ((int * obj)) set)) (fieldRead (FuncTree_left :: (obj => obj)) (this :: obj)))) Un (fieldRead (FuncTree_content :: (obj => ((int * obj)) set)) (fieldRead (FuncTree_right :: (obj => obj)) (this :: obj))))))) & comment ''FuncTree_PrivateInv null implies empty'' (ALL (this::obj). (((this : Object_alloc) & (this : FuncTree) & ((this :: obj) = null)) --> ((fieldRead (FuncTree_content :: (obj => ((int * obj)) set)) (this :: obj)) = {}))) & comment ''FuncTree_PrivateInv no null data'' (ALL (this::obj). (((this : Object_alloc) & (this : FuncTree) & ((this :: obj) ~= null)) --> ((fieldRead (FuncTree_data :: (obj => obj)) (this :: obj)) ~= null))) & comment ''FuncTree_PrivateInv left children are smaller'' (ALL (this::obj). (((this : Object_alloc) & (this : FuncTree)) --> (ALL k. (ALL v. (((k, v) : (fieldRead (FuncTree_content :: (obj => ((int * obj)) set)) (fieldRead (FuncTree_left :: (obj => obj)) (this :: obj)))) --> (intless k (fieldRead (FuncTree_key :: (obj => int)) (this :: obj)))))))) & comment ''FuncTree_PrivateInv right children are bigger'' (ALL (this::obj). (((this : Object_alloc) & (this : FuncTree)) --> (ALL k. (ALL v. (((k, v) : (fieldRead (FuncTree_content :: (obj => ((int * obj)) set)) (fieldRead (FuncTree_right :: (obj => obj)) (this :: obj)))) --> ((fieldRead (FuncTree_key :: (obj => int)) (this :: obj)) < k))))))) & comment ''t_type'' (((t :: obj) : (FuncTree :: obj set)) & ((t :: obj) : (Object_alloc :: obj set)))) --> ((comment ''TrueBranch'' (((t :: obj) = null) :: bool) --> (comment ''ProcedureEndPostcondition'' ((((fieldRead (FuncTree_content :: (obj => ((int * obj)) set)) (null :: obj)) = ((fieldRead (FuncTree_content :: (obj => ((int * obj)) set)) (t :: obj)) - {p. (EX x y. ((p = (x, y)) & (x = (k :: int))))})) & (ALL (framedObj::obj). (((framedObj : Object_alloc) & (framedObj : FuncTree)) --> ((fieldRead FuncTree_content framedObj) = (fieldRead FuncTree_content framedObj))))) & comment ''FuncTree_PrivateInv content definition'' (ALL (this::obj). (((this : Object_alloc) & (this : FuncTree) & ((this :: obj) ~= null)) --> ((fieldRead (FuncTree_content :: (obj => ((int * obj)) set)) (this :: obj)) = (({((fieldRead (FuncTree_key :: (obj => int)) (this :: obj)), (fieldRead (FuncTree_data :: (obj => obj)) (this :: obj)))} Un (fieldRead (FuncTree_content :: (obj => …

And 200 more kilobytes…

Infeasible to prove directly

Splitting heuristic

Verification condition is big conjunction conjunctions in postcondition proving each invariant proving each branch in program

Solution: split VC into individual conjuncts Prove each conjunct separately Each conjunct has form

H1 /\ … /\ Hn Gi

Tree.Remove has 230 such conjuncts How do we prove them?

Detupling (cont’d)

Complete rules:

Handling of Fields (cont’d)

We dealt with field updates New function expressed in terms of old one

Base case: field variables Natural encoding in FOL using functions:

x = y.f ! x = f(y)

Future work

Verify more examples balanced trees fancy priority queues (binomial, Fibonacci, …) hash table with dynamic resizing

hash function verify clients of data structures Improve assumption filtering

take rarity of symbols into account check for occurring polarity …