exploiting sat solvers in unbounded model checking k. l. mcmillan cadence berkeley labs

Exploiting SAT solvers in unbounded model checking

K. L. McMillanCadence Berkeley Labs

Overview

• We will examine the performance of two SAT-based abstraction methods– Counterexample-based abstraction– Proof-based abstraction

• This tells us something about– The nature of decision heuristics in SAT solvers– The strengths are weaknesses of SAT solvers

for bounded model checking and related applications.

Outline

• Background: SAT and BMC• Localization abstraction

– Cex-based– Proof-based

• Performance study– what it tells us about SAT

DPLL-style SAT solvers

• Objective:– Check satisfiability of a CNF formula

• literal: v or v• clause: disjunction of literals• CNF: conjunction of clauses

• Approach:– Branch: make arbitrary decisions– Propagate implication graph– Use conflicts to guide inference steps

SATO,GRASP,CHAFF,BERKMIN

The Implication Graph (BCP)

(a b) (b c d)

Decisions

Assignment: a b c d

Resolution

a b c a c d

When a conflict occurs, the implication graph isused to guide the resolution of clauses, so that thesame conflict will not occur again.

Conflict Clauses

(a b) (b c d) (b d)

Decisions

Assignment: a b c d

Conflict!

(b c )

resolve

Conflict!(a c)

resolve

Conflict!

Conflict Clauses (cont.)

• Conflict clauses:– Are generated by resolution– Are implied by existing clauses– Are in conflict in the current assignment– Are safely added to the clause set

Many heuristics are available for determiningwhen to terminate the resolution process.

Basic SAT algorithm

empty clause?

yUNSAT

conflict?Deduce conflict

clause andbacktrack

is Atotal?

Branch:add some literal

Generating refutations

• Refutation = a proof of the null clause– Record a DAG containing all resolution steps

performed during conflict clause generation.– When null clause is generated, we can extract a

proof of the null clause as a resolution DAG.

Original clauses

Derived clauses

Null clause

Circuit SAT

Can the circuit output be 1?

inputvariables output

variable

(a g) (b g)(a b g)

(g p) (c p)(g c p)

CNF(p)

p is satisfiable when theformula CNF(p) pis satisfiable

Bounded Model Checking

• Given– A finite transition system M– A property p

• Determine– Does M allow a counterexample to p of k

transitions of fewer?

This problem can be translated to a SAT problem

BCCZ99

Models

Transition system described by a set of constraints

Each circuit element is a constraintnote: a = at and a' = at+1

g = a b

p = g c

c' = p

Model:

C = { g = a b, p = g c, c' = p }

Properties

• We restrict our attention to safety properties.

• Characterized by:– Initial condition I– Final condition F (representing "bad" states)

• A counterexample is a path from a state satisfying I to state satisfying F, where every transition satisfies C.

Unfolding

• Unfold the model k times: Uk = C0 C1 ... Ck-1

...I0 Fk

• Use SAT solver to check satisfiability of I0 Uk Fk

• A satisfying assignment is a counterexample of k steps

BMC applications

• Debugging:– Can find counterexamples using a SAT solver

• Proving properties:– Only possible if a bound on the length of the

shortest counterexample is known.• I.e., we need a diameter bound. The diameter is the

maximum lenth of the shortest path between any two states.

– Worst case is exponential. Obtaining better bounds is sometimes possible, but generally intractable.

Localization abstraction

• Property: G (c X c)

Model:

C = { g = a b, p = g c, c' = p }

free variable

C'property, C C' C property

Kurshan

Constraint granularity

g Model:

C = { c' = (a b) c }

Most authors use constraints at "latch" granularity...

...however, techniques we will consider can be applied atboth "gate" and "latch" granularity.

Localization, cont• C' may refer to fewer state variables than C

– reduction in the state explosion problem

• Key issue: how to choose constraints in C'– counterexample-based– proof-based

Algorithm

Model checkabstraction C'

Choose initial C'

Can extend Cexfrom C'to C?

Add constraintsto C'

true, done

yes, Cex

SAT uses

Kurshan

Abstract counterexamples

• Assume simple safety property:– initial condition I and final condition F– w.l.o.g., assume I and F are atomic formulas

• to make this true, add constraints in C: vI I vF F

• Abstract variables V' = support(C',I,F)• Abstract counterexample A' is a truth

assignment to: { vt | v in V', t in 0..k }

where k is the number of steps.

Counterexample extension

• Abstract counterexample A' satisfies: I0 U'k Fk where U'k = C'0 C'1 ... C'k-1

• Find A consistent with A', satisfying: I0 Uk Fk where Uk = C0 C1 ... Ck-1

• That is, A is any satisfying assignment to:

A' I0 Uk Fk

I.e., to extend an abstract counterexample, we justapply it as a constraint in BMC. If unsat, abstractcounterexample is "false".

CGJLV 2000

Abstraction refinement

• Refinement = adding constraints to C' to eliminate false counterexamples.

• Many heuristsics used for this.

– Too many to cover here.

– Recall that a SAT solver can produce a resolution-based refutation in the UNSAT case....

Proof-based refinement

• Recall, to extend abstract Cex A', we check: A' I0 Uk Fk

• If UNSAT, we obtain refutation proof P– proof that A' cannot be extended to concrete Cex

• Let E be set of constraints used in proof P:E = { c C | some ci occurs in P }

• A' cannot be extended to a Cex for E– P is the proof of this.

Thus, add E to C' and continue...

In other words...

The refutation of the formula: A' I0 Uk Fk

gives us a sufficient set of constraints to rule out the abstract counterexample.

We continue ruling out counterexamples until either theabstraction C' proves the property or we can extend anabstract counterexample to a concrete one.

CCKSVW approach (FMCAD02)• Find the shortest prefix of Cex A' that cannot be extended.

• That is, A' I0 Uk Fk

is feasible for all k < i, but not for k=i.

s0 s1 s2 si-1 si...

OK OK OK OK NO!

CCKSVW approach cont.

• Let P be a refutation of A' I0 Ui Fi

• Let E be set of constraints used in proof P only on state si-1:

E = { c C | ci-2 occurs in P }

s0 s1 s2 si-1 si...

OK OK OK OK NO!

add constraints used here

Weakness of Cex-based approach

• Arbitrarily chosen abstract Cex may be refutable for many reasons not related to property.– Thus, may add irrelevant constraints.– To remedy, may try to characterize a set of

Cex's rather than just one (e.g., GKM-HFV,TACAS03).

Alternative: don't use counterexamples

Proof-based abstraction

BMCat depth k

Cex?done

No Cex?

Use refutation to choose abstraction

MC abstraction doneTrue?

False?

kMA,TACAS03

BMC phase

• Unfold the model k times: U = C0 C1 ... Ck-1

• Use SAT solver to check satisfiability of I0 U Fk

• If unsatisfiable:• property has no Cex of length k• produce a refutation proof P

Abstraction phase

• Let C' be set of constraints used in proof P:C' = { c C | some ci occurs in P }

• C' admits no counterexample of length k– let U' = C'0 C'1 ... C'k-1

– P is a refutation of I0 U' Fk

• Model check property on C'– property true for C' implies true for C– else Cex of length k' > k (why?)

• restart for k = k'

Algorithm

BMCC at depth k

Cex?done

No Cex?

Refutation P inducesabstraction C'

Model check C' doneTrue?

Cex of depth k'?

Notice: MC counterexample is thrown away!

Termination

• Depth k increases at each iteration• Eventually k > d, diameter of C'• If k > d, no counterexample is possible

In practice, termination uses occurs when k d/2

Usually, diameter C' << diameter of C

Weakness of proof-based abs

• BMC must refute all counterexamples of length k, while in Cex-based, BMC must refute only one (partial) counterexample.

PicoJavaII benchmarks

• Hardware Java virtual machine implementation• Properties derived from verification of ICU

– handles cache, instruction prefetch and decode

• Original abstraction was manual• Added neigboring IFU to make problem harder

ICU IFUMem,Cache

Integerunit

properties

No properties can be verified by standard model checking!

Abstraction results

solid = original, gray = manual, open = proof-based abstraction

305 306 306 305

Inference

• SAT solver seems to be very effective at narrowing down the proof to relevant facts.

In most cases, it did better than manual abstraction.

Comparing CBA and PBA

• Apples-apples comparison– same SAT solver– same model checker– only differences are:

• For CBA previous A' is kept as a constriaint for BMC, C' is cumulative.

• For PBA previous A' and C' are thrown away each iteration.

Note these are my implementations. This says nothingabout performance of specific tools!

Run time comparison

0.01 0.1 1 10 100 1000

Counterexample-based abstraction

Abstraction comparison

0 10 20 30 40 50 60

Counterexample-based abstraction

Possible explanation

• Internally, SAT solver is really doing CBA

a=0b=1c=0d=1

decision stack= abstract Cex A'

refutation of A'

decision heuristicmoves proof variablesup, into A'

PBA run-time breakdown

solid = BMC time, open = MC time

CBA run-time breakdown

solid = BMC time, open = MC time

A (fuzzy) hypothesis

• Parameterized models allowing no abstraction

SAT-based BMC "succeeds" when number of relevant variables is small, and fails otherwise.

"success" is BMC for k = diameter of relevant logic

Model Max state vars

German protocol 42

"swap" 21

Industrial benchmarks

0 100 200 300 400 500 600 700

Original state variables

Implications for model checking

• Most of the time if bounded model checking succeeds, unbounded model checking also succeeds using abstraction.

• No need to settle for time bounded result

Conclusions

• SAT solvers are very effective at ignoring irrelevant facts– Can think of decision heuristic as a form of CBA

• implications for improving heuristics?

– Hence very similar performance of CBA and PBA for localization abstraction

• SAT solver performance is tied to number of relevant variables– Performs well if there is a small UNSAT "core"– Performs badly when all variables relevant.

exploiting sat solvers in unbounded model checking k. l. mcmillan cadence berkeley labs

b c p g

c d b c d

g c x c

b b c d b d

c c c property kurshan

b c d d slide

b g g p c p g c p cnfp

b c resolve conflict

Documents

equation solvers

bounded and unbounded...

ode solvers

mcmillan w6a1

unbounded symmetric homogeneous domains in...

mcmillan portfolio

1 introduction to model checking ken mcmillan cadence...

sat solvers

mcmillan magnifier

21st century unbounded university

addition - unbounded rationality

robert mcmillan - backup

numerical methods for the dynamics of unbounded …...

ingles mcmillan

coupled feti/beti solvers for nonlinear potential problems...

duncan mcmillan

footprints 3 mcmillan

170840621 mcmillan movers

chapter 12 unbounded linear...

ilp solvers