towards informed swarm verification€¦ · • for large state spaces, we can focus on debugging...

Towards Informed Swarm Verification

Anton Wijs SET meeting 13 april 2010

Table of Contents

•  Model checking •  Swarm Verification •  Informed Swarm Verification •  Conclusions / Future work

Model Driven Software Engineering (MDSE) PAGE 1 01-06-2011

Model Checking (1)

System satisfies p property S

Model M f Formal property

(Modelling Language)

(Temporal Logic)

State space explosion -  reduction techniques needed

-  E.g. Partial Order Reduction


We will deal mainly with explicit state, action-based model checking (LTSs)

Model Checking (2)

•  For large state spaces, we can focus on debugging instead of verification (guaranteeing bug-absence)

•  Techniques such as swarm verification (SV) [Holzmann et al., ‘08] are very effective for debugging in parallel

•  Contribution: extend SV for verification •  Make SV robust for bug-free state spaces

PAGE 3 01-06-2011 Model Driven Software Engineering (MDSE)

Properties

-  For safety properties, goal is to find “bad” states -  Reachability analysis suffices -  BFS / DFS are suitable -  BFS high memory reqs. / short counter-examples -  DFS low memory reqs. / long(er) counter-examples -  BFS very suitable for distributed / multi-core computing


Parallel Model Checking

•  These days, main memory is a decreasingly important bottleneck

•  Instead, a time explosion problem emerges •  Parallel model checking techniques become

appealing: •  Multi-core model checking: still a lot of work to do •  Distributed model checking: often needs lots of

communication between workers (another bottleneck!)


Grid Computing

•  SETI@Home

PAGE 6 01-06-2011

Embarrassingly Parallel! [Foster, ‘95]

Model Driven Software Engineering (MDSE)

Grid Computing [image: mcrl2.org]

•  MC@Home?

PAGE 7 01-06-2011

Not obvious


Embarrassingly Parallel Verification

•  Swarm Verification (SV) [Holzmann et al., ‘08] •  Parallel Randomized DFS

[Sivaraj et al., ‘03, Dwyer et al., ‘07]

•  Require no communication between workers •  No need for synchronisation, workers can start

whenever they want •  May perform duplicate work, but this is tolerated


Bug-hunting

PAGE 9 01-06-2011

If multiple bugs present, BFS finds the one with shortest trace first. Can be parallelised.

DFS can find “deep” bugs quicker, BUT it depends on the area. “Inherently sequential”.

Hard to detect!


Swarm Verification (1)

PAGE 10 01-06-2011

Main idea: To improve fast detection of deep bugs.

DFS inherently sequential, but we can launch multiple DFS based searches

Searches use unique ordering of successors (not just LIFO)!


Swarm Verification (2)

PAGE 11 01-06-2011

Very successful In bug-hunting! [Holzmann et al., ‘08]

- DFTP - DEOS - Fleet

But what about verification?! - No chance of early termination - All searches are exhaustive


Towards Informed SV (1)

•  Can we bound workers to specific areas? •  + No worker will exhaustively search •  BUT: workers together must be exhaustive •  Should remain embarrassingly parallel •  Cumulatively Exhaustive Sets (CESs) of searches

PAGE 12 01-06-2011

{A,B} :R(A)∪ R(B) = SIdeally: R(A)⊂ S ∧ R(B)⊂ S

Iterative searches, but:

Random searches, but:

∃A :R(A) = S{...} →∞


Towards Informed SV (2)

•  : Non-exhaustive searches! •  Directed Model Checking [Edelkamp et al., ‘04]

−  Informed search has guiding function −  Can change state ordering −  Exhaustive: e.g. Dijkstra’s Search, A* −  Non-exhaustive: Nearest-Neighbour, Beam Search −  + Very efficient, useful for bug-hunting − BUT cannot guarantuee bug-absence!

PAGE 13 01-06-2011

R(A)⊂ S

f :S→ N

What about e.g. sets of Beam Searches?!


System of Non-Communicating Processes (1)



PAGE 19 01-06-2011

Informed swarm of two workers: Each worker can ignore 1/3 of states


Informed SV Method (1)

•  Analyse subsystem M of system N •  Subset of set of parallel processes •  Graph of M can still be very small

•  Result is set of traces Σ through behaviour M •  Each worker explores graph of N restricted to σ •  Traces as guiding functions


Informed SV Worker Algorithm (1)


Informed SV (1)

PAGE 22 01-06-2011

Given: - Model N - Trace σ through M <a, b, c, …>


a b c

b b

Informed SV Method (2)

•  Analyse graph subsystem M of system N •  Subset of set of parallel processes

•  Result is set of traces Σ through graph of M •  Each worker explores graph M x N restricted to σ

•  Trivial for non-communicating processes •  What about communicating processes?

PAGE 23 01-06-2011

PROBLEM: if two traces σ, σ’ through graph M lead to same state s, this may not be the case in graph N !


System of Communicating Processes (1)



PAGE 25 01-06-2011

{< push_button(2), get_tea, push_button(2), get_tea, walk_away > }



PAGE 26 01-06-2011

{< push_button(2), get_tea, push_button(2), get_tea, walk_away >,

< push_button(2), get_tea, push_button(1), get_coffee > }



PAGE 27 01-06-2011

{< push_button(2), get_tea, push_button(2), get_tea, walk_away >,

< push_button(2), get_tea, push_button(1), get_coffee >,

< push_button(1), get_coffee > }



PAGE 28 01-06-2011

Never reached! Model Driven Software Engineering (MDSE)


•  Perform analysis graph M with DFS with only stack?

PAGE 29 01-06-2011

Very inefficient: every state with n incoming traces needs to be explored n times!

Alternative: construct weighted graph M


Identifying Traces graph M

PAGE 30 01-06-2011

Traces can be identified with IDs!

E.g. Trace 3: - 0 [0-4> - 2 [2-4> - 3 [2-4> - 5 [3-4> - 6 [3-4> - 7 [3-4>


Informed SV Method

PAGE 31 01-06-2011

- Weighted graph M - Set of unexplored traces Σ

?

Model N + unexplored trace

Feedback: What was seen?

Prune in Σ


Informed SV (2)

PAGE 32 01-06-2011

Given: - Model N - Trace σ through M <a, b, c, …>


a b c

c d

Feedback sets: 0: { a, b, c }

1: { c, d }

Informed SV Worker Algorithm (2)


Experimental Results

•  Prototype worker in LTSmin toolset [Blom et al., ‘10] •  Standalone prototype of manager •  Experiments mostly “simulate” grid environment


-  Searches are still diverse and reach great depths quickly -  Each worker explores a fraction of the state space

-  ½ % DRM, ⅙ % 1394

estimated time

125,139 s 19,477 s

17,325 s 105,020 s 60,784 s

7,294 s

Related Work

•  [Lerda & Sista, ‘99] distribute work based on single process behaviour, not embarrassingly parallel

•  [Groce & Joshi, ‘08] Restrict analysis of program based on trace of events (slicing on C program)

•  [Staats & Păsăreanu, ‘10] Generate test cases for software testing based on “shallow” analysis of symbolic execution trees


Conclusions & Future Work

•  Presented an informed swarm verification technique suitable for grid model checking

•  Experimental results very promising

•  Plans: •  Perform more experiments (compare with SV) •  Improve method −  Support infinite subsystem behaviour −  Reduce over-approximations −  Automatic selection of suitable subsystem

•  Investigate manager selection of σ •  State-based method


01-06-2011 Model Driven Software Engineering (MDSE)

towards informed swarm verification€¦ · • for large state spaces, we can focus on debugging...

Documents