software verification tools: current state and the nearest future

Victor KuliaminInstitute for System Programming Russian Academy of SciencesMoscow

Growth of software complexity Degradation of software quality

2/34

SYRCoSE 2009, May 28

6 10 15 16

138

5145

303

105

216

283

59

25

180

140

80

3020

89

45

40

0

50

100

150

200

250

300

Windows Code, MLOC Debian Linux Code, MLOC

Windows Team, 10p Debian Team, 10p

Bugs per 1000 LOC

Checking consistency between differentdevelopment artifacts, relevant

standards,and between them all and real system

operation

3/34


Design Source Code System OperationRequirements

Development Processes

Standards and Rules

Review (inspection) Static analysis

Correctness rules checking Bug search

Dynamic analysis Monitoring Testing

Formal methods Theorem proving Model checking

4/34


Static analysis

Dynamic analysis

5/34


Requirements & Rules

Source CodeAnalysis Tool

System Operation

Requirements & Rules Monitoring

Environment

UsersTest Generation

Theorem proving [R. Floyd 1967, C. A. R. Hoare 1969] Hoare logic – {Pre} Program {Post} Inference rules

Model checking [E. M. Clarke & E. A. Emerson 1980, J. P. Queille & J. Sifakis 1982] Analysis of reachable states

6/34


Model based testing

Extended static analysis

Runtime verification

Compound structured testing

Auxiliary Symbolic execution Abstract interpretation Constraint inference Constraint resolution

7/34


Formal models

Testing

Static analysis

Monitoring

[J. C. King, L. A. Clark 1976]

8/34


if(x > 0) { y := x+2; }else if(x > -1) { y := x+1; }else { y := x; }

[(x > 0) (y = x+2)]

& [(x ≤ 0 & x > -1) (y = x+1)]

& [[(x ≤ -1) (y = x) ]

[P. Cousot & R. Cousot 1977]

Abstract Domains Octagons x y ≤ a Polyhedra Heap structures Bit vectors …

9/34


...while ( (x == 0) && (2*f(x) <= z+g(y/2.5)) ){ ... x++;}...

...while ( B ){ ... B = false;}...

Daikon 1999 MITM. D. Ernsthttp://groups.csail.mit.edu/pag/daikon/ =, ≠, <, ≤, ax + by + cz + d = 0, x = y2, x % y = 0 , , , , AB=, subsequence, no duplicates,

reverse

Houdini 2001 Flanagan, Leino DIDUCE 2002 Stanford University Agitator 2003 Agitar DySy 2007 Tillmann, Csallner, Smaragdakis

10/34


SAT solvers – DPLL algorithm 1962SMT solvers – Satisfiability modulo Theory CVC 2002 Stanford

D. L. Dill, C. W. Barrett, A. Stump Yices 2005 SRI International

B. Dutertre, L. de Moura Z3 2006 Microsoft Research

L. de Moura, N. S. Bjørner

11/34


SMT-LIBhttp://combination.cs.uiowa.edu/smtlib/

Model based testing

Extended static analysis

Runtime verification

Compound structured testing

12/34


13/34


State model and oracle

State model

System under test

Behavior model

Test action generator

Adequacy metric

12%

Adequacy criteria

36%57%87%

FSM-based testing [F. C. Hennie 1964, M. P. Wasilevsky 1973]

Automata (FSM, LTS, ASM) TGV 1997 INRIA

T. Jéron et al.http://www.inrialpes.fr/vasy/cadp/man/tgv.html

TorX 1999 University of TwenteJ. Tretmans et al.http://fmt.cs.utwente.nl/tools/torx/introduction.html

Gotcha-TCBeans 1999 IBM ResearchA. Hartman et al.

Automata + Software Contracts UniTESK 2000 ISP RAS

A. Petrenko et al.http://www.unitesk.com

SpecExplorer 2004 Microsoft ResearchW. Schulte et al.+ symbolic executionhttp://research.microsoft.com/en-us/projects/SpecExplorer/

14/34


ModelJUnit 2004 Waikato University

M. Utting et al.http://czt.sourceforge.net/modeljunit/index.html

NModel 2007 Univ. of WashingtonMicrosoft Research

J. Jacky, M. Veanes et al.http://nmodel.codeplex.com/

15/34


namespace ClientServer { [Feature] public partial class Server { public static Socket serverSocket = Socket.None; public static Phase phase = Phase.Send; public static bool ServerSocketEnabled() { return (serverSocket == Socket.None); } [Action] public static void ServerSocket() { serverSocket = Socket.Created; }

public static bool ServerBindEnabled() { return (serverSocket == Socket.Created); } [Action] public static void ServerBind() { serverSocket = Socket.Bound; }

public static bool ServerListenEnabled() { return (serverSocket == Socket.Bound); } [Action] public static void ServerListen() { serverSocket = Socket.Listening; }

public static bool ServerAcceptEnabled() { return (serverSocket == Socket.Listening); } [Action] public static void ServerAccept() { serverSocket = Socket.Connected; }

public static bool ServerReceiveEnabled() { return (serverSocket == Socket.Connected && phase == Phase.ServerReceive); } [Action] public static void ServerReceive() { phase = Phase.Send; } }

16/34


17/34


[Feature] public partial class Client { public static Socket clientSocket = Socket.None; public static double clientBuffer = double.MaxValue; public static bool ClientSocketEnabled() { return (clientSocket == Socket.None); } [Action] public static void ClientSocket() { clientSocket = Socket.Created; }

public static bool ClientConnectEnabled() { return (clientSocket == Socket.Created); } [Action] public static void ClientConnect() { clientSocket = Socket.Connecting; }

public static bool ClientSendEnabled() { return (clientSocket == Socket.Connected); } [Action] public static void ClientSend() { phase = Phase.ServerReceive; }

public static bool ClientReceiveEnabled() { return (clientSocket == Socket.Connected); } [Action] public static double ClientReceive(double datum) { clientBuffer = datum; return datum; }

public static bool ClientCloseEnabled() { return (clientSocket == Socket.Created || clientSocket == Socket.Connected); } [Action] public static void ClientClose() { clientSocket = Socket.Closed; } }

18/34


[Feature] public partial class Server {

public static bool ClientConnectEnabled() { return (serverSocket == Socket.Listening); } public static bool ClientSendEnabled() { return (phase == Phase.Send); } [Action] public static void ClientSend() { phase = Phase.ServerReceive; }

public static bool ClientReceiveEnabled() { return (phase == Phase.ClientReceive); } [Action] public static void ClientReceive() { phase = Phase.Send; } }

[Feature] class Values2 { readonly static Set<double> Values = new Set<double>(99.9, 100.0); [Action] static void ClientReceive([Domain("Values")] double datum) {} }

19/34


[Feature] public partial class Client {

public static bool ServerAcceptEnabled() { return (clientSocket == Socket.Connecting); } [Action] public static void ServerAccept() { clientSocket = Socket.Connected; } }}

20/34


Server

21/34


[G. Nelson & J. B. Saxe et al. 1991]

Search for bugs Sound analysis

22/34


Requirements & Rules

Source Code

Behavior model

Analysis Tool

Provers, Solvers,

Model Checkers

ESC/Modula 3 1995 DECG. Nelson, J. B. Saxe, K. R. M. Leino, D. Detlefs

ESC/Java 2000 CompaqK. R. M. Leino, C. Flanagan

ASTREE 2002 CNRSP. Cousot http://www.astree.ens.fr/

ESC/Java 2 2004http://kind.ucd.ie/products/opensource/ESCJava2/

Simplify Spec# Checker 2004 Microsoft Research

K. R. M. Leinohttp://research.microsoft.com/SpecSharp/

Boogie Calysto 2008 University of British Columbia

23/34


Boolean satisfiability (SAT) (x1~x2) & (~x1x3)

Linear integer arithmetics x1 < x2 + 3

Floating-point arithmetics x1 · x2 = 2.5

Polyhedra 0.2x1+3x3 > x2–3.7x4

Ellipsoids 2x12+1.3x2

2 ≤ 76.9

Heap structures x1→p→p ≠ x2→p …

24/34


Counterexample guided abstraction refinement

CEGAR [E. M. Clarke & O. Grumberg et al 2000,

T. Ball & S. K. Rajamani 2000]

25/34


do { nPacketsOld = nPackets; ... if(request) { ... nPackets++; }} while (nPackets != nPacketsOld);

do { b = true; ... if(request) { ... b = b?false:*; }} while (!b);

26/34

SYRCoSE 2009, May 2826 / 14

Behavior Model

Model Checking

Counterexample

Behavior Model

Test Data and ScenarioApproval

Bug!

Refutation

Model Refinement

Correctness Rules

Code under check

SLAM 2001 Microsoft ResearchT. Ball, S. K. Rajamani et al.http://research.microsoft.com/en-us/projects/slam/ Static Driver Verifier 2007 Microsoft

http://www.microsoft.com/whdc/devtools/tools/sdv.mspx BLAST 2003 UC Berkeley

T. A. Henzinger, R. Jhala, R. Majumdar, G. Sutrehttp://mtc.epfl.ch/software-tools/blast/

MAGIC 2003 SCS Carnegie MellonE. M. Clarke, S. Chaki et al.http://www.cs.cmu.edu/~chaki/magic/

27/34


[K. Havelund & W. Visser 1999]

Java Path Finder

+ symbolic execution + test generation

http://javapathfinder.sourceforge.net/

System under check

28/34


Behavior model

Monitoring Environment

State model and oracle

[??? 2003-2004] Targeting to cover various paths in source/byte code Test oracle

No exceptionsNullPointer, IndexOutOfBounds, ClassCast, DivideByZero, IllegalArgument

Annotations and formal models Test data and sequences generation

Random Symbolic execution + constraint resolution State abstraction Heuristic search

29/34


[Y. Smaragdakis, C. Csallner] JCrasher 2004 Check-n-Crash 2005 DSD-Crasher 2006

http://ranger.uta.edu/~csallner/dsd-crasher/

30/34


DaikonDaikon ESC/Java 2ESC/Java 2 solversolver

[P. Godefroid, G. Agha, K. Sen 2005] CUTE 2005

Consolic testing (concrete + symbolic) jCUTE 2006

31/34


Execution

Program

Symbolic Execution

Looking for new paths

Tests

SAGE 2007 Pex 2007

N. Tillmann, W. Schulte, J. de Halleuxhttp://research.microsoft.com/Pex/default.aspx

32/34


00000000h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000030h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000040h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000060h: 00 00 00 00 ; ....

Generation 0 – initial input – 100 bytes of “00”

00000000h: 52 49 46 46 00 00 00 00 00 00 00 00 00 00 00 00 ; RIFF............00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000030h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000040h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000060h: 00 00 00 00 ; ....

Generation 1

00000000h: 52 49 46 46 00 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF....*** ....00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000030h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000040h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000060h: 00 00 00 00 ; ....

Generation 2

00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...*** ....00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000030h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000040h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000060h: 00 00 00 00 ; ....

Generation 3

00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...*** ....00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 00 00 00 00 ; ....strh........00000040h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000060h: 00 00 00 00 ; ....

Generation 4

00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...*** ....00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 76 69 64 73 ; ....strh....vids00000040h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000060h: 00 00 00 00 ; ....

Generation 5

00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...*** ....00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 76 69 64 73 ; ....strh....vids00000040h: 00 00 00 00 73 74 72 66 00 00 00 00 00 00 00 00 ; ....strf........00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000060h: 00 00 00 00 ; ....

Generation 6

00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...*** ....00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 76 69 64 73 ; ....strh....vids00000040h: 00 00 00 00 73 74 72 66 00 00 00 00 28 00 00 00 ; ....strf....(...00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000060h: 00 00 00 00 ; ....

Generation 7

00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...*** ....00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 76 69 64 73 ; ....strh....vids00000040h: 00 00 00 00 73 74 72 66 00 00 00 00 28 00 00 00 ; ....strf....(...00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 C9 9D E4 4E ; ............ÉäN00000060h: 00 00 00 00 ; ....

Generation 8

00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...*** ....00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 76 69 64 73 ; ....strh....vids00000040h: 00 00 00 00 73 74 72 66 00 00 00 00 28 00 00 00 ; ....strf....(...00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 ; ................00000060h: 00 00 00 00 ; ....

Generation 9

00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...*** ....00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; ................00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 76 69 64 73 ; ....strh....vids00000040h: 00 00 00 00 73 74 72 66 B2 75 76 3A 28 00 00 00 ; ....strf²uv:(...00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 ; ................00000060h: 00 00 00 00 ; ....

Generation 10 – bug ID 1212954973!Found after only 3 generations starting from “well-formed” seed file

EXE 2005 Stanford Univ.D. Dill, D. Engler et al.

Randoop 2007 MIT + MS ResearchT. Ball, M. D. Ernst, C. Pacheco, S. Lahirihttp://people.csail.mit.edu/cpacheco/randoop/1.2/doc/

…

33/34


Modern verification tools Use basic services of component technologies Integrate a lot of techniques Become highly modular Can be modules of each other

Next step – integration frameworks ? Java PathFinder Bogor

http://bogor.projects.cis.ksu.edu/ Microsoft RiSE

http://research.microsoft.com/en-us/um/redmond/groups/rise/

34/34


software verification tools: current state and the nearest future

Documents

x syrcose

mlocdebian linux code

windows nt

mlocwindows team

pdebian team

red hat linux

smt solvers satisfiability

hoare logic