programming for everyone: from solvers to solver-aided...

Emina Torlak U.C. Berkeley

[email protected]

http://people.csail.mit.edu/emina/

Programming for everyone: from solvers to solver-aided languages and beyond

mailto:[email protected]

http://people.csail.mit.edu/emina/

visiona little programming for everyone

A little programming for everyone

3

We all want to build programs …

‣ spreadsheet data manipulation [Flashfill, POPL’11]


3


social scientist


‣ models of cell fates [SBL, POPL’13]


3


biologist social scientist



‣ cache coherence protocols [Transit, PLDI’13] ‣ memory models [MemSAT, PLDI’10]


3


hardware designer




‣ cache coherence protocols [Transit, PLDI’13] ‣ memory models [MemSAT, PLDI’10]


3

code

time

effort


hardware designer



4

less code

less time

less effort

solver-aided languages


‣ spreadsheet data manipulation ‣ models of cell fates ‣ cache coherence protocols ‣ memory models

hardware designer


software crisis

program logics (Floyd, Hoare, Dijkstra)

mechanization of logic (Milner, Pnueli)

mechanized tools (Clarke, Emerson, Sifakis)

software gap

SAT/SMT solvers and tools


A little history

5

better programs

software crisis




software gap



A little history

5

better programs

1960

1970

1980

1990

2000

software crisis




software gap



A little history

5

better programs

1960

1970

1980

1990

2000

ASTRÉE [AbsInt]

SLAM [MSR]

6TH SENSE [IBM]

software crisis




software gap



A little history

5

better programs

1960

1970

1980

1990

2000

software crisis




software gap



A little history

5

better programs

more easily

1960

1970

1980

1990

2000

2010

outlinesolver-aided tools, languages and beyond

outlinesolver-aided tools, languages and beyondsolver-aided tools

outlinesolver-aided tools, languages and beyondsolver-aided tools, languages

outlinesolver-aided tools, languages and beyondsolver-aided tools, languages and demos

storysolver-aided tools

P(x) { … …

}

Programming …

8

specification

P(x) { … …

}

Programming …

8

specificationtest case

assert safe(P(2))

Programming with a solver-aided tool

9

?SAT/SMT

solvertranslate(…)

P(x) { … …

} assert safe(P(x))

Solver-aided tools: verification

10

Find an input on which the program fails.

∃x . ¬ safe(P(x))

CBMC [CMU], Dafny [MSR], Miniatur [IBM], Klee [Stanford], etc.

SAT/SMT solver

P(x) { … …

} assert safe(P(x))

Solver-aided tools: verification

10

Find an input on which the program fails.

∃x . ¬ safe(P(x))

CBMC [CMU], Dafny [MSR], Miniatur [IBM], Klee [Stanford], etc.

modelvalues

SAT/SMT solver

P(x) { … …

} assert safe(P(x))

42

SAT/SMT solver

Solver-aided tools: debugging

11

Localize bad parts of the program.

x = 42 ⋀ safe(P(x))

BugAssist [UCLA / MPI-SWS]

P(x) { v = x + 2 …

} assert safe(P(x))

42

SAT/SMT solver

Solver-aided tools: debugging

11

min core

Localize bad parts of the program.

x = 42 ⋀ safe(P(x))

BugAssist [UCLA / MPI-SWS]

P(x) { v = x + 2 …

} assert safe(P(x))

x + 2

42

expressions

Solver-aided tools: angelic execution

12

Find values that repair the failing execution.

∃v . safe(P(42, v))

Kaplan [EPFL], PBnJ [UCLA], Skalch [Berkeley], Squander [MIT], etc.

SAT/SMT solver

P(x) { v = choose() …

} assert safe(P(x))

42

Solver-aided tools: angelic execution

12

Find values that repair the failing execution.

∃v . safe(P(42, v))

modelvalues

Kaplan [EPFL], PBnJ [UCLA], Skalch [Berkeley], Squander [MIT], etc.

SAT/SMT solver

P(x) { v = choose() …

} assert safe(P(x))

42 40

P(x) { v = ?? …

} assert safe(P(x))

Solver-aided tools: synthesis

13

Synthesize code that repairs the program.

∃e . ∀x . safe(Pe(x))

Comfusy [EPFL], Sketch [Berkeley / MIT]

SAT/SMT solver

P(x) { v = ?? …

} assert safe(P(x))

Solver-aided tools: synthesis

13

Synthesize code that repairs the program.

∃e . ∀x . safe(Pe(x))

Comfusy [EPFL], Sketch [Berkeley / MIT]

SAT/SMT solver

x − 2

modelexpressions

wantedmore solver-aided tools …

Building solver-aided tools: state-of-the-art

15

tools expert

execute synth


15

learn the problem domain

I need a tool to create models of biological cells …

tools expert domain expert

execute synth


15


design a domain language


Abstractions for cells, components, interactions, …


execute synth

build a symbolic compiler from the domain language to constraints


15



months or years of work



verify debug

execute synth

A solver-aided tool for creating biological models

Can we do better?

16


weeks

domain expert

verify debug

execute synth

implement an interpreter for the language, get a symbolic compiler for free

Can we do better?

16


weeks

domain expert

a solver-aided host language

a solver-aided domain-specific language (SDSL)

verify debug

execute synth

implement an interpreter for the language, get a symbolic compiler for free

designsolver-aided languages

domain-specific language (DSL)

Layers of languages

18

host language

A formal language that is specialized to a particular application domain and often limited in capability.

A high-level language for implementing DSLs, usually with meta-programming features.

interpreterlibrary

Scala, Racket, JavaScript


artificial intelligence Church, BLOG

databases SQL, Datalog

hardware design Bluespec, Chisel, Verilog, VHDL

math and statistics Eigen, Matlab, R

layout and visualization LaTex, dot, dygraphs, D3

Layers of languages

18

host language

interpreterlibrary


Layers of languages

18

host language

C = A * B

C / Java

Eigen / Matlab

[associativity]C = A * B

for (i = 0; i < n; i++) for (j = 0; j < m; j++) for (k = 0; k < p; k++) C[i][k] += A[i][j] * B[j][k]

interpreterlibrary

solver-aided domain-specific language (SDSL)

Layers of solver-aided languages

19

solver-aided host language

symbolic virtual machine

interpreterlibrary



19



ROSETTE[Torlak & Bodik, Onward’13, PLDI’14]

interpreterlibrary

spatial programming Chlorophyll

data-parallel programming SynthCL

web scraping WebSynth

secure stack machines IFC



19



ROSETTE[Torlak & Bodik, Onward’13, PLDI’14]

interpreterlibrary

deve

lopm

ent

time

(wee

ks)

0

4

8

12

16

Chlorophyll SynthCL WebSynth IFC

SDSLs developed with ROSETTE

20

(first-year grad) (expert) (undergrad) (expert)

deve

lopm

ent

time

(wee

ks)

0

4

8

12

16


Spatial programming for a low-power chip, using synthesis to partition code and data across 144 tiny cores.


20


GreenArrays GA144

x + z

deve

lopm

ent

time

(wee

ks)

0

4

8

12

16


Spatial programming for a low-power chip, using synthesis to partition code and data across 144 tiny cores.

Optimal partitioning synthesized in minutes, while manual partitioning takes days [Phothilimthana et al., PLDI’14].


20


GreenArrays GA144

x + zx + zx + z

deve

lopm

ent

time

(wee

ks)

0

4

8

12

16


Verification and synthesis for data-parallel programming with OpenCL.


20


deve

lopm

ent

time

(wee

ks)

0

4

8

12

16


Verification and synthesis for data-parallel programming with OpenCL.

Used by a novice to develop new vectorized kernels that are as fast as expert code.


20


deve

lopm

ent

time

(wee

ks)

0

4

8

12

16


Synthesis of web scraping scripts from examples (PBE).


20


deve

lopm

ent

time

(wee

ks)

0

4

8

12

16


Synthesis of web scraping scripts from examples (PBE).Works on real web pages (e.g., iTunes) in seconds.


20


deve

lopm

ent

time

(wee

ks)

0

4

8

12

16


Verification for executable specifications of secure stack machines.


20


deve

lopm

ent

time

(wee

ks)

0

4

8

12

16


Verification for executable specifications of secure stack machines.

Finds all bugs reported by a specialized tool [Hritcu et al., ICFP’13].


20


Modern descendent of Scheme with macro-based metaprogramming.

Anatomy of a solver-aided host language

21

Racket

ROSETTE


21

!(define-symbolic id type)

(assert expr)

(verify expr) (debug [expr] expr) (solve expr) (synthesize [expr] expr)

ROSETTE


21

symbolic constants!(define-symbolic id type)

(assert expr)


ROSETTE


21

symbolic constants

assertions


(assert expr)


ROSETTE


21

symbolic constants

assertions

queries


(assert expr)


ROSETTE


21

symbolic constants

assertions

queries


(assert expr)


∃ v . property(P(v))

ROSETTE


21

symbolic constants

assertions

queries


(assert expr)


∃ v . property(P(v))∃ v . property(P(v))

(define-symbolic v number?)

ROSETTE


21

symbolic constants

assertions

queries


(assert expr)


∃ v . property(P(v))∃ v . property(P(v))∃ v . property(P(v))


(assert (odd? (P v)))

ROSETTE


21

symbolic constants

assertions

queries


(assert expr)


∃ v . property(P(v))∃ v . property(P(v))∃ v . property(P(v))∃ v . property(P(v))


(assert (odd? (P v)))

(solve (check P))

A tiny example SDSL

22

def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6

BV: A tiny assembly-like language for writing fast, low-level library functions.

debug synth

A tiny example SDSL

22


BV: A tiny assembly-like language for writing fast, low-level library functions.

test verify

debug synth

def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 !> bvmax(-2, -1)

A tiny example SDSL:

23

ROSETTE



23

(define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5)))

parse

ROSETTE



23


parse

ROSETTE

(out opcode in ...)


def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 !> bvmax(-2, -1) -1

(define (interpret prog inputs) (make-registers prog inputs) (for ([stmt prog]) (match stmt [(list out opcode in ...) (define op (eval opcode)) (define args (map load in)) (store out (apply op args))])) (load (last)))


24

ROSETTE

interpret

(define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5))) `(-2 -1)




24

ROSETTE

interpret


0 -21 -12 03 04 -25 06 -1




24

ROSETTE

interpret


0 -21 -12 03 04 -25 06 -1

(2 bvge 0 1)




24

ROSETTE

interpret


0 -21 -12 03 04 -25 06 -1

(2 bvge 0 1)(define bvmax `((2 bvge 0 1) (3 bvneg 2) (4 bvxor 0 2) (5 bvand 3 4) (6 bvxor 1 5)))



25

ROSETTE


‣ pattern matching ‣ dynamic evaluation ‣ first-class &

higher-order procedures

‣ side effects


(define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (verify (assert (= (interpret bvmax inputs) (apply max inputs))))

def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 !> verify(bvmax, max) (0, -2) !> bvmax(0, -2) -1


26

ROSETTE

query




26

ROSETTE

Creates two fresh symbolic constants of type number and binds them to variables n0 and n1.

query





26

ROSETTE

Symbolic values can be used just like concrete values of the same type.

query






26

ROSETTE

(verify expr) searches for a concrete interpretation of symbolic constants that causes expr to fail.

query







26

ROSETTE

query




def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r2) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 !> debug(bvmax, max, (0, -2))

!(define inputs (list 0 -2)) (debug [input-register?] (assert (= (interpret bvmax inputs) (apply max inputs))))


27

ROSETTE

query

def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(??, ??) r5 = bvand(r3, ??) r6 = bvxor(??, ??) return r6 !> synthesize(bvmax, max) !!!!!

(define-symbolic n0 n1 number?) (define inputs (list n0 n1)) (synthesize [inputs] (assert (= (interpret bvmax inputs) (apply max inputs)))))


28

ROSETTE

query

def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(??, ??) r5 = bvand(r3, ??) r6 = bvxor(??, ??) return r6 !> synthesize(bvmax, max) !!!!!

def bvmax(r0, r1) : r2 = bvge(r0, r1) r3 = bvneg(r2) r4 = bvxor(r0, r1) r5 = bvand(r3, r4) r6 = bvxor(r1, r5) return r6 (define-symbolic n0 n1 number?)

(define inputs (list n0 n1)) (synthesize [inputs] (assert (= (interpret bvmax inputs) (apply max inputs)))))


28

ROSETTE

query

techsymbolic virtual machine (SVM)

Anatomy of a symbolic virtual machine

30

SVM

SDSL + program

synthesize

debug

verify

solve

solver

evaluate & compile to constraints

ROSETTE

[Torlak & Bodik, Onward’13]

[Torlak & Bodik, PLDI’14]


30

SVM

SDSL + program

synthesize

debug

verify

solve

solver


lift solution to program level

ROSETTE

[Torlak & Bodik, Onward’13]



30

SVM

program

synthesize

debug

verify

solve

solver


lift solution to program level

ROSETTE

[Torlak & Bodik, Onward’13


(3, 1, -2) (1, 3)

Translation to constraints by example

31

solve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

reverse and filter, keeping only positive numbers

vs ps

(3, 1, -2) (1, 3)


31


vs ps


31


vs psconstraints

a>0 ∧ b>0 (a, b)


31


vs psconstraints

Design space of precise symbolic encodings

32

symbolic execution

bounded model checkingsolve: ps = () for v in vs: if v > 0: ps = insert(v, ps) assert len(ps) == len(vs)

{ }a > 0 b ≤ 0 false


32

a > 0

b ≤ 0

ps ↦ (a)

ps ↦ (a)

vs ↦ (a, b) ps ↦ ( )

symbolic execution


{ }a > 0 b ≤ 0 false

∨ ∨ ∨


32

a > 0

b ≤ 0

ps ↦ (a)

ps ↦ (a)

vs ↦ (a, b) ps ↦ ( )

b > 0b > 0

ps ↦ (b) ps ↦ (b, a)

{ }a ≤ 0 b > 0 false

{ }a > 0 b > 0 true

a ≤ 0

b ≤ 0

ps ↦ ( )

ps ↦ ( )

{ }a ≤ 0 b ≤ 0 false

symbolic execution


{ }a > 0 b ≤ 0 false

∨ ∨ ∨


32

vs ↦ (a, b) ps ↦ ( )

ps ↦ (a)

ps ↦ ps0

a > 0a ≤ 0

!ps0 = ite(a > 0, (a), ( )) ps1 = insert(b, ps0) ps2 = ite(b > 0, ps0, ps1) assert len(ps2) = 2

a > 0

b ≤ 0

ps ↦ (a)

ps ↦ (a)

vs ↦ (a, b) ps ↦ ( )

b > 0b > 0

ps ↦ (b) ps ↦ (b, a)

{ }a ≤ 0 b > 0 false

{ }a > 0 b > 0 true

a ≤ 0

b ≤ 0

ps ↦ ( )

ps ↦ ( )

{ }a ≤ 0 b ≤ 0 false

symbolic execution

bounded model checking

ps ↦ ( )


{ }a > 0 b ≤ 0 false

∨ ∨ ∨


32

vs ↦ (a, b) ps ↦ ( )

ps ↦ (a)

ps ↦ ps0

a > 0a ≤ 0


a > 0

b ≤ 0

ps ↦ (a)

ps ↦ (a)

vs ↦ (a, b) ps ↦ ( )

b > 0b > 0

ps ↦ (b) ps ↦ (b, a)

{ }a ≤ 0 b > 0 false

{ }a > 0 b > 0 true

a ≤ 0

b ≤ 0

ps ↦ ( )

ps ↦ ( )

{ }a ≤ 0 b ≤ 0 false

symbolic execution


ps ↦ ( )

ps ↦ ps1

b > 0


{ }a > 0 b ≤ 0 false

∨ ∨ ∨


32

vs ↦ (a, b) ps ↦ ( )

ps ↦ (a)

ps ↦ ps0

a > 0a ≤ 0


a > 0

b ≤ 0

ps ↦ (a)

ps ↦ (a)

vs ↦ (a, b) ps ↦ ( )

b > 0b > 0

ps ↦ (b) ps ↦ (b, a)

{ }a ≤ 0 b > 0 false

{ }a > 0 b > 0 true

a ≤ 0

b ≤ 0

ps ↦ ( )

ps ↦ ( )

{ }a ≤ 0 b ≤ 0 false

symbolic execution


ps ↦ ( )

ps ↦ ps1

ps ↦ ps2

b > 0b ≤ 0

ps ↦ ps0


A new design: type-driven state merging

33

{ }a > 0 b > 0 true



33

{ }a > 0 b > 0 true


Merge values of ‣ primitive types: symbolically ‣ immutable types: structurally ‣ all other types: via unions

ba


33

{ }a > 0 b > 0 true



⁊g g

c

ba (c, d)(a, b)


33

{ }a > 0 b > 0 true



⁊g g

c(e, f)

ba (c, d)


33

{ }a > 0 b > 0 true



⁊g g

c(e, f){ ¬g ⊦ a, g ⊦ () }

()



34



a > 0a ≤ 0


34

vs ↦ (a, b) ps ↦ ( )

ps ↦ (a)ps ↦ ( )



Symbolic union: a set of guarded values, with disjoint and exhaustive guards.

g0 = a > 0 g1 = b > 0 g2 = g0 ∧ g1 g3 = ¬(g0 ⇔ g1) g4 = ¬g0 ∧ ¬g1 c = ite(g1, b, a) assert g2

a > 0a ≤ 0 g0¬ g0


34

vs ↦ (a, b) ps ↦ ( )

ps ↦ (a)ps ↦ ( )


ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) }


Execute insert concretely on all lists in the union.


a > 0a ≤ 0

g1

g0¬ g0


34

vs ↦ (a, b) ps ↦ ( )

ps ↦ (a)ps ↦ ( )


ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) }

ps ↦ { g0 ⊦ (b, a), ¬g0 ⊦ (b) }



a > 0a ≤ 0

¬ g1 g1

g0¬ g0


34

vs ↦ (a, b) ps ↦ ( )

ps ↦ (a)ps ↦ ( )


ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) }

ps ↦ { g0 ⊦ (b, a), ¬g0 ⊦ (b) }

ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) }



a > 0a ≤ 0

¬ g1 g1

g0¬ g0


34

vs ↦ (a, b) ps ↦ ( )

ps ↦ (a)ps ↦ ( )


ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) }

ps ↦ { g0 ⊦ (b, a), ¬g0 ⊦ (b) }

ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) }

ps ↦ { g2 ⊦ (b, a), g3 ⊦ (c), g4 ⊦ ( ) }


Evaluate len concretely on all lists in the union; assertion true only on the list guarded by g2.


a > 0a ≤ 0

¬ g1 g1

g0¬ g0


34

vs ↦ (a, b) ps ↦ ( )

ps ↦ (a)ps ↦ ( )


ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) }

ps ↦ { g0 ⊦ (b, a), ¬g0 ⊦ (b) }

ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) }

ps ↦ { g2 ⊦ (b, a), g3 ⊦ (c), g4 ⊦ ( ) }



a > 0a ≤ 0

¬ g1 g1

g0¬ g0


34

vs ↦ (a, b) ps ↦ ( )

ps ↦ (a)ps ↦ ( )


ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) }

ps ↦ { g0 ⊦ (b, a), ¬g0 ⊦ (b) }

ps ↦ { g0 ⊦ (a), ¬g0 ⊦ ( ) }

ps ↦ { g2 ⊦ (b, a), g3 ⊦ (c), g4 ⊦ ( ) }

polynomial encoding

partial evaluation

Effectiveness of type-driven state merging

35

Merging performance for verification and synthesis queries in SynthCL, WebSynth and IFC programs

0

10000

20000

30000

40000

number of control flow joins

0 4500 9000 13500 18000

R² = 0.9884

R² = 0.95

number of unionssize of all unions

Effectiveness of type-driven state merging

36

SVM and solving time for verification and synthesis queries in SynthCL, WebSynth and IFC programs

runn

ing

time

(sec

)

0

75

150

225

300

FWT

3 B1 B2 J2 B3

MM

1 J1

FWT

1 B4 SF3

CR

1

CR

2

CR

3

FWT

4

CR

4

MM

2

SF1

FWT

2

iTun

es

IMD

b

MM

3

AlA

n

SF4

SF2

SVMZ3

demoa little SDSL for finite state automata

thanks

[email protected]

mailto:[email protected]

programming for everyone: from solvers to solver-aided...

Documents