synthesizing memory models framework sketches · 2018-08-25 · framework sketches define a class...

Synthesizing Memory Models from Framework Sketches and Litmus TestsJames BornholtEmina Torlak University of Washington

Memory consistency models define memory reordering behaviors on mul>processors

Memory consistency models define memory reordering behaviors on mul>processors

…correctness of my compiler…

Compiler writers!

Memory consistency models define memory reordering behaviors on mul>processors

…correctness of my compiler…

Compiler writers!

…rules to verify against…

Verifica@on tools🤖

Memory consistency models define memory reordering behaviors on mul>processors

…correctness of my compiler…

Compiler writers!

…rules to verify against…

Verifica@on tools🤖

…possible low-level behaviors…

Kernel/library developers#

Memory consistency models define memory reordering behaviors on mul>processors

Litmus tests and prose

…correctness of my compiler…

Compiler writers!

…rules to verify against…

Verifica@on tools🤖

…possible low-level behaviors…

Kernel/library developers#

Memory consistency models define memory reordering behaviors on mul>processors

Litmus tests and prose

∀∃ ∈

∧

∨

∩∪⊂

⋈⇒

Formalspecifica@ons

…correctness of my compiler…

Compiler writers!

…rules to verify against…

Verifica@on tools🤖

…possible low-level behaviors…

Kernel/library developers#

Memory consistency models define memory reordering behaviors on mul>processors

Litmus tests and prose

∀∃ ∈

∧

∨

∩∪⊂

⋈⇒

Formalspecifica@ons

…correctness of my compiler…

Compiler writers!

…rules to verify against…

Verifica@on tools🤖

…possible low-level behaviors…

Kernel/library developers#

x86 [Sewell et al, CACM’10]

PowerPC [Alglave et al, CAV’10, etc]

ARM [Flur et al, POPL’16]

Litmus tests Formalspecifica@ons

∀∃ ∈

∧

∨

∩∪⊂

⋈⇒

MemSynth

Litmus tests Formalspecifica@ons

Synthesize specifica>ons∀

∃ ∈∧

∨

∩∪⊂

⋈⇒

MemSynth

Litmus tests Formalspecifica@ons

Synthesize specifica>ons

Framework sketch

∀∃ ∈

∧

∨

∩∪⊂

⋈⇒

MemSynth

Litmus tests Formalspecifica@ons

Synthesize specifica>ons

Detect ambigui>es

Framework sketch

∀∃ ∈

∧

∨

∩∪⊂

⋈⇒

MemSynth

Litmus tests Formalspecifica@ons

Framework sketch

Synthesize specifica>ons

Detect ambigui>es

∀∃ ∈

∧

∨

∩∪⊂

⋈⇒

MemSynth

MemSynth

Synthesize specifica>ons

Detect ambigui>es

∀∃ ∈

∧

∨

∩∪⊂

⋈⇒

MemSynth

Framework sketches define a class of memory models

Synthesize specifica>ons

Detect ambigui>es

∀∃ ∈

∧

∨

∩∪⊂

⋈⇒

MemSynth

Framework sketches define a class of memory models

MemSynth engine verifica@on, equivalence, synthesis, ambiguity

Synthesize specifica>ons

Detect ambigui>es

∀∃ ∈

∧

∨

∩∪⊂

⋈⇒

MemSynth

Framework sketches define a class of memory models

MemSynth engine verifica@on, equivalence, synthesis, ambiguity

Results synthesize real-world memory model specs

Synthesize specifica>ons

Detect ambigui>es

∀∃ ∈

∧

∨

∩∪⊂

⋈⇒

Memory models and framework sketches

Litmus tests illustrate memory model behaviorThread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0?

Litmus tests illustrate memory model behaviorThread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0?

Sequen>al consistency: no

Litmus tests illustrate memory model behaviorThread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0?

Sequen>al consistency: no

x86: yes!

Litmus tests illustrate memory model behaviorThread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0?

Sequen>al consistency: no

x86: yes!

A memory model M is a set of constraints that define the possible execu@ons (outcomes) of a program.

Litmus tests illustrate memory model behaviorThread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0?

Sequen>al consistency: no

x86: yes!

A memory model M is a set of constraints that define the possible execu@ons (outcomes) of a program.

Memory model M allows litmus test T if there exists an execu@on that sa@sfies M’s constraints.

Litmus tests illustrate memory model behaviorThread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0?

Sequen>al consistency: no

x86: yes!

A memory model M is a set of constraints that define the possible execu@ons (outcomes) of a program.

MeMemory model M allows test T:

∃ E. M(T,E)

Memory models, formallyCommon formaliza@ons based on rela>onal logic

Example for sequen>al consistency:

no ^(ws + fr + po + rf + fences) & iden

[Alglave et al, CAV’10]

Memory model M allows test T:

∃ E. M(T,E)

Memory models, formallyCommon formaliza@ons based on rela>onal logic

Example for sequen>al consistency:

no ^(ws + fr + po + rf + fences) & iden

[Alglave et al, CAV’10]

Memory model M allows test T:

∃ E. M(T,E)

Binary rela@ons over program instruc@ons

happens-before order

Memory models, formallyCommon formaliza@ons based on rela>onal logic

Example for sequen>al consistency:

no ^(ws + fr + po + rf + fences) & iden

[Alglave et al, CAV’10]

Memory model M allows test T:

∃ E. M(T,E)

Binary rela@ons over program instruc@ons

happens-before order is acyclic

Memory models, formallyCommon formaliza@ons based on rela>onal logic

Example for sequen>al consistency:

no ^(ws + fr + po + rf + fences) & iden

[Alglave et al, CAV’10]

Memory model M allows test T:

∃ E. M(T,E)

Binary rela@ons over program instruc@ons

happens-before order is acyclic

Memory models, formallyCommon formaliza@ons based on rela>onal logic

Example for sequen>al consistency:

no ^(ws + fr + po + rf + fences) & iden

[Alglave et al, CAV’10]

From program syntax

Memory model M allows test T:

∃ E. M(T,E)

Binary rela@ons over program instruc@ons

happens-before order is acyclic

Memory models, formallyCommon formaliza@ons based on rela>onal logic

Example for sequen>al consistency:

no ^(ws + fr + po + rf + fences) & iden

[Alglave et al, CAV’10]

From program syntax

Memory model M allows test T:

∃ E. M(T,E)

Thread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0?

Binary rela@ons over program instruc@ons

happens-before order is acyclic

Memory models, formallyCommon formaliza@ons based on rela>onal logic

Example for sequen>al consistency:

no ^(ws + fr + po + rf + fences) & iden

[Alglave et al, CAV’10]

po = {( , ), ( , )}3 421

Program order:

From program syntax

Memory model M allows test T:

∃ E. M(T,E)

Thread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0?

Binary rela@ons over program instruc@ons

happens-before order is acyclic

Memory models, formallyCommon formaliza@ons based on rela>onal logic

Example for sequen>al consistency:

no ^(ws + fr + po + rf + fences) & iden

[Alglave et al, CAV’10]

po = {( , ), ( , )}3 421

Program order:

From program syntax

Part of execu@on; implicitly existen@ally quan@fied

Memory model M allows test T:

∃ E. M(T,E)

Thread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0?

Binary rela@ons over program instruc@ons

no ^(ws + fr + po + rf + fences) & iden

Framework sketchesA framework sketch defines the search space for synthesizing a memory model M by including holes in constraints

no ^(ws + fr + po + rf + fences) & iden

Framework sketchesA framework sketch defines the search space for synthesizing a memory model M by including holes in constraints

Expression holes for a synthesizer to complete

?? ?? ??

no ^(ws + fr + po + rf + fences) & iden

Framework sketchesA framework sketch defines the search space for synthesizing a memory model M by including holes in constraints

Expression holes for a synthesizer to complete

Framework sketches are the key design tool for synthesizing memory model specifica@ons — they define the “interes@ng” candidate models

?? ?? ??

Memory model frameworks

no ^(ws + fr + po + rf + fences) & iden

[Alglave et al, CAV’10]

?? ?? ??

Memory model frameworks

no ^(ws + fr + ppo + grf + fences) & iden

[Alglave et al, CAV’10]

Preserved program order (same-thread reorderings)

Global reads from (inter-thread order)

Fence cumula>vity (for Power, ARM, etc)

Memory model frameworks

no ^(ws + fr + ppo + grf + fences) & iden

[Alglave et al, CAV’10]

Sequen>al consistency

Preserved program order (same-thread reorderings)

Global reads from (inter-thread order)

Fence cumula>vity (for Power, ARM, etc)

po rf ∅

Memory model frameworks

no ^(ws + fr + ppo + grf + fences) & iden

[Alglave et al, CAV’10]

Sequen>al consistency

Preserved program order (same-thread reorderings)

Global reads from (inter-thread order)

Fence cumula>vity (for Power, ARM, etc)

po rf ∅

Total store order (x86)

po - (Wr→Rd) rf & SameThd ∅

Memory model frameworks are common

Global @me rela@onal model

[Alglave et al, CAV’10]

Axioma@c “must-not-reorder”

func@ons [Mador-Haim et al,

DAC’11]

Exexcutable distributed

consistency models [Yang et al, IPDPS’04]

…

Ocelot: rela>onal logic with holesA rela>onal logic DSL with synthesis support

no ^(ws + fr + ppo + grf + fences) & iden?? ?? ??

Expression holes for a synthesizer to complete

Built on the Roseoe solver-aided language [Torlak & Bodik, PLDI’14]

Available as a Racket package: raco pkg install ocelot

Ocelot: rela>onal logic with holesA rela>onal logic DSL with synthesis support

no ^(ws + fr + ppo + grf + fences) & iden?? ?? ??

Expression holes for a synthesizer to complete

Comple@ons are expressions in rela@onal logic with chosen operators, terminals, and depth.

Built on the Roseoe solver-aided language [Torlak & Bodik, PLDI’14]

Available as a Racket package: raco pkg install ocelot

Ocelot: rela>onal logic with holesA rela>onal logic DSL with synthesis support

no ^(ws + fr + ppo + grf + fences) & iden?? ?? ??

Expression holes for a synthesizer to complete

Comple@ons are expressions in rela@onal logic with chosen operators, terminals, and depth.

Built on the Roseoe solver-aided language [Torlak & Bodik, PLDI’14]

operators = {+, &}

terminals = {po, ws}

depth = 1

Available as a Racket package: raco pkg install ocelot

Ocelot: rela>onal logic with holesA rela>onal logic DSL with synthesis support

no ^(ws + fr + ppo + grf + fences) & iden?? ?? ??

Expression holes for a synthesizer to complete

Comple@ons are expressions in rela@onal logic with chosen operators, terminals, and depth.

Built on the Roseoe solver-aided language [Torlak & Bodik, PLDI’14]

operators = {+, &}

terminals = {po, ws}

depth = 1

po ws

po + ws po & ws

Available as a Racket package: raco pkg install ocelot

Queries‣ Verifica@on ‣ Equivalence ‣ Synthesis ‣ Ambiguity

Verifica>on and equivalenceMemory model M allows test T:

∃ E. M(T,E)Common queries for automated memory model reasoning tools

Herd [Alglave et al, CAV’10]; MemAlloy [Wickerson et al, POPL’17]; etc.

Verifica>on and equivalenceMemory model M allows test T:

∃ E. M(T,E)Common queries for automated memory model reasoning tools

Herd [Alglave et al, CAV’10]; MemAlloy [Wickerson et al, POPL’17]; etc.

Litmus test

Memory modelVERIFY

SAT or UNSAT

Verifica>on and equivalenceMemory model M allows test T:

∃ E. M(T,E)Common queries for automated memory model reasoning tools

Herd [Alglave et al, CAV’10]; MemAlloy [Wickerson et al, POPL’17]; etc.

Litmus test

Memory modelVERIFY

SAT or UNSAT

Reduces to SAT (since litmus tests are loop-free)

Verifica>on and equivalenceMemory model M allows test T:

∃ E. M(T,E)Common queries for automated memory model reasoning tools

Herd [Alglave et al, CAV’10]; MemAlloy [Wickerson et al, POPL’17]; etc.

Litmus test

Memory modelVERIFY

SAT or UNSAT

EQUIVLitmus test or UNSATMemory model MB

Memory model MA

Reduces to SAT (since litmus tests are loop-free)

Verifica>on and equivalenceMemory model M allows test T:

∃ E. M(T,E)Common queries for automated memory model reasoning tools

Herd [Alglave et al, CAV’10]; MemAlloy [Wickerson et al, POPL’17]; etc.

Litmus test

Memory modelVERIFY

SAT or UNSAT

EQUIVLitmus test or UNSATMemory model MB

Memory model MA

Reduces to SAT (since litmus tests are loop-free)

UNSAT = bounded equivalence (“equivalent up to tests of size k”)

SynthesisFind a memory model consistent with a set of litmus tests

Memory modelSYNTH

Framework sketch

Allowed litmus tests

Forbidden litmus tests

SynthesisFind a memory model consistent with a set of litmus tests

SYNTH

Framework sketch

SynthesisFind a memory model consistent with a set of litmus tests

SYNTH

Framework sketchx86

SynthesisFind a memory model consistent with a set of litmus tests

SYNTH

Framework sketch

53

2 allowed tests

1 2 4 6 7 8 9 10

8 forbidden tests

x86

SynthesisFind a memory model consistent with a set of litmus tests

SYNTH

Framework sketch

53

2 allowed tests

1 2 4 6 7 8 9 10

8 forbidden tests

Total store order

x86

SynthesisFind a memory model consistent with a set of litmus tests Memory model

M allows test T: ∃ E. M(T,E)

Allowed litmus tests

Forbidden litmus tests

Framework sketchM

T+

T-Memory model

SynthesisFind a memory model consistent with a set of litmus tests Memory model

M allows test T: ∃ E. M(T,E)

Allowed litmus tests

Forbidden litmus tests

Framework sketchM

T+

T-

∃ E. M(T,E)⋀T∈T+

Memory model

SynthesisFind a memory model consistent with a set of litmus tests Memory model

M allows test T: ∃ E. M(T,E)

Allowed litmus tests

Forbidden litmus tests

Framework sketchM

T+

T-

∃ E. M(T,E)⋀T∈T+

∀ E. ¬M(T,E)⋀T∈T-

Memory model

SynthesisFind a memory model consistent with a set of litmus tests Memory model

M allows test T: ∃ E. M(T,E)

Allowed litmus tests

Forbidden litmus tests

Framework sketchM

T+

T-

∃ E. M(T,E)⋀T∈T+

∀ E. ¬M(T,E)⋀T∈T-

Memory model

Solved incrementally, like counterexample-guided induc@ve synthesis (CEGIS)

AmbiguityFind a dis@nguishing litmus test that exposes an ambiguity in a model

AMBIG

Key idea: axer synthesis, is there a different memory model that explains the tests?

AmbiguityFind a dis@nguishing litmus test that exposes an ambiguity in a model

AMBIG

Allowed litmus tests

Forbidden litmus tests

Key idea: axer synthesis, is there a different memory model that explains the tests?

AmbiguityFind a dis@nguishing litmus test that exposes an ambiguity in a model

AMBIG

Allowed litmus tests

Forbidden litmus tests

Key idea: axer synthesis, is there a different memory model that explains the tests?

Memory model MA

AmbiguityFind a dis@nguishing litmus test that exposes an ambiguity in a model

AMBIG

Framework sketch

Allowed litmus tests

Forbidden litmus tests

Key idea: axer synthesis, is there a different memory model that explains the tests?

Memory model MA

AmbiguityFind a dis@nguishing litmus test that exposes an ambiguity in a model

AMBIG

Framework sketch

Allowed litmus tests

Forbidden litmus tests

Key idea: axer synthesis, is there a different memory model that explains the tests?

Memory model MALitmus test

Memory model MB

AmbiguityFind a dis@nguishing litmus test that exposes an ambiguity in a model

AMBIG

Framework sketch

Allowed litmus tests

Forbidden litmus tests

Key idea: axer synthesis, is there a different memory model that explains the tests?

Memory model MALitmus test

Memory model MB

The new memory model must be seman>cally different from the input: MA and MB must disagree about a new test T

Similar to oracle-guided synthesis [Jha et al, ICSE’10]

AmbiguityFind a dis@nguishing litmus test that exposes an ambiguity in a model

AMBIG

Total store order (x86)

✓Thread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0?

AmbiguityFind a dis@nguishing litmus test that exposes an ambiguity in a model

AMBIG

Total store order (x86)

✓

Is there another seman>cally different memory model that also allows this test?Thread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0?

AmbiguityFind a dis@nguishing litmus test that exposes an ambiguity in a model

AMBIG

Total store order (x86)

Par@al store order (SPARC)✓

Is there another seman>cally different memory model that also allows this test?Thread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0?

AmbiguityFind a dis@nguishing litmus test that exposes an ambiguity in a model

AMBIG

Total store order (x86)

Par@al store order (SPARC)

✓ PSO ✗ TSO

✓

Is there another seman>cally different memory model that also allows this test?Thread 1 Thread 2

X = 11

r1 = Y2

Y = 13

r2 = X4

Can r1 = 0 ∧ r2 = 0? Thread 1 Thread 2

X = 11

Y = 12

r1 = Y3

r2 = X4

Can r1 = 1 ∧ r2 = 0?

The Synthesis-Ambiguity Cycle

53

1 2 4

Litmus tests

The Synthesis-Ambiguity Cycle

53

1 2 4

Litmus tests

Documenta@on

🎲🎲Random/systema@c

genera@on

%& Architects

The Synthesis-Ambiguity Cycle

53

1 2 4

Litmus tests

The Synthesis-Ambiguity Cycle

53

1 2 4

Litmus tests Memory modelspecifica>on

SYNTH

The Synthesis-Ambiguity Cycle

53

1 2 4

Litmus tests Memory modelspecifica>on

SYNTH

AMBIG

6

The Synthesis-Ambiguity Cycle

53

1 2 4

Litmus tests Memory modelspecifica>on

SYNTH

AMBIGUnique memory model

(within framework sketch)

6

Results

Synthesizing exis>ng memory models

PowerPC

x86

Synthesizing exis>ng memory models

PowerPC

x86

768 tests[Alglave et al, CAV’10]

10 tests

Synthesis

Synthesizing exis>ng memory models

PowerPC

x86

768 tests[Alglave et al, CAV’10]

10 tests

✓ 12 seconds

✓ 2 seconds

Search space: 21406

Search space: 2624

Synthesis

Synthesizing exis>ng memory models

PowerPC

x86

768 tests[Alglave et al, CAV’10]

10 tests

✓ 12 seconds

✓ 2 seconds

Not equivalent to published model!

Search space: 21406

Search space: 2624

Synthesis

Synthesizing exis>ng memory models

PowerPC

x86

768 tests[Alglave et al, CAV’10]

10 tests

✓ 12 seconds

✓ 2 seconds

Not equivalent to TSO!

Not equivalent to published model!

Search space: 21406

Search space: 2624

Synthesis

Synthesizing exis>ng memory models

PowerPC

x86

768 tests[Alglave et al, CAV’10]

10 tests

✓ 12 seconds

✓ 2 seconds

Not equivalent to TSO!

9 new tests

4 new tests

Ambiguity

Not equivalent to published model!

Search space: 21406

Search space: 2624

sync, lwsync, etc.

mfence, xchg

Other resultsImplemented another framework sketch [Mador-Haim et al, DAC’11]

Found typo in paper; couldn’t fix by hand, but synthesized repair

Other resultsImplemented another framework sketch [Mador-Haim et al, DAC’11]

Found typo in paper; couldn’t fix by hand, but synthesized repair

Order of magnitude faster than the Alloy general-purpose rela>onal solver for verifica@on and equivalence

Ocelot offers finer-grained control over rela@onal constraints

Other resultsImplemented another framework sketch [Mador-Haim et al, DAC’11]

Found typo in paper; couldn’t fix by hand, but synthesized repair

Order of magnitude faster than the Alloy general-purpose rela>onal solver for verifica@on and equivalence

Ocelot offers finer-grained control over rela@onal constraints

Comparable performance to exis@ng custom memory model tool for verifica@on (Herd [Alglave et al, CAV’10])

∀∃ ∈

∧

∨

∩∪⊂

⋈⇒MemSynth

Framework sketches define a class of memory models

MemSynth engine verifica@on, equivalence, synthesis, ambiguity

Results synthesize real-world memory model specs

memsynth.uwplse.org