bloomunit declarative testing for distributed programs peter alvaro uc berkeley
TRANSCRIPT
![Page 1: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/1.jpg)
BloomUnit
Declarative testing for distributed programs
Peter AlvaroUC Berkeley
![Page 2: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/2.jpg)
Team and Benefactors
• Peter Alvaro• Andy Hutchinson• Neil Conway• Joseph M. Hellerstein• William R. Marczak
• National Science Foundation• Air Force Office of Scientific Research• Gifts from Microsoft Research and NTT Communications
![Page 3: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/3.jpg)
Distributed Systems and Software
To verify or to test?
![Page 4: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/4.jpg)
Verification
• Formally specify program and behavior– e.g., in Promela
• Systematically test inputs and schedules– e.g., using SPIN
High investment, high returns
![Page 5: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/5.jpg)
Testing
• Concrete inputs• Concrete assertions over outputs / state• No control over asynchrony, scheduling
Pay-as-you-go investment, diminishing returns
![Page 6: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/6.jpg)
Sweet Spot?
• Investment similar to unit tests• Payoff closer to formal methods
![Page 7: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/7.jpg)
Context
• Disorderly programming• Computation as transformation• Uniform representation of state: collections
<~ bloom
![Page 8: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/8.jpg)
Hypothesis
Database foundations can simplify distributed systems programming
Successes to date:
1. Compact, declarative protocol implementations [Alvaro et al., NetDB `09, Alvaro et al., Eurosys `10]
2. Static analyses for distributed consistency [Alvaro et al., CIDR `10]
3. Software quality assurance? Yes.
![Page 9: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/9.jpg)
The database view of testing
1. Declarative assertions– Correctness specs as queries
2. Constraint-guided input generation– Synthesize inputs from FDs and FKs
3. Exploration of execution nondeterminism– Apply LP-based analyses to reduce state space
![Page 10: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/10.jpg)
Testing a protocol with BloomUnit
![Page 11: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/11.jpg)
An abstract delivery protocol
module DeliveryProtocol state do interface input, :pipe_in, [:dst, :src, :ident] => [:payload] interface output, :pipe_sent, pipe_in.schema interface output, :pipe_out, pipe_in.schema endend
![Page 12: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/12.jpg)
A delivery protocol
sender
receiverpipe_in
pipe_sent
pipe_out
![Page 13: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/13.jpg)
A delivery protocol –Best effort
sender
receiverpipe_in
pipe_sent
pipe_out
![Page 14: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/14.jpg)
A delivery protocol –Reliable
sender
receiverpipe_in
pipe_sent
pipe_out
![Page 15: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/15.jpg)
A delivery protocol –FIFO
sender
receiverpipe_in
pipe_sent
pipe_out
1
12
2
![Page 16: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/16.jpg)
Declarative Assertions
– Specifications: queries over execution traces• Timestamped log of flow at interfaces
– Specifications encode invariants• Queries capture incorrect behaviors
![Page 17: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/17.jpg)
pipe_out_log
Declarative Assertionsmodule FIFOSpec bloom do fail <= (pipe_out_log * pipe_out_log).pairs do |p1, p2| if p1.src == p2.src and p1.dst == p2.dst and p1.ident < p2.ident and p1.time >= p2.time ["out-of-order delivery: #{p1.inspect} < #{p2.inspect}"] end end endend
create view fail asselect 'out-of-order delivery: ' + p1 + ' < ' + p2
from pipe_out_log p1, pipe_out_log p2where p1.src = p2.src and p1.dst = p2.dstand p1.ident < p2.ident and p1.time >= p2.time
![Page 18: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/18.jpg)
Declarative Assertionsmodule FIFOSpec bloom do fail <= (pipe_out_log * pipe_out_log).pairs do |p1, p2| if p1.src == p2.src and p1.dst == p2.dst and p1.ident < p2.ident and p1.time >= p2.time ["out-of-order delivery: #{p1.inspect} < #{p2.inspect}"] end end endend
``delivery order (timestamps) never deviates from sender order (encoded into ident)’’
pipe_out_log
![Page 19: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/19.jpg)
Input Generation
![Page 20: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/20.jpg)
Input Generation
Idea:• User supplies constraints (in FO logic)• Search for models of the given formula
– Let a SAT solver do the hard work• Convert models into concrete inputs
– Ensure that the models are interestingly different
Implementation:• Use the Alloy[Jackson ’06] language & solver
![Page 21: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/21.jpg)
Input Generation
Exclusion constraints
What records cannot appear in an input instance
all p1, p2 : pipe_in | (p1.src = p2.src and p1.ident = p2.ident) => p1.payload = p2.payload
(ident functionally determines payload)
![Page 22: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/22.jpg)
Input Generation
Inclusion constraints:
What records must appear in an input instance
some p1, p2 : pipe_in | p1 != p2 =>
(p1.src = p2.src and p1.dst = p2.dst)
(there are at least two messages between two endpoints)
![Page 23: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/23.jpg)
Execution Exploration
![Page 24: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/24.jpg)
Execution Exploration
• All distributed executions are nondeterministic
• Each concrete input => set of executions• Message timings / orderings may differ
• Too large a space to search exhaustively!
![Page 25: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/25.jpg)
Execution Exploration
A C
B
Message orderings: N!Loss scenarios: 2N
D
σπ
Messages: N
![Page 26: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/26.jpg)
Execution Exploration
CALM Theorem[Hellerstein ‘10, Ameloot ‘11]: Consistency as logical monotonicity
– Monotonic logic (e.g. select, project, join) is order-insensitive
– Monotonic => race-free
![Page 27: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/27.jpg)
Execution Exploration: Monotonic program
A C
B
D
σπ
Message orderings: 1
![Page 28: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/28.jpg)
Execution Exploration:Hybrid program
A C
B
D
σπ
Messages: KMessage orderings: 1
![Page 29: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/29.jpg)
Execution Exploration
Only explore messages orderings when downstream logic is nonmonotonic
Search only ``interesting’’ orderings
![Page 30: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/30.jpg)
The system
Module under test Inputs
![Page 31: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/31.jpg)
Do you need to learn Bloom?
• Yes. – But you can take advantage of these techniques
without adopting the language• Requirements:
– A high-level query language– Monotonicity analysis capabilities
• Prove (or assert) that program fragments are order-insensitive
![Page 32: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/32.jpg)
Queries?
![Page 33: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/33.jpg)
The fold
![Page 34: BloomUnit Declarative testing for distributed programs Peter Alvaro UC Berkeley](https://reader036.vdocument.in/reader036/viewer/2022062806/56649c765503460f9492ad56/html5/thumbnails/34.jpg)
FIFO delivery in bloommodule FifoPerSource state do scratch :enqueue_src, [:source, :ident] => [:payload] scratch :dequeue_src, [:source] => [:reqid] scratch :dequeue_resp_src, [:reqid] => [:source, :ident, :payload]
table :storage_tab, [:source, :ident] => [:payload] scratch :tops, [:source] => [:ident] end
bloom :logic do storage_tab <= enqueue_src tops <= storage_tab.group([:source], min(storage_tab.ident)) end bloom :actions do temp :deq <= (storage_tab * tops * dequeue_src).combos(storage_tab.source => tops.source, storage_tab.ident => tops.ident, tops.source => dequeue_src.source) dequeue_resp_src <+ deq do |s, t, d| [d.reqid, d.source, s.ident, s.payload] end storage_tab <- deq {|s, t, d| s } endend
module FifoProto state do interface input, :enqueue, [:source, :ident] => [:payload] interface input, :dequeue, [] => [:reqid] interface output, :dequeue_resp, [:reqid] => [:source, :ident, :payload] scratch :chosen_src, [:ident] endend