a hierarchical method for verifying software transactional memory implementations serdar tasiran...
TRANSCRIPT
![Page 1: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/1.jpg)
![Page 2: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/2.jpg)
![Page 3: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/3.jpg)
![Page 4: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/4.jpg)
A Hierarchical Method for Verifying A Hierarchical Method for Verifying Software Transactional Memory ImplementationsSoftware Transactional Memory Implementations
Serdar TasiranKoç University Istanbul, Turkey
joint work with
Tim HarrisMicrosoft Research,
Cambridge, UK
Thanks: Rustan Leino, Tim Harris, Shaz Qadeer, Mike Barnett, Dave Detlefs, Mike Magruder, Yossi Levanoni, Kapil Vaswani
![Page 5: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/5.jpg)
TransactionsTransactions
![Page 6: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/6.jpg)
The STM Verification Problem
• Software transactional memory (STM): – Code blocks marked “atomic” – STM implementation guarantees atomic, serialized,
non-blocking execution, composability
• Complexity shifted to STM implementer– Complicated, tricky code: Conflict detection, rollback,
ordering, ...
• TM will be used widely– Correctness as critical as the rest of the computing platform
• Runtime, compiler, processor, ...
• Real STM implementations are ~10K lines– Interact with the runtime, garbage collector,...
• Goal: Devise modular, re-usable method for proving correctness– Mechanically check most error-prone parts
![Page 7: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/7.jpg)
Typical program-STM interactionTypical program-STM interaction Thread i STM .
BEGIN(TRANSACTION)
OpenForRead(O1)Read(O1.f1)Read(O1.f2)
Local computationOpenForRead(O2)Read(O2.f3)
Local computationOpenForUpdate(O2)Write(O2.f3)Read(O1.f2)
Local computationOpenForUpdate(O3)Write(O3.f4)
COMMIT(TRANSACTION) ValidateRead(O1) ValidateRead(O2) CloseUpdatedObject(O2) CloseUpdatedObject(O3)
Thread i continues ...
![Page 8: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/8.jpg)
Typical program-STM interactionTypical program-STM interaction Thread i STM .
BEGIN(TRANSACTION)
OpenForRead(O1)Read(O1.f1)Read(O1.f2)
Local computationOpenForRead(O2)Read(O2.f3)
Local computationOpenForUpdate(O2)Write(O2.f3)Read(O1.f2)
Local computationOpenForUpdate(O3)Write(O3.f4)
COMMIT(TRANSACTION) ValidateRead(O1) ValidateRead(O2) CloseUpdatedObject(O2) CloseUpdatedObject(O3)
Thread i continues ...
Write O1 metadatasnapshot into read log
Modify O2 metadataBecome exclusive owner
Write field update into log
![Page 9: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/9.jpg)
Approach
• At the algorithm-level, STM’s are well understood – Key ideas in correctness proof known
• Algorithm level: Large-grain atomic actions– Field write, read– Transaction commit, roll back
• Example: Bartok STM, a write-in-place, lazy-invalidate STM
• Idea/earlier work:– Do algorithm-level proof once– Boil down to properties STM must satisfy: EW, VR, CU
• Sufficient condition for correctness
– Check if STM implementation satisfies EW, VR, CU• Formulate each as sequential assertion check, verify with Boogie
![Page 10: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/10.jpg)
Not so simple! • Implementation-level executions Algorithm-level executions
• Problem 1: More variables at implementation level
– STM implementation variables: Logs, per-object metadata, …
• Problem 2: Finer-grain, smaller actions more interleavings
– Atomic action: One instruction in an STM function implementation
![Page 11: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/11.jpg)
STM ImplementationSTM Implementation
![Page 12: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/12.jpg)
![Page 13: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/13.jpg)
Not so simple!
• Reasoning at implementation level more difficult– Serializability proofs messy to write, check
• Approach:
1. Define algorithm-level semantics, prove serializability.• Extract necessary conditions that STM must satisfy.
2. Define, prove correspondence between algorithm- and implementation-level executions
Algorithm-level proof carries over to implementation-level executions.
![Page 14: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/14.jpg)
Proof Approach
Implementation-level execution
Algorithm-level execution satisfying NOWS +VRS + CU
![Page 15: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/15.jpg)
Proof Approach
Implementation-level execution
“Coarse-atomic” execution
Abstract Read operations
Verify NOWS, VRS properties
“Coarse-atomic” execution with serial undo’s
Merge chains of STM internal state transitions
Insert marker actions: “commit”, “undoLastLogEntry”
Algorithm-level execution satisfying NOWS + VRS + CU
![Page 16: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/16.jpg)
KOS + SEMCON
SEMALG, ABSSEMALG,CON
KOSABS + SEMCON
ATOMIC BLOCKS,STRONG, SEQUENTIALLY CONSISTENT SEMANTICS
sr THM. 1
(I)
(I)
Sec. 4.3
Sec. 4.4
(II)
Sec. 4.5
THM. 1
![Page 17: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/17.jpg)
Intuition for proof steps
• Abstract some actions• Prove non-interference• Commute• Prove larger blocks atomic
• Abstract some actions• Prove non-interference• Commute• Prove larger blocks atomic
![Page 18: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/18.jpg)
Outline
• OTFJ: “Our” Transactional Featherweight Java
– Algorithm-level semantics
• Correctness
– Serializability
– Algorithm-level proof• Distill to three required properties
• Relating implementation and algorithm levels
– OTFJ: Implementation-level semantics
– What to abstract, verify at implementation level?
• Discussion
![Page 19: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/19.jpg)
19
““Our” Transactional Featherweight Java (OTFJ) : SyntaxOur” Transactional Featherweight Java (OTFJ) : Syntax
P ::= 0 | P|P | t[p]
L :: = Class C{f1,f2,...,fn; M1, M2,...,Mk}
M :: = m(x1,x2, ..., xp){ p; }
s :: = x | v | s.f | s.m(s1,...,sn) | s.f := s | s; s | new C() | null
p :: = spawn p | lbl: onacid; s; commit | p; p
v :: = v | v.f | v.m(v1,...,vn) | v.f := v | v; v
![Page 20: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/20.jpg)
OTFJ: Algorithm-level semantics
![Page 21: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/21.jpg)
Algorithm-Level OTFJ Semantics
21
OTFJ ProgramAbstract
STM
• Begin Transaction• Field write• Field read• Can I commit?
• OK2Commit • Invalidate Tx - Rollback - Undo using log
Configuration: P, ,
![Page 22: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/22.jpg)
22
OTFJ Algorithm-Level Semantics: Read Field
Program State STM State
P1, 1 1
2
2
read r.fi
Open4Read(r)
Abstract STM:This transitionleft unspecified
P1, 1
P2, 2
![Page 23: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/23.jpg)
23
OTFJ Algorithm-Level Semantics: Read Field
Program State STM State
1
2
2
read r.fi
Open4Read(r)
In actual implementation,”read r.fi” added to transaction read log
P1, 1
P1, 1
P2, 2
![Page 24: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/24.jpg)
24
OTFJ Semantics: Successful Field Write
Program State STM State
1
2
2
v.fi:= rnew
Open4Update(v)
OK2Write(v)
Abstract STM:This transitionleft unspecified
P1, 1
P1, 1
P2, 2
![Page 25: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/25.jpg)
25
OTFJ Semantics: Successful Field Write
Program State STM State
1
2
2
v.fi:= rnew
Open4Update(v)”write (v.fi, rnew, rold)” added to transaction write/undo log
OK2Write(v)
P1, 1
P1, 1
P2, 2
![Page 26: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/26.jpg)
26
OTFJ Semantics: Failed Field Write
Program State STM State
1
2
Open4Update(v)
OK2Write(v)
Invalid(s2)
P1, 1
P1, 1
![Page 27: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/27.jpg)
27
OTFJ Semantics: Transaction Successful Commit
Program State STM State
1
2
2
OK2Commit(1) Logs • appended to parent transaction’s logs• discarded if top-level transaction
Commit
P1, 1
P1, 1
P2, 2
![Page 28: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/28.jpg)
28
OTFJ Semantics: Transaction Failed Commit
Program State STM State
1
2 OK2Commit(1)
Commit
Invalid(s2)
P1, 1
P1, 1
![Page 29: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/29.jpg)
29
OTFJ Semantics: Transaction Rollback
Program State STM State
1
2
Invalid(1)
Program rewound to beginning of transaction • Program variable transitions unspecified
P1, 1
P1, 1
![Page 30: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/30.jpg)
Algorithm-level semantics: What is atomic?
• Atomic actions at this level
– Open4Read, Open4Write, CommitTransaction
– Field Read, Field Write, Commit, Rollback
30
![Page 31: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/31.jpg)
Outline
• OTFJ: “Our” Transactional Featherweight Java
– Algorithm-level semantics
• Correctness
– Serializability
– Algorithm-level proof• Distill to three required properties
• Relating implementation and algorithm levels
– OTFJ: Implementation-level semantics
– What to abstract, verify at implementation level?
• Discussion
![Page 32: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/32.jpg)
Correctness: Equivalence of Executions
• Equivalence: Executions ξ and ξ’ equivalent according to a semantics SEM if ξ and ξ’
– are both consistent with SEM
– have the same set of threads
– have the same end state
– For each thread: Same sequence of actions
• Alternative view:
– Commuting movers in the right direction yields equivalent execution
– Dependent: Access same program variable, at least one is a write
32
![Page 33: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/33.jpg)
Correctness: Serializability
• Serial execution:
• Conflict-free: No transaction is rolled-back
• Serializability
– Is each execution equivalent to a serial execution?
33
. . . . . . . . .
Action by transaction Tx
. . .
Action by transaction Tx
Must belong to Tx, ora child of Tx
![Page 34: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/34.jpg)
The Life of a TransactionThe Life of a Transaction
First read successfully validated
![Page 35: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/35.jpg)
Non-interference properties Non-interference properties
• STM implementation code must ensure that these properties hold
1. Exclusive writes (EW)
2. Valid reads (VR)
3. Correct undo (CU)
![Page 36: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/36.jpg)
Write and Read Spans• Execution ξ = 0, 1, ..., n
• Write span of an object o within transaction Tx: Interval [i, j] ξ = 0, ..., i, ..., j, ..., n
i : First write by Tx to a field of o
j : Completion action (commit or undo) of Tx
• Read span of an object o within transaction Tx: Interval [i, j] ξ = 0, ..., i, ..., j, ..., n
i : First read of o by Tx
j : Completion action (commit or undo) of Tx
![Page 37: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/37.jpg)
Algorithm-Level Serializability: Sufficient Conditions
1. Non-Overlapping Write Spans (NOWS) For any obj, Tx Tx’WriteSpan(Tx,obj) and WriteSpan(Tx’,obj) do not overlap.
2. Valid Read Spans (VRS) For succeeding Tx, for all obj, Tx Tx’ ReadSpan(Tx,obj) does not overlap WriteSpan(Tx’,obj)
3. Correct Undo’s (CU) Open4Update(Tx, obj) ....... Undo(Tx)
Theorem: NOWS + VRS + CU ==> Pure serializability
State of obj same at these two points
![Page 38: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/38.jpg)
Need for abstract algorithm-level semantics
• Concrete semantics– Serial execution: No conflicts, no rollbacks– No concurrent execution with conflicts and rollbacks
can be equivalent to a serial one. • Solution: Allow non-deterministic rollback of transaction• Not enough!
T1 T2 read(Tx1, v=0) read(Tx2, v=0) write(Tx2, v:=1) commit(Tx2) write(Tx1, v:=2) invalidate(Tx1)
• Cannot commute actions of T1 together
• Solution: Allow reads in rolled-back transactions to return non-deterministic values.
38
![Page 39: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/39.jpg)
Algorithm-level serializability proof sketch
• Show actions to be movers of certain types– α is a right mover if
for every ξ = ……. α β ……. ξ’ = ……. β α ……. is equivalent to ξ
– Left mover similar.
• Argue: Actions of a transaction have the following form
RRRRR N LLLLLL
• Case split:
(i) Successful transactions
(ii) Rolled-back transactions
• Ordering of actions within transaction comes from semantics
39
![Page 40: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/40.jpg)
Algorithm-level serializability proof sketch
• Right movers:
– Open4Read, Open4Write: STM state update left non-det
– Field Write: Because of NOWS property
– Field Read:• Because of VRS property (Successful transactions)• Non-deterministic (Rolled-back transactions)
• Non-movers:
– Commit action (Successful tx)
– Undo action (Rolled-back tx)
• Left movers:
– Thread-local bookkeeping actions after commit, rollback
• Correct Undo (CU) property trivially implies pure serializability
40
![Page 41: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/41.jpg)
Algorithm-level serializability proof
• NOWS + VRS + CU imply pure serializability– For abstract algorithm-level semantics
• What does serializability of an abstract model mean?– Extreme case: Completely non-deterministic program
trivially serializable– Must limit how much non-determinism is added.
• RemRB(ξ) = ξ’• ξ’ same as ξ except actions by rolled back transactions removed
• Correct undos (CU) ξ crucial
– Every execution ξ equivalent to a serial ξ*– RemRB(ξ*) is consistent with strong semantics for
transactions41
![Page 42: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/42.jpg)
Outline
• OTFJ: “Our” Transactional Featherweight Java
– Algorithm-level and implementation-level semantics
• Correctness
– Pure serializability
– Algorithm-level proof• Distill to three required properties
• Relating implementation and algorithm levels
– What to abstract, verify at implementation level?
• Discussion
![Page 43: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/43.jpg)
Proof Approach
Implementation-level execution
“Coarse-atomic” execution
Abstract Read operations
Verify NOWS, VRS properties
“Coarse-atomic” execution with serial undo’s
Merge chains of STM internal state transitions
Insert marker actions: “commit”, “undoLastLogEntry”
Algorithm-level execution satisfying EW +VR + CU
![Page 44: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/44.jpg)
Implementation-Level Semantics
• Open4Read: Bartok’s OpenForRead function• Open4Update: Bartok’s OpenForUpdate function
• Commit: Bartok’s DTMCommit(Tx) function– For all objects o in Tx’s read log
• ValidateRead(o)– If all succeed
• For all o in write log– CloseUpdatedObject(o)
• Clear all logs– Otherwise, invalidate Tx, roll back
• Rollback: Bartok’s DTMAbort function– For all entries (v.fi, rnew, rold) in
Tx’s undo log, in reverse chronological order• v.fi = rold
– Clear log44
![Page 45: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/45.jpg)
Semantics: Implementation-level Executions
45
High Level
Implementation Level
(th1,1 )
(th2,2 )
……
1
2
m
1
2
n
…
…
1
2
m
1
2
n
……
![Page 46: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/46.jpg)
1
2
3
45
6
7
8
9
10
11
Implementation-level Atomic Actions
![Page 47: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/47.jpg)
123
Implementation-level Atomic Actions
![Page 48: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/48.jpg)
Atomic read
The rest: Localvariableaccesses:
All movers
![Page 49: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/49.jpg)
Atomic read
Atomic compare and swap
The rest:Localvariableaccesses:Bothmovers
Desired: STM Methods Acting on Single Objects are AtomicDesired: STM Methods Acting on Single Objects are Atomic
![Page 50: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/50.jpg)
Read
CAS
The need for abstraction: Actions do not commute
CAS
• If CAS fails, can’t commute Read or CAS across CAS.
![Page 51: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/51.jpg)
NDRead
CAS
The need for abstraction: Actions do not commute
CAS
• NDRead:– for succeeding CAS: works like regular read– for failing CAS: reads non-deterministic value
• NDRead for failing CAS commutes with all actions• But how do you “implement” NDRead:
• Prophecy variable: CASWillFail• Similar trick for reads within failing transactions
• Prophecy variable: willInvalidate(Tx)• Read returns non-deterministic value if willInvalidate(Tx) is true.
![Page 52: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/52.jpg)
Every execution is equivalent to a coarse atomic one
• Abstract read actions• Prove non-interference• Commute• Prove larger blocks atomic
Valid
ateR
ead(
Tx1,o
bj 1)
Ope
n4W
rite(
Tx2,o
bj 2)
Valid
ateR
ead(
Tx3,o
bj 3)
Valid
ateR
ead(
Tx1,o
bj 4)
Ope
n4W
rite(
Tx2,o
bj 5)
Valid
ateR
ead(
Tx3,o
bj 6)
![Page 53: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/53.jpg)
Proof Approach
Implementation-level execution
“Coarse-atomic” execution
Abstract Read operations
Verify NOWSI, VRSI properties
“Coarse-atomic” execution with serial undo’s
Merge chains of STM internal state transitions
Insert marker actions: “commit”, “undo”
Algorithm-level execution satisfying NOWS +VRS + CU
![Page 54: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/54.jpg)
Proof Approach
Implementation-level execution
“Coarse-atomic” execution
Abstract Read operations
Verify NOWSI, VRSI properties
“Coarse-atomic” execution with serial undo’s
Merge chains of STM internal state transitions
Insert marker actions: “commit”, “undo”
Algorithm-level execution satisfying NOWS +VRS + CU
Boogie!
![Page 55: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/55.jpg)
55
Read and Write Spans at Implementation Level
• Execution ξ = 0, 1, ..., n
• Write span of an object o within transaction Tx: Interval [i, j] ξ = 0, ..., i, ..., j, ..., n
i : First OpenForWrite(Tx,o)
j : CloseUpdatedObject(Tx,o)
• Read span of an object o within transaction Tx: Interval [i, j] ξ = 0, ..., i, ..., j, ..., n
i : First OpenForWrite(Tx,o)
j : ValidateRead(Tx,o)
![Page 56: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/56.jpg)
56
Non-Overlapping Write Spans – Impl. Level
[OpenForUpdate(Tx, obj), CloseUpdatedObj(Tx, obj) ]
does not overlap with
[OpenForUpdate(Tx_bad, obj), CloseUpdatedObj(Tx_bad, obj) ]
Approach:• Assume Tx executed OpenForUpdate(Tx,obj) and not
CloseUpdatedObj(Tx,obj) yet
• ExclusiveOwner(Tx,obj): A formula that says obj metadata indicates that Tx is the exclusive write owner and other good things.
• Prove: [Remember all STM methods are atomic]
Any possible method execution by another thread Tx_bad leaves ExclusiveOwner(Tx,obj) unchanged.
![Page 57: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/57.jpg)
57
Checking NOWSI with Boogie
Havoc(obj’s state and metadata)
OpenForUpdate(Tx_good, obj)
assume( ExclusiveOwner(Tx_good, obj) );
![Page 58: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/58.jpg)
58
Checking NOWS with Boogie
Havoc(obj’s state and metadata)
OpenForUpdate(Tx_good, obj)
assume( ExclusiveOwner(Tx_good, obj) );
assume( Tx_bad != Tx_good);
OpenForUpdate(Tx_bad, obj);
assert(ExclusiveOwner(Tx_good, obj));
![Page 59: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/59.jpg)
59
Valid Read Spans (VRS) ==> VR
[OpenForRead(Tx1,obj), (Successful)ValidateRead(Tx1,obj) ]
does not overlap with
[OpenForUpdate(Tx2, obj), CloseUpdatedObj(Tx2, obj) ]
![Page 60: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/60.jpg)
60
Checking VRS with Boogie
InterfereWith(Tx, obj);
OpenForRead(Tx,obj);
InterfereWith(Tx, obj);
if (*) OpenForUpdate(Tx,obj);
InterfereWith(Tx, obj);
ValidateRead(Tx,obj);
assert(interferedWith ==> Tx.invalid);
![Page 61: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/61.jpg)
61
Checking VRS with Boogie
InterfereWith(Tx, obj);
OpenForRead(Tx,obj);
InterfereWith(Tx, obj);
if (*) OpenForUpdate(Tx,obj);
InterfereWith(Tx, obj);
ValidateRead(Tx,obj);
assert(interferedWith ==> Tx.invalid);
![Page 62: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/62.jpg)
62
Checking VRS with Boogie
InterfereWith: Represents effects of
( + Close) (Open . Close)* ( Open + )
InterfereWith (Transaction Tx, Object obj) {
while (*) {
Tx_bad = non-deterministically chosen transaction
assume(Tx_bad != Tx);if (*) OpenForUpdate(Tx_bad, obj);if (*) CloseUpdatedObj(Tx_bad, obj);
}
}
![Page 63: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/63.jpg)
63
Checking VRS with Boogie
InterfereWith(Tx, obj);
OpenForRead(Tx,obj);
InterfereWith(Tx, obj);
if (*) OpenForUpdate(Tx,obj);
InterfereWith(Tx, obj);
ValidateRead(Tx,obj);
assert(interferedWith ==> Tx.invalid);
![Page 64: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/64.jpg)
64
Checking VRS with Boogie
• Challenge: Finding pre- and post-conditions for InterfereWith()
– Boogie has the loop invariant inference (using abstract interpretation) capability to automate this
InterfereWith (Transaction Tx, Object obj) {
while (*) {
Tx_bad = non-deterministically chosen transaction
assume(Tx_bad != Tx);if (*) OpenForUpdate(Tx_bad, obj);if (*) Close*Obj(Tx_bad, obj);
}
}
![Page 65: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/65.jpg)
65
Checking VRS with Boogie
• Wrote pre/post-condition pair, verified to be inductive, again using Boogie but on straightline code.
– Post-condition: If object opened or closed by Tx_bad, – version number is bigger or
– obj (metadata) is quiescent
– or obj metadata has info related to Tx_bad in it.
InterfereWith (Transaction Tx, Object obj) ensures (interferedWith ==>
PostCondition(Tx, obj));ensures (!interferedWith ==>
Unchanged(Tx,obj))
{ InterfereWith(Tx, obj);if (*) OpenForUpdate(Tx_bad, obj);if (*) Close*Obj(Tx_bad, obj);
}
![Page 66: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/66.jpg)
66
Checking VRS with Boogie
• Omissions in STM pseudocode showed up when checking this proof obligation with Boogie
– Some interleavings had been overlooked in pseudocode– Tx_bad: opens for update
– Tx_good: opens for read
» Tx_bad: modifies object
» Tx_bad: closes updated object
– Tx_good: opens for update
– Tx_good: validates read (shouldn’t have)
– Transaction should be invalidated, it isn’t.
• After putting in fixes, check passed
– Possible to make errors in STM design at this level
• Checks took ~5 minutes
![Page 67: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/67.jpg)
What do we have so far?
• Implementation-level executions have the
– NOWSI
– VRSI
properties
• These are enough to prove
– OpenForWrite, Field Writes right movers
– OpenForRead, Read right movers (in successful Tx)
– CloseUpdatedObject left mover
– ValidateRead left mover (in successful Tx) Successful transactions can be made atomic
• What about rolled back transactions?
– These can be made atomic/sequential after some abstraction (on implementation-level model)
![Page 68: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/68.jpg)
ImplementationAlgorithm-level: Committed Transactions
Valid
ateR
ead(
Tx,o
bj 1)
Valid
ateR
ead(
Tx,o
bj 2)
Valid
ateR
ead(
Tx,o
bj 3)
. . . Valid
ateR
ead(
Tx,o
bj n)
. . .Commit(Tx)
Commute all of Tx’s actions here
![Page 69: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/69.jpg)
Implementation Algorithm Level: Rolled Back Transactions
Writ
eFie
ld(T
x, o
bj 1.f 1
)
Writ
eFie
ld(T
x, o
bj 2.f 2
)W
riteF
ield
(Tx,
obj 3
.f 3)
. . . Writ
eFie
ld(T
x, o
bj n.f n
)
. . .
Tx being rolled back
![Page 70: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/70.jpg)
Implementation Algorithm Level: Rolled Back Transactions
Writ
eFie
ld(T
x, o
bj 1.f 1
)
Writ
eFie
ld(T
x, o
bj 2.f 2
)W
riteF
ield
(Tx,
obj 3
.f 3)
. . . Writ
eFie
ld(T
x, o
bj n.f n
)
. . .
Move all WriteField’s here• WriteField’s commute to left of all intervening actions
cannot be a write: Tx has object open for write, NOWSI
– If is a read by Tx’, Tx’ rolled back is a non-det read
![Page 71: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/71.jpg)
Need for abstraction
Writ
eFie
ld(T
x, o
bj 1.f 1
)
Writ
eFie
ld(T
x, o
bj2.f 2
)W
riteF
ield
(Tx,
obj 3
.f 3)
. . . Writ
eFie
ld(T
x, o
bj n.f n
)
. . .
Commute all of Tx’s actions here
Rea
d(Tx
_oth
er, o
bj2.f 2
)
![Page 72: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/72.jpg)
ND
Rea
d(Tx
_oth
er, o
bj2.f 2
)
Need for abstraction
Writ
eFie
ld(T
x, o
bj 1.f 1
)
Writ
eFie
ld(T
x, o
bj2.f 2
)W
riteF
ield
(Tx,
obj 3
.f 3)
. . . Writ
eFie
ld(T
x, o
bj n.f n
)
. . .
Commute all of Tx’s actions here
• NDRead: Non-Deterministic Read– Acts like regular read, or– Reads arbitrary value
• Only if part of failing (rolled-back) transaction
• Commutes with any action• Only adds to behaviors safe for modeling
![Page 73: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/73.jpg)
Rolled Back Transactions
• Rolled-back implementation-level execution with NOWSI and VRSI properties
– Equivalent (according to abstracted implementation-levelsemantics) to an atomic, serial one.
• Can show correct undo’s property using sequential analysis only
– By NOWSI, transaction has exclusive write control of object until CloseUpdatedObj
– Must show that • updates entered to logs and • undoing them in reverse chronological order
has no net effect on program state.
![Page 74: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/74.jpg)
Proof Approach
Implementation-level execution
“Coarse-atomic” execution
Abstract Read Operations
Verify NOWS, VRS properties
“Coarse-atomic” execution with serial undo’s
Merge chains of STM internal state transitions
Insert marker actions: “commit”, “undo”
Algorithm-level execution satisfying EW +VR + CU
![Page 75: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/75.jpg)
Proof Approach
Implementation-level execution
“Coarse-atomic” execution
Abstract Read Operations
Verify NOWS, VRS properties
“Coarse-atomic” execution with serial undo’s
Merge chains of STM internal state transitions
Insert marker actions: “commit”, “undo”
Algorithm-level execution with serial undo’sNOWSI ==> NOWS, VRSI ==>VRS, CU ==> CU
Straightforwardmanualproofs
![Page 76: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/76.jpg)
76
Discussion• Correctness proof for STM’s involves quantification over
– threads, transactions– all possible sequences of accesses in a transaction – objects
• This is taken care of – mostly in the manual part of proof– and by “Tx_bad”(s) in the mechanized part
• Initial focus: Do STM methods guarantee non-interference? – Most error prone part– Checked mechanically
• Checks with Boogie computationally manageable– Simple sequential scenarios:
• One object, one transaction, environment• interfereWith models all other threads/transactions
![Page 77: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/77.jpg)
Summary
• Formalizing correctness: – OTFJ language, “pure serializability”
• Proving correctness at high-level: – Reduced algorithm-level proof of pure serializability to three
sufficient conditions: • NOWS + VRS + CU
• Relating implementation and algorithm levels:– Provided low-level semantics for OTFJ– Devised sequence of steps to transform
implementation-level executions to algorithm-level executions– Correctness of most error-prone step checked with Boogie
• Modeling tricks:– Identified abstractions in modeling required for approach to work
![Page 78: A Hierarchical Method for Verifying Software Transactional Memory Implementations Serdar Tasiran Koç University Istanbul, Turkey joint work with Tim](https://reader035.vdocument.in/reader035/viewer/2022062519/5697bfcf1a28abf838ca9b8a/html5/thumbnails/78.jpg)
78
Future Work
• Mechanize more of proof
• Non-transactional accesses, buffered-write STM’s– Privatization problem, etc.– Strong isolation for non-transactional accesses– What is the correctness criterion?
• Formalizing relationship between model and implementation – What exactly is assumed of the GC and CLR? – How to establish correspondence between Spec# model and
10K lines of C++?