the relative power of synchronization operations companion slides for the art of multiprocessor...
TRANSCRIPT
![Page 1: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/1.jpg)
The Relative Power of Synchronization Operations
Companion slides forThe Art of Multiprocessor Programming
by Maurice Herlihy & Nir Shavit
![Page 2: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/2.jpg)
Art of Multiprocessor Programming
2
Shared-Memory Computability
• Mathematical model of concurrent computation• What is (and is not) concurrently computable• Efficiency (mostly) irrelevant
1001110011
Shared Memory
![Page 3: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/3.jpg)
Art of Multiprocessor Programming
3
Wait-Free Implementation
• Every method call completes in finite number of steps
• Implies no mutual exclusion
![Page 4: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/4.jpg)
Art of Multiprocessor Programming
4
From Weakest Register
1
0 11
Single readerSingle writer
Safe Boolean register
![Page 5: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/5.jpg)
Art of Multiprocessor Programming
5
All the way to a Wait-free Implementation of Atomic
Snapshots
MRMW
MRSW
SRSW
SafeRegular
Atomic
M-valued
Boolean
Snapshot
![Page 6: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/6.jpg)
Art of Multiprocessor Programming
6
Rationale for wait-freedom
• We wanted atomic registers to implement mutual exclusion
![Page 7: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/7.jpg)
Art of Multiprocessor Programming
7
Rationale for wait-freedom
• We wanted atomic registers to implement mutual exclusion
• So we couldn’t use mutual exclusion to implement atomic registers
![Page 8: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/8.jpg)
Art of Multiprocessor Programming
8
Rationale for wait-freedom
• We wanted atomic registers to implement mutual exclusion
• So we couldn’t use mutual exclusion to implement atomic registers
• But wait, there’s more!
![Page 9: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/9.jpg)
Art of Multiprocessor Programming
9
Why is Mutual Exclusion so wrong?
![Page 10: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/10.jpg)
Art of Multiprocessor Programming
10
Asynchronous Interrupts
Swapped outback at
??? ???
![Page 11: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/11.jpg)
Art of Multiprocessor Programming
11
Heterogeneous Processors
??? ???yawn
supercomputersupercomputer
toaster
![Page 12: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/12.jpg)
Art of Multiprocessor Programming
12
Fault-tolerance
??? ???
![Page 13: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/13.jpg)
Art of Multiprocessor Programming
13
Machine Level Instruction Granularity
Amdahl’s Law
![Page 14: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/14.jpg)
Art of Multiprocessor Programming
14
Basic Questions
• Wait-Free synchronization might be a good idea in principle
![Page 15: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/15.jpg)
Art of Multiprocessor Programming
15
Basic Questions
• Wait-Free synchronization might be a good idea in principle
• But how do you do it …
![Page 16: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/16.jpg)
16
Basic Questions
• Wait-Free synchronization might be a good idea in principle
• But how do you do it …– Systematically?
Art of Multiprocessor Programming
![Page 17: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/17.jpg)
17
Basic Questions
• Wait-Free synchronization might be a good idea in principle
• But how do you do it …– Systematically?– Correctly?
Art of Multiprocessor Programming
![Page 18: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/18.jpg)
18
Basic Questions
• Wait-Free synchronization might be a good idea in principle
• But how do you do it …– Systematically?– Correctly?– Efficiently?
Art of Multiprocessor Programming
![Page 19: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/19.jpg)
19
FIFO Queue: Enqueue Method
q.enq( )
Art of Multiprocessor Programming
![Page 20: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/20.jpg)
20
FIFO Queue: Dequeue Method
q.deq()/
Art of Multiprocessor Programming
![Page 21: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/21.jpg)
21
Two-Thread Wait-Free Queuepublic class WaitFreeQueue { int head = 0, tail = 0; Item[QSIZE] items; public void enq(Item x) { while (tail-head == QSIZE) {}; items[tail % QSIZE] = x; tail++; } public Item deq() { while (tail-head == 0) {} Item item = items[head % QSIZE]; head++; return item;}}
Art of Multiprocessor Programming
0 1capacity-1
2
head tail
y z
![Page 22: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/22.jpg)
22
What About Multiple Dequeuers?
Art of Multiprocessor Programming
![Page 23: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/23.jpg)
23
Grand Challenge
• Implement a FIFO queue
Art of Multiprocessor Programming
![Page 24: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/24.jpg)
24
Grand Challenge
• Implement a FIFO queue– Wait-free
Art of Multiprocessor Programming
![Page 25: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/25.jpg)
25
Grand Challenge
• Implement a FIFO queue– Wait-free– Linearizable
Art of Multiprocessor Programming
![Page 26: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/26.jpg)
26
Grand Challenge
• Implement a FIFO queue– Wait-free– Linearizable– From atomic read-write registers
Art of Multiprocessor Programming
![Page 27: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/27.jpg)
27
Grand Challenge
• Implement a FIFO queue– Wait-free– Linearizable– From atomic read-write registers– Multiple dequeuers
Art of Multiprocessor Programming
![Page 28: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/28.jpg)
28
Grand Challenge
• Implement a FIFO queue– Wait-free– Linearizable– From atomic read-write registers– Multiple dequeuers
Only new aspect
Art of Multiprocessor Programming
![Page 29: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/29.jpg)
29
Puzzle
Art of Multiprocessor Programming
While you are ruminating on the grand challenge …
We will give you another puzzle …
Consensus!
![Page 30: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/30.jpg)
Art of Multiprocessor Programming
30
Consensus: Each Thread has a Private Input
32 19
21
![Page 31: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/31.jpg)
31
They Communicate
Art of Multiprocessor Programming
![Page 32: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/32.jpg)
Art of Multiprocessor Programming
32
They Agree on One Thread’s Input
19 19
19
![Page 33: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/33.jpg)
33
Formally: Consensus
• Consistent:– all threads decide the same value
Art of Multiprocessor Programming
![Page 34: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/34.jpg)
34
Formally: Consensus
• Consistent:– all threads decide the same value
• Valid:– the common decision value is some
thread's input
Art of Multiprocessor Programming
![Page 35: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/35.jpg)
Art of Multiprocessor Programming
35
No Wait-Free Implementation of Consensus using Registers
??? ???
![Page 36: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/36.jpg)
36
Formally
• Theorem – There is no wait-free implementation of n-thread consensus from read-write registers
Art of Multiprocessor Programming
![Page 37: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/37.jpg)
37
Formally
• Theorem – There is no wait-free implementation of n-thread consensus from read-write registers
• Implication– Asynchronous computability different
from Turing computability
Art of Multiprocessor Programming
![Page 38: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/38.jpg)
Proof Strategy
38Art of Multiprocessor Programming
Assume otherwise …Reason about the properties of any
such protocol …Derive a contradictionQuodEratDemonstrandum
Enough to consider binary consensus and n=2
![Page 39: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/39.jpg)
Art of Multiprocessor Programming
39
Protocol Histories as State Transitions
time
x.read() y.read()
y.read(y) x.write()
time
A
B
init state final state
![Page 40: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/40.jpg)
Art of Multiprocessor Programming
40
Wait-Free Computation
• Either A or B “moves”
• Moving means– Register read– Register write
A moves B moves
![Page 41: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/41.jpg)
Art of Multiprocessor Programming
41
The Two-Move Tree
Initial state
Final states
![Page 42: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/42.jpg)
Art of Multiprocessor Programming
42
Decision Values
1 0 0 1 1 1
![Page 43: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/43.jpg)
43
Bivalent: Both Possible
1 1 1
bivalent
1 0 0Art of Multiprocessor
Programming
![Page 44: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/44.jpg)
44
Univalent: Single Value Possible
1 1 1
univalent
1 0 0Art of Multiprocessor
Programming
![Page 45: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/45.jpg)
45
x-valent: x Only Possible Decision
0 1 1 1
1-valent
01Art of Multiprocessor
Programming
![Page 46: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/46.jpg)
46
Summary
• Wait-free computation is a tree
Art of Multiprocessor Programming
![Page 47: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/47.jpg)
47
Summary
• Wait-free computation is a tree
• Bivalent system states– Outcome not fixed
Art of Multiprocessor Programming
![Page 48: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/48.jpg)
48
Summary
• Wait-free computation is a tree
• Bivalent system states– Outcome not fixed
• Univalent states– Outcome is fixed– May not be “known” yet
Art of Multiprocessor Programming
![Page 49: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/49.jpg)
49
Summary
• Wait-free computation is a tree
• Bivalent system states– Outcome not fixed
• Univalent states– Outcome is fixed– May not be “known” yet
• 1-Valent and 0-Valent states
Art of Multiprocessor Programming
![Page 50: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/50.jpg)
50
Claim
• Some initial state is bivalent
Art of Multiprocessor Programming
![Page 51: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/51.jpg)
51
Claim
• Some initial state is bivalent
• Outcome depends on– Chance
– Whim of the scheduler
Art of Multiprocessor Programming
![Page 52: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/52.jpg)
52
Claim
• Some initial state is bivalent
• Outcome depends on– Chance
– Whim of the scheduler
• Multicore gods procrastinate …
Art of Multiprocessor Programming
![Page 53: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/53.jpg)
53
Claim
• Some initial state is bivalent
• Outcome depends on– Chance
– Whim of the scheduler
• Multicore gods procrastinate …
• Let’s prove it …
Art of Multiprocessor Programming
![Page 54: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/54.jpg)
Art of Multiprocessor Programming
54
What if inputs differ?
10
![Page 55: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/55.jpg)
Art of Multiprocessor Programming
55
Must Decide 0
In this solo execution by A
0
![Page 56: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/56.jpg)
Art of Multiprocessor Programming
56
Must Decide 1
1
In this solo execution by B
![Page 57: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/57.jpg)
Art of Multiprocessor Programming
57
Mixed Initial State Bivalent
• Solo execution by A must decide 0
• Solo execution by B must decide 1
0 1
![Page 58: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/58.jpg)
Art of Multiprocessor Programming
58
0-valent
Critical States
1-valent
critical
![Page 59: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/59.jpg)
Art of Multiprocessor Programming
59
From a Critical State
c
If A goes first, protocol decides 0
If B goes first, protocol decides 1
0-valent 1-valent
![Page 60: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/60.jpg)
60
Reaching Critical State
CA
CA
CB
c
CB
univalent
univalent
univalent
univalent
0-valent 1-valent
initially bivalent
![Page 61: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/61.jpg)
61
Critical States
• Starting from a bivalent initial state
Art of Multiprocessor Programming
![Page 62: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/62.jpg)
62
Critical States
• Starting from a bivalent initial state
• The protocol can reach a critical state
Art of Multiprocessor Programming
![Page 63: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/63.jpg)
63
Critical States
• Starting from a bivalent initial state
• The protocol can reach a critical state– Otherwise we could stay bivalent forever– And the protocol is not wait-free
Art of Multiprocessor Programming
![Page 64: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/64.jpg)
64
Model Dependency
• So far, memory-independent!
• True for– Registers– Message-passing– Carrier pigeons– Any kind of asynchronous computation
Art of Multiprocessor Programming
![Page 65: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/65.jpg)
Closing the Deal
• Start from a critical state
Art of Multiprocessor Programming
![Page 66: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/66.jpg)
Closing the Deal
• Start from a critical state
• Each thread fixes outcome by– Reading or writing …– Same or different registers
Art of Multiprocessor Programming
![Page 67: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/67.jpg)
Closing the Deal
• Start from a critical state
• Each thread fixes outcome by– Reading or writing …– Same or different registers
• Leading to a 0 or 1 decision …
Art of Multiprocessor Programming
![Page 68: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/68.jpg)
Closing the Deal
• Start from a critical state
• Each thread fixes outcome by– Reading or writing …– Same or different registers
• Leading to a 0 or 1 decision …
• And a contradiction.
Art of Multiprocessor Programming
![Page 69: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/69.jpg)
69
Possible Interactions
x.read() y.read() x.write() y.write()
x.read() ? ? ? ?
y.read() ? ? ? ?
x.write() ? ? ? ?
y.write() ? ? ? ?
Art of Multiprocessor Programming
![Page 70: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/70.jpg)
70
Possible Interactions
x.read() y.read() x.write() y.write()
x.read() ? ? ? ?
y.read() ? ? ? ?
x.write() ? ? ? ?
y.write() ? ? ? ?
A reads x
Art of Multiprocessor Programming
![Page 71: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/71.jpg)
71
Possible Interactions
x.read() y.read() x.write() y.write()
x.read() ? ? ? ?
y.read() ? ? ? ?
x.write() ? ? ? ?
y.write() ? ? ? ?
A reads xA reads y
Art of Multiprocessor Programming
![Page 72: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/72.jpg)
72
Some Thread Reads
c
Art of Multiprocessor Programming
![Page 73: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/73.jpg)
73
Some Thread Reads
A runs solo, eventually decides 0
0
c
Art of Multiprocessor Programming
![Page 74: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/74.jpg)
74
Some Thread Reads
A runs solo, eventually decides 0
B reads x
0
c
Art of Multiprocessor Programming
![Page 75: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/75.jpg)
75
Some Thread Reads
A runs solo, eventually decides 0
B reads x
1
0
A runs solo, eventually decides 1
c
Art of Multiprocessor Programming
![Page 76: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/76.jpg)
76
Some Thread Reads
A runs solo, eventually decides 0
B reads x
1
0
A runs solo, eventually decides 1
c
States look the same to A
Art of Multiprocessor Programming
![Page 77: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/77.jpg)
77
Some Thread Reads
A runs solo, eventually decides 0
B reads x
1
0
A runs solo, eventually decides 1
c
States look the same to A Contra
diction
Art of Multiprocessor Programming
![Page 78: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/78.jpg)
78
Possible Interactions
x.read() y.read() x.write() y.write()
x.read() no no no no
y.read() no no no no
x.write() no no ? ?
y.write() no no ? ?
Art of Multiprocessor Programming
![Page 79: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/79.jpg)
Art of Multiprocessor Programming
79
Writing Distinct Registers
c
![Page 80: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/80.jpg)
80
Writing Distinct Registers
A writes yc
Art of Multiprocessor Programming
![Page 81: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/81.jpg)
81
Writing Distinct Registers
A writes yc
B writes x
Art of Multiprocessor Programming
![Page 82: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/82.jpg)
82
Writing Distinct Registers
A writes yc
B writes x
0Art of Multiprocessor
Programming
![Page 83: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/83.jpg)
83
Writing Distinct Registers
A writes y B writes xc
B writes x
0Art of Multiprocessor
Programming
![Page 84: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/84.jpg)
Art of Multiprocessor Programming
84
Writing Distinct Registers
A writes y B writes xc
A writes yB writes x
0
![Page 85: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/85.jpg)
Art of Multiprocessor Programming
85
Writing Distinct Registers
A writes y B writes xc
A writes yB writes x
0 1
![Page 86: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/86.jpg)
86
Writing Distinct Registers
A writes y B writes xc
Same story
A writes yB writes x
0 1
![Page 87: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/87.jpg)
87
Writing Distinct Registers
A writes y B writes xc
Same story
A writes yB writes x
0 1Contra
diction
![Page 88: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/88.jpg)
Art of Multiprocessor Programming
88
Possible Interactions
x.read() y.read() x.write() y.write()
x.read() no no no no
y.read() no no no no
x.write() no no ? no
y.write() no no no ?
![Page 89: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/89.jpg)
Art of Multiprocessor Programming
89
Writing Same Registers
States look the same to A
A writes x B writes x
1A runs solo, eventually decides 1
c
0
A runs solo, eventually decides 0 A writes x
Contradictio
n
![Page 90: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/90.jpg)
Art of Multiprocessor Programming
90
That’s All, Folks!
x.read() y.read() x.write() y.write()
x.read() no no no no
y.read() no no no no
x.write() no no no no
y.write() no no no noQED
![Page 91: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/91.jpg)
Art of Multiprocessor Programming
91
Recap: Atomic Registers Can’t Do Consensus
• If protocol exists– It has a bivalent initial state– Leading to a critical state
• What’s up with the critical state?– Case analysis for each pair of methods– As we showed, all lead to a contradiction
![Page 92: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/92.jpg)
Art of Multiprocessor Programming
92
What Does Consensus have to do with Concurrent Objects?
![Page 93: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/93.jpg)
Art of Multiprocessor Programming
93
Consensus Object
public interface Consensus<T> { T decide(T value);}
![Page 94: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/94.jpg)
Art of Multiprocessor Programming
94
Concurrent Consensus Object
• We consider only one time objects: – each thread calls method only once
• Linearizable to consensus object:– Winner’s call went first
![Page 95: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/95.jpg)
Art of Multiprocessor Programming
95
Java Jargon Watch
• Define Consensus protocol as an abstract class
• We implement some methods
• You do the rest …
![Page 96: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/96.jpg)
Art of Multiprocessor Programming
96
Generic Consensus Protocol
abstract class ConsensusProtocol<T> implements Consensus<T> { protected T[] proposed = new T[N];
protected void propose(T value) { proposed[ThreadID.get()] = value; }
abstract public T decide(T value);}
![Page 97: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/97.jpg)
Art of Multiprocessor Programming
97
abstract class ConsensusProtocol<T> implements Consensus<T> { protected T[] proposed = new T[N];
protected void propose(T value) { proposed[ThreadID.get()] = value; }
abstract public T decide(T value);}
Generic Consensus Protocol
Each thread’s proposed value
![Page 98: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/98.jpg)
Art of Multiprocessor Programming
98
abstract class ConsensusProtocol<T> implements Consensus<T> { protected T[] proposed = new T[N];
protected void propose(T value) { proposed[ThreadID.get()] = value; }
abstract public T decide(T value);}
Generic Consensus Protocol
Propose a value
![Page 99: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/99.jpg)
Art of Multiprocessor Programming
99
abstract class ConsensusProtocol<T> implements Consensus { protected T[] proposed = new T[N];
protected void propose(T value) { proposed[ThreadID.get()] = value; }
abstract public T decide(T value);}
Generic Consensus Protocol
Decide a value: abstract method means subclass does the real work
![Page 100: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/100.jpg)
Art of Multiprocessor Programming
100
Can a FIFO QueueImplement Consensus?
![Page 101: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/101.jpg)
Art of Multiprocessor Programming
101
FIFO Consensus
proposed array
FIFO Queue with red and black balls
8
Coveted red ball Dreaded black ball
![Page 102: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/102.jpg)
Art of Multiprocessor Programming
102
Protocol: Write Value to Array
0 10
![Page 103: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/103.jpg)
Art of Multiprocessor Programming
103
0
Protocol: Take Next Item from Queue
0 18
![Page 104: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/104.jpg)
Art of Multiprocessor Programming
104
0 1
Protocol: Take Next Item from Queue
I got the coveted red ball, so I will decide my value
I got the dreaded black ball, so I will decide the other’s
value from the array8
![Page 105: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/105.jpg)
Art of Multiprocessor Programming
105
public class QueueConsensus<T> extends ConsensusProtocol<T> { private Queue queue; public QueueConsensus() { queue = new Queue(); queue.enq(Ball.RED); queue.enq(Ball.BLACK); } …}
Consensus Using FIFO Queue
![Page 106: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/106.jpg)
Art of Multiprocessor Programming
106
public class QueueConsensus extends ConsensusProtocol { private Queue queue; public QueueConsensus() { this.queue = new Queue(); this.queue.enq(Ball.RED); this.queue.enq(Ball.BLACK); } …}
Initialize Queue
8
![Page 107: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/107.jpg)
Art of Multiprocessor Programming
107
public class QueueConsensus<T> extends ConsensusProtocol<T> { private Queue queue; … public T decide(T value) { propose(value); Ball ball = queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; }}
Who Won?
![Page 108: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/108.jpg)
Art of Multiprocessor Programming
108
public class QueueConsensus<T> extends ConsensusProtocol<T> { private Queue queue; … public T decide(T value) { propose(value); Ball ball = queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-ij]; }}
Who Won?
Race to dequeue first queue item
![Page 109: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/109.jpg)
Art of Multiprocessor Programming
109
public class QueueConsensus<T> extends ConsensusProtocol<T> { private Queue queue; … public T decide(T value) { propose(value); Ball ball = this.queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; }}
Who Won?
I win if I was first
![Page 110: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/110.jpg)
Art of Multiprocessor Programming
110
public class QueueConsensus<T> extends ConsensusProtocol<T> { private Queue queue; … public T decide(T value) { propose(value); Ball ball = this.queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; }}
Who Won?
Other thread wins if I was second
![Page 111: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/111.jpg)
111
Why does this Work?
• If one thread gets the red ball
• Then the other gets the black ball
• Winner decides her own value
• Loser can find winner’s value in array– Because threads write array– Before dequeueing from queue
Art of Multiprocessor Programming
![Page 112: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/112.jpg)
112
Theorem
• We can solve 2-thread consensus using only– A two-dequeuer queue, and– Some atomic registers
Art of Multiprocessor Programming
![Page 113: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/113.jpg)
Art of Multiprocessor Programming
113
Implications
• Given– A consensus protocol from queue and registers
• Assume there exists– A queue implementation from atomic registers
• Substitution yields:– A wait-free consensus protocol from atomic
registers
contradiction
![Page 114: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/114.jpg)
114
Corollary
• It is impossible to implement – a two-dequeuer wait-free FIFO queue– from read/write memory.
Art of Multiprocessor Programming
![Page 115: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/115.jpg)
115
Consensus Numbers
• An object X has consensus number n– If it can be used to solve n-thread
consensus• Take any number of instances of X • together with atomic read/write registers• and implement n-thread consensus
– But not (n+1)-thread consensus
Art of Multiprocessor Programming
![Page 116: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/116.jpg)
116
Consensus Numbers
• Theorem– Atomic read/write registers have
consensus number 1
• Theorem– Multi-dequeuer FIFO queues have
consensus number at least 2
Art of Multiprocessor Programming
![Page 117: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/117.jpg)
117
Consensus Numbers Measure Synchronization Power
• Theorem– If you can implement X from Y– And X has consensus number c– Then Y has consensus number at least c
Art of Multiprocessor Programming
![Page 118: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/118.jpg)
Art of Multiprocessor Programming
118
Synchronization Speed Limit
• Conversely– If X has consensus number c– And Y has consensus number d < c– Then there is no way to construct a wait-
free implementation of X by Y
• This theorem will be very useful– Unforeseen practical implications!
Theoretical
Caveat: Certain
weird exceptions exist
![Page 119: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/119.jpg)
119
Earlier Grand Challenge
• Snapshot means– Write any array element– Read multiple array elements atomically
• What about– Write multiple array elements atomically– Scan any array elements
• Call this problem multiple assignment
Art of Multiprocessor Programming
![Page 120: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/120.jpg)
Art of Multiprocessor Programming
120
Multiple Assignment Theorem
• Atomic registers cannot implement multiple assignment
• Weird or what?– Single write/multi read OK– Multi write/multi read impossible
![Page 121: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/121.jpg)
Art of Multiprocessor Programming
121
Proof Strategy
• If we can write to 2/3 array elements– We can solve 2-consensus– Impossible with atomic registers
• Therefore– Cannot implement multiple assignment
with atomic registers
(1)
![Page 122: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/122.jpg)
Art of Multiprocessor Programming
122
Proof Strategy
• Take a 3-element array– A writes atomically to slots 0 and 1– B writes atomically to slots 1 and 2– Any thread can scan any set of locations
![Page 123: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/123.jpg)
Art of Multiprocessor Programming
126
Initially
Writes to 0 and 1
Writes to 1 and 2
A
B
![Page 124: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/124.jpg)
Art of Multiprocessor Programming
127
Thread A wins if
A
BThread B
didn’t move
![Page 125: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/125.jpg)
Art of Multiprocessor Programming
128
Thread A wins if
A
BThread B
moved later
![Page 126: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/126.jpg)
Art of Multiprocessor Programming
129
Thread A loses if
A
BThread B
moved earlier
![Page 127: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/127.jpg)
139
Summary
• If a thread can assign atomically to 2 out of 3 array locations
• Then we can solve 2-consensus
• Therefore– No wait-free multi-assignment– From read/write registers
Art of Multiprocessor Programming
![Page 128: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/128.jpg)
140
Read-Modify-Write Objects
• Method call– Returns object’s prior value x– Replaces x with mumble(x)
Art of Multiprocessor Programming
![Page 129: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/129.jpg)
141
public abstract class RMWRegister { private int value;
public int synchronized getAndMumble() { int prior = value; value = mumble(value); return prior; }}
Read-Modify-Write
Art of Multiprocessor Programming
![Page 130: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/130.jpg)
Art of Multiprocessor Programming
142
public abstract class RMWRegister { private int value;
public int synchronized getAndMumble() { int prior = value; value = mumble(value); return prior; }}
Read-Modify-Write
Return prior value
![Page 131: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/131.jpg)
Art of Multiprocessor Programming
143
public abstract class RMWRegister { private int value;
public int synchronized getAndMumble() { int prior = value; value = mumble(value); return prior; }}
Read-Modify-Write
Apply function to current value
![Page 132: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/132.jpg)
144
RMW Everywhere!
• Most synchronization instructions– are RMW methods
• The rest– Can be trivially transformed into RMW
methods
Art of Multiprocessor Programming
![Page 133: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/133.jpg)
145
public abstract class RMWRegister { private int value;
public int synchronized read() { int prior = value; value = value; return prior; }
}
Example: Read
Art of Multiprocessor Programming
![Page 134: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/134.jpg)
Art of Multiprocessor Programming
146
public abstract class RMW { private int value;
public int synchronized read() { int prior = this.value; value = value; return prior; }
}
Example: Read
apply f(v)=v, the identity function
![Page 135: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/135.jpg)
147
public abstract class RMWRegister { private int value;
public int synchronized getAndSet(int v) { int prior = value; value = v; return prior; } …}
Example: getAndSet
Art of Multiprocessor Programming
![Page 136: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/136.jpg)
Art of Multiprocessor Programming
148
public abstract class RMWRegister { private int value;
public int synchronized getAndSet(int v) { int prior = value; value = v; return prior; } …}
Example: getAndSet (swap)
f(x)=v is constant
![Page 137: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/137.jpg)
149
public abstract class RMWRegister { private int value;
public int synchronized getAndIncrement() { int prior = value; value = value + 1; return prior; } …}
getAndIncrement
Art of Multiprocessor Programming
![Page 138: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/138.jpg)
Art of Multiprocessor Programming
150
public abstract class RMWRegister { private int value;
public int synchronized getAndIncrement() { int prior = value; value = value + 1; return prior; } …}
getAndIncrement
f(x) = x+1
![Page 139: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/139.jpg)
151
public abstract class RMWRegister { private int value;
public int synchronized getAndAdd(int a) { int prior = value; value = value + a; return prior; } …}
getAndAdd
Art of Multiprocessor Programming
![Page 140: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/140.jpg)
Art of Multiprocessor Programming
152
public abstract class RMWRegister { private int value;
public int synchronized getAndIncrement(int a) { int prior = value; value = value + a; return prior; } …}
Example: getAndAdd
f(x) = x+a
![Page 141: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/141.jpg)
153
public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = value; if (value==expected) { value = update; return true; } return false; } … }
compareAndSet
Art of Multiprocessor Programming
![Page 142: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/142.jpg)
Art of Multiprocessor Programming
154
public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = value; if (value==expected) { value = update; return true; } return false; } … }
compareAndSet
If value is as expected, …
![Page 143: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/143.jpg)
Art of Multiprocessor Programming
155
public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = value; if (value==expected) { value = update; return true; } return false; } … }
compareAndSet
… replace it
![Page 144: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/144.jpg)
Art of Multiprocessor Programming
156
public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = value; if (value==expected) { value = update; return true; } return false; } … }
compareAndSet
Report success
![Page 145: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/145.jpg)
Art of Multiprocessor Programming
157
public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = value; if (value==expected) { value = update; return true; } return false; } … }
compareAndSet
Otherwise report failure
![Page 146: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/146.jpg)
Art of Multiprocessor Programming
158
public abstract class RMWRegister { private int value;
public void synchronized getAndMumble() { int prior = value; value = mumble(value); return prior; }}
Read-Modify-Write
Lets characterize f(x)…
![Page 147: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/147.jpg)
159
Definition
• A RMW method– With function mumble(x)
– is non-trivial if there exists a value v
– Such that v ≠ mumble(v)
Art of Multiprocessor Programming
![Page 148: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/148.jpg)
160
Par Example
• Identity(x) = x– is trivial
• getAndIncrement(x) = x+1– is non-trivial
Art of Multiprocessor Programming
![Page 149: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/149.jpg)
161
Theorem
• Any non-trivial RMW object has consensus number at least 2
• No wait-free implementation of RMW registers from atomic registers
• Hardware RMW instructions not just a convenience
Art of Multiprocessor Programming
![Page 150: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/150.jpg)
162
Reminder
• Subclasses of consensus have– propose(x) method
• which just stores x into proposed[i]• built-in method
– decide(object value) method• which determines winning value• customized, class-specific method
Art of Multiprocessor Programming
![Page 151: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/151.jpg)
163
Proof
public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public T decide(T value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }}
Art of Multiprocessor Programming
![Page 152: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/152.jpg)
Art of Multiprocessor Programming
164
public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public T decide(T value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }}
Proof
Initialized to v
![Page 153: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/153.jpg)
Art of Multiprocessor Programming
165
Proof
public class RMWConsensus extends Consensus { private RMWRegister r = v; public T decide(T value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }}
Am I first?
![Page 154: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/154.jpg)
Art of Multiprocessor Programming
166
public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public T decide(T value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }}
Proof
Yes, return my input
![Page 155: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/155.jpg)
Art of Multiprocessor Programming
167
public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public T decide(T value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }}
Proof
No, return other’s input
![Page 156: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/156.jpg)
168
Proof
• We have displayed– A two-thread consensus protocol– Using any non-trivial RMW object
Art of Multiprocessor Programming
![Page 157: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/157.jpg)
169
Interfering RMW
• Let F be a set of functions such that for all fi and fj, either
– Commute: fi(fj(v))=fj(fi(v))
– Overwrite: fi(fj(v))=fi(v)
• Claim: Any set of RMW objects that commutes or overwrites has consensus number exactly 2
Art of Multiprocessor Programming
![Page 158: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/158.jpg)
Art of Multiprocessor Programming
170
Examples
• “test-and-set” getAndSet(1) f(v)=1
• “swap” getAndSet(x) f(v,x)=x
• “fetch-and-inc” getAndIncrement() f(v)=v+1
Overwrite fi(fj(v))=fi(v)
Overwrite fi(fj(v))=fi(v)
Commute fi(fj(v))= fj(fi(v))
![Page 159: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/159.jpg)
Art of Multiprocessor Programming
171
Meanwhile Back at the Critical State
c
0-valent 1-valent
A about to apply fA
B about to apply fB
![Page 160: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/160.jpg)
Art of Multiprocessor Programming
172
Maybe the Functions Commute
c
0-valent
A applies fA B applies fB
A applies fAB applies fB
0 1
C runs solo C runs solo
1-valent
![Page 161: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/161.jpg)
173
Maybe the Functions Commute
c
0-valent
A applies fA B applies fB
A applies fAB applies fB
0 1
C runs solo C runs solo
1-valent
These states look the same to CThese states look the same to C
Contradictio
n
Art of Multiprocessor Programming
![Page 162: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/162.jpg)
Art of Multiprocessor Programming
174
Maybe the Functions Overwrite
c
0-valent
A applies fA B applies fB
A applies fA
0
1
C runs solo
C runs solo
1-valent
![Page 163: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/163.jpg)
Art of Multiprocessor Programming
175
Maybe the Functions Overwrite
c
0-valent
A applies fA B applies fB
A applies fA
0
1
C runs solo
C runs solo
1-valent
These states look the same to CThese states look the same to C
Contradictio
n
![Page 164: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/164.jpg)
176
Impact
• Many early machines provided these “weak” RMW instructions– Test-and-set (IBM 360)– Fetch-and-add (NYU Ultracomputer)– Swap (Original SPARCs)
• We now understand their limitations– But why do we want consensus anyway?
Art of Multiprocessor Programming
![Page 165: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/165.jpg)
177
public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = value; if (value==expected) { value = update; return true; } return false; } … }
compareAndSet
Art of Multiprocessor Programming
![Page 166: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/166.jpg)
Art of Multiprocessor Programming
178
public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = this.value; if (value==expected) { this.value = update; return true; } return false; } … }
compareAndSet
replace value if it’s what we expected, …
![Page 167: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/167.jpg)
Art of Multiprocessor Programming
179
public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public T decide(T value) { propose(value); r.compareAndSet(-1,i); return proposed[r.get()]; }}
compareAndSet Has ∞ Consensus Number
![Page 168: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/168.jpg)
Art of Multiprocessor Programming
180
public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public T decide(T value) { propose(value) r.compareAndSet(-1,i); return proposed[r.get()]; }}
compareAndSet Has ∞ Consensus Number
Initialized to -1
![Page 169: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/169.jpg)
Art of Multiprocessor Programming
181
public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public T decide(T value) { propose(value); r.compareAndSet(-1,i); return proposed[r.get()]; }}
compareAndSet Has ∞ Consensus Number
Try to swap in my id
![Page 170: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/170.jpg)
Art of Multiprocessor Programming
182
public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public T decide(T value) { propose(value); r.compareAndSet(-1,i); return proposed[r.get()]; }}
compareAndSet Has ∞ Consensus Number
Decide winner’s preference
![Page 171: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/171.jpg)
Art of Multiprocessor Programming
183
The Consensus Hierarchy
1 Read/Write Registers, Snapshots…
2 getAndSet, getAndIncrement, …
∞ compareAndSet,…
.
.
.
![Page 172: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/172.jpg)
184
Multiple Assignment
• Atomic k-assignment
• Solves consensus for 2k-2 threads• Every even consensus number has an
object (can be extended to odd numbers)
Art of Multiprocessor Programming
![Page 173: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/173.jpg)
185
Lock-Freedom
• Lock-free:– in an infinite execution– infinitely often some method call finishes
• Pragmatic approach
• Implies no mutual exclusion
Art of Multiprocessor Programming
![Page 174: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/174.jpg)
186
Lock-Free vs. Wait-free
• Wait-Free: each method call takes a finite number of steps to finish
• Lock-free: infinitely often some method call finishes
Art of Multiprocessor Programming
![Page 175: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/175.jpg)
187
Lock-Freedom
• Any wait-free implementation is lock-free.
• Lock-free is the same as wait-free if the execution is finite.
Art of Multiprocessor Programming
![Page 176: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/176.jpg)
188
Lock-Free Implementations
• Lock-free consensus is as impossible as wait-free consensus
• All these results hold for lock-free algorithms also.
Art of Multiprocessor Programming
![Page 177: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/177.jpg)
Art of Multiprocessor Programming
189
There is More: Universality
• Consensus is universal
• From n-thread consensus– Wait-free/Lock-free– Linearizable– n-threaded– Implementation– Of any sequentially specified objectStay tu
ned…
![Page 178: The Relative Power of Synchronization Operations Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit](https://reader035.vdocument.in/reader035/viewer/2022081414/5514792c550346ea6e8b45c6/html5/thumbnails/178.jpg)
Art of Multiprocessor Programming
190
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License.
• You are free:– to Share — to copy, distribute and transmit the work – to Remix — to adapt the work
• Under the following conditions:– Attribution. You must attribute the work to “The Art of
Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work).
– Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license.
• For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to– http://creativecommons.org/licenses/by-sa/3.0/.
• Any of the above conditions can be waived if you get permission from the copyright holder.
• Nothing in this license impairs or restricts the author's moral rights.