![Page 1: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/1.jpg)
Introduction
Companion slides forThe Art of Multiprocessor
Programmingby Maurice Herlihy & Nir Shavit
![Page 2: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/2.jpg)
Art of Multiprocessor Programming 2
Moore’s Law
Clock speed
flattening sharply
Transistor count still
rising
![Page 3: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/3.jpg)
Art of Multiprocessor Programming 3
Still on some of your desktops: The Uniprocesor
memory
cpu
![Page 4: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/4.jpg)
Art of Multiprocessor Programming 4
In the Enterprise: The Shared Memory
Multiprocessor(SMP)
cache
BusBus
shared memory
cachecache
![Page 5: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/5.jpg)
Art of Multiprocessor Programming 5
Your New Desktop: The Multicore Processor
(CMP)
cache
BusBus
shared memory
cachecacheAll on the same chip
Sun T2000Niagara
![Page 6: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/6.jpg)
Art of Multiprocessor Programming 6
Multicores Are Here
• “Intel's Intel ups ante with 4-core chip. New microprocessor, due this year, will be faster, use less electricity...” [San Fran Chronicle]
• “AMD will launch a dual-core version of its Opteron server processor at an event in New York on April 21.” [PC World]
• “Sun’s Niagara…will have eight cores, each core capable of running 4 threads in parallel, for 32 concurrently running threads. ….” [The Inquirer]
![Page 7: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/7.jpg)
Art of Multiprocessor Programming 7
Why do we care?
• Time no longer cures software bloat– The “free ride” is over
• When you double your program’s path length– You can’t just wait 6 months– Your software must somehow exploit
twice as much concurrency
![Page 8: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/8.jpg)
Art of Multiprocessor Programming 8
Traditional Scaling Process
User code
TraditionalUniprocessor
Speedup1.8x1.8x
7x7x
3.6x3.6x
Time: Moore’s law
![Page 9: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/9.jpg)
Art of Multiprocessor Programming 9
Multicore Scaling Process
User code
Multicore
Speedup 1.8x1.8x
7x7x
3.6x3.6x
Unfortunately, not so simple…
![Page 10: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/10.jpg)
Art of Multiprocessor Programming 10
Real-World Scaling Process
1.8x1.8x 2x2x 2.9x2.9x
User code
Multicore
Speedup
Parallelization and Synchronization require great care…
![Page 11: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/11.jpg)
Art of Multiprocessor Programming 11
Multicore Programming: Course Overview
• Fundamentals– Models, algorithms, impossibility
• Real-World programming– Architectures– Techniques
![Page 12: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/12.jpg)
Art of Multiprocessor Programming 12
Multicore Programming: Course Overview
• Fundamentals– Models, algorithms, impossibility
• Real-World programming– Architectures– Techniques
We don’t necessarily
want to m
ake
you experts…
![Page 13: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/13.jpg)
Art of Multiprocessor Programming
13
Sequential Computation
memory
object object
thread
![Page 14: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/14.jpg)
Art of Multiprocessor Programming
14
Concurrent Computation
memory
object object
thre
ads
![Page 15: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/15.jpg)
Art of Multiprocessor Programming 15
Asynchrony
• Sudden unpredictable delays– Cache misses (short)– Page faults (long)– Scheduling quantum used up (really
long)
![Page 16: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/16.jpg)
Art of Multiprocessor Programming 16
Model Summary
• Multiple threads– Sometimes called processes
• Single shared memory• Objects live in memory• Unpredictable asynchronous
delays
![Page 17: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/17.jpg)
Art of Multiprocessor Programming 17
Road Map
• We are going to focus on principles first, then practice– Start with idealized models– Look at simplistic problems– Emphasize correctness over
pragmatism– “Correctness may be theoretical, but
incorrectness has practical impact”
![Page 18: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/18.jpg)
Art of Multiprocessor Programming 18
Concurrency Jargon
• Hardware– Processors
• Software– Threads, processes
• Sometimes OK to confuse them, sometimes not.
![Page 19: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/19.jpg)
Art of Multiprocessor Programming 19
Parallel Primality Testing
• Challenge– Print primes from 1 to 1010
• Given– Ten-processor multiprocessor– One thread per processor
• Goal– Get ten-fold speedup (or close)
![Page 20: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/20.jpg)
Art of Multiprocessor Programming 20
Load Balancing
• Split the work evenly• Each thread tests range of 109
…
…109 10102·1091
P0 P1 P9
![Page 21: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/21.jpg)
Art of Multiprocessor Programming
21
Procedure for Thread i
void primePrint { int i = ThreadID.get(); // IDs in {0..9} for (j = i*109+1, j<(i+1)*109; j++) { if (isPrime(j)) print(j); }}
![Page 22: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/22.jpg)
Art of Multiprocessor Programming 22
Issues
• Higher ranges have fewer primes• Yet larger numbers harder to test• Thread workloads
– Uneven– Hard to predict
![Page 23: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/23.jpg)
Art of Multiprocessor Programming 23
Issues
• Higher ranges have fewer primes• Yet larger numbers harder to test• Thread workloads
– Uneven– Hard to predict
• Need dynamic load balancingre
jected
![Page 24: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/24.jpg)
Art of Multiprocessor Programming
24
17
18
19
Shared Counter
each thread takes a number
![Page 25: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/25.jpg)
Art of Multiprocessor Programming
25
Procedure for Thread i
int counter = new Counter(1); void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); }}
![Page 26: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/26.jpg)
Art of Multiprocessor Programming
26
Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); }}
Procedure for Thread i
Shared counterobject
![Page 27: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/27.jpg)
Art of Multiprocessor Programming 27
Where Things Reside
cache
BusBus
cachecache
1
shared counter
shared memory
void primePrint { int i = ThreadID.get(); // IDs in {0..9} for (j = i*109+1, j<(i+1)*109; j++) { if (isPrime(j)) print(j); }}
code
Local variables
![Page 28: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/28.jpg)
Art of Multiprocessor Programming
28
Procedure for Thread i
Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); }}
Stop when every value taken
![Page 29: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/29.jpg)
Art of Multiprocessor Programming
29
Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); }}
Procedure for Thread i
Increment & return each new
value
![Page 30: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/30.jpg)
Art of Multiprocessor Programming
30
Counter Implementation
public class Counter { private long value;
public long getAndIncrement() { return value++; }}
![Page 31: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/31.jpg)
Art of Multiprocessor Programming
31
Counter Implementation
public class Counter { private long value;
public long getAndIncrement() { return value++; }} OK for single thread,
not for concurrent threads
![Page 32: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/32.jpg)
Art of Multiprocessor Programming
32
What It Means
public class Counter { private long value;
public long getAndIncrement() { return value++; }}
![Page 33: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/33.jpg)
Art of Multiprocessor Programming
33
What It Means
public class Counter { private long value;
public long getAndIncrement() { return value++; }}
temp = value; value = value + 1; return temp;
![Page 34: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/34.jpg)
Art of Multiprocessor Programming
34
time
Not so good…
Value… 1
read 1
read 1
write 2
read 2
write 3
write 2
2 3 2
![Page 35: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/35.jpg)
Art of Multiprocessor Programming
35
Is this problem inherent?
If we could only glue reads and writes…
read
write read
write
![Page 36: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/36.jpg)
Art of Multiprocessor Programming
36
Challenge
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; }}
![Page 37: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/37.jpg)
Art of Multiprocessor Programming
37
Challenge
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; }}
Make these steps atomic (indivisible)
![Page 38: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/38.jpg)
Art of Multiprocessor Programming
38
Hardware Solution
public class Counter { private long value;
public long getAndIncrement() { temp = value; value = temp + 1; return temp; }} ReadModifyWrite()
instruction
![Page 39: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/39.jpg)
Art of Multiprocessor Programming
39
An Aside: Java™
public class Counter { private long value;
public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; }}
![Page 40: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/40.jpg)
Art of Multiprocessor Programming
40
An Aside: Java™
public class Counter { private long value;
public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; }}
Synchronized block
![Page 41: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/41.jpg)
Art of Multiprocessor Programming
41
An Aside: Java™
public class Counter { private long value;
public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; }}
Mutual Exclusion
![Page 42: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/42.jpg)
Art of Multiprocessor Programming
42
Mutual Exclusion or “Alice & Bob share a pond”
A B
![Page 43: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/43.jpg)
Art of Multiprocessor Programming
43
Alice has a pet
A B
![Page 44: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/44.jpg)
Art of Multiprocessor Programming
44
Bob has a pet
A B
![Page 45: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/45.jpg)
Art of Multiprocessor Programming
45
The Problem
A B
The pets don’tget along
![Page 46: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/46.jpg)
Art of Multiprocessor Programming 46
Formalizing the Problem
• Two types of formal properties in asynchronous computation:
• Safety Properties– Nothing bad happens ever
• Liveness Properties – Something good happens eventually
![Page 47: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/47.jpg)
Art of Multiprocessor Programming 47
Formalizing our Problem
• Mutual Exclusion– Both pets never in pond
simultaneously– This is a safety property
• No Deadlock– if only one wants in, it gets in– if both want in, one gets in.– This is a liveness property
![Page 48: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/48.jpg)
Art of Multiprocessor Programming 48
Simple Protocol
• Idea– Just look at the pond
• Gotcha– Trees obscure the view
![Page 49: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/49.jpg)
Art of Multiprocessor Programming 49
Interpretation
• Threads can’t “see” what other threads are doing
• Explicit communication required for coordination
![Page 50: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/50.jpg)
Art of Multiprocessor Programming 50
Cell Phone Protocol
• Idea– Bob calls Alice (or vice-versa)
• Gotcha– Bob takes shower– Alice recharges battery– Bob out shopping for pet food …
![Page 51: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/51.jpg)
Art of Multiprocessor Programming 51
Interpretation
• Message-passing doesn’t work• Recipient might not be
– Listening– There at all
• Communication must be– Persistent (like writing)– Not transient (like speaking)
![Page 52: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/52.jpg)
Art of Multiprocessor Programming
52
Can Protocol
cola
cola
![Page 53: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/53.jpg)
Art of Multiprocessor Programming
53
Bob conveys a bit
A B
cola
![Page 54: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/54.jpg)
Art of Multiprocessor Programming
54
Bob conveys a bit
A B
cola
![Page 55: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/55.jpg)
Art of Multiprocessor Programming 55
Can Protocol
• Idea– Cans on Alice’s windowsill– Strings lead to Bob’s house– Bob pulls strings, knocks over cans
• Gotcha– Cans cannot be reused– Bob runs out of cans
![Page 56: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/56.jpg)
Art of Multiprocessor Programming 56
Interpretation
• Cannot solve mutual exclusion with interrupts– Sender sets fixed bit in receiver’s
space– Receiver resets bit when ready– Requires unbounded number of
inturrupt bits
![Page 57: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/57.jpg)
Art of Multiprocessor Programming
57
Flag Protocol
A B
![Page 58: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/58.jpg)
Art of Multiprocessor Programming
58
Alice’s Protocol (sort of)
A B
![Page 59: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/59.jpg)
Art of Multiprocessor Programming
59
Bob’s Protocol (sort of)
A B
![Page 60: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/60.jpg)
Art of Multiprocessor Programming 60
Alice’s Protocol
• Raise flag• Wait until Bob’s flag is down• Unleash pet• Lower flag when pet returns
![Page 61: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/61.jpg)
Art of Multiprocessor Programming 61
Bob’s Protocol
• Raise flag• Wait until Alice’s flag is down• Unleash pet• Lower flag when pet returns
dang
er!
![Page 62: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/62.jpg)
Art of Multiprocessor Programming 62
Bob’s Protocol (2nd try)
• Raise flag• While Alice’s flag is up
– Lower flag– Wait for Alice’s flag to go down– Raise flag
• Unleash pet• Lower flag when pet returns
![Page 63: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/63.jpg)
Art of Multiprocessor Programming 63
Bob’s Protocol
• Raise flag• While Alice’s flag is up
– Lower flag– Wait for Alice’s flag to go down– Raise flag
• Unleash pet• Lower flag when pet returns
Bob defers to Alice
![Page 64: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/64.jpg)
Art of Multiprocessor Programming 64
The Flag Principle
• Raise the flag• Look at other’s flag• Flag Principle:
– If each raises and looks, then– Last to look must see both flags up
![Page 65: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/65.jpg)
Art of Multiprocessor Programming 65
Proof of Mutual Exclusion
• Assume both pets in pond– Derive a contradiction– By reasoning backwards
• Consider the last time Alice and Bob each looked before letting the pets in
• Without loss of generality assume Alice was the last to look…
![Page 66: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/66.jpg)
Art of Multiprocessor Programming
66
Proof
time
Alice’s last look
Alice last raised her flag
Bob’s last look
QED
Alice must have seen Bob’s Flag. A Contradiction
Bob last raised flag
![Page 67: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/67.jpg)
Art of Multiprocessor Programming 67
Proof of No Deadlock
• If only one pet wants in, it gets in.
![Page 68: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/68.jpg)
Art of Multiprocessor Programming 68
Proof of No Deadlock
• If only one pet wants in, it gets in.• Deadlock requires both continually
trying to get in.
![Page 69: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/69.jpg)
Art of Multiprocessor Programming 69
Proof of No Deadlock
• If only one pet wants in, it gets in.• Deadlock requires both continually
trying to get in.• If Bob sees Alice’s flag, he gives
her priority (a gentleman…)
QED
![Page 70: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/70.jpg)
Art of Multiprocessor Programming 70
Remarks
• Protocol is unfair– Bob’s pet might never get in
• Protocol uses waiting– If Bob is eaten by his pet, Alice’s pet
might never get in
![Page 71: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/71.jpg)
Art of Multiprocessor Programming 71
Moral of Story
•Mutual Exclusion cannot be solved by
–transient communication (cell phones)– interrupts (cans)
•It can be solved by– one-bit shared variables – that can be read or written
![Page 72: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/72.jpg)
Art of Multiprocessor Programming 72
The Arbiter Problem (an aside)
Pick a point
Pick a point
![Page 73: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/73.jpg)
Art of Multiprocessor Programming 73
The Fable Continues
• Alice and Bob fall in love & marry
![Page 74: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/74.jpg)
Art of Multiprocessor Programming 74
The Fable Continues
• Alice and Bob fall in love & marry• Then they fall out of love & divorce
– She gets the pets– He has to feed them
![Page 75: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/75.jpg)
Art of Multiprocessor Programming 75
The Fable Continues
• Alice and Bob fall in love & marry• Then they fall out of love & divorce
– She gets the pets– He has to feed them
• Leading to a new coordination problem: Producer-Consumer
![Page 76: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/76.jpg)
Art of Multiprocessor Programming
76
Bob Puts Food in the Pond
A
![Page 77: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/77.jpg)
Art of Multiprocessor Programming
77
mmm…
Alice releases her pets to Feed
Bmmm…
![Page 78: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/78.jpg)
Art of Multiprocessor Programming 78
Producer/Consumer
• Alice and Bob can’t meet– Each has restraining order on other– So he puts food in the pond– And later, she releases the pets
• Avoid– Releasing pets when there’s no food– Putting out food if uneaten food
remains
![Page 79: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/79.jpg)
Art of Multiprocessor Programming 79
Producer/Consumer
• Need a mechanism so that– Bob lets Alice know when food has
been put out– Alice lets Bob know when to put out
more food
![Page 80: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/80.jpg)
Art of Multiprocessor Programming
80
Surprise Solution
A B
cola
![Page 81: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/81.jpg)
Art of Multiprocessor Programming
81
Bob puts food in Pond
A B
cola
![Page 82: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/82.jpg)
Art of Multiprocessor Programming
82
Bob knocks over Can
A B
cola
![Page 83: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/83.jpg)
Art of Multiprocessor Programming
83
Alice Releases Pets
A B
cola
yum… Byum…
![Page 84: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/84.jpg)
Art of Multiprocessor Programming
84
Alice Resets Can when Pets are Fed
A B
cola
![Page 85: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/85.jpg)
Art of Multiprocessor Programming
85
Pseudocode
while (true) { while (can.isUp()){}; pet.release(); pet.recapture(); can.reset();}
Alice’s code
![Page 86: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/86.jpg)
Art of Multiprocessor Programming
86
Pseudocode
while (true) { while (can.isUp()){}; pet.release(); pet.recapture(); can.reset();}
Alice’s code
while (true) { while (can.isDown()){}; pond.stockWithFood(); can.knockOver();}
Bob’s code
![Page 87: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/87.jpg)
Art of Multiprocessor Programming 87
Correctness
• Mutual Exclusion– Pets and Bob never together in pond
![Page 88: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/88.jpg)
Art of Multiprocessor Programming 88
Correctness• Mutual Exclusion
– Pets and Bob never together in pond
• No Starvationif Bob always willing to feed, and pets
always famished, then pets eat infinitely often.
![Page 89: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/89.jpg)
Art of Multiprocessor Programming 89
Correctness• Mutual Exclusion
– Pets and Bob never together in pond• No Starvation
if Bob always willing to feed, and pets always famished, then pets eat infinitely often.
• Producer/ConsumerThe pets never enter pond unless there
is food, and Bob never provides food if there is unconsumed food.
safety
liveness
safety
![Page 90: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/90.jpg)
Art of Multiprocessor Programming
90
Could Also Solve Using Flags
A B
![Page 91: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/91.jpg)
Art of Multiprocessor Programming 91
Waiting
• Both solutions use waiting– while(mumble){}
• Waiting is problematic– If one participant is delayed– So is everyone else– But delays are common &
unpredictable
![Page 92: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/92.jpg)
Art of Multiprocessor Programming 92
The Fable drags on …
• Bob and Alice still have issues
![Page 93: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/93.jpg)
Art of Multiprocessor Programming 93
The Fable drags on …
• Bob and Alice still have issues• So they need to communicate
![Page 94: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/94.jpg)
Art of Multiprocessor Programming 94
The Fable drags on …
• Bob and Alice still have issues• So they need to communicate• So they agree to use billboards …
![Page 95: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/95.jpg)
Art of Multiprocessor Programming
95
E1
D2C
3
Billboards are Large
B3A
1
LetterTiles
From Scrabble™ box
![Page 96: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/96.jpg)
Art of Multiprocessor Programming
96
E1
D2C
3
Write One Letter at a Time …
B3A
1
W4A
1S
1
H4
![Page 97: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/97.jpg)
Art of Multiprocessor Programming
97
To post a message
W4A
1S
1H
4A
1C
3R
1T
1H
4E
1
whew
![Page 98: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/98.jpg)
Art of Multiprocessor Programming
98
S1
Let’s send another message
S1E
1L
1L
1L
1V
4
L1 A
1
M3
A1
A1
P3
![Page 99: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/99.jpg)
Art of Multiprocessor Programming
99
Uh-Oh
A1
C3
R1
T1H
4E
1S
1E
1L
1L
1
L1
OK
![Page 100: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/100.jpg)
Art of Multiprocessor Programming 100
Readers/Writers
• Devise a protocol so that– Writer writes one letter at a time– Reader reads one letter at a time– Reader sees
• Old message or new message• No mixed messages
![Page 101: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/101.jpg)
Art of Multiprocessor Programming 101
Readers/Writers (continued)
• Easy with mutual exclusion• But mutual exclusion requires
waiting– One waits for the other– Everyone executes sequentially
• Remarkably– We can solve R/W without mutual
exclusion
![Page 102: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/102.jpg)
Art of Multiprocessor Programming 102
Why do we care?
• We want as much of the code as possible to execute concurrently (in parallel)
• A larger sequential part implies reduced performance
• Amdahl’s law: this relation is not linear…
![Page 103: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/103.jpg)
Art of Multiprocessor Programming 103
Amdahl’s Law
OldExecutionTimeNewExecutionTimeSpeedup=
…of computation given n CPUs instead of 1
![Page 104: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/104.jpg)
Art of Multiprocessor Programming 104
Amdahl’s Law
− + pp
n
1
1Speedup=
![Page 105: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/105.jpg)
Art of Multiprocessor Programming 105
Amdahl’s Law
− + pp
n
1
1Speedup=
Parallel fraction
![Page 106: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/106.jpg)
Art of Multiprocessor Programming 106
Amdahl’s Law
− + pp
n
1
1Speedup=
Parallel fraction
Sequential fraction
![Page 107: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/107.jpg)
Art of Multiprocessor Programming 107
Amdahl’s Law
− + pp
n
1
1Speedup=
Parallel fraction
Number of processors
Sequential fraction
![Page 108: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/108.jpg)
Art of Multiprocessor Programming
108
Example
• Ten processors• 60% concurrent, 40% sequential• How close to 10-fold speedup?
![Page 109: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/109.jpg)
Art of Multiprocessor Programming
109
Example
• Ten processors• 60% concurrent, 40% sequential• How close to 10-fold speedup?
106.0
6.01
1
+−Speedup=2.17=
![Page 110: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/110.jpg)
Art of Multiprocessor Programming
110
Example
• Ten processors• 80% concurrent, 20% sequential• How close to 10-fold speedup?
![Page 111: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/111.jpg)
Art of Multiprocessor Programming
111
Example
• Ten processors• 80% concurrent, 20% sequential• How close to 10-fold speedup?
108.0
8.01
1
+−Speedup=3.57=
![Page 112: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/112.jpg)
Art of Multiprocessor Programming
112
Example
• Ten processors• 90% concurrent, 10% sequential• How close to 10-fold speedup?
![Page 113: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/113.jpg)
Art of Multiprocessor Programming
113
Example
• Ten processors• 90% concurrent, 10% sequential• How close to 10-fold speedup?
109.0
9.01
1
+−Speedup=5.26=
![Page 114: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/114.jpg)
Art of Multiprocessor Programming
114
Example
• Ten processors• 99% concurrent, 01% sequential• How close to 10-fold speedup?
![Page 115: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/115.jpg)
Art of Multiprocessor Programming
115
Example
• Ten processors• 99% concurrent, 01% sequential• How close to 10-fold speedup?
1099.0
99.01
1
+−Speedup=9.17=
![Page 116: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/116.jpg)
Art of Multiprocessor Programming 116
The Moral
• Making good use of our multiple processors (cores) means
• Finding ways to effectively parallelize our code– Minimize sequential parts– Reduce idle time in which threads
wait without
![Page 117: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/117.jpg)
Art of Multiprocessor Programming 117
Multicore Programming
• This is what this course is about… – The % that is not easy to make
concurrent yet may have a large impact on overall speedup
• Next week: – A more serious look at mutual
exclusion
![Page 118: Art of Multiprocesor Programming · Introduction Companion slides for The Art of Multiprocessor Programming ... twice as much concurrency. Art of Multiprocessor Programming 8 Traditional](https://reader030.vdocument.in/reader030/viewer/2022041107/5f097e257e708231d4271604/html5/thumbnails/118.jpg)
Art of Multiprocessor Programming
118
This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License.
• You are free:– to Share — to copy, distribute and transmit the work – to Remix — to adapt the work
• Under the following conditions:– Attribution. You must attribute the work to “The Art of
Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work).
– Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license.
• For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to– http://creativecommons.org/licenses/by-sa/3.0/.
• Any of the above conditions can be waived if you get permission from the copyright holder.
• Nothing in this license impairs or restricts the author's moral rights.