concurrency patterns
DESCRIPTION
TRANSCRIPT
![Page 1: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/1.jpg)
Author: Ryan Stout
ConcurrencyAn overview of concurrent processing models
Date: June 22nd, 2013
Wednesday, July 10, 13
![Page 2: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/2.jpg)
Wednesday, July 10, 13
![Page 3: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/3.jpg)
single core performance isn’t increasing anymore
Concurrency Models
No More Free Lunch
Wednesday, July 10, 13
![Page 4: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/4.jpg)
single core performance isn’t increasing anymore
Concurrency Models
No More Free Lunch
Wednesday, July 10, 13
![Page 5: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/5.jpg)
Concurrency Models
Question: How do we take advantage of the extra cores?
Wednesday, July 10, 13
![Page 6: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/6.jpg)
Concurrency Models
Wednesday, July 10, 13
![Page 7: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/7.jpg)
Concurrency Models
Answer: Concurrency & Parallelism
A
Concurrent Parallel
B
Sequential
A
A
B
B
A B
Wednesday, July 10, 13
![Page 8: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/8.jpg)
• Only difficult when sharing state
• Hard to reason about
• Race conditions
• Deadlock
Concurrency Models
Problem: Parallel can be difficult
Wednesday, July 10, 13
![Page 9: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/9.jpg)
1. Processes
2. Threads and Locks
3. Evented-io
4. Vectorization/SIMD
5. Actor Model - Erlang, Celluloid, Akka
6. CSP - Google Go
7. Software Transactional Memory
Different way to do things at the same time
Concurrency Models
Solution (sort of): Better Concurrency Models
Wednesday, July 10, 13
![Page 10: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/10.jpg)
separate memory model
Concurrency Models
1. Processes
Wednesday, July 10, 13
![Page 11: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/11.jpg)
separate memory model
Concurrency Models
1. Processes
Wednesday, July 10, 13
![Page 12: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/12.jpg)
• Pros:
• Easy
• OS handles everything
• Good for stateless operations (Responding to web requests)
• No Memory Leaks
• Cons:
• Communication between processes is difficult
• Ram gets used for each process
The way unix intended
Concurrency Models
1. Processes Process A
ThreadA
Process B
Thread B
Wednesday, July 10, 13
![Page 13: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/13.jpg)
Concurrency Models
2. Threads
Wednesday, July 10, 13
![Page 14: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/14.jpg)
Concurrency Models
2. Threads
Process
Thread A
Thread B
Wednesday, July 10, 13
![Page 15: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/15.jpg)
require 'thread'
class ThreadExample def initialize @count = 0 @count_mutex = Mutex.new
# Start a few readers threads = [] 3.times do threads << Thread.new do read_values end end
threads.each(&:join) end
...
Concurrency Models
2. Threaded/Mutex Example... def read_values loop do @count_mutex.synchronize do local_count = @count
# Get the "value" local_count += rand(10) @count = local_count
puts "Value: #{local_count}" end end endend
ThreadExample.new
Wednesday, July 10, 13
![Page 16: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/16.jpg)
Shared Memory Model
• Pros:
• Less ram than processes
• Cons:
• Manually handling locks
• Deadlocks
• Non-determanistic
• Hard to Debug
the way Java intended
Concurrency Models
2. Threads
Wednesday, July 10, 13
![Page 17: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/17.jpg)
Concurrency Models
3. Evented-ioProcessReactor
A
B
A
Wednesday, July 10, 13
![Page 18: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/18.jpg)
require 'socket'
server = TCPServer.open(host, port)
while client = server.accept Thread.new do line = client.gets client.puts(line) client.close endend
Concurrency Models
Threaded Echo Server
Wednesday, July 10, 13
![Page 19: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/19.jpg)
require 'eventmachine'
class Echo < EventMachine::Connection def receive_data(data) send_data(data) endend
EventMachine.run { EventMachine.connect '127.0.0.1', 8081, Echo}
Concurrency Models
Evented-io - Echo Server
Wednesday, July 10, 13
![Page 20: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/20.jpg)
module ProxyConnection def initialize(local_connection) @local_connection = local_connection end
def receive_data(data) @local_connection.send_data(data) endend
module ProxyServer def post_init # Make a connection to the remote server @connection = EventMachine.connect 'www.zeebly.com', 80, ProxyConnection, self end
def receive_data(data) @connection.send_data(data) endend
EventMachine::run do EventMachine::start_server "127.0.0.1", 8080, ProxyServerend
Concurrency ModelsEvented-io - Proxy Server
Wednesday, July 10, 13
![Page 21: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/21.jpg)
A form of cooperative multitasking
• Pros:
• Great for heavy IO work
• Saves thread switching cost
• Good OS level support (epoll, kqueue)
• Cons:
• Long running operations block all other operations
• Does not use multiple cores
• Somewhat difficult to reason about
Libraries: EventMachine (ruby), Tornado (python), nodejs, libev (C)
the way nginx intended - concurrent, not parallel
Concurrency Models
3. Evented-io ProcessReactor
A
B
A
Wednesday, July 10, 13
![Page 22: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/22.jpg)
Concurrency Models
4. Vectorization/SIMDProcess
Data A
Data B0.51 0.98 1.5 0.29 0.75 0.11
0.39 0.19 1.22 1.6 0.84 0.90
Instruction: Multiply(A,B)
Wednesday, July 10, 13
![Page 23: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/23.jpg)
void MatrixMulOnHost(float* M, float* N, float* P, int Width) { for (int i = 0; i < Width; ++i) { for (int j = 0; j < Width; ++j) { float sum = 0; for (int k = 0; k < Width; ++k) { float a = M[i * Width + k]; float b = N[k * Width + j]; sum += a * b; } P[i * Width + j] = sum; } }}
Concurrency Models
Example Matrix Multiplication - C
Wednesday, July 10, 13
![Page 24: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/24.jpg)
__global__ void MatrixMulKernel(float* d_M, float* d_N, float* d_P, int Width) { int row = threadIdx.y; int col = threadIdx.x; float P_val = 0; for (int k = 0; k < Width; ++k) { float M_elem = d_M[row * Width + k]; float N_elem = d_N[k * Width + col]; P_val += M_elem * N_elem; } d_p[row*Widthh+col] = P_val;}
Concurrency Models
Example Matrix Multiplication - CUDA
Wednesday, July 10, 13
![Page 25: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/25.jpg)
SIMD = Single Instruction Multiple Data
• Pros:
• Very fast for some operations (~50x speed imporvement in some
cases)
• Cons:
• Only works when working with vector data
• No standard way to program (CUDA, OpenCL, SSE, BLAS)
• Time to copy data between CPU and GPU adds up
the way Nvidia intended
Concurrency Models
4. Vectorization/SIMD
Wednesday, July 10, 13
![Page 26: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/26.jpg)
“Don’t communicate by sharing state; share
state by communicating”
http://www.igvita.com/2010/12/02/concurrency-with-actors-goroutines-ruby/
Concurrency Models
5. & 6. Actor Model & CSP
Wednesday, July 10, 13
![Page 27: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/27.jpg)
• State is encapsulated in “Actors”
• Actors send messages to communicate
• Messages are asynchronous
• Messages buffer in a “mailbox”
Libraries: Akka (scala), Celluloid (ruby), Erlang (built in)
the way Erlang intended
Concurrency Models
5. Actor Model
Wednesday, July 10, 13
![Page 28: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/28.jpg)
the way Erlang intended
Concurrency Models
5. Actor Model
Process
Actor A Actor B
Actor C
Wednesday, July 10, 13
![Page 29: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/29.jpg)
require 'actor'ping = nil
ping = Actor.spawn do loop do msg = Actor.receive puts "Ping #{msg}" pong << (msg + 1) endend
...
Concurrency Models
Actor Example
...
pong = Actor.spawn do loop do msg = Actor.receive puts "Ping #{msg}" break if msg > 10000 ping << (msg + 1) endend
ping << 0
Wednesday, July 10, 13
![Page 30: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/30.jpg)
• Pros:
• No deadlocks (if you follow the rules)
• Fairly easy to reason about
• Cons:
• Connecting actors is difficult
• Error handling is tricky
the way Erlang intended
Concurrency Models
5. Actor Model
Wednesday, July 10, 13
![Page 31: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/31.jpg)
Concurrency Models
Process
A
C
6. Communicating Sequential Processes
B
Channel A
Wednesday, July 10, 13
![Page 32: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/32.jpg)
• Google Go’s Model
• Variables are always thread local
• State is shared by sized channels
• Functions can be run as a “go routine”
• Anonymous thread of execution
• Can not be referenced or joined
• Communication takes place over channels
• Channels are sized synchronized buffers, which can be “closed”
the way google intended
Concurrency Models
6. Communicating Sequential Processes
Wednesday, July 10, 13
![Page 33: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/33.jpg)
• Pros:
• Easy to reason about
• Simple model for distributing data processing work
• Simple to implement by hand (sized queue’s)
• Cons:
• Can be extra code for simple synchrnoization
Libraries: Google Go (language), Agent (ruby)
the way google intended
Concurrency Models
6. Communicating Sequential Processes
Wednesday, July 10, 13
![Page 34: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/34.jpg)
• In Actor we name the Actors
• In CSP we name the communication channels
• Both allow us to keep out non-determinism (if we follow
the rules)
Concurrency Models
Difference between CSP and Actor?
Wednesday, July 10, 13
![Page 35: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/35.jpg)
Concurrency Models
7. Software Transactional Memory
Wednesday, July 10, 13
![Page 36: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/36.jpg)
Concurrency Models
Process
MemoryVariable A
7. Software Transactional Memory
Block Bread
process
transactional write
Block Aread
process
transactional write
Wednesday, July 10, 13
![Page 37: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/37.jpg)
Concurrency Models
Process
MemoryVariable A
7. Software Transactional Memory
Block Bread
process
transactional write
Block Aread
process
transactional write
Wednesday, July 10, 13
![Page 38: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/38.jpg)
Concurrency Models
Process
MemoryVariable A
7. Software Transactional Memory
Block Bread
process
transactional write
Block Aread
process
transactional write
Wednesday, July 10, 13
![Page 39: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/39.jpg)
Concurrency Models
Process
MemoryVariable A
7. Software Transactional Memory
Block Bread
process
transactional write
Block Aread
process
transactional write
Wednesday, July 10, 13
![Page 40: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/40.jpg)
Concurrency Models
Process
MemoryVariable A
7. Software Transactional Memory
Block Bread
process
transactional write
Block Aread
process
transactional write
Wednesday, July 10, 13
![Page 41: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/41.jpg)
Concurrency Models
Process
MemoryVariable A
7. Software Transactional Memory
Block Bread
process
transactional write
Block Aread
process
transactional write
SUCCESS
Wednesday, July 10, 13
![Page 42: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/42.jpg)
Concurrency Models
Process
MemoryVariable A
7. Software Transactional Memory
Block Bread
process
transactional write
Block Aread
process
transactional write
SUCCESS FAIL
Wednesday, July 10, 13
![Page 43: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/43.jpg)
Concurrency Models
Process
MemoryVariable A
7. Software Transactional Memory
Block Bread
process
transactional write
Block Aread
process
transactional write
SUCCESS FAIL
Wednesday, July 10, 13
![Page 44: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/44.jpg)
Concurrency Models
7. Software Transactional Memory
Wednesday, July 10, 13
![Page 45: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/45.jpg)
Concurrency Models
Process
MemoryVariable A
7. Software Transactional Memory
Block Bread
process
transactional write
Wednesday, July 10, 13
![Page 46: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/46.jpg)
Concurrency Models
Process
MemoryVariable A
7. Software Transactional Memory
Block Bread
process
transactional write
Wednesday, July 10, 13
![Page 47: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/47.jpg)
Concurrency Models
Process
MemoryVariable A
7. Software Transactional Memory
Block Bread
process
transactional write
Wednesday, July 10, 13
![Page 48: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/48.jpg)
Concurrency Models
Process
MemoryVariable A
7. Software Transactional Memory
Block Bread
process
transactional write
SUCCESS
Wednesday, July 10, 13
![Page 49: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/49.jpg)
• Pros:
• Easy to reason about
• Cons:
• More resource contention = worse performance
• In order to prevent contention, we must use immutable data
structures -- which makes it more difficult to reason about again
the way Clojure intended
Concurrency Models
7. Software Transactional Memory
Wednesday, July 10, 13
![Page 50: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/50.jpg)
• Functional Reactive Programming
Concurrency Models
8. Bonus Model
Wednesday, July 10, 13
![Page 51: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/51.jpg)
Concurrency Models
Which Model is Best?
Wednesday, July 10, 13
![Page 52: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/52.jpg)
• Depends on the problem
Concurrency Models
Which Model is Best?
Wednesday, July 10, 13
![Page 53: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/53.jpg)
• Depends on the problem
• Depends on the language
Concurrency Models
Which Model is Best?
Wednesday, July 10, 13
![Page 54: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/54.jpg)
• http://www.igvita.com/2010/12/02/concurrency-with-actors-goroutines-ruby/
• http://en.wikipedia.org/wiki/Dining_philosophers_problem
• Dining Philosophers Problem - http://rosettacode.org/wiki/Dining_philosophers
Concurrency Models
More Info
Wednesday, July 10, 13
![Page 55: Concurrency Patterns](https://reader035.vdocument.in/reader035/viewer/2022081800/54813726b4af9fa0158b5f0e/html5/thumbnails/55.jpg)
Author: Ryan Stout
Thanks!any questions?
Date: June 22nd, 2013
Wednesday, July 10, 13