thread blocking is evil
DESCRIPTION
This is the simple version of the orinial one addressed at Deview2011.TRANSCRIPT
Thread Blocking Is Evil
Game Network Component Team
Kim, J.H.
CONCURRENCY
Why is Thread Blocking
a Concern?
“The Free Lunch is OVER.”
Herb SutterDecember 2004
Clock speedExecution optimizationCache
Cache
HyperthreadingMulticoreCache
Write Multithreaded Application
CONCURRENCY
Waiting in Line
Thread Blocking
Starvation
Deadlock
Starvation
Task A Task B
1: Lock 1: Sleep 0 msec
2: Sleep 1msec
3: Unlock
Pro
cess
ti
me(m
s)
# of Thread
1 2 3 4 5 6 7 8 9984986988990992994996998
1000
Task BTask A
DeadlockHold & Wait
Thread A Thread B
1A: Lock 1B: Lock
2A: Request 2B: Send Response
3A: Wait for Response 3B: Unlock
4A: Do something with Response
5A: Unlock
Thread A Thread B
1A: Lock 1B: Lock
2A: Request 2B: Send Response
3A: Wait for Response 3B: Unlock
4A: Do something with Response
5A: Unlock
My Prob-lem
Framework for Lobby-Room style
Game
Remote Procedure Call
Easy to write code be-cause it’s all contained
inside one function.
Game Client
LobbyRoom‘Enter Room’
RPC‘Authentication’
RPC
Game Client
LobbyRoom
‘Enter Room’
RPC func-tion body
‘Authenti-cation RPC
‘Function
body
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
Game Client
LobbyRoom
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
‘Enter Room’
RPC func-tion body
‘Authenti-cation RPC
‘Function
body
Game Client
LobbyRoom
‘Enter Room’ RPC
function body
‘Authenti-cation RPC
‘Function
body
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
Response DelayFrom Lobby Server
Game Client
LobbyRoom
‘Enter Room’ RPC
function body
‘Authenti-cation RPC
‘Function
body
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
Room Serverthroughput Decline
Response DelayFrom Lobby Server
What is theAxis of Evil?
Thread Block-
ing
Game Client
LobbyRoom
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
Inherent Limitation of RPC
Easy to write code be-cause it’s all contained
inside one function.
Can not split code be-cause it’s all contained
inside one function.
RPC::Result* JoinRoom(…){ … RPC::Result answer = RPC::SyncCall(“VerifyLobbyUserToken”, …); …. return result;}
What We Want
RPC Function Body
Non-Blocking
What is the Essence of Problems?
Request & Wait
Solution
Asynchronous
Programming
Request(callback, …)
Pros:Non-Blocking
Cons:Can’t Write Sequential CodeCan’t Use Stack VariableCan’t Split Programming Construct…
Too Difficult!!
Coroutine
Similar to Thread
Line of Execution
Stack&
Local Variable
But
Non-preemptive
Instruction:YieldYield BreakResume
Who does provide Coroutine?
C#ErlangHaskell
JavaScript(since 1.7)LuaPerl
Python(since 2.5)Ruby
…
What about C++?
On Your Own!
Windows: Fiber
Linux: getcontext/setcontextmakecontextswapcontext
Asynchronous Pro-gramming
+Coroutine
Non-Blocking&
Writing Sequential Code
Solution for Starva-
tion
Task A Task B
1: Lock 1: Sleep 0msec
2: Sleep 2msec
3: Unlock
Before
Task A Task B
Thread A 1: Sleep 0msec
1: Create & Resume Coroutine
2: Request Lock
3: Yield
Thread A’
4: Resume
5: Sleep 2msec
6: Yield Break
7: Unlock
Non-Blocking
Sequential Code
After
BeforePro
cess
ti
me(m
s)
# of Thread
1 2 3 4 5 6 7 8 9984986988990992994996998
1000
Task BTask A
AfterPro
cess
ti
me(m
s)
1 2 3 4 5 6 7 8 90
200
400
600
800
1000
1200
Task BTask A
# of Thread
Solution for
DeadlockHold & Wait
Thread A Thread B
1A: Lock 1B: Lock
2A: Request 2B: Send Response
3A: Wait for Response 3B: Unlock
4A: Do something with Response
5A: Unlock
Before
Thread A Thread B
1A: Lock 1B: Lock
2A: Create & Resume Coroutine 2B: Send Response
3A: Request 3B: Unlock
4A: Yield
5A: Unlock
Thread A’ (invoked by response)
6A’: Lock
7A’: Resume
8A’: Do something with response
9A’: Yield Break
10A’: Unlock
Non-Blocking
Sequential Code
After
Solutionfor
My Problem
Thread Block-
ing
Game Client
LobbyRoom
Call ‘Enter Room’ RPC
Call ‘Authentica-tion’ RPC
Return ‘Authentication’ RPC
Return ‘Enter Room’ RPC
Room Thread Lobby Service
1R: Call ‘Authentication’ RPC function 1L: Process the request
2R: Wait for the response 2L: Send back the response
3R: Process
Before
Room Thread Lobby Service
1R: Create & Resume Coroutine 1L: Process the request
2R: Request 2L: Send back the response
3R: Yield
Room Thread’(invoked by response)
4R’: Resume
5R’: Process
6R’: Yield Break
Non-Blocking
Sequential Code
After
Request & Wait
Request & Yield
Non-Blocking&
Writing Sequential Code
Coroutine(Fiber)
Thread
CPU CPU CPU CPU
User-mode SchedulingKernel-mode Scheduling
Corou-tine Pool
Network I/O Thread
Pool
RPC Task Queue
RPC Exe-cution
Thread Pool
RPC Task Queue
RPC Execu-tion Thread
Network I/O Thread
enqueuetask
dequeuetask
acquire corou-tine
enqueue
re-sumetask
dequeue
re-sumetask
resume
yield break
release corou-tine
Network Event
Network Event
Coroutine Pool
pool-ing
corou-tine
RPC Execu-tion
yield return
start
RPC Task Queue
RPC Execu-tion Thread
1
Coroutine Pool
RPC Execu-tion
enqueuetask
dequeuetask
acquire corou-tine corou-
tine
yield return
enqueue
re-sumetask
dequeue
re-sumetask
release corou-tine
Network Event
Network Event
pool-ing
RPC Execu-tion Thread
2
resume
yield break
resume
Network I/O Thread
RPC::Result* JoinRoom(…){ … RPC::Result answer = RPC::SyncCall(“VerifyLobbyUserToken”, …); …. return result;}
RPC::Result* JoinRoom(…){ … RPC::Result answer = RPC::SyncCallYield(“VerifyLobbyUserToken”, …); …. return result;}
TPS
# of Thread
Blocking
Yield
Wrap-up
“The free lunch is OVER.”Write Multithreaded
Application
CONCURRENCY
Thread Blocking is Evil
Starvation Deadlock
Asynchronous Programming
+Coroutine
Non-Blocking+
Writing Sequen-tial Code
Request & Wait
Request & Yield
Inspired by Jeffrey Richter
‘Simplified APM With The Asyn-cEnumerator’
Pictures fromwww.Istockphoto.comwww.Flickr.com