event stream processing with multiple threads
TRANSCRIPT
MULTIPLEMULTIPLEMULTIPLEMU
THREADSTHREADSTHREADSTHRMULTIPLETHREADS
EVENT STREAM PROCESSING
with
Sylvain HalléRaphael KhourySébastien Gaboury
Université du Québec a Chicoutimi
Find ways to split P into parts thatcan be processed in parallel
1
output event streamP
computation
input event stream
The Problem
genericFind ways to split P into parts thatcan be processed in parallel
1
output event streamP
computation
input event stream
The Problem
Input events arriveone by one2
genericFind ways to split P into parts thatcan be processed in parallel
1
output event streamP
computation
input event stream
The Problem
Input events arriveone by one2
generic
Output events areproduced one by one3
Find ways to split P into parts thatcan be processed in parallel
1
output event streamP
computation
input event stream
The Problem
→
→
→
+
→
Fork f = new Fork(2);FunctionProcessor sum =new FunctionProcessor(Addition instance);
f
The System
→
→
→
+
→
Fork f = new Fork(2);FunctionProcessor sum =new FunctionProcessor(Addition.instance);
CountDecimate decimate = new CountDecimate(n);
f
n
The System
→
→
→
+
→→
Fork f = new Fork(2);FunctionProcessor sum =new FunctionProcessor(Addition.instance);
CountDecimate decimate = new CountDecimate(n);Connector.connect(fork, LEFT, sum, LEFT)
f
n
The System
→
→
→
+
→→
→
Fork f = new Fork(2);FunctionProcessor sum =new FunctionProcessor(Addition.instance);
CountDecimate decimate = new CountDecimate(n);Connector.connect(fork, LEFT, sum, LEFT). connect(fork, RIGHT, decimate, INPUT)
f
n
The System
→
→
→
+
→→
→
Fork f = new Fork(2);FunctionProcessor sum =new FunctionProcessor(Addition.instance);
CountDecimate decimate = new CountDecimate(n);Connector.connect(fork, LEFT, sum, LEFT). connect(fork, RIGHT, decimate, INPUT). connect(decimate, OUTPUT, sum, RIGHT);
f
n
The System
→
→
→
+
→→
→
Fork f = new Fork(2);FunctionProcessor sum =new FunctionProcessor(Addition.instance);
CountDecimate decimate = new CountDecimate(n);Connector.connect(fork, LEFT, sum, LEFT). connect(fork, RIGHT, decimate, INPUT). connect(decimate, OUTPUT, sum, RIGHT);Pullable p = sum.getOutputPullable(OUTPUT);while (p.hasNext() != NextStatus.NO) {Object o = p.next();
. ..}
f
n
The System
The System
FunctionApplies a function toevery input event
CumulativeComputes theprogressive "sum" ofall input events
TrimRemoves the first ninput events
DecimateOutputs everyn-th event
GroupEncloses a group ofconnected processors
ForkDuplicates a streaminto multiple copies
SliceSplits a stram into multiplesub-streams
WindowApplies a function toa sliding window ofn events
f
Σ.f
n
FilterA first, Boolean stream, decides if each event of a second stream should be output
{f
Existing processors should be changed !4 not
P P PPPP PPP
Enclose them within new, processors
Solution:
thread-aware
P
The Problem
Runnable r = ...ManagedThread t = m.tryNewThread(r);if (t != null) t.start(); else r.run();
Returns a new thread that can be started, or null ifall N threads are busy
If t is null, the desired processing must be donein the current thread
Thread Manager
Responsible for a poolof N threads M
ψ
φ.pull
φ W
φ.pull
e1
e2
φ.pulle3
W.pull
e1
W.pull
e2
W.pull
e3
ψ.pull
ψ.pull
ψ.pull
Pre-emptive Pulling
TIM
E
ψ
φ.pull
φ W
φ.pull
e1
e2
φ.pulle3
W.pull
e1
W.pull
e2
W.pull
e3
ψ.pull
ψ.pull
ψ.pull
Pre-emptive Pulling
TIM
E
ψ
push(e1)
φ.pull
φ2φ1
φe1e2e3
S
φ.pull
push(e2)
e'1
e'1
e'2
e'2
φ2
push(e3)
e'3
Pull Pipeline
TIM
E
ψ
push(e1)
φ.pull
φ2φ1
φe1e2e3
S
φ.pull
push(e2)
e'1
e'1
e'2
e'2
φ2
push(e3)
e'3
e'3φ.pull
Pull Pipeline
TIM
E
Window ψ
push(e)
φ1.push(e)
φn-1.push(e)
. . .
collect e'shiftcopy
push(e')
φ2.push(e)
Blocking Push
TIM
E
Window ψ
push(e)
φ1.push(e)
φn-1.push(e)
. . .
collect e'shiftcopy
push(e')
φ2.push(e)Event e is pushed to the n copiesof φ sequentially. Each call topush is blocking.
Blocking Push
TIM
E
Window ψ
push(e) φ0.push(e)φn-1.push(e)
φ1.push(e)
φ0
φ0.waitFor
φ1 φn-1. . .
Non-blocking Push
TIM
E
Window ψ
push(e) φ0.push(e)φn-1.push(e)
φ1.push(e)
φ0
φ0.waitForφn-1.waitFor
φ1 φn-1. . .
Non-blocking Push
TIM
E
Window ψ
push(e) φ0.push(e)φn-1.push(e)
collect e'shiftcopy
φ1.push(e)
φ0
φ0.waitForφn-1.waitFor
φ1 φn-1. . .
Non-blocking Push
TIM
E
Window ψ
push(e) φ0.push(e)φn-1.push(e)
collect e'shiftcopy push(e')
φ1.push(e)
φ0
φ0.waitForφn-1.waitFor
φ1 φn-1. . .
Non-blocking Push
TIM
E
Window ψ
push(e) φ0.push(e)φn-1.push(e)
collect e'shiftcopy push(e')
φ1.push(e)
φ0
φ0.waitForφn-1.waitFor
φ1 φn-1. . .
Calls to push are non-blocking;each runs in a separate thread.
Non-blocking Push
TIM
E
3 new thread-aware processors
Non-blockingpush
Pullpipeline
Pre-emptivepull
Implemented in a separateindependent from the rest of the code
palette
→
→ NP → PP → PE
Multi-threading in a Query
→
→ {
f
→→
+
Σ.f
10
FunctionProcessor sum = new FunctionProcessor( new CumulativeFunction(Addition.instance));
WindowProcessor win = new WindowProcessor(sum, 10);
ThreadManager m = new ThreadManager(4);FunctionProcessor sum = new FunctionProcessor( new CumulativeFunction(Addition.instance));NonBlockingProcessor nbp = new NonBlockingProcessor(sum, m);WindowProcessor win = new WindowProcessor(nbp, 10);
→
→ {
f 10
Multi-threading in a Query
M→
→ →
→
+
Σ.f4
Experimental Results
Auction biddingCandidate selectionEndless bashingSpontaneous Pingu creationTurn around
1.074.441.451.051.86
Benchmark (CRV 2016) Speedup
A single thread-aware processor inserted in the query
Experimental Results
Turn around
→
A/a/b//character[status=Walker]/id/text()
→ p1
→
A
→ p2
→ →→
→
/a/b
//character[status=Blocker]/id/text()
→
→→
3
→
→
→
<?
→
→
→
→
→
→
→
→
f1
f2
→
→ →
→
→
→
→
M4
Multi-threaded
Experimental Results
Turn around
→
A/a/b//character[status=Walker]/id/text()
→ p1
→
A
→ p2
→ →→
→
/a/b
//character[status=Blocker]/id/text()
→
→→
3
→
→
→
<?
→
→
→
→
→
→
→
→
f1
f2
→
→ →
→
→
→
→
Take-home Points
Multi-threading capabilities added to BeepBeep 3
Existing processors do not need to be aware of the presence of threads
Minimally intrusive: less than 5 lines to add multi-threading to part of a query
5% - 400% speedup on a small sample of queries
Requires manual tuning and intuition
Not all queries can gain from parallelism!