![Page 1: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/1.jpg)
Event Processing at Massive Scale Approaches to Concurrency
Uri Cohen @uri1803 | github.com/uric
Head of Product @ GigaSpaces
![Page 2: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/2.jpg)
This is What It Used to Be Like
![Page 3: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/3.jpg)
That’s What
It’s Like Now
![Page 4: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/4.jpg)
Some Numbers
15 Billion Trades / Day on
NYSE alone
http://www.nytimes.com/2011/08/27/business/as-trade-volumes-soar-exchanges-cash-in.html
![Page 5: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/5.jpg)
Some Numbers
That’s
641K Trades / Second
http://www.nytimes.com/2011/08/27/business/as-trade-volumes-soar-exchanges-cash-in.html
![Page 6: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/6.jpg)
Some Numbers
12 Billion Shares change
hands every day
http://www.bloomberg.com/news/2012-01-23/stock-trading-is-lowest-in-u-s-since-2008.html
![Page 7: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/7.jpg)
Some Numbers
$4 Million The cost of 1
millisecond of latency to a broker
http://www.tabbgroup.com/PublicationDetail.aspx?PublicationID=346
![Page 8: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/8.jpg)
The Problem
Massive stream of events
Time is money, literally
Can’t lose a single message
Fairness is a must
![Page 9: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/9.jpg)
Order Book -
Simplistic Example
Buy Sell 50, $12 60, $10
60, $11 100, $11
30, $10 30, $12
![Page 10: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/10.jpg)
Buy Sell 50, $12 60, $10
60, $11 100, $11
30, $10 30, $12
Price: $10
Order Book -
Simplistic Example
![Page 11: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/11.jpg)
Buy Sell 60, $11 10, $10
30, $10 100, $11
30, $12
Price: $10
Order Book -
Simplistic Example
![Page 12: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/12.jpg)
Buy Sell 60, $11 10, $10
30, $10 100, $11
30, $12
Price: $10
Order Book -
Simplistic Example
![Page 13: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/13.jpg)
Buy Sell 50, $11 100, $11
30, $10 30, $12
Price: $10
Order Book -
Simplistic Example
![Page 14: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/14.jpg)
Buy Sell 50, $11 100, $11
30, $10 30, $12
Price: $11
Order Book -
Simplistic Example
![Page 15: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/15.jpg)
Buy Sell 30, $10 50, $11
30, $12
Price: $11
Order Book -
Simplistic Example
![Page 16: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/16.jpg)
What it Really Means
Low latencyIn memory, GC tuning
Scalability Multi-coreMulti-node
OrderingBy price, order time
Exclusivity
Resiliency
![Page 17: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/17.jpg)
Trading is Just One Use Case
All things FCFS, with a limited stock
Flight booking
Betting
Online Auctions
Cloud Spot Instances
eCommerce
![Page 18: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/18.jpg)
Let’s Talk Solutions
![Page 19: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/19.jpg)
Queue (SEDA/Actor Style)
Not Validated Validated Processed
Validator Processor
![Page 20: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/20.jpg)
Queue (SEDA /Actor Style)
The Good: Ordered (Is it fair?)Multi-threaded
The Bad: Not very scalable
LockingContext
switchingTransient
![Page 21: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/21.jpg)
The Cost of Locking
Method Time in msec
Single Thread 300
Single Thread w/ Lock 10.000
2 Threads w/ Lock 224.000
Single Thread w/ CAS 5.700
2 Threads w/ CAS 30.000
Single Thread w/ Volatile Write 4.700
http://disruptor.googlecode.com/files/Disruptor-1.0.pdf
![Page 22: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/22.jpg)
Queue (Lack of) Fairness
Consumer Thread 1
Consumer Thread 2
60
50 Buy Sell 100
![Page 23: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/23.jpg)
Queue (Lack of) Fairness
Consumer Thread 1
Consumer Thread 2
50
60
Buy Sell 100
![Page 24: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/24.jpg)
Queue (Lack of) Fairness
Consumer Thread 1
Consumer Thread 2
Buy Sell 100
60 50
Can you tell which order will be executed 1st?
![Page 25: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/25.jpg)
Single-Threaded Queue
Validator Processor
![Page 26: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/26.jpg)
Single- Threaded
Queue
The Good: Fast, very fast
No contentionNo context
switchesAlways fair
The Bad: Multi-core?
Not fit for Intense compute & I/O
Need to be async.Transient
![Page 27: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/27.jpg)
Single- Threaded
Queue
They do it…
![Page 28: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/28.jpg)
Disruptor (LMAX)
![Page 29: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/29.jpg)
Segmented Queue
Symbol=A-H Symbol=I-S Symbol=T-Z
Validator Processor Processor
Processor thread pool per segment
![Page 30: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/30.jpg)
Segmented Queue - Optimization Single Processor thread pool, pick random segment
Symbol=A-H Symbol=I-S Symbol=T-Z
Processor
![Page 31: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/31.jpg)
Segmented Queue
The Good: Scalable
But segments can get hot
Minimizes contention
The Bad: Not trivial to
implementStill unfair
Is total ordering needed?
Transient
![Page 32: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/32.jpg)
What about
Fairness?
Exclusivity is Key Process one message for each segment at the same time
No exclusivity across segments
![Page 33: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/33.jpg)
Implicit ExclusivitySingle processor thread per segment
Symbol=A-H Symbol=I-S Symbol=T-Z
Processor Processor Processor
![Page 34: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/34.jpg)
Explicit ExclusivityShared thread pool, mark segments under processing (CAS)
Segment 1 Segment 2 Segment 3
Processor
Segment 1
Segment 2
Segment 3
![Page 35: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/35.jpg)
Explicit ExclusivityShared thread pool, mark segments under processing (CAS)
Segment 1 Segment 2 Segment 3
Processor
Segment 1 X
Segment 2
Segment 3
![Page 36: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/36.jpg)
Explicit ExclusivityShared thread pool, mark segments under processing (CAS)
Segment 1 Segment 2 Segment 3
Processor
Segment 1 X
Segment 2
Segment 3 X
![Page 37: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/37.jpg)
Explicit ExclusivityShared thread pool, mark segments under processing (CAS)
Segment 1 Segment 2 Segment 3
Processor
Segment 1
Segment 2
Segment 3 X
![Page 38: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/38.jpg)
Explicit Exclusivity
Num. of segments is keyToo few: little concurrencyToo many: wasting memory
![Page 39: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/39.jpg)
Dynamic Segmentation Segments are created and removed as needed
Processor
![Page 40: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/40.jpg)
Dynamic Segmentation Segments are created and removed as needed
“GOOG”
Processor
“GOOG”
![Page 41: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/41.jpg)
Dynamic Segmentation Segments are created and removed as needed
“GOOG”
Processor
“GOOG”
![Page 42: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/42.jpg)
Dynamic Segmentation Segments are created and removed as needed
“GOOG”
Processor
GOOG
AAPL
“AAPL”
![Page 43: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/43.jpg)
Dynamic Segmentation Segments are created and removed as needed
“GOOG”
Processor
GOOG X
AAPL
AMZN
“AAPL” “AMZN”
![Page 44: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/44.jpg)
Dynamic Segmentation
Segments created as needed
Randomize on segments until available one found
Fast, scalable, fair
We call it “FIFO groups” or “Virtual Queues”
![Page 45: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/45.jpg)
It Can (and Does) Get Much
More Complex
Memory state can get corrupt on errors
It’s not always as simple as “pop off the queue”
limits, priorities, circuit breakers, etc.
Resiliency is always a pain
![Page 46: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/46.jpg)
A Bit about Usability
What you don’t want to do
Implement data structuresHandle concurrencyHandle HA Handle transactions
![Page 47: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/47.jpg)
A Bit about Usability
What you want to control
Event flow Grouping attribute (e.g
symbol)Event handlers
![Page 48: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/48.jpg)
Data Grid as a
Foundation
Transactional
Highly available
Supports complex matching
![Page 49: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/49.jpg)
How We Thought
of It
![Page 50: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/50.jpg)
How We Thought
of It
![Page 51: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/51.jpg)
How We Thought
of It
![Page 52: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/52.jpg)
How We Thought
of It
![Page 53: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/53.jpg)
How We Thought
of It
![Page 54: Trade and Event Processing at a Massive Scale - QCon NY 2012](https://reader033.vdocument.in/reader033/viewer/2022061218/54b71cb84a795903798b4688/html5/thumbnails/54.jpg)
Thank You!
References: http://martinfowler.com/articles/lmax.htmlhttp://www.nytimes.com/2011/08/27/business/as-trade-volumes-soar-exchanges-cash-in.html http://disruptor.googlecode.com/files/Disruptor-1.0.pdfhttp://www.gigaspaces.com/wiki/display/XAP9/FIFO+Grouping