jax london 2013
DESCRIPTION
Fast Data: Parallel Processing on the Grid. You may think of in-memory datagrids as a big cache, a key value store accessed using put and get. However to really reap the power and scale of modern data grids you need to move beyond cache semantics and turn your architecture on its head. Moving the processing to the data rather than pulling the data across the grid massively increases parallelism and reduces network latency, leading to huge increases in throughput. In this session, we will explore why traditional cache semantics sometimes struggles to scale and how through using parallel processing and event processing on the grid we can rearchitect our systems for massive scalability and utilise all our grid cores for parallel processing.TRANSCRIPT
Steve Millidge | C2B2
Fast Data: Parallel Processing on the Grid
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
SAFE HARBOR
This talk may contain nonsense.
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
WHO AM I
Founder C2B2Java since 1.0 EG Member JSR107 and JSR347 (bit dead)
Our MissionWe ensure the middleware that forms the foundation of our customer’s mission critical systems is Fast, Reliable, Manageable and Secure.
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Data PerformanceProblem?
What do you mean?
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Diagnose Before
You Prescribe
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Measure Everything!
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Algorithm?
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
CAPACITY?Can you add more?
CPUDiskMemoryBandwidth
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
SCALABILITY?
1 2 3 40
100
200
300
400
500
600
700
800
900
Linear Scalability
Typical Scalability
Ability to add more capacity in response to more demand.
Without a reduction in performance!
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
LATENCYTime delay in requesting an operation and it being initiated
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
1Gb Ethernet : 25 – 30MB/s10Gb Ethernet : 250 – 350MB/s
Infiniband : 6GB/s (maybe)
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
PING TIMESLocal : 57µsLAN segment : 300µsLAN: switches : 4msUK : 30msUSA : 100ms3G : 100s ms
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
GRANULARITY?
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Typical SSD Speed540MB/s
Source: tomshardware.com
Spinning Rust
Disk BUS SpeedsSATA 1.0 : 150MB/sSATA 2.0 : 300MB/sSATA 3.0 : 600MB/sSAS : 600MB/sFibre Channel : 1GB/sInfiniband : 1GB/s
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
DDR3 1600 : 12.8GB/s
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Source www.cs.virginia.edu Approx: 3-5GB/s per cpu still about 1 order magnitude faster than SSD and scales with coresThe general rule for STREAM is that each array must be at least 4x the size of the sum of all the last-level caches used in the run, or 1 Million elements -- whichever is larger.
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
MEMORYIS
FAST!Cache Data
InMemory
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
“As an extreme example, several current high-end machines run simple arithmetic kernels for out-of-cache operands at 4-5% of their rated peak speeds --- that means that they are spending 95-96% of their time idle and waiting for cache misses to be satisfied”http://www.cs.virginia.edu/stream/ref.html
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
PARALLELIS
THE FUTURE!
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Other manufacturers are available ;-)
Settle into soft grain leather seats with 10/6 way electric adjustment and heating for driver and passenger. The soft grain stitched instrument panel and Gloss Rich Oak veneer complete the luxurious look, while assertive 18 inch Vela alloy wheels further enhance the dynamic exterior.
The rear parking aid and Touch-screen visual indicator make slipping into tight spaces easy, while distractions at night are reduced by auto-dimming rear view mirrors. The Meridian™ 380W Sound System, with Digital Audio Broadcasting (DAB) radio receiver and Front Media Interface with USB and iPod® connectivity, lets you listen to your favourite music in rich detail.
XF PREMIUM LUXURY
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
http://www.datasciencecentral.com/forum/topics/the-3vs-that-define-big-data
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Application
Key Partition
Application
Key Partition
Application
Key Partition
Application
Key Partition
PU
T
BB
GE
T
B
BBC
PU
T C
C
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
public static void main( String[] args ) { CacheManager cacheManager = Caching.getCachingProvider().getCacheManager();
MutableConfiguration<String, String> config = new MutableConfiguration<String, String>();
cacheManager.configureCache("C2B2",config);
Cache cache = cacheManager.getCache("C2B2");
cache.put("Key", "Value");
System.out.println(cache.get("Key")); }
http://blog.c2b2.co.uk/2013/08/jsr107-getting-started.html
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Consistent Hashing
Key SpaceHash Ring
Node 1Hash
Node 2Hash
Node 3Hash
Node 4Hash
Key Hash
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Key Partitioning
• Keys Assigned Partition via Hash
• Nodes Own a Partition
• Number of Partitions Fixed
• Number of Nodes isn’t
Partition
Partition
Partition
Partition
Partition
Partition
Partition
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Application
Key Partition
Application
Key Partition
Application
Key Partition
Application
Key Partition
PU
T
BB
BB
NODECRASH!!!
B
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
Application
Cache
21 Amazon m2.xLargeMemory Instance with 17Gb RAM3 Nodes Per Instance with 4Gb 64bit JVM Heap + 5 Gb OS63 Cluster Nodes252 Gb JVM Heap AvailableApprox 125Gb Data in the Grid when taking into account duplicates42 vCPUsApprox $3633 per month
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Application
Key Partition
Application
Key Partition
Application
Key Partition
Application
Key partition
Code
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
1Gb Ethernet : 25 – 30MB/s10Gb Ethernet : 250 – 350MB/s
Infiniband : 6GB/s (maybe)
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Application
Cache
Application
Cache
Application
Cache
Application
Cache
CodeCodeCodeCode
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Class Size of code to Serialize is a few Kb
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
// Invoking a MapObject oResult = map.invoke(key, agent);
// Invoke on a subset of the cacheNamedCache map = CacheFactory.getCache("trades");Filter filter = new AndFilter(new EqualsFilter("getTrader", traderid), new EqualsFilter("getStatus", Status.OPEN));Map mapResults = map.invokeAll(filter, agent);
// The Agent Interfacepublic interface EntryProcessor extends Serializable { public Object process(Entry entry); Map processAll(Set setEntries);}
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
<T> T invoke(K key, EntryProcessor<K, V, T> entryProcessor, Object... arguments) throws EntryProcessorException;
<T> Map<K, T> invokeAll(Set<? extends K> keys, EntryProcessor<K, V, T> entryProcessor, Object... arguments);
T process(MutableEntry<K, V> entry, Object... arguments) throws EntryProcessorException;}
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
• Financial Risk Calculation– Can I book this Order
• Bet Placement– Can I place this Bet
• Hotel Availability Search– What hotels are available
• Web Translation– Gather Translated Text
Application
Keys
Application
Keys
Application
Keys
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
• Consumer Trend Analysis– Who orders what when
• Route Optimisation– Delivery Trucks
• Social Graph Search– Patterns in social media
graphs
• Fraud Detection– Transaction analysis
Application
Keys
Application
Keys
Application
Keys
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
public interface CacheEntryListener<K, V> extends EventListener {
}
public interface CacheEntryUpdatedListener<K, V> extends CacheEntryListener<K, V> {
/*** Called after one or more entries have been updated.** @param events The entries just updated.* @throws CacheEntryListenerException if there is problem executing the listener*/ void onUpdated(Iterable<CacheEntryEvent<? extends K, ? extends V>> events) throws CacheEntryListenerException;}
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
• Product Recommendations– Related products
• Alerting– Truck has Stopped!
• Financial Position Calc– Check regs during booking
• Nudge Purchasing– 5 people are looking at this
hotelApplication
Keys
Application
Keys
Application
Keys
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
http://blog.c2b2.co.uk/2012/06/infinispan-event-push-to-tomcat.htmlhttp://demo.c2b2.co.uk:8080/infinispan/
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
CoherenceGlassFishWebSocket
http://demo.c2b2.co.uk:7080 or Full URLhttp://demo.c2b2.co.uk:7080/CleanStockTicker/stockticker.jsphttp://blog.c2b2.co.uk/2012/10/java-one-2012-slides-and-code.html
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
MEMORYIS
FAST!
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Cache DataIn
Memory
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
PARALLELIS
THE FUTURE!
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Send ProcessingAcross theData Grid
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
PUSH IS HERE
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
Data Grids + Event Listeners +
WebSockets =
Real Time Push
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts
RememberYou Have a
Latency Problem!
www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting
The Leading Independent Middleware Experts