real time analytics with spring
DESCRIPTION
Speakers: Luke Taylor and David Turanski Today's solutions must provide the ability to interpret related events and understand trends that are happening right now. This session will cover some of the out of the box capabilities of Spring XD to tap into big data streams and generate metrics such as simple counters, aggregate counters, moving averages, rates of change, and histograms. Hands-on demos will show you how Spring XD uses Redis and GemFire's Continuous Query and Function Execution to incorporate real-time analytics into event-driven applications.TRANSCRIPT
![Page 1: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/1.jpg)
Real Time Analytics WithSpring
David TuranskiLuke Taylor
![Page 2: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/2.jpg)
Real Time AnalyticsWhat's happening right now?
Event drivenNotify humans or systems in real timeSimple counters and metricsAggregate CountersGauges, Moving AverageComplex Event Processing
![Page 3: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/3.jpg)
Spring XD OverviewOne stop shop for:
Data IngestionRealtime AnalyticsWorkflow OrchestrationData ExportBuilt on existing Spring Assets
Spring Integration, Batch, Data
XD='extreme data'
![Page 4: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/4.jpg)
Spring XDOpen source Apache licenseOut of the box executable servers
not a bunch of jars
Pluggable Module systemConfiguration via High level DSL, CLI, andREST services
![Page 5: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/5.jpg)
Spring XD DSLBased on Unix pipes and filters syntax
http | filehttp --port=8080 | file --rootDir=/data/
'http' and 'file' are Module names
![Page 6: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/6.jpg)
Spring XD Streams
![Page 7: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/7.jpg)
Anatomy of a Stream
![Page 8: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/8.jpg)
XD Single Node
![Page 9: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/9.jpg)
Spring XD TapsA Tap is a stream whose source is a point in anexisting streamTaps are primarily used for real time analysisExample:twittersearch --query=Bieber | file
tap:twittersearch > field-value-counter --
fieldName=entities.hashtags.text
![Page 10: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/10.jpg)
Spring XD MetricsSpring XD provides persistent and inmemoryMetrics RepositoriesMetrics are modules
commonly act as sinks for taps
XD includes the following MetricsCounterField Value CounterAggregate CounterGaugeRich Gauge
See fordetails.
Spring XD Analytics Documentation
![Page 12: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/12.jpg)
GemFire FunctionExecution
Efficient way to process large data volumes"where the data lives"Java code deployed to GemFire server nodesThe Function runs on local cache node andtransports results over the network to theprocess that executed the function (Execution)Function Execution aggregates results using aResultCollector
![Page 13: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/13.jpg)
GemFire Continuous QueryGood for asynchronous notification of dataupdatesClient subscribes to a data query and binds it toa QueryListener callbackGemFire invokes the QueryListener wheneverthe query's result set changes state
![Page 14: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/14.jpg)
The DemoPerform real time analytics on the Twitter streamCreate a Twitter ingest stream with XDCreate taps on the Twitter stream to feed XDMetricsCreate a tap to pull selected data into GemFirefor deeper analysis
Show how to build and deploy a custom processor
Execute a GemFire Function to perform acomplex analysis of hash tagsDashboard web app to view analytics
![Page 16: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/16.jpg)
Questions?
![Page 17: Real Time Analytics with Spring](https://reader034.vdocument.in/reader034/viewer/2022042606/5455f74cb1af9fcf338b4bb7/html5/thumbnails/17.jpg)
© 2013 SpringOne 2GX Do not distribute without permission
Learn More. StayConnected
Project SitesSpring XD Spring Data GemFire
projects.spring.io/springxd
projects.spring.io/springdatagemfireRelated sessionsThu 12:45 Real Time Event Processing andDecision Making Lyndon AdamsXD and Pivotal HD Demo Demo lounge