ahead of the stream: how to future-proof real-time analytics

Grab some coffee and enjoy the pre-show banter

before the top of the

hour!

The Briefing Room

Ahead of the Stream: How to Future-Proof Real-Time Analytics

Twitter Tag: #briefr The Briefing Room

Welcome

Host: Eric Kavanagh

[email protected] @eric_kavanagh


  Reveal the essential characteristics of enterprise software, good and bad

  Provide a forum for detailed analysis of today’s innovative technologies

 Give vendors a chance to explain their product to savvy analysts

  Allow audience members to pose serious questions... and get answers!

Mission


Topics

August: REAL-TIME DATA

September: HADOOP 2.0

October: DATA MANAGEMENT


The Value of Future-Proofing

Ø  Storm is hot

Ø  Spark is hotter

Ø More innovation coming

Ø  But keep in mind the latency


Analyst: Robin Bloor

Robin Bloor is Chief Analyst at The Bloor Group

[email protected] @robinbloor


Impetus

  Founded in 1991, Impetus offers a variety of products and services across the big data ecosystem

StreamAnalytix is its open source real-time streaming capability for big data analytics

 The platform leverages multiple Apache components, including YARN, Spark, Storm and Kafka


Guest: Anand Venugopal

Anand Venugopal Product Head - StreamAnalytix, Impetus Technologies Anand Venugopal has been working with Fortune 1000 enterprises to deliver real business benefits and ROI from Big Data Solutions at Impetus. He has been helping IT and line-of-business executives in large enterprises understand and extract the enormous value embedded in their static and "in-motion" Big-Data assets. Before Impetus, since 1995 – Anand has been in techno-business evangelism roles in various industries including telecom, gaming, media and entertainment and hi-tech.

© 2015 Impetus Technologies - Confidential 10

Webinar: Future-Proof Your Streaming Analytics Architecture

Robin Bloor, Principal Analyst

Aug 25, 2015

Twitter: @

Anand Venugopal, Product Head -‐ StreamAnaly3x Twi8er: @streamanaly3x


IMPETUS INTRODUCTION

Mission critical technology

solutions since 1996

Global Leaders are our Big Data clients

1600 people – US, India,

Global reach

Unique mix of Big Data

products and Services


REAL-TIME STREAMING ANALYTICS PLATFORM

Why ?

Build vs. Buy ?

What to buy ?

From whom to buy ?

How to Integrate ?


TOPICS COVERED TODAY

Business need for streaming

analytics

Industry verticals and

use cases Architecture

Streaming platform options

StreamAnalytix approach and

benefits

Some announcements!


WHY STREAMING ANALYTICS ?

Because it is now possible! Batch

only is old

Customer Experience

Operational Intelligence


WHY ? à BATCH VS. REAL-TIME BUSINESS PROCESS

SENSE Days ANALYSE Weeks ACT

SENSE ANALYSE ACT

Sec/ ms

Batch

Real time

Sec/ ms


WHY ? à CONTEXT AWARE: POSITIVE CUSTOMER EXPERIENCE

Multi-channel engagement in

real-time

Context Sensitive service

Happy customers, Loyalty, Revenue,

Profits, Growth


TYPICAL USE CASES FOR STREAMING ANALYTICS

•  Predictive Maintenance •  Clinical care and patient management •  Sensor analytics •  Fleet operations •  Fraud and anomaly detection •  Gaming •  Churn Analytics •  Network traffic analysis and optimization •  Internet Advertising

Verticals

•  Customer experience •  Clickstream Analytics •  Context-sensitive offers and recommendations •  IT Log analytics •  Security

Horizontals

•  Internet of Things •  Mobile app analytics •  Call Center Monitoring and Analytics

Combo


BUILD Vs BUY ?

•  Needs time, skills, budget

•  Upfront costs and long term maintenance costs

•  Total flexibility and control

•  Do you have the time to wait ?

Build

Vs

Buy

•  Are ready options available that meet your needs ?

•  Selection Criteria ? (Show Thumbnail of Ten considerations white paper)


Architecture Considerations


t

now

Hadoop works great back here RT-Ax works here

BLENDED VIEW – HISTORICAL AND NOW

Blended view Blended view Blended View


LAMBDA ARCHITECTURE : BIG AND FAST DATA COMBINED

Batch Layer

All data Pre-computed information

Batch re-compute

Speed Layer

All data Pre-computed information

Real time increment

Batch view

Serving Layer

Batch view

Mer

ge

Real time view

Real time view

All Incoming

Data Query


AN INTEGRATED APPROACH BLENDING CURRENT AND NEXT GENERATION TECH

Landing and ingestion

Structured

Unstructured

External Social

Machine Geospatial

Time Series

Streaming

Provisioning, Workflow, Monitoring and Security

Enterprise

Data Lake

Predictive applications

Exploration & discovery

Enterprise applications

Real-Time applications

Traditional

data repositories

RDBMS MPP

Compliance, Governance, Information Lifecycle, Data Lineage, Enterprise Meta Data Management


Streaming Platform Options and StreamAnalytix approach


“DEFAULT” APPROACHES TO STREAMING ANALYTICS

•  No leverage of open source •  Vendor lock-in •  Could be high cost •  Limited flexibility

Proprietary Platforms

•  Native Open source •  No Vendor Support •  Integration & maintenance

nightmare •  Significant delays in time-to-market

“Do it yourself”


THE 3RD APPROACH: BEST OF BOTH WORLDS

StreamAnalytix mitigates the disadvantages of the "default" approaches and offers the benefits of both worlds to enterprises for streaming analytics.

Abstraction of functional components like Ingest, CEP, Analytics, Visualization


StreamAnalytix – GIVES YOU A FUTURE PROOF OPTION

STORM SPARK OTHERS

NOW

Time


Future proof – Enterprise Grade – Open source based – Streaming Analytics platform

NEXT

Unified Business Interfaces Common Utilities Smart Workflows


StreamAnalytix Screenshots


CONFIGURABLE MESSAGE DEFINITION


CONFIGURATION FIELD DEFINITION


CONFIGURABLE ALERT DEFINITION


SAMPLE DATA PIPELINE (USING DATAFABRIC )

Supported Ingest Channels



Supported Processors



Supported Emi8ers/ Output Channels


CUSTOM CODE DEVELOPMENT/INTEGRATION

Download Sample Project

Custom Java Component Development

Reuse Exis3ng Storm Bolt Code


UPLOADING WORKFLOW

Configurable Workflow Upload (Ac3vi3 BPM support )


MONITORING A PIPELINE


DASHBOARD (SAMPLE)


USER MANAGEMENT


FROM WHOM TO BUY ? IMPETUS

?

Right size Independent Services

Track record of Long term partnerships and value

Recent success stories


Call Center Solution


HOSTED CALL CENTER SOLUTION

•  Call “Stitching” in real-time

•  IVR dominant path analytics

•  Analyze behaviour of Call Centre infrastructure

•  Business driven SLA based alerts in real-time

•  Historical reports for future pricing models

•  Trace complete call flow

•  Advanced Search on Call facets

•  Sentiments and alerts on email/chat conversations

•  Individual events scattered in different media servers.

•  Change the SLA alert definition and apply new definition in real-time without restart.

•  Sequence of events to be maintained at processing, storage and query level.

•  Media server logs contains only 1% of data which is relevant. Platform should have capability to filter the data at source level.

Key Features

Challenges Solved


ACCESS FREE VERSION OF STREAMANALYTIX

StreamAnalytix Lite

A production-ready version of StreamAnalytix for Developers to use a powerful visual tool-kit for developing real-time streaming analytics applications free of cost. •  Limited Functionality •  Unlimited Scale •  Free for ever

StreamAnalytix Developer Fully functional version of 'StreamAnalytix Enterprise' for Developers to quickly try out all the platform features by putting all their data at work to uncover new insights. •  Full Functionality •  Restriction on scale •  Free for 1 year

Enterprise Trial Fully loaded version of StreamAnalytix with a rich set of advanced visualization tools to easily develop & analyze real-life enterprise applications with minimal coding.

•  Full Functionality •  Unlimited Scalability •  60 days trial

For more details visit at http://streamanalytix.com/download

Platform Editions


Log Monitoring App


LOG-MONITORING DASHBOARD


SYSTEM MONITORING(CPU,MEM,DISK)


SEARCH


Real Time Social Media Analytics


CREATING NEW SEARCH FROM DATA SOURCE


REAL-TIME SENTIMENTS AND CLASSIFICATION


REAL-TIME TOPIC CATEGORIZATION


Thank you. Questions? ?

[email protected] www.StreamAnalytix.com

Contact us for an On-premise OR Cloud based trial and/or Proof of concept

Meet us at Strata Hadoop World in New York in September


Perceptions & Questions

Analyst: Robin Bloor

Of Lakes and Streams

Robin Bloor, PhD

The Division

Analytics and streaming analytics are not at all the same

The Biological Analog

u Our human control system works at different speeds: •  Internal systems – enteric nervous system •  Instant external reflex – spinal cord •  Fast external response – motor systems •  Considered response – the brain

u  Swift external response is predictive analytics & triggers

u Considered response is analytics

A While Ago…

The Hadoop Disruption

Then Spark Disrupts Hadoop

u  Spark has become the de facto vehicle for many distinct Hadoop projects because of its in-memory scale-out capability

u  It can do streaming to a degree, but it is not ideal for very low latency applications

u  For that you need to scale-up, not out, and a high level of optimization is necessary

u Nevertheless, it has its place

Spark and Storm

u Along with Spark comes with Shark (a Hive-compatible version of Spark)

u  Storm provides batch and streaming (event processing capabilities) concurrently via the lambda architecture

u  Lambda: batch layer, serving layer, speed layer

u  Spark now also has lambda architecture and can thus behave in a similar manner

u  Spark currently seems to be more fashionable

My Current View

Streaming is more about DATA/SOFTWARE ARCHITECTURE than

anything else

u  There’s clearly a trend to low latency analytics. How do you see this developing?

u  Aside from predictive analytics and typical CEP applications, are there any other application areas that you are encountering?

u  Please describe a typical implementation, from adoption through development to implementation.

u  How much integration work is necessary?

u  What is your current largest customer in terms of streaming volume, and what is the application?

u  Do you find yourselves competing directly with Spark or Storm?


Upcoming Topics

www.insideanalysis.com

August: REAL-TIME DATA

September: HADOOP 2.0

October: DATA MANAGEMENT


THANK YOU for your

ATTENTION!

Some images provided courtesy of Wikimedia Commons and http://arapisacz.blogspot.com/2009/10/floating-house-from-brad-pitt.html

ahead of the stream: how to future-proof real-time analytics

Technology