real-time+communication service providers = monetizing big data with streaming analytics
DESCRIPTION
As the flow of mobile data traffic engulfs wireless networks, carriers large and small suddenly find themselves sitting on a goldmine of CDR, customer, device and network data that, if actioned correctly, holds the keys to greater customer intimacy, and in a broader sense, the successful future and evolution of telecom providers. A new, four-dimensional approach to Operational Intelligence from Big Data can help analyze the relationships between your customer, network, device and location data in order to make faster, better decisions in real-time. This presentation is taken from a Webinar by SQLstream's Telecoms & Media specialists Ronnie Beggs and Chris Clabaugh, and explains the new revenue-generating opportunities that lie within Big Data and the technology challenges to operational efficiency. The presentation explores how to leverage streaming Big Data technology to integrate and analyze streams of CDR, device, network and service data in real-time, ways to enrich the customer experience, deepen subscriber relationships, and drive automated actions to optimize service and network performance in real-time, optimized Customer Care workflows for efficient troubleshooting and reduced costs, and real-time fraud detection and prevention from CDR analytics for cost reduction and improved customer satisfaction.TRANSCRIPT
If you haven’t dialed into the audio portion, please do so now:
U.S.A +1 646.558.2119 | code 884.143.848
T H A N K YO U F O R J O I N I N G ! T H E W E B I N A R I S A B O U T TO S TA R T
REAL TIME + Communication Service Providers = Monetizing Big Data with Streaming Analytics
¤ TWEET: during and after the webinar, please
use #streaminganalytics for live discussions
¤ DIRECT QUESTIONS: please use the box to the
right of your screen
¤ RECORDINGS: an edited version of the webinar
recording will be emailed and posted on
company website
| 3 Copyright © 2013 | +1 877 571 5775 | [email protected]
¤ Explain real-time Big Data and streaming analytics
¤ Explore real-time applications in the Telecoms industry
SCOPE
¤ Share our thoughts, experiences and use cases
| 4 Copyright © 2013 | +1 877 571 5775 | [email protected]
¤ Over twenty years experience of product management, marketing and business development in the real-time software business.
¤ Worked for a number of successful start-ups, from early stage through to acquisition, including Metrica (ADC) and Cramer Systems (Amdocs).
Today ’ s P re sen te r s
¤ Over 30 years of corporate development experience
» Managing member of 25 years at Chapman Investment Company
» Advisory Board member of 14 years at Tulane University
» Advisor and consultant for a wide variety of companies in the technology !eld, including Kabira, CollabNet and Progress Software
¤ CEO and co-founder of Allegrix, Inc.
Ronnie Beggs – Vice President, SQLStream
Chris Clabaugh– Vice President, SQLstream
| 5 Copyright © 2013 | +1 877 571 5775 | [email protected]
About SQLs t ream
facts
o Launched 2009 o Multiple deployments
across many industries
o Real world benchmarks
capabilities
o Unstructured and structured data
o Accelerates and extends Hadoop & RDBMS
o Not only SQL
innovations
o Massively scalable streaming data platform
o Only standard SQL streaming engine
o Six patents for stream processing
A Streaming Big Data Management Platform for real-time Operational Intelligence
from high-velocity machine data.
| 6 Copyright © 2013 | +1 877 571 5775 | [email protected]
¤ Stored information doubling every 18-24 months
¤ “Internet of Things” is creating new data with no human interaction
¤ Business decisions need to happen faster driven by actionable intelligence
¤ Streaming and Predictive Analytics is changing the way we “see” into the future
What ’s happen ing wi th B ig Data
| 7 Copyright © 2013 | +1 877 571 5775 | [email protected]
¤ Businesses have been managed based on prior history delivered at the end of the day, month or quarter - like driving while only looking in your rear-view mirror.
Look ing Forward wi th S t reaming Analy t i c s
¤ Streaming Analytics allows organizations to drive their business in real-time, reacting to the changes and opportunities as they happen.
| 8 Copyright © 2013 | +1 877 571 5775 | [email protected]
Te lecoms and B ig Data
AREAS OF NEEDS ¤ Customer Experience ¤ Fraud prevention ¤ IP Network & Service Performance ¤ Call Center Experience
Improved operational efficiency and customer satisfaction, with a real-time 360o customer view, and continuous data silo integration
Streaming Value
Call Centers| Telecommunications | Data Centers
ISSUES ¤ Untapped New (Big) Data ¤ Ease of churn ¤ Lacking a 360o customer view
$5.4B Communications analytics market by 2019.
Market Research Report.biz
| 9 Copyright © 2013 | +1 877 571 5775 | [email protected]
S t reaming Analy t i c s as a Complement to Tradi t iona l Data Management
DATABASE Collect, translate,
classify ANALYSIS PATTERN
DETECTION ACTION
TIME
STREAMING DATA ANALYTICS PLATFORM
DATA
DATA
Action Action C-ETL Aggregation Classification Profiling Deep analytics Trend detection Trend correlation Extrapolation Alerts Actions
Repeated Queries
Seconds Hours -> Days
DATA
DATA
| 10 Copyright © 2013 | +1 877 571 5775 | [email protected]
OPERAT IONAL INTEL L IGENCE I n t eg ra t i ng Opera t i on s and Ana ly t i c s i n Rea l - t ime
As we move toward a real-time business environment, the capability to process data flows swiftly and flexibly will become increasingly important. SQLstream leads the industry in this kind of capability. ”
Robin Bloor Chief Analyst for Bloor Group
Business Intelligence
Operations
Real-time Operational Intelligence Continuous monitoring and analytics
Improve decision-making
Automate operational processes
Billing Rating QoE
Network analysis Fraud Monitoring
”
| 11 Copyright © 2013 | +1 877 571 5775 | [email protected]
T he I n fo r ma t i on Va l ue Cha i n
What is happening?
What might happen?
What just happened?
Make stuff happen!
| 13 Copyright © 2013 | +1 877 571 5775 | [email protected]
Te l co I ndu s t r y Cha l l enge s i n t he D ig i ta l Wor ld D r i v e r s f o r t h e r i s e i n t h e B i g D a t a Te l c o a n a l y t i c s m a r k e t
• Declining core revenue streams
• Widening data monetization gap
Source: Wipro analysis based on Ovum Mobile Broadband Forecast: 2011-2016, and Cisco Global Mobile Data Traffic Forecast Update, 2011-2016
| 14 Copyright © 2013 | +1 877 571 5775 | [email protected]
Ac t ionable in s igh t s - the ideal s cenar io
SYSTEMS & APPS
SOURCES
Alerts
Triggers
Reports
Dashboards
CORPORATE
• Marketing • Operations • Customer care
• Unstructured • Semi-structured • Structured
QoE
STREAMING
ANALYTICS
| 15 Copyright © 2013 | +1 877 571 5775 | [email protected]
What ’s poss ib le f rom xDRs | APPL ICAT IONS
NETWORK SERVICE CUSTOMER BUSINESS • Optimization of
network utilization
• Network Capacity planning
• Anomaly detection & troubleshooting
• Monitoring and protection
• Self-healing networks
• Partner routing
• Subscriber profiling and information
• New product rollout visibility
• Product development and tariff optimization
• Yield management and dynamic pricing
• Service personalization
• Customer loyalty management
• Churn prevention
• Device analysis
• Campaign management and precision marketing
• Contact center alerts
• New customer experience monitoring
• Billing accuracy and revenue
• SLA management
• Interconnect billing analysis
• Real-time reports
• Fraud and suspicious traffic detection
| 17 Copyright © 2013 | +1 877 571 5775 | [email protected]
Da ta Ana ly s i s Today – fa r f rom Rea l T ime
Current architectures o Multi-stage process o Offline ETL o Interim storage with no analytics capability
ETL / RDBMS process challenges o Volume and Velocity o Variable, changing formats o New types and formats
IMPACT o High Cost of Ownership o Delays to process the billing information o Delays in external distribution to partners
WAREHOUSE
Near-term data storage
PLATFORMS
Real-time ETL
| 19 Copyright © 2013 | +1 877 571 5775 | [email protected]
H igh - ve lo c i t y B ig Da ta Ana ly t i c s M a s s i v e l y p a r a l l e l w i t h i n c r e m e n t a l e v a l u a t i o n
Enhancing with historical information
Storage of intermediate & final query results
Ope
ratio
nal I
ntel
ligen
ce
Logs
Sensors
Mobile
Networks
Wireless
Radio
M2M
Internet
Security gateways
¤ Continuous queries on unstructured & structured streaming data ¤ Incremental query results ¤ Predictive analytics & automated actions
| 20 Copyright © 2013 | +1 877 571 5775 | [email protected]
CDR and I PDR Ana ly s i s W h e r e i s t h e i n t e l l i ge n c e ?
TRANS,2013-02-17-15:30:22,3458783,2347897953,128.56.0.253,STATUS:-15, DE69975, 4157588342 Transaction Log Details
Web Server Logs
CDRs
Device Locations
Twitter {"created_at:Thu Feb 17 15:30:55 +0000 2013,id:304612775055998976,id_str:304612775055998976,text:@MyServiceProvider today sucks, keeps dropped!,source:u006ca href=http:www.url.com rel=nofollow,followers_count:147,friends_count:10142, location: San Francisco, time_zone: Pacific, geo_enabled:true, location:u00dcT: -6.1987552,106.8661953, screen_name:APerson
<id>1597831220</id><deviceid>0198873465</deviceid><lat>lat=47.643957</lat><lon>lon= -122.3269</lon><time>2013-02-17T15:37:26Z</time><bearing>223.4535</bearing>
<id>1597865781</id><deviceid>0198873465</deviceid><lat>lat=47.645982</lat><lon>lon=-122.327500</lon><time>2013-02-17T15:37:26Z</time><bearing>200.6138</bearing>
<id>1597940125</id><deviceid>0198873465</deviceid><lat>lat=47.647381</lat><lon>lon=-122.326501</lon><time>2013-02-17T15:37:26Z</time><bearing>87.4357</bearing>
[Sun Feb 17 15:30:49 2013] [notice] srv-sfo-08 caught SIGTERM, shutting down [Sun Feb 17 15:30:49 2013] [notice] Apache/2.2.21 -- resuming normal operations
TERMINATE,ctl09gsx,01299796304,GMT-08:00,02-17-13,15:21:00,9,387,64ms,02-17-13,15:30:55,0005, IP-TO-IP,4157588342,8775715775,1,0,4157588342,RD_AXY_NN0_001,SFR01AAG34,40.50.245.60, 234.234.60.75,65678,411,399,SIP,SANFRANCISCO,0x4B1698,0x0005E,0x49768,4157588342,0198873465
Timestamp
Timestamp
Timestamp
Timestamp
Timestamp
Mobile # Customer
Mobile # Device ID Term Reason
Device ID Location
Location
Service Provider
Fail Code
Server
| 21 Copyright © 2013 | +1 877 571 5775 | [email protected]
CLEANING & FILTERING
STREAMING ANALYTICS
STREAMING AGGREGATION
CONTINUOUS INTEGRATION
QoE Rating Fraud Monitoring
Billing
Traffic analysis
An Opera t i ona l I n t e l l i gen ce P la t fo r m
Log Sensors Mobile Networks M2M Radio towers
| 22 Copyright © 2013 | +1 877 571 5775 | [email protected]
SQL s t ream s -S t ream ing P roduc t Po r t fo l i o
s-Server Data Management Platform for Streaming Big Data
s-Analyzer Drag and Drop Application Builder for
Streaming Analytics Applications
s-Transport Geo-Analytics for Location-based
Applications
s-Visualizer Advanced Enterprise
Visualization
s-Cloud s-Server EC2 AMI Deployment
s-St
udio
D
evel
oper
& A
dmin
Con
sole
Stre
amA
pps
Fast
Sta
rt S
trea
min
g A
pps
Dashboards
| 24 Copyright © 2013 | +1 877 571 5775 | [email protected]
Case s tudy : f raud prevent ion
Alerts
Triggers
Reports
STREAMING
ANALYTICS
• Call suspension • Acct. suspension • Email Alerts Destination
Location
IP spoofing alerts
Customer call profile
dura
tion
Mo Tue Wed Thu Fri Sat Sun
① LA ② Nairobi ③ NY ④ …..
① LA ② SF ③ NY ④ ….
① LA ② Detroit
① LA ② LA1
Dashboards
| 25 Copyright © 2013 | +1 877 571 5775 | [email protected]
“SQLstream allows Veracity to provide vital real-time reports to our customers that previously took hours to create. SQLstream also provides real-time monitoring and insight into network concerns allowing Veracity to proactively address any such issues”
Rea l - t ime Cal l Rat ing & Fraud
Veracity Networks ¤ Internet provider ¤ Residential and business ¤ Range of IP-based services
OPPORTUNITIES ¤ CDR/IPDR real-time analytics ¤ Real-time rating and QoE ¤ Fraud prevention
BENEFITS ¤ Improved customer satisfaction ¤ Improved bandwidth utilization ¤ Improved fraud detection times
| 26 Copyright © 2013 | +1 877 571 5775 | [email protected]
SQLstream
Cus tomer Benc hmarked Performance L a r ge N e t wo r k & Te l e c o m E q u i p m e n t M a n u f a c t u r e r
Network Data
Network Data
Network Data
Network Data
Network Data
ENRICH SHARE ANALYZE
Remote Agent
Remote Agent
Remote Agent
Remote Agent
Remote Agent
Data Warehouse
External Systems
External Data
PERFORMANCE STATISTICS System Throughput: 1.35M events / sec
Server Configuration: 1 x 4-core CPU
Event Size: ~1KB
Data Sources: Many
SYSTEM CHARACTERISTICS Collection: Intelligent Remote Agents (Distributed)
Enrichment: Streaming data augmentation
Analytics: Temporal & spatial pattern detection
Output: Data warehouse + applications (JDBC)
| 27 Copyright © 2013 | +1 877 571 5775 | [email protected]
¤ Drivers for Streaming Data Analytics » Declining revenue streams » Increasing data monetization gap » A Big Data Problem: Volume, Velocity and Variety » Current technology and solutions far from real-time
¤ SQLstream’s Real-time advantage » Low latency correlation, alerts and actions across all data sources » Streaming enrichment » Continuous integration with existing platforms » Drives real-time rating, billing, QoE and QoS, and fraud
Conc lus ions
Chris Clabaugh
Email | [email protected]
Website | www.sqlstream.com
DOWNLOADS | http://www.sqlstream.com/downloads/