tibco spotfire: data science in the enterprise
Post on 27-Jan-2015
115 Views
Preview:
DESCRIPTION
TRANSCRIPT
From Data to Insights
in Internet Time
Data Science in the Enterprise
Eric Novik Industry Analytics Group TIBCO Spotfire
21st CENTURY PLATFORM
RV, EMS, FTL
ActiveMatrix
BPM
MDM
BusinessEvents
ActiveSpaces
LogLogic
In-Memory
Spotfire
Visual Analytics
Data Discovery
Statistics
Silver Fabric
Private Cloud
Public Cloud
SAAS Apps
tibbr
Social Media Collaboration
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Are Your Models Behaving Badly?
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Data Mashups
Exploratory Data Analysis
Features Spotfire
Dashboards Model & Predict
Deploy Champion
Model
Test & Learn
Sales
Loyalty
Social Media
Behavior
Filter
Map
Merge
Shape
Social
Network
Members
Improve Guided -------- Deploy -------- In-Line Explore Data
Aggregate
Prepare Data Business Case
Increase
Productivity
Grow
Revenue
Ensemble
Forest
Regression
Additive
Models Transaction
Visualize
Geo
Location
Service
Challenger
Models
At Rest
In Motion
Value Theses
Reduce
Risk
ROI
Transaction
Dashboard Updates
Data Science 2.0: Guided and In-Line Analytics
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Time
The TIBCO Spotfire Experience Reduces Latency
Business
Value
Business Event
Data Ready for Analysis
Analysis Completed
Decision Made
$$$$
$$$
$$
$ Action Taken
TIBCO Spotfire
Analytics
Speeds Action
and Increases
Business Value
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Data a Information a Knowledge a Wisdom
Single BI Platform
Traditional BI and Discovery
Design Data Models
Source The Data
Build Reports
Business Users Joe Data-Modeler
CRM ERP Financials
ETL
OLAP Cube
Requests
Data Warehouse
Iterate the Loop
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Business Analysts Joe Data-Modeler
OLAP Cube
BI Platform
Guided, Self-Service Analytics
Design Data Models
Source The Data CRM ERP Financials
ETL
Data Warehouse
The Spotfire Experience: Guided, Self-Service Analytics
Spotfire 5.x
In-Database Analytics
In-Memory Analytics
Hybrid On-demand Analytics
Data Virtualization
TIBCO Enterprise Runtime for R
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Dynamic Visual Analytics: Data and Models
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
My Kind of Pie Chart….
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
“Numerical quantities focus on expected values, graphical summaries on unexpected values.” - John Tukey, Exploratory Data Analysis, 1977.
Visual Analytics
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Visualizing Risk
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Visualizing Risk
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Visualizing Risk
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Visualizing Correlations and Linear Models
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Visualizing Correlations and Linear Models
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Customer Acquisition/Retention ► Understand transactions and attractive
products for individual investors ► Identify prospects through data-mashups
of in-house and industry sources ► Understand client dissatisfaction
#1. Business and Customer Insight
Customer Growth ► Identify persuadable customers for
cross-sell / up-sell programs ► Drive campaign efficiencies ► Discover sensitivities of acceptance to
demographics and behavior ► Test and learn marketing and loyalty
Business leaders and marketing groups need insight in to prospects and customer behavior and to identify “persuadables” for growing the business
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Customer Life Cycle
Cross-sell / Up-sell Sentiment & Intent Analytics Sales Force Detailing Optimization Test & Learn
Market Mix Analytics Price Elasticity Campaign Effectiveness Attrition & Social Churn
Customer Acquisition
Customer Retention
Relationship Growth
Customer Lifecycle
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Customer Growth & Cross-Sell
• Response
• Acceptance of product (wealth management services, credit card)
• Analysis Unit
• Customer
• Features
• Bank transactions: ATM, checking; deposit, withdrawal, balance, …
• Demographics: sex, age, profession, nationality
• Model
• Classification trees
• Association & machine learning
• Value
• Customers * Value * Conversion
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Citibank saw a 21% increase in customer spend, and shortened time-to-market for new campaigns by 92% after implementing TIBCO Spotfire cross-sell programs
Citibank Asia
http://www.tibco.com/customers/#citibank-asia
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Citibank COP: Delivered Business Functionality
COP360 Degree
Customer View
DWH
ACWS
AFEWS
CBOL
ECCM
GRB
ECS
Oracle
Master
data
download
Master
data upload
Offer
palette
Service Request &
Offer Feedbacks
Addr
change
Status
Changes
Offer
response
Financial &
Non-financial
transaction
Campaign
Definition, Offer
Request &
Feedbacks
Customer Citibank Phone Officer (CPO)
DM Staff Members
CTI
Offer
palette
Offers
Files
Campaign
Master &
Offer List
Business Rules
Definitions
KPIs, Performance
Metrics, Campaign
FeedbacksCampaign
Definitions &
Offer list
Status
Changes
Offer
Request &
Feedbacks
Offer Request &
Feedbacks
Offer
response
Offer
response
Offer
response
Offer
request
Login
Internet offers
Campaign
Definition,
Limited Master
data upload
FMS
Offer
Feedbacks
Campaign
Definition,
Offer
Feedback
Fulfillment
info
Intelligent Customer/Scenario Dialogue Real-time synchronization of customer contact / touch-points (inbound and outbound)
Dynamic Offer Management Real-time (re) scoring Real-time offer eligibility and prioritization
Conversion Tracking and Analysis Real-time dash boarding / tracking
Dynamic Channel Synchronization and Management Infrastructure to address time-sensitive offers (e.g.. real-time cross-offers on SMS based on card transactions at specific locations)
Sales Process Tracking Close-loop tracking of intermediate fulfillment processes
On-Board Measures, Statistics and Visibility Reliability, Availability, Scalability Hot – Dynamic Configuration
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
BMO saw a 15% increase in customer spend on other products after implementing TIBCO Spotfire cross-sell programs.
Bank of Montreal
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
BE Discover pattern in data
…
Build predictive statistical model
Deposits correlate with opening wealth account in some demographics
Predict new wealth mgt customer from account transfers and demographics
Export optimal rules/thresholds
and deploy with BE
Make predictions and act on them
at every customer interaction
Measure campaign success
Business Analyst Data Scientist
Manager
5-10% increase in wealth account conversion rate
Male 28-42; median deposit; $ transfers; then extend offer
Refine/Extend
Extend selective offers to open new wealth account based on real-time predictions of propensity to accept offer at time of events e.g. transfer/deposit
Analyst
Event Cloud
Teradata Oracle BPM
Tibbr
Analytics-to-Action: Cross-Sell / Up-Sell
TERR
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Customer Insight: Social Media & Sentiment Analytics
• Response
• Customer sentiment / intent
• Analysis Unit
• Tweets
• Features
• Favorability
• Recommendation
• Influencers / Followers
• Models
• tm, twitteR, wordcloud
• Value
• Attribution : contribution of channel
• Optimization : scenario planning
• Allocation : media mix
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Sentiment and Social Media Analytics
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Influencers and Followers
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Business Insights: Procter and Gamble Business Sphere
Source: McKinsey Quarterly, Nov 2011
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Business, Customer and Consumer Insights
• Analyses
• Business Insights
• Customer Insights
• Consumer Insights
• Analysis Unit
• Product * Market
• Customer behavior
• Features
• Product/channel attributes
• Promotions, pricing
• Value
• Focus on product * market combinations
• Identify managers / attributes with issues
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Enterprise Risk Aggregation ► Aggregated view of multiple risk positions by
geography, product, … ► Assess loss given default for products and
counterparties
#2. Risk Management
Credit and Counterparty Risk ► Handle unexpected changes in markets with
rapid rebalancing ► Incorporate predictive analytics e.g. stress
tests, economic capital ► Collaborate between traders and managers
Operational Risk ► Track desk in near real time ► Identify violations, patterns, rule-brushing
traders
Risk managers need interactive and aggregate views on risk; and ability to visualize complex risk analyses with management
Market Risk ► Mark to market, VaR, scenario and sensitivity
analysis ► Interest rate and FX-risk ► Collaborate across organization for rapid
response to market conditions
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Royal Bank of Scotland
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Royal Bank of Scotland
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Equity Trading ► Analyze price / volume across holdings with
drill down into individual securities ► Calculate statistical and technical indicators,
benchmark comparisons ► Incorporate indicators and financial data e.g.
Bloomberg, Thomson, FactSet ► Collaborate across trading teams
#3. Trading Analytics
Fixed Income Analysis ► Identify unknowns: bond prices, interest,
durations, convexity and yield ► Compare fixed income instruments with
economic indicators ► Incorporate statistical methods applied to
new and existing instruments
Trade Fraud and Compliance ► Track desk in near real time ► Identify violations, patterns, rule-brushing
traders
Traders need rapid and comprehensive view of risk and sensitivities
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Portfolio Optimization ► Analyze risk-return sensitivities ► Efficient frontier statistics ► Drill to efficient portfolios of interest,
incorporating predictive analytics ► Rebalance on –demand
#4. Portfolio Management
Performance and Attribution ► Dimension-free exploration of assets across
time ► Real-time analysis and review of fund
strategies impact on performance ► Rapid transaction reconciliation with
exceptions analysis ► Integrate with CRM data for customer insight
Managers require multiple assets and funds in single view; need to prove value and comply with new regulations
Regulation Compliance ► Comply with new regulations that require
thorough understanding of trading across assets and liabilities
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Anti-Money Laundering ► Identify / trace unknown and suspect
transactions across context e.g. business lines and geographies
► Set thresholds, define segments and automate alert scoring
► Comply with OCC guidelines
#5. Fraud and Compliance
Credit Card Fraud ► Identify fraudulent transactions by vendor,
location ► Highlight patterns, context and identify root
cause
Trade Fraud and Compliance ► Track desk in near real time ► Identify violations, patterns, rule-brushing
traders
Regulators expect banks to have powerful and robust analytics for compliance and Anti Money Laundering (AML)
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
TERR, Open Source R
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Sources: NY Times : “The Age of Big Data” (Feb 11) NY Times: “How Companies Learn Your Secrets” (Feb 16)
TIBCO Enterprise Runtime for R (TERR)
• TIBCO owns the S language and have built an R engine
• TIBCO engineers created and support S+
• TIBCO engineers have created the world’s only commercial R engine: TIBCO Enterprise Runtime for R (TERR)
• TERR built from ground up in C++
• Faster than R; better memory management
• Optimization for known deficiencies in R constructs (e.g. looping)
• Clean room: no code from R
• MKL, lapack, linpack, eispack inside
• Licensing
• IP owned, supported and maintained product
• Scalable, embeddable, redistributable
• TERR is now inside Spotfire and TIBCO Products
• Spotfire Professional
• Spotfire Statistics Services
• TIBCO Business Events, TIBCO Grid Server
Engine Pool
Service dbase
Spotfire
Statistics
Services
TERR
SAS® MATLAB®
C# Java
TERR
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Speed Comparison
Model Fitting: 5 Million Rows Model Scoring: 20 Million Rows
TERR
AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Data Viz Challenge
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
People Who Write Longer Emails Earn Higher Salaries!
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information. Average Email Size
Sala
ry x
10
,00
0
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information. Average Email Size
Sala
ry x
10
,00
0
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Sala
ry x
10
,00
0
Staff
Senior Managers
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Thank You!
Eric Novik Spotfire Industry Analytics Group
Twitter: @ericnovik
enovik@tibco.com :: www.ericnovik.com
top related