the fast data competitive advantage - fst...
TRANSCRIPT
The Fast Data
Competitive
Advantage
Kevin Pool
CTO – TIBCO Asia
© Copyright 2000-2015 TIBCO Software Inc.
The frontiers between digital and physical are blurring
IoT in Business
How Big is Big?
It’s All about getting BIG VALUE
New Data Sources
New Types of Data
New Insights
New Business Value
BIG VALUE from Data
Every Business Should Collect and Analyze Big Data
Visual Analytics
Single BI Platform
Traditional BI and Discovery
Design Data Models
Source The Data
Build Reports
Business Users Joe Data-Modeler
CRM ERP Financials
ETL
OLAP Cube
Requests
Data Warehouse
Iterate the Loop Iterate the Loop
Iterate the Loop Iterate the Loop
Iterate the Loop Iterate the Loop
OLAP Cube
Guided, Self-Service Analytics
Design Data Models
Source The Data CRM ERP Financials
ETL
Data Warehouse
TIBCO Spotfire
Guided, Self Service Analytics
Give Analytics Power directly to the Business Analysts
Business Analysts
Give interactive power directly to
the End Users
Fast Data
A little bit of the right information,
just a little bit beforehand,
is more valuable than all of the information in the
world six months later…
This is the Two-Second Advantage
the Two-Second Advantage
Short-lived opportunities
Critical timeframes
FAST DATA
• Rapidly Changing Values
• Quickly Declining Business Value
• Short Response Timeframes
• BIG VALUE comes from processing and reacting in Real Time
BIG DATA LETS YOU UNDERSTAND WHAT’S ALREADY HAPPENED
FAST DATA LETS YOU INFLUENCE WHAT’S ABOUT TO HAPPEN
Fast Data
CONTINUOUS PROCESSING OF BIG DATA
TO GAIN INSTANT AWARENESS AND TAKE ACTION
Response Time
Value
Fast Data
Fast Response = Big Value
Time
Fast Response = Big Value
Business
Value
Business Event
Data Ready for Analysis
Analysis Completed
Decision Made
$$$$
$$$
$$
$ Action Taken
Value Increases
with Shorter
Response Times
Driving a Competitive Advantage
Fast Data Architecture
Fast Data Architecture
Connecting Mobile, Marketplaces & Everything
API Gateway
DMZ Gateway
Internet
Your Enterprise
Things
Mobile Users
Machines
21
Supplier
Apps, Apps, Apps
Apps, Apps, Apps
• Fast Data can be
– Available in Apps
– Callable from Apps
– Generated by Apps
– Used to influence outcomes
Fast Data Architecture
Fast Data Architecture
Fast Data Architecture
Fast Data Architecture
Fast Data Architecture
Ask Every Interesting Question Every Millisecond
Events
App
Sensors Mobile / Web
Automated Offers
Streaming Analytics
Social Social
Automated Decisions
Fast Data Architecture
The One and Only data mart for Fast Data
LiveView Continuous Query Processor
Dynamic aggregation
Live visualization
Alerts
Action
Ad-hoc continuous
query
Fast Data Architecture
Predictive Analytics
Fast Data Architecture
Fast Data Architecture
© Copyright 2000-2013 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
1001001010010010101001010101010101010101001010010010101010100101010010101001010100100101100100100110010100101010101010101010110010101010101010101010010101010010
MACHINE DATA
TIBCO ENTERPRISE SERVICE BUS AND IN MEMORY DATA GRID
REAL TIME APPLICATION EVENTS AND DATA
UNDERSTAND
EVENT
SERVER
HISTORICAL DATA
ANTICIPATE
RULES /
MODELS
RESULTS
Financial Services Example
VISUALIZE
LIVEVIEW DASHBOARDS
101001010101010101010101010100101010101010101010110000
10101010
LOG FILE MANAGEMENT 11010101010101010101001010101010101010101100001010101 HADOOP / ASTER
1
0
1
1
0
1
0
0
1
1
0
1
0
1
0
0
1
SPOTFIRE ANALYTICS
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
1
0
1
0
0
1
0
1
0
1
0
1
1
0
0
1
RESULTS
STATS
SERVER
ACT
Identify triggers for opening wealth mgt account
Predict new wealth mgt customer from account transfers and demographics
Extend offers to open wealth mgt account based on real-time predictions of propensity to accept offer at time of events e.g. transfer/deposit
Measure increase in wealth account conversion rate
REAL-TIME PROMOTIONS
SHORTENED TIME-TO-MARKET BY
92%
21% INCREASE
CONSUMER SPENDING
• One of the larges department
stores in the USA
• 171,000 employees
• FY 2011 USD$26.4 billion
• 800 Macy’s department stores
and furniture
• 45 Bloomindales Stores and
Outlets
Macy’s
Stock Price Comparison - 2011
JC Penny’s Sears Macy’s Target
“The loyalty implementation was the single greatest technology achievement in the corporate history of Macy's” Peter Sachse – Macys CMO
Macy’s CMO Quote
Stock Price Comparison - 2014
JC Penny’s
Sears
Macy’s
Target
Fast Data Use Case Trade Fraud and Compliance
BNY ConvergEx: Fraud / Compliance Analytics
• Situation: Rule 15c3-5 Compliance Requirement
– VortEx, the BNY ConvergEx dark liquidity agency brokerage, required a new risk and compliance application to comply with SEC Rule 15c3-5
• Problem: Existing Commercial Products Lacking
– Most existing products are not designed to provide an effective, intuitive way of filtering and investigating large amounts of real-time events.
• Solution: CRSS (Compliance, Risk, Surveillance System), Based on TIBCO StreamBase
– Developed and deployed the CRSS which provides a simple, intuitive, ad-hoc way to view and investigate real-time events.
Trading Fraud Detection Algorithms
Layering
Marking-The-Close
Securities that have 2 million shares or less in 30-day ADV and 10 or more cancels (on the opposite side of the market) by a client within a 3-second rolling time window
Securities that have 1 million or less in 30-day ADV and execution prices or order price less than or greater than the Last Trade Prices at 3:50PM by 10%
Large Orders Identify orders and / or executions when Order Quantity exceeds the 30-day average daily volume (ADV) by a predefined percentage.
Ramping on Close
Ramping on Open
Spiking
Identifies Order Limit Price or Execution Price that exceeds todays Open Price by a predefined percentage
Identifies Order Limit Price or Execution Price that exceeds previous days Close Price by a predefined percentage
Identifies Order Limit Price or Execution Price that exceeds Last Traded Price by a predefined percentage
Spoofing Securities that have 2 million shares or less in 30-day ADV and has 25 or more cancels between 9:25AM and 9:29:59AM
Wash Sales Securities that have 1 million shares or less in 30-day ADV and have the client simultaneously buying and selling the same quantity within a 3 second time window
The Bank New York / ConvergEx Solution
TIBCO Trade Correlation
Monitoring
Compliance Risk Surveillance
System
Key Performance Metrics • Over 100 million orders/day with 90% cancellation rates • Over 500 million messages/day • Peaks of 100,000 messages/second • Market data rates of 500,000 messages/second
Streaming Analytics Continuous time-bound windows for real-time analysis
Trade Cancels
“WHEN any security with 2 million shares or less in 30-day average daily
volume AND 10 or more cancels by any client within a 3-second TIME WINDOW, send a HIGH PRIORITY ALERT”
5 3 10 Aggregation
Real-Time Correlation Based on Timing and Windows
Trade Orders
Trade Executions
Trade Cancels
Average Daily Volume
“WHEN any security that has 2 million shares or less in 30-day ADV AND has 25 or more cancels
BETWEEN 9:25AM AND 9:30AM, log as possible Spoof and STOP TRANSACTION”
ALERT
9:25…………………………… 9:29:59
50
“TIBCO makes it easy for our operations staff to whittle
100 million trading events down to the hundreds a day matter. That enables us to fix compliance issues during the day instead of waiting for the end of the day to do a forensic dig.”
- Bruce Osterover, Head of IT, BNY ConvergEx”
The tools and technologies to put in place
to get BIG VALUE from your Fast Data!
Fast Data
In Memory
Data Grid
Event
Server
Enterprise
Service Bus
Enterprise
API
Streaming
Analytics
LiveView
Dashboards
Want to Play With It?
Fast Data Product Evaluation Downloads http://tap.tibco.com/ • TIBCO BusinessWorks • TIBCO Enterprise Message Service • TIBCO BusinessEvents • TIBCO ActiveSpaces • TIBCO Spotfire • TIBCO Web Messaging for TIBCO EMS • TIBCO FTL® / TIBCO eFTL™ • TIBCO StreamBase
• More Products being added to download site all of the time
