the unsung hero of big data in manufacturing: unstructured content
DESCRIPTION
Presented by: Rik Tamm-‐Daniels, VP Technology, Attivio TIBCO Spotfire and Teradata: First to Insight, First to Action; Warehousing, Analytics and Visualizations for the High Tech Industry Conference July 22, 2013 The Four Seasons Hotel Palo Alto, CATRANSCRIPT
The Unsung Hero of Big Data in Manufacturing: Unstructured Content
Rik Tamm-‐Daniels, VP Technology TwiBer: @riktammdaniels
Proprietary
Big Data
Big Data encompasses geEng insight and making smarter decisions from any informaFon that is not easily tapped using
tradiFonal BI and AnalyFcs technology stacks.
Source: 'Big Data' Is Only the Beginning of Extreme Informa7on Management, April 7, 2011, Gartner Group
Proprietary
Big Data
Structured Data
Unstructured Data
Unstructured Data
Unstructured Content
Beyond Structured vs. Unstructured
<xyz> </xyz>
1-‐2-‐2013 abc/bed/fnd 87 998 1-‐2-‐2013 abc/bed/fnd 87 998 1-‐2-‐2013 abc/bed/fnd 87 998 1-‐2-‐2013 abc/bed/fnd 87 998
57° 64° 71° 120°
Understanding the nature of data and how to get insight from it is fundamental to succeeding with Big Data
Proprietary
Unstructured Data
• Brings the promise of inferring customer intent, idenFfying paBerns of behavior and predicFng future acFons
• But gaining this insight requires large amounts of raw data, much of which is noise, to get what amounts to indirect insight
1000 rows of impression logs to get 1 click-‐thru; 6 months of click data to be able to develop staFsFcally meaningful user segments
Proprietary
Unstructured Content
• Consider this CRM case: Customer: Joe Customer [email protected] Product: Mobile Phone X1000 Summary: BaBery is dying aber only 1 hour of use Date: 3-‐1-‐2013 01:35:00 Comments: I just bought the new X1000 because I saw it had fantasFc reviews online, but aber only one week, the baBery dies aber just an hour of use. This is completely unacceptable and I’ve been unable to get help from ACME mobile where I bought the phone. They keep telling me that depending on the apps I’m running, this might be expected. I’m only running the basic pre-‐installed apps! Also, I’ve noFced that the screen is always super bright and starts to hurt my eyes aber 20 minutes of use. I would really appreciate it if you could send me a new phone or tell me how to fix mine.
Proprietary
What does this single CRM case tell us?
Customer: Joe Customer [email protected] Product: Mobile Phone X1000 Summary: BaBery is dying aber only 1 hour of use Date: 3-‐1-‐2013 01:35:00 Comments: I just bought the new X1000 because I saw it had fantasFc reviews online, but aber only one week of use, the baBery dies aber just an hour. This is completely unacceptable and I’ve been unable to get help from ACME mobile where I bought the phone. They keep telling me that depending on the apps I’m running, this might be expected. I’m only running the basic pre-‐installed apps! This is completely unacceptable from a retailer selling your products. Also, I’ve noFced that the screen is always super bright and starts to hurt my eyes aber 20 minutes of use. I would really appreciate it if you could send me a new phone or tell me how to fix mine.
Customer email Customer name
PosiFve SenFment
NegaFve SenFment
Retail Outlet
PotenFal Liability
Key concept
Industry term
Proprietary
What’s the business value of this CRM case?
• When a help desk user gets assigned this case, their view could be: Customer: Joe Customer Email Address: [email protected] Summary: … Case: I just bought the new X1000 because I saw it had fantasFc reviews online, but aber only one week of use, the baBery dies aber just an hour of use. This is completely unacceptable and I’ve been unable to get help from ACME mobile where I bought the phone. They keep telling me that depending on the apps I’m running, this might be expected. I’m only running the basic pre-‐installed apps! Also, I’ve noFced that the screen is always super bright and starts to hurt my eyes aber 20 minutes of use. I would really appreciate it if you could send me a new phone or tell me how to fix mine.
Related CRM cases: Screen and baBery issue … Screen too bright ….
Issue trend line for screen and baBery issues for the X1000
Proprietary
What could the value of a lot of CRM cases be?
LifeFme Customer Value
% NegaF
ve Customer
Service InteracFon
s
ACME Mobile
PhoneX
CoolPhones
Joe’s Phones
BigBell
MallMobile
Percent nega.ve / posi.ve per retailer
BaBery dies
Apps I’m running
Screen
Super Bright
Proprietary
Unstructured Content in Customer Experience Management
Every item from every one of these channels is your customer telling you directly what they like, what they don’t like, where they would like to see you take your products, etc…
Proprietary
Unstructured Content is Everywhere!
CRM case notes Email Surveys
Warranty claims
Product manuals
Maintenance reports
Online reviews Social Media
Unlocking Unstructured Content
Proprietary
Unlocking Unstructured Content
• Fundamental: Structure the Unstructured • For Unstructured Content this is done through Text AnalyFcs: – SenFment Analysis – ClassificaFon – EnFty ExtracFon – Key Concept Analysis – Taxonomies and Ontologies
New Dimensions/New Insights
Key Concept Analysis
EnFty ExtracFon
SenFment Analysis
Proprietary
Enterprise Text AnalyFcs Spectrum
Taxonomies/ Ontologies
Gazeteer/ Dic.onary-‐Driven En.ty Extrac.on
Pa@ern Based En.ty Extrac.on
Machine-‐Learning:
Sen.ment & Classifica.on
Sta.s.cal En.ty
Extrac.on
Language Model-‐Based Keyphrase Analysis
Directed Discovery
Proprietary
Enterprise Text AnalyFcs in PracFce
• IteraFon, iteraFon, iteraFon – the faster you can iterate, the greater the ROI
Text Analy.cs are Itera.ve
Proprietary
Enterprise Text AnalyFcs Best PracFces
• Domain-‐specific analyFcs • Extensible/flexible frameworks • Directed and Discovery text analyFcs • Agile data environments
– Text analyFc data can produce high-‐degrees of cardinality – Metadata will be variable by “row”/“object”
The Big Picture
Proprietary
Big Data Maturity Model
Level 0: Current State
Level 1: Single source of Big Data Analyzed
Level 2: MulFple types of Big Data sources Analyzed
Level 3: Unified Big Data Architecture
Looking at a single source of Big Data: unstructured data or unstructured content
Looking at mulFple sources of Big Data: unstructured data and unstructured content
Single point of access for all Big Data access and analysis with variety of access modes to support variety of business cases
Structured data analysis: EDW + BI
18
Proprietary
• Unified informaFon access plauorms provide single point of access to informaFon from mulFple sources, integraFng and finding relaFonships across sources – Efficiently combines features of database, business
intelligence and search technologies in a single architecture in real-‐Fme
Unified InformaFon Access
Unified Informa.on Access
Note: IDC June 2012.
Unified Informa.on Access & Analysis
Content Analy.cs
Databases Search and Discovery
Decision Management
Business Intelligence
Data Warehouses
Unified Informa.on Access & Analysis
• Serves as foundaFon for new informaFon management and access stack for the enterprise
• May replace data warehouses if applicaFons require quick ad hoc access to collecFons of heterogeneous informaFon
• Will eventually replace the tradiFonal enterprise search engine
Key Disrup.ve Elements
Customer Experience Management for Complex Device Manufacturers
Proprietary
Case Study Global Manufacturer
• Create new BI platform to proactively manage customer engine fleets at new level of breadth and depth of detail beyond just data trends
• Greatly improve efficiency, customer satisfaction and repeat business
• Analyze Everything platform for complete agile BI: integrates, correlates and presents data and content, with no advance data modeling required:
• Engine sensor data, generated in “Big Data” volumes
• Service status data, quality metrics, CRM and other databases
• Customer case management notes
• Engine maintenance system notes by service technicians
• Supports BI tools with native SQL support & ODBC/JDBC connectivity
• BI pilot completed in just 5 weeks – “a new standard for BI time to market”
• Managers analyze and discover new correlations between changes in engine KPIs, sensor data, recurring key phrases from service notes & more
• New insights into root causes behind service issues – not just the numbers
• “No data left behind…Time from ‘data to decision’ drastically reduced”
Pro
blem
W
hy A
IE?
Res
ults
Proprietary
SEARCH API SQL over ODBC/JDBC
QUERY/RESPONSE WORKFLOWS
INGESTION WORKFLOWS
CONTENT API
TIBCO Spotfire
DATA & CONTENT CONNECTORS
UNIVERSAL INDEX
EDW Customer
Service Email CRM
OperaFonal Events
Device Generated Data Complex Event
Processing Engine Maintenance Reports
Textual data is enriched with text analyFcs (senFment, keyphrases, enFty
extracFon)
CUSTOMERS DEVICES OWNED EMAILS CRM CASES OPERATIONAL EVENTS
KPIs
Dashboards using TIBCO Spouire contain a mix of in-‐memory tables loaded from AIE with on-‐demand detail
drill down
Logical Tables from the BI tool
perspecFve, AIE does not store data in physical tables
TIBCO Spouire and AEvio AIE
Proprietary
Benefits of combining AEvio and TIBCO Spouire
VisualizaFon of unstructured content and data that does not currently exist in the BI stack, providing criFcal business process and analysis context
Rich Text AnalyFcs to gain insight from unstructured content that can be visualized in Spouire
Full-‐text search within Spouire Dashboards -‐ providing a complete Data Discovery experience
Agile, complete, UIA technology stack
Proprietary
Spotfire Server
Extracted or On-‐Demand via JDBC
AIE and TIBCO Spouire Reference Architecture
Direct via ODBC
Spotfire���Professional
Spotfire���Web Player
Hadoop A/RDBMS/EDW Web Server File Server Email Server CMS, CRM, ERP
ACTIVE INTELLIGENCE ENGINE (AIE)
Proprietary
Learn more
AEvio Big Data Resources
hBp://go.aEvio.com/analyze-‐everything/
Proprietary
Thank you!