innovation of social value through big data - pldt...
TRANSCRIPT
Innovation of social value through
Big Data NEC analysis technologies for discovering hidden value
March 2015
Yoshiki Seo
Big Data Strategy Division, NEC
Page 2 © NEC Corporation 2014
NEC Vision “Innovative Social Infrastructure”
Traffic solutions Energy solutions Natural-resource solutions Financial solutions Medical solutions
ICT
ICT ICT
ICT
ICT
ICT
ICT that organically integrates individual infrastructure to be
the foundation of a future society that creates efficiency and
fairness in all regions and communities
Airport
Undersea Towers
Retailers Bank
Distribution
bases Trains
Companies
Seaport Post office
Broadcaster
Important
facilities
Hospital
Fire dept.
Municipal
offices
Dam/Water
Roads
Tele-
communications
Factories
Change Human Life through “Data Driven Approach”
Page 3
Collection of large-scale data
Analysis and prediction
Solution of social issues
• Invariant analysis
• Heterogeneous mixture learning
• Facial image analysis
• Behavior analysis
• Textual entailment
recognition
• Surveillance cameras
• Smart devices
• Diverse sensors
SDN : Software-Defined Networking
・Network virtualization
・Cyber security
Diverse sensors and human interface technologies
High-performance/high-
reliability core IT technologies
Next-generation network technologies
CLOUD BIG
DATA
SDN
* Rated as No. 1 among organizations participating in an evaluation task organized by the U.S. National Institute of Standards and Technology (NIST)
Unique
Unique
Unique
No.1
No.1
*
*
From the seafloor
to outer space
World’s first SDN switches
Essential to future
information system
Innovation of Social Infrastructure via ICT
Leveraging information captured by our unique and highly competitive ICT assets
to become a social value innovator
• Accumulated data
© NEC Corporation 2009 Page 4
NEC’s Big Picture of Smart Water Solution
Integration
Control
Prediction / Diagnosis
Detection
Demand Prediction
Water Purification
Management
Energy Power
Saving Operation
Deterioration Analysis Monitoring usage at home
Remote valve
control
Predictive diagnosis
Maintenance
Quick repair
Monitoring Sensor
of water leakage
Life extension of water pipe
Page 5
Monitoring of Social Media
I’m smoking 420*!
Cities can be safeguarded by preventing terrorism and accidents through
the analysis of video and audio data captured using surveillance cameras, as well as integrated analysis of textual data (e.g. twitter) and sensor data
Example of APAC Safer Cities Trial
Detection of abnormalities in communities using video surveillance. Discovery of calls for demonstrations via
twitter through cyber information monitoring. → Integrated analytics engine used to extrapolate the likelihood and
objectives of illegal demonstrations and alert the authorities.
Example
Video surveillance (Facial image analysis)
BlogsTwitterBlogsTwitter
Com
ple
x E
vent P
rocessin
g
Event detection
(Acoustic analysis)
Detection of other events
(Image analysis of
fingerprints etc.)
Cyber
information monitoring
(Recognizing
textual entailment) RISK Events
Disasters Hazardous materials
Fights Crime
Demonstrations
*420: Slang term for cannabis
© NEC Corporation 2014
Page 7 © NEC Corporation 2014
The Process of Delivering Social Benefits from Big Data
Real world
Sensing Actuation/
Optimization Social
benefits
Anticipation/predictions;
decisions
Cyberspace
Remote sensing
Vibration
sensors
Mobile
sensors
Open data
Social sensing
Face
authentication
Language
analysis
Voice
recognition Information
systems
Human sensing
Co
llec
ting
A
cc
um
ula
ting
Au
tom
atic
eva
lua
tion
s
Eva
lua
tion
as
sis
tan
ce
Heterogeneous-
Mixture
Learning
Textual-
Entailment
Recognition
Simulation
Technologies
Automatic
control
Robotics
Automatic
operation
Quality-
improvement
measures
Resource
planning
Digital
signage
Incentives
Analytics
Invariant
Analyzer
RAPID Machine
Learning
© NEC Corporation 2015
Overview of NEC’s Big Data Offerings
Page 8
Creating solution menu for each domain based on advanced field-proven use cases and
consolidate in a structured manner including Platform
サービス提供
, Isilon
NEC Big Data Solutions
オペレーション 高度化 / 最適化
Operation Advancement/Optimization
情報管理の強化、 犯罪・不正の検知
Enhancing Information Governance, Detection of Crime/Fraud
製品 / サービス 価値向上・改善
Product/Service Improvement in value
顧客獲得・維持、 販売促進
Acquiring & keeping customers, Promotion
Platform
SDN
SDN Product
(UNIVERGE PF Series, etc.)
IaaS
NEC Cloud IaaS
Professional Service
Solution Menu
- Monitoring & Predicting Failures in Plant
- ICT in Agriculture - M2M for
Manufacturing
- Product Demand Prediction/Automatic Ordering
- Energy Demand Prediction - Demand Prediction of Repair
Parts - Quality Analysis - Demand Prediction (SAS)
- Human Resources Matching - Facial Recognition
Technology - Customer
Analysis/Campaign Management
- EBM Analysis - Web Access Analysis
Analytics Technology
SAS MicroStrategy Dr.Sum Oracle BI Business Objects MS SQL BI SAS 等 ISV 製品
Invariant Analysis
Rapid Machine Learning
Heterogeneous Mixture Learning
Text Inference
Recognition
Server Storage: iStorage
Network Operation & Maintenance
Real time Event
Processing (Oracle CE
P)
Scale-out DB (IR
S)
Parallel Distributed Processing
(Hadoop)
In-memory DB
(SAP HANA DataBooster)
Ultra High SpeedDWH
(Data Platform for Analytics )
ETL (PowerCenter
DataCoordinator)
Parallel Integrated DB(Oracle
Exadata Oracle DB SL)
M2M Platform(CONNEXIVE
)
Security (Anonymization
Technology, Anonymized Calculation)
Memory DB (TAM)
Big Data Discovery
Program
Big Data Education Program
Platform Planning Service
- Enhancement of Information Governance
- Medical/Healthcare - Vehicle Traffic Control - Big Data Archive
Optimization
© NEC Corporation 2015 Page 9
1.小売業における需要予測 Value 1
Detection of abnormalities in facilities
Invariant Analysis
Page 10
* Collection and analysis of large volumes of sensor data to detect when
operations are “different than normal”
Detection of abnormalities in facilities
Social infrastructure
(Bridges, expressways, etc.)
IT systems, data centers, telecom
networks Power plants
Manufacturing plants (Assembly, chemical, etc.)
Automobiles, trains,
aircraft, ships, etc.
Detection of signs of failure and/or abnormalities in domains in
which failures could have a high economic and/or social impact
Sensor data Operation logs
Provision of safety and security solutions that contribute
to society
Page 11
Visualization of “normal”
operations
[Invariant model]
Detection of signs indicating that
operations are “different than
normal”
[Real-time failure detection]
Mechanical and automatic
visualization of all relationships
between each sensor data
Comprehensive viewing of all
relationships enables
abnormalities to be detected
Detection of abnormalities by comparing past data and
real-time data
Invariant Analysis Technology
Detection of
abnormalities
Page 12
Effectiveness of “Invariant analysis technology” for Large-scale Plants confirmed through field trials at
Shimane Nuclear Power Station
Chugoku Electric Power Co., Inc.
No. of sensors per power plant: 3,500
100 of data from each sensor in a second
Collaboration between experts in power
plant operations working for the customer
and NEC’s own analysts.
Discovery of signs indicating that operations
are “different than normal” from correlations
between 3,500 x 3,499 sets of sensors
Advanced Analysis
Real-Time
On-site know-how
Page 13 Page 13 © NEC Corporation 2013
1.小売業における需要予測 Value 2
Demand forecasting
Heterogeneous Mixture Learning
Page 14 © NEC Corporation 2014
Heterogeneous-Mixture Learning Technology
Sunday
Monday
Tuesday
Saturday
⁞
Sunny
Cloudy
Snowy
Condition A=Y
Condition A=N
Condition B=Y
Condition B=N
Sunday
Monday
Saturday
⁞
Day
Night
Day
Night
Day
Night
Conventional method: categorizing patterns manually
Trying to
categorize
data by day
Trying to
categorize data
even more
minutely
Trying to
categorize data
by weather
Heterogeneous
mixture of data
Hard to find accurate
patterns if data types
mixed
○
○
○
○
⁞
Manual categorization methods limited
to trial and error and inaccurate for
categorization
⇒ High-precision prediction difficult
○
Efficient trial-and-error process
achieved through automatic
categorization preventing
inaccurate prediction
⇒ High-precision prediction
achieved
○
⁞
NEC’s new technology: heterogeneous-mixture learning technology
Automatically categorizing data patterns into classes
Page 15 © NEC Corporation 2013
Fresh products
Disposal loss problems in the retail industry
�Á�ï�
ú�À
Due to short shelf-lives
and high frequency of
orders, losses due to
disposal of unsold items
significantly affect costs.
Food retail
Appropriate product demand forecasting is strongly required.
Calendar attributes Weather changes
Human judgment based on intuition and experience has limitations due to the large number and
complexity of products.
Changes according
to day of the week
Page 15
Disposal loss problems in the retail industry
Page 16
Use of heterogeneous mixture learning technology to predict
demand for goods to be delivered after three days
From verification result calculations, a 30% reduction in losses was achieved,
compared to previous purchases of cream puffs
Verification
Analysis example
Discovery of a negative correlation between cream puff sales
trends and the minimum temperature
Cream puff sales fall when the minimum temperature rises
0 Negative correlation
Correlation between cream puff sales and explanatory factors
Positive
correlation
Minimum temperature
Same category sales
Same product sales
Page 17 © NEC Corporation 2013
Customer’s Problem
Existing prediction model can not handle dynamic changes in the market, and
Error Ratio is significantly high. Low Profit & High Operational Cost
Conditio
n &
Para
mete
rs
Input
Pric
e
Pre
dic
tion
Dete
rmin
e
Tra
de-in
Pric
e
Oth
er
Consid
era
tions
Investig
ate
Pro
ducts
Current Customer’s Price Prediction for Trade-in CertainProducts
Auction Price Prediction using Heterogeneous Mixture Learning
NEC is now proposing a new Price prediction system using Heterogeneous mixture learning technology.
Page 18 © NEC Corporation 2014
Churn Retention Cycle
collect
design
predict
act
evaluate
optimize
understand
implement
customer’s churn
customer segmentation
statistical analysis
predictive modeling
KPI
campaign/promotion
Plan/bundle/package
loyalty program
churn propensity
resource prediction/allocation
campaign policy
KPI review
update prediction model
call operators’ performance
Call/Text/Chat/IVR
Page 19 © NEC Corporation 2014
Big Data Drives Churn Management
Call Detail Record
Users’ Profile
Billing History
Churners’ History
Campaign
Promotion
Price Plan
Loyalty Program
Call Trace
HSS/HLR
Performance
Counter
inbound
outbound
IVR
Chat Churn
Management
Page 20 © NEC Corporation 2013
Fair price forecasting
Deterioration forecasting
Product demand forecasting
Power demand
forecasting
Forecasting of members’
purchasing trends
Applications for heterogeneous mixture learning technology
Enables the realization of demand forecasting, adjustments
and various forecasting solutions for society:
Page 20
Page 22
Textual Entailment Technology
Based on an understanding of the total sentence meaning rather than single words, more
sophisticated analysis and use become possible independently of differences in expressions
×
×
○
○
○
○
In emails, sentences, and daily reports, expressions that can create suspicions of dishonest transactions
(bid-rigging, etc.) are automatically monitored, and a warning is sent to the writer.
Adjusting the sales price with
other companies in the same
industry
Expect cooperation of other
companies in the same industry
regarding the price decision
Adjust other companies
and price
To adjust the sales price within the
company, while adjusting the
prices of similar products of
other companies
Problematic expressions (semantic)
that need to be extracted Sentences
recognized
Correct
understanding
Control needed
Incorrect
Understanding
Control
unnecessary
(leak)
Incorrect
Understanding
Control
necessary
(False alarm)
Correct
understanding
Control needed
Correct
understanding
Control needed
Correct
understanding
Control unnecessary
Correct
understanding
Control needed
Correct
understanding
Control needed
Correct
understanding
Control
unnecessary
(Conventional)
Word based
recognition result
NEC
Text inference
recognition result
Ex.
Differences between conventional word-based recognition and NEC textual inference
recognition
NEC textual inference
recognition Takes into consideration the importance of words and their semantic correspondences as well as the sentence structure involving
the subject, predicate, etc. It recognizes relations between these two types of meanings in a sentence.
Page 22
Information Governance Usecase
Document
Score
Doc A 0.9
Doc B 0.7
Doc C 0.55
Doc D 0.48
Doc E 0.33
Has
required
information
Writers do not always have full knowledge of
document management standards and thus are
not fully competent to judge on their own
Conventional
method (Human judgment)
NEC method
(Human +
machine
judgment)
Decision on need for information control by management
Document author alert
Normal
processing
▌Because the number of documents is so large,
it is difficult for a few document managers to
check all of them.
Problem
1
Emails
Sales
reports
Information
control
required
Information
control
not
required
Has
required
information
Only all suspicious documents are
checked,
and efficient, strict management is
achieved
Encourages
verification with
writer, reduces
judgment mistakes.
Effect
2
Effect
1
Busines
s
law
violation
Informatio
n
leakage
Documen
t
detection
Confirmation by authorizing organization
Emails
Sales
reports
Automatic decision whether required
management information is included in the text Solution
Problem
2
Page 23
Page 24
Public risk detection
Expansion of applications to compliance enhancement
and customer voice analysis
Compliance enhancement
Areas of application for textual inference recognition technology
Customer voice analysis for call centers
Page 25 Page 25 © NEC Corporation 2013
1.小売業における需要予測 Value 4
Matching solution
RAPID Machine Learning
RAPID Machine Learning Original
Technology Deep Learning Engine optimized by NEC
achieving high speed and light processing
High Speed and Light Precious analysis
Global No.1 Decreasing error rate
Analyzing unstructured data such as images and texts!
competitor RAPID
accuracy 97.24 % 97.29 %
Processing Time (sec)
833 4
Memory 2200 MByte
32 MByte
Increase processing speed while
keeping accuracy and small
memory
RAPID can deal with big data easily and improve accuracy
Collaboration Filter
RAPID
parameters 9000 210000
Error rate 29% 14%
Page 26
Human Resource Matching Solution
Page 27
Registration RAPID Machine Learning
Job seekers (Students, seniors, career
changers, workers from overseas)
Placement agencies Companies
seeking employees
Job seekers
Data
Blogs
Information
on job
listings
NEC independently developed RAPID machine learning from deep learning technology
Entry
Companies suited
to each job seeker
Top class personnel
that match corporate needs
Page 27
Page 28
Matching of employers
and people Customer behavior analysis
Areas of application for RAPID machine learning
Image monitoring
of urban areas※ Tourism matching
Development of numerous test projects in each business domain Expansion of matched to needs and test results
Future Development Plan
Upgrading and optimization of
operations
Public risks Product & services risks
Monitoring of urban
areas
Prediction of energy demand
Optimal distribution of
resources
Prediction of transaction
prices
Page 29
Quality control for transportation
vehicles
Manufacturing quality control
Detection of infrastructure
abnormalities
Monitoring of plant
repair predictors
Strengthening of information governance
Human resources matching
Automatic issuance of orders based on demand predictions
NEW
NEW
NEW
NEW
Strengthening of information
management, Detection of crime
and fraud
Kaizen, improvement in
value of products & services
Matching of products &
people
Tourism matching
Customer behavior analysis
Customer acquisition
and support, sales promotion
© NEC Corporation2013 Page 30
NEC Group Vision 2017
To be a leading global company
leveraging the power of innovation
to realize an information society
friendly to humans and the earth