fujitsu laboratories’ r&d targeting intelligent computing
TRANSCRIPT
0 Copyright 2015 FUJITSU
Human Centric Innovation
in Action
Fujitsu Forum 2015
18th – 19th November
1 Copyright 2015 FUJITSU
Fujitsu Laboratories’ R&D Targeting Intelligent Computing
Dr. Hirotaka Hara Director and Heard of Knowledge Information Processing Laboratories
Fujitsu Laboratories Ltd.
2 Copyright 2015 FUJITSU
1. FUJITSU AI Technology Brand “Zinrai”
2. Systematizing AI technologies and deploying them for products/services
3. Cutting-edge AI technologies developed by Fujitsu Laboratories
Agenda
3 Copyright 2015 FUJITSU
FUJITSU AI Technology Brand “Zinrai”
1.
4 Copyright 2015 FUJITSU
Enables customers’ digital transformation Incorporates cutting-edge cloud, mobile, big data, IoT, and AI
technologies
AI: Artificial intelligence IoT: Internet of Things SoE: Systems of Engagement SoR: Systems of Record
FUJITSU Digital Business Platform “MetaArc” Announced in
September 2015
Existing information systems (SoR)
Work efficiencies
Cost reductions
Digital transformation of business (SoE) Transformation of existing business processes
Creation of new products and services
Ecosystem of multiple companies
Digital Business Platform “MetaArc” Cloud
Links between SoE and SoR
IoT Big Data Mobile AI
5 Copyright 2015 FUJITSU
AI technology development activities
Applied for more than 100 AI-related patents since 2008 Technology
▲Risk analysis prediction software “QRMining”(2011/12)
~2013 2015
▲SCM model predictive control(2014/3)
▲Analysis scenario recommendation(2012/8) ▲Predictive model generation from massive datasets (2015/9)
▲Jointly sets up center of excellence in Singapore(2014/10)
▲Establishes joint research unit on mathematical techniques with Kyushu University (2014/9)
2014
▲Field trial at Fukuoka Airport(2015/9)
▲Participates in Artificial Brain Project (2012/9)
▲Voice search software “VoiceTracking/KeywordFinder”(2009/2)
▲Synthesis of various voices(2014/3) ▲Chinese handwriting recognition
exceeds human levels(2015/9)
▲3-D gesture analysis(2013/5) ▲Human flow recognition(2015/3)
▲Gaze tracking “EyeExpert”
(2015/10)
▲Revolutionize access to Open Data (2013/4)
▲Assisted independent living project in Ireland “KIDUKU” (2013/6) ▲Early detection of motor function anomaly(2015/3)
▲Professional baseball game image search service
(2015/4)
▲Spotting applicable areas in material from voice (2015/4)
▲High-speed malware detection(2014/4)
▲Identifying users vulnerable to cyber attack based on behavioral and psychological characteristics(2015/1)
▲Medical image diagnosis(2015/2)
▲Automatic generation of image analysis program(2014/9) Technology
development
Collaboration
▲Consumer behavior analysis service “Do-Cube” (2014/3)
▲Small-enough gaze tracking(2012/10)
▲Automatic tagging of scenes(2010/11)
▲Phone scam detection(2012/3)
6 Copyright 2015 FUJITSU
PoC: Proofs of Concept PoB: Proofs of Business
Customer analysis /Marketing About 60 cases
Traffic/Disaster
About 40 cases
Top 8 fields of PoC/PoB
Product traceability About 30 cases
Visualizing factories About 30 cases
Watching over the elderly and children About 30 cases
Equipment surveillance /maintenance About 20 cases
Flow analysis of store customers About 20 cases
Advanced agriculture /livestock farming About 30 cases
About 300 PoC/PoB are in action
Collaborations with customers toward digital innovation
7 Copyright 2015 FUJITSU
What AI Fujitsu aims to provide
AI that is in collaboration with
people and human centric
AI that grows continuously
AI that can be incorporated in products/services and provided
8 Copyright 2015 FUJITSU
Meaning Being agile and intense
Idea behind the brand name
Realizing innovations in business and society dynamically by supporting people’s decisions and actions speedily
FUJITSU AI Technology Brand
9 Copyright 2015 FUJITSU
Systematizing AI technologies and deploying them for products/services
2.
10 Copyright 2015 FUJITSU
People/Businesses/Society
Systematizing AI technologies of Fujitsu
Sensing Actuation
Sensing and Recognition
Image recognition
Voice recognition
Emotion/state recognition
Knowledge processing
Natural-language processing Knowledge processing & discovery
Pattern discovery
Decision and support
Inference & planning Prediction & optimization Interactivity & recommendation
Social receptivity Simulation Neuroscience Advanced research
Machine learning Reinforcement learning Deep Learning Learning
11 Copyright 2015 FUJITSU
Knowledge processing
Sensing and Recognition
Decision and support
Strengths of Zinrai
Learning technology Supports continuous growth of AI by drawing useful knowledge and patterns through daily learning
Chinese handwriting recognition
Detecting cyber attacks
Preventing resignations Stock optimization
Affective media processing technology
Knowledge processing
Mathematical technology
Uses five senses like people and processes people’s feelings, recognition, and consideration
Understanding human emotions
Phone scam detection
Creates knowledge that not only people can understand but also computers can process
Medical decision -making support
Improvement of banking supervision
Finds solutions to public or business problems using a supercomputer Can a robot pass
the Univ. of Tokyo entrance exam?
Tsunami inundation prediction Gaze tracking
Finger-operated interface Call centers
Visualizing regional features
Restoration planning
Mitigating congestions
12 Copyright 2015 FUJITSU
Provides Zinrai as a service on top of Digital Business Platform “MetaArc”
Deployment for “MetaArc”
IoT: Internet of Things SoE: Systems of Engagement SoR: Systems of Record
Existing information systems (SoR)
Work efficiencies
Cost reductions
Digital transformation of business (SoE) Transformation of existing business processes
Creation of new products and services
Ecosystem of multiple companies
Digital Business Platform “MetaArc” Cloud
Links between SoE and SoR
IoT Big Data Mobile Zinrai
13 Copyright 2015 FUJITSU
Deployment for products/services
Provides products and services with Zinrai technologies incorporated
Industry/Business Applications
Security
Manufacturing
Healthcare
Marketing
City/Traffic
Finance
Products/ Services
14 Copyright 2015 FUJITSU
Application of anomaly detection Fault prognosis of plant facility using huge sensor data
Prevent sudden halt of production line and improve productive efficiency
Apply machine learning to generate 'normal state model' from sensor data
Detect ‘anomaly’ (unusual state) based on deviance from normal state model
Big Data Solution for Anomaly Detection powered by Zinrai
Machine Learning Modeling
Apply machine learning technology to multiple (tens of thousands) series of sensor data to model normal state of machinery/facility
Accumulated Sensor Data
Real-time Anomaly Detection Operation
Normal State Model Anomaly Detection Apply real-time sensor data to ‘normal
state model’ and detect precursor of failure
Real-time
Sensor Data Alert
15 Copyright 2015 FUJITSU
New service-based solutions: AI Application Consulting Availability date: December 2015
Customer Finance
Public/Government
University/ Research institution
Manufacturing
Distribution
Medical etc.
Fujitsu
AI Application Consulting Department Formed on November 1, 2015
Researcher Engineer Curator
Total of 200 people
Creation of new products/services
Reform of existing business
AI-specialist consultants co-create innovation together with the customer
“Co-creation”
16 Copyright 2015 FUJITSU
Cutting-edge AI technologies developed by Fujitsu Laboratories
3.
17 Copyright 2015 FUJITSU
Why AI now?
1950 1970 1980 1990 2000 2010 1960
★”Artificial Intelligence” (Dartmouth Conf.)【’56】
★Big AI projects (Japan ICOT ’82, US MCC ’83, UK Alvey ’84)
★AI failure【’73】 (Lighthill report)
★DEC R1 【’82】
★Victory in chess 【’97】
★ELIZA 【’64】
★Turing test【’50】
★Bayesian network【’88】 ★Lisp【’58】 ★Prolog【’72】
★Back propagation【’86】 ★Perceptron【’62】
Projects
★Deep Learning in limelight【’12】
★Brain Science projects (BRAIN Initiative, Human Brain)
★Japanese version of “Siri”【’12】
1st AI era
Search and reasoning Knowledge
Systems
★MYCIN 【’73】
Paradigms
2nd AI era 3rd AI era
Machine learning
Could not solve practical problems Ended in 1st AI winter
Technologies are advancing rapidly (Big data × Computer power × Machine learning). Practical applications are now possible.
Could not teach experts’ knowledge Ended in 2nd AI winter
18 Copyright 2015 FUJITSU
Knowledge processing
Sensing and Recognition
Decision and support
Strengths of Zinrai
Learning technology Supports continuous growth of AI by drawing useful knowledge and patterns through daily learning
Chinese handwriting recognition
Detecting cyber attacks
Preventing resignations Stock optimization
Affective media processing technology
Knowledge processing
Mathematical technology
Uses five senses like people and processes people’s feelings, recognition, and consideration
Understanding human emotions
Phone scam detection
Creates knowledge that not only people can understand but also computers can process
Medical decision -making support
Improvement of banking supervision
Finds solutions to public or business problems using a supercomputer Can a robot pass
the Univ. of Tokyo entrance exam?
Tsunami inundation prediction Gaze tracking
Finger-operated interface Call centers
Visualizing regional features
Restoration planning
Mitigating congestions
19 Copyright 2015 FUJITSU
Deep Learning
Leaning Technology
Modeling
Nerve Cells (Neurons)
Neuron Synapse
Neural Network
A Mobile Robot SATORU-Kun (In 1988, Three-Layer NN)
29
232
Object Recognition in Image (In 2015, Seven-Layer NN)
1.1M
730M
# of Neurons
# of Synapses
38,000 times
3,150,000 times
The latest technology of neural network (NN)
20 Copyright 2015 FUJITSU
Deep Learning: Object Recognition in Image
Leaning Technology
1.1 M neurons 730 M synapses
Data:1.3M images 1,000 categories
Higher Accuracy than Conventional Machine Learning Automatic Acquisition of Object Features for Recognition
Projection Only
Demo
21 Copyright 2015 FUJITSU
22 Copyright 2015 FUJITSU
Deep Learning: Object Recognition in Image
Leaning Technology
1.1 M neurons 730 M synapses
Data:1.3M images 1,000 categories
Higher Accuracy than Conventional Machine Learning Automatic Acquisition of Object Features for Recognition
Projection Only
Demo
23 Copyright 2015 FUJITSU
Chinese Handwriting Recognition
Technology
Effect
Efficiency of processing of Chinese handwritten document
“Handwriting recognition” using Fujitsu original Deep Leaning
Achieved 96.7% of recognition accuracy that exceeds the human performance
Visualization of the features that are learned among nerve cells in the process of character recognition
Learned character sample generation based on 3D-Distortion
Purpose
Leaning Technology (Use Case)
24 Copyright 2015 FUJITSU
Application of Deep Learning
Learning Technology
Deep Learning
Marketing
Manufacturing
CRM
Finance
Medical Care
Urban transport
Image recognition/Video recognition/Voice recognition/Text recognition/Diagnostic imaging
Security
…
Future prediction/Automatic operation/Anomaly detection/Optimal control/・・・
25 Copyright 2015 FUJITSU
Machine Learning
Detecting Cyber Attacks Enterprise are continuously exposed to various cyber-attacks.
Sophisticated attacks, are operated behind major attacks.
It is difficult to detect novel sophisticated attacks with manually analyzing IDS log.
Enterprise System IDS
DoS (major)
Virus (major)
Scanning vulnerability(major)
Attackers …
Foretaste of attack
Sophisticated attack
Novel-type attack
IDS log (Big-data)
Various attacks are mixed
Security operation center
26 Copyright 2015 FUJITSU
Detecting Cyber Attacks: "Outlier clustering"
11 1 11 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 3 1 1 3 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 3 1 1 1 1 2 1 1 1 3 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 2 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 3 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 1 1 2 1 1 1 2 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1
3 2 3 1 1 11 1
11 1 1
1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 2 21 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 3 2 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 4 4 1 1 1 1 1 3 2 1 1 1 31 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 14 1 1 1 1 1 1 1 1 1
1 1 1 1 1 11 1 1 1 1 1
1 11 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 11 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1
1 1 11 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 11 1
11 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 11 1 1
Distribution of original data* Previous method
Our method
1 11 1 1 1 1 1 1
1 1 1 1 1 1 1 11 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1
1 11
1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1
1 1 11 1 1
Evaluating low-frequency data sets as “outlier clusters”
Simplifying and grouping frequently-appearing data
Attackers’ IP address
Port
num
ber
*Actually n-dimension data
Detecting low-frequently attacks with “outlier clustering” technique.
Machine Learning
27 Copyright 2015 FUJITSU
Detecting Cyber Attacks: Adopting for FUJITSU cloud
Machine Learning (Use Case)
Rare shape Same attack
Our method has extracted novel-type attack at short times.
“A novel type of distributed brute-force attacks” manually
extracted in three months last year.
Perceiving the contexts of “Outlier Clusters” with visualization.
Applying to security monitoring for FUJITSU cloud services.
Result of manually extraction Visualization of “Outlier Clusters ”
28 Copyright 2015 FUJITSU
Machine Learning for Massive Datasets
Machine Learning
Parallel Data Processing (Apache Spark)
Quickly Generating Accurate Predictive Models Latest technology delivers rapid highly accurate results
Learning from datasets of 50 million records in a few hours
Selecting learning algorithms and configuration combinations for the most accurate results in an automatic manner.
Analysis
Result Big
Data
Machine
Learning
Select machine config.
Select
algorithm
Select
operating
condition Assess
accuracy
2. Selection and Tuning
1. Estimation during execution
Performance
knowledge end time, accuracy
Combi-
nations Tune
sampling rate
model Sampling Training set
Most accurate
Test set
model
size
4000
Learning time
size
1000
size
2000
size
4000
size
8000
size
1000
size
2000
size
1000
size
2000
size
4000
size
8000
size
16000 size
32000
Now
Eliminate from
Candidate
Accura
cy
Current Best
Demonstrated
at L19
29 Copyright 2015 FUJITSU
Machine Learning for Massive Datasets
Machine Learning (Use Case)
Optimization
Energy Management
Weather
Power
Consumption the correlation
patterns
automatically learns and Data Sources produces predictive models
Humidity RPM
Voltage Temperature
sensors for components in
a factory
Predictive
Model
Predictive
Model
Failure
Prediction
Manufacturing bxwxwxwy nn 2211
sensor data
weight
30 Copyright 2015 FUJITSU
Affective Media Processing Technologies
Technologies that Sense and Understand You
A detailed user monitoring scheme realized by new visual sensing technologies and analysis algorithms.
Laser Sensing/Analysis 3D-sensing of user behavior from a distance
Gaze Tracking/Analysis Understanding users’ interests from a distance
Equip existing ICT systems with human visual systems
31 Copyright 2015 FUJITSU
Gaze Tracking and Analysis
Affective Media Processing Technologies
Long-range sensor for gaze tracking at a distance
Low cost, compact sensor for wide use in various applications
Gaze Database
By zooming in to track gaze, interests of customers browsing at a distance can also be understood
Multiple sensors are installed in-store to analyze which items are seen and compared.
Details ●●●●●●● ●●●●
Sensor Sensor
Sensor
Zoom in to track from several meters away
Info about item seen by customer
Use gaze data to analyze customer behavior
Store Display Sensor
1. Face parts detection
2. Eye region analysis & gaze calculation Pupil Corneal Reflection
Demo
32 Copyright 2015 FUJITSU
33 Copyright 2015 FUJITSU
Gaze Tracking and Analysis
Affective Media Processing Technologies
Long-range sensor for gaze tracking at a distance
Low cost, compact sensor for wide use in various applications
Gaze Database
By zooming in to track gaze, interests of customers browsing at a distance can also be understood
Multiple sensors are installed in-store to analyze which items are seen and compared.
Details ●●●●●●● ●●●●
Sensor Sensor
Sensor
Zoom in to track from several meters away
Info about item seen by customer
Use gaze data to analyze customer behavior
Store Display Sensor
1. Face parts detection
2. Eye region analysis & gaze calculation Pupil Corneal Reflection
Demo
34 Copyright 2015 FUJITSU
Affective Media Processing Technology (Use Case)
Understanding users’ feelings
Objective
Technology
Future development
Customer service improvement and automation Combines data from multiple sensing technologies to infer users’ mental state (intent, emotions)
Service release planned for FY2016
Results Provide fine grained services adapted to users’ characteristics and conditions
Compact Gaze Tracking Sensor
Behavior sensing
Gaze tracking
Projection display
Apply
Accumulate
Timely display of relevant information when the user seems to be confused
Technology to provide feedback to the user
Technology to understand the user’s mental state
User Characteristics Database
Smart guidance
Demo
35 Copyright 2015 FUJITSU
36 Copyright 2015 FUJITSU
Affective Media Processing Technology (Use Case)
Understanding users’ feelings
Objective
Technology
Future development
Customer service improvement and automation Combines data from multiple sensing technologies to infer users’ mental state (intent, emotions)
Service release planned for FY2016
Results Provide fine grained services adapted to users’ characteristics and conditions
Compact Gaze Tracking Sensor
Behavior sensing
Gaze tracking
Projection display
Apply
Accumulate
Timely display of relevant information when the user seems to be confused
Technology to provide feedback to the user
Technology to understand the user’s mental state
User Characteristics Database
Smart guidance
Demo
37 Copyright 2015 FUJITSU
Affective Media Processing Technologies
Affective Speech Processing
Speaker’s intent and emotion can be understood with cutting-edge speech analytics / acoustic sensing technologies from speech signal.
Scam Detection -- Detect emotional instability
Speech Analytics -- Understand speaker’s feelings
“Pleasure”
“Sorrow”
“Haste”
…
“Anger”
“Laughter”
“Excitement
”
…
speech signal
Ability to read between the lines is added to
conventional ICT.
38 Copyright 2015 FUJITSU
Phone scam detection technology
Object
Technology
Effect
Prevents phone scams, a serious social problem in Japan
Detects phone scams by integrating results of keyword detection and emotional instability analysis by the false alarm rate of less than 1%
In the field trial, the number of cognitive case of phone scams was reduced by
half(Deterrence effect)
Detection system
FUJITSU original conversation analysis technology
(Processing spontaneous conversation)
Emotional instability Analysis
Keyword detection
Combined Decision
“Come here to explain
about the compensation.”
Notifying victim’s family
or relevant parties
Fraudster
Detecting remittance-solicitation phone scams
Conversation speech
Contact and other support
“Well, even
if you say that…”
Victim
Affective Media Processing Technologies (Use Case)
39 Copyright 2015 FUJITSU
LOD(Linked Open Data)
Knowledge Processing
Network of data, published in a standard format on Web
Data can link to, and be linked from other data
LOD consists of More than
1,000 datasets
60 billion triples
500 million links
From various domains ,
not only governmental data
Machine Readable Knowledge Representation
Media
Geographic Life science
Publications Cross-domain
Government
LOD cloud net
40 Copyright 2015 FUJITSU
LOD4ALL:LOD Utilization PF
Knowledge Processing (Use Case)
Renewal on July, 27, 2015: http://lod4all.net/
41 Copyright 2015 FUJITSU
Evaluation of Local Cities based on LOD
Purpose
Technology
Result Support to plan city policies by comparing with similar cities
Fast graph-database engine for billions of LOD (LOD4ALL)
Evaluate dozens of indices from 1,000 kinds of statistical data
(EvaCva)
Evaluate local cities from economy, society and environment
<EvaCva> <LOD>
Knowledge Processing (Use Case)
42 Copyright 2015 FUJITSU
Improvement of Banking Supervision
Purpose
Technology
Result
Grasp of invisible malicious illegal activities such as hidden people network surrounding insider trading
Discovery of the insider trading that has been overlooked
Knowledge Processing(Use Case)
Detect hidden network of people relates to insider trading.
Heterogeneous Data Federation Framework using LOD
Dynamic Graph Network Analysis to Detect Similarity and Anomaly
Demonstrated
at J4
43 Copyright 2015 FUJITSU
Collaborative Research with San Carlos Hospital
Purpose: Establishment of new treatments for mental disease
Technology:
Knowledge discovery by a fusion of medical records and public open data
Anonymization for keeping patients’ privacy
Result: Discovery of triggers of the pathogeny
E.g.: Relation between the moon’s wax and wane and the visit to the hospital
Knowledge Processing (Use Case)
FUJITSU Spain
Angeles Delgado
San Carlos Hospital
Julio Mayol
FUJITSU Laboratories
Hideyuki Saso
Demonstrated
at M7
44 Copyright 2015 FUJITSU
"Can a Robot Pass the University of Tokyo Entrance Exam?"
Mathematical Technology (Use Cae)
Target
Technology
Effect
Pass the University of Tokyo (Todai) entrance exam
Automatic natural language math problem solving base on our unique “QE* inference tech”
Deviation value: about 65 at a prep school practice exam [ improvement: about 15 points compared to 2014 ]
*QE: Quantifier Elimination
entrance exam
解答
input Output Answer Todai robot QE inference technology
automatic problem-solving Participation as the "math team"
for project
Math
Todai robot project
45 Copyright 2015 FUJITSU
「東ロボ」技術詳細説明 10
Inference by computer algebra
Answer r = 2
circle(C) ∧ area(C) = πr2 radius(C) = r
Math knowledge base
word meaning
(x is) circle circle(x)
area (of x) area(x)
radius (of x) radius(x)
Database Math problems
Learning
Ex:Formula for a area of a circle
Find(r)[∀s. πs2 = 4π ∧ s > 0 → s = r ]
equivalent conversion of logical formulas
Language Understanding
Logical form
Problem “Find the radius of a circle with area 4π.”
Find(r)[ ∀C. circle(C)∧ area(C) = 4π → radius(C) = r ]
QE inference technology
Todai robot technologies for math problems
dictionary
Mathematical Technology (Use Case)
Demo
46 Copyright 2015 FUJITSU
47 Copyright 2015 FUJITSU
「東ロボ」技術詳細説明 10
Inference by computer algebra
Answer r = 2
circle(C) ∧ area(C) = πr2 radius(C) = r
Math knowledge base
word meaning
(x is) circle circle(x)
area (of x) area(x)
radius (of x) radius(x)
Database Math problems
Learning
Ex:Formula for a area of a circle
Find(r)[∀s. πs2 = 4π ∧ s > 0 → s = r ]
equivalent conversion of logical formulas
Language Understanding
Logical form
Problem “Find the radius of a circle with area 4π.”
Find(r)[ ∀C. circle(C)∧ area(C) = 4π → radius(C) = r ]
QE inference technology
Todai robot technologies for math problems
dictionary
Mathematical Technology (Use Case)
Demo
48 Copyright 2015 FUJITSU
*Cooperative research : “Fujitsu Social Mathematics Joint Research Unit” in Kyusyu university
Mathematical science, Applied Math
modeling Control
Analytics/Prediction Optimization
Social issues
human behavior and psychology
Solution
Social system design (Game theory/matching)
Social system evaluation (Agent-based social Simulation)
Social system modeling (Operational models that consider human’s mind and behavior)
Refine policies
Mathematical Technology (case example)
Raising Passenger Satisfaction at Fukuoka Airport
Target
Technology
Effect
Raising Passenger Satisfaction in terms of multiple aspects: congestion, security and staff assignment etc.
Social system design framework based on math technologies
Aiming at deploying the field trial results at Fukuoka airport to other airports
49 Copyright 2015 FUJITSU
Tsunami Inundation Prediction
Mathematical Technology (case example)
Target
Technology
Effect
Improvement of tsunami early warning systems
High-efficiency parallel computing of High-resolution tsunami inundation simulation
Tsunami’s inundation is predictable within 2 min after tsunami source is estimated.
Provided by IRIDeS, Tohoku University
Demo
50 Copyright 2015 FUJITSU
51 Copyright 2015 FUJITSU
Tsunami Inundation Prediction
Mathematical Technology (case example)
Target
Technology
Effect
Improvement of tsunami early warning systems
High-efficiency parallel computing of High-resolution tsunami inundation simulation
Tsunami’s inundation is predictable within 2 min after tsunami source is estimated.
Provided by IRIDeS, Tohoku University
Demo
52 Copyright 2015 FUJITSU
Real-time Tsunami Hazard Map
Inundation
analysis
Inundation analysis
Assumptions
of possible
earthquakes
Earthquake
Historical
earthquakes
hazard maps
of local authorities
Real-time
hazard map
Execution time:
Days on a PC
10 sec - minutes
This technology
Execution time:
2 min on supercomputer
More than 5 min
Observation
Instant tsunami
source analysis
Mathematical Technology (case example)
53 Copyright 2015 FUJITSU
Brain Science
Pursuing human’s processing mechanisms
Board pattern recognition
Next-move decision
“Neural Basis of Intuitive Best Next-Move Generation in Board Game Experts”, Science 2011 Collaborative research between RIKEN BSI and Fujitsu
Big projects of brain science will reveal functions of neural circuits in the next decade.
Research with neuroscientists would bear a computer having ‘intuition’ required for complex problem solving.
Striatum
Precuneus
Differences between professionals, amateur experts, novices
Highly activated areas of professional board game players (Japanese ‘Shogi’ players)
54 Copyright 2015 FUJITSU
People/Businesses/Society
Systematizing AI technologies of Fujitsu
Sensing Actuation
Sensing and Recognition
Image recognition
Voice recognition
Emotion/state recognition
Knowledge processing
Natural-language processing Knowledge processing & discovery
Pattern discovery
Decision and support
Inference & planning Prediction & optimization Interactivity & recommendation
Social receptivity Simulation Neuroscience Advanced research
Machine learning Reinforcement learning Deep Learning Learning
55 Copyright 2015 FUJITSU