emotion recognition - a new design paradigm
TRANSCRIPT
![Page 1: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/1.jpg)
GRAPHING CUSTOMER EMOTION IN TURBOTAX IN REAL TIME
1
![Page 2: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/2.jpg)
Agenda1. background
2. technology
3. experiments, insights and customer benefits
4. opportunities, risks and next steps
2
![Page 3: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/3.jpg)
Part 1. Background
3
![Page 4: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/4.jpg)
4
![Page 5: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/5.jpg)
43 muscles and 23 action units (AUs) – combinations of facial muscle activity that correspond to seven core emotions – are studied on a
second-by-second basis.
Pursuant to US Patent #7,113,916
5
![Page 6: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/6.jpg)
6
![Page 7: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/7.jpg)
Why now?
7
![Page 8: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/8.jpg)
SxSW 2015 + CONVERGENCE
8
![Page 9: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/9.jpg)
9
Gathering emotion response data will allow us to see our customers in a completely new way. It will unlock the unique individual behind the faceless demographic.
![Page 10: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/10.jpg)
This would allow us to…
10
Serve up different versions our tax-prep experience according to emotional responses
Capture a completely new dimension of data in analytics
Change marketing offers based on emotion response
Better facilitate customer care (especially agent interaction)
![Page 11: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/11.jpg)
Baseline requirements
11
real time fast accurate seamless integration
![Page 12: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/12.jpg)
Part 2. Technology
12
![Page 13: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/13.jpg)
13
![Page 14: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/14.jpg)
14
![Page 15: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/15.jpg)
Technology Highlights
Valence, Engagement, and other emotions
SDK runs native/local on mobile + desktop platforms
Real-time, reads every 250 ms
Transient — no images are stored
15
![Page 16: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/16.jpg)
Hands-on demo!
16
![Page 17: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/17.jpg)
Part 3. Experiments, insights and customer benefits
17
![Page 18: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/18.jpg)
Experiment A positive reaction (valence >0)
negative reaction (valence <0)
AFFECTIVA measure
page 1
page 2A
page 2B
![Page 19: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/19.jpg)
19
WHAT THE CUSTOMER SEES
WHAT THE CAMERA SEES
Experiment A
![Page 20: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/20.jpg)
positive reaction negative reaction
![Page 21: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/21.jpg)
Experiment Apositive reaction
(valence >0)
negative reaction (valence <0)
measure
![Page 22: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/22.jpg)
22
negative valence
positive valence
![Page 23: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/23.jpg)
we needed data from real people
23
Experiment B
![Page 24: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/24.jpg)
24
Experiment B
-48
+35
![Page 25: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/25.jpg)
25
Experiment B
![Page 26: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/26.jpg)
we needed a controlled test on a specific event at a specific time
26
![Page 27: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/27.jpg)
We measured emotions between 3 points for a 3 second interval
Experiment C
number starts rolling number stops rolling 2 seconds after number stops rolling
THE EVENT THE TIME INTERVAL
0 1 3
When the EIC credit appears
![Page 28: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/28.jpg)
Stt
28
Experiment C
![Page 29: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/29.jpg)
noisy data - wide range of deviation on a single URL giving low
confidence on what the customer is experiencing
event driven data
With data tied to intervals around events we get an accurate reflection of emotion response.
29
Experiment C
![Page 30: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/30.jpg)
Stt
30
Experiment C
![Page 31: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/31.jpg)
31
Experiment B
![Page 32: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/32.jpg)
Customer benefitswe can understand our customers on a deeper level
we can respond to a negative or positive outcome in a way that builds confidence
we can provide customers with support when they need it most
we can and help in ways we never did before
we can identify areas of concern that are otherwise invisible
32
![Page 33: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/33.jpg)
Conclusions / insights
We can gather a completely new understanding of the end-to-end customer journey AND key moments in the customer experience
We can gather valuable, actionable analytics giving us a radically new data dimension, creating a strategic advantage that we have never had before.
33
![Page 34: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/34.jpg)
Part 4. Opportunities, risks and next steps
34
![Page 35: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/35.jpg)
35
There is no art/ to find the man's
construction in the face William Shakespeare, Macbeth
![Page 36: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/36.jpg)
36
![Page 37: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/37.jpg)
37
-12
![Page 38: Emotion recognition - a new design paradigm](https://reader030.vdocument.in/reader030/viewer/2022032620/55cd5928bb61eb940e8b4617/html5/thumbnails/38.jpg)
38
Thanks for your time!!
Damien Amir
Sharon Clarence