using sensors to infer how people feel
DESCRIPTION
Overview of the current research around emotionsense.orgTRANSCRIPT
![Page 1: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/1.jpg)
how can sensor data be used toinfer how people feel?
@neal_lathiauniversity of cambridge
![Page 2: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/2.jpg)
how many smartphones do you have?
![Page 3: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/3.jpg)
how often do you carry your phone with you (%) ?
![Page 4: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/4.jpg)
how often do you carry your phone with you (%) ?
research: 90% of the time it is in the same room as you
making it a powerful, personal device that can sense, monitor, and interact with you
![Page 5: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/5.jpg)
sense (infer): activities, social interactions, mobility, ambient sound
in the moment
![Page 6: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/6.jpg)
monitor: green transportation habits and modes, fitness levels
individuals over time
![Page 7: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/7.jpg)
interact: social network broadcasts, ubiquitous technology evaluation, persuasive interfaces
understand & persuade
![Page 8: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/8.jpg)
now: a tool for psychology research“ubiquitous, intimate, sensor-rich, computationally
powerful, remotely accessible”
![Page 9: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/9.jpg)
sense
interact
mental processes
infer
![Page 10: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/10.jpg)
![Page 11: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/11.jpg)
sensed context
reported mood
“in the moment”
infers
(responds)
![Page 12: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/12.jpg)
system → challenges
user → challenges
inference → challenges
![Page 13: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/13.jpg)
system → challenges
(kiran rachuri)
![Page 14: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/14.jpg)
system → challenges
battery
processing
memory
network
smartphone sensors weren't built for continuous monitoring
![Page 15: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/15.jpg)
system → challenges:: adaptive sensor sampling
sense sleep
continuous interval (50%)adaptive
-45% battery -5 % battery-2% accuracy - 40% accuracyco-location
-44% battery -8 % battery-1% accuracy - 34% accuracyaccelerometer
![Page 16: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/16.jpg)
inference → challenges
(ubicomp 2010)
![Page 17: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/17.jpg)
inference → challenges:: machine learning {sensed context} → emotion
microphone sample
emotional prosody-based model
daily diary inferred emotion
+ 71% accuracy
MAP likelihood
5 broad emotions
![Page 18: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/18.jpg)
user → challenges
![Page 19: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/19.jpg)
user → challenges:: protocols for experience sampling
signal respond
compliance, bias, delay
experience sampling
![Page 20: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/20.jpg)
system → challenges
user → challenges
putting it together
inference → challenges
![Page 21: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/21.jpg)
adaptive sampling
experience sampling
putting it together?
emotion inference learning
conflict?
conflict?
conflict?
![Page 22: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/22.jpg)
{sensed context} → {current emotion}
in the moment
![Page 23: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/23.jpg)
{sensed context} → {current emotion}
{ } → {current emotion}
{sensed context} → { }
![Page 24: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/24.jpg)
{sensed context} → {current emotion}
design for participant compliance
design for sensor data
![Page 25: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/25.jpg)
{sensed context} → {current emotion}
design for participant compliance
design for sensor data
just finishing: a/b test with a variety of triggers. results?
![Page 26: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/26.jpg)
so, to conclude..
![Page 27: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/27.jpg)
who canmisinterpret
your emotions?
you
your social circlesa psychologist
![Page 28: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/28.jpg)
who canmisinterpret
your emotions?
you
your social circlesa psychologist
your phone
![Page 29: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/29.jpg)
sensed context
reported mood
“in the moment”
infers
responds
![Page 30: Using Sensors to Infer how People Feel](https://reader033.vdocument.in/reader033/viewer/2022060200/5597e26a1a28abf30e8b458d/html5/thumbnails/30.jpg)
emotionsense.org
@neal_lathiauniversity of cambridge