quality of sensory experience (quase)
Post on 02-Dec-2014
227 Views
Preview:
DESCRIPTION
TRANSCRIPT
Quality of Sensory Experience (QuaSE)
Chris8an Timmerer
Alpen-‐Adria-‐Universität Klagenfurt (AAU) w Faculty of Technical Sciences (TEWI) w Department of Informa8on Technology (ITEC) w Mul8media Communica8on (MMC) w Sensory Experience Lab (SELab)
blog.&mmerer.com w selab.itec.aau.at w dash.itec.aau.at chris&an.&mmerer@itec.aau.at
QUALINET Final Workshop October 10, 2014
Acknowledgments. This work was supported in part by the European Commission in the context of the NoE INTERMEDIA (NoE 038419), the P2P-‐Next project (FP7-‐ICT-‐216217), the ALICANTE project (FP7-‐ICT-‐248652), the SocialSensor project (FP7-‐ICT-‐287975),
and the COST Ac&on IC1003 QUALINET.
Sensory Experience • Consump&on of mul&media content may s&mulate also other senses – Vision or hearing – Olfac&on, mechanorecep&on, thermocep&on, …
• Annota&on with metadata providing so-‐called sensory effects that steer appropriate devices capable of rendering these effects
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 2
Sensory Experience • Consump&on of mul&media content may s&mulate also other senses – Vision or hearing – Olfac&on, mechanorecep&on, thermocep&on, …
• Annota&on with metadata providing so-‐called sensory effects that steer appropriate devices capable of rendering these effects
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 3
… giving her/him the sensa8on of being part of the par8cular mulsemedia ➪ worthwhile, informa8ve user experience
General Principle – Outline
• General principle: there is a need for a scien8fic framework to capture, measure, quan,fy, judge, and explain the quality of (sensory) experience
• Outline – [How to create, delivery, consume?] – How to capture and measure? – How to quan&fy? – How to judge and explain?
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 4
How to create, delivery, consume? • Sensory Effect Descrip&on Language (SEDL)
– Basic building blocks to describe, e.g., light, wind, fog, vibra&on, scent – MPEG-‐V Part 3, Sensory Informa&on: Effects, GroupOfEffects – Adopted MPEG-‐21 DIA tools for adding &me informa&on
(synchroniza&on) • Descrip&on conforming to SEDL :== Sensory Effect Metadata (SEM)
– Can be associated to any kind of mul&media content (e.g., movies, music, Web sites, games)
– Support to be included in file (MP4) and transport (M2TS) formats • Tool support for crea&ng (annota&on tools) and consump&on
(players, Web plugins) ➜ selab.itec.aau.at • Devices: e.g., amBX (Ambient Experience) system + SDK,
Gameskunk, Scentscape, etc.
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 5
How to capture and measure? • Subjec&ve quality assessments
– Methodology: based on standard methods – Test content: different genres, manually annotated (cf. QUALINET DB)
• Experiment I – Aim: Demonstrate sensory effects as a vital tool for enhancing the quality of
experience depending on the actual genre • Experiment II
– Aim: inves&gate the rela&onship of the QoE to various video bit-‐rates of mul&media contents annotated with sensory effects.
– Subjec&ve quality gap between video resources annotated with and without sensory effects at different bit-‐rates
• [Experiment III] ambient lights & different color calcula&on seqngs • Experiment IV
– Aim: inves&gate the enhancement of the QoE and how users’ emo&ons are elicited and influenced by Web videos annotated with and without sensory effects
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 6
Experiment II: Results
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 7
Sequence Babylon A.D. Earth Duration 35s 21s Resolution 1280 x 544 1280 x 720 Motion High Low Nr. of Effects W: 7; V: 9 W: 8; V: 1 Bit-rates Kbit/s PSNR Kbit/s PSNR Low Quality 2154 38.93 2204 38.11 Medium Quality 3112 41.27 3171 40.65 High Quality 4044 42.95 4116 42.27 Highest Quality 6315 N/A 6701 N/A
Test Sequences
MOS vs. PSNR/bit-‐rate for Earth.
How to quan&fy?
• Experiment V – Aim: towards a quality/u&lity model for QuaSE
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 8
• S&muli with all combina&ons of sensory effects – Vibra&on higher impact than light & wind
– Highest QoE with all effects present
• General QuaSE model
How to judge and explain? • Experiment VI – Aim: understand QuaSE
• Biosensor-‐based QoE evalua&on system
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 9
J. Donley, C. Ritz, M. Shujau, "Analysing the Quality of Experience of Mul&sensory Media from Measurements of Physiological Responses,” QoMEX2014, Singapore, Sep. 2014.
How to judge and explain?
• Experiment VII – Aim: understand QuaSE
• EEG Correlates of Pleasant and Unpleasant Odor Percep&on
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 10
E. Kroupi, A. Yazdani, J.-‐M. Vesin, T. Ebrahimi, "EEG Correlates of Pleasant and Unpleasant Odor Percep&on," ACM TOMM, vol. 11, no. 1s, Sep. 2014.
How to judge and explain?
• Experiment VIII – Aim: understand QuaSE
• Mul&ple-‐Scent Enhanced Mul&media Synchroniza&on
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 11
N. Murray, B. Lee, Y. Qiao, and G.-‐M. Muntean, "Mul&ple-‐Scent Enhanced Mul&media Synchroniza&on," ACM TOMM, vol. 11, no. 1s, Sep. 2014.
General temporal boundaries: -‐10s to +15s are “in-‐sync”, skew values beyond are “out-‐of-‐sync”
Conclusions
• From the need for a scien8fic framework to capture, measure, quan,fy, judge, and explain the quality of experience
• To … – How to create, delivery, consume? – How to capture and measure? – How to quan&fy? – How to judge and explain?
• Open issues? 2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 12
✔︎ ✔︎ ✔︎ ✔︎
Many!
Sensory Experience Lab hyp://selab.itec.aau.at/ Sozware and Services Standardiza&on Publica&ons Media Funding
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 13
References • Markus Waltl, Chris&an Timmerer, Hermann Hellwagner, “A Test-‐Bed for Quality of Mul8media Experience Evalua8on of
Sensory Effects”, Proceedings of the First Interna&onal Workshop on Quality of Mul&media Experience (QoMEX 2009), San Diego, USA, July 29-‐31, 2009.
• C. Timmerer, J. Gelissen, M. Waltl, H. Hellwagner, “Interfacing with Virtual Worlds”, Proceedings of the NEM Summit 2009, Saint-‐Malo, France, September 28-‐30, 2009.
• M. Waltl, C. Timmerer H. Hellwagner, “Increasing the User Experience of Mul8media Presenta8ons with Sensory Effects”, Proceedings of the 11th Interna&onal Workshop on Image Analysis for Mul&media Interac&ve Services (WIAMIS’10), Desenzano del Garda, Italy, April 12-‐14, 2010.
• M. Waltl, C. Timmerer, H. Hellwagner, “Improving the Quality of Mul8media Experience through Sensory Effects”, Proceedings of the 2nd Interna&onal Workshop on Quality of Mul&media Experience (QoMEX2010), Trondheim, Norway, June 21-‐23, 2010.
• C. Timmerer, M. Waltl, B. Rainer, H. Hellwagner, “Assessing the Quality of Sensory Experience for Mul8media Presenta8ons”, Signal Processing: Image Communica8on, 2012.
• M. Waltl, C. Timmerer, B. Rainer, H. Hellwagner, ”Sensory Effects for Ambient Experiences in the World Wide Web”, Mul&media Tools and Applica&ons (MTAP), 2012.
• B. Rainer, M. Waltl, Eva Cheng, Muawiyath Shujau, C. Timmerer, Stephen Davis, Ian Burney, Chris&an Ritz, H. Hellwagner, ”Inves8ga8ng the Impact of Sensory Effects on the Quality of Experience and Emo8onal Response in Web Videos”, Proceedings of 4th Int’l. Workshop on Quality of Mul&media Experience (QoMEX2012), 2012.
• M. Waltl, C. Timmerer, B. Rainer, H. Hellwagner, “Sensory Effect Dataset and Test Setups”, Proceedings of 4th Int’l. Workshop on Quality of Mul&media Experience (QoMEX2012), 2012.
• C. Timmerer, B. Rainer, M. Waltl, “A U8lity Model for Sensory Experience”, In Proceedings of the 5th Interna&onal Workshop on Quality of Mul&media Experience (QoMEX2013), 2013.
• J. Donley, C. Ritz, M. Shujau, “Analysing the Quality of Experience of Mul8sensory Media from Measurements of Physiological Responses,” In Proceedings of the 5th Interna&onal Workshop on Quality of Mul&media Experience (QoMEX2014), Singapore, 2014.
• E. Kroupi, A. Yazdani, J.-‐M. Vesin, T. Ebrahimi, "EEG Correlates of Pleasant and Unpleasant Odor Percep8on," ACM TOMM, vol. 11, no. 1s, Sep. 2014.
• N. Murray, B. Lee, Y. Qiao, and G.-‐M. Muntean, "Mul8ple-‐Scent Enhanced Mul8media Synchroniza8on," ACM TOMM, vol. 11, no. 1s, Sep. 2014.
• G. Ghinea, C. Timmerer, W. Lin, and S. R. Gulliver. "Mulsemedia: State of the Art, Perspec8ves, and Challenges," ACM TOMM, vol. 11, no. 1s, Sep. 2014.
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 14
Thank you for your ayen&on
... ques&ons, comments, etc. are welcome …
Assoc-‐Prof. Dipl.-‐Ing. Dr. Chris&an Timmerer Klagenfurt University, Department of Informa&on Technology (ITEC)
Universitätsstrasse 65-‐67, A-‐9020 Klagenfurt, AUSTRIA chris&an.&mmerer@itec.uni-‐klu.ac.at
hyp://research.&mmerer.com/ Tel: +43/463/2700 3621 Fax: +43/463/2700 3699
© Copyright: Chris.an Timmerer
15 2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 16
2014/10/10 Chris&an Timmerer, Alpen-‐Adria-‐Universität, Austria 17
top related