SENSORY EFFECT DATASET AND TEST

SETUPSMarkus Waltl, Christian Timmerer, Benjamin Rainer, Hermann Hellwagner

Multimedia Communication (MMC) Research Group, Institute of Information Technology (ITEC)

Alpen-Adria-Universität Klagenfurt, Klagenfurt, Austria

E-mail: firstname.lastname@itec.uni-klu.ac.at

Acknowledgments. This work was supported in part by the EC in the context of the ALICANTE (FP7-ICT-248652), SocialSensor (FP7-ICT-287975), and QUALINET (COST IC 1003) projects and partly performed in the Lakeside Labs research cluster at AAU.

OUTLINE Motivation and Context Sensory Effect Dataset

Sensory Effect Metadata Generation Dataset Review Process

Test Setups [Evaluations see prev. QoMEX papers]➪ Conclusion

Sensory Effect DatasetApril 13, 2023 2

MOTIVATION From traditional multimedia to sensory-enriched

multimedia E.g., wind, vibration, and light effects

Goal: increasing the Quality of Experience

Lack of common test content for performing subjective quality assessments

April 13, 2023 Sensory Effect Dataset 3

4April 13, 2023 Sensory Effect Dataset

CONTEXT

MPEG-V (ISO/IEC 23005): interoperability between virtual worlds and the real world

Seven parts: (1) architecture, (2) control information, (3) sensory information, (4) virtual world object char., (5) data formats for interaction dev., (6) common types and tools, (7) conformance + ref. software

Part 3 Sensory Information Sensory Effect Description Language (SEDL) Sensory Effect Vocabulary (SEV)

➪ Sensory Effect Metadata (SEM) May be associated to any kind of multimedia content (e.g., movies,

music, Web sites, games) Steer sensory devices like fans, vibration chairs, lamps, etc. via an

appropriate mediation device

Currently, no common content for evaluations of sensory effects available

76 video sequences from 5 genres 38 action 12 documentary 8 sports 5 news 13 commercial

Enriched with light,wind, and vibration

~1.6GB, availableunder NDA

April 13, 2023 Sensory Effect Dataset 5

MPEG-V: MEDIA CONTEXT AND CONTROLSENSORY EFFECT DATASET

Video sequences enriched with sensory effects using the Sensory Effect Video Annotation (SEVino) tool

Advantages Easy to use Platform independent Support all formats

and codecs (from VLC) Create MPEG-V-

compliant SEMdescriptions

April 13, 2023 Sensory Effect Dataset 6

SENSORY EFFECT METADATA GENERATION

April 13, 2023 Sensory Effect Dataset 7

DATASET REVIEW PROCESS

Evaluations with sensory effects need special devices, e.g., ambient light, fans, vibration chairs

Three setups using amBX system and Cyborg Gaming Lights

Setups have been used in user studies or evaluated internally

April 13, 2023 Sensory Effect Dataset 8

TEST SETUPS

April 13, 2023 Sensory Effect Dataset 9

TEST SETUPS

April 13, 2023 Sensory Effect Dataset 10

TEST SETUPS

April 13, 2023 Sensory Effect Dataset 11

TEST SETUPS

Sensory effect dataset can be used to perform assessments with enriched multimedia content

Provides a number of test sequences from various genres in different bit-rates and resolutions

Most sequences from the dataset were already used in user studies

Proposed a number of test setups that can be used for performing QoE evaluations accompanied by sensory effects

April 13, 2023 Sensory Effect Dataset 12

CONCLUSIONS

SENSORY EXPERIENCE LAB

http://selab.itec.aau.at/

Software and Services

Standardization

Publications

Media

Funding13April 13, 2023 Sensory Effect Dataset

THANK YOU FOR YOUR ATTENTION

Sensory Effect DatasetApril 13, 2023 14