Mixed-Reality Technology and Whole-Body Interaction for Active Lifestyles

Mario Romero2015/12/03

Photo: Fredrik Lilkaer. Project: SolarSense (AGI2013)

A note:

• All of the images in this presentation have hyperlinks to original content.

2015/12/03 Romero 2

Outline

• What is Mixed Reality?

• A vision of the near future

• Select portfolio

• Discussion: how can tech promote healthier lifestyles?

2015/12/03 Romero 4

Disclaimer

• Not talking about aging– Talking about active live styles for all ages

• Not talking about all technology– Talking about mixed-reality

2015/12/03 Romero 5

2015/12/03 Romero 6

Is this you, sitting, now?Enjoying Real Virtuality, but sedentary?

What is mixed-reality?

It is the merging real and virtual objects andinterfaces to produce new experiences where thephysical and digital co-exist and interact in real-time.

2015/12/03 Romero 7

Mixed-Reality Spectrum

2015/12/03 Romero 8

2015/12/03 Romero 9

Augmented and Virtual Reality Spectrum

2015/12/03 Romero 10

Augmented and Virtual Reality Spectrum

2013

2015/12/03 Romero 11

Augmented and Virtual Reality Spectrum

2013

2014

2015/12/03 Romero 12

Augmented and Virtual Reality Spectrum

2013

2014

2015

2015/12/03 Romero 13

Augmented and Virtual Reality Spectrum

2013

2014

2015

2015/12/03 Romero 14

A vision of the future

2015/12/03 Romero 15

Skin Input

Harrison 2010

2015/12/03 Romero 16

Muscle Input

Saponas 20092015/12/03 Romero 17

Bone Output

Walker 2005

2015/12/03 Romero 18

Multi-surface Input

Synaptics Fuse 2006

2015/12/03 Romero 19

Air periphery phone input

Fuffr 2015

2015/12/03 Romero 20

Virtual Reality

Oculus Rift 2016

2015/12/03 Romero 21

Mobile VR

2015/12/03 Romero 22

Mobile 3D Intelligence

Google Tango 2014

2015/12/03 Romero 23

Full Mixed Reality

Microsoft Hololens 20152015/12/03 Romero 24

Muscle Output

Lopes 20132015/12/03 Romero 25

Shared Augmented Reality without wearing gear

Magic Leap 2017 (?)2015/12/03 Romero 26

My studio

2015/12/03 Romero 27

2015/12/03 Romero 28

• Research

• Visualization Supported Collaborative Work

• Foundational Technology

• User Evaluations

• Showcase and classroom environment

• Outreach

Students’ Resources:Visualization Studio

2015/12/03 Romero 29

• High-resolution projection wall with stereoscopy

• Oculus Rift

• Cinema quality audio

• High-definition video communications with eye contact

• Holographic display

• Multi-touch interactive surfaces

• Eye tracking

• GPU-based computing cluster

• Diverse interaction and sensor systems (haptic, mocap, etc.)

• Haptic Devices

• 3D printer

• Epson Moverio

Technologies in

2015/12/03 Romero 30

2015/12/03 Romero 31

VisualizationStudio

2015/12/03 Romero 32

Students

2015/12/03 Romero 33

Select Portfolio

2015/12/03 Romero 34

Virtual Sculpting

2015/12/03 Romero 35

2015/12/03 Romero 36

2015/12/03 Romero 37

2015/12/03 Romero 38

Embodied Sculpting

2015/12/03 Romero 39

MegaMind Tekniska Museet – Forma i Luften

Hands-on interaction with 3D, haptics, sound

2015/12/03 Romero 40

Base Jumper

• Octree rendering

• Computer vision body control

• Link (not on the image)

2015/12/03 Romero 41

DangeRoads

2015/12/03 Romero 42

FaceUp

2015/12/03 Romero 43

PodRacer

2015/12/03 Romero 44

PodRacer

2015/12/03 Romero 45

2Pacs

2015/12/03 Romero 46

From the stormtroopers: "Devoid of colour differentiation (Stormtroopers see everything

in shades of green) made it hard to spot the ghosts." A universal

consideration for interaction design: accessibility and color

blindness.

2Pacs

2015/12/03 Romero 47

InvadAR

2015/12/03 Romero 48

YA3

2015/12/03 Romero 49

2015/12/03 Romero 50

2015/12/03 Romero 51

2015/12/03 Romero 52

2015/12/03 Romero 53

Mad Sand

2015/12/03 Romero 54

Teamtris

2015/12/03 Romero 55

BrARwl

2015/12/03 Romero 56

Padawan 101

2015/12/03 Romero 57

Shmooning

2015/12/03 Romero 58

Blopper

2015/12/03 Romero 59

Resources• ACM SIGGRAPH

• ACM CHI

• ASSETS: ACM International Conference on Computers and Accessibility

• CSCW: ACM conference on Computer Supported Cooperative Work

• DIS: ACM conference on Designing Interactive Systems

• ECSCW: European Conference on Computer-Supported Cooperative Work: ACM conference on supporting group work

• HRI: ACM/IEEE International Conference on Human–robot interaction

• ICMI: International Conference on Multimodal Interfaces

• ITS: ACM conference on Interactive Tabletops

and Surfaces

• IUI: International Conference on Intelligent User Interfaces

• MobileHCI: International Conference on Human–Computer Interaction with Mobile Devices and Services

• NIME: International Conference on New Interfaces for Musical Expression

• Ubicomp: International Conference on Ubiquitous computing

• UIST: ACM Symposium on User Interface Software and Technology

• i-USEr: International Conference on User Science and Engineering

2015/12/03 Romero 60

2015/12/03 Romero 61

Thank you!

marior@kth.se

Discussion?

2015/12/03 Romero 62

https://goo.gl/ZkOI7t