ux and interaction in virtual reality

103

Upload: devgamm-conference

Post on 14-Apr-2017

72 views

Category:

Software


1 download

TRANSCRIPT

UX and Interaction in Virtual Reality

Ciro ContinisioTechnical Evangelist@[email protected]

Before we startTakeaway: Hints and ideas on how to create good user experience in VR, through the analysis of the solutions adopted by several VR games and apps.

This talk is code-free

4

2016 - The Year of VR(be quick though, because were in November!)

Since last year, everyone has been saying that 2016 would have been the year of VR, and in part they were right.We have seen the launch of Oculus and Vive to the public, and soon PlayStation VR.

Sadly, or luckily, its also almost over.5

Takeaways of 2016No market for it... yet85k-100k Vive owners 3 monts after launch40k+ Rift on Steam (how many devkits?)1M monthly users on GearVR (May 2016), 5M Cardboard app downloadsNo standards... yetNo killer application yetSource: http://www.roadtovr.com/htc-vive-sales-figures-data-100000-steamspy-data/http://www.roadtovr.com/oculus-rift-growth-spurt-has-halted-htc-vive-market-share-growth/

So, since its almost over, what are the takeaways of this 2016? One thing is sure, we know that the market is very small, with around 100 thousands Vive sold, probably 40 thousand active Rifts, and mobile VR that sits just in the tens of millions users.Theres no standard, like we have the controllers on consoles, which makes cross-headset development practically impossible.Finally, VR hasnt got its killer application, yet.6

Get ready for 2017

The good news is, you can still be part of the VR revolution in 2017,7

VR killer application

and possibly, create the first real VR killer application8

Audience is fresh

9

No magic formula

10

Interactivity

11

InteractivityTwo-way flow of information, between the player and the game.

The interactivity happens through the User Interface (UI) which, when put to action, defines the User Experience (UX).Source: the Art of Game Design by Jesse Schell

12

InteractivityThe User Interface is made-up of 3 parts:The real-world physical input (controllers) and output (screen, audio)The simulated game worldThe in-game virtual interface (often called UI)

The relationship between them is called mapping. When it works well, the player feels in control, the UI becomes intuitive and the resulting UX is great.

13

Interactivity in VRMotion-controlled physical input (HMD and hand controllers)Now we are inside the simulated game worldNew complications to deal withMotion sicknessFatigue and session lengthLocomotion in the game world

14

Interactivity in VRis largely uncharted

15

Build a vocabulary

16

Creating Presence

17

Presence... a psychological state or subjective perception in which even though part or all of an individuals current experience is generated by and/or filtered through human-made technology, part or all of the individuals perception fails to accurate acknowledge the role of the technology in the experience.Source: ISPR - https://ispr.info/about-presence-2/about-presence/... a psychological state or subjective perception in which even though part or all of an individuals current experience is generated by and/or filtered through human-made technology, part or all of the individuals perception fails to accurate acknowledge the role of the technology in the experience.

18

Types of VR platformsMobile (Cardboard, Daydream, Gear VR, other plastic shells)Low processing power, less pixels on-screenSimplified inputTethered (Oculus, Vive, PlayStation VR)High power, but high hardware requirementsHead trackingMore refined physical interactions

When we look at the current VR platforms available, theres a clear technological divide between the mobile ones, which include []. Making content for these platforms requires less effort, but at the same time we have to optimise for phones, and try to maintain a consistent 60fps. Not easy.

On the other side, we have the VR platforms that have a tethered HMD, [...], which have much more power but at the same time require expensive hardware to run. This greatly limits the potential audience, but on the other side they offer a more refined range of physical interactions.19

Map VR platformsby type of experience

20

Types of VR experiencesOne-button/touch controlsCardboard, Gear VR, other plastic shellsControllers, hand tracking or gesturesOculus Rift, Daydream, PlayStation VRRoom-scale, full-body movementsHTC Vive

Means of interactivity in VR

22

Visuals

23

VisualsSame techniques on all platformsCubemaps, baking light, smokes and mirrors,Reduce post-processing effects, but absolutely use AADifferent fidelity per platformEven static images work wonders!

24

Motion sickness

25

How to induce motion sickness

26

Motion sicknessIn order of importance:Rotational accelerationRotational speedMovement accelerationMovement speed

Roll being particularly bad

Building the VR legs

28

Motion sickness: some tipsStart slow to let the player get used to VRHave a frame of reference

Frame of reference

30

Motion sickness: some tipsStart slow to let the player get used to VRHave a frame of referenceShake that instead of the camera

Screen shake

SubLevel Zero32

Motion sickness: some tipsStart slow to let the player get used to VRHave a frame of referenceShake that instead of the cameraPrefer fast and short to long but slow

Fast movements

SubLevel Zero34

Audio

35

Audio in 3D gamesEasy to do in UnitySounds positioned in the 3D sceneSpread over 2 audio channels depending on listener positionReverberation filtersDoppler effect

36

Audio in VR - Spatial AudioSpatialisation is created mainly by 3 elements:Interaural time differenceInteraural level differenceSpectral filtering done by outer ears

Source: https://developers.google.com/vr/concepts/spatial-audio

37

Audio in VR - Spatial AudioDirect soundEarly reflectionLate reverb

Source: https://developers.google.com/vr/concepts/spatial-audio

38

Audio in VR - Proprietary SDKsThey offer a lot more control over sound:They create Spatial AudioAlter levels, and timePerform spectral filteringThey allow to define materials and size of the virtual environmentOften optimised (Google VR SDK for Daydream, which uses a diff. thread)Source: https://developers.google.com/vr/concepts/spatial-audio

39

General tips for good VR soundsUse mono files with no pre-built reverbUse recognisable soundsAnimate the sound sourceRepeat the soundUse complex sounds

Source: https://developers.google.com/vr/concepts/spatial-audio

40

Input

41

GazeMost basic form of inputVery simple to implementApplies to all types of VRSometimes the only choice

42

Reticule + target feedback

43

Gaze

44

Involuntary interactions

45

Gaze aiming

46

Off-screen events feedback

47

Off-screen events feedback

48

One-button controls

49

Games for the motion impaired

50

Motion controls

51

Hacking the physics

52

Accelerometer

53

Gyroscope

54

Tracked hands

55

New Vive controllers prototype

56

New Vive controllers prototype

57

New Vive controllers prototype

58

NewtonVR

Source:github.com/TomorrowTodayLabs/NewtonVR

59

Never use a buttonif you can go physical

60

Physical tool swap

61

Physical inventory

62

Physical inventory

63

Physical inventory

64

No real-life feedback

65

Haptic feedback

The gallery66

Perceive the Z axis

67

Z axis issues

The gallery68

Locomotion

69

Teleport and reorient

The gallery70

Walking in place

Source: Eelke Folmer - https://www.youtube.com/watch?v=lWqA_4crAEA

71

Freezing the world

Source: Bumble - https://www.youtube.com/watch?v=p0YxzgQG2-E

Cosmic Wandering72

Holosphere

Source: Frooxius - https://www.youtube.com/watch?v=oSP5sjg9TIs

73

Trick the players mind

74

Unlimited corridor

Source: Yohei Yanase - https://www.youtube.com/watch?v=THk92rev1VA

Yohei Yanase75

Unlimited corridor

Source: Yohei Yanase - https://www.youtube.com/watch?v=THk92rev1VA

76

Unlimited corridor

Source: Yohei Yanase - https://www.youtube.com/watch?v=THk92rev1VA

77

Locomotion as a game mechanic

78

Budget Cuts

79

Unseen Diplomacy

Redirected walking80

Escher

81

Clipping through walls

82

Clipping feedback

Source: REFLEKT at Vision Summit - https://www.youtube.com/watch?v=At_Zac4Xezw

83

Comfort

84

HMD comfort

85

HMD comfortNot all HMDs allow for the same degree of freedom in head movements.

86

HMD cables strangling the player

87

HMD cablesWe know the direction the player is facing, and the one he was facing before.

The developer can try to prevent the player from entangling in the cable by using (audio) cues in the reverse direction.

Not every gameplay can use this trick.

88

89

Now the magic formula is

90

Challenge everything I said

91

The future

92

The futureNear future:Larger install base on all platformsNext-level mobile VR thanks to more powerful phonesMore online VRTracked objects in room-scale VR

Tracked objects

94

The futureNear future:Larger install base on all platformsNext-level mobile VR thanks to more powerful phonesMore online VRTracked objects in room-scale VRVR as a social experience

Social VR

96

Entertain the audience

97

Entertain the audience

98

Smooth the camera!

99

Entertain the audience

100

The futureFar future:No cables on tracked HMDsReduced motion sickness thanks to higher resolutions and frameratesGeneral audience will be used to interacting in VRHololens will bring new interest to AR and Mixed Reality

Mixed RealitySource: Owlchemy Labs - http://owlchemylabs.com/owlchemyvr-mixed-reality-tech/

The gallery102

Thank [email protected]@CiroContns