virtual techniques 2018 - virtual dimension center€¦ · advantages hmds: cheap mobile very high...

© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)

Virtual Techniques 2018 VDC-Trend Report

Dr.-Ing. Dipl.-Kfm. Christoph Runde Virtual Dimension Center (VDC) Fellbach Auberlenstr. 13 70736 Fellbach www.vdc-fellbach.de

Mission Display & Projectors Tracking & Capturing 3D Data Supply 3D Controller TOP 1 HMDs & Smart Glasses Other Outputs Market

Virtual Techniques 2018

VDC-Trend Report

Agenda

2

Display and

Projection technology

Head Mounted Displays

and Smart Glasses

Inputs and Controller

Haptics and

Force Feedback

Tracking, MoCap,

Face Capture, Eye Tracking

Scanning,

Digitizing

Markets



VDC-Trend Report

Combinations of LASER projection and Infitec

technology

Advantages:

Very low brightness losses due to filter technology,

since the color frequencies of the glasses

and the projector are coordinated (coherently)

High light output

Very good channel separation

Very high contrast

Very low wear

Passive glasses

Any projection medium

No calibration of two projectors

Display and Projection technology

3

Barco DP4K-60L <

INFITEC

Display & Projection

2500:1



VDC-Trend Report

Combination LCD projectors and

polarizing filters

Reason: pre-polarization of the LCD

panels

Double projection with two projectors

with 2 complementary polarizing LCD

panels

But: LCD replaced by

High-end sector: LASER, LED, 3-chip

DPL projectors,

In mid-range through single-chip DLP

LCD still represented in consumer

products


4

Barco iQ R500 LCD projector

5000 lumen

BARCO Stereo Lumens Boost

20%

2000 stereo lumen

5000 lumen

4000 stereo lumen

100% improvement

60%

External polarization of DLP projectors

Polarization filter glasses




VDC-Trend Report

Combination plasma screens and shutter glasses

Plasma technology:

Light generation by phosphors encouraged by

UV radiation from plasm discharges

The discharge takes place in cells located between 2

panes of glass, which are filled with neon, xenon and

possibly helium, after electrical ignition of the gas for

changing the state of aggregation in plasma

Different levels of brightness are achieved by the

number of ignitions

Synchronization ignitions - transparency pane of the

glasses

Thus: increase in brightness with active stereo


5

Nvidia 3D Vision 2




VDC-Trend Report

But: termination of the production of plasma and

LCD TVs

Pioneer: 2009, likewise: LCD TVs

Panasonic: 2014

LG: 2014

Furthermore:

3D TV discontinued in 2016 by LG, Sony,

Samsung

3D channel of pay-TV station Sky will also be

discontinued in mid 2017

Remained: Panasonic


6




VDC-Trend Report


7

It is and always was valid:

3D needs size




VDC-Trend Report

LED Walls

Very high contrasts

Very high brightness

Very long lifetime

Very fast commissioning

Stereo compatible

Pixel 0.97mm x 0.97mm

Resolution basically

unlimited by tiling

Complex AV technology to

control high-resolution walls


8

Viscon

Barco




VDC-Trend Report

Bench systems: products expired long ago

Planning desks: never really established in the

industry

CAVE systems:

oDismantling @ Fraunhofer IAO

oDismantling @ BMW

All-around projections: more often only in

driving simulators because of peripheral view

Most common industry standard:

Powerwall, possibly as an

array, but seldom ground

projection

Projection systems: Types of construction

9

BARCO

Consul

BARCO

Baron

TAN

Responsive

Workbench

TAN

Holobench

Working

situation at

planning

desk

Scheme

planning desk

Imsys scale XL with

41 million pixels

on 5.5 x 2.2m

6- sided CAVE @

Fraunhofer IAO




VDC-Trend Report


10

HMDs & Smart Glasses



VDC-Trend Report

Advantages HMDs:

Cheap

Mobile

Very high immersion

Very good interior

atmosphere

Disadvantages HMDs:

Isolation

Partially acceptance and

ergonomics

Image quality


11


Spatial Exploration

Testing Training

Product presentation

Group / Individual Individual

Styling Review Group

Ergonomics Individual Individual

Production planning Group

Buildability inspections

Group Individual

Briefing Individual Individual Individual / Group

Driving simulation Individual Individual / Group

Visualization of simulation data

Group

Architecture (office, factory)

Group

Areas of application vs. 3D interface

Group: decission and discussion at the presentation



VDC-Trend Report

Good research approaches

HMDs and Displays: Holographics

12

Polymer-based screen renews three-dimensional projections every two

seconds (Source: University of Arizona, 2010) Qinetiq developed a holographic display prototype based on spatial light

modulation technology years ago (year 2000). It used an active tiling system with

two different spatial light modulators to provide all the depth cues needed to

produce a 3D image. It was expensive to produce and was discontinued shortly

after development, yet still is the closest true holographic display to be

demonstrated (Source: (Qinetiq , 2000)

Holographic video of a butterfly. The core of the novel holographic display are anisotropic

waveguides made of lithium niobate crystals. Compared to older approaches to holographic

video displays, these light modulators allowed a ten times higher pixel rate of up to 50 billion

dots per second. This is necessary in order to be able to transport and project the interference

patterns, which are important for moving holograms, from a reference wave and an object

wave in large numbers. ((Image: D. Smalley, MIT, Juni 2013)




VDC-Trend Report

First products, good prototypes

HMDs and Dislays: Plenoptics

13

Nvidia’s Near-Eye lightfield display prototype was presented in July 2013

for the first time in the course of the SIGGRAPH computer graphics

conference:

Technical details

Display: 2 Sony ECX332A OLED micro displays

Display size: 15,36 x 8,64 mm

Display resolution: 1280 x 720 pixels (24-bit color depth, 2100 ppi)

Effective resolution: 146 x 78 pixels

Field of vision: up to 70° (29° x 16° in Demo)

Lightfield technology HMD

(University of Stanford, 2015)

Holografika HoloVizio C80 3D cinema system




VDC-Trend Report

Focal HMDS: prototypes

Layer displays (multiplanar 3D displays): product status

HMDs and Displays: Focal & Layer displays

14

Scheme multi-layer display

(Source: Lightspace

Technologies)

View through focal display (Source: College of Optical

Sciences, University of Arizona): “The focal plane of the

display is dynamically controlled, matching the depth cue

of virtual objects with the focal distance of the display.




VDC-Trend Report

VR displays with 4K x 4K pixels, viewing angle of

140 °, pixel density 30p / °

Fixed depth of focus will be variable, from 25cm to

(research: holography, plenoptics (light field),

progressive lens, layer displays)

Foveated Imaging (Multi-Res Shading, such as

Nvidias GameWorks VR), therefore necessary:

o Re-Design of the render pipeline

o Integrated Eye-Tracking

Right auralization instead of only stereophony

Current 3D controller: probably still in 5 years

good standard

Currently not in sight: inexpensive hand tracking

with all 22 degrees of freedom of the hand

HMDs: Several forecasts - in 5 years

15

Image: Nvidia

Multi

Resolution

Shading

Image: Nvidia Timewarp


Michael Abrash: Microsoft, id

Software, Valve, Oculus VR



VDC-Trend Report

Topic currently hits the ground running (esp. regarding hardware)

exception: Magic Leap

Smart Glasses

16

Microsoft Hololens

ODG R7

Daqri Smart Glasses

Magic Leap / Google (Feb. 2017)

META


Magic Leap Rendering (Dez. 2017)



VDC-Trend Report

Image: BMW Image: Picavi Image: Volvo

Requirements?

What is the target result of the application of data glasses? How will the success be

measured? And what is the user´s benefit?

Business models?

Smart Glasses

17


Logistics

• Faster

• Error-free commissions

Maintenance, Repair, Service

• Reduce qualification requirements

• Concentrate experts in strategic projects

• Reduce travel costs

• Provide solutions ad-hoc

• Documentation procedure

Production

• Accelerate training

• Increase quality

• Be in control of customized products

• Documentation procedure



VDC-Trend Report

Long-established systems such as

the Spacemouse variants

User interfaces for driving and

flight simulators from the

consumer electronics

Force-feedback systems:

available as mature products

Main applications:

o Implicit design

o Teleoperations

o Simulators

3D-Controller: Desktop

18

3DConnexion

Spacemouse

3DConnexion

SpacePilot

3DConnexion

SpaceTraveller

Saitek Pro Flight X-56

Rhino H.O.T.A.S.

System

Logitech

G27

Geomagic

TouchTM

Haptic 3D

Stylus

Haption

Virtuos

3D-Controller



VDC-Trend Report

Prototypical ideas for

force-feedback systems: force

generation at distance

oVia ultrasonic

oVia air pressure

3D-Controller: Desktop

19

Disney Research Aireal

Ultrahaptics

3D-Controller



VDC-Trend Report

Well established industrial

systems such as the A.R.T

Flystick generations

For a few years very good,

powerful competitors from

the consumer electronics

Also systems with tactile

feedback on the market

3D-Controller: Hand-held controllers

20

Tactical Haptics

LLC Reactive Grip

Valve/HTC Vive

Controller Oculus

Touch

Sony

Playstation

Move

Nintendo

Wii Mote

Razer

Hydra

A.R.T.

Flystick 3

3D-Controller



VDC-Trend Report

Game consoles from consumer

electronics: 2 x 2DOF and multiple binary

inputs

Tek Gear Twiddler: Fist keyboard

Novint XIO: Exoskeleton

3D-Controller: Hand-held controllers

21

Novint XIO

Tek Gear Twiddler

Logitech

F710

Microsoft

Xbox 360

Controller

Mad Genius

MGC

Controller

3D-Controller



VDC-Trend Report

Many technological variants

o Optical

o Electromechanical

o Myoelectrical signals of the muscles

o IMUs

Different degrees of detection

o Only hand

o Finger 5DOF

o Hand 22DOF

3D-Controller: Gesture recognition

22

Intellect

Motion

iMotion

Dexta

Robotics

Dexmo

Exoskeleton

A.R.T. finger

tracking

Thalmic Labs Myo strap

Valve/HTC

Vive Tracker

3D-Controller



VDC-Trend Report

Advantages depth imaging sensors:

oMeasurement at a distance and contactless

o Simple MoCap

Functional principles

o Time-of-Flight

o Structured (IR-)light

Two of the big IT companies on the

market (Google, Microsoft)

3D-Controller: Gesture recognition with depth imaging sensors

23

MindMaze:

Mindleap

Nimble VR:

Nimble Sense

Microsoft :

Kinect and

Handpose

Google :

Leap Motion

3D-Controller



VDC-Trend Report

Illusion of running by sliding of the foot in a

running cage

Always in combination with HMDs

Driven by gaming sector

Attention: for the orientation and for the

construction of a conceptual spatial model,

the information of the vestibular perception

(inner ear) is processed with further sensory

impressions:

o the visual perception,

o the sense of touch

o depth sensitivity (position,

movement, power sense)

3D-Controller: Treadmills - I

24

Cyberith:

Virtualizer

Virtuix :

Omni

Wizdish:

Rovr

3D-Controller



VDC-Trend Report

Further technological approaches:

moving grounds such as

o 2DOF treadmills

o 2DOF ground of a sphere

o Inside area of a sphere

3D-Controller: Treadmills - 2

25

Infinadeck

Omnifinity

Omnideck

Walkmouse

Virtuesphere

3D-Controller



VDC-Trend Report

Availability of „exotic“

controllers is increasing

o Brain-computer interfaces,

whose performance is

steadily increasing

o Bike mount and sensor

o Bird flight simulator

3D-Controller: Flight, Bike, Brain computer interface

26

Somniacs:

Birdly

Widerun

Emotiv

3D-Controller



VDC-Trend Report

Tactile vests that generate pulses or

vibration in case of collision or fire

Partially easy pairing with audio

output

Concept is old, but currently the

number of products on the market is

rising again

Possibly conceivable also for

"accessibility studies" (for example:

does probant permeate geometry?)

Other outputs: Tactile feedback

27

Immerz:

Kor FX

IFTECH:

Araig

Woojer:

Strap

Woojer:

Vest

Other Outputs



VDC-Trend Report

Old idea, new products

Main problem: synthesization

of a very high number of

odorants

Currently driven by the

gaming sector

Other outputs: Smell

28

Current:

Feelreal Helmet

Current: Olorama

Stationary desktop system: prototype of France

Telecom (extension WWW-output)

Portable odorant emitter for the

training of firemen

(question: „what is burning?“)

Stationary system for rows of

chairs (cinema application)

Inition

Other Outputs



VDC-Trend Report

Optical systems have prevailed

over other technologies

(electromagnetic, ultrasound, ...)

for a long time

Since a few years strong

competition by price-favorable

optical systems from the consumer

electronics

Possibly combination with 9DOF-

IMUs, whose prices have

meanwhile arrived at some €

Tracking

29

Valve/HTC Lighthouse

Oculus

Tracking

A.R.T.

Tracking

Systems

Vicon Optitrack

9DOF IMUs

Tracking & Capturing



VDC-Trend Report

Here, too, established vendors such

as Vicon have experienced

tremendous growth of competition

through systems such as Sixense,

PrioVR, Perception Neuron, Stompz,

Xsens, ...

Motion-Capturing

30

Noitom

Perception

Neuron

Vicon

Yei

Technology

Prio VR Pro

Sixense

Entertainment

Stem System

talaries

Technologies

Stompz

Tesla

Studios

Tesla Suit

Xsens Xsens

MTI-Series/

Xsens MVN

Xsens




VDC-Trend Report

HMD's integrated eye tracking was

immediately an issue with the advent

of low cost HMDs

Main areas of application:

usability, design studies, market

research

Solutions for

o integrated HMDs (Rift, Vive)

o smartphone systems

o complete systems (FOVE)

Provider: SMI, tobii, FOVE, Arrington

Research

VR-integrated Eye Tracking

31

SMI for

Samsung‘s

Gear VR

SMI for

Oculus Rift at

Carrefour

Arrington

Research

FOVE:

HMD with

integrated eye

tracking




VDC-Trend Report

Mimic: essential for animated film

Facial expressions also important as

nonverbal communication in

cooperation

Good hardware and software

solutions from the film industry

Purchase of companies

Relevance for VR:

Mimic as additional channel in

distributed virtual environments

Missing facial expressions as a

limitation in cooperation with HMDs

Solutions:

Face Capture

32

Combination Vicon

and Dynamixyz

November 2015

Facebook

Live

Reactions

November 2016




VDC-Trend Report

Still crucial:

efficient 3D content supply chain

3D Data supply

33

3D laser scan of a chemical plant 3D construction of a machine

3D simulation casting process

3D terrain model through

airborne laser scanning

3D measuring data presentation for

deposit inspection

3D Data Supply

3D city model through driving

and overflying



VDC-Trend Report

Number of possible sources is

steadily increasing

3D Data supply

34

3D body scan 3D ultrasound

3D-CT

3D-MRT

3D reconstruction with depth

imaging sensor (Google Tango)

3D reconstruction with photos

(Autodesk 123D Catch, Packshot)

3D Data Supply



VDC-Trend Report

With industry- and application-specific

functions in the engineering sector are ...

Software market VR: Established solution provider

35

ESI in use

….

Market



VDC-Trend Report

... increasingly challenged by service

providers (in large numbers!) and

solutions based on

Game Engines

VR middleware solutions

Software market VR Game Engines

VR-Middlewares

Market



VDC-Trend Report

Democratization of VR via cardboard

systems and smartphones: elimination of

any price barrier for simple VR

VR-Communities,

collaborative VR/AR environments

Software market VR

VR-Headset in combination with smartphone:

Samsung GearVR.

Market



VDC-Trend Report

AR now finally mass market (hardware

smartphone and tablet PC)

Challenges for software manufacturers:

variety of platforms + hardware

Software market: Augmented Reality

Re‘flekt ONE in use

AR platforms:

AR provider:

Market



VDC-Trend Report

Market movements: Industries

39

1980 1990 2000 2010 2017

Time

Training

Product

development,

Training

PD,

Training

PD

Training

PD

Market

research

Presentation

Presentation

Service

Training

Market



VDC-Trend Report


Market



VDC-Trend Report

Industry 4.0 is the vision

of the industrial platform of the

future

Nine technologies are transforming

industrial production


Market



VDC-Trend Report

Base case user

and software

revenue

assumptions

Market

42

Source: Goldman Sachs Global

Investment / Data point starts from:

Gartner, IDC, World bank, US Bureau

of Labour Statistics, National Center

for Education Statistics, Nielsen, FIFA,

Americam Medical Association,

Research and Markets, National

Associations of Realtors, OC&C

Strategy Consultants, the Japan

Ministry of Land, Infrastructure,

Transport and Tourism, the Land

Institute of Japan, Borrell

Associations, CAE, Eurostat, and

Statistics Japan.

Market



VDC-Trend Report

Dominating manufacturers in the consumer-VR market

Market

43

Source: The Nunatak Group

Market



VDC-Trend Report

Progression of base case hardware and software forecasts

Market movements: Growth

44

Source: Goldman

Sachs Global

Investment Research.

Market



VDC-Trend Report

Significant growth of VR market, promoted by high invests

Market movements: Invests

45


Market



VDC-Trend Report

Market movements: Invests and acquisitions

46

Source: News sources, compiled by Goldman Sachs Global Investments Research

Market



VDC-Trend Report

162 million HMD-users worldwide

Market movements: 2020

47

Source: Nunatak forecasts (June 2016), BGR (2016), IDC (2016)

Market



VDC-Trend Report

Predicted composition of the VR/AR

market, grouped by applications


48

Source: Goldman Sachs Global InvestmentResearch.

Goldman Sachs 2025 VR/AR software estimates by use case.

„Consumer-driven use cases in videogames, live events and video-driving ~60% of software spend with the remainder from enterprise and public sector“

Market



VDC-Trend Report

Market movements: Invests and acquisitions

Engin

eering-I

T

Ele

ctr

onic

s

Pla

tform

s

Ente

rtain

ment

Market



VDC-Trend Report

Consideration of HMDs for consumer applications

Market movements: Ecosystems

50


Market



VDC-Trend Report

The number of digital ecosystems in the field of AR will rise

Market movements: Ecosystems

51

ODG R7

Epson Moverio BT300

Microsoft Hololens

Daqri Helmet

META 2

Apple (announced)

Market



VDC-Trend Report

Trends VR & AR: Merging of disciplines

AR projection on

3D-print or clay

model [Source:

Volkswagen]

Market



VDC-Trend Report

53

Use of VR-HMD in

seat box [Source:

BMW]

Trends VR & AR: Merging of disciplines

Market



VDC-Trend Report

Market



VDC-Trend Report


56

In an exclusive interview with ABC News, Tim Cook reaffirms

Apple's attitude to virtual and augmented reality. The lower

immersion with Augmented Reality may be the big advantage of

the technology, says Cook. This would allow to perceive digital

content and other people simultaneously. "Maybe it's something

we're talking about, maybe something that happens somewhere

else that you could let appear. There are many cool things”, says

Cook. He could, for example, allude to phone calls with hologram-

like illustrations, which Microsoft already showed in the research

project for “Holoportation".

"There is Virtual Reality and Augmented Reality. I think augmented reality will be the bigger market - by far." September 2016: Virtual Reality

seems to have a lower priority for

Apple than AR:

Source: Goldman Sachs Global InvestmentResearch.

Goldman Sachs 2025 software estimates by VR and AR. „VR

use cases driving 75% of our software estimates; as AR

technology matures we expect more enterprise use cases to

emerge“

?

Market

© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)

Thank you very much for your interest!

You are interested in this topic and you are looking for contact persons/

implementation partners? Please contact us.

Virtual Dimension Center (VDC) Fellbach Auberlenstraße 13 70736 Fellbach www.vdc-fellbach.de

virtual techniques 2018 - virtual dimension center€¦ · advantages hmds: cheap mobile very high...

Documents