Upload File

near-eye light field displays - nvidia...title: take your head out of that box: sunglasses-like...

  • Home
  • Documents
  • NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We
54
NEAR-EYE LIGHT FIELD DISPLAYS Douglas Lanman and David Luebke NVIDIA Research

Upload: others

Post on 05-Oct-2020

0 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NEAR-EYE LIGHT FIELD DISPLAYS

Douglas Lanman and David Luebke

NVIDIA Research

Page 2: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 3: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

5DT HMD 800 Cybermind NVIS nVisor SX60 Rockwell Collins SimEye

Sensics piSight Sensics xSight Sensics zSight

Oculus Rift

Virtual Realities VR2000

Sony HMZ-T1 Carl Zeiss Cinemizer Silicon Micro Display ST1080

Page 4: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 5: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 6: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

microdisplay

microlens array

Page 7: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 8: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 9: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 10: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 11: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Benefits • thin

• lightweight

• immersive (wide fields of view)

• comfortable - addresses accommodation-

convergence conflict

- approximates retinal defocus cues

- no need for corrective eyewear

Limitations • reduced spatial resolution - high-resolution or tiled microdisplays

- careful microlens array selection

• requires larger microdisplays

• requires user calibration

Page 12: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Designing Near-Eye Light Field Displays

Page 13: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

microdisplay

microlenses

microdisplay

microlenses ?

Page 14: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

microdisplay

lens

microdisplay

microlenses ?

Page 15: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Magnifier Array

we

eye

f

eye relief (de)

(virtual) object distance (do)

dl

display

lens array virtual object

(plane of focus)

retinal image

Page 16: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Magnifier Array: Light Field Synthesis

we

eye

f

eye relief (de)

(virtual) object distance (do)

dl

display

lens array virtual object

(plane of focus)

a

retinal image

Page 17: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 18: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

LVT Film-based Prototype

Page 19: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential Backlit LVT Film

Microlens Array

Page 20: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 21: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 22: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 23: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 24: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 25: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Binocular OLED-based Prototype

Hardware Construction

Page 26: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 27: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 28: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 29: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 30: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 31: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 32: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 33: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 34: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 35: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 36: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 37: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 38: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 39: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 40: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Binocular OLED-based Prototype

Experimental Results

Page 41: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Binocular OLED-based Prototype

Experimental Results

Page 42: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Binocular OLED-based Prototype

Stereoscopic Light Field Rendering

Page 43: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Near-Eye Light-Field Rendering

Page 44: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Rendering as Off-Axis Perspective Projections

Left Elemental Image

Page 45: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Left Elemental Image Center Elemental Image

Rendering as Off-Axis Perspective Projections

Page 46: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Left Elemental Image Right Elemental Image Center Elemental Image

Rendering as Off-Axis Perspective Projections

Page 47: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Ray Tracing for Near-Eye Light Fields

u s

(u,s) s

t

u

v

Real-Time Ray Tracing for Near-Eye Light Field Displays

• Implemented in NVIDIA OptiX SDK 3.0.0

• Added support for quad-buffered stereo

• Renders 1280×720 rays per eye (simulating 13×7 off-axis perspectives)

• Runs at 30 – 60 frames per second (GeForce GTX TITAN)

• Implemented interactive calibration to compensate for optical aberrations

Page 48: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 49: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 50: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Page 51: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Backward Compatibility Case Study:

Resampling Doom 3 BFG Edition

Page 52: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Benefits • thin

• lightweight

• immersive (wide fields of view)

• comfortable - addresses accommodation-

convergence conflict

- approximates retinal defocus cues

- no need for corrective eyewear

Limitations • reduced spatial resolution - high-resolution or tiled microdisplays

- careful microlens array selection

• requires larger microdisplays

• requires user calibration

#GTC2014

@douglaslanman @davedotluebke

Page 53: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NEAR-EYE LIGHT FIELD DISPLAYS

Douglas Lanman and David Luebke

NVIDIA Research

#GTC2014

@douglaslanman @davedotluebke

Page 54: NEAR-EYE LIGHT FIELD DISPLAYS - NVIDIA...Title: Take Your Head Out of That Box: Sunglasses-Like Virtual Reality Eyewear Using Light Field Displays Author: Douglas Lanman Subject: We

NVIDIA Confidential

Realizing Practical Applications


Supplementary Appendices - NVIDIA · 2020. 6. 26. · Supplementary Appendices Cascaded Displays: Spatiotemporal Superresolution using Offset Pixel Layers Felix Heide Douglas Lanman

Lanman 3dim07

Light-Field Displays: a View-Dependent Approach...vergence–accommodation conflict.Optics Express 26, 23 (2018). Gordon Wetzstein, Douglas Lanman, Matthew Hirsch, and Ramesh Raskar

Manufacturing Flexible Light-emitting Polymer Displays With Conductive Lithographic Film Technology

Light Tech Displays

Texture Synthesis and Manipulationalumni.media.mit.edu/~dlanman/courses/en256/Lanman-Quilting-Proposal.pdfDouglas Lanman 9 Pixel-based Texture Synthesis References: [3,4,5,6] Pixel-based

Sanskrit Reader Lanman

Optical Properties II: Emission of Light, Displays and Transparent Conductors

Texture Synthesis and Manipulation - Brown Universitymesh.brown.edu/dlanman/courses/en256/Lanman-Quilting-Proposal.pdfDouglas Lanman 9 Pixel-based Texture Synthesis References: [3,4,5,6]

rendering and interaction on projection-based light field displays

Near-Eye Light Field Displays-0 - NVIDIA Research …research.nvidia.com/sites/default/files/publications/neld-abstract.pdf · Near-Eye Light Field Displays Douglas Lanman David Luebke

Bidi Screen: A thin, Depth-Sensing LCF for 3D Interaction using Light Fields Mattew Hirsh, Douglas Lanman, Henry Holtzman, Ramesh Raskar MIT Media Lab

Light-Field Displays: a View-Dependent Approach · Tuotuo Li, Qiong Huang, Santiago Alfaro, Alexey Supikov, Joshua Rat-cliff, Ginni Grover, and Ronald Azuma. 2020. Light-Field Displays:

LCD displays/projectors Adjust light Adjust color

NEAR-EYE LIGHT FIELD DISPLAYSon-demand.gputechconf.com/.../S4337-near-eye-light-field-displays.pdf · NEAR-EYE LIGHT FIELD DISPLAYS Douglas Lanman and David Luebke NVIDIA Research

SIGGRAPH 2012 Computational Display Course - 3 Computational Light Field Displays

Shape from Depth Discontinuities under Orthographic …hauagge/files/pdf/Lanman-orthoflash-3DIM-2009.pdfShape from Depth Discontinuities under Orthographic Projection Douglas Lanman,

Light Field, Focus-tunable, and Monovision Near-eye Displays | SID 2016

Organic light emitting diode (OLED) displays

Near-Eye Light Field Displays-0 - NVIDIA · We propose near-eye light field displays that enable thin, lightweight head-mounted displays (HMDs) capable of presenting accommodation,

Reporting Guide for Laser Light Shows and Displays

Quantum Dot Light-emitting Diode for Displays with Vivid Color · Quantum Dot Light-emitting Diode for Displays with Vivid Color Emission color Quantum dot size Figure 2: Light emission

Perceptually-Guided Foveation for Light Field Displays

Neural Light Field 3D Printing - quan-zheng.github.io · Modern 3D printers are capable of printing large-size light-ield displays at high-resolutions. However, optimizing such displays

Data Sheet€¦ · Segment Displays, 7.6 mm/10.9 mm Seven Segment Displays, 14.2 mm Seven Segment Displays, 20 mm Seven Segment Displays, or High Light Ambient Seven Segment Displays

Surround Structured Lighting for Full Object Scanningmesh.brown.edu/dlanman/research/3DIM07/lanman-Surround...Surround Structured Lighting for Full Object Scanning Douglas Lanman,

Lanman 3dpvt06-reconstruction

RFID Network Infrastructure: Backgrounder IEEE LANMAN 2007

Lanman In Painting Final Report

High Performance Liquid Crystals for Displays and Spatial Light Modulators

Compressive Light Field Displays

Recognition of animal locomotion from dynamic point-light displays

Light-Field Displays - JPEG - Light-field... · Light-field displays • HoloVizio by Holografika • Optical modules –Project light beams to hit the points of a screen with multiple

MEMS Spatial Light Modulator for Holographic Displays

ENGN 2911 I: 3D Photography and Geometry Processing ...alumni.media.mit.edu/~dlanman/courses/cs220/Lanman-structured-li… · di erent bit planes of a Gray code structured light sequence,