a photon accurate model of the human eye michael f. deering
TRANSCRIPT
![Page 1: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/1.jpg)
![Page 2: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/2.jpg)
A Photon Accurate Model
Of The Human Eye
Michael F. Deering
![Page 3: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/3.jpg)
Use Graphics Theory To Simulate VisionUse Graphics Theory To Simulate Vision
![Page 4: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/4.jpg)
MotivationMotivation
• Understanding the interactions between rendering algorithms, cameras, and display devices with the human eye and visual perception.
• Use this to improve (or not improve) rendering algorithms, cameras, and displays.
![Page 5: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/5.jpg)
Graphics/Vision SystemGraphics/Vision System
Display
ImageGeneration
PostProduction
NeuralProcessing
Display Photons
Eye
![Page 6: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/6.jpg)
Concrete Software DeliverableConcrete Software Deliverable
A computer program to:
• Simulate, photon by photon, several frames of video display into an anatomically accurate human eye retinal sampling array.
![Page 7: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/7.jpg)
OverviewOverview
• Photon counts
• Display device pixel model
• Eye optical model
• Rotation of eye due to “drifts”
• Retinal synthesizer
• Diffraction computation
• Results: rendering video photons into eye
![Page 8: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/8.jpg)
Photons In This Room:4K Lumens: 1019 Photons/SecPhotons In This Room:4K Lumens: 1019 Photons/Sec
14’
17’
75’
~600 photons/60th sec per pixel per cone
![Page 9: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/9.jpg)
Display Pixel ModelDisplay Pixel Model
Each pixel color sub-component has:
• Spatial envelope (shape, including fill factor)
• Spectral envelope (color)
• Temporal envelope (time)
![Page 10: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/10.jpg)
Trinitron™ CRT PixelTrinitron™ CRT Pixel
![Page 11: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/11.jpg)
Direct View LCD PixelDirect View LCD Pixel
![Page 12: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/12.jpg)
3 Chip DLP™ Pixel3 Chip DLP™ Pixel
![Page 13: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/13.jpg)
1 Chip DLP™ Pixel1 Chip DLP™ Pixel
![Page 14: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/14.jpg)
1 Chip DLP™ In This Room1 Chip DLP™ In This Room
![Page 15: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/15.jpg)
Optical Model Of The Eye: Schematic EyesOptical Model Of The Eye: Schematic Eyes
• Historically comprised of 6 quadric surfaces.
• Real human eyes are quite a bit more complex.
• My model is based on:“Off-axis aberrations of a wide-angle schematic eye Model”, Escudero-Sanz & Navarro, 1999.
![Page 16: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/16.jpg)
Eye ModelEye Model
![Page 17: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/17.jpg)
Rotation Of The Eye Due To “Drift”Rotation Of The Eye Due To “Drift”
• When seeing, the eye almost always is slowly drifting at 6 to 30 minutes of arc per second relative to the point of fixation.
• The induced motion blur is important for perception, but rarely modeled.
• (The eye also has tremor, micro-saccades, saccades, pursuit motions, etc.)
![Page 18: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/18.jpg)
Why The Eye Sampling Pattern MattersWhy The Eye Sampling Pattern Matters
![Page 19: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/19.jpg)
Roorda And Williams ImageRoorda And Williams Image
![Page 20: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/20.jpg)
Synthetic Retina GenerationSynthetic Retina Generation
• Some existing efforts take real retinal images as representative patches then flip and repeat. Others just perturb a triangular lattice.
• I want all 5 million cones
– New computer model to generate retinas to order (not synthesizing rods yet).
![Page 21: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/21.jpg)
Retina Generation AlgorithmRetina Generation Algorithm
For more details, attend implementation sketch
“A Human Eye Cone Retinal Synthesizer”
Wednesday 8/3 10:30 am session Room 515B (~11:25 am)
![Page 22: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/22.jpg)
Growth Sequence MovieGrowth Sequence Movie
![Page 23: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/23.jpg)
Growth Movie ZoomGrowth Movie Zoom
![Page 24: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/24.jpg)
Retinal Zoom Out MovieRetinal Zoom Out Movie
![Page 25: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/25.jpg)
3D Fly By Movie3D Fly By Movie
![Page 26: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/26.jpg)
Roorda Blood VesselRoorda Blood Vessel
![Page 27: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/27.jpg)
Roorda Verses SyntheticRoorda Verses Synthetic
![Page 28: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/28.jpg)
The Human Eye Verses Simple Optics TheoryThe Human Eye Verses Simple Optics Theory
All eye optical axes unaligned:
• Fovea is 5 degrees off axis
• Pupil is offset ~0.5 mm
• Lens is tilted (no agreement on amount)
• Rotational center: 13 mm back, 0.5 mm nasal
Eye image surface is spherical
![Page 29: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/29.jpg)
Blur And DiffractionBlur And Diffraction
Just blur Blur and diffraction
![Page 30: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/30.jpg)
Generating a Diffracted Point Spread Function (DPSF)Generating a Diffracted Point Spread Function (DPSF)
• Trace the wavefront of a point source as 16 million rays.
• Repeat for 45 spectral wavelengths.
• Repeat for every retinal patch swept out by one degree of arc in both directions.
![Page 31: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/31.jpg)
Diffracted Point Spread Functions MovieDiffracted Point Spread Functions Movie
![Page 32: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/32.jpg)
Putting It All TogetherPutting It All Together
• Generate synthetic retina.
• Compute diffracted point spread functions by tracing wavefronts through optical model.
• Simulate, photon by photon, a video sequence into eye cones.
• Display cone photon counts as colored images.
![Page 33: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/33.jpg)
Simulating Display And EyeSimulating Display And Eye
• For each frame of the video sequence:
• For each pixel in each frame:
• For each color primary in each pixel:
• From the color primary intensity, compute the number of photons that enter the eye
• For each simulated photon, generate a sub-pixel position, sub-frame time, and wavelength
![Page 34: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/34.jpg)
Simulating Display And EyeSimulating Display And Eye
• From the sub-frame time of the photon, interpolate the eye rotation due to “drift”.
• From the position and wavelength of the photon, interpolate the diffracted point spread function.
• Interpolate and compute the effect of pre-receptoral filters: culls ~80% of photons.
![Page 35: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/35.jpg)
Simulating Display And EyeSimulating Display And Eye
• Materialize the photon at a point within the DPSF parameterized by a random value.
• Compute cone hit, cull photons that miss.
• Apply Stiles-Crawford Effect (I), cull photons.
• Compute cone photopigment absorptance; cull photons not absorbed.
• Increment cone photon count by 1.
![Page 36: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/36.jpg)
30x30 Pixel Face Input30x30 Pixel Face Input
![Page 37: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/37.jpg)
Retinal Image ResultsRetinal Image Results
![Page 38: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/38.jpg)
Lumen Ramp MovieLumen Ramp Movie
![Page 39: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/39.jpg)
30x30 Pixel Movie30x30 Pixel Movie
![Page 40: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/40.jpg)
Result MovieResult Movie
![Page 41: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/41.jpg)
How To Test Model?How To Test Model?
![Page 42: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/42.jpg)
How To Test Model?How To Test Model?
• Test it the same way we test real eyes.
![Page 43: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/43.jpg)
20/27
20/20
20/15
20/12
20/9
20/20
20/27
20/15
20/12
20/9
![Page 44: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/44.jpg)
Sine Frequency RampSine Frequency Ramp
20 cycles 80 cycles40 cycles
![Page 45: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/45.jpg)
Maximum Drift MovieMaximum Drift Movie
![Page 46: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/46.jpg)
Maximum Track MovieMaximum Track Movie
![Page 47: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/47.jpg)
Next StepsNext Steps
• Continue validation of model and adding features.
• Simulate deeper into the visual system:
– Retinal receptor fields
– Lateral geniculate nucleus
– Simple and complex cells of the visual cortex.
![Page 48: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/48.jpg)
AcknowledgementsAcknowledgements
• Michael Wahrman for the RenderMan™ rendering of the cone data.
• Julian Gómez and the anonymous SIGGRAPH reviewers for their comments on the paper.
![Page 49: A Photon Accurate Model Of The Human Eye Michael F. Deering](https://reader036.vdocument.in/reader036/viewer/2022062718/56649eba5503460f94bc2581/html5/thumbnails/49.jpg)