defesa de tese

Interactive Measurements and Tailored Displays for Optical Aberrations of the Human Eye

Vitor PamplonaAdvisor: Manuel M. Oliveira Neto

2

Thesis Contributions

NETRA CATRA TailoredDisplays

Cell Phone-based Refractive Measurements

The Inverse of Shack-Hartmann

with: Manuel Oliveira, Ankit Mohan, Ramesh Raskar

Unique, low-costquantitative lens mapping

CATRA: Quantitative Cataract Maps

with: Erick Passos, Jan Zizka, Everett Lawson, Esteban Clua , Manuel M. Oliveira, Ramesh Raskar

5with: Manuel Oliveira, Daniel Aliaga, Ramesh Raskar

6



NETRA: Measuring Refractive Errors and Focal Range

Thermometer for vision

Vitor F. Pamplona Ankit Mohan Manuel M. Oliveira Ramesh Raskar

9

Millions have poor vision, but are not getting corrected…

Kenya

India

7 Billion people

5B have a cell phone

2B haverefractive errors

0.6B have URE

Source: World Heath Organisation, Vision 2020 Tech Report.

10

Emerging Asia

Africa & Middle East

Latin America

Europe North America

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60

1.801.70

0.50

0.13 0.10 0.02

Billions of People with Uncorrected Refractive Error, by

Region

Source: Essilor, Infomarket 2009, CPB Research, numbers may not add due to rounding

1.6 Billion 2.4 Billion

2.4 Billion People w/out Glasseswho need them around the world

11

Shack-Hartmann Wavefront Sensor

Expensive; Bulky, Requires trained professionals

Wavefront aberrometer

12


Laser

Sensor Microlens Array

Planar Wavefront

Spot Diagram

13

Laser

Sensor

Displacement = Local Slope

of the Wavefront

Spot Diagram


14

NETRA = Inverse of Shack-Hartmann

Spot Diagram on LCD

Cell Phone Display

Eye Piece

15

Spot Diagram on LCD

Inverse of Shack-HartmannUser interactively creates the Spot Diagram

16

Spot Diagram on LCD

Inverse of Shack-HartmannUser interactively creates the Spot Diagram

18

Eye

Relaxed Eye with Myopia

Blurred point

Pointat infinity

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

19

Eye


Distinctimage points

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Pinholes

Pointat infinity

Scheiner’s Principle

20

Eye


Virtual pointat infinity

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

A

B


Display

21


A

B

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Virtual pointat finite distance

Move spots towards each

other

Display

Eye


22

B

A

Points overlap


Display

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Eye



other

23

B

A

Points overlap


Display

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Eye



other

d

24

Points overlap


perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Eye

d

d1

Point at infinity

25

B

A

Points overlap


Display

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Eye

Virtual red pointat finite distance


other

d

f t

ac

26

Interactive Method

Farthest Focal Point(myopia, hyperopia)

27

Interactive Method


28

Interactive Method


Astigmatism: angle-dependent refractive error

http://www.elizabethpope.co.uk/eyeinfo/astigmatism.html

32

Astigmatism

Cross or points may never meet with a 1d search !

33

Astigmatism

Lines reduce the problem to a 1d search

34

Interactive Method

Farthest Focal Point(myopia, hyperopia, astigmatism)

35

Interactive Method


36

Interactive Method


37

Interactive Method


38

Best fitting on a Astigmatic Curve

Cylinder SphereAxis Cyl.Unknowns:

SCP )(sin( 2

Interactive Method

Farthest Focal Point(myopia, hyperopia, astigmatism) 39

40

Measuring Accommodation Range

Perfect vision

~10cmInfinity

Myopia

Hyperopia

Step 1: Far limit Step 2: Near limit

41


Perfect vision

~10cmInfinity

Myopia

Hyperopia


42


Perfect vision

~10cmInfinity

Myopia

Hyperopia


43

Points overlap

A

B

Display

Virtual Point at the far limit

Relaxed Eye

44

Points overlap

A

B

Display

Accommodated Eye

Virtual pointgetting closer

Move points towards each other

Subject Accommodates to fix the “blur”

45

Points overlap

A

B

Display

Accommodated Eye



Subject Accommodates to fix the “blur”

46

Points overlap

A

B

Display

Accommodated Eye



Subject cannot accommodate more than the previous point

47

Patterns for Alignment Task

Displayed

Subject view

Displayed

Subject view

Visual Cryptography

A B A B A B A B A B

A B A B A B A B A B

[NaorShamir94]

48


Displayed

Subject view

Displayed

Subject view

Visual Cryptography

A B A B A B A B A B

A B A B A B A B A B

[NaorShamir94]

49


Displayed

Subject view

Displayed

Subject view

Visual Cryptography

A B A B A B A B

A B A B A B A B A B

A B

[NaorShamir94]

52

Summary of Interaction

Farthest Point(myopia, hyperopia, astigmatism)

Nearest Point(presbyopia)

Accommodation Range

53

Device Resolution

Channel Size25um

Resolution is a function of the display DPISamsung Behold II – 160 DPI = 0.35DGoogle Nexus One – 250 DPI = 0.20DApple iPhone 4 – 326 DPI = 0.14D

55

Limitations

• Children• Ability to align lines

• Resolution is a function of the display DPI– Samsung Behold II – 160 DPI – 0.35D– Google Nexus One – 250 DPI – 0.2D– Apple iPhone 4G – 326 DPI – 0.14D

Evaluation Prototype

LCD Display

Minification

56

Camera simulatesthe perfect eye

Trial lenses simulate lens aberration

57

ACM SIGGRAPH 2010

58

US + International Patent

60

Chad FowlerFounder and CEO - YouTube

61

Early Awards + Recognition

NETRA: $50K Research Grant

NETRA: $50K Innovation Grant

63

NETRA Prototypes Worldwide29 partners in 14 countries.

http://www.youtube.com/watch?v=PvT9tD3NZvo

64

Mumbai Slum Outreach: Lotus Eye Hospital in Dharavi

Hyderabad Eye Hospital: LV Prasad Eye Institute

Patient’s home, Teresina, PI, Brazil

School, Nairobi, Kenya

Mumbai Optical Shop

Chennai Optical Shop

Mumbai Optical Shop

Hyderabad Eye Clinic: LV Prasad Eye Institute Primary vision center

65

NETRA vs Manifest Refraction

-5.00 -4.00 -3.00 -2.00 -1.00 0.00 1.00 2.00 3.00 4.00 5.00

-5.00

-4.00

-3.00

-2.00

-1.00

0.00

1.00

2.00

3.00

4.00

5.00

Subjective Refraction (D)Ne

tra

(D)

13 adults – 0.3D Average Difference

66

Frontiers In Optics & American Academy of Optometry

2010

67

mHealth Summit 2010

68

NETRA Team at LVPEI, India

-7.00 -5.00 -3.00 -1.00 1.00 3.00 5.00 7.00

-7.00

-5.00

-3.00

-1.00

1.00

3.00

5.00

7.00

Retinoscopy (D)

Netr

a (D

)


NETRA vs Retinoscopy

69

Association for Research on Vision and Ophthalmology

70

OneSight, Kenya

71

NETRA Team at Conceição, Brazil

-4.00 -3.00 -2.00 -1.00 0.00 1.00 2.00 3.00

-5.00

-4.00

-3.00

-2.00

-1.00

0.00

1.00

2.00

3.00

AR1 Linear (AR1) Linear (AR1)AR2 Linear (AR2) Linear (AR2)Linear (AR2) SR Linear (SR)

NETRA


NETRA vs Auto-Refractor

72

Pan-American and Brazilian Ophthalmology Congress

73

Association for Research on Vision and Ophthalmology

74

NETRA team at NECO

-8.00 -7.00 -6.00 -5.00 -4.00 -3.00 -2.00 -1.00 0.00 1.00

-8.00-7.00-6.00-5.00-4.00-3.00-2.00-1.000.001.00

AR Linear (AR) Linear (AR)Subj Linear (Subj)

NETRA

11 adults – 0.34D Average Difference from Subjective Evaluation with no cycloplegia

71% of the measurements have a max error of 0.5D

75

American Optometry Academy 2012

2012

78

$300K Vodafone Award

http://project.vodafone-us.com/

79

http://www.youtube.com/watch?v=PvT9tD3NZvo

80Ron GaranISS Astronaut

http://launch.org/forum/2/health/innovators/18/netra

8383

83

http://eyenetra.com

Picture: Anderson Maciel.

87



CATRA: Interactive Measuring and Modeling of Cataracts

Vitor F. Pamplona Erick B. Passos Jan Zizka Manuel M. Oliveira Everett Lawson Esteban Clua Ramesh Raskar

MIT Media Lab – Camera Culture

89

CATRA: Quantitative Maps for Self-assessment of Early Cataracts

90

Main Cause of Preventable Blindness

92

Slit-Lamp Microscope

94

CATRA

95

Four Resulting Maps

Opacity Map(position, size)

Attenuation Map(brightness)

Contrast Map(contrast) PSF Map

CCCCCC

CCC

CC

CCC

CCC

CC

CC

Occlusion Scattering

99

Four Stages of Interaction

Opacity Map(position, size)

Attenuation Map(brightness)

Contrast Map(contrast) PSF Map

CCCCCC

CCC

CC

CCC

CCC

CC

CC

Occlusion Scattering

3mm0.6mm

100

Forward Scattering Sensed on Fovea

LCD1 LensLCD2

Projection on the Fovea

Light Box

Testing Sections

101

Forward Scattering Sensed on Fovea

LCD1 LensLCD2

Projection on the Fovea

Light Box

Testing Sections

102

Testing Sections of the Eye Lens

LCD1 LensLCD2

103

Trading Resolution for Brightness

LCD1 LensLCD2

Collimated

Beams

Testing Section

108

Cataract Screening

Presence of Cataracts

(Binary Answer)

109

Detecting the Presence of Cataracts

Moving patterns on LCD1

LCD1 LCD2

110


LCD1 LCD2

111


LCD1 LCD2 Eye lens

112


Perceived ImageLCD1 LCD2 Eye lens

113

Cataract Screening


(Binary Answer)

114

Interactive Techniques and Maps

Position, Size and Shape

(Opacity Map)


(Binary Answer)

115

Estimating an Opacity Map

Moving patterns on LCD1

LCD1 LCD2

116



(Opacity Map)


(Binary Answer)

117



(Opacity Map)

Brightness Test(Attenuation Map)


(Binary Answer)

118

Estimating Attenuation Maps

LCD1 LCD2

Decreasing Brightness

on the clear path

119


LCD1 LCD2

Same PerceivedBrightness

Decreasing Brightness

on the clear path

120


Perceived ImageLCD1 LCD2

Decreasing Intensity of the Clear Path

To Match Brightness

Eye

121



Decreasing Intensity of the Clear Path

To Match Brightness

Eye

122



(Opacity Map)



(Binary Answer)

124



(Opacity Map)


Sub-apertureContrast Test

(Contrast Map)


(Binary Answer)

CCCCCC

CCC

CC

CCC

CCC

CC

CC

125

Contrast Test


Rotated Low Contrast Letter C

Increasing Contrast

Eye

126

Contrast Test



Increasing Contrast

Eye

127

Contrast Test



Increasing Contrast

Eye

128

Contrast Test



Increasing Contrast

Press the right key

Eye

129



(Opacity Map)



(Contrast Map)


(Binary Answer)

CCCCCC

CCC

CC

CCC

CCC

CC

CC

131


Sub-aperture PSF Match(PSF Map)


(Opacity Map)



(Contrast Map)


(Binary Answer)

CCCCCC

CCC

CC

CCC

CCC

CC

CC

132

Point Spread Function Matching

Perceived ImageLCD1 LCD2 Eye

133



134



135



Sub-aperture Point Spread Function

Eye

136



(Opacity Map)



(Contrast Map)

Sub-aperture PSF Match(PSF Map)


(Binary Answer)

CCCCCC

CCC

CC

CCC

CCC

CC

CC

139

Reducing Search Space for PSF


(Opacity Map)



(Contrast Map)

Sub-aperture PSF Match(PSF Map)Low

Attenuation

High Attenuation


(Binary Answer)

CCCCCC

CCC

CC

CCC

CCC

CC

CC

141

Dual Monitor Stack

142

Dual Monitor Stack

143

Dual Monitor Stack

Lens

Medical Monochrome

Monitors LCD Stack

144

DLP-Mask Prototype

Lens

Mask

Eye

High ContrastDLP Projector

145

Cell Phone Prototype

Lens

Pinhole Mask

Stephen WolframCEO – Wolfram Research

146

Validation using Cameras

Thin Diffuser to Simulate Cataracts

Estimated Attenuation Map Estimated PSF Map

147

Validation using Cameras

Simulated Cataract

EstimatedOpacity Map

Measured Attenuation

(Pictures)

Estimated Attenuation

(User Interaction)

148

Scratching Contact Lenses

Advanced Cataracts

MildCataracts

EarlyCataracts

Measured: 0.5mm2

Estimated: 0.45mm2

149

Elderly Evaluation

18 volunteers- 6 with early cataracts - 12 with no cataracts

157

Limitations

• Active user participation

• One clear light path

• Retinal diseases

Patience and concentration are required to follow steps

158

ACM SIGGRAPH 2011

159

US + International Patent

160

LVPEI CATRA Early Tests.

161

Lotus University Trial

162

Lotus University Trial

163

Mexico VOSH/Rotary Trip.

0.00 5.00 10.00 15.00 20.00 25.00

0.00

5.00

10.00

15.00

20.00

25.00

LOCS III Linear (LOCS III)CATRA

84% Correlationfor 37 eyes.

164

mHealth Summit 2011

165

Awards: MIT Global Challenge & MIT Ideas Competitions

EyeCatra: $5K Winner Award MIT Ideas Competition 2011.

EyeCatra: $5K Public Choice Award MIT Global Challenge 2011.

Unique, low-costquantitative lens mapping

CATRA: Quantitative Cataract Maps

with: Erick Passos, Jan Zizka, Everett Lawson, Esteban Clua , Manuel M. Oliveira, Ramesh Raskar

eyecatra.com

170



171with: Manuel M. Oliveira, Daniel Aliaga, Ramesh Raskar

173

Strong Myopic View

174

Presbyopic View

175

Emmetropic View

176

Farsighted View

177

Farsighted View with Tailored Dashboard

Computer Generated Glasses

178

Perfect vision MyopiaFocal Range Focal Range

Focusing HereFocusing Here

Subject’s Focal Point Does Not Change

179

Computer Generated Glasses

Hyperopia Presbyopia

Perfect visionFocal Range

Focal Range

Focusing HereFocusing Here

Subject’s Focal Point Does Not Change

Tailoring is the easiest way to create a hologram

Because uncorrected individuals are already focusing where you want.

182

Tailoring Process

Focusing HereMyopic View: -3D

He can focus up to 33cm (12in)

Distance Display-Eye: 50cm

183

Tailoring Process

Light-fieldDisplay




184

Tailoring Process

Light-fieldDisplay




185

Tailoring Process

Light-fieldDisplay




1-arc minute Resolution

Pixel Size of 96um at 33cm

186

Working Resolution: 1800 DPI

Channel Size4.7um!

$150 Vuzix HMD LCD

187

Astigmatism Correction

188

Tailoring for Astigmatism

Light-fieldDisplay

Subject’s prescription -2D -1D @ 90

He focus at 30cm to 50cm.Two Points in Focus

Where the Subject’s

Accommodate

30cm50cm

189

Single-Focus Multi-Depth Displays

For a given depth in focus(accommodation),

a single object may be splitted intoanisotropic instances that are placed

at distinct depths

190

Wavefront Maps

0 degrees

90 degrees

Sphere: -2DCylinder: -1DAxis: 90°

Lens focal length in k

k

Zernike Functions

191

Light-fieldDisplay

f(k)

InputsTailoring Process

LCD1

LCD1 LCD2

197

Single-Focus Multi-Depth Displays

Eye

198

B

A

Builds the Image Pixel by Pixel

Display

Eye

td

d

199

B

A

Scaling Pixels by Depth

Display

Eye

td

d

Scale

200

Mapping Light-Field Display <-> Retina

Display

Eye

S1 S2

R

k

t a

f(k)

202

Cataract Density Maps

Nuclear Cataract Cataract density in k

k

Sub-capsular Cataract

203

Avoiding Cataracts

Display

Eye

204

Normalization Step

Display

Eye

205Dual Stack of LCDs

Projector

Camera – the “eye”

206

Vuzix Head Mounted Display

Lens Array

Vuzix Head Mounted Display

207

Input Image

As Seen on a Regular Display As Seen on a Tailored Display

3.25D Myopic Eye

208

No Cataract Tailoring Tailored for CataractsNo Cataracts

Input

3.5D Myopic Eye with Nuclear

Cataracts

209

1D Astigmatic Eye at 180 No Correction Corrected

210

Input Image

Display at 20cm from the eye.Subject focus at 50cm.

211

Regular DisplayPresbyopic +3D View


212

Tailored DisplayPresbyopic +3D View


213

Input Image


214

Regular DisplayPresbyopic View


215



216

Input Image


217

Regular DisplayPresbyopic +3D View


218



219

Users Prefer the Tailored Image

13 Volunteers, 16 votes each.

220


10 Volunteers, 16 votes each.

221


1 Volunteer, 80 votes.

222

Multi-Focus Multi-Depth Display

Input Image Input Depth

+0.5D from the Image Plane -0.5D from the Image Plane

223

ACM SIGGRAPH 2012

224

INPI Patent

226

Tailored Display Limitations

• Eyes fixed relative to the display– Similar to 3D Displays– Depends on the eye aberrations

• High-resolution LCD panels (PPI) – Giga-pixel displays for monitors

• Other ocular diseases may affect our results.

227

Measurement Correction

Need

228

Thesis Conclusions• NETRA: Optics and UI for Refraction

– The Inverse of Shack-Hartmann Aberrometer– Myopia, Hyperopia, Astigmatism, Focal range– Accuracy and Resolution close to Standard Practice

• CATRA: Optics and UI for Cataracts– Forward Scattering and Foveal Projection– Four brand new Maps

• Tailored Displays: Compensate for Aberrations– First-of-its-kind Multi-Depth Display– High-order Aberrations and Cataracts

defesa de tese

Technology