polarization-based shape estimation of transparent objects by using raytracing and plzt camera...

Polarization-based Shape Estimation of Transparent Objects by Using Raytracing a

nd PLZT CameraDaisuke Miyazaki The University of TokyoNoriyuki Takashima Furuuchi Chemical CorporationAkira Yoshida Furuuchi Chemical CorporationEiki Harashima Furuuchi Chemical CorporationKatsushi Ikeuchi The University of Tokyo

The 1st part of this talk

• PLZT polarization camera• Measures the polarization state (Stokes

vector) of the light• Is controllable from the computer

Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(11)Introduction(1/2)

The 2nd part of this talk

• Inverse polarization raytracing• Estimates the 3D shape of transparent obj

ects• Solves the inverse problem of the polarizati

on raytracing

Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(11)Introduction(2/2)

PLZT polarization camera

Mueller calculusLight: 4D vector (Stokes vector)Material: 4x4 matrix (Mueller matrix)

3

2

1

0

ssss Intensity

Power of 0 linear polarized lightPower of 45 linear polarized lightPower of right circular polarized light

DOP(degree of polarization)

2 2 21 2 3

0

s s ss

Introduction(2) Conclusion(2)

Algorithm(6) Experiment(4)

Inverse polarization raytracing(11)PLZT polarization camera(1/12)

Intro(1/2)

Mueller matrixND (neutral density) filter

1000010000100001

NWN

WN: alpha value (0~1)

Retardation1 0 0 00 1 0 0

( )0 0 cos sin0 0 sin cos

D

δ: retardation value

Horizontal linear polarizer

1000010000110011

LWL

WL: 0~0.5 (ideally 0.5)

Rotation1 0 0 00 cos 2 sin 2 0

( )0 sin 2 cos 2 00 0 0 1

C

Introduction(2) Conclusion(2)

Algorithm(6) Experiment(4)

Inverse polarization raytracing(11)PLZT polarization camera(2/12)

Intro(2/2)

PLZT

• Lanthanum-modified lead zirconate titanate• Made from 4 kinds of metal compound

Pb: leadLa: lanthanumZr: zirconiumTi: titanium

• Transparent ceramics• Birefringent media depending on the voltage

3x/4lylyxx-l O)Ti(ZrLaPb

Introduction(2) Conclusion(2)

Experiment(4)

Inverse polarization raytracing(11)PLZT polarization camera(3/12)

Algorithm(1/6)Intro(2)

PLZT and ND filter

Camera Target scene(Light, object, ...)

PLZTNDfilter

x

y

z

+90

Opticalaxis

Introduction(2) Conclusion(2)

Experiment(4)

Inverse polarization raytracing(11)PLZT polarization camera(4/12)

Algorithm(2/6)Intro(2)

Mueller matrix of the system

: Amount of the phase shift of PLZT(depends on the voltage)

cossin00sincos0000100001

)(ˆ)( NND WDNMSystem

NND filter

ˆ ( ) ( 90 ) ( ) ( 90 ) D C D CPLZT rotated 90

( )CRotation( )DRetardation

Introduction(2) Conclusion(2)

Experiment(4)

Inverse polarization raytracing(11)PLZT polarization camera(5/12)

Algorithm(3/6)Intro(2)

PLZT and linear polarizer

Camera Target scene(Light, object, ...)

PLZTLinearpolarizer

+22.5x

y

z

+90

Opticalaxis

Opticalaxis

Introduction(2) Conclusion(2)

Experiment(4)

Inverse polarization raytracing(11)PLZT polarization camera(6/12)

Algorithm(4/6)Intro(2)

Mueller matrix of the system

ˆ ( )DPLZT rotated 90

( )CRotation

: Amount of the phase shift of PLZT(depends on the voltage)

System

0000

sin21cos

21

21

22

sin21cos

21

21

22

sin22cos

22

221

)(ˆˆ)(

LPL WDLM

Linear polarizer rotated 22.5ˆ ( 22.5 ) ( 22.5 ) L C LC

Horizontal linear polarizer L

Introduction(2) Conclusion(2)

Experiment(4)

Inverse polarization raytracing(11)PLZT polarization camera(7/12)

Algorithm(5/6)Intro(2)

Computing Stokes vectorPLZT with ND filterPLZT with linear polarizer [ retarder]PLZT with linear polarizer [1/4 waveplate]PLZT with linear polarizer [1/2 waveplate]

NDM

( )PL M

( 2)PL M

( )PL M

3

2

1

0

2/1,,0

4/1,,0

,,0

,0

022

22

220

22

sin22cos

22

22

000

ssss

WWW

WWW

WWWW

W

ssss

LLL

LLL

LLLL

N

PL

PL

PL

ND

Inversematrix

Stokes vectorfrom 4 images

Introduction(2) Conclusion(2)

Experiment(4)

Inverse polarization raytracing(11)PLZT polarization camera(8/12)

Algorithm(6/6)Intro(2)

Experiment setup

CameraSlider

ND filterLinear polarizer

PLZT unitBand-pass filter

UV-cut filterIR-cut filter

+x

-x+y

-y

Introduction(2) Conclusion(2)Inverse polarization raytracing(11)PLZT polarization camera(9/12)

Intro(2) Algorithm(6) Experiment(1/4)

Experiment result

s0 s1 s2 s3

DOP

Introduction(2) Conclusion(2)Inverse polarization raytracing(11)PLZT polarization camera(10/12)

Intro(2) Algorithm(6) Experiment(2/4)

Evaluation

• DOP of linear polarizer• True value = 1.0• Measurement result

= 0.72

• s3/s0 of left circular polarizer• True value = -1.0• Measurement result

= -0.25

Introduction(2) Conclusion(2)Inverse polarization raytracing(11)PLZT polarization camera(11/12)

Intro(2) Algorithm(6) Experiment(3/4)

Related work

• Liquid crystal polarization camera• Wolff, Mancini, Pouli

qen, Andreou (1997)• Fujikake, Takizawa,

Aida, Kikuchi, Fujii, Kawakita (1998)

• Harnett, Craighead (2002)

• Our PLZT polarization camera• Obtain whole Stokes

parameters• PLZT has higher

response time than LC

Introduction(2) Conclusion(2)Inverse polarization raytracing(11)PLZT polarization camera(12/12)

Intro(2) Algorithm(6) Experiment(4/4)

Inverse polarization raytracing

Reflection and transmissionNormal

Unpolarized

AirObject

Partially polarized

Partially polarized

LightDepends

upon

Introduction(2) Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(1/11)

Intro(1/4) Algorithm(3) Experiment(4)

Interreflection

[Miyazaki 2004]Reflection only

[This method]Reflection & transmission

Introduction(2) Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(2/11)

Intro(2/4) Algorithm(3) Experiment(4)

Polarization raytracing

• Raytracing with polarization• Gondek et al. 1994, Wolff & Kurlander 199

0, Tannenbaum et al. 1994, Guy & Soler 2004, Chipman 1995, Wilkie 2001

• Commercial software

• We use: raytracing + Mueller calculus

Introduction(2) Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(3/11)

Intro(3/4) Algorithm(3) Experiment(4)

Reflection/Transmission matrix

Transmission

|| ||

|| ||

||

||

2 2 0 0

2 2 0 0

0 0 0

0 0 0

T T T T

T T T T

TT

TT

T

Reflection

RRRR

RRRRRRRR

||

||

||||

||||

00000000220022

R

Fresnel coefficients: ||R R ||T T

Introduction(2) Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(4/11)

Intro(4/4) Algorithm(3) Experiment(4)

Cost function

min

Calculate height and normal

dxdy

dxdyqHpHII yxRE222min

Input Calculated

RE II

Relationshipbetween

normal & heightxHp yHq

Introduction(2) Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(5/11)

Experiment(4)Intro(4) Algorithm(1/3)

Update normal

Light ray

Object

Ray changes

Ray changes

Changenormal

Input DOPCalculated DOP

Error

1p p E 2RE IIE 2q q E

Introduction(2) Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(6/11)

Experiment(4)Intro(4) Algorithm(2/3)

Algorithm overviewInitial height

Output height

Normal fromheight

Height fromnormal

is small

2

Input Calc.

Stop when

Minimize

Update normal

2

Input Calc.

Introduction(2) Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(7/11)

Experiment(4)Intro(4) Algorithm(3/3)

Monochromecamera IR/UV cut-off

filter

Linearpolarizer

Transparent object inside

40Wlamp

Plasticsphere

Experimental setupIntroduction(2) Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(8/11)

Intro(4) Algorithm(3) Experiment(1/4)

Experimental result

Acrylic hemisphere (r=15mm) 10 loop

Initial (Miyazaki 2004) 50 loop

Error(height)2.8mm 0.61mm

Error(normal)147.0

Frontal shape(estimated)Frontal shape(truth)Rear shape(known)

Refractive index 1.5& Illumination (known)

Introduction(2) Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(9/11)

Intro(4) Algorithm(3) Experiment(2/4)

Experimental result

Initial (Miyazaki 2004)

10 loop

Glass (n=1.5)

Introduction(2) Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(10/11)

Intro(4) Algorithm(3) Experiment(3/4)

Related work

• Shape estimation of transparent object• Murase 1992, Hata et al.

1996, Ohara et al. 2003, Ben-Ezra & Nayar 2003, Kutulakos 2005, Saito et al. 1999, Miyazaki et al. 2002, Miyazaki et al. 2004

• Shape from polarization• Koshikawa & Shirai 198

7, Wolff & Boult 1991, Rahmann 1999, Rahmann 2000, Rahmann & Canterakis 2001, Rahmann 2003, Drbohlav & Sara 2001, Miyazaki et al. 2003

Target is transparentEstimate arbitrary shape[Our method]

Introduction(2) Conclusion(2)PLZT polarization camera(12) Inverse polarization raytracing(11/11)

Intro(4) Algorithm(3) Experiment(4/4)

Conclusion

Summary[Inverse polarization raytracing][PLZT polarization camera]

PLZTND/LP filter

0 1 2 3Ts s s s

Stokes vector

Voltage

Shape

Iteration

min2

Input Calc.

Introduction(2) PLZT polarization camera(12) Inverse polarization raytracing(11) Conclusion(1/2)

Future work[Inverse polarization raytracing][PLZT polarization camera]

Estimating refractive index

?

Realtime measurement

Improve the accuracy

Introduction(2) PLZT polarization camera(12) Inverse polarization raytracing(11) Conclusion(2/2)

© Daisuke Miyazaki 2005All rights reserved.

http://www.cvl.iis.u-tokyo.ac.jp/Daisuke Miyazaki, Noriyuki Takashima, Akira Yoshida, Eiki Harashima, Katsushi Ikeuchi, "Polarization-based Shape Estimation of Transparent Objects by Using Raytracing and PLZT Camera," in Proceedings of SPIE (Polarization Science and Remote Sensing II, Part of SPIE's International Symposium on Optics and Photonics 2005), Vol. 58

88, pp. 1-14, San Diego, CA USA, 2005.8