image refocus in geometrical optical phase spacecwachan/fio2010.pdf · chan, lam (univ. hong kong)...

Image Refocus in Geometrical Optical Phase Space

Aaron C. W. Chan Edmund Y. Lam

Imaging Systems Laboratory,Department of Electrical and Electronic Engineering,

University of Hong Kong.

http://www.eee.hku.hk/isl

2010 OSA Frontiers in Optics

Chan, Lam (Univ. Hong Kong) Image Refocus Oct 27, 2010 1 / 18

http://www.eee.hku.hk/isl

Introduction

Objectives

To provide a convenient mathematical framework for light-fieldanalysis based on Hamiltonian Optics.

To demonstrate some mathematical symmetries that arise —

useful for computation and algorithm design.

To explain the refocus process in this formulation — Ren Ng,

Stanford (2006). Simply the free space operation on the light fieldbefore imaging; or equivalently, the thin-lens operation in theFourier domain followed by a slice.

Introduction

Objectives

Introduction

Objectives

Introduction

Presentation Outline

Theoretical Background: Hamilton Equations; Lie Operator

formulation of ray equation.

Analysis of imaging system in spatial and spatial frequency

domains. The symmetries will be highlighted.

Refocus process derived. The computational costs will briefly bediscussed.

Concluding remarks, merits, limitations and further work.

Introduction

Theoretical Background

This analysis is based on the geometrical optics (paraxial)approximation — light is incoherent, non-monochromatic...

Theoretical Background Coordinates

Coordinates

x

y

z

α

Optical direction cosines: p1 = n dxds and p2 = ndyds .

p1 = n dxds = n sinα ≈ nα and similarly for p2.

n is the refractive index.

q = (x, y)T , p = (p1, p2)T and u = (q,p)T .

Theoretical Background Coordinates

Coordinates

x

y

z

α

Optical direction cosines: p1 = n dxds and p2 = ndyds .

p1 = n dxds = n sinα ≈ nα and similarly for p2.

n is the refractive index.

q = (x, y)T , p = (p1, p2)T and u = (q,p)T .

Theoretical Background Hamilton Equations

Hamilton Equations

The vectors q and p obey the Hamilton equations

Hamilton Equations

dqdz =

∂H(q,p)∂p

dpdz = −

∂H(q,p)∂q .

The Hamiltonian describes the environment in which the ray istravelling.

General Optical Hamiltonian

H(n, θ) = −n cosθ

Hamilton Equations

dqdz =

∂H(q,p)∂p

dpdz = −

∂H(q,p)∂q .

Hamilton Equations

dqdz =

∂H(q,p)∂p

dpdz = −

∂H(q,p)∂q .

Hamilton Equations

dqdz =

∂H(q,p)∂p

dpdz = −

∂H(q,p)∂q .

Theoretical Background Phase Space Density

Hamilton Equations for a Ray and a Bundle of Rays

Poisson Bracket {f , g} = ∂f∂q∂g∂p −

∂f∂p∂g∂q

Lie Operator L̂H = {·,H}.

Hamilton Equations forSingle Ray

dudz = +L̂Hu

Hamilton Equations forBundle of Rays

∂ρ∂z = −L̂Hρ

Solution

u(z) = exp[(z − zi)L̂H]u(zi)

Solution

ρ(z) = exp[−(z − zi)L̂H]ρ(zi)

Goto Details on Hamiltonian

∂f∂p∂g∂q

dudz = +L̂Hu

Solution

∂f∂p∂g∂q

dudz = +L̂Hu

Solution

∂f∂p∂g∂q

dudz = +L̂Hu

Solution

∂f∂p∂g∂q

dudz = +L̂Hu

Solution

∂f∂p∂g∂q

dudz = +L̂Hu

Solution

There are three basic solutions: the free space propagationsolution, the thin lens solution, and the magnification solution.

Choosing H = 12p21 +

12p

22 , then L̂H = p.∂q, one obtains

Free Space Propagation Solution

T̂ (ξ) = exp[

ξp.∂q]

Solution for Ray

u(z) = T̂(z − zi)ui

Solution for Ray Bundle

ρ(q,p, z) =T̂ [−(z − zi)]ρi(q,p).

Goto Table: Optical Lie Group

12p

T̂ (ξ) = exp[

ξp.∂q]

Solution for Ray

ρ(q,p, z) =T̂ [−(z − zi)]ρi(q,p).

12p

T̂ (ξ) = exp[

ξp.∂q]

Solution for Ray

ρ(q,p, z) =T̂ [−(z − zi)]ρi(q,p).

12p

T̂ (ξ) = exp[

ξp.∂q]

Solution for Ray

ρ(q,p, z) =T̂ [−(z − zi)]ρi(q,p).

qq

p p

q-shear

T̂(ξ) and performs q-shears in phase space, that is q → q + ξp orρ (q, p)→ ρ (q − ξp, p).

q

z

Imaging Integral

I(q) =∫

Ωpρ (q,p)dp

Imaging is the integration of all the rays from all directions pimpinging onto a single point with position q = (x, y)T on theimage plane.

The integral projection of ρ(q,p) onto the q-plane.

q

z

Imaging Integral

I(q) =∫

Ωpρ (q,p)dp

q

z

Imaging Integral

I(q) =∫

Ωpρ (q,p)dp

Analysis of Imaging System

f

u v

object image

Analysis of Imaging System Single Lens Imaging System

Single Lens Optical System

If we consider a simple biconvex lens optical system, that acts onthe phase space position vector, it is evident from the thin lensequation 1f =

1u +

1v that if one wants to refocus to a further

distance v, u must be shortened.

fu v

object image

Analysis of Imaging System Refocus

Refocus – In Spatial Domain

unewuold

vnewvold ξ

object image

Imaging of Refocused Light Field

I′(q) =∫

ΩpT̂(−ξ)ρ(q,p)dp =

∫

Ωpρ(q − ξp,p)dp.

Analysis of Imaging System Refocus

Refocus – In Spatial Domain

unewuold

vnewvold ξ

object image

Imaging of Refocused Light Field

I′(q) =∫

ΩpT̂(−ξ)ρ(q,p)dp =

∫

Ωpρ(q − ξp,p)dp.

Analysis of Imaging System Spectral Analysis

Imaging and Refocus in Spatial Frequency Domain

Definitions

Fq =1

2π

∫ ∞

−∞

∫ ∞

−∞dq exp(−ikq.q), similarly for p.

kq = (kx , ky)T as the spatial frequency of q, similarly for p.

ρ̂(

kq, kp)

= FpFqρ(q,p).

Realistically, max(q)� max(p), and the allowable angles are such that|p| < n(q) under the paraxial approximation, so it is better to define

F(L)p =

12π

∫ L

−L

∫ L

−Ldp exp(−ikp.p).

Definitions

Fq =1

2π

∫ ∞

−∞

∫ ∞

ρ̂(

kq, kp)

= FpFqρ(q,p).

F(L)p =

12π

∫ L

−L

∫ L

Definitions

Fq =1

2π

∫ ∞

−∞

∫ ∞

ρ̂(

kq, kp)

= FpFqρ(q,p).

F(L)p =

12π

∫ L

−L

∫ L

Definitions

Fq =1

2π

∫ ∞

−∞

∫ ∞

ρ̂(

kq, kp)

= FpFqρ(q,p).

F(L)p =

12π

∫ L

−L

∫ L

Definitions

Fq =1

2π

∫ ∞

−∞

∫ ∞

ρ̂(

kq, kp)

= FpFqρ(q,p).

F(L)p =

12π

∫ L

−L

∫ L

1 FpFqT̂ (−ξ)ρ(q,p)2 = FpFq exp(−ξp.∂q)ρ(q,p)

3 = exp(

ξkq.∂kp)

ρ̂(

kq, kp)

4 = ρ̂(kq, kp + ξkq).

Free space operator.

Thin lens operator L̂(ξ) ink-space.p-shear in k-space.

F(L)p FqT̂ (−ξ)ρ(q,p) = ρ̂(kq, kp + ξkq) ∗

[

sin(kp1 L)kp1

sin(kp2 L)kp2

]

if p is defined over a finite interval.Alternative Derivation Table of Operators

3 = exp(

ξkq.∂kp)

ρ̂(

kq, kp)

4 = ρ̂(kq, kp + ξkq).

[

sin(kp1 L)kp1

sin(kp2 L)kp2

]

3 = exp(

ξkq.∂kp)

ρ̂(

kq, kp)

4 = ρ̂(kq, kp + ξkq).

[

sin(kp1 L)kp1

sin(kp2 L)kp2

]

3 = exp(

ξkq.∂kp)

ρ̂(

kq, kp)

4 = ρ̂(kq, kp + ξkq).

[

sin(kp1 L)kp1

sin(kp2 L)kp2

]

3 = exp(

ξkq.∂kp)

ρ̂(

kq, kp)

4 = ρ̂(kq, kp + ξkq).

[

sin(kp1 L)kp1

sin(kp2 L)kp2

]

3 = exp(

ξkq.∂kp)

ρ̂(

kq, kp)

4 = ρ̂(kq, kp + ξkq).

[

sin(kp1 L)kp1

sin(kp2 L)kp2

]

3 = exp(

ξkq.∂kp)

ρ̂(

kq, kp)

4 = ρ̂(kq, kp + ξkq).

[

sin(kp1 L)kp1

sin(kp2 L)kp2

]

Defining the projection operator Pkp as the operation that sets kp = 0,

1 FqI′(q) = Fq∫

ΩpT̂(−ξ)ρ (q,p)dp

2 = 2πPkpFpFqT̂(−ξ)ρ(q,p)

3 = 2πPkpL̂(ξ)ρ̂(kq, kp)4 = 2πρ̂(kq, ξkq)⇒

Fourier Slice

I′(q) = 2πF −1q ρ̂(kq, ξkq).

Fourier Slice – For finite p

I′(q) = 2πF −1q ρ̂(kq,U) ∗[

sin(U1L)U1

sin(U2L)U2

]

.U = ξkq.

1 FqI′(q) = Fq∫

Fourier Slice

I′(q) = 2πF −1q ρ̂(kq, ξkq).

I′(q) = 2πF −1q ρ̂(kq,U) ∗[

sin(U1L)U1

sin(U2L)U2

]

.U = ξkq.

1 FqI′(q) = Fq∫

Fourier Slice

I′(q) = 2πF −1q ρ̂(kq, ξkq).

I′(q) = 2πF −1q ρ̂(kq,U) ∗[

sin(U1L)U1

sin(U2L)U2

]

.U = ξkq.

1 FqI′(q) = Fq∫

Fourier Slice

I′(q) = 2πF −1q ρ̂(kq, ξkq).

I′(q) = 2πF −1q ρ̂(kq,U) ∗[

sin(U1L)U1

sin(U2L)U2

]

.U = ξkq.

1 FqI′(q) = Fq∫

Fourier Slice

I′(q) = 2πF −1q ρ̂(kq, ξkq).

I′(q) = 2πF −1q ρ̂(kq,U) ∗[

sin(U1L)U1

sin(U2L)U2

]

.U = ξkq.

1 FqI′(q) = Fq∫

Fourier Slice

I′(q) = 2πF −1q ρ̂(kq, ξkq).

I′(q) = 2πF −1q ρ̂(kq,U) ∗[

sin(U1L)U1

sin(U2L)U2

]

.U = ξkq.

1 FqI′(q) = Fq∫

Fourier Slice

I′(q) = 2πF −1q ρ̂(kq, ξkq).

I′(q) = 2πF −1q ρ̂(kq,U) ∗[

sin(U1L)U1

sin(U2L)U2

]

.U = ξkq.

1 FqI′(q) = Fq∫

Fourier Slice

I′(q) = 2πF −1q ρ̂(kq, ξkq).

I′(q) = 2πF −1q ρ̂(kq,U) ∗[

sin(U1L)U1

sin(U2L)U2

]

.U = ξkq.

Conclusion

Conclusion:

Hamiltonian Optics and Operator methods provide a powerfulway of analysing incoherent optical systems.

This can be extended into k-space.

There is a symmetry between the operators in physical space andk-space. This means in computation, one only needs to optimizefor three or fewer operations.

However, computation in 4 dimensions still expensive with theFFT. Still, it is often desirable to work in Fourier Domain in ImageProcessing.

Conclusion

Conclusion – Further Work

Further Work

Provide optimized image processing algorithms for light fieldscaptured in 4D.

Compressed sensing/sampling of phase space in ordinary camerasand reconstruction algorithms.

Provide analysis to incorporate stochastic elements/noise/rayscattering (Mie Scattering and Rayleigh Scattering).

References

1 J. N. Mait, Optics and Photonics News 17, 22 (2006).

2 J. Mait, R. Athale, and J. van der Gracht, Optics Express 11, 2093(2003).

3 T. Mirani, D. Rajan, M. P. Christensen, S. C. Douglas, and S. L.Wood, Applied Optics 47, B86 (2006).

4 E. R. Dowski andW. T. Cathey, Applied Optics 34, 1859 (1995).

5 A. Accardi and G. Wornell, J. Opt. Soc. Am. A 26, 2055 (2009).

6 A. Torre, Linear Ray and Wave Optics in Phase Space (Elsevier,2004).

7 K. B.Wolf, Geometric Optics on Phase Space (Springer, 2004).

8 R. Ng, ACM Transactions on Graphics pp. 19 (2005).

References

9 M. Levoy and P. Hanrahan, in ACM SIGGRAPH (1996), pp. 3142.

10 R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P.Hanrahan, Light field photography with a hand-held plenopiccamera, Tech. rep., Stanford University (2005).

11 A. L. Rivera, S. M. Chumakov, and K. B.Wolf, J. Opt. Soc. Am. A12, 1380 (1995).

12 K. B.Wolf, J. Opt. Soc. Am. A 8, 1389 (1991).

13 E. H. Adelson and J. R. Bergen, The Plenoptic Function and theElements of Early Vision (1991), pp. 320.

14 E. H. Adelson and J. Y. Wang, IEEE Trans. Pattern Anal. Mach.Intell. 14, 99 (1992).

Acknowledgment

Acknowledgement

Research Grants Council of the Hong Kong Special AdministrativeRegion, China under Projects HKU 713906 and 713408.

Acknowledgment

Acknowledgement

Research Grants Council of the Hong Kong Special AdministrativeRegion, China under Projects HKU 713906 and 713408.

Group Website

http://www.eee.hku.hk/isl/

Appendix Proof of Liouville’s Theorem

Detour - Proof of Liouville’s Theorem

The total derivative of ρ is given bydρdz =

∂ρ∂z +

∂ρ∂q

dqdz +

∂ρ∂p

dpdz .

Using the Hamilton Equations, this can be written asdρdz =

∂ρ∂z +

∂ρ∂q∂H∂p −

∂ρ∂p∂H∂q =

∂ρ∂z + {ρ,H}.

The rate of change ofNV(z) with respect to z of this volume isequal to the rate of flow out of the 3D surface.

Consider a 4D volume in phase spaceV ∈ R4. The number of rayscontained in this volume isNV(z) =

∫

Vρ(q,p, z)dV .

∂ρ∂z +

∂ρ∂q

dqdz +

∂ρ∂p

dpdz .

∂ρ∂z +

∂ρ∂p∂H∂q =

∂ρ∂z + {ρ,H}.

∫

Vρ(q,p, z)dV .

∂ρ∂z +

∂ρ∂q

dqdz +

∂ρ∂p

dpdz .

∂ρ∂z +

∂ρ∂p∂H∂q =

∂ρ∂z + {ρ,H}.

∫

Vρ(q,p, z)dV .

∂ρ∂z +

∂ρ∂q

dqdz +

∂ρ∂p

dpdz .

∂ρ∂z +

∂ρ∂p∂H∂q =

∂ρ∂z + {ρ,H}.

∫

Vρ(q,p, z)dV .

Detour - Proof of Liouville’s Theorem 2

The rate of change ofNV(z) with respect to z of this volume isequal to the rate of flow out of the 3D hyper-surface S∂∂zNV(z) = −

∫

Sρ(q,p, z)dudz .n̂dS .

∫

From the generalized divergence theorem∂∂zNV(z) = −

∫

V∇.

(

ρ(q,p, z)dudz)

dV .

∫

From the generalized divergence theorem∂∂zNV(z) = −

∫

V∇.

(

ρ(q,p, z)dudz)

dV .

Taking all arguments to the left hand side and moving the partialderivative to inside the integral gives∫

V

(

∂ρ∂z + ∇.(ρ

dudz )

)

dV = 0.

Using the result ∇.(ρ udz ) =∂ρ∂q

dqdz +

∂ρ∂p

dpdz =

∂ρ∂p∂H∂q = {ρ,H},

Using the result ∇.(ρ udz ) =∂ρ∂q

dqdz +

∂ρ∂p

dpdz =

∂ρ∂p∂H∂q = {ρ,H},

and the fact that the volume integral is true for anyV,dρdz =

∂ρ∂z + {ρ,H} = 0.

Appendix Details of Hamiltonian

Detour - More on the Hamiltonian

Return to Presentation

Explicitly, the optical Hamiltonian derived from Fermat’sprinciple is

H (x, y, p1, p2, z) = −√

n2(x, y, z) − p21 − p22 ≈

12n(x,y,z)

(

p2x + p2y

)

− n(x, y, z).Terms up to the second order give the paraxial approximation.

Note p(n, θ) =√

p2x + p2y = n sinθ and H(n, θ) = −n cosθ, where

θ is the angle the ray makes with the optic axis.

Detour - More on the Hamiltonian

Explicitly, the optical Hamiltonian derived from Fermat’sprinciple is

H (x, y, p1, p2, z) = −√

n2(x, y, z) − p21 − p22 ≈

12n(x,y,z)

(

p2x + p2y

)

− n(x, y, z).Terms up to the second order give the paraxial approximation.

Note p(n, θ) =√

p2x + p2y = n sinθ and H(n, θ) = −n cosθ, where

θ is the angle the ray makes with the optic axis.

Detour - More on the Hamiltonian 2

Taylor expanding the refractive index from the optic axis, andassume that there is no z variation, one obtainsn(x, y) ≈ n(0, 0) − ∂n∂x x −

∂n∂y y −

12

(

∂2n∂x2 x

2 + ∂2n∂y2 y

2 + ∂2n∂x∂y xy

)

.

Ignoring the first order terms, which relate to shifts and tilts of theoptic axis,

H(x, y, p1, p2) ≈ 12n(0,0)p21 +

∂2x n|(0,0)2 x

2 + 12n(0,0)p22 +

∂2y n|(0,0)2 y

2.

In general, in the paraxial approximation, we make use of thequadratic monomials q2, p2 and qp.

∂n∂y y −

12

(

∂2n∂x2 x

2 + ∂2n∂y2 y

2 + ∂2n∂x∂y xy

)

.

H(x, y, p1, p2) ≈ 12n(0,0)p21 +

∂2x n|(0,0)2 x

2 + 12n(0,0)p22 +

∂2y n|(0,0)2 y

2.

∂n∂y y −

12

(

∂2n∂x2 x

2 + ∂2n∂y2 y

2 + ∂2n∂x∂y xy

)

.

H(x, y, p1, p2) ≈ 12n(0,0)p21 +

∂2x n|(0,0)2 x

2 + 12n(0,0)p22 +

∂2y n|(0,0)2 y

2.

Appendix Solutions to Optical Hamilton Equations

Solutions to Optical Hamilton Equations

The solutions to these equations take a simple form. H is a linearcombination of q2, p2 and qp under the paraxial approximation.

If we take H = q2, H = p2 and H = qp separately, then we obtain aSp(2,R) Lie group of solutions.

Solutions to Optical Hamilton Equations

The solutions to these equations take a simple form. H is a linearcombination of q2, p2 and qp under the paraxial approximation.

If we take H = q2, H = p2 and H = qp separately, then we obtain aSp(2,R) Lie group of solutions.

sp(2,R) Optical Lie Algebra

Return to Presentation Return to Spectral Analysis

Table: This table shows the sp(2,R) optical Lie algebras. They are the generators foroptical operations as shown in the next slide

H L̂H K matrix

12p

2 L̂− = p ∂∂q K− =(

0 10 0

)

12q

2 L̂+ = −q ∂∂p K+ =(

0 0−1 0

)

qp L̂3 = p ∂∂p − q∂∂q K3 =

(

1 00 −1

)

.

This operator is of interest.

H L̂H K matrix

12p

2 L̂− = p ∂∂q K− =(

0 10 0

)

12q

2 L̂+ = −q ∂∂p K+ =(

0 0−1 0

)

qp L̂3 = p ∂∂p − q∂∂q K3 =

(

1 00 −1

)

.

H L̂H K matrix

12p

2 L̂− = p ∂∂q K− =(

0 10 0

)

12q

2 L̂+ = −q ∂∂p K+ =(

0 0−1 0

)

qp L̂3 = p ∂∂p − q∂∂q K3 =

(

1 00 −1

)

.

Sp(2,R) Optical Lie Group

Return to Presentation Return to Spectral Analysis Return to Phase Space Discussion

Table: This table shows the Sp(2,R) Lie groups of optical operations. They aregenerated by the elements in the previous table through exponentiation.

exp(ζL̂H) exp(ζK)

Free Space T̂(ξ) = eξL̂− T(ξ) = eξK− =(

1 ξ0 1

)

Thin Lens L̂(f) = e1f L̂+ L(f) = e

1f K+ =

(

1 0−1f 1

)

Magnifier M̂(s) = esL̂3 M(s) = esK3 =(

es 00 e−s

)

Anti-homomorphism. Ray Transfer Matrices.

1 ξ0 1

)

1f K+ =

(

1 0−1f 1

)

es 00 e−s

)

1 ξ0 1

)

1f K+ =

(

1 0−1f 1

)

es 00 e−s

)

1 ξ0 1

)

1f K+ =

(

1 0−1f 1

)

es 00 e−s

)

1 ξ0 1

)

1f K+ =

(

1 0−1f 1

)

es 00 e−s

)

Appendix Algebra

Composition Property

exp(

ζL̂H)

ρ(q,p) = ρ(

exp(

ζL̂H)

q, exp(

ζL̂H)

p)

.

Appendix Spectral Analysis

Sheared phase space density in k-space

Return to Spectral Analysis

F(L)p Fqρ(q − ξp,p) = F

(L)p exp(−iξkq.p)Fq (ρ(q,p))

= 12π

∫ L

−L

∫ L

−Ldp exp(−i(kp + ξkq).p)Fq (ρ(q,p))

= ρ̂(kq, kp + ξkq) ∗[

sin(kp1L)kp1

sin(kp2 L)kp2

]

Appendix Microlens Array

Off-axis lens

The Operator for shifting the z-axis is: Ŝ(s) = exp(

s∂q)

. Note that

Ŝ−1(s) = Ŝ(−s). Ŝ has no matrix counterpart.The action on the phase space density of an off-axis lens distance sfrom the optical axis is ρ̃(q, p) = Ŝ(−s)L̂(−f)Ŝ(s)ρ(q, p).Using the result exp(−tÂ )B exp(−tÂ) = exp(−t[Â , ·])B , it can beshown thatρ̃(q, p) = exp

(

q−sf∂∂p

)

ρ(q, p) = ρ(

q, p + q−sf)

.

IntroductionTheoretical BackgroundCoordinatesHamilton EquationsPhase Space Density

Analysis of Imaging SystemSingle Lens Imaging SystemRefocusSpectral Analysis

ConclusionReferencesAcknowledgmentAppendixAppendixProof of Liouville's TheoremDetails of HamiltonianSolutions to Optical Hamilton EquationsAlgebraSpectral AnalysisMicrolens Array

image refocus in geometrical optical phase spacecwachan/fio2010.pdf · chan, lam (univ. hong kong)...

Documents