1 / 28

Seize the Moments for Subdiffraction Incoherent Imaging

Ranjith Nair, Xiao-Ming Lu, Shan Zheng Ang, Mankei Tsang

National University of Singapore

Paris, July 2018

Imaging of Spatially Incoherent Sources

2 / 28

� Fundamental problems: Diffraction Limit and Photon Shot Noise� Image processing?

Sparse (Good Regime, PALM, STED,compressed sensing, etc.) Subdiffraction (Worst Nightmare)

� Limits to image processing (classical Cramér-Rao bound)� Enhancements by better measurements in the far-field� Discovered via quantum metrology

Telescopes, Fluorescence Microscopy

3 / 28

(images stolen from the internet)

Publications

4 / 28

1. Tsang, Nair, Lu, PRX 6, 031033 (2016).2. Nair, Tsang, OE 24, 3684 (2016).3. Tsang, Nair, Lu, SPIE 10029, 1002903 (2016).4. Nair, Tsang, PRL 117, 190801 (2016).5. Ang, Nair, Tsang, PRA 95, 063847 (2017).6. Tsang, NJP 19, 023054 (2017).7. Yang, Nair, Tsang, Simon, Lvovsky, PRA 96, 063829 (2017).8. Tsang, JMO 65, 104 (2018).9. Tsang, PRA 97, 023830 (2018).

10. Lu, Krovi, Nair, Guha, Shapiro, arXiv:1802.02300 (2018).11. Tsang, arXiv:1806.02781 (2018).

Diffraction Limit

5 / 28

Diffraction-limited

1. Fluorescence microscopy2. Space telescopes (Webb, $10 billion)3. Ground-based telescopes corrected by adap-

tive optics:

(a) Large Binocular Telescope (LBT) (Strehl ra-tio > 80%, $120 million)

(b) Giant Magellan Telescope (GMT)(c) Thirty Meter Telescope (TMT)(d) European Extremely Large Telescope (E-

ELT) (>$1 billion each)

Esposito et al., SPIE 8149, 814902 (2011).

Photon Shot Noise

6 / 28

� Superresolution via image processing (e.g., SVD, basispursuit) is possible but extremely sensitive to noise

� EMCCD: efficiency > 90%, extremely low dark counts� Thermal sources (stars, etc.)

– Poisson, bunching negligible at optical– Goodman, Statistical Optics; Zmuidzinas, JOSA A

20, 218 (2003)

� Fluorophores (GFP, dye molecules, quantum dots, etc.)

– Poisson, negligible anti-bunching– Pawley ed., Handbook of Biological Confocal Mi-

croscopy ; Ram, Ober, Ward, PNAS (2006)

Two Point Sources

7 / 28

(a)

(b)

Cramér-Rao Bound for Direct Imaging

8 / 28

� Cramér-Rao bound:

MSE ≥ J−1, Jµν = N

∫ ∞

−∞

dx1

f(x|θ)

∂f(x|θ)

∂θµ

∂f(x|θ)

∂θν. (1)

� J is diagonal for centroid and separation of the two sources.

θ2/σ0 2 4 6 8 10

Fisher

inform

ation/(N

/4σ2)

0

0.5

1

1.5

2

2.5

3

3.5

4Classical Fisher information

J(direct)11

J(direct)22

θ2/σ0 0.2 0.4 0.6 0.8 1

Mean-squareerror/(4σ2/N

)

0

20

40

60

80

100Cramér-Rao bound on separation error

Direct imaging (1/J(direct)22 )

� Rayleigh’s curse� Tsai and Dunn, Lincoln Lab. Tech. Note AD-A073462 (1979); Bettens et al.,

Ultramicroscopy 77, 37 (1999); Van Aert et al., J. Struct. Biol. 138, 21 (2002); Ram,Ward, Ober, PNAS 103, 4457 (2006).

Quantum Cramér-Rao Bound

9 / 28

� Quantum: Direct imaging is just one of the infinitelymany possible measurements.

� Helstrom (1967), Nagaoka (1989), Braunstein &Caves (1994): For any measurement,

MSEµ ≥(J−1

)

µµ≥

(K−1

)

µµ, (2)

Kµν =M Re (trLµLνρ) , (3)

∂ρ

∂θµ=

1

2(Lµρ+ ρLµ) . (4)

� K(ρ) is the quantum Fisher information, the ulti-mate amount of information in the photons.

Quantum Measurement

on image plane

Quantum Optics

10 / 28

� Thermal optical source: average photon number per mode ǫ≪ 1

image plane

� Quantum state in M temporal modes = ρ⊗M ,

ρ = (1− ǫ) |vac〉 〈vac|+ǫ

2(|ψ1〉 〈ψ1|+ |ψ2〉 〈ψ2|) +O(ǫ

2),

|ψs〉 ≡

∫ ∞

−∞

dxψ(x−Xs) |x〉 .

� derived from zero-mean Gaussian P function, mutual coherence, paraxial� Multiphoton coincidence: rare in practice, little info as ǫ ≪ 1 (homeopathy)� Hanbury Brown-Twiss: obsolete for decades (poor SNR)� Agrees with Poisson model� Similar model in Gottesman, Jennewein, Croke, PRL 109, 070503 (2012); Tsang, PRL 107,

270402 (2011).

Plenty of Room at the Bottom

11 / 28

θ2/σ0 2 4 6 8 10

Fisher

inform

ation/(N

/4σ

2)

0

1

2

3

4Quantum and classical Fisher information

K11

J(direct)11

K22

J(direct)22

θ2/σ0 0.2 0.4 0.6 0.8 1

Mean-squareerror/(4σ2/N

)

0

20

40

60

80

100Cramér-Rao bounds on separation error

Quantum (1/K22)

Direct imaging (1/J(direct)22 )

� Tsang, Nair, Lu, PRX 6, 031033 (2016).� arbitrary ǫ: Nair and Tsang, PRL 117, 190801 (2016); Lupo and Pirandola

(York), ibid. 117, 190802 (2016); Editors’ Suggestions.

Optimal Measurement

12 / 28

� Measurement in Hermite-Gaussian(TEM) basis is optimal in Fisher informa-tion.

– Gaussian PSF: Tsang, Nair, Lu, PRX(2016).

– any even PSF: Rehacek et al., OL 42,231 (2017).

� Finite N [Tsang, JMO 65, 104 (2018)]:

– proved MSE ≤ 4× QCRB– Simulations: < 2× QCRB– Similar results with Bayesian/minimax

Spatial-Mode Demultiplexing (SPADE)

13 / 28

image plane

...

...

Estimator

Image

Inversion

� Nair and Tsang, OE 24, 3684 (2016): SuperLocalization via Image-inVERsioninterferometry (SLIVER)

� Many other ways (in optical comm., photonic circuits, etc.)� Labroille et al. (CAILabs/Bell/LKB), OE 22, 15599 (2014):

� Classical sources, Far-field linear optics/photon counting, Importantapplications

Elementary Explanation

14 / 28

� Incoherent sources: energy in 1st-order mode is

∝

(d

2

)2

+

(

−d

2

)2

=d2

2. (5)

� 0th-order mode is just background noise; filtering it improves SNR.� Coherent noise filtering

Experimental Demonstrations

15 / 28

1. Tham, Ferretti, Steinberg (Toronto),PRL 118, 070801 (2017).

� ∼ 2× QCRB

2. Tang, Durak, Ling (CQT Singapore),OE 24, 22004 (2016).

3. Yang, Taschilina, Moiseev, Simon,Lvovsky (Calgary), Optica 3, 1148(2016).

4. Paúr, Stoklasa, Hradil, Sánchez-Soto, Rehacek (Palacky/Madrid/MaxPlanck), Optica 3, 1144 (2016).

5. Parniak et al. (Warsaw),arXiv:1803.07096 (2018).

6. Donohue et al. (Paderborn),arXiv:1805.2491 (2018).

Arbitrary Source Distribution

16 / 28

� Direct imaging of arbitrary sources with distribution = F (X|θ):

f(x|θ) =

∫

dX|ψ(x−X)|2F (X|θ), Jµν =

∫ ∞

−∞

dx1

f

∂f

∂θµ

∂f

∂θν. (6)

� Parameterization?� Simple analytic J?

Defining Subdiffraction Regime

17 / 28

object support width ≡ ∆ ≪ 1. (7)

Cramér-Rao Bound for Direct Imaging

18 / 28

� Define parameters as object mo-ments:

θµ =

∫

dXXµF (X|θ) (8)

� CRB in subdiffraction regime:

(J−1)µν =O(1)

N. (9)

� Tsang, NJP 19, 023054 (2017); PRA97, 023830 (2018).

� Special case: for 2 point sources withseparation d, θ2 = d

2/4,

J (d) =

(∂θ2∂d

)2

J22 = NO(d2).

(10)

d0 2 4 6 8

Fisher

inform

ation/(N

/4)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5Classical Fisher information

J(direct)11

J(direct)22

SPADE for Moment Estimation in 2D Imaging

19 / 28

� Gaussian PSF [Tsang, NJP (2017)]:

– For moments with even µ1 & even µ2: TEM basis

⊲ See also Yang et al., Optica (2016)

– For other moments: interference of pairs of TEM modes

More General PSFs

20 / 28

� Any centrosymmetric and separable PSF [Tsang, PRA (2018)]:

– For moments with even µ1, even µ2: “PSF-adapted” (PAD) basis(Rehacek et al. OL (2017), generalizes TEM)

– For other moments: interference of pairs of PAD modes

PAD

iPAD1 iPAD2 iPAD3

iPAD4 iPAD5 iPAD6

00 10 20 30

01

02

03

11 21 31

12 22 32

13 23 33 00 10 20 30

01

02

03

11 21 31

12 22 32

13 23 33

00 10 20 30

01

02

03

11 21 31

12 22 32

13 23 33

00 10 20 30

01

02

03

11 21 31

12 22 32

13 23 33

00 10 20 30

01

02

03

11 21 31

12 22 32

13 23 33

00 10 20 30

01

02

03

11 21 31

12 22 32

13 23 33

00 10 20 30

01

02

03

11 21 31

12 22 32

13 23 33

(10) (50)

(12) (52)

(54)

(56)

(14)

(16)

(01) (21) (41) (61)

(05) (25) (45) (65)

(11) (31) (51)

(15) (35) (55)

(36)

(34)

(32)

(30)

(03) (23) (43) (63) (13) (53)(33)

(00) (20) (40) (60)

(02) (22) (42) (62)

(04) (24) (44) (64)

(06) (26) (46) (66)

Performance of SPADE

21 / 28

� MSE of SPADE:

MSEµµ =O(∆2⌊µ/2⌋

)

N︸ ︷︷ ︸

variance

+O(∆2µ+4

)

︸ ︷︷ ︸

bias2

.

much lower than direct-imaging CRB when

– ∆ ≪ 1 (subdiffraction)– µ ≡

∑

j µj ≥ 2 (2nd- and higher-order mo-ments)

– bias is negligible.

� Caveat: θ2µ = O(∆2µ), fractional error is still

large, need many photons� Object size and shape estimation

|θ1| / θ0

10−2 10−1

MSE

/[θ

2 0(∆

/2)2

µ]

×10−3

1

2

3

4

567

µ = 1

θ2 / θ0 ×10−3

2 4 6

MSE

/[θ

2 0(∆

/2)2

µ]

10−3

10−2

10−1

100µ = 2

θ2 / θ0 ×10−3

2 4 6

MSE

/[θ

2 0(∆

/2)2

µ]

100

102

µ = 3

θ4 / θ0

10−6 10−5

MSE

/[θ

2 0(∆

/2)2

µ]

100

105µ = 4

Direct imagingDirect imaging (CRB)SPADESPADE (theory)

Tsang, NJP (2017); PRA(2018).

Elementary Explanation

22 / 28

� Wavefunction from each point source:

� For one point source,

Energy in first-order mode ∝ X2. (11)

� A distribution of incoherent sources:

Total energy in first-order mode ∝

∫

dXF (X|θ)X2. (12)

Quantum Limit

23 / 28

� One-photon density operator:

ρ1(θ) =

∫

dXF (X|θ) |ψX〉 〈ψX | , |ψX〉 =

∫ ∞

−∞

dxψ(x−X) |x〉 . (13)

– mixed state– infinite number of spatial modes– infinite number of parameters

� Quantum Cramér-Rao bound [Tsang, arXiv:1806.02781 (2018)]:

(J−1)µµ ≥O(∆2⌊µ/2⌋)

N? (14)

– Choose a nice Choi-Kraus representation/purification of ρ1

� See also Zhou and Jiang (Yale), arXiv:1801.02917 (2018)

Future Directions

24 / 28

� Clean up math, multiparameter problems, exactly optimal measurements, etc.� QCRB for 3D localization?

– Backlund, Shechtman, Walsworth (Harvard/Technion), arXiv:1803.01776 (2018)– Yu and Prasad (New Mexico), arXiv:1805.09227 (2018)– Napoli, Tufarelli, Piano, Leach, Adesso (Nottingham), arXiv:1805.04116 (2018)

� Control?

SPADE

photon-counting

array

image

plane

image

plane

object

plane

� Experiments� FAQ: https://sites.google.com/site/mankeitsang/news/rayleigh/faq

https://sites.google.com/site/mankeitsang/news/rayleigh/faq

Quantum Technology 1.5

25 / 28

Stellar Interferometry

26 / 28

� Astrophotonics: photonic cir-cuits for stellar interferometry

� Conventional wisdom: lesssensitive to atmospheric turbu-lence

� Our work: Fundamental ad-vantage with diffraction +photon shot noise

� Singapore: fluorescence mi-croscopy “Dragonfly,” Jovanovic et al.,

Mon. Not. R. Astron. Soc. 427, 806(2012)

Misalignment

27 / 28

∆ = centroid displacement+ object size ≪ 1 (15)

Rough Sketch

28 / 28

ρ1 =

∫

dXF (X|θ)e−ikX |ψ〉 〈ψ| eikX (16)

=∑

q,p

∫

dXF (X|θ)Xq+p

︸ ︷︷ ︸

moment matrix

(−ik)q

q!|ψ〉 〈ψ|

(ik)p

p!(17)

=∑

q,p

θq+p︸︷︷︸

Cholesky

(−ik)q

q!|ψ〉 〈ψ|

(ik)p

p!(18)

=∑

q,p,r

ΛqrΛpr︸ ︷︷ ︸

(−ik)q

q!|ψ〉 〈ψ|

(ik)p

p!(19)

=∑

r

Ar |ψ〉 〈ψ|A†r, Ar ≡

∑

q

Λqr(−ik)q

q!(Choi-Kraus), (20)

K̃µν = trΠ∂Λ

∂θµ

∂Λ⊤

∂θµ, Πpq ≡ 〈ψ|

(ik)p

p!

(−ik)q

q!|ψ〉 . (21)

Imaging of Spatially Incoherent SourcesTelescopes, Fluorescence MicroscopyPublicationsDiffraction LimitPhoton Shot NoiseTwo Point SourcesCramér-Rao Bound for Direct ImagingQuantum Cramér-Rao BoundQuantum OpticsPlenty of Room at the BottomOptimal MeasurementSpatial-Mode Demultiplexing (SPADE)Elementary ExplanationExperimental DemonstrationsArbitrary Source DistributionDefining Subdiffraction RegimeCramér-Rao Bound for Direct ImagingSPADE for Moment Estimation in 2D ImagingMore General PSFsPerformance of SPADEElementary ExplanationQuantum LimitFuture DirectionsQuantum Technology 1.5Stellar InterferometryMisalignmentRough Sketch