nick beavers project manager deconvolution from andy molnar software engineer

Nick BeaversProject Manager

Deconvolution

from

Andy MolnarSoftware Engineer

Basics of Fluorescent Imaging

What is Fluorescence Imaging?

• Unlike reflection or absorption imaging.

• Use molecules called fluorophores:

– Upon illumination by light of a specific wavelength they emit light of longer wavelengths.

Absorption (Excitation) Spectrum

Emission Spectrum

Wavelength

Stokes Shift

Lens

Light Source

Digital Camera

Emission Filter

Excitation Filter

Dichromatic Mirror

Objective Lens

Fluorescence Microscopy

Fluorescence Microscopy

Microscope Slide

Specimen

Objective Lens

Coverslip

Immersion Medium

Slide

Fluorescence Microscopy

Microscope Slide

Specimen

Coverslip

SlideThe Object: A collection of fluorophores small

enough to be considered point sources of light.

Image Formation

The Object (reality)

The Image (reality observed)

XY

XZ

XY

XZ

What Happened?

Aperture Diffraction

One Dimension - Diffraction

Aperature d

Point Source of Light

Two Dimensions - Airy Disk

PSF (Point Spread Function)

• Point in object Point Spread Function (PSF) in image.– The image is built out of PSFs, not points.

• Effect of PSF– Features blur together– Measurements are corrupted

• Deconvolution– Undo effect of PSF

Resolution and Blur

Effect of the PSF

• Object -> Image

PSF in 3 Dimensions

• In 3d Widefield, light from out of focus planes is detected in focal planes.

• In Confocal, pinholes help but there is still some z-blur especially when SA is present.

XY

XZ

Widefield Confocal

XY

XZ

The Problem

How to make a measured image better represent the real object ?

1) Counteract the PSF – that is put the light that spread (Airy disk) out back to its 2D location

2) Put light into the proper Z plane 3) Reduce the noise

This is best accomplished using deconvolution

Deconvolution ofFluorescence Images

Why Deconvolution?

• Increased Resolution

• Better Contrast

• Improved Signal-to-Noise Ratio

Noise

• Random nature of Photon Emission– Poission Noise– Usually dominant– Low light levels

• Digital Imaging System– Gaussian Noise

Noise

Widefield Example

• Max Projections

Confocal Example

Deconvolution Methods

DECON METHOD

FEATURE

Speed Subtractive Quantitative Requires PSF

Produces PSF S/N Noise

No/Nearest Neighbor Fast Yes No No No Lower Higher

Inverse Filter Moderate No No No No Higher Higher

Non-BlindConstrainedIterative *

Mod-Long No Yes Yes/No No Higher Lower

BlindConstrained Iterative *

Long No Yes No Yes Higher Lowest

* Can use measured or theoretical PSF

Imaging Equation

• Mathematical operation called a convolution.• The Microscope is a convolution operator.• The inverse of a convolution is deconvolution

=

Object convolved with the PSF equals the Image

ImagePSFObject

Iterative Non-Blind Deconvolution

Object Estimate ImagePSF Image Estimate

ErrorUpdate

-

First Iteration

(Fixed PSF)

Blind Deconvolution

Object Estimate ImagePSF Image Estimate

ErrorUpdate

-

First Stage - Image

(Adaptive PSF)

Blind Deconvolution

Object Estimate ImagePSF Image Estimate

ErrorUpdate

-

Second Stage - PSF

(Adaptive PSF)

• Benefits– Can adapt to imperfections in the Microscope.– Double optimization: Object and PSF are

modified– Can adapt to changes in refractive index,

specimen, environment.– Handles SA well. – No need to measure the PSF.– The only certain benefit of non-blind over

blind is that it is faster.

Benefits of Blind Deconvolution(Adaptive PSF)

Comparison

Original Image Nearest Neighbors

Inverse Filter Iterative

2D Widefield Example

Example: Slice View

slice 29

slice 37

raw deblurred

The Point Spread Function (PSF)

Determining the PSF

• Need a input PSF for • Inverse filter• Constrained Iterative• Statistical.

• Need PSF “first guess” for• Blind

PSF - Widefield vs. Confocal

XY

XZ

Widefield Confocal

XY

XZ

What effects the shape of PSF??

• Lens parameters• Modality• Specimen Parameters

XY

XZ

Widefield

XY

XZ

Confocal

Spherical Aberration

Rays at the periphery of the lens have a closer focal point

Rays entering near the center of the lens have a

farther focal point

Spherical Aberration

Z

n1 n1n2

Index of Refraction Mismatch

• Caused by– Changes in refractive

index– Thick specimens– Incorrect cover slip

thickness– Worse deeper in the

sample

• Solve by– Match RI– SA correction collar– Use biased PSF and

deconvolution

Spherical Aberration

PSF

• Theoretical• Measured• Blind

Theoretical

• Theoretical– Scope Properties

• NA, Lens RI, Pixel Spacing, Emissive Wavelength,

– Specimen properties• Specimen RI, Distance from Coverslip

Measured

• Bead Image• Must be taken under the same conditions as the

real sample will be.– Same embedding RI– Same distance from coverslip

Bead Image

Blind PSF

• PSF also changes as part of optimization• Blind: 2 step process in Autoquant

– Spherical Aberration Detection– PSF modified further as part of Deconvolution

• Determine PSF without measuring it.

PSF Comparison

Measured Theoretical Blind

Example

• What happens if you use the wrong PSF??

Raw Imagemax = 2,614

XY

XZ

Deconvolved no SAmax = 12,134

XY

XZ

Deconvolution with SA Correctionmax = 23,938

XY

XZ

Geometric Distortion

• With SA PSF changes with depth• Actual focus location also changes (PSF

Asymmetry)• Images look stretched.• Scale in Z• Many papers on this (Model 2010)

Deconvolution of Confocal Images

• Yes Confocal images can be deconvolved• Confocal images

– some blur.– noisy

• Accurate statistical model is important• Poisson model• Deconvolution reduces both blur and noise• Improves the Signal to Noise Ratio.• Multi-Photon, Spinning Disc, Structured

Illumination.

Confocal Examples

Collecting Optical Sections

• Setting the top and bottom of the scan

• Use Nyquist sampling– Sampling rate needed to collect all information

• Don’t break rules for convenience only out of necessity

Imaging Approach

• First try Widefield + deconvolution– Better for low light levels– Living cells– Proteins

• Next Confocal + deconvolution– Thick specimens (Tissues)

• Multiphoton + deconvolution– Very thick specimens

Deconvolution Tips

• Check the image stack’s metadata before decon.

• Use the line profile to prove that decon works.

• Use statistics to show…– Average image intensity remain relatively the same– Max pixel intensity can increase by 100 times.

Deconvolution Tips

• 99% of the time Blind is best.

• Severe spherical aberration correction is best done by CGC on the objective.

• Use an AOI for SAC testing or decon testing.

• AutoQuant 30 Day trial program:– Use it to your advantage!– Customers receive the entire product to evaluate.

Q&A

Questions?

Nick BeaversProject Manager

Deconvolution

from

Andy MolnarSoftware Engineer