problems in biological imaging: opportunities for signal processing jelena kovačević bimagiclab...
Post on 03-Jan-2016
213 Views
Preview:
TRANSCRIPT
Problems in Biological Imaging: Problems in Biological Imaging: Opportunities for Signal ProcessingOpportunities for Signal Processing
Jelena KovačevićJelena Kovačević
bimagicLabbimagicLabCenter for Bioimage InformaticsCenter for Bioimage InformaticsDepartment of Biomedical EngineeringDepartment of Biomedical EngineeringDepartment of Electrical and Computer EngineeringDepartment of Electrical and Computer EngineeringCarnegie Mellon UniversityCarnegie Mellon University
Cast of CharactersCast of Characters
The RoadmapThe Roadmap
Tasks
Issues
Framework
Tools
Revolution in biology
What can we do?
Revolution in BiologyRevolution in Biology
Focus in biologyFocus in biology Vertical to horizontal approachVertical to horizontal approach ““Omics”: genomics, proteomics, …Omics”: genomics, proteomics, …
Fluorescence microscopyFluorescence microscopy Hugely successfulHugely successful Allows for live-cell imagingAllows for live-cell imaging Fluorescent markers, starting with GFPFluorescent markers, starting with GFP Allows for collection of high-dimensional data setsAllows for collection of high-dimensional data sets
2D images and 3D volumes2D images and 3D volumes At multiple time instantsAt multiple time instants Multiple channelsMultiple channels
Analysis and interpretation Analysis and interpretation Cumbersome, nonreproducible, error proneCumbersome, nonreproducible, error prone
GoalGoal
Imaging in systems biologyImaging in systems biology
Use informatics toUse informatics to acquire, store, manipulate acquire, store, manipulate
and share large and share large bioimaging databasesbioimaging databases
Leads toLeads to automated, efficient and automated, efficient and
robust processing robust processing
NeedNeed Host of sophisticated tools Host of sophisticated tools
from many areasfrom many areas
RegistrationMosaicing
SegmentationTracking
AnalysisModeling
PSF h
A/D
Denoising
Deconvolution
RestorationDenoising +
Deconvolution
The RoadmapThe RoadmapIssues
Revolution in biologyNoise levels and typesNoise levels and typesLack of ground truthLack of ground truthLarge deviationsLarge deviationsLow definition and contrastLow definition and contrastWide range of time-frequency featuresWide range of time-frequency features
Noise Levels and TypesNoise Levels and Types
Shift towards noninvasiveShift towards noninvasive Data collected farther from the sourceData collected farther from the source Signals typically corrupted by Signals typically corrupted by
high levels of noisehigh levels of noise Weak biosignalsWeak biosignals Standard SP techniques not used Standard SP techniques not used
but even those will not work well but even those will not work well with such signalswith such signals
Types of noiseTypes of noise Electrical, neuronal, …Electrical, neuronal, … Modeling of noise a problemModeling of noise a problem
Lack of Ground TruthLack of Ground Truth
Shift towards noninvasive Shift towards noninvasive No access to ground truthNo access to ground truth
Large DeviationsLarge Deviations
Humans and/or animals as ``customers'‘Humans and/or animals as ``customers'‘ Wide range of states considered ``normal'‘Wide range of states considered ``normal'‘ Looking for is a range rather than a single stateLooking for is a range rather than a single state Large deviations from the range of normal states may Large deviations from the range of normal states may
characterize what we are looking forcharacterize what we are looking for
normal delayed abnormal
Low Definition and ContrastLow Definition and Contrast
Images typically have low contrast Images typically have low contrast and are poorly definedand are poorly defined Lack of consistent edgesLack of consistent edges
Wide Range of Time- and Frequency-Wide Range of Time- and Frequency-Localized FeaturesLocalized Features
BioimagesBioimages Global behaviors together with spikes and transientsGlobal behaviors together with spikes and transients Puts time-frequency tools to the testPuts time-frequency tools to the test ““Speckled” nature---stochastic representationSpeckled” nature---stochastic representation
The RoadmapThe RoadmapIssues
Framework
Revolution in biology
Continuous-domain image processingContinuous-domain image processingFrom continuous to discrete domainFrom continuous to discrete domainDiscrete-domain image processingDiscrete-domain image processing
Continuous-Domain Image ProcessingContinuous-Domain Image Processing
Specimen (object) vs Specimen (object) vs image of it (projection)image of it (projection)
LSI systemsLSI systems Impulse response of the Impulse response of the
microscope: PSFmicroscope: PSF
Fourier viewFourier view FT or FSFT or FS
RegistrationMosaicing
SegmentationTracking
AnalysisModeling
PSF h
A/D
Denoising
Deconvolution
RestorationDenoising +
Deconvolution
From Continuous to DiscreteFrom Continuous to Discrete
Resolution in microscopyResolution in microscopy
Filtering before samplingFiltering before sampling
Sources of uncertaintySources of uncertainty
RegistrationMosaicing
SegmentationTracking
AnalysisModeling
PSF h
A/D
Denoising
Deconvolution
RestorationDenoising +
Deconvolution
Discrete-Domain Image ProcessingDiscrete-Domain Image Processing
LSI system, digital LSI system, digital filteringfiltering
Consider the signal asConsider the signal as Infinite signal with finite Infinite signal with finite
number of nonzero number of nonzero coefficientscoefficients
Finite signalFinite signal
Fourier viewFourier view DTFTDTFT DFTDFT
RegistrationMosaicing
SegmentationTracking
AnalysisModeling
PSF h
A/D
Denoising
Deconvolution
RestorationDenoising +
Deconvolution
The RoadmapThe RoadmapIssues
Framework
Revolution in biology
Signal and image representationsSignal and image representationsFourier analysisFourier analysisGabor analysisGabor analysisMultiresolution analysisMultiresolution analysisData-driven representation and analysisData-driven representation and analysis
Tools
t
fDirac basisWPWT
ER
Actin
STFTFT
Signal RepresentationsSignal Representations
““Holy Grail” of signal Holy Grail” of signal analysis/processing analysis/processing Understand the “blob”-like Understand the “blob”-like
structure of the energy structure of the energy distribution in the time-distribution in the time-frequency spacefrequency space
Design a representation Design a representation reflecting thatreflecting that
Data Driven Representation & AnalysisData Driven Representation & Analysis
Use representations based on training data and Use representations based on training data and automated learning approachesautomated learning approaches Wavelet packetsWavelet packets PCA & variationsPCA & variations ICAICA ……
Estimation FrameworkEstimation Framework
Random variations introduced by system noise, Random variations introduced by system noise, artifacts, uncertainty originating from the biological artifacts, uncertainty originating from the biological phenomena lead to statistical methodsphenomena lead to statistical methods
Seek the solution optimal in some probabilistic Seek the solution optimal in some probabilistic sensesense
Optimality criterionOptimality criterion MSE, often depends on unknown parametersMSE, often depends on unknown parameters Bayesian framework, MAP estimatorsBayesian framework, MAP estimators
The RoadmapThe Roadmap
Tasks
Issues
Framework
Tools
Revolution in biology
AcquisitionAcquisitionDeblurring, denoising, restorationDeblurring, denoising, restorationRegistration and mosaicingRegistration and mosaicingSegmentation, tracing and trackingSegmentation, tracing and trackingClassification and clusteringClassification and clusteringModelingModeling
AcquisitionAcquisition
Issues in acquisition of Issues in acquisition of fluorescence microscope imagesfluorescence microscope images
Increase resolutionIncrease resolution Total data acquisition is reduced, speeding up image acquisitionTotal data acquisition is reduced, speeding up image acquisition Allows a higher frame rate (increased temporal resolution)Allows a higher frame rate (increased temporal resolution) Allows us to spend more time acquiring the regions of interest (which gives increased spatial Allows us to spend more time acquiring the regions of interest (which gives increased spatial
resolution)resolution)
Acquire for longer periodsAcquire for longer periods Acquisition process damages both the signal (photobleaching) and the cell (phototoxicity)Acquisition process damages both the signal (photobleaching) and the cell (phototoxicity) Efficient acquisition reduces the total amount of data acquired, thus reducing damage to the cellEfficient acquisition reduces the total amount of data acquired, thus reducing damage to the cell This allows us to observe cellular processes for longer periodsThis allows us to observe cellular processes for longer periods
Intelligent acquisitionIntelligent acquisition Acquire only Acquire only wherewhere and and whenwhen needed needed adaptivity adaptivity Model driven (microscope model & data model)Model driven (microscope model & data model)
Model-Driven AcquisitionModel-Driven Acquisition
AcquisitionAcquisition Grid acquisitionGrid acquisition MR adaptive acquisitionMR adaptive acquisition Markov Random FieldsMarkov Random Fields Example-based enhancementExample-based enhancement
ReconstructionReconstruction Simple interpolation methodsSimple interpolation methods Wavelet reconstructionWavelet reconstruction Model-based reconstructionModel-based reconstruction
Knowledge Extraction
Reconstruction
Efficient Acquisition
Mo
del
ing
MR AcquisitionMR Acquisition
ProblemProblem Why acquire in areas of Why acquire in areas of
low fluorescence?low fluorescence? Acquire only Acquire only whenwhen and and
wherewhere needed needed
Measure of successMeasure of success Problem dependentProblem dependent Here: Here:
Strive to maintain the Strive to maintain the achieved classification achieved classification accuracyaccuracy
ApproachApproach Mimic “Battleship”Mimic “Battleship” Compression Ratio
Accuracy
[Merryman & Kovačević, 2005][Merryman & Kovačević, 2005]
as well as design intelligent acquisition systems based on those models
Develop a mathematical framework and algorithmsto build accurate models of fluorescence microscope data sets
Efficient Acquisition and Learning of Fluorescence Microscope Data Models
2. Choose acquisition regions that allow us to construct an accurate model in the shortest amount of time
1. Use all the input from the microscope to model the data set
2.Intelligent Acquisition
1.Model Building Model
Model satisfactory?
Yes
No
Efficient Acquisition and Learning of Efficient Acquisition and Learning of Fluorescence Microscope Data Models Fluorescence Microscope Data Models
Predict the distribution of fluorescence in the subsequent Predict the distribution of fluorescence in the subsequent frame and acquire accordinglyframe and acquire accordingly Predict likelihood of object moving to any given position Acquire those positions with the highest likelihood
Too small an acquisition region may not find the object Too large an acquisition region is inefficient
Motion modelsMotion models Three motion models commonly observed in practice
Random walk Constant velocity Constant acceleration
[Jackson, Murphy & Kovačević, 2007][Jackson, Murphy & Kovačević, 2007]
Efficient Acquisition and Learning of Efficient Acquisition and Learning of Fluorescence Microscope Data ModelsFluorescence Microscope Data Models
Learning the motion modelLearning the motion model Prediction: Based on current beliefs about motion model,
find likelihood of each object appearing at any given pixel in the subsequent frame
Acquisition: Acquire the pixels that have the highest overall likelihood of containing an object
Observation: Observe the actual location of each object, if found
Update: Use this information to update our beliefs about the motion models for each object
Efficient Acquisition and Learning of Efficient Acquisition and Learning of Fluorescence Microscope Data ModelsFluorescence Microscope Data Models
Known motion modelKnown motion model Single object, random walk of known variance Probability distribution of it appearing in any given location
in the subsequent frame Acquisition regions capture the locations where the object
is expected with the highest probabilities
Efficient Acquisition and Learning of Efficient Acquisition and Learning of Fluorescence Microscope Data ModelsFluorescence Microscope Data Models
Known motion modelKnown motion model If the object is detected, repeat, centering the new
acquisition region at the object’s most recent location If the object is not detected, estimate where it is Probability distribution given that the object was not in the
acquisition region
Efficient Acquisition and Learning of Efficient Acquisition and Learning of Fluorescence Microscope Data ModelsFluorescence Microscope Data Models
Known motion modelKnown motion model Predict this object’s location in the next frame Probability distribution
1D case: choose two disconnected acquisition regions 2D case: choose to acquire between the two black circles
Deblurring, Denoising & RestorationDeblurring, Denoising & Restoration
Microscope images contain artifactsMicroscope images contain artifacts Blurring caused by a PSFBlurring caused by a PSF Noise from the electronics of digitizationNoise from the electronics of digitization
Deblurring/deconvolutionDeblurring/deconvolution Widefield microscopyWidefield microscopy Effect of depthEffect of depth
DenoisingDenoising
Deconvolution + Denoising = RestorationDeconvolution + Denoising = Restoration
Registration & MosaicingRegistration & Mosaicing
RegistrationRegistration Find spatial relationship and alignment between imagesFind spatial relationship and alignment between images
MosaicingMosaicing Used when fine resolution is needed within a global viewUsed when fine resolution is needed within a global view Stitching together pieces of an imageStitching together pieces of an image Usually requires registration, given overlapping piecesUsually requires registration, given overlapping pieces
Segmentation, Tracing & TrackingSegmentation, Tracing & Tracking
SegmentationSegmentation Methods used: thresholding and watershedMethods used: thresholding and watershed Edge-based, region-based, combinationEdge-based, region-based, combination Active contoursActive contours
TracingTracing Mostly tracing of axonsMostly tracing of axons Typical, path following approachesTypical, path following approaches Fail in the presence of noiseFail in the presence of noise
TrackingTracking Molecular dynamics and cell migrationMolecular dynamics and cell migration Tracking of objects over timeTracking of objects over time
SegmentationSegmentation
Separate objects of interest Separate objects of interest from each other and the from each other and the backgroundbackground
Fundamental step in Fundamental step in microscopymicroscopy
Hand segmentationHand segmentation Not reproducibleNot reproducible Not tightNot tight Piecewise linearPiecewise linear Cannot compute statisticsCannot compute statistics Time-consumingTime-consuming
Current standardCurrent standard Watershed segmentationWatershed segmentation
Active Contour SegmentationActive Contour Segmentation
Active contour algorithmsActive contour algorithms Contour comparable to an elastic stringContour comparable to an elastic string Moved under external and internal forcesMoved under external and internal forces
External: derived from the image (edges)External: derived from the image (edges) Internal: geometric properties of the contour (curvature)Internal: geometric properties of the contour (curvature)
Level Set method: A way to track the contour as it evolvesLevel Set method: A way to track the contour as it evolves
Positive inside the contour Positive inside the contour (mountain)(mountain)
Negative outside the contour Negative outside the contour (valley)(valley)
Zero on the contour, Zero on the contour, C embedded at its zero (sea) levelC embedded at its zero (sea) level
n
Fc > 0
Fc < 0
> 0
< 0
= 0
STACSSTACS Combines energy minimization approach with statistical modelingCombines energy minimization approach with statistical modeling
Model matchingModel matching Pixels inside and outside the contour follow different statistical Pixels inside and outside the contour follow different statistical
modelsmodels Modified STACs for fluorescence microscopy imagesModified STACs for fluorescence microscopy images
No edge informationNo edge information No obvious shape informationNo obvious shape information Segmentation driven by statistics of the image and contour Segmentation driven by statistics of the image and contour
smoothnesssmoothness
MSTACSMSTACS: Our level-set evolution equation: Our level-set evolution equation
Topology needs to be preserved Topology needs to be preserved TPSTACS TPSTACS
TPSTACS: ResultsTPSTACS: Results
SuccessfulSuccessful
ProblemProblem Extremely slowExtremely slow
SolutionSolution MRSTACSMRSTACS
Hand-segmented
TPTACS
[Coulot, Kirschner, Chebira, Moura, Kovačević, Osuna & Murphy, 2006][Coulot, Kirschner, Chebira, Moura, Kovačević, Osuna & Murphy, 2006]
37
MRSTACSMRSTACS Decompose image Decompose image
to L levelsto L levels
Smoothing renders cell Smoothing renders cell easier to discerneasier to discern
Detect cells using Detect cells using morphological operationsmorphological operations
Get coarse version of Get coarse version of contour (TPSTACS)contour (TPSTACS)
Refine contour iteratively Refine contour iteratively faster faster segmentationsegmentation Coarse result < 3 secCoarse result < 3 sec Fine result < 30 minFine result < 30 min
horizontalvertical
↓2g
↓2h↓2h
↓2g
↓2h
↓2g2D Filter bankLevel 1 decomposition
A Critical Review of Active Contours
Flexible
Can be tuned to be accurate
Adapt to topological changes in the image
But… Tuning of parameters is involved Updating the level set function – inefficient What is the ‘contour’ in a digital image? Discrete topological rules – external constraints can cause
abruptness Multiresolution – how do we reconstruct the level set function?
New math needed
Active Mask Framework: No ContoursActive Mask Framework: No Contours
Fluorescence microscope images speckled in natureFluorescence microscope images speckled in nature Estimate densities of bright pixels in local neighborhood Estimate densities of bright pixels in local neighborhood
at different scalesat different scales
Recast computation of force as a transformationRecast computation of force as a transformation No need for the time consuming extension functionNo need for the time consuming extension function
For image f, transform T isFor image f, transform T is
Windowing function Windowing function and scale factor and scale factor aa Different conditions (cell lines, resolution, etc.) Different conditions (cell lines, resolution, etc.) Different Different and and aa TPSTACS: Rectangular TPSTACS: Rectangular , a, a = 1 and suitable operands = 1 and suitable operands
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
Original ImageOriginal Image A slight blurA slight blur
Enough to discern the cell Enough to discern the cell boundaryboundary
Too much blur – Edges Too much blur – Edges roundedrounded
Active Masks: ResultsActive Masks: Results
SuccessSuccess Initialization: Level set function is identically zeroInitialization: Level set function is identically zero
Iterations: 3Iterations: 3
Time taken: 6.5 sec per iterationTime taken: 6.5 sec per iteration
HeLa cells – Total protein imageHeLa cells – Total protein image HeLa cells – Membrane protein imageHeLa cells – Membrane protein image
Active Masks
Pros Framework suited to digital images Can be made specific with the choice of suitable forces,
windows and scale factors Performance not critically dependent on initialization Easy and fast to compute Translation, dilation and rotation invariance can be
preserved
Cons Topology preservations hard
Multiple active mask framework
Multiple Active Masks
Initialization Random initialization with M»M0 masks,
where M0 = expected number of objects in the image
Evolution: driven by distributor functions
Can incorporate multiresolution/multiscale
Convergence Experimentally Working on a proof
Results of Results of STACS on Different ModalitiesSTACS on Different Modalities
Yeast DIC Cardiac MRI: Endocardium and epicardium
Axial Coronal
True Positive False Positive False Negative
Saggital
Brain fMRI
Classification Problems in BioimagingClassification Problems in Bioimaging
Determination of Determination of protein subcellular location patternsprotein subcellular location patterns[Chebira, Barbotin, Jackson, Merryman, Srinivasa, Murphy & Kovačević, 2007][Chebira, Barbotin, Jackson, Merryman, Srinivasa, Murphy & Kovačević, 2007]
Detection of developmental stages in Detection of developmental stages in DrosophilaDrosophila embryos embryos[Kellogg, Chebira, Goyal, Cuadra, Zappe, Minden & Kovačević, 2007][Kellogg, Chebira, Goyal, Cuadra, Zappe, Minden & Kovačević, 2007]
Classification of histological stem-cell teratomasClassification of histological stem-cell teratomas[Ozolek, Castro, Jenkinson, Chebira,, Kovačević, Navara, Sukhwani, [Ozolek, Castro, Jenkinson, Chebira,, Kovačević, Navara, Sukhwani, Orwig, Ben-Yehudah & Schatten, 2007]Orwig, Ben-Yehudah & Schatten, 2007]
Fingerprint recognitionFingerprint recognition [Hennings, Thornton, Kovačević & Kumar , 2005] [Hennings, Thornton, Kovačević & Kumar , 2005] [Chebira, Coelho, Sandryhalia, Lin, Jenkinson, MacSleyne, Hoffman, Cuadra, [Chebira, Coelho, Sandryhalia, Lin, Jenkinson, MacSleyne, Hoffman, Cuadra, Jackson, Püschel & Kovačević , 2007]Jackson, Püschel & Kovačević , 2007]
Develop an Develop an automated systemautomated system capable of capable of
fast, robust and accurate classificationfast, robust and accurate classification
Multiresolution ClassificationMultiresolution Classification
Hypothesis: Better classification accuracy obtained if we use the space-Hypothesis: Better classification accuracy obtained if we use the space-frequency information lying in the MR subspacesfrequency information lying in the MR subspaces Compute features in the MR-decomposed subspaces (subbands) insteadCompute features in the MR-decomposed subspaces (subbands) instead
Would like to use wavelet packetsWould like to use wavelet packets Do not have an obvious cost measureDo not have an obvious cost measure Do it implicitly insteadDo it implicitly instead
Generic Classification System
FeatureExtraction
ClassificationMRWeightingAlgorithmFE CMR W
shorthand
MR BlockMR Block
Grow full tree to L levelsGrow full tree to L levels
Use all nodes Use all nodes
MR Bases MR Bases DWTDWT DFTDFT DCTDCT … …
MR FramesMR Frames SWTSWT DT-CWTDT-CWT DD-DWTDD-DWT Our design: LTFTOur design: LTFT
FE CMR W
Lapped Tight Frame Transforms
Build MR transforms for these problems
Not many nonredundant ones exist
Seed them from higher-dimensional bases
Feature Extraction and ClassifierFeature Extraction and Classifier
Feature ExtractionFeature Extraction New Haralick texture features (TNew Haralick texture features (T33, 26 features), 26 features)
Morphological features (M, 16 features)Morphological features (M, 16 features) Zernike features (Z, 49 features)Zernike features (Z, 49 features)
ClassifierClassifier Neural networksNeural networks
No hidden layersNo hidden layers
FE CMR W
FE CMR W
Weighting ProcedureWeighting Procedure
Local decisionsLocal decisions Decision vectors for each subband Decision vectors for each subband
of each training image containing C numbersof each training image containing C numbers Goal: combine local decisions into a global oneGoal: combine local decisions into a global one
AlgorithmsAlgorithms Open form (iterative)Open form (iterative) Closed form (analytical)Closed form (analytical)
Per data setPer data set Per classPer class
Pruning criteriaPruning criteria
FE CMR W
Determination of PSL Patterns: Determination of PSL Patterns: ResultsResults
MR significantly MR significantly outperforms outperforms NMRNMR
MRF outperform MRF outperform MRBMRB
Per-Dataset CF Per-Dataset CF slightly slightly outperforms OFoutperforms OF
Trend is flatTrend is flat→ → TT33 set enough set enough
Why Do MR Frames Work?
Looking into classes of signals where bases/frame perform better
Simple example Real plane Two classes Decision rule Union of nonoverlaping parallelograms, bases,
otherwise, frames
Conclusions and OpportunitiesConclusions and Opportunities
Tasks
Issues
Framework
Tools
Revolution in biology
What can we do?
Conclusions & OpportunitiesConclusions & Opportunities
The “dream”:The “dream”:automated, efficient andautomated, efficient andreliable processing as wellreliable processing as wellas knowledge extractionas knowledge extraction
from large bioimage from large bioimage databasesdatabases
Dig in!Dig in!
Gaps to fillGaps to fill Need tools adapted to Need tools adapted to
specific bioimaging specific bioimaging applications applications
Need to adapt state-of-the-Need to adapt state-of-the-art techniques and/or art techniques and/or come up with new ones for come up with new ones for bioimaging tasksbioimaging tasks
top related