segmentationsegmentation c. phillips, institut montefiore, ulg, 2006

SegmentationSegmentationSegmentationSegmentation

C. Phillips, Institut Montefiore, ULg, 2006C. Phillips, Institut Montefiore, ULg, 2006

In image analysis, segmentation is the partition of a digital image into multiple regions (sets of pixels), according to some criterion.

The goal of segmentation is typically to locate certain objects of interest which may be depicted in the image.

Segmentation criteria can be arbitrarily complex, and take into account global as well as local criteria. A common requirement is that each region must be connected in some sense.

DefinitionDefinitionDefinitionDefinition

A simple example of segmentation is thresholding a grayscale image with a fixed threshold t: each pixel p is assigned to one of two classes, P0 or P1,

depending on whether I(p) < t or I(p) ≥ t.

t=.5

Example: medical imaging...

How to fix the threshold ?

Goal of brain image segmentation

Split the head volume into its « main » components:

• gray matter (GM)• white matter (WM)• cerebrol-spinal fluid (CSF)• the rest/others

• (tumour)

Manual segmentation:an operator classifies the voxels manually

Segmentation approachesSegmentation approachesSegmentation approachesSegmentation approaches

Semi-automatic segmentation:an operator defines a set of parameters, that are passed to an algorithm

Example: threshold at

t=200



Automatic segmentation:no operator intervention

objective and reproducible

Model the histogram of the image !

Intensity based segmentationIntensity based segmentationIntensity based segmentationIntensity based segmentation

Segmentation - Segmentation - Mixture ModelMixture ModelSegmentation - Segmentation - Mixture ModelMixture Model

• Intensities are modelled by a mixture of K Intensities are modelled by a mixture of K Gaussian distributions, parameterised by:Gaussian distributions, parameterised by:– meansmeans– variancesvariances– mixing proportionsmixing proportions

1

23

12 3

ii

i

Segmentation - Segmentation - AlgorithmAlgorithmSegmentation - Segmentation - AlgorithmAlgorithm

Starting estimates for belonging probabilities

Compute Gaussian parameters from

belonging probabilities

Compute belonging probabilities from

Gaussian parameters

Converged ?No Yes STOP

1

23

12 3

Segmentation - Segmentation - ProblemsProblemsSegmentation - Segmentation - ProblemsProblems

Noise & Partial volume effect

MR images are corrupted by a smooth MR images are corrupted by a smooth intensity non-uniformity (bias).intensity non-uniformity (bias).

Image with bias

artefact

Corrected image

Segmentation - Segmentation - ProblemsProblemsSegmentation - Segmentation - ProblemsProblems

Intensity bias field

Segmentation - Segmentation - PriorsPriorsSegmentation - Segmentation - PriorsPriors

Overlay prior belonging probability maps to Overlay prior belonging probability maps to assist the segmentationassist the segmentation– Prior probability of each voxel being of a Prior probability of each voxel being of a

particular type is derived from segmented particular type is derived from segmented images of 151images of 151subjectssubjects• Assumed to be Assumed to be

representativerepresentative

– Requires initialRequires initialregistration toregistration tostandard space.standard space.

• Bias correction informs segmentation

• Registration informs segmentation

• Segmentation informs bias correction

• Bias correction informs registration

• Segmentation informs registration

Unified approach:Unified approach:segmentation-correction-registrationsegmentation-correction-registration

Unified approach:Unified approach:segmentation-correction-registrationsegmentation-correction-registration

Unified SegmentationUnified Segmentation

• The solution to this circularity is to put The solution to this circularity is to put everything in the same everything in the same Generative ModelGenerative Model..– A MAP solution is found by repeatedly A MAP solution is found by repeatedly

alternating among classification, bias alternating among classification, bias correction and registration steps.correction and registration steps.

• The Generative Model involves:The Generative Model involves:– Mixture of Gaussians (MOG)Mixture of Gaussians (MOG)– Bias Correction ComponentBias Correction Component– Warping (Non-linear Registration) ComponentWarping (Non-linear Registration) Component

Gaussian Probability DensityGaussian Probability DensityGaussian Probability DensityGaussian Probability Density

• If intensities are assumed to be If intensities are assumed to be Gaussian of meanGaussian of mean kk and varianceand variance 22

kk,, then the probability of a valuethen the probability of a value yyii is:is:

Non-Gaussian Probability DistributionNon-Gaussian Probability DistributionNon-Gaussian Probability DistributionNon-Gaussian Probability Distribution

• A non-Gaussian probability density function A non-Gaussian probability density function can be modelled by a Mixture of Gaussians can be modelled by a Mixture of Gaussians (MOG):(MOG):

• A non-Gaussian probability density function A non-Gaussian probability density function can be modelled by a Mixture of Gaussians can be modelled by a Mixture of Gaussians (MOG):(MOG):

Mixing proportion - positive and sums to one

Mixing ProportionsMixing ProportionsMixing ProportionsMixing Proportions

• The mixing proportionThe mixing proportion kk represents the prior represents the prior

probability of a voxel being drawn from classprobability of a voxel being drawn from class kk - irrespective of its intensity.- irrespective of its intensity.

• So:So:

• The mixing proportionThe mixing proportion kk represents the prior represents the prior

probability of a voxel being drawn from classprobability of a voxel being drawn from class kk - irrespective of its intensity.- irrespective of its intensity.

• So:So:

Non-Gaussian Intensity Non-Gaussian Intensity DistributionsDistributions

Non-Gaussian Intensity Non-Gaussian Intensity DistributionsDistributions

• Multiple Gaussians per tissue class allow Multiple Gaussians per tissue class allow non-Gaussian intensity distributions to non-Gaussian intensity distributions to be modelled.be modelled.

• Multiple Gaussians per tissue class allow Multiple Gaussians per tissue class allow non-Gaussian intensity distributions to non-Gaussian intensity distributions to be modelled.be modelled.

Probability of Whole DatasetProbability of Whole DatasetProbability of Whole DatasetProbability of Whole Dataset

• If the voxels are assumed to be If the voxels are assumed to be independent, then the probability of the independent, then the probability of the whole image is the product of the whole image is the product of the probabilities of each voxel:probabilities of each voxel:

• It is often easier to work with negative It is often easier to work with negative log-probabilities:log-probabilities:

• If the voxels are assumed to be If the voxels are assumed to be independent, then the probability of the independent, then the probability of the whole image is the product of the whole image is the product of the probabilities of each voxel:probabilities of each voxel:

• It is often easier to work with negative It is often easier to work with negative log-probabilities:log-probabilities:

Modelling a Bias FieldModelling a Bias FieldModelling a Bias FieldModelling a Bias Field

• A bias field is included, such that the A bias field is included, such that the required scaling at voxelrequired scaling at voxel ii,, parameterised byparameterised by , is, is ii(())..

• Replace the means byReplace the means by kk//ii(())

• Replace the variances byReplace the variances by ((kk//ii(())))22

• A bias field is included, such that the A bias field is included, such that the required scaling at voxelrequired scaling at voxel ii,, parameterised byparameterised by , is, is ii(())..

• Replace the means byReplace the means by kk//ii(())

• Replace the variances byReplace the variances by ((kk//ii(())))22

Modelling a Bias FieldModelling a Bias FieldModelling a Bias FieldModelling a Bias Field

• After rearranging:After rearranging:• After rearranging:After rearranging:

()y y ()

Tissue Probability MapsTissue Probability MapsTissue Probability MapsTissue Probability Maps

• Tissue probability maps (TPMs) are used Tissue probability maps (TPMs) are used instead of the proportion of voxels in each instead of the proportion of voxels in each Gaussian as the prior.Gaussian as the prior.

• Tissue probability maps (TPMs) are used Tissue probability maps (TPMs) are used instead of the proportion of voxels in each instead of the proportion of voxels in each Gaussian as the prior.Gaussian as the prior.

ICBM Tissue Probabilistic Atlases. These tissue probability maps are kindly provided by the International Consortium for Brain Mapping, John C. Mazziotta and Arthur W. Toga.

““Mixing Proportions”Mixing Proportions”““Mixing Proportions”Mixing Proportions”

• Tissue probability maps Tissue probability maps for each class are for each class are available.available.

• The probability of The probability of obtaining classobtaining class kk at voxelat voxel ii, given weights, given weights is then:is then:

Deforming the Tissue Probability Deforming the Tissue Probability MapsMaps

Deforming the Tissue Probability Deforming the Tissue Probability MapsMaps

• Tissue probability Tissue probability images are images are deformed deformed according to according to parametersparameters ..

• The probability of The probability of obtaining classobtaining class kk at voxelat voxel ii, given , given weightsweights and and parametersparameters is is then:then:

The Extended ModelThe Extended ModelThe Extended ModelThe Extended Model

• By combining the modifiedBy combining the modified P(cP(cii=k|=k|)) and and P(yP(yii||

ccii=k,=k,)), , the overall objective functionthe overall objective function ( (EE) ) becomes:becomes:

The Objective Function

OptimisationOptimisationOptimisationOptimisation

• The “best” parameters are those that The “best” parameters are those that minimise this objective function.minimise this objective function.

• Optimisation involves finding them.Optimisation involves finding them.• Begin with starting estimates, and Begin with starting estimates, and

repeatedly change them so that the repeatedly change them so that the objective function decreases each time.objective function decreases each time.

Schematic of optimisationSchematic of optimisationSchematic of optimisationSchematic of optimisation

Repeat until convergence...Repeat until convergence...

HoldHold ,, ,, 22 andand constant, and minimiseconstant, and minimise EE w.r.t.w.r.t. - - Levenberg-Marquardt strategy, usingLevenberg-Marquardt strategy, using dE/ddE/d andand dd22E/dE/d22


HoldHold andand constant, and minimiseconstant, and minimise EE w.r.t.w.r.t. ,, andand 22

-Use an Expectation Maximisation (EM) strategy.-Use an Expectation Maximisation (EM) strategy.

endend

Repeat until convergence...Repeat until convergence...



HoldHold andand constant, and minimiseconstant, and minimise EE w.r.t.w.r.t. ,, andand 22

-Use an Expectation Maximisation (EM) strategy.-Use an Expectation Maximisation (EM) strategy.

endend

(Iterated Conditional Mode)

Levenberg-Marquardt OptimisationLevenberg-Marquardt OptimisationLevenberg-Marquardt OptimisationLevenberg-Marquardt Optimisation

• LM optimisation is used for the nonlinear LM optimisation is used for the nonlinear registration and bias correction components.registration and bias correction components.

• Requires first and second derivatives of the Requires first and second derivatives of the objective function (objective function (EE).).

• ParametersParameters andand are updated byare updated by

• Increase Increase to improve stability (at expense of to improve stability (at expense of decreasing speed of convergence).decreasing speed of convergence).

EM is used to update EM is used to update , , 22 and and EM is used to update EM is used to update , , 22 and and

For iteration For iteration (n)(n), alternate between:, alternate between:– E-stepE-step: Estimate belonging probabilities by:: Estimate belonging probabilities by:

– M-stepM-step: Set : Set (n+1)(n+1) to values that reduce: to values that reduce:

Voxels are assumed independent!

Hidden Markov Random FieldHidden Markov Random FieldHidden Markov Random FieldHidden Markov Random Field

Voxels are NOT independent:GM voxels are surrounded by other GM voxels, at least on one side.

Model the intensity and classification of the image voxels by 2 random field:

• a visible field y for the intensities• a hidden field c for the classifications

Modify the cost function E:

And, at each voxel, the 6 neighbouring voxels are used to to build Umrf, imposing local spatial constraints.


T1 image T2 image


White matter

T1 & T2: MoG + hmrf T1 only: MoG only


Gray matter



CSF


PerspectivesPerspectivesPerspectivesPerspectives

•Multimodal segmentation :1 image is good but 2 is better ! Model the joint histogram using multi-

dimensional normal distributions.

•Tumour detection :contrasted images to modify the prior imagesautomatic detection of outliers ?

Thank you for your attention !

segmentationsegmentation c. phillips, institut montefiore, ulg, 2006

Documents

goal of segmentation

segmentation criteria

gaussian distributions

segmentationprior probability

grayscale image

digital image

image analysis

bias correctionbias