ter haar romeny, fev mr slice hartcoronair scale toppoints graph theory edge focusing

ter Haar Romeny, FEV

MR slice hartcoronair

scale

• toppoints

• graphtheory

Edge focusing


Structures exist at their own scale:

Original = e0 px = e1 px = e2 px = e3 px

Noise edges


100 200 300 400 500

4

2

2

4

6

8

10

The graph of the sign-change of thefirst derivative of a signal as a function of scale is denoted the scale-space signature of the signal.

Zero-crossingsof the secondorder derivative= max of firstorder derivative,

as a functionof scale


The notion of longevity can be viewed of a measure of importance for singularities [Witkin83]. The semantical notions of prominence and conspicuity now get a clear meaning in scale-space theory.

In a scale-space we see the emergence of the hierarchy of structures. Positive and negative edges come together and annihilate in singularity points.


Example:

Lysosomesegmentationin noisy 2-photonmicroscopy3D images ofmacrophages.


Marching-cubes isophote surface ofthe macrophage.

slice 24 slice 21 slice 25 slice 18 slice 22 slice 21

slice 24 slice 23 slice 24 slice 20 slice 18 slice 24

Preprocessing:- Blur with = 3 px- Detect N strongest maxima


We interpolatewith cubic splinesinterpolation35 radial tracksin 35 3Dorientations


The profiles are extremely noisy:

Observation: visually we can reasonably point the steepest edgepoints.


Edge focusingover all profiles.

Choose a startlevel based onthe task, i.e. finda single edge.


Detected 3D points per maximum.

We need a 3D shape fit function.


1

2

,1

23

2 sin, 1

23

cos,

1

23

2 sin, 1

42 15

2 sin2,

1

215

2 cossin, 1

45

3 cos2 1,

1

215

2 cossin, 1

42 15

2 sin2

The 3D points are least square fit with 3D spherical harmonics:


Resulting detection:


An efficient way to detect maxima and saddlepoints is found inthe theory of vector field analysis (Stoke’s theorem)


Topological winding numbers

p Li1dLi2. . .dLin i1i2...inL jpL jpn2p L1dL2 L2dL1

L12 L22

N-D

2-D

is the wedge product (outer product for functionals)


In 2D: the surrounding of the point P is a closed path around P.

The winding number of a point P is defined as the number of times the image gradient vector rotates over 2 when we walk over a closed path around P.

maximum: = 1minumum: = 1regular point: = 0saddle point: = -1monkey saddle: = -2


The notion of scale appears in the size of the path.

Winding number = +1 extremum

Winding number = -1 saddle


Generalised saddle point (5th order): (x+i y)5

The winding numbers add within a closed contour, e.g.A saddle point (-1) and an extremum (+1) cancel, i.e. annihilate.

Catastrophe theory

Winding number = - 4 monkey saddle


2 4 6 8 1010200

300

500

700

1000

Slopefor MR image: 1.66555

2 4 6 8 1010

1000

1500

2000

Slopefor white noise: 1.91549

The number of extrema and saddlepoints decrease as e-N over scale

Decrease of structure over scale scales with the dimensionality.


Fertility Prospects

In most developed countries a postponement of childbearing is seen.

E.g. in the Netherlands: Average age of bearing first child is 30 years.

Computer-Assisted Human Follicle Analysis for Fertility Prospects with

3D Ultrasound

ter Haar Romeny et al., IPMI 1999

Application:


pelvis

oviduct

ovary

uterus

rectum

vagina

anusbladder

vulvaureter

clitoris

Femalereproductive anatomy


Ovary

Oviduct

Uterus wall Uterus

Endometrium

Uterus neck


The number of follicles decreases during lifetime


1. As female fecundicity decreases with advancing age, an increasing number of couples is faced with unexpected difficulties in conceiving.

• Approx. 15000 couples visit fertility clinics annually• In 70% of these cases age-related fecundicity decline

may play a role• A further increase is expected

2. In our emancipated society a tension between family planning and career exists.

• Being young, till what age can I safely postpone childbearing?• Getting older, at what age am I still likely to be able to conceive

spontaneously?• A further increase is expected

Menopausal age


Resting 0.03 mm initiation of growth> 120 days?

Early growing 0.03 - 0.1 mm

Preantral 0.1 - 0.2 mm basal growth~ 65 days

Antral 0.2 - 2 mm

Selectable 2 - 5 mm rescued by FSH window~ 5 days

Selected 5 - 10 mm

Dominance 10 - 20 mm maturation~ 15 days

Ovulation

A follicle’s life


3D Ultrasound is a safe, cheap and versatile appropriate modality

Kretz Medicor 530D


Two 3D acquisition strategies:

1. Position tracker on regular probe

2. Sweep of 2D array in transducer

Trans-vaginal probe

Regular sampling from irregularly space slices


Manual counting is very cumbersome Automated follicle assessment

• 2-5 mm hypodense structures• structured noise• vessels may look like follicles• ovary boundary sometimes missing


Automated method:

1. Detection of intensity minima by 3D ‘winding numbers’2. Isotropic ray tracing (500 directions) from detected centra3. Edge detection by 1D winding numbers4. Edge focusing to detect most prominent edge5. Fit spherical harmonics to edgepoints6. Calculate follicle shape/size parameters and visualize


Detection of a singularity (i.e. a minimum)

From theory of vector fields several important theorems (Stokes, Gauss) exist that relate something happening in a volume with just its surface.We can detect singularities by measurements around the singularity.

P

1-D: difference of signs of the gradient i zero crossing or extremum

yxi

,

i i

The surrounding of the point P are just 2 pointsleft and right of P 1D sphere.


ijjiW

d

.21

21

1221

dd

dWe consider a unit gradient vector, so 1

2+22=1.

ijjidd In subscript notation:

where ij is the antisymmetric tensor.

01

10ij


For regular points, i.e. when no singularity is present in W, the winding number is zero, as we see from the Stokes’ theorem:

0:21

321 W W

iiiiiii dStokesddd d

d

where the fact that the (d-1)-form is a closed form was used.

So, as most of our datapoints are regular, we detect singularities very robustly as integer values embedded in a space of zero's.


Example of a result:

1 cm

Dataset 2563, radius Stokes’ sphere 1 pixel, blurring scale 3 pixels


• a conservation of winding number within the closed contour.We measure the sum of the winding numbers. E.g. enclosing a saddlepoint and a minimum adds up to zero.

• the winding number is independent of the shape of W.It is a topological entity.

• the winding number only takes integer values.Multiples of the full rotation angle.Eeven when the numerical addition of angles does not sum up to precisely an integer value, we may rightly round off to the nearest integer.

The winding number has nice properties:


• the winding number is a scaled notionThe neighbourhood defines the scale.

• the behaviour over scale generates a tree-like structureTypical annihilations, creations and collisions, from which much can be learned about the ‘deep structure’ of images.

• the winding number is easy to compute, in any dimension.

• the WN is a robust characterisation of the singular points in the image: small deformations have a small effect.


Detection of follicle boundaries:

• generation of 200 - 500 rays in a homogeneous orientation distribution• determine most pronounced edge along ray by winding number focusing• fit spherical harmonics to get an analytical description of the shape• calculate volume and statistics on shape

Distance along rayDistance along ray

Sca

le

US inte

nsi

ty

Sca

le

Distance along ray


3D scatterplot ofdetected endpoints

3D visualisation of fittedspherical harmonics function


Validation with 2 bovine ovaria

• anatomincal coupes• high resolution MR• 3D ultrasound

Follicle#

xcenter

ycenter

zcenter

distance toneighbor(pixels)

volume fromspherical harmonics

(mm3)

volumefrom MRI

volumefrom

anatomyv00 99 51 35 25.4 259.7 250.0 262.1

93 28 44 45.0 28.4 27.0 29.8113 55 74 41.6 56.2 54.9 59.3

v01 33 41 66 25.3 242.347 22 75 44.5 34.064 49 44 38.9 54.7

v44 72 49 84 25.4 239.769 28 70 45.2 28.432 51 82 40.1 59.3


Patient studies:

Performance of the algorithm compared with a human expert. Number of follicles found. Data for 6 patients. The datasets are cut off to contain only the ovary. Scales used: = 3.6, 4.8, 7.2 and 12 pixels.

Patient # manual Computer Patient # Manual computer1 17 15 4 14 92 10 8 5 9 73 7 5 6 9 7


Conclusions:

• 3D ultrasound is a feasible modality for follicle-based fertilitiy state estimation• automated CAD is feasible, more clinical validation needed• winding numbers are robust (scaled) singularity detectors• a robust class of topological properties emerges


Multi-scale watershed segmentation

Watershed are the boundaries of merging water basins, when theimage landscape is immersed by punching the minima.

At larger scale the boundaries get blurred, rounded and dislocated.


Regions of different scales can be linked by calculating thelargest overlap with the region in the scales just above.


The method is often combined with nonlinear diffusion schemes

E. Dam, ITU


Nabla Vision is an interactive 3D watershed segmentation tooldeveloped by the University of Copenhagen.

Sculpture the 3D shape by successively clicking precalculatedfiner scale watershed details.


11

22

1

2 )tan(,tan

We expand the left and right hand side of the last equation in a Taylor series

up to first order in and 1 respectively. For the left hand side we obtain

)(cos

1tan)tan( 2

12

O

).(

)(

212

1

1221

1

2

2112

1

22

1

22

11

22

O

O

3D winding number

And for the righthand side

.21

21

1221

dd

d


In n-D: d

d

iiiiiiiW

ddd 21

321

In 3-D:

))(

)(

)((

dxdz

dzdy

dydx

kzjxkxjz

kyjzkzjy

kxjykyjxiijk

ijkmlkmjli

ijkjji dxdxdd

This expression has to be evaluated for all voxels of our closed surface. We can do this e.g. for the 6 planes of the surrounding cube. On the surface z = constant the previous equation reduces to

dydxkxjykyjxiijk )(

Contraction of indices:


Performing the contraction on the indices i, j and k gives

)(2

)(2

)(2

xyyxyyxxz

zyxxzxxyy

yyzxzyyxx

Calculation of• the gradient vector elements i = { x, y, z} • the derivatives of the gradient field, e.g. x y = y/xis done by neighbour subtraction.The single pixel steps dx and dy are unity.

dydxkxjykyjxiijk )(

ter haar romeny, fev mr slice hartcoronair scale toppoints graph theory edge focusing

Documents

fev slide

d slide

d ultrasound ter haar

fev edge

functionals slide

missing slide

lifetime slide

fev structures