department of computer science computer vision & pattern recognition group iapr workshop on...

Department of Computer ScienceDepartment of Computer ScienceComputer Vision & Pattern Recognition Group Computer Vision & Pattern Recognition Group http://cvpr.uni-muenster.dehttp://cvpr.uni-muenster.de

IAPR Workshop on Graph-based

Representations in Pattern Recognition

June 11th -13th, 2007 – Alicante (Spain)

GbR ’07GbR ’07

Separation of the Retinal Vascular Graph

in Arteries and Veins

Speaker: Kai RothausCo-authors: P. Rhiem, X. Jiang

CVPR Group, University of Münster

Homepage: cvpr.uni-muenster.de

Rothaus, Rhiem and Jiang: Separation of the Vascular Graph in Arteries and Veins

GbR ’07GbR ’07


22

Outline

Introduction

– Medical purpose

– Image-processing

Method

– SAT-problem specification (vessel labelling)

– Operations for graph manipulation (edge labelling)

– Solving Conflicts

Results

Conclusions and further work


GbR ’07GbR ’07


33

Medical Purpose

Why retinal vessel are of interest?– Vessels of retina and brain are conjuct– Only on retina vessels are visible directly– Conclusions on diseases are possible

Anatomy of the eye– Vessels enter the eyeball at the optic disc– Vessels only branch (no reconnection)– Capillars are invisible

Differences of two vessel types on retina:

Arteries Veinsoxygenated blood oxygen-deficient blood

thinner thickerlight-red dark-red

stronger central reflex poor central reflexnever crossing arteries never crossing veins


GbR ’07GbR ’07


44

Vessel segmentation

Input: Retinal Image Output: Binary vessel image Many segmentation algorithms, based on

– Matched-filter– Tracking– Intensity riges or (1st moment deviations)– Curvature (2nd moment deviations)

Special difficulties– Handling of bifurcations and crossings– Central-light reflex– Different vessel width– Wide intensity spectrum– Pathological objects nearby

Mainly, we use hand-segmented images


GbR ’07GbR ’07


55

Graph-based representation of the vasculature

Input: Binary vessel image Output: Vasculature graph1. Compute the skeleton of the vasculature2. Classify skeleton pixel in

– End pixel (form vertices of degree 1)– Connection pixel (form edges)– Branching pixel (form vertices of degree 3)– Crossing pixel (form vertices of degree 4)

3. Construct graph-based representation Arising Problems:

– Segmentation errors could lead to small cycles– Discontinuous segmentation leads to an over-

fragmented graph representation– Skeleton of a crossing could lead to two branches

binary vessel image

skeleton image

vasculature graph


GbR ’07GbR ’07


66

SAT-Problem Specification (vessel labelling)

Problem: Classify each vessel as artery (a) or vein (v) Mainly recent approaches are based on local features

– Colour, cross-profile, thickness, etc.– Work only good for thick vessels nearby the optic disc

We propose a structure-based approach (on vasculature graph)– Label each vessel segment vi as artery (Li = a) or vein (Li = v) – Formalise anatomical properties of the vasculature:

1. At branches only edges of the same labelling are involved2. At crossings an artery crossing a vein

– Construct logical clauses that describe the properties– Cumulate above rules for all vertices and formulate the SAT-problem– Solve this as a CSP (Constraint Search Problem) with AC-3

a a

a

v v

vv

v

a

aa

av

v


GbR ’07GbR ’07


77

The labelling process (AC-3*)

1. Add the incident vertices of few manually labelled vessel segments in the process queue Q

2. While Q is not empty– Take the first vertex and corresponding logical rule– Reduce set of labels of the incident vessels consistent to the rule– If there is a conflict try to solve it (details later)– Otherwise add the new vertices to Q

Order of processing the vertices (rules) is important

conflict

manuallabel

conflict

manuallabel


GbR ’07GbR ’07


88

Q={ v6 }Q={ v3, v8 }Q={ v4, v8 }Q={ v8, v7 }Improvement: Introduce an intelligent initial edge labelling to detect split crossings Q={ v7 }

conflict


GbR ’07GbR ’07


99

Operations for graph manipulation (edge labelling)

Segmentation or skeleton errors lead to unsolvable SAT-problem

Graph structure has to be manipulated slightly Allowed operations should handle:

1. Split crossings(instead of 1 deg. 4 vertex 2 adjacent deg. 3 vertices)

2. Missing segments(crossing degenerated to vertex of degree 3)

3. Falsely detected branches4. Falsely detected segments

Instead of manipulating the graph directly we introduce a second order labelling (edge labelling):

vessel labelling

resolve problem label graph manipulationop1 1 c melt 2 branches to one crossingop2 2+3 e split a branchingop3 4 f delete an edge

– – n nothing (normal edge)

vasculature graph

edge labelling


GbR ’07GbR ’07


1010

Steering the labelling process (Belief propagation)

Plausibility weights [0,1] for each vertex– Assign crossing vertex the plausibility 1 - P1(d)

– Assign branch vertex the plausibility (with β = max αi) P1(d)+P2(β) - P1(d)P2(β)

Plausibility weights [0,1] for each a/v-labelled vessel – Assign hand-labelled vessels plausibility 1– During AC-3* algorithm use a multiplicative propagation

scheme (with weights of corresponding vertex and edge)

Use weights as heuristic to order Q as priority-queue

Use the average vessel weights to rate labelling results

P1(d)

P2(β)


GbR ’07GbR ’07


1111

Initial edge labelling

Decide on plausibility measures P1(d) and P2(β) if a connection edge between to branches is probably a crossing

No false c-label should be introduced Label edge with c-label iff [ d<3 ] or [ P1(d)<0.75 and P2(β)<P2(30°) ]

0

20

40

60

80

100

120

140

160

180

0 20 40 60 80 100 120

Max Angle

Seg

men

t L

eng

th

n-edges

c-edges

c-edges (missed)

C N

CGT 117 25

NGT 0 621

Confusion matrix on 10 training images

Accuracy of >96 %


GbR ’07GbR ’07


1212

Solving Conflicts

Conflicts cannot been avoided (even not with initial labelling) Conflicts are basically introduced by cycles in the vascular graph Topology is responsible for conflicts

Solving-strategy:– Search cycle (vertex set V’), where all vessel labels are defined– Establish edge candidate set E’={ e | e incident to a v in V’ }– Choose a “suitable” n-labelled edge of E’, with minimum weight and

change edge label to c (crossing)– Otherwise label the conflict edge with e (end-segment)– Restart the AC-3* algorithm


GbR ’07GbR ’07


1313

Interactive labelling tool

Requirement: binary vessel image

Physician mark single vessel segments as arteries an veins

Propagation of the manual labelling as far as possible

Solve logical conflicts automatically

If the result is not good enough for the observer, more vessel label

could be manually added

Presenting results in two different ways:artery (auto.) vene (auto.) artery (man.) vene (man.)

Original image

Binary image


GbR ’07GbR ’07


1414

Results on manually segmented images

STARE data set of A. Hoover et al. image im0082

manuel label init. c-label final c-label final e-label solved confl. avg. weight4 17 21 2 6/10 0.18

manuel label init. c-label final c-label final e-label solved confl. avg. weight6 17 21 3 7/9 0.21


GbR ’07GbR ’07


1515

Discussion results on manual segmentations

image manuel label init. c-label final c-label final e-label solved confl. avg. weight

0002 2 13 16 1 4/4 0.14

0003 4 7 8 0 1/1 0.24

0044 5 9 12 1 4/4 0.23

0077 5 7 15 1 9/10 0.14

0081 4 22 25 1 4/4 0.20

0162 7 25 35 8 18/20 0.15

0163 8 16 23 7 14/17 0.20

Most conflicts could be solved by introducing c-label Only few conflicts could not been solved Problematic regions are even hard to been labelled by experts Normally few hand-labels are necessary


GbR ’07GbR ’07


1616

Results on automatic segmentations

Method of Soares et al. and test database DRIVE of Staal High demands on segmentation algorithm:

Different vessel width, no gaps in segmentation, low false positive rate, etc. Some segmentations leads to poorly connected graphs (less rules)


GbR ’07GbR ’07


1717

Summary and Conclusions

We have developed a method for propagating vessel classification Requirement is a binary vessel image Problem is formulated as Constraint Search Problem Arising conflicts are solved by manipulating graph structure Interactive environment is developed for physicians

Methods works good for tested image databases Quality depends strongly on segmentation result

Further works– Statistical foundation of plausibility function– Realise initial labelling with Bayesian classifier– Justify method by comparison with ground-truth data– Enhance conflict solver– Classify strong vessel automatically as artery or vein – Integrate method in a framework for vascular structure analysis


GbR ’07GbR ’07


1818

Final slide

Thank you for your attention!

Are there any questions?

department of computer science computer vision & pattern recognition group iapr workshop on...

Documents