Perceptual Organization Perceptual Organization based method in vessel based method in vessel extraction from real retina extraction from real retina imagesimages

Revised on Sept 17,2004 Revised on Sept 17,2004

Frank Tao Frank Tao

Motivation & Objectives: Motivation & Objectives:

• Retina vessel map segmentation is very Retina vessel map segmentation is very important to medical applications, such as important to medical applications, such as diabetic retinopathy, aging related retina analysis diabetic retinopathy, aging related retina analysis etc.etc.

• Available effective solutions will either cause high Available effective solutions will either cause high computational cost or need users intervention computational cost or need users intervention

• Our objectives:Our objectives:– Develop an efficient, accurate automatic solution based Develop an efficient, accurate automatic solution based

on perceptual organization principle: perceptual curve on perceptual organization principle: perceptual curve partition & groupingpartition & grouping

• Edge trace partitionEdge trace partition• Generic edge token groupingGeneric edge token grouping

ReviewReview

• Available researches can be grouped Available researches can be grouped into following classes based on a into following classes based on a review paper:review paper:– Pattern recognitionPattern recognition– Matched filter related methods (MFR)Matched filter related methods (MFR)– Regional growingRegional growing– Vessel trackingVessel tracking– Artificial intelligentArtificial intelligent– Others Others

ReviewReview -continuing-continuing

• All the available systems can also be re-grouped into following All the available systems can also be re-grouped into following classes based on the different features they are trying to search classes based on the different features they are trying to search for: for: – Linear segment structure Linear segment structure

• MFR related methodsMFR related methods• Morphology models: snake, water shadeMorphology models: snake, water shade• Regional growingRegional growing• Some tracking methodsSome tracking methods

– Center line and/or edgeCenter line and/or edge• Zhou matched filter edge trackingZhou matched filter edge tracking• Quebec parallel matching edge trackingQuebec parallel matching edge tracking• Sobel edge detection and trackingSobel edge detection and tracking• OthersOthers

– OthersOthers• Artificial intelligenceArtificial intelligence

– Fuzzy c mean Fuzzy c mean – othersothers

Pro and cons of current systemsPro and cons of current systems

• Line segment structure based : MFR, Pattern recognition, Artificial Line segment structure based : MFR, Pattern recognition, Artificial intelligence etc.intelligence etc.– AdvantagesAdvantages

• Automatic systemAutomatic system• Good noise suppression and vessel segmentationGood noise suppression and vessel segmentation• Continues vessel map including junction structures Continues vessel map including junction structures

– DisadvantagesDisadvantages• High computational cost High computational cost

• Center line and/or edge based : Vessel trackingCenter line and/or edge based : Vessel tracking– AdvantagesAdvantages

• Computational efficiencyComputational efficiency• Good noise suppression and vessel segmentationGood noise suppression and vessel segmentation

– DisadvantagesDisadvantages• Non automaticNon automatic• Non continues map with poor junction detection and breaking of Non continues map with poor junction detection and breaking of

vessel segmentsvessel segments

Our proposed systemOur proposed system

• System design:System design:– Robust vessel feature extraction based on Perceptual Robust vessel feature extraction based on Perceptual

OrganizationOrganization– Effective vessel junction and breaking fixing and Effective vessel junction and breaking fixing and

extracting using limited numbers of guided matched extracting using limited numbers of guided matched filtersfilters

• Targets:Targets:– Fully automaticFully automatic– Very low computational costVery low computational cost– Good noise suppression and vessel segmentationGood noise suppression and vessel segmentation– Continues vessel map including junctions and low Continues vessel map including junctions and low

intensity vessel segmentsintensity vessel segments

Perceptual Organization based Perceptual Organization based method for vessel segment method for vessel segment extractionextraction• GAO’s Curve Partitioning Methods GAO’s Curve Partitioning Methods

– Image processing with edge map obtained from an Edge Image processing with edge map obtained from an Edge Tracker software which detected and extracted all the Tracker software which detected and extracted all the edge traces based on the following rules:edge traces based on the following rules:• Intensity similarityIntensity similarity• Shortest distanceShortest distance• Direction similarity Direction similarity • Noise removal principleNoise removal principle

– LinearityLinearity– lengthlength

– Curve Partitioning and GroupingCurve Partitioning and Grouping

Perceptual Partition & Perceptual Partition & GroupingGroupingGibson’s Observation:Gibson’s Observation:The qualities of a simple line observed by Gibson: The qualities of a simple line observed by Gibson:

(a) “Left slant… Zero slant… Right Slant”(a) “Left slant… Zero slant… Right Slant”

(b) “Convex…straight…concave”(b) “Convex…straight…concave”

Perceptual Partition & Perceptual Partition & GroupingGroupingPsychological experiments of 2-D curve Psychological experiments of 2-D curve

partitioning:partitioning:1) best mark those locations at which distinctive curve 1) best mark those locations at which distinctive curve

segments are “glued” together; segments are “glued” together;

2) best allow the reconstruction of the complete curves; 2) best allow the reconstruction of the complete curves;

3) best allow a viewer to distinguish a given curve from the 3) best allow a viewer to distinguish a given curve from the others. others.

GCS Partition ModelGCS Partition Model

• Analytic descriptors of curves: Analytic descriptors of curves: f (x,a) = 0f (x,a) = 0where where xx denotes an image point, denotes an image point, aa is a vector of is a vector of parameters.parameters.

• A generic curve segment : A generic curve segment : GCS = { x | p GCS = { x | p (x) }(x) }where where x x is an edge point, is an edge point, p (x)p (x) indicates the point indicates the point satisfies the property satisfies the property pp. . This property p can be This property p can be represented by the following function:represented by the following function: p (x) = { f (x), j (y), f’ (x), j’ (y)} p (x) = { f (x), j (y), f’ (x), j’ (y)} Where Where y = f (x)y = f (x) is a curve, is a curve, x = j (y)x = j (y) is its inverse function, is its inverse function, f’ f’

(x)(x) and and j’ (y)j’ (y) are their first derivatives are their first derivatives

GCS Partition ModelGCS Partition Model

GCS GCS f(x)f(x) (y)(y) f’(x)f’(x) ’’(y)(y)

CS1CS1 M+M+ M+M+ M+M+ M-M-

CS2CS2 M-M- M-M- M+M+ M-M-

CS3CS3 M+M+ M+M+ M-M- M+M+

CS4CS4 M-M- M-M- M-M- M+M+

LS1LS1 M-M- M-M- cc cc

LS2LS2 M+M+ M+M+ cc cc

LS3LS3 cc N/AN/A 00

LS4LS4 N/AN/A cc 00

A set of generic curve segments (GCS)

Definition of GCS, M+ is monotonic increase and vice verse

GCS GroupingGCS Grouping

Rule #Rule # DefinitionsDefinitions

G1G1 (CPP1, CS1, CS2)(CPP1, CS1, CS2)

G2G2 (CPP2, CS2, CS3)(CPP2, CS2, CS3)

G3G3 (CPP3, CS3, CS4)(CPP3, CS3, CS4)

G4G4 (CPP4, CS4, CS1)(CPP4, CS4, CS1)

G5G5 (CPP5, CS1, CS3)(CPP5, CS1, CS3)

G6G6 (CPP6, CS2, CS4)(CPP6, CS2, CS4)

G7G7 (CPP7, CS, LS)(CPP7, CS, LS)

G8G8 (CPP8, LSi, LSj)(CPP8, LSi, LSj)

Definition of CPPs and Curve Grouping Rules:

Extra CPPs (dark dots) introduced to increase the sensitivity of junction detection

Retinal Image Based Retinal Image Based KnowledgeKnowledge• Vessel map definition Vessel map definition

– Junctions and EndingsJunctions and Endings• Junctions: Branching junctions (including Y junction and T junction); Junctions: Branching junctions (including Y junction and T junction);

Crossing junctionsCrossing junctions• EndingsEndings

– Vessel SegmentsVessel Segments

• Perceptual Partitioning and Grouping of the edge Perceptual Partitioning and Grouping of the edge trace maptrace map– Original CPP detection: Original CPP detection: Aligned CPP, Junction CPP, Ending CPPAligned CPP, Junction CPP, Ending CPP– Virtual CPP creation through two-sides-parallel-scanningVirtual CPP creation through two-sides-parallel-scanning

• Two Sides Parallel Scanning: stretched out from both side of the detected CPP, using the Two Sides Parallel Scanning: stretched out from both side of the detected CPP, using the gradient of the original pixel to do a parallel scanning, try to find matching pair pixels gradient of the original pixel to do a parallel scanning, try to find matching pair pixels with reverse gradient within a pre-defined vessel widthwith reverse gradient within a pre-defined vessel width

– Associated parallel GCS grouping based on original and virtual CPPsAssociated parallel GCS grouping based on original and virtual CPPs

• How to find out the Vessel segments in the edge How to find out the Vessel segments in the edge trace map?trace map?– Extracting vessel segments through connecting all the directly linked Extracting vessel segments through connecting all the directly linked

associated parallel GCS pairsassociated parallel GCS pairs

Vessel Map DefinitionVessel Map Definition

Original Retina Image Vessel Map definition

CPP and related structureCPP and related structure

Junction CPP and related structure Non-Junction CPP and related structure

CPP detection and Virtual CPP CPP detection and Virtual CPP creationcreation

Original CPP Virtual CPP creation via Two-Side-Parallel-Scanning

Associated parallel GCS Associated parallel GCS grouping and Vessel segment grouping and Vessel segment extractionextractionAssociated parallel

GCS groupingVessel segment

extractionOriginal edge trace

map

Vessel junction, breaking Vessel junction, breaking detection and extraction using detection and extraction using guided matched filtersguided matched filters• Assume vessel segment has:Assume vessel segment has:

– Gaussian shaped gradient profile perpendicular to it’s Gaussian shaped gradient profile perpendicular to it’s length directionlength direction

– Piecewise linear structurePiecewise linear structure– Vessel width very close thus can be treated as same Vessel width very close thus can be treated as same

• Assume the junction, vessel breaking structure: Assume the junction, vessel breaking structure: – Vessel breakings: Sit between any two detected vessel Vessel breakings: Sit between any two detected vessel

segmentssegments– Vessel junctions: intersection, crossing or overlapping of Vessel junctions: intersection, crossing or overlapping of

different vessel branchesdifferent vessel branches• Using the direction information from detected Using the direction information from detected

vessel segments to build up matched filter and vessel segments to build up matched filter and convolving it over the junction and vessel breaking convolving it over the junction and vessel breaking areas to detect then extract junctions, breakingsareas to detect then extract junctions, breakings

System ArchitectureSystem Architecture

Pre-ProcessingExtract Edge

Traces

Vessel Map Extraction

Original CPP detection and GCS partition

Virtual CPP creation via two

sides parallel scanning, GCS

further partition

Associated parallel GCS

pair grouping and Vessel segment

extraction

Junction & breaking detection with guided matched filters

Gaussian Blurring

Noise removal

System ArchitectureSystem Architecture

• Extract edge traces from retina image:Extract edge traces from retina image:– Smooth image by Gaussian blurringSmooth image by Gaussian blurring– Apply the edge tracker to extract edge traces Apply the edge tracker to extract edge traces – Remove short and non-linear noise tracesRemove short and non-linear noise traces

• Vessel map extraction:Vessel map extraction:– Original CPP detection and GCS partitioningOriginal CPP detection and GCS partitioning– Virtual CPP creation through two-side-parallel-scanning Virtual CPP creation through two-side-parallel-scanning

and GCS further partitioningand GCS further partitioning– Associated parallel GCS grouping and vessel segment Associated parallel GCS grouping and vessel segment

extractionextraction– Vessel junction and breaking fixing with limited, guided Vessel junction and breaking fixing with limited, guided

Matched FiltersMatched Filters

System evaluationSystem evaluation

• General performance:General performance:– Automation: Automation:

• No user provided start or ending point needed for our No user provided start or ending point needed for our systemsystem

– Fast: Very efficient system: Fast: Very efficient system: • It only takes 2 seconds (average time) for step 1 and 3 It only takes 2 seconds (average time) for step 1 and 3

seconds for step 2 (average time)seconds for step 2 (average time)

– Accuracy: Accuracy: • Avoid human created noise VS from global MF enhancementAvoid human created noise VS from global MF enhancement

– Continues vessel map structure: Continues vessel map structure: • Junctions and breakings were correctly detected or fixed Junctions and breakings were correctly detected or fixed

Result comparison with A.Hoover’s Result comparison with A.Hoover’s systemsystem

• Two standard sets of manual drawing Two standard sets of manual drawing retina vessel map from two expertsretina vessel map from two experts– A.Hoover (normal one)A.Hoover (normal one)– V. Kouznetsova (rich vessel map)

• By compare with the rich manual drawing By compare with the rich manual drawing vessel map, our system obtained high vessel map, our system obtained high positive rate while the negative rate positive rate while the negative rate remain lower than AH systemremain lower than AH system

• Our system proved to be good at detecting Our system proved to be good at detecting even low intensity vessel mapeven low intensity vessel map

Image 0163- negative rate Image 0163- negative rate (green)(green)

A.Hoover’s standard vessel map

A.Hoover’s standard vessel map

V.K’s standard vessel map

V.K’s standard vessel map

Our System Our System

A.Hoover’s System A.Hoover’s System

Matched Filter Enhance Image

Original Retina Image

Image 0163 Positive rate Image 0163 Positive rate (brown)(brown)

Original Retina Image

Matched Filter Enhance Image

A.Hoover’s standard vessel map

A.Hoover’s standard vessel map

V.K’s standard vessel map

V.K’s standard vessel map

Our System Our System

A.Hoover’s System A.Hoover’s System

Negative Rates Negative Rates

Positive RatePositive Rate

SummarySummary

• Perceptual Curve Partitioning method provides an Perceptual Curve Partitioning method provides an robust way in handling vessel map extractionrobust way in handling vessel map extraction– The proposed system has achieved the following targets:The proposed system has achieved the following targets:

• AutomationAutomation• EfficiencyEfficiency• High Accuracy even for low intensity retina vessel mapHigh Accuracy even for low intensity retina vessel map

• Limitation:Limitation: – For some abnormal retina images, like some strong bright For some abnormal retina images, like some strong bright

patches in the background, this system will receive some false patches in the background, this system will receive some false detected vessel segments. detected vessel segments.

• Future works:Future works:– Further verification method could be applied to minimize the Further verification method could be applied to minimize the

negative detection ratenegative detection rate– Investigate how to combine more domain heuristics of retina Investigate how to combine more domain heuristics of retina

images into the perceptual edge tracking mechanism for images into the perceptual edge tracking mechanism for improving our implementationimproving our implementation

AcknowledgementAcknowledgement

The authors gratefully acknowledge that thisresearch received funding support from both NSERC and Deep Vision Inc. Deep Vision Inc. also provided the authors with their edge tracker software which was used for producing original edge trace data.

References:References:

[1] Ferris FL, "How effective are treatments for diabetic retinopathy?", [1] Ferris FL, "How effective are treatments for diabetic retinopathy?", JAMAJAMA 269, 1993, pp.1290-1291. 269, 1993, pp.1290-1291.[2] L. Pedersen, M. Grunkin, B. Ersbøll, K. Madsen, M. Larsen, N. Christoffersen, U. Skands, “Quantitative measurement of [2] L. Pedersen, M. Grunkin, B. Ersbøll, K. Madsen, M. Larsen, N. Christoffersen, U. Skands, “Quantitative measurement of

changes in retinal vessel diameter in ocular fundus images”, changes in retinal vessel diameter in ocular fundus images”, Patt. Recog. Lett.,Patt. Recog. Lett., 21, 1215-1223, 2000. 21, 1215-1223, 2000.[3] Khoobehi, B., Peyman, G.A., Vo, K.D., “Relationship between Blood Velocity and Retinal Vessel Diameter”, [3] Khoobehi, B., Peyman, G.A., Vo, K.D., “Relationship between Blood Velocity and Retinal Vessel Diameter”, ARVO ARVO

Abstract, Invest. Opthalmol. Vis. SciAbstract, Invest. Opthalmol. Vis. Sci., 33, 4, 804, 1992.., 33, 4, 804, 1992.[4] M. Lalonde, L. Gagnon, M.-C. Boucher, “Non-recursive paired tracking for vessel extraction from retinal images”, [4] M. Lalonde, L. Gagnon, M.-C. Boucher, “Non-recursive paired tracking for vessel extraction from retinal images”,

Proceedings of the Conference Vision InterfaceProceedings of the Conference Vision Interface 2000, 61-68, 2000. 2000, 61-68, 2000.[5] Luo Gang, Opas Chutatape*, and Shankar M. Krishnan,"Detection and Measurement of Retinal Vessels in Fundus [5] Luo Gang, Opas Chutatape*, and Shankar M. Krishnan,"Detection and Measurement of Retinal Vessels in Fundus

Images Using Amplitude Modified Second-Order Gaussian Filter,"Images Using Amplitude Modified Second-Order Gaussian Filter,"IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOLVOL. 49, . 49, NONO. 2, . 2, FEBRUARYFEBRUARY 2002. 2002.

[6][6] QI-GANG GAO and A.K.C. WONG, Curve Detection Based On Perceptual Organization, QI-GANG GAO and A.K.C. WONG, Curve Detection Based On Perceptual Organization, Pattern RecognitionPattern Recognition, , VolVol. 26, . 26, NoNo. 7, . 7, pppp.1039-1046, 1993..1039-1046, 1993.

[7] A. Hoover, V. Kouznetsova, and M. Goldbaum, “Locating blood vessels in retinal images by piecewise threshold probing [7] A. Hoover, V. Kouznetsova, and M. Goldbaum, “Locating blood vessels in retinal images by piecewise threshold probing of a matched filter responseof a matched filter response,” IEEE Trans. Med. Imag., Vol. ,” IEEE Trans. Med. Imag., Vol. 19, 19, NoNo. 3, . 3, pp.pp. 203–210, 2000. 203–210, 2000.

[8] F. Zana and J.-C. Klein. A multimodal registration algorithm of eye fundus images using vessels detection and Hough [8] F. Zana and J.-C. Klein. A multimodal registration algorithm of eye fundus images using vessels detection and Hough transformtransform. IEEE. IEEE Trans. Medical ImagingTrans. Medical Imaging, 18(5):419-428, 1999., 18(5):419-428, 1999.

[9] S. Chaudhuri, S. Chatterjee, N. Katz, and M. Goldbaum, “Detection of blood vessels in retinal images using two-[9] S. Chaudhuri, S. Chatterjee, N. Katz, and M. Goldbaum, “Detection of blood vessels in retinal images using two-dimensional matched filtersdimensional matched filters,” IEEE Trans. Med. Imag., Vol. ,” IEEE Trans. Med. Imag., Vol. 33, pp. , pp. 263–269263–269, Sept, Sept. 1989.. 1989.

[10] O. Chutatape, L. Zheng, and S. M. Krishnan, “Retinal blood vessel detection and tracking by matched Gaussian and [10] O. Chutatape, L. Zheng, and S. M. Krishnan, “Retinal blood vessel detection and tracking by matched Gaussian and Kalman filters,” Kalman filters,” in Proc.20th Annu Conf. IEEE Engineering in Medicine and Biology Societyin Proc.20th Annu Conf. IEEE Engineering in Medicine and Biology Society, 1998, , 1998, pppp. 3144–3149.. 3144–3149.

[11] Can, H. Shen, J. Turner, H. Tanenbaum, and B. Roysam, “Rapid automated tracing and feature extraction from retinal [11] Can, H. Shen, J. Turner, H. Tanenbaum, and B. Roysam, “Rapid automated tracing and feature extraction from retinal fundus images using direct exploratory algorithms,” fundus images using direct exploratory algorithms,” IEEE Trans. Inform. Technol. Biomed.,VolIEEE Trans. Inform. Technol. Biomed.,Vol. 3, . 3, pppp. 125–138, . 125–138, June June 1999.1999.

[12] V. Rakotomalala, L. Macaire, J-G. Postaire, and M. Valette. Identification of retinal vessels by color image analysis[12] V. Rakotomalala, L. Macaire, J-G. Postaire, and M. Valette. Identification of retinal vessels by color image analysis. . Machine Graphics and VisionMachine Graphics and Vision, 7:725-742, 1998., 7:725-742, 1998.

[13] L. Gagnon, M. Lalonde, M. Beaulieu, M.-C. Boucher,Procedure to Detect Anatomical Structures in Optical Fundus [13] L. Gagnon, M. Lalonde, M. Beaulieu, M.-C. Boucher,Procedure to Detect Anatomical Structures in Optical Fundus Images,Images,Proc SPIE VolProc SPIE Vol 4322 Med Imaging:Img Processing4322 Med Imaging:Img Processing 2001 1218-25. 2001 1218-25.

[14][14] D. H. Ballard, “Generalizing the Hough Transform To Detect Arbitrary Shapes”, D. H. Ballard, “Generalizing the Hough Transform To Detect Arbitrary Shapes”, Pattern RecognitionPattern Recognition 13, p111-122, 1981. 13, p111-122, 1981.