masters' whole work(big back-u_pslide)

Computer Assisted Screening of Microcalcifications In Digitized Mammogram For Early Detection of Breast CancerThesis Presentation

Nashid AlamRegistration No: [email protected]

Supervisor: Prof. Dr. Mohammed Jahirul Islam

Department of Computer Science and EngineeringShahjalal University of Science and TechnologyFriday, December 25, 2015

Driving research for better breast cancer treatment “The best protection is early detection”

010

2030

020

40-0.1

0

0.1

5; 0.5; 0.7854

5; 0.5; 0.7854

Introduction

Breast cancer:The most devastating and deadly diseases for women.

o Computer aided detection (CADe) o Computer aided diagnosis (CADx) systems

We will emphasis on :

Background Interest

Background Interest

Interest comes from two primary backgrounds

1. Improvement of pictorial information- - For Human Perception

How can an image/video be made more aesthetically pleasing

How can an image/video be enhanced to facilitate:extraction of useful information

Background Interest

Interest comes from two primary backgrounds

2. Processing of data for:Autonomous machine perception- Machine Vision

Micro-calcification

Mammography

Mammogram

Micro-calcification

Background knowledge

Micro-calcification

Micro-calcifications :- Tiny deposits of calcium- May be benign or malignant- A first cue of cancer.

Position:1. Can be scattered throughout the mammary gland, or 2. Occur in clusters.(diameters from some µm up to approximately 200 µm.)3. Considered regions of high frequency.

Micro-calcification

They are caused by a number of reasons:

1. Aging –The majority of diagnoses are made in women over 50

2. Genetic –Involving the BRCA1 (breast cancer 1, early onset) and

BRCA2 (breast cancer 2, early onset) genes

Micro-calcifications Pattern Determines :The future course of the action-

I. Whether it be further investigatory techniques (as part of the triple assessment), or

II. More regular screening

Mammography


Mammography Machine

Mammography

USE:I. Viewing x-ray imageII. Manipulate X-ray image on a computer screen

Mammography :

Process of using low-energyx-rays to examine the human breast

Used as a diagnostic and a screening tool.

The goal of mammography :The early detection of breast cancer

Mammography Machine

Mammogram


mdb226.jpg

Mammogram

Mammogram:An x-ray picture of the breast

Use:To look for changes that are not normal.

Result Archive:The results are recorded:

1. On x-ray film or 2.Directly into a computer

mdb226.jpg

Literature Review

To detect micro-calcifications in an automatic manner-A number of methods have been proposed

These include:

Global and local thresholding

Statistical approaches

Neural networks

Fuzzy logic

Thresholding of wavelet coefficients and related techniques.

Literature Review

Literature Review

Camilus et al.(2011)[1] propose an efficient method

To identify pectoral mussel using:

Watershed transformation

Merging algorithm to combine catchment basins

MIAS database(84 mammograms)

Literature Review

Literature Review

Pronoj et al.(2011)[2] reviews on :

Thresholding techniquesBoundary based methodHybrid techniquesWatershed transformationEdge detection:

SobelPrewittRobertsLaplacian of GaussianZero-crossCanny

Goal:oTo improve quality of imageoFacilate further processingoRemove noiseoRemove unwanted part from the background

Literature Review

Oliver et al.(2010)[3] worked on:

Local feature extraction from a bank of filters.

Performs training steps:-To automatically learn and select:

The features of microcalcifications.

Literature Review

Goal:oTo obtain different microcalcification morphology

Literature Review

Oliver et al.(2012)[4] :MC Detection based on:

microcalcifications morphologyLocal image features-

Set of feature is trained a pixel-based boosting classifier

Pixel-based boosting classifier:At each round automatically selects the most

salient microcalcifictions features.

Literature Review

Goal:oDetect microcalcification and cluster

Literature Review

Oliver et al. (2012)[4] :

Testing new mammogram:Only salient fractures are computed

Microcalcification clusters are found:By inspecting the local neighborhood of

each microcalcification.

Literature ReviewLiterature Review

Papadopoulus et al. (2008)[5] :

Microcalcification detection using neural network

Preprocessing image enhancement

Got best result by applying:The local range modification algorithmRedundant discrete wavelet linear stretching

and shrinkage algorithm.


Pal et al.(2008)[6] :To detect microcalcification cluster used:

oWeighted density function:-Position of microcalcifications

(take into account)Used:

oMulti-layered perception network for selecting 29 features

Features are used :- To segment mammograms


Razzi et al.(2009)[7] proposed :

A two-stage decomposition wavelet filteringFirst stage:Reduce background noise

Second stage:A hard thresholding technique:

-To identify microcalcificationCluster was considered if more then 3 microcalcifications were

detected in a 1cm2 area

Literature Review

Yu et al.(2010)[8] :

Clustered microcalcification detectionused combined :

-Model-based and statistical texture features

Firstly:Suspicious region containing microcalcification were detected using-

Wavelet filter and two thresholds

Literature Review

Yu et al. 2010 [8] proposed :

Secondly:Textural features were extracted:

-From each suspicious regionFeatures classified by:

-A back propagated neural network

Texture features based on both:oMorkov random fields and oFractal models

Literature Review

Wang et.al.(1989) [9]:

The mammograms are:-Decomposed into different frequency subbands.

The low-frequency subband discarded.

Literature Review

Literature Review

Daubechies I.(1992)[10]:

Wavelets are mainly used :

-Because of their dilation and translation properties-Suitable for non stationary signals.

Strickland et.at (1996)[11] :

Used biorthogonal filter bank-To compute four dyadic and -Two cinterpolation scales.

Applied binary threshold-operator -In six scales.

Literature Review

Heinlein et.al(2003)[12]:Goal: Enhancement of mammograms:

Derived The integrated wavelets:- From a model of microcalcifications

Literature Review

Zhibo et.al.(2007)[13]:A method aimed at minimizing image noise.

Optimize contrast of mammographic image featuresEmphasize mammographic features:

A nonlinear mapping function is applied:-To the set of coefficient from each level.

Use Contourlets:For more accurate detection of microcalcification clusters

The transformed image is denoised-using stein's thresholding [18].

The results presented correspond to the enhancement of regions with large masses only.

Literature Review

Fatemeh et.al.(2007) [14]:

Focus on:-Analysis of large masses instead of microcalcifications.- Detect /Classify mammograms:

Normal and Abnormal

Use Contourlets Transform:For automatic mass classification

Literature Review

Balakumaran et.al.(2010) [15] :

Focus on:- Microcalcification Detection

Use :- Wavelet Transform and Fuzzy Shell Clustering

Literature Review

Literature Review

Zhang et.al.(2013)[16] :

Use Hybrid Image Filtering Method:- Morphological image processing- Wavelet transform technique

Focus on:- Presence of microcalcification clusters

Literature Review

Lu et.al.(2013) [17]:

Use Hybrid Image Filtering Method:- Multiscale regularized reconstruction

Focus on:- Detecting subtle mass lesions in Digital breast

tomosynthesis (DBT)- Noise regularization in DBT reconstruction

Literature Review

Leeuw et.al.(2014) [18]:

Use:- Phase derivative to detect microcalcifications - A template matching algorithm was designed

Focus on:- Detect microcalcifications in breast

specimens using MRI - Noise regularization in image reconstruction

Literature Review

Shankla et.al.(2014)[19] :

Automatic insertion of simulated microcalcification clusters-in a software breast phantom

Focus on:-Algorithm developed as part of a virtual clinical trial (VCT) :

-Includes the simulation of breast anatomy, - Mechanical compression- Image acquisition- Image processing, displaying and interpretation.

Problem Statement

Burdensome Task Of Radiologist : Eye fatigue:

-Huge volume of images-Detection accuracy rate tends to decrease

Non-systematic search patterns of humansPerformance gap between :

Specialized breast imagers andgeneral radiologists

Interpretational Errors:Similar characteristics:

Abnormal and normal microcalcification

Problem Statement

Reason behind the problem( In real life):

The signs of breast cancer are:

Masses CalcificationsTumorLesionLump

Individual Research Areas

Problem Statement

Motivation to the Research

Motivation to the research: Goal

Better Cancer Survival Rates(Facilitate Early Detection ).

Provide “second opinion” : Computerized decisionsupport systems

Fast,Reliable, andCost-effective

QUICKLY AND ACCURATELY :Overcome the development of breast cancer

Challenges

Develop a logistic model:

Feature extraction Challenge:

-To determine the likelihood of CANCEROUS AREA -- From the image values of mammograms

Challenge:Occur in clusters

The clusters may vary in size from 0.05mm to 1mm in diameter.

Variation in signal intensity and contrast.May located in dense tissue

Difficult to detect.

Challenges

Database: mini-MIAS databasehttp://peipa.essex.ac.uk/pix/mias/

Class of Abnormality

Severity of Abnormality

The Location of The

Center of The

Abnormality and It’s

Diameter.

1 Calcification(25)

1.Benign(Calc-12)

2 Circumscribed Masses

3 Speculated Masses

4 Ill-defined Masses

5 Architectural Distortion

2.Malignant(Cancerous)

(Calc-13)

6 Asymmetry

7Normal

mdb223.jpg mdb226.jpg

mdb239.jpg mdb249.jpg

Figure01:X-ray image form mini-MIAS database

Database: Mini-MIAS Databasehttp://peipa.essex.ac.uk/pix/mias/

Mammography Image Analysis Society (MIAS) -An organization of UK research groups

• Consists of 322 images-- Contains left and right breast images for 161 patients

• Every image is 1024 X 1024 pixels in size

• Represents each pixel with an 8-bit word

• Reduced in resolution(Is not good enough for MC to be detectable)

•Very Poor Quality with .jpg compression effects(Original MIAS doesn’t have such artifacts)

Mini-MIAS Database

Mammography Image Analysis Society (MIAS) -An organization of UK research groups

Database: http://peipa.essex.ac.uk/pix/mias/

http://see.xidian.edu.cn/vipsl/database_Mammo.html

Plan of Action

Where Are We? Our Current Research Stage

Thesis SemesterM-3

Chart 01: Gantt Chart of this M.Sc thesis Showing the duration of task against the progression of time

Where Are We? Our Current Research Stage

Thesis SemesterM-3

Schematic representation of the system

Sche

mat

ic r

epre

sent

atio

n of

the

syst

em

Materials and Tools

Matlab 2014

Database: mini-MIAS

Removing Pectoral MuscleAnd

X-ray Label

Image SegmentationGoal: Removing X-ray Labeling And Pectoral muscles

Partitioning a digital image into multiple regions (sets of pixels).

GOAL OF SEGMENTATION:• To locate objects and boundaries (lines, curves, etc.) in

images.

• Result of image segmentation-A set of regions that collectively cover the entire image. (a) -A set of contours extracted from the image. (C)

• Each of the pixels in a region(1, 2, 3) are similar with respect to some characteristic or computed property, such as color, intensity, or texture.

• Adjacent regions(1, 2, 3) are significantly different with respect to the same characteristic(s).

Image Segmentation K-means Clustering

Goal: Removing X-ray Labeling And Pectoral muscles

(intensity <130) (intensity >200)

1

2

3

(a) Segmentation Part

(C) Final Segmented Image

(b)Original image

Why Segmentation?



Proposed framework for breast profile segmentation

Plan of Action:

1. Original Image 2. Segmentation Part3. Final Segmented Image

4. Binary ImageLactiferous Sinus, Ducts, lobule

(After removing pectoral muscles, fatty tissues, Ligaments)


Separating the Pectoral muscle



Keeping the biggest Cluster

(K-means clustering)(mdb256.jpg)

(a)Without Noise (b)With Noise

5.Final Segmented Image

BINARY Thresholding

For Two Different Ranges

1.Morphological Analysis:The Basic Operations are -

I.EROSIONII.DILATION

Using the basic operations we can perform -a)OPENINGb)CLOSING

Advanced Morphological Operation can then be implemented using Combinations Of All Of These

2.Image Smoothing/Filtering(Low pass):-Averaging (Drawback: Can vanish interesting details)

Lactiferous Sinus, Ducts, lobule(After removing pectoral muscles, fatty tissues, Ligaments)

5.Final Segmented Image

(a)Without Noise (b)With Noise

Techniques:

Noise Removing

More On Image Morphology Later

Image Morphology:-Deals with the shape (or morphology)of features in an image-Operate on bi-level images

Structuring Elements, Hits & Fits

B

AC

Structuring Element

Fit: All on pixels in the structuring element cover on pixels in the image

Hit: Any on pixel in the structuring element covers an on pixel in the image

All morphological processing operations are based on these simple ideas

Image Morphology Noise Removing

Structuring elements can be any size and make any shape

However, for simplicity we will use rectangular structuring elements with their origin at the middle pixel

1 1 1

1 1 1

1 1 1

0 0 1 0 00 1 1 1 01 1 1 1 10 1 1 1 00 0 1 0 0

0 1 0

1 1 1

0 1 0



0 0 0 0 0 0 0 0 0 0 0 00 0 0 1 1 0 0 0 0 0 0 00 0 1 1 1 1 1 0 0 0 0 00 1 1 1 1 1 1 1 0 0 0 00 1 1 1 1 1 1 1 0 0 0 00 0 1 1 1 1 1 1 0 0 0 00 0 1 1 1 1 1 1 1 0 0 00 0 1 1 1 1 1 1 1 1 1 00 0 0 0 0 1 1 1 1 1 1 00 0 0 0 0 0 0 0 0 0 0 0

B C

A

1 1 1

1 1 1

1 1 1Structuring Element 1

0 1 0

1 1 1

0 1 0Structuring Element 2



•The structuring element is moved across every pixel in the original image to give a pixel in a new processed image(very like spatial filtering)

•The value of this new pixel depends on the operation performed

•There are two basic morphological operations:

Erosion and Dilation



Erosion of image f by structuring element s is given by f sThe structuring element s is positioned with its origin at (x, y) and the new pixel value is determined using the rule:

Erosion

=otherwise 0

fits if 1),(

fsyxg


A morphological opening of an image is an erosion followed by a dilation

Noise Removing 1. Morphological Analysis

What Is Erosion For?

Erosion can split apart joined objects

Erosion can split apart

%noise removingse = strel('disk',25);for i=1:19erode_bolb = imerode(largest_bolb,se);end

Original imageErosion by

3*3square

structuring element

Erosion by 5*5

square structuring

element


Watch out: Erosion shrinks objects

Erosion Example

Structuring Element

Original Image Processed Image With Eroded Pixels


Erosion Example

Structuring Element

Original Image Processed Image

Dilation

Image Morphology X-ray Label Removing

Dilation of image f by structuring element s is given by f sThe structuring element s is positioned with its origin at (x, y) and the new pixel value is determined using the rule:

⊕

=otherwise 0

hits if 1),(

fsyxg

A morphological closing of an image is a dilation followed by an erosion

bw_image = im2bw(Binary_image);imtool(bw_image)se1 = strel ('line', 3,0);se2 = strel ('line', 3,90);

for i=1:9BW2= imdilate (bw_image, [se1

se2], 'full')BW2 = imfill(BW2,'holes');

end



Structuring Element

Original Image Processed Image

Dilation Example


Dilation Example

Structuring Element

Original Image Processed Image With Dilated Pixels

Dilation ExampleOriginal image

Hole fillingInside the blob(dilation)

Result image(Label Removed)

mdb240.jpg

Binary image

A morphological closing of an image is an dilation followed by a erosion

%hole filling with in the bolbse = strel('disk',39);for i=1:19closeBW_largest_bolb = imclose(largest_bolb,se);

After Removing Some NoiseImage Containing Noise(mdb041.jpg)

Noise Removing 2.Image Smoothing/Filtering(Low pass):

After Removing Some NoiseImage Containing Noise(mdb041.jpg)

Noise Removing

Chosen Technique 2D MEDIAN FILTERING FOR SALT AND PEPPER NOISE

I = medfilt2(I, [1 5]); Median filtering is a nonlinear operation often used in image processing to reduce "salt and pepper" noise. A median filter is more effective than convolution when the goal is to simultaneously reduce noise and preserve edges. Since all the mammograms are in high quality images, there is no need to perform median filtering

Why choosing?

2.Image Smoothing/Filtering(Low pass):

1. Morphological AnalysisOVER

-Does not work will on all the image [I = medfilt2(I, [1 5]);] •No effect most of the time•Absence of salt and peeper noise

-Tendency of loosing interesting details

Class: Benign

Image segmentation K-means Clustering


mdb212 150 200mdb214 150 200mdb218 150 210mdb219 150 210mdb222 150 210mdb223 150 210mdb226 150 210mdb227 150 210mdb236 150 210mdb240 150 210mdb248 150 210mdb252 140 210

(inte

nsity

<15

0) (

inte

nsity

>20

0)

1

2

3

(b) Segmentation Part


(a)Original image

mdb236.jpg

(b)Segmentation Part (c) Final Segmented Image(a)Main Image (e)Image containing Duct, Lobules, Sinus

mdb001.jpg

mdb254.jpg



(d)Binary Image

Achievement: X-Ray Label removedClass: Benign

(b)Segmentation Part (c) Final Segmented Image(a)Main Image (e)Image containing only Pectoral muscle



(d)Binary Image

What we need

mdb212.jpg

Issues 1.Biggest Cluster Does Not Contain BreastProduce Artifacts In Pectoral muscle And Breast Region

What we have

Class: Benign

(b)Segmentation Part (c) Final Segmented Image(a)Main Image(e)Image Containing Only Pectoral muscle

mdb001.jpg



(d)Binary Image

mdb214.jpg

mdb218.jpg

What We Need What We Have


Class: Benign

(b)Segmentation Part (c) Final Segmented Image(a)Main Image(e)Image containing duct, lobules,

sinus & Pectoral muscle

mdb001.jpg



(d)Binary Image

mdb222.jpg

mdb223jpg

mdb226jpg

Wha

t we

wan

t What w

e Have


Class: Benign

(b)Segmentation Part (c) Final Segmented Image(a)Main Image

mdb001.jpg



(d)Binary Image

mdb240.jpg

mdb248.jpg

mdb252.jpg

Wha

t we

wan

t What w

e Have

(e)Image containing duct, lobules, sinus & Pectoral musc


Class: Benign

Class: Malignant

mdb209 140 210mdb211 140 210mdb213 140 210mdb216 140 210mdb231 140 210mdb233 140 210mdb238 140 210mdb239 140 210mdb241 140 210mdb245 140 210mdb249 140 210mdb253 140 210mdb254 140 210mdb256 140 210



(inte

nsity

<14

0) (

inte

nsity

>21

0)

1

23

(b)Segmentation Part


(a)Original imagemdb209.jpg

(a) Original image

mdb241.jpg

(b)Segmentation Part (c) Final Segmented Image(a)Main Image(e)Image containing duct, lobules, sinus



(d)Binary Image

Class: Malignant

Achievement: X-Ray Label removed

mdb238.jpg

mdb245.jpg

Labe

l Rem

oved

(a)Main Image



mdb212.jpg

Issues Level Remain In The ImageProduce Artifacts In Pectoral muscle And Breast Region

Class: Malignant

mdb209.jpg

(b)Segmentation Part

(c) Final Segmented Image

(d)Binary Image

(e)Image Containingonly label



mdb212.jpg

Issues Pectoral muscle Remain In The Image

Produce Artifacts In Breast Region

Class: Malignant

(b)Segmentation Part (c) Final Segmented Image(a)Main Image(e)Image containing only pectoral muscle(d)Binary Image

mdb216.jpg

mdb213.jpg

(b)Segmentation Part (c) Final Segmented Image(a)Main Image(e)Image containing only pectoral muscle(d)Binary Image



Expected output: PECTORAL muscle, DUCT, LOBULES, SINUS,LIGAMENTS

mdb256.jpg

Output ImageMain image



Issues

Missing part Main image Image containing Output Image

Fatty tissue area,ligaments

Duct, Lobules, Sinus

Fatty tissue area,Duct, Lobules, Sinus,

ligaments

X-ray Labels

Fatty tissue area,Duct, Lobules, Sinus,

ligaments

Pectoral muscle

mdb001.jpg

mdb209.jpg

mdb213.jpg

Challenge

Find the binary threshold values.



2. Segmentation Part


(K-means clustering)

1.Need A Non-supervised Method

mdb212 150 200

mdb214 150 200

mdb218 150 210

mdb219 150 210

mdb222 150 210

mdb223 150 210

mdb226 150 210

mdb227 150 210

mdb236 150 210

mdb240 150 210

mdb248 150 210

mdb252 140 210

No pre-defined threshold value



Figure: Internal breast structure

2.Keeping fatty tissues and ligamentsChallenge

mdb001.jpg

mdb212.jpg

mdb223jpg

mdb238.jpg

mdb209.jpg

(a)Main Image (b)Result Image

(a)Main Image (b)Result Image

X-ray Label Removing Finding The Big BLOB

The types of noise :High Intensity Rectangular LabelLow Intensity LabelTape Artifacts

1.Binarizatin of original image.

2.Find the biggest blob.

Plan of Action:

function [outim] = bwlargestblob( im,connectivity)if size(im,3)>1,

error('bwlargestblob accepts only 2 dimensional images');end

[imlabel totalLabels] = bwlabel(im,connectivity);sizeBlob = zeros(1,totalLabels);for i=1:totalLabels,

sizeblob(i) = length(find(imlabel==i));end[maxno largestBlobNo] = max(sizeblob);

outim = zeros(size(im),'uint8');outim(find(imlabel==largestBlobNo)) = 1;

end

img=im2bw(img);(threshold luminance level-=0.5)


1.Binarizatin of original image.

2.Find the biggest blob.

Plan of Action:

(threshold luminance level-=0.5)

Original image Binary Image

mdb219.jpg

(a) Artifacts (Hole) in ROI

(b)Absence of Ligaments and fatty tissue

mdb231.jpgmdb253.jpg

(c) Absence of pectoral muscles

Original image Binary Image

Label successfully removed Issues


Original image Binary Image(threshold luminance level-=0.5)

mdb212.jpg

mdb214.jpg

mdb218.jpg

Original image Binary Image(threshold luminance level-=0.5)

mdb219.jpg

mdb222.jpg

mdb223.jpg

Class: Benign Issue with fatty tissues and ligaments existence


Original image Binary Image(threshold luminance level-=0.5) Original image Binary Image


mdb226.jpg

mdb227.jpg

mdb236.jpg

mdb240.jpg

mdb248.jpg

mdb252.jpg

Class: Benign Issue with fatty tissues and ligaments existence


Original imageBinary Image

(threshold luminance level-=0.5) Original image Binary Image(threshold luminance level-=0.5)

Issue with fatty tissues and ligaments existenceClass: Malignant

mdb209.jpg

mdb211.jpg

mdb213.jpg

mdb216.jpg

mdb231.jpg

mdb233.jpg


Original image Binary Image(threshold luminance level-=0.5) Original image Binary Image


Issue with fatty tissues and ligaments existenceClass: Malignant

mdb238.jpg

mdb239.jpg

mdb241.jpg

mdb245.jpg

mdb249.jpg

mdb253.jpg

mdb256.jpg


Moving towards solution

Issue With Fatty Tissues And Ligaments Existence

X-ray Label Removing

Plan of Action:

1.Binarize the image

2.Fill inside the hole region of the binary image

3.Finding the largest Blob:

4.Keep the Largest Blob and discard other blobs(to remove X-ray level)

function [outim] = bwlargestblob( im,connectivity)if size(im,3)>1,

error('bwlargestblob accepts only 2 dimensional images');end

[imlabel totalLabels] = bwlabel(im,connectivity);sizeBlob = zeros(1,totalLabels);for i=1:totalLabels,

sizeblob(i) = length(find(imlabel==i));end[maxno largestBlobNo] = max(sizeblob);

outim = zeros(size(im),'uint8');outim(find(imlabel==largestBlobNo)) = 1;


Image Morphology

Experimental results:

Goal: Region filling(Region inside the blob)

Original imageFinding biggest blob

(Level removed)Hole filling

Inside the blob(dialation)Result image

(Label Removed)

mdb240.jpg

mdb219.jpg

mdb231.jpg

Binary image

Direct Binarization Without Image enhancement



Original imageResult image

(Label Removed)

mdb240.jpg

Issues

mdb219.jpg

mdb231.jpg

Binary image

1.Does not always produce appealing output

2.Some details are missing(Details around Edge region )

Image MorphologyGoal: Region filling(Region inside the blob)



Original image Result image (Label Removed)

mdb240.jpg

mdb219.jpg

mdb231.jpg

Issues

1.Does not always produce appealing output

2.Some details are missing(Details around Edge region )

mdb212.jpg

mdb214.jpg

mdb219.jpg

mdb226.jpg


Image MorphologyGoal: Region filling(Region inside the blob)



-To find largest blobUse -Otsu’s thresholding technique (graytrash) [20]

-Finding Bi-level the image(im2bw)

To Achieve The Desired Final Result:

-ApplyA Range Of Techniques on original image

[20] Otsu, N., "A Threshold Selection Method from Gray-Level Histograms," IEEE Transactions on Systems, Man, and Cybernetics, Vol. 9, No. 1, 1979, pp. 62-66.



1. Histogram equalization of the original X-ray image2. Adjust image contrast3. Apply Otsu's Thresholding Method [20] and

find bi-level the image which has several blobs in it. 4. Finding the Largest blob (Bwlargest.bolb)5. Hole filling within the blob region6. Keep the true pixel value covering only the area of largest

blob and discard other features from the original image7. X-ray label is successfully removed

Plan of Action


1.Original image

2.HistogramEqualization

3.Contrast Image

4.Binary Image

mdb239.jpg

Combining Range of techniques

J = histeq(I); %histogram equalization

contrast_image = imadjust(J, stretchlim(J), [0 1]); %high contrast image

%Apply Thresholding to the Image level = graythresh(contrast_image);

%GRAYTHRESH Global image threshold using %Otsu's methodbw_image = im2bw(contrast_image, level);%getting binary image


5.Finding biggest blob

6.Hole fillingInside the blob

7.Result image(Label Removed)

Combining Range of techniquesX-ray Label Removing

Result image(Label Removed)

Original image

Compare the original and final image


Experimental results


X-ray Label Removing1.Original image

2.HistogramEqualization 3.Contrast Image 4.Binary Image 5.Finding biggest blob



mdb212.jpg

mdb214.jpg

mdb214.jpg

mdb218.jpg

mdb219.jpg

Benign

X-ray Label Removing Benign1.Original image




mdb222.jpg

mdb223.jpg

mdb226jpg

mdb227jpg

X-ray Label Removing Benign1.Original image




mdb226.jpg

mdb240.jpg

mdb248.jpg

mdb252.jpg

X-ray Label Removing Malignant1.Original image




mdb209.jpg

mdb211.jpg

mdb213.jpg

mdb216.jpg

mdb231.jpg





mdb233.jpg

mdb238.jpg

mdb239.jpg

mdb241.jpg





mdb245.jpg

mdb249.jpg

mdb253.jpg

mdb256.jpg

Successful


Finally!

Removing pectoral muscleKeeping fatty tissues and ligaments

mdb212.jpg(a)Main Image (b)Result Image

mdb213.jpg(a)Main Image (b)Pectoral Muscle

mdb214.jpg

Main Image

Result Image

o Fat t y t i s s ue are ao Duc to Lobul e so Si nuso l i gam e nt s

Extraction of ROIRemoving pectoral muscle

Why removing pectoral muscle?

o Pectoral muscle will never contain micro-calcification

o Less Computational Time And Cost-Operation on small image area

Existence of micro-calcification:

ROI

Edge Detection of pectoral muscleRemoving pectoral muscle

Possible Approach To Edge-detection:

1.Scanning pixel value intensity at each points2.find out the sudden big intensity change at the edge location3.Mark the pixels at edge location4.Estimate a straight line depending on the marked edge points

Approach-01:

Problem faced in Approach-01:-Finding appropriate Thresholding value is an unsupervised method,

which will work on every image -The threshold value must be found in an unsupervised manner-Any predefined threshold value will not produce desired output for all image

mdb212 150 200mdb214 130 205mdb218 150 210mdb219 120 200mdb222 150 210mdb223 150 225mdb226 110 210mdb227 150 230mdb236 160 210mdb240 150 200mdb248 150 210mdb252 140 210


Possible Approach To Edge-detection:

1.Segment the image

2.Separate the pectoral muscle form the Duct, Lobules, Sinus region Making all the pixels black(zero)resides in the fatty tissue and ligament area

3.Find the binary image of image found in step 2(it will be used as outer image)

4.Erode the image found in step-3 (it will be used as inner image)

5.Subsract the inner image from the outer image to get the edge

Approach-02:

Visualization in next slide


1.Original imagemdb212.jpg

2.Segmentation Part

3.Fatty tissue& Ligament removed

Possible Approach To Edge-detection(Approach-02):


4.Binary Version(outer)

5.Binary Version(inner)

6.Edge(outer-inner)



mdb212.jpg

mdb214.jpg

mdb218.jpg

mdb252.jpg

1.Original image 2.Segmentation Part 4.Binary Version(outer)

3.Fatty tissue& Ligament removed 5.Binary Version(inner) 6.Edge(outer-inner)


mdb223.jpg

mdb226.jpg

mdb240.jpg

mdb248.jpg

1.Original image 2.Segmentation Part3.Fatty tissue

& Ligament removed4.Binary Version(outer)5.Binary Version(inner) 6.Edge(outer-inner)


1.Pectoral muscle and ligaments in fatty tissue area got merged

mdb218.jpg

mdb240.jpg



Problems faced in (Approach-02):


2.Discontinuity in Pectoral muscle edge

mdb252.jpg

mdb226.jpg

mdb252.jpg

mdb248.jpg






3.Same thresholding value(i.e.,130-210,) does not work well on all the images and Produce improper output(complete black image as output)


& Ligament removed4.Binary Version(outer)

5.Binary Version(inner)6.Edge(outer-inner)


Points to be noted from approach-2:

-Pectoral muscle a Triangular areamdb212.jpg

mdb214.jpg

Based on this point: Moving on to approach -03

mdb209.jpg

(2)Binary Image(1)Original Image

Triangle Detection of pectoral muscleRemoving pectoral muscle

1.Fing the triangular area of the pectoral muscle region

I. Finding white seeding pointII. Finding the 1st black point of 1st row after getting a white seeding pointIII. Draw a horizontal line in these two points.IV. finding the 1st black point of 1st column after getting a white seeding pointV. Draw a vertical line and angular line.

2.Making all the pixels black(zero)resides in the pectoral muscle area

Approach-03(Triangle Detection of pectoral muscle):

Visualization in next slide



mdb212.jpg1.Original image

2.Contrast stretching

3.Binary of contrast image

stratching_in_range=uint8(imadjust(I,[0.01 0.7],[1 0]));

BW=~stratching_in_range;



4.Triangle

5.Triangle Filled

6.muscle removed


Removing pectoral muscleApproach-03(Triangle Detection of pectoral muscle):

Triangle Detection of pectoral muscle


mdb212.jpg

mdb214.jpg

1.Original image 2.Contrast stretching 3.Binary of contrast image 4.Triangle

mdb240.jpg

mdb248.jpg

5.Triangle Filled 6.muscle removed

Class: Benign


mdb222.jpg

mdb226.jpg

mdb227.jpg

2.Contrast stretching1.Original image 3.Binary of contrast image 4.Triangle



The triangle does not always indicates the proper pectoral muscle area.Reason: Discontinuity in edges (First 3 or 4 rows and columns)

it is caused by artifacts in mammogram

Class: Benign



mdb218.jpg


mdb219.jpg


mdb218.jpg


Class: Benign




mdb222.jpg

mdb219.jpg



Class: Benign

mdb223.jpg

2.Contrast stretching1.Original image 3.Binary of contrast image 4.Triangle 5.Triangle Filled 6.muscle removed



Defects in mammogram

Class: Benign

2.Contrast stretching1.Original image

3.Binary of contrast image 4.Triangle




Defects in mammogram

mdb227.jpg

Class: Benign


1.Original image 2.Contrast stretching 3.Binary of contrast image 4.Triangle 5.Triangle Filled 6.muscle removed

Class: Malignant

mdb241jpg

Mdb249.jpg

mdb211.jpg

Problems faced in (Approach-03): 2.Discontinuity in edge lines causes false output


1.Original image

2.Contrast stretching

3.Binary of contrast image

4.Triangle

5.Triangle Filled

6.muscle removed

Class: Malignant

mdb256.pg


mdb212.jpg

mdb214.jpg


mdb240.jpg

mdb248.jpg


Class: Benign

Successful

Pectoral Muscle Removing

Finally!

Improved Computer Assisted Screening

Enhancement of digitized mammogram

Goal

MAIN NOVELTY

Input image

BandpassDirectionalsubbands


Main Novelty

-Contourlet Transform

- Specific Edge Filter (Prewitt Filter):To enhance the directional structures of the image in

the contourlet domain.

- Recover an approximation of the mammogram (with the microcalcifications enhanced):

Inverse contourlet transform is applied

Details in upcoming slides

Based on the classical approach used in transform methods for image processing.

1. Input mammogram

2. Forward CT

3. Subband Processing

4. Inverse CT

5. Enhanced Mammogram

Schematic representation of the system

Contourlet transformation

Implementation Based On :

• A Laplacian Pyramid decomposition followed by -

• Directional filter banks applied on each band pass sub-band.

The Result Extracts:-Geometric information of images.


Main Novelty

Enhancement of the Directional Subbands

The Contourlet TransformLaplacian Pyramid: 3 level

DecompositionFrequency partitioning of a directional filter bank

Decomposition level l=3

The real wedge-shape frequency band is 23=8.

horizontal directions are corresponded by sub-bands 0-3

Vertical directions are represented by sub-bands 4-7




Decomposition

Laplacian Pyramid Level-1



8 Direction

4 Direction

4 Direction

(mdb252.jpg)



Decomposition

Wedge-shape frequency band is 23=8.

Horizontal directions are corresponded by sub-bands 0-3

(1) sub-band 0

(2) sub-band 1

(3) sub-band 2(4) sub-band 3

Contourlet coefficient at level 4



DecompositionContourlet coefficient at level 4

Wedge-shape frequency band is 23=8.


(5) sub-band 4

(6) sub-band 5

(7) sub-band 6

(8) sub-band 7



Decomposition

(a) Main Image(mdb252.jpg)

(b) Enhanced Image(Average in all 8 direction)

(a) Main image(Toy Image)

Contourlet Transform Example

(b) Horizontal Direction

(c) Vertical Direction

Directional filter banks: Horizontal and Vertical

Contourlet Transform ExampleDirectional filter banks

Horizontal directions are corresponded by sub-bands 0-3

(1) sub-band 0

(2) sub-band 1

(3) sub-band 2

(4) sub-band 3

Contourlet Transform ExampleDirectional filter banks


(5) sub-band 4

(6) sub-band 5

(7) sub-band 6

(8) sub-band 7

Why Contourlet?

Why Contourlet?

•Decompose the mammographic image:-Into directional components:

To easily capture the geometry of the image features.


Target


• This decomposition offers:

-Multiscale localization(Laplacian Pyramid) and -A high degree of directionality and anisotropy.

Why Contourlet? Usefulness of Contourlet

Directionality:Having basis elements Defined in variety of directions

Anisotrophy:Basis Elements having Different aspect ration

Contourlet Transform Concept

(a)Wavelet(Require a lot of dot for fine resolution)

(b)Contourlet(Requires few different elongated shapes

in a variety of direction following the counter)

3 Different Size of Square Shape brush stroke(Smallest, Medium, Largest) to provide Multiresolution Image

Example: Painter Scenario

Why Contourlet?

2-D Contourlet Transform (2D-CT) Discrete WT

Handles singularities such as edges in a more powerful way

Has basis functions at many orientations has basis functions at three orientations

Basis functions appear a several aspectratios

the aspect ratio of DWT is 1

CT similar as DWT can beimplemented using iterative filter banks.

Advantage of using 2D-CT over DWT:


Input image



Plan-of-Action

For microcalcifications enhancement :

We use-The Contourlet Transform(CT) [12]

The Prewitt Filter.

12. Da Cunha A. L., Zhou J. and Do M. N,: The Nonsubsampled Contourlet Transform: Theory, Design, and Applications, IEEE Transactions on Image Processing,vol. 15, (2006) pp. 3089-3101

Art-of-Action

An edge Prewitt filter to enhance the directional structures

in the image.

Contourlet transform allows decomposing the image in

multidirectional and multiscale subbands[21].

21. Laine A.F., Schuler S., Fan J., Huda W.: Mammographic feature enhancement by multiscale analysis, IEEE Transactions on Medical Imaging, 1994, vol. 13, no. 4,(1994) pp. 7250-7260

This allows finding • A better set of edges,• Recovering an enhanced mammogram with better visual characteristics.

Microcalcifications have a very small size a denoising stage is not implemented

in order to preserve the integrity of the injuries.

Decompose the digital mammogram

Using Contourlet transform

(b) Enhanced image(mdb238.jpg)

(a) Original image (mdb238.jpg)

Method

CT is implemented in two stages:

1. Subband decomposition stage

2. Directional decomposition stages.


Method


For the subband decomposition:- The Laplacian pyramid is used [22]

Decomposition at each step:-Generates a sampled low pass version of the original-The difference between :

The original image and the prediction.

22. Park S.-I., Smith M. J. T., and Mersereau R. M.: A new directional Filter bank for image analysis andclassification, Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '99), vol.3, (1999) pp. 1417-1420

Details ……..

Method


Details ……..

1. The input image is first low pass filtered

2. Filtered image is then decimated to get a coarse(rough) approximation.

3. The resulting image is interpolated and passed through Synthesis filter.

4. The obtained image is subtracted from the original image :To get a bandpass image.

5. The process is then iterated on the coarser version (high resolution)of the image.

Plan of Action

Method

2.Directional Filter Bank (DFB)

Details ……..

Implemented by using an L-level binary tree decomposition :resulting in 2L subbands

The desired frequency partitioning is obtained by :Following a tree expanding rule

- For finer directional subbands [22].

22. Park S.-I., Smith M. J. T., and Mersereau R. M.: A new directional Filter bank for image analysis andclassification, Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '99), vol.3, (1999) pp. 1417-1420

The Contourlet Transform

The CT is implemented by:Laplacian pyramid followed by directional filter banks (Fig-01)

Input image



Figure 01: Structure of the Laplacian pyramid together with the directional filter bank

The concept of wavelet:University of Heidelburg

The CASCADE STRUCTURE allows:- The multiscale and

directional decomposition to be independent

- Makes possible to:Decompose each scale into

any arbitrary power of two's number of directions(4,8,16…)

Figure 01

Details ………….

Decomposes The Image Into Several Directional Subbands And Multiple Scales

Figure 02: (a)Structure of the Laplacian pyramid together with the directional filter bank(b) frequency partitioning by the contourlet transform(c) Decomposition levels and directions.

(a) (b)

Input image



Details….

(c)

DenoteEach subband by yi,j

Wherei =decomposition level and J=direction

The Contourlet TransformDecomposes The Image Into Several Directional Subbands And Multiple Scales

The processing of an image consists on:-Applying a function to enhance the regions of

interest.

In multiscale analysis:

Calculating function f for each subband :-To emphasize the features of interest-In order to get a new set y' of enhanced subbands:

Each of the resulting enhanced subbands can be expressed using equation 1.

)(', , jiyfjiy = ………………..(1)

-After the enhanced subbands are obtained, the inverse transform is performed to obtain an enhanced image.



Denote

Each subband by yi,jWherei =decomposition level and J=direction Details….



Details….

The directional subbands are enhanced using equation 2.

=)( , jiyf)2,1(

,1 nnWjiy

)2,1(,2 nnWjiy

If bi,j(n1,n2)=0

If bi,j(n1,n2)=1………..(2)

Denote

Each subband by yi,jWherei =decomposition level and J=direction

W1= weight factors for detecting the surrounding tissueW2= weight factors for detecting microcalcifications

(n1,n2) are the spatial coordinates.

bi;j = a binary image containing the edges of the subband

Weight and threshold selection techniques are presented on upcoming slides



The directional subbands are enhanced using equation 2.

=)( , jiyf)2,1(

,1 nnWjiy

)2,1(,2 nnWjiy

If bi,j(n1,n2)=0

If bi,j(n1,n2)=1………..(2)

Binary edge image bi,j is obtained :-by applying an operator (prewitt edge detector)

-to detect edges on each directional subband.

In order to obtain a binary image:A threshold Ti,j for each subband is calculated.

Details….

Weight and threshold selection techniques are presented on upcoming slides

Threshold Selection


Details….

The microcalcifications appear :

On each subband Over a very

homogeneous background.

Most of the transform coefficients:

-The coefficients corresponding to theinjuries are far from background value.

A conservative threshold of 3σi;j is selected:where σi;j is the standard deviation of the corresponding subband y I,j .

Weight Selection


Exhaustive tests:-Consist on evaluating subjectively a set of 322 different mammograms

-With Different combinations of values,

The weights W1, and W2 are determined:-Selected as W1 = 3 σi;j and W2 = 4 σi;j

These weights are chosen to:keep the relationship W1 < W2:

-Because the W factor is a gain -More gain at the edges are wanted.

Experimental Results

Applying Contourlet Transformation Benign

Original image Enhanced image

Goal: Microcalcification Enhancement

mdb222.jpg

mdb223.jpg


mdb248.jpg

mdb252.jpg



mdb226.jpg

mdb227.jpg


mdb236.jpg

mdb240.jpg



Original image Enhanced image Original image Enhanced image

mdb218.jpgmdb219.jpg


Applying Contourlet Transformation MalignantGoal: Microcalcification Enhancement


mdb209.jpg

mdb211.jpg


mdb213.jpg

mdb231.jpg



mdb238.jpg

mdb239.jpg


mdb241.jpg

mdb249.jpg


mdb253.jpg



mdb256.jpg

Applying Contourlet Transformation NormalGoal: Microcalcification Enhancement


mdb003.jpg

mdb004.jpg


mdb006.jpg

mdb007.jpg



mdb009.jpg

mdb018.jpg


mdb027.jpg

mdb033.jpg



mdb046.jpg

mdb056.jpg


mdb060.jpg

mdb066.jpg



mdb070.jpg

mdb073.jpg


mdb074.jpg

mdb076.jpg



mdb093.jpg

mdb096.jpg


mdb101.jpg

mdb012.jpg



mdb128.jpg

mdb137.jpg


mdb146.jpg

mdb154.jpg



mdb166.jpg

mdb169.jpg


mdb224.jpg

mdb225.jpg



mdb263.jpg

mdb294.jpg


mdb316.jpg

mdb320.jpg

Wavelet Transformation

Use Separable Transform

2D Wavelet Transform

Visualization

Label ofapproximation

HorizontalDetails

HorizontalDetails

VerticalDetails

DiagonalDetails

VerticalDetails

DiagonalDetails



Decomposition at Label 4

Original image(with diagonal details areas indicated)

Diagonal Details



Vertical Details

Decomposition at Label 4

Original image(with Vertical details areas indicated)


DWT

1.Original Image(Malignent_mdb238) 2.Decomposition at Label 4

2.Decomposition at Label 1 3.Decomposition at Label 2 3.Decomposition at Label 3


DWT

1.Original Image(Malignent_mdb238) 2.Decomposition at Label 4


1.Original Image(Benign_mdb252) 2.Decomposition at Label 4


DWT


1.Original Image(Malignent_mdb253.jpg) 2.Decomposition at Label 4


Metrics: Quantitive Measurement

Metrics

To compare the ability of :Enhancement achieved by the proposed method

Why?

1. Measurement of distributed separation (MDS)2. Contrast enhancement of background against target (CEBT) and3. Entropy-based contrast enhancement of background against target (ECEBT) [23].

Measures used to compare:

23. Sameer S. and Keit B.: An Evaluation on Contrast Enhancement Techniques for Mammographic Breast Masses, IEEETransactions on Information Technology in Biomedicine, vol. 9, (2005) pp. 109-119

Metrics

1. Measurement of Distributed Separation (MDS)

Measures used to compare:

The MDS represents :How separated are the distributions of each mammogram

…………………………(3)MDS = |µucalcE -µtissueE |- |µucalc0 -µtissue0 |

µucalcE = Mean of the microcalcification region of the enhanced imageµucalc0 = Mean of the microcalcification region of the original image

µtissueE = Mean of the surrounding tissue of the enhanced imageµtissue0 = Mean of the surrounding tissue of the enhanced image

Defined by:

Where:

Metrics

2. Contrast enhancement of background against target (CEBT) Measures used to compare:

The CEBT Quantifies :The improvement in difference between the background and the target(MC).

…………………………(4)

0µucalcEµucalc

0µtissue0µucalc

EµtissueEµucalc

CEBT

σσ

−=

Defined by:

Where:

Eµucalcσ

0µucalcσ

= Standard deviations of the microcalcifications region in the enhanced image

= Standard deviations of the microcalcifications region in the original image

Metrics

3. Entropy-based contrast enhancement of background against target (ECEBT)Measures used to compare:

The ECEBT Measures :- An extension of the TBC metric- Based on the entropy of the regions rather

than in the standard deviations

Defined by:

Where:

…………………………(5)

0µucalcEµucalc

0µtissue0µucalc

EµtissueEµucalc

ECEBT

εζ

−=

= Entropy of the microcalcifications region in the enhanced image

= Entropy of the microcalcifications region in the original image

Eµucalcζ

0µucalcε

MDS, CEBT and ECEBT metrics on the enhanced mammograms


CT Method DWT Method

MDS CEBT ECEBT MDS CEBT ECEBT0.853 0.477 0.852 0.153 0.078 0.555

0.818 0.330 0.810 0.094 0.052 0.382

1.000 1.000 1.000 0.210 0.092 0.512

0.905 0.322 0.920 1.000 0.077 1.000

0.936 0.380 0.935 0.038 0.074 0.473

0.948 0.293 0.947 0.469 0.075 0.847

0.665 0.410 0.639 0.369 0.082 0.823

0.740 0.352 0.730 0.340 0.074 0.726

0.944 0.469 0.494 0.479 0.095 0.834

0.931 0.691 0.936 0.479 0.000 0.000

0.693 0.500 0.718 0.258 0.081 0.682

0.916 0.395 0.914 0.796 0.079 0.900

Table 1. Decomposition levels and directions.

0

0.2

0.4

0.6

0.8

1

1.2

TBC

Mammogram

MDS Matrix

CT DWT

The proposed method gives higher results than the wavelet-based method.


Experimental Results Analysis

0

0.2

0.4

0.6

0.8

1

1.2

TBCE

Mammogram

CEBT Matrix

CT DWT




0

0.2

0.4

0.6

0.8

1

1.2

DSM

Mammogram

ECEBT Matrix

CT DWT




Experimental Results AnalysisMesh plot of a ROI containing microcalcifications

(a)The original mammogram

(mdb252.bmp)

(b) The enhanced mammogram

using CT



(mdb238.bmp)


using CT



(mdb253.bmp)


using CT

More peaks corresponding to microcalcifications are enhanced

The background has a less magnitude with respect to the peaks:-The microcalcifications are more visible.

Observation:



(a)Original image (b)CT method (c)The DWT Method

These regions contain :• Clusters of microcalcifications (target)• surrounding tissue (background).

For visualization purposes :The ROI in the original mammogram are marked with a square.

ACHIEVEMENT

Improved Computer Assisted screen of mammogram

Achievements!

Enhancement of MC in digitized mammogramfor diagnostic support system

Figure: Diagnostic support system

MC

Suspected

Digital mammography systems :Presents images to the Radiologist with properly image processing applied.

Achievements!

(b) Enhanced image(mdb238.jpg)

(a) Original imageROI

(mdb238.jpg)

(a) Original imageWHOLE IMAGE (mdb238.jpg)

Digital mammography systems :Presents images to the Radiologist with properly image processing applied.

Hard to find MC Easy to find MC

Whilephysicians

interact with

The information in an image During interpretation process

Achievements!!

Enhancement of MC in digitized mammogram

With improved visual understanding, we can develop :

ways to further improve :o Decision making ando Provide better patient care

Improved Computer Assisted Screening

Goal Accomplished

Another Step Ahead..how about training a machine?

Dealing with Features

Why Feature Extraction?

Finding a feature:That has the most

discriminative information

The objective of feature selection:

Differs from its immediate surroundings by texture colorintensity

Fig: MC features (Extracted Using Human Visual Perception)


Finding a feature:That has the most

discriminative information

The objective of feature selection:

Differs from its immediate surroundings by texture colorintensity

Fig: MC (Irregular in shape and size)(Extracted Using Human Visual Perception)

MoreFeatures: Shape Size


Problems With MC Features:Irregular in shape and sizeNo definite patternLow Contrast -

Located in dense tissueHardly any color intensity variation

MC Feature

Fig: MC (Irregular in shape and size)(Extracted Using Human Visual Perception)

Why Feature Extraction? MC Feature

How radiologist deals with feature Detection/Recognition issue ?

Using Human Visual Perception


How Radiologist (Using Human Eye) deals with feature detection/Recognition issue ?

Using Human Visual Perception

Humans are equipped with sense organs e.g. eye-Eye receives sensory inputs and -Transmits sensory information to the brain

http://www.simplypsychology.org/perception-theories.html


Teach the machine to see like just we doObjective:

Irregular in shape and sizeNo definite patternLow Contrast -

Located in dense tissueHardly any color intensity variation

Machine Vision Challenges:-To make sense of what it sees

In Real:MC is Extracted Using Human Visual Perception

SURF Point Algorithm

Speeded-Up Robust Features (SURF) Algorithm

Point feature algorithm (SURF)Approach:

Improving the prediction performance of CAD Providing a faster, reliable and cost-effective prediction

Features will facilitate:

Fig: MC Point features (Extracted Using SURF point feature algorithm)

Point feature algorithm (SURF)Approach:

SURF point algorithm

Detect a specific object

Speeded-Up Robust Features (SURF) algorithm to find blob features.

Objective

based on Finding point correspondences

between . The reference and the target image

Reference Image Target Image

Feature Extraction

Context in using the features:I. Finding Key pointsII. Matching key pointsIII. Classification

Strongest feature point(Reference Image) Strongest feature point

(Target Image)

SURF point algorithmSpeeded-Up Robust Features (SURF) algorithm to find blob features.

Feature Extraction

Strongest feature point(Reference Image)

Strongest feature point(Target Image)

SURF point algorithmSpeeded-Up Robust Features (SURF) algorithm to find blob features.

Code Fragment (Detect and visualize feature points.)

%Detect feature points in the reference imageelephantPoints = detectSURFFeatures(elephantImage);

%Detect feature points in the target imagescenePoints = detectSURFFeatures(sceneImage);

% visualize feature points in the reference image.figure;imshow(elephantImage);hold on;plot(selectStrongest(elephantPoints, 100));title('100 Strongest Feature Points from Elephant Image');

% Extract Feature Points% Extract feature descriptors at the interest points in both images.

[elephantFeatures, elephantPoints] = extractFeatures(elephantImage, elephantPoints);[sceneFeatures, scenePoints] = extractFeatures(sceneImage, scenePoints);

Context in using the features:

Feature ExtractionSURF point algorithmSpeeded-Up Robust Features (SURF) algorithm to find blob features.

I. Finding Key pointsII. Matching key pointsIII. Classification

Fig. Putatively Matched Points (Including Outliers )


elephantPairs = matchFeatures(elephantFeatures, sceneFeatures, 'MaxRatio', 0.9);

% Display putatively matched features.matchedElephantPoints = elephantPoints(elephantPairs(:, 1), :);matchedScenePoints = scenePoints(elephantPairs(:, 2), :);figure;showMatchedFeatures(elephantImage, sceneImage, matchedElephantPoints, ...

matchedScenePoints, 'montage');title('Putatively Matched Points (Including Outliers)');extractFeatures(sceneImage, scenePoints);

Code Fragment (Find Putative Point Matches)




Estimate Geometric Transformation and Eliminate Outliers


% Estimate Geometric Transformation and Eliminate Outliers% estimateGeometricTransform calculates the transformation relating the matched points, % while eliminating outliers. This transformation allows us to localize the object in the scene[tform, inlierElephantPoints, inlierScenePoints] = ...

estimateGeometricTransform(matchedElephantPoints, matchedScenePoints, 'affine');figure;% Display the matching point pairs with the outliers removedshowMatchedFeatures(elephantImage, sceneImage, inlierElephantPoints, ...

inlierScenePoints, 'montage');title('Matched Points (Inliers Only)');

% Get the bounding polygon of the reference image.elephantPolygon = [1, 1;... % top-left

size(elephantImage, 2), 1;... % top-rightsize(elephantImage, 2), size(elephantImage, 1);... % bottom-right1, size(elephantImage, 1);... % bottom-left1,1]; % top-left again to close the polygon

newElephantPolygon = transformPointsForward(tform, elephantPolygon);

figure;imshow(sceneImage);hold on;line(newElephantPolygon(:, 1), newElephantPolygon(:, 2), 'Color', 'g');title('Detected Elephant');

CodeFragment

Moving Towards MC Feature DetectionUsing

SURF Point Algorithm

Local feature

Details In Next slide

To keep in mind

Local Feature Detection and Extraction

Local features :

A pattern or structure :Point, edge, or small image patch.

- A pattern or structure found in an image,

Differs from its immediate surroundings bytexture colorintensity

- Associated with an image patch that:

Fig.1 : Some Image Patch We used for Feature Point Detection Purpose


Applications: Image registration Object detection and classification TrackingMotion estimation

Using local features facilitates: handle scale changes rotation occlusion

Detectors /Methods :• FAST• Harris• Shi & Tomasi• MSER

• SURF

Feature Descriptors:SURFFREAKBRISKHOG descriptors

Detecting corner features

detecting blob/point features.


Detector Feature Type Scale IndependentFAST [24] Corner No

Minimum eigen value algorithm[25]

Corner No

Corner detector [26] Corner NoSURF [27] Blob/ Point YesBRISK [28] Corner YesMSER [29] Region with uniform

intensityYes


Why Using SURF Feature?Trying to identify MC cluster Blob


detectSURFFeatures(boxImage);

selectStrongest(boxPoints, 100)

extractFeatures(boxImage, boxPoints)

matchFeatures(boxFeatures, sceneFeatures);


Read the reference image containing the object of interest

Read the target image containing a cluttered scene.

Detect feature points in both images.

Select the strongest feature points found in the reference image.

Select the strongest feature points found in the target image.

Extract feature descriptors at the interest points in both images.

Find Putative Point Matches using their descriptors

Display putatively matched features.

Locate the Object in the Scene Using Putative Matches

Start

End

SURF Point Detection

1.Read the reference image

containing MC cluster

2.Target image containing MC.

2.Strongest feature point

in MC cluster

2. Strongest Feature point in Target Image

3. No match point Found


Are we getting less feature points?

Figure: No match point Found

No. of SURF feature points: 2 No. of SURF feature points: 47

Image Size256*256

Image Size 549*623

Image mdb238.jpg

More features from the image extracted(most points are mismatched)

To extract relevant feature point from the image

Case 1: Consider Big Reference Image

To get more feature points


Case 2: Consider A bigger Reference Image andWhole mammogram as Target Image

1. Image of MC Cluster(mdb238.jpg) (256*256)

2. Main mammogram (mdb238.jpg) 1024*1024

3. 100 strongest point of ROI) (256*256) 4. 300 strongest point of Main mammogram (mdb238.jpg) 1024*1024

To get more feature points

What we finally have? No putative match Point


Case 2: Consider A bigger Reference Image andWhole mammogram as Target Image To get more feature points

1. Image of an Microcalcification Cluster

Too small ROI will cause less feature points to match

2. 23 strongest pointsAmong 100 Strongest Feature Points

from reference image

Reference image: mdb248.jpgImage size: 256 *256

detectSURFFeatures(mc_cluster);

Problem 1: less number of feature points to matchSURF Feature Point

4. Only 1 strongest pointsAmong 300 Strongest Feature Points

from Scene Image

Too small ROI will cause less feature points to match

3. Image of a Cluttered Scene

Scene image: mdb248.jpgImage size: 427*588

detectSURFFeatures(sceneImage)


Result of small ROI (256*256):No Putative Point Matches

[mcFeatures, mc_Points] = extractFeatures(mc_cluster, mc_Points);[sceneFeatures, scenePoints] = extractFeatures(sceneImage, scenePoints);mcPairs = matchFeatures(mcFeatures, sceneFeatures);matchedmcPoints = mc_Points(mcPairs(:, 1), :);matchedScenePoints = scenePoints(mcPairs(:, 2), :);showMatchedFeatures(mc_cluster, sceneImage, matchedmcPoints, ... matchedScenePoints, 'montage');


Image Image Size Number of feature points

1190*589 15

588*427 23

256*256 1

541*520 86

Varying image size to see the effect to get SURF feature points

Approach-01 to solve: Considering the Whole image(Label and Pectoral Muscle)

Image size No. of SURF feature points

1024*1024 63

Target:To acquire more feature

2. Irrelevant Feature Points


1024*1024 63

1. Less Feature points

Approach-01 to solve: Considering the Whole image(Label and Pectoral Muscle)


Result:


255*256 2

Approach-02 : Detect feature from the cropped image



256*256 2


2. Relevant Feature Points

1. Less Feature pointsResult:

Approach-02 : Detect feature from the cropped image

Observation from approach 1 and 2

1. Image Size does not affectThe number of Feature Points

2. Zooming an image mayhelp to extract relevant featuresfrom the image(very few points to match)

mdb238.jpgImage Size: 1024*1024

mdb238.jpgImage Size: 256*256

Observation:Varying image size is not helping to get feature points

Image of an Microcalcification Cluster

23 strongest pointsAmong 100 Strongest Feature Points

from reference image

Reference image: mdb248.jpgImage size: 256 *256

Only 1 strongest pointsAmong 300 Strongest Feature Points

from Scene Image

Scene image: mdb248.jpgImage size: 427*588

Observing SURF Drawback

This method works best for :-- Detecting a specific object

(for example, the elephant in the reference image,rather than any elephant.)

-- Non-repeating texture patterns-- Unique feature

This technique is not likely to work well for:-- Uniformly-colored objects-- Objects containing repeating patterns.

detecting blob /point features. AIM Failed


Image Correlation Technique

Alternate ApproachImage Correlation Technique

Correlation

∑∑ ++=⊗k l

kjkihlkfhf ))((),(=f Image

=h Kernel/Mask

f1 f2 f3

f4 f5 f6

f7 f8 f9

h1 h2 h3

h4 h5 h6

h7 h8 h9

f1h1 f2h2 f3h3

f4h4 f5h5 f6h6

f7h7 f8h8 f9h9

=⊗ hf⊗

Experimental ResultsImage Correlation Technique

Image no: Benign mdb218.jpg

1. Original image

2. Kernel/ Mask/Template

3. Correlation Output

4. Identified MC(High value of sum.)

(Fixed Template Problem)..

Image no: Benign mdb222.jpg(Fixed Template Problem) Cont….

(Fixed Template Problem)..

Using Gabor Filter

Using Gabor Filter

• Make Gabor patch:

2; 2; 0.7854

2; 0.5; 0.7854

2; 2; 1.5708

5; 0.5; 1.5708

5; 2; 0.7854

2; 0.5; 1.5708

5; 0.5; 0.7854

5; 2; 1.5708

• Correlate the patch with image-To extract features of MC

⊗ =

0 10 20 30 40 50 60 70 80 90 100-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

Creating Gabor Mask

1. Linear RAMP

2. Linear RAMP values across: Columns Xm (left) and Rows Ym (Right)

3. Linear RAMP values across - Columns(Xm)

The result in the spatial domain

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5Xm (Across Columns) Ym- (Across rows)

4. Across Columns, Xm :a) Increase frequencyb )Use gray color map

6. Adding Xm and Ymtogether in

different proportions5. Across Rows, Ym :

a) Increase frequencyb )Use gray color map

Creating Gabor Mask

7. Create Gaussian Mask

8. Multiply Grating and Gaussian

Grating Gaussian Mask

Creating Gabor Mask

7. GABOR Mask

Creating Gabor Mask

Alternate ApproachUsing Gabor Filter Gabor kernel

2; 2; 0.7854

2; 0.5; 0.7854

2; 2; 1.5708

5; 0.5; 1.5708

5; 2; 0.7854

2; 0.5; 1.5708

5; 0.5; 0.7854

Scale , frequency, orientation

5; 2; 1.5708

MatrixSize = 26; %always scalar!

Scales = [2, 5];Orientations = [pi/4, pi/2];Frequencies = [0.5, 2];

CenterPoints = [13 13]; %int type (eg. [5 5; 13 13])

CreateMethod = FilterBank.CREATE_CROSSPRODUCT;

010

2030

020

40-0.5

0

0.5

2; 2; 0.7854

010

2030

020

40-0.2

0

0.2

2; 0.5; 0.7854

010

2030

020

40-0.2

0

0.2

2; 0.5; 1.5708

010

203

020

40-0.2

0

0.2

5; 2; 0.7854

010

2030

020

40-0.2

0

0.2

5; 2; 1.5708

010

2030

020

40-0.1

0

0.1

5; 0.5; 0.7854

010

2030

020

40-0.1

0

0.1

5; 0.5; 1.5708

010

2030

020

40-0.5

0

0.5

2; 2; 1.5708

Using Gabor Filter Gabor kernel

; 0 5; 5 08

Using Gabor Filter

⊗

⊗

⊗

=

=

=

Using Gabor Filter

⊗

⊗

⊗

=

=

=

⊗ =

Image In Spatial DomainUsing Gabor Filter Final Scenario

mini-MIAS drawbacksExperimental Realization

mini-MIAS drawbacksBenign mdb218

Original Enhanced

Gabor Effects


Original

Enhanced

2; 2; 0.7854

2; 0.5; 0.7854

2; 2; 1.5708

5; 0.5; 1.5708

5; 2; 0.7854

2; 0.5; 1.5708

5; 0.5; 0.7854

5; 2; 1.5708

Gabor Effects


Original

Enhanced

Gabor Effects


Original Enhanced

- NO definite Feature found

Observations:

Gabor Effects


Original Enhanced

Are these really enhanced?

-There is more detail, but could be noise.

-Enhanced versionseems to contain compression artifacts.

Question Arise?

Gabor Effects

mini-MIAS drawbacks

Enhanced version can contain Noise

Experimental Realization

1.Very Poor Quality with .jpg compression effects

a) Original image b) Enhanced image b) Enhanced imagea) Original image

mdb209

mdb213

mdb219

mdb249

mini-MIAS drawbacks

Not good enough for MC to be detectable


2. Reduced in resolution

Benign mdb218

Original Enhanced

mini-MIAS drawbacks

Not good enough for MC to be detectable


2. Reduced in resolution

Benign mdb218

Original

Enhanced

Where is MC?

OBSERVATION:

-There is more detail, but could be noise.

-Enhanced versionseems to contain compression artifacts.

More Evaluation (Gabor)mdb219.jpgBenign

OBSERVATION:

-Image Smoothing to remove edge will

Vanish the existenceof MC


OBSERVATION:

-NO definite feature of MC


OBSERVATION:

-NO definite feature of MCFalse contour


OBSERVATION:


No feature


OBSERVATION:


No feature

Several similar area false positive o/p


OBSERVATION:

- Bad resolution- Noise dominant- No definite feature of MC


OBSERVATION:

- Bad resolution/Poor quality image

- No definite feature of MC


OBSERVATION:

- Bad resolution-No definite feature of MC- Noise dominant


OBSERVATION:



OBSERVATION:

-feature of MC-But MC has different

orientationin different image

More Evaluation (Gabor)mdb209.jpgMalignant

OBSERVATION:



OBSERVATION:


More Evaluation (Gabor)Malignant mdb231.jpg

OBSERVATION:

- No definite feature of MC- Noise dominant


OBSERVATION:

-No definite feature of MC- Noise dominant


OBSERVATION:





OBSERVATION:

- No definite feature of MC- Noise dominant

Observation &

Drawing Conclusion

Future detection

Observation & Drawing Conclusion Feature Detection


• Very Poor Quality with .jpeg compression effects(Original MIAS doesn’t have such artifacts)

Limitations of mini-MIAS:

What can be done using mini-MIAS ?

• Can be used for big object detection(Pectoral Muscle, X-ray Label, Tumor, Mass detection)

Conclusion: mini-MIAS is not a good choice for:MC feature extraction


Any alternative to mini-MIAS?


Database Name Authority

MIAS ( Mammographic Image Analysis Society Digital Mammogram Database)

Mammography Image Analysis Society- an

organization of UK research groups

DDSM (Digital Database for Screening Mammogram) University Of South Florida

NDM (National Mammography Database) American College Of Radiology

LLNL/UCSF Database

Lawrence Livermore National Laboratories

(LLNL), University of California at San Fransisco (UCSF)

Radiology Dept.


Database Name Authority

Washington University Digital Mammography Database Department of Radiology at the

University of Washington

Nijmegen Database Department of Radiology at the

University of Nijmegen, the

Netherlands

Málaga mammographic database University of Malaga Central Research

Service (SCAI) ,Spain

BancoWeb LAPIMO Database Electrical Engineering Department at

Universidad de São Paulo, Brazil


These databases are NOT FREE

Research Findings

5; 0.5; 0.7854

Research FindingsImproved computer assisted

screening of mammogram

Detection and removal of big objects:- Pectoral Muscle - X-ray level

MC

Suspected

Observation & Drawing Conclusion On

Feature Detection


• Very Poor Quality with .jpeg compression effects(Original MIAS doesn’t have such artifacts)

Limitations of mini-MIAS:

What can be done using mini-MIAS ?

• Can be used for big object detection(Pectoral Muscle, X-ray Label, Tumor, Mass detection)

Conclusion: mini-MIAS is not a good choice for:MC feature extraction

BesideResearch Findings…

Published PaperAvailable Online:

http://cennser.org/IJCVSP/paper.html

Submitted Paperhttp://www.journals.elsevier.com/image-and-vision-computing/

1. Find Attribute/Feature From the enhanced mammogram:To train the machine:

-ANN (Artificial Neural Network)-SVM (Support Vector Machine)- GentleBoost Classifier [30]

2. Based on feature(size/shape), will move on to classification( benign or malignant)

MicrocalcificationIdentification

MicrocalcificationClassification

Plan of action as follows:

Further Research ScopeThere is always more to work on..In Research:

Future Plan

1. Segment the image

2. Find out the feature from the segmented image

3. Train the machine with features:

-ANN (Artificial Neural Network)-SVM (Support Vector Machine)

- GentleBoost Classifier [30]

4. Identify the MC5. Classify the MC

Available options

[2]D.Narain Ponraj, M.Evangelin Jenifer, P. Poongodi, J.Samuel Manoharan “A Survey on the Preprocessing Techniques of Mammogram for the Detection of Breast Cancer”, Journal ofEmerging Trends in Computing and Information Sciences, Volume 2, Issue 12, pp. 656-664,2011

[1]K. Santle Camilus , V. K. Govindan, P.S. Sathidevi,” Pectoral muscle identification inmammograms”, Journal of Applied Clinical Medical Physics , Vol. 12 , Issue No. 3 , 2011

[3]Arnau Oliver, Albert Torrent, Meritxell Tortajada, Xavier Llad´o,Marta Peracaula,Lidia Tortajada, Melcior Sent´ıs, and Jordi Freixenet,” Automatic microcalcification and clusterdetection for digital and digitised mammograms”, Springer-Verlag Berlin Heidelberg, 36,pp. 251–258, 2010

Reference

[4]Arnau Olivera, Albert Torrenta , Xavier Lladóa , Meritxell Tortajada, Lidia Tortajadab,Melcior Sentísb, Jordi Freixeneta, Reyer Zwiggelaarc,” Automatic microcalcification and clusterdetection for digital and digitised mammograms”,Elsevier:Knowledge-Based Systems,Volume 28, pp. 68–75, April 2012.

[5]A.Papadopoulos, D.I . Fotiadis, L.Costrrido,” Improvement of microcalcification clusterdetection in mammogaphy utilizing image enhancement techniques”.Comput.Bio.Med.10,Vol 38,Issue 38,pp.1045-1055,2008

[6]N.R.Pal,B.Bhowmik, S.K.Patel, S.Pal, J.Das,”A multi-stage nural network aided system fordetection of microcalcification in digitized mammogeams”,Neurocomputing, Vol 11,pp.2625-2634,2008

[7]M.Rizzi, M.D’Aloia, B.Castagnolo,” Computer aided detection of microcalcification in digitalMammograms adopting a wavelet decomposition ”,Integr.Comput.-Aided Eng.,Vol 16,Issue 2,pp.91-103,2009

Reference

[8]S.N.Yu, Y.K. Huang,” Detection of microcalcifications on digital mammograms using combinedModel-based and statistical textural features”, Expert Syst.Appl. , Vol 37,Issue 7,pp.5461-5469,2010

[9]Wang T. C and Karayiannis N. B.: Detection of Microcalci¯cations in Digital Mammograms Using Wavelets, IEEE Transaction on Medical Imaging, vol. 17, no. 4,(1989) pp. 498-509

[10]. Daubechies I.: Ten Lectures on Wavelets, Philadelphia, PA, SIAM, (1992)

[11] Strickland R.N. and Hahn H.: Wavelet transforms for detecting microcalcificationsin mammograms, IEEE Transactions on Medical Imaging, vol. 15, (1996) pp. 218-229

[12]Heinlein P., Drexl J. and Schneider Wilfried: Integrated Wavelets for Enhancement of Microcalcifications in Digital Mammography, IEEE Transactions on Medical Imaging, Vol. 22, (2003) pp. 402-413

[13]. Zhibo Lu, Tianzi Jiang, Guoen Hu, Xin Wang: Contourlet based mammographicimage enhancement, Proc. of SPIE, vol. 6534, (2007) pp. 65340M-1 - 65340M-8

Reference

[14]Fatemeh Moayedi, Zohreh Azimifar, Reza Boostani, and Serajodin Katebi: Contourlet-based mammography mass classification, ICIAR 2007, LNCS 4633,(2007) pp. 923-934

[15] Balakumaran T., Vennila ILA, Shankar C.G: Detection of Microcalcification in Mammograms Using Wavelet Transform and Fuzzy Shell Clustering, International Journal of Computer Science and Information Security, Vol 7,Issue 1,pp.121-125,2010

[16] Zhang X., Homma N., Goto S.,Kawasumi Y., Ihibashi T.,Abe M.,Sugita N.,Yoshizawa M: A Hybrid Image Filtering Method for Computer-Aided Detection of Microcalcification Clusters in Mammograms, Journal of Medical Engineering, Vol 3,Issue 1,pp.111-119,2013

[17] Lu J., Ikehara T., Zhang Y,Mihara T., Itoh T.,Maeda R:High quality factor silicon cantilever driven by piezoelectric thin film actuator for resonant based mass detection, Micro system Technologies , Vol 15, Issue 8, pp:1163-1169., 2009

[18] Leeuw H.D., Stehouwer BL, Bakker CJ, Klomp DW, Diest PV, Luijten PR, Seevinck PR,Bosch MA, Viergever MA, Veldhuis WB:Detecting breast microcalcifications with high-field MRI, NMR in Biomedicine,Vol 27, Issue 5, pages 539–546,2014

Reference

[19] Shankla V, David D. P, Susan P. Weinstein; Michael D., Tuite C, Roth R., Emily F:Automatic insertion of simulated microcalcification clusters in a software breast phantom, , Proc. SPIE 9033, Medical Imaging 2014: Physics of Medical Imaging, 2014


[21]Laine A.F., Schuler S., Fan J., Huda W.: Mammographic feature enhancement by multiscale analysis, IEEE Transactions on Medical Imaging, 1994, vol. 13, no. 4,(1994) pp. 7250-7260

22. Park S.-I., Smith M. J. T., and Mersereau R. M.: A new directional Filter bank forimage analysis and classification, Proceedings of IEEE International Conference onAcoustics, Speech, and Signal Processing (ICASSP '99), vol. 3, (1999) pp. 1417-1420

23. Sameer S. and Keit B.: An Evaluation on Contrast Enhancement Techniques forMammographic Breast Masses, IEEE Transactions on Information Technology inBiomedicine, vol. 9, (2005) pp. 109-119

Reference

24. Rosten, E., and T. Drummond. "Machine Learning for High-Speed Corner Detection." 9th European Conference on Computer Vision. Vol. 1, 2006, pp. 430–443.

25. Shi, J., and C. Tomasi. "Good Features to Track." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. June 1994, pp. 593–600.

26. Harris, C., and M. J. Stephens. "A Combined Corner and Edge Detector." Proceedings of the 4th Alvey Vision Conference. August 1988, pp. 147–152.

27 Bay, H., A. Ess, T. Tuytelaars, and L. Van Gool. "SURF: Speeded Up Robust Features." Computer Vision and Image Understanding (CVIU). Vol. 110, No. 3, 2008, pp. 346–359.

28.Leutenegger, S., M. Chli, and R. Siegwart. "BRISK: Binary Robust Invariant Scalable

29.Matas, J., O. Chum, M. Urba, and T. Pajdla. "Robust wide-baseline stereo from maximally stable extremal regions."Proceedings of British Machine Vision Conference. 2002, pp. 384–396.

Reference

30. Oliver A.; Torrent A. , Tortajada M, Liado X, R., Preacaula M , Tortajada L., Srntis M.,Ferixenet J: A Boosting based approach for automatic Microcalcification Detection,Springer-Verlag Berlin Heldelberg,Lecture notes on Computer Science (LNCS 6136), (2010)pp. 251- 258

Reference

Thank you foryour time and attention

masters' whole work(big back-u_pslide)

Education