seeded region growing using line scan algorithm - stack base implementation

Image Segmentation- Region Growing

Algorithm

Pi19404

March 1, 2013

Contents

Contents

2 | ??

Image Segmentation - Region Growing Algorithm For UnderWater ImageSegmentation

Image Segmentation - RegionGrowing Algorithm For UnderWaterImage Segmentation

0.1 IntroductionIn this document we will look at task of image segmentation anduse region growing algorithms to achieve this task and its applicationto underwater image segmentation.

0.2 Image Segmentation

The goal of image segmentation is to partition the image into sub-regions with homogeneous intensity (color or tex- ture) propertieswhich will hopefully correspond to objects or object parts.

Region growing algorithms is a method to extract similar partsof image. The region growing algorithms checks for similarity of theneighborhood pixels of given initial seed point and labels the pixelsthat meet the similarity criteria and mean value of region foorsimilarity evaluation is determined.

The neighborhood pixels of the newly labelled points are anal-ysed and similarity comparison is performed with newly computedmean value. In this way neighborhood can grow in a iterative mannertill stopping criteria is met.

0.3 Region Growing Algorithm

Conider a set R which contains pixels that belongs to the regionthat is required to be segmented.Let set S represent a set ofpixels called seed points. Seed points are pixels that is known tobelong to object desired to be segmented.

3 | ??


For the present application we assume that we need identifythe region in the image to which seed pixels belong ie other regionof the image are not required.

For every seed points 4/8 connected neighbors of seed pointare analyzed for similarity with the seed pixel using some pre-defined criteria.

Thus all the neighborhood pixels are marked to be compared withseed pixel.

If the connected neighbor satisfies the criteria then it is in-cluded labelled as desired region and neighbor is unmarked. Whileall the connected neighbors of the present neighbor are marked.

This process continues till no marked pixels remains which results insegmentation of region to which seed pixel belongs.

As the region grows the similarity comparision can be peformedwith the mean value of region rather than just the seed pixels.

0.4 4/8 conneced neighbors Recurive Algorithm

We will use a recurive algorithm to implement the task. Input toalgorithm is input image and seed pixel location,the output is alabelled image.

Input to recurive function will be location of pixel required tobe analyzed. Inital call to recurive function is the seed pixel location.

The function calls a ColorDistance function to check for simi-larity of the given pixel with the seed pixel provided the pixel hasnot been analyzed already.

If the distance is less than a specified threshold pixel is markedin the output labelled image and recursive calls are initiated toneighbors of the current pixel.

Labelling the output image indicates the pixel has been analyzedand any call to function labelled pixel locations will return withoutperforming any action.

4 | ??


As algorithm progress all connected neighbors of seed pixels areused to initiate recursive calls .If these pixels satisfy the similaritycriteria use recurive calls for their connected neighbors and so on.

In this way all the connected pixel of seed pixel satisfying thesimilarity criteria are labelled in the output image and segmentationis achieved.

The we can either consider 4/8 connected beighbors for analy-sis.

In the present implementation we use a threshold of 20 . Ifeuclidean distance between two pixels is less than 20 we considerthe pixels to be similar.

We can see that region growing algorithm stop at places whenreflections of object fall on the water.Increasing the threshold isone option to overcome this problem.But it is not guranteed towork in all scenarios.This is shown below.

The seed is choosen at location within the shown object. In

(a) Image (b) thresh 20 (c) thresh 50

(d) input 2 (e) thresh 50 (f) thresh 80

Figure 1: Standard Seeded region growing

the second image we can see than even with high threshold of80,the white over exposed region will never be labelled as belonging

5 | ??


to the object.

0.5 Modification of OverExposed Regions

The standard region growing algorithms use only color measurementsfor similarity criteria.Underwater images are affected by reflectionsand crinkle effects which leads to over exposure in some parts ofthe object being segmented. Due to these effects a large intensitychange in observed in region withing the object being segmented iesharp edge are observed within the object region.

The standard Region growing algorithms would terminate at theseregion since large color difference is encountered wrt to seed pixelcolor or color of object required to be segmented.

Thus to overcome this we compute the similarity measure basedon the color,exposedness of region as well as local edge intensity.

One simple technique is to check the distance of pixel being consid-ered to white color.If the distance is close to white than seed pixelcolor and within a specified threshold ,which can be independentsimilarity threshold then we will include pixel as belonging to theobject.

However this modification may not always provide desired sementa-tion as shown in the images below.

0.6 Using The Edge Map

In the second image we can see that due to connected whitecolored pixels the region grows belong the desired object.

To over the effect of seeded region growing algorithm growingbelond the object we use edge information. We compute the edgemap using canny edge detection technique. mean value of edgeabout a small neighborhood of point being analysed is computed andadded to the distance computed to white colored pixels. Thus ifthe color is white but region has a edge the distance will increaseand if region has strong edge it will increase the distance belongthe threshold and region growing will stop. though pixel is white in

6 | ??


(a) input 2 (b) thresh 80 (c) modified

(d) Image (e) thresh 20 (f) thresh 50

Figure 2: Modified Seeded Region Growing Algorithm

color and connected to original object.

Also we make one more modification of reducing the distancethreshold to white pixels keeping it half of color threhold.

In the above image the hand along with the object is de-

(a) Image (b) thresh 20 (c) modified SRG

(d) edge map

Figure 3: Using Edge Map

tected since the color threshold is kept high.But the main point

7 | ??


to note is that due to addition of edge map the region does notoverflow significantly from the desired object boundary

0.7 Implementation Details

0.7.1 Color Distance Computation

The color distance used is simply euclidean color distance betweentwo color vectors. and is defined by function colorDist

1 float seed_fill :: ColorDist3(Scalar a,Scalar b)

2 {

3 float a1=a[2];

4 float a2=a[1];

5 float a3=a[0];

6 float b1=b[2];

7 float b2=b[1];

8 float b3=b[0];

9 float dist =(((a1-b1)*(a1 -b1)+(a2 -b2)*(a2-b2)+(a3-b3)*(a3 -b3)))

;

10 dist=sqrt(dist /3);

11 return dist;

12 }

0.7.2 Edge Map Computation

The edge map computation is performed before calling the seededregion growing algorithm.

First A Laplacian of the RGB image is taken and mean valueof all 3 channels is computed.

Since edge detection also enhances the noise in the image gaussianblur is applied on the image before taking the laplacian.

1 Mat edge1;

2 cv:: GaussianBlur (input ,input ,Size (3,3) ,1);

3 cv:: Laplacian (input ,edge1 ,input.depth () ,1,1);

4 vector <Mat > em;

5 cv:: split(edge1 ,em);

6 cv::add (em[0],em[1],edge);

7 cv::add (edge ,em[2],edge);

8 edge=edge /3;

8 | ??


The requirement is to obtain a normalized edge map that returnsa value between 0-255.

The tranformation function should give a value must be large forstronger edges and small for weak edges. The transformationfunction is monotonic.

The transformation function should give a variation of edge map valuemust gradually increase/decrease according to the edge strength.

A exponential transformation function satisfies these criteria.

The transformation function used is

g0(x) = exp(�I(x; y)=K)

K is parameter that determine how fast the edge strength falls

Since value corresponding to the maximum edge intensity isg0(x) = exp(�1=K)

all the value below this are thresholded to 0

g(x) = 1� g0(x)(1)

1 double K=1;

2 edge.convertTo (edge ,CV_32FC (3) , -1.0/(255.0*K) ,0);

3 cv::exp (edge ,edge);

4 cv:: threshold(edge ,edge , std::exp(-1/K)+0.001 , 0, THRESH_TOZERO );

5 edge=1-edge;

6 edge =255* edge;

7 edge.convertTo (edge ,CV_8UC (1) ,1,0);

0.7.3 Similarity Measure Implementation

In the code the main method to calculate the similarity is ColorDist

The Color based similarity measure is computed by the functionColorDist3 which is just the euclidean distance between two RGBcolor vector.

9 | ??


The Euclidean distance of the current pixel with the seed pixelcolor and with mean pixel intesity value about small neighborhood orcurrent pixel with the white pixel is computed

We define a small neighborhood size over whih the mean is computedthe default value is 5.

The ROI is determined as follows1 int s1=x-bsize /2 >=0?x-bsize /2:0;

2 int s2=y-bsize /2 >=0?y-bsize /2:0;

3 int s3=(x-bsize /2+ bsize) <=width?bsize:x-bsize /2+bsize -width -1;

4 int s4=(y-bsize /2+ bsize) <=height?bsize:y-bsize /2+bsize -height -1;

5 Rect r=Rect(s1 ,s2 ,s3 ,s4);

The mean value of the image pixels and edge pixels are computed inthe defined ROI about the current pixel.

1 Mat rx=edge;

2 uchar *ptr1=(uchar *)input.data;

3 uchar *ptr=(uchar *)rx.data;

4 for(int i1=0;i1 <r.height;i1++)

5 {

6 for(int j1=0;j1 <r.width;j1++)

7 {

8 f1=f1+ptr[(i1+r.y)*rx.step+rx.channels ()*(j1+r.x)];

9

10 f4=f4+ptr1[(i1+r.y)*input.step+input.channels ()*(j1+r.x)];

11 f5=f5+ptr1[(i1+r.y)*input.step+input.channels ()*(j1+r.x)+1];

12 f6=f6+ptr1[(i1+r.y)*input.step+input.channels ()*(j1+r.x)+2];

13

14 }

15 }

16 f1=f1/(r.width*r.height);




Next we compute the distance of the mean intensity value withwhite color value

1 Scalar cc=Scalar(ceil(f4),ceil(f5),ceil(f6));

2 float dist2=ColorDist3(cc ,c);

10 | ??


The final step is to specify a threshold .If the distance dist2

to white color value plus mean edge intesity is less than specifedthreshold then it belongs to object and we return a similarity valueof 0 else we compare the color distane measure to seed pixel valuewith threshold and decide if the pixel belongs to object or not.

0.8 Stack Based scan line methodIn the line scan method ,the pixels in the same vertical/horizontalas the seed/current pixel are scanned.The length of scan is predefined For the present example we set the minmum extent to be0 and maximum extent to be image height/width.

Here we look at horizontal scan algorithm.

Consider a object to be segmented.

Consider a seed pixel chosen within the object being segmented.and a horizontal line containing the seed pixel.

The point S shown the figure is the seed pixel.

(a) Sample Object

The input to scan line algorithm are a pair of seed points pointslying on a horizontal line. The pair of points have different x co-ordinates and same y co-ordinates. and if line above/below currentline is to be scanned next.

The scan line algorithm consists of two step ,finding the seedpoints and next is to mark the pixels between the seed points.

11 | ??


0.8.1 InitializationThe Seed Points can be found by identifying the points at whichthe scan line intersects the edges of image.

Initial pair of seed points are the same as input seed pixel point.

From the current point we keep on checking all the pixels tothe left and right of current pixel using similarity criteria. Thepoints at which the similarity criteria is not satisfied are local edgepoints.

Points A,B shown in the figure determine the seed points.

0.8.2 Finding the Seed points

Let us assume we have seed points of current line. Next step isto determine extents of line above and below.Since pixel intensitycannot change abrubtly hence extents will be close to the points Aand B.

A point with same x co-ordinate as A and y co-ordinate of lineabove is one of seed points and other seed point is point withsame co-ordinate as B and y co-ordinate is line above current line.

(b) Line Scanning

Thus for the above line we obtain the left edge as point Aor point to left of A. and the right edge as point B or point toright of B. Let these new points be denoted as C and D.

From point C we move towards left filling points which satisfysimilarity criteria. If we reach the point D.Then Line has beensuccessfully filled and scan can be performed for the line aboveand below the present line,

12 | ??


From point C as we move towards left we may encounter a edgebefore we reach point D Line segment (E;F ) and (G;H) denotessuch a case.

Thus line segment (E;F ) and (G;H) are considered the scannedline and line above/below this line segment will be scanned.

Similarity if we move from the point D and end before reach-ing point C, then this new line segment will be filled and lines aboveand below this line segment will be scanned next.

In this way entire image will be parsed by using connected linesegments.

0.9 Implementation

In present application we use a stack based implementation. Theelements of stack contain iformation above the line to be scanned.

A class Line_Element is used to define the entry.It will con-tain the right and left co-ordinates of current line segment,yco-ordinate line segment above/below

1 class line_element

2 {

3 public :

4 int left;

5 int right;

6 int y;

7 }

The first operation is to obtain the left and right edges of the line

This is done by scanning left/right from the left and right pixelvalues.

1 for(xa=p.left;xa >=nMinX && check_color(xa , p.y)==true;xa --);

2 if(xa <p.left)

3 xa=xa+1;

4

5 for(xb=p.right;xb<nMaxX && check_color(xb , p.y)==true;xb++);

6 if(xb >p.right)

7 xb=xb -1;

13 | ??


From left end of the line we determine the line segments en-countered as we move towards the right and add pixel of line aboveand below these line segments to stack.

In the below code xa and xb are left and right ends of linesegment.

14 | ??


1 left=xa;

2 do

3 {

4 for(xa=left;xa<nMaxX && check_color(xa , p.y) == true;xa++)

5 {

6 SetPixel(xa , p.y, fill_color);

7 }

8 if(xa >left)

9 {

10 lines.push (line_element(left ,xa -1,p.y-1,-p.nexty));

11 lines.push (line_element(left ,xa -1,p.y+1,-p.nexty));

12 }

13

14 for(;xa <xb && check_color(xa, p.y) == false;xa++);

15 left=xa;

16

17

18 }while(xa<xb);

0.10 Future WorkStatic color threholds are being used presently.A adaptive thresholdwould be required based on information content present in theimage as well as strenght of edge .

Look for further improvements in stack based algorithms.

A better color distance criteria for similarity matching is required.Analysis in different color space and distance other than eculideanwill be considered in the future work.

0.11 CodeThe class seed_fill_stack defines a class for stack based scan linealgorithm. The input to algorithm is input image and seed pixellocation and output is labelled image. Code is available in repositoryhttps://github.com/pi19404/m19404/tree/master/RegionGrowing

15 | ??

https://github.com/pi19404/m19404/tree/master/RegionGrowing

seeded region growing using line scan algorithm - stack base implementation

Documents