evolution of a local boundary detector for natural …sipper/publications/boundary.pdf[2] p. dollar,...

Evolution of a local Boundary Detector for Natural Imagesvia Genetic Programming and Texture Cues

Track: Genetic Programming

Ilan KadarDept. of Computer Science

Ben-Gurion UniversityBeer-Sheva, Israel

ilankad@cs.bgu.ac.il

Ohad Ben-ShaharDept. of Computer Science

Ben-Gurion UniversityBeer-Sheva, Israel

ben-shahar@cs.bgu.ac.il

Moshe SipperDept. of Computer Science

Ben-Gurion UniversityBeer-Sheva, Israel

sipper@cs.bgu.ac.il

ABSTRACTBoundary detection constitutes a crucial step in many com-puter vision tasks. We present a learning approach for au-tomatically constructing high-performance local boundarydetectors for natural images via genetic programming (GP).Our GP system is unique in that it combines filter kernelsthat were inspired by models of processing in the early stagesof the primate visual system, but makes no assumptionsabout what constitutes a boundary, thus avoiding the needto make ad hoc intuitive definitions. By testing our evolvedboundary detectors on a highly challenging benchmark set ofnatural images with associated human-marked boundaries,we show performance that outperforms most existing ap-proaches.

Categories and Subject DescriptorsI.4.6 [Computing Methodologies]: Image Processing andComputer Vision—Segmentation

General TermsAlgorithms, Measurement, Performance, Design

KeywordsBoundary Detection, Computer Vision, Machine Learning,Evolutionary Algorithms

1. INTRODUCTIONBoundary detection in images is a fundamental problem in

computer vision. The performance of many high-level com-puter vision tasks, such as segmentation and object recog-nition, is highly dependent upon the boundary map of animage. A boundary is a contour in the image plane thatrepresents a change in the pixel’s “ownership” from one ob-ject or surface to another. In general, there are different

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.GECCO’09, July 8–12, Montréal, CanadaCopyright 2009 ACM X-XXXXX-XX-X/XX/XX ...$5.00.

types of boundaries: for example, those formed betweentwo regions with an abrupt change in the image bright-ness, and those formed between two regions with a changein the texture. Clearly, boundaries in natural images aremarked by change both in brightness and texture. There aresome attempts in computer vision to address both bright-ness and texture cues using complex and computationallyintensive schemes [5]. In contrast, humans have an out-standing ability to detect boundaries pre-attentively, andhence very fast. Correspondingly, evidence from behavioralscience and neuroscience strongly suggests that this pro-cess occurs in early stages of visual processing. This pa-per presents an approach that aims to use genetic program-ming (GP) as a learning framework for evolving detectors.The detectors are evaluated against human-marked bound-ary maps in order to accurately detect and localize bound-aries in grayscale natural images. The evolving programsuse both linear and non-linear operators to combine mul-tiple cues from the early stages in the visual cortex. Thepresented learning framework was developed based on in-sights from our recent work [3] with the critical improve-ment of incorporating texture cues. Our results show thatthis approach is highly effective at automatically generatingboundary detectors. By testing the evolutionary algorithmon a highly challenging benchmark set of natural imageswith associated human-marked boundaries, we show perfor-mance to be quantitatively human competitive [4].

2. METHODWe present a visual learning approach to automatically

construct a boundary detector using GP and texture cues.Each individual in the GP population represents a candidateboundary cue, which is then combined with a texture gradi-ent cue into a single detector on a trained logistic regressionclassifier. Fitness is defined as the F-measure, computed fora set T = {Ii} of n images taken from the training set of theBerkeley data set. The terminal set is image independent,such that the terminals for image Ii , given as an array ofmatrices, are the convolution of the image Ii with filter ker-nels tuned to various orientations. These filter kernels areinspired by models of processing in the early stages of theprimate visual system, which model both odd- and even-symmetric simple cell receptive fields at various orientations(see Figure 1). The function set contains both unary andbinary functions. The input and output of all functions arearrays of length N of data matrices with the same size as

Draft

images in T (see Table 1). In order to measure the degreeto which texture varies at a location (x, y) in direction θ,we used a texture gradient approach inspired by Martin andMalik [5].

(a) (b)

Figure 1: (a) Odd-symmetric simple cell with preferredorientation 0◦. (b) Even-symmetric simple cell with pre-ferred orientation 0◦.

Table 1: Function set.

M∀i∈1...N = Add(M1i ,M2i) Matrix additionM∀i∈1...N = Sub(M1i ,M2i) Matrix subtractionM∀i∈1...N = Mul(M1i ,M2i) Matrix multiplicationM∀i∈1...N = Max(M1i ,M2i) Largest elements taken

from M1i or M2i

M∀i∈1...N = Min(M1i ,M2i) Smallest elements takenfrom M1i or M2i

M∀i∈1...N = Pow2(M1i) Elements are 2 raised to thepower M1i

M∀i∈1...N = Sqrt(M1i) Square root of each elementof M1i

M∀i∈1...N = M(i+N/2)%N R90 (90◦ Rotation)M∀i∈1...N = M1i * Koddi M1i convolved with ori-

ented odd symmetric kernel

3. RESULTSThe output of each detector for a given image Ii is a soft

boundary map, which provides the probability of a bound-ary at each image location. The fittest individual generatedwith our approach is presented in Figure 2. The presented

Figure 2: The fittest individual generated with our ap-proach, with performance F-measure = 0.62.

detector was tested on the Berkeley test set of 100 images [6],and the overall performance was computed using the Berke-ley benchmark algorithm [1], which computes a score basedon the F-measure. Table 2 shows the obtained score of ourapproach compared with other existing approaches (Notethat like us, most of these approaches use training). One ofthe obtained soft boundary maps is shown in Figure 3.

Analyzing the top individuals might provide additionalinsights that could improve the selection of function and

(a) (b)

Figure 3: (a) Sample test image (b) Extracted softboundary map, using our best evolved detector, withF = 0.81.

Table 2: Performance summary table of local bound-ary detectors

.

Method Performance

Brightness and texture gradient 0.63GP/TG Detector 0.62OE/TG Detector 0.61Learning of the brightness distribution 0.60(brightness Gradient)GM/TG 0.58Texture gradient (TG) 0.58Multiscale gradient magnitude 0.58Second moment matrix 0.57Gradient magnitude (GM) 0.56Segmentation induced by 0.48scale invariance

terminal sets. Similarly, adding mid- and high-level cuesthat have been shown [2] to improve overall performanceon the Berkeley benchmark [1] might also contribute to theevolutionary process. These, and the informed use of addi-tional computational resources to improve performance, allconstitute our short-term future work.

4. REFERENCES[1] The berkeley segmentation dataset and benchmark

http://www.cs.berkeley.edu/projects/vision/grouping/segbench/.

[2] P. Dollar, Z. Tu, and S. Belongie. Supervised learningof edges and object boundaries. IEEE ComputerSociety Conference on Computer Vision and PatternRecognition, 2006.

[3] I. Kadar, O. Ben-Shahar, and M. Sipper. Evolvingboundary detectors for natural images via geneticprogramming. In Proc. of the 19th InternationalConference on Pattern Recognition, Tampa, Florida,2008.

[4] J. R. Koza, M. A. Keane, M. J. Streeter, W. Mydlowec,J. Yu, and G. Lanza. Genetic Programming IV:Routine Human-Competitive Machine Intelligence.Kluwer Academic Publishers, 2003.

[5] D. Martin, C. Fowlkes, and J. Malik. Learning to detectnatural image boundaries using local brightness, colorand texture cues. PAMI04, 2004.

[6] D. Matin, C. Fowlkes, D. Tal, and J. Malik. A databaseof human segmented natural images and itsapplications to evaluating segmentations algorithmsand measuring ecological statistics. In Proc. ofinternational Conference on Computer Vision, 2001.