leaf classification from boundary analysis

1

Leaf Classification from Boundary Analysis

Anne JorstadAMSC 663 Project ProposalFall 2007

Advisor: Dr. David Jacobs, Computer Science

2

Background

Electronic Field Guide for Plants University of Maryland Columbia University National Museum of Natural History

Smithsonian Institution

Project in development over 4 years

3

Background

Current System: Inputs photo of leaf on plain

background Segments leaf from background Compares leaf to all leaves in database,

using global shape information Returns images of closest matches to

the user

4

Background

Sean White, Dominic Marino, Steven Feiner. Designing a Mobile User Interface for Automated Species Identification. Columbia University, 2007.

5

Background

All leaves assumed to be from woody plants the Baltimore-Washington, DC area

245 species, 8000 images

The proof of concept has been implemented successfully

6

Proposal

Current System: All shape information is compared at a

global level, no specific consideration of edge types

My Project: Incorporate local boundary information

to complement existing system

7

Proposal

Leaf edges:

smooth

serrated

serrated, finer teeth

“double-toothed”

wavylobed and serrated

8

Proposal Specifics

Start with boundary curves as discrete points (already have this data with good accuracy)

Represent as , to use 1-D techniques

Classify!

)()( tiytx

9

Method 1: Harmonic Analysis

Harmonic Analysis Decompose boundary into wavelet

basis Different families of species have

distinct serration patterns in the frequency domain

What wavelet basis to choose?

10

Aside: What is a wavelet?

Fourier Transform: decomposes a function into frequency components

Wavelet Transform: similar to Fourier, but with quickly decaying or compactly supported basis functions good for feature detection

11

Method 1: Harmonic Analysis

Think of the boundary as a texture Several Computer Vision algorithms

exist for classifying textures Example:

Describe texture in terms of a set of fundamental features or patterns (sound like a wavelet basis?), search for them throughout the image

12

Method 2: Inner-Distance

“Inner-Distance” on multiple scales Measures the shortest distance between

two points on a path contained entirely within a figure

Good for detecting similarities between deformable structures

13

Method 2: Inner-Distance

The inner-distance has been successfully applied in several situations

Used already as part of the global classification

New: sample points on several scales and look for shape discrepancies not previously measured

14

Method 2: Inner-Distance

Examining inner-distances over a hierarchy of scales will capture new local information

Large scale: similar inner-distances

Small scale: distinct inner-distances

15

Method 3: Convexity

A serrated leaf is much less convex than a smooth one; use convexity measure as a pre-processing classification tool

May not prove useful, but might be worth exploring

16

Method 3: Convexity

Several ways to assign a convexity number to a shape:

etc.

))((

)(

objectConvexHullArea

objectAreaConvexity

)(

))((

objectPerimeter

objectConvexHullPerimeterConvexity

object

ConvexHull(object)

17

Algorithm Verification

Create artificial “leaves” with known properties

Prove algorithm correctness on these simple known cases

18

Algorithm Verification

Run new algorithm on current data sets Demonstrate “reasonable”

classification accuracy for relevant examples

Global information not considered, so expect that not all distinguishing features will be recognized

19

Algorithm Verification

Incorporate into existing system

Ideally: Provide classification results

independent from current results, so together a better overall classification is achieved

20

Specifications

Current system: MATLAB and C

My contribution: mostly MATLAB Image Processing Toolbox Wavelet Toolbox

21

Specifications

End product to run on portable computer Code must run quickly on a small

processor Development and testing from PC

22

References “A New Convexity Measure for Polygons”. Jovisa Zunic, Paul L. Rosin.

IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 7, July 2004.

“Contour and Texture Analysis for Image Segmentation”. Jitendra Malik, Serge Belongie, thomas Leung, Jainbo Shi. International Journal of Computer Vision, vol. 34, no. 1, July 2001.

“Designing a Mobile User Interface for Automated Species Identification”. Sean White, Dominic Marino, Steven Feiner. Proceedings of the SIGCHI, April 2007.

“First Steps Toward an Electronic Field Guide for Plants”. Gaurav Agarwal, Haibin Ling, David Jacobs, Sameer Shirdhonkar, W. John Kress, Rusty Russell, Peter Belhumeur, Nandan Dixit, Steve Feiner, Dhruv Mahajan, Kalyan Sunkavalli, Ravi Ramamoorthi, Sean White. Taxon, vol. 55, no. 3, Aug. 2006.

“Using the Inner-Distance for Classification of Articulated Shapes”. Haibin Ling, David W. Jacobs. IEEE Conference on Computer Vision and Pattern Recognition, vol. II, June 2005.

23

Questions? Comments?