image indexing and retrieval using moment invariants imran ahmad school of computer science...

Post on 17-Dec-2015

215 Views

Category:

Documents

1 Downloads

Preview:

Click to see full reader

TRANSCRIPT

Image Indexing and Retrieval using

Moment Invariants

Imran AhmadSchool of Computer Science

University of Windsor – Canada

I. Ahmad - Windsor, Canada 2

Outline

IntroductionShape-based RetrievalImage RepresentationMoment InvariantsProposed ApproachExperimental ResultsConclusions

I. Ahmad - Windsor, Canada 3

Introduction

Information CharacteristicsNatureMultiple formatsComplexTypes

Image DatabaseTime dependent sequence

Video Database

I. Ahmad - Windsor, Canada 4

Introduction (contd.)

Still information

Dynamic informationTemporal Evolution of informationChanges in features and characteristics

I. Ahmad - Windsor, Canada 5

Introduction (contd.)

Image contentsReal world informationUnique featuresNeed retrieval based on contents

I. Ahmad - Windsor, Canada 6

Image Retrieval

Exact matchSimilarity-based retrievals

Color / texture similarity-based retrievalsSpatial similarity-based retrievalsShape similarity-based retrievals

I. Ahmad - Windsor, Canada 7

Shape-based retrievals

Model-based Object Recognition ApproachModels based on global and local featuresUnknown object compared against known ones

Data-Driven ApproachAn index for known shapesSearch utilizes such indices

I. Ahmad - Windsor, Canada 8

Image Representation

Images can be defined in terms of:Global features

Based on overall image compositionEasier to compute

Local featuresBased on individual image componentsIncorporate spatial informationComputationally expensive

I. Ahmad - Windsor, Canada 9

Shape

How to define a shape?A geometric property of a figureFormal definition – independent of language

Described in terms of properties invariant under a group of coordinate transformationsLet is a characteristic function such that

0

1),( yxf

For points in the figure

Otherwise

),( yxf

I. Ahmad - Windsor, Canada 10

Shape (contd.)

Definition: Let be a group of coordinate transformations. The function I is invariant w.r.t. if

for all characteristic functions and all transformations

Definition: A shape of a figure is a pair <I, >, where I is invariant under the group of coordinate transformations

yxfIyxfI ,,

),( yxf

I. Ahmad - Windsor, Canada 11

MomentsCan capture global information about image Do not require closed boundaries.

Regular moments – introduced by Hu.Invariant to translation, rotation and scaling

Algebraic momentsDo not depend on actual values of the coefficients

Central momentsEquivalent to regular moments of an image that has been shifted

I. Ahmad - Windsor, Canada 12

Moments (contd.)

ApplicationsImage reconstructionShape identification such as aircrafts, etc.Shape recognitionClassifiers

I. Ahmad - Windsor, Canada 13

Moments (contd.)

Let be the image intensity distribution functionp +q is the order of moments (for p, q =0, 1, 2, …)

the algebraic moment of functions are given as:

For a digital image of size M x N

yxf ,

R

qppq dxdyyxfyxm ,

M

x

N

y

qppq yxfyxm

0 0

,

I. Ahmad - Windsor, Canada 14

Moments (contd.)

For centralized moments, we can write:

with its digital form as:

dxdyyxfyyxxqp

pq ,

M

x

N

y

qppq yxfyyxx

0 0

,

I. Ahmad - Windsor, Canada 15

Moments (contd.)

M

x

N

y

M

x

N

y

M

x

N

y

M

x

N

y

mymymyy

mymmmmyy

mmmmyy

m

0 001

20203

303

01020 0

0020102

202

0 00000010101

0 00000

23

0

1

Central moments up to 2nd order are defined as:

I. Ahmad - Windsor, Canada 16

Moments (contd.)

010 0

2201121

221

10200 0

0021020

220

100 0

2021112

212

10110 0

0001101111

0 00000101010

22

22

0

mxmymxmyyxx

mymmmmxx

mymxmymyyxx

mymmmmmyyxx

mmmmxx

M

x

N

y

M

x

N

y

M

x

N

y

M

x

N

y

M

x

N

y

I. Ahmad - Windsor, Canada 17

Moments (contd.)

Algebraic moments by Hu

2

30122

210321033012

20321

21230123003217

03211230112

03212

123002206

23012

2210321032103

20321

21230123012305

20321

212304

20321

212303

211

202202

02201

33..........

33

4

33..........

33

33

4

M

M

M

M

M

M

M

I. Ahmad - Windsor, Canada 18

Moments (contd.)

Moment Invariants:Time complexity in computing MI is directly proportional to the number of pixels in the silhouette or forming the boundary.Let N be the perimeter of the closed boundaryTo calculate 2nd order moments, we need:

4(N-1) real additions and 3N real multiplicationsTo calculate 3rd order moments, we need:

6(N-1) real additions and 12N real multiplications

I. Ahmad - Windsor, Canada 19

Moments (contd.)

A Sample binary image & moment invariants

F={0.259179343138514, 0.00801986505055, 0.012354456089699, 0.00827468547136, -0.000000750728194, -0.00005777268349,

0.00000025369430}

I. Ahmad - Windsor, Canada 20

Clustering

Main CategoriesHierarchical methods

A nested sequence of partitionsInvolves multiple iterations to cluster objects

Non-hierarchical methodsAssume desired number of clusters at the beginningData is reallocated until a particular clustering criteria is optimized.Objects in a cluster are more similar to each other.

I. Ahmad - Windsor, Canada 21

Clustering (contd.)

K-means clusteringLet a set of N objects in d-dimensional space Rd

k is an integerDetermine a set of k points in Rd, called centers, so as to minimize the mean squared distance from each data point to its nearest center.

Also known as squared-error distortion.

I. Ahmad - Windsor, Canada 22

K-means Clustering (contd.)

ALGORITHM: K-means clustering 1. For pattern vectors P1, P2,…, Pm, set first

k pattern vectors to the initial clusters C1=P1, C2=P2, C3=P3,…, Ck=Pk, where m >= k

2. Assign each pattern vector to the nearest cluster

3. Compute new cluster means 4. If new cluster means = old cluster means

stop, else

go to step 2

I. Ahmad - Windsor, Canada 23

K-means Clustering (contd.)

K-means Clustering Tree (KCT)The KCT is a hierarchical data structure similar to a combination of binary search tree and B+ -tree.

Data pointers are stored only at the leaf nodes of tree. Non-leaf nodes that contain weight vectors. Non-leaf nodes have links to other nodes.Unidirectional linksLeft child nodes with lesser threshold valuesRight child nodes with greater threshold values.

I. Ahmad - Windsor, Canada 24

K-means Clustering (contd.)

K-means Clustering Tree (KCT) – contd.KCT has a single root nodeLeaf nodes of KCT contain features for an image shape and a pointer to the images. Non-leaf nodes of the tree correspond to the other levels of the index. The nodes correspond to disk pages and the structure is designed so that search requires visiting only a small number of pages.

I. Ahmad - Windsor, Canada 25

K-means clustering (contd.)

KCT Tree creationAssume 2-D feature vector for each of 19 object as: { {-3.0, 3.0}, {-2.5, 3.0}, {-2.0, 2.0}, {-1.5, 2.0}, {-3.5, 1.5}, {-4.0, 1.0}, {-3.0, 0.5}, {-3.0, 0.0}, {-1.5, 0.5}, { 2.5,-0.5}, { 2.5,-1.0}, { 4.5,-1.0}, { 0.5,-1.5}, { 1.0,-2.0}, { 3.0,-2.0}, { 4.0,-2.0}, { 0.5,-2.5}, { 1.0,-2.5}, { 2.0,-3.0}}

I. Ahmad - Windsor, Canada 26

K-means clustering (contd.)

Same 2-dimensional numerical values Each object value will be in leaf node of KCT Object 1 2 3 4 5 6 7 8 9

X -3.0 2.5 -2.0 -1.5 -3.5 -4.0 -3.0 -3.0 -1.5

y 3.0 2.0 2.0 2.0 1.5 1.0 0.5 0.0 0.5

Object 10 11 12 13 14 15 16 17 18 19

2.5 2.5 4.5 0.5 1.0 3.0 4.0 0.5 1.0 2.0

-0.5 -1.0 -1.0 -1.5 -2.0 -2.0 -2.0 -2.5 -2.5

-3.0

I. Ahmad - Windsor, Canada 27

K-means clustering (contd.)

Corresponding KCT treeA

B C

D E F G

H I J

K

5 6 7 8 1 2

3 4

10 11

9 12

13

15 16 19

14 17 18: Non-leaf node

: leaf node

I. Ahmad - Windsor, Canada 28

K-means Clustering (contd.)

Insertion possibilities1. A leaf node is full and a new object is to be inserted

in that leaf node. In such case, an insertion results in overflow and, therefore, the node must split. As a result, a new non-leaf node and two leaf nodes are constructed and linked to that part of the tree. We also need to train the non-leaf node to get weight values.

2. When the node in which the object has to be inserted has only one object in it. In this case, object can be simply added into node without any additional cost.

I. Ahmad - Windsor, Canada 29

K-means Clustering (contd.)

Deletion possibilities1. When the sibling node is full-leaf node or non-leaf

node and we delete an object from full leaf node delete object from that node.

2. When the sibling node is leaf node with an object and we delete an object from full- leaf node delete object from that node, combine two leaf nodes, delete its parent non-leaf node, and connect remaining full-leaf node to deleted non-leaf node's parent node.

I. Ahmad - Windsor, Canada 30

K-means Clustering (contd.)

Deletion possibilities – contd.3. When the sibling node is full-leaf node and we

delete an object from leaf node with itself delete object from that node, delete its parent non-leaf node, and connect remaining full-leaf node to deleted non-leaf node's parent node.

4. When sibling node is non-leaf node and we delete an object from leaf node with one object Delete an object from that node, delete its parent non-leaf node, and connect deleted non-leaf node's child node to deleted non-leaf node's parent node.

I. Ahmad - Windsor, Canada 31

K-means Clustering (contd.)

Algorithm for retrieving images// Search an object with feature F using KCT1. b block containing root node of a KCT2. read block b3. while (b is not a leaf node of the KCT) do4. next recall Backpropagation using weights

in block b5. b next6. read block b7. search block b for the most similar object with

feature F // search leaf node8. if found then9. read index file block; display images with object

I. Ahmad - Windsor, Canada 32

Experimental Results

Experimental SetupEnvironment: PC with Microsoft Windows 98Language: C / C++Images

Normalized to grayscale and 128 x 128Grayscale images to binary images for chain-codes

Data set size100 original images5 variants involving translation, rotation and scaling.

I. Ahmad - Windsor, Canada 33

Experimental Results (contd.)

Sample image shapes and their seven moment invariants

a b c d

1 0.2136250052

0.2202478858

0.3404800296

0.6092586028

2 0.0007960114

0.0031965503

0.0836683672

0.2806988755

3 0.0061832619

0.0010778334

0.0009795464

0.0710062966

4 0.0000318972

0.0001001921

0.0004405216

0.0345626089

5 -0.000000014

1

-0.000000024

2

0.0000002893

0.0017118779

6 -0.000000892

1

0.0000047618

0.0001274228

0.0183105689

7 0.0000000000

0.0000000222

0.0000000000

0.0000339550

I. Ahmad - Windsor, Canada 34

Experimental Results (contd.)

A subset of grouping results using database images at the root level of KCT. Objects in top row occupy the left subtree while the bottom row objects become right subtree.

I. Ahmad - Windsor, Canada 35

Experimental Results (contd.)

Grouping results with database images at the 3rd level of KCT. The top row of objects forms the left subtree of KCT while the bottom row is the right subtree.

I. Ahmad - Windsor, Canada 36

Experimental Results (contd.)

Results of sample queries. Query shape is given in row 1, column 1 of each image while the retrieved images include the query shape.

I. Ahmad - Windsor, Canada 37

Experimental Results (contd.)

I. Ahmad - Windsor, Canada 38

ConclusionsPresented a moment invariants based image indexing scheme.Chain codes are used to reduce size of databaseIndexing is based on K-means clustering with k = 2 and Backpropagation to get weights for each node.Retrieval of images was based on finding the leaf node that includes similar images.Limitation: Small image collection and limited training data.

top related