selection of the proper compact composite descriptor for improving content based image retrieval

SELECTION OF THE PROPER COMPACT COMPOSITE DESCRIPTOR FOR IMPROVING CONTENT BASED IMAGE RETRIEVAL

Savvas Chatzichristofis, Mathias Lux and Yiannis Boutalis

Department of Electrical & Computer Engineering Democritus University of Thrace – GreeceInstitute of Information Technology ‐ Klagenfurt University Klagenfurt, Austria

Signal Processing, Pattern Recognition and Applications SPPRA 2009

Presenter: Savvas A. Chatzichristofis

• Compact Composite Descriptors (CCD) are global image descriptors capturing more than one feature at the same time, in a very compact representation.

Natural ImagesCEDDFCTH

Artificial ImagesSpCL

Medical ImagesBTDH


Overview• In this paper we propose a combination of two

recently introduced CCDs (CEDD and FCTH) into a Joint Composite Descriptor (JCD).

• We further present a method for auto descriptor selection.

• Similar techniques were applied to select the most appropriate MPEG-7 descriptor, by extracting information from all the images of a dataset.


CEDD and FCTH Descriptors• The CEDD length is 54 bytes per image while FCTH

length is 72 bytes per image.

• The structure of these descriptors consists of n texture areas. In particular, each texture area is separated into 24 sub regions, with each sub region describing a color.

• CEDD and FCTH use the same color information, as it results from 2 fuzzy systems that map the colors of the image in a 24-color custom palette.


CEDD and FCTH Descriptors



0 1 2 3 4 5 6 7

CEDD

Linear

Non

Directional

Horizontal

Activation

VerticalA

ctivation

45 Degree

Diagonal

135 Degree

Diagonal

- -

FCTH

Linear L

owE

nergy

Horizontal L

owE

nergy

Vertical Low

Energy

Both D

irectionsL

ow E

nergy

Linear

High E

nergy

Horizontal

High E

nergy

Vertical High

Energy

Both D

irectionsH

igh Energy



WANG UCID NISTERCCDCEDD 0.25283 0.28234 0.11297FCTH 0.27369 0.28737 0.09463MPEG-7DCD MPHSM 0.39460 - -DCD QHDM 0.54680 - -SCD 0.35520 0.46665 0.36365CLD 0.40000 0.43216 0.2292CSD 0.32460 - -EHD 0.50890 0.46061 0.3332HTD 0.70540 - -


Joint Composite Descriptor (JCD)• Based on the fact that the color information

given by the 2 descriptors comes from the same fuzzy system, we can assume that joining the descriptors will result in the combining of texture areas carried by each descriptor.

• JCD is made up of 7 texture areas, with each area made up of 24 sub regions that correspond to color areas.


Joint Composite Descriptor (JCD)• The texture areas are as follows:

▫ JCD(0) Linear Area▫ JCD(1) Horizontal Activation▫ JCD(2) 45 Degrees Activation▫ JCD(3) Vertical Activation▫ JCD(4) 135 Degrees Activation▫ JCD(5) Horizontal and Vertical Activation▫ JCD(6) Non directional Activation


Descriptor Implementation• We model the problem as follows:• CEDD and FCTH be available for an image. The

indicator m symbolises the bin of the color of each descriptor.

• The indicators n and n’ determine the texture area for the CEDD and FCTH respectively

CONTENT BASED MEDICAL IMAGE INDEXING AND RETRIEVAL USING A FUZZY COMPACT COMPOSITE DESCRIPTOR

[0, 23]m∈

[0,5]n∈ ' [0,7]n ∈


Descriptor Implementation• Each descriptor can be described in the

following way:

'( ) , ( )m mn nCEDD j FCTH j

The algorithm for the Joint Composite Descriptor can be analysed as follows:

54( ) (2 24 5) (53)CEDD j bin bin= × + =


Descriptor Implementation


Auto Descriptor Selection (ADS)

• (i) The descriptor for search is chosen based on the query image.

• (ii) The most appropriate descriptor is chosen at similarity assessment time, so within a single query the chosen descriptor may be different for different image pairs.


Auto Descriptor Selection (ADS)• In retrieval scenarios a

combination of different feature spaces within a single query is often not possible.

• Experiments on the Wang data set have shown that with normalized similarities (mean of 0 and standard derivation of 1) distributions are similar enough to be combined.


Distribution of (a) CEDD, (b) FCTH and (c) JCD similarities / Wang 1000 image

database.

Auto Descriptor Selection (ADS)• Given that the color information in all two

descriptors is the same, the factor that will determine the suitability and capability of each descriptor is mainly found in the texture information.

• The system that determines the most appropriate descriptor is a Mamdani fuzzy system of three inputs and one fuzzy output. The centroid method was used to defuzzify the output of the Mamdanimodel.


Criterion 1: Maximum amount of information.

• The first criterion shows which CCD contains the largest quantity of information.


Criterion 2: Percentage of information in non-uniform texture areas.

• The most appropriate descriptor is the one that contains the smallest percentage of non uniform image blocks.


Criterion 3: The percentage of information in texture areas.

• The third criterion considers the most appropriate descriptor to be the one that has the smallest percentage of image blocks present in linear areas.


Experiments• The proposed methods have been implemented and are available as

open source libraries under GNU - General public License (GPL) in the image retrieval system img(Rummager) the on line application img(Anaktisi) and image retrieval library LIRe.


http://savvash.blogspot.com/2008/06/imgrummager-in-now-available-for.html

http://savvash.blogspot.com/2008/07/imganaktisi-update.html

•• CEDD• FCTH• JCD• Ranking


For use of multiple different descriptors within one query, the ADS unit also needs to normalize the similarities based on their distribution. Based on

experiments we used the normalization values given in paper.

Experiments• To evaluate the performance of the proposed methods, the objective measure called ANMRR is used.

WANG UCID NISTERCEDD 0.25283 0.28234 0.11297

FCTH 0.27369 0.28737 0.09463

JCD 0.25606 0.26832 0.085486

ADSBased on Query descriptor 0.24948 0.27952 0.09291

ADSBased on Pair wise descriptor 0.24876 0.27722 0.09291


Conclusions• JCD and ADS methods are not suggested to improve the

retrieval procedure.

• The goal is to approach the best ANMRR that would result from CEDD and FCTH.

• Nevertheless, the new JCD shows an increase in retrieval performance.

• The methods for automatic selection of the most appropriate descriptor (ASD) for retrieval increases retrieval performance in all 3 experiments.


Download the img(Rummager) application from http://www.img-rummager.com

Thank YouΕυχαριστώ Πολύ


selection of the proper compact composite descriptor for improving content based image retrieval

Technology

image retrieval cedd

chosen descriptor

query image

appropriate descriptor

image retrieval overview

image distributions

global image descriptors

different image pairs