1 list – dtsi – service réalité virtuelle, cognitique et interfaces sensorielles a conceptual...

LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles 1

A Conceptual Approach to A Conceptual Approach to Web Image RetrievalWeb Image Retrieval

Adrian PopescuAdrian PopescuGregory GrefenstetteGregory Grefenstette

CEA LIST, FranceCEA LIST, France

OntoImage 2008OntoImage 2008Marrakech, MoroccoMarrakech, Morocco

Contacts : [email protected]

[email protected]

LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles 2

Some facts

60% of Internet image queries include only one conceptPeople are stupid or lazy?

What are they looking for?Common nounsPlacesOther (famous) peopleProductsSexual content

LIST – DTSI – Service Réalité virtuelle, Cognitique et Interfaces sensorielles

3May 2008DTSI

Our Image Browsing Strategy

Introduce conceptual hierarchy into image search applications

Expand common words by more specific terms

Present results reflecting hierarchy Dual browsing: image or text query Exploit conceptual hierarchy and image

spidering to structure image base


4May 2008DTSI

Architecture of OLIVE

Olivepicture retrieval

framework employing a large scale conceptual hierarchy

to provide a dual access to Web images


5May 2008DTSI

Adapting WordNet

Concept rankingBased on Web counts

Relevance (C1) = webCount(C1, parent)

Web counts and WordNet structure Relevance (C1, C2) = webCount(C1, parent) *

dist(C1,C2)/senses(C1)

Example for dog (Web counts) pooch, pug, Newfoundland, basset (Web+WordNet) collie, basset, german shepherd, doberman

Query reformulationQuery concept + representative leaf

nodes


6May 2008DTSI

Knowledge for dog

Organism, canine, domestic animal, living thing, physical entity

Parents

Wolf, fox, hyena, wild dog, domestic cat, bitch, jackal

Related types

Poodle, corgi, spitz, cur, hunting dog, working dog, toy dog, Dalmatian, griffon

Narrower concepts

pooch, pug, Newfoundland, basset, beagle, cairn, Airedale, Doberman, German shepherd

Leaf Subtypes

yesPolysemic

Cat, monkey, elephant, tiger, lion, golden retriever, kittens, Chihuahua, snake

Expanded search

Rabbit, bird, Scooby DooRelated names

Puppy, Alaskan huskyNarrower

search


7May 2008DTSI

First response page for dog - Google Image


8May 2008DTSI

First response page for dog - Olive


9May 2008DTSI

First response page for Doberman - Olive


10May 2008DTSI

Visual similarity search for image of Doberman


11May 2008DTSI

Evaluation – Precision on first two pages


12May 2008DTSI

Precision over ten pages


13May 2008DTSI

Qualitative evaluation

Experiment setting 10 users compared Olive and Google

Image Questionnaire about: results structuring;

enriched interaction; automatic reformulation;

Positive reactions from the users to the new features introduced in Olive

The users also provided useful comments under the form of free text.


14May 2008DTSI

Conclusions

A semantic retrieval architecture can cover a significant part of Web picture queries.

Dual access to picture content, textual and visual

Simple but efficient solution to introduce CBIR in large scale architectures

With semantic structures, existing image indexes can be better used than in current applications

OLIVE outperforms Google Image, qualitatively and quantitatively

Future work: The extension of the hierarchy (e.g. using

Wikipedia)


15May 2008DTSI

A glimpse of the future


16May 2008DTSI

Advertising

Websitehttp://moromete.net

I am looking for workStarting November 2008

1 list – dtsi – service réalité virtuelle, cognitique et interfaces sensorielles a conceptual...

Documents