extracting an ontology of portrayable objects from wordnet

MUSCLE/ImageCLEF workshop 2005

Extracting an Ontology of Portrayable Objects from

WordNet

Atomic Energy Agency of France (CEA)LIC2M (Multilingual Multimedia Knowledge Engineering Laboratory)BP6, 18 Route du Panorama 92265, Fontenay aux Roses, France

sveta_zinger@yahoo.com, {milletc,mathieub,grefenstetteg,hedep,moellicp}@zoe.cea.fr

S. Zinger, C. Millet, B. Mathieu, G. Grefenstette, P. Hède, P.-A. Moëllic

2MUSCLE/ImageCLEF workshop

2005

Goal:creation of a large-scale image ontology

• WordNet lexical resourses

• Image collections acquisition through web-based image mining

3MUSCLE/ImageCLEF workshop

2005

Building a large-scale image ontology for object recognition:

list of portrayable objects

WordNet lexical resources

basis of ontology

4MUSCLE/ImageCLEF workshop

2005

Building a large-scale image ontology for object recognition:

visual features

semantic filtering

clustering

classification

web-based image mining

large-scale visual dictionary

5MUSCLE/ImageCLEF workshop

2005

Pruning approach to WordNetENTITY

has a distinct separate existence (living or nonliving)

OBJECT physical object (a tangible a visible entity)

simplifying connections

selecting branches

object living thing life wildlifeobject living thing plant … tree tree of knowledgeobject artifact creation classic

deleted

6MUSCLE/ImageCLEF workshop

2005

Extraction from top-level ontology of portrayable objects

ENTITY

object

living thing

natural object artifact floater

organism celestial body

rock

article commodity

consumer goods

102 nodes in total

7MUSCLE/ImageCLEF workshop

2005

VIKA (Visual Kataloguer)

indexing (PIRIA – LIC2M)

clustering (shared nearest neighbor)

visualisation of clusters

web-image search engine(Alltheweb)

8MUSCLE/ImageCLEF workshop

2005

List of portrayable objects(24000 items)

queries to the web (e.g. google image)

VIKA (Visual Kataloguer)

9MUSCLE/ImageCLEF workshop

2005

Composition of queries:

upper node +word identifying portrayable object

Example of queries:

• kino tree• red sandalwood tree • carib wood tree• Japanese pagoda tree• palm tree• ...

Japanese pagoda

buildings

Japanese pagoda tree

trees

10MUSCLE/ImageCLEF workshop

2005

VIKA (Visual Kataloguer)

11MUSCLE/ImageCLEF workshop

2005

Web-image search at present Desired results

query « chair»

12MUSCLE/ImageCLEF workshop

2005

Future work:• face detection (adaboost learning) – to filter web-image search results semantically (images of objects without people)

• testing VIKA system performances

• automatic cluster classification – ignoring irrelevant clusters

• introducing new connections to the ontology: vision principles (scale), co-occurrence rules

13MUSCLE/ImageCLEF workshop

2005

Future workVision principles (scale)

14MUSCLE/ImageCLEF workshop

2005

Future workCo-occurrence rules