![Page 1: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/1.jpg)
Ontology Based Object Learning and Recognition
PhD Defence
14/12/2005
Supervised by Monique Thonnat
Nicolas MAILLOTOrion team
INRIA Sophia Antipolis
![Page 2: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/2.jpg)
N. Maillot 2/54
• Introduction• State of the Art• Knowledge Acquisition• Visual Concept Learning• Object Categorization• Results• Conclusion
Outline
![Page 3: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/3.jpg)
N. Maillot 3/54
Introduction
• Context: Semantic image interpretation• Goal: Object recognition • More precisely: object categorization (i.e. finding the category of an object) and not object identification (i.e. recognition of an individual)
• Approach: Cognitive vision techniques [ECVision Roadmap 04]
• Mixing knowledge representation, machine learning, image processing and reasoning techniques
![Page 4: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/4.jpg)
N. Maillot 4/54
Introduction: Semantic Image Interpretation
Oslo Accords (1993)
• Semantics is not inside the image:
handshake agreement
Need of a priori knowledge in international politics
![Page 5: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/5.jpg)
N. Maillot 5/54
Introduction: object categorization
• Assigning a category (e.g. Aircraft, Galaxy) to a region of the image
• Categories are discrete entities characterized by properties shared by their members
Aircraft
![Page 6: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/6.jpg)
N. Maillot 6/54
Introduction: Goal
• Issues:• Knowledge acquisition• Semantic gap• Use of acquired knowledge for performing object categorization
• Goal: Enabling experts to build object categorization systems dedicated to his/her domain of interest (e.g. biology)
• Restricted scope:• One main object per image • Need of a well-defined expertise
![Page 7: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/7.jpg)
N. Maillot 7/54
Introduction: Proposed Approach
• Decomposition of the object categorization problem in three levels of abstraction:
High-Level Interpretation
Mapping
Image Processing
Domain knowledge
Knowledge about the mapping between domain knowledge and image processing knowledge
![Page 8: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/8.jpg)
N. Maillot 8/54
Introduction: Proposed Approach
• Use of ontological engineering combined with machine learning techniques
Reduction of the knowledge acquisition problem and of the semantic gap
Performing categorization as experts do
![Page 9: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/9.jpg)
N. Maillot 9/54
• Introduction• State of the Art• Knowledge Acquisition• Visual Concept Learning• Object Categorization• Results• Conclusion
Outline
![Page 10: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/10.jpg)
N. Maillot 10/54
State of the Art : Object Recognition
• [Brooks83] Object modeling by ribbons. Geometric reasoning
• [Havaldar96] use of qualitative geometric relationships (e.g. proximity, symmetry)
• [Basri96] Combination of alignment method with recognition by prototypes
• [Sangineto03] Recognition based on the shape invariants of object categories
Geometric model alignment
• Geometric Methods
![Page 11: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/11.jpg)
N. Maillot 11/54
State of the Art : Object Recognition
• Appearance-Based Methods
Implicit objects modelsUse of multiple views • [Swain and Ballard 91] Objects represented by color histograms
• [Schmid97] Local features. Introduction of a voting algorithm
• [Schiele00] Receptive field histograms for approximating the local appearance
• [Fergus03] Local features. Objects modeled as constellations of parts
![Page 12: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/12.jpg)
N. Maillot 12/54
State of the Art : Object Recognition
• Knowledge-Based Methods [Crevier97]
• [Draper89] Blackboard architecture. Schemas (frames + procedures). Hypothesis generation/verification.
• [Matsuyama90] Three expert systems. Frames + rules. Both model driven and data driven. hypotheses generation/verification.
• [Hudelot05] Cooperation between three knowledge-based systems (Frames + rules). Data management functionalities.
![Page 13: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/13.jpg)
N. Maillot 13/54
State of the Art : Object Recognition
• Summary:• Geometric Methods
+ Strong theoretical foundations- identification of individuals and not categorization- Reliable Extraction of geometric primitives is very difficult
• Appearance-Based Methods+ Effective- Need of large number of samples- Lack of explicitness
• Knowledge-Based Methods+ Explicit+ Separation between knowledge and reasoning
- Knowledge acquisition bottleneck (mapping knowledge is difficult to acquire)
![Page 14: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/14.jpg)
N. Maillot 14/54
• Introduction• State of the Art• Knowledge Acquisition• Visual Concept Learning• Object Categorization• Results• Conclusion
![Page 15: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/15.jpg)
N. Maillot 15/54
Domain Expert
Visual Concept
Ontology
Knowledge Acquisition
Knowledge Base
Knowledge acquisition guided by a visual concept ontology (i.e geometry, texture, color) to
describe the objects of the domain.
Knowledge Acquisition
![Page 16: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/16.jpg)
N. Maillot 16/54
Knowledge Acquisition
• Ontology • Definition: An explicit specification of a conceptualization [Gruber93]
• Composed of:• A set of concepts• A set of relations between concepts• A set of axioms (e.g. transitivity, reflexivity)
• Ontological Commitment [Bachimont2000] Shared reference to align with
![Page 17: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/17.jpg)
N. Maillot 17/54
Knowledge Acquisition
• Visual Concept Ontology• 144 concepts :
• spatial concepts (geometry, size, position, orientation)
• color concepts (hue, brightness, saturation)
• texture concepts (pattern, contrast, repartition)
Object classes are described by visual concepts
![Page 18: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/18.jpg)
N. Maillot 18/54
Texture
Repartition Pattern
Repetitive Random Regular Oriented Granulated Coarse Complex
Visual concept ontology content: some texture concepts
Knowledge Acquisition
Based on cognitive experiments [Bhushan et al 97]
![Page 19: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/19.jpg)
N. Maillot 19/54
Subpart Tree
Poaceae :
• Circular Shape
• Granulated Texture
• Pink Color
Poaceae
Pollen
Pore Cytoplasm
Pore:
• Subpart of Poaceae
• Elliptic Shape
• Small Size
Domain knowledge described using visual concept ontology
Knowledge Acquisition
![Page 20: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/20.jpg)
N. Maillot 20/54
• Knowledge Formalization• Domain class hierarchy: from general to specialized classes
• Domain Partonomy: subparts linked to domain classes
• Class: a category (e.g. aircraft, pollen grain) described by visual concepts
• Representation by frames with slots
Knowledge Acquisition
![Page 21: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/21.jpg)
N. Maillot 21/54
Knowledge Acquisition
Visual Concept Examples Numerical Features
ShapePolygonal, Straight SIFT Features [Lowe 99]
TextureGranulated, Smooth
Gabor Features [Manjunath 96]Co-Occurrence Matrices
ColorBlue, Bright, Dark
HistogramsColor Coherence Vectors [Pass96]
Each visual concept is associated with numerical features:
![Page 22: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/22.jpg)
N. Maillot 22/54
Knowledge Acquisition
• Importance of acquisition context• Visual description is valid for an image acquisition context
Acquisition Context
Point of View Sensor
RearView
FrontView
ProfileView
MicroscopeCamera
CCD Camera IR Camera
![Page 23: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/23.jpg)
N. Maillot 23/54
Domain
class
hierarchy
Subparts
hierarchy
Ontology
driven
description
Image
samples
management
Knowledge Acquisition
![Page 24: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/24.jpg)
N. Maillot 24/54
Poaceae
Composition Link
Specialization Link
Pollen
Grain
Pori
NonAperturedPollen
Cupressaceae
Pori of Poaceae
Pori of Parietaria
Knowledge Base(18 domain classes + 17 visual concepts)
Cytoplasm OfCupressaceae
Pollen with Pori
Pollen with Pori and Colpi
Apertured
PollenParietariaOlea
Colpi Colpi of Olea
Knowledge Acquisition
Context:
Sensor: Microscope
Magnification: 60
Dye: Fuchsin
![Page 25: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/25.jpg)
N. Maillot 25/54
High-Level Interpretation
Mapping
Image Processing
Domain knowledge
Completely Acquired
Mapping Knowledge
Partially Acquired
Knowledge Acquisition
• Conclusion:
![Page 26: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/26.jpg)
N. Maillot 26/54
• Introduction• State of the Art• Knowledge Acquisition• Visual Concept Learning• Object Categorization• Results• Conclusion
Talk Overview
![Page 27: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/27.jpg)
N. Maillot 27/54
Visual Concept Learning
• Visual Concept Learning• Goal: Producing visual concept detectors• Why: Mapping knowledge is difficult to acquire
• How: Training of Support Vector Machines (SVM) with annotated samples
Granulated
Texture
Detector
GranulatedTexture
Confidence=0.8
![Page 28: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/28.jpg)
N. Maillot 28/54
• Image Sample Segmentation and Annotation using visual concepts
• Three Approaches:• Manual approach• Use of 3-D models• Weakly-supervised approach
Visual Concept Learning
![Page 29: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/29.jpg)
N. Maillot 29/54
Selection of an image sample of Poaceae object
Interactive selection of region of interest with a drawing tool
• Image Sample Segmentation and Annotation: Manual Approach
Annotation of selected region by visual
concepts:
- Pink- Large- Circular
Visual Concept Learning
![Page 30: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/30.jpg)
N. Maillot 30/54
Visual Concept Learning
• Image Sample Segmentation and Annotation: Use of 3-D Models (meshes)
![Page 31: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/31.jpg)
N. Maillot 31/54
Image training set
Automatic Segmentation
Feature
Extraction
Clustering
(k-means)
Cluster Visualization and Annotation
Annotated
Clusters
Visual concept Ontology
Visual Concept Learning
• Image Sample Segmentation and Annotation: weakly-supervised approach
![Page 32: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/32.jpg)
N. Maillot 32/54
Automatic Segmentation
Size Computation
k-means
Small
Cluster Visualization and Annotation
• Example: clustering for visual concept category Size
Visual concept Ontology
Visual Concept Learning
Image Training Set
…
… …
… …Large
![Page 33: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/33.jpg)
N. Maillot 33/54
• Learning (for each visual concept C used during knowledge acquisition)
Get
Positive
and
Negative
Samples Of
C
Visual
Concept
Detector
SVM
Training
Feature
Extraction
And Selection
Annotated
Regions
Visual Concept Learning
SVM based on Radial Basis Function Kernels
![Page 34: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/34.jpg)
N. Maillot 34/54
Granulated
Texture
Detector
LDA SVM
• Example: Learn the visual concept Granulated Texture
Visual concept detectors are used to complete the mapping knowledge
Gabor
Filter
Get Positive
and Negative Samples
of Concept Granulated Texture
Annotated
Regions
Visual Concept Learning
![Page 35: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/35.jpg)
N. Maillot 35/54
• Introduction• State of the Art• Knowledge Acquisition• Visual Concept Learning• Object Categorization• Results• Conclusion
Talk Overview
![Page 36: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/36.jpg)
N. Maillot 36/54
Object Categorization
• Object categorization based on:• Acquired knowledge (domain knowledge + mapping knowledge)
• Visual concept detectors• Mechanism: Hypothesis Generation/Verification
Object Categorization
Input
Image
Class +
Visual
Description
![Page 37: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/37.jpg)
N. Maillot 37/54
• Algorithm: Hierarchical exploration of object classesFor each class of the class hierarchy (from root
class)
1. Hypothesis generation: generation of a set of hypothetic visual concepts
2. Visual detection of the hypothetic visual concepts in the segmented image
3. Recursion on sub-parts
4. Hypothesis verification: object/class matching w.r.t. a matching threshold
5. If the class is verified then consider sub-classes
Object Categorization
![Page 38: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/38.jpg)
N. Maillot 38/54
• Matching (matching threshold=0.5)
Feature
Extraction
Automatic
Segmentation
Poaceae :
• Circular Shape
• Granulated Texture
• Pink Hue
Current Hypothesis:
CircularShape
Detector
GranulatedTextureDetector
Pink HueDetector
0.63
Σ
Object Categorization
0.5
0.6
0.8
(0.5+0.6+0.8)/3
0.63>0.5 : hypothesis verified
?
![Page 39: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/39.jpg)
N. Maillot 39/54
Object
CategorizationKnowledge
Base
Automatic
Segmentation
Feature
Extraction
Input Image
Poaceae 0.63Circular 0.5
Pink 0.8
Granulated 0.6
Object Categorization
Visual Concept Detectors
Mapping
![Page 40: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/40.jpg)
N. Maillot 40/54
• Introduction• State of the Art• Knowledge Acquisition• Visual Concept Learning• Object Categorization• Results• Conclusion
Talk Overview
![Page 41: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/41.jpg)
N. Maillot 41/54
Results
• Application: Semantic image indexing and retrieval
• Domain: Transport Vehicles (aircrafts, motorbikes, cars) in their environment
• Goal: Enabling Retrieval/Indexing by concept • User-friendliness• Efficiency: no need to store pre-computed feature vectors
• Issue: trade-off between semantic richness and amount of work needed to build semantic indexing and retrieval systems
![Page 42: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/42.jpg)
N. Maillot 42/54
Results
• Semantic Indexing
ImageDatabase Object Categorization
Indexed Images
Use of categorization results as index for images
Indexing time: 1 sec for a 600x400 image on a Intel Pentium IV 3.06Ghz
![Page 43: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/43.jpg)
N. Maillot 43/54
Results
• Query by concept (opposed to query by example):
Indexed Images
Semantic Query:Object Class /
Object Description
• Example of semantic queries: “Aircraft ”, “Gray Aircraft and Blue Sky ”
Retrieved Images
Retrieval
![Page 44: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/44.jpg)
N. Maillot 44/54
Results
• [Fauqueur03] Retrieval/Indexing based on region templates
• [Town04] Supervised learning used for mapping image data to a domain ontology
• [Mezaris04] Querying based on an object ontology (color, position, size, shape). Machine learning and user feedback are used for improving system efficiency
No approach combines weak supervision with a rich high-level knowledge layer
![Page 45: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/45.jpg)
N. Maillot 45/54
Results
Composition Link
Specialization Link
Outdoor
Scene
Transport
Vehicles
Background
Sky
Aircraft
Tarmac
Grass
Sea
Car
Motorbike
Knowledge acquisition
![Page 46: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/46.jpg)
N. Maillot 46/54
Results
Knowledge acquisitionHue Brightn
essGeometry Position Pattern
Aircraft Polygonal Center
Car Polygonal
Center
Motorbike Polygonal
Center
Sky Blue Grey
Dark Light
Top Smooth
Tarmac Grey Black
Bottom Uniform
Grass Green Bottom Uniform
![Page 47: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/47.jpg)
N. Maillot 47/54
Results
• Use of the Caltech image database• Training Set : 850 images (aircraft, car, motorbike)• Test Set : 2000 images (contains 300 images of each class and 800 background images)
Background images
Images containing objects of interest
![Page 48: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/48.jpg)
N. Maillot 48/54
Results: Caltech Database on 3 object classes
• Precision/Recall curve
Precision=n/ARecall=n/N
n: number of relevant retrieved images
A: number of retrieved images
N: number of relevant images
![Page 49: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/49.jpg)
N. Maillot 49/54
• Introduction• State of the Art• Knowledge Acquisition• Visual Concept Learning• Object Categorization• Results• Conclusion
Talk Overview
![Page 50: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/50.jpg)
N. Maillot 50/54
Conclusion
• Approach: Use of ontological engineering combined with machine learning techniques
• Three phases:
1. Knowledge acquisition
2. Visual concept learning
3. Object Categorization
• Applications:
• Semantic image indexing and retrieval
• Knowledge acquisition in the domain of palynology
![Page 51: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/51.jpg)
N. Maillot 51/54
• Contributions:• An extensible and reusable visual concept ontology
[maillot04]144 visual concepts (color, texture and spatial concepts)
• Original combination of knowledge and learning techniques for explicit domain knowledge elicitation and automatic visual concept detector learning [maillot04] In particular, no inference rules to define for mapping
• A weakly-supervised annotation approach [maillot05]enables easy image sample annotation
• An object categorization algorithm [maillot05]reproduces the way expert reasonindependent of the application domain
Conclusion
![Page 52: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/52.jpg)
N. Maillot 52/54
Conclusion
• Strengths and Weaknesses:+ Elicitation of domain knowledge+ Reduction of the knowledge acquisition bottleneck
+ Reduction of the semantic gap
- Spatial reasoning missing- Image processing algorithms not adaptive- Geometric models not used during categorization
![Page 53: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/53.jpg)
N. Maillot 53/54
Future Works
• Short-term:• Integration in a cognitive vision platform [Hudelot
05]• data management• top-down and bottom-up mechanisms• spatial reasoning
• Learning for adaptive image segmentation [Martin et al. 06]
• Long-term:• Extension to video content (e.g. temporal concepts)• Dynamic knowledge bases (no closed-world assumption)
• Use of 3-D models for categorization
![Page 54: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/54.jpg)
N. Maillot 54/54
Thank you for your attention
![Page 55: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/55.jpg)
N. Maillot
Publications
[1] Ontology Based Complex Object Recognition N. Maillot, M. ThonnatImage and Vision Computing JournalUnder Minor Revision
[2] Towards Ontology Based Cognitive Vision (Long Version) N. Maillot, M. Thonnat, A. BoucherMachine Vision and Applications Journal (MVA) Springer-Verlag Heidelberg, December 2004, 16(1), pp 33--40
[3] A Weakly Supervised Approach for Semantic Image Indexing and Retrieval N. Maillot, M. ThonnatInternational Conference on Image and Video Retrieval (CIVR 2005) Singapore, 20-22 July 2005
[4] Ontology Based Object Learning and Recognition : Application to Image Retrieval N. Maillot, M. Thonnat, C.Hudelot16th IEEE International Conference on Tools for Artificial Intelligence (ICTAI 2004)Boca Raton, Florida, 15-17 November 2004
[5] Towards Ontology Based Cognitive Vision N. Maillot, M. Thonnat, A. BoucherThird International Conference on Computer Vision Systems (ICVS 2003)Graz, Austria, April 2003, LNCS 2626, pp.44-53, Springer-Verlag Berlin Heidelberg 2003
![Page 56: Ontology Based Object Learning and Recognition](https://reader033.vdocument.in/reader033/viewer/2022061220/54bc9ccb4a79590c748b456c/html5/thumbnails/56.jpg)
N. Maillot 56/54
Proposed Approach
Data Management Knowledge Base
of Visual Concepts and Data
Data Management Engine
Interpretation Knowledge Base
of Application Domainand Visual Concepts
Interpretation Engine
Program SupervisionLibrary of
vision programs
Knowledge Base of Program Utilization
Program Supervision Engine
CurrentImage
Interpretation
ObjectHypotheses
Image Processing
Request
Numerical data
Image description
Visual Concept
Ontology
Cognitive vision platform [Hudelot 05]