3D Model Indexing in Videos for Content-Based Retrieval via

X3D-Based Semantic Enrichment and Automated Reasoning

Leslie F. Sikos, Ph.D. Flinders University, Australia

Web3D 2017, 6 Jun 2017 Brisbane, QLD, Australia

3D Model Indexing in Videos

• The Semantic Gap in 3D Models and Scenes

• The X3D Vocabulary and Its OWL

Mappings

• 3D Modeling Ontology (3DMO)

• Structured 3D Model and Scene Annotation

• Scene Interpretation via Reasoning

3D Model Indexing in Videos for Content-Based Retrieval 2/23

Outline

Unstructured 3D Annotation

• Modeled or 3D scanned model in VRML or PLY

3D Model Indexing in Videos for Content-Based Retrieval 3/23

The Limitations of Popular Formats

Strengths

– Representation of

geometry and

properties

– Visualization

Weaknesses

– No semantics

– Automated processing

is difficult and

inefficient

Semistructured 3D Annotation

• 3D models with X3D XSD annotation

3D Model Indexing in Videos for Content-Based Retrieval 4/23

The X3D Vocabulary (XSD)

– Machine-processable

– Not machine-interpretable

Example. The permissible values of l

are real numbers in the range [0,1].

Semistructured 3D Annotation

• 3D models with X3D OWL annotation

3D Model Indexing in Videos for Content-Based Retrieval 5/23

The X3D Vocabulary in OWL

– Machine-processable

– Machine-interpretable

Example. The permissible values of directional

light intensity are real numbers in the range [0,1].

Structured 3D Annotation

• OntologyX3D (2004)

• RDFS mapping of X3D (2006)

• Common Shape Ontology (2007)

• Kinect Ontology (2013)

• 3D Modeling Ontology (2016)

3D Model Indexing in Videos for Content-Based Retrieval 6/23

A Brief History of 3D Ontologies

Structured 3D Annotation

OWL 2 mapping of entire X3D v3.3

complemented by terms from the 3D

modeling industry not covered by the

X3D standard (ISO/IEC 19775, 19776,

and 19777)

3D Model Indexing in Videos for Content-Based Retrieval 7/23

The 3D Modeling Ontology (3DMO)

Structured 3D Annotation

3D Model Indexing in Videos for Content-Based Retrieval 8/23

Application Potential

• Engineering

• CAD

• Medical

training

• Virtual museums

Structured 3D Annotation

3D Model Indexing in Videos for Content-Based Retrieval 9/23

Application Potential

• Engineering

• CAD

• Medical

training

• Virtual museums

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 10/23

3D Model Representation

Rich semantics with

3DMO terms for

content-based

retrieval and

interpretation via

reasoning

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 11/23

3D Model Representation

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 12/23

3D Model Representation

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 13/23

3D Model Representation

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 14/23

3D Model Representation

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 15/23

3D Model Representation in Video

• 3DMO terms

• Spatiotemporal

annotation

• Media

Fragment URIs

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 16/23

3D Model Representation in Video

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 17/23

3D Model Representation in Video

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 18/23

3D Model Representation in Video

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 19/23

3D Model Representation in Video

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 20/23

3D Model Representation in Video

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 21/23

3D Model Representation in Video

3D Animation Semantics

3D Model Indexing in Videos for Content-Based Retrieval 22/23

3D Model Representation in Video

3D Models Indexing in Videos Conclusions

• Semantic enrichment of 3D models adds rich

semantics to animations, and virtual reality and

augmented reality applications

• Automated annotation?

• Application to real-world videos?

23/23 3D Model Indexing in Videos for Content-Based Retrieval

Thank you.

Questions?