rhizomik semantic integration and retrieval of multimedia metadata roberto garcía and universitat...

Rhizomik

Semantic Integration and Retrieval of Multimedia Metadata

Roberto García and Universitat de Lleida, Lleida, Spain

Òscar CelmaUniversitat Pompeu Fabra, Barcelona, Spain


Rhizomik

Contents

Introduction Ontological framework Integration framework Integration & Retrieval Scenario Conclusions Future Work


Rhizomik

Introduction

Digital Media requires Multimedia Metadata. Main requirements:

Great amounts of metadata Low-level to user-friendly metadata Interoperability

Proposal: semantic metadata and an ontological framework for integration.


Rhizomik

Contents



Rhizomik

Ontological Framework

Multimedia ontology. Profit from non-semantic initiative MPEG-7. XML Semantic Web:

XSD2OWL: schema to ontology. XML2RDF: XML instance data to RDF instances.

Complete MPEG-7 OWL Ontology


Rhizomik

MPEG-7: Multimedia Content Description Standard content-based description for audiovisual information content searching, filtering and identification Descriptors:

low-level features structure semantic


modelscollectionscreation...


Rhizomik


Other semantic multimedia metadata initiatives: Hunter02: not complete, RDF+DAML. Tsinaraki04: not complete, semantic part of MDS. Troncy03: not complete, from an ontology to MPEG-7.

Direct benefits: XQuery vs. Semantic Query, example next…


Rhizomik


One XQuery for each type

vs.

One semantic query for supertype.

SegmentType hierarchy.


Rhizomik

Ontological Framework ReDeFer

XSD2OWLMappings:

owl:intersectionOfowl:unionOf

sequencechoice

owl:maxCardinalityowl:minCardinality

@maxOccurs@minOccurs

rdfs:subClassOfextension@base|restriction@base

owl:RestrictioncomplexType//element

owl:ClasscomplexType|group|attributeGroup

rdfs:rangeelement@type

rdfs:subPropertyOfelement@substitutionGroup

rdf:Propertyowl:DatatypePropertyowl:ObjectProperty

element|attribute

OWLXML Schema


Rhizomik

Contents



Rhizomik

Integration Framework

MPEG-7 OWL Ontology as integration base for: Semantic metadata: align ontologies with MPEG-7

ontology. MusicBrainz (http://musicbrainz.org) Simac (http://www.semanticaudio.org) Music Vocabulary (http://www.kanzaki.com/ns/music) Etc.

XML2RDF: RDF from MPEG-7 XML instance metadata.


Rhizomik

Integration Framework. Architecture

Signal ProcessingAudio

Video

MPEG-7 XML

Content-based metadata

XML2RDF

Another Ontology

RDF

RDFContext-based

metadata

MPEG-7 Ontology

XML

Integration

Retrieval

Higher-level metadata

DL Classifier

SWRL Engine

XSD2OWL

XMLSchemas: MPEG-7...

RDFS / OWL: MusicBrainz...


Rhizomik


MusicBrainz to MPEG-7 OWL mappings:

musicbrainz:duration ≡ mpeg7:MediaDuration

musicbrainz:sortName mpeg7:Name

dc:title mpeg7:Title

dc:author mpeg7:Creator

musicbrainz:Track mpeg7:AudioSegmentType

musicbrainz:Album mpeg7:CollectionType

musicbrainz:Artist mpeg7:CreatorType


Rhizomik


Simac to MPEG-7 OWL mappings:

simac:meter ≡ mpeg7:Meter

simac:tempo ≡ mpeg7:Beat

simac:key ≡ mpeg7:Key

simac:mode ≡ mpeg7:Scale

simac:Descriptor ≡ mpeg7:AudioDSType

simac:duration ≡ mpeg7:MediaDuration

simac:title ≡ mpeg7:Title

simac:Track mpeg7:AudioSegmentType simac:name ≡ mpeg7:GivenName

simac:Artist mpeg7:CreatorType


Rhizomik


Music Vocabulary ontology to MPEG-7 OWL mappings:

music:meter ≡ mpeg7:Meter

music:key ≡ mpeg7:Key

music:Artist mpeg7:CreatorType

music:sections ≡ mpeg7:AudioSegment

music:Music_Unit mpeg7:AudioSegmentType


Rhizomik

Integration Framework. Architecture


Video

MPEG-7 XML


XML2RDF

Another Ontology

RDF

RDFContext-based

metadata

MPEG-7 Ontology

XML

Integration

Retrieval


DL Classifier

SWRL Engine

XSD2OWL




Rhizomik


XML2RDF: XML tree RDF graph.

Deduce blank node types from XSD2OWL ontologies restrictions.

Root

elem elemelem

elem elem

Empty Text

elemattr

Empty Text Text Text

Blank nodes

rdf:Properties

XML tree model RDF graph model


Rhizomik


• XML2RDF example

mpeg7:DescriptionMetadataType

1.0mpeg7:Confidence

mpeg7:PlaceTypempeg7:CreationLocation

2000-10-10T19:45:00+09:00mpeg7:CreationTime

mpeg7:CreatorTypempeg7:Creator

mpeg7:CreationToolTypempeg7:Instrument

2001-09-20T03:20:25+09:00

mpeg7:LastUpdate

completeDescriptionExample

mpeg7:PrivateIdentifier

098f2470-bae0-11cd-b579-08002b30bfeb

mpeg7:PublicIdentifier

mpeg7:RightsType

mpeg7:Rights

1.1

mpeg7:Version

Tokyompeg7:AdministrativeUnit

jpmpeg7:Region

mpeg7:AgentType, mpeg7:PersonTypempeg7:Agent

mpeg7:ControlledTermUseTypempeg7:Role

mpeg7:Setting

mpeg7:TermUseType

mpeg7:Tool

RID#mpeg7:RightsID

mpeg7:Name

mpeg7:Namempeg7:href

mpeg7:name

mpeg7:value

mpeg7:Name

Creator

creatorCS

Shibata Yoshiaki

sensitivity

0.5

Wizzo Extracto ver. 2


Rhizomik

Contents



Rhizomik

Integration & Retrieval Scenario

Low-level descriptors from audio track signal:

Track editorial metadata:

<http://example.org/track#1> a simac:Track;simac:tempo “122”;musicbrainz:trmid “e3c41bc1-4fdc-4ccd-a471-243a0596518f”.

<http://example.org/track#2> a musicbrainz:Track;dc:title “Blowin’ in the wind”;dc:author [musicbrainz:sortName “Bob Dylan”];musicbrainz:trmid “e3c41bc1-4fdc-4ccd-a471-243a0596518f”.


Rhizomik

Integration & Retrieval Scenario Generic MPEG-7 based integration rules.

Example: audio track with the same fingerprint are the same individual, their metadata is merged.

mpeg7:AudioType(track1) mpeg7:AudioType(track2) musicbrainz:trmid(track1, trm1) musicbrainz:trmid(track2, trm2) (trm1 = trm2) owl:sameIndividualAs(track1, track2)


Rhizomik



Video

MPEG-7 XML


XML2RDF

Another Ontology

RDF

RDFContext-based

metadata

MPEG-7 Ontology

XML

Integration

Retrieval


DL Classifier

SWRL Engine

XSD2OWL




Rhizomik

Integration & Retrieval Scenario Transparent retrieval, MPEG-7 based queries.

RDQL example:

SELECT ?titleWHERE (?track,<rdf:type>,mpg7:AudioSegmentType), (?track,<mpeg7:Title>,?title),

(?track,<mpeg7:Creator>,?author), (?author,<mpeg7:Name>,“Bob Dylan”),(?track,<mpeg7:Beat>,?tempo)

AND (?tempo >= 120)USING mpeg7 FOR

<http://rhizomik.upf.edu/ontologies/2005/03/Mpeg7-2001.owl#>


Rhizomik

Contents



Rhizomik



Video

MPEG-7 XML


XML2RDF

Another Ontology

RDF

RDFContext-based

metadata

MPEG-7 Ontology

XML

Integration

Retrieval


DL Classifier

SWRL Engine

XSD2OWL




Rhizomik

Conclusions

MPEG-7 OWL ontology, an upper-level multimedia ontology for: Semantic metadata integration

align ontologies to MPEG-7 OWL. XML metadata integration

XML2RDF. Semantic queries for retrieval

generic queries based on MPEG-7.


Rhizomik

Contents



Rhizomik

Future Work Semantics-based recommendation systems

http://foafing-the-music.iua.upf.edu http://rhizomik.net

Infer user-friendly metadata from low-level descriptors.

mpeg7:AudioType(track) mpeg7:Beat(track, t) (t > 120) mpeg7:Loudness(track, l) (l > 0.9) genres:Rock(track)danceability(track, “high”)

Rhizomik

Thank you for your attention

Contact:

[email protected]

[email protected]

rhizomik semantic integration and retrieval of multimedia metadata roberto garcía and universitat...

Documents

rhizomik semantic integration

semantic metadata

integration base

semantic query

owl ontology slide

title mpeg7

key mpeg7

meter mpeg7