icme2011 industrial poster
Post on 01-Jul-2015
271 Views
Preview:
TRANSCRIPT
A web system for ontology-based multimedia annotation,browsing and search
M. Bertini, G. Becchi, A. Del Bimbo, A. Ferracani, D. Pezzatini
University of Florence - MICC
System architecture
The system provides a service-oriented architecture(SOA) that allows for multiple viewpoints of multimedia datainside repositories, providing better ways to reuse, repurposeand share rich media.
The analysis layer is responsible for extracting low levelfeatures and semantic annotations from media files,through a series of processing pipelines that can be executedin a cloud of servers, orchestrated by apposite services.
Annotation of visual content is performed using theBag-of-Visual-Words paradigm, based on a fusion of MSER,SURF and SIFT features and the Pyramid Matching Kernel.Audio annotation is based on a fusion of features like ZCR,MFCCs, chroma and spectral features and SVM classifiers.CBIR retrieval is performed using rhythm and pitch featuresfor audio and MPEG-7 features for visual data.
Semantic search / browse
Syntactic search / browse
Database
Media storage
Video/Image analysis pipeline
Audio analysis pipeline
user-defined pipeline
Local SOA architecture
System SOA
System repository
Corporate CMS
End-user interfaces
Authoring environment
File storage
Analysis
Interface layer Authoring layer
SOA Architecture layer
Analysis layer
Semantinc annotation, browsing and search tools
All the web applications of the system havebeen developed according to the RIAparadigm. In particular the applications of theInterface and Authoring layers are developedin AJAX and Flash/Flex, while data isexchanged using SOAP, RSS and JSON formetadata and RTMP for video streaming.
Interfaces allow to perform manualannotation (to check automaticannotations, add metadata or create groundtruth annotations to train new automaticconcept detectors), browse , search(semantic and content based) and tagging.
Usability evaluation
The goal of the field trials was to assess the usability of the system, in particular letting the users to interact withthe search engine and its interfaces, to pose semantic and syntactic-level queries, but also to annotate, automaticallyand manually, some videos.
http://www.micc.unifi.it/ This work was partially supported by the EU IST IM3I project (contract FP7-222267). http://www.im3i.eu
top related