intelligent information services

Berliner XML Tage. Humboldt Universität zu Berlin, 11-13 Oktober 2004

SWEB2004 – Int’l Workshop on Semantic Web Technologies in Electronic Business

Intelligent information services

The marriage of multimedia presentation and the Semantic Web

Dipl.-Inf. Lyndon J B NixonAG Netzbasierte Informationssysteme

Freie Universität BerlinTakustrasse 9, D 14195 Berlin

[email protected]

Lyndon Nixon


Berliner XML Tage, 11-13 Okt. 2004

SWEB2004 Workshop

Introduction: multimedia presentation

Multimedia presentation is about communicating an idea to a user through the use of synchronized mixed media

The World Wide Web is becoming both ubiquitous and multimedia based

Users want access to the information on the Web, and that access must be flexible to differing needs and contexts

Text Video Image Audio

Lyndon Nixon



SWEB2004 Workshop

Introduction: Semantic Web

Knowledge representation techniques allow machines to process data according to its‘ meaning, as modelled in an ontology

The Semantic Web is about extending the existing Web of content into a Web of knowledge that can be understood, reasoned about and intelligently processed by computers.

An ontology for geneaology

(1) A person can be a parent or child of another person

(2) If another person is my child, I am their parent

(3) The parent of a parent is a grandparent

Reasoning on knowledge

John is a parent of Lucy Maria is a child of Lucy

It follows that ->

Lucy is a parent of Maria John is a grandparent of Maria

Lyndon Nixon



SWEB2004 Workshop

Introduction: a beautiful marriage?

Idea: combine Semantic Web and multimedia presentation to offer users a flexible means for access to their desired information through Web-based knowledge

acquisition and reasoning Multimedia-based

presentation of the information result

User expresses an information wish

Semantic Web of Knowledge

Data

Web of content

Internet Service

Lyndon Nixon



SWEB2004 Workshop

State of the Art: IMMPS*

KNOWLEDGESERVER•semantics

•presents

LAYERS

application

context

user

design

control

content

design

realizes

presents

Goal formulation

User

KNOWLEDGE SOURCES

EXTERNAL CONTENT

EXTERNAL APPLICATION

* „A Standard Reference Model for Intelligent Multimedia Presentation Systems“, Bordegoni, M. et al

•A proposed generic model for Intelligent Multimedia Presentation Systems (IMMPS)

•Decision making at every stage based on knowledge provided from a server

•No concrete specification for knowledge representation, querying or manipulation

Lyndon Nixon



SWEB2004 Workshop

State of the Art: Cuypers*

Multimedia IR System

Semantic Structure

Communicative Devices

Qualitative Constraints

Quantitative Constraints

Final Form Presentation

HTTP Web Client * „Towards Second and Third Generation Web-Based Multimedia”, van Ossenbruggen, J. et al

•A multimedia generation system using a constraint logic programming approach to support backtracking and best fit results

Extensions into Semantic Web technologies:

•RDF metadata used as basis to infer spatio-temporal relationships between media•OWL ontologies used to define domain and discourse models for the organisation and presentation of media

•Acts as an interface to a multimedia database

•Semantics limited to annotated media objects

•Knowledge for use in generation is fixed before execution

Lyndon Nixon



SWEB2004 Workshop

State of the Art: CHIME*

* „CHIME: Service-oriented Framework for Adaptive Web-based Systems“, Chepegin, V.; Aroyo, L.; De Bra, P.; Houben, G.-J.

•An Adaptive Hypermedia System

•A modular, interoperable architecture

•An ontological approach for a shared conceptual understanding

•Research direction multi-agent systems / Semantic Web Services

•Project focus limited: user modelling & personalisation

Lyndon Nixon



SWEB2004 Workshop

SWeMPS: architecture

•1Query Tool

Reasoner

Service Planner

Modeller Formatter

Multimedia presentation

Conceptual space

Multimedia model

Service Space

Resource Space

Rules-based

Application L

ogic

Information requestBased on a:

•Generic rule-base

•Conceptualisation of process

•Sets of resources and services

•Abstraction of the result presentation

Using „plugged-in“ components to interface the application with external, distributed, heterogeneous data

Lyndon Nixon



SWEB2004 Workshop

SWeMPS: ontology

•The ontology provides a shared understanding of the definition of the multimedia generation process

•An instantiation of the ontology („knowledge base“) defines the operation of an individual multimedia generation task

•Three key distinctions made in the model, between subjects, resources and services.

Lyndon Nixon



SWEB2004 Workshop

SWeMPS: rules

Rule base written using PROVA, a language combining rule-based (Prolog) and object-oriented (Java) approaches. It supports extension with all Java APIs, such as Jena (Semantic Web) or Web Services.

Rulebase

Factbase

assert/retract

trigger rules

query handler

conceptual space

service space

knowledge acquisition

reasoner (Jena)

service invocation

Fact(X) :-

Block of Prova code

Lyndon Nixon



SWEB2004 Workshop

Case Study: Searching on a user goal

Query handler asserts an information goal in the factbase

assert(goal(me:betty,gen:relationship,X))

This means „who is related to Betty?“

A rule is triggered by the goal assertion

goal(X,Y,Z) :- [code for querying conceptual space]

The executed code queries on the conceptual space, including the ontology for family relationships and metadata on my family

SELECT ?k_obj WHERE (me:betty, gen:relationship, ?k_obj)

Lyndon Nixon



SWEB2004 Workshop

Case Study: Mediation and Reasoning

The family metadata is using the GEDCOM ontology which states that Individuals are spouseIn or childIn a Family

The query fails, as the metadata has no gen:relationship properties

The rulebase supports backtracking: as the rule fails, an alternative is triggered which attempts to map an unknown to a known ontology, e.g.:

X gen:wife Y <- X spouseIn Z, X sex „F“, Y spouseIn Z

The ontology for family relationships states that the derived properties are all subProperties of gen:relationship, and hence it can be reasoned that:

X gen:relationship Y <- X gen:wife Y

Lyndon Nixon



SWEB2004 Workshop

Case Study: Resource resolution

The information goal‘s answer is that Betty is the wife of Bob, sister of Barb and daughter of Ben and Bessy.

The conceptual space references a family photo album, in which the photos are annotated with RDF. A service is invoked to extract knowledge about the family members.

Found resources are asserted in the factbase triggering a rule to adapt and add them to the multimedia model.

The multimedia model has mappings from the conceptual relationships to communicative abstractions e.g.

m_directLeft(R1,R2) :- represents(R1,S1), represents(R2,S2),query(S1,gen:wife,S2)

Lyndon Nixon



SWEB2004 Workshop

Case study: Presentation result *

Ben Bessy

Bob Betty Barb* Images from clipart.freedesktop.org

Notes

• result derived from existing knowledge and annotated media

• information goal can be changed and other presentations generated

• gaps or mismatches in knowledge can be resolved with dynamic acquisition / mapping

• presentation can be adapted to additional knowledge

Lyndon Nixon



SWEB2004 Workshop

Conclusion: Needs

A complete framework needs extensions to meet real world needs, e.g.:

Mediation services for resolving ontology mismatches Semantic Web services framework for the discovery and

invocation of services Ontology and rule management tools

Lyndon Nixon



SWEB2004 Workshop

Conclusion: Gains

The SWeMPS framework intends to allow implementers to leverage the (soon..) existing knowledge infrastructure of the Semantic Web

It offers the Semantic Web community a means for making the stored knowledge intuitively accessible to users

It is to be built upon to realise individual Intelligent Information Services for communicating knowledge from given domains to users

Lyndon Nixon



SWEB2004 Workshop

Conclusion: An e-business vision

Data providers contractually agree to provide trustworthy content and knowledge to a service provider, who is acting as the host for an Intelligent Information Service

Users make information requests to the service (for a charge, through registration, with an agreement to receive advertising) through various UIs on various devices, to which the final multimedia presentation is adapted

Intelligent Information Service

Document Repository

Media Repository

Knowledge Repository

Interactive Television

PC-based Web UI

Mobile Phone

Lyndon Nixon



SWEB2004 Workshop

That’s all folks!

More at

http://page.mi.fu-berlin.de/~nixon/

Thank you for your attention!

intelligent information services

Documents