solutions linux 2011 merged

8/6/2019 Solutions Linux 2011 Merged

1/68


2/68

Agenda

A pragmatic introduction to the SemanticWeb

Experience report and demos from Nuxeo

Apache tools for Big Linked Data

Wednesday, May 11, 2011


3/68

1. Introduction to theSemantic Web



4/68

Prelude



5/68

Source: Mills Davis, Semantic Social Computing, sept. 2007Wednesday, May 11, 2011


6/68

History



7/68Wednesday, May 11, 2011


10/68

Historical perspective

From web 1.0: web of sites and pages,aka the World Wide Web

To web 2.0: web of people and ofparticipation, aka the Social Web (Blogs,RSS, tags, Facebook, Wikipedia, etc.)

To web 3.0: web of data, of meaning andconnected knowledge, aka the SemanticWeb



11/68

Semantics & Ontologies



16/68

Some examples

FOAF: relationships between people (socialnetwork)

SIOC: relationships between websites,articles, blogs, comments

Rich Snippets: syndicate RDFa content forSEO by Google, Yahoo

good-relations: e-commerce (Ebay...)

rNews: metadata for news agencies (AFP,

Reuters...)Wednesday, May 11, 2011


17/68

How is it related tothe Web?



18/68

The traditional Web

A principle: hypertext

A protocol: HTTP

An identification scheme: URNs/URIs

A language: HTML



19/68

To a computer, then, the web is a flat,

boring world devoid ofmeaning

Tim Berners Lee, http://www.w3.org/Talks/WWW94Tim/Wednesday, May 11, 2011
http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/


20/68

This is a pity, as in fact documents on the

web describe real objects and imaginaryconcepts, and give particular relationships

between them

http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/


21/68

Adding semantics to the web involves two things:

allowing documents which have information inmachine-readable forms, and allowing links to be

created with relationship values.

http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/


22/68

The Semantic Web is not a separate Web but an

extension of the current one, in which informationis given well-defined meaning, better enabling

computers and people to work in cooperation.

http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/http://www.w3.org/Talks/WWW94Tim/


23/68

The traditional Web


A protocol: HTTP


A language: HTML



24/68

The semantic Web


A protocol: HTTP


A language: HTML RDF



25/68

The W3C Layer Cake



26/68

The W3C Layer Cake

Alreadystandardized



27/68

URIs and the

Web of Things

URIs (Unique Resource Identifiers) are

used to identify things (also called

entities) in the real world

For instance: people, places, events,

companies, products, movies, etc.



28/68

The RDF model

Subject ObjectPredicate

RDF is used to describe relationships

between objects, identified by their URIs



29/68

Example

Source: http://www.slideshare.net/AntidotNet/web-smantique-web-de-donnes-web-30-linked-data-quelques-repres-pour-sy-retrouver

http://www.slideshare.net/AntidotNet/web-smantique-web-de-donnes-web-30-linked-data-quelques-repres-pour-sy-retrouverhttp://www.slideshare.net/AntidotNet/web-smantique-web-de-donnes-web-30-linked-data-quelques-repres-pour-sy-retrouverhttp://www.slideshare.net/AntidotNet/web-smantique-web-de-donnes-web-30-linked-data-quelques-repres-pour-sy-retrouverhttp://www.slideshare.net/AntidotNet/web-smantique-web-de-donnes-web-30-linked-data-quelques-repres-pour-sy-retrouverhttp://www.slideshare.net/AntidotNet/web-smantique-web-de-donnes-web-30-linked-data-quelques-repres-pour-sy-retrouverhttp://www.slideshare.net/AntidotNet/web-smantique-web-de-donnes-web-30-linked-data-quelques-repres-pour-sy-retrouver


30/68

RDF serialization

As XML:

Others, ex: N3:



31/68

SPARQL

Query language for RDF databases

Several implementations

OSS: Apache Jena, Sesame, 4Store,

Virtuoso, Mulgara, Redland, Open Anzo...

Proprietary: 5Store, AllegroGraphRDFStore, Stardog, Dydra, OWLIM...

More expressive than SQL, scalability is still

an open question



32/68

SPARQL Sample



33/68

Where and howto find these data?



34/68

Solution 1: Lift

One can use HTML scrapping and naturallanguage processing (NLP) technique toextract semantic information from existingcontent / sites

Generic solutions: OpenCalais, Zemanta,Apache Stanbol

Pro: no need to change existing content

Con: error rone, needs human checks



35/68

Example: DBPedia



36/68

Solution 2: export

RDFa and microformats are used to embed

semantic information (expressed using the

RDF model) into regular web pages

RDFa does it using existing (rel) and

additional (about, property, typeof)

attributes Microformats only use usual HTML

attributes (class)



37/68

Solution 3: reuse

Linked Open Data: (usually large) data

repositories available on the web (for freeor not), expressed using the RDF model

Interoperability between these repositories

(their ontologies) must be defined



38/68

Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/

Linked Open Data in 2007

http://lod-cloud.net/http://lod-cloud.net/


39/68


2008


2009


40/68


2009

http://lod-cloud.net/http://lod-cloud.net/http://lod-cloud.net/


41/68


2010



42/68

Good for Enterprise apps too!

Diagram source: http://www.w3.org/2007/Talks/0130-sb-W3CTechSemWeb/



43/68

Why now?



44/68

Key Enablers

Open Data and Linked Online Data

Advances in automatic content analysis

(linguistics, image processing) and machinelearning

Classical logic and classical AI

Computing power (Moores law +

MapReduce)



45/68

Lets put them to use!

The technologies and data

are available,



46/68

2. Nuxeo &Semantic ECM



47/68

Nuxeo: an open source

ECM vendorOur Focus is Enterprise Content Management

ECM as a Platform for Content Applications

Open Source as Efficient Development Model

Modern architecture for 21st Centurybusiness

Lean, mobile, social, interoperable

ASocial Marketplace in action

Innovation driven by community of customers, partners,

and our core developers



48/68

45



49/68

Ma or Customers



50/68

47

Goals for Semantic ECM

Repurpose existing content better

Improve search and collaboration

Make information more contextual

Extract and use information from content

Leverage Open and Linked Data, contribute

Make ECM users content smarter!

> Gain efficiency, effectiveness and strategic

positioning on the ECM market



51/68

48

Demo



52/68

49

IKS project

European project under theFP7, with 13 partners (6 SMEs) and a 8.5 MEURbudget

Goal: create a semantic software stack that will be

used by CMS vendors to add semantic features totheir products

Started in Jan. 2009, will last until Dec. 2012

First tangible result: Apache Stanbol, alreadyintegrated in a Nuxeo plugin



53/68

50

The Semantic Engine

From unstructured content to Knowledge

Language guessing

Topic classification (Business, Sports, Media, ...)

Named Entities extraction and linking

Relationships and properties extraction



54/68

51



55/68

52



56/68

53



57/68

54



58/68

55

=

Semantic Engines(Apache OpenNLP)

+Fast Linked Data local index

(Apache Solr)+

Semantic Rule Engine

(Apache Jena)Wednesday, May 11, 2011


59/68

56

12

3

DBpedia

Freebase

Geonames

LDAP



60/68

3. Apache tools for

processingBig and/or Linked Data


Training statistical models for NER with


61/68

58

Training statistical models for NER with

Wikipedia and DBpedia

Extract sentences with link positions in Wikipedia articles

DBPedia to the find type of the target entity(Person,

Location, Organization)

Apache Pig scripts to compute thejoin + format the result as

training files for OpenNLP

Apache OpenNLP to build and evaluate the models

Apache Hadoop for distributed processing

Apache Whirr for deployment and management on Amazon

EC2 cluster



62/68

59



63/68

60



64/68

61



65/68

62


Training statistical models for topic


66/68

63

Training statistical models for topic

classification from Wikipedia and DBpedia

Filter category tree from DBpedia SKOS entries (~500k)

Pig scripts to compute thejoins with articles abstracts for all

the articles categorized in Wikipedia

Export as 2.8GB TSV file to be indexed inApache Solr

Use Solr MoreLikeThisHandler to find the top 5 most related

Wikipedia category for any kind of text

Apache Whirr & Hadoop for deployment and management on

Amazon EC2 cluster


Wh t t?


67/68

64

Whats next?

Integrate the R&D results into Stanbol / Nuxeo

Work on user interface / high level javascript toolkits for Linked

Data editing

http://github.com/bergie/VIE based on backbone.js

Experiment / Integrate / Refine


R
http://github.com/bergie/VIEhttp://github.com/bergie/VIEhttp://incubator.apache.org/projects/whirr.html


68/68

Resources

http://iks-project.eu

http://stanbol.demo.nuxeo.com

http://incubator.apache.org/stanbol

http://blogs.nuxeo.com/dev

http://hadoop.apache.org/

http://incubator.apache.org/opennlp/

http://github.com/ogrisel/pignlproc
http://incubator.apache.org/projects/whirr.htmlhttp://incubator.apache.org/projects/whirr.htmlhttp://incubator.apache.org/opennlp/http://hadoop.apache.org/http://blogs.nuxeo.com/devhttp://incubator.apache.org/stanbolhttp://fise.demo.nuxeo.com/http://incubator.apache.org/projects/whirr.htmlhttp://incubator.apache.org/projects/whirr.htmlhttp://incubator.apache.org/opennlp/http://incubator.apache.org/opennlp/http://hadoop.apache.org/http://hadoop.apache.org/http://blogs.nuxeo.com/devhttp://blogs.nuxeo.com/devhttp://incubator.apache.org/stanbolhttp://incubator.apache.org/stanbolhttp://fise.demo.nuxeo.com/http://fise.demo.nuxeo.com/http://iks-project.eu/http://iks-project.eu/

solutions linux 2011 merged

Documents