towards a linked open data infrastructure for science, technology & innovation studies

x

Towards a Linked Open Data Infrastructure for Science, Technology & Innovation

Studies

Ali Khalili, PhD Department of Computer Science/Artificial Intelligence

Knowledge Representation & Reasoning Research Group

Outline

• Linked (Open) Data

• RISIS Project

• Semantically Mapping Science (SMS) Platform

• Workflow

• Use Cases

• Adaptive Functional Urban Areas (FUAs) to Study Innovative Activities

• Gendered Dimensions in Grant Selection

Evolution of the Web

https://mcgratha.wordpress.com/

Linked (Open) Data

• A set of best practices for publishing data on the Web.• Follows 4 simple principles:

https://www.ted.com/talks/tim_berners_lee_on_the_next_web

• Use HTTP URIs so that users can look up (dereference) those names. • When someone looks up a URI, provide useful information, using the

open standards.• Include links to other URIs, so that users can discover more things.

• Use URIs as names (identifiers) for conceptual things.

https://www.ted.com/talks/tim_berners_lee_on_the_next_web

Linked (Open) Data: Principles

WWW World

5 Open Data

make your stuff available on the Web (whatever format) under an open license

make it available as structured data (e.g., Excel instead of image scan of a table)

make it available in a non-proprietary open format (e.g., CSV instead of Excel)

use Linked Data format (URIs to identify things, RDF to represent data)

link your data to other people’s data to provide context

http://5stardata.info/

http://5stardata.info/

5 Open Data

Linked Open Data Cloud

8Linked Open DataAli Khalili

http://lod-cloud.net/


Linked Open Data Cloud



Linked Open Data: Statistics

http://lodlaundromat.org/

http://stats.lod2.eu/

more than 3426 datasets

http://lodlaundromat.org/

http://stats.lod2.eu/

Linked Open Data: Examples

https://en.wikipedia.org/wiki/Paris


Resource Property Value

https://en.wikipedia.org/wiki/Paris

Linked Open Data: Exampleshttp://dbpedia.org/resource/Paris

http://dbpedia.org/resource/Paris

Linked Open Data: Examples• Give me a list of capital cities in Europe with population more than 500,000 • Who are mayors of central European towns elevated more than 1000m? • Which movies are starring both Brad Pitt and Angelina Jolie? • All soccer players, who played as goalkeeper for a club that has a stadium with

more than 40.000 seats and who are born in a country with more than 10 million inhabitants

• …


https://www.google.com/cse/

http://www.apple.com


http://www.wolframalpha.com/

http://www.wolframalpha.com

http://risis.eu

http://risis.eu

RISIS EU Project (http://risis.eu)

http://datasets.risis.eu/

http://risis.eu

http://datasets.risis.eu/

RISIS Datasets: Entity Types

Organization Product Agreement

Person PolicyPolicy

Evaluation Location

CIB ETER EUPRO JOREP Leiden-Ranking

MORE I Nano Profile SIPER VICO

Higher Education

Firm Funding Body

Publication

Patent

Project

Investment

Funding Program

Semantically Mapping Science (SMS) Platform

http://sms.risis.eu

http://sms.risis.eu

RISIS WP9 Vision

ProposingS&TmapofEurope

Functional Urban Areas (FUAs)

defined by OECD in collaboration with EC/Eurostat consider factors beyond the predefined city boundaries to better reflect the economic geography of where people live and work


OECD Metropolitan eXplorer: http://measuringurban.oecd.org

http://www.oecd.org/gov/regional-policy/Definition-of-Functional-Urban-Areas-for-the-OECD-metropolitan-database.pdf

http://measuringurban.oecd.org

defined by OECD in collaboration with EC/Eurostat consider factors beyond the predefined city boundaries to better reflect the economic geography of where people live and work

population area GDP environment (CO2 emissions and air pollution) labour market (employment and unemployment growth) innovation (patent intensity) urban form and territorial organization


OECD Metropolitan eXplorer: http://measuringurban.oecd.org

http://www.oecd.org/gov/regional-policy/Definition-of-Functional-Urban-Areas-for-the-OECD-metropolitan-database.pdf

http://measuringurban.oecd.org

FUAs Example: Netherlands

FUAs: Building Blocks

Municipalities

Problem

Address FUA?

Problem

Address FUA?

• Vrije Universiteit Amsterdam • De Boelelaan 1105, 1081 HV Amsterdam Amsterdam (NL002)

Problem

Address FUA?


OECD FUAs List

http://demo.ld-r.org/browse/http%3A%2F%2Fgeo.risis.eu%2Foecd

Problem

Address FUA?


- Geocode to LAU (municipality)

OECD FUAs List


Problem

Address FUA?


- Geocode to LAU (municipality)

- Shapefiles for FUAs or LAUs?OECD FUAs List


Linked Open Data 28Ali Khalili

Linked Open Data

Interlinking

Enrichment

Quality Analysis

Evolution

Exploration

Extraction

Storage/Querying

Authoring

Linked (Open) Data Lifecycle

http://stack.linkeddata.org/

http://stack.linkeddata.org/


Linked Open Data Lifecycle

Exploration



• Search • Browse • Visualize

Exploration

Search for Linked Data


Linked Open Data Lifecycle Exploration

http://lov.okfn.org/





http://schema.org/

http://bl.ocks.org/danbri/1c121ea8bd2189cf411c

http://schema.org





Data hub http://datahub.io search for data, register published datasets, create and manage groups of datasets…

Exploration

http://datahub.io/




http://lotus.lodlaundromat.org

http://lotus.lodlaundromat.org/




• OpenStreepMap (OSM) • Database of Global Administrative Areas (GADM) • Flickr Shapefiles Dataset • Published Shapefiles for Individual Countries • Published Geospatial RDF Datasets

Example

OpenStreetMap (OSM)

• https://www.openstreetmap.org • built by a community of mappers that contribute and

maintain data about roads, trails, cafés, railway stations, and much more, all over the world.

• Administrative Boundaries • Level 1: super-national administrations e.g. European Union. • Level 2: country borders based on the political entities listed on

the ISO 3166 standard. • Level 3 to 11: subnational borders such as ``state'', ``province'',

``region'' and ``district''. • Data Access • Nominatim Web API for querying OSM • The Overpass API for fetching specific OSM data • Planet.osm Data (over 617GB uncompressed!)

https://www.openstreetmap.org

OSM: Nominatim Web API

• a tool to search OSM data by name and address and to generate synthetic addresses of OSM points (reverse geocoding)

• Several companies provide hosted instances of Nominatim query API, e.g MapQuest Open Initiative, PickPoint or the OpenCage Geocoder

• API documentation • Example usage:

• http://nominatim.openstreetmap.org/search.php?q=amsterdam&polygon=1&country=Netherlands&format=json&addressdetails=1

• MapQuest API

http://wiki.openstreetmap.org/wiki/MapQuest

http://pickpoint.io/

http://geocoder.opencagedata.com/

http://wiki.openstreetmap.org/wiki/Nominatim

http://nominatim.openstreetmap.org/search.php?q=amsterdam&polygon=1&country=Netherlands&format=json&addressdetails=1

http://goo.gl/kZZdlt

OSM Data: Example

GADM (Global Administrative Areas)

• http://www.gadm.org • GADM is developed by University of California, Berkeley

Museum of Vertebrate Zoology, the International Rice Research Institute and the University of California, Davis, and with contributions of many others.

• uses other existing sources: http://www.gadm.org/links • Administrative Boundaries • Level 0: countries. • Level 1 to 5: lower level subdivisions such as provinces, departments,

counties, etc. depending on the size and availability of data for the underlying country.

• Data Access • data is available globally and for each individual country, in different

formats: geopackage,R SpatialPolygonsDataFrame, ESRI file geodatabase, Google Earth

http://www.gadm.org

http://www.gadm.org/links

Flickr geo-tagged pictures

• Data from 190M geo-tagged photos on Flickr • new smart phone do not only have a camera but also the ability to capture

location information. • plotted all the geotagged photos associated with a particular place to

generate a mostly accurate contour of that place (something more fine-grained than a bounding box!).

• Where On Earth (WOE) IDs • correspond to the hierarchy of places where a photo was taken: from

country (level 1), region (level 2) county (level 3), locality (level 4) to neighborhood (level 5).

• for a given WOE entity, approximate shape of that place is inferred. • shapes in GeoJSON format • view shapes at http://polymaps.org/ex/flickr.html • download at http://www.flickr.com/services/shapefiles/2.0.1/ • more info: http://code.flickr.net/2012/10/24/2273/

http://polymaps.org/ex/flickr.html

http://www.flickr.com/services/shapefiles/2.0.1/

http://code.flickr.net/2012/10/24/2273/

Published Shapefiles for Individual Countries

• Local administrative offices or Geo-related research centres might provide shape files specific to a country.

• E.g. for the Netherlands, shapefiles are provided by Centraal Bureau voor de Statistiek (CBS)

• Data collection needs to be done by a group of people in contact with Geo-related organization in countries.

• Current status

http://www.cbs.nl/nl-NL/menu/themas/dossiers/nederland-regionaal/publicaties/geografische-data/archief/2014/2013-wijk-en-buurtkaart-art.htm

https://docs.google.com/spreadsheets/d/1xkwe3Y_H_rm_-9xTdRDlhmADZTgtkU3iHKiPUQBdiAA/edit#gid=0

Published Geospatial RDF Datasets

•http://linkedgeodata.org and http://geoknow.eu •a large spatial knowledge base (>400m geo elements)

which has been derived from OpenStreetMap. •provides unique URIs and has Mappings to DBpedia.

•GeoVocab.org • GADM-RDF: Global Administrative Areas • NUTS-RDF: EU's Nomenclature of Territorial Units for

Statistics

http://linkedgeodata.org

http://geoknow.eu

http://geovocab.org

Published Geospatial RDF Datasets

•http://linkedgeodata.org and http://geoknow.eu •a large spatial knowledge base (>400m geo elements)

which has been derived from OpenStreetMap. •provides unique URIs and has Mappings to DBpedia.

•GeoVocab.org • GADM-RDF: Global Administrative Areas • NUTS-RDF: EU's Nomenclature of Territorial Units for

Statistics Outdated!

No Shapefiles!

http://linkedgeodata.org

http://geoknow.eu

http://geovocab.org

Extraction



from Semi-structured sources


Linked Open Data Lifecycle Extraction

Resource Property Value


Linked Open Data Lifecycle Extraction DBpedia



Persian DBpedia?

Persian DBpedia (mapping Wiki)





• Ad-hoc• DBpedia extraction framework

• Generic• OpenRefine

from Semi-structured sources

https://github.com/dbpedia/extraction-framework

http://openrefine.org/

from Unstructured sources



…After leaving Apple, Jobs took a few of its members with him to found NeXT, a computer platform development company based in Redwood City, specializing in state-of-the-art computers for higher-education and business markets. In addition, Jobs helped to initiate the development of the visual effects industry when he funded the spinout of the computer graphics division of George Lucas's company Lucasfilm in 1986. The new company, Pixar, would eventually produce the first fully computer-animated film, Toy Story…

NLP, Text mining, Annotation






Named Entity Recognition







foundedBy

Relation Extraction




http://spotlight.dbpedia.org

http://bioportal.bioontology.org/annotator

http://spotlight.dbpedia.org

http://bioportal.bioontology.org/annotator


from Structured sources: Triplification



• Relational Database to RDFR2RML: RDB to RDF Mapping Languagehttp://www.w3.org/TR/r2rml/

• D2R Server: Accessing databases with SPARQL & as Linked Datahttp://d2rq.org/

• Sparqlifydefining RDF views on relational databaseshttp://sparqlify.org/

http://www.w3.org/TR/r2rml/

http://d2rq.org/

http://sparqlify.org/

DATA EXTRACTION & CONVERSION

GeoJSON

Enrichment Functions

MappingConfigurations

OSM XML

PBF

ESRI shapes

triplify

mapshaper

osmtogeojson

osmosis

DATA EXTRACTION & CONVERSION

Metadata about different levels provided by OSMhttp://wiki.openstreetmap.org/wiki/Tag:boundary%3Dadministrative

http://wiki.openstreetmap.org/wiki/Tag:boundary%3Dadministrative

Storage & Querying



Relational Databases vs. Triple Stores


Linked Open Data Lifecycle Storage/Querying

• A relational databases’ (e.g. MySQL, PostgreSQL, Oracle) natural representation is a collection interlinked tables.

• A triple stores’ (e.g. OpenSesame, AllegroGraph, Neo4j) natural representation is a multi-relational network, or graph.

* Triple Store: it is called a triple store because in RDF, the facts are represented in form of a triple (Subject-Predicate-Object).

Existing Triple Stores



• Native triple stores4Store, AllegroGraph, BigData, Jena TDB, Sesame, Stardog, OWLIM and uRiKa

• RDBMS-backed triple storesJena SDB, IBM DB2 and OpenLink Virtuoso

• NoSQL triplestoresCumulusRDF

DATA STORAGE & QUERYING

Virtuoso Geo Spatial Geometry as SMS

internal representation for Geo-data in RDF

http://docs.openlinksw.com/virtuoso/sqlrefgeospatial.html

SPARQL – SQL for the Linked Data



What can be done with SPARQL that can't be done with SQL?




What can be done with SPARQL that can't be done with SQL?

• SPARQL queries are considerably better aligned with users’ mental models of a domain.




• SPARQL allows the conceptual data model to be fully explored through queries.




• SPARQL allows the conceptual data model to be fully explored through queries.

- example:workPhone rdfs:subPropertyOf example:phone- example:cellPhone rdfs:subPropertyOf example:phone- example:homePhone rdfs:subPropertyOf example:phone




• Queries that have to traverse a chain of connections are particularly complex in SQL while very simple in SPARQL.




• In addition to SELECT, INSERT and DELETE, SPARQL supports ASK queries.

• SPARQL includes syntax (i.e. SERVICE) to call two or more data sources within a single query.

• …

SPARQL Query Interface



http://yasgui.org/

http://yasgui.org/

Interlinking



Interlinking



• The degree to which entities that represent the same concepts are linked to each other.

• “Connecting things that are somehow related” • Methods

• Automatic, Semi-automatic, Manual • Universal, Domain-specific

<http://dbpedia.org/resource/VU_University_Amsterdam>

<https://www.wikidata.org/entity/Q1065414>

SameAs

http://dbpedia.org/resource/VU_University_Amsterdam

https://www.wikidata.org/entity/Q1065414

Interlinking Methods



• Ontology Matching • establish links between ontologies underlying two

data sources.

• Instance Matching (Link Discovery) • discover links between instances contained in two

data sources.

DATA LINKAGE

- Query on metadata about the administrative boundaries

- Find the alignment between levels in different datasets

DATA LINKAGE

- used the possible mappings between datasets at different levels.

- check the overlaps of areas at the similar level, and for the matching areas apply string matching to make sure that they refer to the same administrative boundary.

DATA LINKAGE

OECDFUAs

DBpedia

GeoNames

WikiData

GADM

FlickrShapes

OpenStreetMapAdministrative

Boundaries

Scientific Lenses

DBpedia

Wikidata

OrgRef

GRID FundRef

Geoname

ISNI

VIAF

Cordis

?

Semantically Mapping Science (SMS) Platform

http://sms.risis.eu

http://sms.risis.eu

Linked Data Serviceshttp://api.sms.risis.eu/

http://api.sms.risis.eu/

SERVICE TO APPLICATIONhttp://sms.risis.eu/demos

http://sms.risis.eu/demos

SERVICE TO APPLICATIONhttps://docs.google.com/spreadsheets/d/1XhXzdAf-veqHPj0kIaeZoXE3AJa8nwNugW_nOHH1jtk/edit?usp=sharing

https://docs.google.com/spreadsheets/d/1XhXzdAf-veqHPj0kIaeZoXE3AJa8nwNugW_nOHH1jtk/edit?usp=sharing

Use Cases

https://hyperir.cartodb.com/viz/13b5f3da-4356-11e6-a365-0e5db1731f59/public_map

(research) and innovation subsidies for organizations and companies in the Netherlands

https://hyperir.cartodb.com/viz/13b5f3da-4356-11e6-a365-0e5db1731f59/public_map

Use Cases(research) and innovation subsidies for organizations and companies in the Netherlands

People Hybrid OECD FUAsBusinesses

People Hybrid OECD FUAsBusinesses

Use Cases

Universities + Companies + Projects + Boundaries

Properties of container administrative boundaries

Collaboration between Universities and Companies

Properties of Universities and Companies

Use Cases

Universities + Companies + Projects + BoundariesRVO-NL

DBpedia OpenStreetMap

GADMFlickr

OECD FUAs

CBS-NL




Use Cases

Universities + Companies + Projects + BoundariesRVO-NL

DBpediaLeiden-Ranking

ETER

OrgRef Cordis OpenStreetMap

GADMFlickr

OECD FUAs

Grid

CBS-NL




Eurostat

Summary of the Use Case

Address

FUA

Administrative Boundaries

Coordinates

geocode

References


Linked Open Data

• http://slidewiki.org/deck/11936_semantic-data-web-lecture-series • Introduction to linked data and its lifecycle on the web • http://euclid-project.eu/ • http://videolectures.net/wims2011_auer_interlinked/ • https://vimeo.com/76257120 • http://www.slideshare.net/slidarko/evolving-the-web-into-a-giant-global-

database-3880018 • http://www.dataversity.net/introduction-to-triplestores/ • http://www.topquadrant.com/2014/05/05/comparing-sparql-with-sql/

http://slidewiki.org/deck/11936_semantic-data-web-lecture-series

http://dl.acm.org/citation.cfm?id=2529646

http://euclid-project.eu/

http://videolectures.net/wims2011_auer_interlinked/

https://vimeo.com/76257120

http://www.slideshare.net/slidarko/evolving-the-web-into-a-giant-global-database-3880018

http://www.dataversity.net/introduction-to-triplestores/

http://www.topquadrant.com/2014/05/05/comparing-sparql-with-sql/

towards a linked open data infrastructure for science, technology & innovation studies

Data & Analytics