moldeas at city college

TOWARDS A PAN-EUROPEAN E-PROCUREMENT PLATFORM TO AGGREGATE,

PUBLISH AND SEARCH PUBLIC PROCUREMENT NOTICES POWERED BY

LINKED OPEN DATA:

THE MOLDEAS APPROACH

Dr. Jose María Alvarez Rodríguez

Research Fellow, SEERC

Thessaloniki, 22-02-2012

Background and Glossary e-Procurement

• A public procurements initiated, negotiated and/or concluded using electronic means, i.e. using electronic equipment for the processing and storage of data, in particular through the Internet.

Public procurement

• A procedure initiated by a contracting authority with a view of acquiring goods, services or public works for the fulfillment of its tasks.

Public Procurement Notice, notice, public contract, etc.

• Being strict are not the same.

• There are distinct definitions depending on the stage: PriorNotice, AwardNotice, etc.

• For the sake of a better understanding we will use these terms to refer the same thing “an announcement” of a new public procurement process (first stage and notice).

• CPV

– Common Procurement Vocabulary

• LOD

– Linking Open Data or Linked Open Data

• NUTS

– Nomenclature of Territorial Units for Statistic

• OWL

– Ontology Web Language

• PSI

– Public Sector Information

• RDF

– Resource Description Framework

• SME

– Small and Medium-sized Enterprise

• TED

– Tenders Electronic Daily

Source: http://ec.europa.eu/internal_market/consultations/docs/2010/e-procurement/siemens-study_en.pdf

22/02/2013 Thessaloniki, Greece 2

The problem…

I have a family business that produces beds and other

bedroom furniture…

I think we could sell our products in other countries…

but I do not have clients due to the crisis…

and we could also try to sell beds to public administrations…

Let’s search…


I have amily


Some help…an expert in e-Procurement

We are a Spanish SME that sells an alert service about public procurement opportunities…

…the region…

We need the type of contract…

And other variables: value, duration, etc.

We will deliver to you a daily report…


The interview… “I can provide different types of “beds” and bedroom furniture”

“Ok! Let’s see some CPV codes…” • 33192100-3 - Beds for medical use • 39143116-2 – Cots • 39143310-2 - Coffee tables • …

I have amily

“Do you have any target region?”

“Well, Thessaloniki, Greece…but maybe other countries”

“Ok! Let’s see some NUTS codes…” • GR-Greece • GR1: Voreia Ellada • GR12 Kentriki Makedonia • GR122 Thessaloniki Prefecture, etc.

I have amily


… Is it a familiar business? Isn’t it?

“Great!...a SME…

“yes, we are 10 people…”

“Ok! I am going to collect all these features and I will report you the new opportunities…”

“Do you have any thinking about the duration of the contract or the value?

“I suppose we could assume contracts about 60000€ of

one year duration…”

“Great! I hope to get some business opportunity…”

“For sure! Don’t hesitate about it!


Let’s start…

• We need public procurement opportunities that fulfill these requirements:

Feature Value

Type of “object” (CPV Codes):

33192100-3, 39143116-2, 39143310-2

Location (NUTS Codes) GR, GR1, GR12, GR122 and other European countries

Type of company SME

Duration 1 year

Value 60,000 €

… …


Building the alert…

• We have to retrieve information from different

– Data sources or providers

• Official Bulletins, Official web pages, Newspaper, etc.

– Formats

• PNG, JPEG, PDF, MSOffice, OpenOffice, CSV, RSS, etc.

– Languages

• 23 official languages in Europe

– Models, services and APIs

• XML-Schema, SQL, REST, WSDL/SOAP, etc.


Could you understand this notice?

http://bit.ly/Yw0Rpm 22/02/2013 Thessaloniki, Greece 10

…and this one?

http://bit.ly/WTIRYA 22/02/2013 Thessaloniki, Greece 11


Yes, you can speak/read/write Spanish…

…and the location, where is “Asturias”?

…and the format? you have software to read PDFs, PNGs, etc. files

…and what is the meaning of “2012”? “2012” it is clearly a year

…and what is the meaning of “3.371.282,99 €”? It is clearly a value (~three million of Euros) using “.” as decimal separator

Yes, but…we seek for an alert service… • The information and data should be…

– Automatically processed • Machine-processable format

– Validated against a common data model

– Available for querying via a formal language such as SQL

– Usable to build added-value services

– …

• Someone could say: ”Ok! But I can search by myself in the web and manually check the features”

– Yes, why not? You can perfectly check an average of 16K notices per day in the European Union


and Why?

• e-Procurement is a strategic sector – 17% of the GDP

• Action Plans 2004 and 2020

• Projects – E-Certis, Fiscalis 2013, E-Prior, PEPPOL, STORK, etc.

• Other actions – TED, RAMON metadata server, CPV, NUTS, etc.

• Legal framework (to be transposed in each European country)

• Boost participation (specially SMEs) – First action could be to alert about new public procurement notices


…But • …a tangled realm of data and information…

– Formats, models, APIs, providers, classifications, locations, etc.

It is not easy to reuse this valuable

public sector information (PSI)

We should make this information/data available to be machine-processable…


Semantic Web

Linked Data

Open Data


8 principles-Open Data

1. Data Must Be Complete. 2. . . . Primary. 3. . . . Timely. 4. . . . Accessible. 5. . . . Machine processable. 6. Access Must Be Non-Discriminatory. 7. Data Formats Must Be Non-Proprietary. 8. Data Must Be License-free.


Public Procurement Data is a clear example of Open Data

…and due to its relevance for the economic sector we should ensure all the principles of this initiative.


Semantic web Common & shared data model

Graph (subject, object, predicate) RDF with different serialization

formats Implicit multilinguism support

Knowledge-representation Ontologies OWL (Ontology Web Language) Logic formalism: DL, F-Logic, etc. Reasoning

Knowledge-management Expert systems

Standards Query languages Vocabularies Datasets …

#me “Jose”

“Alvarez”

#diego

foaf:name

foaf:family:name

foaf:knows

“Diego”

foaf:name


RDF triples

jose-foaf: http://www.josemalvarez.es/foaf.rdf#. diego:foaf: http://berrueta.net/foaf.rdf#.

jose-foaf:me foaf:name “Jose”. jose-foaf:me foaf:family_name “Alvarez”. jose-foaf:me foaf:knows diego-foaf:me. diego-foaf:me foaf:name “Diego”.

foaf:name

#me “Jose”

“Alvarez”

#diego

foaf:family:name

foaf:knows

“Diego”

foaf:name


“Can we represent the information of our user using

RDF?”



User RDF Graph

e:topic

#u1

“Beds and bedroom furniture”

“Greece, Thessaloniki, others, etc.”

e:located-in

“60000 euros” e:value

…

e:topic

#e1

“33192100-3, 39143116-2, 39143310-2”

“GR, GR1, GR12, GR122, …”

e:located-in

“60000 euros”

e:value

…

Expert RDF Graph

“There is not so much gain, it is just another way to represent

information…”

“Yes, but it is machine-processable (properties have semantics)

and we can do better!” • Re-using well-know vocabularies, properties, etc. • Making use of data properties • Labeling all resources • …


#e1

GR e:located-in

…

RDF Graph

33192100-3 “Beds for medical use”@en

rdfs:label

33192100-3 “Cots”@en

rdfs:label

33192100-3 “Coffee tables”@en

rdfs:label

foaf:topic

#v1

e:value

GR1

GR12

GR122

“Greece”@en rdfs:label

“Voreia Ellada”@en

rdfs:label

“Kentriki Makedonia”@en

rdfs:label

…

“60000” ^^xsd:double

#c1

e:quantity e:currency

“Euro” “€”

rdfs:label e:symbol

24

“It seems better but…”

“Can we also represent the data in public procurement notices?”

“Yes, of course, we can follow

the same approach!” 22/02/2013 Thessaloniki, Greece 25

“Firstly we are going to introduce the concept of

Linked Data…”


Linked Data Principles

1. Use URIs to name things 2. When someone looks up a

URI, provide useful information, using the standards (RDF*, SPARQL).

3. Include links to other URIs.

4. Use HTTP URIs.

5* Model

http://www.youtube.com/watch?v=ga1aSJXCFe0 (Tim Berners-Lee and “The bag of crisps”)


http://www.youtube.com/watch?v=ga1aSJXCFe0

http://www.youtube.com/watch?v=ga1aSJXCFe0

“These principles can be achieved by applying RDF to represent data and

we can reach 5*!”

“Yes, but you should make links to existing datasets”

“Where Can I find them?”

“In the LOD Cloud there are some RDF

datasets and they are also open!” 22/02/2013 Thessaloniki, Greece 28

Linked Open Data Cloud

http://richard.cyganiak.de/2007/10/lod/

203 datasets ( 25 Billions of rdf triples) and

395 millions of links (Sept. 2010).

Domains: Media, Geographic, Government (42,09 %),

Publications, Cross-domain, Life sciences, etc. (Ago. 2011).

393 datasets (Jun. 2012).




“Let’s link our data to existing datasets…”

CPV 2008 (URI: http://purl.org/weso/pscs/cpv/2008/resource/id)

Example: http://purl.org/weso/pscs/cpv/2008/resource/33192100

NUTS (URI: http://nuts.psi.enakting.org/id/id)

Example: http://nuts.psi.enakting.org/id/GR

We are going to use prefixes to ease the reading of URIs…


http://purl.org/weso/pscs/cpv/2008/resource/id

http://purl.org/weso/pscs/cpv/2008/resource/id

http://purl.org/weso/pscs/cpv/2008/resource/33192100

http://purl.org/weso/pscs/cpv/2008/resource/33192100

http://nuts.psi.enakting.org/id/id

http://nuts.psi.enakting.org/id/id

http://nuts.psi.enakting.org/id/GR

http://nuts.psi.enakting.org/id/GR

#e1

nuts:GR

…

RDF Graph

cpv-2008: 33192100

“Beds for medical use”@en

rdfs:label

“ΕΛΛΑΔΑ”@el

“33192100”

dc:identifier

“33192100-3”

dc:subject

foaf:topic

cpv-2008: 33192000

skos:broader

“Ιατρικές κλίνες”@el

nuts:GR1

rdfs:label

contains

e:located-in

External Datasets


“Great! We can reuse the information and data…but…

Can we enrich that data?”

“Yes, you can create a “proxy” resource with new data and link to the existing one”

“For instance, we are going to add lat/long

to the NUTS code GR”


#e1 GR

…

RDF Graph

“ΕΛΛΑΔΑ”@el

nuts:GR

rdfs:label

contains e:located-in

External Dataset

nuts:GR1

e:has-nuts-code

“39.074208” “21.824312”

wgs84_pos:lat wgs84_pos:long


“We can easily extend the RDF model to represent information keeping the

semantics”

“Yes, exactly.”

“Wait, wait, wait…we have a data model, an implicit semantics and a query

language…so this is like a traditional database”


“Yes, there are common similarities…”

And…this is the

Web of Data!

Table Graph

E/R model RDF/OWL semantics

SQL SPARQL


DBPedia

GEOLD

UK Gov

ACM PubMED

NATURE

DBLP

MOLDEAS

And more… +300

SPARQL endpoints …

Webindex

AEMET

http://labs.mondeca.com/sparqlEndpointsStatus/




Some Use Cases…


http://www.data.gov/

http://data.gov.uk/

http://open-data.europa.eu/open-data/linked-data

http://datos.gob.es/datos/

http://geodata.gov.gr/geodata/index.php

http://www.w3.org/QA/2012/05/interview_bbc_on_publishing_an.html

http://www.renault.co.uk/ownerservices/shop/item/renaulttoys/pedalcar/eco2pedalcar/default.aspx

http://www.bestbuy.com/sitemap_p_index.xml

http://www.bioontology.org/

http://datos.bcn.cl/es

http://data.webfoundation.org/

http://data.worldbank.org/

http://aims.fao.org/standards/agrovoc/linked-open-data

http://eurostat.linked-statistics.org/

http://data.nature.com/

…



http://data.gov.uk/

http://data.gov.uk/










































“Can I execute SPARQL queries?”

“Yes, you could ask….”

“Give me gymnasts, born in Thessaloniki that have won an Olympic gold medal, including their name, date of birth and

some comment about them”


PREFIX dbo: <http://dbpedia.org/ontology/>

PREFIX dc-terms: <http://purl.org/dc/terms/>

SELECT ?name ?birthdate ?comment WHERE

?person dbo:birthPlace :Thessaloniki.

?person dc-terms:subject

<http://dbpedia.org/resource/Category:Olympic_gymnas

ts_of_Greece> .

?person dc-terms:subject

<http://dbpedia.org/resource/Category:Olympic_gold_m

edalists_for_Greece> .

?person foaf:name ?name .

?person rdfs:comment ?comment .

?person dbpedia2:birthDate ?birthdate .

FILTER (lang(?comment)='en').

SPARQL query …


s-subject p-predicate o-object l-literal

Results…

http://bit.ly/Xz3asA




Methods On Linked Data

for E-procurement

Applying Semantics


XML

TED

RDFizing

CPV

Services (e.g. Searching, Matchmaking &

Prediction) BOE

…

NUTS

Organizations

BOPA

RDFizing

Eurovoc

Linked Data Api

Pubby+Snorql

1 3

2

Produce

Consume

Publish

Semantic Methods

4 Validate

Overview


The ongoing example…

http://bit.ly/WTIRYA 22/02/2013 Thessaloniki, Greece 43

Partial view in RDF


What we did… Define the processes to produce, publish, consume and validate the Linked Data

generated from public procurement notices Design an ontology for representing domain knowledge

Entities and relationships ..

Apply the aforementioned points to public procurement data: 1M of Public Procurement Notices 9 Product Scheme Classifications (PSCs) from UN, EU, etc. 50K companies/people +200 Countries

Validate the generated Linked Data and make a comparison with existing approaches A survey of 196 criteria

Consume and exploit the generated Linked Data creating a matchmaking service using different methods Syntactic search, concept query expansion and a recommending engine


If we talk the same language (RDF) we can easily fulfill the requirements of our

“bed manufacturer”.

We would report the possibility of tendering in “Asturias” (ES12)

to provide “Beds” (CPV-33192100)

and other furniture (CPV-39143116 & CPV-39143310-2).


Results http://purl.org/weso/moldeas/ (it is now being updated)


http://purl.org/weso/moldeas/



Publishing Linked Data


22/02/2013 Thessaloniki, Greece #49

SELECT DISTINCT * WHERE

?ppn rdf:type ppn-def:ppn.

?ppn ppn-def:nutsCode ?nutsCode.

FILTER(?nutsCode = <http://nuts.psi.enakting.org/id/ES12> OR

?nutsCode = <http://nuts.psi.enakting.org/id/GR>) .

?ppn cpv-def:codeIn2008 ?cpvCode.

FILTER(?cpvCode = cpv:33192100 OR

?cpvCode = cpv:39143116 OR

?cpvCode = cpv:39143310) .

?ppn dc:date ?date .

Example of a simple SPARQL query

*This is the old version of MOLDEAS. New procurement notices, etc. are coming soon…

http://bit.ly/XQ0uUV http://purl.org/weso/moldeas/

(see Demo queries)

http://bit.ly/XQ0uUV

http://bit.ly/XQ0uUV


What we got… New way for representing the valuable information of public

procurement notices applying semantic technologies New datasets that are now part of the LOD Cloud Dissemination and networking Expertise, know-how generation and new research lines …maybe

A step forward to a new way of publishing public data, more specifically procurement data

Enabling cross-border business opportunities


Main Conclusion We can represent information and

data in public procurement notices using semantic technologies (vocabularies, datasets, etc.) Overcoming most of the problems

in public procurement notices


Questions

Thank you for

your attention!

Credits •

–

–

–

•

–

•

–

–

•

–

http://www.slideshare.net/josemalvarez

http://www.slideshare.net/josemalvarez

http://creativecommons.org/licenses/by-nc-sa/3.0/es/





moldeas at city college

Technology