making the most of existing and mashable data sources

21
Making The Most of Existing and Mashable Data Sources Dimitri van Hees, APIcon UK 2014

Upload: dimitri-van-hees

Post on 19-Jun-2015

76 views

Category:

Data & Analytics


6 download

DESCRIPTION

Did you know you could access all Wikipedia data in a machine readable format? Did you know you could do the same with the Google Knowledge Graph? When was the last time you explored the APIs for DBpedia or the seemingly infinite number of knowledge domains (sports, politics, business, film, biology, etc) for which Freebase is known. When building your next app, your inclination might be to build your own database to power it. But name the information you're looking for and chances are, there's already an API for incorporating it into Web or mobile apps. In this workshop we'll explore ways to enrich applications with the plethora of public data sources that are already out on the Web. By the time this workshop is over, you won't be questioning if the data you need is available. It'll just be where to get it from. And if we can't find it, we'll show you how to generate it.

TRANSCRIPT

Page 1: Making The Most of Existing and Mashable Data Sources

Making The Most of Existing and Mashable Data Sources

Dimitri van Hees, APIcon UK 2014

Page 2: Making The Most of Existing and Mashable Data Sources

About me

• Technical Architect• Data Specialist• Freshheads BV• API supporter

Page 3: Making The Most of Existing and Mashable Data Sources

About me

“The only bad thing about API is that it’s the inverse of IPA”

Page 4: Making The Most of Existing and Mashable Data Sources

About me

• Brewmaster• MQTT API driven brewery

“De Brouwtoren”

Page 5: Making The Most of Existing and Mashable Data Sources
Page 6: Making The Most of Existing and Mashable Data Sources

Buzzwords

• #BigData• #OpenData• #LinkedOpenData• #SemanticWeb• #Web30• #IoT

Page 7: Making The Most of Existing and Mashable Data Sources

Buzzwords

• #BigData: lots of data• #OpenData: free data• #LinkedOpenData: free Linked Data• #SemanticWeb: Linked Data• #Web30: Linked Data• #IoT: Connected devices, sensor data

Page 8: Making The Most of Existing and Mashable Data Sources

Real world

Quality

PriceQuantity

Page 9: Making The Most of Existing and Mashable Data Sources

Data quality

• Machine readable?• Accessible via webservices?• Up-to-date?• Verified?

Page 10: Making The Most of Existing and Mashable Data Sources

Data quantity

• Can we store it?• Can we process it?• Is it representative?

Page 11: Making The Most of Existing and Mashable Data Sources

Data price

• What does it cost?• Do we need an API key?• Are we allowed to use it?• And also for commercial purposes?

Page 12: Making The Most of Existing and Mashable Data Sources

Privacy

Page 13: Making The Most of Existing and Mashable Data Sources
Page 14: Making The Most of Existing and Mashable Data Sources

Freebase

Page 15: Making The Most of Existing and Mashable Data Sources

Freebase

• REST interface• MQL (Metaweb Query Language)• Free API key• Free RDF download• Search widget• http://www.freebase.com/m/09889g• https://

www.googleapis.com/freebase/v1/topic/m/09889g

Page 16: Making The Most of Existing and Mashable Data Sources

DBPedia

Page 17: Making The Most of Existing and Mashable Data Sources

DBPedia

• Wikipedia as Linked Data• Synchronizes Wikipedia periodically• Free RDF download• SPARQL endpoint (dbpedia-owl:wikiPageID)• DBPedia lookup API• http://

lookup.dbpedia.org/api/search.asmx/KeywordSearch?QueryString=Michael%20Jackson• http://dbpedia.org/sparql

Page 18: Making The Most of Existing and Mashable Data Sources

Semantic API’s

• Dandelion.eu• 1000 free requests/day• Provides links to DBPedia as well• https

://dandelion.eu/products/datatxt/nex/demo/?exec=true#

Page 19: Making The Most of Existing and Mashable Data Sources

Or just add it yourself…

…but never ever re-invent the wheel!

Page 20: Making The Most of Existing and Mashable Data Sources

Thank you!

• Twitter: http://twitter.com/dvh• E-mail: [email protected]• GitHub: http://github.com/dvh• LinkedIn: https://linkedin.com/in/dimitrivanhees

Page 21: Making The Most of Existing and Mashable Data Sources

Thank you!

• Twitter: http://twitter.com/dvh• E-mail: [email protected]• GitHub: http://github.com/dvh• LinkedIn: https://linkedin.com/in/dimitrivanhees

• Brewery: http://facebook.com/brouwtoren