being open, accessible, and understandable by jonathan challener, oecd - #imaodbc2014
DESCRIPTION
With the recent introduction of Open Data APIs, namely SDMX-JSON, and other web services planned, the OECD is moving from a pure ‘browser centric’ architecture towards a more ‘web service oriented’ architecture, with powerful and scalable data delivery capabilities in machine-to-machine exchanges, enabling a much wider use of our data by third parties. This focus has, and will further enable more and more users to assemble content and data themselves, compare or integrate data, or collaborate with each other to produce new content. This also allowed for and has led to a number of new and accessible data experiences, now targeting those who do not understand, or have the desire to understand, specialised formats of a statistical nature. This is where the real value will be found in being ‘open, accessible, and understandable”.TRANSCRIPT
Being Open, Accessible, and Understandable
International Marketing And Outputs DataBase Conference Évora, Portugal21-25 September 2014
Prepared and presented by Jonathan Challener, OECD
#imaodbc2014
Three distinct concepts of data as defined by the ODI data definitions
Three distinct concepts of data as defined by the ODI data definitions
Open data can be big, but doesn't have to be. Big data is often based on personal data, but not necessarily.
Big Open Personal (BOP) Data
Big Open Personal (BOP) Data
The data types are merging. Social media being the main contributor to this potential future ‘big open, and personal’ paradigm.
Open DataInformation that is available for anyone to use, for any purpose, at no cost.
For now we are interested in Open Data
Data deluge is only getting bigger
We are all putting up the open signs, inviting users into our data supermarkets
All too much for the average user
Not everyone is a data scientist!
Easy to consume
Easy to understand
1 2 3
Find Free FoodIn my city
Accessible
Turn Data Into Free Food!What if a city released a list of all trees planted on its public property. You could filter the data into a list of all the fruit and nut trees in the city, transfer it into an online database, and create a smartphone app that helps anyone find free food.
Learn about the trees of Paris
A complete list of 300,000 trees maintained by the City of Paris is available via an android app.
Project Delta: making OECD data…
Accessible
Open
Free
FindUnderstandUse
Machine-readableIndexableRe-Useable
Available without charge
Open
• All data open-ready via .Stat• Open Data premium services
Free
• Freemium strategy to maximise dissemination and impact
• 100% data & content available for free in a basic form
• Premium services on a cost-recovery basis• OECD iLibrary v2 (2015)• Sustainable financing for publishing
Accessible
• Data Lab (March 2013)• Data Portal (2014)• Search• User-journeys across OECD online
Project Delta: Objectives and deliverables
Open Data readiness
Open data governance
Community of end-users
Portal accessibility
46%
63%
26%
86%
77%
82%100%
OECD 2013
OECD Target 2015
Open data evolution of the OECD
Low evolution expected in regard to the community of end-users!
Public sector21%
IGOs3%
NGOs7%
Academia 42%
Media2%
Corporate24%
Other 1%
Audience breakdown visiting oecd.org since 2009
Significant increase in the proportion of academics, up 34%.
User needs…
Expertuser
Non-expert
Non-expert user needs:• only interested in main indicators and most relevant graphs• looks for answers, no matter if analytical from pubs, particularly overviews and summaries• shares graphs• …used to consumer tools (less forgiving)
Expert user needs:• looks for specific data• know what they are looking for• wants fast and complete download to use in own tools• citation is important• embeds graphs• …used to complicated tools
OECD Data Products•.Stat browser• Branded views• Data API
• Key tables• Easy to embed charts• Factbook and “At a Glance” publications• Ready-made tables• Mobile apps• Data portal indicators• Simple API usage
Expertuser
Non-expert
Onceuponatime…
First we began with this
Audience > expert users
Audience > expert users
I’m a policy analyst
I’m a researcher/statistician
I’m a university librarian
With both time and skills!
Audience > expert users with machines
I’m a policy analyst
I’m a researcher/statistician
I’m a university librarian
Audience > Org2Org via machine to machine <?xml version="1.0"?><message:MessageGroup xmlns:message="http://www.SDMX.org/resources/SDMXML/schemas/v2_0/message" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.SDMX.org/resources/SDMXML/schemas/v2_0/generic http://www.sdmx.org/docs/2_0/SDMXGenericData.xsd http://www.SDMX.org/resources/SDMXML/schemas/v2_0/message http://www.sdmx.org/docs/2_0/SDMXMessage.xsd" xmlns:common="http://www.SDMX.org/resources/SDMXML/schemas/v2_0/common" xmlns="http://www.SDMX.org/resources/SDMXML/schemas/v2_0/generic"><Header xmlns="http://www.SDMX.org/resources/SDMXML/schemas/v2_0/message"><ID>none</ID><Test>false</Test><Truncated>false</Truncated><Prepared>2013-04-15T11:46:19</Prepared><Sender id="OECD"><Name xml:lang="en">Organisation for Economic Co-operation and Development</Name><Name xml:lang="fr">Organisation de coopération et de développement économiques</Name></Sender></Header><DataSet
With both time and skills!
Simple
Lightweight
data
formats <b>This is bold</b>
{ "header": { "id": "8cd18a83-c86b-43ec-99ea-0db199e185f0",
"test": false, "prepared": "2014-08-22T10:14:15Z",
"sender": { "id": "OECD", "name": "OECD“ }, "request": { "uri":
"http://stats.oecd.org/sdmx-json/data/SNA_TABLE1/AUS+AUT+BEL+FRA.G
DP+B1_GA+B1G_P119+B1G.C+V+VP+VOB.A/all?json-
lang=en&detail=Full&dimensionAtObservation=AllDimensions&startPerio
d=2007" } }, "dataSets": [ {
"action": "Informational", "observations": { "0:0:0:0:0": [ 1178952, 0
], "0:0:0:0:1": [ 1258654, 0
], "0:0:0:0:2": [ 1296324, 0
], "0:0:0:0:3": [ 1406671,
{“SDMX”:“JSON”}
{"odata.metadata": "http://stats.oecd.org/OECDStatWCF_OData/OData.svc
/$metadata#DotStat.OData.DataSource.DatawarehouseE
ntities/REFSERIES", "value": [ { "FREQUENCY": "A", "LOCATION": "FRA", "SUBJECT": "YPTTTTL1_ST", "TIME": "2010", "Flags": "", "Metadata": null, "Value": 62927.11}, { "FREQUENCY": "A", "LOCATION": "FRA", "SUBJECT": "YPTTTTL1_ST", "TIME": "2011", "Flags": "", "Metadata": null, "Value": 63249.09} ]}
API services
Audience > developers
Who don’t have the desire to understand data…
…in specialised formats of a statistical nature
A good start…
{"odata.metadata": "http://stats.oecd.org/OECDStatWCF_OData/OData.svc/$metadata#DotStat.OData.DataSource.DatawarehouseEntities/REFSERIES", "value": [ { "FREQUENCY": "A", "LOCATION": "FRA", "SUBJECT": "YPTTTTL1_ST", "TIME": "2010", "Flags": "", "Metadata": null, "Value": 62927.11}, { "FREQUENCY": "A", "LOCATION": "FRA", "SUBJECT": "YPTTTTL1_ST", "TIME": "2011", "Flags": "", "Metadata": null, "Value": 63249.09} ]
}
{ "header": {
"id": "8cd18a83-c86b-43ec-99ea-0db199e185f0",
"test": false,
"prepared": "2014-08-22T10:14:15Z",
"sender": {
"id": "OECD",
"name": "OECD“
}, "request": {
"uri":
"http://stats.oecd.org/sdmx-json/data/SNA_TABLE1/AUS+AUT+BEL+FRA.GDP+B1_GA+
B1G_P119+B1G.C+V+VP+VOB.A/all?json-
lang=en&detail=Full&dimensionAtObservation=AllDimensions&startPeriod=2007"
} }, "dataSets": [
{ "action": "In
formational",
"observations": {
"0:0:0:0:0": [
1178952,
0 ], "0:0:0:0:1": [
1258654,
0 ], "0:0:0:0:2": [
1296324,
0 ], "0:0:0:0:3": [
1406671,
{“SDMX”:“JSON”}
to becoming open ready
but more is needed
I’m a student I’m a
journalist
I’m a citizen with a cause
I’m an inquisitive
user
To reach a wider audience
Easy to consume
Easy to understand
1 2 3
Find Free FoodIn my city
Accessible
data.oecd.org
data.oecd.org
Data portal vision (from the point of view of the user: “Let me easily access the OECD statistical data and analysis. Let me find, understand and use the data I need.”
A complex back office layer bringing together various pieces of information from several sources.
Enabling a front office compilation for consumption of key indicators with links to relevant summaries and publications, as well as access to complete databases in .Stat.
Quick access to key indicators via search or browse
Filtered search to display different result types and by topic
Topic pages
Country profiling
Full screen view
Quickly share something of interest
and there’s more….
Custom visualisations – Better Life Index
Custom visualisations – Better Life Index
How happy is my country?
One of our most popular publications.
Environmental indicators
Embeddable tables
Mobile apps
Many interfaces to data
Is this too much?
Integrated experience
I’m a student
I’m a journalist I’m a citizen
with a cause
I’m an inquisitive
user
I’m a policy analyst
I’m a researcher/statistician
I’m a university librarian
Users
In control
I’m a student
I’m a journalist I’m a citizen
with a cause
I’m an inquisitive
user
I’m a policy analyst
I’m a researcher/statistician
I’m a university librarian
Data Portal next version? Personalised dashboard
Allow users to copy visualisations to a clipboard and create their own dashboard.
Find Free FoodIn my city
1 2 3
Build on the momentum
Larger user engagement
#citizenengagement
Collaborate
Produce
#crowdsourcing
Will the worlds of Official Statistics and statistics produced by the public converge?
Being Open, Accessible, and Understandable
Thank You
International Marketing And Outputs DataBase Conference Évora, Portugal21-25 September 2014
Prepared and presented by Jonathan Challener - Project Manager, OECD
@Challener
slideshare.net/JonathanChallener
Sources used• Image: http://yowpyowp.blogspot.com/2010/09/they-drew-flintstones.html• Image: https://www.adaptiveportfolios.com/media/1_momentum-716.jpg• Image: http://www.snoutypig.com/2013/04/16/snouty-pride-thats-not-all-folks • http://www.oecd.org/site/envind/ • http://www.oecdbetterlifeindex.org/ • http://data.oecd.org • Image: http://www.telegraph.co.uk • http://stats.oecd.org/opendataapi/OData.html • http://stats.oecd.org • Reference: SDMX DataStructure-OECD.FDI(1.2).xml• Reference: OECD data publishing with APIs• Reference: Study by the OECD on users of OECD.org (2013)• Reference: OECD Delta-Benchmark - final report• Reference: http://www.wired.com/2014/08/dat/?utm_source=dlvr.it&utm_medium=tumblr• Image: http://blog.etq.com/Portals/41636/images/MrData.jpg• Reference: http://seradigm.co.nz/category/opendata• Image: http://commons.wikimedia.org/wiki/File:Planking_in_supermarket.jpg• Reference: https://github.com/theodi/data-definitions • #imaodbc2014