an introduction to open data
DESCRIPTION
Presentation for School of Data training in the Philippines, May 2014 in collaboration with the Open Data PH Taskforce.TRANSCRIPT
An introduction to open data
Anders Pedersen @anpe / @okfn[.org]
CC-By v3 Licensed (all jurisdictions)
Government data: Not a new thing
Government data: Not a new thing
http://dabrownstein.files.wordpress.com/2013/02/visualizing-slavery-carpenter.jpg
Fast forward: Is
it safeto cycle?
Where Does Our Money Go?
We notice when government data collection fails
• US government is prohibited by law to collect data on gun related deaths
• Argentina’s inflation statistics cannot be trusted
• Greek financial figuresuntil 2011 were unreliable
• April 2014: Nigeria adjust their size of GDP by 80 per cent
3 global challenges
Data Locked UpData Hard to UseNo-one to Do It
We are an global network working for since 2004 to open up data and see it used to empower citizens and organizations to answer questions that matter and drive positive change
“Central” team of more than 35 on 4 continents.
Community network including civil servants, civil-society researchers and citizens with presence in more than 40 countries - including the Philippines, Indonesia, South Korea, Japan and China!
Advocacy and Expertise inOpening up Data
Create tools, skills and communities
Data=>
Knowledge & Insight
OKFestivalOpen Knowledge Festival
Largest open data event in the world
Berlin 2014
What?
Data is open if anyone is free to use, reuse and redistribute it for any
purpose without restriction or charge
http://OpenDefinition.org/
Why?
Challenge: Exploding Complexity of information
In 1820 all UK bank clearing done in a single room in London once a day
Today, billions of transactions a minute.
=> componentization to divide and conquer complexity
Opportunity: Information Technology
Smart phone = system for the Apollo moon landings
1TB of storage < $100 - in 1994 $450,000.
Mass participation in information access, processing and production. Decentralization.
Image: ItoWorld OpenStreetmap Edits
Many Eyes Will Bring Knowledge To Society And
Government
Source: http://data.gov.uk/data/openspending-report/index
£200m potentialsaving for NHS
Credit: Ben Goldacre &Open Healthcare UKhttp://openprescribing.org/http://prescribinganalytics.com/
Who committed aid for Yolanda?
The Many Minds PrincipleBest Thing to Do With Your Data Will Be
Thought of By Someone Else
£200m potentialsaving for NHS
Credit: Ben Goldacre &Open Healthcare UKhttp://openprescribing.org/http://prescribinganalytics.com/
3 E’s
EconomyEfficiency
Empowerment
Huge Growth in Open Data in Last Few Years
Especially for Government Data
Data Catalogs Around the World as of July 2012
Source: http://datacatalogs.org/ http://datahub.io/dataset/datacatalogs-org
Data Catalogs Around the World as of July 2012
Is open data simply “open washing”?
Open washing was coined by Christian Villum at Open Knowledge in this blog post: http://blog.okfn.org/2014/03/10/open-washing-the-difference-between-opening-your-data-and-simply-making-them-available/
Open Data Census/Index“Russian officials use the ranking on the Index as one of KPIs of data openness. It’s been very helpful for us - open data activists - to promote open data and open knowledge in Russia.”
Ivan Begtin, Ambassador, Open Knowledge Russia
“Since the Index came out, a number of countries - including the Russian, Indonesian, German and Belgian Governments - are using it as a yardstick for their achievements or lack of it.”
Andrew Stott, former Director for Transparency and Digital Engagement for the UK Government, & Open Data Advisor at the World Bank
In October 2013, ahead of the annual Open Government Partnership (OGP) summit in London, Open Knowledge launched the Open Data Index, the first major assessment of the state of open government data in the world.
The Index ranked 70 countries according to the availability and accessibility of data in ten key categories, and is based on peer reviewed submissions from the Open Data Census.
No natural order: leaders are those who act
Source: Daily Nation, Kenya, November 10, 2012, http://www.nation.co.ke/business/news/Open-data-initiative-has-hit-a-dead-end/-/1006/1617026/-/n18uhrz/-/index.html
Walk the walk on open data● Better data
● Publish data that matters - local and granular
● Release in open formats
● Know how well you score
Better data● Improve formats eg. PDF → CSV
● Documentation: What is in your data
● Low data quality and closed formats will result in reduced reuse
For driving local data use: Granularity is key
Release data that matters: eg. grades at
school level
Source: Twaweza, Tanzania, List of worst schools, http://www.shule.info/schools/worst
Tip: the story is almost always buried in granular data
Source: Mapumental
Case I: fair distribution of government subsidies Government: Provincial breakdown shows“equal distribution”
Breakdown by postal code =rich gained 20 times highersubsidies than poor areas
Case II: US Farm subsidy data
Data is open if anyone is free to use, reuse and redistribute it for any
purpose without restriction or charge
http://OpenDefinition.org/
Without open formats data will not be reusable and truly open data
Example: geodata portal
Source: national.census.okfn.org
Know your baseline: assess how to improve
In doubt about priorities?
Release raw data first...
CSOs or journalists will then take care of the rest
Tackling questions about open data:
The global top 3 excuses why data should not be
opened up
1) We cannot release data due to privacy or commercial sensitivity
2) Data quality is not good enough. It’s complicated!
Response: Just clean it up!
3) We do not have the resources
Response: You do not need to release nice visualisations, just release the bare CSVs.
So besides all these tips:
What makes an open data initiative successful?
Change makers in government working together to make it happen!
http://okfn.org/http://ckan.org/
http://schoolofdata.org/