an introduction to open data

An introduction to open data

Anders Pedersen @anpe / @okfn[.org]

CC-By v3 Licensed (all jurisdictions)

Government data: Not a new thing

Government data: Not a new thing

http://dabrownstein.files.wordpress.com/2013/02/visualizing-slavery-carpenter.jpg

Fast forward: Is

it safeto cycle?

Where Does Our Money Go?

We notice when government data collection fails

• US government is prohibited by law to collect data on gun related deaths

• Argentina’s inflation statistics cannot be trusted

• Greek financial figuresuntil 2011 were unreliable

• April 2014: Nigeria adjust their size of GDP by 80 per cent

3 global challenges

Data Locked UpData Hard to UseNo-one to Do It

We are an global network working for since 2004 to open up data and see it used to empower citizens and organizations to answer questions that matter and drive positive change

“Central” team of more than 35 on 4 continents.

Community network including civil servants, civil-society researchers and citizens with presence in more than 40 countries - including the Philippines, Indonesia, South Korea, Japan and China!

Advocacy and Expertise inOpening up Data

Create tools, skills and communities

Data=>

Knowledge & Insight

OKFestivalOpen Knowledge Festival

Largest open data event in the world

Berlin 2014

What?

Data is open if anyone is free to use, reuse and redistribute it for any

purpose without restriction or charge

http://OpenDefinition.org/

Why?

Challenge: Exploding Complexity of information

In 1820 all UK bank clearing done in a single room in London once a day

Today, billions of transactions a minute.

=> componentization to divide and conquer complexity

Opportunity: Information Technology

Smart phone = system for the Apollo moon landings

1TB of storage < $100 - in 1994 $450,000.

Mass participation in information access, processing and production. Decentralization.

Image: ItoWorld OpenStreetmap Edits

Many Eyes Will Bring Knowledge To Society And

Government

Source: http://data.gov.uk/data/openspending-report/index

£200m potentialsaving for NHS

Credit: Ben Goldacre &Open Healthcare UKhttp://openprescribing.org/http://prescribinganalytics.com/

Who committed aid for Yolanda?

The Many Minds PrincipleBest Thing to Do With Your Data Will Be

Thought of By Someone Else

£200m potentialsaving for NHS

Credit: Ben Goldacre &Open Healthcare UKhttp://openprescribing.org/http://prescribinganalytics.com/

3 E’s

EconomyEfficiency

Empowerment

Huge Growth in Open Data in Last Few Years

Especially for Government Data

Data Catalogs Around the World as of July 2012

Source: http://datacatalogs.org/ http://datahub.io/dataset/datacatalogs-org

Data Catalogs Around the World as of July 2012

Is open data simply “open washing”?

Open washing was coined by Christian Villum at Open Knowledge in this blog post: http://blog.okfn.org/2014/03/10/open-washing-the-difference-between-opening-your-data-and-simply-making-them-available/

Open Data Census/Index“Russian officials use the ranking on the Index as one of KPIs of data openness. It’s been very helpful for us - open data activists - to promote open data and open knowledge in Russia.”

Ivan Begtin, Ambassador, Open Knowledge Russia

“Since the Index came out, a number of countries - including the Russian, Indonesian, German and Belgian Governments - are using it as a yardstick for their achievements or lack of it.”

Andrew Stott, former Director for Transparency and Digital Engagement for the UK Government, & Open Data Advisor at the World Bank

In October 2013, ahead of the annual Open Government Partnership (OGP) summit in London, Open Knowledge launched the Open Data Index, the first major assessment of the state of open government data in the world.

The Index ranked 70 countries according to the availability and accessibility of data in ten key categories, and is based on peer reviewed submissions from the Open Data Census.

No natural order: leaders are those who act

Source: Daily Nation, Kenya, November 10, 2012, http://www.nation.co.ke/business/news/Open-data-initiative-has-hit-a-dead-end/-/1006/1617026/-/n18uhrz/-/index.html

Walk the walk on open data● Better data

● Publish data that matters - local and granular

● Release in open formats

● Know how well you score

Better data● Improve formats eg. PDF → CSV

● Documentation: What is in your data

● Low data quality and closed formats will result in reduced reuse

For driving local data use: Granularity is key

Release data that matters: eg. grades at

school level

Source: Twaweza, Tanzania, List of worst schools, http://www.shule.info/schools/worst

Tip: the story is almost always buried in granular data

Source: Mapumental

Case I: fair distribution of government subsidies Government: Provincial breakdown shows“equal distribution”

Breakdown by postal code =rich gained 20 times highersubsidies than poor areas

Case II: US Farm subsidy data

Data is open if anyone is free to use, reuse and redistribute it for any

purpose without restriction or charge

http://OpenDefinition.org/

Without open formats data will not be reusable and truly open data

Example: geodata portal

Source: national.census.okfn.org

Know your baseline: assess how to improve

In doubt about priorities?

Release raw data first...

CSOs or journalists will then take care of the rest

Tackling questions about open data:

The global top 3 excuses why data should not be

opened up

1) We cannot release data due to privacy or commercial sensitivity

2) Data quality is not good enough. It’s complicated!

Response: Just clean it up!

3) We do not have the resources

Response: You do not need to release nice visualisations, just release the bare CSVs.

So besides all these tips:

What makes an open data initiative successful?

Change makers in government working together to make it happen!

http://okfn.org/http://ckan.org/

http://schoolofdata.org/