Transparency and Open Data:GSS Response

Iain BellHoP MoJ

What will I cover today?

• GSS transparency Group– Stock take– Action plan– Support for departments

• Case studies• Issues/ lessons learned

Main findings from stock take

• Nearly all official statistics can be found via the Government’s public face for the Transparency Agenda, the data.gov.uk website

• However GSS products are not easily identifiable as such and the use of tags is inconsistent

• The vast majority of official statistics are published as spreadsheets ( 2* or 3* format)

• Number of departments trialling or moving towards linked data

• Some concerns about whether departments have the skills to fully deliver on transparency

Proposed way forward

• as part of their statistical planning processes set out a timetable for when under-pinning datasets from our key statistical outputs will be released. All HoPs should compile their commitment as part of their business plan to creata a GSS overview

• all sensitive datasets for release will be accompanied by a Privacy Impact Assessment

• A central body (possibly ONS) should own the issue of the mosaic effect on de-anonymisation

• use GSS standard linked data formats where appropriate to ease re-use and release in linked data formats

Supporting the GSS

• using exemplars and case studies• creating a community of experts in undertaking this

work• developing training for Government Statisticians

– Strategic level– All GSS training (web-based)– Expert community

• a central repository through data.gov.uk of approved linked definitions across the GSS.

Implementing Open Data

• Examples:• DWP: Tabulation tool• Home Office and MoJ• ONS: Consumer Prices Index• DCLG: open data cabinet

Dissemination

Production & PublicationPrivacy Protection

Aggregate & MicrodataSDMX Standard

Web-based Analysis

TRUST & EXPLORATIONAd hoc AnalysisData Integration

Web Service Access

Visualization

Publication & ConsumptionPrivacy & Security

Aggregate DataMetadata Rich

Interactive Visualization

EXPERIENCEGuided AnalysisUser StimulationData Confidence

Stat-Xplore Stat-Tab

New Software

• two access routes

• for both casual and experienced user

DWP Tabulation Tool

Choose any variable

Build tables interactively, then download or save

results

Visualize

Stat-Tab

DWP Tabulation Tool

.

More than 47 million visits / 450 million hits since launch

Application Programming Interface (API) opened to more than 1000 developers.

Visits to Police.uk are more than 10 times higher than previous crime mapper.

Over 4300 individual pieces of

feedback into the system.

Home Office and MoJ

Caution (2)

MoJ - Re-offending

For the first time released• re-offending rates for every local authority• prison by prison re-offending rates• Probation Trust and Youth Offending Team re-offending

rates• for prisons - individual re-offending rates• for Probation Trusts individual re-offending rates

Next up – sentencing data

AGE court Offence_type Sentence_Outcome amount force sex Ethnicity offence_to_completion18-24 Croydon Other Theft Community sentence Metropolitan Police Male Black less than 6 months18-24 Croydon Violent Crime Community sentence Metropolitan Police Female White less than 6 months35+ Croydon Violent Crime Community sentence Metropolitan Police Male Black less than 6 months18-24 Croydon Burglary Immediate custody up to and including 3 monthsMetropolitan Police Male White Not Available25-34 Croydon Driving Offences Suspended sentence Metropolitan Police Male Black less than 6 months35+ Croydon Driving Offences Fine over £100 up to Up to £150Metropolitan Police Female Unknown greater than 2 years35+ Croydon Driving Offences Fine over £25 up to Up to £50Metropolitan Police Male Unknown greater than 2 years18-24 Croydon Driving Offences Fine over £100 up to Up to £150Metropolitan Police Female Unknown 1 to 2 years35+ Croydon Driving Offences Fine over £500 up to Up to £750Metropolitan Police Male Unknown 1 to 2 years25-34 The Justice Rooms Driving Offences Fine over £500 up to Up to £750London City of Other Unknown Not Available25-34 The Justice Rooms Driving Offences Fine over £500 up to Up to £750London City of Other Unknown Not Available25-34 Richmond-upon-ThamesDriving Offences Fine over £750 up to Up to £1000Metropolitan Police Other Unknown less than 6 months25-34 Havering Driving Offences Fine over £500 up to Up to £750Metropolitan Police Other Unknown less than 6 months35+ Highbury Corner Violent Crime Community sentence Metropolitan Police Male White 6 months to 1 year35+ Richmond-upon-ThamesPublic Disorder and Possession of weaponsother Metropolitan Police Female White Not Available35+ Croydon Shoplifting Fine over £301 up to Up to £500Metropolitan Police Male White 6 months to 1 year35+ Croydon Shoplifting Fine over £301 up to Up to £500Metropolitan Police Male White 6 months to 1 year25-34 Highbury Corner Burglary Immediate custody Over 3 months and up to and including 6 monthsMetropolitan Police Male White less than 6 months18-24 Horseferry Road Violent Crime Community sentence Metropolitan Police Female White 6 months to 1 year25-34 Horseferry Road Other Crime Community sentence Metropolitan Police Male White less than 6 months18-24 Acton Violent Crime Fine over £100 up to Up to £150Metropolitan Police Male White less than 6 months25-34 Richmond-upon-ThamesDriving Offences Fine over £500 up to Up to £750Metropolitan Police Male Unknown less than 6 months25-34 Thames Burglary Suspended sentence Metropolitan Police Male White less than 6 months18-24 The Justice Rooms Violent Crime other London City of Male Asian Not Available35+ Stratford Violent Crime Fine over £200 up to Up to £250Metropolitan Police Male Asian 6 months to 1 year18-24 Croydon Violent Crime Community sentence Metropolitan Police Male White 6 months to 1 year35+ Horseferry Road Shoplifting Community sentence Metropolitan Police Male Asian 6 months to 1 year18-24 Camberwell Green/Tower BridgeViolent Crime Suspended sentence Metropolitan Police Male White less than 6 months35+ Highbury Corner Violent Crime other Metropolitan Police Male White 6 months to 1 year18-24 Highbury Corner Violent Crime Fine over £150 up to Up to £200Metropolitan Police Male White 6 months to 1 year

Consumer Price Index

• what is the CPI? • measures the changes in the level of prices for goods &

services bought in the UK

• what data are collected?• 180,000 price quotes for 700 items from 20,000 retailers &

service providers in 150 locations

• user demand• increasing interest in easier access to more detailed

CPI data

Releasing CPI Data - Issues

• Risk of identification of retailers - the Mosaic Effect

• Legal and Policy Position

• Ongoing maintenance of data publication

• Capability of the publishing tool to handle large data files

• Data format

• User needs

• Guidance for users

CPI: Reactions from Users

• data received over 8,000 hits

• well received

• largely being used by more expert users

• improve forecasting models and granularity of inflation analysis

DCLG Localities management area profiling tool

• Provides evidence base for DCLG locality management team.

• Brings together DCLG and external datasets

• Queries external sources, over the web, in real-time

• E.g. NOMIS sources on unemployment, mid-year population estimates

• Developed using free, open tools

Common themes

• Know your users and engage them– Focus on the benefits

• Consider privacy– Disclosure control, code of practice, assurances to

data providers

• Value of a phased approach- won’t necessarily get it right first time

Final thoughts

• Privacy and mosaic effects are real – but often a way forward

• Open and linked data can be tricky…but it works – and is vitally important for reputation of official statistics

• Real progress has and continues to be made right across the GSS

• Computer based training and GSS experts are available

• Don’t get left behind!