quality assurance activities at eurostat
DESCRIPTION
Quality assurance activities at EUROSTAT. CCSA Conference Helsinki, 6-7 May 2010 Martina Hahn, Eurostat. Structure of the presentation. Code of Practice Institutional environment Process quality Output quality ESS Quality Architecture. 11/8/2014. European Statistics Code of Practice. - PowerPoint PPT PresentationTRANSCRIPT
Quality assurance activities Quality assurance activities at EUROSTATat EUROSTAT
CCSA ConferenceHelsinki, 6-7 May 2010
Martina Hahn, Eurostat
204/20/23
Structure of the presentation
Code of Practice
Institutional environment
Process quality
Output quality
ESS Quality Architecture
4
Content of the Code of Practice
3 headings – 15 principles – 77 indicators
Institutional environmentProfessional independence, Mandate for data collection, Adequacy of Resources, Quality commitment, Statistical confidentiality, Impartiality and objectivity
Statistical processesSound methodology, Appropriate statistical procedures, Non-excessive burden on respondents, Cost effectiveness
Statistical outputRelevance, Accuracy and reliability, Timeliness and punctuality, Coherence and comparability, Accessibility and clarity
5
Institutional environment
ESS-wide peer reviews during 2006-08– to monitor the implementation of the Code of Practice– to boost quality, enhance credibility– to share ESS best practices and transfer knowledge
Establishment of the European Statistical Governance Advisory Board (ESGAB):– to provide an independent overview of the implementation of the
Code of Practice by Eurostat– to advise Eurostat and the ESSC on updating and communicating
the Code to users and data providers Establishment of general guidelines (e.g. confidentiality
manual)
7
ESS Quality Coaching
Two quality managers work together to address specific improvement actions
Based on experience, dialogue and information exchange
Agreement on commitments from both sides
Joint working – not doctor/patient relation
17 coaching pairs took part
Coaching exercise completed at the end of 2009
Appropriate where there are clear standards and guidelines
8
Process and output quality: the Eurostat Quality Assurance Framework
A system of coordinated methods and tools to ensure a sustainable level of quality of processes and outputs in line with the Code of Practice:
– product/ output quality requirements are explicitely documented– processes are defined and made known to all staff– the correct implementation of the processes is monitored– users are informed on the quality of the products and possible
deficits– a procedure to guarantee that the necessary improvement
measures are planned, implemented and evaluated
1010
The Eurostat Quality Assurance Framework
User Requirement Standards
III. CONFORMITY Evaluation function
Quality assessments
Rolling reviews Cost/benefit 5YP
Best practices
II. EVALUATION CVD steps | | | | | | Quality diagram
Improvement actionsQuality
Barometer
AMP
I. DOCUMENTATION MEASUREMENT
Process description
Quality reports
SDMX
Production processess Statistical products User perception
Eurostat Institutional/Legal environment
Labelling
ASSESSMENT CHECKLIST
User satisfaction survey
1111
Process Quality Assessments (1)C
on
cep
tua
l fra
me
wo
rk (
2)
IT conditions (11) – Management, planning and legislation (12) – Staff, work conditions and competence (13)
User needs(3)
Data collection
(4)
Validation
Country level (5)International level (6)
Confidentiality (7)
Dissemination (9)
Documentation (8)
Follow-up (10)
Common tool: The Process Quality Assessment Checklist (based on the International ISPAC)
Method: chronologically examining the process
Outputs:– Assessment Diagram– Summary Assessment report with improvement actions– Highlight of a good practice
1212
Process Quality Assessments (2)
Each process is matched with an assessment category
Self-Assessment
Prod. unit: fills in Checklist, produces outputs
B1: provides support on demand
Supported Self-Assessment
Prod. unit: in control of assessment,participates in all steps
B1: pre-fills the Checklist, produces the outputs
Peer reviewSame as
Supported Self-Assessment with one or more peers (int/ext)
taking part in all steps
Rolling reviewA comprehensive reviewbased on the checklist
implemented by ext.contractor, involving partner & user surveys
and cost-benefit analysis
Process 1
Characteristics:- Periodicity- Legal Basis- Output- ESTAT intervention-availability of quality report
Process 4Process 2 Process 3 Process 5
13
Process Quality Assessments Implementation Plan: Principles
Minimize burden for units
– Test of the approach in advance
– Support from the Quality unit
– Flexibility
Build on existing information
– Quality reports
– Process analysis
Integrate QA with other horizontal activities
15
Process Quality Assessment Outputs (2)
Coherence General coherence
Timeliness Timeliness of final publication
Accuracy Overall accuracy
Relevance User satisfaction
Accessibility and clarityOverall quality of metadata
Comparability
Comparabilityover time
Comparabilityacross countries
0
1
2
3
4
5
Assessment Diagram
1616
The Eurostat Quality Assurance Framework
User Requirement Standards
III. CONFORMITY Evaluation function
Quality assessments
Rolling reviews Cost/benefit 5YP
Best practices
II. EVALUATION CVD steps | | | | | | Quality diagram
Improvement actionsQuality
Barometer
AMP
I. DOCUMENTATION MEASUREMENT
Process description
Quality reports
SDMX
Production processess Statistical products User perception
Eurostat Institutional/Legal environment
Labelling
ASSESSMENT CHECKLIST
User satisfaction survey
17
ESS Standard & Handbook for Quality Reports (1)
Provide (mainly producer oriented) recommendations / guidelines for the preparation of comprehensive quality reports for statistical processes and their outputs by distinguishing between 6 types of statistical processes:
– Sample survey– Census– Statistical process using administrative sources– Statistical process involving multiple data sources– Price or other economic index process– Statistical compilation
18
ESS Standard & Handbook for Quality Reports (2)
Promote harmonized quality reporting across statistical processes and Member States => facilitate comparisons
Are applicable to National Statistical Offices and Eurostat in their roles as producers of statistics
Also include 21 quality and performance indicators Handbook: is more detailed, with lots of examples and
explanations, describes the quality concepts, explains how the different quality documents relate to each other and is aligned with the 15 Principles of the Code of Practice