the revised oecd health systems performance framework: methodological issues and implications for...
Post on 23-Jan-2018
591 Views
Preview:
TRANSCRIPT
The revised OECD Health Systems Performance Framework: methodological issues and implications for comparative public reporting at a global levelFabrizio CarinciProfessor of Health Systems and PolicySchool of Health Sciences, University of SurreyMember of the OECD Expert Group on Health Care Quality Indicators
ForumAgency for Clinical InnovationSydney, AustraliaWednesday, 24th February 2016
f.carinci@surrey.ac.uk
Fabrizio Carinci
OECD Health Care Quality Indicators Project Under the umbrella of the Organisation for Economic Cooperation
and Development (OECD), the ‘Health Care Quality Indicators’ (HCQI) Project was initiated in 2001
The general objective was to help Member States (MS) identify priority areas for quality improvement to provide achievable standards by examining results among best performing countries
In 2006, the OECD released a common conceptual framework for health system performance. Nested “quality matrix” with vertical dimensions of ‘effectiveness’, ‘patient safety’ and ‘responsiveness/patient-centeredness', horizontally subdivided according to levels of health care needs over the life cycle: ‘staying healthy’ for healthy subjects, ‘getting better’ for people affected by a disease, ‘living with illness or disability’ for those with a chronic condition and ‘coping with end of life’ for terminal patients.
Fabrizio Carinci
Matrix Dimensions: Quality
Effectiveness
Achieving desirable outcomes, given the correct provision of evidence-based health care services to all who could benefit
Achieving desirable outcomes, given the correct provision of evidence-based health care services to all who could benefit
Safety
System has the right structures, renders services and attains results in ways that prevent harm to the user, provider, or environment
System has the right structures, renders services and attains results in ways that prevent harm to the user, provider, or environment
Responsiveness/
Patient centeredness
System actually functions by placing the patient/user at the center of its delivery of health care
System actually functions by placing the patient/user at the center of its delivery of health care
Fabrizio Carinci
HCQI Outputs
Since 2007, results of the HCQI project have routinely contributed to international comparisons through the publication of the OECD series ‘Health at a Glance’ and the release of OECD Health Statistics alongside other international health data on expenditure, resources, utilization and outcomes.
In 2013, the OECD HCQI data collection process included a total of 70 indicators covering the following ‘themes’: Primary Care (PC); Acute Care (AC); Mental Health (MH); Cancer Care (CC); Patient Safety (PS) and Patient Experiences (PEs). The collection reports data from 34 countries, including non-OECD member countries eg Singapore and Latvia.
Fabrizio Carinci
NEW DEFINITIONS
Revised OECD-HCQIHealth Systems Performance Framework 2015
Fabrizio Carinci
Criteria used to select HCQIs
Fabrizio Carinci
Carinci F, Van Gool K, Mainz J, Veillard JH, Januel JM, Kim SM, Arispe I, and Klazinga N, Towards actionable international comparisons of health system performance: expert revision of the OECD framework and quality indicators, Int J Qual Health Care. 2015 Apr;27(2):137-46.
Fabrizio Carinci
Effectiveness – Getting better
Carinci F, Van Gool K, Mainz J, Veillard JH, Januel JM, Kim SM, Arispe I, and Klazinga N, Towards actionable international comparisons of health system performance: expert revision of the OECD framework and quality indicators, Int J Qual Health Care. 2015 Apr;27(2):137-46.
Fabrizio Carinci
Effectiveness – Chronic care
Carinci F, Van Gool K, Mainz J, Veillard JH, Januel JM, Kim SM, Arispe I, and Klazinga N, Towards actionable international comparisons of health system performance: expert revision of the OECD framework and quality indicators, Int J Qual Health Care. 2015 Apr;27(2):137-46.
Fabrizio Carinci
Safety – Primary and secondary prevention
Carinci F, Van Gool K, Mainz J, Veillard JH, Januel JM, Kim SM, Arispe I, and Klazinga N, Towards actionable international comparisons of health system performance: expert revision of the OECD framework and quality indicators, Int J Qual Health Care. 2015 Apr;27(2):137-46.
Fabrizio Carinci
Safety – Getting better
Carinci F, Van Gool K, Mainz J, Veillard JH, Januel JM, Kim SM, Arispe I, and Klazinga N, Towards actionable international comparisons of health system performance: expert revision of the OECD framework and quality indicators, Int J Qual Health Care. 2015 Apr;27(2):137-46.
Fabrizio Carinci
Responsiveness / Patient centrednessPrimary and Secondary Prevention
Carinci F, Van Gool K, Mainz J, Veillard JH, Januel JM, Kim SM, Arispe I, and Klazinga N, Towards actionable international comparisons of health system performance: expert revision of the OECD framework and quality indicators, Int J Qual Health Care. 2015 Apr;27(2):137-46.
Fabrizio Carinci
HCQI 2015
Fabrizio Carinci
Mortality after AMISource: OECD Health at a Glance 2015http://www.oecd.org/health/health-systems/health-at-a-glance-19991312.htm
Fabrizio Carinci
Mortality after StrokeSource: OECD Health at a Glance 2015http://www.oecd.org/health/health-systems/health-at-a-glance-19991312.htm
Fabrizio Carinci
OECD Patient Safety Indicators 2015
Fabrizio Carinci
Performance Dashboards: AustraliaSource: OECD Health at a Glance 2015http://www.oecd.org/health/health-systems/health-at-a-glance-19991312.htm
Fabrizio Carinci
Emerging international issues in public reporting
The ability to report on health systems performance is directly related to the capacity of the information infrastructure (eg possibility to perform data linkage at varying degrees of complexity)
The level of implementation may also depend from specific cultural, organizational and political conditions that can vary both within and across countries
A range of OECD studies show that the applicability of standardized definitions is still limited: more efforts are required
Public reporting as a cost effective solution can hardly be demonstrated in general at regional/state/national level
Fabrizio Carinci
Ketelaar NA, Faber MJ, Flottorp S, Rygh LH, Deane KH, Eccles MP. Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations. Cochrane Database Syst Rev. 2011 Nov 9
Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med. 2008 Jan 15;148(2):111-23.
Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement, Med Care. 2003 Jan;41(1 Suppl):I30-8.
Selection pathway Limited evidence
Change pathway Good evidence for hospitals No evidence for individual
providers Scant evidence on improved
clinical outcomes
Is Transparency evidence-based?
Fabrizio Carinci
Why public reporting?
So, why doing public reporting? To implement legislation passed by Parliaments around the world
to achieve higher levels of transparency “no matter what” to respect the citizens' rights to be informed
To achieve greater efficiency by making all public servants directly accountable for their performance
To support pay for performance schemes that can routinely use open benchmarking for continuous quality improvement
To provide the basis (eg hospital reporting system) for specific solutions that can be proved to be cost effective in selected clinical areas or for targeted population subgroups (RCTs)
To improve data quality through a continuous relation with a prurality of users
Fabrizio Carinci
OECD HCQI Bureau Meetings
Rome, ITALY - Friday 26th September 2014
OPPORTUNITIES AND CHALLENGES OF HOSPITAL PERFORMANCE PUBLIC REPORTING AT THE NATIONAL LEVEL: INTERNATIONAL EXPERIENCES AND FUTURE PERSPECTIVES
International workshop
organized by: Italian National Agency for Regional Health Services (AGENAS) , Italian Ministry of HealthProgetto Mattone Internazionale
Seoul, KOREA - Thursday 17th September 2015INTERNATIONAL SYMPOSIUM ON EXPERIENCES OF HOSPITAL PERFORMANCE MEASUREMENT OF OECD MEMBER COUNTRIESOrganized by: Health Insurance Review and Assessment Service (HIRA)
Fabrizio Carinci
Canada: Health Systems Performance Framework
Fabrizio Carinci
Canada: Interactive OECD tool
Fabrizio Carinci
Canada: benchmarking Provinces vs Canada OECD variation of HCQI
Fabrizio Carinci
UK: NHS Choices
Fabrizio Carinci
UK: NHS Consultant choice
Fabrizio Carinci
UK: Funnel Plots Risk-adjusted in-hospital mortality after elective abdominal aortic aneurysm repair: surgeon figures in comparison to national average
Source: National Vascular Registry. 2013 Report on Surgical Outcomes. Consultant level Statistics. June 2013 c
Fabrizio Carinci
Italian National Outcome Programhttp://95.110.213.190/PNEed14_EN/index.php
Outcome measures by Hospital/Local Health Unit
Audit tools
Report Card by Hospital/Local Health Unit
ER Information System
Pilot studies
Fabrizio Carinci
Italy: National Outcomes Program (PNE)
Fabrizio Carinci
Denmark: sundhed.dk
Fabrizio Carinci
Denmark: Danish Quality Registries
Data entryData entry
Clinical observations
Nationalaudit
Regionalaudit
Dataanalysis
Database
Quality improvement
Detailed feedback:Once per year
Standard feedback:Monthly
Results made publically available
Fabrizio Carinci
Korea: decreased variation of AMI quality score
Fabrizio Carinci
Dartboard diagram
To visually represent the results of the six areas, each organization is presented with a specific “target” diagram, divided into five assessment bands.
The closer the organization is to the target level of each performance indicator, the nearer the relative circle will be to the centre.
This approach is nice, but it has known methodological limitations and interpretative drawbacks. There is no perfect graphical representation!
Fabrizio Carinci
Exchanging best practices for fair, transparent and effective communication to the public
PERSONAL JUDGMENT
LEAGUE TABLES SCIENCE
Fabrizio Carinci
Why international collaboration To share best practices that would improve:
understanding of the methods interpretation of national results and interregional variability through
common benchmarks understanding of data limitations and targeted strategies to
strengthen the information infrastructure and overall data quality uptake of effective strategies for quality and outcomes improvement
To gain efficiency through fostered international cooperation, allowing to share costs and gain access to high level skills/tools that are difficult to obtain at national level
To help defining the “Essential Levels of Health Information” for the definition of international standards of performance evaluation
Fabrizio Carinci
Essential levels of health information in EuropeCarinci F. Health Policy. 2015 Apr;119(4):530-8.
Recently, the European Commission coordinated a discussion between MS and Expert Networks on how (and whether) to realize a EU health information system.
The conceptual framework for the „Essential Levels of Health Information” originated from this experience
Fabrizio Carinci
Essential levels of health information in EuropeCarinci F. Health Policy. 2015 Apr;119(4):530-8.
Fabrizio Carinci
Open questions
Can system-level indicators be used at provider level? How can a system-oriented performance framework be adapted to hospitals or specific providers? Which OECD quality indicators can be adopted for hospital reporting?
How can we identify international benchmarks for continuous hospital performance reporting? Which indicators shall be routinely communicated to the public? How to share relevant data, and to which level of detail for reliable analyses?
Which countries shall be routinely compared (EU, OECD, universal/insurance, or any rule to define “peer countries”)?
How to communicate results to the public (best practices, new approaches)? Is public reporting effective, sustainable and practically convenient? Which forms of new communication and how can citizens contribute to the process?
Lessons from the world......your turn!
Fabrizio Carinci
ForumAgency for Clinical InnovationSydney, AustraliaWednesday, 24th February 2016
Thanks for your attention!
top related