health systems and policy analysis policy brief 27 · 2018-02-21 · keywords: delivery of health...

No. 27ISSN 1997-8073

POLICY BRIEF 27

HEALTH SYSTEMS AND POLICY ANALYSIS

How to make sense of health system efficiency comparisons?

Jonathan CylusIrene PapanicolasPeter C Smith

Keywords:

Delivery of Health Care – economics

Efficiency, Organizational – economics

This policy brief is one of a new series to meet the needs of policy-makers and health system managers. The aim is to develop key messages to support evidence-informed policy-making and the editors will continue to strengthen the series by working with authors to improve the consideration given to policy options and implementation.

© World Health Organization 2017 (acting as the host organization for, and secretariat of, the European Observatory on Health Systems and Policies)

Address requests about publications of the WHO Regional Office for Europe to:

Publications WHO Regional Office for Europe UN City, Marmorvej 51 DK-2100 Copenhagen Ø, Denmark

Alternatively, complete an online request form for documentation, health information, or for permission to quote or translate, on the Regional Office web site (http://www.euro.who.int/pubrequest).

All rights reserved. The Regional Office for Europe of the World Health Organization welcomes requests for permission to reproduce or translate its publications, in part or in full.

The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries.

The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters.

All reasonable precautions have been taken by the World Health Organization to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either express or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for damages arising from its use. The views expressed by authors, editors, or expert groups do not necessarily represent the decisions or the stated policy of the World Health Organization.

What is a Policy Brief?A policy brief is a short publication specifically designed to provide policy-makers with evidence on a policy question or priority. Policy briefs:

• Bring together existing evidence and present it in an accessible format

• Use systematic methods and make these transparent so that users can have confidence in the material

• Tailor the way evidence is identified and synthesised to reflect the nature of the policy question and the evidence available

• Are underpinned by a formal and rigorous open peer review process to ensure the independence of the evidence presented.

Each brief has a one page key messages section; a two page executive summary giving a succinct overview of the findings; and a 20 page review setting out the evidence. The idea is to provide instant access to key information and additional detail for those involved in drafting, informing or advising on the policy issue.

Policy briefs provide evidence for policy-makers not policy advice. They do not seek to explain or advocate a policy position but to set out clearly what is known about it. They may outline the evidence on different prospective policy options and on implementation issues, but they do not promote a particular option or act as a manual for implementation.

EditorsErica Richardson

Series EditorErica Richardson

Associate EditorsJosep FiguerasHans KlugeSuszy LessofDavid McDaidElias MossialosGovin Permanand

Managing EditorsJonathan NorthCaroline White

Contents

Boxes, figures and tables 3

Key messages 4

Executive summary 5

Policy brief 7

Introduction 7

Findings 9

Discussion 19

Conclusions 21

References 22

Appendix 1 23

Authors

Jonathan Cylus, European Observatory on HealthSystems and Policies and London School of Economics & Political Science, United Kingdom

Irene Papanicolas, London School of Economics& Political Science, United Kingdom

Peter C Smith, Imperial College Business School,United Kingdom

page


Acknowledgements

The authors and editors aregrateful to Matthew Jowett(WHO) who commented on thispublication and contributed hisexpertise.

ISSN 1997-8073

PB_health_system_comparisons_v2_Policy_brief_A4 1/06/17 10:19 Page 1

2

Policy brief

How do Policy Briefs bring the evidence together?

There is no one single way of collecting evidence to inform policy-making. Different approaches are appropriate for different policy issues, so the Observatory briefs draw on a mix of methodologies(see Figure A) and explain transparently the different methods usedand how these have been combined. This allows users to understand the nature and limits of the evidence.

There are two main ‘categories’ of briefs that can be distinguishedby method and further ‘sub-sets’ of briefs that can be mapped alonga spectrum:

• A rapid evidence assessment: This is a targeted review of theavailable literature and requires authors to define key terms, setout explicit search strategies and be clear about what is excluded.

• Comparative country mapping: These use a case study approach and combine document reviews and consultation withappropriate technical and country experts. These fall into twogroups depending on whether they prioritize depth or breadth.

• Introductory overview: These briefs have a different objective tothe rapid evidence assessments but use a similar approach. Litera-ture is targeted and reviewed with the aim of explaining a subjectto ‘beginners’.

Most briefs, however, will draw upon a mix of methods and it is forthis reason that a ‘methods’ box is included in the introduction toeach brief, signalling transparently that methods are explicit, robustand replicable and showing how they are appropriate to the policyquestion.

V

Rapidevidence

assessment

Introductoryoverview

SystematicReview

Meta-NarrativeReview

RapidReview

ScopingStudy

NarrativeReview

MultipleCase Study

InstrumentalCase Study

V VV V

Countrymapping(breadth)

Countrymapping(depth)

POLICY BRIEFS

Source: Erica Richardson

Figure A: The policy brief spectrum


3


Acronyms

DEA data envelopment analysis

DRG diagnosis-related group

EQ-5D EuroQol Five Dimensions questionnaire

OECD Organisation for Economic Cooperation andDevelopment

PPP purchasing power parity

PROM patient-reported outcome measure

QALY quality-adjusted life year

SF-36 Short Form-36

WHO World Health Organization

Boxes, tables and figures

Boxes

Box 1: Methods

Box 2: Generic prescribing

Box 3: Operations per specialist

Box 4: Duplicate tests

Box 5: Average length of stay for particular conditions

Box 6: Expenditures per case

Box 7: Cost per QALY

Box 8: Hospital readmission rates

Tables

Table 1: Results from exploratory efficiency analysis

Figures

Figure 1: Visualization of analytical framework

Figure 2: The production process in hospital care with common indicators along the production pathway


4

Policy brief

Key messages

• The inexorable growth in health expenditure has led to awidespread demand for efficiency improvements.

• There is no single metric or set of indicators that will givethe complete picture of health system efficiency in acountry.

• The real causes of any identified inefficiencies need to becarefully identified and analysed to inform good policy-making.

• More nuanced indicators require more standardized anddetailed cost accounting data and linked datasets and registries.

• This policy brief gives a useful framework for understand-ing and interpreting the healthcare efficiency metrics thatare widely used.


5


Executive summary

There is ample evidence to suggest that inefficiency is amajor problem in all health systems. Identifying variability inefficiency is therefore of great importance, and becomes increasingly relevant to health systems grappling with significant or sudden resource constraints.

But how exactly can we understand and evaluate efficiencyin health systems? Although the core idea of efficiency iseasy to understand in principle – maximizing valued outputsrelative to inputs, or vice versa – in practice it can be challenging to measure and interpret metrics, as well as toidentify a course of action to remedy any observedinefficiencies.

In this brief we propose and apply an analytic frameworkthat seeks to facilitate the interpretation of health systemefficiency measures. We recommend that for any efficiencyindicator, five aspects should be explicitly considered:

• the entity to be assessed

• the outputs (or outcomes) under consideration

• the inputs under consideration

• the external influences on attainment

• the links with the rest of the health system.

By thinking through each of these in detail, we argue that itbecomes easier to understand, interpret and respond to vari-ability in health system efficiency measures.

For instance, it may be that the accountable entity responsi-ble for the observed production of health care outputs oroutcomes is not obviously identifiable. Likewise, it may bethat an indicator is impaired by some of the numerous, well-known challenges associated with measuring and matchingthe inputs and outputs of health care organizations. Fur-thermore, measurement may be complicated by externalinfluences that have effects on health. The health sectorhas little direct opportunity to influence many of these fac-tors, such as social determinants of health, so in principleany measure of efficiency must take account of their impacton measured levels of attainment. Finally, there are impor-tant links within the health system itself that must betaken into account. It is quite conceivable that some compo-nents are functioning efficiently within an inefficient broaderhealth system.

Recognizing that it will always be necessary to look at sev-eral metrics across various levels and sectors of the healthsystem in order to determine the magnitude and nature ofinefficiency, we apply the framework across a wide range of16 indicators that capture various production processeswithin the health system. These include common measures,such as average length of patient stay, incidence of duplicatetesting, expenditures per case and generic prescribing rates,as well as popular frontier-based methods. As an illustrativeexample, using Organisation for Economic Cooperation andDevelopment (OECD) data, we apply the framework toexplore the challenges associated with frontier-based cross-country efficiency comparisons at the system level. Theframework helps to shed light on what each metric can andcannot reveal about health system efficiency, and providesguidance on how to pinpoint the sources of any observedvariability.

Understanding how proficiently discrete processes within thehealth system are completed will offer important glimpsesinto how well the system functions and provide some evi-dence of where it may be possible to make efficiency gains.However, the difficulties in measuring and operationalizingefficiency metrics and practices across health systems havegiven rise to a number of efficiency myths and oversimplifi-cations regarding how to improve health system efficiency,and even what it means to be efficient. To understand themany nuances of well-known statements about health sys-tem efficiency (as well as to dispel common health policy‘myths’), it is essential to review and fully understand themethodological techniques that are used to measure healthsystem efficiency, as well as the ways in which efficiency metrics can be used to make well-informed policy and man-agerial decisions. These important topics are explored indetail in a full volume on measuring health system efficiency,produced by the European Observatory on Health Systemsand Policies and entitled ‘Health system efficiency: How tomake measurement matter for policy and management’ [1],as part of the Observatory’s large programme of research onperformance assessment [2,3]. Building on this work, in thispolicy brief we aim to provide a more nuanced approachtowards the pursuit of efficiency – one that seeks to identifyefficient practice and understand the reasons for inefficiency,allowing the reader to critically evaluate existing metrics andchallenge persistent myths.


6

Policy brief


7


Introduction

Why is health system efficiency important?

Health system efficiency seeks to capture the extent towhich the inputs to the health system, in the form of expen-ditures and other resources, are used to secure valued healthsystem goals. Its pursuit is one of the central preoccupationsof health policy-makers and managers, and is justifiably acause for concern. In many other sectors of the economy,consumer preferences help to ensure that the most valuedoutputs are produced at market prices. However, there arenumerous, well-known market failures in the health sectorthat mean that the traditional market mechanisms cannotwork, allowing poor quality or inappropriate care to persistat high prices if no policy action is taken [4].

Inefficient use of health system resources poses serious concerns, for a number of reasons:

• It may limit health gains for patients who have receivedtreatment, because they do not receive the best possiblecare available within the health system’s resource limits.

• By consuming excess resources, inefficient treatment maydeny treatment to other patients who could have bene-fited from treatment had resources been better used.

• Inefficient use of resources in the health sector may sacrifice consumption opportunities elsewhere in theeconomy, such as education.

• Particularly in higher income countries where publicsources dominate funding for health, suboptimal use ofresources may reduce society’s willingness to contribute tothe funding of health services, thereby harming social solidarity, health system performance and social welfare.

Therefore, as well as its instrumental value, tackling inefficiency has an important accountability value: to reassure payers that their money is being spent wisely, andto reassure patients, caregivers and the general populationthat their claims on the health system are being treated fairlyand consistently. Also, health care funders (including govern-ments, insurance organizations and households) areinterested in knowing which systems, providers and treat-ments contribute the largest health gains in relation to thelevel of resources they consume. Efficiency becomes particu-larly important in light of financial pressures and concernsover the long-term financial sustainability of many healthsystems, as decision-makers seek to ensure and demonstratethat health care resources are put to good use. When usedappropriately, efficiency indicators can be important tools tohelp decision-makers determine whether resources are allocated optimally, and to pinpoint which parts of thehealth system are not performing as well as they should be.

How is health system efficiency measured?

The measurement of health system efficiency is beguilinglysimple, represented at its most straightforward as a ratio ofresources consumed (health system inputs) to some measure

of the valued health system outputs they create. In effect,this creates a generic metric of ‘resource use per unit ofhealth system output’. Yet putting this simple notion intopractice can be complex. Within the health system as awhole, there is a seemingly infinite set of interlinkedprocesses that could independently be evaluated and foundto be efficient or inefficient. This has given rise to a plethoraof apparently disconnected indicators that give glimpses ofcertain aspects of efficiency, but rarely offer a comprehensiveoverview.

Economists view the transformation of inputs into valuedoutputs as a ‘production function’, which indicates the maxi-mum feasible level of output for a given set of inputs. Anylevel of production below that maximum is an indication ofinefficiency [5]. The concept of a production function can beapplied to the functioning of very detailed micro units (suchas an individual physician’s practice) through to huge macrounits (such as the entire health system). Whatever level ischosen, the intention is to offer insights into the successwith which health system resources are transformed intophysical outputs (such as patient consultations), or (moreambitiously) into valued outcomes (such as improved health).

However, a health system may not perform as well as itcould; processes in the health system may be inefficient fortwo distinct, but related, reasons. Firstly, health systeminputs, such as expenditures or other resources, may bedirected towards creating some outputs that are not priorities for society; for example, providing very high-costend-of-life cancer treatments may create benefits for theindividuals involved, but society may judge that the limitedmoney available to the health system could be better spenton other interventions that create (in aggregate) largerhealth gains. Secondly, there could be waste of inputs in theprocess of producing valued health system outputs. Misuseof inputs at any stage of the production process will meanthat there will be less output than could have been achievedfor a given initial level of resources. If a health system doesnot secure the lowest cost of medicines and other inputs,less output in terms either of: 1) the quantity of patientstreated; or 2) the quality of care provided, will be possiblefor a given level of expenditure. Economists refer to these asallocative efficiency and technical efficiency, respectively.

Allocative efficiency

Allocative efficiency can be used to scrutinize either thechoice of outputs or the choice of inputs. On the outputside, it examines whether limited resources are directedtowards producing the ‘correct’ mix of health care outputs.On the input side it guides decisions about what to includeor exclude from the package of benefits offered.

Allocative efficiency is central to the work of Health Technol-ogy Assessment agencies, which often use expected gains inquality-adjusted life years (QALYs) as the central measure ofthe benefits of a treatment, and ‘cost per QALY’ as a maincost–effectiveness criterion for determining whether or notto encourage the adoption of a new treatment or discour-age an existing one. The assumption underlying thisapproach is that payers wish to see their financial contributions used to maximize health gain. Under thesecircumstances, a provider would not be allocatively efficient

Policy brief


8

Policy brief

if it produces treatments with low levels of cost–effective-ness, because the inputs used could be better deployedproducing outputs with higher potential health gains.

Allocative efficiency can also be considered at a broad sectoral level to examine whether the correct mix of healthservices is funded, so that at a given level of total expendi-ture, health outcomes are maximized. For example, anallocatively efficient health system allocates funds betweensectors like prevention, primary care, hospital care and long-term care to maximize health-related outcomes in line withsocietal preferences. Allocative efficiency indicators at thislevel should indicate whether a health system is performingpoorly because of a misallocation of resources between sectors. Metrics such as excessive antibiotic prescribing, orexcessive referrals to hospital specialists, might be indicatorsof allocative inefficiency.

Allocative efficiency can also examine whether an optimalmix of inputs has been used – for example, the mix of labourskills – to produce its chosen outputs, given the differentprices of those inputs. On the input side, there may also bepotential for a wide range of indicators of allocative ineffi-ciency, in the form of ‘inappropriate’ use of health systemresources; for example, metrics relating to the skill mix oflabour inputs can be prepared at a whole system level or alocal level.

Consideration of the different levels of allocative efficiencyhighlights the fact that the health system may containorganizations (such as clinical teams) that are performingperfectly efficiently in producing what has been asked ofthem. However, consideration of a broader societal perspec-tive may indicate that strategic decision-makers havemisallocated resources between, for example, preventive andcurative services, and that the efficient teams are operatingwithin an inefficient system.

A great deal of emphasis regarding allocative efficiency hasbeen placed on treatment guidelines and clinical pathways.Assuming guidelines have been prepared in line with theprinciples of cost–effectiveness, they can also be used retrospectively to explore whether provider organizationsand practitioners have deviated from policy intentions anddelivered what can be thought of as ‘inappropriate’ care.This may take the form of obviously suboptimal allocation ofresources, such as hospital treatment for conditions that donot typically require such a resource-intensive setting. Thewide range of metrics for treatment taking place in thewrong setting (such as emergency admissions for ambula-tory care sensitive conditions or delayed discharges fromhospital to a community setting) is an indication of theheightened interest in this area.

Retrospective allocative efficiency could also take the form oftreatments that confer health benefits but which policy- makers have decided not to be priorities, perhaps implicitlybecause their cost–effectiveness ratios are above the system’schosen cost–effectiveness threshold. End-of-life cancer drugsare emerging as a particularly challenging example of suchtreatments in some systems, but there are many other poten-tial metrics indicating deviation from cost-effective treatments.

Technical efficiency

In contrast, technical efficiency indicates how far the systemis minimizing the use of inputs in producing its chosen outputs, regardless of the value placed on those outputs. Analternative but equivalent formulation is to say that it is maximizing its outputs given its chosen level of inputs. Anyvariation in performance from the highest feasible level ofproduction is an indication of technical inefficiency, or waste.The main interest in technical efficiency is therefore in theoperational performance of the entity, rather than its strate-gic choices about the outputs it produces or the inputs itconsumes.

The analysis and measurement of technical efficiency mayappear less demanding than that of allocative inefficiency. It does not require prior specification of guidelines and,instead, is usually entirely an examination of whether theoutputs produced by the entity under scrutiny were maxi-mized, given its inputs and external circumstances.Comparative performance therefore lies at the core of mostanalyses of technical inefficiency.

A major class of technical efficiency indicators examines thetotal costs of producing a specified unit of output, in theform, for example, of costs per patient within a specifieddisease category. The most celebrated version of such ‘unitcost’ indicators forms the basis for the various systems ofdiagnosis-related groups (DRGs), initially for use in the hospi-tal sector in the USA [6]. These methods cluster patients intoa manageable number of groups that are homogeneouswith respect to medical condition and expected costs. In thefirst instance, a hospital’s average unit cost within a DRGcategory can be compared with a national ‘reference’ costfor that DRG, often the average of unit costs across all comparable institutions [7]. This metric in itself may provideuseful information on the functioning of individual specialities within the hospital.

Moreover, the number of cases in each DRG can then bemultiplied by the relevant reference cost to derive the‘expected’ aggregate costs of treating all the hospital’spatients (if the national reference costs applied). These normative costs can be compared with actual costs to yieldan index of the hospital’s relative efficiency. This approachhas usually been used in the hospital sector, but can beextended to many other units of analysis in the health system. An important barrier to applying the DRG methodeffectively is the great complexity of hospital cost structures.This has led to major challenges in allocating many hospitalcosts to specific patients and activities, and the associatedvariation in accounting practices is one of the reasons for theapparent variation in unit costs. Where it is feasible, greaterstandardization of accounting practices would seem to be animportant priority.

Unit cost metrics offer insights into the overall technical efficiency of the entity (relative to other such entities), butgive little operational guidance as to the reasons why suchinefficiency arises, nor any insights into the allocative effi-ciency of the entity. Therefore, aggregate measures oftechnical inefficiency can usefully be augmented by more


9


specific metrics of operational waste, either in some specified form (such as excessive prices paid for inputs, comparatively long lengths of stay or unnecessary duplication).

Common myths about health system efficiency

The difficulties in measuring and operationalizing efficiencymetrics have contributed to the promulgation of a numberof myths and oversimplifications about efficiency that arenot supported by evidence. These include assertions such as:

• improved efficiency is synonymous with cost containment

• improved quality of health care will necessarily lead toimproved efficiency

• increased efficiency will necessarily lead to poorer healthoutcomes and less equitable access to care

• increased intensity of resource use (for example, increasedbed occupancy) is always a sign of increased efficiency

• increased emphasis on preventive medicine will inevitablyreduce health system costs and increase efficiency

• adopting methods such as DRGs for paying providers willalways improve system efficiency.

To understand why these are myths, it is necessary first tounderstand the concept of, as well as methodological tech-niques used to measure, efficiency in health systems. Toassist in this endeavour, in this policy brief we present andapply a framework that is designed to guide an analyst ordecision-maker’s approach to understanding and interpret-ing efficiency indicators (see Box 1).

Box 1: Methods

This narrative review of health system efficiency metrics builds on afull volume on measuring health system efficiency produced by theEuropean Observatory on Health Systems and Policies [1], as part ofthe Observatory’s large programme of research on performanceassessment [2,3]. This policy brief provides an analytical frameworkfor decision-makers and researchers who want to understand howhealth system efficiency measurements work in practice and whatthey really mean.

Findings

Framework

Whether inefficiency takes the form of inputs misdirectedtowards relatively low-value health outputs, or inputs lost inthe production of valued health outputs, a first step towardsremedial actions is to properly recognize the nature of anysuch inefficiency. It is important to be keenly aware of whata specific efficiency indicator does or does not tell you, andto be able to identify ways in which an indicator may beinformative, misleading or reflect only some aspect of a production process. To this end, it is necessary to understandwhat is actually being measured and, importantly, how tointerpret the findings from an efficiency analysis.

To facilitate this, we have developed an analytical frameworkshowing the five aspects of any efficiency indicator thatshould be explicitly considered (see Figure 1). Thesecomprise:

• the entity to be assessed

• the outputs (or outcomes) under consideration

• the inputs under consideration

• the external influences on attainment

• the links with the rest of the health system.

Figure 1: Visualization of analytical framework

i

i

i

i

i

Externalinfluences?

Links to the health system?

Inputs? Outputs?

Entity?

Links to the health system?

i

i

i i iii

i


10

Policy brief

In the following sections we discuss each of these in greaterdetail.

Identifying the accountable entity: what is being evaluated?

An assessment of efficiency first depends crucially on under-standing the boundaries of the entity under scrutiny. At thefinest level of analysis, an entity could be a single treatment,where the goal is to assess its cost relative to its expectedbenefit. At the other extreme, the entity could be the entirehealth system, defined by the World Health Organization(WHO) as ‘all the activities whose primary purpose is to pro-mote, restore or maintain health’ [8].

Most often, however, efficiency measurement takes place atsome intermediate level, where the actions of individuals orgroups of practitioners, teams, hospitals or other organiza-tions within the health system are assessed. Whatever thechosen level, as a general principle, it is important that anyanalysis reflects an entity for which clear accountability canbe determined, whether it is the whole health system, ahealth services organization or an individual physician. Onlythen can the relevant agent, whether it is the government,management board or physician, be held to account for thelevel of performance revealed by the analysis.

Almost all efficiency analysis relies on comparisons, so it isimportant to ensure that the entities being compared aregenuinely similar. A great deal of efficiency analysis is con-cerned with securing such comparability. If organizationalentities are operating in different circumstances, perhapsbecause the population cared for or the patients beingtreated differ markedly, some sort of adjustment will beneeded to ensure that like is being compared with like (see‘What are the external influences?’ on page 11).

Almost all organizations and practitioners operate withinprofound operational constraints, created by the legal, pro-fessional and financial environments within which they mustfunction. In assigning proper accountability for efficiencyshortcomings, it is important to identify the real source ofthe weakness, which may lie beyond the control of theimmediate entity under scrutiny; for example, a communitynurse practising in a remote rural area may necessarilyappear less efficient when assessed using a metric such as‘patient encounters per month’. However, local geographymay preclude any increase and the nurse may be performingas well as can be expected within the constrained circum-stances.

When choosing the entity to evaluate, there is often a diffi-cult trade-off to be made between scrutiny of the detailedlocal performance of the system and scrutiny of the broadersystem-wide performance. In general terms, the perform-ance of individual clinicians and clinical teams may be highlydependent on the inputs from other parts of the system; forexample, the performance of the emergency department insupporting the work of a maternity unit. Furthermore, deter-mining the resources used by local teams can be challengingfrom an accountancy perspective. On the other hand, mov-ing the analysis to a more aggregate level, whilst obviatingthe need to identify in detail who undertakes what activity,can make it difficult to identify what is causing apparentlyinefficient care.

What are the outputs under consideration?

In the context of efficiency analysis in the health sector, twofundamental issues need to be considered:

1. How should the outputs of the health care sector bedefined?

2. What value should be attached to them?

The consensus is that, in principle, health care outputsshould properly be defined in terms of the health gains pro-duced. However, organizations rarely collect relevant routineinformation about health gains and, in any case, the conceptof health gain has proved challenging to make operational.In most circumstances, it is rarely possible to observe a base-line or a counterfactual (the health status that would havebeen secured in the absence of an intervention).Furthermore, the heterogeneity of service users, the multidi-mensional nature of ‘health’ and the intrinsic measurementdifficulties add to the complexity.

Recent progress in the use of patient-reported outcomemeasures (PROMs) offers some prospect of making moresecure comparisons, at least of providers delivering a specifictreatment [9], and a number of well-established measure-ment instruments have been developed that could be usedto collect before/after measures of treatment effects, such asthe EuroQol Five Dimensions questionnaire (EQ-5D) and theShort Form-36 (SF-36) [10,11]. Although there remain manyunresolved issues surrounding the precise specification andanalysis of such instruments, their use should be consideredwhenever there are likely to be material differences in theclinical quality of different organizations.

In practice, however, analysts are often limited to examiningefficiency by measuring activities; for example, in the form ofpatients treated, operations undertaken, or outpatients seen.Such measures are manifestly inadequate, as they fail to cap-ture variations in the effectiveness (or quality) of the healthcare delivered. Yet there is often in practice no alternative tousing such incomplete measures of activity in lieu of healthcare outcomes.

Measuring activities can also address a fundamental difficultyof outcome measurement – identifying how much of thevariation in outcomes is directly attributable to the actions ofthe health care organization; for example, mortality after asurgical procedure is likely to be influenced by numerous fac-tors beyond the control of the provider or even the healthsystem. In some circumstances, such considerations can beaccommodated by careful use of risk-adjustment methods.However, there is often no analytically satisfactory way ofadjusting for environmental influences on outcomes, inwhich case analysing care activities may instead offer a moremeaningful insight into organizational performance, provid-ing there is some confidence that the activities will, in time,lead to health improvement.

What are the inputs under consideration?

The input side of efficiency metrics is usually considered tobe less problematic than the output side. Physical inputs canoften be measured more accurately than outputs, or can besummarized in the form of a measure of costs. However,even the specification of inputs can give rise to serious conceptual and practical difficulties.


11


A fundamental decision that must be taken is the level ofdisaggregation of inputs to be specified. At one extreme, asingle measure of aggregate inputs (in the form of totalcosts) might be used. The input side of the efficiency ratiothen effectively becomes ‘costs’. This approach assumes thatthe organizations under scrutiny are free to deploy inputsefficiently, taking account of relative prices. In practice, someaspects of the input mix are often beyond the control of theorganization, at least in the short term; for example, thestock of capital can usually be changed only in the longerterm. In these circumstances, it may be important to disag-gregate the inputs in order to capture the different inputmixes that organizations have inherited.

Labour inputs can usually be measured with some degree ofaccuracy, often disaggregated by skill level. An importantissue is therefore how much aggregation of labour inputs touse before pursuing an efficiency analysis. Unless there is aspecific interest in the deployment of different labour types,it may be appropriate to aggregate into a single measure oflabour input, weighting the various labour inputs by theirrelative wages. There may be little merit in disaggregationunless there is a specific interest in the relationship betweenefficiency and the mix of labour inputs employed. Undersuch circumstances, metrics using measures of labour inputdisaggregated by skill type may be valuable. Such analysismay yield useful policy insights into the gains to be securedfrom (say) substituting some types of labour for another.

Although labour inputs can be measured readily at an organizational level, problems may arise if the interest is inexamining the efficiency of subunits within organizations,such as, for example, operating theatres within hospitals. Asthe unit of observation within the hospital becomes smaller(department, team, surgeon or patient), it becomes increas-ingly difficult to attribute labour inputs. Staff often workacross a number of subunits but information systems cannotusually track their input across those units with any accuracy.Particular care should be exercised when developing metricsthat rely heavily on input measures of self-reported alloca-tions of professional time.

In general, capital is a key input whose misuse can be amajor source of inefficiency. However, incorporating meas-ures of capital into the efficiency analysis is challenging. Thisis partly because of the difficulty of measuring capital stockand partly due to problems in attributing its use to any par-ticular activity or time period. Measures of capital are oftenvery rudimentary and even misleading; for example,accounting measures of the depreciation of physical stockusually offer little meaningful indication of capitalconsumed. Indeed, in practice, analysts may have to resortto very crude measures – for example, the number of hospital beds or floor space as a proxy for physical capital.Regardless of the cost accounting method employed, it isimperative that all entities being compared use the sameapproach in order not to introduce unwarranted variability.Furthermore, non-physical capital inputs (such as health promotion efforts) are important capital investments thatcan be difficult to attribute directly to health outcomes.

As with all modelling, efficiency metrics should be developedaccording to the intentions of the analysis. If the interest is innarrow, short-run use of existing resources, then it may be

relevant to disaggregate inputs in order to reflect theresources currently at the disposal of management. If alonger-term, less constrained analysis is required, then a single measure of ‘total costs’ may be a perfectly adequateindicator of the entity’s physical inputs.

What are the external influences?

In many contexts, a separate class of factors affects organizational capacity – the external or ‘environmental’determinants of performance. These are influences on theentity, beyond its control, that reflect the external environ-ment within which it must operate. For example:

• Population mortality rates are heavily dependent on thedemographic structure of the population under consider-ation and the broader social determinants of health.

• Intensity of resource use is usually highly contingent onthe severity of disease.

• The costs to emergency ambulance services of satisfyingservice standards (such as speed of attendance) maydepend on local geography and settlement patterns.

• Health outcomes achieved by clinical teams may be highlydependent on the health and social characteristics of thepatient group they serve.

There is often considerable debate as to what environmentalfactors are considered to be ‘controllable’. This will be a keyissue for any scrutiny of efficiency and for holding relevantmanagement to account. The choice of whether to adjustfor such external influences is likely to be heavily dependenton the degree of autonomy enjoyed by management, aswell as on whether the purpose of the analysis is short-runand tactical, or longer-run and strategic. In the short run,almost all input factors and external constraints will be fixed;in the long run, depending on the level of autonomy, manymay be changeable. In many circumstances it will be appro-priate to consider efficiency metrics both with and withoutadjustment for external factors.

Broadly speaking, environmental factors can be taken intoaccount in efficiency analyses in three ways:

• restrict comparison only to entities operating within asimilarly constrained environment

• model the constraints explicitly, using statistical methodssuch as regression analysis

• undertake risk adjustment to adjust the outcomesachieved to reflect the external constraints.

The first approach to accommodating environmental influ-ences is to select only entities in similar circumstances, withthe intention of comparing only like with like. But what criteria should be used to select the ‘similar’ entities? Thesemight simply be readily observable characteristics (such asurban/rural location); or, statistical techniques such as clusteranalysis might be used to identify similar organizationsaccording to a larger number of observable characteristics[12].

A shortcoming of comparing only similar entities is that thiswill reduce the sample size as it allows comparison of performance only with similar types. A second approach istherefore to incorporate environmental factors directly into a


12

Policy brief

regression model of organizational efficiency. The regressionanalysis makes allowance for the uncontrollable factors at anorganizational level, and the residual in the model (thatwhich cannot be statistically explained) is the adjusted measure of efficiency. While this leads to a more generalspecification of the efficiency model than the clusteringapproach, the use of such techniques gives rise to modellingchallenges that are discussed in detail elsewhere [5].

The final method to control for variation in environmentalcircumstances is the family of techniques known as ‘riskadjustment’. These methods adjust organizational outputsfor differences in circumstances before they are used in anyefficiency indicator and are – where feasible – often themost sensible approach to dealing with environmental fac-tors. In particular, they permit the analyst to adjust eachoutput for only those factors that apply specifically to thatoutput, rather than use environmental factors as a generaladjustment for all outputs. However, risk adjustment usuallyhas demanding data requirements, ideally in the form ofinformation on the circumstances of individual patients [13].The DRG adjustments described in the preceding section onoutputs are one such example of risk-adjustment methods.

Links with the rest of the health system

No outputs from a health service practitioner or organizationcan be considered in isolation from their impact on the restof the health system in which they operate. For example:

• the effectiveness of preventive services will affect thenature of demand for curative services

• the performance of hospital support services, such asdiagnostic departments, will affect the efficiency of func-tional areas such as surgical services

• the actions of hospitals, for example in creating care plansfor discharged patients, may have profound implicationsfor primary care services

• the performance of rehabilitative services may haveimportant implications for future hospital readmissions.

Likewise, cost-effective treatment is often secured only ifthere is effective coordination between discrete organiza-tions. The need for such coordination is becomingincreasingly important as the number of people with comor-bidities and complex care needs rises. The frequent calls forbetter integration of patient care reflect the concern thatsuch coordination often fails to meet expectations. That failure may in itself be an important cause of inefficiency.Failures of integration of care for patients with complexlong-term needs pose an especially serious barrier to goodefficiency assessment. Indeed, the very act of measuring theefficiency of separate entities may frustrate efforts toencourage cooperation between different parts of the healthsystem unless successes of care integration are properly recognized in performance assessment. Organizations thatare held to account with partial measures of efficiency thatignore coordination activities may be reluctant to divertefforts towards integration of future patient care. Linkingpatient data across multiple care settings is an importantprerequisite for beginning to address this issue.

Scrutiny of a health system entity in isolation, be it a team ofsurgeons or a hospital, may ignore the important implica-tions of its impact on whole system efficiency. For example,if a primary care practice is held to account only by metricsof costs per patient, it might secure apparently good levelsof efficiency by inappropriately shifting certain costs (such asemergency cover) onto other agencies, such as hospitals orambulance services. The chosen metric creates perverseincentives for the practice, and may fail to capture its seriousnegative impact on other parts of the health system. Thatconsequence should in principle be accounted for in anyassessment of that practice’s efficiency. In theory, it shouldbe feasible to accommodate such negative effects – whicheconomists conceive as externalities – within the analyticframework. However, in practice, it is rarely done, withpotentially important consequences for bias in efficiencyassessment, perverse incentives and misdirected managerialresponses.

Application

Assessing (and scrutinizing) common efficiency indicators

In this section we review a number of common indicatorsthat reflect some of the stages of production where wastecan occur within the health system and apply the analyticalframework.

From a simplistic viewpoint, efficiency represents the ratio ofthe inputs an organization consumes to the valued outputsit produces. For health production processes of any complex-ity, there are usually a number of stages in thetransformation of resources to outcomes that can be evalu-ated. To illustrate this, Figure 2 represents a typical (butsimplified) sequence of processes associated with the treat-ment of acute care patients, such as those treated in ahospital, along with a set of common indicators correspon-ding to the production pathway. The overarching concern iscost–effectiveness, which summarizes the transformation ofexpenditures (on the left hand side) into valued health out-comes (the right hand side); in practice, this could bemeasured using the ‘cost per QALY’ indicator. However, thedata demands of a full system cost–effectiveness analysis areoften prohibitive, and the results of such endeavours may inany case not provide policy-makers with relevant informationon either the causes of inefficiency or where to makeimprovements. In order to take remedial action, decision-makers require more detailed diagnostic indicators ofdiscrete parts of the transformation process.

Inefficiency might occur at any stage of the productionprocess. Take first the transformation of money into physicalinputs. The principal question (given the mix of choseninputs) is whether those inputs are purchased at minimumcost; for example, is the organization using branded ratherthan (presumably) less expensive generic medicines? Data ongeneric prescribing rates could be useful to assess this, if weassume that generic medicines are providing comparablevalued outputs (in the form of health outcomes) at lowerprices (Box 2). A similar question is whether the organizationis paying wage rates above local market rates? A metric such


13


as average hourly wage might shed light on such issues,though it is important to adjust for skill-mix to ensure thatthe outputs, in the form of health care workers, are compa-rable across organizations. If no adjustment is made forskill-mix, the index may also capture information about theallocative efficiency of input choices: is the right mix of doc-tors, other professionals and administrators being deployed?

Box 2: Generic prescribing

What is it?

Measures of generic prescribing give information on whetherproviders and pharmacists are prescribing and dispensing genericmedicines more often than brand name medicines. As generic medi-cines are typically less expensive than brand name drugs, if genericmedicines comprise a large share of total medicines, it is indicative ofgreater efficiency – or more outputs (in this case, medicines are anoutput) for less input (cost).

What does it tell you, and what does it not tell you?

The share of generic prescribing gives an indication of whether asystem is obtaining medicines at low cost. However, although this isnot really a problem in European countries, it is important to be surethat generic medicines are biologically equivalent to brand namedrugs; otherwise, there may be differences in the quality of drugs,which will be reflected in the output under consideration.

What should you do next if you find variation?

Generic prescribing is almost always a useful metric. However, ifgeneric prescribing is low, there are various questions that must beasked to understand why. Generic prescribing may reflect externalinfluences that can influence the production or consumption of

generics. For example, patients may have health needs for which themost effective medicines are not yet available in generic forms, inwhich case, aggregate rates of generic prescribing across providersmay appear low. Another example would be if patients may perceivegeneric drugs to be of inferior quality. If patients have expectationsthat generics are not equivalent to brand name drugs, it is importantto either improve patient information or reduce (or eliminate) co-pay-ments for generics while raising (or instituting) co-payments forbrand name drugs.

Finally, it is important to consider links with the health system andhow these may influence generic prescribing, such as incentives thatencourage or discourage generic prescribing; for example, ifproviders get bonuses for prescribing brand name drugs. If pharma-cists receive higher margins for dispensing brand name drugs, or ifpharmaceutical companies compensate providers for prescribingbrand name drugs, they may also be incentivized to offer them pref-erentially.

The production process now moves to the activitiesperformed, such as diagnostic tests or surgical procedures.Possible sources of waste here may include the use of highlyskilled (and therefore costly) workers for activities that couldbe done by less specialized workers, or using excessive hoursof labour or other physical inputs in the creation of a partic-ular activity, which may lead to a reduction in the totalnumber of procedures that can be produced. One of count-less possible indicators could be the number of operationsby a specialist over a certain time frame (Box 3).

Figure 2: The production process in hospital care with common indicators along the production pathway

i

EXPENDITURE

i

PHYSICALINPUTS

• Capital

• Labour

• Medicines

i

ACTIVITIES

• Diagnostic test

• Surgical procedure

i

PHYSICALOUTPUTS

• Episode of hospital care

OUTCOME

• Improvement in length and quality of life

• Patient experience

i i ii

i

i i iii

ii

ii

Use of generic medicinesOperations per specialist

Duplicate tests

Average length of stay

Expenditures per case

Cost per OALY

Emergency readmissions


14

Policy brief

Box 3: Operations per specialist

What is it?

The metric ‘operations per specialist’ provides information on thenumber of operations any one provider is performing. The entityunder consideration is the provider. Particularly for conditions wherethere is high demand, it is assumed that providers performing moreoperations are more efficient.


This indicator assumes that outputs or activities (i.e. operations) aresufficiently comparable across providers and that externalinfluences, such as the case-mix of the populations served, are alsosimilar. However, not all operations require the same intensity of careand/or preparation time. Some operations may take longer becauseof different patient characteristics, resulting in fewer operationsbeing performed overall.

The indicator also assumes that producing more operations is effi-cient. It does not consider other outputs on which the specialist maybe spending time; for example, some may be involved in teaching, orsupervising operations. They may be preparing for operations byreviewing case files or recommending necessary tests and/or consult-ing with patients. Finally, they may spend time conducting research,which can improve the efficiency of operations in the future.

Furthermore, while performing more operations may be indicative ofhigh demand, it may also reflect supplier-induced demand. In thiscase, more operations would be costly and without gain, thus ineffi-cient.

The indicator also does not consider other relevant inputs, such asother health care workers, and how they may contribute to opera-tions.


If there is large variation between providers in the number of opera-tions they are performing, it is important to understand why this ishappening. Firstly, consider how the assumptions made about theoutputs, inputs and external influences highlighted above mayinfluence variation. It may be that the prevalence of the illness forwhich the operation is performed differs across the regions wherethe variation occurs. It may be because some of the specialists onlyperform the operation on a particular subset of patients, or becausethey are engaged in other activities. However, it is also important toexplore potential sources of inefficiency related to links with otherparts of the health system; for example, whether fewer operationsare happening because of lack of operating theatre space or otherclinical staff (such as anaesthetists or nurses).

Next, physical outputs are created by aggregating activitiesfor a particular patient. In a hospital setting, this usuallyrefers to single episodes of patient care, an aggregation ofmany actions such as tests, procedures, nursing care andphysician consultations. There is great scope for waste in thisstep, for example in the form of duplicate or unnecessarydiagnostic tests and similar (Box 4), or unnecessarily longlength of stay (Box 5). Much depends on how the internalprocesses of the hospital are organized in order to maximizeoutputs using the given inputs. Note that the ‘unit costs’metric (e.g. expenditure per case) usually links costs to physical outputs (Box 6). Numerous efficiency indicators havebeen developed that seek to shed some light on the reasonsfor variations in unit costs.

Box 4: Duplicate tests

What is it?

Duplicate testing indicators provide information on whether a partic-ular test has been administered repeatedly to the same patient – theentity – within a short period of time. The data is often collectedthrough patient surveys, though it can also be collected using patientrecords. Duplicate tests can indicate inefficient use of resources, astests can be expensive, and if the results of a test are already known,there might not be a good reason to conduct the test again.


While duplicate tests may be indicative of a waste of resources, thisassumes that there is no benefit from conducting the same testwithin a certain time period; that is, that additional inputs do notproduce more meaningful outputs. However, this may not always betrue. For example, some tests may have high rates of false positivesor false negatives, or be inconclusive; therefore, on some occasions, aprovider might need to administer a second test in order to make anaccurate diagnosis. Additionally, patients may require a test to beredone if enough time has passed since the previous test and there isa possibility that the results could have changed.


If patients are receiving the same tests more than once, it is impor-tant to find out where this is happening and what types of test arebeing repeatedly administered; for example, it may be related toother factors in the health system, for example if patients arereceiving the same tests within a hospital because patient records arenot being shared across wards. Or, it could be that patients visit dif-ferent primary care providers who are unaware that the patient hasalready been given a particular test in another setting, again becauseinformation is not being shared. Alternatively, it could be that testsare being repeated because of the possibility the results may havechanged; this should be investigated before taking action that limitsaccess to repeat testing.

Box 5: Average length of stay for particular conditions

What is it?

As the name suggests, this measure provides information on thenumber of days per inpatient stay, on average. The entity underconsideration is the hospital, where average length of stay can beused both as a general measure (to cover all conditions), and/or byparticular condition/treatment (e.g. average length of stay for hipreplacement).


Average length of stay for particular conditions can help to highlightvariations in resource use across providers. However, in most cases, itis not clear what the ideal average length of stay should be. It is com-monly assumed that a shorter length of stay is more efficient, as ashorter stay implies reduced costs, however this may not always holdtrue for a number of reasons, relating to:

• Variations in external influences: even within the same condi-tion, cases are different in terms of their severity and the intensityof treatment required. For example, it is likely that an older, morefrail patient will need to stay in the hospital longer than a younger,healthier patient who undergoes the same treatment. It is thuslikely that across hospitals there is a difference in the case-mix ofthe population being treated.

• The time frame being considered to measure the output underconsideration: while shorter length of stay may seem desirable in


15


the short run, as it is indicative of lower resource use, it may beinefficient in the long run. For example, discharging patients earlymay result in an increased probability of complications or slowerrecovery, which can cost more in the longer term through expen-sive readmissions or accumulated outpatient services.

• The assumptions made about inputs under consideration: in itscurrent form, the indicator assumes that every additional day inhospital expends the same resource. However, hospital costs arenot the same across all days of an inpatient stay. It is likely thatcosts in the initial days of the stay are more expensive, as these arethe days where diagnostic tests and/or interventions are more likelyto occur. Later days may entail necessary bed rest and continuingmedications, which will be cheaper.


If providers, or indeed countries, have different average lengths ofstay for particular conditions or overall, it is important to find outwhich of the above explanations may apply.

Are the hospitals truly comparable? Differences in length of staybetween providers may reflect differences in external influencessuch as case-mix. It is thus important to adjust for patient characteris-tics before concluding that longer length of stay is necessarilyinefficient.

When comparing average length of stay across health systems, it maybe that hospitals as entities are not sufficiently comparable; forexample, rehabilitation is performed in hospitals in some countries,while in other countries it occurs in rehabilitation facilities. Indeed, itmay also be that countries have different definitions for what theyconsider to be ‘hospitals’ and thus do not record the same informa-tion for this indicator. Comparisons should only be made betweensimilar types of facility.

The role of health system factors should also be considered. Forexample, long-term care bed availability outside the hospital maycause variations in delays to discharge, which can influence length ofstay. Longer length of stay can also reflect inefficiencies linked toadministrative delays in other areas, such as delays in schedulingtests, coordinating care across providers, and/or treatment. Otherstructural factors related to the health system that may be importantto consider are payment systems or targets. Different payment systems put in place to reimburse hospital stays, such as budgets,per-diem payments or DRGs, produce different incentives for early orlate discharge; for example, hospitals who are paid on a per-diembasis can generate more income by discharging patients later thanhospitals who are paid through global budgets.

Box 6: Expenditures per case

What is it?

Expenditure per case provides information on how much money isspent to deliver various health care services. Different types of healthcare services will require very different types and amounts of inputs;as a result, directly comparing the costs of a hip replacement with thecosts of a hernia operation is not informative. To appropriately com-pare expenditures, it is important to standardize the entity beingassessed, to ensure that we compare like with like. Two generalapproaches can be taken. If there is an interest in comparing aggre-gate expenditures, for example across hospitals, expenditures can besummed after using weights to account for differences in patientcase-mix; DRGs are a useful tool to account for differences in theintensity of services. Alternatively, while less common, to compareexpenditures for specific types of care, clinical vignettes that describeparticular diagnoses, procedures and patient characteristics can beused to cost hypothetical episodes of care.


Comparing expenditures gives a sense of whether too much input isbeing used to provide similar health care activities or output. How-ever, countries that spend the same amount of money deliveringservices do not necessarily provide equivalent services. There is aquestion of whether the outputs being considered are truly compa-rable. It is very difficult to account for differences in the quality ofcare by only comparing expenditures by case, so the assumption isthat the quality of care is uniform.


There are a number of reasons why expenditures per case couldappear too high. It is possible that input costs, such as providersalaries or the prices of drugs and diagnostic tests are too high, orthat too many inputs are being used to treat a condition. For exam-ple, providers may order unnecessary diagnostic tests that increasecosts without additional health gains.

Likewise, the indicator relies on the assumption that the entities arecomparable, for example that expenditures have been adjusted torender services fully comparable across systems, but this may nothave been done sufficiently. It may be that health systems treat thesame conditions using very different approaches, or that costs havenot been properly adjusted to account for regional differences inoverall prices.

Most importantly, there could be large differences in the outputsbeing compared, for example because of the quality of care provided;if a system is found to produce the same health services for muchlower expenditure, it is important to ensure that patients are enjoyingsatisfactory health outcomes.

The final stage of the health system production process isthe quality of the outputs produced. There is growing interest, particularly from a managerial perspective, in under-standing how to maximize the value produced by healthcare [14]. Even when they employ the same physical inputs,activities or physical outputs, there is great scope for varia-tion in effectiveness between providers. The notion ofquality in health care has a number of connotations, includ-ing the clinical outcomes achieved (usually measured interms of the gain in the length and quality of life, sometimesusing QALYs) and the patient experience (Box 7). So, forexample, even though two hospitals produce identical num-bers of hip replacements, variations in clinical practice andcompetence mean the value they confer on patients (in theform of length and quality of life and the patient experience)can vary considerably. Quality-adjusted output is usuallyreferred to as the ‘outcome’ of care in the literature. Thequality of care has become a central concern of policy- makers, and its measurement, while contentious, is essentialfor a comprehensive picture of efficiency. Nevertheless, if wecan agree on a measure of aggregate-valued outputs, thenwe are able to calculate a summary measure of ‘efficiency’as the ratio of valued outputs to inputs – what is oftenreferred to as cost–effectiveness.


16

Policy brief

Box 7: Cost per QALY

What is it?

Cost per QALY measures the cost of an additional year of goodhealth. This measure is generally used to assess the cost–effectivenessof a particular health intervention. The entity under scrutiny in thiscase is the intervention itself. Data used to construct this measure isoften from clinical trials.


The cost per QALY indicator can be used to choose between differentinterventions on the basis of cost–effectiveness. On the output side,however, there may be factors that influence health outcomes that arenot controllable by the providers. For example, adherence to the inter-vention or family support may influence outcomes and not beadequately factored into the analysis. On the input side, differentproviders may use different cost accounting techniques and/or differentlevels of resources, which may lead to variability in observed costs.


If we see variations in cost per QALY, we want to ensure that thisvariation is because one treatment is truly more effective thananother. To ensure this we should consider that all external influences on outcomes are sufficiently controlled for.

It is important to note that the production of the majority ofhealth care outputs rarely conforms to the production-linetype technology implied in Figure 1, in which a set of clearlyidentifiable inputs is used to produce a standard type of out-put. Instead, the majority of health care is tailor-made to thespecific needs of an individual patient, with consequent varia-tions in clinical needs, social circumstances and personalpreferences. This means that there is often considerable varia-tion amongst patients in how inputs are consumed andoutputs or outcomes are produced. For example,contributions to the care process may be made by multipleorganizations and caregivers; an ‘episode’ of care may occurover an extended period of time and in different settings; andthe responsibilities for delivery may vary from place to placeand over time.

Additionally, any specific indicator of efficiency may seek toaggregate all inputs into a single measure of costs, or it mayconsider only a selection of inputs; for example, labour pro-ductivity measures such as ‘operations per specialist’ or‘patient consultations per full-time equivalent physician permonth’ ignore the many other inputs into the consultationand the many outputs other than patient consultations pro-duced by the physician (Box 2). In effect, such measurescreate efficiency ratios using only a subset of the inputs andoutputs represented by the arrows in Figure 1. Here the out-put measure is partial in several senses: a physician mayundertake many other activities; there are many other inputsinto the patient’s care; and there is no information on thehealth gain achieved by the consultation. In short, the indica-tor shows only a fragment of the complete transformation ofresources into desired outcomes (improved health).

This stylized example of a production pathway looks only atthe hospital sector and is therefore focused mainly on hospitaltechnical efficiency, making no judgement on broader

allocative efficiency issues within the health system, such aswhether patients might have been treated more cost-effec-tively in different settings (for example, primary care ornursing homes). Evidence of allocative inefficiencies may beseen by investigating the frequency of hospital admission forconditions that can typically be managed in less intensive set-tings, or even potentially from data on emergencyreadmissions (Box 8). Further, by focusing on the curative sec-tor, we cannot shed light on the success or otherwise of thehealth system’s efforts to prevent or delay the onset ofdisease. Similarly, the impact of the hospital’s performance onother sectors within the health system is ignored. For exam-ple, it may be the case that apparently high levels of efficiencyin average length of stay are being secured at the expense ofheavy workloads for rehabilitative and primary care services,which may or may not be efficient from a ‘whole system’ per-spective. To understand how efficiently an entire healthsystem is performing, it is important to review multiple indica-tors covering a plethora of production pathways.

Box 8: Hospital readmission rates

What is it?

Measures of hospital readmission provide information on whether apatient has been readmitted for any cause to any hospital within ashort period of time. Often this period of time is around 30 days,however in practice this can vary. This data is often reported inadministrative data or patient records, and is sometimes adjusted forpatient characteristics that make readmission more likely (such aspatient age or known comorbidities).


Readmission rates are used to assess the efficiency of the hospital, asreadmissions may be indicative of poor-quality care received duringthe initial admission or of patients being discharged too early and/orreceiving substandard care in hospital. If patients are readmitted tothe hospital because of these factors, this additional visit can repre-sent an inefficient use of resources.

However, readmissions may also be linked to external factors that areoutside the hospital’s control, such as other complicating illnesses,patient lifestyle and behaviours, as well as links with other parts ofthe system, such as the quality of care provided to patients after dis-charge.

Finally, readmissions may also be indicative of good care; if, for exam-ple, hospitals are better able to successfully treat very ill patients, andindeed save their lives, they are likely to have higher readmissionrates than hospitals that have higher mortality rates.


Before coming to the conclusion that providers with high readmissionrates are inefficient, it is important to rule out some of the otherexplanations outlined above. As a first step it may be useful toexplore the causes of readmissions for particular hospitals: are theydriven by particular clinical conditions or events? Do they change forparticular subgroups of patients? Exploring the patient characteristicsof different hospitals may also be useful and help to ensure that apopulation with greater health needs does not account for the higherreadmission rates. This should be done even if readmission rates areadjusted for patient characteristics; for example, it is likely that a hos-pital in a deprived area will have different readmission rates whencompared to a hospital in an affluent area, even when controlling forage and comorbidity. This may also capture external factors, such


17


as the presence of a social network after discharge or the patient’scapacity for managing their own care. Exploring the proportion ofpatients readmitted through emergency care versus being transferredfrom other facilities may also help to better understand the nature ofreadmissions.

It may also be useful to explore the relationship between readmissionrates and other measures of hospital quality to obtain greater clarityabout the hospital’s performance. Other output measures to be con-sidered may be mortality rates or even process and throughputmeasures, such as patient discharge consultations, medical errors,waiting times or length of stay.

To explore variation in readmission rates across countries, it may beuseful to explore additional avenues. For example, it may be useful toensure that the entities being considered are comparable: Are read-missions measured in the same way and for the same time period?Are the providers and conditions included in the national definitionsalso consistent?

The description of a hospital production pathway abovelargely considers the production of outputs per inputs in iso-lation; that is, it reflects how well a single input (forexample, in the form of costs) produces a single output (inthe form of an intervention). It is nevertheless possible tocombine multiple inputs and/or outputs into a single metricusing methodological tools known as frontier-based

methods. Data envelopment analysis (DEA) and other regres-sion-based techniques are tools that can be used to estimatea production possibilities frontier (how much output can the-oretically be produced at a given level of inputs) and assesshow close a provider is to the frontier. While appearing morecomplex, this is essentially a simple extension of the indica-tors described in Figure 2. An example of frontier-basedanalysis comparing how well health systems produce lifeexpectancy is described below.

Learning from system-level efficiency comparisonsusing frontier-based techniques

Many analyses have sought to identify the countries thatclaim the best overall health outcomes given their levels ofhealth-influencing inputs, such as health expenditures, edu-cation, lifestyle and wealth [8,15–18]. Analysts can usesophisticated modelling techniques in an effort to estimatehow far a country’s life expectancy is attributable to a selec-tion of these observable factors. For example, an additionalyear of compulsory schooling might be estimated to increaselife expectancy across a range of countries by a certainamount, while an increase in alcohol consumption across thepopulation could be found to decrease life expectancy byanother amount.

Table 1. Results from exploratory efficiency analysis

CountryAccounting for

alcohol and tobacco use

Accounting for alcohol, tobaccoand per person

health expenditure

Change in rankingafter accounting

for health expenditure

Appears worseafter accounting


Appears betterafter accounting


A 1 1 0

B 2 2 0

C 3 3 0

D 4 7 3 D

E 5 4 -1 E

F 6 5 -1 F

G 7 6 -1 G

H 8 9 1 H

I 9 11 2 I

J 10 8 -2 J

K 11 10 -1 K

L 12 12 0

M 13 13 0

N 14 14 0

O 15 16 1 O

P 16 15 -1 P

Q 17 17 0

Note: All models are generalized least squares with AR(1) correlation structure and heteroskedastic error structure. Models include country fixedeffects which control for factors that are constant within each country across the entire sample period, and year fixed effects which account forfactors which affect all countries over time. Country names are not shown.


18

Policy brief

To illustrate what frontier analysis can and cannot tell us, weuse data from 1960 to 2013 from the OECD Health data-base and estimate our own simple models with the intentionof understanding how health systems and other factors con-tribute to life expectancy. There is limited data available forall years, so we seek to explain variation in life expectancyusing only a few variables: tobacco grams consumed perperson age 15+, litres of alcohol consumed per person age15+, and total health expenditures per person in US$ at2005 purchasing power parity (PPP) rates. We then rankcountries, where countries are more efficient if their actuallife expectancy is greater than the model would predict,given the inputs selected. The results of this exploratory exer-cise based on data from 2010 are shown in Table 1.

In column 2, we find that after controlling for the detrimen-tal health effects of alcohol and tobacco consumption,Country A is the most efficient at producing years of life (i.e.actual life expectancy in Country A is the longest, givenwhat the model predicted life expectancy should be). Country Q is the least efficient in this scenario.

To evaluate the contribution of the health system to lifeexpectancy, we next add per capita health expenditures intothe model (Column 3). There are some slight changes in therankings. Countries E, F, G, J, K and P all seem to performbetter, suggesting that in these countries, health spendingcontributes comparatively effectively to longer lifeexpectancy. In countries D, H, I and O, health expendituredoes not contribute as many years of additional life as themodel would have expected. Country A is the top performerin both models, in the sense that its actual life expectancy isthe highest above that predicted by the model.

So does this mean that Country A has the most efficienthealth system? Based on this analysis, Country A appears toproduce the longest lifespans at its given levels of inputs.But, before firmly drawing this conclusion, we need moreinformation to understand the reasons for cross-country vari-ations. It must be noted, for example, that Country A spendsamong the least amount of money per capita on health careof all OECD countries and is also among the lowest in termsof density of doctors and nurses, as well as number of beds.Country A also has comparatively high cardiovascular diseasemortality and is among the highest in hospital fatality ratesfor acute myocardial infarction and for stroke. Cervical can-cer screening, mammography and colorectal screening areall low as well [19].

Applying the framework to interpret the analysis

To better understand the results of this analysis we return tothe analytical framework. The accountable entity beingassessed here is an entire country. One important considera-tion is that it is not clear that all of the countries included inthe analysis are comparable to such an extent that theybelong to the same production possibilities frontier; that is,in all likelihood, the entities are not sufficiently comparableto be modelled together.

Moreover, the output considered is life expectancy, which isa rather blunt measure in that it does not reflect otherhealth outcomes besides the population’s average age ofdeath. The main input of interest in the analysis is expendi-

ture, which serves as an imperfect proxy for the health sys-tem’s many inputs.

Remember that the rankings are determined by the size ofthe unexplained variation in life expectancy. Life expectancyis also a product of many inputs in addition to the threeconsidered in this analysis. While we have attempted to -control for external influences in the form of alcohol andtobacco consumption, there are innumerable observable andunobservable factors that we have not considered, whichundoubtedly play an important role in determining population health.

Additionally, aggregate analyses such as this can provideinteresting information about how well systems are perform-ing overall and can highlight unexpected variations thatmight not be observed by only focusing on specific healthcare processes. Yet at the same time, these studies are usefulonly as a starting point before conducting further analysis,since they cannot give any clear indication about whereproblems might be occurring within the health systemand they are susceptible to missing information. Also, thelevel where an efficiency issue becomes apparent is not necessarily the level where policy-makers should take actionif they want to make improvements.

So what should be done next?

The analysis presented here is only intended to be illustra-tive. To properly conduct a comparison of health systemefficiency, we would need to take different modellingapproaches and take advantage of a wider range of data,while searching for repeated patterns. Any analyst runningsuch a study should ask themselves a variety of questionsbefore coming to any strong conclusions, such as: Are wemeasuring the right things? Are we using good metrics? Arewe adjusting the metrics correctly? Are there other thingsgoing on that we are observing? Is our analytic model cor-rectly specified?

In this particular analysis, Country A appears to be most effi-cient, partially because its health expenditure is relativelylow, while life expectancy, despite being lower than in mostOECD countries, is considerably greater than theapproximately 70 years predicted by the full model.

The quickest way for another country to improve in therankings would therefore be to simply reduce the level ofinputs (i.e. health expenditure). Assuming that lifeexpectancy does not change immediately, technically speak-ing, a country’s ranking should improve. However, while thismay improve a country’s efficiency ranking in the short term,indiscriminately reducing health-producing inputs like healthcare expenditure can in the longer term lead to reductions inboth allocative and technical efficiency. Highly productiveparts of the health system may be affected just as much asinefficient sectors. The inefficiencies may lead to unnecessar-ily severe reductions in health outcomes and still worse levelsof efficiency. It is important to clearly distinguish expenditurereduction and cost savings from efficiency improvement, andto note that – if required by policy-makers – any expenditurereduction should be carefully targeted to reduce the sourcesof allocative and technical inefficiency.


19


Discussion

How can health systems monitor efficiency?

In this policy brief we have reviewed common metrics thatassess efficiency within and across health systems. On itsown, each indicator provides only a limited amount ofactionable evidence on the potential for efficiency gains. Tofully grasp health system performance, one cannot rely solelyon a single metric to try and assess efficiency, no matter howmacro or micro that indicator is. Rather, we must ‘triangu-late’ available information by making use of many indicatorsthat offer glimpses into the efficiency of processes across thehealth system.

There is great interest in identifying the ‘right’ set of indica-tors that give the most complete snapshot of health systemefficiency. However, there is no one-size-fits-all ‘dashboard’of efficiency metrics. To determine the most useful metrics, itis necessary to have a clear understanding of many particularaspects of the health system. That being said, in Appendix 1,we provide a list of all indicators mentioned in this reportand, using the analytic framework, we consider some of thequestions analysts and decision-makers may ask themselvesto understand the results, facilitate further analysis andbegin to identify policy options.

The way forward

Interest in efficiency has been heightened by the apparentlyinexorable growth in health system expenditure in mostcountries, as well as the widespread belief that majorimprovements in efficiency can be made. However, althoughit is one of the most fundamental health system perform-ance metrics for researchers and policy-makers, the conceptof health system efficiency is, in practice, heavily contestedand its accurate measurement across countries difficult torealize. It has proved challenging to develop robust measuresof comparative efficiency that are feasible to collect or esti-mate, offer consistent insight into comparative health systemperformance and that are usable in guiding policy reforms.

Two broad types of inefficiency have been discussed – alloca-tive and technical inefficiency. Allocative inefficiency ariseswhen the ‘wrong’ mix of services is provided, given societalpreferences and the limited budget available for health care(or the ‘wrong’ mix of inputs is used to produce services).Allocative inefficiency can occur at the level of the healthsystem, the provider organization or the individualpractitioner, and may arise from inadequate priority setting,faulty payment mechanisms, lack of clinical guidelines, weakperformance reporting or simply inadequate governance ofthe system. Technical inefficiency arises most notably at theprovider and practitioner levels, and may result from inap-propriate incentives, weak or constrained management, andinadequate information. Either type of inefficiency may haveprofoundly adverse consequences for the patients, who areconsequently given poor-quality treatment or denied treat-ment altogether because of the associated loss of resources.

Efficiency analysis first requires a choice of accountableentity to scrutinize, whether this be an individual

practitioner, team, provider organization or the entire healthsystem. Indicators of the entity’s efficiency then usually rep-resent the ratio of some input (or inputs) to some output(or outputs). We have shown that many indicators are par-tial, in the sense either that they seek to measure only partof the care pathway, or they capture only part of the inputsor outputs. Furthermore, to secure comparability, it is usuallynecessary to adjust for variations in the uncontrollable exter-nal factors that affect the performance of providers andpractitioners, using techniques such as risk adjustment.Because of these limitations, we have argued that meaning-ful scrutiny of efficiency indicators must be accompanied bymore in-depth analysis to ascertain the magnitude, natureand causes of any apparent inefficiency.

Clarity about the entity under scrutiny ensures that theboundaries of any analysis are clearly drawn. However, thereare inevitable links between any single production process inthe health system and other parts of the health system.The detection of inefficiency at a local level (say of a localfamily practitioner) does not necessarily mean that the localentity should be held responsible for that inefficiency. It willoften be the case that the inefficiency arises from constraintsimposed on the local organization or practitioner by higherlevels of authority, for example in the form of clinical guide-lines, legal requirements, performance targets or financingmechanisms. Therefore, as well as identifying the nature andmagnitude of inefficiency, the analysis should also correctlyidentify the entity ultimately responsible for the causes ofinefficiency. We therefore strongly underline the importanceof seeking to ‘unpack’ the sources of inefficiency wheneverit is found.

Scope for improving measurement

We have identified enormous scope for improvement inmeasuring efficiency. Conceptually, there is much work stillto be done in creating indicators that conform to the usualrequirements of specificity, validity, reliability, timeliness,comparability and avoidance of perverse incentives. On theinput side, there is a need for more consistent and detailedcosting of the care given to individual patients. Managementaccountants have a key role to play in this respect. On theoutput side, the use of PROMs might offer great scope forimproved quality measurement. Furthermore, we have notedthe tendency for most indicators to reflect only part of thepatient pathway. The increased use of electronic healthrecords, linked datasets and registries, capturing entirepatient treatments, offers considerable scope for developingmore complete efficiency metrics, capable of assessing therelative merits of alternative approaches to care.

A particularly important challenge relates to the lack of information about care outside the hospital setting. Theincreasing cost of meeting the complex care needs of peoplewith multimorbidities makes resolution of this issueespecially urgent. Policymakers have a clear need for betterinformation concerning the most efficient balance ofresources between hospital and community. Furthermore,there is an almost complete absence of useful evidence onefficiency in mental health care.


20

Policy brief

A general challenge to better information on efficiency is thelack of agreement on information standards and protocols.Even within countries, there is considerable variation in theinterpretation of accountancy rules and the use of patient-level information systems. International comparison is evenmore problematic and there would be major gains if therecould be international agreement on basic reporting andinformation standards beyond achievements such as EuroDRG and the System of Health Accounts.

A return to the efficiency myths

At the start of this policy brief we listed six statements com-monly made in health policy that should not be taken asuniversal truths. We must emphatically underline that we arenot suggesting that the search for cost-effective new innova-tions and reforms should be abandoned. Rather, that theyshould be properly evaluated, using as full an array of efficiency metrics as is practicably feasible.

Improved efficiency is synonymous with cost containment

Cost containment and improved efficiency are quite distinct.The need to contain costs may heighten the urgency of find-ing areas that can do ‘more with less’. However, costcontainment refers only to the inputs utilized and makes noreference to any associated changes to either the outputsof the entity under scrutiny or the impact of cost contain-ment on the rest of the health system. Cutting costsindiscriminately may be detrimental to the volume or qualityof care in some sectors and ultimately lead to worse healthoutcomes relative to costs. Furthermore, it may shift costsonto other health system sectors, with adverseconsequences for overall system efficiency; for example, ifcuts in primary care give rise to extra inefficient use of thehospital sector.

Improved quality of health care will necessarily lead toimproved efficiency

This argument hinges on the belief that improved quality willlead to better health outcomes and will reducecomplications, which will lead to lower health care costs forthe health system as a whole. However, if securing betterquality is itself costly, perhaps requiring more timely or inten-sive intervention, then the benefits must be weighed againstthese additional costs. There are no guarantees thatimproved quality of health care will lead to health gains andfuture cost reductions that are commensurate with its imme-diate costs. Thus, the important requirement is to match allthe additional inputs associated with the quality improve-ment to the associated outputs, including both health gainsand the links with the rest of the health system. Ifimproved quality is costly in the short term, then efficiencymay appear to be impaired if it is measured in terms of thecost per activity, even though it has longer-term benefits.This underlines the importance of having absolute clarityabout how the entity under scrutiny is defined, both interms of the scope of its activity and the time period underconsideration. Too narrow a definition may fail to capture allthe costs and benefits associated with increased quality.

Increased efficiency will necessarily lead to poorerhealth outcomes and less equitable access to care

It is sometimes argued that the pursuit of improvedefficiency may be detrimental to outcomes and access. Thereis no reason that this must be the case. Increased efficiencyfrees up resources that can be used for patient care. If theimprovement is in technical efficiency, then increased out-puts can be used for the same inputs (or reduced inputsused for the same outputs, releasing resources for use else-where). If the improvement is in allocative efficiency (saybetween different programmes of care), then there is a needto identify carefully the appropriate entity under scrutiny, inthis case the whole health system. With a reallocation ofinputs, some programmes of care may indeed be increasedat the expense of others. However, if efficiency principles arerespected, the net effect will be to improve the outputs ofthe system as a whole, which should in this situation be thefocus of attention.

Increased intensity of resource use (for example, increased bed occupancy) is always a signal of increased efficiency

Bed occupancy rates capture the extent to which availablebeds are being used; however, the indicator provides noinformation on whether beds are being used appropriatelyor efficiently. For example, high bed occupancy rates couldreflect external influences on performance, such as thecase-mix of patients, or an inability to discharge patients nolonger needing hospital care because they have no satisfac-tory social care at home. Also, if bed occupancy is too high,hospitals may be unable to cope with fluctuations indemand, with serious knock-on effects for the performanceof emergency services, health system costs and health out-comes. In this case, without broadening the definition of theentity under analysis, important links with the rest of thehealth system may be ignored. Moreover, hospital beds areonly one of the many inputs to hospital care, so the indica-tor may be seriously misleading, for example, by ignoringthe implications it has for the use of clinical staff. In short,without further analysis, there is no ‘magic target number’that indicates the optimal use of hospital beds.

Increased emphasis on preventive medicine will inevitably reduce health system costs and increase efficiency

Preventive services are important to reduce the incidence ofdisease or delay its onset, and improve health outcomes.Intrinsically, they have important links with other(curative) sectors of the health system. They may reducesome health sector costs in the long run (e.g. an individualwho takes statins is less likely to experience an expensiveand traumatic cardiovascular episode). However, whetherthis will in fact reduce total expenditures over time is notguaranteed, given the longer life expectancy of the individu-als. Furthermore, the effectiveness of preventiveinterventions is likely to be highly dependent on externalinfluences, such as compliance with advice or medication.For preventive services, therefore, it is particularly importantfrom an efficiency perspective that a broad definition of the


21


entity under scrutiny is adopted. This definition is likely toembrace the health system as a whole, considered over anextended time period, and is therefore likely to be achallenging undertaking.

Adopting methods such as DRGs for paying providerswill always improve system efficiency

As a prospective method of reimbursing providers, DRGswere initially used in the Medicare program in the USA as analternative to the manifestly wasteful ‘fee-for-service’arrangements, which reimbursed providers irrespective ofthe effectiveness of the service. DRG payment offered a sin-gle fee for an entire episode, usually regardless of servicesprovided, and clearly blunted the incentives for excessivewasteful care in US hospitals. However, this hospital perspec-tive adopts a narrow view of the entity under analysis. Forexample, when introduced in other health systems that pre-viously relied on fixed budgets for providers, DRG methodsmight instead stimulate additional demand for hospital carethat is not necessarily cost-effective and lead to allocativeinefficiency between the hospital and ambulatory sectors.Therefore, although the efficiency of the hospital sectormight be improved through the introduction of DRGs, theefficiency of the system as a whole might be adverselyaffected.

Conclusions

The discussion of these ‘myths’ indicates that there is an element of truth in each of the statements. However, ascrutiny of the efficiency arguments underlying them, usingthe framework explained in this policy brief, indicates that ineach case the policy message should be much morenuanced. In particular, any efficiency indicator associatedwith such statements is likely to be partial in one or morerespects. There is therefore a need for careful considerationof any limitations, and we would recommend that the five-point analytical framework here sets out a useful startingpoint for any such analysis.

In conclusion, we would underline the central importance ofefficiency metrics in governing, managing and reforming anyhealth system. There is a massive agenda to improve thescope, comparability, timeliness, quality and usefulness ofsuch metrics – but they should also be treated with caution.In many health systems, managers and practitioners operatewithin constraints that limit their scope for radical improve-ments, at least in the short term. There is therefore a clearneed for policy-makers to set out clearly what they mean byefficiency, to give local decision-makers the leadershipcapacity and autonomy needed to pursue improvedefficiency, and to put in place information systems thatmeasure progress accurately and in a timely fashion.


22

Policy brief

References

1. Cylus J, Papanicolas I, Smith PC (2016). Health systemefficiency: How to make measurement matter forpolicy and management. Health Policy Series, 46.Brussels, European Observatory on Health Systemsand Policies.

2. Smith PC et al. (eds) (2009). Performance measure-ment for health system improvement: Experiences,challenges and prospects. Health Economics, Policyand Management. Cambridge, Cambridge UniversityPress.

3. Papanicolas I, Smith PC (2013). Health system performance comparison: An agenda for policy, information and research. European Observatory onHealth Systems and Policies Series. Buckingham,Open University Press.

4. Arrow KJ (1963). Uncertainty and the welfareeconomies of medical care. American Economic Review, 53(5):941–973.

5. Jacobs R, Smith PC, Street A (2006). Measuring efficiency in health care: Analytic techniques andhealth policy. Cambridge, Cambridge University Press.

6. Fetter RB (1991). Diagnosis related groups: Understanding hospital performance. Interfaces,21(1):6–26.

7. Busse R et al. (eds) (2011). Diagnosis-related groupsin Europe: Moving towards transparency, efficiencyand quality in hospitals. Maidenhead, McGraw-HillOpen University Press.

8. WHO (2000). The World Health Report 2000. Healthsystems: Improving performance. Geneva, WorldHealth Organization.

9. Smith PC, Street AD (2013). On the uses of routinepatient-reported health outcome data. Health Economics, 22(2):119–131.

10. EuroQol Group (1991). EuroQol – a new facility forthe measurement of health-related quality of life.Health Policy, 16(3):199–208.

11. Ware J, Sherbourne C (1992). The MOS 36-itemshort-form health status survey (SF-36). Conceptualframework and item selection. Medical Care,30(6):473–483.

12. Everitt B et al. (2001). Cluster analysis. London,Arnold.

13. Iezzoni LI (2003). Risk adjustment for measuringhealthcare outcomes, 3rd edn. Baltimore, Health Administration Press.

14. Porter M (2010). What is value in health care? NewEngland Journal of Medicine, 363:2477–2481.

15. OECD (2010). Health care systems: Efficiency and policy settings. Paris, Organisation for Economic Co-operation and Development Publishing(http://www.oecd-ilibrary.org/social-issues-migration-health/health-care-systems_9789264094901-en, accessed 20 May 2017).

16. Joumard I et al. (2008). Health status determinants:Lifestyle, environment, health care resources and efficiency. OECD Economics Department Working Papers, 627. Paris, Organisation for Economic Co-operation and Development.

17. European Commission (2015). Comparative efficiencyof health systems, corrected for selected lifestyle factors. Final report. Brussels, European Union.

18. Jowett M et al. (2016). Spending targets for health:No magic number. Health Financing Working PaperNo. 1. Geneva, World Health Organization(http://apps.who.int/iris/bitstream/10665/250048/1/WHO-HIS-HGF-HFWorkingPaper-16.1-eng.pdf, accessed20 May 2017).

19. OECD (2014). Health at a glance: Europe 2014. Paris,Organisation for Economic Co-operation and Development (http://www.keepeek.com/Digital-Asset-Management/oecd/social-issues-migration-health/health-at-a-glance-europe-2014_health_glance_eur-2014-en#page93, accessed20 May 2017).


23


Appendix 1

Efficiencymeasure

Accountable entity

Outputs InputsExternal

influencesLinks with the rest of

the health system

Antibiotic prescribingrate

Is the entity of interest a provideror an entire system?

Are patient outcomescomparatively improveddue to antibiotics?

Are patients receiv-ing antibiotics whodo not need them?

Are there clinical guide-lines or other factorsthat drive antibioticprescribing?

What incentives exist for antibiotic prescribing? Is thereevidence of high levels of antibiotic resistance infections?

Averagelength ofstay

Are theproviders/case-mixsufficiently compa-rable?

Are patient outcomescomparatively improveddue to remaining inhospital? Is the risk ofreadmission reduced?

Are additional daysin hospital expensivein comparison toother care settings?

Are there differences incase-mix that are notaccounted for?

Can other less intensive settingstreat these patients effectively?Are there capacity issues inlong-term care or communitycare settings?

Comparisonof providerexpected vsactual costs

Are theproviders/case-mixsufficiently comparable?

Are data available onpatient outcomes?

Is cost accountingcomparable acrossentities?


Does the entity being evaluatedhave spillover effects on otherparts of the health system? Isthis entity dependent on coordi-nation across multiple providers(or, alternatively, can it shiftsome of its costs to otherproviders)?

Costs per patient (bycondition)





Does the entity being evaluatedhave spillover effects on otherparts of the health system? Isthe entity dependent on coordi-nation across multiple providers(or, alternatively, can it shiftsome of its costs to otherproviders)?

Cost perQALY

Is the entity of inter-est a particularhealth intervention?If so, are multipleproviders involved?If the entity is a single provider, is itfully accountable forthe health outcomebeing assessed?

Is the health outcomemeasure appropriate?

Is cost accountingcomparable acrossentities? Are thereinputs that affectthe health outcomewhich are not ac-counted for?

Are there factors whichaffect the health out-come that are control-lable/ uncontrollable byproviders? How arethese factors ac-counted for in theanalysis?


Delayed dischargesfrom hospitalto commu-nity setting

Are theproviders/case-mixsufficiently comparable?

Are patient outcomescomparatively improveddue to remaining inhospital? Is the risk ofreadmission reduced?

Are additional daysin hospital expensivein comparison toother care settings?


Can other less intensive settingstreat these patients effectively?Are there capacity issues inlong-term care or communitycare settings?

Emergencyadmissionsfor ambula-tory sensitiveconditions

Is the entity of inter-est a provider or anentire system?

Are patient outcomescomparatively improveddue to emergency ad-missions?

Can patients beseen in less intensivesettings?


Can other less intensive settingstreat these patients effectively?

Emergencyreadmissions

Are providers sufficiently compa-rable (e.g. are theytreating similartypes of patients)?


Is there informationon a patient's initialadmission to assesswhether care wassuboptimal?



Using the analytic framework to interpret selected efficiency measures

Continued on next page >


24

Policy brief

Efficiencymeasure

Accountable entity

Outputs InputsExternal

influencesLinks with the rest of

the health system

Expenditureper case

Are cases sufficiently comparable?




Does the entity being evalu-ated have spillover effects onother parts of the health sys-tem? Is the entity dependenton coordination across multipleproviders (or, alternatively, canit shift some of its costs toother providers)?

Excessive referrals tospecialists

Is the entity of in-terest a provider oran entire system?

Are patient outcomescomparatively improveddue to specialist referrals?

Can patients beseen in less intensivesettings?


What incentives exist for primary care providers to referpatients?

Frontieranalysis (e.g. DEA)

Are the entities suf-ficiently comparable(i.e. do they belongto the same produc-tion possibilitiesfrontier)?

Are the outputs produced entirely by theinputs? Is there information on healthoutcomes?

Do the inputs fullyaccount for the outputs?

Are there factors whichaffect the outputs thatare controllable/ uncon-trollable by the enti-ties? How are thesefactors accounted forin the analysis?

Does the entity being evaluatedhave spillover effects on otherparts of the health system? Is the entity dependent on coordination across multipleproviders (or, alternatively, can itshift some of its costs to otherproviders)?

Generic prescribingrates


Are generic medicinesof comparable quality?

Are generic medi-cines less expensivethan branded?

Are there factors (suchas economic interests)which affect the pro-duction or consump-tion of medicines?

What provider incentives existfor generic prescribing?

Operationsper specialist


What other outputs besides operations isthe specialist spendingtime on?

Are there inputs notaccounted for (e.g.other health workersparticipating in anoperation)?



Prices paidfor inputs (e.g. averagehourly wage)

Are prices adjustedsufficiently to ac-count for variabilityin cost-of-livingacross entities?

Are the inputs (e.g.labour, capital) compa-rable in terms of quality?


Are there factors whichaffect the prices of in-puts broadly (e.g. fac-tors affecting wages forother sectors)?

Are prices fixed (e.g. public- sector salary scales)?

Skill-mix ofhealth workers

Is the entity able toalter its mix ofhealth workers?

Does the mix of healthworkers reflect popula-tion needs?

Are wages forhealth workers appropriate giventheir skill level?

Are there factors (suchas migration, educa-tion) which affect theskill-mix of healthworkers?

If the entity is a system, to whatextent are there variationsamong providers?

Unnecessaryduplicationof an activity(e.g. diagnos-tic tests)

Is the entity (e.g.provider or a system) aware ofduplicate activitiesby other entities?

Is data available on pa-tient outcomes?

Is a duplicate activitydefinitely not neces-sary (e.g. has suffi-cient time passedsince a prior test oris there a high likelihood of falsepositives?)

Are there differences incase-mix that are notaccounted

Are there issues related to sharing of patient information?Are there incentives for over- utilization of services?

> Continued from previous page


1. How can European health systems support investment in and the implementation of population health strategies? David McDaid, Michael Drummond, Marc Suhrcke

2. How can the impact of health technology assessments be enhanced? Corinna Sorenson, Michael Drummond, Finn Børlum Kristensen, Reinhard Busse

3. Where are the patients in decision-making about their own care? Angela Coulter, Suzanne Parsons, Janet Askham

4. How can the settings used to provide care to older people be balanced? Peter C. Coyte, Nick Goodwin, Audrey Laporte

5. When do vertical (stand-alone) programmes have a place in health systems? Rifat A. Atun, Sara Bennett, Antonio Duran

6. How can chronic disease management programmes operate across care settings and providers? Debbie Singh

7. How can the migration of health service professionals be managed so as to reduce any negative effects on supply? James Buchan

8. How can optimal skill mix be effectively implemented and why? Ivy Lynn Bourgeault, Ellen Kuhlmann, Elena Neiterman, Sirpa Wrede

9. Do lifelong learning and revalidation ensure that physicians are fit to practise? Sherry Merkur, Philipa Mladovsky, Elias Mossialos, Martin McKee

10. How can health systems respond to population ageing? Bernd Rechel, Yvonne Doyle, Emily Grundy, Martin McKee

11. How can European states design efficient, equitable and sustainable funding systems for long-term care for older people? José-Luis Fernández, Julien Forder, Birgit Trukeschitz, Martina Rokosová, David McDaid

12. How can gender equity be addressed through health systems? Sarah Payne

13. How can telehealth help in the provision of integrated care? Karl A. Stroetmann, Lutz Kubitschke, Simon Robinson, Veli Stroetmann, Kevin Cullen, David McDaid

14. How to create conditions for adapting physicians’ skills to new needs and lifelong learning Tanya Horsley, Jeremy Grimshaw, Craig Campbell

15. How to create an attractive and supportive working environment for health professionals Christiane Wiskow, Tit Albreht, Carlo de Pietro

16. How can knowledge brokering be better supported across European health systems? John N. Lavis, Govin Permanand, Cristina Catallo, BRIDGE Study Team

17. How can knowledge brokering be advanced in a country’s health system? John N. Lavis, Govin Permanand, Cristina Catallo, BRIDGE Study Team

18. How can countries address the efficiency and equity implications of health professional mobility in Europe? Adapting policies in the context of the WHO Code and EU freedom of movement Irene A. Glinos, Matthias Wismar, James Buchan, Ivo Rakovac

19. Investing in health literacy: What do we know about the co-benefits to the education sector of actions targeted at children and young people? David McDaid

20. How can structured cooperation between countries address health workforce challenges related to highly specialized health care? Improving access to services through voluntary cooperation in the EU. Marieke Kroezen, James Buchan, Gilles Dussault, Irene Glinos, Matthias Wismar

21. How can voluntary cross-border collaboration in public procurement improve access to health technologies in Europe? Jaime Espín, Joan Rovira, Antoinette Calleja, Natasha Azzopardi-Muscat, Erica Richardson, Willy Palm, Dimitra Panteli

22. How to strengthen patient-centredness in caring for people with multimorbidity in Europe? Iris van der Heide, Sanne P Snoeijs, Wienke GW Boerma, François G Schellevis, Mieke P Rijken. On behalf of the ICARE4EU consortium

23. How to improve care for people with multimorbidity in Europe? Mieke Rijken, Verena Struckmann, Iris van der Heide, Anneli Hujala, Francesco Barbabella, Ewout van Ginneken, François Schellevis. On behalf of the ICARE4EU consortium

24. How to strengthen financing mechanisms to promote care for people with multimorbidity in Europe? Verena Struckmann, Wilm Quentin, Reinhard Busse, Ewout van Ginneken. On behalf of the ICARE4EU consortium

25. How can eHealth improve care for people with multimorbidity in Europe? Francesco Barbabella, Maria Gabriella Melchiorre, Sabrina Quattrini, Roberta Papa, Giovanni Lamura. On behalf of the ICARE4EU consortium

26. How to support integration to promote care for people with multimorbidity in Europe? Anneli Hujala, Helena Taskinen, Sari Rissanen. On behalf of the ICARE4EU consortium

27. How to make sense of health system efficiency comparisons? Jonathan Cylus, Irene Papanicolas, Peter C Smith

The European Observatory has an independent programme of policy briefs and summaries which are available here: http://www.euro.who.int/en/about-us/partners/observatory/publications/policy-briefs-and-summaries

Joint Policy Briefs

World Health OrganizationRegional Office for EuropeUN City, Marmorvej 51,DK-2100 Copenhagen Ø,DenmarkTel.: +45 39 17 17 17Fax: +45 39 17 18 18E-mail: [email protected] site: www.euro.who.int

The European Observatory on Health Systems and Policies is a partnership that supports and promotes evidence-based health policy-making through comprehensive and rigorous analysis of health systems in the European Region. It brings together a wide range of policy-makers, academics and practitioners to analyse trends in health reform, drawing on experience from across Europe to illuminate policy issues. The Observatory’s products are available on its web site (http://www.healthobservatory.eu).

No. 27ISSN 1997-8073

health systems and policy analysis policy brief 27 · 2018-02-21 · keywords: delivery of health...

Documents