1

Efficiency of Health Care Systems – what it is and how to measure and compare it

by Gernot Pehnelt

Gernot.pehnelt@ecipe.org

--DRAFT--

Please do not cite without author’s permission. Comments and feedback

will be much appreciated

Introduction

Health – often referred to as the „highest good‟ (summum bonum) – and the provision of

health care services have been a major issue in national and international politics and, not

least, for the very individual, indeed. The question on how effective, efficient and fair health

care systems around the globe perform has attracted much attention and an intensive and

controversial debate not only in academic and political circles. In recent years, quite a few

studies have been conducted in order to evaluate and compare the efficiency of health care

systems. Given the different theoretical and methodological approaches, it is rather

unsurprising that the results of these studies are far from conclusive.

In this study, we will discuss the question what efficiency of health care systems actually

means, how it can be measured and – finally – how the performance of health care systems

can be compared on a reasonable and fair basis. In order to do so, we will first develop a

theoretical framework of the very nature of the „production process‟ of health care systems.

Based on these theoretical considerations, various approaches to evaluate the performance of

health care systems will be assessed and a procedure to measure the efficiency of different

health care systems will be developed and discussed.

The Health Care System – A Productive Entity

The Output Dimension

According to the World Health Report 2000 (WHO 2000) health systems are defined as

comprising all the organizations, institutions and resources that are devoted to producing

health actions. A health action is defined as any effort, whether in personal health care, public

2

health services or through intersectoral initiatives, whose primary purpose is to promote,

restore or maintain health. In other words, health care systems can be described as productive

entities, consisting of various components or sub-divisions, whose primary goal and –

economically speaking – output is the improvement of the health care status of the (potential)

patients.1 The value of healthy time gained by health actions can manifest itself in various

forms. Healthy time can be used for productive work generating income for the very

individual and creating an output and additional wealth for the society. Healthy time could

also be used for leisure activities yielding to additional perceived personal utility. Last but not

least, living in a better health state has a value to the individual in its own right, e.g. the

absence of pain, higher mobility and better quality of life (Drummond et al. 2005).

Sure, there are various other goals that contribute to the performance of a health care system

including accessibility, fairness, a sustainable and just financing, customer satisfaction,

transparency, respect for the patient etc. However, these instrumental goals are rather means

to the very end of health care „production‟ which is protecting and improving the health status

of the entire population over people‟s whole life cycle. Once the achievement of this final

goal can be measured, there might be no reason to integrate every single outcome or sub-

ordinate target explicitly into the efficiency analysis. Unfortunately, the improvement of the

health status of the patients due to health actions cannot be measured directly. This is, first,

because there is no exact measure or even a good proxy of the (virtual) health status of the

population in the absence of any health care system.2 Second, the actual health status of any

individual within a society with all the relevant physical and mental dimensions3 can hardly be

quantified at a certain point in time, let alone in a dynamic perspective. Third, as we will

discuss later, there are many factors outside the health care system that might influence the

health status of individuals. To sum up, many questions about health system performance

have no clear or simple answers because outcomes are hard to measure and it is hard to

disentangle the health system‟s contribution from other factors (WHO 2000).

Different outcome measures have been used to approximate the performance of health care

systems. Generic outcome measures often used in utility analysis are quality-adjusted life-

years (QALYs)4, healthy-years equivalent (HYE)5, the disability-adjusted life-years (DALY)6,

1 The term „potential patients‟ basically includes the entire population of a country since every individual has

a certain demand for health services, let it be in cases of urgent emergencies, a demand for general

diagnostic procedures, basic services or some sort of optional demand. 2 Even very simple and indigenous societies have some sort of a health care system and use resources to

perform health actions. 3 By the way, following the definition of health given by the World Health Organization (WHO) whereby,

“health is a state of complete physical, mental and social well-being and not merely the absence of disease

or infirmity”, almost every human being might “suffer” from certain health deficits. 4 QALYs basically take into account the quality and the quantity of the life years added by a medical

intervention. Each year in perfect health is assigned the value of 1 down to a value of 0 for death. If the extra

years were not being lived in full health, for example if the patient suffers constant pain or be confined to a

wheelchair, then the extra life-years would be weighted with a value between 0 and 1 to account for this. 5 HYE is calculated o the basis of the number of years of perfect health that has the same utility as the lifetime

path of health states under consideration (Gold et al. 1996). 6 The DALY is a health gap measure that extends the concept of potential years of life lost due to premature

death to include equivalent years of “healthy” life lost by virtue of being in states of poor health or disability.

It combines in one measure the time lived with disability and the time lost due to premature mortality. One

3

age specific mortality rates, infant mortality or simply the life expectancy at birth. On the one

hand, these indicators are appropriate proxies for the final outcome of a health care system

and other factors that influence the health status of human beings. On the other hand, the

output of health care systems is multidimensional, by all means. First, because of the massive

technological progress and accumulation of knowledge about the factors that influence the

health status, certain diseases and their transmission, etc. the scope of modern health care

systems is much more diversified than a couple of decades before. Health education, for

instance, becomes more and more important when it comes to combating some of the most

widespread diseases in Western society, such as obesity and diabetes. Second, people do no

longer occupy the health care system for just the relief of pain and treatment of physical

limitations and emotional disorders. They are also looking for diagnostic procedures on a

regular basis without a direct indication, for advice on diet, child-rearing and other behaviour.

Third, maintaining and improving a functioning health care system with sufficient capacities,

decent facilities and well educated experts meets a certain optional demand that every

individual has. That is why offering state-of-the-art health care services or providing the

opportunity to use health care resources is an output itself. 7 Fourth, product and process

innovation and technical progress in diagnostics, surgery, medication etc. also contribute to

the performance of health care systems. The higher the rate of technological progress and

accumulation of knowledge, the more effective will be the identification of disease patterns

and, finally, the treatment of patients. Furthermore, under certain (unfortunate) circumstances,

such as a natural disaster or a high prevalence of certain diseases (diabetes, HIV/Aids etc.)

because of the population‟s genetic exposure or life-style patterns the health care system of a

particular country might have to carry out a lot of health actions in order to maintain a decent

level of health amongst the population whereas the health care system of another country

simply does not face these rather structural problems that cannot be influenced in the short run.

Saying this, it has to be recognized that to perform health actions is an (intermediate) output

itself. Overall, there might be a case for integrating some of these additional or intermediate

outputs when it comes to the assessment of the performance of health care systems.

The Input Dimension

In order to assess the productivity of health care system, one has to define the actual

boundaries of a health system regarding the very inputs that are used to „produce‟ the

multidimensional output. There have been quite a few attempts to define boundaries that

separate the health system from elements outside of it (e.g. Terris 1978, Hsiao 1992,

Chernichovsky 1995). There is little doubt about that certain facilities and services such as

hospitals and primary care are elementary parts of the input domain of health care systems.

There is more controversy around components that have health-enhancing benefits but are not

DALY can be thought of as one lost year of „healthy‟ life and the burden of disease as a measurement of the

gap between current health status and an ideal situation where everyone lives into old age free of disease and

disability (WHO 2008). 7 One important objective of modern health systems is to respond to people‟s expectations regarding not only

the actual quantity and quality of care but also the availability of decent treatment – just in case.

4

primarily intended to influence the overall level of health within a society.8 Examples for such

factors are the educational system or urban infrastructure. Furthermore, there are some

regulations and actions that clearly aim at the safety or health of individuals but are still not

referred to as inputs within the health care system. For example, the prohibition of smoking in

public buildings does most probably influence the smoking habits of individuals and – most

definitely – contribute to the health of non-smokers who work in public buildings. However,

the costs that are related to the enforcement of non-smoking rules are usually not integrated as

an input of the production process of health care systems. The same holds for environmental

and labour standards, certain safety rules and other regulations. One could argue that health

education at school is primarily intended to promote health and should therefore be

recognized in productivity analyses. Well, looking straight forward at first glance, this issue is

– again – far from trivial. First, the question is whether deeper understanding of how diseases

are spread, how important individual hygienic practices and a decent nutrition are and how

certain habits may influence the risk of getting ill is an input or an output of health care

system. On the one hand, the resources used to teach the students these things are means to an

intended end; that is improved health. Therefore, these resources can be referred to as inputs

in the production process of the health system. On the other hand, enhancing people‟s

knowledge about health related issues and giving them the opportunity to choose their

individual life-style and habits rationally on the basis of a well-founded cognition of the

complex sphere of health is most definitely beneficial and therefore an output itself. Second,

from a practical point of view, one must confess that it would be almost impossible to

separate the resources that have been used primarily in order to – directly or indirectly –

promote health, let it be in public schools, businesses or sports clubs. How many hours a year

does the 2nd

grade teacher at school spend to teach things that are primarily intended to

promote children‟s health or their knowledge about health related issues? How many

resources does a medium-sized enterprise invest to maintain or improve the health of its

employees and what will be the result? Even if some schools or businesses account for these

things, comprehensive information about these health-related inputs/outputs do not exist on a

broader scale and will most probably never exist.

Each health system should be judged according to the resources actually at its disposal, not

according to other resources which in principle could have been devoted to health but were

used for something else (WHO 2000), even if these activities outside the health care system

do have an impact on the health status of the population. We call factors that are outside the

defined boundaries of the health care system non-health care determinants. To make one thing

clear, these non-health care determinants may play a major role in assessing the overall

performance of health care systems and are of special importance in explaining differences

between countries. That is why they surely have to be recognized in any analysis, though

maybe not explicitly as direct inputs in the very production process of health care systems.

However, given the considerations discussed above, we prefer a rather narrow definition of

the boundaries of health care systems. It is the resources that are employed in the health care

8 “Clearly, all boundary definitions are arbitrary but to undertake an assessment of health system performance

an operational boundary must be proposed.” (Murray and Frenk 1999)

5

sector and, indeed, health care expenditures that matter most when it comes to an assessment

of the efficiency of health care systems. Some of the resources used in the production process

in the narrower sense can be measured relatively easily. It is the labour employed in the health

sector, the capital that has been invested in health care facilities, the utilities and other health

care expenses that can be measured in monetary terms. However, there are still some

important inputs that are rather intangible, such as the specific know-how and the level of

technology. These inputs can be estimated by certain proxies as the number of high-tech

equipment such as MRT or CT or the ratio of examined medical personnel employed in the

health care sector.

The health care production process at a glance

The production process of health care systems can be described as follows. Resources that lie

within the boundaries, referred to as health care resources, are used to provide health care

services in order to improve the health status of the population.9 The primary output of this

process is the difference between the initial health status of the population (without any health

services) and the actual health status that has been achieved due to the health actions

performed by the health system (the “delta” in Figure 1).

Figure 1: The health care system as a productive entity

The initial (and actual) health status of the population as well as the health care system itself

are highly influenced by factors that lie more or less outside the boundaries of the health care

systems, such as the average income, the educational level, sanitation and nutrition, the

9 The patients are the most important physical input of the production process, so to speak.

population (potential patients)

initial health status

non-health care determinants:

average income

education

sanitation, nutrition

environment, polution etc.

life-style

other socioeconomic factors

health care system

health care resources

regulation and other policies

population

actual health status

∆ =

ou

tpu

t o

f th

e h

ealt

h c

are

sys

tem

6

environment, genetic exposure, the life-style and other socioeconomic factors.10 These non-

health care determinants might be even more important for the health status of a country‟s

population. In fact, some evidence suggests that health systems make little or even no

difference regarding the overall level of health. Comparisons across countries show that while

per capita income, education, income inequality or cultural characteristics are strongly related

to the actual health status there seems to be little independent connection between the level of

health and health care inputs such as doctors or hospital beds (Cochrane et al. 1978), total

health expenditure (Musgrove 1996), expenditure only on conditions amenable to medical

care (Mackenbach 1991), or public spending on health (Filmer and Pritchett 1999). It is not

surprising to find that these relations are weak in rich countries, since many causes of death

and disability are already controlled and there are many different ways to spend health system

resources, with quite varying effects on health status. But health system expenditure often

seems to make little difference even in poor countries with high infant and child mortality,

which should be a priority to reduce (WHO 2000).11

Figure 2 translates the complex health care system into terms of production. The initial

(secondary) inputs such as medical personnel and other labour, equipment, facilities and other

capital, know-how, utilities and other resources are used and combined in the production

process. The very nature of this production process in terms of a production function remains

pretty much unclear and can be referred to as a “black box”. The intermediate output of this

production process are the health actions that are performed to reach the final goal. These

secondary outputs such as diagnoses, surgeries and other procedures, outpatient treatments

and care days, medication, information and documentation are used as primary inputs in order

to cure the patients, ease their pain, weaken the symptoms and maintain or improve overall

health which is the final and primary output of the health care system.

10 Maybe, some of these factors can be influenced by certain health policies and regulations, at least in the long

run. However, many of these non-health care determinants lie completely outside even a broad definition of

what health systems are. 11 There is little evidence for a significant impact of higher cost on beneficiary outcome. Some regions in the

US have markedly higher cost than others, with no higher patient satisfaction or health improvement

(Dartmouth 1998).

7

Figure 2: Inputs and outputs in the health care production process

As discussed before, the secondary inputs can be measured relatively easily by counting the

number employees or the full-time equivalents (FTE) in the different sub-sectors of the health

care system and collecting data about health care expenses. The same is true for the secondary

outputs that are basically well documented in health care statistics. The difficult part is to

quantify the primary output. The generic measures or a combination of some of these

measures mentioned above seem to be the most appropriate way to estimate the final outcome

of the production process.

The assessment of health system performance

The concept of relative performance

Given the tremendous importance of non-health care determinants for the initial and actual

health status of the population, international comparisons of the performance of health care

systems are not a trivial task. Just comparing the outcome or output-input-ratios might be not

appropriate to get a reasonable and fair assessment of the overall performance and efficiency

of different health systems around the world. In the following we explain the difficulties that

come around and develop an approach that is able to overcome some of these problems.

Murray and Frenk (1999) introduced a concept that compares the actual performance of a

health care system with its potential, taking into account that the actual outcome – that is the

health status of the population – would not be zero if there was no health care system at all.

Even without any health care expenditure or a functioning health care system, health levels

process 1 process 2

process n

production process („Black Box")

(combination of secondary inputs)

• capital

• labour

• know-how

• utilities

• other resources

• diagnose

• procedures and

surgeries

• patient & care days

• medication

• documentation

• cure

• easing the pain

• weakening the

symptoms

• improve overall health

health care resources

/ secondary inputs

primary input /

secondary

output

primary output

…

8

would not be zero because the entire population would not be dead or seriously ill. The initial

health status, so to speak, of a country‟s population is supposed to be highly correlated with

various factors such as the average income, the educational level, environmental issues,

infrastructure, sanitation, nutrition and the like (non-health care determinants). The outcome

of a health care system could be defined as the difference between the observed health status

of the entire population and the initial (virtual) health status of the population given a certain

level of these factors. Therefore, performance of the health system involves relating goal

attainment to what could be achieved under certain circumstances that cannot be addressed by

the health care system directly. In other words, performance is a relative concept. A rich

country has higher levels of health than a poor one, but which country has a higher level of

performance relative to health system resources? Murray and Frenk (1999) argue that

performance should be assessed relative to the worst and best that can be achieved for a given

set of circumstances.

Figure 3 illustrates an example.12 The upper lines (hmaxi) show the maximum attainable level of

health (y-axis) for the given level of health expenditures (x-axis), given the non-health care

determinants of health status such as the average income, the educational level or

environmental pollution for every country i. The stars (hmini) represent the minimum level of

health given the specific non-health system circumstances in this particular country i. The

circles (Hi) show the actual observed health status in each country i. What can be seen is that

country A certainly has a lower level of health than country B – initially (hminA < hmin

B) and

actually (HA < H

B). However, both countries show exactly the same health care system

performance in absolute terms, no matter if this performance is measured by the distance

between minimum level and the actual level of health (p* = improvement due to the health

care system) or the distance from the observed health status to the maximum attainable

frontier (p° = potential improvement / level of inefficiency). In relative terms, the

performance of country A‟s health care system is even better than the performance of B.

Notice that even if country A invested as much health care resources as country B, the initial

health status of country A‟s population would still quite below the health status of country B‟s

population (hminA < hmin

B). That is due to unfavourable circumstances such as certain

socioeconomic, life-style or environmental factors. That is why the maximum attainable

health status – the individual efficient frontier – at a given amount of health expenditures is

also lower in country A than in country B (FhmaxA < Fhmax

B). 13

12 The examples and illustration given by Murray and Frenk (1999) and elsewhere (e.g. Evans et al. 2001) are

somehow misleading. That is why we use a different approach to illustrate the very problem. 13 The shape of the efficient frontier implicitly assumes diminishing marginal returns to investments in health

care services. We will discuss this issue later in this chapter. The slope of the minimum level of health

(Fhmin) can be expected to have a curved slope as well. However, since this detail is not relevant here, we

skip this discussion.

9

Figure 3: Health system performance

There might be another country (C) that has the same actual health status as country B (HC =

HB). At first glance, country C would probably ranked first among the three countries if it

comes to a simple ratio analysis of the performance of the health care system since country C

uses fewer resources than country B and reaches the same overall health status. However, the

performance of country C‟s health care system seems to be not much better or even weaker

than B‟s performance, since the improvement due to the health care system is lower in

country C than in country B (p*C < p

*B) and the lack of efficiency compared to the individual

efficient frontier (Fhmax) is higher in country C (p°C > p°

B). Because of the favourable non-

health care determinants of health status in country C, this country could reach a much higher

overall health status than it does (see the efficient frontier FhmaxC). C faces the highest

individual efficient frontier among the three countries.

To assess the relative performance of health care systems, it would be desirable to develop an

individual benchmark, an upper limit or “frontier”, corresponding to the best practice that

could be achieved by the health system under consideration. This frontier would represent the

level of attainment which a health system might achieve under the given non-health care

determinants, but which no country surpasses (Donabedian et al. 1982). With such a frontier

approach it would be possible to see how much of the potential has been realized. In other

words, comparing actual attainment with potential shows how far from the relevant frontier of

maximal performance is each country‟s health system (WHO 2000). In economic terms, the

distance between the actual outcome and the potential would be referred to as slack or

inefficiency.

Now we draw our attention to the slope of the best-practice frontiers in Figure 3 which leads

us to a very important fact that have not been recognized by the vast majority of empirical

studies. It is very reasonable to assume diminishing returns in the production of health

hmaxA

hmaxB

hmaxC

HB

HC

HA

p*C

hminA

hminB

hminC

Country C hea

lth s

tatu

s

health system resources

Country A

Country B

p*A

p*B p°

A

p°B p°

C

FhminA

FhminB

FhminC

FhmaxB

FhmaxA

FhmaxC

10

concerning the final outcome, regardless the measure applied. Coming from a very poor level

of overall health, i.e. a low life-expectancy, it might take just a few relatively inexpensive

measures to improve the average health and life-expectancy of the population. The

improvement of hygiene in hospitals, for instance, has dramatically reduced hospital mortality

in the 20th

century. The cost of the related measures, such a sterilizing surgical instruments or

simply encouraging medical personnel to wash one‟s hands and provide the necessary

facilities to do so, are comparably low. The same holds for basic vaccination. Some diseases

have almost been wiped out in many industrialised countries in the last century due to

vaccination. By the way, some of the most effective measures that are able to enhance the

overall health status and improve the life-expectancy lie outside the health care system. The

tremendous increase in life-expectancy and the quality of life since the mid-1800s where due

to better nutrition, improvements in urban sanitation and personal hygiene that came along

with a better understanding of how diseases spread and, not least, the rise of capitalism, an

enhanced division of labour and the subsequent economic growth and increasing social

welfare. Technically speaking, economic growth pushed the frontiers upwards and increased

the overall level of health by improving the initial health status. Furthermore, technological

progress and a more efficient use of resources made it possible to invest more in health, which

would be represented by a country‟s move to the right in Figure 3.

Once a country has already reached a high level of health, it takes much more effort to further

reduce morbidity and increase life-expectancy. Compared to the rather basic health actions

mentioned above, the treatment of certain diseases like cancer are very expensive. Although

modern chemo and radiation therapy together with subsidiary medication are quite successful,

they are not able to increase the average life expectancy within the society by years or even

decades like some of the measures discussed above could. The same is true for almost every

intensive care measure. Intensive care, a surgery or decent medication can save one‟s life,

ease the pain or even cure a disease and therefore add some additional life-time. However, in

the end, even if one disease can be successfully defeated, it just means replacing one

particular cause of death by another one. The higher the average level of health (e.g. the life-

expectancy) is, the harder and more expensive it gets to further improve the health and life

expectancy of the population.

Why is this important for efficiency analyses of health care systems? Well, economically

speaking, the health system faces diminishing marginal returns – beyond a certain level, the

production function becomes very flat. That is why country B in Figure 3 has to invest much

more into its health care system (in absolute and relative terms) to get a marginal

improvement of the overall health status than country A. In other words, the marginal

productivity of country A‟s health care system at this certain point of economic development

is by nature higher than the marginal productivity of country B‟s health system. Any

investment in health actions in country A (movement to the right in Figure 3) would add quite

a lot to the average level of health in this country, whereas country B must invest quite a lot to

increase the overall health of its population just a bit. What can be drawn is that, because of

various reasons, it is a trivial task to actually compare the efficiency of country A and country

B regarding the production of health.

11

However, there have been attempts to compare the efficiency of different health care systems

and some of them are quite promising. Since some studies do not take the relative nature of

efficiency into account, we, in the following, rather focus on studies that do incorporate the

relative concept of performance.

Data Envelopment Analysis – an appropriate tool to measure relative efficiency

Based on the work of Farell (1957)14 Charnes et al. (1978) developed the Data Envelopment

Analysis (DEA) to measure the efficiency of productive entities, so called decision making

units (DMUs). This method, which has been advanced during the last decades 15, is very

appropriate in assessing the (relative) efficiency of DMUs that produce multidimensional

outputs and use certain inputs that cannot be measured in monetary terms.

Model (1) calculates the relative efficiency of DMU 0 in comparison to every other DMU in

the observed sample of n DMUs. Every DMU j is a productive entity that uses the i = 1,..., m

inputs (xij > 0) to produce r = 1,..., s outputs (yrj > 0). The model chooses the individual

weights of every single input and output (qi, pr) in a way that maximizes the efficiency score

ho.

m.,...,1is,...,1r0q ,p

n...,,1 j1

xq

yp

NB:

xq

yp

hmax

ir

m

1i

iji

s

1r

rjr

m

1i

oi

s

1r

ror

oq ,p

(1)

The optimization model (2) – the so called envelopment-form – calculates the DEA-Score θo*

that shows to which level the DMU 0 has to reduce the inputs x to become efficient. Efficient

units get a DEA-Score of 1.

n.,...,1j0λ

m,...,1i0xλθx

s,...,1ryyλNB:

θmin

j

n

1j

ijjio

ro

n

1j

rjj

oλ

(2)

14 For an overview on the origins of Data Envelopment Analysis see Forsund and Sarafoglou (2002). 15 See for instance Löthgren and Tambour (1996), Andersen and Petersen (1993), Cooper et al. (1996) and

Sengupa (1989). Seiford (1996) provides an outline of the early developments and a bibliography of the

DEA.

12

Figure 4 shows an example with seven DMUs that produce the same output (quantity and

quality) with two inputs.

Figure 4: DEA – Example with 7 DMUs, two inputs and one output

The efficient DMUs A, B, C, D and E lie on the frontier.16 The frontier “envelops” the two

inefficient DMUs F and G. In order to be ranked efficient, F and G had to reduce both inputs

to reach point C and G‟, respectively. As can be seen, the DEA is a relative concept of

efficiency. The efficiency of every DMU is assessed relatively to the performance of real

DMUs or convex combinations of existing DMUs. The benchmark, so to speak, for DMU F is

DMU C. The benchmark for DMU C is constituted by a combination of the efficient peers C

and D.

DEA explicitly incorporates the relative concept of efficiency by measuring the relative

distance of every DMU to the best-practice frontier. Furthermore, the peers or benchmarks of

every inefficient DMU are generally similar to that DMU in terms of the relative importance

of the inputs and outputs in the production process.

With DEA one can also take into account variable returns to scale. Some technologies possess

increasing returns to scale, whereas other production processes might come along with

constant or decreasing returns to scale. Under the assumption of constant returns to scale

(CRS), the efficient frontier FCRS in Figure 5 is determined by the position vector from the

origin through the points C and A. The efficient DMUs A and C lie on this frontier and are

identified as efficient (θACRS* = θC

CRS* = 1). The other DMUs (B, D and E) are ranked inefficient

by the CRS-model (θBCRS* , θD

CRS* , θECRS* < 1). The magnitude of the inefficiency is quantified by

the relative distance of the inefficient DMU to the frontier. In case of DMU B this is the ration

B°B‟‟/B°B. In order to become efficient, DMU B had to reduce the input by the distance BB‟‟.

16 The frontier is determined by linear sections (convex combinations) between two efficient DMUs at a time.

Note that DMUs A and E lie on the frontier although they could reduce one input by one unit. The DMUs A

and E exhibit some slack and are called “weakly efficient”.

Input 1

Input 2

A

B

C

D E

F

G

G’

1 2 3 4 5

1

2

4

3

0

13

Figure 5: DEA – CRS- and VRS-model

Under the assumption of variable returns to scale (VRS) the best-practice frontier FVRS is

determined by the DMUs A, B, C and D and the respective convex combinations of the

neighbouring DMUs. The VRS-model identifies 4 of the 5 DMUs as efficient, taking into

account that up to a certain size, the production process possesses increasing returns to scale

and beyond a certain threshold decreasing returns to scale. In other words, the DMUs B and D

produce technically efficient but are too small (DMU B) or too large (DMU D) to be fully

efficient. DMUs B and D are scale inefficient. The magnitude of the scale inefficiency of a

DMU 0 can also be measured using the DEA-scores calculated by CRS- and VRS models:

*VRS

o

*CRS

oo (3)

One of the biggest advantages of DEA is the possibility to integrate various multidimensional

inputs AND outputs into the efficiency analysis. This is of special importance in the health

care sectors where the output is multidimensional and some inputs and especially the output

can hardly be measured in monetary terms. Furthermore, there is no need to specify a certain

type of production function. Since very little is known about the functional form of health

care production, the endogenous – so to speak empirical – way to identify the best-practice

frontier is another major advantage of this non-parametric technique.

DEA can also be used to measure productivity changes over time. Figure 6 illustrates how the

dynamic productivity change can be measured in a disaggregated way. DMU D, for instance,

is identified in both points of time (t and t+1). The technical efficiency of the inefficient DMU

D at time t is 0J/0Dt and 0M/0D

t+1at time t+1, respectively (θD

CRS* < 1). In order to assess the

dynamic productivity change of DMU D, its technical efficiency relatively to the frontier in

time t and time t+1 will be measured. The geometric mean of these two rates of change shows

Output

A

B

C

D

E

FVRS

FCRS

0

B’’ B°

Input

14

the change of the total factor productivity of DMU D (∆TFPD) over the whole period. This is

the Malmquist-Index of this particular DMU (MID).

Figure 6: Disaggregated Malmquist-Index

The Malmquist-Index can be separated into two measures of efficiency change, an efficiency

change („catch up‟) component and a technology change („frontier shift‟) component:

2

1

t1tDL0

J0

M0

K0

D0

J0

D0

M0MI (4)

The first term on the right hand side („catch up‟) of equation (4) measures the change of the

distance of DMU D to the frontier in time t+1 (0M/0Dt+1

) relative to the distance to the

frontier in time t (0J/0Dt). If the score for this catch up component is higher than 1, the DMU

has enhanced its relative technical efficiency, it has cough up.17 The term on the right hand

side of equation (4) measures the shift of the frontier over time. Values of this component

bigger than 1 indicate dynamic productivity gains within the sample or, so to speak, the

underlying production technology. Using the disaggregated Malmquist-Index one cannot only

identify the productivity change of every single DMU that are due to individual efforts but

also efficiency changes that are due to technological progress or other factors that influence

the productivity of the whole sample. Again, this tool is very useful to assess the efficiency of

productive entities in health care because it incorporates a relative efficiency concept and can

handle multidimensional inputs and outputs.

17 If the score for the catch up component iss maller than 1, one actually had to call it „fall behind‟.

Input 1

Input 2

At

Bt

Ct = C

t+1

At+1

Bt+1

Dt

Dt+1

0

FCRS – time t +1

FCRS – time t

J

K

L

M

„catch up“ (∆E) „frontier shift“ (∆F)

15

Efficiency analysis in health care – some applications and results

Because of its specific advantages, during the last 20 years, Data Envelopment Analysis and

other non- or semi-parametric approaches have been used increasingly to measure the

efficiency of health care services. The vast majority of these studies focus on certain facilities

such as hospitals (e.g. Grosskopf and Valdmanis 1987, Borden 1988, Ozcan and Luke 1993,

Burgess and Wilson 1995, Grosskopf et al. 2001, Biorn et al. 2003 and Prior 2006), nursing

homes (e.g. Nyman and Bricker 1989, Kooreman, P. 1994, Rosko et al. 1995 and

Chattopadhyay and Ray 1996) or primary health care centers (e.g. Huang and McLaughlin

1989, Szczepura et al. 1993 and Athanasios et al. 2002).18

Interestingly, there are not many applications of non-parametric techniques to measure and

compare the efficiency of health care systems as a whole.19

Färe et al. (1997), analyzing a sample of 19 OECD countries, found a very widespread and

rapid productivity growth between 1974 and 1989 among the 10 countries with complete data

with Denmark and the US showing the highest cumulative productivity growth in this period

(> 30%). The main driver of this productivity growth was technical change.

Alfonso and St. Aubyn (2006) estimate semi-parametric models of health production using a

DEA-based two-stage approach for OECD countries for the years 2000-2003. Input variables

include medical technology indicators and health employment. Output is measured by

indicators such as life expectancy and infant mortality. The authors use a principal component

analysis (PCA) of the outputs life expectancy, infant survival rate and potential number of

years of life not lost. This reduces the dimensionality of multivariate output to one single

measure which has its advantages when the sample size is relatively small. However, by

running PCA one gets an artificial output measure that might not represent the output of every

health care system well enough. In the basic model, among 21 OECD countries, Canada,

Finland, Japan, Korea, Spain, Sweden and the US show the highest efficiency. The efficiency

of the Czech Republic, the Slovak Republic and especially Hungary falls way behind the

sample median. Inefficiency in the health sector has been found strongly related factors that

are, at least in the short to medium run, beyond the control of the health system: the GDP per

capita, the level of education, smoking habits, and obesity. In a next step, the authors take into

account differences in national income, obesity and tobacco consumption and correct the

efficiency score for these factors. Comparing the ranks and efficiency scores of the

uncorrected model with the scores and ranks resulting from the correction, significant changes

occur. Some countries poorly ranked previously are now closer to the production possibility

frontier – e.g. Denmark, the Czech Republic, Hungary, the Slovak Republic, and the UK. On

the other hand, countries like Canada, Japan, Sweden and the US receive remarkably lower

efficiency scores after the correction.

18 For a review of non-parametric studies in health care see Hollingsworth et al. (1999) and Hollingsworth

(2003). 19 Gupta et al. (1997) use an approach that is similar to DEA (free disposal hull – FDH) to analyze the

efficiency of government spending on health and education in Africa with life expectancy and infant

mortality as the main outputs.

16

Spinks and Hollingsworth (2005) go even further in integrating socioeconomic factors into

the models. They use measures for education (school expectancy years), employment

(unemployment rates), and income (GDP per capita) as inputs, additional to total health

expenditure. Life expectancy at birth is the only output. The authors calculated the technical

efficiency of OECD countries based on the data provided by the OECD and the WHO

respectively. In the WHO setting, they use a different output measure (disability adjusted life

expectancy – DALE). Turkey, Mexico, Korea, Greece, Spain and Japan reach full efficiency

in both models (OECD and WHO dataset). Italy and France lie on the frontier only in the

WHO setting. Iceland and Switzerland are identified as efficient only in the OECD setting.

The most remarkable efficiency deficits have been found for Denmark, the US, Belgium and

Germany.20

Spinks and Hollingsworth (2005) also calculate changes in efficiency between 1995 and 2000

(OECD dataset) and 1993 and 1997 (WHO database) using the disaggregated Malmquist-

Index. The mean value of the catch up component is 0.961 for the OECD dataset indicating

that overall, member countries have moved slightly away from the frontier, representing a

decrease in technical efficiency. Similarly, the mean technological change value (frontier shift)

of 0.995 would suggest that that the efficiency of the whole technology has declined slightly.

Overall, the mean efficiency of the whole sample have decreased between 1995 and 2000

(mean Malmquist-Index = 0.956). Interestingly, results using the WHO dataset move

conversely to results for the OECD dataset. Overall, the technical efficiency has improved

(mean catch up component = 1.041), whereas the efficient frontier has retracted (mean

frontier shift = 0.974). The total factor productivity seems to have increased slightly between

1993 and 1997. (MI = 1.014). Looking at the individual scores of the disaggregated

Malmquist-Index calculated by Spinks and Hollingsworth (2005) on the basis of the two

different datasets draw a rather dubious light on the results since for the majority of countries

the scores move to the opposite direction using the two different datasets. Since the two

periods do not differ considerably, the odd result maybe due to the different output measure

(OECD: life expectancy, WHO: DALE).

Both approaches discussed above (Alfonso and St. Aubyn 2006; Spinks and Hollingsworth

2005) incorporate broad socioeconomic or life-style measures more or less explicitly into the

efficiency analysis. As argued before, some measures of socioeconomic status, such as the

average income, the level of overall economic development, education and environmental

factors have clearly shown to be associated with health status (Macintyre 1993, Kaplan et al.

1996, Laporte 2002). In other words, different countries may face different individual

efficiency frontiers. That is why the idea to correct the efficiency scores for some important

factors that cannot be influenced by the health care system, at least not in the short run,

follows the concept introduced above (see again Figure 3). However, the dramatic changes of

the ranks and especially some individual efficiency scores due to the correction draw a rather

dubious light on the methodological approach used by Alfonso and St. Aubyn (2005).

Furthermore, integrating factors that are not under control of the productive entity to be

20 Notice that some of these findings are quite contrary to the results documented by Alfonso and St. Aubyn

(2006).

17

assessed directly into the efficiency analysis (Spinks and Hollingsworth 2005), is in conflict

with the very nature of efficiency analysis.

Towards a concept to measure the efficiency of health care systems – discussion

Efficiency in health care is relative. That is why the actual performance of a health system has

to be assessed relative to the performance of a group of peers that can be reasonably

compared to the system under evaluation. Although factors that lie outside the boundaries of

the health system might be even more important in determining the health status of the

population than the actions performed by the health system, it does not make sense to

incorporate factors that cannot be controlled by the decision makers within the health system

into the efficiency analysis, at least not explicitly as inputs. However, these factors surely

have to be taken into account when it comes to an assessment of the (relative) performance of

the health system. Therefore, we suggest a clear cut two-step procedure. First, the relative

efficiency of health care systems has to be measured by including all inputs that are used to

actually produce the output. Since the output of health systems is multidimensional, it might

be not enough to focus solely on life expectancy or other rather broad measures of the health

status of the population as the output measure. It could be necessary to integrate some of the

secondary outputs (e.g. some health actions that have been actually performed) into the

analysis in order to get a reasonable and fair evaluation of the efficiency of health systems.

This is not least due to the diminishing marginal returns to investment in health care that most

probably causes a lower marginal, though not necessarily average, productivity in countries

that already invest a lot into the health system.

Once the efficiency of the health systems has been calculated on the basis of factors that are

under control of the health system, the causes of differences in the relative efficiency can be

investigated, using regression analysis with the efficiency scores as the dependent variable

and certain socioeconomic and other factors as explanatory variables. The results can be used

to further elaborate and interpret the efficiency assessment. Using an appropriate research

design, one may even calculate specific elasticities of certain factors that affect the efficiency

of health care systems. These elasticities could be used to correct the efficiency scores derived

in the first step in order to take into account the different best practice frontiers the different

countries might face.

Because of its relative and non-parametric concept and the possibility to integrate multiple

inputs and outputs of different dimension into the analysis, the Data Envelopment Analysis

(DEA) is a very appropriate and powerful tool to assess the efficiency of health care systems.

Chapter on the design or reference to the second paper

18

19

References

Afonso, A. and St. Aubyn, M. (2006): Relative Efficiency of Health Provision: a DEA

Approach with Non-discretionary Inputs. Working Papers 2006/33, Department of

Economics at the School of Economics and Management (ISEG), Technical

University of Lisbon.

Athanasios, I.Z.; Tsakos, G.; Economou, C. and Kyriopoulos, J. (2002): Using DEA to

Evaluate Efficiency and Formulate Policy Within a Greek National Primary Health

Care Network. Journal of Medical Systems, Vol. 26(4), pp. 285-292.

Biron, E.; Hagen, T.P.; Iversen, T. and Magnussen, J. (2003): The Effect of Activity-Based

Financing on Hospital Efficiency: A Panel Data Analysis of DEA Efficiency Scores

1992-2000. Health Care Management Science, Vol. 6(4), pp. 271-283.

Borden, J.P. (1988): An Assessment of the Impact of Diagnosis-Relatd-Groups (DRG)-Based

Reimbursement on the Technical Efficiency of New Jersey Hospitals Using Data En-

velopment Analysis. Journal of Accounting and Public Policy, Vol. 7(2), pp. 77-96.

Burgess, J.F. and Wilson, P.W. (1995): Decomposing Hospital Productivity Changes, 1985 -

1988: A Nonparametric Malmquist Approach. Journal of Productivity Analysis, Vol.

6(4), pp. 343-363.

Chattopadhyay, S. and Ray, C.S. (1996): Technical, scale and size efficiency in nursing home

care: a non-parametric analysis of Connecticut homes. Health Economics, Vol. 5(4),

pp. 363-373.

Charnes, A; Cooper, W.W. and Rhodes, E. (1978): Measuring the efficiency of decision

making units. European Journal of Operational Research, Vol. 2(6), pp. 429-444.

Cochrane, A.L.; Leger, A.S.St. and Moore, F. (1978): Health service "input" and mortality

"output" in developed countries. Journal of Epidemiology and Community Health,

32(3), pp.200-205.

Cooper, W.W.; Thomson, R.G. and Thrall, R.M. (1996): Introduction: Extensions and new

developments in DEA. Annals of Operations Research, Vol. 66(1), pp. 3-45.

Dartmouth (1998): The Dartmouth Atlas of Health Care 1998. The Dartmouth Institute for

Health Policy and Clinical Practice, Center for Health Policy Research, Dartmouth

Medical School.

Dartmouth (2008): Tracking the Care of Patients with Severe Chronic Illness. The Dartmouth

Atlas of Health Care 2008. The Dartmouth Institute for Health Policy and Clinical

Practice, Center for Health Policy Research, Dartmouth Medical School.

Donabedian, A.; Wheeler, J.R.C.; Wyszewianski, L. (1982): Quality, cost, and health: an

integrative model. Medical Care, Vol. 20, pp. 975-992.

20

Drummond, M.F.; Sculpher, M.J.; Torrance, G.W.; O'Brien, B.J. and Stoddart, G.L. (2005):

Methods for the Economic Evaluation of Health Care Programmes. 3rd

Edition,

Oxford University Press.

Evans, D.B.; Tandon, A.; Murray, C.J.L. and Lauer, J.A. (2001): The comparative efficiency

of national health systems in producing health: an analysis of 191 countries. Global

Programme on Evidence for Health Policy Discussion Paper No. 29, World Health

Organization, Geneva.

Färe, R.; Grosskopf, S.; Lindgren, B. and Poullier, J.-P. (1997): Productivity Growth in

Health-Care Delivery. Medical Care, Vol. 35(4), pp. 354-366.

Farrel, M.J. (1957): The measurement of productive efficiency. Journal of Royal Statistical

Society, Series A, Vol. 120(3), pp. 253-281.

Filmer, D. and Pritchett, L. (1999): The impact of public spending on health: does money

matter? Social Science and Medicine, Vol. 49(10), pp. 1309-1323.

Forsund, F.R. and Sarafoglou, N. (2002): On the Origins of Data Envelopment analysis.

Journal of Productivity Analysis, Vol. 17(1-2), pp. 23-40.

Gold, M.R.; Siegel, J.E.; Russel, L.B.; Weinstein, M. (eds) (1996): Cost-effectiveness in

health and medicine. New York: Oxford University Press, 1996.

Grosskopf, S.; Margaritis, D. and Valdmanis, V. (2001): Comparing Teaching and Non-

teaching Hospitals: A Frontier Approach (Teaching vs. Non-teaching Hospitals).

Health Care Management Science, Vol. 4(2), pp. 83-90.

Grosskopf, S. and Valdmanis, V. (1987): Measuring Hospital Performance. A Non-parametric

Approach. Journal of Health Economics, Vol. 6(2), pp. 89-107.

Gupta, S.; Honjo, K. and Verhoeven, M. (1997): The efficiency of government expenditures:

Experiences from Africa. IMF Working Paper No. WP/97/15, International Monetary

Fund, Washington D.C.

Huang, Y.G. and McLaughlin, C.P. (1989): Relative efficiency in rural primary health care:

an application of data envelopment analysis. Health Services Research, Vol. 24(2), pp.

143-158.

Hollingsworth, B. (2003): Non-Parametric and Parametric Applications Measuring Efficiency

in Health Care. Health Care Management Science, Vol. 6(4), pp. 203-218.

Hollingsworth, B.; Dawson, P.J. and Maniadakis, N. (1999): Efficiency measurement of

health care: a review of non-parametric methods and applications. Health Care

Management Science, Vol. 2(3), pp.161-172.

Kaplan, G.A.; Pamuk, E.R.; Lynch, J.W.; Cohen, R.D. and Balfour, J.L. (1996): Inequality in

income and mortality in the United States: analysis of mortality and potential

pathways. British Medical Journal, Vol. 312(7037), pp. 999-1003.

Kooreman, P. (1994): Nursing home care in The Netherlands: a nonparametric efficiency

analysis. Journal of Health Economics, Vol. 13(3), pp. 301-316.

21

Laporte, A. (2002): A note on the use of a single inequality index in testing the effect of

income distribution on mortality. British Medical Journal, Vol. 55(9), pp. 1561-70.

Löthgren, M. and Tambour, M. (1996): Alternative Approaches to Estimate Returns to Scale

in DEA-Models. Working Paper No. 90, Stockholm School of Economics, Stockholm.

Macintyre, S.; Maciver, S. and Sooman, A. (1993): Area, class and health: should we be

focusing on places or people? Journal of Social Policy, Vol. 22, pp. 213-34.

Mackenbach, J.P. (1991): Health care expenditure and mortality from amenable conditions in

the European Community. Health Policy, Vol. 19, pp. 245-255.

Murray, C.J.L. and Frenk, J. (1999): A WHO framework for health system performance

assessment. Global Programme on Evidence for Health Policy Discussion Paper No. 6,

World Health Organization, Geneva.

Musgrove, P. (1996): Public and private roles in health: theory and financing patterns.

Washington D.C., The World Bank, 1996 (World Bank Discussion Paper No. 339).

Nyman, J.A. and Bricker, D.L. (1989): Profit incentives and technical efficiency in the

production of nursing home care. The Review of Economics and Statistics, Vol. 71(4),

pp. 586-594.

Ozcan, Y.A. and Luke, R.D. (1993): A National Study of the Efficiency of Hospitals in Urban

Markets. Health Services Research, Vol. 27(6), pp. 719-740.

Prior, D. (2006): Efficiency and total quality management in health care organizations: A dy-

namic frontier approach. Annals of Operations Research, Vol. 145(1), pp. 281-299.

Rosko, M.D.; Chilingerian, J.S.; Zinn, J.S. and Aaronson, W.E. (1995): The effects of

ownership, operating environment, and strategic choices on nursing-home efficiency.

Medical Care, Vol. 33(10), pp. 1001-1021.

Seiford, L.M. (1996): Data Envelopment Analysis: The evolution of the state of the art (1978-

1995). Journal of Productivity Analysis, Vol. 7(2-3), pp. 99-137.

Sengupta, J.K. (1989): Nonlinear measures of technical efficiency. Computers & Operations

Research, Vol. 16(1), pp. 55-65.

Spinks, J. and Hollingsworth, B. (2005): Health production and the socioeconomic

determinants of health in OECD countries: the use of efficiency models. Centre for

Health Economics, Working Paper 151.

Szczepura, A.; Davis, A.; Fletcher, C.J. and Boussofiane, A. (1993) Efficiency and

effectiveness in general practice, Journal of Management in Medicine, Vol. 7(5), pp.

36-47.

WHO (2000): The World Health Report 2000. Health Systems: Improving Perforance. World

Health Organization, Geneva, 2000.

WHO (2008): Health statistics and health information systems. Disability adjusted life years

(DALY). [http://www.who.int/healthinfo/boddaly/en/]