1

HEALTH SYSTEMS STRENGTHENING AND HEALTH OUTCOMES:

THEORETICAL FRAMEWORK AND EMPIRICAL STRATEGY

AIM

JUNE 11, 2010

2

TABLE OF CONTENTS

BACKGROUND ............................................................................................................................................................... 3

THEORETICAL FRAMEWORK ......................................................................................................................................... 4

FIGURE 1: CONCEPTUAL FRAMEWORK OF A HEALTH SYSTEM ................................................................................................... 5

AN APPLIED EXAMPLE OF THE THEORETICAL FRAMEWORK: VACCINATION COVERAGE ........................................... 7

EMPIRICAL EVIDENCE SUPPORTING THEORETICAL FRAMEWORK .............................................................................. 9

FIGURE 2: RESULTS FOUND IN THE EMPIRICAL LITERATURE REVIEW: ACCESS ............................................................................ 11 FIGURE 3: RESULTS FOUND IN THE EMPIRICAL LITERATURE REVIEW: COVERAGE ....................................................................... 12

EMPIRICAL MODEL ...................................................................................................................................................... 13

APPENDIX A: SUMMARY AND ANNOTATED BIBLIOGRAPHY..................................................................................... 16

TABLE 1: ANNOTATED BIBLIOGRAPHY OF STUDIES USING MORTALITY RATE AS OUTCOME ......................................................... 18 TABLE 2: ANNOTATED BIBLIOGRAPHY OF STUDIES USING ACCESS INDICATORS AS OUTCOMES .................................................... 32 TABLE 3: ANNOTATED BIBLIOGRAPHY OF STUDIES USING COVERAGE INDICATORS AS OUTCOMES ................................................ 42 TABLE 4: ANNOTATED BIBLIOGRAPHY OF STUDIES USING LIFE EXPECTANCY AS OUTCOME.......................................................... 52 TABLE 5: ANNOTATED BIBLIOGRAPHY OF OTHER STUDIES INCLUDED IN LITERATURE REVIEW ...................................................... 56

APPENDIX B: SOURCES FOR DETERMINANTS OF VACCINATION COVERAGE ............................................................ 72

REFERENCES ................................................................................................................................................................ 76

3

Background

Funding for the health sector in developing countries has grown significantly in the last decade. While health sector investments have increasingly focused on alleviating the impact of HIV/AIDS, tuberculosis, and malaria, both development actors and the governments of developing countries are shifting their priorities toward more holistic, integrated development approaches, linking disease-specific interventions with longer-term investments in health systems strengthening (HSS). With increased HSS funding, there comes a greater need for informed evidence-based decision-making in health systems investments. This includes building consensus on a core set of indicators measured across programs and countries that provide sufficient information to monitor health systems. Such an effort is particularly useful in the context of the Millennium Development Goals, the “Countdown to 2015” initiative, and the growing threat of non-communicable diseases (NCDs), as NCDs account for the major burden of disease in most E&E and LAC countries. Furthermore, particularly in the area of HIV/AIDS, the recent reauthorization of PEPFAR is another example of greater interest in examining and strengthening health systems in the effort towards preventing and treating HIV/AIDS.

This analysis stems from a specific interest by the Bureau of Global Health at USAID to understand the role that health systems indicators play in helping countries achieve targeted levels of health outcomes. For USAID specifically, a gap persists in demonstrating the relationship between their HSS interventions, intermediate outcomes, and final health outcomes. This gap needs to be bridged in order to accurately determine the need and priority areas for investments in health systems, as well as the length of time to provide assistance before the responsibility can be passed onto the host governments.

The USAID HSS Working Group tasked AIM and a technical advisory group with the development of an empirical HSS input/health outcome model based on current literature and available data to examine the impact of health system inputs on outcomes. To ensure gains in investments to strengthen health systems over time, USAID’s interest is to identify where the greatest gaps in a country’s health system exist and when a country will be capable of sustaining a well-functioning health system without technical assistance.

Towards this effort, AIM has developed a comprehensive literature review of Health Systems Strengthening indicators (available at http://www.usaid.gov/our_work/global_health/hs/publications/ impact_hss.pdf). To build on this work, the current analysis includes a literature review of health systems performance indicators and data sources to identify indicators of health system performance and quantifies the relationship between HSS indicators and health outputs. This literature review is primarily based on the framework developed by the World Health Organization, framing the six building blocks that compose a health system1, and the Control Knobs framework2, as well as drawing from other health systems frameworks.

As a first step to the analysis of HSS indicators and improved health, we identify three specific health output or intermediate outcome indicators, including SBA, ANC and vaccination coverage, to be used as outcomes in the empirical model. These three output indicators were addressed in several studies reviewed. We will test the null hypothesis that no statistically significant relationship exists between health systems inputs and health outcomes. Assuming that the null hypothesis can be rejected, we will explore alternative hypotheses and test the strength and magnitude of the correlation between various HSS inputs for each of these output indicators. The results of this analysis will help identify specific positive and negative deviant countries which will be examined in greater depth for the role played by their country and health system characteristics and their influence on health outcomes. These findings will provide the basis for making decisions about the emphasis to be given to each of the building blocks

4

and the level of investment needed to strengthen health systems in countries. Time series data available from the Brandeis’ International Health Database and data compiled by AIM will be analyzed to answer these questions. Future analysis as an extension to this work could determine “tipping points” or graduation thresholds for HSS indicators in USAID funded countries, which will show when countries can assume ownership of programs, without adversely affecting the ability to achieve program objectives and maintain outcomes.

Theoretical Framework The 2000 World Health Report3 notes that health systems are an integral component to improving health, acting as drivers of scientific, technical, and organizational progress. Interventions motivated by health systems have been cited as the most significant reason for decreased mortality since the 1960’s, more important than income increases or the rise in female education.

However, it is often hard to prove the contribution of health systems empirically, as the route from health systems inputs to final health outcomes is through intermediate outcomes, such as increased access to and coverage of health programs1. The process of improving health status is influenced by a number of factors not specific to health such as education or economic development. The factors affecting health operate at multiple levels, on both the supply and demand side. While this makes it difficult to accurately describe the relationship between specific inputs and final outcomes, several frameworks have attempted to illuminate the paths between the two. Our conceptual model relies primarily on two main models, the World Health Organization Building Blocks model and the Control Knobs model, and takes into consideration other existing health systems models45.

Figure 1 visually describes our conceptual model. The six WHO building blocks are used as a backbone for this model, with slight modifications, and several Control Knobs processes are used as pathways for interactions between the blocks. The purpose of the Control Knobs framework is to identify the parts of health systems that are most amenable to change and would yield the greatest health benefits. Concepts from the Control Knobs model help identify the health systems variables that should be targeted for HSS interventions.

A great deal of overlap exists between currently available models in the concepts that make up the health system. This theoretical framework attempts to reflect all of the major influences on health outcomes, though it is important to note that some components do not have currently available data with which they can be measured and evaluated.

The framework is divided into supply and demand side functions, with the supply side encompassing both public and private provisions of health care services. Private services are inclusive of both for-profit and not-for-profit providers, as well as community-based service delivery mechanisms. The role of the community in providing health services also needs to be recognized, due to the significant role this dimension plays in many countries. For example, Nepal has approximately 49,000 female community health volunteers who have played a significant role in the increase in distribution of vitamin A6. The interactions between these three sectors are complex and differ across countries, but in many

1 To maintain continuity with the WHO Building Block framework, we will use the term access. Here, access is meant to cover actual use of services, along the lines of “realized access” as described in Andersen, R. (1995), ‘Revisiting the behavioural model and access to medical care: does it matter?’, Journal of Health and Social Behaviour, 36(1): 1–1. This definition includes utilization measures.

5

developing nations we assume that the public sector will act as the “first mover” and choose its level of investment in the health sector7. Based on the existing demand for health services, the private and community sectors act as complements or substitutes for the public sector, depending on the health system.

Figure 1: Conceptual framework of a health system

Health system inputs have different pathways for influencing outcomes. The building blocks of the health system are tied together by cross-cutting processes, which can influence the quantity, quality, and efficiency of health services, and, ultimately, health outcomes. Among the six building blocks specified by the WHO framework, information primarily acts on health outputs by supporting other building blocks; it does not necessarily have a measurable impact on its own, although a few studies from developing countries support such a direct relationship8. Health financing connects to health

Health status

Demand-i

Financing

Governance

Service delivery

Health workforc

Medicines/

Quality

Access

Coverage

Safety

Efficiency

Equity

DEMAND Defined by socio-

demographics, politics, economy, epidemiology

Interme

Final

Inputs

Information

Organization Innovation

SUPPLY (public/private/community)

6

outputs directly, but also provides the necessary funds for the other building blocks. This building block comprises a wide range of indicators; it includes both payment schemes for providers and payments made by customers, which in turn influence the supply of health workers, number of hours health workers choose to work, and, to a certain degree, the demand for health services based on the out-of-pocket expense. Governance, leadership, and stewardship of the system determine the policies put in place by the state, influencing the functions and priorities within health systems, behavior, choice, and health outcomes, as well as regulation done by trade or other sub-national organizations 1 2.

Service delivery relies on health workforce as well as other building blocks, and may affect measurable individual outcomes both through its effects on the health services available and through interactions between different service delivery indicators. For instance, a change only in the number of hospital beds may not affect outputs, but it may be a significant factor when combined with increased physicians. Another key input into the health system is available medicines and technology, which also have a strong link to service delivery.

At a broad level, the interactions between the different inputs need to be examined in the context of the organization and management of the system as a whole. Public, private, and community-driven elements influence the capacities of and competition within the system9 1011 . Innovation, including new medical technologies as well as new methods of delivering medical services to a population, innovative payment and financing schemes, revised methods of managing supply chains, and other arenas, also influences the system as a whole. In developed nations, diffusion of innovations throughout the population is often managed through the private sector, while the government often fills this role in developing nations11. Though this process has not been explicitly included in past health systems frameworks, its role is instrumental in improving health outcomes.

Consumer demand itself influences health outcomes. The level of demand is influenced by the socio-demographic, political, economic, and epidemiological makeup of a country4. Beyond the supply side components, persuasion and demand generation in the form of social marketing act directly to influence demand for health services2. The control knobs framework defines persuasion as both the direct and indirect influences on individuals’ beliefs, expectations, lifestyle, and preferences for health services, viewing it as the best channel for influencing individual behaviors. These inputs account for the wants and needs of the population served by the health system.

Community comprises two key components which each play an important role in the health system: community facilities and community mobilization. First, decentralization of the health system and increases in service provision through community facilities builds local capacity. Community health workers and local health centers provide primary medical care and other counseling and support to community members. Second, as a component of demand generation, community mobilization can create demand for services at a local level. Community members, groups, and organizations participate in or promote activities to improve their health over time212 ; these activities may be initiated by particular community members or other catalysts, and often play a prominent role in generating demand for specific health services.

The current health concerns in a country can influence both the demand for and supply of health services. It is important to recognize how these supply and demand-side factors can influence one another, shifting the inputs and outcomes of the health system, and how health outcomes can

7

ultimately affect inputs. For example, when the prevalence of malaria increases in typically low-prevalence countries, government and donor financing for malaria programs is reactive and increases, demand for antimalarial drugs increases, and other inputs shift to account for the changing health needs of the community13 . Furthermore, overarching characteristics such as a country’s level of development also play a role in the above mentioned relationships between health inputs and outcomes14.

The intermediate outcomes in this framework define elements of a well-functioning health system which will eventually lead to improved health status. They include coverage and access, both drawn from the WHO building block framework, and describe the success of providing health services, like vaccinations or antenatal care visits, and the patients’ ability to reach health providers when needed. Measurements of the quality and safety of health services are additional intermediate outcomes. Equity within the system can also be considered an intermediate outcome, and includes gender equity and equity in financial contribution, where those with fewer resources pay less for basic health services. Equity can be seen as a cross-cutting theme across a well-functioning health system.

Our final health outcomes are comprehensive measures of health status, such as mortality rates, life expectancy, or DALYS. Note the control knobs framework includes two additional final outcomes, independent of health status, which we do not explicitly include in our model: financial risk protection and consumer satisfaction2. While both are legitimate goals within a health system, our framework includes consumer satisfaction as a component of quality of health services and financial risk protection as part of equity.

Current data constraints allow only the examination of the correlation between a few isolated health systems indicators and a few selected outputs. Given this, we believe the best option for measuring the contribution of health systems is through the relationship between inputs and intermediate health outcomes, such as SBA, ANC, and vaccination coverage.

An applied example of the theoretical framework: Vaccination coverage The theoretical framework provides a general scheme for understanding the inputs, intermediate outcomes, and the final outcomes within a health system. These different aspects may be best understood by considering one of our intermediate outcomes of interest and describing its different determinants, as cited in the literature. Many determinants of vaccination coverage have been well documented. Differences in how factors influence one another and the final health outcome vary across countries, making a single global causal chain for increased vaccination coverage difficult to construct. For this analysis, we will outline the factors influencing vaccination coverage and how they are reflected in our theoretical framework. For our purposes, we examine how health systems inputs and population factors influence vaccination of children, which ultimately results in reductions in child mortality. The Andersen Behavioral Determinants of Health model is commonly used to describe the different influences on health behavior; here, vaccination of children 15 16. The model highlights the environment (health system and external environment), population characteristics, health behaviors (including use of health services), and outcomes, all linked in a causal chain. The Mosley and Chen Proximate Determinants of Health model is also frequently used, and can be applied to vaccination.17 While both models show a relatively linear relationship between inputs, it is important to note that the many factors influencing a given health outcome interact with one another, rather than in a simple causal chain. Appendix A presents an overview of some of the studies in the literature that elaborate on the details of health outcomes. These can be applied to vaccination coverage as a health outcome.

8

On the supply side of our theoretical framework, many health systems factors are relevant. An effective national immunization program requires adequate government and donor financing. 18 19 As donor funds play a very large role in financing vaccination campaigns through global partnerships, like GAVI and the Global Polio Eradication Initiative, personal income and GDP per capita do not always have a significant impact on vaccination coverage. 20 21 While financing plans do not play a prominent role for this outcome of interest, due to strong influences of donor support and investment, they are very relevant to other health outcomes. Strong leadership and governance also underpin a successful vaccination program, minimizing corruption like leakages, bribery, health worker absenteeism, and overpayment. 20 22 Information available to the government and to individuals also influences coverage. At the government level, information about the cost-effectiveness of an immunization program, current disease burden, and international consensus on the use of a vaccine influence policies are key. 23 At the individual level, a mother’s knowledge of the benefits and safety of vaccination influence her decision to have her child vaccinated and can be a more powerful predictor of behavior than formal education level. 19 24 25 A strong health workforce is also necessary to meet the constant need for vaccination services. The density of health workers, including physicians, nurses, and sometimes community health workers, is an indicator of the system’s capacity to distribute vaccine to its constituents. Distribution of health workers is also important, though more difficult to measure. 21 26 27 Note there are many influences on the health workforce, including government policies, financing of workers’ salaries and training programs, capitation schemes and bonuses for health workers, and other factors. The strength of the health workforce influences effective service delivery, as does access to storage facilities, cold chain equipment, size of health facilities in relation to the population served, and access to necessary medical supplies. 28 22 29 On the demand side of the framework, community level factors must be taken into consideration as well, due to the role of cultural perceptions of vaccination. Socio-cultural acceptability of a vaccine, attitudes of local leaders towards vaccination, and community resources, like access to a health facility can impact vaccination coverage. 18 24 28 Community oversight can also minimize corruption at a local level, improving outcomes. 20 Data for these inputs is scarce, making them difficult to measure quantitatively across countries, but they are worthy of note. Finally, population characteristics influence the demand for and ability to access services, and must be controlled for when empirically estimating health systems’ inputs effect on vaccination coverage. These include demographic factors, family characteristics, enabling factors, need, and knowledge and satisfaction with services. 19 Child gender, child age, family size, and, in select studies, income affect vaccination coverage. 19 24 30 31 The education level of the mother and female literacy have also been repeatedly found to be strong predictors of a child’s vaccination status. 19 21 23 24 25 26 29 30 32 The many factors influencing the outcome, vaccination coverage, on both the supply and demand sides, do not operate independently of each other. Their influence also occurs at different points of the causal chain. Many of these factors operate truly independently from the outcome and other independent variables; they are exogenous to the outcome of interest. However, many others interact with each other and are endogenous to the system. We need to take this endogeneity into account in our empirical model; otherwise the parameter estimates will be biased. Endogeneity can be addressed

9

through a series of structural equations, as discussed in the section entitled “An Alternate Empirical Approach”. However, our ability to apply structural equations may be limited by data availability.

Empirical Evidence Supporting Theoretical Framework An extensive literature review was conducted to find existing empirical work on the relationships between health systems inputs and health outcomes. Based on this review, Figures 2 and 3 illustrate the significant relationships found between health systems inputs and outcome indicators. The review covers both intermediate outcomes like skilled birth attendance (SBA), antenatal care (ANC) and vaccination coverage as well as final outcomes such as mortality and life expectancy.

Figure 2 shows the significant correlations reported by the literature between ANC and SBA, two intermediate outcomes that measure access, and health inputs. Vaccination coverage, specifically represented by measles and DPT3 immunization, was also often examined in the literature for its relationship with HSS inputs. Figure 3 shows the relationships that were found to be significant for this intermediate coverage outcome. A more in-depth discussion of these findings can be found in Appendix A. An annotated bibliography by health outcome is presented in Appendix Tables 1-5.

Not represented in the review are studies looking at non-communicable diseases (NCD) outcomes, such as the prevention and treatment of diabetes, heart disease, or obesity. This is in part due to the fact that measures of health systems outcomes related to NCDs are not currently as relevant for developing countries as other health outcomes like SBA or ANC. However, for regional analyses for LAC and the E &E regions, where there is widespread prevalence of such diseases, there is a small body of literature that can be referenced (e.g. results of the WHO MONICA project, European Observatory studies). The WHO’s 2005 book Preventing Chronic Disease: A Vital Investment elaborates on the links between governance, service delivery, and community-based interventions and NCD outcomes. 33 Sources of NCD indicators would be the CDC’s chronic disease indicator database, as well as the European Community Health Indicators Monitoring (ECHIM) database.34 35

According to the literature, health workforce indicators are by far the most consistently significant predictors of better access and coverage of health services. However, there is contradictory evidence about the type of health worker (nurses/midwives, physicians) that is most important to vaccination coverage. Research studies have typically chosen either nurses/midwives or physicians in their analysis, though some use an aggregate measure of health workers. Where both nurses/midwives and physicians were included separately, some study results support the key role played by the availability of nurses/midwives, while others find that physicians are a stronger predictor of better health access and coverage.

The literature review also uncovers some empirical evidence supporting the relationship between public health spending and access outcomes, while there was little evidence of the effect of public spending on coverage outcomes. Most studies that looked at the effect of financing were concerned with its link to mortality rather than to intermediate outcomes.

Governance had the widest range of empirically tested indicators, few of which were found to be significant. The breadth of choices belies the difficulty in finding an effective measure of governance. A handful of studies explore the interaction between finance and governance, and find that corruption does seem to moderate the positive effect of public spending.

The literature review found no studies examining health information’s relationship to intermediate outcomes in the developing world. Studies examining such a relationship appear to be limited to the

10

U.S., and focus on clinical decision support systems36 37. These systems are not yet a prevailing source of health information in the developing world, and while health information systems (HIS) do have a relationship with safety and quality, studies for developed countries were not included in this review. Further studies examining the correlation between information and health outputs are needed in developing countries.38

Community involvement and community mobilization interventions have a positive effect on access and coverage outcomes in some select countries, but the body of literature examining this area of health systems relies primarily on sub-national interventions, and include few if any additional health systems control variables. Private sector involvement had mixed effects on health outcomes.

Several of the studies reviewed measured health system efficiency. These studies did not focus on specific relationships between inputs and outcomes, but rather the level of health that could be produced by a health system given a mix of inputs. This is a similar to the approach we will be using to identify positive and negative deviant countries. However, in this body of literature the authors are concerned mainly with the final health outcome of life expectancy. To examine this, the authors used very long panels of data over multiple decades, to allow for changes in this slow moving outcome. We will be focusing on intermediate outcomes to take advantage of a shorter, richer panel of data.

The majority of the studies in the review tested relationship on cross-sectional data using linear models. Others, particularly those looking at final health outcomes, used panel data over time.

11

Figure 2: Results found in the Empirical Literature Review: Access

Service Delivery

Financing

Quality

(22)

Health Inputs Access Outcomes

Health Workforce

Medicines

Governance

SBA

ANC

Empirical Tested Relationships between Health Inputs and Outputs

Public health spending

% THE that is OOP

Km to a health facility (5,25)

HRH densityPhysician density (54, 32)Nurse/Midwife density

Corruption

Pharmaceutical consumption (40)

Governance Matters index (36)Decentralization (9,25,38,51)Accountability index/V&A (5,21, 36)

CPIA (8, 56, 59) Government effectiveness (36)

Pharm. logistics systems (61)

(22, 25, 31, 33)

(32)

(54, 32)

# outpatient visits (37)

Catastrophic health expenditure(60)

Primary care score (37)

Drug/equip. availability(25)

Information* = No literature found tested direct relationships between Information and outcomes.

Connecting lines represent statistically significant relationships Numbers represent relevant supporting literature- see list of references.Green=positive effect Red=negative effect

12

Figure 3: Results found in the Empirical Literature Review: Coverage

Service Delivery

Information* = No literature found tested direct relationships between Information and outcomes.

Financing

Vaccinations

Quality

Health Inputs Coverage OutcomesConnecting lines represent statistically significant relationships Numbers represent relevant supporting literature- see list of references.Green=positive effect Red=negative effect

Health Workforce

Medicines

Governance

Empirical Tested Relationships between Health Inputs and Outputs

Public health spending (22, 25, 31, 33)

% THE that is OOP

Km to a health facility

HRH densityPhysician densityNurse/Midwife density

Corruption

Pharmaceutical consumption (40)

Governance Matters index (36)Decentralization (9,25,38,51)Accountability index/V&A (5,21, 36)

CPIA (8, 56, 59) Government effectiveness

Pharm. logistics systems

(54, 32)

# outpatient visits (37)

(2)

(5)

(36, 62)

Catastrophic health expenditure(60)

(5, 25)

(61)

Primary care score (37)

Drug/equip. availability(25)

13

Empirical Model Based on our theoretical framework and the results of the literature review, we construct empirical models that control for the effect of available health systems strengthening inputs on intermediate outcomes such as SBA, ANC and vaccination coverage (as measured by DPT3 and measles coverage) at the national level. These models will primarily be used to derive predicted probabilities, which will be used to identify countries that are both positively and negatively deviant in their ability to turn available health inputs into health outcomes. Ideally, all aspects of the health system will be measured in one model. However, given significant data constraints, the role of less quantifiable inputs (e.g. community involvement; the supply of medicines) may not be examined in the empirical models. These aspects of the health system will be better examined in the qualitative deviant analysis in specific countries that will supplement the empirical model.

Data constraints limit the proposed analysis in several ways.

• A clear picture at the national level of the distribution of health services provided by the public, private and community sectors is not available.

• Although there is much evidence of the role played by community health workers, data are not available.

• A strong cross-national measure of medical products, vaccines, and technology is lacking. • There is no consensus on a predictive measure of national information systems. • There is heterogeneity in demand and demand generation activities across the world’s

countries.

These aspects of the health system will therefore play a diminished role in our predictive models.

We present three empirical models, one predicting SBA, one for ANC, and one predicting measles and DPT3 coverage.

(1) SBA = α + β1HWit + β2Hospit + β3(HWit*Hospit) + β4THEpercapi(t-1) + β5Corrit +β6(THEpercap i (t-1) *Corrit) + β7Privit + β11Controlsit + υi + εit

(2) ANC = α + β1HWit + β4THEpercapi (t-1) + β5Corrit +β6(THEpercap i (t-1) *Corrit) + β7Privit + β10Donorit + β11Controlsit + υi + εit

(3) Vaccination = α + β1HWit + β2%Ruralit + β3(HWit*%Ruralit) + β4Donorit+ β5Corrit +β6(Donorit*Corrit) + β7THEpercap i (t-1) + β10Controlsit + υi + εit

Data primarily come from Brandeis’ International Health Database, which has compiled cross-national health systems information from databases such as WHOSIS, World Bank World Development Indicators, UN Millennium Development Goals, US Census’ International Database, and WHO National Health Accounts. We have augmented this with data from the World Bank’s World Governance Indicators. We will use data from 1995-2007.

Sources f or the specific variables used in the model below are mentioned below.

Key Dependent Variables:

Measures of access: SBA and ANC

Measures of coverage: Measles and DPT3 coverage

14

Key Independent Variables:

Hosp- Number of hospital beds per 10,000 population. Ideally, we would use outpatient visits or distance to health facilities as our control for service delivery. Unfortunately, internationally collected measures of service delivery are scarce. As such, we will use hospital beds as a proxy for availability of health facilities in that country. Because coverage outcomes are less tied to facilities, we will only include this measure in our access model.

HW- Health workers density per per 10,000 population. This indicator comes from the WHO Statistical Information System (WHOSIS), and is available for most countries. It combines physicians, nurses, and midwives per 10,000 population. Given the expected interaction between service delivery and health workforce, an interaction term is proposed.

Info – No indicators for information were tested in the literature, and very few are available across countries. We could include the World Bank’s Statistical Capacity index, which is a normalized index available for 143 countries. However, in our conceptual framework, information does not operate along the direct channels with any of our intermediate outcomes, and so we would prefer to leave it out.

THEpercap - Per capita government expenditure on health at average exchange rate (US$). This indicator is available in WHO’s National Health Account (NHA) data for most countries over several years. We may consider using lagged effects, depending on our data.

Donor and Priv – For our coverage model, we expect donor spending to be influential, and therefore it is included and interacted with corruption. For our access model, we would expect private health expenditure to be more influential, at least for SBA, since there are greater private fund outlays for delivery and pregnancy costs. We can also include donor funding for these models as an additional control, particularly for ANC.

For THE per capita as well as donor spending, we propose using a lagged variable to represent the time delay between influx of funding and changes in outcomes, even at the intermediate level.

Corr – Control of Corruption measure from the World Bank’s World Governance Indicators. These data are available for all countries and over multiple years. Based on empirical evidence that corruption can adversely affect use of funding in a country, an interaction term between health expenditure and the corruption is also proposed.

We will explore the possibility of adding a variable representing community mobilization into the model.

Control Variables:

We will include controls for several major covariates at the national level. The control variables most supported by theory and the literature are:

Percent of population that is rural- in lieu of a distance to facility variable, this measure of geographic isolation may be a better proxy for service delivery in the coverage model

Income per capita Maternal education or female literacy Income distribution ( e.g. Gini coefficient ) Total Fertility Rate – for SBA and ANC.

15

Cell phone subscribers per 100 population – this may be used to control for non-health specific level of information technology/development

Fixed vs. random effect models

It is reasonable to expect that the relationship between health systems inputs and health outcomes will vary from country to country in some non-measurable way. To control for these unobservable country level characteristics, country fixed effects can be included in the model. Random effects are an alternative if the country/regional fixed effects still over-identify the model, though the assumptions are more restrictive and computation may be more complicated.

An Alternate Empirical Approach As an alternative to our simplified empirical model, a series of structural equations could be used to estimate each of the intermediate health outcomes: SBA, ANC, and vaccination coverage. This approach would account for the effect of endogenous variables influencing inputs into our original empirical model, and better account for the interactions between the variables in our equation. Continuing our earlier example, we will briefly illustrate this alternate approach through examination of select factors influencing vaccination coverage. This is for illustrative purposes only for discussion by the TAG. Assuming there are no data constraints, if one were to examine all possible determinants of vaccination coverage, some are endogenous factors and determined by a combination of both exogenous and other endogenous factors. These endogenous factors include donor funding, USAID support, access to medical care (ANC, post-natal care, or SBA), and health workforce. For each, there are numerous predictors that could be used to calculate a predicted value which will be entered into the final empirical model. For example, donor funding may be affected by the current level of vaccination coverage, which illustrates need for the funding of additional programs, and poverty level, measured through households living on less than two dollars per day. At the government level, total health expenditure per capita, oil, and presence of a civil war in the country may also impact decisions about donor investment. After calculating the predicted value for donor funding, it would need to be used for some of the other structural equations, as other endogenous variables (e.g. USAID support, health workforce) are influenced by levels of donor funding. If this alternate approach is viable based on available data, a complete set of structural equations will need to be constructed and identified. A potential set of structural equations that is based on the studies and frameworks reviewed will be sent to the TAG separately. The purpose of reviewing a set of structural equations will be to determine whether this approach will be feasible given the data available. Next Steps Predicted probabilities calculated based from the above models will highlight positive and negative deviant countries for each of the health outcomes. The health systems and other conditions responsible for better/poorer health outcomes in these countries will be further analyzed in a qualitative analysis.

16

Appendix A: Summary and Annotated Bibliography The following literature review covers studies that use cross-country analysis to examine the effect of health system inputs on specific health outcomes. Outcomes include mortality rates (MMR, U5MR, IMR, NNMR), skilled birth attendance, vaccination rates/coverage, life expectancy, and antenatal care. The studies are presented below by main outcomes. For the studies relevant to the development of a model, most of the data used are at the national level. The majority of the data come from the DHS and WHO health indicator database. Other sources include OECD, World Bank, and NHA data, as well as nationally representative datasets for single country studies.

Literature searches were conducted on Google Scholar, Ebsco Host and PubMed, using key search terms “Health Systems”, “analysis”, “impact,” “interaction”, “health outcomes”, “panel data”, “cross-country”, and “cross-national”, “production function,” “indicators,” and “performance assessment,” as well as keywords particular to each building block, and for community involvement and private sector. Specific searches for key authors on health systems were also conducted. Literature was largely selected from the last decade.

Mortality Rates

The majority of the studies reviewed (29 of 87) use mortality rates as one of their health outcomes of interest (See Table 1). All of the six building blocks are represented in the independent variables. In nearly every study in this group, income per capita has a significant inverse relationship with mortality rates. The majority of these studies also find a strong inverse relationship between maternal education/literacy and mortality. Bhalotra (2007), Filmer and Pritchett (1999), Bokhari et al (2008), Zakir and Wunnava (1999), and Alvarez et al (2009), Gupta, Verhoeven and Tiongson (2002) focus on health financing, and all find public health spending to have a negative effect on various measures of mortality (infant mortality, under-5 mortality, and/or maternal mortality). Health aid was also found to have a negative effect on mortality rates (Mishra and Newhouse (2009)) while out of pocket expenditure on health had a positive correlation with IMR (Alvarez et al (2009)).

Six of the studies focus on how governance affects mortality rates. Rajkumar and Swaroop (2008), Gupta, Davoodi, and Tiongson (2000) and Wagstaff and Claeson (2004) note a significant interaction between governance indicators and public spending, suggesting government corruption, management and equity play important roles in efficient use/disbursement of health spending. Robalino, Picazo and Voetberg (2001) find that fiscal decentralization is significantly related to infant mortality rates, though the relationship seems to be non-linear, with poor and rich countries benefiting more than middle income countries. Lewis (2006) does not find any significant relationships between her governance indicators and under-5 mortality, but does note a significant positive relationship with ethno-religious fractionalization.

Mercer et al (2004) is the only HIS study to look at mortality rates. The authors do not conduct any formal analysis, but point out the importance of a central HIS to support government priorities on improving health outcomes. In the realm of service delivery, Macinko, Starfield, and Shi’s (2003) results show that the strength of a country’s primary health care system is significantly related to mortality

17

rates. The only study on private sector service delivery, Limwattananon (2008) find a consistent positive correlation between U5MR and the informal private sector’s treatment share, and a negative correlation with the formal private sector share. Anand and Bärnighausen (2004), Or (2001), and Speybroeck et al (2006) note that physician density per capita is significantly inversely related to mortality. Non-health system risk factors such as alcohol, tobacco, and air pollution are also significant direct predictors of mortality (Or (2001) and Arah et al (2005)). Arah et al (2005) and Cavalini and Ponce de Leon (2008) both observe significant relationships between non-personnel health service coverage measures and mortality.

Regarding community involvement, all studies found reported results or a national or sub-national intervention or health policy change. Perry et al (2006) notes that in Haiti community based care inversely relates to U5MR, in a treatment area where there are fewer physicians and hospital beds per capita than is available for Haiti nationwide, but with many more nurses per capita and community health volunteers. Macinko, Guanais, & de Fatima Marinho de Souza (2006) also note that a national program in Brazil involving community in health care decreased IMR, holding several health systems inputs constant.

18

Table 1: Annotated Bibliography of Studies Using Mortality Rate as Outcome Study Building

Block (s) Health outcome

Objective Data Source Methods Findings

Abraha and Nigatu (2009)

Health workforce, service delivery

mortality rates (MMR, IMR, NNMR, U5MR); Intermediate outcome: SBA

To analyze trends of and develop model for the prediction of health indicators in Ethiopia, through describing indicator trends, identifying determinants of mortality and morbidity indicators, and developing a model for predicting future values of MDG indicators.

National Ethiopia data from annual reports of health and health related indicators (1987-2000)

ARIMA model, Descriptive statistics, trend lines. Morbidity/mortality indicators considered to be outcomes, and all potential determinants fitted into ARIMA model. Best-fit equations of trend lines used to predict future values.

Only MMR had statistically significant changes from 1987-2000 once confidence intervals were taken into consideration (IMR, U5MR, crude death rate, and life expectancy at birth did not). Twelve variables were found to have statistically significant correlation with MMR; three of these were found to be predictor time series variables with significant association with MMR. TRF and physicians per 100,000 are positively associated with MMR; SBA and postnatal care coverage are negatively associated with MMR. Adjusted beta coefficients for determinants of MMR included TFR (723.6), physicians/100,000 pop (40.8), SBA (-16.3, and postnatal care coverage (-11.89). Confidence intervals available. Authors suggestion that not seeing larger changes in MMR may be in part due to the fact that SBA and post-natal care coverage were two of the health coverage variables with the least increment.

Ali et al (2000) Health Workforce, Service Delivery, (Community)

mortality rates (U5MR); Intermediate outcome: acute lower respiratory infection (ALRI)

To evaluate the effects of health care provision on acute lower respiratory infection (ALRI) mortality in very young children in rural Bangladesh.

National Bangladesh surveillance data, ALRI-specific, on mortality for very young children; 1988 - 1993. Matlab GIS database, and Matlab data on workforce.

OLS regression - multivariate and bivariate. Also GIS to conduct spatial analysis. Controls: Allopathic worker/pop ratio (per 1000), Indigenous worker/pop ratio (per 1000), Bari (cluster) pop, Population density around baris (per km2), in-migrants to baris 1988- 93, In-migration around baris 1988-93 (per km2), Cost, distance to the nearest ALRI treatment center

The results show that the ALRI mortality rate is significantly inversely related to the community-based ALRI control intervention. Greater access to allopathic practitioners was inversely related to ALRI mortality rates while access to indigenous practitioners was positively associated with mortality. There were no significant socioeconomic influences on disease mortality in the context of the ALRI intervention.

19

Alvarez et al (2009)

Financing, service delivery

mortality rates (MMR, IMR); Intermediate outcomes: ANC, SBA

To compare MMR in sub-Saharan Africa, and assess the health system, economic, and developmental determinants of MMR across countries.

WHO, UNICEF, World Bank and UNDP for national level indicators, from 1997-2006.

Bivariate analysis, descriptive analysis. Peformed Kolmogorov-Smirnov test, and used log-transformation for non-normal variables. Performed pearson/Spearman correlation analysis to ascertain degree of relation between MMR and variables.

Bivariate analysis demonstrated strong direct correlation between MMR and IMR; inverse correlation between MMR and adult literacy rate, rate of enrolment in primary school, GNI per capita, and per-capita gov expenditure on health; significant direct correlation between out-of-pocket expenditure on health and MMR. Highlighted importance of focusing on both supply-side factors influencing MMR (availability of high-quality services) as well as demand-side factors (educational factors). Because of ecological study design, cause-effect relationship cannot be confirmed and conclusions cannot be inferred to individual countries.

Anand and Bärnighausen (2004)

Workforce mortality rates

Authors examine how health workforce affects maternal mortality rates (MMR), infant mortality rates (IMR), and under-five mortality rates (U5MR).

DHS data- 83 country cross-sectional panel

Log-linear OLS regression. The authors include the measures of HRH used in the 2007 paper. Controls: income, female adult literacy, and absolute income poverty.

All measures of HRH were significantly inversely correlated with mortality rates except nurses. Doctors' effect on MMR, IMR, and U5MR, was -0.386, -0.174, and -0.216, respectively (ie 1% change in MDs= -0.386% change in MMR). The HRH density had a slightly larger effect for all dependent variables. Female literacy was marginally significant for IMR and U5MR , but not for MMR.GNI per cap was significant for all, while poverty was significant for MMR and U5MR.

Arah et al (2005)

Other/multi mortality rates, PYLL

Assess the overall population-level associations between mortality and factors amenable to public health.

OECD data on 18 countries during the period 1970 to 1999

Multi-level analysis with fixed effects. Variables: Addictive and nutritional lifestyle, air quality, public health spending, health care coverage, and immunizations.

Alcohol, tobacco, and fat consumption, and sometimes, air pollution were significantly associated with higher all-cause mortality and premature death. Immunizations, health care coverage, fruit/vegetable and protein consumption, and collective health expenditure had negative effects on mortality and premature death, even after controlling for the elderly, density of practicing physicians, doctor visits and GDP per cap. However, tobacco, air pollution, and fruit/vegetable intake were sensitive to adjustments.

20

Baqui et al. (2008)

Service Delivery, Health workforce, (Community)

mortality rates (NNMR); Intermediate outcomes: ANC visits from trained providers, use of iron and folic-acid supplements, clean cord-cutting instruments, delay in newborn first bath, initiation of breastfeeding within 1 hour of birth, TT immunization

Evaluation of the project for advancing the health of newborns and mothers (Projahnmo), a community-based intervention package through government and non-government organization infrastructures to reduce neonatal mortality

Project data from Bangladesh: baseline, interim, and endline household surveys 2003-2006, along with CHW visit records

Multinomial Logit Regression. T-tests to compare differences in NNMR between intervention and comparison clusters. Adjusted relative risk (RR) of NNMR, constructing 95% CIs for RR with Taylor series approximated variance and log transformed RR. Controls: mother's age and years in education, sex and birth order of index child, wealth index.

Significant inverse relationship between home-care intervention and neonatal mortality. Neonatal mortality was reduced in the home-care intervention by 34% during the last 6 months versus that in the control. No mortality reduction was noted in the community-care intervention. ( In the home-care intervention: female CHWs (one per 4000 population) identified pregnant women, made two ANC home visits to promote birth and newborn-care preparedness, made PNC home visits to assess newborns on the first, third, and seventh days of birth, and referred or treated sick neonates. Community-care intervention: birth and newborn-care preparedness and careseeking from qualified providers were promoted solely through group sessions held by female and male community mobilisers.)

Bhalotra (2007) Financing mortality rates

Bhalotra examines the relationship between state health spending (per cap.) and childhood mortality.

National Indian Family Health Survey-2. 1998-1999

Cross sectional and time series probit regression with fixed effects. Controls: dummies for child gender/birth, mother age at childbirth, maternal and paternal education, ethnicity, religion, logged Gini coefficient (rural/urban), logged sector-specific headcount ratio, ratio of logged state agricultural to nonagricultural income, consumer prices inflation and quadratic of newspaper circulation per cap.

Using the traditional method for measuring this relationship, Bhalotra finds no significant result. However, when the author allows for a lagged effect of state spending, includes fixed effects and only looks at a sample of rural households, the author finds a significant effect of health expenditure on infant mortality. In the long run, a one standard deviation increase in log health expenditure per cap. at a given level of state income is estimated to reduce the risk of mortality by 1%. The author finds that attitudes and information, along with income, influence the effect on infant mortality.

21

Bokhari et al (2008)

Financing mortality rates

Authors examine links between country's per cap. government health expenditures and per cap. income to two health outcomes: under-five mortality and maternal mortality.

WHO and World Bank data - 127 countries from 2000. Additional data from UNESCO, CIA factbook and others.

GMM estimators (robust OLS and 2SLS for instrumental variables. logged right hand vars). Instruments: consumption–investment ratio. Interaction term instrument: military expenditures of the neighboring countries. Controls: GDP per cap., education, paved roads, % pop with access to improved sanitation, CPIA, donor funding per cap., interaction terms (govt health expenditure X donor funding, and govt health expenditure X paved roads).

The elasticity of under-five mortality with respect to government expenditures ranges from -0.25 to -0.42 ( mean = -0.33). For maternal mortality the elasticity ranges from -0.42 to -0.52 (mean = -0.50). For developing countries, results imply that while economic growth is important to health outcomes, government spending on health is just as important a factor. Other significant variables include education and roads, and GDP per cap. The coefficient for the interaction between funding and donor funding is positive and significant for U5M.

Cavalini and Ponce de Leon (2008)

Other/multi mortality rates

Paper examines how socioeconomic and health care variables effect morbidity and mortality indicators in Brazil.

National Brazilian Hospital IS; Census; other national political and health data, 2000.

Multi-level Poisson regressions. Controls: local population, education, income per cap., Theil index, access to sanitation, municipal health funding, vaccine coverage, primary care, hospital and outpatient services, workforce variables. State and Federal: urban/rural, hospital export rate, and political party of representatives.

In the infectious disease mortality regression, income per cap., Theil index, number of liberal mayors, and high-complexity outpatient procedures per cap. were all positively associated with mortality rates. Access to water, sanitation, and basic medical procedures all had an inverse effect on mortality rates, as did hospital export rate and access to high-complexity hospital procedures. The results seem to be contradictory on the role of health care in mortality risk.

22

Filmer and Pritchett (1999)

Financing mortality rates

The authors examine the impact of public spending on both health and non-health factors (economic, educational, cultural) in determining U5MR and IMR.

UNICEF and the World Bank data- cross-national data.

OLS and two-stage least squares (2SLS) models. Controls: national income per cap., inequality of income distribution, extent of female education, level of ethnic fragmentation, and predominant religion (whether the country is more than 90% Muslim).

Limited impact of public spending on health. Estimates find that doubling public spending from 3 to 6 % of GDP would improve mortality by only 9-13 %. E.g., it would cost between $50,000-$100,000 in public spending to avert one additional child death in developing countries. This is a much larger figure than what has been found to be the cost of most essential child health interventions. Several possible explanations: health services are increased by spending; inefficiency of health systems in developing countries and cost-effectiveness of interventions. *Lewis (2006) suggests poor governance may have contributed to this unexpected finding.*

Gakidou and King (2002)

Other/Multi mortality rates

The authors define the concept of total health inequality measure at the country level, and demonstrate how to measure it by the variation in health status across individuals (within a country as a whole or any subgroup within a country).

DHS Data- children under age two in 50 countries

Beta-binomial analysis. Results transformed to expected survival in first two years of life. Controls: mother's age, number of children, level of education, and average birth interval. Other variables are World Health socio-economic and Polity II scores on Regime Change and Political Authority, 1800–1994, Gini, and GDP per cap.

This method adopted by WHO and implemented in surveys around the world; preliminary estimates have appeared in the 2000 WHR. Total health inequality estimates should be routinely reported alongside average levels of health in populations and groups, as they reveal important policy-related information. Countries with similar average child mortality differ in total health inequality. The relationship between health inequality and absolute poverty (< $1 per day) appears to be more linear, with much variation in inequality at each level of poverty. Health inequality seems uncorrelated with income inequality (Gini Coefficient). This approach enables meaningful comparisons of inequality across countries.

Gupta, Davoodi, and Tiongson (2000)

Governance SBA, mortality rates, and immunization coverage

The authors look at the relationship between corruption and several health outcomes.

WDR, WDI, UNESCO + data on 89 countries for 1985 and 1997. PRS/ICRG and TI corruption indices.

Linear-log OLS and 2SLS regressions. Instruments for 2SLS: initial income per cap., share of protestants, and exposure to democracy. Controls: Income per cap., public health spending, female education, dependency ratio, and urbanization.

Corruption indicators are negatively associated with child and infant mortality, the likelihood of an attended birth, immunization coverage and low-birth weight. This effect is reduced once factors such as mother’s education, public health and education spending and urbanization are controlled for but remains significant.

23

Gupta, Verhoeven and Tiongson (2002)

Financing mortality rates

This paper examines whether public expenditures on health and education have an effect on outcomes in these areas.

WDI, GFS and UNESCO databases, the World Bank, and PETS- Cross-sectional data for 50 countries was used.

log-log regressions (both in OLS and 2SLS). Instruments: foreign aid and military spending. Controls: adult illiteracy, income per cap., urbanization, and access to sanitation. Chosen from core indicators proposed by Development Assistance Committee (DAC).

Increased public expenditure on health care is associated with reduced mortality rates for infants and children. The relationship between health care spending and mortality rates is weak. They find some evidence that government health expenditure is negatively correlated with childhood mortality, but this relationship is not robust. Illiteracy and income per cap. were also significant.

Klomp and de Haan (2009)

Governance Life expectancy, mortality rates, disease prevalence, quality of health sector

The authors analyze whether the political system and its stability are related to cross-country differences in health.

WHO, World Bank

structural equation model, Factor analysis of composite outcome variable. Created two new variables ("individual health" and "quality of health care sector"). Controls: income, education, population living in rural areas, total population, fertility rate, trade openness, aid, public/private spending on health, tobacco use, life circumstances (water/san, food), lifestyle (tobacco/alcohol use), climate, share of gov expenditure in GDP, dependency ratio, and share of women in total population.

Results across 171 countries suggest that democracy has a positive relationship with individual health, and regime instability has a negative relationship with the health of individuals. Government instability is also negatively related to individual health via its link with the quality of the health care sector. Democracy is positively related with individual health through its link with income.

24

Lewis (2006) Governance mortality rates and vaccination coverage

The author examines the effectiveness of government and specifically the efficiency of its role in producing health care services. She first reviews health care markets, the role of government and the current definition and measurement of governance.

WDI 2005, World Bank EdStats, Kaufmann, Kraay and Mastruzzi (2005); CID International Data on Educational Attainment; and Stanford University (Alesina et al (2003)

Simple correlations and OLS regressions. Governance indices: government effectiveness, control of corruption, and voice and accountability. Controls: log GDP per cap., average years of schooling for women, primary completion rate, ethnic and religious fractionalization, and road density.

Voice and accountability does not appear to have any correlation to either of the health outcomes in the simple correlations. This governance indicator also was not significant when included in Lewis’ multivariate regression on measles vaccination. Government effectiveness explains more of the differences in measles immunization and has a slightly stronger effect than overall corruption index, which would be expected since immunization is a relatively simple technology and requires basic government capacity to do it well. In a separate regression on child mortality, Lewis uses aggregate governance indicator with the same controls. This indicator was not significant in full regression (income per cap. and ethno-linguistic fractionalization explained most of variation) . This analysis is part of a larger work in which Lewis also outlines policy possibilities and implications for the sector.

Limwattananon (2008)

Service Delivery (Private)

mortality rates (U5MR), MCPR; Delivery care; Diarrhea; ARI;

This study aims to portray the relative importance of the private and public sectors in providing health care to women and children in 25 low income countries.

DHS data on 25 countries, 1996-2006 and World Health Statistics (WHO 2008)

Bivariate analysis There was consistent positive correlation between under-five mortality and the informal private sector’s treatment share for fever and cough and for diarrhea. There was a negative correlation with under-five mortality and the formal private sector’s treatment share for fever and cough and for diarrhea. Baseline illness prevalence (for diarrhea and ARI) was positively correlated with child mortality, and overall treatment (for diarrhea and ARI) coverage was negatively with mortality.

25

Macinko, Starfield, and Shi (2003)

Service Delivery

Mortality rates, disease-specific outcomes

The authors assess the contribution of primary care systems to a variety of health outcomes from 1970-1998.

OECD Health Data - pooled, cross-sectional time-series data on 18 high income countries, 1970-1998 (504 country-year obs)

Fixed effects differencing regression. Controls: macro-level (GDP per cap., total physicians per 1,000, % of elderly) and micro-level (average number of ambulatory care visits, income per cap., alcohol and tobacco consumption)

The strength of a country's primary care system is negatively associated with all-cause mortality, all-cause premature mortality, and premature mortality from asthma and bronchitis, emphysema and pneumonia, cardiovascular disease, and heart disease. This relationship is significant but reduced when the macro-level and micro-level determinants of population health were included. The authors conclude that a strong primary care system and practice characteristics, such as geographic regulation, longitudinality, coordination, and community orientation, are associated with improved population health.

Macinko, Guanais, & de Fatima Marinho de Souza (2006)

Service Delivery, Workforce, (Community)

mortality rates (state IMR)

The authors use publicly available datasets to evaluate the impact of Brazil's Family Health Program (PSF) on infant mortality, while controlling determinants over time.

National Brazilian Ministry of Health website and yearly population surveys by Brazilian Institute of Geography and Statistics, 1990- 2002

Longitudinal analysis using a fixed effects model. Unit of analysis is the state. 1 year time lag. The period 1990-2002 is examined because it includes pre-PSF implementation, early PSF development, and late PSF expansion. Controls: Socioeconomic conditions, women's development indicators, and health service indicators (physicians and nurses per 10,000 pop and hospital beds per 1000)

From 1990 to 2002 IMR declined from 49.7 to 28.9 per 1000 live births, and PSF coverage increased from 0 to 36%. There was a significant inverse relationship established between PSF coverage and IMR, with a 10% increase in PSF coverage associated with a 4.5% decrease in IMR, controlling for all other health determinants. Other main determinants with an inverse relationship with IMR included female literacy (the most significant determinant), hospital bed availability, clean water, and women's literacy. Higher fertility and income per capita had a modest, positive association with IMR.

26

Masanja et al (2008)

Other/multi Mortality rates

Authors analyze the trends in child mortality in Tanzania.

National Tanzania DHS 1990-2004; household budget surveys in 1992 and 2002; 2003 Service provision survey; Public Expenditures Review

Lowess regression, logged dependent variable. Estimated trends in mortality 1990-2004, to forecast trend to 2015. Descriptive statistics then compared to trend. Factors considered: coverage of interventions, policy changes in management, services, and decentralization, governance, financing, resource allocation, and service delivery, fertility, GDP per person, rates of poverty and food security, climate shock, maternal education and literacy, access to improve water sources, and HIV/AIDS.

Analysis showed the reduction in mortality between 1990 and 1999 was 1·4% per year whereas for 2000 to 2005, this trend accelerated to 10·8% per year. There was 24% improvement in child survival between 1994-2005. Authors hypothesize as to the causes: Tanzania more than doubled its public expenditure on health; such increased expenditure has been strongly correlated with increased survival for children under five; decentralization of management may have helped scale up IMCI and increased coverage of health interventions. Gains in GDP per capita were marginal and were not likely a factor. Fertility, birth spacing, and education did not change, and were not factors. Nutrition only improved slightly and has played a small role.

Mercer et al (2004)

Information Mortality rates

The authors describe effectiveness of an NGO primary health care program in rural Bangladesh through the use of information obtained from an MIS.

National Bangladesh DHS, MIS data

Descriptive Statistics. Compared NGO data to DHS national averages on the following variables: ANC, SBA, Tetanus vaccination for pregnant women, PNC, measles vaccination for children, ARI, MCPR, and mortality rates.

Central coordination of HIS design is essential to make sure that information systems are aligned with government priorities and can deliver the information required by managers. Data analysis can stimulate greater interest in information, which can improve the quality of reporting and improve planning and monitoring of health services. Senior decision-makers and political leaders play an important role in creating a culture of information use.

Miller and Frech (2002)

Medicines mortality rates and Life expectancy

Authors examine a relationship between increased pharmaceutical consumption and life expectancy.

OECD and WHO - pooled data on 21 countries from the 1990s

log-log robust OLS regression, with lags. Medicines variables: pharmaceutical and non-pharmaceutical health care expenditures. Controls: GDP and consumption of tobacco and alcohol and fat.

Pharmaceutical consumption has a positive effect on increasing the DALE and life expectancy, although this varies by disease. Increasing per capita pharmaceutical expenditures by 10 % would increase life expectancy at age 40 by 0.3 %, and life expectancy at age 60 by 0.6 %. Non-pharmaceutical medical care consumption does not have a statistically significant effect on life expectancies.

27

Mishra and Newhouse (2009)

Financing, Payment

mortality rates (IMR)

To examine the relationship between health aid and infant mortality.

OECD data. Net Official Development Assistance (ODA) for 118 countries (1973-2004); does not use data from OECD (CRS). Aid recipient countries selected based on criteria for World Bank "developing country" according to 2005 GNI per capita. IMR data from UN (2004).

OLS regression, time-lagged. Dynamic panel data with fixed effects as second empirical model, controlling for country-specific/time-invariant factors; Generalized Method of Moments (GMMs); Control variables included: war dummy, HIV/AIDS rate, lagged log health aid per capita, lagged log infant mortality, lagged log per capita income, lagged log population, lagged log fertility, lagged log female literacy (> 15 years old), log undernutrition (% pop), log physicians (per 1000 pop), log sanitation (% pop with access). Multiple checks for robustness (alternative samples, instruments, explanations, lags in health aid, and additional controls) made to confirm findings.

Health aid has a discernable effect on IMR; on average, doubling of per capita health aid is associated with a 2 percent reduction in IMR. Currently, countries are not on track to meet the MDG for IMR by 2015, and US$5.8 billion would be needed in additional aid to meet the goal. Additional weaker finding that health aid has become more effective in reducing IMR since 1990, and is more effective in countries with higher quality policies and institutions. Over time, health aid allocated to health policy and administrative management has decreased while medical services and basic health care have become more important. There is not evidence of any significant effect of other types of aid (besides health aid) reducing IMR; aid for economic infrastructure has weak (not significant) negative effect on IMR; peace and security aid have weak positive associations with IMR, probably due to deteriorating health outcomes in conflict situations. Both general health aid and basic health aid (only for basic health care services, not administration) have negative effects on IMR.

Or (2001) Workforce mortality rates, disease specific outcomes

The author explores the effect of variations in the volume of health care (using physicians per 1,000) and in certain characteristics of health systems on mortality and other health outcomes.

OECD Health Data, ILO Yearbook of Labor Statistics and national surveys - panel of 21 OECD countries covering the years 1970-1995.

GLS regressions. Controls: MDs per 1000 pop, Share of public expenditure in THE, % of countries where wage/salary, FFS, or cap. is dominant means of payment in primary care, % countries with physicians acting as gatekeepers, % countries with global budget and bed-day as dominant means of payment in secondary care, % countries with FFS as dominant means of payment in hospital care, GDP per cap., Share of white-collar workers in total work force,

The author suggests that over the past 25 years, increasing doctor numbers have been strongly and significantly associated with lower mortality, after allowing for other determinants of health status for which we have data. In addition, the results suggest that the relative importance of the determinants varies with the type of mortality. With respect to the infant mortality, public health expenditure, GDP per cap., and white collar workers also had a significant negative effect, while alcohol, tobacco, and NOx emissions had a significant positive effect.

28

alcohol, tobacco, and NOx emissions per cap.

Perry et al (2006)

Service Delivery, (Community)

mortality rates (U5MR)

The authors compare the under-five mortality in the Hospital Albert Schweitzer (HAS) Primary Health Care Service Area with that for Haiti in general. HAS provides an integrated system of community-based health care services, hospital care, and community development.

National Haiti DHS data, and data obtained from interviews with HAS staff, HAS internal program reports, HAS surveys. 2000

bivariate analysis. Odds ratios with exact Fisher 95% confidence intervals and the likelihood-ratio chi square to calculate statistical significance of odds ratios. No controls, just comparisons of health care resources per 10,000 (various health personnel, community health workers, supervisors, hospital beds), socioeconomic (education, electricity, sanitation, drinking water), malnutrition rates (UW, stunting, wasting), and comparisons and odds ratios of key MCH services (all vaccinations, Vit A, ORT, ARI treatment, EBF, TT immunization, ANC, SBA, MCPR) between HAS area, and Haiti nationwide.

U5 mortality and mortality for children 12-59 months were significantly inversely associated with HAS intervention. Under-5 mortality was 58% less in the HAS service area, and mortality for children 12-59 months was 75% less. The population coverage of targeted child survival services was also positively significantly associated with HAS. These results were achieved with an input of fewer physicians and hospital beds per capita than is available for Haiti nationwide, but with twice as many graduate nurses and auxiliary nurses per capita than are available nationwide, and with three cadres of community health workers that do not exist nationwide: Physician Extenders, Health Agents, and Community Health Volunteers.

29

Rajkumar and Swaroop (2008)

Governance mortality rates

Examines the interaction between government corruption and public health and education spending on development outcomes.

WDI, UNESCO, WIDER, ICRG, others. Data from 91 countries over three years (1990, 1997 and 2003)

OLS regressions. Governance variables: corruption index, quality of bureaucracy, and interaction (governance indicator X public health spending). Controls: % GDP spent on health, GDP per cap., ethno-linguistic fractionalization, the % of the population of Muslims, the % of females aged 15+ that are literate, the % of the population living in urban areas, Gini coefficient, and % of the population under the age of 5.

Two major results: rich countries have lower child mortality, and link between public health spending and child mortality is negative. Differences in the efficacy of public spending can be largely explained by the quality of governance. Public health spending lowers child mortality rates more in countries with good governance. Authors propose that poor governance may help to explain the inconclusive findings of Filmer and Pritchett (1999) on the lack of association between public health expenditures and infant and child mortality.

Robalino, Picazo and Voetberg (2001)

Governance mortality rates

Authors explore the effect of fiscal decentralization on infant mortality rates.

WDI and IMF panel data from low- and high-income countries, covering the period 1970-1995

log-log model with fixed effects. Control: GDP per cap., ethno-linguistic fractionalization, corruption, and political rights.

Greater fiscal decentralization is consistently associated with lower mortality rates. The significance of the square of logged GDP per cap. suggests that fiscal decentralization at the margins benefits poor and rich countries more than middle income countries. Positive effects of fiscal decentralization on infant mortality may be greater in institutional environments that promote political rights. Fiscal decentralization also appears to be a mechanism for improving health outcomes in environments with a high level of corruption.

30

Speybroeck et al (2006)

Workforce Mortality rates, SBA, vaccination coverage

The authors examine the relationship between health outcomes (SBA, measles vaccinations, and mortality rates) and HRH.

WHO data -health systems indicators, supplemented with surveys sent out to the WHO offices, labor and census surveys.

logit–log and arcsine-log models. Several measures of workforce density. Controls: GDP, income poverty, female literacy and land area.

Statistically significant relationship between the aggregate density of health workers and coverage for both measles immunization and SBA. They plot the results and define a minimum threshold for health workers to be 2.28 workers per 1,000, to meet the goal of 80% coverage of SBA. They also find a significant inverse relationship between aggregated health workers and mortality rates. Disaggregated, doctors have a significant positive effect on survival rates, while nurses and midwives have a positive effect on maternal survival.

Wagstaff and Claeson (2004)

Governance mortality rates, malnutrition

Authors explore the impact of increased spending that has resulted from the worldwide effort to reach the MDGs.

WHO, UNICEF, World Bank, CID International Data on Educational Attainment

Log-log OLS regression. Controls: GHE as % of GDP interacted with CPIA, govt health expenditure as % of GDP, income per cap., maternal education, access to safe water, Gini coefficient, ethno-linguistic fractionalization, industrialized country, and regional dummies.

Spending reduces U5MR, but only where governance, as measured by the World Bank’s CPIA score, is high (a CPIA above 3.25). The authors find even higher CPIA thresholds for MMR, TB mortality, and malnutrition, but the interaction term was not significant. The authors conclude that more spending in medium and low CPIA countries would not be expected to reduce mortality rates. More than the other MDG indicators, U5MR was more responsive to income per cap.

Zakir and Wunnava (1999)

Financing mortality rates

The authors test for factors affecting infant mortality rates.

WDR and HDR data- from 1993, cross-section data from 117 low, middle, and high income countries, 6 continents

GLS model. Controls: fertility rates, per cap. GNP, female share of the labor force, female literacy rates, and government expenditure on health care as a fraction of annual GNP.

Significant relationships between infant mortality and fertility rates, GNP per cap., female share of the labor force, and female literacy rates. Quadratic relationship exists between female labor force participation and infant mortality, but the effect of a rise in labor force participation is much greater (4.135) than the subsequent drop off (-0.04604). Finally, while expenditure on health as a percentage of GNP has a negative impact on infant mortality rates, the effect of the expenditure is statistically insignificant.

31

Access: Skilled Birth Attendance (SBA), Antenatal Care (ANC)

Nine studies look at SBA as a health outcome of interest, while four of those also look at ANC (See Table 2). Three studies each examined underweight and health care utilization. The majority of the studies look at questions of health workforce, but financing and governance are also examined. Gupta, Davoodi, and Tiongson (2000) and Speybroeck et al (2006) both find that female literacy/education has a significant positive effect on SBA, while Kruk et al (2009) find this association for ANC but not SBA. Gupta, Davoodi, and Tiongson (2000), Kruk et al (2008), and Kruk et al (2007) report that SBA is significantly positively related to public spending, while at the individual level Hotchkiss et al (2003) saw a negative impact on SBA from increased out-of-pocket health spending. Kruk et al (2008) examine public health spending as it interacts with equity and female education, and find that interaction to be significantly positive. Hutchinson, P, J Akin, F Ssengooba (2006) find this same relationship between ANC and public health spending in Uganda. Kruk et al (2009) and Speybroeck et al (2006) also find significant direct relationships between nurse/midwife and health worker density and SBA. In addition, both papers find a positive relationship between income per capita and SBA.

Gupta, S., H. Davoodi, and E. Tiongson (2000) find a negative relationship with corruption and SBA. Kruk et al (2009) find a negative relationship with land area for both ANC and SBA. Regarding community involvement, Bjorkman and Svensson (2009) and Dubowitz et al (2007) report positive effects of community related system interventions on ANC, underweight and in Uganda and India.

32

Table 2: Annotated Bibliography of Studies Using Access Indicators as Outcomes Study Building Block

(s) Health outcome Objective Data Source Methods Findings

Bjorkman and Svensson (2009)

Workforce, Service Delivery (community)

Underweight, U5MR; vaccination; quantity and quality of health care

The authors explore whether community-based monitoring strengthens the relationship of accountability between health service providers and citizens and hence the quantity and quality of health care. The authors were also interested in whether communities' ability to exercise accountability increased; and whether the intervention resulted in behavioral changes of staff.

National Ugandan QSDS and Household surveys were conducted pre (2004) and 1 yr post-intervention (2006?). Sampling based on facilities.

Difference-in-differences. Controls: Distance from health facility; distance to other government health facilities in area, district fixed effects and facility and household characteristics variables

They hypothesize that efforts of a health worker could complement or substitute the effort of other health workers. Complementarities in workers effort combined with a more engaged and supportive community can therefore result in a virtuous circle, and as service quality improves, community members in turn shift from self-treatment toward facility care, theoretically improving health outcomes. One year into the program, average utilization was 16 percent higher in the treatment communities; the weight of infants higher, and the number of deaths among children under 5 markedly lower. The program also increased measles, polio, dpt3 and BCG vaccinations. This can be seen as a demand generating intervention. Treatment communities became more extensively involved in monitoring the providers following the intervention and the results suggest that the health unit staff responded by exerting a higher effort in serving the community.

Chen et al (2004) Workforce vaccination coverage, SBA

See JLI (2004) WHO data- see JLI et al (2004)

33

Dubowitz et al (2007) Service delivery, workforce, (Community)

underweight, ANC, immunization

The authors assess the effectiveness of the Dular program in Jharkand, India, which is establishing a community-based tracking system of the health status of women and of children 0 to 36 months of age by neighborhood-based local resource persons (LRPs).

Project data from India, surveys of intervention and comparison groups. Included sociodemographic information as well as health outcome data.

logistic regression, analysis of covariance with Bonferroni adjustments. Controls: Various measures of socioeconomic status (caste, work for pay, husband works for pay, religion, education, type of house, type of cooking fuel, site of defecation, electricity in HH.

Significant differences were found between Dular and non-Dular villages in all major outcomes. Dular villages had significant positive association with more women practicing prenatal care, delivering breastmilk, delivering colostrum, using iodized salt, iodized salt status (when tested), having up-to-date immunizations, and using delivery kits. Particularly significant was the inverse relationship between Dular areas and prevalence of severe malnutrition, and the positive relationship with Dular villages and receiving colostrum.

Gupta, Davoodi, and Tiongson (2000)

Governance SBA, mortality rates, and immunization coverage

The authors look at the relationship between corruption and several health outcomes.

WDR, WDI, UNESCO + data on 89 countries for 1985 and 1997. PRS/ICRG and TI corruption indices.

Linear-log OLS and 2SLS regressions. Instruments for 2SLS: initial income per cap., share of protestants, and exposure to democracy. Controls: Income per cap., public health spending, female education, dependency ratio, and urbanization.

Corruption indicators are negatively associated with child and infant mortality, the likelihood of an attended birth, immunization coverage and low-birth weight. This effect is reduced once factors such as mother’s education, public health and education spending and urbanization are controlled for but remains significant.

34

Hotchkiss et al (2003) Financing, (Private)

SBA The authors examine the role of household out-of-pocket costs and structural attributes of quality on the use of maternity care in Morocco.

National Morocco DHS with Service Availability Module, 1995

Nested mixed multinomial logit model. Dependent variable categories: public fac delivery, private fac delivery, SBA home delivery , SBA home delivery . Facility level controls: User fees, Travel time, availability of health care workers, availability of medical equipment/infrastructure; availability of drugs; availability of services other than obstetric care. Individual and household level controls: Age, Education, marital status, birth history, household size, number of adult females in household, and wealth.

Quality of care measures were not substantially better in private facilities than public facilities. Delivery in a health facility was significantly positively associated some structural attributes of quality, with urban, wealthier women, another adult woman in the house, and women giving birth for the first time or a c-section during previous birth. Simulations show that increases in out-of-pocket costs for public facilities would be expected to have very little impact on women living in better-off households, but would have a substantial detrimental effect on poor. Among poor, the net effect of any strategy that involved increases of more than 10 percent in out-of-pocket costs would have a detrimental influence on utilization rates, regardless of quality improvements. Use of private facilities barely shifted when price of public facilities changed. The main substitution affect was seen in home births, with home births increasing as the price of public facilities increased. Most use of private facilities occurred among top 50 % of wealth spectrum.

35

Hutchinson, Akin, Ssengooba (2006)

Governance vaccination coverage, ANC, primary HC outcomes

The authors examine the impacts of a public sector decentralization program on health care-seeking behaviors in Uganda in the 1990s.

National Uganda Household Surveys (1995 and 1997), Annual district health workplans (World Bank).

Probit regressions. Decentralization variables: total district health budgeted expenditure per cap., the total district public and private goods budgeted expenditure per cap., and share of district’s budget to public and private goods and services. Controls: maternal education, rural/urban, sex, facility capacity, distance of the health facility, and government ownership of facility.

The authors' analysis indicates that decentralization is positively associated with greater use of curative care services but shows no significant association with basic primary health care activities, such as childhood vaccination, access to safe water, or access to adequate sanitation. A longer period of analysis is required to determine the persistence of these effects.

Joint Learning Institute (2004)

Workforce vaccination coverage, SBA

This is a wide-ranging document covering many aspects of the global workforce. This paper was cited by the WHR 2006 as the source for the often cited threshold of the 2.3 health workers per 1,000 population.

WHO data- see Speybroeck et al (2006)

No formal analysis done. The analysis that led to that threshold is actually from Speybroeck et al (2006), which is described below. JLI does not disclose its methods in this work.

Two health outcomes are significantly associated with health worker densities: measles immunization and SBA. 1.5 workers per 1,000 are associated with 80 % coverage with measles immunization, and 2.5 workers per 1,000 are associated with 80 % coverage of SBA. This is only suggestive; it does not cover more labor-intensive health outputs, and not all aspects of the health system are controlled for in this analysis.

36

Jutting (2003) Financing, Service Delivery

health care utilization

The author analyzes whether rural Senegal members of a community-based health insurance scheme are better off than non-members, in terms of health care utilization and financial protection.

National Senegal survey data, 2000.

logit/log-linear regression. Control variables include individual level (age, sex, education, illness freq. and illness type); household level (income, religion, ethnicity); village level (cost of seeking health care (including distance from facility) and specific design of mutuelle insurance schemes)

Community-based insurance programs can work, with members having a significant positive association with health care utilization services. Members have a higher probability of using hospitalization services than nonmembers and pay substantially less when care is needed. Young people are significantly inversely associated with health care utilization, and women are positively associated with health care utilization.

Kruk et al (2007) Financing SBA, ANC, Caesarean section

Authors determine the effects of government versus private financing of health services on utilization of antenatal care (ANC), SBA (SBA) and Caesarean section.

DHS and NHA data - from 42 low-income and lower-middle-income countries

OLS regression. Controls: Total per cap. health spending and female literacy

Government health expenditure as a percentage of THE is significantly associated with utilization of SBA and Caesarean section but not ANC. The authors find that THE is also significantly associated with utilization of SBA and Caesarean section.

Kruk et al (2008) Financing SBA The authors examine SBA by wealth quintile in developing countries to assess whether redistributive government policies, in the context of higher levels of health spending, are associated with more equitable use of maternal health care.

DHS data - from 45 developing countries

OLS regressions. Controls: logged national health care expenditures, the wealth distribution of women's fifth grade education (a proxy for redistributive policy environment within gov't) and the overall proportion of women with fifth grade education (a proxy for female literacy and an indicator of gov'ts' commitment to girls' education). Also included was a 3-way interaction variable.

Interaction term is a significant predictor of increased equity in utilization of SBA. This finding supports the hypothesis that higher health expenditures need to be accompanied by redistributive policies to reduce the gap in utilization of SBA between poorer and richer women in developing countries.

37

Kruk et al (2009) Workforce vaccination coverage, ARI, ANC (4+), SBA, caesarean section, and TB case detection under DOTS

The authors explore the relationship between HRH and six health outcomes (Measles immunization, visits of children with ARI, ANC (4+), SBA, caesarean section, and TB case detection under DOTS).

WHO data- cross- national data of low- and middle income countries (those under $11,116 GDP per cap.)

logistic-log regressions. Controls: GDP per cap., female literacy rates, land area, and rural location

Aggregate health worker density is significantly related to only SBA and Measles vaccination. Breaking HRH out by training, nurses and midwives seems to drive this association for SBA, while doctors are not significantly related to SBA. The opposite is true for Measles vaccination, with only doctors showing a significant relationship to that outcome. These results are in opposition to findings of other studies on vaccination rates and HRH, where nurses are more important to vaccination coverage.

Limwattananon (2008) Service Delivery (Private)

mortality rates (U5MR), MCPR; Delivery care; Diarrhea; ARI;

This study aims to portray the relative importance of the private and public sectors in providing health care to women and children in 25 low income countries.

DHS data on 25 countries, 1996-2006 and World Health Statistics (WHO 2008)

Bivariate analysis There was consistent positive correlation between under-five mortality and the informal private sector’s treatment share for fever and cough and for diarrhea. There was a negative correlation with under-five mortality and the formal private sector’s treatment share for fever and cough and for diarrhea. Baseline illness prevalence (for diarrhea and ARI) was positively correlated with child mortality, and overall treatment (for diarrhea and ARI) coverage was negatively with mortality.

38

Liu et al (2007) Financing, service delivery, (private)

general: Access, Quality, Coverage

The authors review the research literature on the effectiveness of contracting-out primary health services and its impact on both program and health systems performance in low and middle income countries.

N/A Literature review. Majority of contracting-out initiatives reviewed have the goal of improving access to health services. Past studies found contractors were consistently more effective in improving access to health care. Access measured by coverage rates, availability of services, and quantity of services provided and utilized (measured through population-based utilization rates). Contracting-out did not harm health systems performance with respect to access of non-focal services.

Matsubayashi et al. (2009)

Service delivery

Immunization (DPT3, measles), TB control (TB detection rate, TB successful treatment rate), SBA

The objective of this study was to identify the levels of sustainable change that low/middle income countries can expect to achieve in health services, as well as how changes in one priority health service are associated with changes in a different type of health service.

WHO, World Bank, DHS, UNICEF, and other UN sources. National level panel data (1990-2004) for all 162 countries considered LMIC in 1990 according to World Bank definitions.

Multilevel longitudinal regression with random effects, and a GEE model for marginal effects. Controls: national population, gross national income (GNI) per capita, official development assistance (ODA) per capita, health expenditure per capita, governance ratings, adult literacy rate, and the number of people affected by disasters

National governance ratings were significantly positively related to Dpt3 coverage and TB case detection. Individual country differences (in delivery of health services, health status, and macro context) may be more important than international averages imply. Rather than showing a common pattern of progress for health services, countries tend to follow their own pathways; large differences in country starting points affect their prospects of achieving targets and their rates of change. Improvements in health services are usually not associated with changes in other health services.

39

Speybroeck et al (2006) Workforce Mortality rates, SBA, vaccination coverage

The authors examine the relationship between health outcomes (SBA, measles vaccinations, and mortality rates) and HRH.

WHO data -health systems indicators, supplemented with surveys sent out to the WHO offices, labor and census surveys.

logit–log and arcsine-log models. Several measures of workforce density. Controls: GDP, income poverty, female literacy and land area.

Statistically significant relationship between the aggregate density of health workers and coverage for both measles immunization and SBA. They plot the results and define a minimum threshold for health workers to be 2.28 workers per 1,000, to meet the goal of 80% coverage of SBA. They also find a significant inverse relationship between aggregated health workers and mortality rates. Disaggregated, doctors have a significant positive effect on survival rates, while nurses and midwives have a positive effect on maternal survival.

Trujillo, Portillo, and Vernon (2005)

Financing, Service Delivery, Governance, (Private)

Medical care utilization: Preventive care visit, hospitalization in last 12 months, outpatient visit in last 30 days.

The author evaluates the impact of Colombia's subsidized health insurance program (SUBS) on medical care use. SUBS is a demand-side subsidy intended for low-income families that increases the participation of the private sector in a way that is unique in the LAC region. It utilizes a decentralized structure to identify beneficiaries.

National Colombian data at the household, community, and regional level from the 1997 Colombia Living Standards Survey.

2SLS regression, with IVs. Propensity score matching was used on data first to deal with the self-selection in non-experimental data. Controls: Individual characteristics, Household characteristics, and local characteristics (municipal per capita health expenditure, community health center, government-supported grassroots organization in community, voter turnout in 1994 Municipal elections, % of state population affiliated to subsidized regime)

SUBS significantly increased medical care use (outpatient care, hospitalization, preventive care) among the country's poor and uninsured, including among children, women and elderly groups. These results are consistent across statistical methods. (In only 3 instances there is no significant difference between treatment and control: outpatient care among 0-4; hospitalization among 5-15; and preventive care 60-98). In assessing impact differences between rural and urban areas, the relationship again is positive and statistically significant, with those in urban areas obtaining greater gains in all types of medical uses.

40

Wagstaff and Claeson (2004)

Governance mortality rates, malnutrition

Authors explore the impact of increased spending that has resulted from the worldwide effort to reach the MDGs.

WHO, UNICEF, World Bank, CID International Data on Educational Attainment

Log-log OLS regression. Controls: GHE as % of GDP interacted with CPIA, govt health expenditure as % of GDP, income per cap., maternal education, access to safe water, Gini coefficient, ethno-linguistic fractionalization, industrialized country, and regional dummies.

Spending reduces U5MR, but only where governance, as measured by the World Bank’s CPIA score, is high (a CPIA above 3.25). The authors find even higher CPIA thresholds for MMR, TB mortality, and malnutrition, but the interaction term was not significant. The authors conclude that more spending in medium and low CPIA countries would not be expected to reduce mortality rates. More than the other MDG indicators, U5MR was more responsive to income per cap.

41

Coverage: Vaccination Coverage, Modern Contraceptive Prevalence Rate (MCPR)

The 13 studies that use vaccination coverage as their variable of interest fall within just three building blocks: health workforce, service delivery, and governance. These studies, along with three studies on MCPR and two studies on sector specific coverage outcomes can be found in Table 3. Nearly all of these studies find a significant positive relationship between female literacy/education and vaccination coverage. Kruk et al (2009) and Speybroeck et al (2006) find that physician density per capita and income per capita have a significant positive effect on vaccination coverage, while nurse/midwife densities have no effect. In contrast, Anand and Bärnighausen (2007) report that nurse/midwife densities are significant while physician density is not. Using panel data from Turkey, Mitchell et al (2008) find a positive relationship between health worker densities in the early years of the data, but a negative relationship in the later years. Bjorkman and Svensson (2009) hypothesize that complementarities between health worker effort and a more engaged and supportive community will improve service quality and shift from patients from self-treatment toward facility care, improving health outcomes. They report that in Uganda, a community mobilization program did indeed increase vaccination coverage one year into the intervention.

Azfar and Gurgur (2008) note an incongruous finding: The wealth index in the Philippines has a negative effect on vaccination. This same study finds negative effects on vaccination for corruption, urbanization rates, inequality, and distance to health facilities, as well as for interactions of the latter two with corruption. Related to corruption, Lewis (2006) finds a direct relationship between government effectiveness and vaccination coverage. The author also finds a negative effect of ethno-religious fractionalization on coverage.

Karim (2005) focuses on the effect of contraceptive logistics systems performance on MCPR, and finds that it does indeed have a significant positive effect on MCPR. He also finds foreign assistance to be a significant predictor. Douthwaite and Ward (2005) note a significant positive association between community health workers and MCPR, though they do not control for many health system variables. According to bivariate tabulations, the formal private sector plays a major role in provision of family planning in countries with relatively poor governance in terms of government effectiveness (Limwattananon (2008)).

42

Table 3: Annotated Bibliography of Studies Using Coverage Indicators as Outcomes Study Building

Block (s) Health outcome

Objective Data Source Methods Findings

Amazigo et al (2007) Workforce, Governance

dewormer coverage

The authors evaluate the sustainability of APOC community-directed treatment projects.

Project data from Cameroon, Nigeria, Tanzania and Uganda, 1998-2000. Interviews with relevant staff and community members, and data checks of financial records and treatment registers. Evaluation instruments developed to measure each indicator.

Bivariate analysis. Quantitative and qualitative assessments of projects (41 projects, based on countries, states/provinces, or groups of districts) to obtain individual community scores on nine aspects of sustainability, as well as an overall sustainability score for projects, on a scale of 0-4. Nine indicators: 5 assessed routine project activities and processes (planning, leadership, supervision, ivermectin supply and distribution, and training); 3 assessed resources available (financing, HR, transport and material resources); and "coverage"- 65% the threshold to achieve control within 15 years

Therapeutic coverage achieved in each project was significantly positively associated with the ratio of volunteer ivermectin distributors per population served. Community level performance is also significantly positively associated with overall project sustainability. Of 41 projects graded, 70% scored "satisfactorily" to "highly sustainable" at the community level. Performance of distributors not affected by direct incentives offered. Community ownership is among the important determining factors of sustainability in community-based programs.

43

Anand and Bärnighausen (2007)

Workforce vaccination coverage

The authors look at the relationship between health workforce density and the coverage of three different vaccines: measles, DPT3 and polio.

DHS data- multi country cross-sectional panel dataset

Logistic-log regression. Three different measures of HRH (From WHO): Doctor density per cap., nurse/midwife density per cap., and agg. measure of health workers, which combines first two measures by pop. The authors leave out CHW and other workers due to lack of data. Controls: GNI per cap, female adult literacy (HDR), and land area.

The aggregate measure is significant. However, when the measure is split, nurses/midwives are significant, but doctors are not. This is consistent with the assumption that doctors are less involved than nurses/midwives in vaccination coverage. For Dpt3, female adult literacy and land area are also significant. Authors posit that income, as well as other related factors it might proxy (nutrition, safe water, sanitation, and housing), do not affect vaccination rates the same way they would affect other health outputs.

Azfar and Gurgur (2008)

Governance vaccination coverage, QOC outcomes

The authors examine the effect of corruption on immunization rates and 5 other health outcomes in the Philippines.

National Philippine household survey conducted in 2000, Ministry of Health data.

MLE random effects model for the multi-level regressions, logit and probit models for others. Controls: health care resources provided by local government, public resources capacity index , # of medical personnel, maternal education, household wealth, and rural/urban location, wealth index, accountability index, external accountability mechanisms, operational autonomy of health centers and municipalities in decision making, social cap.l, political institutions , whether private health

With regard to corruption, the authors find it significantly reduces immunization rates, delays the vaccination of newborns, discourages the use of public health clinics, reduces satisfaction of households with public health services, and increases waiting time at health clinics. Corruption also affects public services in rural areas in different ways than urban areas, and corruption harms the poor more than the wealthy. Urbanization rates, unequal distribution of wealth, distance to health centers (as reported by public health facilities), and surprisingly, the wealth index, also negatively influence immunization coverage. Authors note an interaction between corruption and wealth inequality, and corruption and distance to a private health facility, for immunization rates.

44

care exists in municipality, and distance (in km) from household to the closest alternative (public or private) health facility.

Bjorkman and Svensson (2009)

Workforce, Service Delivery (community)

Underweight, U5MR; vaccination; quantity and quality of health care

The authors explore whether community-based monitoring strengthens the relationship of accountability between health service providers and citizens and hence the quantity and quality of health care. The authors were also interested in whether communities' ability to exercise accountability increased; and whether the intervention resulted in behavioral changes of staff.

National Ugandan QSDS and Household surveys were conducted pre (2004) and 1 yr post-intervention (2006?). Sampling based on facilities.

Difference-in-differences. Controls: Distance from health facility; distance to other government health facilities in area, district fixed effects and facility and household characteristics variables

They hypothesize that efforts of a health worker could complement or substitute the effort of other health workers. Complementarities in workers effort combined with a more engaged and supportive community can therefore result in a virtuous circle, and as service quality improves, community members in turn shift from self-treatment toward facility care, theoretically improving health outcomes. One year into the program, average utilization was 16 percent higher in the treatment communities; the weight of infants higher, and the number of deaths among children under 5 markedly lower. The program also increased measles, polio, dpt3 and BCG vaccinations. This can be seen as a demand generating intervention. Treatment communities became more extensively involved in monitoring the providers following the intervention and the results suggest that the health unit staff responded by exerting a higher effort in serving the community.

Chen et al (2004) Workforce vaccination coverage, SBA

See JLI (2004) WHO data- see JLI et al (2004)

45

Douthwaite and Ward (2005)

Workforce, Service Delivery, (Community)

MCPR The authors assess the impact of the Lady Health Worker Programme (LHWP) on the uptake of modern contraceptive methods. , initiated under the Ministry of Health in the early 1990s, aimed at integrating family planning into the doorstep provision of primary health care, especially in rural areas. This paper presents findings from the first national evaluation of this program.

National Pakistan data from evaluation of the LHWP, completed in 2002

Logistic regression with random effects. Controls: Community: LHW program or not, distance to nearest shop, province. Individual/HH: education, # living children, ratio of boys to girls, wants more children, watches TV, type of HH flooring.

There is a significant positive association between LHWP and use of modern reversible methods, with women served by Lady Health Workers are significantly more likely to use a modern reversible method than women in communities not served by the program. The data provide strong evidence that the LHWP has succeeded in increasing modern contraceptive use among rural women.

Dubowitz et al (2007) Service delivery, workforce, (Community)

underweight, ANC, immunization

The authors assess the effectiveness of the Dular program in Jharkand, India, which is establishing a community-based tracking system of the health status of women and of children 0 to 36 months of age by neighborhood-based local resource persons (LRPs).

Project data from India, surveys of intervention and comparison groups. Included sociodemographic information as well as health outcome data.

logistic regression, analysis of covariance with Bonferroni adjustments. Controls: Various measures of socioeconomic status (caste, work for pay, husband works for pay, religion, education, type of house, type of cooking fuel, site of defecation, electricity in HH.

Significant differences were found between Dular and non-Dular villages in all major outcomes. Dular villages had significant positive association with more women practicing prenatal care, delivering breastmilk, delivering colostrum, using iodized salt, iodized salt status (when tested), having up-to-date immunizations, and using delivery kits. Particularly significant was the inverse relationship between Dular areas and prevalence of severe malnutrition, and the positive relationship with Dular villages and receiving colostrum.

46

Gupta, Davoodi, and Tiongson (2000)

Governance SBA, mortality rates, and immunization coverage

The authors look at the relationship between corruption and several health outcomes.

WDR, WDI, UNESCO + data on 89 countries for 1985 and 1997. PRS/ICRG and TI corruption indices.

Linear-log OLS and 2SLS regressions. Instruments for 2SLS: initial income per cap., share of protestants, and exposure to democracy. Controls: Income per cap., public health spending, female education, dependency ratio, and urbanization.

Corruption indicators are negatively associated with child and infant mortality, the likelihood of an attended birth, immunization coverage and low-birth weight. This effect is reduced once factors such as mother’s education, public health and education spending and urbanization are controlled for but remains significant.

Hutchinson, Akin, Ssengooba (2006)

Governance vaccination coverage, ANC, primary HC outcomes

The authors examine the impacts of a public sector decentralization program on health care-seeking behaviors in Uganda in the 1990s.

National Uganda Household Surveys (1995 and 1997), Annual district health workplans (World Bank).

Probit regressions. Decentralization variables: total district health budgeted expenditure per cap., the total district public and private goods budgeted expenditure per cap., and share of district’s budget to public and private goods and services. Controls: maternal education, rural/urban, sex, facility capacity, distance of the health facility, and government ownership of facility.

The authors' analysis indicates that decentralization is positively associated with greater use of curative care services but shows no significant association with basic primary health care activities, such as childhood vaccination, access to safe water, or access to adequate sanitation. A longer period of analysis is required to determine the persistence of these effects.

47

Joint Learning Institute (2004)

Workforce vaccination coverage, SBA

This is a wide-ranging document covering many aspects of the global workforce. This paper was cited by the WHR 2006 as the source for the often cited threshold of the 2.3 health workers per 1,000 population.

WHO data- see Speybroeck et al (2006)

No formal analysis done. The analysis that led to that threshold is actually from Speybroeck et al (2006), which is described below. JLI does not disclose its methods in this work.

Two health outcomes are significantly associated with health worker densities: measles immunization and SBA. 1.5 workers per 1,000 are associated with 80 % coverage with measles immunization, and 2.5 workers per 1,000 are associated with 80 % coverage of SBA. This is only suggestive; it does not cover more labor intensive health outputs, and not all aspects of the health system are controlled for in this analysis.

Karim, Ali Mehryar (2005)

Medicines MCPR The author examines the influence of FP logistics systems’ performance on higher contraceptive use.

DHS data- panel data from 17 countries for two time points, 1995 and 1999.

OLS Fixed effects model. Weighted by country pop. Control: Secular Trend, External Assistance, Wanted TFR, Family Planning Effort, Logistics Systems’ Performance Score (LSPS), GDP, Female Education, Female Labor Force

Author finds LSPS and foreign assistance have significant positive effects on MCPR in the adjusted model. On average, for every 10- point increase in the Contraceptive Logistics System Performance Index score, the modern method use rate increases by 1 percentage-point. Other controls do not appear to be significant. The author notes inherent measurement problems with linking program to results, but nonetheless states that investing in the improvement of logistics system is an effective programmatic component to FP interventions.

Kruk et al (2009) Workforce vaccination coverage, ARI, ANC (4+), SBA, caesarean section, and TB case detection under DOTS

The authors explore the relationship between HRH and six health outcomes (Measles immunization, visits of children with ARI, ANC (4+), SBA, caesarean section, and TB case detection under DOTS).

WHO data- cross- national data of low- and middle income countries (those under $11,116 GDP per cap.)

logistic-log regressions. Controls: GDP per cap., female literacy rates, land area, and rural location

Aggregate health worker density is significantly related to only SBA and Measles vaccination. Breaking HRH out by training, nurses and midwives seems to drive this association for SBA, while doctors are not significantly related to SBA. The opposite is true for Measles vaccination, with only doctors showing a significant relationship to that outcome. These results are in opposition to findings of other studies on vaccination rates and HRH, where nurses are more important to vaccination coverage.

48

Lewis (2006) Governance mortality rates and vaccination coverage

The author examines the effectiveness of government and specifically the efficiency of its role in producing health care services. She first reviews health care markets, the role of government and the current definition and measurement of governance.

WDI 2005, World Bank EdStats, Kaufmann, Kraay and Mastruzzi (2005); CID International Data on Educational Attainment; and Stanford University (Alesina et al (2003)

Simple correlations and OLS regressions. Governance indices: government effectiveness, control of corruption, and voice and accountability. Controls: log GDP per cap., average years of schooling for women, primary completion rate, ethnic and religious fractionalization, and road density.

Voice and accountability does not appear to have any correlation to either of the health outcomes in the simple correlations. This governance indicator also was not significant when included in Lewis’ multivariate regression on measles vaccination. Government effectiveness explains more of the differences in measles immunization and has a slightly stronger effect than overall corruption index, which would be expected since immunization is a relatively simple technology and requires basic government capacity to do it well. In a separate regression on child mortality, Lewis uses aggregate governance indicator with the same controls. This indicator was not significant in full regression (income per cap. and ethno-linguistic fractionalization explained most of variation) . This analysis is part of a larger work in which Lewis also outlines policy possibilities and implications for the sector.

Limwattananon (2008)

Service Delivery (Private)

mortality rates (U5MR), MCPR; Delivery care; Diarrhea; ARI;

This study aims to portray the relative importance of the private and public sectors in providing health care to women and children in 25 low income countries.

DHS data on 25 countries, 1996-2006 and World Health Statistics (WHO 2008)

Bivariate analysis There was consistent positive correlation between under-five mortality and the informal private sector’s treatment share for fever and cough and for diarrhea. There was a negative correlation with under-five mortality and the formal private sector’s treatment share for fever and cough and for diarrhea. Baseline illness prevalence (for diarrhea and ARI) was positively correlated with child mortality, and overall treatment (for diarrhea and ARI) coverage was negatively with mortality.

49

Matsubayashi et al. (2009)

Service delivery

Immunization (DPT3, measles), TB control (TB detection rate, TB successful treatment rate), SBA

The objective of this study was to identify the levels of sustainable change that low/middle income countries can expect to achieve in health services, as well as how changes in one priority health service are associated with changes in a different type of health service.

WHO, World Bank, DHS, UNICEF, and other UN sources. National level panel data (1990-2004) for all 162 countries considered LMIC in 1990 according to World Bank definitions.

Multilevel longitudinal regression with random effects, and a GEE model for marginal effects. Controls: national population, gross national income (GNI) per capita, official development assistance (ODA) per capita, health expenditure per capita, governance ratings, adult literacy rate, and the number of people affected by disasters

National governance ratings were significantly positively related to Dpt3 coverage and TB case detection. Individual country differences (in delivery of health services, health status, and macro context) may be more important than international averages imply. Rather than showing a common pattern of progress for health services, countries tend to follow their own pathways; large differences in country starting points affect their prospects of achieving targets and their rates of change. Improvements in health services are usually not associated with changes in other health services.

Mitchell et al (2008) Workforce vaccination coverage

The authors investigate the relationship between HRH and vaccinations in the EPI program in Turkey.

National Turkish data- panel data of 7 years and 80 provinces

Logistic-log regression. Measures of HRH are aggregate of all providers (GPs, nurses, midwives and health officers) by population, and then separate densities by provider type per 10,000 population. Controls: GDP per cap., female adult illiteracy, land area, and time trends.

Unexpected results, with HRH predicting greater EPI coverage in early time periods but predicting less coverage in the later time periods. Female illiteracy is only significant control, an inverse relationship. The authors say measuring HRH by densities is of limited value, and other approaches, such as needs- or demand-based approaches, utilization, target setting, and density benchmarking, may be more useful in predicting health outcomes.

50

Razavi et al (2009) Service delivery, financing

ITN/LLIN coverage, malaria outcomes, funding

The goal of this study was to disentangle the relationship between malaria trends, funding, and interventions.

WHO, WDI data on 94 countries over 7 years (2001-2007). Controls: government effectiveness (Ranson et al country typology) and climate and rainfall.

Random-effect models with panel-data; Hausman test for all models. Modifications to explanatory variables or use of fixed-effect estimates used when random-effect models provide inconsistent estimates. Created three models explaining different steps in feedback loop.

They find total funding does have a significant positive effect on ITN and LLIN coverage, both when lagged and in the same period. Donor funding was significant in the previous periods but not same period. For the final outcome of number of malaria cases, there was a negative relationship between level of funding for malaria and number of cases, as well as between an interaction of coverage factors and high risk countries. Government effectiveness did not have a significant effect. Funding for malaria has increased over past four years, particularly donor funding. Interventions typically funded in response to disease trends, with donor funding being most reactive. High prevalence countries rely mainly on vertical programs, while low case countries leverage existing health system resources (horizontal programs). Divergence of low case countries controlling malaria well and high case countries seeing continued increases. Low prevalence countries able to reduce the number of cases through their health systems. While increase in funding expands the coverage for almost all interventions, the small effect size indicates low efficiency of funds. Once the funding is available the expansion of coverage is partially delayed by as long as one year.

Speybroeck et al (2006)

Workforce Mortality rates, SBA, vaccination coverage

The authors examine the relationship between health outcomes (SBA, measles vaccinations, and mortality rates) and HRH.

WHO data -health systems indicators, supplemented with surveys sent out to the WHO offices, labor and census surveys.

logit–log and arcsine-log models. Several measures of workforce density. Controls: GDP, income poverty, female literacy and land area.

Statistically significant relationship between the aggregate density of health workers and coverage for both measles immunization and SBA. They plot the results and define a minimum threshold for health workers to be 2.28 workers per 1,000, to meet the goal of 80% coverage of SBA. They also find a significant inverse relationship between aggregated health workers and mortality rates. Disaggregated, doctors have a significant positive effect on survival rates, while nurses and midwives have a positive effect on maternal survival.

51

Life Expectancy

five of the seven studies using life expectancy as an outcome are those concerned with describing health system efficiency (See Table 4). These studies do not focus on the specific relationships between inputs and outcomes, but rather the level of health that can be produced by a health system, given a mix of inputs. Systems deemed “most efficient” are those that produced the highest level of health (life expectancy) from the mix of measured inputs (health workers, financing, etc). Evans et al (2001) is one of the first in this category, and come out of the WHO’s work on health system efficiency. (Murray and Evans, eds. (2003) is a compendium of studies that came out of that effort). The study produces a ranking of countries based on their health system efficiency. Tandon et al (2001) build on Evans’ work to construct an efficiency model that included four additional dimensions. This is the technical paper produced for the WHR 2000 ranking. Lauer et al (2004) examine the flexibility of the weights of the ranking. Jamison and Sandbu (2001) critique the WHO method and demonstrate the need to control for other variables. Separate from the WHO’s work on this topic, Greene (2004) does a deeper exposition of different empirical models of efficiency and inputs to efficiency.

One other study uses life expectancy as an outcome. Miller and Frech (2002) find that pharmaceutical consumption increased life expectancy in a sample of OECD countries. After controlling for income and consumption of tobacco, alcohol and fat, increasing per capita pharmaceutical expenditures by 10 % would increase life expectancy at age 40 by 0.3 %, and life expectancy at age 60 by 0.6 %. Klomp and de Haan (2009) use a composite measure of health that includes life expectancy at birth and healthy life expectancy, and find that democracy has a positive relationship with individual health, while regime instability has a negative relationship.

52

Table 4: Annotated Bibliography of Studies Using Life Expectancy as Outcome Study Building

Block (s) Health outcome

Objective Data Source

Methods Findings

Evans et al (2001)

HS Efficiency

life expectancy

The authors make an effort to identify the minimum health status of a country in the absence of a health system in order to rate health systems as a whole on their efficiency at producing health.

WHO data - Minimum health derived from 25 countries before existence of health system, then created using 1997 literacy rates.

Stochastic frontier regression with fixed effects, logged independent variables. (logistic?)-log regression to determine efficiency score. Bootstrapped for mean scores and confidence intervals.

In a 25-country sample, health is mainly correlated with literacy. Efficiency scores compare current population health levels with the maximum possible for observed levels of health expenditure and education in a country. The authors find efficiency is positively related to health expenditure per cap. Performance increase greatly with expenditure up to about $80 per cap. a year, suggesting it is difficult for systems to be efficient at low expenditure.

Greene (2004) HS Efficiency

life expectancy

The author responds to the previous analyses of health system efficiency done by Evans et al (2001) and others.

WHO data- full five-year panel data set

"True" fixed effects stochastic frontier model, with log-linear transformation. Controls: THE per cap, average years of schooling, population density, Gini coefficient, the voice and accountability index, a measure of government effectiveness, a control for tropical location, percentage of health care paid for by government, GDP per cap, and a control for OECD membership.

Previous analyses use stochastic frontier analysis of efficiency in panel data, notably the fixed and random effects models, which fail to distinguish between the between-country heterogeneity and “inefficiency”, or a lower functioning health system. There are considerable differences between OECD and non-OECD countries, though the WHO dataset has in general low variation. A true fixed effects model fits the cross-country heterogeneity well. This approach increases risk of over-identification. After running a preferred model, a parabolic relationship between GDP per cap. and DALEs was found, suggesting income has a diminishing marginal return on health outcomes.

53

Jamison and Sandbu (2001)

HS Efficiency

life expectancy

The authors describe some of the shortcomings of the WHO 2000 algorithm (Evans et al 2001).

WHO data Sensitivity Analysis to re-rank countries. Additional controls: geographical variables.

Favorable performance can result from high levels of health expenditure, high efficiency in use of health expenditures (good health system performance), favorable geography, good governance, or luck. The authors argue three points: First, include additional determinants in order to explain more of the outcome deviation, but by different amounts in different countries. Second, even with controls included, the remaining outcome deviation will result only partly from efficiency variation. Third, multiple arbitrary assumptions define the upper and lower bounds, leaving rankings sensitive to assumptions.

Klomp and de Haan (2009)

Governance Life expectancy, mortality rates, disease prevalence

The authors analyze whether the political system and its stability are related to cross-country differences in health.

WHO, World Bank

Structural equation model, Factor analysis of composite outcome variable. Created two new variables ("individual health" and "quality of health care sector"). Controls: income, education, pop. living in rural areas, total pop., TFR, trade openness, aid, public/private spending on health, tobacco use, life circumstances (water/san, food), lifestyle (tobacco/alcohol use), climate, share of gov exp. in GDP, dependency ratio, and women in total pop.

Results across 171 countries suggest that democracy has a positive relationship with individual health, and regime instability has a negative relationship with the health of individuals. Government instability is also negatively related to individual health via its link with the quality of the health care sector. Democracy is positively related with individual health through its link with income.

Lauer et al (2004)

HS Efficiency

life expectancy

Regarding the WHO efficiency rankings, the authors investigate the feasibility and desirability of using mathematical

WHO data Linear Primal and Dual Programs (weighting scheme). First index is without weight restrictions. The second and third indices have

By global distributional measures, scores and ranks are not found to be very sensitive to changes in weights, although differences can be large for individual countries. Building the flexibility of variable weights into calculation of efficiency index is a useful way to allow countries to represent their particular circumstances.

54

programming techniques that allow weights to vary across countries to reflect their varying circumstances and objectives.

bounds set by Saaty's Analytical Hierarchy Process. The former includes lower bounds on weights, while the latter has both lower and upper bounds.

Miller and Frech (2002)

Medicines mortality rates and Life expectancy

Authors examine a relationship between increased pharmaceutical consumption and life expectancy.

OECD and WHO - pooled data on 21 countries from the 1990s

log-log robust OLS regression, with lags. Medicines variables: pharmaceutical and non-pharmaceutical health care expenditures. Controls: GDP and consumption of tobacco and alcohol and fat.

Pharmaceutical consumption has a positive effect on increasing the DALE and life expectancy, although this varies by disease. Increasing per capita pharmaceutical expenditures by 10 % would increase life expectancy at age 40 by 0.3 %, and life expectancy at age 60 by 0.6 %. Non-pharmaceutical medical care consumption does not have a statistically significant effect on life expectancies.

Tandon et al (2000)

HS Efficiency

life expectancy

The authors expand on Evans et al. (2000) by assessing health systems performance in terms of four other indicators, and measures how well a country achieves all five goals of the health system simultaneously, relative to the maximum it could be expected to achieve.

WHO data Stochastic frontier regression with fixed effects, translog. THE per capita (public and private) in 1997 international dollars, average years of schooling, and yrs of schooling squared.

Ranking reveals more efficiency for health alone compared with overall health. Higher overall goal attainment is associated with increasing health expenditure; however, efficiency on health increases with health expenditure per capita and then declines slightly (possibly diminishing returns).

55

Other Studies

Of the 35 studies reviewed in this category, only nine conduct formal analyses (See Table 5). Subramanian, Peters, and Willis (2006) examine the effect of World Bank assistance on health inputs, such as financing and services, as well as health status. The authors report that project inputs involving equipment, buildings, and software appear associated with improvements in health status. No significant relationship is found between GNI per cap. and CPIA and improvements in health services, financing, or outcomes. However, it is not clear if the results described above are causal or merely associated with better outcomes. Schieber, George and Akiko Maeda (1999) look at the relationship between public health spending and GDP per capita, and find that the income elasticity of public health spending is greater as income rises. The authors find an income elasticity of unity for low income countries, suggesting that for every dollar increase in GDP per capita, public health spending will also rise $1. For high-income countries, this rises to $1.47 per one dollar increase in GDP per capita. Trujillo, Portillo, and Vernon (2005) and Jutting (2003) examine the impact of decentralized, community-based health insurance and find increases in medical care use.

Xu et al’s (2003) work establishes the preconditions for catastrophic health expenditures (CHE). In simple correlations, the authors also find the proportion of households incurring CHE is directly related to the percentage of total health expenditures that is OOP. Mutemwa (2005) takes a qualitative approach to examining what affects the use of HIS. The author finds that people, organizational events, and processes also affect the flow and use of information. Starfield and Shi (2002) find a relationship between stronger primary care and lower costs. Bossert et al (2007) assess 15 different logistics functions and measure the relationship between the degree of choice and indicators of performance for each of the functions, and find decentralization does have different effects for different processes.

Eight of the remaining studies are literature reviews on topics that may inform upcoming analyses. The studies are Alderman and Lavy (1996); Palmer et al (2004); Filmer et al (2000); Lewin et al (2008); Resnick and Birner (2006); Hollingsworth (2003); Atun et al (2009); and Kessler (2004). Vian (2008); Murray and Frenk (2000); Bustreo et al (2003); and Siddiqi et al (2009) present theoretical frameworks on health systems. The remaining studies summarize trends or describe data for health systems indicators or report findings from single-country performance assessments.

56

Table 5: Annotated Bibliography of Other Studies included in Literature Review Study Building Block (s) Health

outcome Objective Data Source Methods Findings

Alderman and Lavy (1996)

Financing general The authors conduct a review of single-country studies that look at the effects of OOP costs on health outcomes.

N/A Literature Review If OOP cost increases because higher quality services become available, health outcomes may increase. This will skew the expected result of higher OOP costs on health incomes. In the reviewed literature, the rise in OOP costs most often produced a drop in demand in low income groups. However, mother’s education had an even greater effect on outcomes.

Amazigo et al (2007)

Health workforce, Governance

dewormer coverage

The authors evaluate the sustainability of APOC community-directed treatment projects.

Project data from Cameroon, Nigeria, Tanzania and Uganda, 1998-2000. Interviews with relevant staff and community members, and data checks of financial records and treatment registers. Evaluation instruments developed to measure each indicator.

Bivariate analysis. Quantitative and qualitative assessments of projects (41 projects, based on countries, states/provinces, or groups fo districts) to obtain individual community scores on nine aspects of sustainability, as well as an overall sustainability score for projects, on a scale of 0-4. Nine indicators: 5 assessed routine project activities and processes (planning, leadership,

Therapeutic coverage achieved in each project was significantly positively associated with the ratio of volunteer ivermectin distributors per population served. Community level performance is also significantly positively associated with overall project sustainability. Of 41 projects graded, 70% scored "satisfactorily" to "highly sustainable" at the community level. Performance of distributors not affected by direct incentives offered. Community ownership is among the important determining factors of sustainability in community-based programs.

57

supervision, ivermectin supply and distribution, and training); 3 assessed resources available (financing, HR, transport and material resources); and "coverage"- 65% the threshold to achieve control within 15 years

Atun et al (2009)

ALL Population, health, and nutrition related indicators

The review evaluates the evidence on the extent and nature of integration in the context of key health system functions; how integration or non-integration in different contexts influenced program success; and how contextual factors affected the extent to which programs were integrated.

N/A Literature review The analysis shows few instances where there is full integration of a health intervention or where an intervention is completely non-integrated. Instead, there exists a highly heterogeneous picture both for the nature and also for the extent of integration. Health systems combine both non-integrated and integrated interventions, but the balance of these interventions varies considerably. There is a wide variety of instances where health interventions are integrated into one or more critical health system functions.

58

Babar et al (2007)

Medicines N/A The authors assess the availability and affordability of medicines in Malaysia.

National Malaysian preliminary surveys (For MSG). LPG collected at facility level. Data for 28 core list medicines and 20 supplementary drugs.

Descriptive Statistics. Medicine prices were compared with international reference prices (IRPs) to obtain a median price ratio. The daily wage of the lowest paid unskilled government worker is used to gauge the affordability.

Mark-ups for generic products are greater than for IBs. Reducing the base price without controlling mark-ups may increase profits for retailers and dispensing doctors without reducing the price paid by end users. In private pharmacies, brand prices are 16 times higher than the IRPs, while generics are 6.6 times higher. In dispensing doctor clinics, the figures are 15 times higher for innovator brands and 7.5 for generics. Dispensing doctors applied high mark-ups of 50%–76% for IBs, and up to 316% for generics.

Bjorkman and Svensson (2010)

Service delivery, (Community)

Health services utilization and summary measures of the extent of community monitoring

Authors test whether social heterogeneity can explain why some communities managed to push for better health service delivery while others did not.

District level Ugandan QSDS Data on utilization. covers 50 PHC providers in nine districts in Uganda, half intervention half control. Uganda National Household Survey 2005 used to construct Gini.

Difference-in-differences. Controls: ethno-linguistic fractionalization, Gini coefficient, district fixed effects and facility and household characteristics variables

The results suggest that income inequality, and particularly ethnic fractionalization, adversely impact collective action for improved service provision.

59

Bossert et al (2007)

Medicines N/A Authors develop a framework and methodology describing decentralization's effect on the performance of logistics systems in Guatemala and Ghana.

Logistics Indicators Assessment Tool (LIAT) and Decision Space Assessment Survey (DSAS) in Ghana and Guatemala. Facility level.

T-tests and Descriptive Statistics. Assessed 15 different logistics functions and measured the relationship between the degree of choice and indicators of performance for each of the functions.

Less choice (i.e., more centralized ) is associated with better performance for two key functions (inventory control and information systems), while more choice (i.e., more decentralized) over planning and budgeting is associated with better performance. Logistics systems can be effectively decentralized for some functions, while others should remain centralized.

Bustreo et al (2003)

Service delivery, (private), (community)

Facility for treatment of ARI and diarrhea

Authors review available evidence on role of private sector in child health in developing countries, and discuss strategies to integrate private actors into initiatives to improve health outcomes.

N/A Theoretical Framework and Literature Review

Private sector plays a significant role in child health in developing countries, and this role ranges from highly beneficial to very harmful. Evidence indicates private and NGO health providers are more frequently visited than public sector providers for childhood illnesses, even in poor households. Of papers using facility for treatment of ARI and for diarrhea the percent of children in the poorest wealth quintile treated outside public sector for most recent episode of ARI ranged from approx. 36% in Namibia to 99% in Chad. A conceptual framework adapted from Claeson et al (2001) illustrates the determinants of child health outcomes, connecting government policies and actions to intermediate factors at the household/community level and health system/related sectors level, which connect to child health outcomes (health, nutritional status, mortality). Intermediate factors are inclusive of building blocks (examples of health finance and service provision), as well as other related factors (household resources, household behaviors, community factors, supply in health-related sectors).

60

Cameron et al (2006)

Medicines (private)

N/A Authors present the findings of an analysis of medicine prices and availability in 45 national and sub national surveys.

WHO/HAI medicines surveys: 45 surveys in 36 countries.

Descriptive Statistics. Results are presented for 15 medicines included in at least 80% of surveys and four individual medicines.

Public and private sector prices for brand and generic medicines are substantially higher than would be expected if purchasing and distribution are efficient and mark-ups are reasonable. The authors recommend policy options to promote generic medicines and alternative financing mechanisms to increase availability, reduce prices, and improve affordability.

Chaulagai et al (2005)

Information N/A The authors describe the installation of a HIS in Malawi, the development of indicators, and a strategy for establishing the system in the country.

N/A Performance assessment.

The HIS installation has provided the health sector with information by facility by month. This has not improved use of information in rationalizing decisions. The design of an information system seems to need to be accompanied by commitment of leadership.

Cibulskis and Hiawalyer (2002)

Other/information N/A Authors describe design and implementation of a national HIS in Papua New Guinea, and how information has been used to support decision-making.

N/A Performance assessment.

The definition of management functions at each level of the health system and the identification of information needs and indicators are crucial steps in revising information systems. Implementation should be done piece-wise. Commitment from senior management is necessary to reinforce a culture of information use. Donor harmonization of indicators and inputs is also important.

61

Dubois and Mckee (2006).

Workforce N/A The authors provide a useful source for discussion on the differences in the categorization of HRH across countries; should cite in discussion of HRH variables.

N/A No formal analysis done

Filmer et al (2000)

Other/multi general The authors review empirical results from other studies on the relationship between primary health care, health policy, and improved health status.

N/A Literature Review Enhancing health outcomes is not only a matter of providing additional funds or increasing access to primary health care services and facilities. Previous work has shown low usage of public health facilities is often correlated with low health worker attendance, poor service, low supplies, etc. The size of the private sector may affect the influence of public health spending; if private providers are dominant, less of an effect will be seen on outcomes when public spending increases. Demand for care by patients will also affect influence of public spending. Health systems must be effective at using those inputs to provide services.

Goodman et al (2007)

Other/medicines N/A The authors assess the challenges of regulating retail drug sellers through a case study of malaria treatment in rural Tanzania.

Regional survey data, structured interviews, and drug census from Tanzania.

Descriptive Statistics. Results of interviews presented.

Eliminating regulatory infringements is unlikely to be feasible, and could be undesirable if access to essential medicines is reduced. Alternatives such as greater use of positive incentives for providers and consumer involvement could be a solution.

62

Haines et al (2007)

Health workforce, (community)

N/A This review focuses on documenting the impact on child survival and cost effectiveness of community health workers (CHWs), and to elucidate factors associated with CHW success and sustainability.

N/A Theoretical Framework and Literature Review

While much variation exists among CHW programming, certain overall lessons on improving CHW effectiveness have been learned. Supervision and support increase the effect and sustainability of CHW programming: training alone for CHWs is not enough. CHWs will also probably perform better with clearly defined roles in the health system and a limited number of specific tasks. A combination of incentives, both monetary and non-financial, also improve performance and reduce CHW attrition. Finally, the active support of communities themselves is essential to the success and use of CHWs, as are policies that support and sustain CHW programs. Important determinants that effect the success of CHW programs are national socioeconomic and political factors (i.e. participatory democracy process), community factors (i.e. mobilization of key populations or persons of a community), health systems factors (especially the relationship of the CHW program with the formal health system), and international factors (i.e. expenditure ceilings, donor policies).

Hanson & Berman (1998)

Service Delivery, workforce, Financing (private)

# of private providers, # of private beds, and public-private composition of provision.

The study assembled data on the number of public and private providers, as well as looked at the determinants of the size and structure of the private health sector.

WHO, MOH and World Bank national level data were compiled on 35 countries

Bivariate analyses. Simple multivariate equations of one independent variable and income on the outcome. Controls used with income: Urban, Urban growth rate, Secondary school enrolment, illiteracy, %pop. >15 yrs., IMR, LE, public HE pc, private HE

Across countries there is wide variation in the level private provision and composition. They find that the supply of private physicians is income elastic, whereas the supply of public physicians was relatively inelastic with respect to income. In fact these results suggest that private providers are a luxury good, and that as income rises a greater % of income is spent on them. this is not the case for public providers. There is some evidence that they are substitutes for each other. The supply of private beds was around unity, but public beds were an inferior good. In the tabulations that control for income, public HE pc, private HE pc, THE pc, and % THE public were all independently found to have a positive effect on total private beds. these did not have an effect on private providers. They do not

63

pc, THE pc, % THE public, %THE private.

find a connection between the public-private mix of financing and mix in provision. The majority of private providers operate in the ambulatory sector, while hospital care seems to be dominated by public.

Hollingsworth (2003)

HS Efficiency N/A The author reviews 188 published papers on frontier efficiency measurement.

N/A Literature Review The techniques used are mainly based on nonparametric data envelopment analysis, but there is increasing use of parametric techniques, such as stochastic frontier analysis. Applications both to hospitals and wider health care areas are reviewed. Results appear to confirm earlier findings that public provision demonstrates less variability than private.

IDRC (2007) Information general Report on the use of wireless communication networks and PDAs (personal digital assistants) to expand the range and quality of the Ugandan HMIS. Technical report to the IDRC.

N/A Performance assessment.

Wireless technology is a viable and cost-effective solution to improve HMIS functioning. Health care workers are able to conduct more outreach to rural areas. Findings of a health impact assessment find that consistent access to professional development materials resulted in improved quality of service to the local population.

Kaufmann, Kraay and Mastruzzi (2007)

Governance N/A The authors' data are presented is an often cited source for other governance papers.

*Data come from 30 different sources, including the World Bank, Afrobarometer, Freedom House and others.

N/A The resulting data set includes information on 212 countries and territories for four time periods: 1996, 1998, 2000, 2002 and 2006. These data are used to construct six measures of perceptions of governance from an unobserved components model in each of the four periods. The governance indicators reported here are an update and expansion of previous research work on indicators initiated in 1998 (Kaufmann, Kraay, and Zoido-Lobaton 1999a, b and 2002). The authors also address various methodological issues, including the

64

interpretation and use of the data given the estimated margins of errors.

Kessler (2004) Medicines N/A The author reviews price controls, the innovation of drugs, drug use, and the quality of life associated with the use of pharmaceuticals.

N/A Literature Review Empirical evidence on the link between regulation and access to, and use of, existing pharmaceuticals suggests that regulation has adverse effects on the cost and quality of medical care and reduces patients’ well-being. Regulation may have a number of consequences that mitigate or outweigh this effect: reduced R&D, delays in the launch of new drugs even after they have already been discovered, and distortion of patients’ and physicians’ choices toward compounds with lower therapeutic value.

Lewin et al (2008)

Service Delivery N/A The authors provide a summary of the evidence from health systems reviews, with a particular focus on evidence relevant to primary health care in such settings.

N/A Literature Review Studies organized into four categories: Governance, Financing, Delivery, and Implementation strategies. The majority of those included relate to the financing. No governance studies deal with governance of primary care systems. Many delivery studies in fact deal with health workforce, but two reviewed work on improving the quality of care through primary- secondary care communication. Though some arrangements and implementation strategies are promising, more rigorous evaluations are needed in order to make a more definitive statement on their worth.

Murray and Frenk (2000)

HS Efficiency N/A The authors propose a framework to advance the understanding of health system performance - developed and improved as part

N/A Theoretical Framework and Literature Review

Authors propose that factors explaining health system performance variation should focus on three categories of performance of the four functional areas: financing, provision, stewardship and resource generation. The three categories of analysis are the strategic design, structural arrangements and implementation management of policies of each area.

65

of the discussions for The 2000 WHR.

Mutemwa (2005)

Other/information N/A The author analyzes the extent to which HMIS information is used in decision making. The author establishes a presence of 5 information forms: written, verbal, observational, experiential and training.

Regional Zambian HMIS and ministry of health documents; key informant interviews from 2 districts.

Multi-level, qualitative comparative case study. Analysis of decisions and decision-making processes with regard to five information forms.

Managers’ decision-making behavior suggests a far more tenuous relationship between the presence of organizational management information systems (such as HMIS) and effective strategic decision making. People, organizational events and processes also affect the flow and use of information.

Nandakumar et al (2004)

Financing N/A The authors summarize the trends found in national health accounts (NHAs) from 26 countries in the MENA, LAC and ESA regions.

NHA data Summary of trends A key finding from the first round of NHA is that, in all 26 countries, expenditures both in per cap. terms and as a percentage of GDP are higher than previous estimates. There is large variation observed across countries (from $18.50 in Tanzania to $709 in South Africa). It is notable that highest and the lowest expenditures come from the same region. A major contribution of the first round of NHA studies is a systematic assessment of donor assistance. According to these assessments, there is a strong negative relationship between donor assistance and GDP per cap. For most countries in this sample, there

66

is a weak and positive relationship between THE as a percentage of GDP and GDP per cap., though this did not hold true for the LAC countries.

Or and Jusot (2007)

Other/multi health care consumption

The objective of this paper is to provide new evidence on the impact of the health care system (its organization and funding) on health-care consumption and social inequalities in health care consumption.

OECD (national) and Eurothine (individual-level) data- 20 countries

Multilevel binominal models and regressions. Utilization variables: visits to general practitioners, visits to medical specialists, visits to dentists, and cholesterol screening in past 12 months. Controls: health care system level: existence of a gatekeeping system; the ratio of public funding to THE, methods of paying ambulatory doctors (fee-for-service, caption or salary). Individual level:

The authors' analysis sheds light on the contribution of health care systems to overall national differences in health care utilization, as well as to cross-country differences in social inequalities in health care utilization.

67

Palmer et al (2004)

Financing general Authors present research since 1995 on the impact of various financing strategies on access to health services or health outcomes in low income countries

N/A Literature Review The limited evidence available suggests user fees deterred utilization. This is in line with studies of user fees in developed countries, such as the RAND insurance experiment. Prepayment or insurance schemes offer potential for improving access but are very limited in scope. Conditional cash payments show promise for improving uptake of interventions but could also create a perverse incentive. The largely African origin of the reports of user fees, and the evidence from Latin America on conditional cash transfers, demonstrate the importance of the context.

Peters et al (2000)

Financing N/A The authors summarize findings from the data the World Bank collected on 48 African countries from 1990-1996.

World Bank Data- on 48 African countries from 1990-1996.

Summary of trends Outdated information but possibly useful as a benchmark.

Resnick and Birner (2006)

Governance N/A Authors review a range of studies that include measures of governance as independent variables and focuses on the dependent variable in at least two of the three dimensions of pro-poor growth: poverty, inequality, and

N/A Literature Review The review of governance includes a discussion of the indicators on governance, including the World Bank and Freedom House measures. Governance indicators that capture the decision-making environment for investment and policy implementation, such as political stability and rule of law, are associated with growth but provide mixed results in regard to poverty reduction. Governance indicators that refer to transparent political systems, such as civil liberties and political freedom, tend to be conducive for poverty reduction, but the evidence is rather mixed. The relationship between these variables and growth remains unclear. The paper discusses the methodological challenges inherent in this literature. There is no discussion of the effects of governance or

68

growth. economic growth on health.

Schieber and Maeda (1999)

Financing N/A The authors analyze trends in health spending, health outcomes, and health delivery system characteristics across six different regions of the world, and across income groupings.

WDI, other World Bank sources.

Simple correlations and descriptive statistics. The authors do calculate income elasticities for countries by income level.

Health spending is directly correlated with GDP per cap. Richer countries spend more and have more health inputs (physicians and inpatient beds). Income elasticities for total per cap. health spending as GDP per cap. changes varies by income of country. In high income countries, health spending responds more to changes in GDP per cap. than in middle income countries, and even more still than in low income countries. The authors find income elasticities of 1.47, 1.19, and 1, respectively, for these three groups. Globally, public health spending (εI=1.21) is more responsive than private health spending (εI=1.02) to changes in GDP per cap.

Siddiqi et al (2009)

Governance general Authors present a framework for assessing health system governance (HSG) at national and sub-national levels.

N/A Theoretical Framework

Four existing frameworks are considered: WHO's domains of stewardship; PAHO's essential public health functions; World Bank’s six basic aspects of governance; and UNDP's principles of good governance. The proposed HSG assessment framework includes the following 10 principles: strategic vision, participation and consensus orientation, rule of law, transparency, responsiveness, equity and inclusiveness, effectiveness and efficiency, accountability, intelligence and information, and ethics.

Starfield and Shi (2002)

Service Delivery health costs The authors analyze the effect of a strong primary care system on overall health costs.

OECD and WHO data - A panel of 13 industrialized country data.

OLS regression and simple correlations. Controls: income inequality, smoking, and other health care characteristics

The authors find a relationship between stronger primary care and lower costs.

69

Subramanian, Peters , and Willis (2006)

Other/multi health services, health financing, and status

The authors report on an analysis of how World Bank assistance at the country level has influenced health services, health financing, and status on peoples’ health.

World Bank implementation completion reports - for all 118 projects completed between fiscal years 2003-2005.

logistic regressions and unadjusted odds ratios. Variables of interest: type of project organization (e.g., disease program, sector wide approach), project inputs (e.g., salaries, TA, equipment, and key project activities. Controls: geographic region, income level, and Country Policy and Institutional Assessment (CPIA), all of which are World Bank indicators.

Few Bank-assisted projects in the health sector evaluate changes in health services (42%), health financing (17%), or health status (33%). Those measuring change demonstrate improvements. Projects measuring improvement in health services delivery are five times more likely to measure improvements in health status than those without improvement. There is significant association between use of a SWAp and health services/health status improvements. Project inputs involving equipment, buildings and software appear associated with improvements in health status. The level of GNI per cap. does not appear related to improvements in health services, financing, or outcomes. The average levels of CPIA do not appear to be statistically associated with improvements in any of the project outcomes. However, it is not clear if results described above are causal or merely associated with better outcomes.

70

Thapa et al (2005)

Service Delivery, Workforce, (Community)

Intermediate outcome: Vitamin A coverage on U5MR

The authors assess the effect of Nepal's vitamin A supplementation program on child mortality at age 12–59 months, which utilized a large cadre of women who served as community-based volunteers and who were integral to the program's rapid expansion.

National Nepal DHS Survey 2001, administrative records

Logistic regression, predicted values (to set Vit A coverage to different levels), odds ratios. 1 year time lag. Controls: Individual child variables (child's age, child's sex, place of residence. Region, wealth index, mother's age at birth, mother's education, exposure to mass media, ; community level variables: incidence of ARI, fever, and diarrhea, and other health interventions: ANC, TT vaccination, SBA, place of delivery, measles, DPT, BCG, and polio vaccinations

Vitamin A is significantly inversely associated with U5 mortality. The estimated effect of 100% community-level vitamin A coverage since the child's birth, relative to no coverage, is to reduce the odds of dying at age 12–59 months by slightly more than half. Vitamin A coverage is correlated at low levels with other health interventions, which is consistent with the nature of Vit A program in Nepal, staffed by female CHWs, not by persons in the medical system. Note that Vit A is actual coverage - community level coverage at time of child's birth.

Vian (2008) Governance general The author provides a framework for how governance affects health and then reviews the benefits and weaknesses of

N/A Theoretical Framework and Literature Review

71

the instruments and indicators used to measure corruption.

Xu et al (2003)

Financing N/A The authors analyze the extent of catastrophic health expenditure (CHE) in 59 countries.

National datasets- pooled household surveys.

log-log OLS models. Proportion of household incurring CHE as the dependent variable. Controls:

Three key preconditions for incurring catastrophic health costs: availability of health services requiring payment, low capacity to pay, and the lack of prepayment or health insurance. CHE is defined as 40% of household income after basic needs are met. In simple correlations, the proportion of households incurring CHE is directly related to percentage of THE that is OOP. Households in transition countries are more at risk of CHE, as well as households in Latin America, though this second finding is possibly due to higher income inequality.

72

Appendix B: Sources for Determinants of Vaccination Coverage

Citation Model Building Blocks Determinants of Vaccination Coverage Acosta-Ramirez N, Duran-Arenas LG, Eslava-Rincon JI, Campuzano-Rincon JC. (2005). Determinants of vaccination after the Colombian health system reform. Revista de Saude Publica. 39:3.

Andersen's Behavioral Determinants of Health

Organization, service delivery

Population characteristics: child age, child gender, family size, age/gender of head of household, income, area of residence, educational level of head of household, regular care source for child, health status perception, attitude towards preventive services, information about immunization, satisfaction with distance/wait time/courtesy/cost; Health system characteristics: Type of insurer, type of provider, distance to provider, waiting time

Anand, S and T Bärnighausen (2007) Health workers and vaccination coverage in developing countries: an econometric analysis. Lancet. 369(9569): 1277-1285.

Varied sources Workforce Nurse density, female literacy, land area, demand for vaccine, trained workforce, distribution of health workers; no association for income/person

Azfar, Omar and Tugrul Gurgur, 2008, Does corruption affect health outcomes in the Philippines? Economics of Governance, 9:3.

Giedon et al (2001) Model of Corruption's Influence on Health Outcomes

Governance Overpayment for supplies, leakages, bribery, absenteeism, dilution of vaccine, public funding, local governance and level of corruption, size of public health facilities, household wealth, health center capacity

Brenzel L, Costales M, Durrheim D, Gordon S, Hyde T, Kumar R, Le Franc E, Levine O, Mantel C, Mounier-jack S, kadama P, de Oliveira L, Steinglass R, and Stenson B. (2010) Impact of new vaccines introduction on immunization and health systems. World Bank.

WHO Building Blocks Model

All building blocks Perceived effectiveness/safety by community, socio-cultural acceptability of vaccine (especially new), social mobilization initiatives, targeting of program, storage of vaccine, capital/training costs for new vaccine, information systems for decision making, cost of medical products (vaccines, delivery mechanisms), cold chain capacity, availability of vaccine storage sites, government financing, donor financing, human resource costs

73

Bustreo F, Harding A, and Axelsson. (2003) Can developing countries achieve adequate improvements in child health outcomes without engaging the private sector?. Bulletin of the World Health Organization. 812:12, p 886-894.

Claeson et al (2001) Determinants of Health Sector Outcomes

Organization, service delivery, governance, demand generation

Contracting, social marketing, training/dissemination of information, health policies, government policies and assets, health finance, health service provision, community factors, household resources, household behaviors, supply in related sectors

Chen, L. T.Evans, S.Anand, J.Boufford, H.Brown, M.Chowdhury, M.Cueto, L.Dare, G.Dussault, G.Elzinga Human resources for health: overcoming the crisis The Lancet, Volume 364, Issue 9449, Pages 1984-1990

Varied sources Workforce Health workforce, health worker density

Gauri V and Khaleghian P. (2002) Immunization in developing countries: Its political and organizational determinants. World Bank and John Hopkins University.

Varied sources Organization, Governance, Financing, demographics, demand generation

Maternal education, socio-economic status, accessibility of immunizations, attitudes of local leaders, household beliefs about immunization, power relationships within the household, fear of immunization, quality of vaccine, incentives for providers, organization of national immunization systems, political will, decentralization, price barriers, information

Hutchinson P, Akin J, Ssengooba F (2006), “The impacts of decentralization on health care seeking behaviors in Uganda,” The International Journal of Health Planning and Management, Volume 21, Issue 3 , Pages239 - 270.

Varied sources Organization Level of private health spending, mother's level of education, rural/urban, gender of child, distance to facility, facility capacity

Kruk et al (2008), Equity of Skilled Birth Attendant Utilization in Developing Countries: Financing and Policy Determinants, American Journal of Public Health, Vol 98, No. 1.

Varied sources Workforce Aggregate doctor/nurse density, country level factors (managerial competence of Ministry of Health and Ministry of Education)

74

Lewis, M (2006). “Governance and Corruption in Public Health Care Systems.” Working Paper Number 78, Center for Global Development.

Varied sources Governance Corruption, government effectiveness, road density, dispersion of population, infrastructure, public/private division of services, absenteeism of health workers, training of doctors/nurses/CHWs, leakage of funds, incentives for health workers (capitation, bonuses), voice/informed citizen action, community oversight; GDP/capita not significant

Mahoney RT and Maynard JE. Introduction of new vaccines into developing countries. Vaccine 17 (1999) 646-652

N/A Governance Dissemination of disease burden data, cost-effectiveness of vaccination, international recommendations for vaccine use, assurance of adequate supply, creation of funding mechanisms to support countries that cannot finance own procurement

Pande RP. (2000) Family composition effects on gender differentials in nutrition and immunization in rural India. World Bank.

Mosley and Chen Proximate Determinants of Health

Service delivery, demand-generation

Gender of child, maternal education, mother's access to media, family structure, women's autonomy, community-level economic development, mother's perception of immunization as beneficial

Sia D, Pournier P, Kobiane JF, and Sondo BK. (2009) Rates of coverage and determinants of complete vaccination of children in rural areas of Burkina Faso (1998-2003). Biomed Central Public Health, 9:416.

N/A Organization Poverty level, continuity of care (prenatal care through institutional delivery)

Speybroeck, N, Y Kinfu, M Dal Poz, D Evans (2006), “Reassessing the Relationship Between Human Resources for Health, Intervention Coverage and Health Outcomes,” Geneva: WHO.

N/A Workforce, information

Density of physicians, female literacy, land area; also found GDP/capita to not be significant

Streatfield K,Singarimbun M and Diamond I. (1990) Maternal education and child immunization. Demography. 23: 3. p.

Mosley and Chen Proximate Determinants of Health

Service delivery, demand-generation, demographics, information

Maternal education, maternal knowledge of vaccine benefits

75

Victora CG, Habicht JP, and Bryce J. (2004) Evidence-based public health: Moving beyond randomized trials. American Journal of Public Health. 94:3, p 400-405.

Own model Service delivery, medicines/technology, information

Health worker training, access to medical supplies, mothers' knowledge of where to access care and why care is important

76

References

1 WHO. (2007) Everybody’s business: strengthening health systems to improve health outcomes: WHO’s framework for action. Geneva: World Health Organization. 2 Hsiao W. (2003) What is a health system? Why should we care? Cambridge, MA: Harvard School of Public Health. 3 WHO 2000. The World Health Report 2000 - Health Systems: Improving Performance. Geneva: World Health Organization. 4 Atun R, Ohiri K, Adeyi O. (2008) Integration of Health Systems and Priority Health, Nutrition, and Population Interventions: A framework for analysis and policy choices. World Bank Health, Nutrition, and Population Discussion Paper.

5 Mills AJ and Ransom MK. (2001) The design of health systems. In Merson H, Black RE, Mills AJ, editors. International Public Health: Diseases, Programs, Systems, and Policies. Gaithersburg: Aspen Publishers.

6 Paige B. (2007) Op-ed: Nation-wide Vitamin A Supplementation. USAID Nepal. 2 November 2007. 7 Hanson, K and Berman, P (1998). Private health care provision in developing countries: a preliminary analysis of levels and composition, Health Policy and Planning. 1998 September; 13(3): 195–211. 8 Chandrasekhar, CP and Ghosh, J (2001). Information and Communication Technologies and Health in Low Income Countries: The Potential and the Constraints. Bulletin of the World Health Organization. Vol. 79:9 9 Haines A, Sanders D, Lehmann U, Rowe AK, Lawn JE, Jan S, Walker DG, and Bhutta Z. (2007). Achieving child survival goals: Potential contribution of community health workers. The Lancet. v. 369, p 2121-31. 10 Nishtar S. (2004) Public-private partnerships in health – a global call to action. Health Research Policy and Systems. v. 2, no. 5. 11 Nandakumar AK, Beswick J, Thomas CP, Wallack SS and Kress D. (2009) Pathways of Health Technology Diffusion: The United States and Low-Income Countries. Health Affairs 28, no. 4 p. 986-995 12 Rosato M, Laverack G, Grabman LH, Tipathy P, Nair N, Mwansambo C, Azad K, Morrison J, Bhutta Z, Perry H, Rifkin S, and Costello A. (2008). Community participation: Lessons for maternal, newborn, and child health. The Lancet, v. 372, p 962-71. 13 Razavi SM, Wallack S, and Gaumer G. (2009) Draft: Achieving a better understanding of the malaria control system. Brandeis University, Waltham, MA. Bill and Melinda Gates Foundation. 14 Mishra P and Newhouse D. (2009) Does health aid matter? Journal of Health Economics. 28, p. 855-872. 15 Anderson R and Newman. (1973) Societal and individual determinants of medical care utilization in the United States. Milbank Memorial Fund Quarterly Journal. 51:95-124.

16 Andersen R (2005). Revisiting the behavioral model and access to medical care: does it matter?. Journal of Health and Social Behavior. Volume 36(1), p. 1-10.

17 Mosley and Chen (1984). An analytical framework for the study of child survival in developing countries. Population Development Review. Volume 10, p. 25-45.

18 Bustreo F, Harding A, and Axelsson. (2003) Can developing countries achieve adequate improvements in child health outcomes without engaging the private sector?. Bulletin of the World Health Organization. 812:12, p 886-894.

77

19 Acosta-Ramirez N, Duran-Arenas LG, Eslava-Rincon JI, Campuzano-Rincon JC. (2005). Determinants of vaccination after the Colombian health system reform. Revista de Saude Publica. 39:3.

20 Lewis, M (2006). “Governance and Corruption in Public Health Care Systems.” Working Paper Number 78, Center for Global Development.

21 Speybroeck, N, Y Kinfu, M Dal Poz, D Evans (2006), “Reassessing the Relationship Between Human Resources for Health, Intervention Coverage and Health Outcomes,” Geneva: WHO.

22 Azfar, Omar and Tugrul Gurgur. (2008) Does corruption affect health outcomes in the Philippines? Economics of Governance, 9:3.

23 Mahoney RT and Maynard JE. (1999) Introduction of new vaccines into developing countries. Vaccine 17 646-652.

24 Gauri and Khaleghian 2002

25 Streatfield K,Singarimbun M and Diamond I. (1990) Maternal education and child immunization. Demography. 23: 3. p.

26 Anand, S and T Bärnighausen (2007) Health workers and vaccination coverage in developing countries: an econometric analysis. Lancet. 369(9569): 1277-1285.

27 Kruk et al (2008), Equity of Skilled Birth Attendant Utilization in Developing Countries: Financing and Policy Determinants, American Journal of Public Health, Vol 98, No. 1.

28 Brenzel L, Costales M, Durrheim D, Gordon S, Hyde T, Kumar R, Le Franc E, Levine O, Mantel C, Mounier-jack S, kadama P, de Oliveira L, Steinglass R, and Stenson B. (2010) Impact of new vaccines introduction on immunization and health systems. World Bank.

29 Victora CG, Habicht JP, and Bryce J. (2004) Evidence-based public health: Moving beyond randomized trials. American Jounral of Public Health. 94:3, p 400-405.

30 Pande RP. (2000) Family composition effects on gender differentials in nutrition and immunization in rural India. World Bank.

31 Sia D, Pournier P, Kobiane JF, and Sondo BK. (2009) Rates of coverage and determinants of complete vaccination of children in rural areas of Burkina Faso (1998-2003). Biomed Central Public Health, 9:416.

32 Hutchinson P, Akin J, Ssengooba F (2006), “The impacts of decentralization on health care seeking behaviors in Uganda,” The International Journal of Health Planning and Management, Volume 21, Issue 3 , Pages239 - 270.

33 WHO (2005). Preventing Chronic Disease: A Vital Investment. Geneva: WHO Press.

34 CDC. Chronic Disease Indicator Database. Accessed Sept 3, 2009, http://apps.nccd.cdc.gov/cdi/DefSearchResults.aspx

35 European Community Health Indicators Monitoring (ECHIM) Project. (2008). ECHI shortlist indicators. http://www.healthindicators.org/healthindicators/root/o13.html

36 Garg, A.X ; Adhikari, Neill K J; McDonald, H. et al. (2005). Effects of Computerized Clinical Decision Support Systems on Practitioner Performance and Patient Outcomes: A Systematic Review, JAMA. 2005;293(10):1223-1238

37 Tan, K., Dear, P. and Newell, SJ. (2009) "Clinical decision support systems for neonatal care,"Cochrane Database of Systematic Reviews, Issue 3, 2009.

78

38 Chandrasekhar, CP and Ghosh, J. (2001). Information and Communication Technologies and Health in Low Income Countries: The Potential and the Constraints. Bulletin of the World Health Organization. Vol. 79:9,