!!!!!THE OUTCOMES OF PAY FOR PERFORMANCE SCHEMES The Ministry of Health in Rwanda implemented a Pay for Performance (P4P) scheme targeted towards neonatal and under age 5 child health indicators. The result of the P4P scheme was a 20% increase in healthcare provider efficiency and a significant improvement in under age 5 child health. Policies should focus on implementing the P4P scheme nationally complemented by additional medical training. !David Dingus !Neonatal care has a significant impact on children's health once born, which has lifelong effects by impacting both physical and cognitive development of the child, and affecting their health, educational attainment, and employment outcomes. An increase in child health will lead to greater growth and development for a nation in the long-run. !P4P schemes are a way to encourage an increased level of provider care. These schemes pay medical facilities based on their performance, and payments can target specific treatments that are deemed a priority. In this case, neonatal and under age 5 child healthcare were targeted. In total there were 14 specific areas of focus. There have been many studies on the effects of P4P schemes in wealthy countries, but there has been relatively little research on the effects in low income countries. To see if P4P could have a significant positive impact on child health outcomes in low income countries, researchers evaluated the national P4P scheme that has been implemented in Rwanda. !In the research paper: Using Performance Incentives to improve Medical Care Productivity and Health Outcomes, the authors evaluate the impact of the P4P scheme on the quantity and quality of healthcare provided by medical facilities, and how this affected under age 5 child health.

POLICY BRIEF 3 J - PAL POLICY BRIEF [ JULY 18 2014 ]

Pay for Performance improved under age 5 child health. 0-11 month health improved in terms of weight for age by .53 standard deviations, and 24-48 month health improved in terms of height for age by .25 standard deviations to the comparison group.

The increase in child health was attributed to incentives, not an increase in financial resources. Part of the benefits of a P4P scheme is increased resources, however, the evaluation concluded that the improvements were isolated to incentives.

Healthcare providers became more efficient. As a result of the P4P scheme, providers administered more of the knowledge they already knew, increasing provider efficiency by 20%.

Increased medical training complements the P4P scheme. Additional training could have yielded even better child health outcomes. The evaluation concluded a positive correlation between administered care and the level of knowledge of the provider.

www.povertyactionlab.org 1

In 2006 the Ministry of Health in Rwanda implemented a P4P scheme, which provided addi t iona l payments based on out l ined performance and quality indicators. The prospective study collected baseline data to develop an internally and externally valid study. The study utilised a randomised offering to select 10 districts that were representative of the country to participate in the P4P scheme. The remaining 9 districts, which were also representative of the country, were chosen as the comparison group. In total, 80 medical facilities were phased into the P4P scheme over a 5 month period, and evaluated for an additional 23 months.

P4P facilities were evaluated with numerous audits to determine the quantity of care provided with payments assigned to fourteen specifically targeted indicators. These payments were then adjusted by the quality of care being administered through observation and auditing. To control for the effects of increased funding, 86 comparison hospitals were given the equivalent increase in funding with no evaluation. !Additionally, an independent body was setup to conduct a quasi-experimental study to examine the impact that the P4P scheme had directly on under age 5 child health outcomes. The prospective survey gathered baseline data beginning in 2006 and randomly surveyed 13 households with under

age 5 children from each treatment and control facility for a total of 2158 households. Since the P4P scheme was originally distributed in an internally and externally valid way, the same holds true for this survey, because the enumerators surveyed an equal number of households from each treatment and control group, and geographic region. There was a 98% participation rate, and 88% of households were able to be reexamined 23 months later.

Using a Difference-in-Difference model the survey was able to control for the effect of the P4P scheme on child health. To isolate for the different impacts of neonatal care and infant care, children were examined in the 0-11 month group and in the 24-48 month group. This allowed observations to be categorised based on the different type of care resulting from the implementation of P4P over the child's lifetime.

EVALUATION

1) Specific cash value for 7 indicators based on the number of visits for a specific treatment at

2) Specific cash value for 7 indicators based on the content of care provided at the facility

[Treatment Group] Total Payment = (1 + 2) x (quality provided) where 0 is the lowest quality and 1 is the highest quality.

[Control Group] Receives the same payment as the treatment group without observation, to control for increased financial resources.

www.povertyactionlab.org 2

Figure 1: !Implementation and evaluation period to child’s age

Implementation !in 2006

Implementation !in 2006

Evaluation !in 2008

Evaluation !in 2008

In utero — 0 ———— 11 24 ————————— 48

Healthcare providers became more efficient. Looking at Figure 2, medical personnel knew 63% of medical protocols, but only administered 45%, representing an 18% gap. This gap did not change over the evaluation period for providers with below-average knowledge. However, the inefficiency gap in administered care decreased by 3.5 percentage points for those with average knowledge under P4P, representing a 20% increase in efficiency. The effect of P4P led to even greater gains in efficiency for providers with above-average knowledge, showing a 6 percentage point decrease in inefficiency.

Increased medical training complements the P4P scheme. Given that providers with below-average knowledge saw no improvements in administered protocols, while providers with above-average knowledge saw even larger efficiency gains, we see that additional medical training would complement the P4P scheme. This increase in efficiency would make the P4P scheme more effective and result in a larger improvement in child health. !Pay for Performance improved under age 5 child health. Looking at figure 3 we see that there had been a significant improvement in child health for those children receiving treatment from a facility

under P4P. On average, there was an improvement of infant weight for age by .53 standard deviations, and an improvement for children between 24-48 months old by .25 standard deviations versus the comparison group. There was a smaller impact on the height for age improvement for infants because of breast milk. Breast milk lacks many of the micro nutrients needed to foster height, but breast milk does improve weight, which helps fight illness. As a result of being able to fight illness, children between 24-48 months were able to grow more than those in the comparison group.

The increase in child health was attributed to incentives not an increase in financial resources. After controlling for an increase in financial resources, it was seen that increases in child health could be attributed to incentives from the P4P scheme, rather than increased resources. As such, the money used to increase the resources of the control group was not effective, and had the same amount of money been delivered in an incentive form under P4P, it would have yielded a greater impact on child health.

0,0%

25,0%

50,0%

75,0%

100,0%

1 2 3 4

25%37%37%37%

12%

14,5%18%18%

63%48,5%45%45%

Implemented Inefficiency Gap No knowledge

1 - No P4P 2 - Below-Average Knowledge 3 - Average Knowledge 4 - Above-Average Knowledge

Figure 2: !Impact of P4P scheme on administered care and inefficiency gap

RESULTS

www.povertyactionlab.org 3

Children 0 - 11 months old Children 24 - 48 months old

0,03

0,53

0,25

0,16

Height for Age Weight for age

Figure 3: !Improvement of P4P on child health indicators by standard deviation

!P4P schemes can be an effective tool in improving child health outcome. P4P schemes create incentives for medical providers to act more efficiently, carrying our additional treatments at an improved quality. This evaluation concludes that incentives are the key influence in these observed improvements in child health and that these improvements were not the result of increased financial resources. It should be noted that the facilitates examined in Rwanda already had 95% of the recommended resources and this factor must be considered independently when P4P is implemented on facilities with fewer resources. However, once a facility has the majority of the recommended resources, further financial measures should focus on creating incentives for the healthcare providers to act more efficiently and thereby improve child health outcomes. !

Complementing a P4P scheme with further medical training could lead to even greater efficiencies. This survey concluded that increases in efficiencies were greater for healthcare providers with higher levels of knowledge. Complementing the P4P with a training program would result in more knowledgable staff. These providers would then administer greater levels of care and lead to greater improvements in child health than P4P alone.

Policy Lessons

www.povertyactionlab.org 4