Mid-term Review Evaluation Report

Born to Survive: Closing the Gaps in Child Health in

Tibet P.R.C

Save the Children China Programme

(Tibet Health Project)

Diki Yudron

December, 2013

1

Contents

Acknowledgement ......................................................................................................... 2

List of acronyms ............................................................................................................. 3

Executive summary ........................................................................................................ 4

Introduction ................................................................................................................... 5

1. Background ......................................................................................................... 5

2. Key project activities carried out during the last two years project intervention

period:..................................................................................................................... 6

3. Mid-term review objectives ............................................................................... 7

Methodology .................................................................................................................. 7

1. Project mid-term review evaluation sites selection ........................................... 7

2. Methods .............................................................................................................. 8

Results and Discussion ................................................................................................... 9

1. Training effectiveness ......................................................................................... 9

1.1 Township Doctors Evaluation .................................................................... 9

1.2 Village Doctor Evaluation ........................................................................ 10

1.3 In-depth Interview with county level health bureau staff ...................... 10

2. Improvement of awareness of community health ........................................... 11

3. Medical equipment delivery and usage evaluation ......................................... 12

Conclusion and Recommendations ................................ ¡Error! Marcador no definido.

1. Conclusions ....................................................................................................... 14

2. Recommendations ............................................................................................ 14

Acknowledgement

Firstly, we would like to extent our heartfelt gratitude to all those who directly or

indirectly contributed and taking their parts in making this mid-term review

evaluation a grand success.

Secondly, as always, we are highly thankful to the two main donors respectively

ANESVAD and Unilever for their generous provision of grant support for this project.

Thirdly, we are greatly grateful to our local government partners: prefecture and

county level Health Bureaus, Centre for Disease Control (CDC), Women’s Federation

(WF), county hospitals, township and village clinics for their generous cooperation

and support in completing our mid-term review evaluation in Nagchu and Nyirong

counties. Without your support, we wouldn’t be able to make this mission

accomplished.

Also, our sincere appreciation goes to all the staff of Save the Children Tibet Health

Project team and two volunteers for their hard work and great efforts on data

collection throughout the evaluation.

Finally, we sincerely thank all the respondents for their support and patience

throughout our interview schedule. We assure you that the information you

provided us will be in good use.

3

List of Acronyms

ANC Antenatal Care

CDC Centre for Disease Control

WF Women’s Federation

TAR Tibetan Autonomous Region

KSA Knowledge and Skill Assessment

IDI In-depth Interview

FGD Focus Group Discussion

MCH Maternal and Child Health

IEC Information, Education and Communication Materials

UNCRC United Nation Convention on the Rights of the Child

ToT Training of Trainers

HBB Helping Babies Breathe

INGO International Non-government Organisation

MDGs Millennium Development Goals

4

Executive summary

Save the Children has been implementing projects in Tibet Autonomous Region (TAR)

since the early 1990s, as the largest child rights focused International

Non-government Organisation (INGO). In health, we work with local communities

and the health system to ensure that women and children have improved health

status, we do this through providing trainings to health workers, promoting health

knowledge and practices among caregivers, and working with government to change

policies and practices.

Currently, Save the Children is implementing a three-year project called Born to

Survive: Closing the Gaps in Child Health in Tibet. The goal of this project is to make a

contribution to the realisation of the right to life and the right to health to which

children and women are entitled to. This will ultimately contribute to the

achievement of Millennium Development Goals (MDGs) 4 and 5 which is to improve

maternal and child health and to reduce maternal and under-five mortality.

The project largely funded by ANESVAD and co-funded by Unilever phases from

January 2012 to December 2014. The project is implementing in two of the eleven

counties of Nagchu Prefecture, namely Nagchu and Nyirong Counties.

Year 2013 is the second project operational year for the project. In collaboration

with local external researcher, the project team and trained volunteers carried out

an interim project evaluation between October and November 2013 in the two

project counties.

The principal objectives of the review was to assess (i) the continuing viability and

appropriateness of the objectives, approach and design of the project, (ii) progress in

its implementation, and (iii) the effectiveness of the project, and to draw lessons of

experience to help guide project planning for the following year 2014.

In order to achieve the expected results of mid-term evaluation, the project team

employed a range of evaluation tools, including a) Knowledge and Skill Assessment

(KSA) for Township and Village Doctors, b) In-depth Interview (IDI) for County Health

Bureau Staff, Township and Village Doctors, c) Focus Group Discussion for Women of

Reproductive Age, d) Health Facility Checklist for County Hospitals, Township and

Village Clinics.

A total of 126 mothers, pregnant women, and women of reproductive age attended

the FGD. 2 county health bureau staff, 15 township doctors, and 56 village doctors

5

were interviewed with the IDI. The health facility assessment was conducted in 11

township clinics and 9 village clinics.

Most (60%) township doctors interviewed attended project organised MCH training

workshops for at least 3 times during the course of 2 years project period and 100%

of interviewed village doctors received at least once training. However, only half

(53%) of them reported be confidence in providing MCH services, including antenatal

care (ANC), normal delivery, newborn home visit, and common childhood illness

management. Most interviewed health workers wish to receive longer and more

frequent trainings.

80% of community members interviewed reported to have received MCH

information from township and village doctors. While 4-time ANC rate is reported

to be high, hospital delivery rate and postnatal care rate are remain low. The

awareness of danger signs during pregnancy and after delivery is also low.

Introduction

1. Background

The Tibet Autonomous Region (TAR) remains one of the most impoverished areas in

China. Due to lack of access to quality health care and a variety of cultural practices

that favour home births, high proportion of Tibetan women give birth entirely

unattended or attended by untrained family members, including mothers and sisters

– putting both mothers and infants at serious risk.

Nagchu Prefecture is located in northern Tibet. The average altitude is about 4,500

meters above sea level, with a total area of 450,537 km². It’s the largest prefecture in

the TAR, accounts for 37% of the total area of the TAR. The prefecture is divided into

11 different counties and has a population of 420,000.

Nagchu Prefecture

Nyirong County

Nagcu County

Figure 1. The project sites of the two counties in Nagchu

prefecture in Tibet.

6

The project aims to address the problems of availability of and access to quality

health care for women and young children in rural nomadic communities by

implementing an essential package of evidence-based, high impact, cost effective

interventions from the four main areas of child survival: 1) skilled attendant during

pregnancy, childbirth, and postpartum, 2) nutrition, 3) immunisation, and 4)

prevention and treatment of common childhood diseases, such as pneumonia and

diarrhoea.

The current on-going project intervention period is considered as a second project

year. In order to assess whether the project is achieving intended results, review its

organisational structures and technical approaches, and formulate

recommendations for future directions, amid-term review was carried out from 4th

to10th of November 2013.

2. Key project activities carried out during the last two years

project intervention period

1) Competency-based training for prefecture and county level MCH service

providers on emergency obstetric and neonatal care, common childhood illness

management, and supportive supervision.

2) Training of Trainers (ToT) for selected prefecture and county level MCH

providers on antenatal care, normal delivery, postnatal care, essential newborn

care and common childhood illness management.

3) Competency-based training courses for township and village level health care

providers on antenatal care, normal delivery, essential newborn care, postnatal

care, and management of common childhood illnesses.

4) Piloting Helping Babies Breathe (HBB) training program to address birth

asphyxia.

5) MCH management training for prefecture and county health bureau staff and

Women’s Federation (WF) staff.

6) Strengthen referral mechanism between village and health facilities.

7) Provision of basic MCH equipment.

8) Develop an adapted best practice health education package.

9) Identify and train community health promoters on behaviour change

communication skills and key MCH health topics.

10) Support community health promoters to conduct health promotion

activities in villages.

11) Support local health promotion campaign on special health days.

7

12) Stimulate awareness of rights and entitlements using different communication

method to promote and support maternal and child health which engage the

general public.

13) Regular contributions to the Global EVERY ONE Campaign.

14) Senior government officials to participate in cross-learning activity and child

survival related forum.

3. Mid-term Review Objectives

1) To evaluation training effectiveness evaluation.

2) To review medical equipment delivery and usage status.

3) To get project administrative partner feedback.

4) To evaluate community members’ knowledge and awareness in practising of

good health/hygiene and MCH home care.

Methodology

1. Project mid-term review evaluation sites selection

Prefecture County Township Village

Nagchu

Nagchu

Kolu Village No.2, 5, 9, and 10

Lhoma Village No.14

Khormar Village No.2, 3, and 5

Shamo Village No.2 and 3

Namochen Village No.4

Sishong Village No.6 and 7

Nyirong

Salong Village No.2, 4, 7, 9, and 10

Shachu Village No.1, 7, 14, and 15

Soshong Village No.1

Serchen Village No.17

Palshong Village No.4

Nyima

Table1. The survey site in Nagchu County and Nyirong County.

The project covered a total number of 22 townships, total population of 110,000

residents, including 10,996 children below 5 years of age. After considering the

population, accessibility to health care facilities (distance; time if by car/motorbike;

altitude, climate; road conditions, etc), availability of township clinics, number of

doctors have trained by project, we selected 6 townships and 13 villages from

8

Nagchu county and 6 townships and 12 villages from Nyirong county for mid-term

review evaluation. See Table1.

2. Methods

Data collection for the mid-term review evaluation survey utilized both qualitative

and quantitative methods including the following tools:

Knowledge and skill assessment for township doctors and village doctors

(Quantitative).

Health facility distributed essential basic MCH equipment assessment

(Quantitative).

Focus Group Discussions (Qualitative).

In-depth Interviews (Qualitative).

See Table 2 below for the survey methods and their implementation.

Respondents Sample Information Methods Tools

Township and

village doctors

who attended

project trainings

15 township

doctors and 56

village doctors

Antenatal care,

normal delivery,

neonatal

resuscitation,

postnatal care,

essential newborn

care and common

childhood illness

management

Test Knowledge and

skill assessment

New mothers,

pregnant women,

women of

reproductive age

126 women Community’s MCH

knowledge and

practices

Focus group

discussion

Mid-term

Review

Evaluation FGD

guide

County health

bureau staff,

township and

village doctors

2 health bureau

officials, 15

township

doctors, 56

village doctors

Training

effectiveness,

project

implementation

partner feedback

In depth interview

with heath bureau

staff and health

workers

Mid-term

Review

Evaluation

In-depth

Interview guide

9

Township and

village clinics

11 township

clinics

9 village clinics

Key MCH

equipment

delivery and usage

Health facility

observation

Health facility

assessments

check list

Table 2. Survey methods and its implementation in the mid-term review evaluation survey

Results and Discussion

1. Training effectiveness

1.1 Township Doctor Evaluation

In-depth Interview Result

Most (9 out of 15) township doctors interviewed attended at least 3 trainings during

the project implementation year 2012 and 2013. The average length of each training

was 6 days and training content included antenatal care, normal delivery, HBB,

danger signs, postnatal care, postpartum haemorrhage, breast feeding, essential

newborn care, common childhood illness management, and referral mechanism.

Most trained township doctors are self-reported to have confidence in providing

services in antenatal care, normal delivery, newborn care, and common childhood

illness management. And four babies’ lives were saved using basic newborn

resuscitation skills. The detailed information is as follows (Table 3).

Township Village Baby Birth Date Birth Time Doctor who

performed newborn

resuscitation

Nyirong Village No.1 Girl Jul, 28, 2013 12:10AM Thanam Nyirong Village No.6 Girl Aug, 13,

2013

4:32PM Thanam Nyirong Village

No.15

Girl Sep.

25,2013

2:08PM Thanam Nagchu Village No. 1 Girl June 12,

2013

3:00PM Tsering Tsomo

Table3. The profiles of the baby who saved the lives with HBB.

All township doctors interviewed used project developed resources, including

training guidelines, manuals and IECs, as key training tools, and found them to be

very useful and informative, because the materials are locally adapted, and with the

pictures in the materials, it is easy to understand, easy to explain to the village

doctor and community members. 9 of the 15 interviewed township doctors

organised and delivered cascading trainings to village doctors.

The further training, the interviewed township doctors suggested:

(a) To have longer training period.

10

(b) To have longer clinical practice at higher level health facilities as part of the

training.

(c) To have more training focus on postpartum haemorrhage, neonatal resuscitation,

and common childhood illness management.

Knowledge and Skill Assessment Result

Knowledge and skill assessment was also conducted with the 15 township doctors

interviewed, to evaluate training effectiveness. The test was divided into three

parts: a multi-choice paper-based test for essential newborn care, a multi-choice

paper-based test and performance observation with standard cases for HBB , and a

multi-choice paper-based standard case for postpartum haemorrhage. Only 10

township doctors tested for HBB, because the other 5 were not trained for HBB yet

at the time of mid-term review.

The knowledge test showed that the interviewed township doctors have the highest

knowledge retention in HBB (averagely 16 correct answers out of 17 questions), then

essential newborn care (averagely 4 correct answers out of 6 questions), and lowest

in postpartum haemorrhage (averagely 3 correct answers out of 6 questions). The

mains areas that the township doctors lost points were newborn temperature

management, cord care, breastfeeding counselling, and essential steps of

postpartum haemorrhage management.

1.2 Village Doctor Evaluation

In-depth Interview Result

A total of 56 village doctors attended In-depth Interview. All of them reported that

they had received one cascading training from township doctors, with an average

duration of 3 days. However, only less than half of them reported to be confidence in

providing health services in common childhood illness and antenatal care.

All the village doctors have been providing MCH information to community members

by conducting health education and promotion activities in their communities,

including conducting home education, organising community meeting and

providing consultation during routine clinic work.

Knowledge Assessment Result

The village doctors interviewed scored averagely 4 out of 9 questions, with

significant improvement needs in areas including antenatal care, breastfeeding as

well as postnatal care.

1.3 In-depth Interview with county level health bureau staff

The involvement of health bureau in the development and delivery of the project

included:

11

(a) Coordinated and organised various capacity building activities at all levels of the

health system, as well as activities during global moments, such as World

Breastfeeding week and Global Handwashing Day, etc..

(b) Involved in activities including project planning, implementation, follow up visits,

and monitoring and evaluation. From the interview, health bureau appreciated the

project’s contribution to the capacity building of township and village doctors.

However, due to low basic training level of township and village doctors, it is hoped

that the trainings could be repeated and prolonged, in order to maintain training

effect in terms of knowledge and skills.

The effort of strengthen referral system between community and different health

system levels, including referral trainings, referral emergency contact cards and

display board, was also recognised as valuable to strengthen the linkage within the

health system. However, it is recommended to conduct more research to

understand the barriers in referral (e.g. transportation, financial difficulty,

communication, etc.), and address accordingly in the future.

However, it is also recognised that unstable human resources at local health

authorities, due to health officials seconded to other places for social security

reasons, could be a disadvantage for health authorities’ consistent engagement in

the project.

2. Improvement of awareness of community health

Focus group discussions (FGD) were held with participants from 25 different villages

divided into 11 groups. The total number of participants was 126, including pregnant

women, women of reproductive age, and mothers with children under age of 2.

Most women attended the FGD reported had received MCH information from

township and village doctors, including importance of ANC, postnatal care, pregnant

woman and child nutrition, danger signs and breastfeeding.

Half of the women reported received more than 4 ANCs during pregnancy, and

mostly at township and village clinics. They could recognise the importance of ANC

as “it is important to ensure the mother and baby’s health, can identify danger signs

and timely intervention”.

During the FGD, most women reported received folic acid tablets from different level

of hospital during their pregnancy, however less women took them or took them

correctly (three months before pregnancy until three months). Some women

12

reported that they didn’t clearly know the functions and importance of taking folic

acid during their pregnancy.

Most women reported had a birth preparedness plan, in terms of money,

transportation, nutritional food, warm cloth for newborn, and in case of had to

deliver baby at home they will also prepare clean place for give birth, some tools for

cord clamping, and fire wood. However, more than half of the women said that they

understood the importance of hospital delivery and plan to deliver baby at different

levels of hospital. Less than half of interviewees reported received postnatal care

within 28 days of delivery.

In terms of breastfeeding, most mothers reported breastfed for more than 2 years,

however few fed their babies with colostrum. Still half mothers reported fed their

babies with other liquid in the first 6 months. Most women could name several

danger signs during pregnancy, including bleeding, swollen and headache. However,

less women could name danger signs at postnatal stage and danger signs for

newborn.

3. Medical equipment delivery and usage evaluation

Essential basic MCH equipment was distributed to health facilities during the past

one and half project years. In the mid-term evaluation, health facility equipment

delivery and usage assessment was conducted in 11 township clinics (6 from Nyirong

and 5 from Nagchu) and 9 village clinics (4 from Nyirong and 5 from Nagchu).

The evaluation results are displayed below. Generally, most of the health facilities

surveyed received the MCH equipment, however, usage of these equipment is

relatively low. The survey found that since the health facilities were scattered, it took

a long time for all the equipment disseminated in the health system from prefecture

level to primary level, that some facilities only received the equipment recently. Plus,

no. of services provided in some facilities is small, therefore it is hard to capture use

of equipment within a month time.

Items

No. of township

clinics received

the equipment

Percentage of

receive

No. of township

clinics used the

equipment in the

month before the

survey

Percentage of

usage

Digital Infant Scale 8 80% 1 12.5%

Obstetric table 9 90% 2 22.2%

Digital Thermometer 7 70% 2 28.5%

Cold box with ice 8 80% 2 25%

13

packs

NeoNatalie Complete

(light) 9 90% 5 56%

NeoNatalie

Resuscitator 9 90% 2 22.2%

NeoNatalie Suction 9 90% 2 22.2%

Is there area for

regular practice with

neonatal

simulator/mannequi

n?

4 40%

Is there recording of

resuscitation

attempts and results?

0 0%

Items

No. of village

clinics received

the equipment

Percentage of

receive

No. of village

clinics used the

equipment in the

month before the

survey

Percentage of

usage

Scissor(M) 8 89% 2 25%

Cord clamp/forceps 8 89% 0 0%

Scalpel and blade 7 78% 1 14%

Stethoscope 9 100% 3 33%

Sphygmomanometer 9 100% 4 44%

Disposable delivery

kits 7 78% 1 14%

Infant scale 8 89% 2 25%

Condoms 4 44% 1 25%

Cold box with ice

packs 5 56%

2 40%

Digital Thermometer 9 100% 4 44%

Safety Boxes 4 44% 1 25%

Head Light 5 56% 2 40%

White Uniform 7 78% 4 57%

14

Conclusion and Recommendations

1. Conclusions

The mid-term review result showed that the project had built the capacity of health

workers, especially at township and village level, that most of them had growth in

confidence of providing essential MCH services. Most of the equipment that the

project provided arrived at the health facilities. The IEC materials were well received

by using local language and culturally adapted images, so that township and village

doctors felt comfortable of using them in conducting health promotion activities in

communities.

In terms of the community, the awareness of ANC, birth plan and hospital delivery

has raised.

The project is also well received by the local health authorities, since it filled the gap

of health professional training needs at primary level.

2. Recommendations

Further trainings could be tailored to focus on topics that health workers are

weak in knowledge and skill.

Continue monitor the use of basic medical equipment. Information gathered

could be a reference to service usage situation at the local community.

Raise in awareness among community members is a good opportunity for

further project design to focus on improve service usage.