end of term evaluation for sustainable water, …...methodology for the evaluation a combination of...

WASH Initiative for the Rural Poor in 21

Districts in Uganda

End of Term Evaluation

FINAL REPORT

Prepared By

Narathius Asingwire (PhD) SWSA Dept. Makerere University

July 2012

List of Acronyms and Abbreviations ACAO Assistant Chief Administrative Officer ADHO Assistant District Health Officer ADWO Assistant District Water Officer AMREF African Medical Research Foundation CAO Chief Administrative Officer CBMS Community-Based Maintenance System CBO Community-Based Organizations CDO Community Development Officer CFR Case Fatality Rate CLTS Community Led Total Sanitation CRS Catholic Relief Services DEO District Education Officer DHI District Health Inspector DHO District Health Officer DWD Directorate of Water Development DWO District Water Officer DWSCC District Water and Sanitation Coordination Committees DWSDCG District Water and Sanitation Conditional Grant ECD Early Childhood Development EPI-INFO Epidemiological Information EU European Union FY Financial Year GDP Gross Domestic Product GFS Gravity Flow Schemes GIS Geographical Information System GoU Government of Uganda HA Health Assistant HC Health Centre HI Health Inspector HMC Health Management Committee HPM Hand-pump Mechanic HSSP Health Sector Strategic Plan HWWS Hand-washing with soap IDPs Internally Displaced Persons IGAs Income Generating Activities IMR Infant Mortality Rate LC Local Council LG Local Government LEARN Literacy Enhancement and Rural Nutrition LRA Lord’s Resistance Army M&E Monitoring and Evaluation MDGs Millennium Development Goals MoES Ministry of Education and Sports

ii

MoFPED Ministry of Finance, Planning and Economic Development MoGLSD Ministry of Gender, Labour and Social Development MoH Ministry of Health MoLG Ministry of Local Government MWE Ministry of Water and Environment NDP National Development Plan NGO Non Governmental Organization NUSAF Northern Uganda Social Action Fund O&M Operation and Maintenance ODF Open Defecation Free PEAP Poverty Eradication Action Plan PHAST Participatory Hygiene and Sanitation Transformation PRDP Peace, Recovery and Development Program P/S Primary School PSOs Private Sector Organizations PTAs Parent-Teacher Associations SFG Schools facilities Grant SFMCs School Feeding Management Committees SMC School Management Committee SPSS Statistical Package for the Social Scientist UDHS Uganda Demographic and Health Survey UGX Uganda Shillings UN United Nations UNICEF United Nations Children Fund USD United States Dollar VHTs Village Health Teams WASH Water, Sanitation and Hygiene WATSUP Water Atlas Update Project WSC Water and Sanitation Committee

iii

Acknowledgement Several people contributed to making this evaluation possible from the preliminary and inception phases to data collection and now to drafting of the report. With profound gratitude we acknowledge all that we cannot mention all the names, but in a special way, we are indebted to Mr. Samuel Madul, Water, Sanitation and Hygiene (WASH) Officer for his time in attending to the constant demands of the consultant and for effectively co-ordinating the entire effort and the comments made on the draft report. With utmost gratitude we acknowledge with appreciation Ms. Daniela Rojas Chaves, the Research and Evaluation Specialist for her tremendous insights and comments on the methodology, tools and draft report. Mr. Prakash Raj Lamsal, WASH Specialist provided invaluable comments of the draft report. The initial reflections on the terms of reference (ToR) and evaluation planby Mr. Chander Badloe enabled the consultant to conceptualize the assignment. Special thanks go to the District Water Officers (DWOs) and Health Inspectors (His) in the 10 districts visited for the wonderful preparations and participation. You willingly and keenly organized consultative meetings with all relevant stakeholders and also provided invaluable information to the evaluation team regarding your experience with the project implementation. To the community members, school administrators and the pupils we interacted with during evaluation, we owe you a huge debt of gratitude for sparing your valuable time to share your experiences and views on the integrated WASH services and activities implemented with the European Union (EU)-United Nations Children Fund (UNICEF) fund. Without the Field Supervisors; Mr. Joseph Kiwanuka and Mr. Christopher Muhoozi it would have been a huge challenge for the Lead Consultant to accomplish all the different tasks of this evaluation. Finally, to all the 18 Interviewers you are greatly acknowledged for having ably traversed the 20 sub-counties locating the respondents and inspecting the performance of WASH facilities.

iv

Table of Contents

List of Acronyms and Abbreviations ....................................................................................................................... i Acknowledgement.................................................................................................................................................... iii Table of Contents ...................................................................................................................................................... iv Executive Summary ................................................................................................................................................. vii 1.0 INTRODUCTION AND COUNTRY CONTEXT ..................................................................................... 1

1.1 Project Background ...................................................................................................................................... 1 1.3 Purpose and Objectives ............................................................................................................................... 2

1.4 Scope of the Evaluation ............................................................................................................................... 3

1.5 Report structure ............................................................................................................................................ 3 2.0 EVALUATION APPROACH AND METHODOLOGY .......................................................................... 4 2.1 Overall Approach ......................................................................................................................................... 4

2.2 Study Areas and Participants ..................................................................................................................... 4 2.3 Sample/Study Participant Selection Procedure ....................................................................................... 5

2.4 Data Collection Methods ............................................................................................................................. 6 2.5 Data Management and Analysis ................................................................................................................ 7

2.6 Analysis of the Socio-Demographic Characteristics of Household Respondents and Pupils ........... 7

3.0 RELEVANCE .............................................................................................................................................. 10

3.1 Introduction ................................................................................................................................................ 10 3.2 Project Consistence ..................................................................................................................................... 10

3.3 Project Coherence with Current Initiatives of GoU and Sector Partners ............................................ 11 3.4 Clarity and Internal Consistency of the Project Objectives................................................................... 12

3.5 Stakeholder Participation .......................................................................................................................... 13 3.6 Appropriateness of the Recommended Monitoring and Evaluation (M&E) Arrangements ........... 14

4.0 EFFECTIVENESS ........................................................................................................................................ 15 4.1 Introduction ................................................................................................................................................ 15

4.2 Delivery on Planned Benefits.................................................................................................................... 15 4.3 Beneficiary Participation in the Intervention .......................................................................................... 20

4.4 Influence of Institutional Arrangements on Project Effectiveness ....................................................... 22 5.0 EFFICIENCY ............................................................................................................................................... 24 5.1 Introduction ................................................................................................................................................ 24

5.2 Operational work-planning, Implementation and Budget management ........................................... 24

5.3 Relations and Co-ordination with Local Authorities, Institutions, Beneficiaries and other Donors ........................................................................................................................................ 25

5.4 Quality of Information Management and Reporting ............................................................................ 25 5.5 Cost-Benefit Analysis ................................................................................................................................. 26

5.6 GOU Contributions to Project Efficiency ................................................................................................ 27 5.7 Appropriateness of Project Technical Assistance .................................................................................. 28

6.0 IMPACT ....................................................................................................................................................... 30

6.1 Introduction ................................................................................................................................................ 30

6.2 Access to Safe Water .................................................................................................................................. 30 6.3 Access to Sanitation Facilities ................................................................................................................... 34

6.4 Hygiene Behaviors and Practices ............................................................................................................. 37 6.5 Knowledge and Reported Occurrence of WASH Related Diseases .................................................... 38

v

6.6 Access to Information on WASH ............................................................................................................. 41

6.7 Factors which Facilitated Attainment of Project Impact ....................................................................... 43 6.8 Factors which Constrained Project Impact ............................................................................................. 44

6.9 Impact on Cross-cutting Aspects ............................................................................................................. 45 7.0 SUSTAINABILITY ...................................................................................................................................... 47 7.1 Introduction ................................................................................................................................................ 47

7.2 Ownership of the Objectives and Achievements ................................................................................... 47 7.4 Institutional capacity .................................................................................................................................. 48

7.5 Socio-cultural factors.................................................................................................................................. 49

7.6 Financial Sustainability .............................................................................................................................. 49 7.7 Technical (Technology) Issues .................................................................................................................. 50

8.0 GOOD PRACTICES, LESSONS, CHALLENGES AND RECOMMENDATIONS ............................ 51

8.1 Introduction ................................................................................................................................................ 51 8.2 Good Practices ............................................................................................................................................ 51

8.3 Lessons ......................................................................................................................................................... 52 8.4 Implementation Challenges ...................................................................................................................... 53

8.5 Recommendations ...................................................................................................................................... 54

References .................................................................................................................................................................. 56

Appendix 1: Map of Uganda showing the project area ....................................................................................... 57

Appendix 2: Key informants at district, sub-county and community level ..................................................... 58

vi

List of Tables

Table 1: Distribution of household respondents by district and sub-county ..................................................... 4 Table 2: Distribution of household respondents by district and sub-county ..................................................... 5

Table 3: Distribution of interviewed pupils by district and school ..................................................................... 5 Table 4: Socio-demographic characteristics of household respondents .............................................................. 8

Table 5: Characteristics of schools and pupils interviewed .................................................................................. 8 Table 6: Activities engaged in the various Sector partners and districts of operation .................................... 11

Table 7: Planned and implemented water activities ............................................................................................. 18 Table 8: Household participation in construction and O&M of the water facilities ........................................ 20

Table 9: Household participation in construction and O&M of water sources ................................................ 21

Table 10: Type of contribution towards O&M by gender and district .............................................................. 21 Table 11: Uganda Country Progress against 7 out of 11 eleven golden MDG indicators

for the WASH sector................................................................................................................................. 30 Table 12: Trends in levels of access to safe water in project supported districts ............................................. 31

Table 13: Source of water by households and school .......................................................................................... 32 Table 14: Access to safe water: Project performance as per MDG indicators ................................................... 33

Table 15: District and project site latrine coverage ............................................................................................... 35 Table 16: Pupil:stance ratio of studied schools ..................................................................................................... 35

Table 17: Access to sanitation: Project performance as per MDG indicators ................................................... 36

Table 18: Hygienic conditions of latrines .............................................................................................................. 36 Table 19: Hygiene practices and indicators ........................................................................................................... 38

Table 20: Pupils knowledge of WASH related diseases and experience ........................................................... 38 Table 21: Respondents knowledge of WASH related diseases........................................................................... 39

Table 22: Reported known WASH related diseases by district .......................................................................... 40 Table 23: Households reported WASH related morbidity and mortality ......................................................... 40

Table 24: Methods of household solid waste management and non-dangers of indiscriminate disposal ... 41 Table 25: Accessibility to information and sources of learning about WASH ................................................. 42

Table 26: Respondents receipt of information by district and gender............................................................... 42

List of Figures Fig 1: Household main water source pre and after Project implementation .................................................... 32

Fig 2: Availability of WUC in sampled communities .......................................................................................... 33

Fig 3: Use of latrine by pupils at school ................................................................................................................. 35 Figure 4: Reported WASH sources of information .............................................................................................. 43

vii

Executive Summary

Introduction This end of project evaluation of the “Sustainable Water, Sanitation and Hygiene (WASH) Initiative for the Rural Poor in 21 Districts in Uganda” was carried out in May and June. The evaluation focussed on assessment of the project’s relevancy, effectiveness, efficiency, impact and sustainability vis-à-vis the 3 objectives that the project aimed at realising, namely, (i.) reduced mortality and morbidity due to WASH related diseases, (ii.) accelerated progress towards meeting the Millennium Development Goals (MDGs) drinking water and sanitation target, and (iii) increased access to integrated WASH services. Out of these three broad objectives, the project had four result areas that included (i) increased access to latrines in the rural population by 5 percent, (ii) hygiene practices improved by 25 percent, (iii) a total of 210,750 new users gain access to safe water, and (iv) institutional development and strengthening of policies, guidelines, and advocacy. To achieve the desired results, UNICEF planned to implement several WASH activities in schools and communities including construction of latrines for demonstration purposes, construction of water facilities, promotion of hygiene practices and institutional development.

Methodology for the Evaluation A combination of quantitative and qualitative methods was adopted in carrying out this evaluation. Out of 21 districts project districts, 10 were selected for in-depth assessment. Selection of the studied 10 districts was largely guided by the activities implemented across the project area and regional representation. Study participants included household members, school pupils, members of water and sanitation committees (WSCs), school authorities, technical staff at district and sub-county level. Other participants included the Sub-county chiefs, Local council (LC) III chairpersons, LCV chairpersons or their deputies. Random sampling techniques were used in the selection of 504 household respondents in 20 sub-counties and 151 pupils in 5 schools. Purposive sampling was employed in selecting stakeholders at national, district and sub-county levels, including members of water and sanitation committees (WSCs), school authorities, technical staff at district and sub-county level. Other participants included sub-county chiefs, local council (LC) III chairpersons, LCV chairpersons or their deputies. Quantitative and qualitative methods of data collection were used. Quantitative data was collected using structured questionnaires and observation checklists that were administered at the household and school level. Quantitative data provided a descriptive summary of the extent to which the project objectives were achieved. Qualitative data was collected using unstructured interview and focus discussion guides administered to technical staff at the district and sub-county levels, as well as school and health facility administrators in sampled study sites.

Key Evaluation Results

Relevance Relevance refers to the extent to which the objectives of the project were consistent with the beneficiaries' requirements, country needs, global priorities and partners' policies. Evaluation

viii

findings indicate that the project was very relevant as it aimed at contributing to the realization of the national WASH goals and objectives enshrined in the then Poverty Eradication Action Plan (PEAP) and later the National Development Plan (NDP). The project complimented Government of Uganda (GOU) programs such as the Peace, Recovery and Development Program (PRDP) and the Northern Uganda Social Action Fund (NUSAF) II both intended to support the northern region to recover from the effects of the conflict. The project targeted an essential area of need i.e., safe water, sanitation and better hygiene practices, which are critical in reversing morbidity and mortality mainly among children. Internationally, the Millennium Development Goals (MDGs) prioritize WASH under MDG 7. The project objectives were in synch with the objectives of the NDP 2010/11-2014/15, which replaced the PEAP, the Health Sector Strategic Plan III 2010/11-2014/15 (HSSP III), and the Water and Sanitation Sector Investment Plan 2000-2015 (SIP 15).

Effectiveness The effectiveness criterion refers to the extent the project's results were attained, and the project's specific objectives achieved. Evaluation results reveal a high degree of effectiveness as reflected by registered achievements in the planned hardware and software WASH activities. Supported districts designed and constructed nearly all the planned model/demonstration latrines (i.e. 201 out of 305) mostly at schools and health centres. All contractors delivered the agreed-on five-stance latrine blocks with adequate privacy and hand-washing facilities for targeted schools. Activities such as sensitizations on WASH services and activities were effectively conducted targeting communities, political, religious and cultural leaders through advocacy workshops. Through the project, local artisans and staff of partner NGOs received training in implementation of WASH activities. Hygiene education outreaches, hand-washing demonstrations as well as co-curricular activities that targeted hygiene promotion in schools were implemented as planned. In addition to demonstrations, UNICEF procured and supplied sanitation kits and detergent to 23,789 rural households, and 309 primary schools reaching about 155,000 children. District and sub-county authorities, particularly health inspectors and assistants, were trained in Participatory Hygiene and Sanitation Transformation (PHAST) approaches. All the 21 districts were supported by UNICEF in the preparation of Sanitation Ordinances and By-laws. However, of all the regions where the WASH project was implemented, the Karamoja sub-region registered low results for sanitation and hygiene. With regard to access to safe water, new water users were reached through construction of boreholes, motorized wells and extension of the gravity flow schemes (GFS) as well as installation of rainwater harvesting systems at primary schools. Operation and maintenance (O&M) workshops were conducted as planned for district politicians and members of the District Water and Sanitation Coordination Committee (DWSCC). Over 230 WSCs were trained in preventive maintenance of the water sources while 51 hand-pump mechanics (HPMs) were trained to handle minor and medium repairs. As a result, all water sources visited during this evaluation were functional except one.

Efficiency The assessment of project efficiency focused on how well the various activities transformed the available resources into the intended results or outputs, in terms of quantity, quality and timeliness. The increased access to safe water from all the functional sources constructed demonstrated the cost-effective nature of the WASH project. With regard to sanitation facilities,

ix

in the face of fast filling pit latrines and in areas with weak and collapsing soils, UNICEF adopted a new technology for pit latrine construction (i.e., lining up the walls), which was cost-saving as it would prevent the sides from collapsing. Planning and implementation of all project activities was jointly done by UNICEF and district stakeholders. District staff actively participated in the planning, implementation and monitoring of all project activities including those that were not directly managed by the district local government like procuring of contractors. In Northern Uganda where the population was returning from IDPs camps implementation of some project activities particularly construction of new water points did not follow the standard guidelines as stipulated by the Ministry of Water and Environment (MoWE). It was prudent that the construction of the water points commenced before implementing software activities such as sensitization and community mobilization for O&M of the installed services.

Impact of the Project The project impact was assessed by examining the relationship between the overall and specific objectives. Overall, the project registered notable increase in access to safe and clean water in all the 21 districts above the national average except the Karamoja sub-region. Majority (88.1%) of household respondents reported accessing safe water, which is higher than the national average estimated at 65%. Majority of household respondents (81.0%) reported accessing their current water source within a distance of less than a kilometer (<1 km). Before the WASH Project, 53.7%

of the sampled households obtained water from unprotected sources. The sources of safe water for supported schools were boreholes and GFS and motorized wells, which were constructed with UNICEF support. Majority of the pupils interviewed (84.1%) acknowledged easy access to safe water while at school i.e., sources located within schools’ compound. Increased access and use of sanitation facilities such as pit latrines, bathing shelters and kitchen drying racks as well as rubbish pits was evident. In the 10 sampled project districts, latrine coverage was estimated at 71.4%, which is slightly higher than the national average estimated at 69.8%. Both in schools and households self-reported use of latrine was considerably very high; 89.3% and 76.6% respectively. Overall, the average pupil-stance ratio of the five schools visited was 59:1. In general, the proportion (29%) of households found without latrines and the pupil-stance ratio of 59:1, represent a big number without access to latrines. The low pupil-stance ratio is attributed to increased enrollment for primary education while for household it is attributed to either high costs of construction or lack of manpower. Almost all the interviewed pupils (98%) reported washing hands after latrine use compared to 67.9% of household respondents. But at the time of this evaluation only 40% of the observed hand-washing facilities (HWFs) at schools had water, and presence of soap was only observed in 18%. At the household level, only 20.1% had HWFs and with only 11.2% containing water at the time of the visit. General awareness of existence of WASH related diseases among pupils was universal (97.4%) among sampled school pupils. The most known WASH related disease as reported by pupils was diarrhea (68%); reported by 69.3% and 66.7% female and male pupils respectively. In terms of morbidity, over three-thirds (69.4%) of the pupils reported not to have suffered from any WASH related diseases in the three (3) months preceding this evaluation; an indicator

x

of the WASH Project’s impact on the targeted schools. Like school pupils, household respondents exhibited almost universal awareness about WASH related diseases (95.4%). Varied differences however, emerged in the knowledge of WASH specific related diseases and cases of death across the 10 sampled districts. Reports of households losing members (dying) due to WASH related diseases mostly came from participants in Moroto, followed by Soroti, Bundibugyo and Gulu; about 58% of respondents in Moroto reported losing a household member. In the entire 10 sampled districts there were reports that in overall terms WASH related diseases such as diarrhea, dysentery, cholera, intestinal worms, skin rush, coughs etc had reduced. Successful implementation and attainment of the WASH Project objectives is attributed to several factors; availability of funding, goodwill from both the political and civil leadership in the districts, and the modus operandi chosen by UNICEF. By working through the district and sub-county LGs, UNICEF ensured that no parallel or ad-hoc systems for Project implementation were setup. All Project activities were primarily planned, implemented and/or supervised by district staff, which approach was appreciated by district stakeholders. The Project was managed and coordinated well, involving all stakeholders including the actual direct beneficiaries.

Sustainability Sustainability refers to continuation of positive outcomes of the project and the flow of benefits after expiry of external funding for activities. Sustainability of the project is traced in the manner it was conceptualized and implemented. Evaluation findings reveal that the project was implemented within the national policy and planning framework with objectives subscribing to those of the sector. Consultation of stakeholders characterized the implementation of the project including communities. Policy support towards exchange visits of stakeholders and beneficiaries from one region to the other was critical in relation to sustainability of benefits. Evaluation findings with regard to financial sustainability present a promising picture in some areas. For instance, initiatives for raising O&M funds such as forming users’ groups which engage in IGAs e.g., brick laying, commercial farming are emerging as good sustainability practices especially in Northern Uganda. Within schools, sanitation/health clubs have been formed with patrons – this approach is envisaged to ensure sustainability of, among others, the hand-washing campaign, which is basically a software activity, but not a hardware that would include latrine sustainability. However, what is glaringly missing are financial resources to undertake major repairs such as replacing whole pipes, pump heads, etc. LGs, especially sub-counties, are ill-equipped to shoulder the responsibility of O&M for boreholes.

Good Practices and Lessons Learnt Beneficiary involvement in project implementation and monitoring, partnership with local government, private contractors and other partners (NGOs) and combining capacity building with service delivery are evidently good practices that characterized the project. Lesson learnt from implementing the WASH project include the following:

Leadership and effective participation of local leaders in project implementation is critical not only in ensuring smooth undertaking of activities but also eventual sustainability.

xi

The WASH project component in schools was meant to create awareness among school children as a way of promoting good sanitation and hygiene practices at school and home. By creating awareness, children became change agents with regard to improved household sanitation and hygiene.

Provision of sanitation facilities especially the construction of latrines contributed greatly to increased school attendance of pupils. This was reported to be particularly so for girls in upper primary.

Formation and training of WUCs is critical for sustainable O&M of water facilities. WUCs need regular monitoring i.e., to be refreshed on their roles and function, which is critical for the functionality of the committees.

Existence of trained and retooled hand-pump mechanics is a critical requirement in sustainability of boreholes.

Private sector and NGOs unlike government agencies can be very efficient and effective in implementation of WASH activities.

Centrally managed procurement process of contractors ensures that even the most remote communities are not shunned by private contractors.

Combining hardware with software activities or the latter coming later given the emergence nature of the communities in northern Uganda made the project produce maximum impact.

Implementation Challenges A few challenges were encountered during the implementation of this project, which were in a way beyond the control of the project.

The much yearned return of peace in northern Uganda resulted into populations returning to communities with no WASH facilities since these had been constructed in the IDPs, which necessitated re-planning. The scattered nature of communities equally made planning, implementing and monitoring of project activities quite challenging.

Beneficiaries’ unqualified expectations and dependency syndrome due to humanitarian aid made it difficult to encourage communities to build independent and sustainable systems without “hand-outs” or “facilitation fees”.

Occasional delays in implementation of project activities during the inception phase and subsequent activities due to weak financial management capacity of district local governments to submit quality accountabilities in time.

Despite the above implementation challenges, the WASH project accomplished its objectives as evidenced in increased access to safe water, improved sanitation and hygiene behaviours and practices in rural target communities and school and strengthened institutional capacity of local partners. Recommendations Drawing from the findings it is evident that project supported districts have better O&M of water facilities, all newly constructed and rehabilitated water facilities have remained functional throughout the project life. For the water facilities to remain functional, the high standards exhibited in O&M needs to be upheld. Similarly, to ensure that communities that have been supported to become model villages with regard to hygiene and sanitation do not

xii

slacken, the need is to sustain and scale up sensitization and community sanitation mobilization campaigns to even cover other communities. Sustainability and scale up of sanitation and hygiene will involve:

Sustaining active involvement of VHTs and engaging of political leaders to endear them to be exemplary cases

Political support and enforcement – there is a need for regular hygiene and sanitation advocacy meetings with LG political leaders

Districts need to be supported to disseminate and enforce sanitation bye-laws.

Some project activities, especially under sanitation, require more frequent follow-up and intensified in places such as Karamoja

To further support ODFC there is a need for sustained monitoring to track the outcome of ODF.

To ensure sustainable O&M the following should be done:

Continued strengthening of LGs technical capacity to support monitoring and mentoring of WSCs. and also building capacity for new districts to manage water facilities.

Train new WSCs and provide refresher training at least after every two years of their formation. Support and empower trained WSCs to enact and enforce bye-laws for O&M of water facilities. Sub-counties should have adequate budget to fund training of WSCs and monitoring visits and provide back-up support to WSCs.

1

1.0 INTRODUCTION AND COUNTRY CONTEXT

1.1 Project Background The WASH Project1, an initiative for 21 rural poor districts in Uganda aimed at contributing to the attainment of the MDGs, was funded by the European Union (EU) since 2007. The project was designed for a four year period, running from October 2007 to October 2011. It was implemented in seven districts of Northern Uganda, namely Amuru, Amolatar, Gulu, Lira, Oyam, Dokolo, and Apac; five districts from Karamoja namely Abim, Kaabong, Kotido, Moroto, and Nakapiripirit; four districts in Teso namely Amuria, Katakwi, Soroti, and Kaberamaido; and five from Western Uganda namely Kamwenge, Kabarole, Bundibugyo, Kasese, and Kyenjojo. See Appendix 1. After a year of Inception Phase, implementation of other planned activities started (October 2008) and ended in March 2012, following a no-cost extension of the project from October 2011. The WASH Project was set out to achieve three broad objectives, namely:

1. Reduce mortality and morbidity due to water and sanitation related diseases. 2. Accelerate progress towards meeting the MDG on drinking water and sanitation targets. 3. Increased access to integrated WASH services in 21 remote poverty stricken and conflict

affected districts of Uganda. Adjustments in the project design after the Inception Phase (2007 to 2008) resulted into four project result areas on which this evaluation focused. The result areas were:

1. Access to latrines in the rural population of 21 districts increased by 5 percent 2. Hygiene practices (safe water chain, hand washing, food hygiene) among the targeted

population in the 21 districts improved by 25 percent. 3. A total of 210,750 new users gain access to safe water in the rural population of the 21

districts. 4. Institutional development and strengthening of policies, guidelines, and advocacy

To achieve the desired results, UNICEF planned to construct 275 latrines at schools and 30 in the communities for demonstration purposes, 190 deep boreholes, and 152 shallow wells, install 86 rainwater harvest tanks in schools, rehabilitate 100 deep boreholes, 100 shallow wells and 8 Gravity Flow Schemes (GFS). In addition, UNICEF planned to supply 21 water quality test kits and chemicals to the supported districts. With regard to institutional development, UNICEF planned to facilitate production of books, flyers on lessons learned, operations research where need arose e.g., in water quality, review of policies, guidelines and standards,workshops and seminars to sensitize/train the various stakeholders (e.g. political, religious and cultural leaders at the district and lower levels, local artisans and HPMs, WSCs) and advocate for WASH services. UNICEF also planned to facilitate 17 radio talk shows per quarter, 41 radio spot messages per month, production of various promotion materials and 12 learning exchange visits across the region. With the official expiry of the WASH Project in March 2012, UNICEF Uganda commissioned this external End of Term Evaluation to assess the extent to which the set objectives were

1 Throughout this report the acronym WASH Project is used to refer to this EU/UNICEF project.

2

achieved, and pick out possible lessons. Data collection for this evaluation was conducted in May and June 2012 through a desk review and field visits to selected 10 districts and at national level.

1.2 Uganda Country Context With a total fertility rate of 6.2 (UDHS, 2011) and average annual population growth rate of 3.2% between 1991 and 2002, it is projected that by mid 2011, Uganda had a population of approximately 32.9 million people (UBOS website). Uganda remains among the poorest countries in the world with an annual average GDP per capita of USD 390. The UN Human Development Index ranks the country 144th out of 175 countries in the world. Far more people live below the poverty line in Northern Uganda (64.8%) than in other regions in the country. For close to a quarter a century, the northern region experienced conflict as a result of armed rebellion by the Lord’s Resistance Army (LRA), which led to massive displacement of almost the entire population of the region. Conditions in internally displaced people’s (IDPs) camps showed one of the dark faces of the LRA conflict. High disease prevalence including HIV/AIDS, poor sanitation, poor access to safe water, education and health services all explained the effects of the conflict. Since the Peace talks in Juba between the Government of Uganda and LRA culminating into the “signing” of the Cessation of Hostilities Agreement (CHA- on 26th August 2006), and the eventual push of the LRA from Northern Uganda and Southern Sudan, a semblance of peace has returned to the region. This has allowed the displaced persons to return to their villages. At the Inception of this WASH Project, the return and resettlement of the formerly IDPs were steadily on course, a process largely that was largely achieved by the end of 2009. At the time IDPs were returning to their original homes, access to safe water and sanitation facilities in communities and schools featured among the leading priorities for resettling the returnees. Consequently development aid focused on increasing access to safe water and sanitation in the return villages away from the camps as well as schools in the community. By increasing access to safe water and improved sanitation, with safe hygiene education, the WASH Project aimed at contributing to reduced mortality and morbidity rates from diseases caused by use of unsafe water and poor sanitation in the general community. Further, through implementation of the WASH activities in schools, it was envisaged that pupils would not only access safe water and proper sanitation facilities while at school, but also act as change agents in their homes.

1.3 Purpose and Objectives 1.3.1 Purpose The overall purpose of this Evaluation was to undertake an in-depth assessment of the extent to which the Project achieved its objectives. 1.3.2 Specific objectives Specifically, this Evaluation was intended to:

1) Assess the extent the project’s objectives were met and factors that facilitated or constrained achievements of the planned activities

3

2) Identify the level of effectiveness and impacts of the project on its target groups and the people it worked with

3) Assess the efficiency with which the project was implemented to achieve results and objectives

4) Document two to three cases and lessons learnt in the implementation of the project for replication of good practices

5) Assess the sustainability of the project in relation to results, outcomes and impact, and identify which activities were the most relevant to ensure sustainability

6) Make recommendations to strengthen the scope of WASH initiatives in Project districts and Uganda in general

1.4 Scope of the Evaluation As provided in the terms of reference (ToR), this evaluation was based on the criteria of relevance (appropriateness of problem and need identification), effectiveness (achievement of purpose or the extent to which the WASH Project activities were implemented), efficiency (sound management and value for money), impact (achievements of wider effects with focus on the Project benefits translated into achievement of the four result areas), and sustainability (likely continuation of the achieved results).

1.5 Report structure This Report is presented in eight sections, which reflect the scope of work of the evaluation as contained in the detailed ToR. The first two sections give the context of the evaluation, objectives and the methodology used in undertaking this evaluation. Sections three to seven present the findings of this Evaluation as per the evaluation criteria, namely Relevance (Section 3) Efficiency (Section 4) Effectiveness (Section 5), Impact (Section 6) and Sustainability (Section 7). The last Section i.e., 8 highlights the good practices, lessons learnt through the implementation of the WASH Project. Further, in Section 8, implementation challenges and recommendations for sustaining and strengthening supported activities and for guiding future WASH activities in Uganda are provided.

4

2.0 EVALUATION APPROACH AND METHODOLOGY

2.1 Overall Approach A combination cross-sectional study design utilizing quantitative and qualitative methods was adopted in carrying out this evaluation. Quantitative methods of sample selection were used to obtain a sample of household respondents and pupils in schools, while qualitative methods were used to select secondary stakeholders at national, district and sub-county levels, as well as for school authorities/teachers.

2.2 Study Areas and Participants 2.2.1 Study areas The Evaluation was carried out in 10 out of the 21 project supported districts to collect data for ensuring an in-depth assessment of the extent to which the project achieved its objectives. The other 11 project supported districts were covered through an extensive desk review. Selection of the studied 10 districts was largely guided by the activities implemented across the project area and regional representation. The intention was to capture as much as possible a representation of the WASH supported activities that were implemented. Out of the 10 districts for household coverage, five had WASH components implemented in primary schools (P.S). Thus, in each of these five districts, a primary school that benefited from the WASH Project was covered. See Table 1.

Table 1: Distribution of household respondents by district and sub-county District and sub-county for household participants District and schools

District Sub-county District School

Abim Bundibugyo Dokolo Gulu Kabarole Kasese Katakwi Lira Moroto Soroti

Lotuke Ntotoro Dokolo Awach/Paicho Kicwamba Lake Katwe Usuk Ngetta Nanduget Arapai

Bundibugyo Gulu Kasese Katakwi Lira

Namugongo P.S Awach P.S Katunguru P.S Usuku P. S Boke P. S

2.2.2 Study participants Study participants included both primary and secondary project stakeholders. Primary stakeholders were direct beneficiaries who included household members in communities and pupils. To some extent members of the WSCs, school authorities and district technical staff from departments of water, health and education as well as extension staff at sub-county level were also direct beneficiaries. Secondary stakeholders included Sub-county Chiefs, Local Council (LC) III chairpersons, LCV chairpersons or their deputies (See Appendix 2 for a detailed list of evaluation participants). These (secondary stakeholders) were particularly targeted to provide an overall Project assessment in their respective their districts, the challenges and lessons

5

learned during implementation, the relevance of project activities to the priority needs of the populace and their level of participation, level of efficiency and effectiveness in implementation. They also helped in documenting the impact made by the project in their districts and sub-counties towards increased access to integrated WASH services.

2.3 Sample/Study Participant Selection Procedure Selection of study participants was through random and non-random procedures. Non-random selection was used to select secondary stakeholders based on their role in the implementation of the WASH Project while random selection was used for selecting respondents who participated in household interviews and pupils in schools. In each selected sub-county where the WASH Project was implemented, two sites were randomly selected. At the site level, the sampling frame comprised of all households in the village. With support of the village leadership, the Evaluation Team ascertained the number of households in the village and used it to determine the sampling interval. In each site/village, 25 households were randomly sampled i.e., a total of 50 households per district. At the selected household, one eligible respondent was picked for participation in the survey; either, the household head or his/her spouse. In instances, where both the head and the spouse were found absent at the time of the visit, any other adult member of the household was recruited into the study. As shown in Table 2, a total of 504 respondents were covered. Table 2: Distribution of household respondents by district and sub-county

District Sub-county Community respondents (N=504)

N %


Lotuke Ntotoro Dokolo Awach/Paicho Kicwamba Lake Katwe Usuk Ngetta Nanduget Arapai

51 54 50 49 50 50 49 50 50 51

10.1 10.7 9.9 9.7 9.9 9.9 9.7 9.9 9.9

10.1

Total 504 100.0

In order to obtain primary data on school sanitation facilities, and assess the impact of the WASH Project on primary school pupils’ sanitation and hygiene practices, a minimum sample of 30 pupils was selected in each of the five schools. The WASH Project targeted school pupils because of their potential as “knowledge centre/change agents” for neighbouring communities. The schools visited and distribution of pupils interviewed is shown in Table 3. Table 3: Distribution of interviewed pupils by district and school

District School Pupils interviewed

N %

Bundibugyo Gulu Kasese Katakwi

Namugongo P.S Awach P.S Katunguru P. S Usuku Boys P. S

30 30 30 30

19.9 19.9 19.9 19.9

6

Lira Boke P. S 31 20.5

Total 5 151 100.0

Apart from covering pupils as possible change agents of proper sanitation and hygiene practices in their homes and communities, it was the intention of this evaluation to assess the extent the WASH Project impacted on individual pupils’ behaviors and practices with regard to personal hygiene and sanitation.

2.4 Data Collection Methods A mix of quantitative and qualitative methods of data collection was deemed appropriate for this evaluation. A team of 18 university graduates interviewers and two field supervisors was constituted and trained to support the consultant in collecting data. Qualitative data, mostly obtained from national, district and sub-county level participants was collected by the Consultant assisted by the Field Supervisors. The interviewers mostly supported the Consultant at the community level; they administered the individual interviews with eligible respondents within selected households. They also moderated and took notes in all discussions held with community level people including members of the WSCs. All interviews and discussions at the community level were conducted in local languages. 2.4.1 Quantitative methods Quantitative data was collected using structured questionnaires and observation checklists from direct beneficiaries of the WASH project i.e. water users, school pupils, WSC members and local artisans. As shown in Appendix 2, there were four structured questionnaires i.e. one for household members, another for the school pupils, the other two were for members of the WSC and HPMs respectively. Observation checklists were used to collect data on the sanitation and hygienic conditions of sampled households and schools. Focus in the observations was on the latrine and related sanitation facilities and practices. At the household level, issues investigated included access to safe drinking water, availability and use of sanitation facilities as well as their condition, household hygiene conditions and practices, changes in prevalence of water and sanitation related diseases, accruing benefits from access to safe water, improved sanitation and hygiene. This data was specifically collected to help in determining the level of effectiveness and the impact of the WASH Project on the targeted population. 2.4.2 Qualitative methods Qualitative techniques of data collection used in this evaluation, included in-depth interviews with district and sub-county level staff in the Directorates of Water, Health and Education as well as head teachers and their deputies in selected schools. These study participants provided insights on the extent to which the project’s objectives were achieved, factors that facilitated/constrained achievements of the planned activities, extent of project efficiency as well as enabling documentation of best practices and lessons learnt in the course of implementation. Sustainability issues were also explored through in-depth interviews. See Appendix 3 for interview guides. To corroborate data from both direct project beneficiaries and the secondary stakeholders that participated in this study, an extensive review of reports, policies and other program

7

documents such as the 4th Annual Progress Report-EUWF 14 Oct 2011, Addendum#2 EUWF UNICEF WASH August 2011, Annex III Revised for Addendum#2 Final EUWF UNICEF WASH 15 August 2011 as well as the EUWF Annex I Donor Proposal, was done. The review also enabled the consultant access data on the 11 project supported districts that were not visited for the evaluation. Photographs of project supported facilities such as water sources and sanitation were taken and have been incorporated in this report to provide a deeper appreciation of the project’s effectiveness and impact where possible. Review of secondary data mainly form relevant sector and project reports was undertaken to complement primary data. The evaluation team reviewed documents availed and others accessed from government archives (like Government of Uganda [GoU] documents on the WASH Sector). Document review was done both at inception phase and during data processing and analysis. The review of the documents informed the development of the inception report (IR) and the evaluation tools. See List of References.

2.5 Data Management and Analysis All completed questionnaires were checked for accuracy and consistency of recorded responses before entry into the computer. A data entry module was designed and developed using the Epidemiological Information (EPI-INFO Version 6.0). After cleaning, the data was exported to computer software programs; SPSS and Ms. Excel2 for analysis. Frequency tables and cross-tabulations were produced. On the other hand, thematic and content approaches were used to analyze all the qualitative data. This involved use of an analysis grid with themes reflecting the Evaluation objectives to delineate salient comments and explanations. All data sources have been triangulated to produce this evaluation report. Since the Project used data in official Government of Uganda (GoU) sector documents as benchmark/baseline data, this Evaluation adopted the same approach. The Project Proposal Document adopted the MoWE Sector Performance Report (2005) to extract baseline figures for WASH in targeted districts. This Evaluation has, therefore, where possible used similar documents to track changes in access to safe water and sanitation and achievement of particular targets. The Sector Performance Report for 2011 has been used to help in tracking progress and achievement of targets.

2.6 Analysis of the Socio-Demographic Characteristics of Household Respondents and Pupils

2.6.1 Household respondents Analysis of the socio-demographic characteristics of the individual household respondents (summarised in Table 4) shows that the majority (65.3%) of household respondents were females while about a third (34.7%) were males. Attempts to have an equal representation of males and females in the sample were made, but in majority of households only women were found available at the time of the visit. However, given that the females bear the major burden of collecting water for domestic use and are vanguards of household sanitation and hygiene practices, this was a plausible justification for the excess female sample. Three quarters (74.8%)

2 This was at the request of UNICEF i.e., to have a dataset in Excel.

8

of the sample indicated that they were in stable relationships (married), where the fetching water, cooking, washing clothes and bathing children are traditional gendered roles assigned to the women in all parts of rural Uganda. See Table 4.

Table 4: Socio-demographic characteristics of household respondents

Characteristics (N=504) Respondents

n %

Gender Male

Female

175 329

34.7 65.3

Education level

None Primary

Secondary –O’ level Secondary – A’ level

Post secondary

141 291 56 5

10

28.0 57.9 11.1 1.0 2.0

Occupation Peasant farmer

Salaried worker Business / commercial

Casual worker Bar Operator

Student Others

422 17 38 13 2 6 6

83.7 3.4 7.5 2.6 0.4 1.2 1.2

As per the Table 4, more than half the sample had attained only primary education, while almost a third had never been to school. This literacy level may not augur well for the predominantly female sample, given that women are required to take key positions in the WASH management structures. Further, that majority of the sample reported to be depending on farming has a bearing on participation of women in maintenance of WASH facilities, especially, where the men could be reluctant to make financial contributions for operation and maintenance (O&M) of WASH facilities. Thus, where women have little or no control over the proceeds of their agricultural labour, their ability to influence O&M could be potentially undermined. 2.6.2 School pupils By design, majority of the pupils covered in this study were selected from primary 5 to 7. As Table 5 shows, slightly over a half being females.

Table 5: Characteristics of schools and pupils interviewed

Characteristics

Pupils

N %

Type of school visited

Day school – mixed Day school – single sex

119 30

79.9 20.1

Ownership status of schools visited

Government school Private school

121 30

80.1 19.9

9

Sex of the pupils

Male Female

73 78

48.3 51.7

Age of the pupils 10-12 years 13-15 years 16-18 years

23 99 29

15.2 65.6 19.2

Class of pupils interviewed

Primary 3-4 Primary 5-7

5

145

3.3

96.7

Pupils interviewed in the five schools were predominantly those in upper primary i.e. primary 5 to 7. These were chosen because of their ability to express themselves. Besides, these were the children who were in involved in the hygiene and sanitation promotional activities in their schools.

10

3.0 RELEVANCE 3.1 Introduction Relevance refers to the extent to which the objectives of the Project were consistent with the beneficiaries' requirements, country needs, global priorities and partners' policies. The analysis of Project relevance in this Evaluation addressed inter alia the issues as shown in Box 1. Box 1: Criteria for project relevance The extent to which the project was consistent with, and supportive of, the national policy and

programme framework.

The Projects' coherence with current and ongoing initiatives

The quality of the problem analysis of the Project's intervention logic and logical framework matrix, appropriateness of the objectively verifiable indicators

The extent to which stated objectives correctly addressed the identified problems and social needs

The quality of the identification of key stakeholders and target groups

The stakeholder participation in the design and in the implementation of the project.

The appropriateness of the recommended monitoring and evaluation (M&E) arrangements

3.2 Project Consistence The relevance of this Project was partly evaluated in as far as it was deemed consistent with and supportive of Government of Uganda (GoU) policies such as the PEAP, and later the NDP 2010/11-2014/15, WASH sector policies, EC Country Strategy Paper, and MDGs. Before the implementation of this WASH Project, UNICEF had a long tradition of support to GoU in the WASH sector, the Education and Health sectors. This European Union (EU) funded WASH Project was therefore a component of UNICEF WASH intervention in Uganda. UNICEF’s goal of reducing morbidity and mortality through increasing accessing to safe water and sanitation is a notable contribution to achieving the core goal of the EU, namely economic and social development and fighting poverty in developing countries, Uganda inclusive. Further in its (Uganda) Country Strategy Paper and National Indicative Program, 2008-2013, the EU pledges to support Uganda’s efforts to attain the MDGs. This pledge is firmly grounded in the third of the three pillars of the EU Strategy for Africa agreed upon in 2005, which prioritizes, improving access to basic social services and protection of the environment to reach the MDGs. To this end, the WASH project rhymes with the EU country strategy for Uganda. Secondary and primary data indicate that the WASH Project was highly relevant as it aimed at contributing to the realization of the national WASH goals and objectives enshrined in the then PEAP, and later the NDP. The relevance of the WASH Project is reflected in the importance that GoU and its Development Partners (DPs) accord to WASH activities. This importance is firmly reflected in key GoU ministries that prioritize WASH, namely MoWE, Ministry of Health (MoH), Ministry of Education and Sports (MoES), Ministry of Local Government (MLoG), Ministry of Gender, Labour and Social Development (MoGLSD) and Ministry of Finance, Planning and Economic Development (MoFPED). At the Local Government (LG) level—district and sub-counties, WASH sector attracts one of the most popular intervention areas for NGOs.

11

In terms of the geographical scope, the WASH Project complimented GoU programs such as the Peace, Recovery and Development Program (PRDP) and NUSAF II both intended to support the Northern region to recover from the effects of the conflict that ravaged the region for more than two decades.

3.3 Project Coherence with Current Initiatives of GoU and Sector Partners The WASH Project was conceived within the overall national policy and planning framework. The purpose and objectives of the Project were intended to contribute to the WASH national objectives. For instance, promotion of hand-washing with soap especially after latrine use has for long been targeted by various sectors. At the district level, hand-washing campaigns have for years been promoted by the Directorates of Health, particularly the health inspectorate, Community-Based Services, Education, and even Works; Water Office falls in the later directorate. Through the MoES, the School Facilities Grant (SFG) has been used to finance construction of sanitation facilities in school. Further, the MoH hosts the Uganda Sanitation Fund which uses the demand-driven approach to promote better sanitation and hygiene practices across the country. Some of the WASH Project supported districts such as Soroti are beneficiaries under the Fund. Overall the Fund supports 16 districts. Apart from GoU, several sector partners including non-government organizations (NGOs) are involved in implementation of WASH activities which, among other outcomes, provides opportunities for synergies that can be prudently exploited. Across the country, NGOs have been supporting the WASH sector, by mobilizing funds for the sector, training and supporting water and sanitation infrastructure development. In addition, the NGOs are involved in capacity building of communities to demand for water sources, develop, operate and maintain WASH facilities. The UNICEF Strategy entered into partnership with the existing NGOs involved in WASH activities to carry out implementation of some of the software and hardware activities. See Table 6 Table 6: Activities engaged in the various Sector partners and districts of operation Sector partners Implementation area/location Activities engaged in by sector partner

Co-operation and Development (C&D)

Operates in 7 districts of Karamoja

Drilling and rehabilitation of boreholes

Training of school clubs and SMCs

Formation and training WSC in O&M

Arbeiter-Samariter-Bund Deutsch Land (ASB)

Agago, Pader, Lamwo (Acholi), Soroti, Serere, Amuria, Katakwi, Kaberamaido (Teso)

Training of school clubs and SMCs

Formation and training WSC in O&M

Dissemination of information in communities on sanitation and CTLS

Source: Interview with UNICEF staff In order not to alienate the districts from NGOs due to the partnership with UNICEF, and to further ensure ownership of NGOs implemented WASH activities by the district, UNICEF provided support to all LGs to monitor the work of NGOs. Synergies were further identified in

12

A privately owned hand pump that has a watering trough for animals, Usuk Sub-county, Katakwi

the NGOs work operating in the Project areas. In 33 schools in the districts of Katakwi and Amuria, the ACDI/VOCA3 School Feeding Program shares the same objective of increasing enrollment, retention and completion rates in primary school by providing food to the pupils. Schools with such ACDI/VOCA Feeding Program found the UNICEF supported activities to provide water to the school, a relevant contribution to the efficiency of their program.

3.4 Clarity and Internal Consistency of the Project Objectives Clarity and consistence of Project objectives entails the extent to which stated objectives correctly addressed the identified problems and social needs. The implemented WASH activities were directly derived from the district development plans and work plans. The objectives addressed the real problems of the targeted areas. For instance, in Northern Uganda, people were just returning from IDPs to villages without safe water sources and sanitation facilities. Schools and health facilities had been re-opened, but overwhelmingly lacked adequate access to safe water, sanitation and hygiene facilities and supplies at the time. Project support towards construction of latrines and safe water sources mainly targeted institutions such as primary schools, local government (Sub-county) headquarters and health units. In terms of schools, through the project’s interventions, the objective was to boost enrollment, retention and completion rates especially of the girl child by improving the school learning environment. Location of WASH services at health units was intended to target mothers and children using such facilities. Whereas most other development partners target the community as the direct beneficiaries, the UNICEF strategy in this WASH Project, and particularly on the issue of improved hygiene and sanitation practices, was to have the institutions as primary beneficiaries and the surrounding communities as secondary beneficiaries. Important to note, UNICEF also has community WASH programs, in which communities are targeted directly, indeed in this WASH Project, increased access to safe water was directly planned for communities as primary beneficiaries. Where O&M roles of the community and the institution were clearly understood, respected and performed, this

approach of benefiting both the community and the institution was a prudent one. However, primary data for Evaluation revealed some untargeted WASH needs which, community members thought were critical as well and the Project could have helped to solve. It was revealed, for instance, that apart from the community need for safe water, communities in water stressed areas also strive to access water for their livestock,

3 In September 2010 ACDI/VOCA was awarded a grant from the United States Department of Agriculture to implement a

three-year Food for Education program in northeastern Uganda. As part of the Literacy Enhancement and Rural Nutrition

(Uganda-LEARN) program, ACDI/VOCA works with schools, Parent-Teacher Associations (PTAs), School Feeding

Management Committees (SFMCs), and local communities to deliver a comprehensive package of school feeding, school

gardens, community training, and infrastructure rehabilitation.

13

mainly cattle. In cases where water for cattle is hardly available, recourse is to the available sources for domestic water needs. There was evidence of constructed water sources which also doubled as sources of water for production. In Kasese, the GFS tap stand was used to water cattle and support a brick-making venture. Similar cases where animals access water sources constructed for domestic use were noted in Katakwi. The incidence of occurrence of such cases is higher during the dry season, when the usual drinking points for livestock dry up.

3.5 Stakeholder Participation Stakeholder participation is traced in the involvement of beneficiaries in the design and implementation of the WASH Project, the level of local ownership and implementation capacity (at school, community, sub-county, district, and national levels). The Inception Phase of the WASH Project was primarily for stakeholder consultation whose one of the major outcomes was a revision in the project design reflected in the addition of a fourth result area. The new result area focused and emphasized participation of a range of stakeholders and beneficiaries in implementation of project activities. According to stakeholders interacted with at district and sub-county level, they actively participated in project planning and implementation phases, thereby strengthening the institutional position of UNICEF-GoU relationship. Within the context of decentralization, such an approach by UNICEF goes a long way in supporting GoU policies and strategies for good governance and poverty reduction, and generally efforts to meet the MDGs. Across the 10 studied districts, WASH officials revealed that consultation on the general technical design of the project did not happen, but rather consultation on some specific aspects such as the most in need sub-counties, schools and health units in the districts to be targeted. But this lack of consultation on technical designs was explained by UNICEF that, technical design depends on type of water sources to be constructed as well as the options taken by the community. In the case of the WASH Project, all water sources opted for did not require any consultations on the technical designs to be adopted. The case of indirect beneficiaries’ participation started right from the planning of activities. District staff revealed that UNICEF always engaged them in the planning and setting of priorities. Prior to any activity, district staff from the water office and health would be provided with a template for the work plan. It was in this template that the district detailed the activities to be implemented, where to implement them, who to involve and incase of latrines or water sources – the number to be constructed. The work plans would be sent back to UNICEF for approval before the release of funds for implementation of project activities. All software related activities such as training workshops, Community Led Total Sanitation (CLTS) campaigns/sanitation improvement campaigns aimed at triggering communities to become Open Defecation Free (ODF), formation and training of school health clubs, roll-out and dissemination of information on sanitation and CLTS, supervising construction/installation of water sources and latrines were directly handled by either the District Water Office or the District Health Inspector. UNICEF limited itself to procuring contractors and participating in joint monitoring activities and advocacy events. Participation of the intended beneficiaries in the intervention greatly fostered ownership and effective implementation of activities. Staff in the Project supported districts acknowledged that

14

due to the fact that activities would be incorporated in their work plans and directly drawn from the Development Plans, it enabled them implement the activities appropriately. There were, however, a few areas where participation of LGs was non-existent or minimal such as procurement of the contractor. This was especially in procurement of contractors for civil works, which was centrally managed by UNICEF. The LGs, however, played a monitoring and supervision role over the contractor. To some extent this limited the capacity of LGs to enforce certain conditions since the Contractor would be less answerable to them. The most cited case was compliance with the DWD Guidelines which requires community mobilization in the pre-civil works Phase for construction of water sources that would not be done, but instead performed after the civil works. .

3.6 Appropriateness of the Recommended Monitoring and Evaluation (M&E) Arrangements

In all the districts visited, it was acknowledged that M&E visits promoted noticeable participation of stakeholders in implementation phase. Field visits which preceded the M&E meeting were always attended by sub-county and lower level leadership. Sites where Project activities were implemented benefited from regular visits at different times by a team comprising of a UNICEF Focal Officer in the region, the DWO, District Health Inspector (DHI), sub-county leadership, and community leadership including School Management Committee (SMC) and Health Management Committee (HMC). M&E meetings allocated two days; one (1) for field visits, and another for discussing findings. The M&E arrangement increased visibility of the funding agencies in the communities due to their involvement in the field visits.

What used to happen is that UNICEF staff would inform us about the nature of activity they wanted us to put in our work plan. So we would do all the planning work, for instance, if it is construction of boreholes, using their template, we would plan for the number of sources we wanted, where to locate them, the pre-mobilization meetings, and supervision of the contractor. We would send the work plan to UNICEF who would in turn review and whatever they would retain is what would be implemented. UNICEF would procure the contractor and for us as the district, we would handle all the software related activities like training of people. District Water Officer, Dokolo.

15

4.0 EFFECTIVENESS 4.1 Introduction The effectiveness criterion refers to the extent the Project's results were attained, and the Project's specific objectives achieved. See Box 2 Box 2: Criteria for project effectiveness The extent the planned benefits were delivered and received, as perceived by key stakeholders

Whether the intended beneficiaries participated in the intervention.

The extent the changed institutional arrangements and characteristics produced the planned improvements

4.2 Delivery on Planned Benefits Planned benefits were of hardware nature such as construction of sanitation and water facilities, and of software nature that involved capacity-building of LGs through workshops/seminars as well as activities geared towards promotion of hygiene and sanitation practices and behaviors. These activities targeted both the communities and institutions—primary schools and health centres. 4.2.1 Sanitation facilities Several activities were planned and executed as part of efforts to promote sanitation, key of which was to support the construction of latrines in institutions and demonstration latrines in communities. The WASH Project designed and constructed model/demonstration latrines mostly at schools and health centres. A total of 94 latrine blocks i.e. separate 5-stance latrine blocks for boys and girls were constructed at primary schools in target districts benefiting over 10,000 children, and in 18 Early Childhood Development (ECD) centres, benefiting approximately 2,700 children. An additional 55 separate latrine blocks for males and females were constructed at 13 health centres in the project supported districts. The choice of schools was to improve the pupil-stance ratio because apparently crowding at toilets leads to lowered attendance and absenteeism from school for especially the girl child. The Project also supported creation of capacity among locals (local artisans) in construction of appropriate latrines. Local artisans and partner NGOs/PSOs in the districts of Amolatar, Lira and Oyam were supported to construct 52 appropriate demonstration sanitation facilities at central locations i.e. 16 in Amolatar, 20 in Lira and 16 in Oyam, benefiting around 1,040 people. In total 201 5-stance latrine blocks were constructed with support from UICEF (4th Annual Progress Report 2011). The Project supported construction of waste disposal facilities at 16 health centres i.e. 8 in Amolatar and another 8 in Lira District, hence improving the handling of medical wastes in an environmentally friendly manner. From the above Table, it is clear that a big number of sanitation facilities were constructed at institutions and demonstration latrines in communities; achieving effectiveness rate of 86% for institutional latrines and 84% for demonstration latrines in communities. The Evaluation findings revealed that other institutions (health centers) which had not been considered in the plan were supported during the period of project implementation. The process of design and construction of sanitation facilities for primary schools was implemented

16

well. This involved construction of at least a five-stance latrine blocks with adequate privacy and hand-washing facilities in targeted primary schools.

With regard to planned software activities such as sensitizations, results show tremendous achievements. For instance, the project supported sensitization of political, religious and cultural leaders in the 21 districts on the benefits of and approaches to safe excreta disposal. This was done through advocacy workshops where leaders were provided with Sector Guideline Documents—e.g., the Updated Sanitation and Hygiene Promotion Guidelines, Ethikwini Declaration, International Year of Sanitation Action Plan and the National Water Quality Standards. Stakeholders talked to at district and community levels acknowledged that the sensitized leaders together with sub-county extension staff and NGO staff used various fora to sensitize community members on the importance of safe excreta disposal, hand-washing and ensuring safe water chain management. This was mostly accomplished through home and village visits/community meetings, targeted drama shows and radio programmes. Through project implementation, local artisans, staff of partner NGOs and private sector organizations (PSOs) were trained in improved construction skills for hand dug wells and sanitation facilities. As a result of the trainings, a total of 52 demonstration sanitation facilities were constructed. The only limitation was that the trained artisans were not provided with construction kits as planned. This limitation, notwithstanding, households were triggered through the sanitation improvement campaigns to construct their own latrines using local materials. Thus, majority of households (71%) in the Project had latrines. Planned activities which were not implemented included the formulation and dissemination of a Sanitation Marketing Strategy and an Outreach Plan with DHIs, DWOs, District Community Development officers, District Health Educators, sub-county extension staff, local leaders, ministry of health officials, NGOs and PSOs. According to UNICEF staff, the Sanitation Marketing Strategy and an Outreach Plan were supposed to be spearheaded by the MoH, which never happened. The role of the Project, therefore, would have been to support stakeholders in using the Strategy and implementing an Outreach Plan. On a positive note, however, a committee has been set up to work on the Sanitation Marketing Strategy (it was constituted as of 31st July 2012). Once the strategy and outreach plan are completed, UNICEF will support its implementation. 4.2.2 Hygiene practices The project planned and implemented a number of activities as part of measures to improve hygiene in targeted communities—food hygiene, safe water chain and hand-washing with soap. Reports (4th Annual Progress Report 2011) indicate that a total of 197 primary schools in the districts of Dokolo, Oyam, Soroti, Katakwi, Bundibugyo, Kyenjojo, Kasese, Lira, Gulu, Amuru, and Apac benefited from the hand-washing demonstrations. A total of 4,358 pupils and 117 teachers in Oyam, 13,679 pupils in Lira, 21,339 pupils and teachers in Apac, 750 pupils in Soroti, benefited from these demonstrations. For the case of Dokolo, intervention was through training of school health clubs. In addition to demonstrations, sanitation kits comprising water cans, hand washing facilities, laundry soap, slashers, wheel barrows, hoes, plastic basins, hard brushes, protective gloves, dust bins, pangas, water buckets, axes and detergent were procured and supplied to 23,789 rural households in Gulu, Amuru, Abim, Kaabong, Kotido, Moroto and Nakapiripirit Districts, and 309 primary schools (about 155,000 children) in Apac, Amolatar,

17

Dokolo, Lira, Oyam, Amuru, Gulu, Kitgum, Pader, Abim, Kaabong and Kotido Districts. All the five (5) schools visited during the Evaluation had hand-washing facilities; Awach Primary School in Gulu, had as many as five hand-washing cans strategically placed in various corners of the school compound. Items such as slashers, wheel barrows, hoes, plastic basins, hard brushes, protective gloves, dust bins, pangas, water buckets, axes and detergent were specifically procured and supplied to schools and health centers, no household is reported to have received any of these items. Households were given water cans and laundry soap. A Training Module was designed to ensure effective delivery of hygiene education. District and sub-county authorities particularly health inspectors and assistants were trained in Participatory Hygiene and Sanitation Transformation (PHAST) approaches. The trainings were carried out in 2008 and 2009 covering 8 districts in Northern Uganda, 3 from North-Eastern part i.e. Amuria, Katakwi and Moroto as well as Bundibugyo in Western Uganda. Districts were supported in the preparation of Sanitation Ordinances and Bye-laws. Model villages were identified through sanitation competitions by Health Assistants in the various project supported districts and publicized to attract community leaders from the poor performing sub-counties and villages to come and learn from the excelling villages. Hygiene education outreaches as well as co-curricular activities that targeted hygiene promotion in schools were implemented as planned. These were implemented mostly through health clubs and inter-school drama competitions. A total of 300 primary schools in the Project area formed health clubs to undertake hygiene promotion activities among their peers at school. In addition to forming and training members of the school health clubs, the Project supported initiation of hand-washing demonstrations and campaigns in several primary schools; aimed at further impacting on the pupils’ hygiene practices and those of the neighboring communities. Reports (4th Annual Progress Report 2011) reveal that, for instance, in Soroti, all the 25 schools which attended training on WASH activities installed hand-washing facilities. Among these, 20 schools had health and hygiene plans while in 16, club meetings were held regularly as evidenced by minutes. At the district and lower levels, advocacy workshops/awareness meetings were held with both district and sub-county political and civic leaders responsible for resource allocation to further emphasize the importance of sanitation and hygiene. These were particularly targeted to ensure that they considered hygiene education among their priorities when drawing budgets. In addition, a learning exchange visit was organized for district staff from the districts of Gulu, Amuru, Oyam, Dokolo, Apac, Amolatar, and Lira to Rakai and Lyantonde to learn lessons on how to adopt and promote household sanitation improvement campaigns. What was not implemented, however, was the review of reports on hygiene behavior including conducting hygiene behavioral studies to identify behavior patterns, attitudes and practices. Given the abundance of data on hygiene behavior possibly it was found not cost-effective to implement this activity.

Of all the regions where the Project was implemented, the Karamoja sub-region registered mixed results for sanitation and hygiene. Whereas there are reported impressive strides in the open defection free (ODF) campaigns, slow adoption of better sanitation and hygiene practices is partly manifested in the outbreak of cholera in the sub region. In 2010, cholera was reported

18

Members of the Evaluation Team checking for functionality of a borehole constructed with UNICEF funds in Achutome, Gulu district

in Moroto, the worst hit being the sub-county of Nadunget, which is located at the foot of the drainage from Moroto municipality. 4.2.3 Access to safe water for new users According to the Uganda Water and Sanitation Sector Performance Report (2005) i.e., baseline figure, districts in the project area especially those in Northern and Eastern Uganda were the least covered with rural safe water. In order to address this need, several activities were planned and to a large extent implemented during the project period, and hence achieving tremendous success. See Table 7. Table 7: Planned and implemented water activities

Activity Number Planned

Number Achieved

Remark

Siting and construction of deep boreholes 190 200

Siting and construction of shallow wells 152 22

Installation of rain water tanks at schools 86 8

Extension of existing GFS/Rehabilitation of GFS

8 9

Rehabilitation of deep boreholes 100 96

Motorized/Production wells 9 5

Training water and sanitation committees 523 230 For old and new sources

Training hand pump mechanics 15 51

Training local artisans in hand-dug wells 12 22

Procure tools for repair of boreholes - 271

Source: Addendum No 2 for Planned Activities and 4th Annual Progress Report (2011)

There were activities where the Project achieved more than planned e.g., siting and construction of boreholes. It was clarified by UNICEF that the activities, which had been planned during the first Addendum were altered in the second Addendum with more focus on construction of water and sanitation facilities—i.e., hardware. Accordingly, a new water technology of motorized wells/production wells was introduced instead of supporting construction of several hand pumps especially the shallow wells. A single motorized/production well has eight tap stands with each serving a total of 150 people, which increases access to a larger population. Thus, eight (8) tap stands of a single well serve a population that would have been served by four (4) boreholes, which demonstrates a high level of effectiveness and efficiency with which the Project was implemented.

Members of the Evaluation Team checking for functionality of a borehole constructed with UNICEF funds in Achutome, Gulu district

19

As per the Project objective, new water users were reached through construction of boreholes, motorized wells and extension of the GFS as well as installation of rainwater harvesting systems at primary schools. Through sub-county staff, communities were mobilized to identify sites where new water sources would be constructed and also to participate in the construction works. The choice of technology used was mostly determined by the targeted users through advocacy/introductory meetings both at the sub-county and beneficiary village. Pre-construction planning and advocacy meetings were managed by the District Water Officers (DWOs) in the 21 Project districts. To inculcate the spirit of community operation and maintenance (O&M) of newly constructed or rehabilitated water sources, O&M workshops were conducted as planned for district politicians and members of the District Water and Sanitation Coordination Committee (DWSCC). Through such workshops members of DWSCCs were equipped with skills to conduct quarterly monitoring visits of water facilities. For O&M works at the water source, capacity was either strengthened or created among members of the WSC, caretakers and HPMs. Over 230 WSCs were trained in preventive maintenance of the water sources while 51 HPMs were equipped with skills to handle repair of the water sources. The trained HPMs, under their association of HPMs, were provided with tool kits; a total of 271 sets of tool kits were procured and distributed to the 21 Project supported districts. Extension staff from various project supported districts also received training in water quality assessment and monitoring. All Project supported districts are undertaking water quality testing as a result of Project support. Water users revealed that whenever they suspected cases of contamination at their water source they promptly report. Majority of water sources constructed with support from UNICEF are still functional and used by the targeted population. 4.2.4 Institutional development At the institutional level, UNICEF planned to facilitate production of lessons’ learned books and flyers, operations research where need arose for instance in water quality, review of policies, guidelines and standards where need arose, workshops and seminars to sensitize/train the various stakeholders (e.g. political, religious and cultural leaders at the district and lower levels, local artisans and HPMs, WSCs) and advocate for WASH services. UNICEF also planned to facilitate 17 radio talk shows per quarter, 41 radio spot messages per month, production of various promotion materials and 12 learning exchange visits across the region. In order to achieve its plans, the project supported the Directorate of Water Development (DWD) in its efforts of training district staff in data collection and mapping of water resources. Untrained/new staff in both the old and the newly created districts were trained in data collection and mapping of water resources using Geographical Information System (GIS). District staff such as the Health Inspectors and Health Assistants were trained in GIS techniques, as well as household level data collection, management, analysis and report writing. Mapping of existing and new water facilities was done in the districts of Apac, Dokolo, Lira, Kaabong, Bundibugyo, Kabarole and Kyenjojo. This assisted in proper targeting of interventions and rational resource allocation. District water and extension staff including health assistants in 15 project supported districts were trained and supported to procure test kits for testing water quality. A total, of 166 district staff i.e. 35 from Gulu, Dokolo (15), Lira (20), Bundibugyo (26), Kasese (35), and a combined 35

20

staff from the districts of Kapchorwa, Kween, Bukwo, Bududa, Manafwa, Pallisa, Mbale, Butaleja, Sironko and Budaka, were trained with support from the WASH Project in water quality assessment and monitoring (4th Annual Progress Report 2011). Review of policies has also been undertaken; the WASH Project supported finalization of a strategic implementation plan for the School Health Policy, and Guidelines for O&M of School WASH Facilities. The project also supported the process of updating the national Water Resources and Water Quality Guidelines, to better guide sector implementation. IEC materials on water quality management were developed for district use in promoting better management of available water resource utilization by different stakeholders. And at a national level, support was given to printing of the updated Uganda Water Supply Atlas in 2010. Within the 21 project supported districts, 17 districts were supported to strengthen the functioning of their DWSCCC. Lessons’ learned books and flyers on various WASH related aspects were produced and disseminated; for instance, during the 2010 Joint WASH Sector Review, 500 copies of a book on the “districts’ positive experiences and lessons learned regarding the use and enforcement of sanitation by-laws, providing insight into best practices and potential risks”, were given out to participants. Further, Radio spot messages and talk shows with key messages on hand-washing were broadcast as planned especially in Gulu and other districts in the region (4th Annual Progress Report 2011).

4.3 Beneficiary Participation in the Intervention As part of efforts to foster ownership of the WASH services, the Project enlisted involvement of both direct and indirect beneficiaries in the various activities. At the community level, beneficiaries participated in various ways including identification of three (3) possible locations for construction of water sources on the basis of accessibility. Except in situations where the water table was not appropriate for the kind of technology that alternative locations were sought by the contractor. Similarly, in institutions (i.e., schools and health facilities), head teachers and in-charges respectively were consulted on their priority WASH needs. Choice of schools and health facilities to support was based on reports submitted by head-teachers and in-charges to the DEO and DHO respectively. Beneficiary participation was further assessed by the extent households in targeted communities were involved in the construction of the water source and O&M. More than a third of household respondents in sampled communities reported that their households participated in the construction of water sources while over two-third (71.7%) indicated participating in O&M of the constructed facilities. There were district differences in the level of participation with Abim scoring the least on the two aspects as shown in Table 8. Table 8: Household participation in construction and O&M of the water facilities District Household participation

Participated in construction (N=168) %

Participates in O&M (N=360) %

Abim Bundibugyo Dokolo Gulu Kabarole

5.9 41.0 36.7 51.1 37.8

43.1 68.5 88.0 85.7 96.0

21

Kasese Katakwi Lira Moroto Soroti

12.5 6.0 58.0 60.0 57.8

70.0 92.0 56.0 53.1 65.3

38.0 71.7

Table 8 shows that most beneficiaries are involved in O&M compared to participation in source construction. Nevertheless, this shows enhanced chances to sustain the installed facility. Households that participated or made a contribution towards source construction provided local materials, labour for cleaning the surroundings etc. Majority (63.9%) of those who participated in O&M reported to be contributing money. The low rates of participation depicted in the construction of water sources are mostly attributed to two factors; (i) transition from IDPs to the villages and, (ii) low capacity within the districts to mobilize communities. It has to be noted that in most of the communities where the project activities were implemented, most of the population was in transition from IDPs. Therefore, it is only the few people who were already in the villages that participated in the construction of water sources. Further, districts especially in the Karamoja sub-region have limited capacity and with very few NGOs engaged in WASH related activities that can mobilize communities for WASH activities. Table 9: Household participation in construction and O&M of water sources

Nature of participation (N=360) Respondents

n %

Nature of contribution /participation for O&M Money

Cleaning surroundings Cleaning source Local materials

Others

230 174 118 16 35

63.9 48.3 32.8 4.4 9.7

There were also district and gender differences with regard to nature of contribution towards O&M as Table 10 shows. Table 10: Type of contribution towards O&M by gender and district Characteristic Type of contribution (N=360)

Money %

Cleaning source Surroundings %

Cleaning source %

Local materials %

Other %

Gender

Female Male

58.5 66.5

54.2 45.5

31.4 33.5

5.1 4.1

12.7 8.3

District Abim Bundibugyo

50.0 43.2

31.8 40.5

22.7 51.4

0.0 18.9

18.2 0.0

22

Dokolo Gulu Kabarole Kasese Katakwi Lira Moroto Soroti

40.9 71.4 100.0 17.1 100.0 60.7 46.2 81.3

72.7 47.6 2.1 71.4 56.5 82.1 38.5 46.9

54.5 23.8 2.1 54.3 13.0 42.9 42.3 34.4

2.3 0.0 0.0 0.0 2.2 0.0 19.2 6.3

22.7 16.7 2.1 2.9 8.7 10.7 3.8 12.5

63.9 48.3 32.8 4.4 9.7

Overall, at the community level, participation of beneficiaries in O&M is visible in the contribution of money if a source breaks down. It was reported in group interviews and key informant interviews that WSC mobilize households to contribute for O&M when a source breaks down as well as cleaning the sources.

4.4 Influence of Institutional Arrangements on Project Effectiveness

Various changes were made in the institutional arrangement between targeted districts and UNICEF, as part of measures to enhance Project effectiveness. The changes ranged from provision of technical focal persons, procuring contractors for the case of water sources and latrines to implementation of Project activities in an integrated manner. Evaluation results show that these changes in institutional arrangements and characteristics greatly contributed to Project effectiveness. For instance, allocation of a technical person from UNICEF to all Project supported districts ensured access to specialized skills. Whenever, there were activities that required specialized skills, UNICEF procured and provided the districts with consultants to assist without unnecessary delays. Procurement of service providers for water source and pit-latrine construction being the responsibility of UNICEF ensured quick access to competent and reputable contractors. Whereas such an approach would limit growth in capacity of the districts, it greatly ensured good workmanship. The Project was implemented in remote sites that are usually unattractive to good contractors but due to the fact that procurement was done centrally by UNICEF, all the targeted rural sites benefited from the services of highly competent contractors. Also cases of funds getting diverted to other development needs of districts were eliminated through the use of this approach. The Project was implemented using an integrated approach. Construction of water supply facilities was combined with training of WSC in hygiene and sanitation, safe water chain particularly use of clean water vessels to draw water. Hygiene education preceded commissioning of water sources, which greatly helped in the promotion of sanitation. In the schools, construction of latrines was accompanied by provision of hand-washing facilities and soap to mitigate cases of poor hygiene and sanitation related diseases among pupils. For the case of Northern Uganda, the Project embraced GoU’s position of shifting focus from humanitarian support to the development approach. For instance, instead of constructing water supply facilities and sanitation facilities in IDPs, they were constructed in the communities, primary schools and health facilities. Given that people were moving away from IDPs back to their villages, taking WASH services to the communities greatly helped increase access to

23

integrated WASH services. Through the ODF campaigns, various households were triggered to construct their own sanitation facilities instead of relying on communal facilities as was the case in IDPs. See Section 6.0 on Project Impact.

24

5.0 EFFICIENCY

5.1 Introduction The assessment of Project efficiency focused on how well the various activities transformed the available resources into the intended results or outputs, in terms of quantity, quality and timeliness. The key issues that were assessed are shown in Box 3. Box 3: Project efficiency Operational work-planning, implementation, and budget management

Relations and co-ordination with local authorities, institutions, beneficiaries other donors

The quality of information management and reporting.

The extent to which the costs of the project were justified by the benefits.

GoU contributions

The extent to which technical assistance helped to provide appropriate solutions and develop local capacities

Quality of monitoring

5.2 Operational work-planning, Implementation and Budget management Given the fact that much of the project activities were concentrated in areas that were coming out of the war, implementation of project activities aimed at accessing safe water to the population did not wholly follow the standard Guidelines of the MoWE. According to the DWOs visited, civil works (the hardware) for the construction of water sources where funding was from UNICEF, in most cases preceded community mobilization (the software). Community mobilization is intended to elicit community demand for safe water facilities and cultivate ownership of the constructed water source, which is critical for O&M and sustainability of the activity. Whereas the software component was later implemented, DWD Guidelines require that community mobilization which is in line with the six critical guidelines precedes construction of the water source. This kind of approach is, however, easily practicable in a relatively stable environment, but not one that is of an emergency nature. In the circumstances,

it was important for the project to begin with the hardware and then revert to software after the service had been provided. Internal changes within UNICEF accounting and financial management system indirectly affected implementation of some project activities, mainly the hardware activities towards the end of Project implementation in 2010, but these were isolated cases, on the whole implementation of planned activities was not affected. Above: A latrine under construction at Usuku Girls Primary School, Katakwi

25

A latrine which is near completion at Kichwamba, Kabarole district

At the time of field visits, the Evaluation Team, found latrines under construction in Katakwi and Kabarole districts.

5.3 Relations and Co-ordination with Local Authorities, Institutions, Beneficiaries and other Donors

All local governments visited acknowledged UNICEF as a development partner that has made significant contribution to development in their respective districts. They cited mainly health, education and WASH sectors as areas that UNICEF has been supporting. As such, relations with the UNICEF during Project implementation were described normal and characterized by mutual respect. Although some of the LG officials argued that they should have been given more say in controlling and directing the UNICEF funded activities in order to hold accountable the contractors sent by UNICEF, they nevertheless stressed that before any contractor could be paid, the district had to issue a Certificate of Approval. Some of the Project reports seen by the Evaluation team showed that UNICEF facilitated the district staff to monitor and supervise such contractors while in some cases, UNICEF procured contractors from a list of prequalified firms submitted by the beneficiary districts. In terms of coordination, UNICEF facilitated coordination meetings of the WASH sector partners at a regional level. In the coordination meetings, achievements, challenges and future plans for the sector in a particular region would be shared and discussed. The meetings brought together NGOs with activities in the region; the Technical Support Units (TSUs) which are the regional backstopping team for DWD and district WASH partners. Such meetings provided an opportunity to review, reflect and learn from each other, among the partners, effectively transforming the coordination meetings into a platform to strengthen interventions in the WASH sector.

5.4 Quality of Information Management and Reporting Stakeholders, especially district local governments were adequately informed about the project activities and participated in the district-based planning and monitoring of activities. Similarly, lower level stakeholders e.g., the sub-county leadership, communities, schools and health facilities were involved in the local level planning and implementation of project activities. No project activity would be implemented without prior knowledge of the lower level stakeholders. Reporting by the districts to UNICEF was activity based rather than time bound. In effect, districts filed reports whenever they received any funding from UNICEF and implemented planned activities. In essence, these were accountability reports. Reporting was not regular like on a quarterly basis since it was activity-based. Given the fact that reporting

26

was activity-based, reports were generated by district level staff and not communities reporting to the districts. Despite the inadequacy of communities reporting to higher level (i.e. district), monitoring visits conducted by UNICEF together with district staff and other WASH stakeholders enabled the verification of data before reports would be forwarded to UNICEF head office. Nonetheless, the reports from the district would be sent to Zonal/Regional Centres; Gulu for the Northern region, Moroto for the Eastern districts and Karamoja area, and Kampala (to the Desk Officer for the Western districts). Upon receipt of these reports a verification of the highlighted challenges and achievements would be carried, and this, according to UNICEF staff would be put into consideration for the next disbursement. In a way, this constituted feedback formal on the reports from the implementing projects.

5.5 Cost-Benefit Analysis4 The Project aimed at reducing morbidity and mortality due to water, sanitation and hygiene related diseases. In the circumstances any life saved by the project intervention would not be equated to any cost i.e., millions of shillings. Accordingly, the realized benefits outweigh the costs incurred. Indeed, cases of morbidity and mortality due to WASH related diseases were reported to be on the decline in the surveyed Project sites. The increased access to safe water from all the functional sources constructed (except 1) demonstrates the cost-effective nature of the Project. The high rate of functionality was attributed to the competency of the selected contractors and effective monitoring by supported LGs. Thus, UNICEF chose highly competent companies to drill the boreholes. According to the Water Officer in Gulu, each borehole was constructed at an estimate of UGX 20 million which was value for money (this is comparable to the EUR.7,550 budgeted by UNICEF as the per unit cost for constructing a borehole). Further, it was revealed that it was more cost effective to involve the beneficiaries in construction of water sources—local labor and materials were provided wherever it was possible voluntarily by the beneficiaries, this was particularly the case in extension of the GFS and communities where the populace was already settled. Consequently, beneficiary participation in construction of water sources was low in Northern Uganda where people were in transition from IDPs to the villages. With regard to sanitation facilities, in the face of fast filling pit latrines and in areas with weak and collapsing soils, UNICEF in 2011 adopted a new technology for latrine construction (i.e. the drainable latrine), which was cost-saving. All latrines constructed in 2011, were lined i.e., the walls of the dug pits were lined with bricks and mortar to prevent the sides from collapsing. When the latrine fills up, it is emptied using a vacuum pump to extract the sludge. This saves the cost of constructing a new latrine as it used to be the practice, yet the latrine superstructure would still be in a good condition to support the continued use of the pit, if emptied. This is also a space saving technology, instead of having space filled with demolished un-lined latrines.

4 A term used in the project document

27

A completed pit latrine, ready for use; the health unit’s request for adjustments in the design to include a mother’s bathing space were not granted. Karambi Health Centre III, Kabarole There were, however, some concerns regarding O&M of the lined pit latrines. Respondents raised questions on the cost of emptying the latrines which requires a vacuum pump that may not be available in all communities where this technology was used. Even where the vacuum pump is available, it comes at a cost, which users need to be prepared to shoulder. But overall, sustainability of drainable latrines costs much less if compared with the cost of constructing a new latrine including the land/space utilized.

5.6 GOU Contributions to Project Efficiency The key line ministries of Water and Environment, Health, Education and Sports provided supervisory role as one of their mandates to ensure that LGs implemented the WASH Project as per the national guidelines. These ministries received funds from UNICEF under the WASH Project to undertake this support supervisory role to ensure quality. This approach was considered more efficient as well as contributing to capacity strengthening of central government structures. Local governments were responsible for planning and implementation of activities that fit in a UNICEF blue print as well as performing regular M&E role. Thus, the contribution of the LGs in the WASH Project was in form of planning, supervision and monitoring works of the construction companies procured by UNICEF, which would have been more costly if UNICEF were to perform it. As part of the planning process, LGs identified beneficiary communities and institutions, which ensured appropriate selection of beneficiaries.

28

Nonetheless, the contribution of GoU was in a way compromised by a thin staff mainly in the newly created districts, where inadequacy of technical staff and inadequate facilitation could have contributed to some level of inefficient and ineffective sector interventions. The most recent WASH Sector Performance Report shows that in FY 2010/11, the total population served through the District Water and Sanitation Conditional Grant (DWSDCG) was 559,136 persons through the construction of 2,863 water points. This is lower than the 670,910 people served in FY 2009/10. The reduction in number of people served in F/Y2010/11 (15%) is mainly the result of creation of new districts where considerable investments from the DWSCDG went into establishing new district water offices, purchase of transport and office equipment (Water and Environment Sector Performance Report 2011, September 2011) Further, the low staffing levels in the District Water Offices (DWOs) affected implementation of water programmes and subsequently sector performance especially with the newly created districts. Up to 90% of staff in the newly created district water offices lack the required qualifications and experience to effectively and efficiently implement water activities under the grant (Water and Environment Sector Performance Report 2011, September 2011).

5.7 Appropriateness of Project Technical Assistance Technical assistance provided by UNICEF to district staff and sector partners was of great use; for instance, project supported districts were equipped with skills in water quality testing which they did not posses before. Through the Project, local artisans were trained and equipped with skills to repair boreholes – i.e., HPMs. Each sub-county in the project area was provided with at least two (2) HPMs to handle all basic repair issues.

Challenges of an Institutional Led O&M, Arapai Sub-county Headquarters

and Arapai Health Centre II

Two boreholes located at Arapai sub-county headquarters and Arapai Health

Centre II provide a case for institutional lead O&M or the lack of it. At first, the

district constructed a borehole at the sub-county headquarters, and the borehole

served the police, the HC II and the surrounding community. Although a user

committee was formed, none of the members demonstrated clear commitment as

a sign of ownership of the water source. When the borehole broke down, the

committee was ill prepared to repair it, and the source lies abandoned despite the

fact that it is located at the seat of the sub-county. In 2011, Soroti district with

funding from UNICEF, constructed another water source less than 200 meters

from the obsolete one. The new one is located at the HC II. According to the Nurse

in-charge of the HC, the water source which has been operating for over a year,

does not have a functional WSC. She expressed concern that this hand pump

would most likely face the fate of previous one located at the sub county

headquarters. One key informant explained that, “where facilities are located on

government land, communities tend to think that such facilities belong to the

government institution, yet the community is the biggest user. When the source

breaks down, the community complains that they are not getting water, yet when it

comes to repair, they say the source belongs to the government”. -Soroti Key

Informant. However, according to the DWO Soroti, functionality of boreholes in

Soroti is at 82%. He added that non-functionality of boreholes is largely attributed

(for an estimated three quarters of the non- functional sources) to limitations of

community based maintenance system (CBMS). Interestingly, the budget for

rehabilitation (reportedly including UNICEF supported activities) seldom includes

support to CBMS activities, mainly revamping WSCs. “You know this thing is very

complicated. When you are rigid and strict on compliance with the guidelines to

include CBMS, some development partners feel scared. So you have to soften

your stance in order to get the services (water source) for the people. It is a trade

off.”

29

The project pursued innovative approaches to stimulate demand for better sanitation and hygiene practices. Such approaches included the Community Led Total Sanitation (CLTS) and the Self-help Group Approach (SGA). The SGA entailed community members forming a pool of labour force and working for one another to improve sanitation and hygiene in a rotation. The host (beneficiary) would buy food for the members, while the Project would facilitate the group with tools and technical knowledge through local artisans resident in the community. The local artisans were also trained with support from UNICEF. In Moroto where challenges had been registered in mobilizing communities to dig pit latrines, the SGA was applauded as having been a viable solution, for it promoted local resourcefulness, leadership and ownership of activities. It took about two weeks for a group of households in a pool to construct one latrine. Other than CLTS which was used to trigger community members into action, exchange visits and involvement of VHTs were effective methods of stimulating demand for better sanitation. For instance, community members in Nachogorum Parish, Nadunget Sub-county, Moroto District were mobilized to participate in an exchange visit to Kacheri village in Kotido district. The visit was deemed a success as the participants in the visit later returned and constructed pit latrines alongside other improvements in sanitation and hygiene. Nachogorum Parish is still hailed as a success story and was visited by the participants in the quarterly regional WASH sector coordination meeting, as a case of successful community led improvement in sanitation and hygiene. It is worth mentioning that the use of VHTs is in itself a prudent strategy to ensure institutionalization of the knowledge and skill in the community. VHTs who are part of the national health structure benefit from facilitation and motivation for transport from the MoH.

30

6.0 IMPACT 6.1 Introduction Impact as per the ToR of this Evaluation was assessed by examining the relationship between the overall and specific Project objectives. In assessing the Project Impact, two broad aspects were analyzed, namely, the extent to which the objectives of the Project were achieved, and the assessment of facilitating and constraining factors vis-à-vis achievement of objectives. See Box 4. Box 4: Project impact o The extent to which the objectives of the Project were achieved as intended.

New water users gain access to safe water in the rural population. Access to latrines in the rural population Hygiene practices (safe water chain, hand washing, food hygiene) among the targeted population

improved. Institutional development and strengthening of policies, guidelines, and advocacy

o Whether the effects of the Project: Have been facilitated or constrained by external factors Have been facilitated by or constrained by Project management, by co-ordination arrangements,

by the participation of relevant stakeholders Have made a difference in terms of cross-cutting issues like gender equality, environment, good

governance, conflict prevention etc.

6.2 Access to Safe Water Data from the MoWE (2007-2011) indicate general progress in attainment of MDGs in relation to WASH i.e., WASH project targeted districts inclusive. Since UNICEF over the period provided support to the sector in over 21districts, it can from the onset be inferred that the project made a noticeable contribution towards the attainment of MDG golden indicators for the WASH sector. See Table 11, for the Uganda Country trend and targets. Table 11: Uganda Country Progress against 7 out of 11 eleven golden MDG indicators for the WASH sector5

Indicator 2006/07 2010/11 2014/15 Target

Access % of people within 1 km (rural) and 0.2 km (urban) of an improved water source

63% 65% 77%

Functionality % of improved water sources that are functional at time of spot-check

83% 83% 90%

Per Capita Investment Cost Average cost per beneficiary of new water and sanitation schemes (US$)

$38 $47 $45

Sanitation % of people with access to improved sanitation (Households).

59% 70% 77%

Sanitation: Pupil to latrine/toilet stance ratio – schools (from DHI reporting)

69:1 66:1 40:1

Hand washing % of people with access to (and using) hand-washing facilities.

HH: 14% 24% 50%

SCH: 41% 33% 50%

5 Since the project area was rural, only values for the rural areas have been used.

31

Management % of water points with actively functioning Water & Sanitation Committees

63% 71% 95%

Gender % of Water User committees with women holding key positions.

87% 81% 95%

Source: MoWE, Water and Environment Sector Performance Reports for 2007 and 2011

In relation to the project area, there is a notable increase in access to safe and clean water and functionality of facilities in almost all the 21 project supported districts except for Karamoja sub-region. However, in Amuru, a drop in functionality rate of water facilities was recorded; mostly attributed to the decommissioning of boreholes in old IDP camps and non-existence of WUCs of some of the abandoned sources in former IDP camps. In most of the newly created districts i.e., those that were not in existence at the commencement of the WASH Project in 2007, increase in access to water could not easily be ascertained due to lack of baseline data. See Table 12. Table 12: Trends in levels of access to safe water in project supported districts No. District Water coverage [%] Functionality [%]

2007 2011 2007 2011

1 Abim 56 84 59.6 78

2 Alebtong 87 74

3 Amolator 57 64 74.6 77

4 Amudat 23 83

5 Amuria 57 57 88.5 87

6 Amuru 67 76 95.0 71

7 Apac 47 66 85.9 75

8 Bundibugyo 63 61 71.1 84

9 Dokolo 64 91 76.7 74

10 Gulu 76 93 95.0 67

11 Kaabong 21 19 82.5 84

12 Kabarole 69 90 82.7 80

13 Kaberamaido 90 74 88.5 84

14 Kamwenge 85 86 94.4 78

15 Kasese 81 61 95.0 83

16 Katakwi 71 85 81.6 93

17 Kole 66 74

18 Kotido 24 43 64.7 82

19 Kyegegwa 66 71

20 Kyenjojo 66 76 83.8 84

21 Lira 56 93 77.8 77

22 Moroto 47 32 82.3 90

23 Nakapiripirit 39 51 49.6 85

24 Napak 49 77

25 Ntoroko 79 62

26 Nwoya 83 72

27 Otuke 93 79

28 Oyam 52 62 76.2 76

29 Serere 71 86

30 Soroti 79 75 86.5 84

Source: MoWE (2007 & 2011), Water and Environment Sector Performance Reports

32

School children collect water at Boke Primary School, Lira district

Records show that all the 21 Project districts were supported to construct safe water supply facilities mainly boreholes, motorized wells, GFS and rainwater harvest tanks in water stressed communities, schools, and health centers. Findings from a sample of 10 districts out of the 21 corroborate reports on access to safe water in the Project supported districts by comparing the pre and after Project implementation situation. See Figure 2. Fig 1: Household main water source pre and after Project implementation

Results show that hand-pumps, GFS taps and motorized wells are currently the main sources of water for use in households. In Figure 2, it can be seen that slightly more than half (53.7%) of the sample used to obtain water from unprotected water sources prior to Project implementation. Household data shows that 53.5% of the respondents noted that they

were accessing safe water from households constructed after 2007 while close to a third (28.1%) were accessing water from some of the sources that had been rehabilitated by the Project.

Similarly, the source of safe water for schools that were sampled in this Evaluation were boreholes and GFS and motorized wells, which were constructed with UNICEF support. Majority of the pupils interviewed (84.1%) acknowledged easy access to safe water while at school i.e., sources located within schools’ compound. Based on the sample of communities and schools that were visited, it is clear that rainwater harvesting tanks were not mentioned as sources of water. RWHTs are seasonal i.e., only functional during the rainy season and in a situation where the population is big e.g., in a school setting, water in the tank lasts a few days.

As much as this is appropriate technology, alone cannot serve the needs of a school with a big population or serve the school through a prolonged dry season. Table 13: Source of water by households and school

Beneficiary RWHT %

Borehole %

Protected spring %

GFS/Motorised wells %

Unprotected source %

Community/districts Abim Bundibugyo Dokolo Gulu

0.0 0.0 0.0 0.0

51.0 0.0 52.0 95.9

2.0 1.9 46.0 0.0

41.2 70.3 0.0 0.0

5.9 0.0 2.0 4.1

33

Kabarole Kasese Katakwi Lira Moroto Soroti

0.0 0.0 0.0 0.0 0.0 0.0

0.0 2.0 100.0 76.0 100.0 62.0

2.0 2.0 0.0 24.0 0.0 30.0

88.0 28.0 0.0 0.0 0.0 0.0

10.0 68.0 0.0 0.0 0.0 0.8

Total 0.0 53.4 10.7 24.1 11.9

School/pupils

Namugongo P.S Awach P.S Katunguru P. S Usuku Boys P.S Boke P.S

0.0 0.0 6.7 0.0 0.0

0.0 100.0 10.0 100.0 100.0

3.3 0.0 0.0 0.0 0.0

96.7 53.3 0.0 0.0 0.0

0.0 0.0 30.0 0.0 0.0

Total 1.3 62.3 0.7 29.8 6.0

As per the MDG golden indicator for the WASH sector of access percentage of people within 1 km of an improved water source, functionality of the water source and percentage of water source with active WSCs and gender percentage of WSCs with women holding key positions, the Project sites posted impressive results as shown in Table 14. Majority of household respondents (81.0%) reported accessing their current water source within a distance of less than a kilometer (<1 km) and 62.8% affirmed that they always got adequate water for their domestic use.

Table 14: Access to safe water: Project performance as per MDG indicators Indicator MDG Target 2010/11 2014/15

MDG Project achieved

MDG

Access % of people within 1 km (rural) of an improved water source

65%

81.0%

77.0%

Functionality % of improved water sources that are functional at time of spot-check

83%

95.0%

83%

Management % of water points with actively functioning Water & Sanitation Committees

71%

Gender % of Water User committees with women holding key positions.

81%

Fig 2: Availability of WUC in sampled communities

All the water sources constructed under the WASH Project were functional at the time of this Evaluation, well operated and maintained, hence contributing to sustainably increased access to safe and clean water for community and schools. However, the major challenge was found with the rainwater

34

harvesting facilities, which are seasonal i.e., they are only fully functional during the rainy season and for a short time because of the large population of users at schools and HCs. The high functionality rate although mostly attributed to the technology used; availability of highly skilled HPMs and existence of WSCs also account for the high functionality rate. The project supported training of 230 WSCs and several HPMs in O&M of water sources. However, many household respondents observed that WSCs were not fully functional; about half (56%) reported having functional WSCs, which implies that the remaining proportion (44%) was inactive WSCs at their water sources. Existence of inactive WUCs despite efforts to revitalize the committees in all project supported districts was partly attributed to lack of training and motivation of WUCs. The Project supported hand-pump mechanics to form associations e.g., hand-pump mechanics in Moroto and Abim were supported to form associations through which HPMs support each other in joint maintenance and construction work. In addition, the Project supported procurement of 271 sets of “Hand-pump Mechanic Tools” for use in 15 districts (4th Annual Progress Report 2011). Apart from providing water facilities, districts were supported in water quality control/surveillance to ensure the safety of water accessed by communities. Testing of water quality to ensure its safety is done at all newly constructed water sources. The practice in these districts is for the Contractor to test the water in the presence of district staff and members of the community/beneficiaries before handing the source over to the community. The districts have also used the support and expertise to carry-out an extensive water quality testing exercise. Surveillance was, however, reportedly constrained by lack of adequate testing kits at district level. It was reported that under the WASH Project each district was given about five (5) testing kits which are kept at the district. Further, the reagents and chemicals used in testing water were reportedly not readily available in the districts, which pose a challenge to timely response to reported cases of water contamination. Given the remote nature of the project supported districts, suppliers of such chemicals are not available within easy reach. Overall, the impact in the area of accessing safe water to communities and institutions (schools and HCs) is visible. In large villages where only one safe water source existed, construction of another water source with support from the Project eased pressure on the existing water source thereby increasing access to safe water. Cases of crowding at water points were observed rare in Project supported communities.

6.3 Access to Sanitation Facilities Evaluation results show that efforts to increase access to sanitation facilities such as pit latrines, bathing shelters and kitchen drying racks as well as rubbish pits achieved commendable results. According to DHIs in all sampled districts there was increased latrine coverage and use especially in Project sites. In the 10 sampled Project districts latrine coverage was estimated at 71.4%, which is slightly higher than the national average estimated at 69.8% (DWD, 2011 SPR). See Table 15.

35

Table 15: District and project site latrine coverage

District District coverage (2011) %

Project site coverage %

Abim Moroto Soroti Katakwi Lira Dokolo Gulu Kabarole Bundibugyo Kasese

39.5 <20.0 60-70 40-60 60-77 60-77 40-60 >77 60-77 >77

47.1 34.0 72.0 72.0 50.0 80.0 73.0 100.0 94.4 86.0

Total 71.4

As noted above, all project sites in respective districts posted higher latrine coverage than the district average. Majority of latrines (66.9%) were traditional, 4.2% were improved while 0.4% of the households only pits with no superstructures were taken as latrines. Regarding latrine use, majority (76.6%) of households indicated that all their members used latrines except in cases of young children who cannot squat on the pit.

Fig 3: Use of latrine by pupils at school

With regard to schools, pupils and teachers met in this evaluation revealed that construction of latrines at schools had greatly improved sanitation behaviors and practices with majority of pupils reached reported to be using latrines. See Figure 3. Improved access to latrine is, however, constrained by the big number of pupils compared to the stances available. Although overall the pupil: stance ratio of the five schools visited was 51:1, in two of the schools, namely Awach P.S and Katunguru P.S, the ratio was very low. See Table 16. Table 16: Pupil:stance ratio of studied schools School Total population of

pupils No. of stance Pupil: Stance ratio

Namugongo P.S Awach P.S Katunguru P. S Usuku Boys P.S Boke P.S

702 957 705 739 1224

14 10 10 18 22

50:1 96:1 70:1 41:1 56:1

Total 4327 74 59:1

When the Project achievements are compared with the MDGs targets, the pupil: Stance ratio, which stands at 59:1 in the Project area, is improved compared to the MDGs targets for 2010/11, but far short from the MDGs target of 40: 1 for 2014/15. For the rest of the two indicators, the Project achieved targets are lower than the targets for the MDGs. See Table 17.

36

Table 17: Access to sanitation: Project performance as per MDG indicators

Indicator MDG Target 2010/11 2014/15

MDG Project achieved

MDG

% of people with access to improved sanitation (household)

70% 71.0% 77%

Pupil to latrine/toilet stance ratio—schools (from DHI reporting)

66:1 59.1% 40:1

Hand-washing % of people with access to (and using) HWFs

24% 14.3% 50%

Both in schools and households in the community use of latrine was considerably high; 89.3% and 76.6% respectively. Thus, for both pupils and households proportions that did not or rarely use the latrines were negligible. For the pupils who did not use the latrine at school, nearly all (93.8%) attributed this to the poor hygienic condition (i.e. being fouled/dirty) of the latrines. As per the Evaluation findings, apart from offering privacy, having separate stances for boys and girls, the hygienic conditions of most latrines were poor. On the other hand, failure to use latrines was basically attributed to either the young age of the member (small children) or the most elderly (92.8%). The Evaluation findings on the hygienic conditions of latrines in the Project sites revealed that most of the school latrines were fouled with human waste, which partly explains the big number of pupils that did not use school latrine. See Table 18. Table 18: Hygienic conditions of latrines

Condition of school and household sanitation facilities

School

Household

Distance of the pit latrine from nearest class block/ main house Less than 10 meters

10 meters and above

20.0 80.0

32.1 67.9

Presence of human waste around homestead Yes, observed

No

60.0 40.0

13.7 86.3

General condition of latrine Offers privacy

Has door/shutter Offers protection from rain

Has repugnant smell Fouled

Has latrine cover

90.0 70.0 90.0

-- 60.0

--

70.4 55.3 71.5 46.4 38.3 17.6

The noticeable fair hygienic sanitation conditions among households in Project area was partly attributed to the sanitation campaigns. Through the Community Led Total Sanitation (CLTS) approach, various communities in Project supported districts were triggered to stop open defecation within their communities, although 100% Open Defecation Free (ODF) villages have not been realized, the improvements are major. In Gulu alone, 32 villages have been triggered

37

A pupil at Namugongo primary school, in Bundibugyo district displaying a hand-washing

facility from UNICEF UNICEF

for the ODF. To maximize the impact of improved access to sanitation facilities particularly pit latrines, a “name and shame”/”walk of shame” campaign was adopted in some Project districts, where names of especially local leaders without pit latrines were read out on Radio – this compelled people to construct pit latrines. In other districts, bye-laws on sanitation have been invoked to arrest, charge and compel people without pit latrines to construct them. Districts have developed sanitation bi-laws and ordinances. Further, schools had received sanitary materials for adolescent girls and 1 teacher in each beneficiary school has been trained in the making of sanitary wears using local materials i.e. cotton and cloth. In addition, members of the school management committee (SMC) in beneficiary schools have been trained and are actively engaged in monitoring availability and use of sanitation facilities within the schools. Despite the above noticeable impact, about a third (29%) of households without latrines and the pupil: stance ratio of 59: 1 represent a big number without access to latrine. The low pupil: stance ratio is attributed to increased enrollment for primary education (MoWE, 2009). On the other hand, lack of household latrines is attributed to either the high costs of construction, lack of manpower or the fact that there were generous neighbors who allowed them to share the latrine. The practice of sharing latrines was observed more common in the districts of Northern Uganda.

6.4 Hygiene Behaviors and Practices Hand-washing with soap was promoted both in schools and communities in the Project area. Hand-washing facilities (HWFs) including soap were distributed to various schools and health centers in Project supported districts. Whereas all the five schools (100%) visited during the evaluation had HWFs only a fifth (20.1%) of them. For instance, for most households with HWFs next to latrine in the communities, only a half had water; implying that washing hands after using a latrine was not yet an inculcated practice among households. However, efforts to inculcate the practice of washing hands with water and soap after visiting a latrine were more evident in schools with 98% of the pupils interviewed reporting washing hands after latrine use compared to 67.9% of household respondents. However, at the time of this Evaluation only 40% of the observed HWFs at schools had water and presence of soap was only observed in 18%. Nonetheless, it was reported by pupils and school authorities that during assembly messages on hand-washing are disseminated to pupils. Pupils refill HWFs and are responsible for their safety. See Table 19.

38

Table 19: Hygiene practices and indicators

Sanitation and hygiene practices

Pupils

%

Household respondents

%

When do you wash your hands After using the toilet

Before preparing food Before eating

After eating After gardening

When they look dirty Before feeding children After changing nappies

Others

98.0

-- 65.0 37.1

-- 21.9

-- --

16.6

67.9 27.6 79.4 51.4 46.4 31.9 4.0 2.2 5.8

Selected household indicators

n

%

Presence of hand-washing facility next to latrine Yes No

72

286

20.1 79.9

Presence of water in the hand-washing facility Yes No

40

318

11.2 88.8

Presence of soap/ash in hand-washing facility Yes No

18

340

5.0

95.0

Presence of bathing shelter Yes No

331 173

65.7 34.3

Presence of kitchen utensils drying rack Yes No

177 327

35.1 64.9

In the communities, respondents acknowledged attending hand-washing sensitization meetings, and the adoption of the practices was reportedly taking root. Other hygienic practices such as handling of food utensils and presence of drying racks for kitchen utensils were observed in only about a third of the sampled households but even where they were available and in use, they were not used properly.

6.5 Knowledge and Reported Occurrence of WASH Related Diseases General knowledge of WASH related diseases among pupils was universal (97.4%) among sampled school pupils in the five primary schools. The most known WASH related disease as reported by pupils was diarrhea (68%); reported by 69.3% and 66.7% female and male pupils respectively. See Table 20.

Table 20: Pupils knowledge of WASH related diseases and experience

Sanitation practices Pupils

Female %

Male %

Total %

39

Do you know any disease caused by drinking /bathing with bad water

Yes No

96.2 3.8

98.6 1.4

97.4 2.6

Known diseases caused by drinking /bathing with bad water Skin rash

Scabies Diarrhea

Eye diseases

4.0

17.3 69.3 5.3

16.7 8.3

66.7 2.8

10.2 12.9 68.0 4.1

Suffered from water and sanitation related diseases in past 3 months

Yes No

30.7 69.3

30.6 69.4

30.6 69.4

In terms of morbidity, over three-thirds (69.4%) of the pupils had not suffered from any water and sanitation related diseases in the three (3) months preceding this Evaluation; an indicator of Project’s impact on the targeted schools. Like school pupils, household respondents exhibited almost universal knowledge of WASH related diseases (95.4%). Although there were no differences by gender on the overall knowledge, there were differences on the specific diseases between male and female respondents. See Table 21. Table 21: Respondents knowledge of WASH related diseases

WASH related diseases

Respondents Total

%

Female %

Male %

Do you know any diseases caused by poor sanitation/hygiene Yes No

97.1 2.9

94.5 5.5

95.4 4.6

Known diseases resulting from poor sanitation/hygiene Diarrhea Cholera

Intestinal worms Typhoid

Dysentery Other

67.5 59.8 11.9 17.4 12.5 34.1

62.9 65.3 10.0 20.0 12.4 34.7

65.9 61.7 11.2 18.3 12.5 19.5

Varied differences emerged in the knowledge of WASH specific related diseases across the 10 sampled districts. See Table 22.

40

Table 22: Reported known WASH related diseases by district District Known WASH disease

Diarrhea %

Cholera %

Intestinal worms

%

Typhoid %

Dysentery %

Other %


77.3 43.4 98.0 75.0 41.3 2.0

88.9 80.9 64.0 91.8

47.7 77.4 12.0 60.4 58.7 95.9 64.4 70.2 94.0 34.7

2.3 5.7 6.0 4.2

28.3 12.2 4.4

19.1 4.0

26.5

6.8 35.8 6.0 0.0

32.6 20.4 28.9 27.7 10.0 14.3

0.0 5.7 2.0

16.7 32.6 6.1

20.0 2.1

28.0 12.2

36.4 17.0 46.0 54.2 21.7 6.1

60.0 36.2 10.0 59.2

Total 65.9 61.7 11.2 18.3 12.5 34.3

Overall, slightly over a quarter (26.3%) of all respondents in the sampled households reported a member in their households who had suffered from WASH related diseases in the last three months preceding this Evaluation. With regard to cases of loss or death of household members due to WASH related diseases in the last four years prior to this study (i.e., the period over which the Project was implemented), less than a tenth (7.6%) of respondents in sampled households in the 10 districts reported death of a member. In other words, cases of someone dying due to a WASH related disease was only reported in 7.6% of sampled households. See Table 23.

Table 23: Households reported WASH related morbidity and mortality

District Morbidity

< 5 yrs of age %

>5 yrs of age %

No %

Don’t know %


13.0 5.7

10.9 33.3 2.2 0.0

11.1 4.3

75.0 34.7

10.9 3.8

10.9 8.3 6.5 0.0 8.9 4.3 6.3

14.3

76.1 90.6 78.3 56.3 91.3

100.0 80.0 91.5 18.8 51.0

0.0 0.0 0.0 2.1 0.0 0.0 0.0 0.0 0.0 0.0

Total 19.0 7.3 73.4 0.2

Mortality in the last 4 years

Yes %

No %

Abim Bundibugyo Dokolo Gulu Kabarole Kasese

0.0 5.6 0.0 4.1 0.0 0.0

100.0 94.0

100.0 95.9

100.0 100.0

41

Katakwi Lira Moroto Soroti

0.0 0.0

58.0 8.2

100.0 100.0 42.0 91.8

Total 7.6 92.4

Households that reported loss of members (dying) due to WASH related diseases were mostly in Moroto, followed by Soroti, Bundibugyo and Gulu; about 58% of respondents in Moroto reported losing a household member. No household reported losing a person due to WASH related diseases in the last 4 years in the rest of the other districts. In all the Project sites there were reports that in overall terms WASH related diseases such as diarrhea, dysentery, cholera, intestinal worms, skin rush, coughs etc had reduced. Records from some of the health facilities visited (the case of Awiri Health Centre in Dokolo) indicated only 3 cases of dysentery that were recorded in a period of 3 months while diarrhea declined from 90 cases in December 2011 to 30 cases in March 2012. In Kabarole district, the Health Assistant Kichwamba sub-county revealed that cases of Bilharzias had considerably reduced in five villages that benefited from the Project supported GFS. Overall, it can be inferred that the Project managed to impact on the level of morbidity and mortality resulting from WASH related diseases in the targeted communities and sites. Other notable impact was observed in the campaign to promote household hygiene such as campaigns to promote proper management of solid waste. Rubbish pits were observed available in about half the households visited. Direct intervention in waste disposal management was done at mostly health centres. Community practices with regard to waste disposal are commendable; 50.5% have rubbish pits (see Table 24).

Table 24: Methods of household solid waste management and non-dangers of indiscriminate disposal Waste disposal Households

n %

Ways rubbish is disposed off among sampled households Rubbish pit

Thrown in bush Covered in ground

Burning Scatter Others

254 144

2 12 83 8

50.5 28.6 0.4 2.4

16.5 1.6

Reported dangers of poor/bad disposal of rubbish/garbage None

Diseases Smelly surroundings

Ridicule from community

5

460 160 10

1.0

91.3 31.7 2.0

6.6 Access to Information on WASH Software activities implemented in the Project districts were, among others, aimed at increasing knowledge and awareness with regard to safe water, proper sanitation and hygiene related behaviors and practices. Disseminated messages were reported to include installing HWFs,

42

washing hands with soap especially after latrine use, cleaning water vessels and generally maintaining the water safety chain. Schools particularly benefited from these campaigns as nearly all pupils (93.4%) interviewed acknowledged receiving information about WASH aspects at school and their homes. The most commonly cited sources of WASH related information by pupils included teachers and science lessons (see Table 25).

Table 25: Accessibility to information and sources of learning about WASH

Information on WASH Pupils

n %

Have you ever got information about WASH related diseases Yes No

141 10

93.4 6.6

Sources of learning about WASH related diseases Science lessons

Teachers at school Fellow pupils/peers

School health club Posters/fliers

Radio Parents at home

Health worker Other

74

107 19 27 9 6 7 11 11

49.0 70.9 12.6 17.9 6.0 4.0 4.6 7.3 7.3

In the communities, slightly over two-thirds (66.8%) of the sampled household members reported to have received information on WASH compared to a third (33%) that had not received the information. See Table 26 for district variations. Table 26: Respondents receipt of information by district and gender District Receipt of information

Yes %

No %


52.9 75.5 80.0 63.3 92.0 84.0 74.0 34.0 26.0 86.0

47.1 24.5 20.0 36.7 8.0

16.0 26.0 66.0 74.0 14.0

Gender

Female Male

65.7 69.0

34.3 31.0

Total 66.8 33.2

43

Household respondents that had not received information on WASH majority were in Moroto (74%) followed by Lira and Abim. With regard to gender, more females than males had not received WASH related information, which could also be attributed to the sources of information that tend to be more accesed by men than women such as the radio as was the case. See Figure 4. Figure 4: Reported WASH sources of information

There was a remarkable increase in the proportion of villages/wards with trained VHTs, according to a Survey in 2010, in line with the expansion to a further 18 districts Health Sector Strategic Plan III 2010/11-2014/15).

6.7 Factors which Facilitated Attainment of Project Impact Successful implementation and attainment of the WASH Project objectives is attributed to several factors. Key among the cited factors was the availability of funding, goodwill from both the political and civil leadership in the districts, and the modus operandi chosen by UNICEF. By working through the district and sub-county LGs, UNICEF ensured that no parallel or ad-hoc systems for Project implementation were setup. All Project activities were primarily planned, implemented and/or supervised by district staff, which approach was appreciated by district stakeholders. Availability of funding was the most focal factor which facilitated attainment of project objectives. The EU-UNICEF fund made available resources for the construction of water and sanitation facilities for the rural poor in the 21 project supported districts. Thus, the effective involvement of district leaders; political, religious and cultural accounted greatly for the attainment of Project objectives. In all districts visited, both the political and civil service leadership were reported to have embraced the sanitation and hygiene campaign. For instance, in all districts leaders in communities worked together with the WUCs to enforce the bye-laws such as prohibiting use of dirty water vessels, stopping animals from sharing waters sources for domestic use, fencing the water sources etc. In particular districts of Gulu and

44

Bundibugyo leaders were reported to have prohibited some community members from washing vehicles near water points to minimize possibilities of water contamination and bathing in streams for purposes of promoting personal hygiene. In some districts such as Abim, the participation of the LCV chairman in supervision and monitoring visits had positively impacted on the Project activities. It was further observed in in-depth discussions that political leaders had even embraced the sanitation ordinances and bye-laws and in some districts participated in the arrest of household heads found without latrines. Appropriate targeting of communities with severe access challenges to safe water also contributed to Project success and rallying ownership of especially water sources. Following the priorities in the district development plans, focused targeting of water sources was achieved. Distances to safe water sources have been tremendously reduced especially for people and school pupils in northern Uganda. However, realization of optimal results from sanitation improvement campaigns such as the ODF campaign was partly constrained by limited follow-up. The ideal would be to visit the triggered communities at least once a month to observe progress in ODF, but this was not possible due to inadequacy of funds for the ODF campaign. Further, district staff reported that it was always difficult to mobilize all community members to be present for an ODF triggering meeting. In addition, there was Project inability to provide communities with tools for construction of sanitation facilities and making slabs as had earlier been envisaged. Similarly, in the schools, whereas members of staff had been trained in making sanitary wear for adolescent girls using cotton and cloth, this was not implemented. As a result of good management and coordination, notable differences were made in such cross-cutting aspects like gender equality. The project requirement that at least a third of the WSC be women has ensured gender equality in WASH activities and choice of water point locations which take into consideration safety of women and the girl-child But overall, both external and internal factors played a big role in the attainment of the anticipated Project outcomes. The Project was managed and coordinated well, involving all stakeholders including the actual direct beneficiaries.

6.8 Factors which Constrained Project Impact

Since the commencement of Project implementation new districts have been created, which have weak capacity to support implementation of Project activities. In 2007 i.e., the starting date of the Project there were 21 districts, but by end 2011 the number of districts had increased to over 30 in the Project area. The districts are associated with weak capacity as reflected in a thin staff in the key departments including the water office. Secondly capacity weakness is reflected in a small resource envelope that such local governments can muster, mainly through lost local revenue. In terms of central government transfers which are based on district total population figures, the small the population, the smaller the size of the transfer from the central government, which dissuades qualified personnel from taking up posts in such districts. Inadequate funding from the Centre affecting district implementation capacity; for instance where an actor does not provide for software activities, DWD requires that the district steps in to cover that activity. However there is less funding for that activity. Whenever there is a budget cut, it is the software part which is greatly affected.

45

6.9 Impact on Cross-cutting Aspects

Cross-cutting aspects that were relevant in the Project on which it had impact include good governance, gender equality, environment protection and conflict prevention. 6.9.1 Good governance

The Project was implemented by decentralized local government structures i.e., districts and sub-counties, which was in line with government decentralization policy. This, way the Project impacted on good governance via making local structures more visible. Further, the Project facilitated the formation and training of grassroot structures particularly the WSCs to be in-charge of governance of the constructed sources. Discussions with stakeholders confirmed that the WASH Project has fostered good governance in the management of constructed water sources. WSC existed even for institutional water sources i.e. schools and health centers. The WSC in-charge of institutional water sources comprise some members of host communities and school administration. The functions of WSC, as revealed in this Evaluation are shown in the Box below.

Box 5: Indicators of good governance.

Mobilise water users for O&M

Enact and enforce O&M bye-laws

Hold meetings with water users to discuss problems affecting the water sources

Plan for and oversee O&M; report problems to sub-county

Collect and utilize O&M funds, and account to the water users

Together with users select caretakers

Engage HPM/plumbers and pay for spares and repairs

Monitor own and external Activities

In FGDs with water users it was confirmed that WSC members hold regular meetings to review and plan for their water source, work with caretakers and the designated HPM to carryout routine repair and maintenance of the water source. Further, there are reports that the WSC, although not very frequently, call for general community meetings to give the users feedback and accountability about the money collected for O&M. All WSC members covered in the study reported collecting O&M funds- an average of Ug.shs.1,000/= per month. Bye-laws are actively enforced; for instance, dirty water vessels are discouraged at water source, animals (cows) that graze around the water source are seized and owners penalized. 6.9.2 Gender equality

Gender equality received a lot of focus in this WASH Project. The Project requirement that at least a third of all the WSC positions be reserved for women was adhered to. Women have mostly taken the position of Secretary and Treasurer on the WSC; but there were communities where they had been selected as Chairpersons of the WSC. This condition of having women on the WSC ensured gender equality in WASH activities and choice of water point locations which take into consideration safety of women and the girl-child.

46

6.9.3 Environmental protection

There was evidence that efforts were made to protect the environment in areas where construction of water facilities took place especially the construction of the drainage channels and soak way pits. Well maintained drainage channels and soak way pits ensure that there is no stagnant water around the source as observed in the water sources constructed with the support of the Project. This, all water sources were observed with well constructed aprons/drainage channels and soak pits, which are a measure of protecting the environment. The contractors were reported to have also handled disposal of drilling waste in a manner that would not degrade the environment. Drilling waste was not left on any of the constructed water sources, but rather transported by the contractor for safe disposal. Similarly, siting of pit latrines in the Project areas was closely monitored to ensure that institutions such as schools and health centers did not construct pit latrines in areas where there was potential of contaminating water sources. For the general population, the Project through ODF triggering campaigns and/or PHAST trainings educated people on the dangers of constructing pit latrines within 30 meters of water points and less than 10 meters to the main house. A case of a pit latrine demolished because of contaminating water was reported in Dokolo District.

6.9.4 Conflict Prevention

Availability of safe water sources was reported, among others, to have contributed in averting sporadic household and community conflict. In large villages or neighboring communities that shared a single source, construction of other source was reported to have eased the pressure on the existing water sources and hence reducing on overcrowding that occasionally resulted into conflicts. Through observations by the Evaluation team, it was clear that sases of crowding at water points were rare in Project supported communities. Further to the above, location of water sources was harmonized under the WASH Project; letters of agreement for land and access roads to water points were made pre-requisites for construction of a borehole. Conflicts that would ordinarily develop between members of the WSC/ water source caretakers and the users were greatly averted through the pre-construction sensitization campaigns. Prior to construction of the water source, the district water office, would visit the community to carry-out a pre-construction sensitization/mobilization to adequately prepare the community for the water source.

We got a complaint from community members from Kwera about a borehole which had been contaminated, sometimes pumping out human waste. We went there and found that a school had constructed a pit latrine near the community borehole. So we gave them 2 options; one to abandon the water source or to demolish the pit latrine. But because the community originally had poor access to safe water and also considering the cost of constructing a borehole, the community together with the owner of the school agreed to demolish the pit latrine. We have actually been advising households not to dig too deep to reach the water table or building latrines to close to water sources. Assistant District Water Officer, Dokolo District

47

7.0 SUSTAINABILITY

7.1 Introduction

Sustainability refers to continuation of positive outcomes of the Project and the flow of benefits after expiry of external funding for activities. The existing support interventions such as policy dialogue and co-ordination also point to sustainability of interventions. This Evaluation made an assessment of the prospects for the sustainability of benefits on basis of areas shown in Box 5. Box 5: Sustainability issues for the WASH project

o The ownership of objectives and achievements including the extent stakeholders were consulted. o Policy support and the responsibility of the beneficiary institutions. o Institutional capacity in terms of policy and budgetary support and that of counterpart institutions. o Socio-cultural factors. o Financial sustainability o Technical (technology) issues.

7.2 Ownership of the Objectives and Achievements

The WASH Project was implemented within the national policy and planning framework. The Project objectives subscribed to those of the sector. Consultation of stakeholders characterized the implementation of the Project including communities that, among others, were accorded opportunity to select appropriate locations for the water sources. Where the location would not be ideal for the technology in question, the communities would be explained the reasons and advised to select other locations. Institutions including schools and health centres also participated in selecting locations for water sources as well as sanitation facilities. 7.3 Policy Support and the Responsibility of the Beneficiary Institutions Recent structural changes at UNICEF which place WASH services under health are likely to affect the operations and priorities of the water and sanitation component in the districts. The restructuring accordingly requires that funds from UNICEF on WASH related activities be channeled to the Directorate of Health through the CAO’s office through MoWE and MoH. The Directorate of Health together with the UNICEF Focal Person could inadvertently prioritize the curative component not water access and sanitation. Suggestions to routinely monitor the various WASH indicators in schools and health facilities were not embraced. In Gulu, the DWO had planned to train Health Assistants and VHTs to undertake the monitoring on a monthly, but in a planning meeting with DHO where they (DWO) were not represented, the idea was dropped. Policy support towards exchange visits of stakeholders and beneficiaries from one region to the other was critical in relation to sustainability of benefits. For instance, one of the lessons learnt by leaders from Northern Uganda, who went for a learning exchange visit to Rakai in Central Uganda, was that local officials can use the existing “Public Health Act” to arrest and charge persons without toilets in their homes, but the political leaders feared the repercussions because of cultural sensitivities especially among the Acholi. They have instead preferred to develop their own Sanitation Ordinances which they have sent to the Attorney General’s Office for approval. They plan to disseminate them widely to secure ownership and thereafter enforce them.

48

7.4 Institutional capacity

There are indicators of opportunities available to support continuation of project activities through strengthening of institutional capacity. The Ministry of Health has facilitated VHTs with bicycles to enable them move across villages. VHTs are in charge of mass awareness, among others, on issues of hygiene and sanitation. The MoH also administers the Uganda Sanitation Fund whose aim is to improve livelihoods through increased access to better hygiene and sanitation practices.6 In line with the WASH Project, the Uganda Sanitation Fund which runs five year funding cycles to beneficiary districts also promotes open defection free communities. Soroti which has been a beneficiary under this Fund is presently running hygiene and sanitation promotion in three sub-counties of Gweri, Asuret and Katine. Outside the GoU structure, development partners, mainly NGOs have been running WASH programs across the country including those in the Project target areas. AMREF gave bicycles to all VHTs in Katine Sub County, Soroti district. According to the District Water Office Soroti, the increased access to safe water in Katine Sub County from 73% to 89% is attributed to AMREF’s total coverage of that sub county. World Vision did the same in Kamuda and Gweri. Despite the above mentioned strengths and opportunities, LGs capacity remains weak. This is manifested in inadequate manpower both in quantity and quality, both of which have been exacerbated by the creation of new districts in pursuit of increasing service delivery. The weak resource base translates into non-prioritirizing of the some activities, mainly the software aspects of O&M as well as monitoring and mentoring of communities by the district and extension staff. Given that aspects of sanitation and hygiene have a lot to do with behavior change, sustained monitoring and mentoring is key to making meaningful progress on stated objectives. However, a small resource base at the LG coupled with a restricted release from the central government makes it difficult for the LG to play the monitoring and mentoring role. The environmental health component aims at contributing to the attainment of a significant reduction of morbidity and mortality due to environmental health and unhygienic practices and other environmental health related conditions. There are noticeable gaps in environmental health staffing at all levels leading to inefficiencies in the areas of environmental health service delivery. Only 10% of the ADHOs in charge of environmental health positions have been filled in the districts (Annual Health Sector Performance Report, FY, 2010/2011)

What is glaringly missing are financial resources to undertake major repairs such as replacing whole pipes, pump heads, etc. LGs especially sub-counties are ill-equipped to shoulder the responsibility of O&M for boreholes – apparently, they can only write and submit complaints of broken water sources to the district. However, there is evidence that WSC are mobilizing water users to collect money for O&M – households contribute between UGX 500 and 1,000 per month.

6 Cf: http://www.wsscc.org/resources/resource-news-archive/uganda-sanitation-fund-receives-us6-million-wsscc-5-

year-programme

http://www.wsscc.org/resources/resource-news-archive/uganda-sanitation-fund-receives-us6-million-wsscc-5-year-programme

http://www.wsscc.org/resources/resource-news-archive/uganda-sanitation-fund-receives-us6-million-wsscc-5-year-programme

49

Within schools, sanitation/health clubs have been formed with patrons – this approach is envisaged to ensure sustainability of, among others, the hand-washing campaign even after the Project has closed

7.5 Socio-cultural factors

The WASH Project was implemented in different contexts with various socio-cultural dynamics; all of which can facilitate or constrain project sustainability. Overall, however the Project was in tandem with local needs and reception was overwhelming especially for water. All changes that occurred in the course of Project implementation were well accepted largely because the level of participation of stakeholders and relations between the Project staff and other stakeholders e.g., communities and LGs were impressive. In Moroto, for instance, support for better sanitation and hygiene from political leadership was reported as lacking or weak in most respects due to the socio-cultural dynamics in the Karamoja region. Whereas political leadership was reportedly supportive of sanitation and hygiene via public rallies, the same leaders did not display practical enthusiasm by erecting sanitation facilities at their own homes. The contradicting acts of the political leaders who are supposed to be exemplary to their followers, have served to undermine efforts of the extension staff as the latter strive to promote ODF villages.

7.6 Financial Sustainability

Financial sustainability taken as the willingness and capacity of beneficiary communities to sustain constructed facilities, the picture that emerges is rather mixed. There are, however, good practices that communities have evolved to ensure sustainability of WASH facilities. Initiatives for raising O&M funds such as forming users’ groups which engage in IGAs e.g., brick laying, commercial farming are emerging as good sustainability practices such as in Lalogi sub-county in Gulu district. Water users agreed to communally lay bricks as their contribution towards O&M of their water sources. At one point the group sold the bricks, which fetched at UGX 500,000/= for O&M, which could not have been raised if the users were to rely on monthly contributions from households. In addition, UNICEF secured a five year commitment from contractors to repair and maintain constructed water sources. Despite some of the evolving practices that can contribute to sustainability, it still remains a big constraint for the user communities. Unless the LG and central government play their respective roles communities and institutions on their own will keep on challenged to ensure continuity of Project provided facilities given the existing levels of poverty in the Project area. For instance, in the Karamoja region, apart from poverty, the nomadic life style undermines user participation in O&M of WASH facilities. Given such a situation, it is not uncommon to have recommendations as one given by an extension worker in Moroto, “areas like Moroto, and community contribution in terms of money (cash) is not feasible”. However, some respondents averred that such a view does not take into account the cultural aspects of the poverty problem. Accordingly, poverty in Karamoja is cultural not essentially economic, considering the amount of wealth kept in livestock-cattle, goats, sheep and donkeys. The Karamojong are less keen at selling their livestock to invest in ventures like WASH, which are not directly linked with the wellbeing of their livestock and/or which are interpreted as the responsibility of the state. Acceptance by some development partners and stakeholders of the popular view that the majority of the people in the sub-region are poor and incapable of making a quarter dollar

50

contribution per month to O&M have tended to support a reversal of WASH strategies. Accordingly, the tendency is for the district to finance repairs for water sources below the mandatory threshold of UG.Shs 300,000 which goes backward to the 1990s approach of total supply-driven. Other factors that affect financial sustainability include:

Beliefs among communities that they have a “right to safe water” whether they make contributions for O&M or not.

Contradictory messages by politicians especially local ones regarding O&M contributions

Absolute voluntarism by WSCs and source caretakers, which is impinging on O&M of facilities.

7.7 Technical (Technology) Issues

The type of technology used has got its own sustainability concerns. Training of hand-pump mechanics and retooling them was one measure the Project adopted to ensure sustainability of boreholes. As per the sector Guidelines, the responsibility of engaging HPMs as well as remunerating them for their services lies with the WSCs. However, WSCs hardly have contracts with the HPMs, and often find it difficult to meet the costs for repairs especially when they are major. All the water sources constructed by the Programme were functional at the time of this Evaluation implying the quality of the facilities. Quality of the facilities has a direct bearing on sustainability of the source. Other technologies, which are cheaper to sustain/maintain include

GFS and hence appropriate technology. GFS do not experience heavy breakdown, and even when it happens, it is mainly replacement of taps, which beneficiary communities easily afford. A Case of Buhara GFS illustrates this point—i.e. an institutional water source. Left: A Case of a Functional Institutional Water Source A water tap at Buhara Catholic Church which also serves a Nursery school. According to a Tap stand Committee member, the tap head got spoilt and was replaced one week before the visit of the Evaluation Team. The Committee also bought a locker to stop unscrupulous persons from vandalizing the tap. The locker cost 7000UGX, each user household contributes 500UGX per month. It is this money that the Committee uses for O&M. The Committee has seven members (3 male and 4 female).

51

8.0 GOOD PRACTICES, LESSONS, CHALLENGES AND RECOMMENDATIONS

8.1 Introduction In this evaluation some good practices as well as lessons that can inform future project implementation have been identified. Also identified are the challenges that were encountered in project implementation, which can form basis for learning and project replicability. Arising out of the evaluation results and challenges encountered in project implementation, recommendations have been generated aimed at promoting sustainability of project benefits, scaling-up and replicability.

8.2 Good Practices7 Three possible good practices that characterized the WASH project include beneficiary involvement, partnership with central and local governments as well as NGOs, and combining advocacy with service delivery.

8.2.1 Beneficiary involvement Given the fact that the project aimed at making a contribution towards the national WASH goal and objectives, it endeared to several partners whose participation was critical in the implementation of the project. Project implementation through partnership at the district level contributed greatly in realization of the project goal and objectives. There is evidence that project districts have increasingly showed signs of owning the project and supporting implementation to increase access to safe water coverage and improvement of hygiene and sanitation both in schools, health centres and in communities. Further, participation of beneficiaries in the project implementation that ensues is a counter-check measure that planned benefits are delivered to the right beneficiaries at the right time thereby promoting accountability and value for money. Pre-construction mobilization and preparation of communities of users to receive the water source fostered a spirit of ownership. The targeted users worked together with the contractor, right from the time of siting, clearing the site to construction of drainage channels and soak-pits. Materials for construction of the channels and soak pits which could be accessed locally, the community willingly provided them. Due to high level of involvement, community ownership of the water points has been largely achieved; none of the WSC indicated that they would wait for UNICEF or district to come and repair the water point if it broke down.

8.2.2 Partnership with local government, private contractors and other partners (NGOs) UNICEF implemented this project through local partners—districts, private contractors and non-governmental organizations, which was cost-effective and a solid ingredient for sustainability. UNICEF particularly involved the District Water Office in the planning for integrated WASH services which gave the district opportunity to fulfill their plans as laid out in their development plans. As a result, local governments were able to guide on where urgent need for WASH services existed—leading to effectiveness in project implementation. Thus, all

7 Good practice criteria comprise of effectiveness, replicability, ethical soundness, sustainability, cost-

effectiveness, relevance and innovativeness (UNAIDS criteria)

52

implemented WASH activities and services were in line with the priority needs of the supported communities.

8.2.3 Combining capacity building with service delivery. The project with two-pronged—i.e., building the capacity of partners especially local governments and the same time delivering services. The motivation for partners to undertake capacity-building interventions was tremendously aroused by construction of water and sanitation facilities in communities and schools. Institutional strengthening particularly among the lower local government is critical for sustainability. As reported elsewhere, training of district authorities and staff through study tours improved the capacity and confidence of the district government authorities in the eastern (Lango) region (GOU/UNICEF, 2011).

8.3 Lessons 8.3.1 Leadership and participation Leadership and effective participation of local leaders in project implementation is critical not only in ensuring smooth undertaking of activities but also eventual sustainability. Available evidence shows that the best performing districts in the project area were characterized by active involvement of community leaders at all levels, including engagement in allocation of budgets to hygiene and sanitation and enforcement of local sanitation by-laws (GOU/UNICEF, 2011). Through exchange visits and joint monitoring higher levels of motivation, commitment and a sense of peer competition were created among district staff and political leaders. On completion of any phase of activities, a team comprising both representatives from UNICEF and the district would spend two days monitoring and producing reports on accomplished projects. This ensured harmonious relations between the donor and the district. It is also noted that a strong district and sub-county technical leadership for coordination and management of limited government resources such as transport, logistics, cash and human resources is crucial for successful promotion of lasting hygiene and sanitation changes (GOU/UNICEF, 2011). 8.3.2 School pupils as an entry to improving household hygiene and sanitation The WASH project component in schools was meant to create awareness among school children as a way of promoting good sanitation and hygiene practices at school and home. By creating awareness children became change agents with regard to improved household sanitation and hygiene. School health clubs that were formed during project implementation ensured active participation of children in hygiene and sanitation promotion. Through school health clubs children’s roles and responsibilities in school sanitation and hygiene were defined. The eventual effect was noticeable improvements to cleanliness in and around latrines and the school compound. According to GOU/UNICEF (2011) there was a clear contribution by school health clubs to the sensitisation of their peers on safe hygiene practices. 8.3.3 Multiple impacts of water on school pupils Discussions with teachers and girls in the five schools that were covered in this evaluation revealed that provision of sanitation facilities especially the construction of latrines had contributed greatly to increased school attendance of pupils. This was reported to be particularly so for girls in upper primary.

53

8.3.4 Building capacity of local grassroot structures—WUCs for O&M of water sources Formation and training of water user committees (WUCs) is critical for sustainable O&M of water facilities. In the project districts WUCs were formed and trained over the period of project implementation and hence recording a higher functionality rate of water sources compared to non-project districts. WUCs need regular monitoring i.e., to be refreshed on their roles and function, which is critical for the functionality of the committees. 8.3.5 Institution of hand-pump mechanics Existence of trained and retooled hand-pump mechanics is a critical requirement in sustainability of boreholes. At least each of the sub-counties in the project district had a hand-pump mechanic to handle repairs and maintenance of boreholes. Because of active involvement of the hand-pump mechanics the failure rate of boreholes in project districts was extremely low. 8.3.6 Private sector and NGO partnering can be efficient and effective Private sector and NGOs unlike government agencies can be very efficient and effective in implementation of activities largely due to lessened bureaucratic processes encountered, among others, in procurement. Further, the fact that UNICEF managed the procurement process of contractors in a very transparent manner and selected from pre-qualified firms by the districts meant that even the most remote communities that would have been shunned by private contractors were served with the services. Use of centrally procured contractors guaranteed access to skilled and experienced service providers for even the remotest of the supported districts. 8.3.7 Holistic and flexibility in project design The holistic and flexible nature of the project—i.e., combining hardware with software activities or the latter coming later given the emergence nature of the communities in northern Uganda made the project produce maximum impact. This combined with sensitizations of communities on water chain management helped ensure that users collect safe water and guard it from contamination throughout the chain of use. People in some supported communities reported that in cases where the borehole produced milky (water with colour, not colourless), reports were made to the contractor through the District Water Office. In such cases, the contractor would return and sink some more pipes to reach better quality water in the ground. The lesson we draw here, is that testing for the quality of water guaranteed sustained use of a water source. No cases of abandoned water sources constructed with the EU-UNICEF fund have been reported. Instead, communities are using the opportunity to even report cases of suspected contamination at other water sources used by the community.

8.4 Implementation Challenges 8.4.1 Return of peace in northern Uganda and dismantling of IDPs camps The project was implemented during the period when peace was gradually returning to the districts in northern Uganda that had experienced a civil strife for almost two decades that had resulted into displacement of the population into internally displaced people’s camps (IDPs). During the era of IDPs services including WASH ones had been provided in the IDPs. The return of populations to their communities resulted into project implementation challenges. First, people returned to communities without WASH services, necessitating providing new services that had been provided in IDP camps. Second, formerly displaced persons were

54

returning to scattered communities that made planning, implementing and monitoring of project activities quite challenging. 8.4.2 Beneficiaries’ unqualified expectations and dependency syndrome The time spent in the IDPs meant that the populations were deprived of their sources of livelihood especially agriculture. A population that used to be self-reliant especially with regard to food supplies increasingly became dependent on external agencies for relief supplies and other hand-outs. Gradually, the once resilient population started developing unqualified expectations that translated into a dependency syndrome for most people. As observed, encouraging communities to build independent and sustainable systems without “hand-outs” or “facilitation fees” proved difficult throughout the north (GOU/UNICEF, 2011). 8.4.3 Low partners’ capacity District local governments spearheaded the implementation of the project activities as well as monitoring amidst a situation of limited staff capacity to execute this function. This situation was exacerbated by the creation of new districts by government, which further affected the districts’ staff capacity to implement and monitor project activities. As noted earlier, at least every sub-county had a trained hand-pump mechanic to undertake repairs of boreholes—minor and medium repairs. In a few instances where the repairs were beyond the capacity of hand-pump mechanics, it took a while before they could be rectified by district authorities. 8.4.4 Delays in implementation There were reported delays in implementation of the project activities during the inception phase and to some extent the subsequent activities. These delays were attributed to inadequate financial management capacity of central and local government partners. In situations where there were delayed district submission of accountabilities meant that UNICEF would not release funds for project activities in time.

8.5 Recommendations The recommendations provided here are not for the improvement of project implementation since this was end of term evaluation, but rather to enable local governments and central government consolidate sustainability of project activities. Second, the recommendations are meant to guide UINCEF and other partners on implementation of future similar WASH projects so as to realize high levels in efficiency, effectiveness and impact. 8.5.1 Sustainable O&M of water facilities The evaluation findings have revealed better O&M of water facilities in the project sites as compared to non-project sites (MWE/DWD/UNICEF, 2011). For O&M to be enhanced in the project sites so as it does not slacken, there is need for the following:

Continued strengthening of LGs capacity and building capacity for new districts, mainly technical capacity to support monitoring and mentoring. This requires investment in transport-vehicles, motorcycles alongside GoU efforts to increase staffing in local governments. Staffing levels for LGs need to significantly be improved for timely, efficient and effective service delivery. District water offices should have increased capacity to monitor and provide technical support to sub-counties

Train new WSCs and provide refresher training at least after every two years of their formation. Support and empower trained WSCs to enact and enforce bye-laws for O&M

55

of water facilities. Sub-counties should have adequate budget to fund training of WSCs and monitoring visits and provide back-up support to WSCs.

Lobby politicians not to misinform direct beneficiaries on their roles and responsibilities in O&M. In a similar manner, sensitize community and institution on their role in O&M, and especially for the former on the importance of collecting money for O&M.

Consider water for production as a strategic intervention for communities with livestock e.g., in Karamoja to help improve O&M of constructed water sources. Small changes that require structural designs may initially lead to substantial gains Right: An example of a once functioning (privately owned) hand-pump that has a watering trough for animals (Usuk Sub-sounty, Katakwi)

8.5.2 Hygiene and Sanitation Household sanitation and hygiene in the project area is yet to pick up especially in Karamoja area, which require special attention. On the other hand, sanitation and hygiene in schools in the project area have picked up, and only need to be sustained. Scaling up sanitation and hygiene will involve sustained active involvement of VHTs and engaging political leaders to endear them to be exemplary cases for sanitation and hygiene best practices. Specifically, the following is recommended:

Political support and enforcement - political leaders are key for the achievements of improved sanitation and, therefore, regular hygiene and sanitation advocacy meetings with LG political leaders are crucial. On the other hand, districts that developed sanitation by-laws should be supported to disseminate and enforce these laws in the lower LGs. Thus, enforce bye-laws at district, sub-county, community and school level and also have more involvement of CSOs especially in hard to reach areas.

Some project activities especially under sanitation require more frequent follow-up. During the project life, several communities were triggered to become ODF free and even model villages, but no follow-up was made to these villages due to lack of facilitation.

To further support ODFC there is a need for sustained monitoring to track the outcome of ODF. Again local leaders and VHTs will be key participants in this exercise.

Peg improved hygiene and sanitation to provision of a water source facility. Households in a potential recipient community for the water source should have acceptable latrine coverage and hand-washing facility next to latrine before a water source is provided

56

References

Government of Uganda, Ministry of Water and Environment, Water and Sanitation Sector

Performance Report 2007, September 2007 Government of Uganda, Ministry of Water and Environment, Water and Environment Sector

Performance Report 2011, September 2011 Government of Uganda, Ministry of Health, Annual Health Sector Performance Report

Financial Year 2010/2011 Government of Uganda, Ministry Of Health, Health Sector Strategic Plan III 2010/11-2014/15 Government of Uganda and UNICEF Country Programme 2010-2014, 4th Progress Report to the

European Union, SC/2007/0817 Sustainable WASH Initiative for the Rural Poor, in 21 Districts in Uganda Contributing to Attainment of the MDGs, October 2011

Government of Uganda and UNICEF Country Programme2010-2014, Justification of Proposed Adjustments, Addendum #2 Contribution Agreement 9 ACP RPR 50/26, Sustainable WASH Initiative for the Rural Poor in 21 Districts in Uganda Contributing to Attainment of the MDGs, June 2011

Ministry of Finance, Planning and Economic Development, Millennium Development Goals Report for Uganda 2010; Special Theme: Accelerating Progress towards Improving Maternal Health, September 2010

Republic of Uganda - European Community Country Strategy Paper and National Indicative Programme, 2008-2013

Soroti District Local Government, Annual Report for the Rural Water and Sanitation Conditional Grant FY 2010/2011

UBOS website (www.ubos.org); browse for statistics, population: accessed on 20th August 2012, last modified on Monday, 20 – Feb – 2012 at 6:48:18.

http://www.ubos.org/

57

Appendix 1: Map of Uganda showing the project area

Moy o

Kibaale

Katakwi

Lira

MubendeKy enjojo

Ntungamo

Sironko

Amolatar

JinjaMity ana

Ssembabule

Luwero

Isingiro

Bududa

Kaabong

Butaleja

Oy am

Kasese

Amuria

Kapchorwa

Kotido

Nakaseke Kaliro

Kanungu

Mbarara

May uge

Pallisa

Nebbi

Ly antonde

Rakai

Kiboga

Namutumba

Bukwa

Maracha

Dokolo

Kabale

AbimGulu

Amuru

Ibanda

Kamwenge

Budaka

Kaberamaido

Yumbe

Busia

WakisoMpigi

Kampala

Masindi

MbaleManaf wa

NakasongolaHoima

Iganga

Arua

Rukungiri

Busheny i

Pader

Buliisa Apac

KabaroleTororo

Kamuli

Kitgum

Kay unga

Masaka

Kalangala

Soroti

Kisoro

Koboko

Kumi

Moroto

Mukono

Bundibugy o

Nakapiripirit

Bugiri

Adjumani

Kiruhura

Bukedea

58

Appendix 2: Key informants at district, sub-county and community level

Category of Participants

Bundibugyo Soroti Gulu Katakwi Lira Moroto Dokolo Kasese Abim Kabarole Total

Teachers 3 - 3 3 3 - - 3 - - 15

HC in-charge - 1 1 - 1 - 1 - 2 1 6

WSC - - 1 - 1 1 - 1 1 - 5

GFS attendant - - - - - - - - - 1 1

Local Artisans/HPMs

- - 2 - 2 1 - - 1 - 6

DWO/ADWO 1 2 3 1 2 1 2 2 1 2 17

District Engineer

- - - - - - - - - 1 1

DHI 1 1 1 - 1 1 1 - 1 1 8

CDO 2 2 1 1 1 2 1 - - 1 11

HI-Sanitation 1 1 - 2

HA - - 1 1 1 - 1 - - 1 5

Sub-county Chief

- 1 - 1 1 1 - 1 - 1 6

PHAST trainees 4 - - - - - - - - - 4

CAO/ACAO - - - 4 - 1 - - - 1 2

DEO/DIO 1 - - 1 - - - - - 2

LC V Chairperson

1 - - - - - - 1 2

LCIII Chairperson

- 1 1 - - - - - 1 3

Councillor LC3 1 - 1 - - - - - - - 2

UNICEF Regional

- - 1 - - 1 - - - - 2