1. 6 2. 3. 4. - goalglobal.org...od open defecation odf open defecation free oecd - dac development...
TRANSCRIPT
i
1. 6
2. 3. 4.
GOAL UGANDA WATER, SANITATION AND HYGIENE (WASH)
PROGRAMME EVALUATION
AND ASSESSMENT AGAINST GLOBAL WASH AIMS AND PRIORITIES
CONDUCTED BY
FINAL REPORT, APRIL 2016
Prepared by: Anton Rijsdijk and Dennis Nabembezi
Quality assurance: Marie Körner, Daniel Svoboda
Director SaafConsult B.V.: Mr. Ele Jan Saaf
WASH COUNTRY PROGRAMME SUMMARY
ii
Country of implementation: Uganda Target Districts: Kampala; Bugiri, Namayingo, Agago,
Kaabong and Abim Districts
Project name: WASH Programme in Uganda Sectoral focus: WASH - Water, Sanitation and Hygiene
Donor: Irish Aid (Major), GOAL, Charity
water, Bank of Ireland (BOI), Nachsteni Lieb
Weltweit (NLW) and Japanese Embassy.
Implementer: GOAL Uganda - an international non-
governmental and non-political humanitarian
organization
Background:
GOAL WASH strategic objective is to continue to deliver holistic Water, Sanitation and Hygiene
programmes that target vulnerable groups in a timely and efficient manner whilst measuring impact,
retaining the capacity to respond to rapid onset emergencies, reducing disaster risk for the chronically
vulnerable and building the capacity of a full range of WASH stakeholders to continue to operate and
maintain installed facilities and carry out work in the future.
GOAL first began working in Uganda in 1979. The current GOAL Uganda (GU) programme has a
strategic focus on the north and east of the country, where poverty is more entrenched and access to
services less, when compared to the rest of the country. GU country programming reached
approximately two million people in 2015, and aims to build resilience and support socio-economic
development in three core programming areas; Livelihoods, WASH, and Health.
First operational in WASH in 2003, GOAL’s WASH programme has transitioned, with the changing
context, from emergency to a development and increasingly uses a systems approach to catalyze
sustainable access to goods and services. Programming is underpinned by national policy and GOAL
WASH strategy and M&E systems. Implementation in five districts, is both direct and through
partners; CSO, private sector and local government.
The current WASH strategy (2015) presents a six pillar approach of: water, Operation and
Maintenance (O&M), sanitation promotion, hygiene promotion, district networking and coordination
and cross cutting issues. GU actively prioritizes early adopting communities over low coverage or
access. In 2015, GU WASH programme reached 18,979 people directly, and 69,352 people indirectly.
GU is currently piloting an initiative to catalyze more sustainable O&M services with local
government and private sector partners.
The objectives of the evaluation:
The main purpose of the evaluation is to obtain objectively substantiated and consistent conclusions
that can be used in the decision making of GOAL on the future direction of GOAL in Uganda in the rural
WASH sector with focus on safe water supply, operation and maintenance of water services,
sanitation promotion (CLTS) and hygiene promotion.
In addition to the specific country evaluation objectives, an additional objective is to assess the
country programme against the strategic GOAL and objective for GOAL globally and against GOAL’s 10
Key WASH Principles with an objective measurement to allow cross-country meta-analysis.
Year IAPF GOAL
Charity
water BOI NLW
Japanese
Embassy
(GGP)
Annual
Budget(€)
2012 690,996 152,952 843,948
2013 503,813 43,282 547,095
implemen
tation
period:
2012–
2016
iii
2014 774,404 48,260 43,325 25,000 80,805 971,794
2015 783,156 66,589 874,821 26,507 1,751,073
2016 893,658 81,588 894,126 24,576 1,893,948
Total(€) 3,646,027 392,671 1,812,272 51,083 25,000 80,805 6,007,858
Figure 1: Map of Uganda showing GOAL Uganda operational areas:
iv
LIST OF ACRONYMS
BH Borehole
CBO Community Based Organization
CC Community Conversation
CHAST Child Hygiene and Sanitation Transformation
CLTS Community Led Total Sanitation
CPE Country Programme Evaluation
CSO Civil Society Organization
DHD District Health Department
DLG District Local Government
DP Development Partner
DRA Demand Responsive Approach
DSCG District Sanitation Conditional Grant
DWD Directorate of Water Development
DWO District Water Office
DWSCG District Water and Sanitation Conditional Grant
ENR Environment and Natural Resources
FGD Focus Group Discussion
FY Financial Year
GDP Gross Domestic Product
GEMA Local Enterprise
GI Galvanized Iron
GOAL International Humanitarian Agency, Ireland
GoU Government of Uganda
GU Goal Uganda
HH Household
HP Hand Pump
HPM Hand Pump Mechanic
HPMA Hand Pump Mechanics Association
HW Hand Washing
HWF Hand Washing Facility
IAPF Irish Aid Programme Funding
IDP Internally Displaced People /Camp
IMR Infant Mortality Rate
IWRM Integrated Water Resources Management
JMP Joint Monitoring Program
KII Key Informants Interview
LDG District Local Government
LG Local Government
M&E Monitoring & Evaluation
MDG Millennium Development Goals
MEL Monitoring, Evaluation and Learning
MMR Maternal Mortality Rate
MWE Ministry of Water And Environment
NDP National Development Plan
v
NEMA National Environment Management Authority
NFA National Forest Authority
NGO Non-governmental Organization
NWSC National Water and Sewerage Cooperation
O&M Operation And Maintenance
OCA Organizational Capacity Assessment
OD Open Defecation
ODF Open Defecation Free
OECD - DAC Development Assistance Committee of The Organization For Economic
Cooperation And Development
Pa Per Annum
PHAST The Participatory Hygiene and Sanitation Transformation
PTA Parent Teachers Association
RUWAS Rural Water and Sanitation Project
RWH Rain Water Harvesting
SCBA Social Cost-Benefit Analysis
SCWB Sub-County Water Boards
SDGs Sustainable Development Goals
SHC School Health Club
Shs Shillings
SM Sanitation Marketing
SMC School Management Committee
SP Spare Parts
SPR Sector Performance Report
SS Stainless Steel
TOC Theory of Change
TOR Terms of Reference
UGX Uganda Shilling
UMURDA Uganda Muslim Rural Development Association
UNICEF United Nations Children’s Fund
UNMA Uganda National Metrology Authority
UWASNET Uganda Water & Sanitation NGO Network
VAT Value Added Tax
VHT Village Health Team
VIP Ventilated Improved Pit-Latrine
VSLA Village Saving and Landing Association
WASH Water, Sanitation and Hygiene
WESWG Water and Environment Sector Working Group
WP Water Point
WSDF Water and Sanitation Development Facility
WSS Water Supply System
WUC Water Users’ Committee
WW Wagwoke Wunu
vi
EXECUTIVE SUMMARY
Introduction
This report sets out findings of the external evaluation of the GOAL Uganda (GU) Water, sanitation
and hygiene (WASH) programme (2012 to 2015) covering four districts of Namayingo, Bugiri, Abim
and Agago excluding Kaabong District where programming begun mid-2014. The evaluation was
conducted by a team comprising of an International WASH Consultant, a local Social Scientist with
extensive experience in WASH (all contracted by SaafConsult BV of Netherlands) and GOAL Uganda
country programme team members while Social Value for Money" was conducted by external
economist whose report is annexed to this report.
The overall objective of the evaluation was to obtain objectively substantiated and consistent
conclusions that can be used in the decision making of GOAL Global on the future direction of GOAL
in Uganda in the rural WASH sector with focus on safe water supply, operation and maintenance of
water services, sanitation promotion (CLTS) and hygiene promotion. The evaluation also set out to
assess the country programme against the 10 GOAL global WASH priorities and principles with an
objective measurement to allow cross-country meta-analysis.
This evaluation coincided with the development of the new Global WASH strategy (2017-2020), so
every attempt was made to conduct it in a way which would support that process, rather than
conflict with. The evaluation was carried out in a fully participatory manner, involving GOAL Uganda
Staff, GU partners, other sector stakeholders and communities throughout the exercise.
Approach
The approach used in this evaluation has been one which looks back and forward to examine the
past, recent and future activities of GOAL Uganda WASH programme as highlighted in the
evaluation terms of reference (ToRs) excluding Kaabong District WASH programme. Overall, the
approach has been one of trying to find the best fit/match between GOAL Uganda Country WASH
programme and performance against Global WASH aims and priorities. The image used in this
evaluation is one of trying to arrange a marriage between the two entities.
The evaluation relied on the OECD/DAC evaluation criteria based on a series of key questions, set
out in an evaluation framework, of relevance, efficiency, effectiveness, impact and sustainability as
well as the evaluation matrix for water, sanitation and hygiene programme. Data was collected on
five broad categories, one of which was secondary (literature review) and the rest primary which
included;
a) Data extraction from GOAL Uganda Monitoring and evaluation system data
b) One-on-one interviews with GOAL Uganda WASH staff and senior management,
c) Consultations with sector professionals,
d) Discussions with partner organizations during field visits, and
e) Discussions with districts and community beneficiaries (WUCs and CLTS teams).
An extensive review of GOAL Uganda’s internal documentation and sector external documents was
carried out. In addition to the interviews with key stakeholders, water points (WP) were inspected
on functionality construction quality, hygiene, water quality and fencing. The itinerary for the entire
vii
evaluation including names and contacts for the persons consulted for this evaluation is annexed to
this report.
Findings
Relevance: The WASH promotion of the GOAL country programme was very relevant due to low
water, sanitation and hygiene coverage figures. Across all districts, baseline figures show low
performance against the national average in relation to access to improved water, sanitation and
hygiene services. GOAL WASH programme intervention in construction of water points,
rehabilitation of water points, training of water user committees and provision of school sanitation
facilities including (hand washing facilities, facilities for the disabled, and a wash room for girls for
menstrual hygiene concerns) and promotion of household hygiene and sanitation through CLTS,
Demand Responsive approach and community conversations is directly contributing to Government
efforts in meeting their WASH objectives.
At national level (MWE and UWASNET) the message is even more clearer, GOAL Uganda through
their research work using a business model (making markets work for the poor- M4P approach) to
address the issues of accountability, collection and custody of water user fees meant for O&M is
already picking momentum. GOAL is recognized as an active and knowledgeable player on issues
related to functionality of water points, especially hand pumps. GOAL is currently leading an
informal learning platform of WASH sector NGOs on O&M of water facilities. .
Effectiveness: To a large extent, this evaluation has found GOAL Uganda WASH programme to have
achieved its goal and objectives. Data from the MEL database and reports indicate that over 263
new boreholes and 70 shallow wells were constructed over the programme period and 83
boreholes were rehabilitated across the target district to increase access to safe water. However,
data from GOAL indicate that during the project period 2012-2015 GU did not use IAPF funds for
new boreholes (these were installed prior to 2011 or with other funding from the Japan Embassy
and charity water).
Data further reveal that over 15,820 latrines were constructed, 321 villages were triggered and 183
villages were declared ODF (57%) during the last four years of the programme excluding Kaabong
district. In addition, 414 water user committees had been formed and trained to monitor and
undertake O&M of their water points constituting about 87% functionality of all GOAL monitored
water sources across the four districts. There has also been a reported reduction in the incidences
of diarrhoea among children below five years from 38.9% to 23% in 2015.
GOAL Uganda WASH programme can only be effective as its partners’ capacity can provide. To a
large extent, GOAL Uganda has built the capacity of local partners to effectively implement the
programme activities. Feedback from partners about the GOAL capacity building component is
rated satisfactory with the exception of the MEL.
GOAL Uganda WASH programme operates monitoring system to collect monitoring data for the
programme code named Monitoring, Evaluation and Learning (MEL) system. Three databases are
reported, updated regularly and verified including, sanitation (CLTS) and water and CHAST for
schools.
viii
Efficiency: GOAL Uganda WASH Programme employed cost effective approaches in the delivery of
services to the target communities. Five approaches stand out among the many and these relate to:
a) the lean and effective organizational structure, b) working through partnerships c) leveraging
existing resources [office equipment, human resources, and transport to mentions], d) selection of
approaches, and e) working with the private sector and early adopters of WASH promotion
activities.
Impact: Five levels of sanitation and hygiene practices were observed at both the community and
school levels, for example data from the MEL data system (2012-2015 follow up surveys), FGDs with
women and the CLTS team and individual household interviews indicate that over 17,155 household
latrines have been constructed and 183 villages have been declared ODF. The data also suggest that
there have been an overwhelming number of people who have started and continued to wash
hands after visiting the latrines and before eating food with over 15,796 hand washing facilities
(HWF) have been built by the community over the last four years.
Access to sanitation has improved to 86% in 2015 from below 50% at baseline (2012), improved
food handling and storage [clean and covered containers] now stands at 79% from 42% at baseline,
observance of the safe water chain has also improved to 18% from 4.7% at baseline, and the
number of households accessing safe water within 30 minutes has improved to 84% from 62% at
baseline.
There has also been reported reduction in the incidences of diarrhoea among children below five
years especially in ODF communities compared to OD communities [% of HH with children suffering
from diarrhoea in the past 2 weeks (3+ loose stools in 24 hrs)] from 38.9% to 23%. At school level,
the programme has equally improved sanitation and hygiene practices of the pupils through the
reduction of pupils-stance ratio (averaging at 1:49 across all the schools visited) from an average of
over 1:75 at baseline, provision of access to menstrual hygiene services (wash rooms) for the girl
child and hand washing.
Sustainability: The sustainability of community sanitation and hygiene programmes strongly
depends on a chain of links beginning with real demand, community participation (especially early
adopters) as well as community contributions related to adequate revenue generation for
maintenance and operation of the facilities established. It has been encouraging in this evaluation
to hear and see community own initiatives geared towards contribution (O&M fees) and sustaining
the ODF status in the respective villages, for example natural leaders, SHCs, private partners and
community facilitators, whose capacities have been built and the roles of women and children
sanitation and hygiene promotion.
The sustainability of the rural water supply is still a thorny issue in most, if not all, Sub Saharan
African countries. Some important factors which could influence the functionality of the GOAL
installed WP include ability of the WUC to collect sufficient funds for the maintenance and repairs
of the HP, leakage of the rising main pipe and the turbidity of water. Overuse could also become
serious constraint to the sustainability of the WPs because several WPs serve between 300 and 500
households, much more than the recommended number (between 250-300 households) for the
handpump. The network of HPM through their district based associations though with some
ix
difficulty especially non-payment of the repair fees from the WUCs, are still functional in some
districts but far from being sustainable.
However, GOAL is developing several promising approaches to improve the sustainability of the HP.
Interventions focus on three interrelated behaviour changes that are crucial to the success of an
alternative O&M model namely, a) O&M service provider(s) adopt and market a new O&M service
contract with WUCs, b) WUCs adopt a new mobile payment system and collect regular water fees
and c) Local Government increases its influence in regulation and enforcement. In the framework of
these changes, GU developed the following activities including contracts with commercial
enterprises, phone banking of O&M fees to improve collection, payment and efficiency of O&M
services and Commercialization of Hand pump mechanics.
Despite some progress, sustainability of the WPs is still vulnerable. The stability and ability to collect
sufficient funds of the WUC are the main bottleneck to sustainability. Recent initiatives with
commercial contractors aiming at a more commercialized approach on HPM and the phone banking
system are encouraging but not yet tested on a larger scale.
Table 1: Summary assessment1
Evaluation criteria Rate of fulfillment/ Score
Relevance High (5)
Effectiveness Rather high (4)
Efficiency Rather high (4)
Sustainability Rather low (3)
Impacts Rather high (4)
GOAL’ WASH programme assessment against global principles and priorities: GOAL WASH
programme performance against the global priorities and principles is rated High (score 5) with all
objectives met and there is an overall satisfaction with the intervention. All key aspects against the
10 principles are met and considered during both planning, implementation and monitoring of
programme activities. Integration, participation, gender mainstreaming, demand creation,
sustainability of approaches, partnership and capacity building principles’ performance stand out
among the 10 principles.
1 1 Very low (there are critical problems, the objectives cannot be reached, there are negative impacts)
2 Low (in spite of significant problems or dissatisfaction the objectives are still partly achievable)
3 Rather low (the procedures, results or assumptions do not fully meet the expectations)
4 Rather high (the intervention brings good results but there are negative external factors)
5 High (the objectives are met and there is an overall satisfaction with the intervention)
6 Very high (the objectives are fully met and the applied practices can be further disseminated)
x
Table 2: Summary rating
GOAL WASH Principles and priorities Score and rating
Principle 1: Addressing the ‘three prongs’ of WASH (Water, Sanitation, and Hygiene),
either as an integrated program, or in collaboration with other partners, including
equal considerations for men and women
High (5)
Principle 2: Community involvement and engagement in all aspects of programming High (5)
Principle 3: Gender mainstreaming Rather low (3)
Principle 4: Creating demand Rather high (4)
Principle 5: Sustainability of WASH services including environmental impact Rather low (3)
Principle 6: Appropriateness of interventions for beneficiaries, particularly vulnerable
groups
Rather high (4)
Principle 7: Focus on behaviour change Rather high (4)
Principle 8: Partnership and capacity building High (5)
Principle 9: Integrated approach to WASH programming Rather high (4)
Principle 10: Focus on reduction of vulnerability to future hazards Rather low (3)
Conclusions and recommendations
Conclusions
To a large extent, the process of WASH promotion at community level was highly active, interactive
and meaningful; save for the limited school sanitation promotion campaigns which focused on
fewer schools compared to demand for services. This level of participation is a precursor for
sustainability of latrine use, hand washing behaviour and functionality of the facilities at both
schools and communities
Although all WASH interventions by GOAL Uganda are undoubtedly important, some aspects need
more attention than others in the next country strategy. Issues related to sustainability of water
points (O&M), menstrual hygiene management such as making of reusable pads at both school and
community level/linking schools to private service providers, sanitation marketing to sustain the
gains created by Open defecation free (ODF) villages, responses to sanitation challenges created by
rapid urbanization through faecal sludge management, and new and appropriate alternative
technologies to water supply such as solar powered schemes ,, piped schemes and self-supply
system should be further explored and reflected.
GOAL Uganda WASH programme implemented effective measures to ensure delivery of the
programme results, for instance, improvement in access to WASH services (water and sanitation
including functionality), hygiene promotion [hand washing, food storage and safe water chain] as
well as reduction in reported diarrhoea incidence among children below five years stood out during
this evaluation.
Generally, the MEL system is working well and robust though with some limitations including
missing out on some key golden indicators collected by the district and MWE like gender on water
user committees (number of water points with women in key positions) and chemical water quality
in view of the corrosion problem which shortens the lifespan of the HPs.
xi
If GOAL’s efficiency is dependent on the performance of its partners, collaborators, contractors and
selected approaches then the nature of partnerships, collaborations, contracts and the
relationships between GOAL and its partners, collaborators and contracts is equally important.
The private sector partners though in their early stages of work show strong attributes for efficiency
and sustainability which should be more embraced than the CSO partnerships.
The GOAL WASH programme was designed to contribute to the Districts and the National
Development Plan and achievement of the longer term effects on the target beneficiaries. Since
there are a number of players in the district and the national development field all targeting to
some extent the same beneficiaries, the programme can only contribute to those longer term goals.
Despite some progress, sustainability of WPs is still vulnerable. The stability and ability to collect
sufficient funds and accountability of the WUC are the main bottlenecks. Recent initiatives with
commercial contractors aiming at a more commercialized approach on HPM and the phone banking
system are encouraging but not yet tested on a larger scale.
Recommendations
Recommendations related to the GOAL Sanitation and Hygiene program in Uganda
1. Focus on issues related to menstrual hygiene management such as making of reusable pads or
linking to a private provider at both school (CHAST manual) and community level, sanitation
marketing to sustain the gains created by Open defecation free (ODF) villages and respond to
sanitation challenges created by rapid urbanization (faecal sludge management using
indigenous micro-organisms – IMOs and other approaches of collection, transportation,
disposal, treatment and reuse).
2. Exploit the knowledge and learning niche at the national level by investing more in cutting
edge research and share widely on appropriate sanitation and hygiene promotion approaches
which are cost-effective and able to generate demand for household sanitation and hygiene
behavioural change as well as addressing sustainability issues (a combination of CLTS, DRA and
CC is a good example).
Recommendations related to the GOAL Water program in Uganda
1. Build capacity of the CSO partners to make meaning on the MEL data collected and appreciate
it uses in planning and monitoring of their projects.
2. Work with UWASNET and other WASH CSOs to influence MWE policy recommendation on the
use of non-corrosive water pipes such as PVC and stainless steel pipes.
3. Standardize functionality criteria with those of District Water Office (DWO).
4. Encourage, support and monitor new initiatives on the sustainability of the WP, such as
commercialization of the HPMA, cooperate with existing commercial partners and develop,
test and rollout the phone banking system fully.
5. Pilot with the installation of small scale (solar powered) reticulation system, gravity flows and
pumped up schemes in locations with a high water demand and poor underground water
quality (salty and hard water).
Recommendations related to the GOAL processes and mechanism
1. Build the capacity of politicians and LG staff (health assistants) on CLTS and other sanitation
xii
promotion approaches to support the programme efforts through enforcement of the public
health act and promote sustainability of the interventions.
2. Apply the sanitation and hygiene promotion approaches (CLTS, DRA and CC approaches) in a
systematic and sequential manner to exploit the comparative advantages of each while
strengthening each other for better results. This evaluation found, the best approach would
be to enter a community with CC to identify and prioritize community problems. This would
be followed by a combination of DRA and CLTS in that order with some overlaps.
3. Build the capacity and involve the school administration (PTA/SMC) on their roles and
responsibilities in monitoring the quality of works of the school sanitation facilities to ensure
quality works, value for money and sustainability.
4. Undertake a phased approach from CSO partners to engage more of the private
sector/business oriented partners for longer term sustainability of interventions.
5. Review the cost, price and pomp of the ODF celebration parties in line with purpose and
intended benefits to the programme’s sustainability and value for money.
6. Exploit the opportunity and take lead of the niche of sector learning on O&M in the country.
UWASNET might be a good vehicle for this. It will provide the power to influence a number of
issues in the sector and also adoption of the different O&M models that have been
developed, piloted and scaled by GOAL Uganda for larger sector replication and adoption.
Recommendations related to GOAL Systems
1. Align the MEL system to the national WASH sector performance golden indicators for
sanitation, collaborate with DLG water and health officials in collection of WASH monitoring
data and also build the capacity of the CSO partners on making value of the data to
appreciate the importance of the system.
xiii
TABLE OF CONTENTS
LIST OF ACRONYMS ......................................................................................................................................... iv
EXECUTIVE SUMMARY .................................................................................................................................... vi
1. INTRODUCTION AND BACKGROUND .......................................................................................................1
1.1. Introduction ...........................................................................................................................................1
1.2 Overview of GOAL Uganda Country WASH programme ....................................................................2
2 COUNTRY CONTEXT AND THE WASH SECTOR .....................................................................................4
2.2 Uganda country context .......................................................................................................................4
2.3 WASH sector in Uganda ......................................................................................................................6
2.3.1 Summary of the current situation .....................................................................................................6
2.3.2 Emerging issues: Needs and priorities for GOAL Uganda’s attention .............................................7
2.3.3 WASH Sector institutional framework 2 . ..........................................................................................9
3 EVALUATION APPROACH AND METHODOLOGY ................................................................................. 11
3.1 Approach ................................................................................................................................................. 11
3.2 Methodology ............................................................................................................................................ 11
3.3 Evaluation Constraints............................................................................................................................. 12
4 EVALUATION FINDINGS .......................................................................................................................... 14
4.1 Relevance ................................................................................................................................................ 14
4.1.1 How was community participation ensured and how did selection criteria reflect the most
vulnerable populations? ................................................................................................................ 14
4.1.2 What specific needs of children and women were considered in designing school latrines and
water facilities? .............................................................................................................................. 15
4.1.3 What roles did women and children play in WASH intervention? ................................................. 16
4.1.4 What are the priorities of the Government, partners and project participants? ............................ 17
4.1.5 Conclusion on relevancy ............................................................................................................... 18
4.2 Effectiveness of interventions ........................................................................................................... 19
4.2.1 Achievement of results and theory of change ............................................................................... 19
4.2.2 What is the most significant behaviour change caused by the programme? ............................... 20
4.2.3 To what extent did the intervention contribute to the increased capacity of local stakeholders to
maintain WASH services? ............................................................................................................. 21
4.2.4 What was the most motivating approach for the changes in sanitation and hygiene behaviour? 24
4.2.5 Is there any monitoring system in place? , Who is using the results? For what kind of decisions
are the results used? Are the results easily available? ................................................................. 24
4.2.6 Conclusions on effectiveness: ....................................................................................................... 25
4.3 Efficiency and value for money ......................................................................................................... 25
4.3.1 What is the project’s operational cost-efficiency? ......................................................................... 25
xiv
4.3.2 How effective were the institutional arrangements for the implementation of the Programme with
focus on roles and responsibilities of GOAL and partners? .......................................................... 28
4.3.3 Conclusion on efficiency ............................................................................................................... 28
4.4 Impact ................................................................................................................................................ 29
4.4.1 What were the main contributions of the programme to changes in sanitation and hygiene
practices of the most vulnerable members of communities? ........................................................ 29
4.4.2 How has the program impacted on access to drinking water for the most vulnerable members of
communities? ................................................................................................................................ 30
4.4.3 What extent has demand for improved sanitation increased as a result of the project (Demand
Responsive Approach)? ................................................................................................................ 30
4.4.4 To what extent has demand for drinking water services increased as a result of the project? .... 30
4.4.5 As a result of the project, are there demonstrated changes in willingness to pay for improved
access to drinking water? .............................................................................................................. 31
4.4.6 Do the beneficiaries register any positive impact on health status as a result of the intervention?
……………………………………………………………………………………………………………...32
4.4.7 Conclusions on impact .................................................................................................................. 32
4.5 Sustainability ..................................................................................................................................... 33
4.5.1 To what extent were the risk factors to sustainability included in the monitoring system? ........... 33
4.5.2 Has an exit strategy been agreed with partners during formulation? ........................................... 33
4.5.3 What is the extent of risk that ODF communities will slip back to non-ODF status? And what are
the main problems in proper latrine maintenance? ....................................................................... 34
4.5.4 Are there any new local (locally owned) initiatives to improve sanitation? ................................... 35
4.5.5 Sustainability of the Water supply systems ................................................................................... 35
4.5.6 New approaches for sustainability of the water points .................................................................. 37
4.5.7 Conclusions on sustainability ........................................................................................................ 38
5 ASSESSMENT OF GOAL WASH PROGRAMME AGAINST 10 KEY WASH GLOBAL PRINCIPLE AND
PRIORITIES ...................................................................................................................................................... 40
5.1 Principle 1: Addressing the ‘three prongs’ of WASH. ....................................................................... 40
5.2 Principle 2: Community involvement and engagement in all aspects of programming .................... 40
5.3 Principle 3: Gender mainstreaming ................................................................................................... 41
5.4 Principle 4: Creating demand ............................................................................................................ 41
5.5 Principle 5: Sustainability of WASH services including environmental impact ................................. 42
5.6 Principle 6: Appropriateness of interventions for beneficiaries, particularly vulnerable groups ........ 42
5.7 Principle 7: Focus on behaviour change ........................................................................................... 43
5.8 Principle 8: Partnership and capacity building .................................................................................. 43
5.9 Principle 9: Integrated approach to WASH programming ................................................................. 43
5.10 Principle 10: Focus on reduction of vulnerability to future hazards .................................................. 44
6 CONCLUSIONS AND RECOMMENDATIONS ......................................................................................... 45
xv
6.1 Conclusions ....................................................................................................................................... 45
6.2 Recommendations ............................................................................................................................ 46
7 ANNEXES .................................................................................................................................................. 48
7.1 Graphs showing GOAL progress on key WASH indicators – Follow up survey data (2012-2015) .. 48
7.2 Short profile of GOAL partners.......................................................................................................... 55
7.3 List of people interviewed .................................................................................................................. 57
7.4 References ........................................................................................................................................ 61
7.5 Terms of reference ............................................................................................................................ 63
7.6 Itinerary for the evaluation ................................................................................................................. 82
List of tables
Table 1: Summary assessment ............................................................................................................................ ix
Table 2: Summary rating ...................................................................................................................................... x
Table 3: GOAL Uganda timeline of significant shifts in the WASH country programme .....................................1
Table 4: Districts and sub-counties where CLTS is implemented by GOAL excluding Kaabong District ..............3
Table 5: Uganda basic statistical data ..................................................................................................................5
Table 6: Ministry of Water and Environment Selected WASH golden indicators ...............................................6
Table 7: Number of site visits and interviews conducted ................................................................................. 12
Table 8: Programme output achievement ........................................................................................................ 19
Table 9: Progress of the rehabilitation of WP's by the DWO in 2015 ............................................................... 36
Table 10: Comparison of functionality data between GOAL and DLGs ............................................................ 36
Table 11: Assessment scores for GOAL WASH global principles and priorities ................................................ 44
List of boxes
Box 1: A Shitty Start: ......................................................................................................................................... 21
Box 2: Brief about partners, contractors and collaborators ............................................................................. 27
List of figures
Figure 1: Map of Uganda showing GOAL Uganda operational areas: ................................................................ iii
1
1. INTRODUCTION AND BACKGROUND
1.1. Introduction
This report sets out findings of the external evaluation of the GOAL Uganda (GU) Water, sanitation
and hygiene (WASH) programme (2012 to 2015) covering four districts of Namayingo, Bugiri, Abim
and Agago excluding Kaabong District where programming begun mid-2014. The evaluation was
conducted by a team comprising of an International WASH Consultant, a local Social Scientist with
extensive experience in WASH (all contracted by SaafConsult BV of Netherlands) and GOAL Uganda
country programme team members.
The overall objective of the evaluation was to obtain objectively substantiated and consistent
conclusions that can be used in the decision making of GOAL Global on the future direction of GOAL
in Uganda in the rural WASH sector with focus on safe water supply, operation and maintenance of
water services, sanitation promotion (CLTS) and hygiene promotion. The evaluation also set out to
assess the country programme against the 10 GOAL global WASH priorities and principles with an
objective measurement to allow cross-country meta-analysis.
This evaluation coincided with the development of the new Global WASH strategy (2017-2020), so
every attempt was made to conduct it in a way which would support that process, rather than
conflict with. The evaluation was carried out in a fully participatory manner, involving GOAL Uganda
Staff, GU partners, other sector stakeholders and communities throughout the exercise.
GOAL has been implementing WASH programmes in Uganda since 2003 and the last Country WASH
Programme Evaluation (CPE) was carried out in 2012. The period since 2003 has been one of
considerable change for the Country Programme approach and staff. This background of change
persists to the present day. Some of the most significant events for the WASH Country Programme
include transitioning from an emergency approach, through crisis recovery to a development
approach through the districts.
Table 3: GOAL Uganda timeline of significant shifts in the WASH country programme
Agago Abim Bugiri Namayingo Kaabong
2003
Emergency
Crisis recovery
Development
CSO
partnership
2008
Crisis recovery
Development
CSO
partnership
2010
Development
CSO partnership
Private sector
partnership(2015)
2010
Development
CSO partnership
Private sector
partnership(2015/16)
2014
Development
CSO
partnership
The evaluation has been carried out in awareness of these changes which GOAL Uganda WASH
programme has experienced over many years. This view has been taken from the outset of the
evaluation that this is the time to build on what exists, to consolidate the best of the change which
has taken place, and to avoid recommending further changes which may undermine or stall the
continuing growth towards full effectiveness of the WASH Country Programme. Nevertheless,
certain changes are proposed, and guidance is offered toward the development of a more focused,
more strategic country programme.
2
1.2 Overview of GOAL Uganda Country WASH programme
GOAL was first operational in Uganda in the late 1970’s. Over the last 30 years, GOAL has worked in
the Northern, Southern, Eastern and Western regions of Uganda. Today GOAL Uganda (GU) has an
administrative office (head office) located in Kampala and satellite offices in Abim, Agago, Kaabong,
Lira and Bugiri Districts. GU has three core competencies and programming areas including
Livelihoods, WASH and Health.
The WASH programme comprises of two major components:
• Safe water supply (provision of water via borehole drilling) and operation and maintenance
of water services (training of WUCs, sustainability of hand pumps) hence improved access to
drinking water.
• Sanitation promotion (CLTS, beginning of sanitation marketing) and hygiene promotion (via
community conversations, demand driven approaches, child hygiene and sanitation
transformation (CHAST) in schools hence improved access to Sanitation and Hygiene at both
school and household level.
In 2013/14 GOAL Uganda (GU) conducted mapping of all water points (community, private and
institutional), and Open Defecation Free (ODF) communities in all operational areas. GOAL’s
intention was to work intensively in a geographic area where other country programme
components are being implemented to stimulate demand and increase coverage of water,
sanitation and hygiene.
The emphasis for GOAL Uganda’s WASH programme over the period (2012-2015) has been to
develop additional and to sustain existing water points and to increase community access to
sanitation and hygiene. This has been implemented both directly and through national partners
(see brief of partners in annex). Additionally, along with partners, GOAL has been engaging more at
a district level in order to strengthen institutional capacity particularly around the area of
coordination.
Since 2010, and across the operational districts [Bugiri, Namayingo, Abim, Agago (since 2012) and
Kaabong (since 2014)], GOAL has built or rehabilitated 390 water points and trained Hand Pump
(HP) mechanics and Water User Committee (WUC) members to maintain these points. No new
water points were planned under the current programme under evaluation. GOAL Uganda also
aimed at further involving the private sector actors in operation and repair of water points through
assisting them in co-funding business propositions, development of business plans and linking them
to WUC and District Local Government.
Sanitation through Community Led Total Sanitation (CLTS) since 2011 has been implemented in 4
districts including Namayingo, Abim, Agago and Bugiri which across 22 sub-counties.
3
Table 4: Districts and sub-counties where CLTS is implemented by GOAL excluding Kaabong District
District Namayingo Bugiri Agago Abim
Sub- county Banda Bulidah Adilang Abim S/C
Mutumba Muterere Kalongo TC ABIM T/C
Kotomor Alerek
Lamiyo Lotuke
Lapono Morulem
Lira Palwo Nyakwae
Lukole
Omiya
Paimol
Parabongo
Patongo
Wol
Totals 02 02 12 06
4
2 COUNTRY CONTEXT AND THE WASH SECTOR
2.2 Uganda country context
This section presents some basic data (see table 6) and highlights some particular features of
Uganda as an operating environment for GOAL Uganda WASH programme.
Population: Uganda is characterized by an extremely high fertility rate (6.2) with a population of
about 34.6 million people with correspondingly high population growth rate estimated as 3.3% per
annum. This represents a doubling every two decades. Urbanization is taking place at an even
higher rate, estimated at 4.5% pa2.
Environment: Demographic and other pressures are putting great stress on the natural and built
environments. Deforestation is taking place at an alarming rate. Inevitably this is leading to soil
erosion in some parts of the country and very likely it is also changing the water balance.
Urbanization, with its attendant challenges of pollution, sanitation, storm water drainage and solid
waste management, is a growing problem especially in low-lying unplanned/informal settlements.
There is little hard evidence yet of climate change, but existing variability combined with other
environmental stresses is leading to greater human impact of floods, droughts and lowering of the
water table in some parts of the country.
Security issues: With peace returning to the northern, eastern regions and the reduction in
Karamojong incursions from cattle rustling to petty theft, three significant unresolved issues
remain: first the plight of those who were in Internally displaced people’s (IDP) camps for so many
years, but who are now resettling onto their former lands, where services are very poor; second the
chronic poverty of the Karamajong themselves, occupying the dry north-east. A third and more
generic issue is the need for services in rural growth centres, small towns and Uganda’s cities. As in
rural areas, water is fundamental, but the problems of sanitation are much wider and more difficult.
Outside of the urban centres and in the other regions there is real need, but the problems of the
north and north-east, and of the towns, are probably the most pressing.
Politics and governance: Uganda is a multi-party democracy, but the opposition is relatively
ineffective and the Head of State has just won a new five-year term of office (2016 general
elections) through his sixth term in power. As Uganda moved towards elections this year many
expected that, as on previous occasions, vote-winning measures will be deployed by the ruling
party. Before the last elections the graduated tax was abolished, thus -constraining the capability of
district and sub-county local Government to offer services to the community through locally
generated revenue. There was also an introduction of Value Added Tax (VAT) - 18% on water which
increased the unit cost of water especially for piped water supply and bottled water.
Seven new districts have been created in 2016 and will be operational in a phased manner over the
next three financial years (2016/2017 to 2018/2019). The total number of districts at the time of
writing this report was 112. Division and creation of new districts creates significant start-up and
transaction costs, diverting investment from public services like drinking water. There is also a
tendency of limited focus on sanitation and hygiene at national and district level (since it is not a
vote-winner) and political interference in issues around water point sustainability is likely to
increase (as politicians tell their constituents that water tariffs/user fees for O&M are not necessary
5
as water is “a free good” provided by the Government while some politicians have offered to repair
boreholes during the recently concluded political campaign period.
Development priorities: In contrast to the basic needs for WASH services highlighted by GOAL
globally, the direct poverty-related needs for improved livelihoods, food security and health
services including WASH as highlighted in the National Development Plan (NDP II) and Vision 2040,
call for an approach which “intertwines economic growth and poverty eradication”, focusing on
“growth, employment and socio-economic transformation for prosperity” and transforming the
nation to a middle class economy by 2040.
The NDP II also highlights wealth creation as a means to a prosperous future, with a particular
emphasis on the water sector; Water for Production. Furthermore, the Head of State has recently
been calling for a greater emphasis on physical infrastructure, specifically roads and energy, and
there is concern that this may further divert budgetary allocations away from drinking water and
sanitation.
Table 5: Uganda basic statistical data2
Aspect Uganda Bureau of Statistics Summary Data
Geography Altitude (min ASL) 620 m (max ASL) 5,110 m
Total surface area 241,551km2
Area under land 199,807 km2
Area under water and swamps 41,743 km2
Temperature 15-31° C
Rainfall 735-1863 mm/year
Economy GDP at current market prices UGX 58,865 billion
Per capita GDP at current market prices UGX 1,638,939
GDP Growth rate 4.7 percent
Per capita GDP growth rate 1.1 percent
Contribution of agriculture to GDP at current market prices 20.9%
Reserves -234.7 million US$
Inflation rate 5.5 percent
Demography Population in households 34,650,070
Population growth rate 3.3%
Urban population 6,426,013 (19%)
Rural population 28,430,800 (81%)
Crude Birth Rate 42.1%
Total Fertility Rate 6.2%
Sex Ratio at birth 103
Population Density (persons per Sq km) 174
Health Infant Mortality Rate 54/1000
Maternal Mortality Rate 438/1000
Contraceptive Prevalence Rate 30%
HIV Prevalence rate 7.3%
Education Net enrolment at pre-primary level: 10.1%
Primary school enrolment: 8.5 million
2 Uganda National Population and Housing Census 2016,
6
Pupil/Book ratio: 4:1
pupil teacher ratio: 49:1
Pupil Classroom ratio: 57:1
Literacy rate: 34%
2.3 WASH sector in Uganda
2.3.1 Summary of the current situation
The recent (2015) Sector Performance Report (SPR) for the Ministry of Water and Environment
(MWE) estimates that about 35% of Ugandans in rural areas don’t have access to safe water and
23% practice open defecation3. Further an estimated 12% of all water sources in the rural areas are
non-functional which further decreases access. Hygiene indicators are even bleaker, with almost
three-quarters of the Ugandan population (67%) and another three quarters (62%) of school
children lacking access to hand washing facilities with water and soap3.
There is also a significant difference between coverage [urban and rural] and figures across the
different sources [Joint Monitoring Program for Water and Sanitation - JMP and Government of
Uganda- GoU] due to the difference in the definition of “improved access”. For example, for access
to water, GoU assesses coverage based on known infrastructure multiplied by the set/fixed number
of users for each improved source (depending on the source type) while the JMP draws upon data
collected across the country. For the case of sanitation, the GoU national estimates include less
rigorous standards such as shared facilities and some unimproved latrines in the definition of
improved sanitation while the JMP does not4. This difference in the definition brings about
significant differences in reported data for both water and sanitation.
GOAL Uganda has demonstrated strong experience in collecting monitoring data related to WASH.
This experience can be widely shared for sector influencing and monitoring at both national and
district level.
Table 6: Ministry of Water and Environment Selected WASH golden indicators
Selected Golden
Indicators
Rounds
Baseline
-2015,
endline
(2012)
JMP5 Government of
Uganda6
WASH 2015
Target Districts (2015)7
GOAL Uganda MEL data in brackets ()*
Rural Urban Bugiri Namayingo Agago Abim
Safe water coverage:
% of people within
1,000 m (rural) and 200
m (urban) of an
improved water source
Endline 61% 65% 65% 73% 55%
(65%)
34% 68%
(92%)
65%
(87%)
Baseline 57% 64% 69% 35% 35% 32% 53%
3 Ministry of Water and Environment Sector Performance Report 2015
4 WHO/UNICEF Joint Monitoring Program for Water and Sanitation
5 WHO/UNICEF Joint Monitoring Program for Water and Sanitation
6 Ministry of Water and Environment Sector Performance Report 2015
7 GOAL Uganda Monitoring evaluation and Learning (MEL) data 2015
7
Sanitation coverage:
% of people with access
to improved sanitation
Endline 35% 75% 77% 84% 82%
(75%)
59%
60%
(89%)*
56%
(80%)
Baseline 32% 70% 81% 57% 57% 58% 26%
Hand washing (soap +
water/ash)
Endline 24% 33% 23%
(13%)
29%
16%
(63%%)
*
38%
(40%)
Baseline 21% 27% 28% 10% 10% 26.5% 27%
Functionality: % of
working improved water
sources
Endline 60% 84% 88% 92% 74%
(89%)*
74%
(89%)*
70%
(92%)*
65%
(83%)*
Baseline 58% 82% 85% 46% 46% 55% 46%
Gender: % of Water
User committees/Water
Boards with women
holding key positions
Endline 84% 67% (93%)*
(90 %)*
(95%)*
(82%)*
Baseline 82% 45%
()* GOAL Uganda MEL data in brackets ()*: Only covers GOAL operational Sub-counties while GoU and
JMP data covers the whole district and the whole country.
The Government considers that WPs which are out of order for 5 years or longer are considered
abandoned and removed from the functionality lists. This leads to the strange situation that when
there is a long backlog of major maintenance, the coverage decreases, but the functionality
increases. Comparing national data with international data: Uganda coverage of access to improved
water sources (65%) is slightly above the Sub-Sahara Africa average (56%).
Concerning sanitation, Uganda’s 35% improved rural sanitation is also above the Sub-Saharan Africa
average of 23%. However, Uganda falls short on achieving the MDG target. There is also a
difference of definition on functionality between Government and GOAL: For Government non-
functional WP doesn’t produce any water, while according GOAL definition a hand pump which
takes more than 10 strokes to produce water or 50 strokes to fill a 20 litre jerry can is not
functional.
2.3.2 Emerging issues: Needs and priorities for GOAL Uganda’s attention
Overall, Uganda’s WASH sector is performing moderately well, but it still faces numerous practical
challenges. Those highlighted here are not exhaustive, but they relate most closely to opportunities
where GOAL Uganda could contribute.
Sanitation and hygiene promotion:
Sanitation in the country has been sub divided into three (3) sectors8 under a memorandum of
understanding between (Ministry of health, Education and Water and Environment) with the
Ministry of Health as lead agency for household sanitation and hygiene9. Some progress has been
8 Sanitation under the memorandum of understanding is sub divided into Health for household sanitation and
hygiene; Water for sanitation in urban areas and rural growth centres, and Education for sanitation in schools. 9 Uganda Sanitation Fund, Country Programme Proposal, Ministry of Health, February 2014
8
made in defining roles and responsibilities of the numerous actors in the sanitation sub-sector, and
a dedicated budget line for sanitation now exists in all three ministries. There is a long way to go
before sanitation is addressed by the Uganda Government as systematically as drinking water
supply.
Activities are very fragmented still, and the budget line has no mother ministry as is scattered in all
three line ministries with limited funds allocated. For example, during FY 2014/15, UGX 2.0 billion
(Approx. 4.5 Million EUR) was released under the District Sanitation Conditional Grant (DSCG), with
89 districts receiving an average of UGX 23 million (approx. 6,000 EUR) under the sanitation grant.
This level of funding is not only meagre but also doesn’t cover all districts. Activities are
implemented in only two sub-counties covering a total of 4 parishes and about forty villages out of
the many sub-counties in the district.
Further, the sector still grapples with issues of capacity; the majority of the district staff is not
trained in new sanitation promotion approaches like CLTS, and secondly the district offices are
heavily understaffed and unequipped (lack transport and facilitation capacities) to promote
sanitation in the districts. The sector is also challenged with finding an effective approach to create
and sustain demand and positive behavioural change. Most household latrines constructed by
communities through CLTS, use local materials which are prone to collapse potentially causing Open
Defecation Free (ODF) communities to lapse to Open Defecation (OD).
If sanitation is the poor relation to water supply, then hygiene promotion has tended to be the
orphan in the sector. Few systematic attempts exist in which sufficient time is taken, and a
structured process followed, to ensure that hygiene ‘messages’ are heard, internalized, practiced
and sustained i.e. behavioural change. Current hygiene promotion approaches focus on household
hygiene [hand washing, dish racks, bath shelters and a safe water chain], with little attention to
other aspects of personal hygiene including menstrual hygiene management for school girls and
women in the community. Although GOAL Uganda has some minimal good examples and
experience in this area [construction of segregated sanitation facilities with wash rooms], much
could be done in terms of research and sharing good experiences and practices with other sector
players for adoption and learning in the next country WASH strategy.
Financing:
The sector receives funds from the Government of Uganda (GoU) composed of treasury releases
known as on budget support including government’s own resources and development partners’
contributions, whereas off budget support is composed mainly of donor funds independently
accessed by organisations 3. The Government of Uganda’s commitment to finance core WASH
programmes has decreased in real terms with the percentage of the national budget spent on
WASH currently at 2.82% from 4% in the recent past (2012/2013).
It is estimated that the majority (94%) or more, of all financial resources are allocated to water,
water production, water resource management, environment and climate sub-sectors, leaving
sanitation with 6% or less. The story is also not very different at community level, with less
prioritization given to sanitation financing in terms of improving household sanitation facilities from
traditional to improved pit latrines (VIP) through acquisition of sanitation products. Although there
is some enthusiasm to improve on the sanitation facilities, lack of access to sanitation products like
9
slabs, vent pipes and skilled labour in rural communities or the low levels of income in the country
side are limiting factors.
GOAL Uganda WASH country programme investment in sanitation, and early work on menstrual
hygiene, sanitation marketing and plans for faecal sludge management (in 2017), is well placed to
close some gap in financing sanitation at district and community levels.
Operation and maintenance:
Last but by no mean least, numerous aspects of sustainability need further work. Software activities
which are supposedly carried out by Districts prior to, during and after water supply construction
are often weak or neglected. It is clear that in many cases), revenues raised by communities for
rural water supplies are insufficient to cover Operation and Maintenance (O&M) and replacement
costs. Undertaking major repairs (any repair above approx. $100), is the responsibility of Local
Government through the District Water and Sanitation Conditional Grant (DWSCG) which allocates
about 13% to O&M.
The district based Hand Pump Mechanic Associations (HPMA) and the newly created Sub-county
Water Boards, where staff is often un-coordinated and not motivated has the responsibility to
undertake O&M works in the districts. Evidence shows that Sub-country Water Boards are
ineffective in increasing functionality (IRC/MWE 2014). The HPMA lack access to spare parts and
there is low business sense. The majority of the WUCs don’t want to pay for services due to
perceived link to government services which are “free”. As is the case with sanitation, O&M is
poorly funded. The result is reduced access to water as water points in need of major repairs
remain broken for extended time periods.
With limited funding to meet the Sustainable Development Goals (SDG) for water and sanitation,
the role of self-supply, focus on high quality research on approaches that can create and sustain
demand for household sanitation improvement, private sector engagement in operation and
maintenance for water supply and faecal sludge management, especially for rural growth centres, is
being supported by the MWE and in which GOAL Uganda has had limited involvement in the past,
offers useful ways in which GOAL Uganda can make positive contribution to the sector in the next
country strategy.
2.3.3 WASH Sector institutional framework 3
Uganda can rightly be proud of what is one of the most advanced Sector Wide Approaches (SWAPs)
in the WASH sector in Africa. Major progress has been made in terms of harmonization and
coordination of donor and GoU approaches and systems, decentralization, and contracting-out of
construction.
Sector processes include the annual Joint Sector Review, the Joint Technical Review, DP
Coordination, Sector Performance reporting, and the activity of national thematic working groups.
At District level the District Water Supply and Sanitation Coordination Committees offer the main
opportunity for WASH sector coordination. Eight regionally based Technical Support Units (TSUs)
provide support to the District Water Offices.
10
The Water and Environment sector consists of two sub-sectors: The Water and Sanitation (WSS)
sub-sector and the Environment and Natural Resources (ENR) sub-sector. The WSS sub-sector
comprises water resources management, rural water supply and sanitation, urban water supply and
sanitation, and water for production. The ENR sub-sector comprises environmental management;
management of forests and trees; management of wetlands and aquatic resources; and weather
and climate. The institutional sector framework consists of:
• The Ministry of Water and Environment with the Directorates for Water Development
(DWD), Water Resources Management (DWRM) and Environmental Affairs (DEA);
• Local Governments (Districts and Town Councils), which are legally in charge of service
delivery under the Decentralization Act;
• A number of de-concentrated support structures related to MWE, at different stages of
institutional establishment, including Technical Support Units (TSUs), Water Supply
Development Facilities (WSDFs), and Water Management Zones (WMZs);
• Four semi-autonomous agencies: (i) National Water & Sewerage Corporation (NWSC) for
urban water supply and sewerage; (ii) National Environment Management Authority
(NEMA) for environment management; (iii) National Forestry Authority (NFA) for forestry
management in Government’s Central Forest Reserves; and (iv) the Uganda National
Meteorological Authority (UNMA) for weather and climate services;
• NGOs/CBOs (coordinated through UWASNET and ENR CSO Network) and Water User
Committees/Associations;
• The private sector (water and sanitation infrastructure operators, contractors, consultants
and goods suppliers) currently under UWASNET but plans are underway to create a
separate and independent department for private sector at the Ministry of Water and
Environment.
Activities undertaken in Sanitation and Water for Production (mainly focusing on agricultural and
animal production) require close coordination with other line ministries including the Ministry of
Health, Ministry of Education and Sports and the Ministry of Agriculture. The Water and
Environment Sector Working Group (WESWG) provides policy and technical guidance and includes
representatives from key sector institutions (GoU), Development Partners and NGOs).
The policies, legislation, structures and guidance documents are in place to allow effective
functioning of the sector (with the exception of sanitation). The greatest challenges consist of the
translation of policy to practice at the District level and below where GOAL Uganda could be of
great contribution.
11
3 EVALUATION APPROACH AND METHODOLOGY
3.1 Approach
The approach used in this evaluation has been one which looks back and forward to examine the
past, recent and future activities of GOAL Uganda WASH programme as highlighted in the
evaluation terms of reference (ToRs). Overall, the approach has been one of trying to find the best
fit/match between GOAL Uganda Country WASH programme and performance against Global
WASH aims and priorities. The image used in this evaluation is one of trying to arrange a marriage
between the two entities.
On one side of the marriage is GOAL Uganda Country WASH programme performance against the
OECD DAC evaluation criteria (relevance, efficiency, effectiveness, impact and sustainability) of the
activities performed over the last WASH country strategy (2012-2015). We have also painted a
picture of the opportunities which exist for consolidation, further strengthening and more effective
contributions to the WASH sector in Uganda while identifying GOAL Uganda’s comparative
advantage. On the other side of the marriage is the country programme’s performance against
Global WASH aims and priorities [here we assess the performance of the WASH country programme
against the 10 global aims and priorities10
].
The approach has been participatory in nature, carried out as a joint activity with both internal and
external stakeholders represented. For example, GOAL staff has added to the team the
organizational perspective/approach, the independent consultants have brought in their expertise
to guide the process, the Ministry of Water and Environment (MWE) and Partner CSOs have added
to the mix the sector priorities and opportunities available for exploring in the next country
strategy, and the target communities (District and community participants/beneficiaries) have
shared their priorities, concerns and experiences of GOAL WASH Country programme.
3.2 Methodology
a) The evaluation relied on the OECD/DAC evaluation criteria11
, based on a series of key
questions, set out in an evaluation framework, of relevance, efficiency, effectiveness,
impact and sustainability as well as the evaluation matrix for water, sanitation and hygiene
programme.. Data was collected on the evaluation questions using primary and secondary
methods: Data extraction from GOAL Uganda M&E data system
b) An extensive review of GOAL Uganda’s internal documentation and sector external documents (
see annex 7.4)
c) One-on-one interviews with GOAL Uganda WASH staff and senior management,
d) Consultations with sector professionals,
e) Discussions with partner organizations during field visits, and
f) Discussions with districts and community beneficiaries (WUCs and CLTS teams).
10
1 - Three prongs’ of WASH (Water, Sanitation, and Hygiene), 2 - Community involvement and engagement, 3 - Gender mainstreaming, 4 - Demand creation, 5 - Sustainability, 6 - Appropriateness of interventions, 7 - Behavioural change, 8 - Partnership and capacity building, 9 - Integration and 10 - Reduction of vulnerability of communities to future hazards. 11
The DAC Principles for the Evaluation of Development Assistance, OECD (1991), Glossary of Terms Used in Evaluation, in ‘Methods and Procedures in Aid Evaluation’, OECD (1986), and the Glossary of Evaluation and Results Based Management (RBM) Terms, OECD (2000)
12
In addition to the interviews with key stakeholders, water points (WP) were inspected on
functionality12
, construction quality, hygiene, water quality and fencing (see table 7 for the sites
visited and households interviewed). The itinerary for the entire evaluation including names and
contacts for the persons consulted for this evaluation is included in Annex 7.3 and 7.6 respectively.
Data analysis was conducted concurrently with fieldwork in a reflexive and a projective manner. A
content-driven and thematic approach was used to make meaning and describe relationships
between variables as they emerged from the respondents. A copy of all field interviews written in a
similar format is included in Annex 7.7. The interviews were carried out by the WASH expert (WUC's
interviews and physical inspections of the WP) and a Social Scientist (CLTS interviews). The field
checks/visits were triangulated with the internal database of GOAL.
In addition to the data collected in the field, the team made use of the GOAL MEL databases on
water points, CLTS, CHAST and water user committees (WUC) as well as of data from the WASH
follow up surveys implemented between 2012 and 2015.
Table 7: Number of site visits and interviews conducted
District # of WUC
interviewed
% of the
WUCs
interviewed
# of CLTS
teams
interviewed
# CLTS people
interviewed (*)
# of HH
latrines
inspected
# of
Interviews
with HPM
Bugiri 4 18% 2 36 10 2
Namayingo 6 9% 2 45 10 2
Agago 12 12% 6 46 20 2
Abim 7 5% 2 50 10 2
Total 29 9% 12 117 50 8
3.3 Evaluation Constraints
a) Attribution
• The evaluation design was non-experimental, one-shot. This design is insufficient to
demonstrate that the intervention (programme) alone caused the change (causality),
but is the only option available in the absence of a reference group randomly identified
before the intervention.
• GOAL Uganda did not work alone in the target communities, other CSOs, Government
and interventions are also taking place which make sole attribution of the results to
GOAL Uganda WASH programme difficult.
• The evaluators used the assumed attribution, self-reporting from respondents and the
follow–up survey data to draw conclusions about impacts.
b) Representativeness
• The surveyed villages have been preselected by GOAL on basis of categories such as
(non) functional, triggered and ODF. This does not allow drawing conclusions for the
whole population.
• The evaluation team went with guides from GOAL and its partners to the target
12
The DWO on definition on functionality was applied: Non functional WP means no water at all
13
communities. Also the local languages translators were staff member from GOAL's
partners. This could have led to "politically correct" answers to please the interviewers.
• Fortunately, most of the information collected in the field could be crosschecked with
the extensive WASH databases of GOAL MEL. As far as this could be crosschecked, the
field information matched the info of the databases.
14
4 EVALUATION FINDINGS
4.1 Relevance
The WASH promotion of the GOAL country programme was very relevant due to low water,
sanitation and hygiene coverage figures especially for latrine coverage, access to safe drinking
water and hand washing (see table 6, selected WASH indicators at baseline/endline). Across all
districts, baseline figures show low performance against the national average in relation to access
to improved access to safe water, sanitation and hygiene services.
4.1.1 How was community participation ensured and how did selection criteria reflect the most
vulnerable populations?
Participation was ensured at three levels (a) planning; b) Implementation and c) monitoring of
water, sanitation and hygiene (WASH) promotion activities. At planning level, the community and
the district local authorities were involved in a baseline survey to determine the state of affairs
including neediest areas and least served areas (WASH baseline survey 2011/2012). The baseline
survey provided the rationale for the selected sub-counties and communities to work with. Through
coordination and start-up meetings with the district officials, some sub-counties which were having
low coverage for WASH and being served by another development partner (CSO) were excluded
from those where GOAL intervened to avoid duplication of resources and efforts.
What is clear from the baseline survey findings and information from the District water/health
office across all the selected districts, Agago and Abim stood out. WASH coverage in some sub-
counties both at community and schools as low as 30%. For example, sub-counties like Wol had
sanitation coverage close to 28%, Lukole 36% and Alum was about 45% according to data from
Agago District health office. This could also be explained by the fact that majority of the population
were starting to re-settle in their original places and communities from a 20 year civil war which
had forced them into IDP camps.
Districts and sub-county engagements also provided key contact persons to work with, including
the District Health inspectors (DHI), Health Assistants (HA) and the Water Officers. The planning
meetings also were vital in mobilizing both political and technical support towards the programme
implementation. The community further participated in making action work plans for hygiene and
sanitation promotion after the CLTS trigger meetings, writing applications for water sources at the
sub-county level, collection of community capital contribution (approximately $ 100) for new water
points for both school and community water points and site selection.
At implementation, community participation was very evident in the CLTS activities [including CLTS
pre-triggering activities, triggering, post triggering and scaling up], Community Conversations (CC)
and Demand Responsive Approaches (DRA). Under CLTS activities the community was involved
selection of natural leaders, CLTS triggering, mapping of open defecation (OD) areas, community
action planning, construction of sanitation and hygiene facilities (latrine and hand washing, bath
shelters and drying racks), monitoring of ODF status and open defecation free (ODF) declaration.
In both Namayingo and Bugiri Districts, DRA was used to drive the community to construct
household sanitation facilities to access water points for example one of the conditions for the
15
community to receive a borehole was 100% sanitation and hygiene coverage, therefore, the process
was used as a catalyst for household hygiene and sanitation improvement. CC on the other hand,
though in its early stages, was meant to help the community through community selected
facilitators to identify their own problems and action plans to solve them. A limited number of
meetings have so far been undertaken and one action plan has been developed in Abim in relation
to whistle blowing in case somebody was found open defecating in the community.
Community participation in water services was rather unique, with majority of the beneficiary
communities during implementation taking part in site selection, formation and selection of the
WUC members, development of an O&M plan and appointment of the caretaker for the water
sources. Other implementation participation activities included, construction of the fence around
the water source, drafting of bylaws for the water point including collection of water user fees
across all the target districts. The contractors undertook the rest of the activities until the water
point was completed and handed over.
Participation during monitoring was ensured through the CLTS teams and health inspectors at Local
Government (LG) levels. The CLTS team members are community volunteers who periodically
undertake household visits to ensure sustainability of ODF status. Some of the key activities are to
encourage households to undertake repair and maintenance of their sanitation facilities and also
feedback to the GOAL partner project staff on progress. The district and sub-county Health
Assistants and Inspectors undertook joint monitoring visits to encourage and enforce sanitation in
the target communities.
However, participation of school authorities save for the training of the children, formation of
health clubs and a committed construction committee (as per the selection criteria for the
beneficiary schools), participation in the construction of sanitation facilities was low and deemed
passive. Interaction with the school authorities indicate that they were not informed on their roles
in supervision, simple operation and maintenance and they only received completed facilities
during handover ceremony. This could probably explain the lack of adherence to approved physical
plans for the school latrines. For example on all school approved designs, the stance for the
disabled was to also double as the washrooms for girls but in all most all the facilities in Namayingo
and Bugiri district the finishing (roughcast not watertight) of these facilities could not support this
purpose yet they had been handed over as completed.
4.1.2 What specific needs of children and women were considered in designing school latrines
and water facilities?
Across all the targeted schools, three children’s needs were put into consideration in the design of
latrines, for example all school latrines were designed with drop holes which were small in size not
to scare away young users and the design of the urinals were gender sensitive (for both
girls/women and boys/men) and child friendly. First, they were segregated for boys/men and
girls/girls to provide appropriate segregation and privacy. Secondly, the boys’ urinals were designed
with a raised platform to allow the urine flow in the drainage away channel and not to dirty their
feet while urinating. Third, the latrine blocks (unlike in the previous approved Ministry of Education
and Sports design) where the boys and the girls side were separated by a brick curtain wall, the
project constructed standalone blocks located about 100 meters apart for boys and girls to further
improve on privacy.
16
The girls’ sanitation facilities were also designed with a washroom to cater for menstrual hygiene
needs. However, across all school latrine constructed in Namayingo and Bugiri Districts this aspect
though clearly indicated on the physical plans where the stance for the disabled would also double
as a washroom for girls, the finishing was roughcast and had not be completed to standard to serve
their intended use despite the facilities being handed over as completed.
At community level, some attempts were made to consider the needs of the children in designing
household latrines. One key aspect was segregation of the children’s latrine from the adult one with
special features. One of the rooms (stances) was slightly smaller compared to the adult stance, with
a small drop hole and in some cases especially for the very young between 2-3 years, there was a
designated place where the children were instructed to defecate and the parents/caretakers drop
the faeces in the latrine. This was meant to maintain the cleanliness of the household latrine as the
children would make it dirty or the fear that the children may fall into the latrine since they were
very young.
This evaluation however found meaningful consideration for children’s and women needs in
relation to water services design. Children like women were included on the WUC purposely to
encourage fellow children on proper use of the water point. The women on the other hand were
mandated to occupy at least three key positions (chairperson, treasurer and secretary) of the
committees. Interaction with children and women found at the water points during inspection also
revealed that the pumps were soft especially those which were well maintained and functioning
making it easy for the children and women to pump. The design of the apron of all the water points
had a raised platform to support the height of the children to effectively pump water from the
source.
4.1.3 What roles did women and children play in WASH intervention?
When water and sanitation services are not functional or far way, the women and children suffer
most in terms of lost time to collect water, risk of diseases and sexual violence when they have to
use the bush for convenience. The evaluation found that, although household latrine construction is
a man’s responsibility, both children and women had clearly identified roles in WASH promotion at
home. Interaction with CLTS groups and women FGDs across the four districts indicate that the
roles of women and children vary between the two regions, as regards support towards
construction and maintenance of the household sanitation and hygiene promotion facilities [latrine,
hand washing, bath shelter and drying rack].
In both Namayingo and Bugiri Districts, the women and girls were heavily involved in the cleaning
and maintenance of the sanitation facilities including sweeping and provision of anal cleansing
materials and ensuring that water for hand washing, soap or ash were available at the facility. While
men and boys ensured that the latrine, drying racks, bath shelter and hand washing facilities were
constructed in the home.
The scenario was however different in the other two districts of Abim and Agago. The roles of the
children [both boys and girls] were identical to the women/mothers in the home. The children and
their mothers ensured that grass for roofing was collected, fetched water to make bricks/mud for
the construction of the super structure, collected logs for the slabs and smeared the walls and the
17
floor with cow dung for all sanitation facilities. In addition, the women and the children were
responsible for the maintenance of the cleanliness of the facilities and provision of water and
ash/detergent for hand washing. The men’s role on the other hand was to undertake the pit
excavation and construction of the facilities.
The evaluation also found significant roles for both children and women in water services
promotion, at community level. The women are involved in management of the water sources
where they serve as committee members [occupy three key positions] even in some cases
chairperson of the WUCs. The evaluation found at least 7 water point committees headed by
women across all the districts. The women and the children were also responsible for boiling
drinking water at households. In schools using the CHAST approach both girls and boys undertake
hygiene and sanitation promotion activities including general cleaning, hygiene and sanitation
parades, and maintenance of personal hygiene. Through the CHAST approach, the children also
make outreaches to the community to sensitize the community on safe water, sanitation and
hygiene.
4.1.4 What are the priorities of the Government, partners and project participants?
In an attempt to provide access to safe water and sanitation services, creating demand for services,
contribute to sector learning and addressing the issues of operation and maintenance for water
supply facilities, the GOAL Uganda WASH programme is undoubtedly relevant.
At national level, the Government of Uganda through National Development Plan II (2015-2020)
and Vision 2040 is committed to improving access to safe water to 80% for rural and 100% for
urban and increasing sanitation coverage to 90% for both urban and rural by 2020.
Uganda is also a signatory to the recent Sustainable Development Goals (SDGs) which aim at
ensuring water and sanitation for all by 2030 (Goal #6). GOAL Uganda WASH programme
intervention in construction of water points, rehabilitation, training of user committees and
provision of school sanitation including (hand washing facilities, facilities for the disabled, and a
wash room for girls for menstrual hygiene concerns) and promotion of household hygiene and
sanitation through CLTS, Demand Responsive Approach and community conversations is direct
contribution to Government efforts in meeting their WASH objectives.
At national level (MWE and UWASNET) the message is even more clear, GOAL Uganda through their
research work using the business model (making markets work for the poor- M4P approach) to
address the issues of accountability, collection and custody of water user fees meant for O&M is
already picking momentum. GOAL is recognized as an active and knowledge player on issues related
to functionality of water points especially hand pumps. GOAL is currently leading an informal
learning platform of WASH sector NGOs on O&M of water facilities. GOAL should therefore
continue to engage in cutting edge research and share learning for the wider sector adoption to
further exploit this niche.
At district level there is overwhelming evidence of relevancy of GOAL’s WASH interventions. All
districts baseline figures show low performance against the nation average in relation to access to
safe water, functionality and access to improved sanitation and hygiene services (see table 4 on
selected WASH golden indicators). All district level human and financial capacity was worryingly low
18
[for example each district is able to drill only 20 boreholes in two sub-counties and on average have
UGX 25 million for sanitation promotion in only 40 villages] each financial year.
The water and sanitation offices at the districts also operate on close to 50% staffing level (District
Staffing Survey of 2014) with limited capacity in sanitation promotion approaches like CLTS. GOAL’s
WASH programme intervention to improve access to WASH services to the target communities
through water point construction, rehabilitation of water sources, training and formation of water
user committees, hygiene and sanitation promotion through CLTS and improvement of school
sanitation and hygiene (Bugiri and Namayingo district) as well also facilitating district sector
coordination meetings (Agago district) is highly appreciated and relevant to the target districts.
GOAL’s rationale for working in all of its selected districts, with all of its selected partners, and in
the particular ways of working which prevails in each project is even more relevant. While all of
GOAL’s selected districts are especially poor, underserved or otherwise challenged compared to
national average statistics for WASH services at baseline, Agago and Abim stand out here more than
the others.
Three main reasons compound Agago and Abim vulnerability: a) both districts are recently
recovering from a 20+ year civil war which made all systems collapse, b) the two districts are
challenged with very low water table and issues that relate underground water (hard water)
including limited water resources (like Namayingo and Bugiri) and c) the population settlement
patterns and low levels of income. The investment over the last Country WASH programme (2012-
2015), shows improvement regarding access to WASH services since the intervention started (see
table 6) for some WASH indicators at baseline and end line across target districts.
4.1.5 Conclusion on relevance
To a large extent, the process of WASH promotion at community level was highly active, interactive
and meaningful save for the limited school sanitation and hygiene promotion (in terms of coverage
compared to the community water supply).. This level of participation is a precursor for
sustainability of latrine use, hand washing behaviour and functionality of the facilities.
Although all WASH interventions by GOAL Uganda are undoubtedly important, some aspects need
more attention than others in the next country strategy. Issues related to sustainability of water
points (O&M), menstrual hygiene management such as making of reusable pads at both school and
community level/ linking schools to private service providers, sanitation marketing to sustain the
gains created by Open defecation free (ODF) villages, responding sanitation challenges created by
rapid urbanization through faecal sludge management and exploring new and appropriate and
alternative technologies to water supply such as solar powered schemes, , piped schemes and self
supply.
Criterion score: 5 – High (the objectives are met and there is an overall satisfaction with the
intervention)
19
4.2 Effectiveness of interventions
This section looks at the extent to which the programme objectives have been achieved taking into
account their relative importance. The effectiveness of the GOAL WASH programme was assessed
by looking at the extent the programme, and its respective annual plans were executed in a timely
manner and according to accepted standards and whether they achieved the intended objectives.
4.2.1 Achievement of results and theory of change
GOAL Uganda WASH programme through an agreed theory of change committed to achieving a
goal of improving the target population’s health with a particular focus on maternal child health
outcomes (MDG 4 and 5) using a public health approach. The achievement of the above goal was
dependant on five key outcome indicators:
a) Adoption and maintenance of appropriate hygiene behaviour by men, women and children
in the target districts;
b) Access to sufficient quantities of safe water for men, women and children within
operational areas;
c) Access to and use of sanitation facilities by men, women and children within operational
areas;
d) Access to and use of hygiene facilities/products by men, women and children within
operational areas;
e) Communities actively manage their own WASH facilities within operational areas.
This evaluation has found GOAL Uganda’s WASH programme to have achieved this goal and
objectives. Data from the MEL database and report indicate that over 263 new boreholes and 70
shallow wells were constructed over the programme period and 83 boreholes were rehabilitated
across the target district to increase access to safe water. Data further reveal that over 15,820
latrines were constructed, 321 villages were triggered and 183 villages were declared ODF during
the last four years of the programme excluding Kaabong district. In addition, 414 water point user
committees had been formed and trained to monitor and undertake O&M of their water points
constituting about 87% functionality of all GOAL monitored water sources across the four districts.
Table 8: Programme output achievement
District # of Bore
Holes
constructed
# of Shallow
Wells
constructed
# of
rehabilitated
WP
# WP
monitored
#Villages
triggered*
# Villages
declared
ODF*
# of latrines
constructed*
# of hand
washing
facilities
constructed*
Bugiri 34 17 0 56 59 36 3,559 2,902
Namayingo 33 31 3 91 100 45 7,147 6,605
Agago 83 15 8 141 85 52 2,844 3,334
Abim 113 7 72 122 77 50 2,270 2,263
Total 263 70 83 410 321 183 15,820 15,104
*Excludes Kaabong district which was not part of this evaluation
20
4.2.2 What is the most significant behaviour change caused by the programme?
Five levels of behaviour change can be attributed to the programme across the four districts; key
among them is the increased latrine use, increased hand washing during critical times, safe water
handling, food handling and adoption of semi-permanent and permanent latrine structures.
Data from the GOAL MEL system further indicate substantial achievement of the goals and
objectives of the country programme. For example, the WASH follow up surveys (2012-2015) clearly
indicate that the number of households washing hands with soap during the five critical times
[before eating, handling food, after visiting the toilet, after cleaning the baby faeces and before
feeding the baby] has increased from below 10% to about 27% across the target districts however
the percentage is higher for each isolated critical time.
Access to sanitation has improved to 86% from below 50% at baseline, improved food handling and
storage [clean and covered containers] now stands at 79% from 42% at baseline, observance of the
safe water chain has also improved to 18% from 4.7% at baseline and the number of households
accessing safe water within 30 minutes has improved to 84% from 62% at baseline (see annex 7.1
for individual district statistics on performance of key golden indicators by the programme).
Interaction with CLTS groups, women FGDs and selected household visits among the ODF declared
villages show that communities have adopted latrine use (see box 1 for an example of the village
initiative for latrine construction and use), in almost all households sampled (although the sample is
not representative to make scientific conclusion), all households had at least some form of faecal
matter containment.
The majority of latrines were traditional in nature (made of mud/wattle, roofed with grass and logs
as floor and the door made of a sack or some old piece of cloth). There was overwhelming evidence
of use including a well cut-out path to the latrine; absence of the faeces around the toilet and a look
inside was a clear indication of recent use. Further still interaction with the household heads
indicated that almost all household members used the latrines more regularly than before and all
children’s faeces were dropped in the latrines as opposed to burying it. This was further evidenced
by the number of ODF declared villages and latrines built by communities in the last four years of
the programme (see table 8).
Through Household visits, the evaluation team also found a hand washing facility either a tippy tap
made of a jerrycan or a used recycled mineral water bottle with majority of households having ash
instead of soap for hand washing. Some of the critical evidence of hand washing found was
dampness of the ground of the hand washing facility, half empty water in the hand washing jerry
cans/bottles and presence of an interrupted heap of ash which indicate that the facility was being
used.
21
Most eye catching of all behavioural change practices has been the adoption of semi-permanent to
permanent latrine structures compared to temporary sleeping houses. Travelling through the two
districts of Agago and Abim, one is treated to this iconic behavioural change characteristic. One will
see many households with
baked/unbaked bricks,
sand/mud and roofed
with iron sheets / grass
latrine structures (with
some having plastic san
plats/slabs) compared to
family houses made of
mud and wattle which
equally need repair and
replacement from time
and again.
Interaction with the CLTS
groups, women and the
individual household visits
revealed that the cost of
establishing a latrine is
high (approximately $70
to excavate the pit) as
opposed to a house which
is almost free using local
materials and household
labour. Without money,
the process is tedious and laborious which has made people to adopt a more permanent structure.
Others have commented that it is all about the value placed on the household latrine in terms of
reducing diarrheal incidences among children and the influence of the shit demonstration during
the CLTS trigger meetings.
Important to note however, the process of behavioural change is slowly taking root across all the
districts. The majority of the household latrine structures are traditional and temporary with
communities mentioning lack of access to affordable sanitation products which may affect the
communities’ momentum to move higher the sanitation ladder.
4.2.3 To what extent did the intervention contribute to the increased capacity of local stakeholders to maintain WASH services?
GOAL Uganda WASH programme can only be effective as its partners’ capacity can provide. To a
large extent, GOAL Uganda has built the capacity of local partners to effectively implement the
programme activities. Feedback from partners about the GOAL capacity building component is
rated satisfactory with the exception of the MEL.
Box 1: A Shitty Start:
Community Led Total Sanitation (CLTS) is a well-tested community
empowerment methodology for eliminating open defecation. CLTS focuses on
the behavioural changes needed to ensure real and sustainable
improvements. The process starts with triggering emotions of disgust and
shame in order to trigger the desire for collective change. This does not
always go as planned…
Agwee, is a small village in Kotomor Sub-county in Agago District in Northern
Uganda. As planned the community undertook their analysis and as expected
felt shame and embarrassment. However, instead of deciding on collective
action, they decided to stop the process.
The powerful influence of early adopters
Meantime four neighbouring villages were successfully triggered and became
Open Defecation Free (ODF). When they held their ODF celebrations, some
community members from Agwee village attended; they were challenged and
asked why they too had not become ODF.
After the celebrations, new water points were drilled in the four ODF villages.
The people from Agwee village had no safe water source and appealed to
their neighbours to allow them to fetch water from the newly constricted
water points, their neighbours refused, and made their neighbours feel
ashamed of their sanitation situation. As a result, the community from Agwee
village decided to construct latrines and hand washing facilities. It did not
take long before they too became ODF and received a water point.
22
Purposeful capacity building measures were taken on four levels:
a) Community (CLTS team members and natural leaders),
b) School level (health clubs and sanitation teachers),
c) Local implementing partners (CSOs), and
d) Though not targeted like the above three due to lack of specific Organizational Capacity
Assessment (OCA) for the district personnel. Equally important was lack of deliberate
targeting of some key offices at Local Government level including Health assistants and
inspectors vital in sanitation and hygiene promotion including enforcement of the Public
Health Act in the district and politicians vital for their influence and development of bylaws
regarding water, sanitation and hygiene promotion.
At community level, between five (5) to ten (10) CLTS members, natural leaders and community
facilitators were identified during the CLTS trigger meetings to support sanitation and hygiene
promotion activities in the village. These were taken through a series of different trainings by the
CSO partners on CLTS approaches, Community Conversations and DRA. Key among the roles of
these community volunteers (facilitators) was to mobilize communities, monitor sanitation
promotion in the village and sensitize fellow community members on sanitation and hygiene
behaviour.
Information gathered from a few sampled households visited reveal that these were instrumental in
influencing the design of the latrines, (for example communities agreed on the depth of the latrine
ranging from 15-25 feet in Agago district) and construction of household sanitation and hygiene
facilities. Their value is further underscored by the different names and titles given to them by the
communities they serve. For example, in Bugiri and Namayingo Districts due to their vigilance,
communities have started referring to them as “Onyawa” which is literately translated as “where
do you defecate” when seen moving around the village.
Their influence further extends beyond their positions on the Goal WASH programme since majority
of them are also community leaders (both political and opinion), resource persons and contact
persons for other different projects, including Government as Village Health Team (VHT) members.
Their position in the community as “gate keepers of information” of the communities they serve is
likely to continue the influence of sanitation promotion in the villages.
Another category of people who benefited from the capacity building initiatives were both the
water user committee members and Hand pump mechanics (HPMs). The WUCs were taken through
a series of different trainings on their roles to manage water points, collection of user fees,
accountability and record keeping. Over 410 WUCs on all GOAL Uganda monitored water points
were trained on these roles. At least 3 HPMs from each sub-county where GOAL is operating were
also trained on operation and maintenance, marketing their business and given entrepreneurial
skills. In other instances like in Abim and Agago District, GOAL offered subsidies to spare parts,
supported HPMs with an office space (container in Abim) and start-up up capital in form of a
revolving loan through their district associations (HPMA) to access spare parts closely and cheaply.
At school level, though with limited intervention (fewer schools were supported compared
communities), GOAL through the partner organizations in the respective districts mobilized, formed
and trained school health clubs (SHCs), school teachers and some School Management Committee
23
(SMC) members to promote WASH in the schools where the new sanitation infrastructure has been
built, based on Demand Responsive Approach (DRA). Key among the responsibilities of the SHCs
was to sensitize fellow pupils on hygiene and sanitation, mobilizing and leading others to ensure
cleanliness of the school sanitation and hygiene facilities, ensuring that there was water, soap/ ash
and anal cleansing materials at the latrine facilities. The SMC members were to support the school
to sustain the facilities in terms of collection of O&M money and support supervision of the
construction works of the school facilities including latrines and boreholes.
The intervention also trained the teachers on how to manage the clubs and the different hygiene
and sanitation promotion activities. Although GOAL Uganda and partners could not accurately
provide information on the number of teachers, schools health clubs and children sensitized on
hygiene and sanitation promotion, a few (6) schools visited [at least three SMC members, one clubs
of about 30-50 pupils and 3 teachers trained] during this evaluation noted that the training of the
sanitation clubs missed an important topic on menstrual hygiene and management including
making of re-usable pads. The clubs, the teachers and SMC members are revolving with new
members recruited in the club every year as the candidates leave the school which is likely to
sustain hygiene and sanitation activities in the schools.
GOAL worked through partners to deliver services to the community. To increase the capacity of
partners to implement WASH programming, extend the scale of programming and leverage
considerable local knowledge and experience. GU’s work on capacity building with local partners
was guided by the organizational Partnership Manual (2013), and GU WASH Strategy (2015), both
provide details on GU capacity building approach detailed under four key initiatives:
a) Organizational Development: All WASH partners were facilitated to conduct periodic
Organizational Capacity Assessments (OCA), make their own plans and GU offered support
in strategic planning, fundraising and logistic systems to name a few. For example in 2015,
GU linked and paid for all partners to subscribe to a fundraising website, which also
provides tutorials on fundraising competencies www.fundsforngos.org.
b) Financial strengthening: Each partner was assessed regularly and provided with a financial
capacity rating. This rating determines the ceiling of funding and the frequency of financial
mentoring/monitoring physical support visits. For partners with less developed systems,
they were visited more frequently by GU Grant Manager to provide mentoring on financial
systems and provide GU with confidence that donor funds are being used transparently.
Each year an internal audit was conducted to test the use of the partners systems and
funding was provided to each partner to support an external audit, essential for
demonstrating partner’s financial credibility and fund raising. In 2015, and across four
WASH partners, 17 finance visits were made, four internal audits conducted, three
financial assessments and one external audit supported.
c) Monitoring, evaluation and learning: The WASH programme operates and uses a single
MEL system and shared tools. GU’s role in this regard is to design, training and quality
assure. Partners are visited routinely to provide a mixture of monitoring and hands on
support on the use of the tools.
24
d) WASH technical backstopping. In each district of WASH programme operation, GU has a
full time WASH Technical Manager. Their role is to provide technical backstopping
including training and oversee the quality of partners WASH implementation. They interact
with partners on a weekly/daily basis. Each year there are 2-3 WASH review meetings with
all partners to review progress, learn and collaborate. For example, in 2014 formal TOT
training in the Community Conversation methodology was facilitated for all WASH
partners.
Although the OCA has eight thematic areas, implementation capacity was the most important to
WASH promotion. For example, all partners were taken through a series of residential trainings on
the three approaches of sanitation and hygiene promotion (DRA, CLTS and CC). The purpose of
capacity building was to strengthen their skills to effectively deliver the project activities.
This evaluation did not permit detailed partner capacity assessment, the skills mix and experience
of some partner staff had some gaps (many were diploma holders, relatively new on the job and
this was their first WASH project undertaken). Besides the skill mix, across all the districts the
staffing structure included four staff; two project officers [with exception of URMUDA who had
eight but operating in two districts], an accountant and the manager covering both water and
sanitation activities in the two target sub-counties per district including schools and the whole
district for water quality monitoring and software activities where new sources were to be sunk.
This evaluation found this staff to be working overly busy which may in the near future affect the
quality of services delivered especially monitoring of works and activities
4.2.4 What was the most motivating approach for the changes in sanitation and hygiene behaviour?
Information available from CLTS teams, women FGDs and individual household visits, all
overwhelmingly mention CLTS approach and specifically “shit experiment and demonstration with
water or food” as the most motivating approach to sanitation and hygiene behaviour change.
Communities mention that one of the key reasons they put up the latrine including the hand
washing, was that they realized they were eating their own “shit”.
Across all the districts, CLTS and Women group members mention with disgust on their faces while
explaining the sanitation triggering meeting and the shit experiment. One CLTS team member put it
rightly by saying that “that same day I ran home and I could not imagine how I was eating my own
shit, the neighbours’ and for that old woman in the corner. I had to take responsibility to mobilize
people to build latrines”.
4.2.5 Is there any monitoring system in place? Who is using the results? For what kind of decisions are the results used? Are the results easily available?
GOAL Uganda WASH programme operates a monitoring system to collect data for the programme
code named Monitoring, Evaluation and Learning (MEL) system. Three databases are reported and
verified including, sanitation (CLTS) and water and CHAST for schools.
A set of data collection tools related to CLTS baseline information, verification and follow up were
developed and shared with the implementing partners. Data is collected periodically [monthly,
25
quarterly, bi-annually], entered into an excel database and analysed by the monitoring, evaluation
and learning (MEL) team at GOAL Kampala office.
Data is collected on four key indicators that is: number of villages triggered, number of ODF villages
verified, latrines built and hand washing facilities built from which cumulative numbers are
calculated to guide project implementation and monitoring of results. All partner staff is taken
through the data collection tools, data collection techniques, data entry and quality control.
Interaction with partner staff revealed that their involvement in making meaning of data (analysis)
for their own use is limited and never trained on data analysis, interpretation which sometimes
makes use of the data in planning minimal.
4.2.6 Conclusions on effectiveness:
Substantially, GOAL Uganda WASH programme implemented effective measures to ensure delivery
of the programme results, improvement on access to WASH services (water and sanitation including
functionality), hygiene promotion [hand washing, food storage and safe water chain] as well as
reduction in reported diarrhoea incidence among children below five years stood out during this
evaluation.
Generally, the MEL system is working well and robust though with some limitations including
missing out on some key golden indicators collected by the district and MWE like gender on water
user committees (number of water points with women in key positions) and chemical water quality
in view of the corrosion problem which shortens the lifespan of the HP.
Further still, sustainability of the MEL system is questionable and how data can be fed into the
national data collection and monitoring system.
4.3 Efficiency and value for money
4.3.1 What is the project’s operational cost-efficiency?
GOAL Uganda WASH Programme employed cost effective approaches in the delivery of services to
the target communities. Five approaches stand out among the many and these relate to: a) the lean
and effective organizational structure, b) working through partnerships c) leveraging existing
resources [office equipment, human resources, and transport to mentions], d) selection of
approaches, and e) working with the private sector and early adopters of WASH promotion
activities.
GOAL Uganda WASH team works within a thin and effective structure that relies on support from
other departments that makeup GOAL Uganda Country Office. The team is headed by the Country
WASH Advisor, below her are four (including Kaabong) WASH Programme Managers in the field
offices to support partners with technical backstopping and monitoring quality of works.
Criterion score: 4 – Rather high (the intervention brings good results but there are negative external
factors)
26
Below the managers are partners´ staffs under the direct supervision of the partners [in most cases
dependant on the scale of the funded programme and geographical coverage], two field officers for
hygiene and sanitation and one for water services. The rest are support staff performing support
functions like accounts, logistics, MEL and human resources supported on a partial basis. This
structure is cost effective and encourages partners to source other funding to share management
and core costs.
The Country programme has gone through a series of strategic shifts over the last 10 years within
the districts of operation from emergency, crisis recovery to development and programming
through CSO partnership since 2012. This is driven by a search for the best approach to deliver the
services to the target population with limited costs while achieving maximum outputs/outcomes
and building CSO capacity. Working through partnerships, collaborators and contractors is one of
those efficient approaches (See box 2 for what we mean by partnerships, collaborators and
contractors).
Working through partners provides GOAL WASH Programme with three distinct opportunities and
advantages: a) local sustainability of interventions through capacity building, b) local relevance, and
c) leveraging resources (wide coverage with fewer resources). Partner organizations undertake
work on behalf of GOAL WASH team within their local context which would otherwise be more
costly in terms of human resources, and local context such as the language of the intervention,
equipment and overhead operation costs.
The year 2014/2015 has seen another shift from working with local NGO partners to more private
and business oriented approach (collaborators and contractors). Take for example, GOAL is working
with national sector organisation to lead an informal sector learning group on O&M of water
facilities to influence policy and learning in the sector. The question of efficiency here is the pooling
of resources, knowledge and numbers for greater voice and influence other than going it alone
which would rather create less impact.
Private contractors, except for the traditional roles in drilling and offering direct consultancy
services, have of late been added to the pool of approaches to deliver social services which were a
preserve for the civil society organization.
Take for example GEMA and Expert Concrete (Bugiri and Namayingo Districts), Airtel wallet Wesa
(mobile money) and HPMAs across all target districts have been brought on board for their quick
results turnaround and increased opportunities for sustainable services, limited costs (since no
overhead in terms of office and availably of a wide range of technical expertise in the areas of
interest).
27
GOAL also during implementation of CLTS activities chose to work with early adopter communities
for the WASH promotion behaviour rather than waiting for all communities to move at the same
pace. For example,
communities which
achieved 100% sanitation
coverage were rewarded
with water points. This
approach incentivised early
adopters and catalysed
them to uptake sanitation
behaviour change in a short
time. Early adopters also
influenced other
communities to speed up
their behavioural change
(see Box 1), hence
demonstrating the power of
early adopters and
contributing to cost
effectiveness.
GOAL works through
structures grounded in the
community and provides
capacity building to support
them to learn and grow
experience, and leverages
available resources. Overall,
GOAL Uganda country office structure and resources helps the WASH programme not to work in
isolation but in a synchronized manner. GOAL Uganda works in five districts (Abim, Namayingo,
Bugiri, Kaabong and Agago) with four field offices. The presence of the field offices and all related
equipment improves the efficiency of the WASH programme in reducing operational costs and
overheads. For example, all the WASH programme managers/technical officers are stationed at the
field offices utilizing the same resources as the other country programmes including transport,
office space, human resources and logistics. Two out of five GOAL field offices are located within
partner offices which also significantly reduce GOAL’s costs and provides with timely technical
support significantly more preferable to remote technical support models.
Selection of approaches which are cost effective and efficient in delivering the services to the target
community is worth mentioning. Sanitation and hygiene services are delivered through CLTS, DRA
and CC. Two common characteristics cut across all these approaches, these are: no subsides and
use of community volunteers. GOAL works with natural leaders, CLTS teams, community facilitators,
VHTs and SHCs to deliver the service on a voluntary basis which reduces the cost of operation and
builds local capacity. Further still, CLTS implementation doesn’t provide subsides and works with
the whole village instead of a household unlike other approaches like The Participatory Hygiene and
Sanitation Transformation (PHAST) and household hygiene improvement campaigns being used by
Box 2: Brief about partners, contractors and collaborators
In this report we distinguish between partners, collaborators and
contractors.
Partners are organizations with which GOAL joins hands to undertake
agreed programmes of work. A partnership agreement is established,
setting out a clear understanding of mutual roles, responsibilities and
deliverables. It is expected that the relationship will be two-way,
contributing to mutual learning and the achievement of common goals.
Transfers of funds take place to the partner for specific agreed activities.
Partners may benefit from GOAL’s international knowledge and
experience, but they should be professionally competent to undertake
the agreed work.
Collaborators are rather more distant from GOAL. They are
organizations which share some common goals with GOAL, and which
GOAL works with, but there would normally not be a formal partnership
agreement or transfers of funds. In appropriate cases a Memorandum of
Understanding may be established which expresses the shared intent of
the collaboration.
Contractors are private sector suppliers of goods or services. The
relationship with GOAL is defined by a contract setting out the goods or
services to be provided, the contract terms and conditions, and the
payment to be made. The relationship with a contractor is essentially
about a transaction, and any closer mutuality of goals is a bonus.
28
other sector players. However, the cost and pomp of the ODF celebration parties are questionable
and their budgetary implications need to be assessed further to ascertain value for money.
4.3.2 How effective were the institutional arrangements for the implementation of the
Programme with focus on roles and responsibilities of GOAL and partners?
“The GOAL WASH programme can only be successful if the partnership delivering it is effective”.
One important criterion for such assessment is the extent to which the partners value and support
each other. GOAL Uganda WASH programme works with three distinct set of partners: the CSO
partners (implementing activities on behalf of GOAL while providing technical assistance,
monitoring and learning), the private sector partners (innovations, research and learning) and the
Government and other sector players (regulation, coordination and collaboration).
At district and national level, the message and the role of GOAL is highly appreciated, GOAL is being
viewed as a strategic and knowledgeable partner especially in the area of CLTS promotion and
innovation related to O&M learning for both water and sanitation facilities. The early efforts in
sanitation marketing and development of lower cost products such as sanitation slabs, bringing
services closer to people at the bottom of the pyramid (e.g. mobile money) and accountability for
water points O&M fees are vivid examples of this.
The message is even clearer from the CSO partners that GOAL is a mutual and respectful partner.
Out of four GOAL CSO WASH partners, GOAL is the sole funder for the organisation Wagwoke Wuno
in Agago District and when the GOAL funds end as planned in December 2016, the organisation may
not survive. For UMURDA, 65% of their organisational funding comes from GOAL and they will
survive beyond GOAL funding.
GOAL is applauded in building the capacity of the partners to deliver services through the periodic
capacity assessment and capacity strengthening. Another area of support which was highly
recommended was the support to organizational and systems development, support with
equipment (motor bikes, computers, printers and photocopiers to mention) and logistics for the
partners to deliver the project benefits to the target community.
Private and business partners though brought on board only recently (2014) show more promising
efficiency and sustainability characteristics. GOAL is providing technical support on the
development of sanitation products (Expert Concrete and Airtel mobile money banking in Bugiri and
Namayingo districts) and technical support on O&M of water points (GEMA Enterprises in Bugiri
and HPMA in Agago and Abim districts). The partners offer services to the community on a purely
commercial basis. This arrangement, if successful, is likely to be more sustainable as CLTS has
created enough demand for sanitation products in ODF communities.
4.3.3 Conclusion on efficiency
If GOAL’s efficiency is dependent on the performance of its partners, collaborators, contractors and
selected approaches then the nature of the partnerships, collaborations, contracts and the
relationships between GOAL and its partners, collaborators and contracts is equally important.
More support for partners and also more their own initiatives are still needed in areas of
fundraising (funding proposal writing) to sustain themselves after GOAL funding comes to an end.
29
The private sector partners though in their early stages of work show strong attributes for efficiency
and sustainability which should be more embraced than the CSO partnerships.
Criterion score: 4 – Rather high (the intervention brings good results but there are negative external
factors)
4.4 Impact
This section looks at both positive and negative changes brought about by the programme, directly
or indirectly, intended or unintended. It looks at an assessment of social and economic long term
benefits of the programme beyond the implementation phase.
4.4.1 What were the main contributions of the programme to changes in sanitation and hygiene
practices of the most vulnerable members of communities?
Five levels of sanitation and hygiene practices were observed at both the community and school
levels, for example data from the MEL data system (2012-2015 follow up surveys), FGDs with
women and the CLTS team and individual household interviews indicate that over 17,155
household latrines (including Kaabong district) have been constructed and 135 villages have been
declared ODF. The data also suggest that there have been an overwhelming number of people who
have started and continued to wash hands after visiting the latrines and before eating food with
over 15,796 hand washing facilities (HWF) built by the community over the last four years.
Data obtained from the individual household visits show that almost all households visited at
random had constructed other hygiene facilities like the dish rack, bathing shelter, soak pit and a
kitchen which are equally important in promoting the household health. Data from the GOAL MEL
system further indicate substantial behaviour change practices across the target districts. For
example the WASH follow up surveys (2012 -2015) clearly indicate that the number of households
washing hands with soap during the five critical times [before eating, handling food, after visiting
the toilet, after cleaning the baby faeces and before feeding the baby] has increased from below
10% to about 27% across the target districts however the percentage is higher for each isolated
critical times.
Access to sanitation has improved to 86% from below 50% at baseline, improved food handling and
storage [clean and covered containers] now stands at 79% from 42% at baseline, observance of the
safe water chain has also improved to 18% from 4.7% at baseline and the number of households
accessing safe water within 30 minutes has improved to 84% from 62% at baseline (see annex 7.1
for individual district statistics on performance of key golden indicators by the programme).
There has also been reported reduction in the incidences of diarrhoea among children below five
years [% of HH with children suffering from diarrhoea in the past 2 weeks (3+ loose stools in 24 hrs)]
from 38.9% to 23%. Important to note however, this improvement is a summary of status in those
few sub-counties where GOAL Uganda WASH programme is being implemented and not necessary
the picture of the whole district.
At school level, the programme has equally improved sanitation and hygiene practices of the pupils
through the reduction of pupils-stance ratio (averaging at 1:49 across all the schools visited),
30
provision and access to menstrual hygiene services (wash rooms and segregation of latrine facilities)
for the girl child and hand washing. However there are still concerns of access to re-usable pads,
faecal sludge management.
4.4.2 How has the program impacted on access to drinking water for the most vulnerable
members of communities?
The reduction in distance and time to safe water points cannot be under estimated, from 62% to
84% [% of HH within 30 minutes or less walking distance to the nearest improved water point]. This
should increase time for women and children to attend to other socio-economic / education
activities. Improvement of the water quality benefitted all community members, not just the most
vulnerable, but new WPs added extra benefits of proximity to the less healthy and older community
members. For example, the poorest of families and the elderly accessed water for free across all
districts as reported by the WUC members’ interviews.
4.4.3 What extent has demand for improved sanitation increased as a result of the project
(Demand Responsive Approach)?
DRA approach was used as a catalyst for communities to establish sanitation facilities as a pre-
condition to access a water point especially in the district of Namayingo and Bugiri. It was pre-
conditioned that when the village achieves 100% sanitation and hygiene coverage, they would
receive a water source.
Data from the monitoring systems indicate that over 36 and 45 villages in both Bugiri and
Namayingo were declared ODF out of 59 and 100 triggered villages respectively thanks to DRA.
However other factors like CLTS especially the shit demonstration were more important in
influencing communities to build latrines.
4.4.4 To what extent has demand for drinking water services increased as a result of the
project?
Although there is noticeable improvement in water coverage across the target districts with respect
to the implementation sub-counties to about 82% coverage and reduction in time and distance
travelled to a safe water source [about 84% of the households access water in less than 30 minutes]
due to construction or rehabilitation of the WP, the demand for water is still high in some target
sub-counties [17 out of 21 (81%)] especially those where GOAL is not operating. Communities
where GOAL has improved sanitation and didn’t receive water sources, access to water is
mentioned as first priority, far more than other priorities such schools and health centres.
Across those communities that received water supply systems, the explanation for this increased
demand can be traced from three stand points, a) In districts like Bugiri and Namayingo the
underground water resources is quite aggressive and highly turbid making water salty to an extent
that some of the newly constructed water points are being abandoned or used less frequently
[Magooli village borehole DWD# 48911 and Bulidha borehole DWD # 45786]. In situations like these
the communities call for more safe and reliable water for use or return to unsafe sources.
b) The lack of sufficient underground water resources have also meant that the community selected
sites (closer to households) have been abandoned by the contractors to site for water in other
places which are further away from the households making the distance longer than expected
31
especially in Bugiri and Namayingo Districts hence continued demand for safe water near their
homes. For example a community borehole in at Nangera A, Banda sub-county, Namayingo district:
DWD # 48910 was constructed next to a swamp where water could be sited about 2 kilometres
away from the original community site and it is also very salty and almost abandoned.
c) Another key factor increasing demand for water especially in Bugiri and Namayingo were the
approach of CLTS where early adopters were awarded boreholes. This has made other communities
which were somewhat slow in achieving the 100% sanitation coverage to come up with demand to
water.
Unlike in the Bugiri and Namayingo Districts, the call for more water in Abim and Agago is related to
both water quality (where it is hard and salty) and communities who returned from camps to their
original settlement areas, where demand for water is high and supply inadequate. Years of previous
investment in WP in camps are now significantly less relevant.
However increased access to safe water at household level has not resulted into substantial
increase in safe water use. Data from GOAL WASH follow-up survey (2015) indicate that average;
utilisation of litres per person per day (LPPPD) increased from 7.1 litres in 2012 to 7.2 litres in 2013
to 9.4 litres in 2014 and dropped to 8.4 litres in 2015. Water used per person per day is still less
than Government of Uganda standard of 20 LPPD. According to SPHERE standard, the basic need of
water ranges from 7.5 – 15 litres per person per day.
4.4.5 As a result of the project, are there demonstrated changes in willingness to pay for
improved access to drinking water?
There is certainly low willingness to pay O&M fees and this willingness to pay is hampered by lack of
accountability for the O&M fees by WUCs, limited training of the WUCs on their roles and political
influence encouraging communities not to pay for water, as it should be provided freely by the
government and in some instances politicians voluntarily make contributions towards O&M fees
especially during electoral periods. Data from GOAL O&M research (2014) also indicate only about
30% of all WUCs had some user fees collected.
This evaluation from the WP visited, the WUCs mention about 30% households in the villages
make contributions at one point in time for O&M especially when the source has broken down
while there are not enough contributions for regular maintenance services. Further still, about 30%
of all water points visited had some O&M money (ranging from 10,000 to 230,000 shillings) kept by
either the treasurer or a SACCO in the village. All communities who had boreholes constructed since
2014 have been required to save 230,000 UGX, as a precondition of being eligible for a new water
point.
The evaluation also contends that in Bugiri and Namayingo Districts where the M4P and mobile
money approaches are being implemented might slightly increase the willingness to pay for
communities where the boreholes are located.
32
4.4.6 Do the beneficiaries register any positive impact on health status as a result of the
intervention?
Meta-analysis of follow up survey data (2012-2015) in the target districts covering only operational
sub-counties indicate a self-reported reduction in the number of diarrhoea incidence among
children below five years [% of HH with children suffering from diarrhoea in the past 2 weeks (3+
loose stools in 24 hrs)] from 39% in 2012 to 23% in 2015 (see annex 7.1). This reduction can also be
triangulated with information from individual household interviews, CLTS teams and women FGDs,
all mention reduction in diarrhoea episodes especially in the ODF villages due to increased latrine
use and hand washing (with majority using ash) especially after visiting the latrine and before eating
food.
Data gathered through household interviews also revealed that there has been an increase in the
management of children’s faeces at household level. It was revealed that children faeces used to be
left out, buried and thrown away unnecessary before the intervention. Information available
indicates that almost all children’s faeces is now dropped in the latrine and there is an increased
latrine use among children between 3 to 5 years and above which further positively impacts on the
health of the community.
There was also an observation on the improvement in the general cleanliness of household latrines;
most households’ latrine facilities were clean, with a drop-hole cover and presence of anal cleansing
materials mostly (papers and leaves) which further control the movement of flies hence breaking
the faecal oral route.
4.4.7 Conclusions on impact
The GOAL WASH programme was designed to contribute to the Districts and the National
Development Plan and achievement of the longer term effects on the target beneficiaries. Since
there are a number of players in the district and the national development field all targeting the
somewhat the same beneficiaries, the programme can only contribute to those longer term goals.
The programme has made tremendous contribution to water, sanitation and hygiene practices for
all community members, the most vulnerable included including reduction in diarrhoea incidences
especially among children which can’t be underestimated.
Improved hand washing at household level, safe water use per person per day, and management of
children feaces at household level are good health promotion behaviours promoted as a result of
the programme across the target population. However the programme still scores low on improving
the willingness to pay for water related O&M services with some promising initiatives undertaken in
Bugiri and Namayingo using mobile money and M4P approaches pilots.
New WPs added extra benefits to the less healthy and older community member because these
were allowed to collect water at the new WP at shorter distances for free. The women and children
have all had their time to do other social economic activities and attend school increased
respectively.
33
4.5 Sustainability
This section looks at the likelihood that GOAL WASH programme actions whether in service delivery
[improved access to water, sanitation, hygiene and functionality of water points], capacity building,
partnerships, collaborations and learning will have lasting impact and sustains the benefits that
were brought about by the programme to the target communities.
4.5.1 To what extent were the risk factors to sustainability included in the monitoring system?
To a large extent, sustainability of the MEL system is plausible but data may cease to be collected
and used by partners when the programme funding stops. The sustainability question here is
whether the partners’ capacity has been built into strong and effective organization to utilize the
system and manage it locally.
The answer in regards to the MEL system data is no. GOAL Uganda relies on partners and
community volunteers to collect data periodically (monthly, quarterly and annually) and transmits
to the GOAL MEL team for analysis and interpretation with limited involvement of the partners in
analysis and interpretation. It is very unlikely unless the partners are trained in analysis and
interpretation of the data collected for planning and monitoring of their programme activities, that
they will continue to collect the information.
Secondly, almost all MEL data is collected by partners with the help of community volunteers [VHTs,
community facilitators and CLTS members] through their routine work in the sub-counties of
operation with funding from GOAL. The limited involvement of district Health Assistants and Water
Officers at sub-county level who equally collect the same data routinely for the national sector
performance report further affects the systems data sustainability and continued data collection.
Strong collaboration with these Government structures would add value to the sustainability of the
MEL system data when GOAL intervention comes to an end, the responsibility for managing and
funding the system would be transferred to local actors.
The present MEL is impressive both in its detailed design and frequency of data collection. It
includes all risk factors with the exception of the ground water level and gender issues on WUC as
detailed in the MWE golden indicators. However interviews with the DWO, it appears that declining
groundwater tables is not yet a reality in northern and eastern Uganda, but this could change due
to the effects of the climate change.
4.5.2 Has an exit strategy been agreed with partners during formulation?
Although there were no formal exist strategies discussed with the partners, this evaluation found
some activities with strong characteristics of exit strategies at both community and partner levels.
Key among these was the partnership funding agreement. The nature of work between GOAL
Uganda and the partners was directed by the funding agreement clearly specifying the activities,
roles and responsibilities, funding, a termination clause and the time frame for the agreement. The
evaluation found that during the period under evaluation, no partner agreement had been
Criteria score: Rather high – 4 [the intervention brings good results but there are negative external
(and internal) factors]
34
terminated but the end of the project implementation period will certainly mark the end of the
partnership including routine data collection and monitoring of water points.
Further still, all WASH partners have been prepared for exit through capacity, systems
strengthening and linking them to a funding organization websites to sustain themselves after the
end of GOAL financial support. Information from GOAL also reveals that all partners have been
informed 11 months in advance that funding will end at the end of 2016 and all partners
encouraged to gear up their fundraising activities now to access new funding from other sources in
2017.
At community level, the exit was determined by the ODF celebrations and handover of water points
to Local Government official and the trained WUCs [with periodic follow up visits (quarterly and bi-
annually) across all the ODF villages], this would mark the end of GOAL intervention in the village.
The responsibility of monitoring sanitation and WPs was handed over to the CLTS teams, WUCs,
natural leaders with an agreed O&M plan for water points and an ODF sustainability plans in
presence of the district authorities.
At school level, the handover of sanitation facilities to SMC/PTA, development of an O&M plan,
possession of an O&M fund, formation and training of the health clubs, appointment and training of
the sanitation focal teacher in the school marks the end of the intervention in the school. These
activities are deemed sufficient to maintain the benefits achieved by the programme.
4.5.3 What is the extent of risk that ODF communities will slip back to non-ODF status? And
what are the main problems in proper latrine maintenance?
The probability is rather very low that ODF villages will slip back to OD status due to the level of
behavioural change that has taken place in the ODF communities. Interaction with the CLTS team,
women FGDs and the individual households mentions that the “shit demonstration experiment” is
still fresh in their minds and they can’t allow “eating own shit again”.
There are also indicators for further sustainability of the ODF status as evidenced from the role of
women and children in operation and maintenance of the household latrines (see section 4.1.4).
The availability of free local materials used in the repair and maintenance of the household latrines
is another important factor. However there may be a few households in the villages which may
temporary slip back to OD during the process of re-constructing their household latrine when they
collapse due to heavy rains and termites.
Two challenges were suggested by almost all CLTS, women and individual household interviews as
key in maintenance of household latrines. Most important was lack of access to affordable
sanitation products like slabs and hand washing jerry cans. CLTS promotes the construction of
household latrines with locally available resources which are in most cases comprise of wood, grass
and mud which are temporary in nature as they eventually rot or are eaten by termites.
For communities to maintain the ODF status there is a need to advance to more affordable
sanitation products which are permanent. However these are rarely available in the community.
Sanitation marketing is also slowly taking place only in Bugiri and Namayingo (pilot phase) with
35
plans to research Agago and Abim in 2016, in order to design appropriate sanitation marketing
interventions for implementation in 2017.
The soil texture is also another challenge that was vividly echoed by all districts, the soil texture
when soft (clay and sand) affects the depth of the latrine and usually collapse when it rains, while
when rocky, it presents an enormous challenge in digging a deep pit. For example, in Bugiri and
Namayingo, the target communities are affected by both rocky and collapsing soils while in Agago
and Abim the soils are collapsing. These two types of soil texture bring in a recurrent cost of pit
excavation to the household which is usually not easy to finance.
4.5.4 Are there any new local (locally owned) initiatives to improve sanitation?
Natural leaders, CLTS teams and community facilitators have come out vividly to mobilize
communities and support households to put up household latrines. It has been reported through
the household interviews, the FGDs with women and the CLTS teams that these influenced the
design of the household latrines and bi-laws have been passed on the type and depth of latrines.
For example, in Bugiri, Namayingo and Agago, the CLTS team and the community agreed on the
depth of the pit to be not less than 20 feet for the household latrines. In Abim, the CLTS team and
the community agreed on a mechanism to use a whistle to scare and shame anybody found open
defecating. It is even more encouraging to learn that all these initiatives are voluntary with no
financial support. The natural leaders have also continued to monitor sanitation and hygiene (ODF
status) in their own villages across all the districts.
The role of the women and the children in O&M of the household latrines cannot go unnoticed. The
involvement of the children in both construction and maintenance is a way of transferring the much
needed skill that when the children grow up, they will find value in maintaining the ODF status in
their own households since they will have grown doing it and having that skill.
Last but by no means least, the involvement of local private business into sanitation and hygiene
promotion is another positive initiative. Take for example Expert Concrete in Bugiri who sees the
business potential to development, piloting and making of affordable sanitation products like slabs,
slab beams and latrine hole covers using a commercial approach.
4.5.5 Sustainability of the Water supply systems
The sustainability of the rural water supply is still a thorny issue in most, if not all, Sub Saharan
African countries. The causes are complicated, diverse and no country has managed to find an
acceptable solution to date. Some common constraints are the poor long term stability of the WUC,
lack of external follow-up support at the end of rural water projects, the lack of preventive
maintenance, the non-availability of spare parts or repair capacity, poor community management
capacity, lack of accountability for O&M funds and the unwillingness to pay for maintenance
(Lockwood et al., 2003).
Both Districts and GOAL constructed/rehabilitated WPs have their specific approaches and
constraints on sustainability. The main issue for the sustainability of the WP's constructed by the
DWO is the limited budget of the district to maintain these WPs. According to MWE guidelines, the
districts are responsible to carry out major maintenance (any repair above approx. $100) and
rehabilitation of the WP in the district, while minor repairs (below approx. $100) should be done by
36
the WUCs. In theory, this creates a perverse situation that lack of maintenance by the community
would be rewarded by free major repairs by the DWO. However it also appears that insufficient
maintenance is not the only cause for non-functioning HP as boreholes inspection yielded that the
main cause of malfunctioning is often the corrosion of rising main pipe due to aggressive water
which has nothing to do with a lack of maintenance.
The district based Hand Pump Mechanic Associations (HPMA) where staff are often un-coordinated
and not motivated has the responsibility to undertake O&M works in the districts. Evidence to date
shows that HPMA lack access to spare parts ( with exception of Namayingo and Bugiri) and there is
low business sense and majority of the WUCs don’t want to pay for services due to perceived link to
Government services which are “free”. The result is reduced access to water as water points in need
of major repairs remain broken for extended time periods (see table 9).
Table 9: Progress of the rehabilitation of WP's by the DWO in 2015
District # of WP
rehabilitated in 2015
# of WP on
waiting list
# of years to process the waiting
list with current speed
Bugiri 20 100 5
Namayingo 14 45 3.1
Agago 10 20 2
Abim 10 46 4.6
To date, the GOAL constructed or rehabilitated Water Supply Systems (WSS) in the communities run
reasonably well. Most WUC are still operating, the WPs are well maintained and the average
functionality of the GOAL WP's in the four districts is significant higher than those of the non GOAL
WP (see table 10). However, long term functionality is not yet ensured.
Table 10: Comparison of functionality data between GOAL and DLGs
District Functionality
of all WP's
(%)13
%
Functionality
of GOAL
WP's*
%
Functionality
of GOAL
WUCs*
% of WUC
which do
monthly O&M
fees collection*
Average amount
in cash by WUC
(UGX / EUR) *
Bugiri 75 89 100 96 191,000 / 49
Namayingo 74 89 62 77 176,000 / 46
Agago 70 92 86 87 70,000 / 18
Abim 65 83 81 69 59,000 / 15
*Data from GOAL MEL system and only indicate GOAL operational areas.
Some important factors which could influence the functionality of the GOAL installed WP include
ability of the WUC to collect sufficient funds for the maintenance and repairs of the HP. Data from
the borehole visited during this evaluation indicates that about 30% of the WUCs mention that they
13
GOAL and the MWE have different definitions on functionality: For Government non functional WP don’t
produce any water, while according GOAL definition a hand pump which takes more than 10 strokes to produce
water, or 50 strokes to fill a 20 l jerry can is not functional.
37
collected some O&M ranging between 20,000 and 230,000 UGX kept with either the treasurer or
saved in a village saving and lending association (VSLA).
However, these collected funds were lower than the average costs of the last repairs. In addition,
the willingness to pay is poor as nearly 70% of the WP suffered from non paying water users due to
lack of accountability and political influence [GOAL O&M research 2014].
WUC during water point visits and interviews frequently complained about leakage of the rising
main pipe and the turbidity of water. 10 out of the 29 interviewed WUC mentioned that leakage
was the main technical constraint they faced. Leakage is caused by corrosion of the commonly used
Galvanized Pipes (GI) due to the aggressiveness (low pH and/or high salt content) of the water.
Next to leakage, corroding pipes could lead to poor taste of the water and staining on laundry. The
constant leakage often frustrates the efforts of the communities on paying O&M fees in addition to
issues of accountability of the O&M fees. However the corrosion problem could technically be
solved by replacing the GI pipes with stainless steel (though expensive) or PVC (vulnerable to proper
handling) and there is currently no government policy on this issue.
Overuse could also become a serious constraint to the sustainability of the WPs because several
WPs serve between 300 and 500 households, much more than the recommended number
(between 250-300 households) for the handpump. These water points will obviously breakdown
more quickly and may lead to a decrease in community water user fee contributions.
The networks of HPM through their district based associations are still functioning but are not
sustainable in the long term due to non-payment by the community of repair fees. The water point
visits revealed that WUCs are in general happy with their HPM, their technical competence is good
and not expensive and communities consider HPM to be community volunteers. Only 2 out of 19
interviewed WUCs complained about the HPM being too expensive. However, the HPM are
infrequently paid by communities and essentially work on a semi voluntary basis. Payment is still a
serious constraint for HPM business because repairing HP's is still seen by many as a volunteer task,
rather than a commercial job. Even if the HPM is paid, delays in payment often occurred. All 8
interviewed HPM complained about little or no payments from the WUCs.
In addition, spare parts are often difficult to get and HPMs have to travel long distances although
this is not the case in Abim, Namyingo and Bugiri. HPMs are confident that they can do the job, but
most cannot handle plastic or stainless steel pipes. In Bugiri / Namayingo, GOAL is piloting an O&M
service contract with an existing business enterprise (GEMA) who will outsource O&M jobs to HPM
and be responsible for the quality of their work.
4.5.6 New approaches for sustainability of the water points
GOAL is developing several promising approaches to improve the sustainability of the HP.
Interventions focus on three interrelated behaviour changes that are crucial to the success of an
alternative O&M model14
, namely:
a) O&M service provider(s) adopt and markets a new O&M service contract with WUCs.
14
GU Training Water User Committee’s A Training Guideline and Report 2016
38
b) WUCs adopt a new mobile payment system and collect regular water fees.
c) (Local) government increases its influence in regulation and enforcement.
In the framework of these changes GU developed the following activities:
a) Contracts with commercial enterprises: In Bugiri, a commercial construction firm (GEMA)
who installs HPs in rural communities will offer the WUC's contracts for the maintenance
and repair of the HPs. For a quarterly fee of 230,000 UGX (59 EUR), GEMA carries out 3
minor services and if necessary one major service/year and all repairs in between. The idea
looks promising, but GEMA has just 2 contracts to date and hence the viability of these
arrangements still has to be proven.
b) Mobile money wallet: One of the main problems of WUCs is the storage and accountability
of O&M funds. Normally, these funds are stored in the house of the treasure or VSLA group
account due to the lack of access to financial services. This system is quite vulnerable as it
can be stolen out of the house or just "borrowed" for other purposes while at VSLA though
the money would be safe, but may not be available at the time when needed especially
when lent out to members who have pa back. In addition, GOAL found in Bugiri and
Namayingo that the lack of trust that maintenance funds are being kept safely was found to
be the most important reason for the communities unwillingness to pay for water user fees.
In this system, the WUC´s maintenance funds are kept in an electronic wallet. Authorized
persons (needs to be 3 WUC members together) can transfer funds by mobile money.
c) Commercialization of Hand pump mechanics: As mentioned above, the standard HP of
Uganda, the Indian Mark II/III HP cannot be repaired at village level but need a trained
mechanic. The low income of these mechanics rather than lack of technical skills appear to
be the bottleneck in the system. Input (spare parts and a container for office and storage),
was provided by GOAL in the past. However, this HPMA is not generating income as clearly
these are not the constraints to providing reliable O&M services. GOAL considers the largest
impediment is the lack of business acumen. Research into this in Agago, Abim and Kaabong
will be undertaken in 2016.
4.5.7 Conclusions on sustainability
The sustainability of community sanitation and hygiene programmes strongly depends on a chain of
links beginning with real demand, community participation (especially early adopters) as well as
community contributions related to adequate revenue generation for maintenance and operation
of the facilities established.
It has been encouraging in this evaluation to hear and see community own initiatives geared
towards contribution (O&M fees) and sustaining the ODF status in the respective villages, for
example natural leaders, SHCs, private partners and community facilitators, whose capacities have
been built and the roles of women and children sanitation and hygiene promotion.
Despite some progress, sustainability of the WPs is still vulnerable. The stability and ability to collect
sufficient funds of the WUC are the main bottleneck to sustainability. Recent initiatives with
commercial contractors aiming at a more commercialized approach on HPM and the phone banking
system are encouraging but not yet tested on a larger scale.
39
The monitoring exercises by GOAL's partners contribute to the relatively good state of the village
WPs but this is likely to cease at the end of the contract between GOAL and its partners. In absence
of other organizations which could monitor and assist WUC in the long term, the role of the LG
should be revised. At present the LG carries out repairs of HP but due to limited funds has a serious
backlog on this. In future the LGs should move away from implementation and should concentrate
on M&E and regulation.
Criterion score: 4 – Rather low (the procedures, results or assumptions do not fully meet the
expectations)
40
5. ASSESSMENT OF GOAL WASH PROGRAMME AGAINST 10 KEY WASH GLOBAL PRINCIPLE AND PRIORITIES
This section draws on information collected on the assessment of the key programme components
related to OECD criteria of evaluation principle (2-8) while other Principles have been assessed at
the programme level. Using information from programme documents, reports and discussions with
GOAL and relevant partners/key informants (principles 1, 9 and 10) also provided vital evaluation
information. To allow for comparison between countries, a matrix of GOAL WASH programme
scores against each principle has been prepared and attached at the end of this section.
5.1 Principle 1: Addressing the ‘three prongs’ of WASH
This principle looks at whether GOAL WASH programme in Uganda is implemented as either an
integrated programme, either by GOAL or in collaboration with other partners, including equal
considerations for men and women. This evaluation has found the programme in cognizance of this
principle as an integrated, collaborative and with equal consideration for both man and women.
GOAL WASH programme is integrated in a manner that water, sanitation and hygiene services
constitute the package of services (see section 1.2, brief about GOAL WASH programme in Uganda)
delivered to the community using a partnership approach (see annex 7.2 for GOAL partners and
section 4.3.1 for partnership approach). Further integration is ensured through a consistent delivery
of all service in the areas of operation. For example in all communities where water points are
constructed [with support from Japan Embassy and charity: water funds], hygiene and sanitation
services were also implemented although the scale of service delivery may have differed. For
example in Bugiri and Namayingo Districts all operational areas received both construction,
rehabilitation and monitoring of water points in addition to hygiene and sanitation promotion
activities while in Agago and Abim some sub-counties did not receive new water points but instead
rehabilitation and monitoring.
Women and men are equally considered in programme implementation at all stages, for example
women are encouraged to take up leadership positions for all WUCs (at least three key positions as
per the MWE guidelines), during CLTS promotion the women are also encouraged to take lead in
sanitation promotion as volunteers and natural leaders. The evaluation also found specific roles
defined by the community in regards to water, hygiene and sanitation promotion (see section 4.13
on the roles of woman and man in WASH promotion).
5.2 Principle 2: Community involvement and engagement in all aspects of programming
GOAL WASH programme ensures participation at three levels (a) planning; b) Implementation and
c) monitoring of water, sanitation and hygiene (WASH) promotion activities [see section 4.1.1: how
was community participation ensured]. At planning level, the community and the district local
authorities were involved in a baseline survey to determine the state of affairs including neediest
areas and least served areas (WASH baseline survey 2011/2012).
At implementation, community participation was very evident in the CLTS activities [including CLTS
pre-triggering activities, triggering, post triggering and scaling up], Community Conversations (CC)
and Demand Responsive Approaches (DRA). Under CLTS activities the community was involved
selection of natural leaders, CLTS triggering, mapping of open defecation (OD) areas, community
41
action planning, construction of sanitation and hygiene facilities (latrine and hand washing, bath
shelters and drying racks), monitoring of ODF status and open defecation free (ODF) declaration.
Community participation in water services was plausible, with the majority of the beneficiary
communities during implementation taking part in site selection, formation and selection of the
WUC members, development of an O&M plan and appointment of the caretaker for the water
sources. Other implementation participation activities included, construction of the fence around
the water source, drafting of bylaws for the water point including collection of water user fees
across all the target districts.
Participation during monitoring was ensured through the CLTS teams and health inspectors at Local
Government (LG) levels. The CLTS team members are community volunteers who periodically
undertake household visits to ensure sustainability of ODF status. Some of the key activities are to
encourage households to undertake repair and maintenance of their sanitation facilities and also
feedback to the GOAL partner project staff on the progress. The district and sub-county Health
Assistants and Inspectors undertook joint monitoring visits to encourage and enforce sanitation in
the target communities.
5.3 Principle 3: Gender mainstreaming
Gender is central in all GOAL WASH interventions for example in all community engagements in the
target population, deliberate efforts were made to in involve women and gender neutral language
and actions are encouraged. Gender sensitivity is also encouraged especially in hygiene and
sanitation promotion trainings where female field officers are advised to dress appropriately and
handle topics which are sensitive to women. Take for example issues to do with the role of women
in hygiene and sanitation promotion at households.
At school level, gender mainstreaming is also encouraged in both design of the WASH facilities and
training of the key personnel to promote sanitation in schools. For example in each school a female
teacher (senior women) trained to handle menstrual hygiene and girl issues (with exception of
access to and making of re-usable pads) in the school as the male teacher handles the men/boys
issues (senior man). All school sanitation facilities, like latrines and hand washing facilities are
designed and segregated for girls and boys. The girls section is also inclusive of washrooms for the
girl child issues for example washing and changing during menstrual periods. However,
improvements and investments are still needed especially on access to and making of re- usable
pads.
5.4 Principle 4: Creating demand
Demand creation was integrated into the approaches used to deliver services to the community,
take for example the DRA and CC approaches. The intention of these two approaches was to
catalyze the community to demand for improved services. In Bugiri and Namayingo districts early
adopters of 100% sanitation coverage demanded for a water point which was delivered by GOAL.
Also the CLTS approach embeds the element of demand creation. For example after triggering
meetings [especially the shit demonstration] the community was driven by the demand created to
get rid of faeces in the community hence driving the momentum for sanitation and hygiene
42
promotion activities like latrine construction and drying racks including hand washing practices at
household level as opposed to other approaches like PHAST and CHAST.
5.5 Principle 5: Sustainability of WASH services including environmental impact
The programme design paid critical attention to sustainability of the interventions, take for example
working through local partners, careful and improved training and formation of WUCs, School
health clubs, school management committees and community volunteers [see section 4.5 on
sustainability and section 4.3 on efficiency].
GOAL is also constantly learning new approaches for O&M for example (change from galvanized
pipes which corrode easily to PVC and PPR pipes, use of the stainless steel to counter the effects of
corrosion and breakdown of the pumps). GOAL is also learning new approaches in management and
collection of user fees through making markets work for the poor (M4P approach) where they are
piloting a business model for operation and maintenance of water points. Through sanitation
marketing, GOAL is trying to encourage communities to move up the sanitation ladder to replace
traditional latrines to more permanent latrines using the available low cost sanitation products like
slabs, pans/flappers, sato-pans, which are more permanent and resistant to termites and collapsing
soils.
Environmental management also took centre stage during CLTS promotion activities. GOAL
encourages the use of dead logs, instead of cutting down trees to make the latrine slabs. During
water point construction encouragement was made for planting of trees around the boreholes, soil
back filling and planting of grass. However there is no clause in the drilling contracts which bind the
contractors to undertake environmental protection apart from management of waste water and
dangerous chemicals used in drilling. The government standards, recommends 5% of the contract
sum for all drilling contracts to be invested on environmental protection. For example planting, and
sensitization of the community about environment, the evaluation did not find any borehole where
the community or the contractor had planted trees to replenish the underground water reserves.
However there are plans in the new country strategy to undertake integrated water resources
management (IWRM) as a key priority.
5.6 Principle 6: Appropriateness of interventions for beneficiaries, particularly vulnerable groups
GOAL is working in poor communities with low levels of income and sparsely populated. The choice
of a hand pumps is an appropriate technological choice for water supply due to a relatively low
investment cost and maintenance as opposed to piped schemes and other technologies. GOAL is
also fitting stainless steel and PPR pipes for all the water points constructed to counter the
corrosion effect where water is aggressive.
With the back ground of accountability for water user fees, the implementation of a business model
for management of water user fees and collection of O&M to address the problem of functionality
is an appropriate measure to address the challenges of functionality though not yet widely spread.
However hand pumps are being affected by low water potential in some sub-counties of the target
districts e.g. in Namayingo, Banda sub-county has low water potential and where there is water; it is
too salty and hard. This has led to all boreholes in Banda sub-county not to be used efficiently with
at least two out of the nine completely abandoned within one year of their construction [Magooli A
and Nangera A].
43
The installation of lined pits for school sanitation is yet another appropriate measure to counter the
effects of collapsing soils and sustainability of the school facilities which used to fill up within two
years of construction, with no emptying facilities available, this substantially reduces the value of
the investment. However, the search for an appropriate and user-friendly faecal management
approach is still a long way off especially in rural areas where access to cesspool emptying services
is a challenge, expensive and nonexistent in some districts.
5.7 Principle 7: Focus on behaviour change
CLTS as an approach used by GOAL to promote sanitation and hygiene and is central to behavioural
change, the community is made aware that they are “eating their own shit” during the triggering
meetings.
This experiment makes the community realize the need to construct latrine and hand washing
facilities and motivates them to do so without subsidies. The selection of such an approach is a clear
testimony that GOAL WASH interventions are central to changing behaviour of the community
where they work.
Information from follow up surveys (2012-2015) also shows interesting results of behaviour change
practices related to hand washing, safe food handling, safe water handling, latrine construction and
above all reduction in the number of children below five years suffering from diarrhoea [see section
4.2.2 most significant behaviour change caused by the programme and annex 7.1 for performance
against golden indicators].
5.8 Principle 8: Partnership and capacity building
GOAL WASH programme works through partners to deliver services to the community. GOAL
supports partners to increase the capacity of partners to implement WASH programming, extend
the scale of programming and leverage considerable local knowledge and experience.
GOAL Uganda’s work on capacity building with local partners is guided by the organizational
partnership manual (2013), and GU WASH Strategy (2015), both provide details on GU capacity
building approach detailed under four key initiatives including organization development, financial
strengthening, monitoring, evaluation and learning and WASH technical backstopping.
GOAL also works with other civil society partners to influence policy, learning and service delivery
respectively [see section 4.2.3: Capacity of local stakeholders to maintain WASH services].
Although the OCA has eight thematic areas, implementation capacity was the most important to
WASH promotion. For example all partners were taken through a series of residential trainings on
the three approaches of sanitation and hygiene promotion (DRA, CLTS and CC). The purpose of
capacity building was to strengthen their skills to effectively deliver the project activities.
5.9 Principle 9: Integrated approach to WASH programming
GOAL WASH programme doesn’t operate in isolation of other country programmes. Save for ACT
health programme which operates in 16 districts under a randomized controlled trail and much of
research and learning work. The WASH programme operates in the same districts as the livelihoods
44
programme although sub-counties of converge may differ from district to district. This integration
helps the WASH programme to leverage resources hence being more efficient [see section 4.3.1:
programme operational cost efficiency].
5.10 Principle 10: Focus on reduction of vulnerability to future hazards
The WASH programme envisaged reduction of vulnerability of the target population to future
hazards in both design and implementation. For example it ensured that both WUCs, HPMs are
equipped with the necessary capacity to ensure that the water sources are functional to provide
access to clean safe water.
The introduction of sanitation marketing where CLTS has taken place is also forward looking in
terms of sustaining the ODF status of the communities to avoid future diarrheal incidences in the
community when the latrine facilities constructed of local materials collapse.
The new GOAL Global WASH strategy (2017-2020) is also cognizant of future hazards and
vulnerabilities where Integrated Water Resources Management (IWRM) is given due attention to
ensure that underground water resource is protected and restored. The drilling contracts also have
a clause on management of waste water and dangerous chemicals used in drilling of water points.
All these are meant to protect the community from future vulnerabilities and hazards.
Table 11: Assessment scores for GOAL WASH global principles and priorities
GOAL WASH Principles and priorities Score and rating
Principle 1: Addressing the ‘three prongs’ of WASH (Water, Sanitation, and
Hygiene), either as an integrated program, or in collaboration with other partners,
including equal considerations for men and women
High (5)
Principle 2: Community involvement and engagement in all aspects of programming High (5)
Principle 3: Gender mainstreaming Rather high (4)
Principle 4: Creating demand Rather low (3)
Principle 5: Sustainability of WASH services including environmental impact Rather low (3)
Principle 6: Appropriateness of interventions for beneficiaries, particularly
vulnerable groups
Rather high (4)
Principle 7: Focus on behaviour change Rather high (4)
Principle 8: Partnership and capacity building High (5)
Principle 9: Integrated approach to WASH programming Rather high (4)
Principle 10: Focus on reduction of vulnerability to future hazards Rather low (3)
45
6. CONCLUSIONS AND RECOMMENDATIONS
6.1 Conclusions
To a large extent, the process of WASH promotion at community level was highly active, interactive
and meaningful save for the limited school sanitation promotion in terms of coverage. This level of
participation is a precursor for sustainability of latrine use, hand washing behaviour and
functionality of the facilities at both community and household levels.
Although all WASH interventions by GOAL Uganda are undoubtedly important, some aspects need
more attention than others in the next country strategy. Issues related to sustainability of water
points (O&M), menstrual hygiene management such as making of reusable pads at both school and
community level/linking schools to private service providers, sanitation marketing to sustain the
gains created by Open defecation free (ODF) villages, responding to sanitation challenges created
by rapid urbanization through faecal sludge management and exploring new and appropriate and
alternative technologies to water supply such as solar powered supply systems, piped schemes and
self supply.
GOAL Uganda WASH programme implemented effective measures to ensure delivery of the
programme results, for instance, improvement in access to WASH services (water and sanitation
including functionality), hygiene promotion [hand washing, food storage and safe water chain] as
well as reduction in reported diarrhoea incidence among children below five years stood out during
this evaluation.
GOAL MEL system is working well and robust though with some limitations including missing out on
some key golden indicators collected by the district and MWE like gender on water user
committees (number of water points with women in key positions) and chemical water quality in
view of the corrosion problem which shortens the lifespan of the HPs.
If GOAL’s efficiency is dependent on the performance of its partners, collaborators, contractors and
selected approaches then the nature of partnerships, collaborations, contracts and the
relationships between GOAL and its partners, collaborators and contracts is equally important.
The private sector partners though in their early stages of work show strong attributes for efficiency
and sustainability which should be more embraced than the CSO partnerships.
The GOAL WASH programme was designed to contribute to the Districts and the National
Development Plan and achievement of the longer term effects on the target beneficiaries. Since
there are a number of players in the district and the national development field all targeting the
somewhat the same beneficiaries, the programme can only contribute to those longer term goals.
Despite some progress, sustainability of WPs is still vulnerable. The stability and ability to collect
sufficient funds and accountability of the WUC are the main bottlenecks. Recent initiatives with
commercial contractors aiming at a more commercialized approach on O&M service providers and
the phone banking system are encouraging but not yet tested on a larger scale.
46
6.2 Recommendations
# Recommendations related to the GOAL Sanitation and
Hygiene program in Uganda
Main
addressee
Degree of
importance*
Reference
section
1 Focus on issues related to menstrual hygiene
management such as making of reusable pads or linking
to a private provider at both school (CHAST manual)
and community level, sanitation marketing to sustain
the gains created by Open defecation free (ODF)
villages and respond to sanitation challenges created by
rapid urbanization (faecal sludge management using
indigenous micro organisms - IMOs).
GOAL
Uganda
1 Relevance
section
2 Exploit the knowledge and learning niche at the
national level by investing more in cutting edge
research and share widely on appropriate sanitation
and hygiene promotion approaches which are cost-
effective and able to generate demand for household
sanitation and hygiene behavioural change as well as
addressing sustainability issues (a combination of CLTS,
DRA and CC is a good example). However this will need
to backed up with a proof concept before wide sharing.
GOAL
Uganda
1 Relevance
section
# Recommendations related to the GOAL Water
program in Uganda
Main
addressee
Degree of
importance
Reference
section
1 Build capacity of the CSO partners to make meaning on
the MEL data collected and appreciate it uses in
planning and monitoring of their projects.
GOAL /
Implementing
partners
2 Sustainability
section
2 Work with UWASNET and other WASH CSOs to
influence MWE policy recommendation on the use of
non-corrosive water pipes such as PVC and stainless
steel pipes.
GOAL/
UWASNET
1 Sustainability
section
3 Standardize functionality criteria with those of District
Water Office (DWO).
GOAL 3 Sustainability
section
4 Encourage, support and monitor new initiatives on the
sustainability of the WP, such as commercialization of
the HPMA, cooperate with existing commercial
partners and develop, test and rollout the phone
banking system fully.
GOAL /
Implementing
partners
1 Sustainability
section
5 Pilot with the installation of small scale (solar powered)
reticulation system, gravity flows and pumped up
schemes in locations with a high water demand and
poor underground water quality (salty and hard water).
GOAL /
Implementing
partners
2 Sustainability
section
# Recommendations related to the GOAL processes and
mechanism
Main
addressee
Degree of
importance
Reference
section
47
1 Build the capacity of politicians and LG staff (health
assistants) on CLTS and other sanitation promotion
approaches to support the programme efforts through
enforcement of the public health act and promote
sustainability of the interventions.
GOAL 1 Effectiveness
section
2 Apply the sanitation and hygiene promotion
approaches (CLTS, DRA and CC approaches) in a
systematic and sequential manner to exploit the
comparative advantages of each while strengthening
each other for better results. This evaluation found that
the best approach would be to enter a community with
CC to identify and prioritize community problems. This
would be followed by a combination of DRA and CLTS in
that order with some overlaps.
GOAL 1 Effectiveness
section
3 Build the capacity and involve the school
administration (PTA/SMC) on their roles and
responsibilities in monitoring the quality of works of
the school sanitation facilities to ensure quality works,
value for money and sustainability.
GOAL 2 Efficiency
section
4 Undertake a phased approach from CSO partners to
engage more of the private sector/business oriented
partners for longer term sustainability of interventions.
GOAL 1 Efficiency
section
5 Review the cost and pomp of the ODF celebration
parties in line with purpose and intended benefits to
the programme’s sustainability and value for money.
GOAL 2 Efficiency
section
6 Exploit the opportunity and take lead of the niche of
sector learning on O&M in the country. UWASNET
might be a good vehicle for this. It will provide the
power to influence a number of issues in the sector and
also adoption of the different O&M models that have
been developed, piloted and scaled by GOAL Uganda
for larger sector replication and adoption.
GOAL 2 Effectiveness
section &
Sustainability
section
# Recommendations related to GOAL Systems Main
addressee
Degree of
importance
Reference
section
1 Align the MEL system to the national WASH sector
performance golden indicators for sanitation,
collaborate with DLG water and health officials in
collection of WASH monitoring data and also build the
capacity of the CSO partners on making value of the
data to appreciate the importance of the system.
GOAL 1 Effectiveness
section &
Sustainability
section
*Remark: 1 is the most important, 3 the least important
48
7. ANNEXES
7.1 Graphs showing GOAL progress on key WASH indicators – Follow up survey data (2012-2015)
a) Number of villages triggered in 2012, 2013, 2014 and 2015
b) Open defecation free villages declared within 12 months
c) Open defecation free villages overall
d) Latrines constructed after triggering in 2012, 2013, 2014, and 2015
49
e) Hand washing facilities constructed after triggering
f) Water source functionality according to the community opinion (water flowing)
50
g) Water source functionality according to the GOAL monitoring data base which follows critical criteria15
.
h) Hand washing at the critical times
i) What is used during hand washing?
15
A water point is functional if it meets all of the following criteria: 1) Sanitary score is equal or less than 7: (2) the
hand pump draws water before 10 strokes of the handle (3) less than 100 strokes to deliver 20 liters.
51
j) Knowledge on the cause of diarrhoea
k) How do you prevent your HH members from getting diarrhoea?
l) Rubbish disposal?
52
m) HH practicing appropriate food storage (clean and covered container)
n) HH practicing appropriate water storage (covered, clean and narrow neck container)
o) HH water consumption (litters per person per day)
53
p) Access to an improved water source
q) HH within 30 minutes or less walking distance to the nearest improved water point
r) HH using an improved latrine (JMP standard)
54
s) HH with latrines that renovated them (made improvement on them)
t) Incidence of diarrhoea in the past 2 weeks before the survey (3+ loose stools in 24 hrs)
u) Incidence of diarrhoea by ODF status
55
7.2 Short profile of GOAL partners
a) Wagwoke Wunu (WW)
A local NGO started in 2005, with a vision of “a community that is well informed about their health,
human rights and accountable“. The organization has six (6) staff and two (2) volunteers. Started
collaborating with GOAL in 2008 to implement a HIV project and later in 2013 started to implement
WASH activities. The organization has an annual budget of about 200,000,000 Uganda shillings with
GOAL Uganda contribution about 85% and UNICEF providing the rest of the funding for maternal and
child health project. Majority of the staffing are graduates in social sciences, finance and development
studies. The programme coordinator also has a post graduate diploma in project management.
b) UMURDA
Faith based NGO started in 1992 as a local community based organization (CBO) with a vision of a
healthy and informed rural community. The NGO now operates in 13 districts of Eastern Uganda with
Namayingo and Bugiri included. The organization started working with GOAL in 2004 as a local partner
to implement a comprehensive HIV programme.
The NGO now operates with 20 staff with qualifications ranging from diplomas, degrees and certificates
in development studies, sociology and Engineering. The NGO has an annual budget of about
600,000,000 shillings (App. $180,000). The NGO is currently receiving grants from both international and
local funding agencies including African development bank, Rotary club of Canada, USAID, GOAL
Uganda, UWEZO education imitative and Ministry of water and environment.
c) GEMA Investments
Started as HPM Association under the RUWAS programme, trained to undertake O&M of water points
at sub-county level and also to support installation of over 200 boreholes in the district of Bugiri and
Namayingo then included as part of Bugiri. They received a tool box on loan from RUWAS which they
had to pay back from their payments for the work undertaken. In addition to support installation works,
they were tasked to train WUC caretakers on simple O&M of the facilities until the programme ended in
2007. With the introduction of multi-party politics, and interference of politicians in the collection of
water user fees, the operation and maintenance of many pumps collapsed since the committees could
not pay for the repairs.
In 2008, Government in an attempt to restore functionality of the water sources, organized all HPM into
an association at sub-county level to provide them work for O&M through short time contracts. This
however failed due to corruption on the award of tenders where many of the tenders went to the
private companies. In 2013, a group of HPM decided to form a private company to compete in business
of O&M from the district. They stared with replacing of GI with PVC which had rusted at a cost of 25
million shillings (App. $7500) for Bugiri district till now.
d) Expert Concrete
The company deals in concrete products, i.e. latrine slabs, culverts, fencing poles, bricks and blocks to
mention. GOAL Uganda is trying to foster a relationship to engage the company to undertake production
of specialized sanitation products which can be marketed to the community through sanitation
marketing. The idea is to move the community up the sanitation ladder where many of the household
toilets which were constructed through CLTS model are traditional and temporary. GOAL is providing
the partner with technical knowledge on how to make these products and undertake technical
supervision. The partner has been able to make at least one latrine slab which is light weight (about 15
kg) and durable which costs only about 25,000 shillings (app. $7).
56
e) MUCOBADI – Multi-Community Based Development Initiative
Local NGO with head offices in Bugiri District established in 2000 with four other branches. The
organization works in 12 districts of Uganda with programmes ranging from HIV/AIDS, WASH, advocacy,
accountability and sustainable livelihoods.
The organization has four staff in Abim including two project officers (WASH), accountant and the
branch manager with qualifications like diploma in Water engineering, diploma in social work and social
administration, diploma in accounts and the manager has a bachelor in counselling and guidance in
addition to a diploma in secondary education.
The organization started working with GOAL Uganda in 2009 on a comprehensive HIV programme and in
2013, started implementing WASH activities as their first grant in WASH. The organization has an
operational budget of about one (1) billion Uganda shillings with other funders including CST, positive
action for children, Path and TASO with GOAL Uganda as the sole funder for WASH activities.
f) Uganda Water and Sanitation NGO Network (UWASNET)
Founded in 2003 from a policy reform to recognize the CSOs’ contribution to the sector through the
sector-wide approach (SWAP). This brought together the Government, donor agencies and the NGOs as
development partners. The network is currently having over 260 members both local and international
NGOs. There are also discussions to include the private sector under this composition which will initially
be coordinated by UWASNET.
The role of the network is to promote partnership, networking, knowledge management and learning
for the sector NGOs. The network is also responsible for documenting and feeding the members’ work
into the national sector performance report with a dedicated chapter for the NGOs.
57
7.3 List of people interviewed
Name Organization Position Phone
Ojoto Simon Peter GOAL WASH project manager Bugiri
Wandera DEO GOAL Techn. Officer BH & latrines
Bugiri
TorephAkoteh GOAL WASH programme Manager
Bugiri
Namukuba Fred GEMA Director investments
Mwerigma Robert DWO District Water Officer 0700554799
Obwapus Stella DWO Engineering officer 0774916922
Mirembe Silver Expert Concrete Director 0783690252
Wabusa Joshua Water Office Assistant water office
Namayingo
0784257806
Opus Emmanuel GOAL WASH project officer 0782140147
Odungt Richard Hajans? Wagwokue Wunu Programme coordinator 0771676188
Bongomin Samuel Otto Water office Agamo District water officer 0772825858
Otio Dominic HPMA Abim Chairman 0773887856
Mwaka Isaak Phillip DWO Abim District Water Officer 078145838
Edmund DWO Abim Assistant District Water Officer 0782957094
Tino Mary Gorretje
Abrahams
GOAL Abim WASH project manager 784563444
NamukoseJoasiMulungi Mucobadi Branch Manager 7754961667
Abura Jimmy Oronmy Mucobadi WASH Officer 777592551
TonnyArima Mucobadi WASH Officer 784750916
Mathias Buteraba Abim District DHI- Environmental health
CLTS/Women Group –
Bugiri District
TakuwaFatumah VHT
MamusubyaHawa CLTS Members
Amiti Mary CLTS Members
Nangoobi Janet CLTS Members
Akooth Christine CLTS Members
Babirye Jenifer CLTS Members
Namukuuna Cate CLTS Members
Jingo Lyidia CLTS Members
NankwangaFatia CLTS Members
NamagandaMwahajibu CLTS Members
NayikobaBayat CLTS Members
Mutumba B CLTS team
and Women’s group
Makoha Ronald VHT
Boogere Bernard VHT
Magengeni Patrick VHT
Owino Joseph VHT
MaginaMicheal VHT/CLTS Member
58
Nabutoono Jessica VHT
NabwireScovia VHT
KatalinaNabutoono VHT
Agutu Evelyn VHT
Mutoonyi Mary VHT
CLTS-WOMEN-FGDS-
ABIM DISTRICT, Aridai
south – CLTS team
meeting
AdrikoFaustina CM-LCI
Stella Grace
Akong Josephine
Omuge Tom Peter
Ochan John
Okalanyi Moses
OchanJovantino
Amuge Rose
Appolot Norah
Oyello Julius
Apio Agnes NyikiNyiki Village
Women /CLTS FGD –
ABIM District
Okule David
Akole Alfred Abia
AkelloMatha Awir
Awor Grace
AmuwaHeln
Akello Christine
Aketch Nancy
Auma Beatrice
Ayo Veronica
Akello Wine
AchanMatayo
Okello Felix Remmy
CLTS/Women groups
Agago district
Tetwala Village,
Kutomor sub-county,
members
Abaa Moses
Ogwal Francis
AtimMorina
Akulullu Celina
AtoCeliberia
Adong Cate
Akello Alice
59
AtimAlican
Agom Susan
CLTS/Women groups
Agago district, Olyelo
Central FGD-CLTS
group members
Acheng Patrick
Okello Fred Jimmy
Okello Nelson
AyugiScovia
Okello Robert
Apio Rose
Ebule Ward/Women
and CLTS group, Agago
District
Achanan Harriet
AdongMildrade
Adur Lucy
Apio Teddy
Akullu Jacinta
Auma Margret
Awiri Lucy
Akello Betty
Apio Sylvia
Acheng Molly
Akuremior – Kutomor
CLTS Group, Agago
District
Opio Jasper
AgangAbudu Ali
Okidi Tony
Otim Joseph
Otim Sylvester
Okello Geoffrey
Opoto James
Angom Agnes
Florence Alimo District Health
Inspector (DHI)
Agago District
Mutumba Robert
Ass. District Health
Officer –
Environmental Health
Bugiri District
Malinga Isaac Senior Health
inspector
Bugiri District
Mathias Mangeni- ASS. District Health officer Namayingo District
Mukyala Veronica
Acting. Ass. DHO –
Maternal and Child
Namayingo District
60
Health
Oundo Humphrey
District Health
Inspector
Namayingo District
MugimbaShamira URMDA project officer WASH 0782889755
Tusiime Christopher URMDA Project Officer-WASH 0788357029
Namukose Sophie URMDA team Leader WASH 0774178951
Malugendo KS URMDA Coordinator 0772604449
SwagoJaberi URMDA Project Officer WASH 0751369875
TagooleGoerge URMDA Finance Manager 0782671801
Waiswa Ahmed URMDA Project officer WASH 0712182650
MasabaRicahrd URMDA Project Officer-WASH 0784150312
NaigagaZaharah URMDA Project Officer-WASH 0700275881
WakateJeeda URMDA Administrative Assistant 0782469960
NandhuHamdan GEMA Investments Director
Nalukuba Fred GEMA Investments Deputy Director
Mirembe Silver Expert Concrete Sales manager
Turihabwe Vincent Expert Concrete Sales manager
Emanuel Opus Gaol Agago WASH Manager – Agago District
GorretiTino Mary Goal Abim WASH Manager –Abim District
DeoWandera Goal Bugiri field office Technical officer
Joseph Akotch Goal Bugiri field office Programme manager WASH
Simon Peter Ojoto Goal Bugiri field office Project manager –WASH
George Mugenyi Goal Bugiri field office Project Officer
DeoWandera Goal Bugiri field office Technical officer
Namukose Joan MUCOBADI – Multi-
Community Based
Development Initiative
Branch Manager
Alimo Tony MUCOBADI- Multi-
Community Based
Development Initiative
WASH Officer
Abra Jimmy MUCOBADI – Multi-
Community Based
Development Initiative
WASH programme officer
Ware Isaiah Nansaga Primary
school – Bugiri District
Head teacher
NabiryeZiria Nansaga Primary
school – Bugiri District
senior woman teacher
Kirigoola Patrick Bulule primary school,
Namayingo
Headmaster
Oundo Wilber Force Bulule primary school,
Namayingo
Senior man teacher
Josephine Mugala
Uganda Water and
Sanitation NGO
Network (UWASNET)
Water Engineer and Research
and Development Officer
61
7.4 References
GOAL strategic documents
GU strategic plan 2012 - 16
History of GU programming
Policies / guidelines reference file / CD ROM (Global, Gov, National):
WASH strategy
A systems approach to sustainable water Operation & Maintenance June 2015
Global WASH strategic framework 2015 draft
GU WASH strategy 2015 – 17 (draft)
GU WASH strategy 2014 – 16
Presentation on market research O&M and san marketing 2015
O&M and Mobile money brochures
WUC training manual old and new
Community Conversations methodology
GU Irish Aid annual reports 2012, 2013, 2014 and 2015
Charity: water proposal SWaSH 2014 – 16
Partner selection criteria
Charity: water proposal SWiM 2014 – 16
MEL
WASH MEL Plan
WASH MEL plan SWiM 2015
GOAL Uganda Monitoring evaluation and Learning (MEL) data 2015
WASH results framework 2012 – 15
Monthly WASH monitoring data sheet Dec 2015
Monthly CC monitoring data sheet Dec 15
Monthly training updates Dec 15
WASH evaluations 2012, 2013, 2014 and 2015 (draft)
North Water external technical evaluation Nov 2015
Irish Aid external evaluation 2015
WASH case studies / learning
Learning presentation on corrosion 2015
Training Water User Committee’s A Training Guideline and Report
GU Training Water User Committee’s A Training Guideline and Report 2016
Partners
Partners proposal in country review
Partnership funding agreement
GEMA business proposition 2015
District Coordination
District Coordination Guideline 2012
District Government MOUs
District reports
Other documents:
Uganda National Population and Housing Census 2014, provisional results
Ministry of Water and Environment Sector Performance Report 2015
WHO/UNICEF Joint Monitoring Program for Water and Sanitation
62
Uganda Sanitation Fund, Country Programme Proposal, Ministry of Health, February 2014
The DAC Principles for the Evaluation of Development Assistance, OECD (1991), Glossary of Terms
Used in Evaluation, in ‘Methods and Procedures in Aid Evaluation’, OECD (1986), and the Glossary of
Evaluation and Results Based Management (RBM) Terms, OECD (2000)
Lockwood, H., Bakalian, A., Wakeman, W., 2003. Assessing sustainability in rural water supply: the
role of follow-up support to communities; Literature review and desk review of rural water supply
and sanitation project documents. [online] WADC: World Bank. Available at:
http://www.aguaconsult.co.uk/uploads/pdfs/WBAssessingSustainability.pdf.
63
7.5 Terms of reference
BACKGROUND ON GOAL’S WASH OPERATIONS
GOAL’s Global operations
GOAL is an international humanitarian and development agency dedicated to alleviating the suffering of
the poorest of the poor. GOAL is a non-denominational, non-governmental and non-political
organization. Since its inception in 1977, GOAL has spent in excess of 910 million EUR on humanitarian
and development programmes in more than 50 countries. GOAL delivers a wide range of humanitarian
and development programmes with focus on the sectors of WASH, Health, Nutrition, Infrastructure,
Livelihoods and Child Empowerment and Protection.
Globally GOAL has a Health Strategy and WASH Strategic Framework that guide programming and areas
of programme development. Within these documents are the following WASH goal and objectives:
WASH Goal (strategic objective): To continue to deliver holistic Water, Sanitation and Hygiene
programmes that target vulnerable groups in a timely and efficient manner whilst impact,
retaining measuring the capacity to respond to rapid onset emergencies, reducing disaster risk for
the chronically vulnerable and building the capacity of a full range of WASH stakeholders to
continue to operate and maintain installed facilities and carry out work in the future.
WASH Objectives:
• Ensure that the most vulnerable and/or disaster-affected, in rural, urban and emergency
contexts, have increased, equitable access to appropriate and sustainable water and
sanitation services and hygiene promotion.
• Strengthening emergency preparedness and contingency planning integrating risk
reduction.
• Ensuring local stakeholders, communities and institutions have increased capacity to
operate, maintain, develop (and possibly replicate) WASH interventions in order to ensure
that WASH interventions are sustainable.
About the country programmes: GOAL operates in 15 countries globally, these are: Ethiopia, Haiti,
Honduras, India, Kenya, Liberia, Malawi, Niger, Philippines, Sierra Leone, South Sudan, Sudan, Syria,
Uganda and Zimbabwe. WASH programming in these locations varies greatly and can cover some of the
following programming areas:
• Water Supply: achieved via a variety of water supply options depending on the context,
including hand dug wells, boreholes (fitted with handpumps or submersible pumps) and
rainwater harvesting. In urban areas GOAL also works with water utilities on extending service
delivery and management of services. GOAL always ensures community management systems
for operation and maintenance are put in place and that government/utility roles are made
clear and linked to other stakeholders.
• Sanitation: GOAL aims to achieve sanitation coverage via Community Led Total Sanitation
(CLTS), although there is acknowledgement that CLTS is not always the most suitable solution
and sometimes an alternative low cost subsidy approach is needed. Sanitation Marketing is
adopted in many countries, both to sustain and improve upon the results of CLTS, but also to
increase sanitation coverage in urban areas. The on-going operation and maintenance (O&M) of
on-site sanitation is also taken into consideration, with faecal sludge management (FSM) being
pursued in urban areas. During emergency response, GOAL is keeping up with the latest
64
developments on emergency sanitation, with the ultimate aim to deliver sanitation as quickly as
possible to those in need.
• Hygiene Promotion: Since 2012 GOAL has actively promoted a Designing Behaviour Change
(DBC) framework approach to hygiene promotion, this has resulted in most countries producing
several DBC frameworks that detail a variety of new types of activity to achieve hygiene
promotion targets. This often can support the use of existing hygiene promotion methodologies
such as participatory hygiene and sanitation transformation (PHAST) or community
conversations. However, in some cases it might lead to more innovative solutions. GOAL also
aims to assess the extent that communities are able to access hygiene hardware; particularly in
emergency response, this is of utmost importance.
• Local Government Engagement: With decentralization taking place and becoming a reality in
many of GOAL’s operational countries the benefits of engaging with local governments on
achieving WASH targets have been greatly enhanced. Although it can present challenges, GOAL
aims to work closely with local governments in planning and coordinating WASH work as well as
in assessing and improving on the quality of work.
• Market-based solutions: GOAL is increasingly interested in market-based WASH solutions with a
focus upon sanitation provision, FSM and O&M of rural water supply as the three main areas.
However, this is also a wider interest should solutions be appropriate to specific country
objectives. The role of government (local and national) is also taken into account in these
approaches, with the aim not to replace government functions but to empower them to see
private sector solutions as something they can utilize to deliver on WASH targets for
communities.
• WASH Advocacy: Although still not a significant part of programming, many GOAL countries
already take part in informal advocacy relating to key WASH issues. This still need to be better
formalized but exists as something many GOAL WASH teams already achieve.
Key areas of WASH programme development: GOAL has been focusing on the following areas of
programme development since 2013 and will continue to adjust as seen fit:
• CLTS and Sanitation Marketing: With the challenges in rural sanitation still significant in many
locations the requirement to come up with solutions to close the sanitation gap, GOAL has used
Community Led Total Sanitation (CLTS) in many countries to achieve change in this area.
Ensuring that households are enabled to climb up the sanitation ladder using their own
resources via sanitation marketing programming is however an ongoing challenge.
• Sustainability of Rural Water Supply: There is still great need to increase water supply
coverage, but one of the greatest challenges facing the WASH sector is the sustainability of such
infrastructure. GOAL has a particular focus on this and on improvement of collaboration with
local government and private sector enterprises that can bring about improvements in results.
• Effective Hygiene Promotion: Through the use of Designing Behaviour Change (DBC)
frameworks GOAL has already catalyzed some change in operations by introducing this
methodology and increasing a focus on behaviour change. The challenge remains to effectively
monitor behaviour and identify a small number of effective activities for certain behaviours.
• Urban WASH: GOAL works in several urban areas globally and has worked on emergency
response, recovery and development programming. Particular interest is paid to faecal sludge
management (FSM).
65
GOAL’s 10 Key WASH Principles
1. Individual programmes will address the ‘three prongs’ of WASH (Water, Sanitation, and
Hygiene), either as an integrated program, or in collaboration with other partners, including
equal considerations for men and women
2. Community involvement and engagement are essential in all aspects of programming
3. Gender mainstreaming is of considerable importance within the WASH sector and all
programmes should be gender sensitive
4. WASH programmes should not only respond to demand, but also create demand
5. Sustainable WASH services should be a primary focus of programming in terms of beneficiaries’
continued use of services and ability to operate and maintain without external assistance. This
should include assessment of a program’s environmental impact
6. Any WASH intervention should be designed and selected so that it is appropriate to the specific
circumstances of the people (particularly the most vulnerable) and the location
7. WASH programming should focus on behaviour change and avoid simply improving knowledge
8. Partnership and capacity building of the whole range of stakeholders within a WASH programme
is crucial for increasing the strength of a programme and also as a programme output
9. WASH does not operate in isolation from other programme teams. Integration of programming
can greatly enhance outputs and thus the well-being of beneficiaries
10. WASH programmes should not only address existing needs of communities, but also be designed
to reduce the vulnerability of communities to future hazards
Methodology for assessing GOAL’s WASH Strategic Objective
The assessment of WASH Strategic Objective will focus on the evaluation of effectiveness – the extent to
which the WASH objectives were attained. This will be done in two stages. During the first stage, actual
contributions of the individual projects within the respective country programme will be assessed. These
contributions will be ranked. In the second stage, the average of rankings from the individual projects
will determine the score for effectiveness of the country programme.
Scale 1-6 (very low, low, rather low, rather high, high, very high), where 1 represents very low and 6
represents very high will be used for ranking the extent of achievement of the WASH Objective. The
ranking will also include an option “not applicable (n.a.)”. The approach is illustrated in the
(hypothetical) table below.
Table 1: Assessing GOAL WASH Strategic Objective
Country Rate of fulfilment
Project 1 Project 2 Project 3 Project 4 Project 5 Project 6 Programme
A Very high (6) High (5) High (5) High (5) Rather low
(3)
Low (2) 26/6 = 4
B High (5) Rather
high (4)
Rather low
(3)
Very low
(1)
n.a. 13/4 = 3
C Rather high
(4)
n.a. 4
Assessment of the 10 Key WASH Principles
Some of the WASH Principles will be assessed in the same manner as the effectiveness of the individual
country programs, based on the assessment of individual projects:
• Principle 2: Community involvement and engagement in all aspects of programming
66
• Principle 3: Gender mainstreaming
• Principle 4: Creating demand
• Principle 5: Sustainability of WASH services including environmental impact
• Principle 6: Appropriateness of interventions for beneficiaries, particularly vulnerable groups
• Principle 7: Focus on behaviour change
• Principle 8: Partnership and capacity building
Other Principles will be assessed at the programme level, using information from programme
documents, reports and discussions with GOAL and relevant partners/key informants:
• Principle 1: Addressing the ‘three prongs’ of WASH (Water, Sanitation, and Hygiene), either as
an integrated program, or in collaboration with other partners, including equal considerations
for men and women
• Principle 9: Integrated approach to WASH programming
• Principle 10: Focus on reduction of vulnerability to future hazards
Sample of summary of conclusions is presented below.
Table 2: Assessing GOAL WASH Principles 1, 9 and 10
Principle 1: Addressing the ‘three prongs’ of WASH (Water, Sanitation, and
Hygiene), either as an integrated program, or in collaboration with other partners,
including equal considerations for men and women
Very high (6)
Principle 9: Integrated approach to WASH programming Rather low (3)
Principle 10: Focus on reduction of vulnerability to future hazards High (5)
Comparison between countries will be possible by the comparison of the individual countries scores. A
matrix will be prepared by countries and by WASH objectives/WASH Principles to allow cross-country
meta-analysis by countries as well as by objectives and Principles within a country and between
countries.
GOAL’S WASH PROGRAMME IN UGANDA
Background
GOAL began working in Uganda in 1979. Employing a market based approach across our focus areas of
Health and Livelihoods, GOAL Uganda’s country programming aims to build resilience and support socio-
economic development. GOAL Uganda (GU) has three core competencies and programming areas;
Livelihoods, WASH and Health. In 2013/14 GOAL Uganda conducted mapping of all water points
(community, private and institutional), and Open Defecation Free (ODF) communities in all operational
areas. GOAL’s intention is to work intensively in a geographic area to stimulate demand and increase
coverage of water, sanitation and hygiene. Water Point Mapping helps to identify the geographical
positions of all water points in a particular location, in addition to management, technical and
demographical information.
Goal, outcomes and outputs
The emphasis for GOAL Uganda’s work in WASH over the period 2012-2015 is to increase and sustain
existing water points and to increase community access to sanitation and hygiene. This has been
implemented both directly and through national partners. Additionally, along with partners, GOAL has
been engaging more at a district level in order to strengthen institutional capacity particularly around
the area of coordination. The key elements of the Theory of change are depicted below.
67
Figure 1: WASH Theory of change
Outputs Outcomes Impact (Goal)
Reduction in
morbidity and
mortality associated
with water related
diseases in GOAL’s
operational areas in
Uganda by
December 2015
Supported communities
have improved access to
an appropriate and
sustainable water supply
Increased capacity of
local communities and
institutions to operate
and maintain, develop
(and possibly replicate)
communal WASH
resources
Increased community
access to saniation
Improved sanitation
practices in supported
communities
Improved community
hygiene practicesImproved hygiene
practises in supported
communities (with a
particular focus on HH
with children < 5 years)
Improved district (meso)
level management of
water points
Increased community
access to drinking
water
Programme focus
Focus of the current program is on sustaining previous investments. GOAL works closely with the Local
District Government (LDG) to support the systems that will maintain existing water infrastructure. Work
on sanitation is community driven and follow up is an integral part of the programme. It is anticipated
that with investment in system strengthening by 2015 communities and LDG will be able to maintain
their own water systems.
Increasing access to drinking water
Since 2010, and across the operational districts16, GOAL has built or rehabilitated 390 water points and
trained Hand Pump (HP) mechanics and Water User Committee (WUC) members to maintain these
points. No more water points are planned under the current programme. GOAL monitors functionality
of water points and WUCs through monthly field visits. Each month, data is collected and analysed on a. Functionality of water points
16
Currently four: Namayingo, Abim, Agago (since 2012) and Kaabong (since 2014)
68
b. Functionality of WUC
This information allows GOAL to plan interventions, such as rehabilitations and refresher trainings. Data
is entered and analysed through an MS Excel Database.
GOAL Uganda aims to further involve the private sector in operation and repair of water points. GOAL
assists the private sector through co-funding business propositions, development of business plans and
linking to WUC and District Government.
Sanitation and hygiene promotion
GOAL Uganda has been implementing CLTS since 2011 in 4 districts. Under the current programme, five
districts are included: Namayingo, Abim, Agago, Pader (phase out in 2013 and since 2014 also Kaabong.
CLTS is operational in specific sub-counties in each district. CLTS is a village-based methodology, which
aims at ‘triggering’ the village population to build latrines for defecation purposes with an ultimate aim
of improving the village health status (particularly reduction in diarrhoea). It uses Participatory Rapid
Appraisal (PRA) methods, which enable the local community to analyze their sanitation condition and
collectively internalize the impact of Open Defecation (OD) on the community’s health outcomes. The
use of crude local word for “Shit” provokes collective local action to become totally ODF. The method
does not involve any hardware subsidy or a hands-on approach by the facilitator but rather, focuses on
igniting a change in sanitation behaviour through social awakening stimulated by a facilitator.
Baseline information is collected before a village is triggered and followed by regular monitoring and
sensitization exercise with the help of district authority and village leaders. This is done to encourage
the community and ensure that the village becomes ODF. An ODF verification exercise is conducted with
the relevant district authority and if a village is found to be ODF, it is then celebrated.
Stages in CLTS:
• Mobilization
• Triggering
• Follow up on ODF rates and latrine construction
• ODF verification
Key monitoring information such as latrines constructed and hand washing facilities built, number of
visits, ratio of households (HH) to latrines etc. are collected on a monthly basis and inputted into an MS
Excel database. The CLTS database is pre formulated to allow for immediate analysis of information to
track progress and assess gaps. The CLTS programme has baselines for all villages triggered since the
project was initiated. Follow up surveys have been implemented to assess progress against key
indicators such as diarrhoea rates, sanitation and hygiene practices and to show trends.
Scope of the Programme
The WASH program 2012-2015 has been – with some alterations over time – implemented in 5 districts,
i.e. Namayingo, Bugiri, Pader, Abim, Agago and recently Kaabong. Evaluation of CLTS will include the
following Districts and Sub-counties:
Table 3: CLTS Program 2012 – 2015: Districts and sub-counties
District Sub-
counties
Parishes Villages In programme
since
#
Population
# Households
Namayingo Banda 5 28 2012 37,328 6,432
Mutumba 7 28 2012 40,967 6,416
69
The drinking water supply program has been implemented in five districts and a larger group of sub-
counties. Kaabong and Pader are again not included since they had no program running throughout the
2012-2015 programme period. The following tables indicate where water points have been constructed
/ rehabilitated. In the course of the programme period, the number of water points has increased by 65
in Agago, Bugiri and Namayingo Districts. Some water points have been submerged or abandoned after
2011. An overview is provided in Table 4 below.
Table 4: Water Supply Program 2012-2015
District Sub-counties Constructed till
2015
Constructed
WP 2011
Still in use of
WP 2011
Still in use of
WP till 2015
Namayingo Banda 48 39 21 30
Mutumba 43 30 19 32
Sub-total 91 69 40 62
Bugiri Bulidah 35 22 21 34
Muterere 21 0 0 21
Sub-total 56 22 21 55
Agago Adilang 5 5 3 3
Kalongo TC 14 14 14 14
Kotomor 25 17 8 16
Lamiyo 9 9 6 6
Lapono 19 19 11 11
Lira Palwo 11 11 9 9
Lukole 6 6 6 6
Omiya 6 6 5 5
Paimol 12 11 4 5
Parabongo 21 21 14 14
Patongo 4 4 1 1
Wol 9 9 6 6
Sub-total 141 132 87 96
Abim Abim S/C 14 14 12 12
ABIM T/C 20 20 16 16
Alerek 14 14 9 9
Lotuke 25 25 22 22
Morulem 31 31 26 26
Nyakwae 18 18 16 16
Sub-total 122 122 101 101
Sub-total 12 56 78,295 12,848
Abim Lotuke 5 51 2012 14,853 2,374
Alerek 4 24 2012 6,251 1,112
Sub-total 9 75 21,104 3,484
Agago Kotomor 6 59 2012 17,594 2,594
Sub-total 6 59 17,594 2,594
TOTAL 5 27 190 116,993 18,926
Kaabong Started in late 2014
Pader Phased out in 2013
70
TOTAL 410 345 249 315
OBJECTIVES OF THE EVALUATION
The main purpose of the evaluation is to obtain objectively substantiated and consistent conclusions
that can be used in the decision making of GOAL on the future direction of GOAL in Uganda in the rural
WASH sector with focus on safe water supply, operation and maintenance of water services, sanitation
promotion (CLTS) and hygiene promotion. Evaluation will be conducted in line with the principles set out
by the Development Assistance Committee of the OECD with reference to programme relevance,
efficiency, effectiveness, impact and sustainability.
Following priorities of GOAL Uganda, particular attention will be paid to:
• Cost efficiency (detailed analysis of value for money – social cost-benefit analysis (SCBA17
)
• Effectiveness of approaches and tools used for increasing demand for drinking water and sanitation-
Effectiveness of Demand Responsive Approach on changing behaviour
• The adequacy of GOAL Uganda monitoring system (data collection mechanism, relevance, quality,
reliability and validity of collected information)
• Use of information from monitoring for planning
• Sustainability - the likelihood of the continuation of the benefits after donor funding has been withdrawn.
(Beneficiaries enjoy the benefits - positive outcomes - as they perceive them.)
• The level of governance (transparency, accountability, participation, rule of law, equity)
• Cooperation with partners
Conclusions inferred from the obtained information can be used for future planning and implementation
of the WASH sector Programme in Uganda.
In addition to the specific country evaluation objectives, an additional objective is to assess the country
programme against the strategic goal and objective for GOAL globally and against GOAL’s 10 Key WASH
Principles with an objective measurement to allow cross-country meta-analysis.
Period covered by this evaluation: 2012-2015.
APPROACH AND METHODOLOGY
Key elements
The assignment will be implemented in accordance with the requirements of the Terms of Reference
(TOR), following the IDEAS Code of Ethics adopted in November 2014. The evaluation team will make
clear to all participating stakeholders especially children of all ages that they are under no obligation to
participate in the evaluation. All participants will be assured that there will be no negative consequences
if they choose not to participate. The team will obtain informed consent, where not possible provisional
consent, from the participants including children by negotiating with their parent.
In case the assessment team does not understand the participants’ first language, an interpreter has to
be used. Team will have to receive prior permission for taking and use of visual still/ moving images for
specific purposes, i.e., for study report and presentations. The evaluation team will assure the
participants the anonymity, confidentiality and will assure the visual data and all other information
obtained is protected and used for agreed purpose only.
17
Guidance in measuring and maximizing value for money in social transfer programmes – second edition,
Department for International Development, April 2013,
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/204382/Guidance-value-for-
money-social-transfers-25Mar2013.pdf
71
The evaluation will be implemented in line with the program design and available project documents. It
will combine primary and secondary research using mixed methods. The approach will be based on
consultations and dialogue. Source of information will be quoted for each finding. Conclusions will be
clearly linked to findings. Own comments by the evaluation team will be marked as such and explained.
Internal and external factors affecting implementation will be addressed. Reliability of data collection
instruments will be verified during discussions within the evaluation team.
Design is non-experimental, one-shot (situation during the evaluation). Before-and-after design without
comparison group may be used if baseline data is available (information on areas of intervention and
beneficiaries will be compared before and after the project implementation– the desired effect should
come only after the beginning of the project. This design is insufficient to demonstrate that the
intervention (program) alone caused the change (causality), but is the only option available in the
absence of a reference group randomly identified before the intervention; the counterfactual methods
cannot be used.
Possible major changes in the approach and methodology will be consulted with GOAL in advance.
All data collected during the evaluation will be appropriately analyzed and electronic copies of data and
calculations made available to GOAL in Excel workbooks. This data will remain under the ownership of
GOAL and will not be allowed to share without expressed permission. At the end of fieldwork,
preliminary findings and conclusions will be shared and discussed with GOAL Uganda team and other
stakeholders as appropriate. An evaluation report will be compiled following the structure outlined in
these TOR.
Language used for the implementation of the evaluation will be English.
Analysis of possible methodological barriers and evaluation limits will be included in the evaluation
report.
Scope of the evaluation
An external evaluation of the Program has been foreseen in the GOAL Uganda proposal 2012 – 2015. In
accordance with GOAL priorities, the evaluation will focus on the following areas of the GU programme:
• Safe water supply (provision of water via borehole drilling) and operation and maintenance of water
services (training of WUCs, sustainability of hand pumps) (further Project Improved access to drinking
water)
• Sanitation promotion (CLTS, beginning of sanitation marketing) and hygiene promotion (via community
conversations) (further Project Improved Access to Sanitation and Hygiene)
The evaluation will include assessment of the two projects on:
• Relevance, effectiveness, efficiency, impacts, sustainability (OECD DAC criteria)
• GOAL 10 Key WASH Principles (see section 1.2 above)
• Ex-post Social Cost Benefit Analysis (SCBA) (Guidance in measuring and maximizing value for money in
social transfer programmes – second edition, Department for International Development, April 2013,
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/204382/Guidance-
value-for-money-social-transfers-25Mar2013.pdf)
Phasing
The evaluation will be implemented in three phases:
Preparatory (inception) phase will take place before the field phase. The preparatory phase
encompasses:
72
• Preparation of data gathering instruments (questionnaires, discussion frameworks, checklists) for
gathering information from relevant institutions, partner organizations, GOAL, WUCs, communities,
school children and other stakeholders
• Gathering of information, review of secondary data – analysis of available data; relevant legislation,
project documentation, periodic project reports, financial reports and budget, monitoring databases,
strategic documents, and other relevant accessible documents
• Speaking with key informants on CLTS programming to bring some thoughts on the benefits and
limitations of the approach from a global perspective
• Consolidation and approval of the Terms of Reference including the methodology by GOAL
• Formulation of hypotheses related to the evaluation questions, based on the résumé of information and
findings. These hypotheses will be verified during the mission in Uganda.
Field phase: The fieldwork will be implemented in accordance with the agreed evaluation questions and
methodology and in compliance with the objectives of the evaluation and expectations of the
contracting authority. This phase encompasses:
• Collection of data from stakeholders in the Programme area
• Detailed consultations with the GOAL Uganda team
• Review of secondary data – including strategic plans, project reports, statistics, monitoring reports,
monitoring reports from previous projects, reports from trainings, information materials and other
relevant documents
• Analysis of information and factors that contributed to successes and failures
• Identification and gathering of missing information
• Verification of hypothesis formulated during the preparatory (inception) phase
• Opening and closing briefings for GOAL Uganda and other relevant stakeholders
During the final phase, the information from the preparatory and field phases will be consolidated,
processed, analyzed, and interpreted in relation to the evaluation questions. This phase encompasses:
• Analysis, synthesis and interpretation of data
• Formulation of findings, conclusions and recommendations
• Drafting and submitting the draft evaluation report
• Processing comments from GOAL and preparing the final report
Important source of information will be feedback from briefings and debriefings and comments on the
draft version of the report.
Evaluation questions
Evaluation questions reflect the objective and scope of the evaluation and encompass the assessment of
relevance, efficiency, effectiveness, impact and sustainability (OECD DAC evaluation criteria), Value for
Money analysis as well as the level of contributions to the GOAL WASH Objective and 10 Key WASH
Principles.
Conclusions drawn from findings for each evaluation criterion will be rated 1-6 (the same rating
applies for the SCBA and 10 GOAL WASH Principles)
Table 5: Rating of conclusions on evaluation criteria and on the 10 GOAL WASH Principles
Rating Interpretation
1 Very low
2 Low
3 Rather low
73
4 Rather high
5 High
6 Very high
Table 6: Evaluation questions
Evaluation
criteria
Evaluation questions
Improved access to sanitation and hygiene
1. Relevance
To what extent
are the
outcomes still
relevant?
1.1. How was community participation ensured? (Provide rating for participation
in planning, implementation and monitoring stage) (Principle 2 – Community
involvement)
1.2. What specific needs of children were considered by designing school
latrines? (Principle 6 – Appropriateness to circumstances & vulnerable)
1.3. What roles play women and children in the intervention? (Principle 3 –
Gender mainstreaming)
1.4. What are the current priorities of the government, partners, beneficiaries
and GOAL Uganda?
2. Effectiveness
To what extent
can the Project
reach the
foreseen
behaviour
change?
(Principle 7)
2.1. What is the most significant behaviour change caused by the Project?
2.2. To what extent did the intervention contribute to the increased capacity of
local stakeholders to maintain sanitation? (Objective) (Principle 8 –
Partnership and capacity building)
2.3. What was the most motivating approach for the changes in sanitation and
hygiene behaviour?
2.4. Is there any monitoring system in place? Sub-questions: Who is using the
results? For what kind of decisions are the results used? Are the results easily
available?
3. Efficiency
How cost-
effective was
the applied
implementation
strategy?
3.1. What is the project’s operational cost-efficiency? Sub-questions: How would
you rate predictability, transparency, and sufficiency of funding?
3.2. How effective were the institutional arrangements for the implementation of
the Project, with focus on roles and responsibilities of GOAL and partners?
(Principle 8 – Partnership & capacity building) Sub-questions: What is the
added value of each implementing partners? What were the main problems
in cooperation?
3.3. What is the main (social) Value for Money?
4. Impacts
What are the
foreseen or
unplanned
benefits?
4.1. What were the main contributions of the programme to changes in
sanitation and hygiene practices of the most vulnerable members of
communities? (Principle 6 – Appropriateness to circumstances & vulnerable)
4.2. To what extent has demand for improved sanitation increased as a result of
the project (Demand Responsive Approach)? (Principle 4 - Responding &
creating demand)
4.3. Do the beneficiaries register any positive impact on health status as a result
of the intervention?
5. Sustainability
How the CLTS
approaches will
continue
5.1. To what extent were the risk factors to sustainability included in the
monitoring system? (Principle 5 – Sustainable services & environmental
impact)
5.2. Has an exit strategy been agreed with partners during formulation? (Principle
74
without GOAL
support?
5 – Sustainable services & environmental impact)
5.3. What percentage of “new” sanitation facilities is still fully functional? Sub-
question: What are the main problems in proper maintenance?
5.4. What is the extent of risk that ODF communities will slip back to non-ODF
status?
5.5. Are there any new local (locally owned) initiatives to improve sanitation?
(Principle 8 – Partnership & capacity building)
Improved access to drinking water
1. Relevance
To what extent
are the project
objectives still
relevant?
1.1. How was community participation ensured? (Please provide rating for
participation in planning, implementation and monitoring stage)? (Principle 2
– Community involvement)
1.2. What specific needs of women and children were considered in designing the
water points? (Principle 3 Gender mainstreaming)
1.3. What roles do women have in the intervention? (Principle 3 – Gender
mainstreaming)
1.4. What are the current priorities of the government, partners, beneficiaries
and GOAL Uganda?
2. Effectiveness
To what extent
can the
programme
reach the
foreseen
behaviour
change?
2.1. What is the most significant behaviour change caused by the project?
2.2. To what extent did the intervention contribute to the increased capacity of
local stakeholders to operate and maintain the water sources? (Objective)
(Principle 8 - Partnership & capacity building)
2.3. How much did the interventions help to ensure a feasible fee collection
system?
2.4. Is there any monitoring system in place? Sub-questions: Who is using the
results? For what kind of decisions are the results used? Are the results easily
available?
3. Efficiency
How cost-
effective was
the applied
implementation
strategy?
3.1. What is the project’s operational cost-efficiency? Sub-questions: How would
you rate predictability, transparency, and sufficiency of funding?
3.2. How effective were the institutional arrangements for the implementation of
the programme, regarding roles of GOAL and partners? (Principle 8 –
Partnership & capacity building) Sub-questions: What is the added value of
each implementing partners? What were the main problems in cooperation?
3.3. What is the main (social) Value for Money?
4. Impacts
What are the
foreseen or
unplanned
benefits?
4.1. How has the program impacted on access to drinking water for the most
vulnerable members of communities? (Principle 6 - Appropriateness to
circumstances & vulnerable)
4.2. To what extent has demand for drinking water services increased as a result
of the project? (Principle 4 - Responding & creating demand)
4.3. As a result of the project, are there demonstrated changes in willingness to
pay for improved access to drinking water? (Principle 7 – Behaviour changes)
4.4. What positive impacts do the beneficiaries register as a result of the project?
5. Sustainability
How the water
sources
operation and
maintenance
5.1. To what extent were the major risk factors to sustainability included in the
monitoring system? (Principle 5 – Sustainable services & environmental
impact)
5.2. Has an exit strategy been discussed and agreed with partners during
formulation? (Principle 5 – Sustainable services & environmental impact)
75
will continue
without GOAL
support?
5.3. What percentage of new water sources is still fully functional? Sub-question:
What are the main problems in proper maintenance?
5.4. How are the cost of operation and maintenance of the water supply systems
covered?
5.5. Are there any new local initiatives to improve access to drinking water?
(Principle 8 – Partnership & capacity building)
Programme level
1. Relevance
To what extent
are the
programme
objectives still
relevant?
1.1. To what extent did the selection criteria for the Program area reflect
vulnerability of beneficiaries? (Objective / Principle 6 – Appropriateness to
circumstances & vulnerable)
2. Effectiveness
To what extent
can the
programme
reach the
foreseen
behaviour
change?
2.1. Has the theory of change been properly formulated?
2.2. Has the theory of change been used for monitoring?
2.3. What is the role of program monitoring in improving program delivery? Sub-
questions: Are data properly used to adjust future programming/planning?
Are lessons learnt from implementation used?
2.4. Has the support to partners been effective in terms of increased internal
capacity?
3. Efficiency
How cost-
effective was
the applied
implementation
strategy?
3.1. Are data collection systems useful for decision making process? Do they
provide reliable data?
3.2. How is the support from GOAL Uganda perceived?
4. Impacts
What are the
foreseen or
unplanned
benefits?
4.1. Is there any indication of mortality or morbidity decrease as a result of the
WASH program? (Reduction of incidences of vector borne diseases for the
population of 6 to 18 years, reduction of infant mortality and morbidity for
the children of 0 to 5 years, other changes since start of the WASH
programme)
WASH
Principle 1
1 a) Have the ‘three prongs’ of WASH (Water, Sanitation, and Hygiene) been
addressed either as an integrated program, or in collaboration with other
partners?
1 b) To what extent were gender concerns (equal consideration for men and
women) integrated into the programme design?
WASH
Principle 9
To what extent is the programme integrated with other programme teams to
enhance the well-being of beneficiaries?
WASH
Principle 10
10 a) What information about WASH conditions or problems prompted the
programme response?
10 b) How does the programme design address the reduction of the vulnerability
to future hazards?
Techniques for data collection and sources of information
76
Techniques for data collection will include:
• Review of the substantive datasets in GOAL Uganda and of other secondary sources (literature,
available documentation, Results framework, Programme progress and financial reports,
statistics, Gov’t policies and strategies i.e. Water policy, District implementation plan, Ministry
of water sector performance report 2012 – 2015. Sector investment plans, Improved hygiene
and sanitation improvement strategy (ISH)) (Review) (List of currently available documents is
provided in Annex A) (water and sanitation)
• Focus group discussions with women (FGDW) (water)
• Key informant interviews with a selection of 10% of WUCs from across the four Districts
(Namayingo, Bugiri, Agago and Abim) (KII WUC); these interviews will be done by independent
enumerators. GOAL Uganda will provide a comprehensive list of water points including
functioning, non-functioning, abandoned and submerged. Selection of WUCs for KII will be done
by the evaluation team in consultation with GU. Completed questionnaires will be submitted to
the evaluation team in electronic form. (water)
• Key informant interviews with households (KII HH) in each of the five sub-counties where the
sanitation programme is implemented (Banda, Mutumba, Lotuke, Alerek and Kotomor). 45 KII
HH interviews will be implemented by independent enumerators (about 9 in each sub-county).
The HHs will be selected from a list of all households by the evaluation team based on criteria
agreed with GU. Completed questionnaires will be provided to the evaluation team in electronic
form.(sanitation)
• Key informants interviews with implementing partners and other key stakeholders included in
the overview of information sources in table 7 below (KII) (water and sanitation)
• Interviews with GOAL staff (GOAL) (water and sanitation)
• Visits and observations (V&O) (water and sanitation) Water points and sanitation facilities for
observation by the evaluation team will be selected on criteria agreed with GU. Selection criteria
for water points will include their operational status. Selection criteria for sanitation facilities
will include stages in the sanitation ladder. There will be at least some overlap with the KII WUC
and KII HH.
Single case studies using qualitative methods will be included to gain in-depth understanding of a
specific process or situation. Cases would be selected by the evaluation team in consultation with GU
and follow on existing change stories in Abim and Agago from 2013 to record possible changes over
time. (CS CLTS, CS Water) (water and sanitation)
The table below indicates techniques to be used for data collection from secondary and primary sources
of information. The table may be revised after initial briefing with GOAL Uganda.
Table 7: Sources of information and data collection techniques
Source of information Techniques for data collection
Secondary sources
TOR Review
Datasets in GOAL Uganda Review
literature, available documentation, Results
framework, Programme progress and financial
reports, statistics, Gov’t policies and strategies i.e.
Water policy, District implementation plan, Ministry
of water sector performance report 2012 –
Review
77
Source of information Techniques for data collection
2015.Sector investment plans, Improved hygiene
and sanitation improvement strategy (ISH)
The internet Review
Follow-up HH Survey Review (Follow up survey will be available in
December)
Primary sources - Water
GOAL Uganda staff GOAL
Beneficiaries
Beneficiaries CS Water
Women (W) FGDW
Water users committees/water points water points
WUC KII WUC
Water points (WP) V&O
Implementing partners
Mucobadi (Abim) KII
UMURDA (Namayingo, Bugiri) KII
WW (Agago) KII
Administration
DWO KII
LDG KII
Ministry of Water and Environment (MWE) KII
Service providers
Hand pump mechanics (HPM) KII
Hand pump Mechanics Association (HPMA) KII
Shops/dealers with spare parts (Suppliers) KII
Experts (such as ex-GOAL staff with good
institutional memory)
KII
Primary sources, CLTS and hygiene
GOAL Uganda staff (GOAL) KII
Beneficiaries
Households (HH) CS CLTS, KII HH
Sanitation facilities V&O
Children FGDC
Community representatives and Organizations
Village Health Committee (VHC) KII
Natural Leaders (NL) KII
Implementing partners
Mucobadi (Abim) KII
UMURDA (Namayingo) KII
WW (Agago) KII
Administration
DHD KII (Health Inspector)
LDG KII
Service providers
Local artisans (Pit-diggers, masons, carpenters, etc.) KII, V&O
78
Source of information Techniques for data collection
(Artisans)
Sanitation product suppliers (RSM, manufacturers,
wholesalers, retailers)
(Suppliers)
KII, V&O
Financial institutions
Village Savings and Loan Associations (VSLA) KII
Formal banks (Bank) KII
Primary sources: Program level
GOAL Uganda staff (GOAL) KII
Ministry of Water and Environment (MWE) KII
Other donors, projects, related interventions
UNICEF KII
UWASNET KII
A schedule for interviews, meetings and observations will be prepared prior to the arrival of the team in
Uganda in consultation between the Team Leader and GOAL Uganda.
Evaluation matrices
Evaluation matrices will be used including evaluation criteria, questions, sub-questions,
indicators/standards (for normative questions), availability of baseline, type of question, design, as well
as data source and data collection instruments. A digitalized version of the matrices includes software
that allows for generating consistent questionnaires and checklists. The matrices are attached in Annex
B.
Methodology
Interviews with GOAL and key informants, and possibly review of statistics will be conducted at the
national level (Kampala) as well as at the Namayingo, Bugiri, Agago and Abim Districts level.
Selection of households and water user committees for KII is described in section 4.5 above.
Selection of water points and sanitation facilities for observation is described in section 4.5 above.
Villages for KII with HPM, suppliers, artisans, VHC, NL VSLA and bank will be selected by the evaluation
team in consultation with GU. Particular attention will be paid to suppliers of sanitation products and
services and to representatives from financial institutions to establish the potential for M4P. For the
selection of villages, the following will be taken in to consideration:
• Highest share of vulnerable population (highest incidence of poverty)
• At least 2 from remote areas
• Diverse ethnicity
• Diverse religion
• Diverse communities (e.g. fishermen, farmers, ...)
5FGDW will be conducted at water points selected by the evaluation team in consultation with GU. The
following may be taken into consideration:
• To save travel time, FGDWs may be conducted at water points in the same villages as selected for KII with
local key informants
• Both, functional and non-functional water points should be included
79
Focus group discussions will be conducted by both consultants, with the support of a facilitator and an
interpreter (for the international consultant). Neither the facilitator nor the interpreter should be from
GOAL or the Implementing Partner. Records from focus group discussions will be discussed and
consolidated at the end of the day. Following transcription, the analysis process involves:
• Thorough review of transcripts and identifying relevant themes and sub themes
• Organizing quotations with accompanying respondent information
Case studies will follow on already recorded cases to record possible changes over time, to the extent
possible in the same villages as selected for the above data gathering. The beneficiaries will be selected
by the evaluation team in consultation with GU. Preferably both, success stories and less successful
cases should be included.
Instruments for data collection for each information source will be generated by the software used for
preparing the evaluation matrix and adjusted as/if required for effective communication with
informants. Information obtained during interviews with key informants will be entered in digital form
during the interviews or transcribed from handwritten to digital versions. Interviews with key
informants will be held by one or by both consultants. If both consultants conduct the interview, their
raw notes will be typed into one e-questionnaire.
GOAL will provide 2 staff to facilitate and organize meetings, overall coordination for 5 days, the
intended stay in one survey area. Since English is widely spoken by KII it is not likely that there is a need.
The tentative number of focus group discussions and interviews with key informants and households in
the Districts is included in Table 8 below. The table does not include interviews with some key
informants included in Table 7 above, such as implementing partners, financial institutions or teachers.
The list with a detailed work plan will be finalized by the Team Leader in consultation with GOAL before
the start of the fieldwork.
Table 8: Tentative number of FGDs and interviews in the Districts
*Optional if time allows. 8 in-depth interviews would probably do.
District Project
sub-
counties
Project
villages
Selected
villages
NL,
CLTS
teams
District
and sub-
county
officials
(DWO, HI,
CDO)
Masons,
HPMs,
supplier
s
FGDs
with
women
HH KII
(including
rapid ODF
check)
Namayingo Banda 28 2 2 2 2 1 5
Mutumba 28 2 2 1 2 1 5
Bugiri
Abim Alerek 24 2 2 2 2 1 5
Lotuke 51 (2)* (2)* (1)* (2)* 1 5
Agago Kotomor 59 2 2 2 2 1 5
Total 5 190 10 10 8 10 5 25
80
Table 9: Preliminary outline of programme
Date Activity Sources of
information
Timing
10-01-16 Travel to Bugiri
11-01-16 Interview with partner staff (2) Morning
Interview with GOAL staff (2)
Instruction interviewers Afternoon
Interview KII Bugiri DWO
Interview KII Bugiri HPMA or HPM
12-01-16 Interview KII Namayingo DWO Morning
Interview KII Namayingo HI
Interview KII Namayingo HPMA or HPM
Interview 5 households on sanitation Village A Afternoon
Physical check water points Village A, Y and Z
13-01-16 Physical check water points Village B, C and D Morning
Interview CLTS team or NL Village B
Interview 5 households on sanitation Village B Afternoon
Discussion with interviewers WUC
14-01-16 Interview Mason/HPM in Namaying0 Morning
Interview CLTS team or NL Village E
Physical check water points Village E, F and G
Interview CLTS team or NL Village G Afternoon
FGD Women in Namayingo Village G
15-01-16 Interview HPM(A) in Bugiri Morning
Physical check water points in Bugiri Village H, I and J
Interview Financial institution In Bugiri Afternoon
16-01-16 Travel to Agago
TENTATIVE TIME FRAME AND STAFF INPUTS
Table 10: Tentative timeframe and staff inputs
Day Activity Input (working
days)
TL TM SCBA
81
04 – 15 January Review of available project data sets and documentation
5 4 2
06 January Travel day (depending on the availability of flights) 1
07 January Meetings in Kampala (GOAL team, key informants) 1 1
08 January
Briefing GOAL, reconciling documents/secondary data,
reviewing implementation schedule and arrangements for
field work, KII
1 1
09 January
Finalizing questionnaires for WUC and HH interviews with
enumerators, KII interviews with implementing partners,
Drafting SCBA
1 1 2
10 January Travel to Bugiri 1 1
11 – 15 January Field work (Bugiri, Namayingo) (KII, FGDs, processing)
information) 3 3
16 January Travel to Agago 1 1
17 January Preparations, consolidation of preliminary findings 1 1
18 – 23 January Field work (Abim, Agago) (KII, FGDs, processing information) 3 3
24 January Travel to Kampala 1 1
25 – 26 January Summarizing key findings, debriefing presentation 2 2
27 January Departure/travel day 1
Consolidation report and Value for Money analysis18 10 4 0.5
15 February Submission of first draft report
28 February Input from IE Committee (GOAL)
Incorporating comments 1
05 March Submission of 2nd draft
Incorporating comments 1
10 March Submission final report
Total working days (estimate) 34 23 4.5
18
9 days are included for drafting findings, conclusions and recommendations on OECD DAC evaluation criteria and
10 GOAL WASH Principles, 4 days for 2 Value for Money assessments as per the Guidance in measuring and
maximizing value for money in social transfer programmes – second edition, Department for International
Development, April 2013
82
7.6 Itinerary for the evaluation
GOAL Uganda
External WASH Evaluation
17th
Jan - 2nd
Feb 2016
Agenda for WASH Evaluation
Consultant: Anton Rijsdijk / Dennis Nabembezi
Field visit plan
Date / time Consultant 1 Consultant 2 Person responsible Docs / Actions
Sunday
17.01.16
13:30 pm Pick up Arrange pick up and
accommodation Sun & Mon
Monday
18.01.16
09:00 am Security brief Charles SOP docs
10:00 am Programme briefing in Kampala Alan, Claire,
Maurits, Mary,
Fiona
Presentations
16:00 pm Finance Julius Per diem
Tuesday
19.01.16
09:00 am Ministry meeting (O&M) Steven / Mary O&M
11:30 am UWASNET (Coordinator)
Doreen / Mary Case studies
12 am GOAL Uganda
Mary Presentations
15:00 am Travel to Bugiri Mary Vehicle / accommodation
Tues, Wed, Thurs, Friday
Wednesday
20.01.16
Bugiri
08:00 am Security brief Joseph
08:30 am Interview with GOAL staff Joseph, Simon,
Mary
10:30 am Interview with CSO partner staff UMURDA Joseph, Simon,
Mary
13:00 pm Field visit
3 water points
3 WUCs
1 HPM
Bukudhulu
Nansaga
Itoolo A
(Fred)
Field visit
3 water points
3 WUCs
1 HPM
Nakyegereike
2
Nakisenyi 2
Isakabisolo 1
Joseph, Simon,
Mary
2 vehicles
2 translators
83
Date / time Consultant 1 Consultant 2 Person responsible Docs / Actions
(Aramanzani)
Thursday
21-01-16
Bugiri
09:00 am GEMA (private sector O&M partner) Joseph, Simon,
Mary
1 vehicle
10:30 am District Water Office Joseph, Simon,
Mary
1 vehicle
12:00 am Expert Concrete
(San marketing partner)
Joseph, Simon,
Mary
1 vehicle
14:00 pm
Institutional Sanitation
(CHAST)
Nansaga P/S
1 CLTS team
8 HH on sanitation
Namungodi
Joseph, Simon,
Mary
2 vehicles
2 translators
15:00 pm 1 CLTS team
2 HH
Bukudhulu
Joseph, Simon,
Mary
2 vehicles
2 translators
16:00 pm Women Focus Group
Discussion
Joseph, Simon,
Mary
2 vehicles
2 translators
8:30am Travel to the field Joseph, Simon,
Mary
2 vehicles
Friday
22.01.16
Namayingo
10:30 am District Health Inspector District Water Officer Joseph, Simon,
Mary
2 vehicle
12:00 am Institutional San (CHAST)
Bulule Primary school
2 water points Bugali B
2 WUCs Hagulu
Joseph, Simon,
Mary
2 vehicles
2 translators
14:00 pm 1 CLTS team
5 HH
Mutumba B
3 water points Bumeru A
Buchimo C
Lugala C
3 WUCs
Joseph, Simon,
Mary
2 vehicles
2 translators
16:00 pm Women Focus Group
Discussion (Mutumba B)
2 HPM Siraji& James Joseph, Simon,
Mary
2 vehicles
2 translators
8:30 am Travel to the field Joseph, Simon,
Mary
Saturday
23.01.16
10:00 am 1 CLTS team Nangera A
2 HH on sanitation
3 HH on sanitation Magooli Joseph, Simon,
Mary
1 vehicle
2 translators
84
Date / time Consultant 1 Consultant 2 Person responsible Docs / Actions
Namayingo
13:00 pm Travel to Soroti –Akello Hotel Mary 1 vehicle Accommodation
Soroti on Saturday
Sunday
24.01.16
08:00 am Travel to Agago
Mary Accommodation Agago
Sunday, Monday, Tuesday
13:30 pm Arrival and Consolidation of findings Consultants
Monday
25.01.16
Agago
08:00 am Security brief Elly
08:30 am Interview with GOAL staff Emma
10:00 am Interview with CSO partner staff WW 1 vehicle
11:30 pm District Health Inspector District Water Officer Emma, Mary 2 vehicles
2 translators
13:00pm Field visit Kotomor
2 water points
2 WUCs
Awong and Akenowor
Field visit Kotomor
2 CLTS teams
5 HH
Tetworo and Okunamor
16:00 pm
8:30am Travel to the field AGAGO Emma Tino
Tuesday
26.01.16
Agago
10.00 am Field visit Kotomor
2 water points
2 WUCs
Awong and Akenowor
Field visit Kotomor
2 CLTS teams Tetworo and
Okunamor
5 HH
Emma, Tino 1 vehicle
12:00 am Institutional San (CHAST)
2pm
5 water points
5 WUCs
OlyeloTekulu
Acwao
Kotomor East
Kulir south
Juklebi
Emma, Tino 2 vehicles
2 translators
14:00 pm 1 CLTS team
5 HH
Olyelo central
Emma, Tino 2 vehicles
2 translators
16:00 pm Women Focus Group
discussion
Ebule
2 HPMs
Mr Omar Michael and Mr
OgwangTarasisto
Emma, Tino 2 vehicles
2 translators
8:30 am Travel to the field Emma
85
Date / time Consultant 1 Consultant 2 Person responsible Docs / Actions
Wednesday
27.01.16
Agago
10:00 am 1 CLTS team
5 HH
Akurimoa
3 water points
3 WUCs
Kenyal Central
Akuri
Abone North
Moneoroma
Emma, 2 vehicles
2 translators
14:00 pm Travel to Abim
Mary 1 vehicle
Accommodation Abim Wed,
Thursday, Friday
Thursday
28.01.16
Abim
08:00 am Security brief Mike
08:30 am Interview with GOAL Staff
9:00 am Interview with CSO partner MUCOBADI
10:30 am District Health Inspector District Water Office Emma, Tino, Mary 1 vehicle
13:00 pm 1 CLTS team
- Aridai South-Lotuke
5 HH
- Aridai South-Lotuke
3 water points
- Ajopiro - Lotuke
- Kathabok TC - Morulem
- Lobolwala - Morulem
3 WUCs
- Ajopiro - Lotuke
- Kathabok TC - Morulem
- Lobolwala - Morulem
Emma, Tino, Mary 2 vehicles
2 translators
16:00 pm Women FGD,
- Aridai South - Lotuke
2 HPMs
- Lotuke Sub County
Emma, Tino, Mary 2 vehicles
2 translators
8:30am Travel to the field Emma, Tino, Mary
Friday
29.01.16
10:00 am
1 CLTS team
- Nyikinyiki - Alerek
5 HH
- Nyikinyiki - Alerek
2 water points
- Tyenopobo west Alerek
- Kathimangor BH - Alerek
2 WUCs
- Tyenopobo west Alerek
- Kathimangor BH - Alerek
Emma, Tino, Mary 2 vehicles
2 translators
86
Date / time Consultant 1 Consultant 2 Person responsible Docs / Actions
14:00 pm 3 water points
- Akwanamor - Abim SC
- Arengkitoi - Abim T/C
- Angwee South - Abim T/C
3 WUCs
- Akwanamor - Abim SC
- Arengkitoi - Abim T/C
- Angwee South - Abim T/C
Emma, Tino, Mary 1 vehicle
1 translator
Saturday
30.01.16
08:00 am Travel to Kampala Mary 1 vehicle Accommodation
Saturday, Sun, Mon and
Tuesday
Sunday
31.01.16
Rest
Monday
01.02.16
Additional interviews
Data base
Maurits
Tuesday
01.02.16
Validation WS?
Debrief / workshop
Maurits
87
7.7 Summary of interviews
Health interviews
Bugiri District
Mutumba Robert – Ass. District Health Officer- Environmental Health
Malinga Isaac – Senior Health inspector
Relevance (sanitation)
82% of the district has access to sanitation, however this varies from sub-county to sub-county with
some being as low as 54% (Nabukalu and Bukudulu). The district can only undertake sanitation
improvement in two (2) sub-counties per year with an annual budget from the sanitation grant of 25
million (app. $7,000). This leaves out all the other 7 sub- counties unattended too.
The district is implementing home improvement campaign which is slightly an expensive approach
where they intervene in a home and ensure that all key sanitation facilities are installed.
Hand washing is only estimated at 25% which is very low compared to the national average of 32%.
Challenges of improving sanitation and hygiene in the district
Limited funding which affect the district interventions across the sub-counties
Collapsing soils in the sub-counties of Nabulalu and Bulidha, while in the town council, the soils are
rocky and clay. Slow uptake of sanitation marketing strategy due to low incomes of the local population.
Upcoming rural growth centers and the town council which present new challenges of sanitation like
fecal sludge management. The district has no plans on how to guide and manage sanitation in those
centers which will cause a problem in the near future with the population explosion.
Goal Intervention (effectiveness)
Through a baseline survey, needs assessment and through the district coordination meeting, Goal is
targeting two (2) sub-counties of Bulidha and Muterere which had the least sanitation coverage at about
45%. World vision another partner is also targeting three (3) sub-counties of Nabukalu, Buwungu and
Nankooma while the district is targeting two (2) sub-counties of Bulesa and Budagaya out of the total 13
sub-counties. Goal is using CLTS as a model of sanitation improvement in the village as a whole and
supplements ODF villages with water sources. Over 69 villages have been declared ODF. Goal is also
promoting school sanitation in three primary schools with sanitation facilities, hand washing facilities,
sanitation and hygiene promotion and improving access to safe water (borehole in each target school).
Strategy to improve sanitation (efficiency)
Adopt CLTS as sanitation improvement approach which is low cost and with a wider coverage
Integrate fecal sludge management into sanitation improvement strategy for the urban council and rural
growth centers.
Namayingo District
Mathias Mangeni – ASS. District Health officer
Mukyala Veronica – Acting Ass. DHO- Maternal and Child Health
Oundo Humphrey – District Health Inspector
District Profile
9 sub counties in total, Average population 260,000 people
88
Sanitation coverage – 63% with some disparity, some sub-counties as low as 5%9 Island sub-counties -
2.8%). Hand washing – 22%
Challenges of sanitation improvement in the district (relevance)
The District staffing is not trained in CLTS and lean, for example only two staff has been trained by the
Ministry of Water and Environment.
School sanitation is almost zero with no hand washing facilities, the pupil to stance ratio is very high for
the district due to collapsing soils which make investment vey expensive .
Sanitation at the health facilities is also limited with many health centers don’t have access to adequate
sanitation.
There is a challenge of O&M of water facilities including water, where the district is mostly a landing site
and boarding the lake
Limited financing for sanitation: The sanitation grant is only 23 million Uganda shillings (app. $600)
annually to promote sanitation and hygiene in the whole district. This level of financing can only cover
two sub-counties and only a few parishes and about 40 villages in the sub-county.
Limited staffing, for example the district has 2 health inspectors instead of 6, 9 health assistants instead
of 22, and 9 CDOs instead of 12. This staffing limits support to sanitation promotion.
Goal intervention (effectiveness)
Targeting two sub-counties of Banda and Mutumba sub-counties with CLTS promotion, training of water
user committees and school sanitation in three schools
Urgent support
Support the district water sector with piped water scheme to deal with issues of water parity.
Support social mobilization campaigns to support household sanitation through integration of CLTS and
sanitation marketing
Support institutional sanitation as key entry points with sanitation facilities and water for example,
health centers, schools including training of water user committees, sanitation clubs, SMC, PTA to
mention on their roles and responsibilities
Support district staff with capacity to rollout CLTS to other sub- counties where CLTS is not working.
However there is need to integrate CLTS with approaches especially for the transit population and
fishing communities.
Support water quality testing and monitoring in the district to ensure water quality.
Abim District –DHI
Mathias Buteraba –DHI- Environmental health
District sanitation profile:
Sanitation coverage is about 56%, hand washing is estimated at 38%, hand washing in schools is about
25% and the pupils to stance ratio is estimated at 1:60.
District sanitation staffing is estimated at 56% and financing is only from sanitation grant of about 22
millions and the DWSCG of about 30 million for the whole district including software activities for
preparation of new water points. Other funding includes UNICEF which target diarrhea reduction in
children less than five years with about 12 million Uganda shillings.
89
Challenges in Hygiene and sanitation promotion:
Negative attitude towards sanitation promotion: the community is recovering from the relief period to
development work which was used to handouts and subsides, with approaches like CLTS without
subsides, it makes the adoption of household latrines quite challenging.
The land texture is another challenge where in some parts it is rocky while in others it is sandy and
collapsing which make excavation of the pits a challenge for the community
Limited financing to the sanitation sector make implementation of the district mandate slow since
money available is only for twenty villages
Human resources gap: the district is operating 56% capacity for the sanitation sector, some sub-counties
don’t have health assistants to promote sanitation, and even those which are present are not
adequately trained in sanitation promotion approaches like CLTS. This has further hampered the efforts
of the district to promote sanitation.
Areas of Goal intervention:
Focus o school sanitation which is still very, most especially the issue of menstrual hygiene, and fecal
sludge management to reduce on the cost of providing sanitation in the schools. The current practice is
that, when the pit is full, instead of emptying, the school ends up constructing a new one. Alternatives
like IMOs would go a long way to reduce the content in the pit by about 50%.
Focus on household sanitation and hygiene improvement in other sub-counties which have very low
coverage ranging between 20 and 30%.
Interview with the MoWe
Stephen – Sector Advisor- Ministry of Water and Environment
Relevance of Goal work to the sector
Goal is trying to address the functionality problem for the rural water supply especially the hand pumps.
The sector is grappling with the issues of O&M and the nexus between functionally and
decommissioning of hand pumps. There is also a problem of accurate data about functionality. Geol
Uganda is trying to collect routine data related to functionality of the water supply
The Government has piloted a number of models to address functionality issues which are not working.
For example, the issue of major [any repair above 300,000/= (app. $85) is left to the government/local
Government to handle and the community can only take over minor repairs below $85). The second
model which the ministry has tried is the Hand pump mechanics associations- HPMA [ don’t have access
to tool, limited training, lack business capacity and skills, and lack management and organization) taking
over the role of managing O&M of the water supply in the rural areas. The other option is the sub-
county water boards which have all failed.
Limited coordination in the sector for water and sanitation supply, for example rural water supply is a
docket of the ministry of local government which have limited capacity to implement the service, they
rely only on contractors to deliver the service
Access to water is also challenged by the settlement pattern where people settle between averages of
500 meters to a kilometer which make access to water through point sources a challenge. It actually
means to meet the SDGs, one need to construct at least a point water source for every five households
90
Goal Uganda is piloting a community based management model which looks at functionality of the
water supply facilities as business rather than a voluntary services as for the cases of the HPM and water
boards.
The sector is also facing a critical issue related to the training of the water user committees and
orientation on their roles, issues of accountability, little collection and access to spare parts for the
repair of the hand pumps. Goal and partners are piloting a mobile banking platform where O&M
collections would be banked online and accessed by the WUC directly to pay for O&M of the point
source. Each hand pump and the committee becomes an account and collect money for O&M of the
pump regularly. It is envisaged that this would reduce on the issues of accountability and none payment
of the HPMs when they have undertaken repair of the pumps.
Recommendations
There is need to modify the model and tag it on the functionality of the sources, for example, if the
Business model Goal is developing is to effectively work, there is need to pay the HPMs for the number
of days the pump works not the number of the pumps they have repaired. This will ensure constant
follow of water at the pumps. Therefore, there is need for regulation on the average cost for
maintaining the borehole all year, then this cost is divided into the days in the year and this will
determine the transfer to the private sector for the maintenance of the hand pump per month based on
the days the pump is functional.
There is need to link the water supply to economic activities with real value to the community, for
example which livelihood of the village depends on, this will also help people to come and settle closer
to the livelihood activity and hence they will ensure the functionality of the source since their livelihood
depends on it
Ensure water quality for the point sources[ without the water quality, people will always abandon the
clean and safe water source if the water is say salty, this will affect the functionality of the source] and
they will always go back to the unsafe water but perceived to be of better quality than the safe water
source.
Share the Goal learning to the wider sector, especially issue related to the O&M [ business model for
addressing functionality of the water sources, mobile money banking for the point sources, spare parts
standardization , say the GI, stainless steel, PVC and PPR pipes in relation to corrosion and breakdown of
the pumps].
Nansaga Primary school – Bugiri District
Ware Isaiah- Head teacher
Nabirye Ziria – senior woman teacher
School profile (relevance)
Population -1338 (2015)- 632 girls and 706 boys, in 2014, the school had a population of 1008 pupils
Toilet stances- 47 and the pupils stance ratio stands at (1:28)
Total of 18 teachers and the staff ceiling would be 21 teachers
Performance, 3 first grades, 45 second grades and 23 third grades in 2015, in 2014, the school got 2 first
grades, 23 second grades and 54 third grades.
91
The school receives a UPE grant of approximately 2 million Uganda shillings (app. $570) per term.
Toilet design, 3.5 meters deep X 2 meters wide X 6.7 meters long and lined with 25 stances.
The project targeted about 8 schools of about 1200 pupils each
Sanitation improvement (effectiveness)
The school received a new sanitation facility with 25 stances with 10 for boys and 15 for girls inclusive of
the urinal and one for the disabled children. The girls’ washroom was not integrated into the design
however the evaluation was informed that the PWD facility will be modified to serve both purposes. The
facilities are also equipped with a hand washing facility.
The school has put in place emergency measures to handle menstrual hygiene issues, for example
emergency uniforms, soap, basin, and emergency sanitary pads. However the pupils haven’t been
trained on the making of re-usable pads.
Sanitation clubs have been trained in the schools through training of the sanitation focal teacher, who in
turn trained the club members on their duties and responsibilities.
Recommendations:
Integrate menstrual hygiene facilities on the design of the new toilet structures to include a washroom
plus all the other requirements to separate them from the disability facilities.
Integrate drinking water for the children at school to give them a complete package
Integrate an incinerator on the design of the school latrines to deal with issues of disposing pads to
avoid filling the latrine very fast and also to ease emptying
Integrate making of reusable pads into the sanitation club training to offer the young stars an alternative
to making re-usable pads which are cheaper and environmentally friendly.
Bulule primary school, Namayingo
Kirigoola Patrick – Headmaster
Oundo Wilber force – senior man teacher
School profile (relevance)
Population 2015, boys 584, girls- 559 and total 1143, in 2014, boys 613, girls- 568, total 1181,
Performance, 1st grade, 1, second grade 30 and 3rd grade 33 in 2015, in 2014, first grade 1, second grade
30, third grade 23.
Pupils to stance ratio 1:39.
Pupils to class room ratio is 1:114
The pupil t teacher ratio is 1: 76 from the national recommended of 1:58
Facilities (effectiveness)
The school was provided with 20 stances, 4 blocks of the toilet each five (5)stance making a total of 20
stances with a urinal for boys and PWD facility, however without a provision for the girls washroom. In
2011, the school also received 2 blocks of five stances each with a washroom for girls but no provision
for the PWDs.
The school sanitation clubs were trained by the teachers who had been trained by Goal through Urmda.
The school has a washroom on the old latrine design which the pupils use equipped with items for
emergency menstruation.
92
The school was also supported with a bore hole to provide access to safe water for the children.
Note: It is not clear why some schools in the new grant got different numbers of stances and why there
were no facilities for girls (washrooms) yet they were reported in the school package. This may raise
questions on the differences in budget allocation and whether correct specifications were followed.
Recommendations,
The new school latrines need to be remodeled to include girls (washroom) as it was stated in the design
of the toilets. There is need also to verify why there are differences in the number of stances across the
schools yet the budget and the design is the same.
Integrate making of re-usable pads in the training of the sanitation clubs to offer the needed skill in
management of menstrual hygiene for the young girls.
CLTS-WOMEN-FGDS-ABIM DISTRICT
Aridai south – CLTS team meeting
Nyiki Nyiki Village Women /CLTS FGD
Village profile and sanitation issues:
Across the villages, households range from 60 and 80 with majority of them having the four critical
hygiene and sanitation facilities, like lateen, hand washing facilities, bath shelter and the drying rack.
Majority of the latrines are temporary constructed out of mad and wattle.
Almost 80% of the villages took part in the CLTS trigger meeting and very vividly remember the shit
demonstration as the most influencing activity to behavior change.
Majority of people in the FGDs indicate that the shit demonstration encouraged them to set up the
latrines after realizing that they were eating their own shit.
Across the two villages, there were no cultural issues that stop women from using the latrine.
Role of men, women and children in household larine construction
The responsibility of constructing the latrine falls on the man who is the head of the household. The
man is responsible for excavating the pit, roofing and putting the door. The women and the children
share the roles and they are sometimes identical. The women are involved in collecting grass, smearing
the wall and maintaining the cleanliness and providing anal cleansing materials. The children also help
in fetching water, making bricks and cutting the logs in addition to supporting the mothers’ roles. The
roles of the children and the women are interchangeably played amongst themselves depending on the
age of the children.
Major changes noticed
Across all the two villages, the members report reduction in diarrhea incidents among children, due to
improved latrine use and hand washing in the community
There is also improved appreciation of the latrine in the community where many people have adopted
more permanent latrine structures with sanplats. Those with sanplats reported to having bought them,
from the market at a price of between 30,000 and 40,000 Uganda shillings.
Menstrual hygiene:
93
All parents acknowledge supporting their girl child with disposable pads and parents are using cloth
while undergoing menstruation, however there were no mention of having skills at both community and
the school girl level on making re-usable pads.
Challenges of maintain or constructing latrines:
Soil texture which makes excavation of pits a challenge
Termites eat away the materials used in construction of the latrines
CLTS/Women groups Agago district
Olyelo Central FGD – CLTS group members
Ebule Ward/Women and CLTS group
Achanan Harriet
Akuremior – Kutomor CLTS Group
Village sanitation profiles
The number of households in each village range between 24 to 75 households, household latrine
coverage is reported to almost 100% with some very old women and disabled people are helped by the
CLTS groups to put up household latrines.
Health /hygiene promoters and natural leaders trained by Goal and other partner organizations are the
main source of knowledge on how to build the latrines.
Across all the villages, the community through the CLTS meeting agreed on the depth of the latrine
which ranged between 10-25 feet to ensure sustainability of the ODF practice. The super structure could
vary with some having rounded structures and others square forms.
Role of women and children in sanitation promotion
Across all the villages, the women and the children share similar roles in construction of household
latrines, for example both women and the children collect water, mould bricks, collect grass and logs
(especially the children) while constructing the latrine. Routine maintenance of the facility is also a role
of the women and the children. For example the women smear the floor and the wall with cow dung,
clean inside the toilet and provide cleansing materials with the help of the children. However no
consultation on the preference for both the children and women needs was carried out.
Construction of the latrine however is the man’s responsibility in the home. The man is responsible for
digging the pit, laying the slab, roofing and putting up the super structure.
Most influencing to behavioral change
All most all households that participated in the sanitation trigger meeting noted that the shit
demonstration (either with food or water) was the most influencing activity to behavioral change.
There is reported reduction in diarrhea especially among children across all households and villages due
to improved sanitation (latrine use) and hand washing at household level.
Challenges in marinating the latrines and sanitation
The temporary nature of the materials like logs and grass make the maintenance of latrine. The logs are
also becoming scare with increased burning of charcoal.
94
There is also a problem of soil texture where in some areas the soils are week and sand which collapse
during the rainy seasons while in some parts the soils are rocky which make excavation of the pits very
hard.
Lack of access to sanitation products like slabs, jerrycans for tippy taps which are permanent and
resistant to termites which eat away the logs and the grass used for latrine construction
Across all communities, there were no mentions of cultural issues that stop women from using a latrine.
Negative influence from previous projects, earlier projects which worked in the camps offered free
subsides to sanitation like sanplats, jerrycans, slabs and other products in the promotion of sanitation at
households using PHAST approaches. This further constrains sanitation promotion especially CLTS which
doesn’t support use of any subsidies.
Menstrual hygiene:
Girls seek help from their mothers and usually use disposable pads which are very expensive. The
parents also mention some using disposable pads and majority use clothes. There was no mention of
training in making disposable pads which would go a long way in promoting menstrual hygiene.
Sustainability of sanitation facilities
Involvement of the children and women in maintenance of the household sanitation facilities provides
cheap labor to ensure that the latrine is sustained. In addition the women and the children support the
construction of other sanitation and hygiene facilities in the home like drying racks, bath shelters etc.
Availability of local materials, although there were mention of the termites that eat these materials,
communities noted that their availability support the maintenance of the latrine facilities since they are
freely available in the community.
Approaches to promote sustainability
VSLA have potential to support communities in accessing sanitation products.
Sanitation marketing, this is likely to bring close sanitation products like slabs which can promote
sustainability
Train CLTS teams in making of slabs to bring them closer to the population who need them to improve
on their latrines.
Meeting with district water office Bugiri (20/11)
Total coverage 58 % June 2015 and 90 % operating
WUC tasks: Collect user fees. Every sub country has a HPM
District level major repairs are done by the local gov.
30 % of the budget goes to repair, that is 20 BH, but there are 100 BH waiting for repairs
WUC should do the minor repairs and no need for the contribution by the village
Now an association of HPM two years old organized by the local gov.
The district has a budget for reactivation of 50 villages / yr. in the district there are 500 WP
70 % of the WUC is active
Main problem is that people do not pay because they expect HP maintenance / repair is a government
Business. The HPM were trained 20 years ago after that no new training
95
11 HPM are in the district but only 8 are active. 1 HPM for 50 WPs. Also the relationship of HPM with
communities is not commercial it is seen as voluntary work
Meeting with water officer Namayingo (22/1)
The district has 350 WP water with coverage of 34 %, functionality of 74 %
In 2015 the district drilled 13 BH
Repaired 14 BH in 2015, but received 35 requests this year by district
Still 45 HP waiting to be repaired and the funds are not sufficient for HP repairs in the village
10 HPM all working and have an association, but these are poorly paid
Only 60 % of the WUC collected some money. Often the HP not greased and breakdowns. But
breakdown due to corrosion is also common. The water level is stable in the district
GOAL is involved in planning to avoid duplicating
Meeting with DWO in Agago (25/1)
WUC not always operational and management is weak. District has stock of pipes at subsidized prices,
but this will be phased out
The district has 1058 BH of which 70 % functional and a 68 % coverage
The district drilled 16 BH and dug 5 shallow wells. About 20 WP need repair
About 100 abandoned BH at places of IDP camps these should be decommissioned
This year district rehab 10 BH and concern 10 in other districts
In total 56 HPM working in the district and have HPM association
HPM get low payment. GOAL is doing well especially on sanitation but this need to be scaled up and
should go to other sub counties
At the moment the district contracts repair this should be the HPMA
Meeting with DWO in Abim (28/1)
The development partners have 1 quarterly meeting with dev. partners on WASH
In 2015 no BH drilled > all funds to pipe scheme. There are 325 BH with 65 % functionality and 85 %
coverage. The district repaired 10 BH, but 46 HP are waiting for rehab. The main problem is the people
do not contribute and wait till the pumps breaks down
Most cases pipes are leaking although the water not really aggressive
Every district has technical supervision unit
HPMA now stores SP District does not have SP in stock and the costs of the transport is high
No change in water table salty water rare locally high Fe content
Goal is one of the prominent WASH partners
Goal trains HPM and district staff in CLTS