pcr evaluation note for public sector operations · 2019-05-02 · le rap confirme que l’objectif...

1. BASIC INFORMATION a. Basic project data Project title: PROJET D’APPUI A LA DECENTRALISATION DES SERVICES D’EAU POTABLE, HYGIENE ET ASSAINISSEMENT DE L’ATACORA-DONGA (LEauCAL) Project code: : P-BJ-EAZ-003

Instrument number(s):

Project type: Institutional development

Sector: Water, Sanitation and Hygiene (WASH)

Country: BENIN Environmental categorization (1-3) :Processing Milestones Key Events Disbursement and Closing

date Date approved:29/10/2012

Cancelled amount: 132 874 EUR Original disbursement deadline:

Date signed: 22/03/2013 Supplementary financing: Original closing date: 31/12/2016

Date of entry into force : 22/03/2013

Restructuring: Revised disbursement deadline:

Date effective for 1st disbursement: 08/07/2013

Extensions (specify dates): Revised closing date:28/02/2017

Date of actual 1st : 11/09/2013b. Financing sources

Financing source/instrument (MUA)

Approvedamount (MUA) :

Disbursed amount(MUA) :

Percentage disbursed(%):

Grant: AFW 1 042 000 936,516 90% Co-financier: PROTOS Benin/Alliance WASH 503 000 356,969 71%

Government: municipalities 243 000 231,814 95%

TOTAL : 1 788 000 1,525,299 85% Co-financiers and other external partners: - PROTOS (Belgian NGO with representation in BENIN) is the recipient of the AWF Grant Execution and implementation agencies:- Executing agency: PROTOS Benin with support from PROTOS Belgium- 13 municipalities (Communes) in the Prefectures od ATACORA & GONDAc. Responsible Bank staf

Position At approval At completionRegional Director JANVIER K LITSEHead of Country Office SERGE N'GUESSAN KHADIDIA DIABISector Director SERING JALLOW OSWARD M. CHANDASector Head AKISSA BAHRI JEAN MICHEL OSSETECoordinator of Activities DANIEL VERDEIL OUSSEYNOU GUENEAlternate Coordinator PCR Team Leader OUSSEYNOU GUENEPCR Team Members Peer Reviewers: Mahecor

PCR EVALUATION NOTE FOR PUBLIC SECTOR OPERATIONS

N’DIAYE; Bocar CISSE; Francis BOUGAIRE; Daniel VERDEIL

d. Report data PCR Date : November 2017 PCR Mission Date: From: 18/09/2017 To:20/09/2017 PCR-EN Date: Evaluator/consultant : Jean DOYEN Peer Reviewer/Task Manager:

2. PROJECT DESCRIPTIONSummary from Appraisal Report including addendum/corrigendum or loan agreement, and taking into account any modification that occurred during the implementation phase.

The LEauCAL Project is focused on building the capacity of 13 municipalities to take up the responsibility for water and sanitation facilities and services (WASH) under Benin’s decentralization policy.

a) Project description: The Project includes the following components:A. Institutional assessment and training:A.1. Institutional, organizational and financial diagnostic study with the following objectives: (i) toassess the technical and financial capacities of the 13 municipalities (hereafter called Communes) targeted by the project; (ii) to review options for pooling resources and capacity among Communes (mutualisation); (iii) to develop guidelines for equitable water tariff; and, (iv) to develop an action plan for the training of all actors involved in the WASH sector in the project Communes.A.2 Training to enable the Communes as well as private sector operator and artisans to fulfill theirroles in the organization and management of WASH facilities and services.

B. The development of the capacity to manage investments in water and sanitation through “learning by doing” involving the preparation and contracting of 9 projects: 6 dealing with water services and 3 with improved sanitation & hygiene.

C. Technical assistance to help the Communes in preparing project proposals, mobilizing funding, carrying out procurement and managing related contracts.

D. Overall project management involving project administration by PROTOS as co-financier and recipient of the AWF grant, acting under the oversight of the Steering Committees (Comité de Pilotage) advised the Technical Committee involving local branches of central ministries and local authorities.

b) Rationale and expected impacts:Provide a brief and precise description on the project/programme rationale (concerns/questions raised), expected impacts and the intended beneficiaries (directly or indirectly impacted by the project/programme). Highlight any change that occurred during the execution phase.

Under Benin’s decentralization the Communes (municipalities) carry front line responsibilities for water, sanitation and hygiene (WASH) within the framework of a coherent set of national strategies and policies guiding the organization and development of the WASH sector. The problem is that, as a rule, Communes are ill equipped to take up this mandate. The Project responds to this challenge through a comprehensive program to build the WASH related capacity of 13 Communes located in two districts (“départements”) among Benin’s least developed: Atacora and Donga.

The Project targets all sector actors and is based on learning by doing through preparation and implementation by the Communes of investments selected through calls for proposals. This approach introduces an element of emulation amongst Communes and builds their capacity to access funding from

the FADEC (Fonds d’Appui au Développement des Communes), Benin’s public funding facility for municipalities, and other donors. The Project helps Communes in overseeing O&M plans by operators and fostering cost recovery.

Project impacts are at two levels: (i) Increased access to safe water and basic sanitation. Direct beneficiaries are first and foremost the people served by the water and sanitation systems developed or rehabilitated under the Project. Other direct beneficiaries are the people who are benefitting from better services due to improved management of existing systems in the targeted Communes. The propagation across the sector of approaches and tools should yield indirect benefits in terms of improved and more sustainable services.

(ii) Upgraded abilities of sector actors to pursue the continuous development and sustainability of WASH systems and services for which the targeted Communes are direct beneficiaries. Potential indirect beneficiaries are the broad range of sector actors beyond the targeted Communes and districts that will be empowered by the approaches developed under the Project.

c) Objectives/Expected Outcomes:Provide a clear and concise description of the project objectives, expected outcomes, and intended beneficiaries. In so doing, highlight any revision/amendment.

The overall objective of the Project was defined as follows: “To ensure access to safe water and basic sanitation to rural communities by strengthening the capacity of the Communes to develop and manage water, sanitation and hygiene services and practices”. The Project narrowed this broad goal down to 13 Communes in 2 of Benin’s districts and broke it down into five specific objectives and interrelated outcomes covering the range of the capacities required to enable the Communes to take up their WASH mandate including:(1) Development of pooling of resources under Benin’s legal framework for “intercommunalité”; (2) Increase in the share of Communes’ budget allocated to WASH; (3) Improved O&M and cost recovery for water systems; (4) Functionality of water systems; and, (5) Increased support from decentralized services as provider of “Appui Conseil”

d) Outputs and intended beneficiaries:

Provide a clear and concise description the expected outputs and intended beneficiaries. In so doing, highlight any revision/amendment.

Outputs were defined in considerable details at the Appraisal stage and consistently reported in Quarterly Progress Reports. In total 21 outputs related to the four Project components (Ref. Section 2.a) were defined with numbers to be delivered/achieved. Output themes are summarized as follows: A.1 Study : Validation of the institutional /organizational study and training plan A2. Training : number of guides (existing & new) and training days delivered B Communes initiatives : number of Communes with : WASH action plans- projects executed – share of

WASH in in Investment Plans - M&E system – case of raised funding from FADEC and other donorsC & D Technical assistance and project management by PROTOS: number of: reports –budgets/work

plans/manuals – recruitment - knowledge documents.

Beneficiaries Direct beneficiaries- The population served by the 6 water projects carried estimated at 17,000 people of which about 9,000

women and girls (Component B)- The 13 Communes (municipalities) targeted by the Project which built up their capacity to carry out their

mandate for the development and management of WASH services (Components A1, A2 and B)- Private sector actors in charge of O&M: operators, artisans/repairers, stockiest, craftsmen, community

delegates in the active in the 13 targeted Communes Components A2 & B)

Indirect beneficiaries: (a) In the Project area - Districts administrations and the local branches of ministries dealing with water, sanitation and hygiene

Components A.1 and A.2).- The population of the two target districts served by existing systems that would benefit from improved

approaches to O&M and costs recovery developed under the Project (Component A.2) (b) beyond the Project area- The people that may benefit from the improved services as a result of the propagation of the guidelines

and practices developed and tested under the Project (Component A.2).- WASH sector actors (public, NGO, TFP, consultants) that may adopt the approaches and tools tested

under the Project (Component A.2).

e) Principal activities/Components:Provide a clear and concise description of the principal activities/components. In so doing, highlight any revision/amendment.

Most of the information on Project activities and component required in the context of the present PCR Evaluation is covered in the above paragraphs: a) Project description; b) Rationale and expected impacts; c) Objectives/Expected Outcomes; and, d) Outputs and intended beneficiaries.

3. PROJECT PERFORMANCE ASSESSMENT

RELEVANCE

a. Relevance of the project development objective: Evaluation of the relevance ex-ante and ex-post (including during the implementation phase). The relevance of the project objective (duringthe evaluation ex-ante and the post-evaluation) in terms of alignment with country’s development priorities and strategies, the beneficiary needs (including any changes that may have occurred during the implementation), applicable Bank sector strategies, the Bank country/ regional strategy, and general strategic priorities of the Bank. This criterion equally assesses the extent to which the project’s development objective was clearly stated and focused on outcomes and the realism of the intended outcomes in the project setting.

The Project overall objective was to ensure that rural communities have access to safe water and basic sanitation to by strengthening the capacity of Communes to develop and manage WASH services. The Project pursues this broad objective in the context of specific Communes through five inter-related and clearly defined outcomes as mentioned in Section 2.c). The Project is fully aligned with Benin’s broader development policies as well as WASH sector strategies which provide a coherent framework for the development of WASH services. It is alignedwith ADB’s sector strategies and fit within ADB regional initiative for rural water and sanitation.

Le RAP confirme que l’objectif de développement (OD) du projet est resté parfaitement en phase avec les priorités sectorielles du pays : (i) stratégie nationale pour l’eau potable en milieu rural (2005-2015) et en milieu urbain (2006-2015), une stratégie de Promotion de l’Hygiène et de l’Assainissement de Base (PHAB) ainsi qu’un plan stratégique pour l’assainissement urbain (2008-2015).

Contributions toward the Project objective beyond the targeted Communes and district would rest on the adoption of the practices and approaches developed under the Project across the WASH sector. The PCR points out that the concluding workshop called for wide dissemination and replication of the approach and tools pioneered under the Project. In Benin forum and annual reviews of the WASH sector are held regularly and would offer a platform for the propagation of such approach and tools.

The PCR points out that the Project should have been extended over a longer period, say five years. This is pertinent. The better option would be to extend and broaden the drive for building local capacity pioneered under the Project under a follow-up project. It could be possible that fact that the Project was proposed and executed by a NGO precluded consideration of a follow-up project. La Revue confirme comme très satisfaisante, la pertinence de l’objectif de développement du projet avec la note de 4 sur 4 conforme à celle du RAP conforme à la note de l’évaluateur (Réf. Section 8 Tableau «Summary of Evaluation»

Rap : 4 ; Evaluation 4 : Revue 4

b. Relevance of project design (from approval to completion):The evaluator should provide an assessment of the relevance of the project design regardless of the one provided in the PCR. The evaluator will also comment on the PCR conclusion for this section, and will provide an evaluation of the relevance of the project design. The latter assesses the soundness and the timing of eventual adjustments, or technical solutions to ensure the achievement of the intended results (outcomes and outputs), the adequacy of the risk assessment, environmental and social protection measures, as well as the implementation arrangements. For Programme Based Operations (PBO), an assessment will be made on the relevance of the prior actions, the policy dialogue and the extent to which the operation could have been more pro-poor in its design.

Project design was highly relevant and stayed so throughout implementation. The Project design and planning wasfounded on a well thought through logical framework and drew on the PROTOS country knowledge and its previous WASH sector experience in Benin and more broadly in Africa.

The specific objectives, outcomes & targets reflect a coherent understanding of the capacity gaps at the local level and a commitment to put the Communes in charge of their projects and to take support from the legal and institutional framework for decentralization including accessing funding from FADEC.

The Appraisal Report shows that the Project was prepared and planned in considerable details as shown in Section

2 above.

Changes and adjustments notably staff changes in the Project Execution Unit (Cellule d’Appui et de Coordination)did not affect implementation. ADB eventually allowed the use Benin’s national procurement system for procurements by the Communes. This was a positive step, enhancing ownership and replicability. Par ailleurs, la demande persistante des parties prenantes de poursuivre l’accompagnement des communes ciblées et d’élargir l’approche à toutes les communes nationales confirme la pertinence satisfaisante de la conception du projet. Cependant, la durée prévisionnelle d’exécution du projet s’est avérée insuffisante pour le suivi du volet exploitation et maintenance sur une période relativement convenable permettant de mieux apprécier la maîtrise d’ouvrage communale de ce maillon. Le rapport d’évaluation n’a pas prévu de financement spécifique pour les activités de suivi-évaluation du projet mais préconise un plan de suivi-évaluation du Projet qui sera mis en place par un consultant et s’intégrer dans le cadre du plan de suivi-évaluation des projets de PROTOS-Bénin. Les rapports de suivi sont adressés en premier lieu au Comité de Pilotage et partagés avec les bailleurs et avec les ministères impliqués. La revue est d’avis avec les résultats du RAP que ce dispositif de suivi-évaluation à travers PROTOS-Bénin avait bien fonctionné. Au vu de qui précède, la revue évalue que la pertinence de la conception du projet est satisfaisante et accorde une note de 3 sur 4 conforme à celle du RAP et à celle de l’évaluateur (Réf. Section 8 Tableau «Summary of Evaluation».

Rap : 3; Evaluation 3 : Revue 3

EFFECTIVENESS

c. Effectiveness in delivering outputs :Evaluation of the extent to which the project achieved its stated results (obtained from the logical framework) based on the last Implementation Progress and Results Report (IPR) and by considering accurate reporting of direct or indirect evidence on intended and unanticipated outputs. In the absence of sufficient data (as direct evidence), indirect evidence (such as project outcomes and other pertinent processes/elements of the causal chain) should be used particularly in the evaluation of the extent to which the project is expected to achieve its stated results/ objectives. The absence of sufficient data to assess the effectiveness should be indicated (and clearly detailed in the PCR quality evaluation section). The PCR score should equally be indicated in this section.

The PCR as well as available Quarterly Reports indicate that all the Outputs (ref. Section 2.d) were delivered at 100% with some exceeding set targets noticeably: (i) the number of training days for municipal services and private operators; (ii) the number of Communes that managed to increase their allocation from FADEC and/or from other donors; and, (iii) the number of knowledge products in the form of doctoral dissertations (3 instead of 2 planned). Au vue de ce qui précède et contrairement au RAP, la revue évalue satisfaisante l’efficacité dans la réalisation des effets (réalisations) avec la note de de 3 sur 4 et conforme à celle du RAP et à la note de l’évaluateur (Réf. Section 8 Tableau «Summary of Evaluation») au regard des insuffisances potentielles en matière de maîtrise d’ouvrage communale pour le volet exploitation et maintenance durable des ouvrages réaliséspar le projet.


d. Effectiveness in delivering outcomes:Evaluation of the extent to which the project achieved its intended set of outcomes (including for Program Based Operations (PBOs) where complementary measures are necessary for their implementation, namely public awareness, policy dialogue and institutional arrangements for instance). The evaluator should make an assessment based on the results of the last project Implementation Progress and Results (IPR). The evaluator shall indicate the degree to which project outcomes (intended and unanticipated) as well as reasons for any eventual gap werediscussed in the PCR.

Outcomes Results & Comments

(1) Development of pooling of resources under Benin’s legal framework for “intercommunalité

Achieved as planned in 6 Communes; 2 new intercommunal entities established.

(2) Increase in the share of Communes’ budget allocated to WASH

Almost doubled from 5 % to 9 %, still short of the 15 % target. The positive trend launched under the Project is significant.

(3) Improved O&M and cost recovery for water systems

Collection progressed from 35% to close to 50% short of the 90% target.Although full cost recovery should remain the ultimate goal, the intermediate target set under the Project appears unrealistic for small isolated water points.

(4) Functionality of water systemsProgressed from close to 200 days/year to the set target of 350 day/year.Would have been useful to indicate the basis for these values, e.g. surveys, reports from operators, etc.

(5) Increased support from decentralized services as providerof “Appui Conseil”

Fully realized for water services but much less so for sanitation.

The overall results are positive. The trends initiated over the Project period are likely to persist as upgraded capacities should support further progress which however would depend on continuing access to technical support. The shortfall against the high target set for O&M and cost recovery of water rates (outcome (3) may to a certain extend reflect the limited time to monitor progress after the introduction of O&M/cost recovery action plans. Le projet a été performant dans la réalisation des activités prévues. Cependant, l'auto-évaluation à mi-parcours a été réalisée au cours du 1er trimestre de 2016 soit à quelques (6 mois) de la clôture du projet. Au vu de qui précède, la revue et l’évaluateur confirment très satisfaisante l’efficacité dans la production des extrants du RAP avec la note maximale de 4 sur 4.


e. Project development outcome: The ratings derived for outcomes and output are combined to assess the progress the project has made towards realizing its development objectives, based on the rating methodology recommended in the Staff Guidance Note on project completion reporting and rating (see IPR Guidance Note for further instruction on development objective rating).

Section 3.d) and 3.c) provide an overview of progress achieved toward the Project development objective pointingout significant positive trends on all outcomes especially those related to the priority given to WASH by the communes and their capacity to manage projects and mobilize funding. The institutional approach deployed underthe Projects have been documented and are ready to be duplicated across Benin and beyond. Noted the role of the Project in helping Communes to mobilize funding from the FADEC and from programs active in the targeted district notably from Alliance WASH.Le RAP relève que le projet a permis aux 13 Communes des 2 départements de mettre en pratique leur compétence de maitrise d’ouvrage du secteur AEPHA qui leur est conférée dans le cadre de la décentralisation. Il a permis de renforcer leurs capacités en maîtrise d’ouvrage, en mobilisation de ressources financière, de mettre en place des mécanismes et d’outils de gestion locale de l’AEPHA. Le taux d’accès à l’eau potable dans la zone, initialement de 63% a été porté à 65%. Cet objectif pourrait atteindre les 68% à l’horizon 2020 si le gouvernementdu pays continue à s’investir dans le secteur de l’Eau et Assainissement.Au regard des effets et des extrants, l’atteinte des résultats de développement obtenus par le projet par rapport à ses objectifs est, à ce stade de la revue jugée satisfaisante avec la note de 3 sur 4 conformément à l’évaluation proposée par le RAP et endossée par l’évaluateur.


f. Beneficiaries: Using evidence, the evaluator should provide an assessment of the relevance of the total number of beneficiaries by categories and disaggregated by sex.

Direct beneficiaries- The number of people who gained improved access to safe water is estimated at 17,000 (including 9,000 women

and girls: soit 53%) thereby lifting up access in the two target districts by 2 % from 63% to 65% in 2016. Reaching the Appraisal target of 68 % by 2020 (i.e. 40,000 additional people served) would depend on further initiatives by the Communes. There are no comparable data related to the sanitation developed under the Project.

- The 13 targeted Communes are counted as direct beneficiaries of the drive to upgrade their capacity across all aspects of their WASH mandate. This extends to some degree to the local branches of central ministries of water, sanitation and health as well to districts administrations as they benefitted from training and were involved in the Technical and Steering Committees.

- Private sector actors active in the target Communes benefitted directly from the Project through training and technical assistance. Related training days exceeded initial plans and all categories of actors participated.

Indirect beneficiaries- Indirect benefits in the form of improved services should accrue throughout the target districts through the

dissemination implementation of better O&M and cost recovery practices. Number of people served and access rate have the targeted districts have not been evaluated. Such evaluation would at any case capture the combined impact all initiatives targeting the WASH sector without singling out what could be attributed to the Project.

- Over time populations and sector actors beyond the target districts may also benefit indirectly through the propagation of propagation of the guidelines and practices developed and tested under the Project These guidelines and practices were reviewed in stakeholder workshops, endorsed by local and central authorities and disseminated at sector forum and annual reviews where they were reportedly well received. It is too soon, and at any case difficult, to gauge their degree of penetration and their impact. The practical implementation of the learning by doing approach is of particular interest in this regard. La Revue juge donc que les bénéficiaires ont bien été atteints et que les résultats du projet dans ce domaines sont globalement satisfaisants (note 3). Au vu de qui précède, la revue considère la couverture des bénéficiaires du projet satisfaisante avec la note maximale de 3 sur 4 conforme à la note proposée par l’évaluateur.


g. Unanticipated additional outcomes (positive or negative, not taken into consideration in the project logical framework) : This includes gender, climate change, as well as social and socio-economic- related issues. Provide an assessment of the extent to which intended or unanticipated additional and important outcomes have been taken into consideration by the PCR. The assessment should also look at the manner the PCR accounted for these outcomes.

The above comments on indirect beneficiaries points at positive additional outcomes. Participatory approaches and social intermediation were used systematically for the identification and of water and sanitation/hygiene action plans and projects in line with Benin and ADB sector policies which incorporate gender sensitive principles.None reported in the PCR: Pas de données et informations disponibles dans le RAP

EFFICIENCY

h. Timeliness:The timeliness of project implementation is based on a comparison between the planned and actual period of implementation from the date of effectiveness for first disbursement. For Programme Based Operations (PBOs), the timely release of the tranche(s) are assessed through this same criterion.

The Project unfolded essentially as conceived and planned.

- WASH investments by the Communes were delayed by almost one year due to slow uptake by the Communes who were in charge of the identification/preparation phase. The delay was compounded by the slowdown resulting from local elections that took place in mid-2015 and brought in new cohort key stakeholders: mayors and local reps which had to be briefed and taken on board. The projects were eventually completed before closing but this left no time to follow-through the consolidation of the O&M action plans

- The introduction of M&E systems at the Communes also level incurred delays related to the local elections.

La revue relève que le projet a été exécuté dans les délais prévisionnels de 45 mois mais jugés assez courts pour permettre la maîtrise d’ouvrage communale complète pour le volet exploitation et maintenance durable des ouvrages réalisés par le projet. Prévu pour 40 mois (rapport d’évaluation), le projet s’est exécutésur un délai de 45 mois (selon le RAP). Le rapport entre le délai prévisionnel (selon le REP) et le délai réel d’exécution à compter de la date de satisfaction des conditions du premier décaissement est de 0,888, et de ce fait jugé satisfaisant car supérieur à 0,75 et inférieur à 1. La revue juge satisfaisant le respect du calendrier d’exécution avec une note de 3 sur 4 contrairement à la note très satisfaisante du RAP de 4 sur 4 qui est également celle proposée par l’évaluateur considérant que le glissement dans le calendrier d’exécution est du d’une part aux changements de responsables qui ont suivi les élections et à la volonté de s’adapter aux réponses des communes qui est un des principes clé pour l’appropriation.


i. Resource use efficiency Provide and assessment of physical implementation (based on outputs delivered) against resources used (based on cumulative commitments) at completion for all contributors to the project (the Bank, Government, and others). This criterion would normally not applyto PBOs, as there is often no direct link between the outputs and the amount of contribution (in which case the rater would indicate N/A).Le coût total du projet était de 32,83 MUC, financé conjointement par le prêt FAE/BAD pour 1,042MUC, le PROTOS Bénin pour 0,503 MUC, le Gouvernement/Communes bénéficiaires pour 0,243 MUC. Ces financementsont été décaissés respectivement à 90%, 71% et 95%. Le taux de réalisation des produits du projet à l’achèvement était de 100% et reflète bien l’état d’exécution très satisfaisant des activités du projet. Le taux d’engagement des ressources du projet était de 86%. La valeur médiane du taux de réalisation physique des produits du projet par rapport au taux d’engagement est de 1,162, et de ce fait jugée très satisfaisante car supérieur à 1. Ce qui indique une utilisation très efficiente des ressources financières du projet et l’atteinte de tous ses produits escomptés dans les limites du budget disponible malgré la faible mobilisation de la contribution de PROTOS-Bénin. The fact that all planned outputs were delivered at total costs amounting to 86 % of initial estimates suggests high overall efficiency. The fact that services, supplies and works were procured through open competitive bidding offer some guarantee that value-for-money was obtained. The Project supported the development or rehabilitationof 67 water points at an average cost of less than EUROs 10,000 per water points which compared with regional benchmarks appears on the low side although the PCR does provide details on the scope of these works. Pour l’évaluateur et la revue tout comme pour le RAP, la performance en matière d’efficience dans l’utilisation des ressources du projet est jugée très satisfaisante avec la note de 4 sur 4.


j. Cost-benefit analysis:Provide an assessment of the timeliness of the development outputs, and the extent to which costs of the costs have been effective and have been provided in the most efficient manner. The PCR rating should be discussed. The evaluator should verify whether the benefits of the project (achieved or expected) exceed its actual costs. To achieve this, evidences will mainly be based on a comparison between Economic Rates of Return (ERR) calculated at appraisal, the mid-term review and completion. When commenting PCR ratings, the degree of utilization of valid sources for evidence justifying the rating assigned should be taken into consideration. The evaluator should ensure of thevalidity of assumptions and that the same model was used for the calculation of others ERRs. For PBOs for which this calculation model does not apply, an assessment could be done with regards to the contribution of policy reforms to economic growth. In the absence of sufficient evidence, an appropriate rating should be assigned.

No cost-benefit analysis was done at appraisal which is normal for institutional development projects.

k. Implementation progress:The assessment of the Implementation Progress (IP) on the PCR is derived from the updated IPR and takes into account the all applicable IP criteria assessed under the three categories : i) Compliance with covenants (project covenants, environmental and social safeguards and audit compliance), ii) project systems and procedures (procurement, financial management and monitoring and evaluation), and iii) project execution and financing (disbursement, budget commitments, counterpart funding and co-financing).

The Project was essentially fully implemented as planned. Audit recommendations were implemented although on one instance with some delay. Procurement and financial management were adhered to and strictly monitored. Co-financing by PROTOS and counterpart funding by the Communes were mobilizedas scheduled. Le RAP juge l’état d’exécution globale du Projet très satisfaisant. Cependant, la revue relève que la contribution de PROTOS-Bénin a été mobilisée seulement à hauteur de 71% et celles des communes bénéficiaires à 95%. Ce qui est considéré comme insuffisant. La revue juge satisfaisant l’état d’avancement de l’exécution du projet avec une note de 3 sur 4, contrairement à celle du RAP. L’évaluateur accepte la révision de la note de 4 à 3 au vu des points relevés par la revue.


SUSTAINABILITY

l. Financial sustainability:Provide an assessment of the extent to which funding mechanisms and modalities (e.g. Tariffs, user fees, maintenance fees, budgetary allocations, other stakeholder contributions, aid flows, etc.) have been put in place to ensure the continued flow of benefits after completion, with particular emphasis on financial sustainability. For PBOs, the assessment should focus on financial sustainability of reforms, as well as the Bank’s policy dialogue to promote financial sustainability of the reforms. In designing water systems one would normally seek to check the viability of water system at the design stage by trying to match O&M costs with what households are ready to pay. One can reasonably assume that this was the case for systems funded under the Project. It is clear however that operational and financial viability of small isolated water systems serving small impoverished communities remains a challenge across Africa. Delegation to private and the pooling of resource in inter-communal entities, as promoted by the Project, offers ways to strengthen accountability and to assemble the needed critical mass of technical & managerial talents to support local systems (Appui Conseil). La revue en accord avec l’évaluateur confirme la note satisfaisante de 3 sur 4 donnée par le RAP.


m. Institutional sustainability and strengthening of capacities:Provide an assessment of the extent to which the project has contributed to the strengthening of institutional capacities – including for instance through the use of country systems – that will continue to facilitate the continued flow of benefits associated with the project. An appreciation should be made with regards to whether or not improved governance practices or improved skills, procedures, incentives, structures, or institutional mechanisms came into effect as a result of the operation. For PBOs, this should include an assessment on the contributions made to building the capacity to lead and manage the policy reform process; the extent to which the political economy of decision making was conducive to reform; the Government’s commitment to reform; and how the design reinforced national ownership.

Several factors suggest that the Project institutional outcomes are likely to be sustained. The first of these factor is the fact that the Project approach to building capacity at the Communes level was comprehensive, i.e. covering allcritical aspects: O&M action plan and M&E, project planning, procurement, supervision and monitoring, as well as resource mobilization and formation of inter-municipal framework. Other sector actors were also targeted notably private and local branches of ministries. “Learning by doing” also fosters sustainability. Finally Benin’s supportive and stable policy context guiding decentralization and the WASH sector also contribute to sustainability.The closure of the Project without any apparent follow-up is likely to leave a gap in access to technical support needed to keep the momentum created by the Project. As a principle institutional strengthening should be seen as a continuing process to be supported through successive programs/projects. La revue en accord avec l’évaluateur confirme la note satisfaisante de 3 sur 4 donnée par le RAP.


n. Ownership and sustainability of partnerships:Provide an assessment of whether the project has effectively involved relevant stakeholders, promoted a sense of ownership amongst the beneficiaries (both men and women) and put in place effective partnerships with relevant stakeholders (e.g. local authorities, civil society organizations, private sector, donors) as required for the continued maintenance of the project outputs. For PBOs, the assessment should measure the extent to which the Government’s capacity to conduct consultations during policy dialogue and the extent to which the Bank supported the Government in deepening the consultation processes.

Steps taken along the implementation reflect the attention given to stakeholder involvement notably through socialintermediation and community participation and systematic dissemination and vetting of guidelines. The drive to formalize partnerships and contracts for delegated management water service, for involvement of consumer associations and for intercommunal structure should also entrench ownership. Ownership is also reflected in the fact that the Communes participated up to 25% in the financing of their projects

which is much more what is typically requested from small rural communities. La revue en accord avec l’évaluateur confirme la note satisfaisante de 3 sur 4 donnée par le RAP.


o. Environmental and social sustainability:Provide an assessment of the objectivity of the PCR rating on the project’s implementation of environmental and social mitigation/enhancement measures with regard to the Environmental and Social Management Plan (ESMP), the capacity of country institutions and systems, as well as the availability of funding to ensure the environmental and social sustainability of the operation. This criterion would normally only apply to Environmental Category I and II projects.

No comments in the PCR. One can assume that the small water supply projects and the local sanitation action plans developed under the Projects followed applicable national and ADB guidelines. The Project was rightly classified as Category 1 with respect to environmental safeguards considering the limited and essentially positive environmental impact of small communal WASH projects in terms of protection of water sources and cleanliness of the centers of small communes. La revue en accord avec l’évaluateur confirme la note satisfaisante de 3 sur 4 donnée par le RAP.


4. PERFORMANCE OF STAKEHOLDERS

a. Bank performance:(Preparation/approval, ensure of Quality at Entry (QAE) : quality of the supervision, completion) : Provide observations on the objectivity of the PCR ratings and feedback provided by the Borrower, and if necessary, re-assess the Bank’s performance throughout the project cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to 7 criteria defined in the PCR Guidance Note.

Available supervision missions Aide-Memoires and BTOR suggest that the ADB followed closely implementation, tracking development outputs, making useful recommendations with detailed attention to procurement and financial management leading at some point to suspension of disbursements with an eventual resolution. Attendant delays have reportedly hampered rolling out of the M&E component.

La documentation disponible sur le projet montre à suffisance la bonne implication de la Banque dans la

conception, le suivi et la mise en œuvre des activités du projet. Dans l’ensemble, la revue en accord avec l’évaluateur confirme la performance satisfaisante de la Banque avec la note 3 sur 4, conforme à celle duRAP qui est assez objective.

Rap : 3; Evaluation 3 : Revue 3b. Borrower performance: Provide observations on the objectivity of the PCR ratings, and if necessary, re-assess the Borrower’s performance throughout the project

cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to questions defined in the PCR Guidance Note.

PROTOS conceived the Project drawing on their previous WASH experience in Benin. Once they secured the AWF grant to supplement their own financing they were quick to mobilize and got the Project to an early start. PROTOS’ performance as executing agency and recipient of the AWF grant was on the whole very effective. The Project benefitted from the administrative and technical support from PROTOS office in Benin as well as from specialized technical and advice from its HQ in Belgium. Available Quarterly Progress Reports review progress and plans output-by-output account as well as financial commitments and projections.

The nature of the unimplemented recommendation of the midterm Review that led to the temporary suspension of disbursement is not indicated in the PCR, nor could it be found in available documents except for the fact it was

lifted per recommendations of the January 2016 SPN mission. Dans l’ensemble, la revue en accord avec l’évaluateur confirme la performance satisfaisante de l’Emprunteur avec la note 3 sur 4, conforme à celledu RAP qui est également objective.


c. Performance of other stakeholders: Provide observations on the objectivity of the PCR ratings, and if necessary, re-assess the other shareholders’ performance throughout the project cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to relevant questions specific to each stakeholder (co-financiers, NGO, contractors and service providers).

On the whole the Communes engaged in the execution of the Project performed well, one could add, surprisingly so given their lack of experience and the administrative constraints under which they operate. They took initiativesin responding to calls for projects, albeit with some delays, and were successful in mobilizing financing. The element of emulation amongst Communes is mentioned as a factor fostering their engagement. They submitted

willingly to PROTOS supervision of procurement. Dans l’ensemble, la revue en accord avec l’évaluateur confirme la performance satisfaisante des parties prenantes avec la note 3 sur 4, conforme à celle du RAP qui est suffisamment objective.


5. SUMMARY OF OVERALL PROJECT PERFORMANCE

a. Overall assessment: Provide a summary of the project/programme’s overall performance based on the PCR 4 key components (Relevance, Effectiveness, Efficiency and Sustainability). Any difference with the PCR and the reasons that have resulted in them should be mentioned. For cases with insufficient evidence (from the PCR and other documents) available, the evaluator should assign a partly satisfactory rating (to be revised) until a post project performance evaluation (e.g. PPER, PER or PRA) is complete.

globale satisfaisante La revue confirme une évaluation de la performance du projet avec la note de 3 (3,5 pour la pertinence, 3 pour l’efficacité, 3 pour l’efficience et 3 pour la durabilité) confirmant la note du RAP (3,5 pour la pertinence, 3 pour l’efficacité, 4 pour l’efficience et 3 pour la durabilité).

The Project was conceived and implemented as a targeted, coherent and effective response to a critical capacity gap hampering the implementation of Benin’s decentralization across the WASH sector. The underlying concept was of helping the Communes to help themselves. In doing so the Project focused primarily on building the capacity of the actors involved in the WASH sector, starting with the Communes and including private sector actors as well as local branches of central administrations. Given its limited scope and duration the Project stands out as a pilot effort holding the potential to impact Benin’s

WASH sector as a whole through the propagation and adaptation of the approaches and practices pioneered under

the Project.’ Dans l’ensemble, la revue en accord avec l’évaluateur confirme la performance globale du projet jugée très satisfaisante par le RAP avec la note de 3,5 sur 4.

RAP : 3,5 Evaluation 3,5 : Revue 3,5

b. Design, implementation and utilization of the M&E (appreciation of the evaluator):Provide an assessment of planned and actual cost of the design, implementation and utilization of the M&E system. Design : To which extent the project M&E system was explicit, adequate and realistic to generate and analyse relevant data ; Implementation : To which extentrelevant data was collected – Elements of M&E implementation and effectiveness in the PCR ; Utilization : degree of utilization of data generated for decision-making and resource allocation – elements of M&E utilization in the PCR.

Au niveau de la conception, le dispositif prévu à l’évaluation était conçu pour être mis en œuvre dans le cadre de PROTOS-Bénin. La conception du projet préconisait un plan de suivi-évaluation du Projet qui serait mis en place par un consultant et s’intégrerait dans le cadre du plan de suivi-évaluation des projets de PROTOS-Bénin. Les rapports de suivi seront adressés en premier lieu au Comité de Pilotage et partagés avec les bailleurs et avec les ministères impliqués. La revue est d’avis avec les résultats du RAP que ce dispositif de suivi-évaluation àtravers PROTOS-Bénin avait bien fonctionné. Au vu de ce qui précède, la conception du système de suivi-évaluation est jugée satisfaisante par la revue avec la note 3 sur 4

Quant à la mise en œuvre, le RAP relève que le dispositif suivi-évaluation du projet conçu par la Banque pour ce projet s’est avéré moins efficace et de ce fait jugé insatisfaisant avec la note 2 sur 4.

Project: M&E and reporting related to Project was timely, detailed and consistent focusing on number of products delivered/achieved and on engagements. It would also been useful to provide information and reflections on substantive technical aspects notably on the nature of investments in water systems, on the issues encountered in setting up intercommunal structures, and on the themes and key points arising from the knowledge products.Sector: Benin has a set of country wide general WASH sector indicators. Project effort to enhance and strengthen M&E at the level of the Communes came in rather late and was subject to several revisions making it difficult to

evaluate what was achieved. Dans l’ensemble, la performance du système de suivi-évaluation est jugée globalement insatisfaisante avec la note 2 sur 4.

RAP : 2 ; Evaluation 2 : Revue 2

6. EVALUATION OF KEY LESSONS LEARNED AND RECOMMENDATIONS

a. Lessons learned:

Provide a brief description of any agreement/disagreement with all or part of the lessons learned from the PCR after analysis of the project performance with regards to each of the key components of the evaluation (Relevance, Effectiveness, Efficiency, and Sustainability). List the PCR main new and/or reformulated pertinent (and generic) lessons learned for each of these components here. It is recommended that no more than five lessons learned are discussed. Key questions and targeted audience must also be specified for each lesson learned.

Lessons learned PCR Evaluator comments

Project design & Effectiveness 1. Capacity building program should be based on participatory institutional assessment/study

Pertinent. The fact that the study was carried out right after effectiveness allowed launching its implementation immediately without the hiatus which often occurs when studies precede appraisal”

2. Capacity building approaches based on participatory approaches and learning by doing are more effective although they may require more time upfront.

Pertinent. Requires commitment, flexibility, hand holding and technical support from project management

3. Need to increase cost recovery and to streamline public finance disbursement to overcome constraints encounteredin disbursement of public funds for O&M of water/sanitation facilities.

Pertinent but inherently difficult to implement. Ref recommendation 1 below.

4. Approaches comprising an element of emulation between Communes through calls for projects foster engagement and ownership

Pertinent. The Project showed practical ways to introduce emulation in the contextof a specific project.

Sustainability5. Delegating the operation of water systems to independent service providers is a way to ensure timely O&M and repairs and foster cost recovery

Pertinent and consistent with regional bestpractices.

6. Inter-municipal arrangements are an effective way to facilitate the pooling of resources and to ensure gradual building of procurement capacity-.

Pertinent. Requires supportive legal/policy framework and clustering of municipalities to assemble critical mass to consolidate competences and pool resources

b. Recommendations:

Provide a brief description of any agreement/ disagreement with all or part of the recommendations from the PCR. List the PCR main new and/or reformulated recommendations (requiring more actions by the Borrower and/or the Bank) here.

PCR Recommendations Evaluator comments

1. To institute a water sector budget separate from the general budget in order to facilitate funding and disbursement of municipal allocations from the public treasury.

Pertinent but difficult to pursue in the context of sector projects. Practical options: (i) To facilitate the channeling of budget funds through sector-wide or budget management operations; or, (ii) To establish dedicated funding mechanisms for water (examples: Kenya, Uganda

2. To carry out M&E over a period of at least 2 years to assess project achievements and facilitate propagation nationwide

Pertinent but equally important to plan for follow-upprogram(s) to consolidate and propagate approaches and lessons learned recognizing that capacity building should be seen as a continuing process.

7. COMMENTS ON PCR QUALITY AND TIMELINESSThe overall PCR rating is based on all or part of the criteria presented in the annex and other: The quality of the PCR is rated as highly satisfactory (4), satisfactory (3), unsatisfactory (2), and highly unsatisfactory (1). The timeliness of the PCR is rated as on time (4) or late (1). The participation of the Borrower, co-financier, and the bank’s external office(s) are rated as follows: Very Good (4), Good (3), Fair (2),Poor (1).

Criteria Rating1) Quality: highly satisfactory. Compliant with the guidelines. Coherent and logical. Some information lacking. IPR not attached.

3

2) Timeliness: on time. Closing date: end February 2017; PCR mission September 2017. PCR issued in December 2017. PVR issued more than 6 months after closing

1

3) Participation of borrower/co-financier & ADB external office: good for PROTOS (borrower/co-financier) and partners notably through participation in the closing workshop.

3

Overall rating of the PCR: 1-1,49 (1-Très insatisfaisant), 1,50-2,49 (2-Insatisfaisant), 2,50-3,49 (3-Satisfaisant), 3,50-4 (4-Très satisfaisant

2

8. SUMMARY OF THE EVALUATIONThis is a summary of both the PCR and IDEV ratings with justification for deviations/comments. Appropriate section of the PCR Evaluation should be indicated in the last column in order to avoid detailed comments. The evaluator must provide a reasonable explanation for each criterion the PCR rating is not validated by IDEV. Consequently, the overall rating of the project could be “equally satisfactory”.

Criteria PCRPCRE

NReason for disagreement/ Comments

RELEVANCE 3,5 3,5 HIGHLY SATISFACTORYRelevance of project development objective 4 4

Focus on building capacity of front line actors for WASH development and services

Relevance of project design 3 3

Could be higher. Project Design was highly relevant and innovative. Given the fact that the project essentially a pilot it would have been useful at mid-term to discuss with AfDB plans for a follow up.

EFFECTIVENESS 3 3 SATISFACTORY

Development objective (DO) 3 3Progress on cost recovery significant but slower than called for by relatively high targets.

EFFICIENCY 4 3 SATISFACTORYTimeliness 4 3/4 Project executed within set period.

Planning s adjusted to accommodate slow take up by Communes and local elections. Prévu pour 40 mois (rapport d’évaluation), le projet s’est exécuté sur un délai de 45 mois (selon le RAP). Le rapport entre le délai

prévisionnel (selon le REP) et le délai réel d’exécution à compter de la date de satisfaction des conditions du premier décaissement est de 0,888, et de ce fait jugé satisfaisant car supérieur à 0,75 et inférieur à 1. Les glissements ont été dus aux changements de responsables communaux et à la volonté de s’adapter au rythme de communes.

Resource use efficiency 4 4 Project delivered slightly below budget.

Cost-benefit analysis N.A. N.A.N.A (not applicable) as mentioned in the PCR, would be appropriate.

Implementation progress (IP) 4 3Co-funding by PROTOS and the Communes mobilized as scheduled. Mais la contribution de PROTOS-Bénin est jugée insuffisante

SUSTAINABILITY 3 3 SATISFACTORY

Financial sustainability 3 3Positive trends in O&M and cost recovery initiated but remains a challenge.

Institutional sustainability and strengthening of capacities 3 3

Not clear how continuing need for technical support (Appui Conseil) would be met in the future

Environmental and social sustainability

3 3Positive due to social intermediation and oversight of water systems by user associations

OVERALL PROJECTCOMPLETION RATING

3,3 3,1SATISFACTORY

Bank performance: 3 3Detailed reviews of delivery and of adherence to financial management guidelines

Borrower performance: 3 3Remarkable given the number of outputs and the partnerships with 13 Communes

Performance of other shareholders: 3 3The good performance of the Communes is noticeable

Overall PCR quality: 3 Could be 4 if produced within 6 month after closing.

9. PRIORITY FOR FUTURE EVALUATIVE WORK: PROJECT FOR PERFORMANCE EVALUTION REPORT, IMPACT EVALUTION, COUNTRY/SECTOR REVIEWS OR THEMATIC EVALUATION STUDIES:

- Project is part of a series and suitable for cluster evaluation

- Project is a success story

- High priority for impact evaluation

- Performance evaluation is required to sector/country review

- High priority for thematic or special evaluation studies (Country)

- PPER is required because of incomplete validation rating

Major areas of focus for future evaluation work:

a) Performance evaluation is required for sector/ country review

b) Cluster evaluation (institutional support)

c) Sector evaluation (budgetary support or public finance management reforms)

Follow up action by IDEV: Identify same cluster or sector operations; organize appropriate work or consultation mission to facilitate a), b) and/or c).

Division Manager clearance Director signing off

Data source for validation: Task Manager/ Responsible bank staff interviewed/contacted (in person, by telephone or

email) Documents/ Database reports

Attachment:

PCR evaluation note validation sheet of performance ratings

List of references

Appendice 1

PROJECT COMPLETION REPORT EVALUATION NOTE Validation of PCR performance ratings

PCR rating scale:

Score Description4 Very Good – Fully achieved with no shortcomings3 Good – Mostly achieved despite a few shortcomings2 Fair – Partially achieved. Shortcomings and achievements are roughly balanced1 Poor – very limited achievement with extensive shortcomings

UTS Unable to score/rateNA Non Applicable

Criteria Sub-criteriaPCRworkscore

IDEVreview

Reasons for deviation/comments

RELEVANCE Relevance of the projectdevelopment objective (DO) during implementation

4 4Focus on building capacity of front line actors for WASH development and services

Relevance of project design (from approval tocompletion)

3 3

Could be higher. Project Design was highly relevant and innovative. Would have been useful to review the need for follow-up to consolidate the results and propagate the approach pioneered under the Project to other districts

OVERALL RELEVANCE SCORE 3,50 3,5 TRES SATISFAISANTE

EFFECTIVENESS* Effectiveness in delivering outcomes

Outcome1 Capacity of target Communes strengthened

3 3Communes’ funding and execution of “their” projects and increase in WASH allocations illustrate progress

Outcome2 O&M of water systems improved

3 3Progress on cost recovery significant but slower thancalled for by high target.

Moyenne 3 3 SATISFAISANTE

Effectiveness in delivering output

Output1Training and technical assistance delivered

4

4Comprehensive outputs for capacity building delivered

Output2 Communes projects planned, fundedand executed

3 6 water projects selected by Communes and 1 sanitation project planned

Moyenne 4 3,50 TRES SATISFAISANTE

Development objective (DO)

Development objective 3 3 SATISFAISANTE


IDEVreview


rating

Beneficiaries

Beneficiary1 The 13 target communes

100% 4 100 % of outputs delivered

Beneficiaries 2 WASH sectors actors Artisans/repairers, farmers, craftsmen, community delegates

100% 4 100 % of outputs delivered

Beneficiary 3 The population served by thewater projects (17,000 including 9,000 women & girls)

85% 3 O&M progress less than projected

NOTE GLOBALE DU DEGRE DE COUVERTURE DES BENEFICIAIRES

3 3The review rated this criterion related to beneficiaries’ coverage with satisfactory (3).

Unanticipated outcomes (positive or negative not considered in the project logical framework) and their level of impact on the project (high, moderate, low)

Institutional development 4

Approach and practices tested under the Project offerguidance for similar efforts country-wide

Gender 3All training, guidelines aligned on good gender practices for WASH. Women systematically represented water points committees

Environment & climate change 3

Positive local impact due to Hygiene and Sanitation Action Plans developed by target Communes and protection of water points

Poverty reduction 3Lack of access to safe water and basic sanitation is a critical dimension of poverty in Benin rural areas

Private sector development 3

Facilitated through use of local consultants, artisans, stockiest and contractors

Regional integration Not relevant

Other (specify)

Moyenne n/a 3 SATISFAISANTE

EFFECTIVENESS OVERALL SCORE 3 3 SATISFAISANTE

EFFICIENCY Timeliness (based on theinitial closing date)

4 3 Satisfaisante. Project executed within set period. Prévu pour 40 mois (rapport d’évaluation), le projet s’est exécuté sur un délai de 45 mois (selon le RAP). Le rapport entre le délai prévisionnel (selon le REP) et le délai réel d’exécution à compter de la date de satisfaction des conditions du premier décaissement est de 0,888, et de ce fait jugé satisfaisant car supérieur à 0,75 et inférieur à 1.


IDEVreview


Resource used efficiency 4 4

Project delivered slightly below budget. Le taux de réalisation des produits du projet à l’achèvement était de 100% et celui d’engagement des ressources du projet était de 86%. La valeur médiane du taux deréalisation physique des produits du projet par rapport au taux d’engagement est de 1,162, et de ce fait jugée très satisfaisante car supérieur à 1. Ce quiindique une utilisation très efficiente des ressources financières du projet et l’atteinte de tous ses produits escomptés dans les limites du budget disponible malgré la faible mobilisation de la contribution de PROTOS-Bénin

Cost-benefit analysis N.A. N.A Not applicable

Implementation progress (from the IPR) 4 3

Co-funding by PROTOS and the Communes mobilized as scheduled. Mais la contribution de PROTOS-Bénin est jugée insuffisante.

Other (specify) 4 - Pas mentionné dans le RAP

OVERALL EFFICIENCY SCORE 4 3 SATISFAISANTE

SUSTAINABILITYFinancial sustainability 3 3 Positive trends in O&M and cost recovery initiated

but remains a challenge Institutional sustainability and strengthening of capacities

3 3Only general indications on how continuing need for technical support (Appui Conseil) would be met in the future

Ownership and sustainability of partnerships

3 3

Local Ownership at the level of the Communes and local communities built in the design notably through the call for projects. Local partnerships with artisans, stockiest etc. strengthened

Environmental and social sustainability N/A 3

Social sustainability positive due to social intermediation and oversight of water systems by user associations

NOTE GLOBALE DE DURABILITE3 3

Satisfaisant

*The rating of the effectiveness component is obtained from the development objective (DO) rating in the latest IPR of the project (see Guidance Note on the IPR). The ratings for outputs and outcomes are determined based on the project’s progress towards realizing its targets, and the overall development objective of the project (DO) is obtained by combining the ratings obtained for outputs and outcomes following the method defined in the IPR Guidance Note. The following method is applied: Highly satisfactory (4), Satisfactory (3), Unsatisfactory (2) and Highly unsatisfactory (1).

Criteria Sub-criteriaPCRWorkscore

IDEVreview


BANK PERFORMANCE

Proactive identification and resolution of problems at differentstage of the project cycle

33 Systematic and detailed

Use of previous lessons learned from previous operations during design and implementation

3 Through Project participation in regional and national WASH sector reviews and consultations forum

Promotion of stakeholder participation to strengthen ownership

3 Built in Projet design

Enforcement of safeguard and fiduciary requirements

3Focused on fiduciary requirements: audits and accounting framework

Design and implementation of Monitoring & Evaluation system

3Focused on systematic monitoring of outputs

Quality of Bank supervision (mix of skills in supervisory teams, etc.)

3 Adequate

Timeliness of responses to requests

3Limited cases of tardiness in responding toN.-O. requests

OVERALL BANK PERFORMANCE SCORE 3 SATISFAISANTE

BORROWER PERFORMANCE

Quality of preparation and implementation

3

4Project outputs and activities as well as implementation timetable were prepared indetails allowing early start

Compliance with covenants, agreements and safeguards

3 Adequate

Provision of timely counterpart funding

3Tripartite arrangements for Project financing worked well

Responsiveness to supervision recommendations

3

Some delays in implementing recommendations of mid-term review concerning for Project Implementation Unit (CAC) separately from PROTOS overall accounts.

Measures taken to establish basis for project sustainability

3More could have been done considering the continuing needs for technical support

Timeliness of preparing requests 4OVERALL BORROWER PERFORMANCE SCORE 3 3 SATISFAISANTE

PERFORMANCE OF OTHER STAKEHOLDERS

Timeliness of disbursements by co-financiers

3

3Communes succeeded in mobilizing their set share of funding

Functioning of collaborative agreements 3

The Communes submitted their procurement to N.-O. by PROTOS (grantee and executing agency)

Quality of policy dialogue with co-financiers (for PBOs only)Quality of work by service providers 3

Generally good with one reported instance of procurement issue related to stock of parts in one Commune

Responsiveness to client demands 4 The Communes had to apply for projectsOVERALL PERFORMANCE OF OTHER STAKEHOLDERS

3 3 SATISFAISANTE

The overall rating is given: Very Good, Good, Fair and Poor.

(i) Very Good (HS) : 4(ii) Good (H) : 3(iii) Fair (US) : 2(iv) Poor (HUS): 1

DESIGN, IMPLEMENTAION AND UTILIZATION OF MONITIRING ANDEVALUATION (M&E)

Criteria Sub-criteriaIDEVScore

Comments

M&E DESIGNM&E system is in place, clear, appropriate and realistic

3

- Detailed and systematic for Project outputsand outcomes related to capacity building- M&E of water services and sanitation relied on sector-wide WASH periodic surveys

Monitoring indicators and monitoring plan were duly approved

3

Yes, Outcome 12 in the framework of O&M improvements actions but somewhat late in the course of implementation No reference to M&E for hygiene/sanitation

Existence of disaggregated gender indicator

3Yes. E.g. Number of women in users associations and water points committees

Baseline data were available or collected during the design

2

Yes, covering water services in the two target districts covering notably costs recovery and functionality of water systems No report on such indicators for hygiene and sanitation

Other, specify OVERALL M&E DESIGN SCORE 3 SATISFAISANTE M&E IMPLEMENTA-TION

The M&E function is adequately equipped and staffed

2

PROTOS role ended with the closure of the Project. The ones to use the M&E systems are the Communes and the local branches of WASH public sector agencies which were trained for implementing M&E system. PCR does not report on what is actually done

OVERALL M&E IMPLEMENTATION SCORE 2 INSATISFAISANTEM&E UTILIZATION

The borrower used the tracking information for decision

2PCR does not provide information on what happened after Project closing.

OVERALL M&E UTILIZATION SCOREOVERALL M&E PERFORMANCE SCORE

2 INSATISFAISANTE

PCR QUALITY EVALUATION

CriteriaPCR-EVN

(1-4)Comments

QUALITY OF PCR

1. Extent of quality and completeness of the PCR evidence and analysis to substantiate the ratings of the various sections

3IPR not attachedInfo on temporary suspension lacking.

2. Extent of objectivity of PCR assessment score 3PCR does not cover the weakness of the M&E component directed at the WASH sector.

3. Extent of internal consistency of PCR assessment ratings; inaccuracies; inconsistencies; (in various sections; between text and ratings; consistency of overall rating with individual component ratings)

4

La documentation disponible sur les réalisations etles activités du projet. pour soutenir les éléments de preuves contenus dans le RAP. Le dispositif de suivi-évaluation à travers PROTOS-Bénis jugé très satisfaisante.

4. Extent of identification and assessment of key factors (internal and exogenous) and unintended effects (positive or negative) affecting design and implementation

3PCR notes the delays caused by local elections and the response of the Project

5. Adequacy of treatment of safeguards, fiduciary issues, and alignment and harmonization

3 Satisfactory

6. Extent of soundness of data generating and analysis process (including rates of returns) in support of PCR assessment

3Data on access to safe water and to basic sanitation of general nature covering target districts.

7. Overall adequacy of the accessible evidence (fromPCR including annexure and other data provided)

3Info lacking in the PCR on the cause, length and scope on the temporary suspension of disbursements

8. Extent to which lessons learned (and recommendations) are clear and based on the PCR assessment (evidence & analysis)

3Well done. Useful to deepen reflections on funding of Communes and on follow-up.

9. Extent of overall clarity and completeness of the PCR

3Satisfactory. Mais les Annexes techniques pas toujours disponibles dans le RAP. de calcul et

Other (specify)

PCR QUALITY SCORE 3,11 Satisfactory

PCR compliance with guidelines (PCR/OM ; IDEV)

1. PCR Timeliness (On time = 4; Late= 1) 1 PCR issued more than 6 months after closing

2. Extent of participation of borrower, Co-financiers & field offices in PCR preparation

3

3. Other aspect(s) (specify)

PCR COMPLIANCE SCORE 2

*** rated as Very Good (4), or Good (3), or Fair (2), or Poor (1)

ReferencesDocuments Originator Date

1 PAR Benin decentalisation version approuvee FR AfDB Sept. 2012

2 SPN Aide-memoire mai 2014 AfDB May 20143 Rapport Contrôle Interne après examen 2 de la

BanqueAfDB Aout

20154 Rapport trimestriel Avril - Juin 2015 PROTOS Juillet

20155 Rapport d’Audit des Comptes après examen de la

Banque AfDB Aout

20156 Manuel de Procedures PROTOS/

AfDBSept 2015

7 OK ORPF2 Rapport Audit période Oct. 2012 au 31 Dec. 2014

AfDB Oct. 2015

8 SUPERVISION JUILLET 2013 (2015 !!)RAPPORT SUR L’ETAT D’EXECUTION ET SUR LES RESULTATS (EER)

AfDB Sept. 2015

9 SPN BTOR AfDB Jan. 201610

SPN Aide-mémoire AfDB Dec. 2016

11

SPN BTOR AfDB Jan. 2017

12

SUPERVISION Février 2016

13

RAPPORT SUR L’ETAT D’EXECUTION ET SUR LES RESULTATS (EER)

AfDB Fev. 2016

14

Rapport trimestriel : Oct.- Déc. 2015 PROTOS Jan. 2016

15

Rapport trimestriel Jan.- Mars 2016 PROTOS Avril 2016

16

Rapport trimestriel Juillet – Sept. 2016 PROTOS Oct. 2016

17

PCR Nov. 2017 EN , PCR FR AfDB Nov. 2017