Inception report – final (redacted)
Nigeria Country Strategy and Program
Evaluation 2004-14
October 2015
Table of contents
3
Inception report
List of abbreviations 5
1 Introduction 6
1.1 Context and background of the evaluation 6
1.2 The socio-economic backdrop for the AfDB’s engagement 6
2 Purpose and objectives 9
3 Evaluation questions and universe 11
3.1 The Evaluation Questions 11
3.2 The evaluation universe 12
4 Methodology 13
4.1 Evaluation design- a theory based evaluation focused on learning 13
4.2 Evaluation components – a multi-layered approach 13
The strategic level 14
Portfolio level 14
Project level 14
4.3 Selecting projects eligible for PRA 14
4.4 Identifying different project types 17
4.5 Data collection 18
4.6 Sampling 19
4.7 Practical issues relating to data collection 20
4.8 Data analysis and ratings- a rigorous approach 21
5 Limitations, risks and mitigation 23
6 The evaluation process – managing for quality, timeliness and impact 25
6.1 Principles 25
6.2 Deliverables and timeline 25
6.3 Roles and responsibilities 26
6.4 Quality assurance 26
6.5 Dissemination – sharing knowledge 27
Annex 1: Project Portfolio Error! Bookmark not defined.
Annex 2: List of persons consulted during September 2015 scoping mission 31
Annex 3: Overview of AfDBs CSPs for Nigeria during 2004-2014 33
Annex 4: Evaluation matrix 35
Annex 5: Country ToC 43
Annex 6: Portfolio review template 46
4
Inception report
Annex 7: Project results assessments templates 49
Public sector project template 49
Private sector projects template 54
Annex 8: Examples of rating guidance: Effectiveness for public and private sector projects 59
Public sector projects 59
a) Achievement of outputs 59
b) Achievement of outcomes 60
Private sector projects 61
a) Achievement of outputs 61
b) Achievement of outcomes 61
Annex 9: Example of project ToC and indicators identified for collection 65
Annex 10: Examining lines of credit 67
Prior to Visit to the bank/FI 72
Visit to the bank/FI 72
Annex 11: Examining the knowledge aspect of the Country Strategy Papers and Programs 73
Annex 12: Interview protocol 75
Annex 13: List of key informants for semi-structured interviews 77
Public sector projects stakeholder mapping 78
Private sector projects stakeholder mapping 79
Annex 14: Document mapping 81
Mapping of available project documents 81
Public sector documents 81
Private sector documents 82
Mapping of available non-project documents 83
AfDB Documents 83
Other sources 85
Annex 15: Possible structure of main technical report and sector reports 86
Annex 16: Timeline 89
5
Inception report
List of abbreviations
ADF Agricultural Development Fund
AFW African Water Facility
CDE Committee on Development Effectiveness
CEDR Comprehensive Evaluation of Development Results
CSP Country Strategy Paper
CSPE Country Strategy and Program Evaluation
DIR Development Impact Report
ECOWAS Economic Community of West African States
EIB European Investment Bank
EQ Evaluation Question
ESW Economic and Sector Work
FI Financial intermediary/institution
GDP Gross Domestic Product
GHS General Household Survey
HNLSS Harmonised Nigeria Living Standards Survey
IDEV Independent Development Evaluation function
IFC International Finance Corporation
LA Loan agreement
LoC Line of Credit
ODA Official Development Assistance
OECD DAC Organisation for Economic Cooperation and Development/Development
Assistance Committee
PAR Project appraisal report
PCRs Project Completion Reports
PRA Project Results Assessment
PSR Project supervision report
RSDT Road Sector Development Team
RISP Regional Integration Strategy Paper
RMC Regional Member Countries
RWSSI Rural Water and Sanitation Initiative
SFR Special Relief Fund
TA Technical assistance
ToC Theory of Change
TYS Ten-Year Strategy
UA Unit of Account
XSR Expanded supervision report
6
Inception report
1 Introduction
1.1 Context and background of the evaluation
At the request of the African Development Bank's (hereafter "the Bank") Committee on
Development Effectiveness (CODE), the Independent Development Evaluation function (IDEV) is
undertaking a comprehensive evaluation of development results (CEDR) for the period of 2004-
2013. As part of this initiative, IDEV identified a purposive sample of 14 countries which represent
approximately 60% of the Bank's approvals over this period. This sample is representative of the
Bank's portfolio with respect to certain characteristics, such as country size, income, language and
fragility.
The Country Strategy and Program Evaluation (CSPE) initially covers the period 2004-14 (adding
an additional year to the CEDR requirements to ensure findings are as up to date as possible). In
order to establish a proper selection of projects to be studied in depth in the Project Result
Assessment (PRA), some projects with an approval date in 2003 have been included as well (cf.
section 4.3). As the project portfolio in Annex 1 illustrates, the Bank’s program during this period is
divided between lending to the private sector (mainly through lines of credit), lending to the public
sector in agriculture and infrastructure (notably power, transport, and water supply & sanitation) and
some programs in the social sector (education and health). In addition, the Bank invested indirectly
via equity funds and directly in financial institutions.
A scoping mission was conducted 7-18 September 2015. This mission focused on validating the
portfolio, collecting documentation, establishing contacts, testing new PRA templates for public and
private sector projects, communicating the evaluation to stakeholders, and seeking feedback on the
data scope and feasibility. A list of people met is available in Annex 2. The results of this scoping
mission have been used to refine and finalize the methodology and work plan for the evaluation.
Herewith the scoping mission paved way for the creation of the final list of projects eligible for PRA.
The stakeholders to be interviewed during the data collection mission will be subsequently identified
on the basis of this final list.
1.2 The socio-economic backdrop for the AfDB’s engagement
Following an April 2014 statistical "rebasing1" exercise – starting from the FY 2012/2013 – Nigeria
has emerged as Africa's largest economy, with 2013 GDP estimated at US$ 502 billion. This
exercise impacted on other key macroeconomic indicators, lowering the Government’s Budget
deficit to 1% a year, the Government’s expenditures to 15% a year and the level of the sovereign
debt to 10.6% a year. This fundamental change in the figures for the Nigerian economy was also
mirrored by the GDP composition with the share of the three main sectors – oil and gas, trade and
agriculture – plummeting from 85% of the GDP to the 54% following the rebasing. In particular, the
contribution of the oil and gas sector shrank from an estimated 37% share of the overall GDP to a
15.8% share, falling behind the contribution of the agriculture sector, now building up as much as
the 22% of the GDP.
1 Re-basing has entailed an updated of the firms assessed to be operating in Nigeria by roughly ten times since 1990, last
year in which the accounts statistics had been rebased (The World Bank).
7
Inception report
Table 1.1 Macroeconomic and sectoral indicators (after rebasing, World Bank calculations)
2010 2011 2012 2013 2014
Economic indicators2
Real GDP growth (%) 7.8 4.9 4.3 5.4 6.3
Inflation (%) 13.8 9.5 9.3 5.9 4.7
Government debt as % of
GDP 9.4 10.2 10.4 10.6
11
(preliminary)
Services as % of GDP 54.1 52.9 54.3 57.0 59.1
Industry as % of GDP 22.0 24.8 23.7 22.0 20.7
Agriculture as % of GDP 23.9 22.3 22.1 21.0 20.2
The internal composition of the non-oil related GDP as also shown fundamental variations, the
weight of agriculture has been reducing with manufacturing and trade gaining bigger shares. Over
the period concerned by the rebasing exercise, the economic performance of the oil and gas sector
has recorded a negative growth rate, falling by 4.9 and 13.1% in 2012 and 2013 respectively. The
overall economy has nevertheless scored positive and sustained growth rates for the same years,
with a non-oil GDP growth of 5.9% and 8.3% that translated in a total GDP growth of 4.3% in 2012
and 5.4% in 2013. Preliminary data for 2014 and projections for 2015 confirm the negative trend for
the oil and gas sector but also a slowing down in the non-oil GDP growth rate as shown in the table
below.
Table 1.2: Impact of the Oil and Gas sector on the Nigerian Economy
2012 2013 2014 (pl.) 2015 (pr.)
Oil and gas impact3
Real GDP growth (%) 4.3 5.4 6.3 4.8
Oil and Gas as % of GDP -4.9 -13.1 -1.3 -1.6
Non-oil as % of GDP 5.9 8.3 7.3 5.5
Total revenues as % of GDP 14.3 11.0 9.9 7.7
Oil and gas revenue as % of GDP 10.4 7.1 5.8 3.4
Total expenditure as % of GDP 14.7 13.4 12.3 10.6
Overall balance -0.3 -2.4 -2.5 -2.8
Non-oil primary balance -11.6 -9.7 -8.1 -5.5
The weight of the oil and gas sector over the economy may have been lowered as a component of
the GDP, but the weight of the sector In terms of government revenues remains heavy, as shown in
the table above. The lowering of the price of crude oil implied lower revenues and a deterioration of
the account balance of the state, that has been falling in negative territory. In this scenario of falling
revenues and increased pressure on the revenues and on the value of the Naira, the Government
has adopted a tightening fiscal policy. It further has allowed the Naira depreciation and unified the
foreign exchange rates.
Compared with the level of economic growth that has been achieved over the past years, the
official poverty rate level remains strikingly high at 46% of the adult population, according to the last
comprehensive household survey Harmonised Nigeria Living Standards Survey ( HNLSS) 2009-
2010, slightly declining from the 2004 level of 48%. However, smaller general household surveys in
panel format (GHS 2010-2011 and 2012-2013) provided more encouraging results as shown in the
table below.
2 Source: Data Portal .The World Bank, 2015 consulted the 29/09/2015. Available at: http://data.worldbank.org/ 3 Source: ‘IMF press release - IMF Executive Board Concludes 2014 Article IV Consultation with Nigeria’ Press Release No.
15/91, March 4, 2015. Available at: https://www.imf.org/external/np/sec/pr/2015/pr1591.htm
8
Inception report
Table 1.3 Impact of the Oil and Gas sector on the Nigerian Economy
Region HNLSS 2009-10 GHS 2010-2011 GHS 2012-2013 GHS Diff
National 62.6 35.2 33.1 -2.1
Rural 69.1 46.3 44.9 -1.4
Urban 51.2 15.8 12.6 -3.2
North Central 65.8 33.4 31.1 -2.3
North East 75.4 47.1 50.2 3.1
North West 74.2 46.9 45.9 -1.0
South East 54.9 31.7 28.8 -2.9
South 53.3 27.7 24.4 -3.3
South West 47.9 21.2 16.0 -5.2
Although the World Bank warns that the abovementioned figures are not totally comparable and the
GHS are not as comprehensive and accurate as the NHLSS would be, they still provide a proxy of
how the poverty headcount ratio has improved overall. The table also highlights two structural
features of the Nigerian socio-economic landscape, the regional divide between a poor north and a
more prosperous south and the persistence of rural poverty compared to a net decrease in the
urban poverty levels. Income inequality overall is high, with the last GINI index calculation available
(2010) showing an increase to 43 from the 40 recorded in year 2000.
Table 1.4 Social and environmental indicators 4
Social and environmental indicators 1990 2000 2013
Unemployment (%) 7.4 7.6 7.5
Maternal mortality ratio (per 100,000 live
births)
1,200 950 560
Total net primary school enrolment (%) - 65 64
Improved sanitation facilities (% of
population)
38 34 29
Improved water source (% of population) 40 52 68
CO2 emissions (kg per PPP $ of GDP) 0.8 1.2 0.5
Other key social and environmental indicators show mixed trends, with improvements in maternal
health and access to water sources but stable or decreasing indicators for the access to sanitation
facilities and school enrolment. A reason for this mixed performance is to be found in the massive
increase of the Nigerian population that has grown from 95 million inhabitants in 1990 to 173.6
million in 2013, with the life expectancy only growing from 46 years to 52 years on the same period.
The scoping mission confirmed the magnitude of challenges in Nigeria. One of the key issue is to
achieve economic growth in sectors not related to oil and gas, in agriculture in particular, to create
jobs that allow for the absorption of millions of young people looking for work, both now and in the
near future. The Bank’s portfolio seems to be in par with this key issue as it operates a diversified
portfolio. While finance sector projects constitute the largest share in the portfolio, the promotion of
agricultural development is pivotal for the Bank. Growth in this sector is directly promoted through
the funding of public projects in the agricultural sector or indirectly through projects in other sectors
like transport or environment. Agri-businesses are also funded by the bank via Lines of Credit
(LoCs) to commercial banks.
4 Source: Data Portal .The World Bank, 2015 consulted the 29/09/2015. Available at: http://data.worldbank.org/
9
Inception report
2 Purpose and objectives
The purpose of the Nigeria CSPE is two-fold. It is an important building block of evidence for the
CEDR but it is also an important evaluation product in its own right, which should be expected to
inform future programming in Nigeria, and indeed in other countries in the region where the Bank is
active with a similar program. The Bank’s program in Nigeria is large. The program is mainly
characterized by lending both to government (one third of the portfolio) and to the private sector
(close to two third of the portfolio). In addition, several grants have been provided by the Bank to
the public and private sector. The multinational operations account for a relatively small share of the
portfolio. The evaluation will explicitly address the issue of added value, and learning from the
Nigeria experience.
Table 2.1 Public Sector Portfolio overview
Nature Sector Number of
projects
Amount allocated % public
portfolio
% total
portfolio
Public
Agriculture 5 124,537,147 14.81% 5.32%
Environment 2 3,123,510.10 0.37% 0.13%
Transport 1 35,270,000 4.19% 1.51%
Water Supply & Sanitation 5 301,304,967.03 35.83% 12.88%
Power 2 132,000,000 15.70% 5.64%
Finance 1 194,848.90 0.02% 0.0083%
Social 4 31,771,353.67 3.78% 1.36%
Multi-Sector 2 212,707,195.22 25.29% 9.09%
Totals 22 840,909,021.92 100% 35.95%
Table 2.2 Private Sector Portfolio overview
Nature Sector Number of
projects
Amount % public
portfolio
% total
portfolio
Private
Power 1 70,854,146.74 2.34% 0.91%
Agriculture 1 21,279,614.13 1.42% 1.50%
Transport 1 35,076,837.02 4.73% 3.03%
Finance 20 1,371,221,037.29 91.51% 58.62%
Totals 23 1,498,431,635.18 100% 64.05%
The specific objectives of the evaluation are as follows:
To provide credible evaluative evidence on the development results of the Bank's engagement
in Nigeria
To provide credible evaluative evidence on how the Bank has managed its engagement with
Nigeria
To identify the factors (both internal and external) that affect good or poor performance.
To identify lessons and recommendations from the performance and management of the Bank's
support to Nigeria to inform the design and implementation of future strategies and operations
(both in Nigeria and potentially other Regional Member Countries, RMCs).
10
Inception report
These objectives serve multiple clients for this evaluation:
1. The Board - The evaluation will provide the Board with an independent and evidence-based
assessment of the development results that the Bank has achieved in Nigeria as well as with
suggestions for potential improvements to better position the Bank in terms of achieving the
objectives outlined in its Ten-Year Strategy (TYS). The volume of the program makes it of
particular interest to the Board.
2. Bank staff and management - The Nigeria CSPE can inform the development of a new Nigeria
CSP as well as new operations there, by providing the Bank’s office in Nigeria and
headquarters based staff working on Nigeria with independent and evidence–based
assessment of what worked and what did not work, and why, as well as suggestions for
improvement. Lessons based on experience in Nigeria should also be useful for the Bank’s
work in other countries with similar features (e.g. other middle income countries, or where the
Bank is focused on private sector operations).
3. IDEV – the evaluation will inform the CEDR by providing independent and evidence-based
assessment of the development results that the Bank has achieved in this country, as well as
lessons of what worked and what did not and the factors associated with good and poor
performance.
11
Inception report
3 Evaluation questions and universe
3.1 The Evaluation Questions
The Evaluation Questions for the Nigeria CSPE will take as their starting point the standard EQs
used for all CSPEs involved in the CEDR. In addition, the evaluation focuses on the extent to which
the Bank is maximising its added value in the Nigerian context. This added value relates to the
Bank’s focus, its instruments it has available in the country and its non-lending work.
Table 3.1. presents an overview of the Evaluation Questions.
Table 3.1 Overview evaluation questions
Criterion Evaluation Question
Achievement of Development Results
Relevance 1. To what extent are the country strategy and Bank operations aligned with:
a. RMC development needs;
b. RMC development strategies and priorities; and
c. The needs of beneficiaries?
2. To what extent are the interventions in the country aligned with the Bank’s
strategy and priorities?
Effectiveness 3. To what extent have the Bank’s interventions achieved their expected results?
4. To what extent have the Bank’s interventions benefited target group members?
5. To what extent have the Bank’s interventions contributed to the achievement of
development objectives and expected development results of the country, including
impacts (both intended and unintended)?
Sustainability 6. To what extent have achieved benefits continued or will be likely to continue
once the Bank’s interventions are completed?
Cross-cutting
issues
7. To what extent are the Bank’s interventions inclusive (i.e., bringing prosperity by
expanding the economic base across the barriers of age, gender and geography) in terms
of gender equality and regional disparity?
8. To what extent are the Bank’s interventions environmentally sustainable and
support the transition to green growth?
Management of the Bank's interventions
Efficiency 9. To what extent are the Bank’s interventions delivered in an efficient manner
(i.e., whether resources and inputs are economically converted to results)?
10. To what extent are the Bank’s interventions implemented in a timely manner
and in compliance with operational standards?
Design 11. To what extent is the quality of the CSP satisfactory?
12. To what extent has the Bank applied selectivity in designing its country portfolio
and focused on areas where it brings added value?
13. To what extent has the Bank been innovative in adapting its approach to the
country’s context and development challenges/needs?
14. To what extent are the Bank’s interventions coherent and well-coordinated
internally?
Knowledge and
policy advice
15. To what extent has the Bank actively engaged in and influenced policy dialogue
through relevant advice?
16. To what extent has the Bank delivered adequate analytical work in support of its
interventions, positioning and policy advice?
12
Inception report
Criterion Evaluation Question
Partnerships and
leverage
17. To what extent are the Bank’s interventions harmonized with those of other
donors (avoiding duplication, simplifying procedures etc.)?
18. To what extent are the Bank’s interventions and resources bringing in other
players and being leveraged for maximizing development effectiveness at country level?
Managing for
results
19. To what extent has the Bank successfully implemented management systems
that focus on results and allow learning from past experience?
20. To what extent has the Bank supported the development of national capacities
and management systems that focus on results?
Maximizing added value
Added value 21. To what extent has the Bank been able identify and maximize its added value in
the Nigeria context, in terms of:
(a) What it finances
(b) The instruments it has available
(c) Its non-lending work
Lessons
Lessons 22. What are the key factors positively and negatively influencing the achievement
of development results?
3.2 The evaluation universe
The scope of the evaluation in terms of timeframe is 2004-2014, i.e. projects approved in this
period. In terms of looking forward, to examine in what direction the portfolio is evolving more
recent approvals may also be considered (up to July 2015, when the portfolio is finalised). In
addition, in terms of contextualisation and since the evaluation cuts across the CSP periods, the
2004-2014 portfolio will be seen within that broader context. For that same reason projects
approved before 2004 but implemented within the period 2004-2014 will be included in this
evaluation when considered of added value for the assessment of development results. However,
2004-2014 will remain the guiding timeframe.
All projects listed under the Nigeria portfolio will be covered by this evaluation. These concern both
public and private sector projects implemented in Nigeria.
The evaluation will examine also multinational projects like the projects on Invasive Aquatic Weeds
and the Road Sector Development Team (RSDT), which were not included as such in the initial
portfolio, once these are approved by the Bank.
The way these projects will be analysed will be further set out in the next chapter on methodology.
13
Inception report
4 Methodology
4.1 Evaluation design- a theory based evaluation focused on learning
This is a theory-based evaluation. A Theory of Change (ToC) will be employed for both the Bank’s
overall programme for Nigeria and for individual projects that are examined in-depth.
A theory-based evaluation is the most suitable approach for undertaking CSPEs since it will not
only address which results were achieved, but also how and why these results were achieved or
not. The ToC for the Bank’s entire programme for Nigeria is set out in Annex 5. This ToC has been
reconstructed on the basis of the insights obtained from desk research as well as from the scoping
mission with the Bank’s staff that took place 7- 18 September. The ToC will be tested and validated
in the course of the evaluation. Of particular interest for learning purposes are the assumptions and
hypotheses listed. These will also be tested and the results as well as the factors affecting
performance will be recorded in the typology section 4.4. This typology facilitates the identification
of patterns that will be further explored.
4.2 Evaluation components – a multi-layered approach
The Bank’s programme for Nigeria for the period under review is composed of composed of 45
national projects, including 23 private and 22 public sector projects, and 7 multi-national projects.
The Bank’s portfolio in Nigeria is one of its most diversified portfolio’s as it covers a wide variety of
sectors and includes various lending and non-lending projects.
A multi-layered strategy will be employed. Hence the evaluation will be composed of three main
components as summarized in Figure 4.1. The components are closely linked to each other.
Figure 4.1 Multi-layered approach to evaluation
Projectportfolio
assessment
Projectresults
assessment
Detailed assessment of
completed projects
Full portfolio assessment,
looking at design, implementation,
results
Broader assessment,
beyond projects, looking at CSP level
Programmaticassessment
14
Inception report
The strategic level
CSPEs will each require an examination of the programme as a whole ad its main pillars. This
involves an analysis of non-project activities – notably policy dialogue as well as knowledge and
analytical work (studies) undertaken and whether the Bank has well identified its niche or added
value, and managed to build constructive relationships in Nigeria. The quality and results achieved
by the Bank’s knowledge work will be achieved through a combination of document review and key
informant interviews. The overall ToC for the country programme will be assessed at this level.
Portfolio level
The full portfolio of projects will be reviewed, based on available documentation. This portfolio will
be broken down into different parts suitably to focus on each of (i) design/quality at entry; (ii)
implementation, supervision/monitoring; (iii) results, including both outputs and outcomes. This
review will be desk based but complemented by information from primary data collection. It will be
crucial to assess most of the evaluation questions. Annex 6 provides an outline of the specific
criteria to be assessed in the portfolio review. This will be piloted with 3-4 projects before being
adjusted and rolled out to the rest of the portfolio.
Project level
Within the national portfolio, all of the completed/closed and near-completed projects (in terms of
disbursement ratios) will be subject to detailed Project Results Assessments (PRAs), including site
visits (where possible) and other additional data collection methods that are focused on results
(both outputs and outcomes). A template will be completed for each of these projects, providing
multiple lines of evidence for assessments along the various evaluation criteria, plus identification
and categorization of lessons. The templates for public and private sector projects slightly differ with
regard to the indicators for each evaluation (sub) criterion. The templates have been designed in
line with international evaluation guidance standards. The two templates are provided in Annex 7.
Given the focus on development results achieved by the Bank in Nigeria, the effectiveness criterion
is of key importance for this assessment. For each PRA project a tailored ToC will be developed.
This work will be especially crucial to assessing results and understanding the factors that affect
performance.
For each project selected for PRA, before the data collection mission a ToC will be developed to
inform the assessment. Indicators will be identified and the gaps in information highlighted.
Assumptions included in the ToC will be tested on the basis of desk research and the data
collection mission. Factors influencing performance will be noted and categorized. An example of a
project ToC is provided at Annex 8. Each project’s ToC is connected to the ToC at country level
through underlying ToC of the respective sector of the project (e.g. transport, agriculture, finance).
4.3 Selecting projects eligible for PRA
In the (revised) ToR, IDEV has already proposed a set of criteria to identify PRA-eligible projects,
which are supposed to be applied to all CSPEs to provide consistent data for the CEDR.
These criteria establish that the project has to be:
Approved in the period 2004-14, and
Closed/completed or near to completion.
To check whether the project is near completion, the disbursement rates stated in the Bank’s
systems have been verified with the project Task Managers. Furthermore, the following
15
Inception report
interventions are excluded from PRA: (i) projects with a value of less than one million UA; (ii)
studies.
On the basis of these criteria, 18 out of 45 interventions have been selected as eligible for PRA in
the CEDR for Nigeria
Comments on the initial selection
The initial selection included four public sector projects and 14 private sector projects. These
projects accounted for 746,9 million UA or 31% of the total portfolio. Overall, the projects selected
for PRA included 3 agricultural projects, 2 transport projects and 13 financial sector projects.
Twelve of these provided lines of credits (LoCs) to banks. This selection reflected the fact that LoCs
constitute an important share of the AfDB’s portfolio for Nigeria. In several cases LoCs are already
in their second and/ or third tranche (Zenith, Access, Guaranty Trust Bank).
The scoping mission revealed that the selection originally proposed requires to be amended as it
insufficiently constitutes a representative selection of the Bank’s interventions in Nigeria. In
particular, the mission contributed to the highlighting of some inconsistencies in the initial selection:
1. A rigid application of the criteria excluded several promising projects in terms of development
outcomes and lessons learned. For example, the Skills Training and Vocational Education
Programme (STVET) is potentially relevant for Nigeria’s education system since secondary
education has long been neglected by (other) donors. STVET was initially not selected as a
PRA project because of the disbursement rate being lower than 75%.5 Other projects were
excluded because of their approval dated back to 2003.
2. The portfolio in Nigeria is one of the Bank’s most diversified portfolios in terms of sectors
covered. The current focus on the financial sector insufficiently mirrors this diversity. More
importantly, it contributes to a bias towards the private sector, diminishing the actual weight of
the public sector in the project selection. The scoping mission moreover showed that the private
sector bias causes insufficient attention to issues like the impact of co-financing on project
ownership and ultimately the results that have been achieved with that project;
3. The agricultural sector plays a pivotal role in the entire project portfolio of the Bank since
projects from other sectors are in many cases mainly intended to improve agricultural
productivity, e.g. water management and transport projects;
4. Some partners seem to be less willing to cooperate with the evaluation team. The Bank’s field
office signalled that most notably Helios Towers would not be willing to provide us with
additional information. The preparation of the data collection mission will clarify the extent to
which partners are willing to cooperate and ultimately which information will be made available
to us.
Rebalancing the selection
While acknowledging that a different application of the selection criteria would set limitations to the
consistency and comparability of data for the CEDR, we propose to improve the selection by simply
mirroring the structure of the portfolio which consists of 40% public projects and 60% private
projects (in number).
In this way the selected sample of projects more adequately represents the theoretical construct
laid down in the country ToC as well as include a wider range of stakeholders involved in designing,
implementing receiving or administering AfDB lending programmes.
5 The final disbursement is planned for the end of September 2015.
16
Inception report
By doing so, we must point out that the project portfolio we have available is still provisional. As a
matter of fact a revised final project portfolio still has to be validated by the AfDB’s field office.
The information made available through the bank
The level of information available to us differs between the projects currently included in the
portfolio. For the majority of projects a lot of information is available. Project Appraisals, Aide
Memoires or Back-to-office Reports, Project Completion or Status Reports are documents that
contain the most valuable information. Sometimes the original Project Proposals and the
Loan/Grant Agreements are available as well. For most of the private sector projects, further
financial documentation like audits, Expanded Supervision Reports (XSRs) and annual financial
reports are available. All these documents together potentially will provide the team with a fairly
complete picture of the project status and life already at the level of desk research. In some cases,
crucial documents are not available and only the financing agreements or a single Back-to-office
Report has been provided.
Proposing a new selection
In order to arrive at a selection that mirrors the portfolio distribution between public and private
sector projects more adequately, more public sector projects need to be included in the selection.
Keeping in mind the caveats and diverging availability of information presented above, IDEV
decided to include public sector projects that have been approved in 2003. This allowed for the
inclusion of three agricultural projects (in addition to the social project STVET). The total number of
PRA projects increased to 21.
Availability of information for selected PRA projects
The level of information currently available for these 21 PRA projects differs. For the vast majority
of projects a high level of information is available. For those projects for which only limited
information is available, we have asked the Bank whether it is possible to provide this information.
Without this information the projects cannot be selected for PRA.
The revised selection of projects eligible for PRA herewith includes:
Table 4.1 List of PRA projects
No. Project name Project
code
Net loan (in
UA)
Previously
included?
Information
availability
Public Sector projects
1 RURAL ACCESS & MOBILITY
PROJECT
P-NG-
DB0-005
35,270,000 Yes High
2 SUPPORT TO AGRIC. & RURAL
INSTITUTIONS
P-NG-
AA0-026
2,636,876.27 Yes Medium
3 COMMUNITY-BASED AGR. &
RURAL DEVELOPMENT
P-NG-
AA0-025
13,000,000 No Medium/High
4 INVASIVE AQUATIC WEEDS –
NIGERIA (multi)
P-Z1-AA0-
084
1,609,000.00 Yes High
5 SKILLS TRAINING AND
VOCATIONAL EDUCATION
P-NG-IA0-
001
30,000,000 No Low
6 FADAMA DEVELOPMENT
PROJECT
P-NG-
A00-005
19,658,135.07 No Low
7 NERICA DISSEMINATION
PROJECT -NIGERIA
P-Z1-
AA0-073
5,570,000 No Insufficient
17
Inception report
No. Project name Project
code
Net loan (in
UA)
Previously
included?
Information
availability
Private Sector projects
8 LOAN LEKKI TOLL ROAD
PROJECT
P-NG-
DB0-008
35,076,837.02 Yes Medium
9 LOAN NIGERIAN EXPORT
IMPORT BANK
P-NG-
HAA-003
35,427,073.37 Yes Medium/High
10 LOC I ZENITH BANK PLC P-NG-
HAB-004
49,597,902.72 Yes Medium
11 LOC I GUARANTEE TRUST
BANK
P-NG-
HAB-005
28,341,658.70 Yes Insufficient
12 LOC II ZENITH BANK P-NG-
HAB-006
70,854,146.74 Yes Medium/High
13 LOC I ACCESS BANK NIGERIA P-NG-
HAB-007
24,798,951.36 Yes High
14 LOC II GUARANTY TRUST
BANK
P-NG-
HAB-009
63,768,732.07 Yes Low
15 EMERGENCY LIQUIDITY
FACILITY ZENITH
P-NG-
HAB-013
35,427,073.37 Yes High
16 EMERGENCY LIQUIDITY
FACILITY UBA
P-NG-
HAB-014
35,427,073.37 Yes High
17 LOC III ZENITH BANK PLC P-NG-
HAB-017
88,567,683.42 Yes Low
18 LOC II ACCESS BANK NIGERIA P-NG-
HAB-019
70,854,146.74 Yes Medium/High
19 FRB SUBSIDIARY IN NIGERIA P-NG-
HAB-022
53,363,332.18 No High
20 HELIOS TOWERS P-NG-
GBO-004
23,000,000.00 No High
21 UBA TRADE FINANCE
INITIATIVE
P-NG-
HAB-015
70,854,146.74 No Low
4.4 Identifying different project types
Overall the portfolio of projects to be examined presents one fundamental division between public
sector projects and private sector projects. The two groups are different in both the level of sector
coverage and the typology and variety of financing instruments deployed.
As shown in table 2.1 from chapter 2 the public sector portfolio covers seven different sectors plus
a multi-sector project. The Bank’s instruments for the public sector include grants and loans. The
funds of these grants or loans stem from specific sources like the Rural Water and Sanitation
Initiative (RSSWI) and the African Water Facility (AFW) or to special funds like the Special Relief
Fund (SFR) or the Agricultural Development Fund (ADF).
18
Inception report
Table 4.2 Public sector portfolio per typology of financing instrument :
The situation of the private sector portfolio is more homogeneous, with a predominance of the
financial projects, 20 over 23 and of the line of credit as a financing instrument. Also, the origin of
the funding is not linked to a special purpose fund as is the case with public sector projects.
Table 4.3 Private sector portfolio per typology of financing instrument
4.5 Data collection
A combination of quantitative and qualitative data collection methods will be used in the evaluation.
Multiple data collection methods is needed to strengthen the quality of the analysis and to be able
to come up with valid findings on programme and project results. Multiple methods will also help
capture the information needed to answer the various EQs to triangulate information sources, and
to support findings with multiple sources of evidence.
14%
27%
18%
4%
5%
9%
23%
MIC Grant ADF Loan SFR Grant RWSSI Grant
AWF Grant ADB Loan ADF Grant
31%
61%
4%4%
ADB Loan ADB LoC Equity Grant
19
Inception report
Data collection methods to be used include: (i) document review, portfolio review, literature review,
(ii) semi-structured interviews, and (iii) focus groups. The evaluation matrix at Annex 4 sets out
how each of these methods will be used to answer the EQs.
(i) Document review
For document review the evaluation team depends heavily on information provided by the Bank. In
addition, documents from third parties such as the World Bank will be analysed. Document reviews
will be undertaken for the PRAs, the portfolio assessments and the assessment at strategic level.
(ii) Semi-structured interviews
Semi-structured interviews will be held with a range of stakeholders, including Bank staff at head
quarters (e.g. project task managers, country economist). In addition, Bank staff in the country
office, project implementing partners at Federal, State and local level, private sector partners,
beneficiaries and other donors (International Finance Corporation (IFC), European Investment Bank
(EIB) will be approached via personal interviews during the data collection mission and clarification
round via email. Before each interview, interview questions will be send to the interviewees. The
interview protocol is included in Annex 11 Annex 12 presents a preliminary list of institutions to be
interviewed. Further review of project level documentation as well as potential snowballing on the
ground should give us further names to be added to this list.
(iii) Focus groups
The team might use focal groups to collect further information. Opportunities for focus groups may
be found in the following areas: development partners, Bank country office staff, and project staff of
implementing partners. The team will determine during the evaluation process whether collecting
information through meetings with focus groups or individual interviews will be more appropriate.
4.6 Sampling
A purposive sampling exercise will be undertaken to identify project sites that will be visited during
the data collection mission. Sites will only be visited for those projects that are selected for PRA. In
addition, both public and private sector projects selected for PRA may include multiple sub-projects,
for example sub-borrowers. The sample will be based on the representation of the main banks and
their sectoral focus (private sector) and the main stakeholders in the sample as well as on the
information available for each project. These sampling criteria are further elaborated below. We will
propose a definite sample as soon as the selection of the PRA projects is approved by IDEV.
For several private sector projects the team needs to consider which of the different sub-borrowers
need to be interviewed. The scoping mission revealed that banks provide a project pipeline when
requesting a LoC from the AfDB. Hence most of the sub-borrowers can be readily identified. All
banks interviewed during the scoping mission indicated to be willing to provide us with this list and
to assist in organising visits to sub-borrowers. We consider that this method might give more
valuable insights than the electronic survey would do. If the sample of sub-borrowers willing to be
interviewed is sufficiently representative and varied compared to the overall list of sub-borrowers,
we propose to skip the electronic survey and limit the data collection to interviews.
When undertaking the visits to sub-borrowers, we aim to guarantee the representativeness by
selecting and visiting sub-borrowers for every bank in the sample and sufficient spread across the
different sectors.
20
Inception report
The table below shows the main sectoral focus of the sub-borrowers that the banks have indicated
in the interviews during the scoping mission:
Table 4.4 Sector focus selected banks
Bank Sector focus
Nexim Manufacturing
Agriculture
Transport and services
Access Agriculture
Manufacturing
Tourism
Zenith Tourism
UBA Oil & gas
GT Bank Construction
Maritime
FRB
A final purposive sampling criterion concerns stakeholders. All important stakeholders should be
represented in our sample. A stakeholder map can be drawn by scanning the available documents
per project and identifying all those actors that are mentioned directly or indirectly. Private and
public sector stakeholder will differ from each other. However, public sector stakeholders will be
relevant for the assessment of private projects and vice versa.
Most of the public sector projects have stakeholders dispersed over the three-tier governance
system of Nigeria, from the Federal down to the State and Local level. The Federal Government
(which is in most cases the borrowing entity) includes the line ministries whose sector is concerned,
the specular regulating authorities and a specific department in the ministry. The same structure
can be found at State level whereas at local level mostly the Local Government Councils are
involved in project implementation.
4.7 Practical issues relating to data collection
There are four main challenges with regards data collection in Nigeria:
The Bank’s own monitoring and documentation varies in quality and detail. Document mapping
has been completed to provide clarity on what documentation is available (annex 13).
The problem of institutional memory and limited hand over will limit the information available for
the first half of the evaluation period. In a number of cases Bank task managers may indicate
that they do not have information pertaining to before they took up their post, indicating that
there may be a broader issue of handover/document storage.
21
Inception report
The data collection mission will be challenging in terms of geography and logistics. The size of
Nigeria is considerable and the Bank’s projects are implemented throughout the country. Public
sector projects in particular are dispersed over Nigeria, for example in Osun State, Cross-River
State and in the area of Kaduna. Visiting these projects is likely to be time-consuming given the
condition of roads and flight connections with Abuja or Lagos. The scoping mission revealed
that it would take at least two days for the team to visit projects in more remote areas. Given the
time and budgetary constraints to this evaluation, the number of site visits will be limited to
approximately six public sector projects. Conducting a proper sampling exercise is therefore
important to identify the most promising projects. In addition, we hired a local consultant to
assist with logistics. Private sector projects are located in the area of Lagos. Visiting these
projects will be mainly arranged via the banks.
The banks that were visited during the scoping mission all indicated to help with arranging visits
to the sub borrowers of their Lines of Credit. This likely generates a bias in the selection of
private sector visits since banks will be inclined to select only good practices. Banks will be
therefore asked whether the evaluation team can select sub borrowers from their project
pipelines.
4.8 Data analysis and ratings- a rigorous approach
Triangulation of data sources will be crucial. It is expected that the evaluation findings should be
based on multiple (i.e. two or more) lines of evidence. The qualitative data generated through
interviews and focus groups will be coded to identify common responses and to allow for robust
analysis.
Factors associated with good and poor performance, for the country program in general and for
Bank projects in particular will be identified and categorized against an agreed typology. A typology
which differentiates between design, implementation, contextual and other factors, is set out in
table 4.5 below. This will also help to test the ToC. Since context will be a crucial aspect of this, one
possible approach is to look at context/ mechanism/ outcome configurations6. The typology will be
adjusted if necessary.
Table 4.5 Typology of factors affecting performance
Stage in results
chain
Design factors Implementation factors External/context factors
Delivery of
inputs
Delivery of
outputs
Intermediate
outcomes
Final outcomes
Impacts
6 Context/mechanism/outcome configurations are used in the Realist Evaluation Approach set out by Pawson and Tilley in
2004. It emphasizes the importance of understanding context in explaining performance patterns.
22
Inception report
In order to understand the contribution of the Bank’s program to development results, a general
ToC has been developed (cf. Annex 5), which will be used to qualitatively assess the Bank’s
contribution. A more quantitative assessment is not feasible, since the Bank’s contribution is in fact
a tiny proportion of finance available in the country. Indeed, overall ODA to Nigeria has remained
below 0.5% of the GNI (World Bank, 2014). Nevertheless the Bank is seeking to unblock particular
bottlenecks, as summarised in the country ToC, so its contribution could be identified in those
areas.
In addition to interviews with key informants, secondary data at national, sector and sub-sector level
available for Nigeria will be analysed both for understanding of context, change over time and to
assist in determining the possible contribution made by the Bank in the key areas in which it has
engaged. Data sources will include for example those relating to investments at national and sector
level, other sectoral data (particularly related to energy, transport, water & sanitation and social
sectors) , exports, employment, poverty, access to finance and financial inclusion.
The primary data that support the evaluation findings will be made available as annexes to the
technical report, and all secondary data will be fully referenced and sourced.
Ratings will be provided at two levels: the individual projects included in the PRA and for the overall
program (the CSP). In both cases a six point rating scale (from highly satisfactory (6) to highly
unsatisfactory (1)) will be employed, against the criteria of relevance, efficiency, effectiveness and
sustainability.
For individual projects, the assessment will be based on the PRA templates (see Annex 7).
Guidance for completing the PRAs and for rating the effectiveness criteria has been made
available by the Bank for both public and private sector projects (see examples in Annex 8). The
three main levels of the evaluation – strategic, portfolio and project - will be brought together to
support an overall assessment of the Bank’s interventions in Nigeria over the period 2004-14. A
six point rating scale will be employed, as with the ratings for individual projects. All ratings will
be clearly justified, and based on multiple lines of evidence. Though ratings will be given for the
four criteria only, the evaluation will also conclude on each of the evaluation questions drawing
from all three levels as appropriate.
23
Inception report
5 Limitations, risks and mitigation
This evaluation has the following limitations, which should be understood and agreed at the outset:
How far down the results chain we can go. The evaluation is looking at a program that is large
for the Bank, but in a context where the funds constitute a small contribution. The evaluation will
seek to look at the outcome of these interventions and their contribution to wider development
results. It will not be possible to establish causality or an unquestionable contribution of
development impacts to this support.
Institutional memory and record storage. The scoping phase has already demonstrated the
difficulty in obtaining records for operations more than a few years old. Every channel will be
explored to establish the existence of documentation, such as supervision reports, including
contacting previous task managers. However, where they are not available, the evaluation
cannot give the benefit of the doubt regarding their existence. Similarly, as with any evaluation
covering more than a decade, there will be problems in finding key informants with institutional
memory beyond the last 3-4 years.
The evaluation has four major risks which will need to be actively managed. These risks and
mitigation strategies are set out in table 5.1 below.
Table 5.1 Risks and their mitigation
Risk Likelihood Potential
impact
Mitigation measure(s)
Lack of data Medium High Although there are serious data limitations the
evaluation design takes these into account. For
example, data available on development outcomes is
limited, which is one of the reasons for focusing on
financial sector aspects.
This inception report is based on an initial review of
documentation and scoping phases, which has
sought to explicitly address weaknesses in data
availability and design accordingly.
Falling behind schedule Medium Medium Because of the requirements for the CEDR the
schedule for this evaluation is tight. Any slippage will
have an impact on the schedule. However, such
slippage will not be material for the aspects of the
CSPE not required for the CEDR. Clear project
planning, sticking to deadlines and sound
communication within the team will all be sued to
minimize the likelihood of slippage.
Particular attention will be paid to those phases which
are often subject to delay: missions (because of the
need to coordinate and be flexible with partners); and
consultation (because of the tendency for comments
to arrive late).
In the event of slippage, priority will also be given to
the PRA to ensure data is available to the CEDR in
early December 2015. In fact, data is not required for
the new CSP’s development until mid-2016.
24
Inception report
Human resource
shortages
Low High The IDEV team working on this evaluation has joined
with the team working on Nigeria. This should allow
for some mutual backstopping. IDEV has engaged a
team of consultants, through the firm ECORYS. This
firm is of reasonable size and has back stopping as
well as quality control functions. The team proposed
is comprised of a mix of senior and junior staff.
Evaluation / review /
assessment fatigue
among interviewees
means that there is a
limited willingness to
grant us time for
interviews and meetings
Low Medium We will work hard to minimise the ‘contact time’,
which we demand of stakeholders by good
preparation of efficient interviews and focus groups.
Also the combined review and evaluation mission for
the energy projects will reduce the time investment for
the interviewees. We will ensure that interviewees of
government institutions will be given ample time to
provide their view on the value added of Bank
interventions in the country.
25
Inception report
6 The evaluation process – managing for quality, timeliness and impact
6.1 Principles
The principles which underwrite IDEV evaluations are in line with international standards, they
include:
Impartiality and independence
Credibility
Usefulness
Participation of stakeholders and partners
Transparency of methods and reporting
6.2 Deliverables and timeline
The deliverables, timelines etc. are set out in the ToRs of the contractual agreement. In summary,
the deliverables for this evaluation will be as follows:
1. Inception report
2. Completed PRA templates
3. Technical report(s) providing overall analysis as well as by sector (a potential outline is provided
at Annex 14).
4. Annexes or separate volumes containing all data used to support the technical report(s),
including coded interview notes, portfolio review files, statistical or financial analysis files.
5. Summary of findings from the South Africa and Nigeria CSPEs relating to lines of credit;
including comparative analysis, trends observed and lessons specific to this instrument.
6. A presentation at a stakeholder workshop.
7. A summary report for CODE (to be drafted by IDEV with consultant input).
The timeline for the evaluation is summarised in table 6.1 and presented in a chart in Annex 15.
Table 6.1 Timeline for the evaluation
Phase and period Main activities
Inception
01 August – 30 September
Scoping visit to Abidjan from 5-7 August, 2015;
Desk review and inception report drafting;
Scoping mission to Nigeria from 7 -18 September, 2015;
Revision inception report (amended and expanded methodology and
work plan);
Sharing draft inception report with reference group and reviewers,
including a meeting to discuss this draft report;
Finalization of the inception report.
Data collection and analysis
1 October – 30 November
Phase 1: desk review;
Review of documents, portfolio review, and identification of data
gaps;
Phase 2: Nigeria visit (1 November to 17 November -TBD);
Key informant interviews, site visits, focus groups, and further data
collection;
Additional data analysis before drafting.
26
Inception report
Phase and period Main activities
Technical report drafting and
finalization
30 November – 18 January
Draft report to IDEV for comment (1). Allow 10 days;
Draft report to reference group and reviewers for comment (2). Allow
14 days;
PRA templates also finalized (2) (by 23 December)
Report finalized (3).
Summary report, workshop and
dissemination
January-February 2016
Summary report drafted by IDEV with Ecorys input
Ecorys attendance at workshop
IDEV to lead on dissemination
6.3 Roles and responsibilities
IDEV works collaboratively with consultants. IDEV staff has organised the scoping mission to
Nigeria in the second week of September in which one of the team members has participated. IDEV
staff will also take part in the data collection mission. IDEV will act as a gateway to Bank staff and
clients. It will also act as guardians on quality control. This Inception Report was drafted jointly.
While the consultants will be tasked with drafting the technical reports, IDEV will lead in drafting the
summary (CODE) report.
6.4 Quality assurance
There are three phases of quality assurance for approved IDEV evaluations, as follows:
Quality threshold 1: Inception report (QT1 draft expected late September, finalization expected
early October)
Quality threshold 2: Technical reports (QT2 draft expected late December, finalization expected
late January)
Quality threshold 3: Summary report (QT3 draft expected early February finalization expected
mid February).
At each stage the drafts are circualted for review and comment from the groups listed below. At
each stage, the comments made are collated and addressed in a matrix, which is provided
alongside the updated draft.
IDEV team, peere reviewers, management
Independent expert reviewer (engaged by IDEV, independently of consulting firm)
Reference group. More information is provided about the reference group below.
Reference Group for Nigeria CSPE
The reference group will meet at each of the three QTs explained above. Reference group members have
been agreed to represent the following:
Nigeria Country Economist;
Nigeria Country Portfolio Officer,
Representative from OPSM,
Representative from ONEC,
Representative from OFSD.
Government of Nigeria: representative of the Ministry of Finance and Planning.
The group is expected to meet three times, though a fourth meeting can be called in order to discuss
recommendations if deemed appropriate. Given the Reference group is geographically split by location,
meetings will make use of VC, and reference group members will also be encouraged to submit comments
in writing.
27
Inception report
The reference group will be chaired by the responsible IDEV manager, and administered by the IDEV Task
manager.
6.5 Dissemination – sharing knowledge
The IDEV team will conduct a workshop to disseminate the evaluation findings, and in particular to
inform the next CSP in early 2016. The next CSP is due in 2017, however, CODE has asked for
CSPEs to come early enough to ensure they inform the early decisions on future CSPs, such as the
choice of pillars.
In addition to the published summary report, a short brief will also be made available both on line
and hard copy. The final inception report and final technical report will also be made available on
the IDEV website. The potential for additional briefs picking up on specific issues will be assessed
based on the strength of evidence in the technical report.
Annex 2: List of persons consulted during September 2015 scoping mission
Note that this list has been established on the basis of business cards available to the consultant
and information provided by the Bank.
Table 0.3 AfDB Country Office
Name Function Email
DORE, OUSMANE Director [email protected]
BARUNGI, BARBARA Lead Economist [email protected]
ALEMU, ZERIHUN GUDETA Chief Country Economist [email protected]
AKINWUMI, EMMANUEL
IBITUASE
Principal Private Sector Specialist [email protected]
OSUBOR, GREGORY Social Sector Expert [email protected]
OPOKU-DARKWA,
PATRICK
Principal Transport Engineer [email protected]
AMADOU, IBRAHIM AHMED Chief Agricultural Economist [email protected]
MOHAMMED, USMAN Principal Disbursement Officer [email protected]
OKPE, BERNARD Administrative Clerk/Receptionist [email protected]
OLAOYE, BOLANLE
PATRICIA
Principal Education Officer [email protected]
OZOR, CORNELIUS Information & Communication
Technology Officer
ANSAH, DENNIS Chief Portfolio Management [email protected]
Table 0.4 Public sector
Federal Ministry of Water and
Natural Resources
Haruna Mohammed Director , Federal Ministry of
Finance
Bala Danshehu Director Highways, Federal
Ministry of Works
Ishaq D. Mohammed Unit manager, RSDT, Federal
Ministry of Works
Federal Ministry of Environment
Federal Ministry of Agriculture
Ubandoma Ularamu National Coordinator, RAMP,
Federal Ministry of Agriculture
Ebrima Njie Commission, Infrastructures,
ECOWAS
Ministry of Education
Table 0.1 Private sector
Nigerian Export Import
Bank – Nexim
Bashir M.Wali Executive Director [email protected]
Dorothy Ogbutor Head, Specialised Business
Dept
Adaeze Ebegbulem Head, Trade and Finance Dept [email protected]
Ahmed D.Modibbo Secretary to the Board [email protected]
Tayo Omidiji Head, Strategic Planning [email protected]
ZENITH BANK PLC
Peter Amangbo GMD/CEO [email protected]
Olusola R. Ayodele Manager, mcp Group [email protected]
Ebenezer Onyeagwu Executive Director [email protected]
Anyimah Michael General Manager [email protected]
UBA
Lape Seriki Head, Bilateral Funding and
Correspondent Banking
Sola Yomi-Ajayi Division Head, Financial
Inst.&Int. Organizations
Uche Ike Chief Risk [email protected]
Adepeju Ajibola [email protected]
Ugo Nwaghodoh Group Chief Finance Officer [email protected]
ACCESS Bank
Ediale Eremionkhale Head Office [email protected]
GT Bank
Razaq Jinadu [email protected]
Lara Ogunlaja [email protected]
Annex 3: Overview of AfDBs CSPs for Nigeria during 2004-2014
There has been some evolution of the program under the four different strategies covering the
2004-2014 period, but also notable continuity. Some of the changes are linked to changes in the
way in which CSPs more generally are written by the Bank.
Table 0.5 Overview of Bank strategic priorities in Nigeria over the period
CSP
period
Pillars Components To note
2002-2004 1. Health sector
Improvement of public health facilities;
Improvement of access to health services.
Overall focus on
poverty reduction.
Strategy does not
contain pillars per
se but makes
reference to areas
of focus
2. Agriculture and
rural
development.
Enhance agricultural production;
Raise income levels; and
Support for food security.
3. Regional
Integration
Development of infrastructure for regional
integration;
Resource mobilization infrastructure projects
that contribute towards enhancing regional and
continental integration.
2005-2009 4. Development of
human capital
through improved
service delivery in
education and
health
Provision of unfettered access to education;
Elimination of gender disparity;
Improve the quality of education at all levels;
Strengthen preventive and curative primary
health care services, to increase access;
Provision of quality primary health care.
In 2010, this
strategy was
extended to 2011.
5. Stimulating non-
oil growth through
enhanced
infrastructure and
agricultural/rural
development
Provision enabling environment for
infrastructure investments;
ensure integrated water management and
development;
protection of water resources and environment;
building public-private partnership for a
sustainable development of water resources.
2011-2013 1. Improving
governance;
Transparency and accountability;
Participation;
Sector governance;
Capacity development;
Judicial reform and democratic governance.
This was a joint
strategy
implemented in
partnership with
the World Bank
2.Maintaining non-
oil growth;
Infrastructure to support growth clusters;
Promoting private sector;
Technical and vocational education to address
the skills gap;
Reducing import bans and high tariff barriers.
3. Promoting human
development
Improve access and utilization of services;
Support to education with particular focus on
girls education;
Support to maternal and child health.
CSP
period
Pillars Components To note
2013-2017 1. Supporting the
Development of a
Sound Policy
Environment;
Public financial management reforms;
Resource mobilization and fiscal federalism;
Private public partnerships;
Financial intermediation;
Gender mainstreaming;
Regional integration.
The CSP
mentions areas of
focus as opposed
to components
under each pillar.
2. Investing in
Critical
Infrastructure to
Promote the
Development of
the Real Sector of
the Economy.
Power;
Transport.
RISP
period
Pillar Components To note
2011-15 1. Linking
regional
markets
Regional transport infrastructure;
Transport and trade facilitation;
Regional energy production and
markets integration.
Pillar II refers also to
strengthening the capacity of
existing Regional Economic
Communities (RECs) in West
Africa. Specifically, the capacity
of ECOWAS/WAEMU, as well as
selected regional institutions, and
national entities where necessary.
2. Capacity
building
Capacity building for effective policy
and regional projects
implementation;
Capacity building for financial
sector integration; and
Support to regional research and
training centres relevant to the
integration agenda.
Consistent features throughout the time frame under review include the following:
There has been emphasis on diversification of growth towards non-oil sectors, through the
provision and enhancement of infrastructure, private sector development, as well as
agricultural/rural development;
There has been emphasis on promotion of public private partnerships, improvement of public
management, sector governance, and institutions.
Annex 4: Evaluation matrix
This matrix is a revised version of the matrix presented in the proposal. It is based on a first assessment on the insights obtained during the scoping mission at the Bank in Abidjan, the
documentation available to us as well as IDEV’s standard templates for analysis. For each of the projects covered in the PRA, mini evaluation matrices will be developed which will identify
indicators for the specific projects. The indicators included in the overarching matrix are not project specific, but the project level evaluations will feed in the overall conclusions.
Relevance
Evaluation question Level of analysis Indicators Methods
1. To what extent are the country strategy and
Bank operations aligned with:
RMC development needs;
RMC development strategies and priorities;
and
The needs of beneficiaries
Strategic, portfolio, project
Strategic level: Alignment of projects rationale and objectives with the Bank’s CSP
and the applicable sector strategies, the country’s development strategies and the
beneficiary needs from design/approval to completion (including any adjustments that
were made to the project in view of changes in the applicable policy environment,
such as project restructuring).
Proportion of CSPs which include analysis (or reference to analysis in
separate document) of beneficiary needs.
Feedback from RMC stakeholders.
Portfolio level: a) Sector distribution and trends volume and size; status of
operations; ratio of lending and non-lending/ private sector and public sector; b)
Percentage and ratio of loans and Grants; ratio of co-financed initiatives with DFIs; c)
Focus of programs: Extent of emphasis on; Regional Integration, financial sector
development, capacity development, response to cross-cutting issues(Gender,
Environment, Governance, knowledge work; d) Contribution of Multinational
operations benefitting the country.
Project level: (i) the relevance of project objectives (to a-d); (ii) relevance of project
design, including - the extent to which the project’s objectives are clearly stated and
focused on outcomes as opposed to outputs; the realism of intended outcomes in the
country’s current circumstances also is assessed; the extent to which project design
adopted the appropriate solutions to the identified problems; (iv) the modifications to
project design; (v) the circumstances prevailing at the time of the evaluation.
Proportion of projects that include satisfactory analysis of beneficiary needs.
PRA,
Portfolio review,
Analysis of AfDB CSP’s and other strategies
and non-bank documentation, semi-structured
interviews
Effectiveness
Evaluation question Level of analysis Indicators Methods
2. To what extent have the Bank’s
interventions achieved their expected
results?
Portfolio, project, strategic Strategic: To what extent has the theory of the Bank’s engagement in
SA held true.
Portfolio level: Sector performance as documented by the AfDB (per
key sector);
Project level:
Achievement of outputs: output execution ratio7 of both public and
private sector projects;
Achievement of outcomes: Achievement of intermediate and final
outcomes covering core sector indicators 8 of both public and
private sector projects.
PRA,
Portfolio review,
Analysis AfDB CSP’s and other strategies and
non-bank documentation, semi-structured
interviews, focus group with AfDB and in-country
3. To what extent have the Bank’s
interventions benefited target group
members?
Project Project contribution to expected achievements for target groups
included in the CSPs
Analysis AfDB CSP’s, PRA, semi-structured
interviews
4. To what extent have the Bank’s
interventions contributed to the
achievement of development objectives
and expected development results of the
country, including impacts (both intended
and unintended)?
Portfolio, project,
strategic.
Contribution of the Bank’s projects and engagement to national
outcomes at sector and project level.
Extent to which project results are aligned to CSP level planned
results.
Unintended outcomes encountered during the project cycle.
PRA,
Portfolio review,
Analysis of AfDB CSP’s and other strategies and
non-bank documentation, semi-structured
interviews
7 Supported by example outputs from the retrospective project theory of change. 8 Supported by example outputs from the retrospective project theory of change..
Sustainability
Evaluation question Level of analysis Indicators Methods
5. To what extent have achieved benefits
continued or will be likely to continue once
the Bank’s interventions are completed?
Portfolio, project Project:
Reliance of project achievements on the procurement of sound
technology;
Extent to which beneficiaries are able to operate funded
equipment autonomously;
Financial sustainability of project’s operations/organisation;
Institutional capacity in executing/operating organisation;
Ownership and partnerships observed;
Likelihood of continuation of benefits after project completion.
The project’s environmental and social performance in meeting the
Bank’s requirements; and ii) the project’s actual environmental and
social impacts.
Portfolio: Extent to which different aspects of sustainability (financial,
technical, institutional, environmental) were considered at appraisal
and monitored during supervision.
PRA, portfolio review, semi-structured interviews
with AfDB
Cross-cutting issues
Evaluation question Level of analysis Indicators Methods
6. To what extent are the Bank’s
interventions inclusive (i.e., bringing
prosperity by expanding the economic
base across the barriers of age, gender
and geography) in terms of gender
equality and regional disparity?
Portfolio, project Share of positive outcomes project/ portfolio for men/ women, age
groups, deprived areas or groups/ geographical distribution (%).
Extent to which cross cutting issues are considered at appraisal and
monitored.
Any unintended impacts (if any) on particular groups.
PRA, portfolio review, analysis AfDB strategies,
semi-structured interviews with AfDB,
beneficiaries and other stakeholders
7. To what extent are the Bank’s
interventions environmentally sustainable
and support the transition to green
growth?
Portfolio, project Environmental performance of projects/ portfolios in terms of pollution
loads, wastes, energy and resource efficiency, biodiversity
conservation, etc.
Compliance of borrowers with social and environmental provisions
included in the loan agreements.
Any unintended environmental impacts (if any).
Extent to which transition to green growth is assessed in appraisal and
monitored.
Efficiency
Evaluation question Level of analysis Indicators Methods
8. To what extent are the Bank’s
interventions delivered in an efficient
manner (i.e., whether resources and
inputs are economically converted to
results)?
Portfolio, project Costs of implementation (overheads)
Project level: cost benefit analysis/cost effectiveness analysis (i) at appraisal;
(ii) at time of evaluation.
Return on investments( financial products)
Evidence of responsiveness of Bank to country context that impacts on project
success eg. Effect of government policies and regulations on performance of
projects
PRA,
Portfolio review, semi-structured
interviews, focus group with AfDB and
in-country
9. To what extent are the Bank’s
interventions implemented in a timely
manner and in compliance with
operational standards?
Portfolio, project Portfolio level:
Evolution of portfolio performance:
Timeliness: Start up delays as well as implementation and closure delays;
IPR ratings;
Overview of project performance(PP/PAR/PPP), disbursement ratio/rating;
Evidence of supervision reporting, identifying and addressing issues of;
procurement, financial management, project design review (changes);
Evidence of frequency of monitoring.
Project management:
Compliance to reporting: audit reports, progress reporting from borrower;
Evidence that problematic projects are well monitored;
Evidence of Response to project management issues.
Project level:
Timeliness: Comparison between the planned and the actual period of
implementation from the date of effectiveness (approval stage – signature
loan agreement- first disbursement);
Dates of disbursement
Supervision and administration.
Design
Evaluation question Level of analysis Indicators Methods
10. To what extent is the quality of the CSP
satisfactory?
Strategic Compliance with AfDB guidelines valid at the time of drafting the CSP and
current (using existing quality at entry criteria in addition to SA specific
analysis).
RBM
Analysis AfDB strategies, semi-structured
interviews
11. To what extent has the Bank applied
selectivity in designing its country portfolio
and focused on areas where it brings added
value?
Portfolio Strategy: The CSPs demonstrate increased focus and selectivity, based on
analysis of where the Bank adds value.
Portfolio: The portfolio is focused on a limited number of sectors and sub-
sectors, as per CSPs and analysis of where Bank adds value.
Project: identification and appraisal includes consideration of added value.
Stakeholder views on Bank success in finding added value/potential for
future comparative advantage.
No of unsigned, almost approved and abandoned operations after appraisal
No of operations approved but cancelled.
Portfolio review, semi-structured interviews with
AfDB
12. To what extent has the Bank been innovative
in adapting its approach to the country’s
context and development challenges/needs?
Project Evidence of changes to project or strategy design following change in
country or borrower context
Evidence of responsiveness of Bank to country context.(eg. Impact of
government policies on success of operations)
PRA, semi-structured interviews with AfDB and
other stakeholders, focus groups with AfDB and in-
country
13. To what extent are the Bank’s interventions
coherent and well-coordinated internally?
Portfolio, project Evidence of country team coordination efforts
Evidence of complementarity and efficient use of instruments in response to
country needs
Evidence of operations aligning to Bank’s emerging initiatives and strategic
direction.
PRA, Portfolio review, semi-structured interviews
with AfDB
Knowledge and policy advice
Evaluation question Level of analysis Indicators Methods
14. To what extent has the Bank actively
engaged in and influenced policy
dialogue through relevant advice?
Strategic Extent to which relevant guidelines (on policy dialogue) are implemented
by the Bank;
#Participation in dialogue with RMC at various levels of governance
#Advice provided to RMC and beneficiaries
Analysis CSPE’s and strategic documentation,
semi-structured interviews with AfDB,
beneficiaries and other stakeholders
15. To what extent has the Bank delivered
adequate analytical work in support of its
interventions, positioning and policy
advice?
Strategic, portfolio, project #Studies, monitoring and evaluations, analyses undertaken by the AfDB
Proportion of operations preceded by Bank funded studies
Views from stakeholders on usefulness, availability and dissemination of
knowledge products.
PRA, portfolio review, analysis CSPE’s and
strategic documentation, semi-structured
interviews with AfDB, beneficiaries and other
stakeholders, focus groups in country
Partnerships and leverage
Evaluation question Level of analysis Indicators Methods
16. To what extent are the Bank’s interventions
harmonized with those of other DPs avoiding
duplication, simplifying procedures etc.)?
Strategic, portfolio #Meetings with other DPs (institutionalised and occasional)
#Agreements on common strategies with other DPs (e.g. CAF)
#Sectoral partnerships with other donors
Views of stakeholders on efforts made for
harmonisation/coordination/complementarity.
Proportion of projects for which procedures are shared (either with GoSa or
other DPs)
Analysis AfDB strategic documentation, portfolio
review, semi-structured interviews with AfDB,
beneficiaries and other stakeholders (donors)
17. To what extent are the Bank’s interventions
and resources bringing in other players and
being leveraged for maximizing development
effectiveness at country level?
Strategic, portfolio #Common interventions with other DPs
# Interventions for which other DPs were encouraged to come on board
following Bank engagement.
Analysis AfDB strategic documentation, portfolio
review, semi-structured interviews with AfDB,
beneficiaries and other stakeholders (donors), in-
country focus group
Managing for results
Evaluation question Level of analysis Indicators Methods
18. To what extent has the Bank successfully
implemented management systems that
focus on results and allow learning from
past experience?
Strategic, portfolio, project Application RBM principles in the AfDB, inclusion of objectives,
expected results and indicators to monitor progress of achievements;
M&E system in place in AfDB;
Proportion of projects for which problems identified through monitoring;
proportion of which projects adjusted to address problem.
Proportion of projects which consider lessons from previous projects in
their design.
Analysis AfDB strategic documentation, portfolio
review, PRA, semi-structured interviews with
AfDB and beneficiaries
19. To what extent has the Bank supported
the development of national capacities
and management systems that focus on
results?
Portfolio, project Portfolio: #Capacity building projects funded (lending & non-lending
activities) by the AfDB that aim to introduce RBM (and alike) principles
in SA
Project: Extent to which project design and monitoring process
encouraged clients to take on more results based approach.
Lessons learned
Evaluation question Level of analysis Indicators Methods
20. What are the key factors positively and
negatively influencing the achievement of
development results?
Portfolio, project Evidence of key program implementation issues linked to project
success/failure from stakeholders perspective.
PRA, portfolio reviews, semi-structured
interviews with AfDB
Annex 5: Country ToC
Strengthening the policies and developing a sound policy environment
and an enabling environment for private
sector development
Investment in climate resilient
infrastructure for road transport ,
energy, agriculture and water, health & sanitation, and
education
Policy advice and analytical work
focusing on resource mobilisation, oil
resources and subsidy manage-ment, job
creation, green growth, gender mainstreaming,
and regional integration
Inreased percentage of Federal, State and local road network in
good conditionProvision of concessional credit
to agribusinessIncrease average travel speed and
commuter travel time reduced
Intermediaries have greater financial
capacity to on lend to SMEs, ensuring their
own sustainability and focusing on
development additionality
Context Input Output Intermediate Outcome Outcome Impact
- One of Africa’s fastest growing economies
- Heavy reliance on oil Need for diversification
- Acess to financing of enterprises/ SMEs is a bottleneck, which might even become stronger due to the drop in the oil price
- Lack of adequate infrastructure in all sectors (transport, energy, water and sanitation) also constraint the growth in all sectors, esp. agriculture
- Fast growing young population facing large unemployment and social deprivation
- Complex governance structure with a Federal level, 36 states and local
authorities
- Authorities’capacities generally face constaints and authority is challenged by Boko Haram in the North-East
Increased production of food and cash crops and
better distribution of fertilizer
Capacity building for PPP
Infrastructure
Federal, State and local roads rehabilitated or
upgraded
Power generation increased and
transmission network extended and strengthened
Increased productivity & income
for farmers
Contribution to improved business
and investment environment
Increase of the actually irrigated area
Reduction in energy losses and additional customers connected
Improved health situation of the
population
Better policy formulation and
implementation by the GoN, leading to
enhanced
macroeconomic environment,
improved public financial
management and improved public-private dialogue,
Increased awareness and skills concerning,
PPP, resource mobilisation, jobcreation, green
growth, gender mainstreaming and regional integration
Increased knowledge-sharing,
in PPP, resource mobilisation, job creation , green
growth, gender mainstreaming and regional integration
Investing in critical, safe and efficient
infrastructure to promote the development of the
real sector of the
economy,
More jobs created in the value chain of agricultural crops
Increase in access to electricity and
reduction in electriciy cost
New urban and rural water connections and
sanitation facilities
Construction and rehabilitation of dams and irrigation schemes
Increased access to clean water and
sanitation facilities
Indirect investments to deepen private sector participa-
tion in infrastruc-ture financing
through private equity funds,
lines of credits and partial risk
guarantees
Increased access to finance in the private
sector
Economic diversification
towards non-oil sectors
Leveraging third-party investment in
the form of co-financing Food security ensured
Inclusive growth and transition to green growth
in Nigeria
Network of public educational and health
facillities expanded
Improved access to education and health
Increas ed share of the population with
education
Workforce has skills in demand
by employers to support job
creation.
Nigerian economy is not
negatively affected by low-oil
price and global economy.
Nigerian politics and public
administration engaged in
reforms.
Government regulation and
policies favorable for successful
implementation, and as a result
the regulatory environment
supports business growth.
No major barriers or stumbling
blocks to regional integration,
trade, and the free flow of
people and capital.
Private sector development supports
inclusive and green growth, not
unequal growth or growth that is not
environmentally sustainable.
Political will for transition to green
growth.
Political and social stability.
Appropriate targeting and
level of resources.
Right instruments available.
Complementarity and
coordination with other
development
partners/sources of finance.
Bank (and other partners)
provide inputs on time and
as planned. This includes
Bank's flexibility in re-
aligning to sudden changes
of government regulations
and policies as well as
adaptability in its approach
to changing market
conditions.
Partner meets conditions to
allow disbursement on time.
Bank inputs are well
designed (evidence
based/ownership/ design)
and sequenced.
AfDB procedures are clear to all relevant
stakeholders
AfDB loans are attractive to borrowers
despite reporting requirements and
procedures.
Implementation on schedule.
Design appropriate and scalable.
Sound management in partner
organisations.
Risks monitored and well managed.
Procurement systems efficient and effective.
Sufficient capacity available to implement.
Training/equipment is effective in
enhancing capacity.
Appropriate institutional
support/capacity/policy space to support
use of funds.
Knowledge work of sufficient quality, and
disseminated.
Improvements in infrastructure
causes economic spill-overs to
other sectors (especially growth
in agricultural production).
Growth in agricultural
production brings along
multiplier effects (e.g. supply to
agribusiness).
New and refurbished
infrastructure is suitable for
harsh tropical conditions (e.g.
resistant to heavy rainfall).
Politics and public
administration are willing to
engage in good governance.
Regulatory environment
encourages take up of access to
finance.
Knowledge shared is relevant to
needs and there is absorption
capacity to use it.
Assumptions
Mainly internal ------------------------------------------------------------------------------------------------------------------> Mainly external
Risks
Political stability challenged by disfunctional Federal government and/ or Boko Haram. Social stability challenged by significant ineqaulities, poverty and high unemployment rates, partiularly among young poeople.The number of jobs created is insufficient to absorb the millions of newcomers on the labour market each year.Low oil price threatens solvency of the Nigerian economy.
Adverse global economic environment leading to increased vulnerability of the country to unexpected capital movements and changes in commodity prices.Fiscal policy of the Government will not be credible and irresponsive.
Annex 6: Portfolio review template
Portfolio review template
Assessment
Area
Focus Criteria Main Sources of Information
and Data
Relevance Alignment with
Bank priorities
(CSPs)
Extent of emphasis on increased job
creation, increased entrepreneurial
activity, and increased regional
Integration
Annual Reports (2004 – 2013)
Country Portfolio & Improvement
Reports
Country Strategy
Credit notes where applicable
Identification or preparation
reports if available
Loan Agreement
Papers(Bank)
Policy and strategy documents
(Bank)
Portfolio data (SAP)
Project Appraisal Reports
Project concept notes
Statistical data (Bank)
Country development plans
(govt)
Alignment with
cross-cutting
issues
Extent of emphasis on inclusive
growth, environmental sustainability,
gender equity, good governance and
knowledge solutions
Alignment with
country
priorities
Degree of alignment with national
policies, sector strategies etc.
Alignment with
beneficiary
needs
Evidence of consideration of
beneficiary needs at appraisal
Evidence that project results meets
beneficiary needs
Bank’s added
value
Evidence of the added value which the
Bank’s strategy and focus provides
Effectiveness Achievement of
actual or
expected
outputs
Output execution ratios
What factors served to enable
outputs?
What factors served to hinder outputs?
Policy reviews(Strategy
documents, MTS 2008-2012,
TYS 2013-2022
Annual Reports (2004 – 2013)
IDEV’s evaluations; SME’s, QaE,
Transport, Energy, ESWs
Development Partners sector
review reports
Achievement of
actual or
expected
outcomes
Degree of contribution to Intermediate
Outcomes, and of Outcomes, in the
country-level ToC
What factors served to enable
outcomes?
What factors served to hinder
outcomes?
Unintended
outcomes
Positive and negative contribution of
unintended outcomes to ToC
Portfolio
distribution
Sector distribution, volume and size;
status of operations; ratio of lending
and non-lending/ private sector and
public sector
Percentage and ratio of loans and
Grants; ratio of co-financed initiatives
with DFIs
Contribution of multinational
operations benefitting the country
Efficiency Cost-benefit
analysis
Economic Rates of Return
Sizes of contribution to economic
growth
SAP portfolio data
Flashlight reports
Outliers report
Supervision reports Cost- Costs of alternative ways to achieve
Assessment
Area
Focus Criteria Main Sources of Information
and Data
effectiveness
analysis
project objectives Progress Reports
Project monitoring Reports
Audit reports
Borrowers reporting documents
Reference to bank’s
standard/criteria for self-
assessment and quality control
(see PCR guidelines and
portfolio improvement action plan
series
Project Appraisal Report(PAR)
Project concept notes and
technical notes
Country Strategy Papers (Bank)
ADOA Guidelines
Evaluation of QaE (IDEV)
PCR guidelines
Readiness Review Guidelines
Bank
investment
profitability
Net profit contribution
Return on equity
Timeliness Start up delays9 as well as
implementation and closure delays
What factors served to cause delays?
Quality at entry Evidence of Quality of Logic or Results
Framework.
Evidence of Risk Assessment and
existing mitigation measures in design.
Evidence of consideration of lessons
from previous or similar interventions.
Adequacy of context analysis and
alignment to rationale for intervention
Evidence of consideration of Bank’s
added value/niche.
Co-financing – extent to which Bank
led and leveraged funds from others,
versus played a supportive role.
Adequacy of reporting requirements.
Discrepancy between ADOA ratings
and PAR (for private sector projects
only)
Implementation
progress
IPR ratings
Overview of project
performance(PP/PAR/PPP10),
disbursement ratio/rating
Project
management
Evidence of supervision reporting,
identifying and addressing issues of:
procurement, financial management,
project design review (changes)
Evidence of frequency of monitoring
Evidence that problematic projects are
well monitored
Evidence of responses to project
management issues:
Responses to complex issues with
large co-financed projects
Evidence of internal coordination to
respond to project management
issues
Compliance to reporting requirements:
audit reports, progress reporting
Portfolio Extent to which the portfolio is
9 Assessment of time taken from Project Approval (approval date) to project effectiveness and 1st disbursement 10 Project Assessment Criteria: PP - Problematic Projects; PAR - Projects at Risk; PPP - Potential Problematic Project
Source: Portfolio Flashlight Report
Assessment
Area
Focus Criteria Main Sources of Information
and Data
coherence and
performance
coherent and well-coordinated in
achieving objectives of CSP;
Effect of decentralisation on portfolio
performance
Sustainability Technical
soundness
Soundness of technology used,
including operations and maintenance
requirements
Supervision reports
Progress Reports
Project monitoring Reports
Project Appraisal Report(PAR)
Project concept notes and
technical notes
Economic and
financial
viability
Extent to which effective mechanisms
and modalities are in place to ensure a
sustained flow of benefits
Institutional
sustainability
Extent to which improved governance
practices, improved skills and
capacities, and improved institutional
mechanisms have come into effect,
and commitment and ownership exists
to ensure a sustained flow of benefits
Annex 7: Project results assessments templates
Public sector project template
Name of project Insert here
Country Insert here
Year approved Insert here
Year Closed (if still open, insert that open and also level of disbursement) Insert here
Sector Insert here
PCR available Yes/No
1.1 RELEVANCE
1.1 Relevance of project objectives Rating
(1 - 6) The relevance of objectives assesses to what extend the project purpose as specified in the RLF was aligned with the Bank’s CSP and the applicable sector
strategies, the country’s development strategies and the beneficiary needs from design/approval to completion (including any adjustments that were made to the
project in view of changes in the applicable policy environment, such as project restructuring).
Insert your evidence here Insert your
rating here
1.2 Relevance of project design to achieve those objective
The relevance of project design should consider: (i) the extent to which the project’s objectives are clearly stated and focused on outcomes as opposed to outputs;
(ii) The realism of intended outcomes in the country’s current circumstances also is assessed; (iii) the extent to which project design adopted the appropriate
solutions to the identified problems; (iv) the modifications to project design; (v) the circumstances prevailing at the time of the evaluation.
Insert your evidence here Insert your
rating here
Overall Rating for Relevance Insert your
rating here
2. EFFECTIVENESS
The assessment of Effectiveness tests the validity of the anticipated links between the project’s activities, outputs, and intended outcomes (the results chain). Actual, expected and
unintended results of an operation are included in the assessment of Effectiveness. For PBOs the assessment should not only review the extent to which outputs were delivered (i.e.
agreed-upon policy reforms took place), but also the degree to which complementary measures necessary for their implementation occurred (e.g. public awareness, policy dialogue
and institutional arrangements).
2.1 Achievement of outputs Rating
(1 - 6) The assessment of outputs is based on the output execution ratio. It should consider the planned (targets) and actual output or those who are considered on track
to be reached. In determining the final rating, no formula based on a pre-determined weight applied to individual outputs is undertaken. It is considered good
practice to select no more than 10 output indicators in the RLF and to take into account the relative importance of the various components of the project in their
selection. The overall output rating is based on the percentage of outputs (output execution ratio) that reached or are on track to meet the end of project target.
Insert your evidence here Insert your
rating here
2.2 Achievement of outcomes
The assessment of outcome puts is based on the direct and intermediate outcomes stated in the retrospective project logic model. It should cover at least the core
sector indicators.
Insert your evidence here Insert your
rating here
2.2 Unintended outcomes (if any)
The assessment will cover all unintended outcome which came out during the project cycle
Insert your evidence here Insert your
rating here
Overall Rating for Effectiveness Insert your
rating here
3. EFFICIENCY
The Efficiency assessment attempts to answer two questions: (i) Did the benefits of the project (achieved or expected to be achieved) exceed project costs; and (ii) Were the benefits
of the project achieved at least cost? Cost-benefits analysis helps to address the first question. To address the second question a cost-effectiveness analysis is carried out. Good
practices suggest also the, In addition to the traditional measures of efficiency (cost-benefit analysis and cost-effectiveness analysis), the Efficiency assessment considers aspects of
project design and implementation that either contributed to or reduced efficiency (Timeless and Implementation progress) to the extent they are not already captured in the
evaluation’s cost-benefit or cost-effectiveness analysis.
3.1 Cost-benefits analysis Rating
(1 - 6) Cost-benefit analysis is carried out to the extent that data is available. The validity of the cost-benefit analysis conducted at appraisal/mid-term review is re-
assessed at completion It is a recommended to use the same model that was developed at appraisal. For PBOs a quantitative assessment will be done if an
Economic Rate of Return (ERR) was calculated at appraisal, otherwise an assessment could be done with regards to the contribution of policy reforms to
economic growth (if not applicable, indicate N/A for this criterion).
Insert your evidence here Insert your
rating here
3.2 Cost-Effectiveness Rating
(1 - 6) The analysis considers the cost of alternative ways to achieve project objectives, unit costs for comparable activities, sector or industry standards, and/or other
available evidence of the efficient use of project resources.
Insert your evidence here Insert your
rating here
3.3 Timeliness Rating
(1 - 6) The timeliness of project implementation is based on a comparison between the planned and the actual period of implementation from the date of effectiveness.
For PBOs, the timely releases of the tranche(s) are assessed through this criterion. The following rating scale applies
Insert your evidence here Insert your
rating here
3.4 Implementation progress (IP) Rating
(1 - 6) The IP rating will be derived from the IPR that shall be updated in tandem with the PCR preparation. The IP rating takes into account all applicable IP criteria
assessed under each of the three main categories: i) compliance with covenants (project covenants, environmental and social safeguards and audit compliance),
ii) project systems and procedures (procurement, financial management and monitoring and evaluation), and iii) project execution and financing (disbursement,
budget commitments, counterpart funding and co-financing). The simple arithmetic average of the individual ratings is calculated to derive the final rating. The
overall IP rating is provided as follows.
Insert your evidence here Insert your
rating here
Overall Rating for Efficiency Insert your
rating here
4. SUSTAINABILITY
The assessment of sustainability considers the extent to which the project has addressed risks during implementation and put in place mechanisms to ensure the continued flow of
benefits after completion. It should also evaluate risks to the sustainability of development outcomes and/or the project’s benefits, including the resilience to exogenous factors. The
overall rating of the sustainability outcome is the mean of the rating of the following four criteria: i) technical sustainability; ii) financial sustainability, iii) institutional sustainability and
strengthening of capacities, iv) ownership and sustainability of partnerships and v) environmental and social sustainability.
4.1 Technical Soundness Rating
(1 - 6) The criterion assesses the extent to which the project achievements rely on sound technology using inputs efficiently and providing productivity gains. It includes
operation and maintenance (O&M) facilitation, availability of recurrent funding, spare parts, workshop facilities etc.)
Insert your evidence here
Insert your
rating here
4.2 Economic and Financial viability Rating
(1 - 6) This criterion assesses the extent to which funding mechanisms and modalities (e.g. tariffs, user fees, maintenance fees, budgetary allocations, other stakeholder
contributions, aid flows, etc.) have been put in place to ensure the continued flow of benefits after project completion, with particular emphasis on financial
sustainability. For PBOs the assessment should focus on the financial sustainability of the reforms, as well as the Bank’s policy dialogue to promote financial
sustainability of the reforms.
Insert your evidence here
Insert your
rating here
4.3 Institutional sustainability and strengthening of capacities Rating
(1 - 6) The criterion assesses the extent to which the project has contributed to strengthen institutional capacities - including for example through the use of country systems
- that will facilitate the continued flow of benefits associated with the project. An appreciation should be made with regards to whether or not improved governance
practices or improved skills, procedures, incentives, structures, or institutional mechanisms came into effect as a result of the operation. For PBOs this should include
an assessment on the contributions made to building the capacity to lead and manage the policy reform process; as well as the extent to which the political economy
of decision making was conducive to reform, the Government’s commitment to reform and how the design reinforced national ownership
Insert your evidence here
Insert your
rating here
4.4 Political and governance environment
This criteria assesses the extent political and governance developments that could impact the government’s priorities with respect to the project. This includes (but is
not limited to) upcoming elections or an impending change in government; and other factors that could impact the political commitment to the operation or operational
engagement and the political decisions required for sustainability of project results (including laws and the provision of counterpart financing). Special attention should
be paid to fraud, corruption and other unethical practices resulting from governance failures.
Insert your evidence here
4.5 ownership and sustainability of partnerships Rating
(1 - 6) The assessment determines whether the project has effectively involved relevant stakeholders, promoted a sense of ownership amongst the beneficiaries (both men
and women) and put in place effective partnerships with relevant stakeholders (e.g. local authorities, civil society organizations, private sector, donors) as required for
the continued maintenance of the project outputs. For PBOs, the assessment should include the extent to which the Government conducted extensive consultations
during the preparation and implementation of the PRSP and the extent to which the Bank supported the Government in deepening the consultation processes
Insert your evidence here
Insert your
rating here
4.6 environmental and social sustainability Rating
(1 - 6) This criterion would normally only apply to Environmental Category I and II projects. It assesses the extent to which the environmental and social
mitigation/enhancement measures of the project were implemented, the capacity of country institutions and systems and the availabil²ity of funding to ensure the
environmental and social sustainability of the operation. The Environmental and Social Safeguards rating in the IPR should be used as a guidance.
Insert your evidence here
Insert your
rating here
4.7 Resilience to exogenous factors and risk management Rating
(1 - 6) This criteria assesses the extent to which the achievements depend on exogenous factors, such as the terms of trade, the world market prices or the political situation
in neighbouring countries.
Insert your evidence here
Overall Rating for Sustainability Insert your
rating here
Private sector projects template
Name of project Insert here
Country Insert here
Year approved Insert here
Project Reached Early Operating Maturity11 (if not, include level of disbursement) Insert here
Sector Insert here
XSR and/or XSREN available XSR: Yes/No XSREN (Yes/No)
1.1 RELEVANCE
1.1 Relevance of project objectives Rating
(1 - 6) The relevance of objectives assesses to what extend the project purpose as specified in the approval document was aligned with the relevant RMC CSP and the
applicable sector strategies, the country’s own development strategies and the beneficiary needs from design/approval to completion (including any adjustments that
were made to the project in view of changes in the applicable policy environment, such as project restructuring).
Insert your evidence here Insert rating
here
1.2 Relevance of project design to achieve project objective (Quality of front-end work and additionality)
The relevance of project design is evaluated via assessing the following:
A) ‘Screening, Appraisal and Structuring’ stage. This sub-dimension assesses how the Bank carried out its work on the project prior to commitment with
reference to the following specific aspects: i) Relevance of the investment to the Bank’s corporate, country and sector strategies (see above); ii) Identification
of risks that the investment would fail to meet the intended development objectives or generate adequate returns; iii) The sponsors, company, management,
country conditions, market dynamics, project concept, configuration and costs; iv) Financing plan, sources of financing, and assumptions used in financial
and economic projections; v) Political risks and mitigation measures; vi) Environmental and social risk assessment and action plans to mitigate adverse
effects; vii) Investment instrument selection, structure, pricing, exit mechanism, security, covenants and other terms and conditions; and viii) Client
satisfaction with the Bank’s pre-commitment work;
B) Additionality: The Bank’s additionality measures what Bank financing brings to the project over and above commercial financiers. It is based on the
counterfactual assessment of how the project would have proceeded without Bank financing. This dimension is measured through two sub-indicators:
financial additionality and non-financial additionality. The rating for additionality is a synthesis of the rating of its two underlying sub-indicators. i) Financial
Additionality measures the special contribution that the Bank’s funding offers the client that would otherwise not have been offered by other financiers which
includes; would the client have been able to obtain sufficient financing from private sources on appropriate terms? Did the Bank catalyse other funding or did
it merely fill a financing gap? Was the Bank’s financing needed to reduce risk or provide comfort thereby encouraging other financiers to invest in the
11 Refers to a point in time at which an investment operation is ready for self-evaluation. See Annex 1 for detailed description of project typology and criteria for early operating maturity by project type.
undertaking? ii) Non-Financial Additionality measures the Bank’s contribution to reducing the projects risk profile, design or functioning. The rating is
determined by considering answers to questions such as: Was the Bank needed to bring about a fair allocation of risks and responsibilities e.g., between
public and private investors? Did the Bank’s participation lead to improved design, enable the client to adopt new or better standards or contribute to the
client’s capacity building objectives through technical assistance, training, etc.?
Insert your evidence here Insert rating
here
Overall Rating for Relevance
Insert rating
here
2. EFFECTIVENESS
The assessment of Effectiveness includes accounting for the actual, expected and unintended results on outcomes level for an operation. For Lines of Credit operations, the
outcomes typically accrue on the level of the partner financial intermediary. The changes in the underlying portfolio of the financial intermediary as well as the increased efficiency
and financial deepening as a result of Bank operations. Finally, for equity/investment funds operations, it is preferred to extend the assessment to investee companies.
2.1 Achievement of outputs Rating
(1 - 6) The assessment of outputs is based on the output execution ratio. It should consider the realization of actual physical outputs of the project. Depending on the type of
project, this could be production line in expansion operations, establishment of plant and/or equipment in greenfield operations, etc. In determining the final rating,
output rating is based on the percentage of outputs (output execution ratio) that reached or are on track to meet the end of project target.
Insert your evidence here Insert rating
here
2.2 Achievement of outcomes Rating
(1 - 6) The assessment of outcome is based on the direct and intermediate outcomes stated in the retrospective project logic model. Typical outcomes of a private sector
operation covers the following areas:
i) Economic benefits; the best indicator of a non-financial market project’s contribution to economic growth is its economic rate of return (ERR) or economic return on
invested capital (EROIC). Ideally, the ERR/EROIC considers and quantifies the projects economic effects on all its economic stakeholders12. For Financial Markets
projects the economic benefits measures the extent to which the sub-projects financed with the proceeds of the line of credit or the investee companies in the case
of equity funds are providing a net economic benefit to stakeholders including and beyond the FI’s owners and financiers. Such benefits include, but are not limited
to: Contribution to government revenues resulting from taxes paid by the intermediary, sub-projects or investee companies; Contribution to poverty alleviation, social
or gender equality and regional development etc.; Delivery of community services such as entrepreneurship training, educational programs and other community
services; employment generated;
ii) Contribution to Private Sector Development; measures the extent to which the project has spread benefits of growth of productive private enterprise beyond the
12 The universe of entities impacted by a project in addition to the financiers and employees include: government, the rest of society, customers, producers of complementary products, competitors,
suppliers and neighbors.
project company, i.e. on issues such as competition, market expansion, private ownership & entrepreneurship, development of financial institutions and markets,
standards of corporate governance, transfer of technology and dispersion of skills, and the development of physical infrastructures used by other private parties.
The project can have positive or negative impacts on private sector development and it is necessary to establish that the impacts are attributable to the project.
Indicators include: Upstream and downstream supply linkages to local private businesses; introduction of new technology and know-how; enhancement of private
ownership and entrepreneurship; contribution to improving the environments for private sector development and an open economy; greater competition and
competitiveness; broad demonstration effects in the local economy and follow-on investments by other investors; domestic capital market development and greater
resource allocation efficiency; improvements in standards for corporate governance and business conduct; and development of physical infrastructures used by
other private parties.
Insert your evidence here Insert rating
here
2.2 Unintended outcomes (if any) Rating
(1 - 6) The assessment will cover all unintended outcome which came out during the project cycle
Insert your evidence here Insert rating
here
Overall Rating for Effectiveness Insert rating
here
3. EFFICIENCY
The Efficiency assessment examines the Bank investment profitability, the timeliness of Bank implementation and disbursement as well as its supervision and administration
arrangements.
3.1 Bank Investment Profitability Rating
(1 - 6) For the Bank to continue to be sustainable, the investments it makes, whether in the form of loans or equity have to be profitable. For loans: The best indicator of the
Bank’s investment (profitability) in a project is the net profit contribution (gross income less financing costs, loan loss provisions/ write-offs, transaction costs and
administrative costs measured in discounted cash flow terms. However, because of the difficulty in estimating transaction and administrative costs associated with
individual projects before the Bank implements a viable cost accounting system, a qualitative approach based on gross profit contribution (gross income less financing
costs, loan loss provisions/ write-offs) is recommended. For equity investments, profitability shall be measured by comparing the nominal internal rate of return (also
referred to as return on equity (ROE)), computed using projected dividends and
Insert your evidence here Insert
rating here
3.2 Timeliness Rating
(1 - 6) The timeliness of project implementation is based on a comparison between the planned and the actual period of implementation from the date of effectiveness. For
LOCs, the timely releases of the tranche(s) are assessed through this criterion. The following rating scale applies
Insert your evidence here Insert
rating here
3.3 Supervision and Administration Rating
(1 - 6) Supervision and administration starts after the investment agreement between the Bank and the beneficiary has been signed. The assessment indicates the extent to
which the Bank has professionally executed its responsibilities post commitment. The factors related to the Bank’s administration of the investment that must be taken
into consideration include: The monitoring of the client company’s compliance with investment covenants and conditions; the completeness of supervision reports in
documenting project implementation and risks; the monitoring of the client company’s environmental and social performance; the adequacy and timeliness of the
Bank’s response to emerging problems or opportunities; the contributions made by Bank representatives on investee company Boards; client satisfaction with the
Bank’s service quality; and the continuity of the Bank’s service delivery when monitoring staff changes occur.
Insert your evidence here Insert
rating here
Overall Rating for Efficiency Insert
rating here
4. SUSTAINABILITY
The assessment of sustainability considers the extent to which the performance of the project as a proxy for its long-term sustainability
4.1 Business Success i.e., financial performance and fulfilment of project business objectives Rating
(1 - 6) For Non-Financial Market projects, this sub-dimension measures the project’s actual and projected financial impact on the project’s financiers (lenders and equity
investors) over the economic life of the project, the project’s contribution to other business goals articulated at appraisal. The effect of the project on the company is
assessed on a “with versus without” basis, or on a “before versus after” basis. The principal indicator for business success is the financial rate of return (FRR) based
on real, after tax cash flows for project loans or the return on invested capital (ROIC) in the case of corporate investments. For financial market operations, business
success measures the effect of the LOC/investment on the FI’s profitability and contribution to other business goals.
Insert your evidence here
Insert rating
here
4.6 Environmental and Social Performance Rating
(1 - 6) The rating of environment and social performance is based primarily on the Bank’s specified standards in effect at approval and secondarily on the Bank’s
environmental and social standards prevailing at the time of evaluation. The assessment should cover i) the project’s environmental and social performance in
meeting the Bank’s requirements; and ii) the project’s actual environmental and social impacts, including pollution loads, wastes, energy and resource efficiency,
biodiversity conservation, workers’ and communities’ health and safety, public consultation and participation, land acquisition and cultural heritage. In the case of
financial intermediation projects, the assessment should consider the adequacy of the FI’s or fund manager’s Environmental and social management system and its
implementation.
Insert your evidence here Insert rating
here
Overall Rating for Sustainability Insert rating
here
Annex 8: Examples of rating guidance: Effectiveness for public and private sector projects
The following is taken from the broader guidance for project results assessment, to provide an example. Full guidance is available
covering four criteria for each of the public and private sector projects.
Public sector projects
EFFECTIVENESS
The assessment of Effectiveness tests the validity of the anticipated links between the project’s activities, outputs, and intended
outcomes (the results chain). Actual, expected and unintended results of an operation are included in the assessment of
Effectiveness. For PBOs the assessment should not only review the extent to which outputs were delivered (i.e. agreed-upon policy
reforms took place), but also the degree to which complementary measures necessary for their implementation occurred (e.g.
public awareness, policy dialogue and institutional arrangements).
a) Achievement of outputs
The assessment of outputs is based on the output execution ratio (see table below) and the quality of outputs. It should consider
the planned (targets) and actual output or those who are considered on track to be reached. In determining the final rating, no
formula based on a pre-determined weight applied to individual outputs is undertaken. If possible, select no more than 10 output
indicators in the RLF and to take into account the relative importance of the various components of the project in their selection.
Any selection need to be clearly justified.
The overall output rating is based on the percentage of outputs (output execution ratio) that reached or are on track to meet the end
of project target.
The following table demonstrates what is expected:
Major Activities Expected Outputs Actual Outputs Outputs execution rate Outputs quality
assessment
The following rating scale applies:
6 – Highly Satisfactory: Based on the output execution ratio all the project output targets were reached or are considered
on track to be reached by the end of the project in accordance with quality standards;
5 – Satisfactory: Based on the output execution ratio between 90% and 99% of the project output targets were reached or
are considered on track to be reached by the end of the project. Corrective actions for off track indicators were implemented
in a timely manner to ensure that the end of project targets could be achieved in accordance with quality standards;
4 – Moderately Satisfactory: Based on the output execution ratio between 75% and 89% of the project output targets were
reached or are considered on track to be reached by the end of the project. Corrective actions for off track indicators were
implemented in a timely manner to ensure that the end of project targets could be achieved in accordance with quality
standards;
3 – Moderately Unsatisfactory: Based on the output execution ratio between 50% and 74% of the project output targets
were reached or are considered on track to be reached by the end of the project. Corrective actions for off track indicators
were not implemented in a timely manner to ensure that the end of project targets could be achieved;
2 – Unsatisfactory: Based on the output execution ratio between 35% and 49% of the project output targets were reached
or are considered on track to be reached by the end of the project. Corrective actions were not implemented and closely
monitored for off track indicators. Poor performance jeopardized the achievement of one or more outcomes of the project;
1 – Highly Unsatisfactory: Based on the output execution ratio less than 35% of the project output targets were reached or
are considered on track to be reached by the end of the project. Poor performance jeopardized the achievement of most
expected outcomes and the possibility of stopping or suspending the project was considered.
b) Achievement of outcomes
Outcomes are assessed against the project’s objectives as contained in the Project Appraisal Report (PAR). The assessment of
outcome is based on the direct and intermediate outcomes stated in the retrospective project logic model. If the statement of project
objectives in the appraisal documents is unclear or is focused on outputs rather than outcomes, the evaluator reconstructs an
outcome-oriented statement of objectives using the project’s results chain, performance indicators and targets, and other
information including country strategies and interviews with government officials and AfDB staff. The anticipated links between the
project’s activities, outputs, and intended outcomes are summarized in the project’s results chain. The results chain is taken from
the PAR. If the results chain is absent or poorly defined, the evaluator constructs a retrospective results chain from the project’s
objectives, components, and key performance indicators.
The evaluator should make sure that data collection remains open to unintended results that have not anticipated by including
some open-ended questions in interviews and questionnaires, and by encouraging reporting of unexpected results. Unexpected
benefits, once known about, can be designed into future interventions.
The ECG's 'Big Book on Evaluation Good Practice Standards'13 recommends the following approach in this area:
1. Assess the causal chain in relation to the needs of the target population, collaborating with stakeholders and experts;
2. Examine the critical assumptions and expectations inherent in the project’s design, reviewing the logic and plausibility of the results chain.
Again, this is done in collaboration with stakeholders;
3. Use available research evidence and practical experience elsewhere, comparing the project with projects based on similar concepts;
4. Observe the project in operation, focusing on interactions that were expected to produce the intended outcomes.
For PBOs the assessment should not only review the extent to which outputs were delivered (i.e. agreed-upon policy reforms took
place), but also the degree to which complementary measures necessary for their implementation occurred (e.g. public awareness,
policy dialogue and institutional arrangements). Since PBOs are typically joint with other donors and RMC governments and are
implemented through country systems, it may be difficult to attribute a direct link between the specific inputs of the Bank Group
(and those of other partners) and the expected results. Therefore, the progress will be measured in terms of the collective efforts of
the RMC and other partners, where applicable, while taking into account other external factors.
The following rating scale applies:
6 – Highly Satisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant
exogenous risks/factors and assumptions, it is plausible to expect that all intended project outcomes were achieved or are
likely to be achieved;
5 – Satisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant
exogenous risks/factors and assumptions, it is plausible to expect that most (75%) intended project outcomes were
achieved or are likely to be achieved;
4 – Moderately Satisfactory: Taking into account the latest value of the outcome indicators and the analysis of other
relevant exogenous risks/factors and assumptions, it is plausible to expect that a substantial (50%-74%) intended project
outcomes were achieved or are likely to be achieved;
3 – Moderately Unsatisfactory: Taking into account the latest value of the outcome indicators and the analysis of other
relevant exogenous risks/factors and assumptions, it is plausible to expect that few (25-49%) intended project outcomes
were achieved or are likely to be achieved;
13 https://www.ecgnet.org/document/ecg-big-book-good-practice-standards.
2 –Unsatisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant
exogenous risks/factors and assumptions, it is plausible to expect that few (5-24%) intended project outcomes were
achieved or are likely to be achieved;
1 – Highly Unsatisfactory: Taking into account the latest value of the outcome indicators and the analysis of other relevant
exogenous risks/factors and assumptions, it is plausible to expect that very few (<5%) of the intended project outcomes
were achieved or are likely to be achieved.
Private sector projects
EFFECTIVENESS
The assessment of Effectiveness includes accounting for the actual, expected and unintended results on outcomes level for an
operation. For Lines of Credit operations, the outcomes typically accrue on the level of the partner financial intermediary.
Measurement of changes in the underlying portfolio of the financial intermediary as well as the increased efficiency and financial
deepening as a result of Bank operations would represent desirable outcomes. Finally, for equity/investment funds operations, it is
preferred to extend the assessment to investee companies.
a) Achievement of outputs
The assessment of outputs is based on the output execution ratio. It should consider the realization of actual physical outputs of the
project. Depending on the type of project, this could be production line in expansion operations, establishment of plant and/or
equipment in Greenfield operations, etc. In determining the final rating, output rating is based on the percentage of outputs (output
execution ratio) that reached or are on track to meet the end of project implementation targets.
The following rating scale applies:
6 – Highly Satisfactory: Based on the output execution ratio all the project output targets were reached or are considered on track
to be reached by the end of the project in accordance with quality standards.
5 – Satisfactory: Based on the output execution ratio between 90% and 99% of the project output targets were reached or are
considered on track to be reached by the end of the project. Corrective actions for off track indicators were implemented in a timely
manner to ensure that the end of project targets could be achieved in accordance with quality standards.
4 – Moderately Satisfactory: Based on the output execution ratio between 75% and 89% of the project output targets were
reached or are considered on track to be reached by the end of the project. Corrective actions for off track indicators were
implemented in a timely manner to ensure that the end of project targets could be achieved in accordance with quality standards.
3 – Moderately Unsatisfactory: Based on the output execution ratio between 50% and 74% of the project output targets were
reached or are considered on track to be reached by the end of the project. Corrective actions for off track indicators were not
implemented in a timely manner to ensure that the end of project targets could be achieved.
2 – Unsatisfactory: Based on the output execution ratio between 35% and 49% of the project output targets were reached or are
considered on track to be reached by the end of the project. Corrective actions were not implemented and closely monitored for off
track indicators. Poor performance jeopardized the achievement of one or more outcomes of the project.
1 – Highly Unsatisfactory: Based on the output execution ratio less than 35% of the project output targets were reached or are
considered on track to be reached by the end of the project. Poor performance jeopardized the achievement of most expected
outcomes and the possibility of stopping or suspending the project was considered.
b) Achievement of outcomes
The assessment of outcome is based on the direct and intermediate outcomes stated in the retrospective project logic model.
Typical outcomes of a private sector operation cover the following areas:
i) Economic benefits; the best indicator of a non-financial market project’s contribution to economic growth is its economic
rate of return (ERR) or economic return on invested capital (EROIC). Ideally, the ERR/EROIC considers and quantifies the projects
economic effects on all its economic stakeholders . For Financial Markets projects the economic benefits measures the extent to
which the sub-projects financed with the proceeds of the line of credit or the investee companies in the case of equity funds are
providing a net economic benefit to stakeholders including and beyond the FI’s owners and financiers. Such benefits include, but
are not limited to: Contribution to government revenues resulting from taxes paid by the intermediary, sub-projects or investee
companies; contribution to poverty alleviation, social or gender equality and regional development etc.; delivery of community
services such as entrepreneurship training, educational programs and other community services; employment generated.
The following rating scale applies:
6 – Highly Satisfactory: Both: (i) the project has succeeded in reaching targeted groups of sub-borrowers; and (ii) there is direct
evidence (from sub-portfolio data) that sub-borrowers have made strong economic contributions, or indirect evidence (from market
data) that market sectors supported by the project and/or more generally by the financial intermediary are major economic
contributors to society.
5 – Satisfactory: Both: (i) the project has succeeded in reaching targeted groups of sub-borrower; and (ii) there is direct evidence
(from sub-portfolio data) that sub-borrowers are economically viable, or indirect evidence (from market data) that market sectors
supported by the project and/or more generally by the financial intermediary are economically viable and do not rely on economic
distortions to maintain their commercial viability.
4 – Moderately Satisfactory: Both: (i) the project has succeeded in reaching targeted groups of sub-borrower; but (ii) there is no
evidence that sub-borrowers do not rely on economic distortions to maintain their commercial viability.
3 – Moderately Unsatisfactory: (i) the project has largely failed to reach targeted groups of sub-borrower; however, there is
evidence that sub-projects are economically viable.
2 –Unsatisfactory: Either: (i) the project has largely failed to reach targeted groups of sub-borrower; or (ii) there is direct evidence
(from sub-portfolio data) that most sub-borrowers are not economically viable, or indirect evidence (from market data) that market
sectors supported by the project and/or more generally by the financial intermediary are weak economic contributors to society.
1 – Highly Unsatisfactory: LoC: Both: (i) the project has largely failed to reach targeted groups of sub-borrower; and (ii) there is
direct evidence (from sub-portfolio data) that most sub-borrowers are not economically viable, or indirect evidence (from market
data) that market sectors supported by the project and/or more generally by the financial intermediary are weak economic
contributors to society.
ii) Contribution to Private Sector Development; measures the extent to which the project has spread benefits of growth of
productive private enterprise beyond the project company, i.e. on issues such as competition, market expansion, private ownership
& entrepreneurship, development of financial institutions and markets, standards of corporate governance, transfer of technology
and dispersion of skills, and the development of physical infrastructures used by other private parties. The project can have positive
or negative impacts on private sector development and it is necessary to establish that the impacts are attributable to the project.
Indicators include: Upstream and downstream supply linkages to local private businesses; introduction of new technology and
know-how; enhancement of private ownership and entrepreneurship; contribution to improving the environments for private sector
development and an open economy; greater competition and competitiveness; broad demonstration effects in the local economy
and follow-on investments by other investors; domestic capital market development and greater resource allocation efficiency;
improvements in standards for corporate governance and business conduct; and development of physical infrastructures used by
other private parties.
The following rating scale applies:
6 – Highly Satisfactory: Considering its size, the project had: (a) substantial positive effects consistent with the Bank’s mandate
objectives of promoting economic development and poverty reduction by furthering the country’s private sector development or the
development of efficient financial / capital markets; and (b) no negative impacts in this respect.
5 – Satisfactory: The project had: (a) demonstrable effects consistent with the Bank’s mandate objectives of promoting economic
development and poverty reduction by furthering private sector development or the development of efficient financial / capital
markets; and (b) a clear preponderance of sustainable positive impacts in this respect.
4 – Moderately Satisfactory: The project had: (a) demonstrable positive effects consistent with the Bank’s mandate objectives of
promoting economic development and poverty reduction by furthering private sector development or the development of efficient
financial / capital markets; However, there is lack of evidence on the sustainability of such effects.
3 – Moderately Unsatisfactory: The project made no discernible contribution, either positive or negative as supported by available
evidence.
2 –Unsatisfactory: The project had mainly negative effects in respect of the Bank’s mandate objectives of promoting economic
development and poverty reduction by furthering private sector development or the development of efficient financial / capital
markets, but these negative effects are not expected to be of long duration or broad applicability.
1 – Highly Unsatisfactory: The project had substantial negative effects in respect of the Bank’s mandate objectives of promoting
economic development and poverty reduction by furthering private sector development or the development of efficient financial /
capital markets, and these impacts are likely to be widespread, of long duration, or both.
Annex 9: Example of project ToC and indicators identified for collection
An example of a project LoC has been taken from the South Africa evaluation. The figure below presents a Theory of Change for
the Medupi power project.
Project Theory of Change – Medupi Power Plant Project
67
Annex 10: Examining lines of credit
First, this Annex will explain what LoCs are and how data is collected for their evaluation at the
strategic level. Second, it explains how a theory of Change (ToC of LoC) can be applied at his
evaluation level.
Definition
LoCs are a method of reaching SMEs and other enterprises that AfDB could either not reach
directly or which would be prohibitively expensive to lend to directly. They involve loans by AfDB to
financial intermediaries (FIs, primarily banks14) that are on-lent by the FIs to SMEs and other
defined categories of beneficiaries. AfDB’s credit risk is with the FI. LoCs in Africa have traditionally
been denominated in US dollars but in recent years there has been a growth in local currency LoCs
in countries where AfDB can raise local currency funding, including both Nigeria and South Africa.
Other objectives for LoCs include:
Strengthening financial intermediaries so that they can better serve SMEs and other
beneficiaries that they are either not supporting or only in a limited way. This may involve the
provision of technical assistance to FIs alongside LoCs.
Improving FI’s access to medium and long term funding that enables them to extend the tenor
of their lending.
Financial and private sector development, including increased access to credit.
Preliminary overview of AfDB LoCs in Nigeria
The table below summarizes the key features of the LoCs in the Nigerian portfolio, divided between
PRA projects and non-PRA projects.
Table 0.6: Overview LoCs
Bank/FI Approve
d
Tenor Focus Sub
Project
Amount
($m)
No of Sub-
loans
Max
($m
)
Min
($m
)
Averag
e
Year
s
Defined
?
$m $m $m
PRAs
Access
Bank 1
2007 General (?) unclear 35 9 12.
7
1.1 3.9
Access
Bank 2
2014 10 SMEs Yes 100 -
GTB 1 2006 7 General No 40 6 20.
4
0.3 6.7
GTB 2 2010 7 General No 100 4 32.
5
7.1 20.1
NEXIM
Bank
2011 -
UBA ELF 2009 5 SMEs,
infrastrucur
e
No 50 4 27.
2
2.8 12.5
Zenith 1 2005 5 Exporters No 70 14 10.
6
1.3 6.1
14 Loans can also be made to micro-finance institutions/banks for on-lending to micro enterprises.
Zenith 2 2006 7 General No 100 11 31 0.5 9.1
Zenith ELF 2009 5 General No 50 12 20.
0
3.0 9.9
Zenith 3 2014 7 SMEs Yes 125
Other Nigerian LoCs
Bank of
Industry
2011
Developme
nt Bank of
Nigeria
2014 400
Fidelity
Bank
2012 75 or
150?
Stanbic
IBTC Bank
2014 7 SMEs and
renewable
energy
100
Africa
Finance
Corporatio
n - regional
2012 10 Infrastructu
re and
industry
By loan
size
200 4 30 25 28.8
Assessing development outcomes
LoCs are provided by the development institution (the AfDB) to a financial intermediary (e.g. a
commercial bank) that subsequently finances subprojects (e.g. SMEs). In practice, it is not always
possible to trace back the origin of funded projects by financial intermediaries and hence of the
development outcomes of the LoCs provide by the AfDB. According to the AfDB this is especially
the case for LOCs that are denominated in foreign currency while the actual lending takes place in
local currency. Difficulties to retrieve information on development outcomes in these cases arise
from money flows and reporting on this by the financial intermediaries.
In Nigeria, LoCs are predominantly provided by the AfDB in US Dollars and on-lending takes place
in US Dollars to designated clients with businesses with foreign exchange accounts. In this way a
clear link between AfDB funding and final beneficiaries can be established, especially if the rates
and tenure periods offered by the AfDB are passed on to clients or if long-term maturities offered by
the AfDB enabled the financial intermediary to have a better ability to lend long-term.
Nevertheless, the assessment of development outcomes of LoCs should take into account the
potential difficulties by differentiating between the levels at which analysis is possible. The ToC for
the LoCs (cf. below) shows the focus of these analyses at different levels. Whereas the analysis of
the level of financial intermediaries focuses mainly on outputs, in particular on the lending capacity
and behavior of financial intermediaries, the analysis at the level of sub borrowers focuses on
intermediate outcomes such as job creation and increased access to finance by SMEs.
69
Figure 0.1 Theory of Change for a line of credit
71
The level at which analysis can take place depends to a significant extent on the information
available and its quality. Below, we have investigated which information is potentially available
through the AfDB.
Project life-cycle documentation
To monitor how its funding is used and to measure development outcomes of the ultimate
beneficiaries of LOCs, the AfDB applies various processes and procedures that are included in the
project life-cycle and summarized in the figure below and further elaborated below the figure.
Figure 0.2 LoC project life-cycle
1. The AfDB Project Appraisal Reports (PARs) for LoCs contain sections on the expected/target
development effects together with Results Based Logical Frameworks (Logframes, in older
LoCs referred to as an Economic and Development Impact Matrix) that set out the results
chains.
2. Loan Agreements (LAs) with banks/FIs set out, inter alia, (i) the purpose of the LoC, (ii) a more
detailed schedule “Objectives and Description” and (iii) a list of the reporting requirements
covering both the bank/FI and the sub-projects. It should be noted that in a sample of LAs for
South African and Nigerian banks/FIs seen by Ecorys, the LAs do not include the detailed
development objectives/goals specified in PAR logframes.
3. Sub-Project Reports that should be provided by banks/FIs on a semi-annual basis (lists of
sub-projects) and annually the “development outcomes” achieved for each project according to
templates in the LAs.
4. Supervision Documents are prepared by the AfDB portfolio division. They are based on the
reports provided by banks/FIs and information gathered during supervision visits to banks/FIs
and a sample of sub-projects. The comprise bank to office reports (BTOs) and the more formal
project supervision report (PSRs).
5. Expanded Supervision Reports (XSRs) are in depth reports that are prepared for mature
projects and assess, inter alia, the development outcomes of the portfolio of sub-projects
financed by LoCs.
Proposed Approach to Assessing LoC Sub-Project Development Outcomes
For each LoC that is studied in-depth (PRA) the sub-project evaluation activities will include, but not
necessarily be limited to:
Prior to Visit to the bank/FI
1. Extraction of logframes and related information in PARs.
2. Comparison of logframes with development outcomes and reporting requirements included in
LAs: (i) the purpose of the LoC, (ii) a more detailed schedule “Objectives and Description” and
(iii) a list of the reporting requirements covering both the bank/FI and the sub-projects.
3. Assessment of the appropriateness of the development indicators contained in both logframes
and LAs. It is of note that IFC’s DOTS systems lists the following essential characteristics of an
adequate indicator known as the “RATE” scale:
Relevant - good indicators must capture the essence of the intended result.
Aggregatable – in order to assess, compare and report development results of all AfDB
supported projects, results from individual projects need to be aggregated.
Time-bound and targeted, i.e. by when and by how much a certain result is expected to be
achieved.
4. Assess whether and how the LoCs score on the development indicators is used in the Financial
Sector Development Strategy. Since all of the LoCs were approved before this Strategy, this
step is primarily to see whether LoCs developed in a similar fashion as foreseen in the Strategy.
5. Verify the most up-to-date, definitive list of sub-projects provided by the bank/FI. This is
important because some LoCs seem to be conflicting with information in the supervision
documents (BTOs and PSRs) with different sub-project lists in the files. In addition, for some
projects the aggregate amount of sub-loans exceeds the amount of the LoC. Also, some sub-
loans may have tenures exceeding those of the LoCs.
6. Reviewing/analysing the most up to date development outcome/impact reports provided by the
bank/FI
7. Review of PSRs and where they have been prepared, XSRs and a comparison with information
provided by banks/FIs as well as other thematic evaluations that might have covered these LoC
operations, for example the SME Evaluation, or the Microfinance evaluation.
8. Preparation of sub-project portfolio specific questions for discussions at meetings with bank/FI.
Visit to the bank/FI
9. Discussion of AfDB logframe. The purpose is to understand how involved the bank/FI was in the
logframe preparation.
10. Preparation of sub-project portfolio specific questions for discussions at meetings with bank/FI.
11. On a sub-project by sub-project basis validation of the development outcomes data reported to
AfDB. In Nigeria, visits will be made to a sample of sub-projects for the bank’s LoCs. During
these visits the tangible, on-the-ground development outcomes will be assessed and compared
with those reported by the banks to AfDB
.
73
Annex 11: Examining the knowledge aspect of the Country Strategy Papers and Programs
Knowledge and policy products (non-lending activities)
Evaluation Questions EQ15 and EQ16, labelled under the category “Knowledge and policy advice”
are part of the broader block of evaluation questions aimed at assessing the Management of the
Bank’s interventions”. Even though currently no significant AfDB project on knowledge and policy
advice is implemented in Nigeria, knowledge sharing constitutes an important part of the CSP
2013-17.
These two questions are also relevant in the light of the recent approval of a Knowledge
Management Strategy (KMS) for the period 2015-2020 since the vision set therein is for the AfDB to
become the premier knowledge institution in Africa in the areas of its mandate.
In particular, the two evaluation questions considered are tightly linked to the second pillar of the
KMS, “Strengthening the quality of the institution’s policy dialogue, advisory services, and
involvement in the development debate”. Indeed, as per the Terms of Reference, the CSPEs will
answer have to assess:
EQ15: To what extent has the Bank actively engaged in and influenced policy dialogue through
relevant advice? and
EQ16: To what extent has the Bank delivered adequate analytical work in support of its
interventions, positioning and policy advice?
Before describing how each question will be addressed, it is important to stress that themes linked
to knowledge management and policy dialogue have been gradually inserted in the Bank’s country
and regional strategies over the period covered by this evaluation and mostly after 2009.
EQ15. To what extent has the Bank actively engaged in and influenced policy dialogue through
relevant advice?
The level of analysis required by this question is programmatic. The examination will require a
distinction between the Bank’s engagement and the Bank’s influence of the policy dialogue.
Active engagement in policy dialogue can be measured through the participation of the Bank to
relevant initiatives occurred in the period 2004-2014 as well as from the production of studies,
papers or forecasts tailored on Nigeria and the region. The presence and the activities of the Bank’s
Country Office can also be examined through semi-structured (telephone) interviews and
documentary reviews. The number of events organized or hosted by the Bank in the country or
focusing on the country will give an overview of how continuous and deep the engagement of the
Bank has been.
Additionally, the overall level of engagement may be examined by contrasting the Bank’s
knowledge products with similar products of comparable development institutions (e.g. World Bank)
to be able to put it in the appropriate relative terms. A brief benchmarking exercise will allow the
evaluators to identify the niches in which the Bank operates or may operate and will feed in the
assessment of the value added.
The capacity of influencing policy dialogue is less straightforward to assess. A review and
synthesis of the reforms and best practices spearheaded by the Bank will provide the base for the
analysis of the policy evolution in the country concerned, but cannot alone provide a direct link
between the activity of the Bank and the reforms carried out in the country. An assessment of the
role of the other donors and (telephone) interviews with relevant stakeholders will prove necessary
to assess the level of influence the Bank has had.
EQ16. To what extent has the Bank delivered adequate analytical work in support of its
interventions, positioning and policy advice?
This question will require an analysis spread over all the three layers of analysis at program, project
and portfolio-level. Part of the documentary review, analysis and synthesis produced in addressing
EQ15 can feed in EQ16, although once again the mere presence of analytical work does not say
much about its inherent adequacy.
Comparative analysis with similar documents produced by other development institutions for the
same typology of projects or for co-financed projects can provide insights on the quality of the
Bank’s analytical support to its interventions. The same exercise can be done at the program level,
comparing different CSPs, Country Analysis, Country Sector Reviews with similar documents
produced by other development institutions. Such a comparison will highlight eventual lacks but
also strengths of the Bank’s analytical approach.
Also the timelines and regularity in the production of the documentation can be a useful indicator to
assess the capacity and the willingness of the Bank to produce knowledge content relating in
particular to the CSP’s “Pillars”.
These findings have then to be confirmed by interviews with relevant stakeholders during the field
mission and additional telephone interviews with the producers of the analytical work. The question
will therefore be answered through a double-layered analysis with a first step centered on desk
research and a second one based on interviews.
It is noted that the mode of communicating and distributing analytical work of the Bank may affect
the responses of the users. This aspect will, however, not be examined in the scope of this
evaluation.
75
Annex 12: Interview protocol
The interviews will follow as much as possible the key evaluation questions. Given the restricted
time for each interview the questions will be related to data and information gaps which will be
identified on the basis of the document review. Before each field mission the following procedure
will be followed:
Interview protocol
1) Based on the document review, a list of pre-interview questions for each sector will be prepared and
shared with IDEV;
2) A list of interview questions will be sent to the interviewees in advance (together with a short description of
the purposes of the evaluation) – see below for list of general questions. The main purpose of these
questions is merely to ‘refresh’ interviewee’s memory prior to the interview, so the interviewee can prepare
himself/herself and understands the main focus of the interview. It is noted that only certain evaluation
questions will be asked to the potential interviewees, depending on their knowledge of the projects and
their involvement. In a few cases interviewees may not know about the projects of the Bank, but has
profound knowledge on the South African economy and respective sector which may provide valuable
insights for the evaluation;
3) The interviewee will be assured of the confidential nature of the interview;
4) The main focus of the interview will be the interview questions. Nonetheless it will be useful to reiterate
shortly the objectives of the evaluation and provide opportunities for feedback and discussion;
5) If needed, the context/ background of AfDB activities in South Africa will be discussed as well as the
institutional setting of the sector and the role of the organisation of the interviewee therein;
6) The main focus will be on the main challenges and success factors encountered when implementing AfDB
projects (using the evaluation questions);
7) Discuss the availability of documents, timeline for obtaining these documents; additional sources; and if
relevant potential other interviewees for data collection;
8) Interviewees of financial institutions will be asked their support in conducting the survey among
beneficiaries of Lines of Credit, the questions included in the survey and to discuss the approach of
undertaking the survey (i.e. ideally by requesting the banks to send the survey request to the
beneficiaries);
9) The interviewee will be informed about the follow-up steps of the evaluation
10) For each interview minutes will be prepared and shared with the interviewee for check on factual errors
and omissions;
11) The data and information in the interview minutes will be coded to enable structured analysis;
12) If possible some follow-up questions and clarifications by telephone may take place, depending on the
availability and preparedness of the interviewed persons.
77
Annex 13: List of key informants for semi-structured interviews
On the basis of the information available to us we developed an initial stakeholder overview. In the
tables below we provide an overview of the key stakeholders of the Bank’s public and private sector
projects disaggregated by sector.
Public sector projects stakeholder mapping
Sector Federal Level State Level Province/Local body Level Donors Private Sector
Agriculture
Office of the Chief Economic Adviser to the President
Federal Ministry of Agriculture and Rural Development (FMARD):
12 departments and one unit, namely: Department of Agriculture; Department of
Rural Development; Department of Planning, Research and Statistics; Department of
Agricultural Science; Department of Livestock and Pest Control Services;
Department of Cooperatives Department of Storage and Strategic Grain Reserve;
Department of Fertilizer; Department of Land Resources; Department of Fisheries;
Department of Finance; Department of Administration and Supplies; and the Project
Coordinating Unit.
The Federal Department of Planning, Research and Statistics (FDPRS)
Agricultural and Rural Management Training Institute (ARMTI)
The Nigerian Institute of Social and Economic Research (NISER)
State Ministry of Agriculture (SMOA), Agriculture Development, Livestock and Veterinary
Services, Engineering Department, Fisheries Department and Forestry Department. Some
states in addition, have the Cooperative Development Department and the Produce Grading,
Inspection and Pest Control Department.
State Agricultural Transformation Implementation Committee (SATIC)
Department of Integrated Rural Development, which promotes rural infrastructure, and
Community Based Organizations (CBO)
Public institutions supporting agriculture and rural development at state level include:
(i) Technical Service Support, provided by the Ministry of Agriculture (MoA), Agricultural
Development Authorities (ADA), Agricultural Development Programmes (ADP), Ministry of Water
Resources and Rural Water Development, Ministry of Commerce, Industry and Cooperatives,
Ministry of Works, Housing and Transport);
(ii) Socio- Economic Support, provided by Ministry of Health, Ministry of Women Affairs and
Social Development, Ministry of Education;
(iii) Sector Planning, Monitoring and Evaluation, provided by Ministry of Local Government or
Chieftaincy Affairs, State Planning Commission or Ministry of Economic Development or
Planning; and
(iv) Commercial/Financial Services
Local Government Councils
(LGC)
Local Government Area
(LGA)
Environment NDCC (Niger Delta Development Commission)
Shell Petroleum Development
Company
Niger Delta Environment Survey
Finance Federal Ministry of Finance
Multi-SectorFederal Ministry of Finance
Sectoral Ministries WHO-AFRO
PowerFederal Ministry of Finance
Federal Ministry of Power (FMP)
Nigeria Bulk Electircity Trading
SocialFederal Ministry of Finance
Federal Ministry of Education: Science and Technology Department
Safe School Initiative (SSI) Secretariat
WHO-AFRO
Transport Cross River State Ministry of Works
Water Sup SanFederal Ministry of Agricultural and Water Resources (FMAWR)
Federal Ministry of Environment
Yobe Basin Trust Fund of Nigeria
Rural Water Supply and Sanitation Agency (RWASSA) is the executing Agency in Yobe State
Osun Rural Water Supply and Environmental Sanitation Agency (RWESA) is the executing
agency in Osun State
Water Corporation of Oyo State
Taraba State Water Supply Agency
Kaduna State Water Board
State Mnistry of Water Resources
State Ministry of Work
State Water Board
Local Government Councils
(LGC)
Local Government Area
(LGA)
Private sector projects stakeholder mapping
Sector Federal Level State Level Province/Local body Level Donors Private Sector
Agriculture Cross River State Forestry Commission Local Council Group ECOWAS Bank for Investment and Development (USD
20 million), the Islamic Corporation for the
Development of the Private Sector (USD 30 million)
and a set of local commercial banks (USD 40 million). In
addition to the sponsor’s 23% contribution to equity,
additional equity is expected to be provided by DEG,
Finfund, PROPARCO, Emerging Capital Partners and
one individual investor.m
Negris Group Ltd
Accugas Ltd
Finance Central Bank of Nigeria
Federal Ministry of Finance
Federal Government of Nigeria
Asset Managent Company of Nigeria (AMCON)
Nigeria Deposit and Insurance Corporation
National Drug Law Enforcement Agency (NDLEA)
Economic and Financial Crime Commission (EFCC)
NIgerian Financial Intelligence Unit (NFIU)
Small and Medium Enterprises Development Agency of
Nigeria (SMEDAN)
Belgian Investment Company for Developing Countries
(BIO)
European Bank for Reconstruction and Development
(EBRD)
Netherlands Development Finance Company (FMO)
Kreditanstalt für Wiederaufbau (KfW)
International Financial Cooperation (IFC)
Fund for the African Private Sector Assistanct (FAPA)
Development finance institutions (“DFIs”): Bank of
Agriculture (“BOA”), the Bank of Industry (“BOI”), the
Federal Mortgage Bank (“FMB”), the Nigerian Export-
Import Bank (“NEXIM”) and the Infrastructure Bank
(“IB”)
Banks: United Bank of Africa, First Bank of Nigeria,
Access Bank, Zenith Bank, ETI, Union Bank of Nigeria.
Power River State Government Host Communities (NGO) Belgian Investment Company for Developing Countries
(BIO)
Netherlands Development Finance Company (FMO)
International Financial Cooperation (IFC)
Kreditanstalt für Wiederaufbau (KfW)
DFID (through the CDC)
Gas Aggrecgation Company Nigeria Ltd
Transport Federal Government of Nigeria
Federal Ministry of Environment
Security and Exchange Commission of Nigeria
Nigerian Corporate Affairs Commission (through Larue
Ltd)
Lagos State Government Asset and Resource Management Company
Africa Infrastructure Investment Fund (Macquarie
Bank)
HITECH
81
Annex 14: Document mapping
Based on the document overview send by the Bank we developed a simplified overview of available project documents. The overview of non-project documents
has been adopted from the Bank’s overview.
Mapping of available project documents
Public sector documents
Sector Project Name Project Reference Funding Status PREPARATION DOCUMENTS IMPLEMENTATION DOCUMENTS COMPLETION DOCUMENTS
Agriculture MIC GRANT ASSET MAPPING OF ECONOMIC OPPORTUNITIES IN NIGERIA P-NG-A00-007 MIC GRANT APVD P
Agriculture SUPPORT TO AGRIC. & RURAL INSTITUTIONS P-NG-AA0-026 ADF COMP P
Agriculture NATIONAL PROGRAMME FOR FOOD SECURITY P-NG-AA0-027 ADF COMP P
Agriculture AGRICULTURAL TRANSFORMATION AGENDA SUPPO P-NG-AAB-003 ADF APVD P
Agriculture SUPPORT COMBATING AVIAN INFLUENZA 2006 P-NG-AAE-002 SRF FUND COMP P
Agriculture INVASIVE AQUATIC WEEDS - NIGERIA P-Z1-AA0-084 ADF COMP P
Environment NIGER DELTA SOCIAL AND ENVIRONMENT STUDY P-NG-C00-001 ADF CLSD P
Finance TRADE MISPRICING THE HIDDEN DRAINAGE P-NG-HB0-006 MIC GRANT OnGo P
Multi-Sector TRANSPORT SECTOR AND ECONOMIC GOVERNANCE P-NG-K00-005 ADB LOAN OnGo P P
Multi-Sector CAPACITY DEVT PROGRAM FOR NASS ON MICTAF P-NG-KF0-002 MIC GRANT COMP P No report yet No report yet: due by October 2015
Power ECONOMIC AND POWER SECTOR REFORM PROGRAM P-NG-FA0-002 ADF LOAN OnGo P P Aide Memoire OK
Power PRG I N SUPPORT OF POWER SECTOR P-NG-FA0-006 ADF LOAN APVD P
Social SKILLS TRAINING AND VOCATIONAL EDUCATION P-NG-IA0-001 ADF OnGo P P Aide Memoire OK
Social EMERG. ASSIST. TO CH ***** P-NG-IA0-003 SRF FUND APVD P Disborsement effectiveness
Social AIDE MULTINAT URG ERADIC POLIOMYELITE P-NG-IBE-001 SRF FUND COMP P
Social EMERGENCY ASSIST EBOLA P-NG-IBE-002 SRF FUND APVD P P
Transport RURAL ACCESS & MOBILITY PROJECT P-NG-DB0-005 ADF LOAN OnGo P Aide Memoire OK
Water Sup SanKOMADUGU-YOBE BASIN STRATEGIC DEVT PLAN P-NG-EAZ-002 AWF Grant APVD P
Water Sup SanRURAL WATER & SAN SUB-PROG (YOBE & OSUN) P-NG-E00-002 RWSSI Loan OnGo P To be sentMission Aide memoires (PCR
supervision) to be sent
Water Sup SanURBAN WATER & SAN IMPROVEMENT PROJECT P-NG-E00-004 ADF LOAN OnGo P PMission Aide memoires (PCR
supervision) to be sent
Water Sup SanZARIA WATER EXPANSION & SAN PROJ, P-NG-E00-005 ADF LOAN OnGo P PMission Aide memoires (PCR
supervision) to be sent
Water Sup SanURBAN WATER REFORM & PORT HARCOURT WSSP P-NG-E00-007 ADB LOAN APVD P
Private sector documents
Sector Project Name Project Reference Funding Status PREPARATION DOCUMENTSIMPLEMENTATION DOCUMENTSCOMPLETION DOCUMENTS
Agriculture OKIPP P-NG-AAG-002 ADB APVD P P
Finance FIDELITY BANK LOC P-NG-HA0-005 ADB/ LOC OnGo P P
Finance BANK OF INDUSTRY - BO P-NG-HAA-002 ADB LOAN APVD P
Finance NIGERIAN EXPORT IMPORT BANK - NEXIM P-NG-HAA-003 ADB LOAN OnGo P
Finance STANBIC IBTC BANK PLC P-NG-HAB-016 ADB LOAN APVD P P
Finance STANBIC IBTC BANK PLC P-NG-HAB-016 CLEAN TECH FUND APVD P P
Finance ZENITH BANK PLC - LOC III P-NG-HAB-017 ADB/LOC APVD P P
Finance ACCESS BANK NIGERIA LOC II P-NG-HAB-019 ADB/LOC APVD P P
Finance FRB SUBSIDIARY IN NIGERIA P-NG-HAB-022 ADB/LOC OnGo
Finance LAPO MICROFINANCE BANK LTD. P-NG-HB0-005 ADB LOAN APVD P P
Finance ABN MICROFINANCE TA GRANT P-NG-HA0-003 Priv Asst Fund/GRANT OnGo P P
Finance LINE OF CREDIT TO ZENITH BANK PLC P-NG-HAB-004 ADB/LOC CLSD P P P
Finance HELIOS TOWERS ADB/LOC OnGo
Finance LOC TO GUARANTEE TRUST BANK P-NG-HAB-005 ADB/LOC CLSD P P P
Finance ZENITH BANK LOC II P-NG-HAB-006 ADB/LOC OnGo P P P
Finance LINE OF CREDIT TO ACCESS BANK PLC P-NG-HAB-007 ADB/LOC OnGo
Finance LINE OF CREDIT II TO GUARANTY TUST BANK P-NG-HAB-009 ADB/LOC OnGo P P
Finance ZENITH EMERGENCY LIQUIDITY FACILITY P-NG-HAB-013 ADB/LOC OnGo P P P
Finance UBA EMERGENCY LIQUIDITY FACILITY P-NG-HAB-014 ADB/LOC CLSD P P P
Finance UBA TRADE FINANCE INITIATIVE P-NG-HAB-015 ADB/LOC CLSD P
Finance DEVELOPMENT BANK OF NIGERIA(DBN) P-NG-HAA-004 EQUITY APVD P P
Finance AFRICA FINANCE CORPORATION(AFC) ADB/LOC OnGo P P
Power INDORAMA FERTILIZER P-NG-FD0-002 ADB LOAN OnGo P P
Transport LEKKI TOLL ROAD PROJECT P-NG-DB0-008 ADB LOAN(PPP) OnGo P
83
Mapping of available non-project documents
AfDB Documents
DOCUMENT TITLE AVAILABILITY
THE ONEBANK GROUP RESULT MEASUREMENT FRAMEWORK 2013-2016 x
INTEGRATED SAFEGUARD SYSTEM POLICY AND OPERATIONAL SAFEGUARDS x
BANK GROUPGENDER POLICY x
BANK GROUP TRANSPORT SECTOR POLICY x
BANK GROUP TRADE FINANCE PROGRAM BUSINESS PLAN x
BANK GROUP OPERATIONS MANUAL x
BANK GROUP GENDER STRATEGY 2014-2018 x
BANK GROUP FINANCIAL SECTOR DEVELOPMENT & STRATEGY x
BANK GROUP GENDER STRATEGY 2014-2018 x
BANK GROUP FINANCIAL SECTOR DEVELOPMENT & STRATEGY x
BANK GROUP ENERGY POLICY x
BANK GROUP AGRICULTURE SECTOR STRATEGY 2010-2014 x
BANK GROUP AGRICULTURE AND RURAL DEVELOPMENT POLICY x
ADOA ADDITIONALITY AND DEVELOPMENT OUTCOMES FRAMEWORKD 2009 x
BANK GROUP REGIONAL INTEGRATION STRATEGY x
IDEV OPERATION MANUAL x
BUSINESS MANUAL - PRIVATE SECTOR OPERATION x
IDEV EVALUATION - REVIEW OF THE BANK ESW 2005 -2010 x
IDEV EVALUATION - TRANSPORT SECTOR x
IDEV EVALUATION - MICROFINANCE + Database + Portfolio Review x
IDEV EVALUATION - NON SOVEREIGN OPERATIONS+ Database+ Portfolio review x
IDEV EVALUATION - QUALITY AT ENTRY x
IDEV EVALUATION - PROJECT RESULT REPORTING x
IDEV EVALUATION - SME x
DECENTRALIZATION x
85
Other sources
Bertelsmann Stiftung. 2014 ‘BTI 2014 — Nigeria Country Report’. Gütersloh: Bertelsmann Stiftung, 2014 EY. 2014 ‘Africa by numbers – A focus on Nigeria Special report issued for: World Economic Forum on Africa 2014’ London, 2014 EYGM Limited KPMG. 2015 ‘Country Report – Nigeria 2014 Quarter 4’ London, KPMG LLP 2015
KPMG. 2014 ‘Nigeria Country Profile – KPMG Africa Region 2012/2013’ London, KPMG LLP 2014 IFAD. 2013 ‘Federal Republic of Nigeria – COSOP MID-TERM REVIEW 2013’ Report No: 3554-NG, Rome IFAD. 2010 ‘Federal Republic of Nigeria – Country strategic opportunities programme’, EB/2010/99/R.11, Rome IMF. 2015,’ IMF press release - IMF Executive Board Concludes 2014 Article IV Consultation with Nigeria’, International Monetary Fund, Press Release No. 15/91, March 4, 2015, Washington, D.C. IMF. 2013, ‘Nigeria: Financial Sector Stability Assessment’, International Monetary Fund, Country
Report No. 13/142, Washington, D.C. IMF, World Bank. 2013. ‘Financial Sector Assessment Program : Nigeria - Crisis Management and Crisis Preparedness Frameworks. World Bank, Washington, DC’. © World Bank. https://openknowledge.worldbank.org/handle/10986/15964 License: CC BY 3.0 IMF. 2013, ‘Nigeria: Publication of Financial Sector Assessment Program Documentation–– Technical Note of Banking Cross-Border Issues’, International Monetary Fund, Country Report No.
13/142, Washington, D.C. IMF. 2013, ‘Nigeria: Publication of Financial Sector Assessment Program Documentation–– Technical Note on Stress Testing’, International Monetary Fund, Country Report No. 13/142,
Washington, D.C. IMF. 2013, ‘Nigeria: Publication of Financial Sector Assessment Program Documentation–– Technical Note on Crisis Management and Crisis Preparedness Frameworks’, International
Monetary Fund, Country Report No. 13/142, Washington, D.C. IMF. 2013, ‘Publication of Financial Sector Assessment Program Documentation–– Technical Note on Strengthening Monetary and Liquidity Management’, International Monetary Fund, Country
Report No. 13/142, Washington, D.C. IMF. 2013, ‘Publication of Financial Sector Assessment Program Documentation–– Detailed Assessment of Implementation of IOSCO Objectives and Principles of Securities Regulation, International Monetary Fund, Country Report No. 13/142, Washington, D.C. World Bank. 2015. ‘Nigeria Economic Report 2014’, July 2, 2015, Washington, DC: World Bank Group. World Bank. 2014. Doing Business 2015: Going Beyond Efficiency. Washington, DC: World Bank Group. DOI: 10.1596/978-1-4648-0351-2. License: Creative Commons Attribution CC BY 3.0 IGO World Bank. 2014. Doing Business in Nigeria 2014: Understanding Regulations for Small and Medium-Size Enterprises. Washington, DC: World Bank Group World Bank. 2011,’Country Partnership Strategy Progress Report For The Federal Republic Of Nigeria For The Period Fy10 – Fy13’, Washington, DC: World Bank Group
Annex 15: Possible structure of main technical report and sector reports
Besides the main technical report, two separate sector reports will be produced each covering:
Infrastructure projects in the energy sector and the transport sector
Lines of Credit projects
Structure of the main technical report
Executive summary
1. Introduction: Description of purpose and objectives; limitations and data; structure of report
2. Country context: Description of social and economic developments
3. Composition of portfolio: trends, types, size, volume, focus
4. Analysis of Portfolio Data
3.1 Portfolio performance: Implementation performance, Sector performance, monitoring
and reporting;
3.2 Private sector operations: by sector: design issues, implementation issues, strategic
outcomes;
3.3 Public sector operations: by sector: design issues, implementation issues, strategic
outcomes;
3.4 Multinational operations: contribution to CSP outcome;
3.5 Technical Assistance/knowledge sharing: performance and contribution to CSP
outcome;
3.6 Comparison of overall performance of Bank Portfolio to Country Portfolio (trends over
the period).
5. Program Outcome and Performance Summary:
4.1 Strategic overview: contribution of portfolio to CSP results; added value of the Bank,
Borrowers Performance, Partnerships;
4.2 Relevance: summary (refer to template details);
4.3 Efficiency and Implementation (refer to template details);
4.4 Potential for results.
6. Key lessons and going forward (this should include issues such as)
5.1 Strategic positioning of the Bank;
5.2 Engagement with the country;
5.3 Institutional arrangement: decentralization, resources, re-structuring.
87
Structure of the sector reports
Executive summary
1. Introduction: Description of purpose and objectives; limitations and data; structure of report
2. Context: Description of sector, key stakeholders, developments in the sector, sector policy in
South Africa
3. Composition of project portfolio in the respective sector: trends, types, size, volume, focus
4. Assessment
a. Relevance
b. Effectiveness
c. Efficiency
d. Sustainability
e. Added value of Bank interventions in the sector
5. Key lessons
89
Annex 16: Timeline
ID Task Name Start Finishjan 2016aug 2015 sep 2015 dec 2015okt 2015 nov 2015
29-1116-8 15-11 22-11 6-128-114-10 18-10 27-1230-8 13-9 25-1020-9 11-10 1-112-8 13-1227-9 20-1223-89-8 10-13-16-9
1 30-9-20155-8-2015Inception phase
2 7-8-20155-8-2015Scoping mission Abidjan
3 4-9-201510-8-2015Desk research and inception report drafting
4 18-9-20157-9-2015Scoping mission Nigeria
30-9-201521-9-2015Finalisation Inception report
6
7
5
13-11-20151-10-2015Data collection and analysis
26-10-20151-10-2015Phase 1: Desk-based review of documents, portfolio review, identification of data gaps
17-11-20152-11-2015Phase 2: Data collection mission Nigeria8
9
10
11
12
13
13-1-201618-11-2015Technical report drafting and finalization
23-12-201518-11-2015Draft report to IDEV for comment
23-12-201518-11-2015PRA templates finalized
14-1-201623-12-2015Draft report to reference group and reviewers for comment
29-1-201615-1-2016Finalisation report
14 4-3-201615-1-2016Summary report, workshop and dissemination
17-1
feb 2016
24-1 31-1
mrt 2016
7-2 14-2 21-2 28-2 6-3