an overview of oda evaluations...formation of a consistent monitoring and evaluation system focusing...

Chapter 1An Overview of ODA Evaluations

Chapter 1An Overview of ODA Evaluations

— � —

Chapter 1 An Overview of ODA Evaluations

Chapter 1 An Overview of ODA EvaluationsChapter 1 An Overview of ODA Evaluations

1.1.1 Trends in ODA Evaluation in Japan

The evaluation of Official Development Assistance (ODA), since its initiation in 19751, has been gaining importance as a means to improve the effectiveness and efficiency of ODA as well as to ensure public accountability.

In August 2006, in order to strengthen its functions in planning ODA, MOFA conducted a structural reorganization, and newly established the International Cooperation Bureau. With this reform, the ODA Evaluation Division became an independent unit to conduct comprehensive operations related to ODA evaluation. Such reforms aimed at further enhancement of the evaluation system, with a view to ensuring more effective and efficient use of ODA through evaluation and high quality of international cooperation. Moreover, with the enactment of JICA law amendment in 2007 and in preparation for the new JICA launch in 2008, continuous efforts have been made to establish comprehensive evaluation system.

(1) Revision of the ODA Charter and the Enhancement of ODA Evaluation

Stringent financial conditions forced budget cuts allocated to the ODA at the beginning of the 21st

century. On the other hand, with the economic rise of emerging countries and the deepening of global issues, the concept of “human security” was proposed in the international society, thereby leading to the formulation of the “MDGs: Millennium Development Goals” in the year 2000. After the 9-11 terrorist attacks in the United States in 2001, poverty reduction has become a highly critical issue, as one component that may lead to terrorism. Faced with such circumstances, revisions were made to the ODA Charter in August 2003, in which it was clarified that the objectives of ODA will include not only contribution to the peace and security of the international community but also

contribution to Japan’s own security and prosperity. Furthermore, high expectations have been placed on the strategic use of ODA, which for instance aims at promoting private-sector economic activities and securing energy and other resources.

The former ODA Charter, which was approved by the Cabinet in 1992, stipulated that “evaluation activities should be expanded to include third-party evaluations and joint evaluations with other countries for the benefit of future international cooperation.” The Charter also called for the “promotion of comprehens ive eva lua t ion o f ODA.” These stipulations reflected the situation in which the former ODA Charter was formulated in 1992. At that time, the MOFA, its overseas embassies, implementation agencies and third-parties were conducting ex-post evaluation. The third-party evaluation, however, accounted for only a small part. It was the time when the importance of a more comprehensive evaluation based upon the results of individual project evaluation has been pointed out within MOFA.

Prior to the revisions made to the ODA Charter in August 2003, there had been incessant changes to the international situation, including worsening poverty driven by the progress of globalization, escalation of regional or internal conflicts after the end of the Cold War and increased international terrorism. As a result, the significance and scope of ODA evaluation underwent substantial changes. In the past, ODA evaluation was conducted in parallel with the planning and implementation procedures. However, as the idea of a cycle of planning (Plan), implementation (Do), evaluation (Check) and reflection (Act) (PDCA cycle) for effective management of government activities became established, the ODA evaluation began to be incorporated as an integral part of the ODA system.

One of the most prominent features of the ODA Charter is the effort to implement a comprehensive evaluation. Specifically, the evaluation system covers

1.1 Recent Trends in ODA Evaluation

1 Ex-post evaluations were initiated on individual projects by the then Overseas Economic Cooperation Fund (OECF, now the Japan Bank for International Cooperation) in 1975. The MOFA began its ex-post evaluation in 1981, and JICA followed it in 1982.


— � —

each stage of ODA activities, from ex-ante to ex-post, and covers a wider spectrum with policies at one end and individual projects at the other. Now that evaluations are incorporated into one of the implementation processes of ODA, consistent evaluations from ex-ante to ex-post, are required for effective implementation of aid policies, whereas, in the past, the evaluation was conducted on an individual project basis and only after its completion. In an effort to enhance its effectiveness, the focus of development assistance has shifted in recent years to a more comprehensive approach, which targets a sector or a country as a whole in addition to individual projects. An evaluation with broader scope becomes possible by targeting not only individual projects but also assistance activities at the sector or country level.

Another feature of the ODA Charter concerning the significance of evaluation is its reference to the self-evaluation by government agencies, in addition to the third-party evaluations required for further objectivity.

The ODA Charter also stipulates that evaluation results should be reflected in the formulation and implementation of policies. The evaluation is only meaningful when its results are used effectively as a feedback in the planning and implementation of policies. While the former ODA Charter did not necessarily stress this point, the new Charter points out the importance of having evaluation results reflected in the subsequent ODA activities.

The ODA Charter emphasizes the necessity of cooperation with other countries to conduct joint evaluations. The importance of joint evaluations is continually expressed in other policies, such as “Japan’s Medium-Term Policy on ODA” which is the subordinate policy under the ODA Charter, and “Basic Policies for Economic and Fiscal Management and

Structural Reform 2006 (the so-called “Basic Policies”2006).” Through these references to the significance of evaluations in various policies including the ODA Charter, evaluations have become one of the vital pillars in the ODA reforms, which have been actively put forward by the government under serious domestic budgetary constraint and the public critical views towards development assistance.

(2) PDCA Cycle

As the significance of evaluation began to be widely acknowledged, the “Government Policy Evaluation Act (GPEA)” was enforced in 2002. Under this Act, the government agencies are obliged to conduct its own policy evaluations to facilitate effective and efficient administrative activities as well as to fulfill its accountability for the public. ODA evaluations are considered as one of these policy evaluations based upon the Government Policy Evaluation Act. In the “Basic Policies 2005” approved by the Cabinet in June 2005, it was announced that “objective third-party evaluations including cost-effectiveness analysis on the outcome of ODA projects should be conducted, with their results opened to the public, in order to establish the PDCA cycle.” Following this trend, in the policy papers MOFA has been announcing since 2005, “Examination and Improvement of ODA – Aiming for Better-Quality ODA,” “Improvement of Checking Systems” was raised as one of the three pillars. It was aimed to enhance the evaluation system and reflect the evaluation in policies, as well as to establish the PDCA cycle.

By conducting evaluations in a PDCA cycle and strengthening feedback on evaluation findings and to implementation agencies (both in Japan and in recipient countries), MOFA is also making efforts

(Feedback to policy implementation/reflection of evaluation results on new policy)

Act

(Policy formulation)Plan

(Evaluations (ex-ante, mid-term, ex-post), audits, other studies)

Check

(Implementation)Do

Figure 1: PDCA Cycle

— � —


to utilize the lessons and proposals derived from evaluations, in the formulation and revision of ODA policies. For instance, since MOFA assumes the role of formulating economic cooperation policies, it focuses on policy-level evaluations such as “country policy evaluations” that examine Country Assistance Programs and “priority issue evaluations” that examine priority issues stipulated in the ODA Charter. Follow-up measures are being taken so that feedbacks on the lessons and proposals derived from evaluation results are reflected in the ODA policy. Such third-party evaluations are conducted mainly by academic experts and NGO members, who constitute the members of the External Advisory Meeting on ODA Evaluation. Furthermore, through ministry-wide meetings, the sharing of know-hows among government ministries and agencies are encouraged. (Please refer to Chapter 1 Section 1.2 “ODA Evaluation Implementation”)

(3) ODA Implementation Reform and Evaluation

In August 2006, in order to strengthen its function in planning ODA, MOFA conducted a structural reorganizat ion to integrate a par t of former Economic Cooperation Bureau and Global Issues Department and newly established the International Cooperation Bureau. This made it possible for MOFA to effectively carry out bilateral and multilateral economic cooperation activities, while making the ODA Evaluation Division an independent section responsible for comprehensive ODA evaluation services where it was formerly a part of the Aid Planning Division of the Economic Cooperation Bureau. The path has been set for MOFA to further enhance its evaluation structure based on an even broader perspective for implementing ODA with “high quality” international cooperation with enhanced effectiveness and efficiency.

In 2007, as part of the ODA Reform, the amendment to the Law Concerning the Japan International Cooperation Agency passed the Diet, and the new JICA is to be established to be responsible for all tasks including continuing technical cooperation as well as yen loans formerly conducted by JBIC and grant aid formerly conducted by MOFA. The government is seeking to create a comprehensive evaluation system towards the establishment of the new JICA.

Although there have been many reforms to ODA

in the past, the current movement outstands all past reforms, affecting ODA at “strategy,” “policy-making,” and “implementation.” At the “strategy” level, basic strategies were discussed at the Overseas Economic Cooperation Council headed by the Prime Minister. At the “policy-making” level, MOFA’s structural reorganization led to the establishment of the International Cooperation Bureau to assume overall responsibilities for both bilateral assistance and multilateral aid via international organizations. Lastly, at the “implementation” level, the three different modalities of assistance are to be implemented under the new JICA. It is expected that coordination between different aid modalities will be conducted more actively, which will lead to the flexible responses to the needs of developing countries. Formation of a consistent monitoring and evaluation system focusing on certain characteristics of each aid modality is also being considered. In terms of program-level evaluations by MOFA and new JICA, adjustments will be made from the planning stages to ensure efficient demarcation and avoid duplications.

1.1.2 Trends in ODA Evaluations in the International Community

(1) Background

Prior to the 1980s, evaluations of Official Development Assistance (ODA) were conducted individually within each country’s administrative activities. With increasing international awareness of the importance of the ODA evaluation, ODA evaluation issues have been discussed at OECD-Development Assistance Committee (DAC) and other international meetings since the 1980s. Subsequently, the importance of the ODA evaluation has been gradually growing as a means to enhance the effects and efficiency of ODA and to accomplish government accountability to the public. In addition, with the emergence of more comprehensive development approaches such as the World Bank Comprehensive Development Framework (CDF) and Sector Programs which cover issues beyond the boundaries of each project, the scope of the evaluation expanded from each project to sector-wide aid activities and to nation-wide operations.

Reflecting the increasing importance of ODA evaluation, evaluation activities originally carried


— � —

out separately from the implementation process of development assistance have been gradually integrated into ODA systems. This change could be brought by the New Public Management (NPM) and Result Based Management (RBM) which attract strong attention from the international community.

New Public Management is an approach that emerged in the 1980s mainly in the United Kingdom and the United States to incorporate business approaches into the public sector, placing emphasis on the use of market mechanism and the principle of competition and advocating active disclosure of management information of central and local governments. The basic principles of this New Public Management are clarification of responsibilities and outcome-based management. “Result Based Management” was increasingly adopted in the 1990s mainly in North American countries, in the wake of accelerated government reform. Results Based Management is a strategy intended to manage organizational activities from the viewpoint of their performances and the results (outputs, outcomes, and impacts).

“New Public Management” and “Result Based Management” are both intended to verify whether initial targets have been achieved by measuring their performance and results, and in turn to properly allocate resources to each government activity based on the findings ascertained from the measured results. Thus, in these two systems, monitoring and evaluation are established tools essential for managing government programs.

In this way, monitoring and evaluation have been gradually integrated into the administrative system as its established mechanisms, when moves to manage governmental activities have been accelerated. Monitoring and evaluation are also seen as means for measuring and verifying the efficiency and effectiveness in the comprehensive development approaches which started attracting attention in the late 1990s.

(2) Recent Trends

In the late 1990s, various development approaches and international development objectives appeared in the international community including DAC’s new development strategy (Shaping the 21st Century: The

Contribution of Development Co-operation), the World Bank CDF and the Millennium Development Goals (MDGs) following the United Nations Millennium Summit.

DAC’s new development strategy was adopted in May 1996 at the DAC 34th high-level meeting. This new development strategy classified the international community’s development targets into three categories, “economic well-being,” “social development” and “environmental sustainability and regeneration” and established the DAC targets together with measurable indicators. To achieve these development targets, it stressed the importance of mutual promises and aid coordination among development partners and policy coherence, encouraging the implementation of monitoring and evaluation for further effective development aid.

CDF was first proposed by the President of the World Bank at the IMF/World Bank Annual Meeting in 1998. It is a framework to realize development assistance through more integrated and country-driven approaches. CDF, as an approach for poverty reduction, has four basic principles: being a comprehensive and long-term strategy, emphasizing the ownership of the recipient country, ensuring a solid partnership among the relevant parties and being results-based. In addition, the World Bank/IMF Development Committee in 1999 determined to introduce Poverty Reduction Strategy Papers (PRSP) as a key tool to realize CDF. Thus heavily indebted poor countries and nations eligible for International Development Association (IDA)2 funds are encouraged to prepare PRSP debt reduction and IDA loan supply. PRSP, whose basic framework is CDF, has similar core principles as of CDF: being country-driven, emphasizing result-based, taking comprehensive approaches, emphasizing partnership and having long-term perspectives. For the implementation of PRSP, there are four basic processes: (1) participatory process, (2) comprehensive understanding of poverty, (3) prioritization of policies, and (4) monitoring and evaluation.

At the UN Millennium Summit held in New York in September 2000, representatives of the 189 member states, including 147 heads of state, unanimously adopted the UN Millennium Declaration which defined

2 A group organization established by the World Bank in 1970. It is responsible for providing long-term interest-free loans to the poorest countries.

— � —


the goals for the international community in the 21st century. The Declaration clarified the directions that the UN will look out in the 21st century and addressed some issues such as peace and security, development and poverty, environment, human rights and good governance, and specific needs from African countries. The MDGs consist of the development targets already adopted at major international conferences and summit meetings in the 1990s and those defined in the UN Millennium Declaration. MDGs will also serve as guidelines for the international community to formulate development policies and programs and also as a common framework to measure their effects. MDGs have 8 basic goals and 18 targets. The UN, OECD, IMF and the World Bank regularly monitor the achievement of these goals. In order to achieve MDGs, UN has set research and recommendations, monitoring and millennium campaigns as its key strategies. Monitoring has been carried out at both the global and the national levels where indicators defined under the MDGs are measured. Based on the results of these monitoring, the UN Secretary General’s reports at the global-level and the Millennium Development Goals Reports (MDGRs) at the country-level are prepared.

This MDGRs is a series of reports on indicator trends in recipient countries that have been published in approximately 50 countries. International Development Banking Groups such as the World Bank has also been monitoring MDGs achievement levels and the results are reported at the development fund meetings.

(3) The DAC Network on Development Evaluation

In the 1980s, DAC founded the Group of Evaluation Correspondents to lead international discussion on ODA evaluation. In conjunction with the reorganization of the DAC in 2003, which aimed at enhancing efficiency, the Working Party on Aid Evaluation was modified as the DAC Network on Development Evaluation.

The DAC Network on Development Evaluation (“the Network” hereinafter) is comprised of representatives from about 30 donor countries and international organizations. The mandate of the Network is to: share knowledge of evaluation and strengthen cooperation on evaluation; contribute to the improved effectiveness of development assistance; provide advice to DAC;

and assist recipient countries in developing their evaluation capacities. The Network meets twice a year.

The Network held its first meeting in January 2004. Issues concerning evaluation were discussed and each member reported its major evaluation activities. Since then, meetings have been held continually to hold the upcoming sixth meeting in Paris in June 2007.

In the past meetings, discussion on the issues facing evaluation centered on quality. The members agreed that minimum standards needed to be established so as to ensure minimum level of quality.

Recently, discussions have been held on the subject of joint evaluation methods, taking into account the increasing adoption of comprehensive development strategies, such as the Poverty Reduction Strategy Process. Participants agreed on the importance of conducting joint evaluation in accordance with the development cycle of recipient countries and coordinated by donor countries. On the other hand, the discussion pointed out obstacles, such as time lag between evaluation and development strategy, attribution of assistance results among donors, and the lack of data necessary for evaluation.

With respect to the major evaluation activities, the members presented reports on evaluation of General Budget Support (GBS) and their collaborative efforts with recipient countries. General Budget Support provides funds directly into the general budget of the recipient countries, and is one of the important forms of assistance under the Poverty Reduction Strategy Process. The increased importance of GBS is backed by the fact that: (1) conventional project/program-level assistance as a whole fails to bring about stronger capabilities and ownership within recipient country governments; (2) conventional assistance methodologies, in general, fail to contribute to nationwide poverty reduction or economic growth; and (3) given this, the importance of pursuing comprehensive Poverty Reduction Strategies (PRS) that promote donor-recipient country partnerships is being increasingly recognized.

Evaluation reports were announced in spring 2006 by sixteen countries and organizations, including Japan, that participated in the evaluation of GBS under the leadership of the United Kingdom and the Netherlands.

The sixth Network meeting held in Paris in June 2007 gathered participants from 26 countries,


— � —

including Japan, and 7 international organizations. Opinions were exchanged regarding: evaluation on the implementation of the Paris Declaration, evaluation of GBS, comprehensive ODA evaluation, and Evaluation Capacity Development (ECD). It was decided to increase involvement in recipient countries’ evaluation capacity development.

As mentioned above, the Network is working together to share knowledge on evaluation and improve the effectiveness of development assistance. The next major agenda includes studies on techniques for country or sector-level comprehensive evaluation and for joint evaluation, as well as review of evaluation quality. The importance of evaluation, both independently or jointly, of development assistance for a country or a sector as a whole will increase along with the deployment of comprehensive development approaches emphasizing the ownership of the development project by recipient countries.

(4) Evaluation of Paris Declaration

Today, within the international development assistance community, there is an increased awareness on the importance of improving aid effectiveness. At the “Paris High-level Forum on Aid Effectiveness” held in Paris in March 2005, the Paris Declaration on Aid Effect iveness was adopted, in which promises among donors and recipient countries were coordinated to raise aid effectiveness. Reform efforts have been made from both donor and recipient countries.

As part of the donors’ efforts to “improve aid effectiveness,” there is a trend toward bringing assistance policy/strategy, and procedures to provide assistance by each donor, organization in line with development objective/strategies and procedures of the recipient country governments as far as possible (alignment) thus harmonizing assistance procedures taken by each donor (harmonization). These trends are expected to help alleviate administrative burdens on recipient countries.

In addition, efforts to strengthen support for recipient countries on development programs are being taken, through formulation of development program with more ownership, and promoting economic and social development while effectively utilizing resources for development. Demonstrating to the people how resources for development were used (accountability)

helps such efforts as strengthening support of donors and the people of recipient countries on development programs that are currently underway.

Currently, the DAC Network on Development Evaluat ion, with Denmark at the center, are conducting evaluations on the implementation of Paris Declaration, and a discussion was held in March and June of 2007 with about 20 countries and international organizations regarding implementation of evaluation, target, and evaluation questions. Following the 2005 “Paris High-level Forum on Aid Effectiveness,” research is being conducted on principal donor and recipient countries in preparation for the 2008 “High-Level Forum on Aid Effectiveness” being held in Ghana (Accra HLF).

(5) ODA Evaluation Workshop

To improve the quality of evaluation and evaluation capacities of the recipient country, as well as to form a result-based evaluation system, Japan has been inviting Asian countries and international organizations every year since 2001 to the “ODA Evaluation Workshop.” The fifth “ODA Evaluation Workshop” was held in Tokyo in January 2005, where Japan contributed to the understanding of ODA evaluation methodologies and challenges, and to capacity building for evaluation in Asian countries.

In October 2006, Japan jointly hosted the “Asian Regional Forum on Aid Effectiveness: Implementation, Monitoring and Evaluation” in Manila, together with the World Bank, Asian Development Bank and the UK Department for International Development (a total of 25 Asian countries participated) with the following major objectives; to highlight outcomes/issues of implementation of the Paris Declaration in the Asia- Pacific region; and to reflect Japan’s experiences and outcomes of assistance in Asia led by Japan into discussion and efforts on the improvement of aid effectiveness on a global basis.

At the forum, an evaluation session was set up under Japan’s suggestion, and “evaluations” were addressed as one of the main agenda, alongside with “implementation” and “monitoring.” Lively discussion was held on topics such as the evaluation capacity building of recipient countries, establishment of the PDCA cycle and the format ion of an evaluation framework of the implementation of the Paris Declaration. By providing an opportunity for

— � —


representatives of evaluation and representatives of policy to gather and hold discussions, it was possible to share the importance of evaluation for enhanced aid effectiveness and efforts to link evaluation with concrete measures. The continual support for the broadening of evaluation capacity of the recipient country was also agreed upon. Furthermore, the significance of evaluation was clearly stated in the

concluding document, as an item which covers ECD of the recipient country and all 5 principals of the Paris Declaration.

In November 2007, the seventh “ODA Evaluation Workshop” was jointly hosted by the governments of Japan (MOFA, JICA and JBIC) and Malaysia in Kuala Lumpur.

Column1Column1ODA Evaluation Workshop

MOFA, aiming to raise understanding of ODA evaluation methodologies and challenges to evaluations, and for the capacity building for evaluation, has been holding an “ODA Evaluation Workshop” every year since 2001, inviting Asian countries and international organizations to participate.

ODA evaluations bring benefit not only by raising aid effectiveness in the donor country, but also improving governance and ownership in the recipient country. In the “ODA Evaluation Workshops” held in the past, there was a common interest to establish an evaluation system under a result-based approach, playing a major role in enhancing Asian recipient country’s evaluation capacity development (ECD).

In 2006, this workshop was held as a separate evaluation meeting in conjunction with the “Asian Regional Forum on Aid Effectiveness: Implementation, Monitoring, and Evaluation” held in October in Manila. By providing an opportunity for representatives of evaluation and representatives of policy to gather and hold discussions, it was possible to share the importance of evaluation for enhanced aid effectiveness and efforts to link evaluation with concrete measures. In 2007, prior to the establishment of new JICA in the following year, the ODA Evaluation Workshop was co-hosted by the Malaysian government and three organizations from Japan (MOFA, JICA and JBIC). Malaysia achieved a remarkable breakthrough after utilizing Japan’s ODA, and the workshop was held as part of “Japan-Malaysia Friendship Year 2007” to commemorate the 50th Anniversary of the establishment of diplomatic relations between Japan and Malaysia. 18 Asian countries (including the host countries) participated, along with international organizations including OECD, ADB, and UNDP, totaling to over 70 participants. Discussions were held on topics including the need for human resources and institution to enhance evaluation capacity and establishment of regional evaluation network.

Opening remarks from Dr. Toyama, Vice-Minister for Foreign Affairs

Past Workshops:1st: November 7- 8, 2001 (Tokyo)2nd: November 13 - 14, 2002 (Tokyo) 3rd: November 12 - 13, 2003 (Tokyo) 4th: January 17 - 21, 2005 (Thailand)

5th: January 26 - 27, 2006 (Tokyo)6th: October 18 - 20, 2006 (the Philippines)*7th: November 28 - 29, 2007 (Malaysia)

* held as a part of Asian Regional Forum on Aid Effectiveness: Implementation, Monitoring, and Evaluation

The Fifth ODA Evaluation Workshop


— 10 —

■ Objectives of ODA evaluation

(1) Support for ODA management: To support the management of ODA and the improvement of its quality through the examination of ODA activities and feedback of the lessons learned and suggestions made in the formulation and implementation process of ODA policies.(2) Accountability: To ensure accountability to the public and enhance transparency of ODA activities through publishing the results of evaluation to promote public understanding and participation in ODA.

■ Inter-agency collaboration: MOFA, the implementation agencies and other ministries and agencies

In order to efficiently and effectively implement ODA evaluat ions , MOFA and implement ing agencies emphasize cooperation and coordination of responsibilities. Since MOFA assumes the role of formulating economic cooperation policies, it focuses on evaluations at the policy-level and program-level. (1) The policy-level evaluation of ODA assesses the basic ODA policy. Accordingly, the MOFA has been

evaluating medium-term policies on ODA, as well as country policies and priority issues. (2) Program-level evaluations of ODA target groups of projects that share common objectives. (3) Project-level evaluation reviews individual activities such as single projects conducted mainly by implementing agencies. The JICA conducts technical cooperation, JBIC provides loan assistance, and in FY2005, MOFA introduced ex-post project evaluations for Grant Aid projects. Figure 2 shows each level of evaluation.

In order to ensure objectivity of ODA evaluation, MOFA essentially conducts third-party evaluations. Since October 2003, MOFA has been commissioning the “External Advisory Meeting on ODA Evaluation,” which consists of academics, as a private advisory body to the Director-General of the Economic Cooperation Bureau (currently Director-General of the International Cooperation Bureau). In addition, evaluations by recipient country governments or agencies (such as think tanks and academic institutions), and joint evaluations with external agencies (such as other donors and NGOs) are being conducted.

With regard to project-level ex-post evaluations

1.2 ODA Evaluation Implementation

EvaluationActivities of MOFA

*including evaluation in accordance with the Government

Policy Evaluation Act

EvaluationActivitiesof JICA

EvaluationActivitiesof JBIC

*JICA and the overseas economic cooperation operations of JBIC

are to be integrated in October 2008.

ODA Evaluation ODA Implementation

ODA CharterMid-Term Policy on ODA

Country Assistance PolicyODA policy on priority issues, etc.

1) Policy Level

Sector assistance programs, individual aid modality, etc.

2) Program Level

Individual projects, etc.

3) Project Level

Figure 2: Evaluation Implementation Systems and Scope of Evaluation

— 11 —


of grant aid introduced in FY2005, trial evaluations were conducted by Japanese embassies. Starting from FY2006, development experts and academic organizations began to participate in evaluations with higher objectivity.

Aside from these third-party evaluations, MOFA also conducts its own evaluations based upon the Government Policy Evaluation Act. The responsible divisions of the International Cooperation Bureau carry out evaluations in accordance with the Ministry’s policy evaluation plan focusing around policy-level evaluation. This is compiled by the Policy Evaluation and Administrative Review Office of the Minister’s Secretariat as the policy evaluation report, which is submitted to the Ministry of Internal Affairs and Communications. (See the website http://www.mofa.go.jp/mofaj/annai/shocho/hyouka/h19.html on the policy evaluation report of FY2006). MIC compiles evaluation reports submitted by each government agency and presents them to the Diet.

The links between third-party evaluations conducted by MOFA and evaluations conducted pursuant to the Government Policy Evaluation Act are shown in Figure 4 (following section).

MOFA was g iven the manda te o f ove ra l l collaboration of ODA administration under the Basic Law on the Administrative Reform of the Central Government (1998). MOFA has been holding the Experts Meeting of ODA Evaluation consisting of ministries and agencies involved in ODA, and compiling ODA evaluation results by these ministries and agencies. Along with human resource development projects, such as accepting trainees and hosting seminars, these ministries and agencies also dispatch experts and conduct research studies. For further information on ODA evaluations, findings conducted by ministries and agencies in the last fiscal year are provided in Chapter 2, Section 2.

Figure 3: Evaluation Implementation System of the Ministry of Foreign Affairs

The Ministry of Foreign Affairs

Government/agenciesin a recipient country

The External AdvisoryMeeting on ODA Evaluation

External agencies

Minister’sSecretariat

InternationalCooperationBureau

The Minister

Linkages under the GPEA (policy evaluation). (Summarized by the Minister’s Secretariat and submitted tothe Minister for Internal Affairs and Communications.)ODA evaluation linkages (other than the GPEA). (Covered by the International Cooperation Bureau)

External Agencies

AdministrativeEvaluation Bureau

The Ministry ofInternal Affairs andCommunications

Table 1 The External Advisory Meeting on ODA Evaluation:(Titles as of April 1, 2005)

Hiromitsu MUTA Member of the Board/Dean, Graduate School of Decision Science and Technology, Tokyo Institute of Technology

Kiyoko IKEGAMI Director, UNFPA Tokyo OfficeYoshikazu IMAZATO Editorial Writer, the Tokyo ShimbunIzumi OHNO Professor, National Graduate Institute for Policy StudiesYayoi TANAKA Associate Professor, National Institution for Academic Degrees and University EvaluationMasato NODA Trustee, Nagoya NGO Center/Associate Professor, Chubu UniversityHiroko HASHIMOTO Professor, Jumonji UniversityKatsuya MOCHIZUKI Director in Charge, Inter-disciplinary Studies Center, Institute of Developing EconomiesTatsufumi YAMAGATA Professor, Institute of Developing Economies Advanced School (IDEAS)


— 12 —

Figure 4: MOFA ODA Evaluation Modalities

Policy-level

Country policyevaluation

Priority issueevaluation

Evaluation of Countryassistance policy

Evaluation of Assistance policy forpriority issues

Sector program evaluation Aid modality evaluation

Evaluations for all aid operationsin a specific sector in a country

Evaluation for Aid modalities

Project evaluation by MOFA and Third-party Project evaluation in accordance withthe Government Policy Evaluation Act

Ex-post Evaluation on Grant Aid Project Grant Aid exceeding 1 billion yen (Ex-ante) [*1]Yen Loan exceeding 15 billion yen (Ex-ante) [*1]Individual projects (Ex-post) [*2]

Project-level

Program-level

Conduct Third-party Evaluation

Conduct Third-party evaluation, Recipient country government/agency evaluation, joint evaluation

[*1] See the following website for information on the MOFA ODA evaluation Implemented in FY 2006. Ex-ante evaluation based on Government Policy Evaluation Act/ ex-post evaluation of individual projects based on this law: http://www.mofa.go.jp/mofaj/gaiko/oda/kaikaku/hyoka.html

[*2] Projects that have not yet concluded loan contracts five years after the policy decision was made, or projects that have not yet begun loan implementation (unstarted projects), etc. and projects that have not yet finished loan implementation ten years after the policy decision (Cabinet decision) was made (unfinished projects).

an overview of oda evaluations...formation of a consistent monitoring and evaluation system focusing...

Documents