chapter 2: an overview of evaluation results · 2020-01-30 · chapter 2: an overview of evaluation...

Chapter 2: An Overview of Evaluation Results2.1 Results of ODA Evaluations by MoFA

2.1.1 An Overview of FY 2004 Evaluations

As mentioned in Chapter 1, MoFA’s ODA evaluation work is divided into three levels: policy-levelevaluations, program-level evaluations and project-level evaluations (evaluations of grant aid-funded projectswere introduced in FY 20051). In FY 2004, MoFA conducted a total of 13 evaluations incorporating third-partyperspectives.

In terms of policy-level evaluations, MoFA carried out four Country Policy Evaluations (Laos,Uzbekistan/Kazakhstan, Bangladesh and Ethiopia), which examine Japanese assistance policy to the respectivecountries, and three Priority Issue Evaluations (anti-personnel mine action assistance, education-related MDGs,and health-related MDGs), which assess Japanese assistance policy on specific priority issues.

Program-level evaluations included two sector program evaluations (the Morocco UNICEF program and transport infrastructure section of Honduras), which basically examine all ODA activities undertaken in a specific sector of a given country. Four aid-modality evaluations (adjustment lending, the Grant Assistance forJapanese NGO projects modality, general budget support, and grassroots/human security-related grant aid toBolivia) were also conducted, assessing the performance of Japan’s aid modalities.

Based on the five OECD/DAC criteria for evaluating development assistance (relevance, effectiveness,efficiency, impact and sustainability), MoFA essentially verifies the relevance of objectives, the effectiveness ofresults, and the appropriateness of process for all third-party evaluations, joint evaluations and evaluations ofrecipient country governments/agencies. This section summarizes the ODA evaluations undertaken by MoFAduring FY 2004 on the basis of these criteria. It should be noted that the recommendations derived from MoFAfindings are based on the implementation status at the moment of evaluations. (The full text of each evaluation isavailable at: http://www.mofa.go.jp/policy/oda/evaluation/index.html)

Chapter

2AnOverview

ofEvalu

ationResults

Table 2: MoFA ODA Evaluations in FY 2004 (policy- and program-level evaluations)EvaluationModality Survey Title (country / sector)

Country Assistance Evaluation of Laos Country policy Country Assistance Evaluation of Uzbekistan and Kazakhstanevaluation Country Assistance Evaluation of Bangladesh

Policy-level Country Assistance Evaluation of Ethiopia evaluation Evaluation of Japan’s Anti-personnel Mine Action Assistance Policy

Priority issue evaluation

Mid-term Evaluation of Japan’s Contribution to the Achievement of the MDGs in the Area of Education Mid-term Evaluation of Japan’s Contribution to the Achievement of the MDGs in the Area of Health

Sector program Morocco-UNICEF Country Programme Evaluationevaluation Evaluation of the Bridge Construction Program for Tegucigalpa and on

Main Highways in Honduras Program-levelevaluation Aid modality

Review of Adjustment Lending – Overview of Structural Adjustment Loansand Sector Adjustment Loans –

evaluation Evaluation of “the Grant Assistance for Japanese NGO Projects Modality”Program Assistance:The Democratic Republic of Timor-LesteEvaluation of Grassroots Human Security Grant Aid for Bolivia

1 With a view to extending more effective and efficient Grant Aid cooperation and to achieving higher levels of publicaccountability, MoFA introduced ex-post project evaluations in FY 2005 (52 projects evaluated). The internationalevaluation standards set forth by the OECD/DAC were used to ascertain implementation status on individual projects, withissues and problem areas identified. This initiative will begin in earnest in FY 2006, with the involvement of developmentexperts and other third parties.

15

Chapter 2: An Overview of Evaluation Results

2.1.2 Policy-level Evaluations Country Policy Evaluations(1) Country Assistance Evaluation of Laos (third-party evaluation)

Evaluation Period: August 2004 – March 2005Chief Evaluator: Hiroko Hashimoto, Professor, Department of Information and Social

Sciences, Jumonji University Member of the MoFA External Advisory Meeting on ODA Evaluation

Advisors: Nobuyoshi Nishizawa, Professor, Faculty of Economics, Kinki UniversitySeiichi Fukui, Professor, Department of Economic Cooperation, KobeUniversity

Consultants: Hisami Nakamura, Overseas Project Management Consultants, Co., Ltd. Kako Inoue, Overseas Project Management Consultants, Co., Ltd.

Evaluation objectives The main objective of this country assistance evaluation was to draw useful lessons and make

recommendations towards more effective and efficient implementation of aid in the future and for theformulation of the Laos Country Assistance Program, as well as to uphold transparency and accountability byreleasing the results to the general public.

Scope of evaluation The evaluation covered Japanese policies and achievements in Laos between 1997 and 2003. The Japanese

Country Assistance Policy for Laos was drawn up in 1998 and covers Japan’s aid to the country during theperiod in question. The report provides an integrated and comprehensive evaluation of the objectives,formulation and implementation process and the outcomes of this policy.

Evaluation results(a) Evaluation of objectives

The priority sectors and sub-sectors identified in the Laos Country Assistance Policy were confirmed to beconsistent with Japanese ODA policy and with the development needs of Laos. Alternatively, it might be saidthat these policy settings enable Japan to provide assistance across a wide range of areas. This all-purpose aidapproach can be appropriate in setting priority sectors to meet every need in the development phase of Laos. Theprocess of developing the National Growth and Poverty Eradication Strategy (NGPES) allowed Japan to engagein discussion with the government of Laos and other donors at roundtable meetings, etc., and Japan has usedvarious opportunities to voice its opinions on poverty reduction. Although project implementation initiativeswere oriented towards this goal, further study on ways of focusing aid on deprived areas remains necessary.

(b) Evaluation of outcomesA logic model used to verify the rationality and causality of the aims and implementation of Japan’s Country

Assistance Policy for Laos demonstrated that the four priority sectors have been largely covered and that effortsare made to address the country’s various developmental needs. However, it was found that among the fourpriority sectors, many projects were implemented in the health sector with Basic Human Needs (BHN) aid and in infrastructure development, primarily of the transport sector. Moreover, when analyzed in terms of (1) the effectiveness of Japan’s policy on the macroeconomic development of Laos, (2) the effectiveness of Japan’spolicy on the achievement of the development aims of the National Socio-Economic Development Plan(NSEDP), and (3) the effectiveness of Japan’s policy on the achievement of the MDGs, the results achieved byJapan’s Country Assistance Policy for Laos were found to be highly relevant to various development targets,proving that the policy served as an effective developmental tool. The financial assistance extended by Japanfurnished critical funds for the development budget of Laos and helped to promote development projects, for the

16

most part in infrastructure, proving highly effective. While Japan’s assistance made significant contributions inthe four priority sectors, there are outstanding issues in terms of the operation and maintenance of developmentprojects and the sustainability of development effects. Japan’s Country Assistance Policy for Laos is consistentwith its assistance policy for East Asia and is thus deemed relevant.

Weak fiscal base of the government of Laos (current budget shortfalls are adversely affecting the effects andsustainability of ODA projects) and the tendency among other major donors to reduce aid volume to Laos (therelative importance of Japanese aid to Laos is on the increase) also need attention. In terms of the impact on bilateral relations with Laos, a questionnaire survey revealed that Japan’s ODA is creating a favorableimpression among the people of Laos and that it is playing a key role in maintaining friendly relations betweenthe two countries. Moreover, Japanese grant aid to the East-West Corridor development plan is expected to facilitate economic development by stimulating goods flows in central Vietnam, south-central Laos and northeastern Thailand. Internationally, Japan’s aid to Laos has been highly commended for meeting the country’sneeds, particularly in terms of infrastructure development, although there are calls for Japan to strengthen thelinkage between “hard” and “soft” assistance, including the operation and maintenance of infrastructure.

(c) Evaluation of processThe process employed in developing the Japanese Country Assistance Policy for Laos is confirmed to have

been appropriate in that the involvement of relevant ministries and agencies in aid affairs in this country wassecured and efforts were made to ascertain the development needs of Laos. The environment in which ODA isextended to Laos is changing rapidly due to such factors as the establishment of the NGPES, initiatives targetingpoverty reduction, more active donor coordination and an increase in aid volume from China. Since the fourpriority sectors were agreed by the Economic Cooperation Mission that was dispatched to Laos in March 1998,though the effectiveness of Japan’s Country Assistance Policy for Laos was confirmed by an EconomicConsultation Mission in July 1999 and again, by an ODA Task Force in FY 2003, no systematic opportunitieshave been set up for regular reviews.

Chapter

2AnOverview

ofEvalu

ationResults

RecommendationsReductions in the Japanese ODA budget mean that the Japanese government will need to implement more

strategic, efficient and selective aid in order to maximize the effects of ODA. In terms of effectiveness, studieson priority sectors for inclusion in the new country assistance policy that is currently under development will need to focus on: (1) support for regional economic integration targeting economic development in Laos, i.e. infrastructure development and human resource development; (2) support for poverty reduction, i.e. BHNsupport and aid to poverty-stricken areas; and (3) support for crosscutting issues; i.e. gender initiatives. In orderto implement highly effective and efficient assistance policy, it is necessary to ensure consistency between astrategic assistance policy and the process of project formation. In developing the new country assistanceprogram, it will be essential to have common understanding through discussion on the areas targeted to receiveJapanese aid. Such discussions should be undertaken after the goals and objectives of Japan’s assistance policyare defined and, as far as possible, goals and objectives are established for specific priority areas. This processwill enable more strategic and logical project formation towards achieving the sector goals, which is expected tolink to higher efficiency in aid to Laos. At the same time, although Japan’s assistance to Laos is provided inresponse to requests from the government of that country, in view of capacity of Laos, it must supportimprovements in local project formation capabilities, including better efforts to support participatorydevelopment.

The strengthening of the fiscal foundations of the government of Laos is a key issue and consideration willneed to be given to incorporating measures to secure operation and maintenance budgets into project plans and inmaking the realization of this target a condition for the provision of ODA. Also, in order to promote autonomyand build awareness of project ownership in Laos, agreement will need to be reached on the duration and extentof Japan’s assistance during the policy formation process. At the same time, the Lao government must be madeto recognize that Japan’s aid will be provided within a limited timeframe subject to the achievement of specificgoals in the priority areas and a system for monitoring progress established.

17


Furthermore, in order to enhance the efficiency of Japan’s ODA to Laos, it is not enough simply toimplement those of Japan’s aid modalities that are practicable in that country; instead, the new assistance planwill need to be developed with an eye to establishing a framework that incorporates approaches that will addresscurrent development needs, including expansions of aid modality targets, the development of new aid modalityand coordination with other donors.

An elementary school built with grant assistance for grassrootshuman security projects (Nakhok Noy elementary school)

The Lao-Japan Human ResourceCenter built by grant aid/project-typetechnical cooperation

(2) Country Assistance Evaluation of Uzbekistan and Kazakhstan (third-partyevaluation)

Evaluation Period: August 2004 – March 2005Chief Evaluator: Yasunaga Takachiho, Professor, College of Business Administration,

Tamagawa UniversityMember of the MoFA External Advisory Meeting on ODA Evaluation

Advisors: Miki Wajima, Senior Researcher, Institute for Economic Studies, JapanAssociation for Trade with Russia and Central Europe (ROTOBO)Natsuko Oka, Research Fellow, Institute for Developing Economies(IDE-JETRO)

Consultants: Daisuke Nagao, Manager, ODA Department, Ernst & Young ShinNihonHiromi Suzuki, Consultant, ODA Department, Ernst & Young ShinNihon

Evaluation objectives This evaluation mainly aims to draw useful lessons and make recommendations towards more effective and

efficient implementation of aid in the future and for the formulation of the two Country Assistance Programs, as well as to uphold accountability by releasing the results to the general public, through a review of the trends and all assistance-related policies targeting Uzbekistan and Kazakhstan.

Scope of evaluation The evaluation covered the period from FY 1997, when the then Prime Minister Ryutaro Hashimoto

announced “Eurasian Diplomacy” as Japan’s policy for developing closer ties with the so-called “Silk RoadRegion”, through FY 2003. It deals with Japan’s policy on Uzbekistan and Kazakhstan agreed through bilateralpolicy consultations held on three occasions with the respective governments of the two countries, and its results.

18

Evaluation results<Uzbekistan>(a) Evaluation of objectives

The priority areas for assistance agreed upon in bilateral policy consultations (namely, (1) assistance for thetransition to a market-oriented economy, (2) infrastructure development, and (3) reconstruction of the socialsector) were consistent with the fundamental principles and priority areas respectively identified in the formerODA Charter, the Silk Road Region Diplomatic Action Plan and the section on the Central Asia and Caucusregion contained in the former Mid-Term Policy on ODA.

The priority areas identified in the assistance guidelines agreed through bilateral policy consultations werecompared with the development needs cited in the annual plans developed during the period of this evaluation,with the Presidential Decree of June 2000, with the various policies and programs on infrastructure, and withsector development plans, including the State Program of Reform in Healthcare (1998-2005) and the NationalSkill Formation Program. This analysis confirmed that the priority areas selected accurately reflecteddevelopment needs in Uzbekistan.

(b) Evaluation of outcomes and impacts[Infrastructure development] Japan’s financial assistance, particularly for infrastructure development,accounts for a considerable portion of budgetary spending. Since infrastructure development is fundedthrough Yen Loan assistance, Japan’s contribution to the development of this sector is growing. Projectscompleted under Japanese funding in the air and transport sectors have achieved certain results and arerecognized to have produced a positive social outcome.

Chapter

2AnOverview

ofEvalu

ationResults

[Social sector reconstruction] (1) Japanese assistance to the healthcare sector has contributed toimprovements in services and to the formulation of related policy and is recognized to have had a positivesocial outcome. (2) In education, equipment provided to academic institutions through grant assistance for grassroots projects is sufficiently utilized, which is contributing to the development of the sector and is recognized to have had results. (3) In regard to agricultural technology, productivity using equipmentprovided under the grant for increased food production has increased relative to that using existingequipment and the real GDP growth rate of the agriculture sector remained positive throughout the period(1997-2001), suggesting that the grant for increased food production has had some positive impacts.

(c) Evaluation of processThe results of policy discussions undertaken on the basis of the assistance guidelines in the bilateral policy

consultations have been duly reflected in JICA’s Country Program for Uzbekistan and in JBIC’s regionalassistance policy for the Central Asia region. At present, the Japanese Embassy in Uzbekistan and the JICAOffice are responsible for coordinating Japan’s ODA to the country, and the results of the bilateral policyconsultations are reflected in the project formation, request, selection and implementation process.

In terms of systemic arrangements, Uzbekistan’s Agency for Foreign Economic Relations is responsible forreceiving foreign aid, while the Japanese Embassy and JICA Office coordinate activities on the ground. Inaddition, the establishment of a local ODA task force in 2003 allows for more localized policy discussions. Thisconforms that the Japanese assistance implementation system in Uzbekistan has been appropriately expanded todate. On the other hand, although the outlines of project formation systems have been developed in Uzbekistan,problems have been pointed out in terms of inexperience in aid operations and administrative staff shortages, andoperational performance remains insufficient.

Efforts have been made to coordinate Japan’s various aid modalities: Yen Loans, Grant Aid and technicalcooperation, and in some cases, Japan Overseas Cooperation Volunteers (JOCV) and senior overseas volunteers(SOV) have identified projects for grassroots assistance, from which it is concluded that due consideration hasbeen given to coordination among the various aid modalities in extending assistance to Uzbekistan.

19


<Kazakhstan>(a) Evaluation of objectives

The priority areas for assistance agreed upon in bilateral policy consultations (namely, (1) development of human resources and institutions, (2) development of social and economic infrastructure, and (3) alleviation ofthe social problems accompanying the transition to a market economy) were consistent with the fundamentalprinciples and priority areas respectively identified in the former ODA Charter, the Silk Road Region DiplomaticAction Plan and the section on the Central Asia and Caucus region contained in the former Mid-Term Policy on ODA. Meanwhile, soaring oil prices gave some room to the economy of Kazakhstan, which could mean thatODA alone will not yield sufficient results in terms of Japan’s bilateral relations with the country. Looking ahead,therefore, Japan will need to examine its stance towards Kazakhstan as a whole in the light of its policyobjectives for the country, giving due consideration to measures to promote foreign direct investment (FDI) andtrade as a means of furthering development.

The priority areas identified in the assistance guidelines agreed through bilateral policy consultations werefound to be consistent with the priorities of Kazakhstan’s long-term development framework: “Kazakhstan –2030”. Japan’s ODA policy was also confirmed to be in line with focal national development programs, namely,the “Mid-term Development Program (1998-2000)” and the “2010 Plan”. The “Industrial Innovation Program”adopted in March 2003 targets economic diversification through the development of manufacturing industries,and this will require due consideration when examining future aid to Kazakhstan. Kazakhstan’s various aiddonors all recognize that the country currently has no overt need for financial assistance and that it is, instead,placing greater emphasis on technical assistance as a means of transferring financial management know-how.

(b) Evaluation of outcomes and impacts[Development of human resources and institutions] With economic reforms making steady progress inKazakhstan, it is inferred that Japan’s assistance is producing certain positive social and economic impactsin the transition to a market economy. However, the focus on more specific goals in the provision ofassistance suggests that Kazakhstan’s development needs are becoming more sophisticated and pragmatic.Accordingly, from now on, Japanese aid initiatives will need to become more focused, for example, onWTO accession, policy finance and promotion of small and medium-sized enterprises.[Development of social and economic infrastructure] Japan’s financial assistance accounts for a considerable portion of budgetary spending in the transportation and telecommunications sectors. YenLoans constitute the main type of assistance in these two sectors, which suggests that these aid modalitieshave had a major impact on the development of transport and telecommunications infrastructure inKazakhstan. The completed rail sector project is also recognized to have had a positive impact on theeconomy.[Alleviation of the social problems accompanying the transition to a market economy] (1) Japaneseassistance to the healthcare sector, starting with the projects undertaken in the Semipalatinsk region, hascontributed to certain improvements in the standards of healthcare and is judged to have been effective. (2) Assistance for environment-related projects, including the introduction of a water quality monitoringsystem, is recognized to have produced certain results. In light of the contribution to local policy-makingcapacity made by the introduction of this monitoring system, Japan’s assistance is judged to have had apositive impact on Kazakhstan’s environment, although conditions are deteriorating.

(c) Evaluation of processSimilar conclusions as for Uzbekistan on the appropriateness of the policy development process were

reached for Kazakhstan. As to Japan’s assistance implementation system, the relocation of the Japanese Embassyin Kazakhstan to Astana and the planned establishment of a JICA office in Astana to coincide with the signing ofa technical cooperation agreement will allow for greater efficiencies in project formation and request procedures.There is currently no local ODA task force, but discussions take place with the Kazakhstan government in theform of local ODA policy dialogue.

Public relations activities are an important part of ODA, but within Kazakhstan public awareness of Japan’s

20

aid work remains low.The Ministry of Economy and Budget Planning of Kazakhstan is responsible for receiving foreign aid and,

whilst certain progress appears to have been made on developing the requisite systems, shortages of experiencedpersonnel with sufficient knowledge of policy implementation procedures remain a problem. Meanwhile, thoughtax exemptions should be applied to grant aid and technical cooperation projects, since it requires congressionalratification in Kazakhstan, procedural time requirements are causing problems.

In terms of coordination among Japan’s various aid modalities, experts have been dispatched to support YenLoan assistance projects and, in some instances, Development Studies have led to the provision of Yen Loanfunding to Kazakhstan. As to coordination with other donors, however, informal meetings do take place but onlyfor information-sharing purposes, and few projects have been undertaken in cooperation with NGOs.

Recommendations<Uzbekistan>

Encourage and support for the formulation of a comprehensive development plan, and near-termresponses: In order for Uzbekistan to realize stable and sustainable growth, a comprehensivemedium-range development plan will need to be formulated, the objectives of the plan shared among thepublic and all stakeholders, and efforts made to achieve steady growth. Accordingly, the governmentshould be counseled to develop a national development plan based on ownership of Uzbekistan, andongoing support provided in the form of expert dispatches and acceptance of trainees.

Chapter

2AnOverview

ofEvalu

ationResults

Support widespread improvements in the government’s administrative capacities, including aid receivingcapacity: Certain progress has been made in the development of systems for receiving foreign aid, butproblems have arisen due to the lack of experience in conducting aid operations and administrative staffshortages. Accordingly, Japan should dispatch experts as necessary and provide ongoing support foradministrative capacity building and improvements in staffing policies.Encourage and support the development and disclosure of statistical data: Statistical data not only providethe basis for the formulation of comprehensive development plans but also serve as a key tool and indexfor bilateral policy consultations. Ongoing support will need to be extended towards improvements in thereliability of Uzbekistan’s statistical data, through the dispatch of experts and acceptance of trainees. Review of the basic policy on “Assistance for the transition to a market-oriented economy”: It is importantthat Japan continue to provide steadfast assistance for the transition to a market economy; however, thefocus of Japan’s support (i.e. whether to continue to focus aid on the development of human resources thatare actively capable of making the transition, or to be more aggressive in pushing for progress on thisissue) must be clarified at the policy level. Once this is achieved, studies on aid requirements should linkto the provision of more effective and efficient assistance.Strengthen knowledge-based support for infrastructure development: To ensure that the infrastructuredeveloped under Japanese assistance can be utilized more efficiently, further efforts should be made to reinforce the provision of knowledge-based support. Assisting with the formulation of a comprehensivepolicy for the transport sector, coordinating the operation of airports with development of the touristindustry, and providing know-how on management and marketing are some examples of this kind of support.The Aid for Increased Food Production (2KR) counterpart fund: 2KR counterpart reserve funds areinsufficient, which has the potential to jeopardize the intended effects of this aid modality. Japan shouldcontinue to urge Uzbekistan to ensure sufficient reserve of counterpart fund.

<Kazakhstan>Support widespread improvements in the government’s administrative capacities, including aid receivingcapacity: In order to address the dearth of policy-implementation experience in Kazakhstan, Japan shoulddispatch experts as necessary to provide ongoing support for administrative capacity building.Support efforts to break away from a resource-dependent economy: Due consideration should be given tothe importance attached by Kazakhstan to its “Industrial Innovation Program” targeting economic

21


diversification and to the emergence of more focused and pragmatic development needs as the countrybecomes increasingly market-oriented, and Japan will need to take full advantage of its strengths in thefield of aid, including transfers of advanced technologies, while giving consideration to a targetedassistance program that is in line with the development needs outlined by Kazakhstan in the aforementioned program.Strengthen technical assistance and encourage efficient use of existing infrastructure (developed using aid funding): The development of Kazakhstan’s petroleum industry and soaring oil prices mean that thecountry has no overt need for financial assistance; on the contrary, the country is seeking technical transferof money management techniques. Based on this, Japan will need to reinforce the technical transfer components when formulating new Yen Loan projects, targeting the identification and design of projectsthat more precisely address Kazakhstan’s needs. In addition, in order to efficiently use the existinginfrastructure developed with past Japanese assistance, it is necessary to actively provide knowledge-basedsupport in such fields as management of facilities.Support measures to address deteriorating health indicators and healthcare infrastructure: Kazakhstan’shealth sector is beset by problems, and Japan will need to intensify its discussions with the government ofKazakhstan, clarify its precise role and the regions particularly in need of aid resources and, taking into account trends among other donors, make active efforts to resolve the various issues.Recommendations on other issues that should be taken into consideration in extending economiccooperation:

a) Coordination with NGOs: Assistance provided in cooperation with NGOs has the potential to improvethe effectiveness and efficiency of Japan’s ODA. Accordingly, Japan will need to draw up well-definedselection criteria in reference to the work of other donors in order to expand the possibilities of its aidby working with reliable NGOs.

b) Improvement of tax exemption procedures: Japan must continue to call for the prompt ratification ofthe Technical Cooperation Agreement (on technical cooperation) and work on the government of Kazakhstan to expedite the tax exemption procedures required for grant aid.

c) Strengthen PR activities: PR activities were undertaken in the past, but public awareness of Japan’s aidwork remains low in Kazakhstan. Accordingly, the techniques and content of Japan’s PR work willneed to be reviewed and further efforts should be made to improve and strengthen these activities.

Staff of the Astana Emergency Medical Centerstand in front of an ambulance provided underthe “Improvement of Emergency MedicalCenter in Astana”

Students intent on their studies at desksprovided under the “Supply of modernequipment to secondary school No.5 inSamarkand”

22

(3) Country Assistance Evaluation of Bangladesh (third-party evaluation)

Evaluation Period: August 2004 – March 2005Chief Evaluator: Teruo Kawakami, certified public accountant, representative director of

Office AsahiMember of the MoFA External Advisory Meeting on ODA Evaluation

Advisor: Tatsufumi Yamagata, Professor, Institute of Developing EconomiesAdvanced School (IDEAS)

Consultants: Haruko Awano, Senior Consultant, IC Net Limited Kenji Momota, Consultant, IC Net Limited

Evaluation objectives This evaluation mainly aims to draw useful lessons and make recommendations towards more effective and

efficient implementation of aid in the future and for the formulation of Country Assistance Program ofBangladesh, as well as to uphold transparency and accountability by releasing the results to the general public.

Scope of evaluation This evaluation covers the Country Assistance Program for Bangladesh (formulated in March 2000), together

with all Japanese ODA extended to Bangladesh on the basis of this plan between April 2000 and August 2004.

Chapter

2AnOverview

ofEvalu

ationResults


The Country Assistance Program is broadly consistent with the priority issues and aid principles outlined inJapan’s former ODA Charter and former Mid-Term Policy on ODA, and with the substance of Bangladesh’sInterim Poverty Reduction Strategy Paper (IPRSP) and other national development plans. It also maintains a favorable balance with the development needs identified by other donors and with their aid policies, and is thusconsidered to be relevant. However, there is room for improvement in terms of the fact that: (1) only limitedreference has been made to cross-cutting issues, such as improvements in governance and the elimination ofgender gaps, and (2) the program lacks clarity on goals and specific approaches.

(b) Evaluation of outcomesThis evaluation covered a relatively short period of four-and-a-half years and, in many cases, recently

implemented projects have yet to yield results; any direct correlation between the performance of these projectsand broad trends in macro indicators in Bangladesh was thus hard to demonstrate. However, improvements inrelevant indicators have been demonstrated for the health sector in some regions of the country.

Since some projects have produced notable results at the regional level, efforts will be needed to expandthese effects, to establish a roll-out model, and to provide information on successes to the government of Bangladesh and other donors. Greater consideration must also be given to the sustainability and self-sufficiencyof projects.

On the other hand, major bridge construction as a means of improving distribution networks is expected toimprove the macro effects of large infrastructure projects implemented in Bangladesh. There are also effortswhich can achieve outcomes by building up the capacity of the implementing agencies of Bangladesh throughtechnical cooperation in the fields of power and rural infrastructure development.

As to the evaluation of results, Japan should not simply adopt approaches that focus on the performance ofindividual projects, but must examine ways of evaluating the outcome of its ODA in the context of the risingtrend toward aid coordination.

(c) Evaluation of processThis evaluation assessed the appropriateness and efficiency of the process employed in formulating and

implementing the Country Assistance Program.

23


The program-making processThe Country Assistance Program was formulated after due investigation of development needs inBangladesh and the work of other donors, which process is considered to have been broadly appropriate.However, improvements and review are needed in respect of the following: (1) the limited involvement ofJapanese implementation agencies and the government of Bangladesh, (2) the failure to address changingneeds due to the absence of any reviews once the Country Assistance Program had been formulated, and (3) the two-year period required to formulate the Country Assistance Program.

The policy-implementation processProjects were formulated on the basis of the JICA Country Program and JBIC’s ODA Loan AssistanceStrategy for Bangladesh - both of which reflect the focus of the Country Assistance Program - and on thesector-based aid policies mapped out by the local ODA task force. Broadly speaking, Japan’s cooperationhas been reviewed and implemented in line with the Country Assistance Program. The local ODA task force has been actively promoting tie-ups among Japan’s different aid modalities and is pursuing aid coordination as well as partnerships with other donors and NGOs. Much of the burden of this work fallson the local ODA task force. Their work, particularly in the development of sector-based policies, is to behighly commended in that it has resulted in localized models that promote the effective implementation ofaid. In order to bolster these initiatives, consideration will need to be given to, among other matters,strengthening local systems and the adoption of flexible aid modalities.

RecommendationsSystemized objectives for Japan’s country assistance and sector programsSupport for Bangladesh’s Poverty Reduction Strategy Paper (PRSP), which is to become operational in FY2005, should be established as the target of the next Country Assistance Program and of the sectorprograms currently being developed by the local ODA task force, and Japan must work out a structure forits aid objectives with the ultimate goal of contributing to the attainment of PRSP goals (i.e. povertyreduction).

Monitoring and evaluation of Japan’s country assistance and sector programsJapan needs to consider establishing mechanisms for monitoring and evaluating the progress and targetattainment status of its Country Assistance Program and sector programs that are linked to the PRSPmonitoring system that has been set up by the government of Bangladesh.

Greater consideration for cross-cutting issuesCross-cutting issues, such as improvements in governance and the elimination of gender gaps, continue tobe a significant problem in Bangladesh, and Japan must give due consideration to these issues by, amongother things, conducting hearings with relevant experts.

Enhancing local functionsSince the local ODA task force is expected to play a key role in implementing and monitoring theassistance program, Japan should consider making functional enhancements by, for example, improvingpersonnel capabilities (dispatching policy advisor experts, making active use of local NGOs) andencouraging decision-making flexibility.

24

A view of the Paksey Bridge,constructed by Japan’s ODA

The Paksey Bridge monument

(4) Country Assistance Evaluation of Ethiopia (third-party evaluation)

Evaluation Period: August 2004 – March 2005Chief Evaluator: Tatsuya Watanabe, Trustee, Japan NGO Center for International

CooperationMember of the MoFA External Advisory Meeting on ODA Evaluation

Advisor: Yuka Kodama, Researcher, Institute of Developing Economies (IDE-JETRO)Consultants: Tetsuya Ishii, KRI International Corp.

Junko Saikawa, KRI International Corp.Masanobu Ninomiya, KRI International Corp.Atsuko Nishida, KRI International Corp.

Chapter

2AnOverview

ofEvalu

ationResults

Evaluation objectives This evaluation mainly aims to draw useful lessons and make recommendations towards the formulation of a

Country Assistance Program for Ethiopia that will serve as a guide to more specific project formation in futureand further the effective and efficient implementation of Japan’s ODA. To this end, the evaluation was intendedto provide a comprehensive review of Japan’s assistance policy for Ethiopia so as to verify the objectives,planning and implementation processes and results of such policy. The second objective was to upholdaccountability by releasing the results of the evaluation to the general public.

Scope of evaluation The evaluation covered Japan’s ODA to Ethiopia between FY 1995 and 2003. No Country Assistance

Program or Policy for Ethiopia has been drawn up to date, and priority areas were agreed upon in conformitywith national development policy and programs at the following policy consultation meetings.

1997 Economic AssistancePolicy Dialogue



(1) Education (2) Health/medical care(3) Food security(4) Infrastructure

restoration/development(5) Environmental conservation

(water supply, forest)

(1) Education (2) Health/medical care(3) Agriculture (food security)(4) Infrastructure development

(road/water/telecommunications)(mid and long-term point of view,environmental conservation)

(1) Education/Human ResourceDevelopment

(2) Health, HIV/AIDS(3) Water(4) Economic infrastructure(5) Food/agriculture/rural

development

25



Consistency with Japan’s ODA policy: The priority areas of Japan’s assistance policy for Ethiopia areconsidered to be consistent with the ODA Charter formulated in 1992 and revised in 2003, and are broadlyconsistent with the former Mid-Term Policy on ODA formulated in 1999.Relevance to international development initiatives: With the exception of “economic infrastructure”, thepriority areas for Japanese assistance are consistent with the priority areas and issues outlined in variousinternational development initiatives, including the TICAD II (Tokyo International Conference on Africa’sDevelopment) Tokyo Action Plan, NEPAD (New Partnership for Africa’s Development), the MillenniumDevelopment Goals (MDGs), the recommendations of the Commission on Human Security, and the G8Africa Action Plan, and are highly relevant.Relevance to Ethiopia’s development plans: With the exception of “development of the private sector” and“decentralization”, the priority areas for Japanese assistance coincide with the Sustainable Developmentand Poverty Reduction Program (SDPRP), and are highly relevant.

(b) Evaluation of outcomes and impactsContributions to priority areas

Japan’s contributions in the road sector and against infectious diseases have been highly commended and itsefforts in respect of the fight against infectious diseases have produced tangible results, reducing the incidence ofpolio and bringing its eradication within range.

Infrastructure development: Despite making little contribution to the telecommunications sector, Japan’swork in the road sector has received maximum recognition and commendation from the Ethiopian government, other donors and local NGOs. Japan’s contribution (output) has been substantial: Ethiopia’sroad network is not well-developed compared to those in other Sub-Saharan countries and even trunkroads are inadequate. Under such circumstances, Japan has made the most significant contribution of allbilateral aid donors in funding road rehabilitation and development. There is also evidence, albeitfragmentary, of an upswing (outcome) in economic activity along the rehabilitated trunk road.Health and hygiene: Japan’s measures against infectious diseases have had tangible effects, reducing polioand measles incidence rates by upgrading the polio checkup system – the result of effective coordinationbetween technical cooperation and grant aid, and raising vaccination rates by providing polio and measlesvaccines through UNICEF. In addition, the establishment of health posts and health centers has producedsome results and made a certain contribution to promoting maternal health.Education: Japan has consistently supported efforts to expand basic education, which constitutes theprimary focus of Ethiopia’s development plans for the education sector. However, Japan’s inputs in thisarea are of small scale and its contribution to basic education as a whole is thus limited.Agriculture and food security: The provision of fertilizer via the Aid for Increased Food ProductionProgram (2KR) was highly appreciated by the Ethiopian government, but unit yields have not risen and itis thus difficult to determine what contribution has been made to raising productivity. Aid for thedevelopment of irrigation, though small-scale, is commended from the perspective of technical transferand sustainability. In terms of food security, Ethiopia is pushing for aid that will contribute directly toincreasing agricultural production (productivity) and achieving balanced food distribution (i.e. foodsecurity) within Ethiopia.Water: Japan has focused on support for safe water supplies. It has made certain contributions in terms offacility construction and technical transfer and these efforts are locally appreciated; however, Japan’scontribution is limited because its input is limited in this area.

Tie-ups between aid modalities and sectors, and aid coordinationInter-modality coordination took place on a number of projects, namely, polio eradication (grant aid –

technical cooperation), road rehabilitation (grant aid – technical cooperation) and groundwater development and water supplies (grassroots grant aid – technical cooperation). These produced synergies and increasedeffectiveness and efficiency of Japanese aid.

26

(c) Evaluation of processesThe policy-making process

Regular policy dialogue was established in 1997, for which Japan prepares guidelines incorporating opinionsfrom JICA experts and Japanese NGO staff members. These policy dialogues proved effective in providingassistance to Ethiopia based on mutual agreement, through an accurate and specific diagnosis of its developmentneeds reflected in Japanese ODA policy.

Though regional governments were consulted on individual projects, they were not invited to participate inpolicy consultations. As progress is made on decentralization, regional governments are increasingly acting as project implementation agencies and it is thus considered appropriate to require their involvement in futurepolicy consultations and the preparatory meetings that precede them.

Meetings among aid donors take place frequently and are regularly attended by Japanese stakeholders forinformation gathering and exchange purposes. Should Japan intend to pursue greater selectivity and focus, it will need to participate actively and take the lead in donor meetings concerning these priority areas.

The policy implementation processThe priority areas were selected in line with Ethiopia’s development plans and requests from government,

and full respect was given to local ownership. The majority of aid projects were selected in conformity with thepriority areas and/or their sub-sectors, and the selection process is deemed appropriate. The selection of aid instruments was also appropriate in that Japan limited its assistance to grant aid and technical cooperation,avoiding Yen Loans in view of fiscal conditions in Ethiopia.

Chapter

2AnOverview

ofEvalu

ationResults

Progress is made on the coordination of grant aid and technical cooperation. The participation of local NGOsis also promoted and Japanese NGOs are providing assistance.

At the 2003 policy consultation, both sides agreed to annual ambassador/minister-level talks and toworking-level follow-up meetings three to four times a year. These consultations and meetings are highlycommendable in that they constitute an extremely useful means of ensuring the steadfast implementation ofJapanese assistance policy in Ethiopia.

RecommendationsPerspectives on assistance policy development: Considering anticipated spending cuts in Japan’s ODAbudget, Japan’s biggest challenge will be to find ways of utilizing its limited aid resources effectively and efficiently and of raising its presence. Its plans will need to be formulated in terms of: (1) articulatestatements on the significance and objectives of its assistance, (2) selectivity and focus on specific aidtargets (sectors and/or regions), and (3) the creation of synergies through coordination.Significance of Japan’s aid to Ethiopia: Ideally, objective goals and targets, inclusive of priority areas,should be structured, and specific goals (including numerical targets) to be attained within the term ofJapan’s aid program should be set in line with the stated objectives of local development plans.Aid targets (sectors/fields): The selection of priority areas should be undertaken on the basis of threecriteria, namely, development needs in Ethiopia, Japan’s past experience and comparative advantages, andachieving a balance with the initiatives of other donors. Ethiopia is considered to have powerful needs inthe following areas: (1) food security (agriculture, water, roads), (2) health and hygiene (infectious disease countermeasures), and (3) education (basic education).Cross-cutting/emerging issues: Given that peace-building, democratization and economic reform(structural adjustments and privatization) are identified as priority areas in Japan’s ODA Charter, it isnecessary to use policy consultations to push for results in these areas, and to incorporate appropriatemeasures at the project implementation level.Aid modalities: In light of Ethiopia’s status as an LLDC, Japan should continue to address its needsthrough grant aid and technical cooperation, and should make increasing use of grassroots human securitygrant aid.Assistance methodologies: Japan needs to: (1) coordinate priority areas and regions (between aidmodalities, sectors, other donors and NGOs), (2) strengthen South-South cooperation, (3) investigate the

27


dispatch of policy advisors and injection of resources into common funds in priority areas (particularly theroad and health sectors), (4) progressively implement direct budgetary support, and (5) pursuestreamlining by harmonizing its aid procedures (common modalities and procedures, proceduralrationalization, etc.) Policy consultations/dialogue: It is recommended that Japan: (1) seek regional participation in priorityareas, (2) provide forums for discussions on priority areas at the regional level, (3) make more effort to reflect the opinions of Japanese experts and NGOs, (4) conduct hearings with and seek opinions of theministries and NGOs involved in aid affairs in Japan, and (5) provide forums for discussion and dialoguewith local NGOs.Improving the transparency and predictability of aid: Efforts should be made to enhance predictability by,for example, providing Ethiopian stakeholders with a breakdown of technical cooperation costs forindividual projects.Improving grant aid: Ethiopia has requested that grant aid be untied in order that it can retain the freedomto select contractors. In addition, normative grant aid structures should be developed in order to allow forthe use of the cost-sharing scheme, which improves ownership among residents and increases projectsustainability. Since the procedures required for grassroots human security grant aid are relatively simpleand it is applicable for small-scale projects, one solution could be to implement such aid in clusters(bundling several similar projects under a single umbrella).Augmenting and enhancing aid personnel: Amidst efforts to give local staff greater autonomy (empower),it is essential that the number of aid personnel be increased so as to ensure the quality of Japan’s assistance.In order to address calls for sector assistance and direct budgetary support, and to play a leading role indonor meetings, efforts must also be made to develop abilities of local staff members.

Kilinto Village Water Supply Scheme An elementary school constructed bygrassroots human security grant aid

28

Priority Issue Evaluations(1) Evaluation of Japan’s Anti-Personnel Mine Action Assistance Policy (third-party

evaluation)

Field Survey Country: CambodiaEvaluation Period: August 2004 – March 2005Chief Evaluator: Yoshikazu Imazato, Editorial writer, The Tokyo Shimbun

Member of the MoFA External Advisory Meeting on ODA EvaluationAdvisor: Katsuhisa Furuta, Professor, Tokyo Denki University Consultants: Hiroshi Ishizato, Senior Consultant, International Project Center,

Mitsubishi Research Institute, Inc. (MRI) Satoshi Usami, Senior Researcher, International Project Center, MRISeiji Yamamoto, Senior Researcher, Science and Technology Research Division, MRI Shinichi Mizuta, Researcher, International Project Center, MRI

Evaluation objectives This evaluation mainly aims to provide a comprehensive assessment of the objectives, process and results of

Japan’s action on anti-personnel mines, and to draw useful lessons and make recommendations towards moreeffective and efficient implementation of aid in the future and for the formulation of Anti-Personnel Mine Action Assistance Policy, as well as to uphold transparency and accountability by releasing the results to the generalpublic.

Chapter

2AnOverview

ofEvalu

ationResults

Scope of evaluation The evaluation covered the “Zero Victim Program”, Japan’s policy on assistance for mine action announced

in December 1997 at the signing of the Anti-personnel Mine Ban Convention (the Ottawa Convention), togetherwith the series of projects undertaken under the Zero Victim Program during the period from January 1998 to March 2004.


It is confirmed that Japan’s anti-personnel mine action assistance policy is in agreement with: Japan’s basicpolicies on economic cooperation, including the ODA Charter and the Mid-Term Policy on ODA; internationallaws, such as the Anti-personnel Mine Ban Convention and Certain Conventional Weapons Convention; and thecurrent trends in assistance provided by the international community such as the promotion of human securityand poverty reduction.

(b) Evaluation of outcomesIntegrated mine action: The Cambodia Mine Action Center (CMAC), which included contribution to thetrust fund, the provision of equipment, mine clearance activities and the dispatch of experts, has beenimproving its functions as a mine action organization and achieving steady outcomes in mine clearanceand awareness activities while donors are recovering their confidence in this organization. In view of this,it is appropriate to conclude that Japan’s assistance in this area has been effective.Mine clearance: The mine clearance organizations supported by Japan have been steadily promoting mineclearance activities in efforts to increase usable land. In this sense, it is appropriate to conclude thatJapan’s assistance in this area has been effective.Japan has contributed to activities implemented by groups involved in assisting victims, and consideringthat these activities are achieving outcomes, it is appropriate to judge that Japan’s assistance has been effective.

29


Impact: The Cambodian economy, as a whole and in individual sectors, has been growing steadily during thepast decade, and it is assumed that mine action has had some significant impact on such economic growth.However, as the methodology for measuring the quantitative impact has not been established, we could notmeasure it in a tangible manner.

(c) Evaluation of processIt was confirmed that Japan gave due consideration to conditions in various regions and sectors targetedfor assistance and that it extended aid having made a careful selection of recipient organizations. Theseincluded government agencies and local organizations of recipient countries, international organizations,regional organizations and both International and Japanese NGOs. Discussions with the recipientorganizations are considered to have been held in an appropriate manner.Mine action requires continuous efforts, but because of the single year budget principle of Japan, medium-to long-term plans are difficult to make. Likewise, the Grass-Roots Human Security Grant Act does notfully cover the costs of NGO activities.The majority of Japan’s assistance for mine clearance is limited to support for demining operations;however, in recent years, projects combining mine clearance with development activities have beenincreasing, and Japan needs to be more active in providing this kind of assistance, which consciouslycoordinates these activities.The government regards goods required for anti-personnel mine clearance activities to be an exception tothe Three Principles on Arms Exports. However, goods related to anti-tank mines and unexplodedordnance are not included in the subjects for exceptions.On the other hand, the disposal of unexploded ordnance is an urgent priority in minefields and, moreover,given conditions in the field, it is essential that the equipment used to clear anti-personnel mines becapable of withstanding the impact produced by antitank mines.The need for technical development in mine detection and clearance is increasing because the currentmethods of clearance, which rely mainly on manual processes, have their limits. On the other hand, whilesome demand more contribution for clearance activities rather than for technical developments, it is appropriate to consider that both are equally important. Japan should continue its assistance giving equalweighting to both issues.In regard to the aid policy-making and implementation processes employed by the Japanese government, ithas become clear that there is no particular section which controls and coordinates in an integrated mannerJapan’s anti-personnel mine action assistance programs and that it has yet to develop a strategy indicatinggeographical and/or sector-based priority levels. However, there was no evidence that this caused anyparticular problems. Furthermore, Japan’s monitoring of related aid projects, which involves regularmeetings with and the submission of reports from the recipient and/or implementing organizations, isconsidered to be broadly appropriate. Primary responsibility for avoiding duplication of its assistance with that of other donors has in generalbeen fulfilled by the recipient and implementing organizations, and it was confirmed that the Japanesegovernment examined whether such coordination efforts have been made before it makes the finaldecisions on providing individual assistance.

RecommendationsContinued assistance in mine actionLong-term continuous efforts are needed for anti-personnel mine action assistance. In recent years, a declining trend of contribution can be seen in Japan’s assistance in these areas. However, aid should beproactively continued in the future as well.

Formulation of general strategies for mine action assistance and the establishment of an assistance frameworkIt is desirable that, in MoFA, a certain division is assigned a coordinating role for planning andimplementing Japan’s mine action assistance policy with clearer division of labour among the bureaus and

30

divisions concerned. Additionally, efforts should be made to extend aid on the basis of a well-definedstrategy that indicates priority regions and sectors.

Appropriate selection of recipient organizations and provision of technical assistance to localorganizationsIt is vital for Japan to assist reliable organizations in carrying out anti-personnel mine action assistance. Atthe same time, if the local organizations are structurally fragile and immature, it is necessary to providethem active technical assistance to improve their functions.

Implementation of assistance with linkage and combination between development and mine clearanceIn the area of mine clearance, it is becoming more common to coordinate and combine mine clearanceactivities and development activities. To make its aid more effective and visible, it is desirable for Japan to be more active in providing assistance which coordinates and combines mine clearance and developmentactivities.

Review and improvement of operations of the Grass-Roots Human Security Grant AidMany of the implementing organizations involved in mine action are international NGOs, and Grass-RootsHuman Security Grant Aid is widely used for supporting them. However, these NGOs indicated that theyhave various problems relating to this aid modality: medium to long-term perspective plans cannot bemade because this aid modality operates on a single fiscal year basis, and necessary costs for mineclearance cannot be adequately provided. Therefore, it is desirable for the government to make efforts to review the system and operation of the aid modality and to endeavour to improve it so that a response can be given to these requests.

Chapter

2AnOverview

ofEvalu

ationResults

Public relations efforts for wider recognition of Japan’s Grass-Roots Human Security Grant AidIn order to enable wider and more appropriate use of this aid modality, further public relations efforts are required to make its existence and details of the system more widely known.

Continuation of assistance for the technological development of mine detection and clearance The development of new mine detection and clearance technology is a necessary initiative for achievingthe goal of “zero victims” more quickly in the near future. It is desirable for Japan to make positive effortsto provide assistance for development of mine detection and clearance technology while continuing tosupport mine clearance activities.

A mine detection dog with a member of the mine clearance unit

A workshop on prosthesis manufacture

31


(2) Mid-term Evaluation on Japan’s Contribution to the Achievement of the MDGs inthe Area of Education (third-party evaluation)

Field Survey Country: VietnamEvaluation Period: September, 2004 – March, 2005 Chief Evaluator: Koichiro Agata, Professor, Faculty of Political Science and Economics,

Waseda UniversityMember of the MoFA External Advisory Meeting on ODA Evaluation

Advisors: Kazuo Kuroda, Associate Professor, Graduate School of Asia-PacificStudies, Waseda University Miki Sugimura, Lecturer, Department of Education, Faculty of Humanities, Sophia University

Consultants: Yoko Ishida, KRI International Corp.Mikiko Nishimura, Independent consultant, KRI International Corp.Yoko Takimoto, KRI International Corp.Yasuyo Hasegawa, KRI International Corp.

Evaluation objectives This evaluation mainly aims to provide a comprehensive assessment of Japan’s contribution to attainment of

Millennium Development Goals (MDGs) in the area of education and to draw useful lessons and makerecommendations towards more effective and efficient implementation of aid in the future and for theformulation of the MDGs, as well as to uphold transparency and accountability by releasing the results to thegeneral public.

Scope and methodology of evaluation In order to achieve the MDGs in the area of education, there is a need to establish an approach not only to

primary education, but also to more general approaches that affect primary education. To systematize the generalapproaches in relation to the two goals mentioned above, a schema of efforts was formulated to present a graphiclogical flow of related efforts. This schema covers assistance targeting poverty reduction, work to improve thereadiness of children (to attend school), improvements in health and nutrition services, and the development ofwater supply and transport infrastructure in the community, together with capacity building initiatives foreducational planning and financing at the administrative level. It shows the prioritized activities aimed at achieving the MDGs and their causal relationships.


Prior to BEGIN, there had been no government ODA policy of Japan specialized in the education sector. Itwas formulated based upon international trends including the EFA (Education for All) initiative. BEGIN wasannounced in 2002 as Japan’s first education ODA policy, with basic education sector as a priority.

Expansion of access to basic education is one of BEGIN’s high priorities. It is expected that implementationof BEGIN would strongly support the achievement of goal 2 of the MDG; i.e., ‘achieve universal education’.The basic principles of BEGIN assist self-help efforts by recipient governments, promoting communityparticipation and mobilization of available resources, and coordination with other development sectors. Thisreflects Japan’s comprehensive approach, which addresses the concerns of both community and administrativestakeholders.

BEGIN intended to challenge coordination and cooperation among G8 Kananaskis summit members tosupport the achievement of EFA. It is clearly stated that Japan will seek to share its development assistance goalswith international agencies and other donors in the education sector.

Under the current circumstances, Japan has difficulties in providing direct financial assistance, especially for

32

target countries’ recurrent costs, which is one of the main supporting pillars in FTI promoted by the World Bank.It is expected to examine this possibility as one of the ways in which Japan contributes to the FTI.

(b) Evaluation of outcomesAs pointed out earlier, education assistance activities by the Japanese government had been following

international aid trends such as the EFA initiative and the Dakar Framework for Action, prior to theannouncement of the MDGs. As a consequence, there is no major difference in ODA funding flows between the1990s and the 2000s. From 2001 onwards, the comprehensive approach as a basic principle has been moreheavily emphasized, and Japan has been continuing to invest in both direct and indirect contributions, whichwould contribute to achieving the MDGs in the area of education.

On a regional basis, sub-Saharan Africa was the lead recipient of Japanese input in initiatives intended tocontribute directly to MDG attainment in the field of education. This region received 25%, followed bySoutheast Asia, which received 19.16%. By contrast, by far the largest portion of Japan’s input in projectsdesigned to make an indirect investment to the MDGs, namely, the development of community infrastructure,secondary education and vocational training, was channeled to the priority regions of Northeast and SoutheastAsia. Moreover, Southwest Asia received substantially less funding: 10.64% for direct investments and 1.39%for indirect investments. Japan’s ODA has aimed at effective contribution to the achievement ofeducation-related MDGs through comprehensive contribution, as is shown in BEGIN and its new Mid Termpolicy on ODA statement. Therefore, it may be desirable to reconsider regional distribution of direct and indirectinvestment to Sub-Saharan African countries and Southwest Asian countries in the context of the developmentneeds of these countries.

Chapter

2AnOverview

ofEvalu

ationResults

(c) Evaluation of processFormulation of Japan’s aid strategy (BEGIN) and its implementation system

BEGIN was released almost immediately after the MDGs were adopted by the UN and internationaldevelopment partners. It represented the strong message that Japan is firmly committed to the MDGs in the areaof education. In the BEGIN policy formulation process, representatives from the Ministry of Foreign Affairs, theMinistry of Education, Culture, Sports, Science and Technology, JICA, JBIC, and academics contributed to thenumerous discussions. Consequently, BEGIN reflects the comprehensive viewpoint of different stakeholders.However, even after BEGIN was announced, various projects were implemented under existing conventionalpolicies and annual plans despite the new policy. The new BEGIN system by which individual projects aremonitored and evaluated with regard to their relevance to BEGIN has not yet been established. It is still expectedthat a system to monitor the progress and outcomes established in BEGIN will eventually be created.

Project formulation and implementation systemsThe staff members of JICA and JBIC now appear to share a relatively adequate understanding about the EFA

initiative and the MDGs in the area of education. As a consequence, the MDGs are recognized in preparingdevelopment programs, implementation plans, programs and projects. At Japan’s internal mission level in eachdeveloping country, a task force consisting of the Embassy, JICA, JBIC and JETRO examines an implementationapproach for each task. The organizational structure for implementation process for primary education assistanceprograms is also being established.

The roles of the task force should be strengthened so that Japan’s aid-related agencies can cooperate and arecoordinated to practice efficiencies and maximum benefits to recipients in implementing programs and projects.Currently, there are some cases where grant aid projects and technical cooperation are coordinated by usingdifferent aid modalities such as JOCVs, counterpart fund, and Grass-Roots Human Security Grant Aid tofacilitate inter-agency cooperation. By combining various aid modalities such as loans, grant aid, technicalcooperation, grassroots grant, JOCVs, senior volunteers, and NGOs, the use of funds contributed to theinternational aid organizations, expertise and efforts make it possible to get a maximum impact on achievingMDG results from total Japanese government input.

Donor coordination is emphasized in the ODA Charter, Mid-Term policy on ODA, and BEGIN. At Japan’sinternal mission level in each developing country, members of aid-related agencies attend education sector donormeetings to present Japan’s aid policies and individual project implementation plans and activity progress.

33


It is important to note that the European aid agencies and multilateral aid agencies have been promotingassistance based upon a sector-wide approach in recent years. In countries where this approach is adopted, otherdonors do not understand the narrower project based approach pursued by Japan. This narrower approach raisesserious questions as to the sustainability by recipient countries of the aid efforts supported by Japan. For thisreason, it is indispensable that at the project formulation stage there must be an agreement with the recipientcountry and other donors as to project purposes, intended outcomes and long term commitments by the countryto sustain the project after it ends.

Japanese aid activities for achieving the MDGs in the area of education, grant aid and grassroots grant aid are the main players for “expanding education access”. These two modalities are also important for “improvingquality” of the school facility rehabilitation programs. Moreover, health and nutrition improvement for thegreater “readiness of children”, the contributions of these two aid funding modalities are relatively high. Schoolconstruction projects through grant aid modality often take place in sites with geographically relatively easy access. This means that it may incur quality improvement, but may not contribute to access improvement.

The Development Study modality relates to quality improvement in primary education, expansion ofeducational opportunities, and the development of education policies and institutions. This modality was in thepast used by JICA for collecting and analyzing data in order to formulate an overall development improvementplan. In recent years, more practical and immediate actions took place, such as the promotion of communityparticipation and capacity development, teacher training, and rehabilitation of school facilities. The technicalcooperation modality had been mainly used for secondary education development. Development Studycontributions have been larger than those of the technical cooperation modality in supporting qualityimprovements of primary education so far.

Contributions to non-formal education, such as literacy programs and pre-school education, have beenprovided through grant aid and grassroots grant aid modalities, but the contributions by partnership programswith NGOs are substantial too. A good example of this is the literacy education program in Vietnam by theNational Federation of UNESCO Associations in Japan. This program is called the ‘World Terakoya Movement’and is deployed in some Asian nations.

On the other hand, total share of fund in ODA through international organizations to leverage its professionalknowledge and know-how is still small. Although grant aid, grassroots grant aid, and technical cooperationmodalities have been implemented in Asia, Africa, and Central and South America, similar procedures, schedules,itemized budget allocation frameworks, and organizational structures have been used throughout. Somecountries in Southeast Asia have achieved marked expansion of access to education opportunities and have mademajor efforts to improve education quality, whilst most of the African nations still lag far behind. Hence,different approaches are appropriate for different regions, according to specific local needs and conditions. Thismore tailored approach should lead to substantial progress toward achieving the MDGs in the area of education.

Xuat Hoa Primary School in Bac Kan province A class in progress at Hoa Van Satellite Schoolin Tam Duong

34

RecommendationsIn order for Japan’s assistance to contribute more effectively and efficiently towards the achievement of the

MDGs in the area of education, a summary of the recommendations made on the basis of the findings of thisevaluation is given below.(1) Review of Organizational Structure and Mainstreaming of BEGIN(2) Reconsideration to Donor Coordination and Direct Financial Assistance(3) Promotion of Comprehensive Approach for Achieving the MDGs in the Area of Education(4) Effective utilization of Multilateral Assistance to Education(5) Development of Plans for Individual Regions and Countries for Achieving the MDGs in the Area of

Education(6) Diversification of Technical Cooperation(7) Utilization of Grant Aid(8) Utilization of Grassroots Grant Aid Scheme(9) Loan Projects for Primary and Secondary Education

(3) Mid-term Evaluation on Japan’s Contribution to the Achievement of the MDGs inthe Area of Health (third-party evaluation)

Chapter

2AnOverview

ofEvalu

ationResults

Field Survey Country: The PhilippinesSurvey Period: August 2004 – March 2005Chief Evaluator: Kiyoko Ikegami, Director of the United Nations Population Fund

(UNFPA) Tokyo officeMember of the MoFA External Advisory Meeting on ODA Evaluation

Advisors: Naruo Uehara, Professor, Graduate School of Medicine, TohokuUniversityTomoko Sakota, Senior Program Director, Program ProductionDepartment, Japan Broadcasting Corporation

Consultants: Yutaka Tagami, Chief Researcher, Mitsubishi Research Institute, Inc.(MRI)Yoshihiko Kato, Senior Researcher, MRIHiroshi Ishizato, Senior Consultant, MRI Kentaro Yamaguchi, Researcher, MRI Shintaro Yamaki, Researcher, MRI Akiko Matsuyama, Researcher, Global Link Management Inc.

Evaluation objectives This evaluation mainly aims to assess how the implementation of Japanese assistance has contributed to the

attainment of the MDGs, development objectives that are shared by the international community and are to bereached by 2015, and to draw useful lessons and make recommendations towards more effective and efficientimplementation of aid in the future and for the formulation of the MDGs in the Area of Health, as well as to uphold transparency and accountability by releasing the results to the general public.

Scope and methodology of evaluation The Millennium Declaration addresses such issues as peace and security, development and poverty, the

environment, human rights and good governance, and meeting the special needs of Africa. In doing so, itpresents a clear direction for the role of the United Nations in the 21st Century. The Millenium DevelopmentGoals (MDGs) were created as a common framework for international development by integrating theMillennium Declaration with the development goals adopted at major international conferences and summitsheld in the 1990s. Of the MDGs, Goals 4 through 6 and Targets 5, 6, 7 and 8 are defined as MDGs in the area of

35


health (or health MDGs). This evaluation focuses on Japan’s contribution toward achieving these health MDGs.Japan’s contribution subsequent to the introduction of the MDGs (FY 2001 – 2003) was analyzed

qualitatively and quantitatively by region and aid modality.

Evaluation results(a) Japan’s assistance in the area of health and its contribution to the achievement of the MDGs

(contribution level)Although overall Japanese Official Development Assistance (ODA) showed a decreasing trend, ODA input

in the area of health as a percentage of overall ODA has risen slightly in recent years. In order to evaluate how Japan has been contributing through bilateral assistance, bilateral ODA projects in

the area of health are classified in accordance with each MDG target, and classified on the basis of “MDGclassification codes” designed for this evaluation according to the MDG target. Through this process it wasrevealed that, for example, General project grant aid assistance toward Target 5 (reduce mortality rate amongchildren under five) and Target 8 (halt and begin to reverse the incidence of malaria and other major diseases) is relatively well-funded and stable; technical cooperation projects addressing Target 8 (halt and begin to reversethe incidence of malaria and other major diseases) are also well-funded and saw a steady increase.

Additionally, it was confirmed that, with respect to Japan’s assistance through international organizations,sizeable contributions to the Global Fund to Fight AIDS, Tuberculosis and Malaria have been made and are increasing, and that Japan is also making a considerable contribution by hosting international conferences.

(b) Whether Japan’s policies and strategies toward attaining the MDGs are clear and made known(policy commitment)The new Mid-Term policy on ODA takes into consideration the MDGs as trends in international society and

further focuses on the perspective of human security. It was confirmed that the Japanese government wasclarifying its commitment to the achievement of the MDGs at a number of conferences. However, Japan has yetto develop any initiatives specifically targeting the achievement of the MDGs.

(c) Whether a strategic approach is being taken to attain the MDGs (Strategy)In the Philippines, which was the subject of a case study, the government highly appreciated Japan’s role,

while it was difficult to find any awareness of Japan’s assertive strategy for the MDGs. The government of thePhilippines has established its own MDG targets and has added “guaranteed access to reproductive health” to thetargets for health; however, it was not confirmed whether or not Japan’s assistance is helping to address thisadditional target during the field survey.

Japan, as a principle, has been implementing international cooperation on a request basis. However, Japan iskeen to provide advice and guidance to recipient countries from project formulation stage through policyconsultations, dispatch of policy advisors, project formulation studies and dispatch of project formulationadvisors and overseas survey specialists under JICA program formulation operation. There are also cases inwhich experts on experience and good knowledge on health assistance become essential.

(d) Whether high quality and its improvement are being pursued in assistance implementation inorder to ascertain results and improve efficiency (Quality assurance) Currently, the common approach to formulating projects is resource-oriented rather than objective-oriented.

Consequently, it seems likely that PDM is not being fully utilized in the manner it should be. Further, at sites where projects are being implemented, there are cases where assistance policy archives, project documents and related reports were not easily accessible. Limited access to required archives may even impede the strategic and effective implementation of assistance projects. Desirable environment is the state in which experienceaccumulated by past assistance can readily be obtained. Therefore, in some cases, it is considered that aninformation management system needs to be improved so as to ensure the systematic accumulation and utilization of institutional memory.

In the process of formulating, selecting, and implementing assistance projects, coordination with the other

36

donors is of vital importance. Since the coordination is essential to ensure more strategic effectiveness and practicality of assistance, donor meetings should be attended at all times, with the aim of looking at morepossibilities for coordination and avoiding duplication of input.

Personnel with specialist knowledge and experience are considered the key to coordination work in partnercountries, and expertise in health assistance in developing countries is essential for such a coordination role. Inorder to place the necessary emphasis on quality and the accumulation of results, it was confirmed that Japan is starting to adopt the trend in which not only the individual experts but the whole management of the projectimplementation is outsourced.

Recommendations(a) Policy commitment to the MDGs and clarification of their significance

[Recommendation 1-1] Integrated with IDI and other initiatives, Japan’s MDGs strategy should bedeclared immediately as a policy.

(b) Promotion of strategic assistance; specify strategic issues related to the MDGs to design theassistance projects[Recommendation 2-1] Input data (number of projects and amount spent) of ODA projects contributing to

the attainment of the MDGs should be classified by MDG targets, so that the inputcan be better monitored. Furthermore, project classification codes by all healthsector objectives should be defined and utilized as proposed in this report, whichcan be further defined to cover the whole sector. Project database to registernecessary information should be introduced.

Chapter

2AnOverview

ofEvalu

ationResults

[Recommendation 2-2] Establish strategic issues, and assign experts to create and design projects in linewith such issues.

(c) Introduction of a quality management system to enable effective contribution to the MDGs;promote multilateral collaboration among specialized organizations including NGOs[Recommendation 3-1] Improve documentation management methods, establishing project database, and

share knowledge and experience.[Recommendation 3-2] Effective utilization of the Project Design Matrix (PDM) as a tool to assist the

formation of objective-oriented projects.[Recommendation 3-3] Enhance domestic capacity building and the utilization of expert organizations.[Recommendation 3-4] Further promote cooperation with UN and other organizations.

37


2.1.3 Program-level Evaluations

Sector Evaluations (1) Morocco-UNICEF Country Programme Evaluation

Target Country: MoroccoEvaluation Period: December 2004 – March 2005CPE team members: Keiko Kita (education and maternal health), Global Link Management Inc.

The survey was conducted by a team of external consultantswith the Evaluation Office of the UNICEF New York Headquarters acting as coordinator

Simon Latraverse (team leader)Karen Hickson (gender and communication)Taufik Bakkala (health and childcare data systems)Azeddine Akesbi (education and child protection)

Evaluation objectives Background to the Country Program Evaluations (CPE)

This joint evaluation was conducted as a consequence of the “Fifteenth Regular Japan-UNICEFConsultation” meeting of February 2003. UNICEF is active in 162 countries and regions, where it coordinatesthe planning, implementation, monitoring and evaluation of aid programs, whose duration is normally years(country programs, hereunder). Country Program Evaluations (CPE) are conducted to obtain strategicrecommendations that will allow the program to be repositioned in accordance with any major changes todevelopment policy and/or needs that have occurred in the recipient country. The decision to undertake a CPE of Morocco was taken in consequence of reforms to the Family Code, the Labor Code and numerous otherdomestic laws affecting the rights of women and children. The purposes of Japan’s involvement in the CPE were: (1) to study UNICEF’s approach to programming and CPE methodologies, (2) to identify needs forprimary education and healthcare of mothers and children in Morocco to feedback to future project formulationand aid policy, and (3) to assess the potential for partnering with UNICEF in Morocco and to obtainrecommendations to facilitate effective coordination.

Objectives of the UNICEF CPE 1. To assess the role and relevance of the Morocco Country Program: (1) the situation of women and children

in the country, (2) national policies and strategies, and (3) international legislation, including theConvention on the Rights of the Child (CRC) and the Convention on the Elimination of All Forms ofDiscrimination Against Women (CEDAW).

2. To assess the design and focus of the Country Program, i.e. the quality and the formulation of objectivesand strategies and, more specifically, the integration of the human rights-based approach to programming.

3. To evaluate the comparative advantages of Morocco Country Program in relation to the strategies and activities of other national and external partners that contribute to the realization of women’s and children’srights.

4. To assess the effectiveness of the program, i.e. the relation between its costs and the results.5. To assess resource mobilization for the program (UNICEF’s Regular and Other Resources) as well as the

annual budget status since the last mid-term review (1999).6. To analyze to what extent the tools and models developed for this pilot project are sustainable at their

respective administrative levels and if they can be replicated at the national level.7. To strengthen the country program evaluation capacities of personnel at the UNICEF office in Morocco and

their counterparts within the government of Morocco and contribute to the development of CPEmethodologies and guidance for all UNICEF operations.

Scope of evaluation The Country Program comprises four programs: the Support for National Priority Programme, the Support

38

for Children in Rural Areas Programme, the Child Protection Programme, and the Monitoring on CRC Promotion Programme. Each program comprised two projects and between two to five sub-projects. The ChildProtection Programme consisted of four projects, but no sub-projects are identified.

Evaluation of resultsI Role:

Two roles of the Country Program are: (1) support to the government of Morocco for the realization of thefour children’s rights stipulated in the Convention on the Rights of the Child (survival, development, protectionand participation), and (2) the implementation of pilot projects with a view to the development of tools andmodels for intervention in the areas of education, health, regional development and child protection.

II Relevance:The objectives and content of the Country Program are relevant to national priorities and policies, to the

Convention on the Rights of the Child and Millennium Development Goals (MDGs), and to the declaration “AWorld Fit for Children.” However, the relevance of the Program to the Convention on the Elimination of AllForms of Discrimination Against Women (CEDAW) is less clear because gender equality was not a primarypriority. In the education sector, the Country Program is supporting national policies and strategies targeting theprovision of pre-school and elementary education for all children, as well as improvements in the quality ofeducation and the elimination of drop-outs. The Country Program is thus also contributing to the attainment oftwo MDGs: “achieving universal primary education” and “promoting gender equality”. The Country Programalso focuses on “comprehensive care of children from birth to the age of three”. Efforts to improve girls’schooling in rural areas have been insufficient. In the health sector, the relevance of Country Program objectivesto national policies is clear since most of the national programs (immunization, maternal health, etc.) establishedby the Ministry of Health have been integrated. The Country Program is also consistent with the MDG indicatoron “reducing infant and maternal mortality.”

Chapter

2AnOverview

ofEvalu

ationResults

III Design and focus:Since neither the human rights-based approach to programming nor a gender approach were identified as

design priorities for the Country Program by UNICEF, neither approach was fully applied in Morocco. Again, interms of the focus of the program, more importance was attached to operational activities than advocacyactivities.

IV Comparative advantages of UNICEF collaboration:UNICEF has comparative advantages in (1) its role as a protector of children’s rights and its advocacy of

government initiatives targeting their realization, (2) its field activities in urban and rural areas and theexperience accumulated therefrom, and (3) its activities based on the implementation of surveys and researchand the findings obtained therefrom, and the development of intervention models and tools.

V Level of achieved results:At the output level, the Country Program has produced many results that were consistent with the activities

and outputs scheduled in the Master Plan of Operation (MPO) and annual action plans. In terms of outcomes, i.e.contributions to the realization of children’s rights, the experience gained through pilot projects has eitheralready been replicated or is in the process of being incorporated into government policy.

VI Efficiency:The absence of both a “results matrix” and data on the cost of activities made it difficult to evaluate the

efficiency of the Country Program. The current Country Program comprises a large number of projects that are,moreover, distributed over sites in five rural provinces and four urban districts. Adequate management andmonitoring is therefore essential, but the UNICEF office in Morocco faces shortages of staff with the requisiteskills and experience to perform this work.

39


At a village in Chaouen, the United NationsDevelopment Programme (UNDP) has set upliteracy education and income-generating classesfor women. A community-run nursery school builtwith UNICEF assistance stands in front of thebuilding in which the classes are held, allowingmothers to devote themselves to their studies.

In rural provinces, most deliveries are assisted bymidwives in the home. The midwives conductmid-pregnancy examinations and check the health of mothers and babies one month afterdelivery. In mountainous areas, it is not unusualfor midwives to walk for two hours or more.

VII Sustainability:Since the Country Program is incomplete, neither the sustainability of the pilot projects nor the prospects for

generalization of intervention models were analyzed. However, the analysis that was made of success factors andconstraints identified several factors affecting sustainability, namely, outside funding, leadership, the motivationof stakeholders, voluntary work, community participation, institutional support for pilot projects, and systemization of results.

VIII Procurement and utilization of operating funds: From 2000 to 2004, the Country Program received 15% more on average in regular resources than the

amount budgeted in the MPO. The mobilization of funds from other donors, on the other hand, has been 60% lower than the annual forecast. However, performance on the use of program resources has been excellent, andall available resources were utilized in 2002 and 2003.

(2) Evaluation of the Bridge Construction Program for Tegucigalpa and on Main Highways in Honduras (recipient country/agency evaluation)

Target Country: HondurasEvaluation Period: December 2004 – March 2005Evaluators: Manuel Euceda, president, GEDESA

Alberto Gutiérrez, civil engineer Fausto Lazo Castillo, environmental economist

Javier Francisco Chávez, technical assistantCooperation from: SOPTRAVI (Ministry of Public Works, Transportation and Housing), Honduras

Evaluation objectives This evaluation mainly aims to provide a comprehensive assessment of Japan’s cooperation in the transport

infrastructure sector in Honduras, and to draw useful lessons and make recommendations towards more effectiveand efficient implementation of aid in the future and for the formulation of the Bridge Construction Project, aswell as to uphold transparency and accountability by releasing the results to the general public.

40

Scope of evaluation This evaluation is positioned as a sector evaluation at the program level. The evaluation covered three

projects executed as a single program, namely, the Project for the Construction of Choluteca Bypass Bridge, theProject for Reconstruction of Bridges in the Tegucigalpa Region, and the Project for Construction of the IlamaBridge and Democracia Bridge.


The primary objective of this program was to rehabilitate those bridges on the main highways that hadsuffered hurricane damage and to build permanent bridges to restore the function of the road network and tostimulate social and economic activities. The Grant Aid conferred on the government of Honduras by theJapanese government is considered relevant in light of the necessity and urgency of tackling the negative effectsof the hurricane, which destroyed virtually all the transport and production infrastructure in the country. ThisGrant Aid has produced important benefits at all levels (national, regional and community); the program is consistent with the priorities set by the government of Honduras in its Master Plan for the reconstruction andtransformation of the country, and is contributing to the priority areas outlined in the government’s programs andPoverty Reduction Strategy Paper (PRSP).

Chapter

2AnOverview

ofEvalu

ationResults

(b) Evaluation of outcomesThe objectives of this program are considered to have been fully achieved. This conclusion is based on the

fact that the constructed bridges overcome any serious problems caused by natural disasters and that they havecontributed to the rehabilitation of infrastructure and to the revitalization of production capacity in Honduras.(1) Effectiveness

Bridge construction work commenced in October 2000 and was completed in April 2003. The efficiencyand quality of the construction of seven bridges are acknowledged.

(2) ImpactFor this evaluation, information was collected through interviews conducted with representatives from thosegovernment agencies and institutions involved, with users and in the communities directly affected by theconstruction of the bridges. This information was analyzed, and it was confirmed that the impacts producedby the program were similar for all the constructed bridges.

Technical impactThe design used in the construction of the bridges is extremely modern in that it guarantees the safety andsustainability of the structures, thereby rendering them resistant to natural disasters. The technologies andprocedures used by the Japanese contractors involved were far superior to what Honduras could offer,which offers the assurance of high levels of quality and performance and makes it possible to limit anyflood or other damage attributable to natural disasters.

Institutional impactThis bridge construction program made a considerable contribution to the introduction of national strategytargeting improvements in the road network. It also enhanced the technical capabilities of those local organizations and companies involved in the development of highway infrastructure and the effectivetransfer of technologies throughout the construction process.

Economic impactThis bridge construction had a favorable impact on the process of reactivating domestic industries,particularly agriculture, tourism development and the service industry, thereby contributing to the creationof employment and to increases in household income.

Political impactThe implementation of this program strengthened the mechanisms for regional integration and made asignificant contribution to domestic and international trade policies. The program facilitated the executionof government policy on highway infrastructure and the Master Plan for rebuilding the nation.

41


Social and cultural impactUnder the principles of discipline, efficiency and safety, the bridge construction process had a favorableimpact on local laborers, and on commercial and community stakeholders in Honduras. The constructedbridges have realized safe movement of people and goods, and reduction of traffic accidents, andimproved access to educational and medical services.

Environmental impactIn several cities, less congestion on the roads has served to reduce pollution and to improve the livingstandards of local residents.

(c) Evaluation of processAll application and negotiation processes involving the governments of Honduras and Japan were

appropriately conducted, thereby facilitating the formulation and implementation of the three projects thatformed the program. This program was more compatible with the assistance extended through other Japanese aidmodalities, i.e. non-project and non-refundable Grant Aid, and allowed for adequate coordination and alignmentwith the projects implemented by other donors in response to the “Hurricane Mitch Emergency” in Honduras.

Besides Japan, assistance for the high-priority restoration of highway infrastructure (bridges) was extendedby Sweden, the UK, the USA, CABEI and IDB. At the request of the Japanese government, donor meetings wereheld throughout the implementation process, thereby allowing different aid modalities, particularly those of Sweden, the UK and the USA, to be properly coordinated.

Lessons learned and recommendations Lessons learned and recommendations on the objectives

Inadequate infrastructure development contributes to the country’s vulnerability, and the destruction of itsmain highway network can essentially paralyze production and service systems.It is critical that the government agencies responsible for the maintenance and monitoring of the nation’shighway infrastructure step up their efforts to accomplish these tasks, and that efforts are made to strengthen local government capabilities in this area.

Lessons learned and recommendations on the resultsNo rigorous and/or systematic efforts are made to apply the laws and regulations pertaining to appropriatemanagement of natural resources in Honduras.In view of the fact that the roles and responsibilities of the various agencies involved in the maintenanceand monitoring of bridge infrastructure have not been defined, it is necessary to strengthen thedecentralization policy initiated by the government.The solidarity demonstrated by the government of Japan in response to the emergency was vital to therehabilitation of highway infrastructure and to the recovery of production capacity in Honduras.

Lessons learned and recommendations on the processThe damage caused by the hurricane that hit Honduras in 1998 clearly proved the critical vulnerability ofthe country’s environment in consequence of the rapid degradation of natural resources and the absence ofany preventive policy.The encounter with Hurricane Mitch demonstrated the limited capacity of Honduras to respond to emergencies and the extent of its dependency on foreign aid.It is important that the government of Honduras increases its efforts in the handling of natural resourcesand in the application of measures for land readjustment in order to reduce its ecological vulnerability andsafeguard its highway and productive infrastructures.Monitoring and control measures must be stepped up to reduce the procurement of construction materials(sand and stones) in areas peripheral to the bridges and prevent riverbed erosion.

42

Aid Modality Evaluations (1) Review of Adjustment Lending: Overview of Structural Adjustment Loans and

Sector Adjustment Loans

Evaluation Period: August 2004 – March 2005Chief Evaluator: Yayoi Tanaka, Associate Professor, Graduate School of Engineering,

University of Tokyo Member of the MoFA External Advisory Meeting on ODA Evaluation

Advisors: Yasunori Fujita, Associate Professor, Faculty of Economics, Keio UniversityYoji Kunimitsu, Chief Researcher, Laboratory of Planning Evaluation,Department of Rural Planning, National Institute for Rural Engineering

Consultants: Fumihiko Kamio, Nomura Research Institute, Ltd. Shigeharu Satake, Nomura Research Institute, Ltd.

Review objectivesIn recent years, budget support has been the subject of much discussion. After having tried it in some cases,

Japan is studying carefully whether to introduce budget support. This evaluation mainly aims to draw usefullessons and make recommendations toward more effective and efficient implementation of aid in the future and for the formulation of Adjustment Lending, as well as to uphold transparency and accountability by releasing theresults to the general public.

Chapter

2AnOverview

ofEvalu

ationResults

Scope and methodology of reviewThe review team set adjustment lending as the main objective and overviewed Structural Adjustment Loans

(SALs) and Sector Adjustment Loans (SECALs) which were perceived as adjustment lending. The PovertyReduction Support Credits (PRSCs) applied in December 2004 were also referred to. This review was carried outin 3 steps; (1) clarification of the object of the review, (2) overview of the object, and (3) simulations of theeffects of PRSCs. The review team attempted to gather necessary information for SALs and SECALs extendedbetween 1986 and 2002. However, there were limitations, since some of the programs date back many years andothers contained undisclosed information.

Review results(1) Overall trend

Adjustment Lending was started by the World Bank in 1980. The original instruments were SALs andSECALs designed to help developing countries suffering from the financial crisis caused by the 1979 oil shockand the sudden rise in the international interest rate. Japan started SALs in 1986 and had provided a total of967.6 billion yen through 78 SALs and SECALs by 2002.

SALs and SECALs reached a peak in terms of number and amount in FY 1988 (14 loans, about 200 billion yen). At that time, adjustment lending was utilized in a series of international supports because of the severeproblem in international balance of payments in many developing countries. The “Baker Plan”1 and Special Program of Assistance for Africa (SPA), established as a support framework for Africa, were major reasons forthe high-level disbursement of SALs and SECALs. It is assumed that Japan increased its SALs and SECALsalong with these international movements. During the first half of the 1990s, up to 1996, SALs and SECALsshowed a slow decline in terms of number and amount. Rising criticism of the failure of structural adjustment to achieve its original purpose might have been the reason for such a decline. The expansion of debt relief anddecrease of need in Asia were also related to this trend. In 1997, when the Asian currency crisis took hold, Japanincreased SALs and SECALs again. After 2000, SALs and SECALs were decreased again to mitigate the economic crisis in Asia and increase focus on poverty reduction.

1 This was a debt relief plan announced by US Treasury Secretary James Baker at the World Bank-IMF annual meeting inSeoul in 1985.

43


(2) Trends in purpose Behind the trend in the late 1980s was a need for resources to be disbursed quickly to decrease imbalances of

payment in recipient countries. Neoclassical economics, which promotes small government, was another reasonfor the trend. In the early 1990s, the purpose of macroeconomic policy support remained to be centered in accordance with the Washington Consensus1, while some programs designed to address public and socialpurposes as well as those targeting the economic and industrial sector were adopted in the light of manycriticisms against adjustment lending. After 1997, programs combining multiple purposes, namely, socialdevelopment, balance of payments support, and support for economic and industrial sectors became prominent to meet various needs. The currency crisis in Asia in 1997 demanded that resources be disbursed in the short term.It was widely accepted that support for vulnerable people was necessary.

(3) Trends in conditionalityWhen looked at on a single-loan basis, in the late 1980s, conditionality was mainly attached to loans

addressing macroeconomic objectives, including the private sector and economic policy, but affected relativelyfew sectors. Since then, the number of sectors affected has increased to 4 or 5, including the public sector andsocial development.

(4) Trend by country and regionIn terms of the amount of Japan’s SALs and SECALs, 44.8% is directed to Southeast Asian countries and

22.6% to Africa. In each region, the loans were directed to a few countries in which economic problems andpoverty were not as serious as in others. It is said that the programs are selected on the basis of requests fromrecipient countries. However, this study could not clarify the core principles and criteria for choosing a programif different programs are requested by these countries. On the other hand, the World Bank directed its resources to countries in Latin America, Southeast Asia and Eastern Europe, where currency crises and serious imbalancesof payments had occurred.

Results of the quantitative analysis of PRSC This quantitative analysis traced the path of effects of PRSCs in four levels of its logic model. As a result, it

was found that sectors that are effective for social development are not necessarily the same as those effective forGDP growth. This shows that financing PRSCs would not automatically result in poverty reduction through GDPgrowth brought about by social capital accumulation. It is suggested social infrastructure and financial sectorsneed to be supported separately fromthe economic sector.

Issues concerning SALs and SECALs 1) Criticisms on theories: the basic conditions implicated in neoclassical economics do not fit realities in

developing countries.2) Criticisms of the social aspects: structural adjustment has negative impacts on the social aspect in recipient

countries, especially on vulnerable people.3) Criticisms of the operational issues: the approach and attitude of donor countries and institutions are

coercive.Conditionality attached to loans for social development is gradually being increased. Criticism of the theories

was taken into consideration. In August 2004 the World Bank renamed SALs, SECALs and other adjustmentlending instruments “Development Policy Lending”. PRSCs, one of the main instruments of Development PolicyLending, focus on social concerns and local ownership. In this sense, criticism of social sectors and operationalissues was taken into consideration.

1 Washington Consensus is a summary of economic adjustment policies advocated by J. Williamson in 1990. He produced it to summarize the majority opinions in Washington, D.C.: the US Ministry of Finance, the FRB, the World Bank and theIMF. It includes 10 instruments such as trade liberalization and privatization.

44

Future issues (a) Operational issues1) Making clear policies on PRSCs

The information that was difficult to collect during this review (information explaining government policydecisions, Japan’s share of the total amount of co-financed programs, etc.) must be made available, andresearch and analysis that will enable policy decisions on PRSC must be conducted.

2) Stabilization of macroeconomy and poverty reductionThe basic purpose of PRSCs is the stabilization of the macroeconomy. It is necessary to clarify its (1)thinking on customization (i.e. formulation of support policies for development appropriate to the recipientcountry situations), and (2) the causal link between macroeconomic growth and poverty reduction.

(b) Issues for future research1) Assessment of the medium- to long-term effects of adjustment lending

There is a need for quantitative and qualitative research and analysis on the extent to which goals have beenachieved and what impact these have had in recipient countries; this should also cover contributions madeto poverty reduction as well as any negative impacts produced by adjustment lending.

2) Investigation that supports policies

Chapter

2AnOverview

ofEvalu

ationResults

To facilitate the feedback of results, information on decision-making in Japan needs to be collected andanalyzed.

3) Research of management issues concerning new modality including PRSCsMethods of addressing the substantial coordination costs generated during the process of buildingconsensus between recipient country and donors, and of securing the necessary experts, need to beexplored.Given the shortage of statistical data in developing countries, methods for evaluating PRSCs need to beestablished and their effectiveness tested to ascertain the need for further improvement.Customization (the extent to which conditionality reflects the recipient country’s culture and society)Research needs to be conducted on the measures taken to address the moral hazards involved in SALs andSECALs, and what effects these are producing.

(c) Direction of PRSCs from quantitative analysisFrom the sensitivity analysis, it is observed that an investment in a specific sector which contributes to GDP

growth is not automatically related to poverty reduction, and hence that it is more important to invest in thosesectors that will have a direct impact on social development indicators. It is also observed that applying the samefunding conditions to all sectors will not necessarily furnish an effective link to poverty reduction. On futurePRSCs, rather than extending equal support to all sectors, clear targets will need to be set for poverty reductionand economic growth and funding directed to those sectors that will have a significant impact on the attainmentof these goals.

45


(2) Evaluation of “Grant Assistance for Japanese NGO Projects Modality” (NGO-MoFAjoint evaluation)

Evaluation Period: September 2004 – March 2005 Evaluation team: This evaluation was conducted by a team of evaluators from NGOs and

the Ministry of Foreign Affairs with the support of a consultant.[NGOs]

Masato Noda: Trustee, Nagoya NGO Center/Associate Professor, ChubuUniversity

Nobuhiko Katayama: Vice-chairman, Japan NGO Center for InternationalCooperation/National Director, World Vision Japan

Yasushi Nozaki: Trustee, Nagoya NGO Center/Associate Professor, Nihon Fukushi University

[Economic Cooperation Bureau of the Ministry of Foreign Affairs] Kanji Kitazawa: Senior Coordinator, Aid Planning DivisionMasanori Nakano: Principal Deputy Director, Non-Governmental Organizations

Assistance DivisionYae Kosugi: Deputy Director, Non-Governmental Organizations Assistance

DivisionTomohiro Nakagaki: Deputy Director, ODA Evaluation Unit, Aid Planning Division

[Consultants]Yukimasa Fukuda: Senior Deputy Director, Foundation for Advanced Studies on

International Development (FASID) Takahiro Miyoshi: Consultant, Foundation for Advanced Studies on International

Development (FASID)

Evaluation objectives This evaluation mainly aims to draw useful lessons and make recommendations for more effective and

efficient implementation of aid in the future, and for the formulation of the Grant Assistance Project, as well asto uphold transparency and accountability by releasing the results to the general public.

Evaluation methodologyThis evaluation study was carried out in light of three perspectives; namely 1) relevance of the Modality

“purpose”, 2) adequacy of the Modality’s operational “process”, and 3) “results” of the Modality’s operation. Itcomprised the following components: literature review, a questionnaire survey, interviews with stakeholders in Japan, and a field survey in Cambodia.

Evaluation results(a) Evaluation of purpose

Relevance to Japanese government policy on development assistance: The Modality aims to providefinancial assistance for social and economic development and emergency humanitarian assistance projectsundertaken by Japanese NGOs in developing countries and regions. It is in line with Japan’s ODA Charter(current and former) and the Mid-Term Policy on ODA.Relevance to the needs of Japan’s NGOs: The Modality supports the needs of NGOs as it addresses thefinancial gap. As for the provision for headquarters project implementation expenses, the Modality is comparable to other supporting modalities. While it is true that many of the supported projects havetangible aspects, this trend is not due to limitations of the Modality. Whether NGOs can obtain funding fortheir proposed projects may depend more on the capacity of NGOs to design and plan effective projects

46

that combine tangible and intangible components, as well as the capacity of MoFA to select such projects.Relevance to recipient country needs (governments, counterparts, local residents): The governments ofrecipient countries expect financial, material and technical assistance from Japanese NGOs as well as NGOs’ partnership role in compensating for the inadequacies of government services. Local NGOs in recipient countries look to Japanese NGOs to enhance skills and capacity through technical support. Localresidents look to Japanese NGOs for improved living standards through project delivery. Principally, theModality is evaluated to be relevant to the needs of recipient countries. The field survey in Cambodiarevealed that from the perspective of “human security”, official aid through Japanese NGOs, such as thisModality, could be especially effective in remote areas and with the underprivileged. The single-yearcommitment is applicable for all Japanese public budgets; nevertheless this Aid Modality can be tailoredto address long-term developmental needs in the recipient country.

(b) Operational “results”Performance of fund provision: Performance of the grant provision through this Modality increased from1.29 billion yen in FY 2002 to 1.76 billion yen in FY 2003. The budget execution rate also improved, from64.5% in FY 2002 to 80% in FY 2003. However, “Assistance for NGOs in Partnership Projects” and“Micro-credit Capital Assistance” remain sluggish.

Chapter

2AnOverview

ofEvalu

ationResults

Attainment of the assistance objectives: The majority of NGOs mentioned the contribution of thisModality to filling the financial gap. However, it is difficult to judge whether this Modality directly linksto the solution of fundamental problems in the financial circumstances faced by NGOs. It is required for NGOs to consider their organizational capacity when receiving assistance. MoFA on the other hand, isrequired not only to assess NGOs’ fund management capacity but to provide appropriate advice. Almostall projects have achieved their objectives according to the questionnaire survey. However, the study teamwas not presented with clear visions or specific steps on project sustainability from NGOs or other stakeholders.Positive and negative impacts: As positive impacts, NGOs remarked that this Modality has strengthenedthe Japanese presence in the respective countries or areas, and promoted constructive relationships amongNGOs and between NGOs and local governments. Other NGOs remarked that it strengthened theorganizational capacity and project management capacity of NGOs themselves through execution of theprojects. On the other hand, some NGOs pointed out the long waiting time between submission of theproposal and project selection, which in effect compelled some changes in the plan. One NGO pointed outthe increased dependency of NGOs themselves on public funds.

(c) Adequacy of operational “process”Adequacy of selection and fund provision procedures: Project proposals submitted by NGOs are comprehensively determined and selected against fair and relevant criteria by MoFA. However, the prepared proposal format is not necessarily made to address the specified criteria, and calls for review.Many NGOs pointed out the lengthy process of project selection, and lack of transparency of the selectionprocess. Through meetings with MoFA and NGOs, the evaluation team agreed that the Modality’sprocedures and management aspects are improving. One issue however, is the need for NGOs to enhancecapacity in proposal drafting and articulation.Adequacy of financial management by NGOs after the provision of funds: The grant money is depositedinto the NGO’s bank account in the country of operation, and NGOs must submit a mid-term financialreport and execute an external audit. No major fraudulence or defects have been reported in the past. Formost projects, monitoring and evaluation activities are questionable. At the same time, few NGOs are in compliance with the requirement for publicizing project information and a project completion report, andimprovement is called for, especially from the perspective of securing accountability.

47


RecommendationsNGO activities reinforcing ODA policy from a “human security” perspectiveThere were no cases of NGO proposals for this Modality deviating significantly from the Country Assistance Program. Nevertheless, if this kind of proposal is presented, such proposals should be viewedfrom a “human security” perspective, and it should be selected if it may fill the gap that the CountryAssistance Program failed to address. NGO activities may complement government ODA policy

NGO’s organizational capacity and provision of public fundsThe provision of public funds should be preceded by prudent assessment of the NGO’s capacity to managethose funds considering the weak structure of NGOs. MoFA assesses NGO’s capacity to manage fundspreceding the submission of the proposal. Also, MoFA is expected to promote its capacity development toprovide comprehensive advice to strengthen the institution of NGOs.

Provisions of “intangible” expenses and administrative/indirect costs, mutual understanding betweenNGOs and MoFAThis Modality provides for “soft” expense of locally spent costs as well as for “project implementationexpenses of headquarters” in Japan; this Modality bears comparison with other assistance modalities onsuch administrative expenses. NGOs should familiarize themselves thoroughly with this Modality, anddraft proposals clearly articulating the need for funds. MoFA personnel need to disseminate informationabout the Modality and develop consulting skills to facilitate NGO application processes to be coherentwith the guidelines of the Modality.

Efforts for multi-year projectsFormally, it is impossible to provide support for multi-year projects under the current budget system.Nevertheless, MoFA holds that this Modality can secure project continuity by accepting proposals for the next term prior to project completion for the current term, provided that the interim report is proved to beadequate. NGOs should develop project plans and proposals with a long-term perspective whileformulating plans according to single-year project formation. In such cases, it is important to conductproject evaluation yearly. It is necessary to set clear goals and enhance the project evaluation capacity.

Capacity development and reinforcement of partnership between MoFA and NGOs in applicationproceduresMany NGOs mention delays in screening procedures. In order to improve this point, it is necessary toenhance the quality of the application form developed by NGOs. In addition, MoFA shouldcomplementarily support NGOs in case they lack the capacity to prepare the application form.Furthermore, capacity to screen proposals at the Japanese Overseas Diplomatic Mission (JODM) shouldbe enhanced through review of the JODM system, and staff training and elaboration of the operationalmanual should be studied and introduced.

NGO-MoFA joint evaluationFor instance, it is useful to select a pilot case for the multi-year projects and implement an NGO-MoFAjoint evaluation to draw concrete lessons. This kind of joint evaluation helps to build constructivepartnership between NGOs and the MoFA through mutual learning.

OthersIt is necessary to strengthen NGOs’ evaluation and monitoring of projects implemented through thisModality. Also, few NGOs disclose project completion reports on their own website, which is arequirement of this Modality. From the perspective of accountability, NGOs should commit toimproving performance.Current application forms are not necessarily consistent with the screening criteria. There is room forimprovement of the current proposal format.

48

Children photographed during the field survey Children who have benefited from the primaryschool construction project by Japan Center for Conflict Prevention

(3) Program Assistance: The Democratic Republic of Timor-Leste (MoFA-USAID joint evaluation)

Chapter

2AnOverview

ofEvalu

ationResults

Evaluation Period: August 2004 – March 2005Study team: This study was carried out by consultants from the two countries

commissioned by the United States Agency for International Development(USAID) and the Ministry of Foreign Affairs, Japan.

Jeff Malick Project Director, Development Information Services (Team Leader)Anne Beasley Senior Research Analyst, Development Information ServicesAndrew Melnyk Senior Economic Analyst, Development Information ServicesShinichi Mizuta Researcher, International Project Center, Mitsubishi Research Institute

Objectives and scope of evaluation This evaluation mainly aims to review the Democratic Republic of Timor-Leste’s experience with program

assistance (General Budget Support), determine whether it has been an appropriate method of disbursing aid, and identify the country conditions that are favorable for such assistance. The specific focus of this study isTimor-Leste’s Transition Support Program (TSP), the primary program that uses budget support for assistance in Timor-Leste.

Evaluation results(1) Objectives of General Budget Support (GBS)

In general, the primary objective of GBS is to provide flexible resources for development based on an agreedprogram and shift the responsibility of development from donors to host governments, encouraging the latter totake a more active role in their own development. Countries with weak and unaccountable governments, lowcapacity, or insufficient public financial management are usually poor candidates for GBS, as are countries withsystematic or large-scale corruption, inadequate or non-existent monitoring and evaluation systems and/or auditagencies.

(2) Outline of Timor-Leste’s Transition Support Program (TSP) The TSP provides GBS and has been in operation for over two years. It is coordinated and managed by the

World Bank and funded by both the World Bank and ten bilateral donors. Total donations were approximatelyUS$91 million as of September, 2006. Though the TSP was expected to stabilize oil and gas revenues and wasoriginally scheduled to be phased out by 2005, the government of Timor-Leste (GOTL) has requested thatfunding be extended to 2007 in consideration of the holdups in oil and gas field development.

49


The goals of the TSP are to; Provide a temporary source of funding to augment government revenue;Enable implementation of Timor-Leste’s National Development Plan (NDP);Build sufficient government capacity so that future oil and gas revenues will be appropriately managed.

(3) Country characteristics and TSP objectives The following conditions were taken into consideration when considering aid modalities for Timor-Leste;

It is a new, post-conflict country, with little revenue outside of foreign assistance, and there is a need forfinancial resources to be provided through a flexible instrument;Human and administrative capacity are especially low, forcing the government to rely on large numbers of foreign advisors and staff;There is little tradition of entrepreneurship and few natural resources that can be exploited;Unlike most post-conflict countries, there is stability and little chance of renewed violence;Off-shore oil and gas deposits are expected to generate significant revenue, most likely by 2007.

(4) Conclusions The TSP is facilitating the GOTL initiatives under the GOTL ownership and serves as an effectiveframework for monitoring. Much of the success of the TSP is attributed to World Bank monitoring andprogram design. Bilateral donors contend that World Bank oversight offers an effective way of managingtheir assistance while requiring minimal staff on their part.Though there is no explicit conditionality, donors and the GOTL meet twice a year to review progress andto set new goals and priorities. During this time donors can discuss issues of importance with high-levelofficials. In this way donors are given a forum in which to voice their concerns.From the GOTL’s perspective, the reviews and strong donor harmony are a source of pressure on the government and are perceived to encroach on sovereignty to some degree. Nonetheless, the GOTL hasachieved positive results and remains actively engaged in the process.One of the reasons that the TSP is functioning so well lies in the fact that budgetary decision-makingpowers are centralized in the Ministry of Planning and Finance (MoPF), which exerts powerful controlover budgetary management. Donors report that the current system of public expenditure is sound and thatthe incidence of corruption is minimal and low-level. However, by centralizing such authority,inefficiencies can occur, and the host government may not release funds in a timely fashion.Local NGOs have not been involved in TSP consultation in a systematic and meaningful way. But the taskof involving a large and diverse group in an organized, representative, and substantive manner presents achallenge and requires effort by the NGO community in concert with the host government and donors.With regard to the relative merits of various types of assistance over others, the GOTL has had too littleexperience with projects and programs to state a strong preference. However, they are leaning towardsprojects which they view as a means of providing needed capacity building assistance if managed jointlywith donors in various sectors.As a new government with fledgling institutions, the capacity building aspects to the TSP were crucial toinstilling the skills of public expenditure management largely through World Bank auspices with supportfrom other donors. In East Timor’s post-conflict environment, this capacity building within MoPF was keyto engendering donor support for the TSP. Furthermore, this type of capacity building ought to beconsidered in other country contexts where there are established institutions, and donors intend to channellarge amounts of their assistance through a GBS mechanism.Because government revenue from oil and gas will soon be generated, Timor-Leste has an obvious andnatural exit strategy for GBS. In part, the strategy is also based on whether the country will have thefoundational capacity to manage its budget, albeit with the assistance of expatriate advisors.

50

(4) Evaluation of Grass-Roots Human Security Grant Aid for Bolivia (evaluation byrecipient country/agency)

Evaluation Period: January 2005 – March 2005Country Evaluated: BoliviaEvaluators: María Eugenio Jurado

Vice-Ministry of Public Investment and External Financing (VIPFE) within the Ministry of Finance, Bolivia (contact point for foreign aid)Julio César Mallón (Consultant), etc.

Evaluation objectives This evaluation mainly aims to review Japan’s achievements in Grass-Roots Human Security Grant Aid by

assessing eleven of 136 projects funded under this aid modality in Bolivia during the period in question (FY2001-2003). The eleven projects cover the areas of education, health and productivity. The evaluation is based on workshops and questionnaires targeting local residents (approximately 456 participants), and is intended to draw useful lessons and recommendations towards more effective and efficient implementation and the formulation ofaid in the future, as well as to uphold transparency and accountability by releasing the results to the generalpublic..

Chapter

2AnOverview

ofEvalu

ationResults

Evaluation resultsEvaluation perspectives

This evaluation study provides a comprehensive assessment of the objectives, results, and process of Japan’sgrassroots human security grant aid projects in the areas of education, health and productivity. This was aparticipatory evaluation and specifically, the following six approaches to realizing human security wereevaluated.

Assistance that puts people at the center of concerns and that effectively reaches the peopleAssistance to strengthen local communitiesAssistance that emphasizes empowering of peopleAssistance that emphasizes benefiting people who are exposed to threats Assistance that respects cultural diversityCross-sectoral assistance that mobilizes a range of professional expertise

(1) Relevance of grassroots human security grant aid objectivesBasic Human Needs, which are the main target of Japan’s Grass-Roots Human Security Grant Aid, are

consistent with the development strategies of the Government of Bolivia and with the development priorities ofthe Bolivian government organizations responsible for the respective target areas (health, education, etc.). They are also consistent with Japan’s ODA Charter and the Mid-Term Policy on ODA.

(2) Effectiveness and impactsThe six approaches to human security outlined in the Mid-Term Policy on ODA are evaluated as follows:(a) Assistance that puts people at the center of concerns and that effectively reaches the people

The assistance targeting people and local residents based on their human needs enabled the safety andcomfort of project beneficiaries. According to the questionnaire survey of project beneficiaries, theperformance rates were: 72% in education, 76% in health, and 84% in production.

(b) Assistance to strengthen local communitiesThe assistance for the communities strengthened their institutional capacity building and helped them topromote local development. Assistance to the production sector was particularly effective instrengthening participation process among farmer and producer organizations. According to thequestionnaire survey, the performance rates were: 60% in education and health, and 76% in production.

51


(c) Assistance that emphasizes empowerment of peopleExperience gained through project formulation, implementation decision mechanisms, teamwork duringconstruction and project management served to strengthen the empowerment of people. According to thequestionnaire survey, the performance rates were: 60% in education and health, and 76% in production.

(d) Assistance that emphasizes benefiting people who are exposed to threats The assistance was targeted at urban and rural communities exposed to threats (deprivation and the senseof anxiety), with priority assigned to those communities whose livelihoods, subsistence, and dignity are endangered. According to the questionnaire survey, the performance rates were: 80% in education, health and production, respectively.

(e) Assistance that respects cultural diversityThe assistance contributed to create a society and communities where both ethnic and cultural diversityare acknowledged and respected. According to the questionnaire survey, the performance rates were: 80%in education, health and production, respectively.

(f) Cross-sectoral assistance that mobilizes a range of professional expertiseCoordination with local government authorities and relevant government ministries located in projectareas are partial and more needs to be done to establish links with local governments in particular.According to the questionnaire survey, the performance rates were: 60% in education, health andproduction, respectively.

(g) The workshop and questionnaire process not only made to raise awareness of the Grass-Roots HumanSecurity Grant Aid known amongst local residents but also strengthened empowerment of the people

(3) Appropriateness of Grass-Roots Human Security Grant Aid plans and implementationprocess

Project formulation stage: The implementation agency was expected to set a general goal and did not haveset goals or strategies specific to individual projects. There is thus a need to set specific goals to implement theprojects efficiently.

Implementation process: Projects were implemented adequately, but greater flexibility is needed for projectsinvolving construction and/or renovation because of occasional design modifications.

RecommendationsThe following recommendations were drawn regarding the objectives, process and results of the Grass-Roots

Human Security Grant Aid modality as well as the effectiveness of the six approaches to human security stated in the Mid-Term Policy on ODA.

(1) This participatory evaluation not only served to publicize the aid modality and the six “human security”perspectives, it also contributed to strengthen the empowerment of local communities after theimplementation of the projects. Participatory evaluations should be conducted on a regular basis.

(2) Although priority was assigned to the areas of health and education, it is also necessary to considerformulating production related projects, as assistance to this sector tends to help strengthen communityassociations.

(3) Even though there might be problems of project formulation and management capacity, projects should beimplemented systematically in rural and vulnerable areas with access limitations.

(4) It is necessary to manage the specific indicators systematically when it is possible to set such indicators inthe area of education, health, and production. This will increase the burden on project participants, but suchdata will facilitate evaluations in the future.

(5) The concept of “Human Security” is not well known among the organizations, including the Ministry ofEconomic Development, while a few organizations such as MOF share the concept. Therefore, informationon the Grass-Roots Human Security Grant Aid modality as a means of materializing this concept needs to

52

be exchanged and disseminated.(6) In order to clarify the extent of contributions made by grassroots human security grant aid, baseline surveys

covering economic and social indicators should be conducted in target areas.(7) A mechanism for coordinating various levels of government (central, provincial and local) needs to be

established in order to ensure sustainability.

Chapter

2AnOverview

ofEvalu

ationResults

Participatory evaluation workshopsin progress (Sucre, Bolivia)

53


2.1.4 Status of Follow-up on Past Evaluation Results

This section presents the status of follow-up activities on the evaluation results reported in the previousAnnual Evaluation Report on Japan’s Economic Cooperation (2004).

The Internal Study Meeting on ODA Evaluation set up in the Economic Cooperation Bureau* of the MoFAdiscusses and decides what measures to take in response to the recommendations obtained from the evaluationsand follows up evaluation results. Some of the recommendations may be responded to immediately, while othersmay require detailed discussion or study and all recommendations are valuable indications for the improvementof future ODA.

In the previous Annual Evaluation Report on Japan’s Economic Cooperation, the evaluation results ofCountry Policy Evaluations, Priority Issue Evaluations, Sector Program Evaluations and Aid ModalityEvaluations were presented.

Evaluations reported in the previous Annual Evaluation Report on Japan’s Economic Cooperation (2004)

Country Policy Evaluations: Indonesia, India, Jordan, PakistanPriority Issue Evaluations: Okinawa Infectious Diseases Initiative (IDI) Mid-term EvaluationSector Program Evaluations: Infrastructure Development Sector Cooperation in Papua New

GuineaWater Resources Development Sector Cooperation in the Kingdom of MoroccoJapan’s ODA to the Education Sector in GhanaEnvironment Sector Cooperation in SenegalBasic Human Needs Cooperation for Bolivia

Aid Modality Evaluations: Japan Disaster Relief TeamsJapan’s Cultural Grant Aid

The main recommendations of these evaluation studies are: (1) the need to specify priority areas andcross-cutting themes; (2) the need to strengthen coordination with other donors, international developmentorganizations and NGOs, etc.; (3) the need to strengthen monitoring and evaluation systems; and (4) the need toestablish an “all-Japan” support system beyond ministerial boundaries for the control of infectious diseases.Based on these recommendations, the MoFA has implemented the follow-up measures below.

(1) Setting priority areas/cross-cutting themes, and creating an objective tree As the volume of Japanese aid is declining, it has been recommended that priority areas and cross-cutting

themes be clarified and assistance provided strategically selecting the area of focus for assistance. Therecommendations drawn from the Country Policy Evaluations for India and Pakistan both suggested that Japanshould create an objective tree in the policy formulation stage as a means of clarifying its policy concepts andstrategies, and ensuring the effective and efficient implementation of ODA.

In response to these recommendations, MoFA specified priority areas and cross-cutting issues in its CountryAssistance Program for Pakistan (formulated in February 2005) and incorporated an objective tree in whichJapan’s aid objectives, the direction of its aid strategies, and the priority issues and areas for its assistance arespecified. Likewise, the objective tree combining priority areas and development issues was also included in theCountry Assistance Program for Indonesia (formulated in November 2004), and the inclusion of this tool is also considered for other Country Assistance Programs in the formulation process.

* International Cooperation Bureau as of 1 August 2006

54

(2) Coordination with other donors, international development organizations and NGOs

The Country Policy Evaluations for Indonesia, Pakistan and Jordan and Okinawa Infectious DiseasesInitiative (IDI) Mid-term Evaluation all pointed out that Japan needs to strengthen its coordination with otherdonors, international development organizations and NGOs.

On the basis of the policy to “strengthen the functions of field missions” outlined in Japan’s new ODACharter, MoFA, through local ODA Task Forces, is working actively to exchange information and reinforce itsdialogue to establish close ties with other donors, international development organizations and NGOs byencouraging local ODA Task Forces by participating in sector-wide donor meetings and exchanging views withNGOs.

(3) Reinforcing monitoring and evaluation systemsThe recommendations drawn from the Country Policy Evaluations for India and Pakistan both advocated the

establishment of monitoring and evaluations systems for Country Assistance Programs as a means of ensuringthe effective and efficient implementation of Japanese aid.

In response, MoFA has been conducting evaluation on the occasion of formulation or revision of the CountryAssistance Programs. In the new Country Assistance Program for Pakistan, for instance, it is clearly stated that the local ODA Task Force shall review the progress of implementation of the assistance program and that theresults of the review will be fed back to the evaluation to be conducted at the moment of revision of the CountryAssistance Programme.

Chapter

2AnOverview

ofEvalu

ationResults

(4) Establishing an all-Japan support system beyond ministerial boundaries for thecontrol of infectious diseases

One of the recommendations made on the Okinawa Infectious Diseases Initiative (IDI) Mid-term Evaluationwas the establishment of an all-Japan support system removing the boundaries among the concerned ministriesfor infectious disease control, in order to facilitate the effective use of limited ODA resources and enhance globalpublic relations activities.

In response, MoFA is leading efforts to strengthen inter-agency coordination. As part of these efforts, MoFA,JICA and JBIC organized a panel exhibition illustrating the AIDS initiatives being taken across Japan at theAsia-Pacific AIDS Congress held in July 2005 to publicize this among domestic and international stakeholders inthe health sector. Roundtable meetings involving representatives from MoFA, MHLW (Ministry of Health,Labour and Welfare) and MEXT (Ministry of Education, Culture, Sports, Science and Technology) are heldregularly to discuss assistance for health and to facilitate coordination and cooperation in the fight againstinfectious diseases.

55


2.2 Results of Evaluations Conducted by Other Ministries and Agencies

This section summarizes the ODA projects and evaluations conducted by other ministries and agencies in FY2004. The ODA implemented by these ministries and agencies can be categorized broadly into: acceptance oftrainees; dispatch of experts; research and surveys; and contributions to international organizations. As of thecurrent fiscal year, all the ODA implemented by these ministries and agencies will be listed in tabular form (see the Reference section at the end of this report). The evaluation reports included in this section are the results ofmajor evaluation studies, while the selection of specific evaluation studies to be included was left to the decisionof each ministry and agency. In principle, the reports included in this section are policy evaluations conducted inaccordance with the Government Policy Evaluations Act (GPEA); however, some non-GPEA evaluations arealso included for reference. As for the level of evaluation, it can be policy-level, program-level, and project-level,and as timing of evaluation can be classified into ex-ante and ex-post evaluation1.

2.2.1 The Financial Services Agency

Technical Assistance to Banking Regulators in Emerging Market Economies(ex-post evaluation)Evaluator: Financial Services Agency

1. Outline and objectives of policies In developing countries, a sound and stable financial system and trouble-free money markets are critical to

sustainable economic development. Moreover, in the midst of ongoing financial globalization, the stability offinancial systems in developing countries plays a vital role in the stabilization of financial systems throughoutthe world, including Japan. Given the importance of supporting the development of proper financial systems andsound money markets in developing countries, Japan is actively engaged in the provision of technical support(training for financial administrators in emerging market economies) to banking regulators/supervisory agenciesin the emerging market economies of the Asia-Pacific region.

2. Outline of evaluation resultsThe training project executed in FY 2004 was planned and implemented on the basis of findings from a

range of past studies and is designed to meet the needs of emerging market economies. In FY 2004, the FSAintroduced, on a trial basis, an ex-post questionnaire survey that is sent to trainees and to the organizations towhich they are affiliated one month after course completion. The findings from this survey reveal that over 70%of respondents are putting their training “to practical use” or “are investigating concrete methods of applying it”.This suggests that the FSA’s policy for FY 2004, in providing technical support, contributed to improvement ofthe capabilities of banking regulators in emerging market economies.

Notes:The above is extracted from the FY 2004 performance evaluation (July 2004 – June 2005) conducted inaccordance with the Government Policy Evaluation Act.For more detailed information, refer to the Financial Services Agency website(http://www.fsa.go.jp/en/index.html)

1 In an ODA-related evaluation, “ex-ante evaluation” refers to an evaluation during ODA policy planning, and “ex-post evaluation” at a certain time after the completion of ODA. This section, however, includes mainly policy evaluations under the GPEA. The evaluation studies are divided into “ex-ante evaluation” and “ex-post evaluation” based on the timing ofpolicy decision, as defined by the GPEA.

56

2.2.2 The Ministry of Internal Affairs and Communications

Coordination for and Contributions to Global Issues in Telecommunications Sector withinBilateral or Multilateral Framework (ex-post evaluation)Evaluator: Ministry of Internal Affairs and Communications

1. Outline and objectives of programs The Ministry implemented the following programs for the promotion of bilateral and multilateral policy

cooperation among countries and international organizations in order to increase the international community’sunderstanding of Japan’s information and communication policies, to eliminate the digital divide in the worldcommunity, and to build a global society integrated through information and communication networks:(1) Bilateral policy dialogues on an ongoing basis, and international coordination among the operational

frameworks of international organizations;(2) Building a network for the support of policies and programs on IT;(3) Implementation of international joint experiments to help eliminate the digital divide in the world

community;

Chapter

2AnOverview

ofEvalu

ationResults

(4) International coordination in support of the activities of the International Telecommunications Union (ITU);and

(5) Steady promotion of the Asia Broadband Program

2. Outline of evaluation resultsThe programs were implemented to contribute to solving international issues in the field of information and

communication, as well as to help build a global society integrated by information and communication networks.Based on the situation with regard to implementation of regular bilateral consultations and policy dialogues,participation in international coordination meetings within the frameworks of international organizations, etc., and the implementation of international projects, it can be confirmed that they have achieved a certain measureof success.

However, the elimination of the international digital divide, which is one of the challenges, requires morebilateral and multilateral dialogues with other countries as well as with international organizations, because the development of IT has not made noticeable progress in some developing countries, especially in Asia, and Japanneeds to step up its contribution. It is also necessary for the Ministry to continue to implement effectiveprograms, such as assistance with policy and program planning, technical cooperation, human resourcedevelopment, and joint experiments.

Additionally, with respect to promotion etc. of the Asia Broadband Program, the Ministry has concludedcooperation agreements with Vietnam, Cambodia, the Philippines, China, India and others based on ministerialtalks, and has held working level policy dialogues concerning specific content of cooperation. It is important forthe Ministry to continue to promote specific measures.

Notes:Non-ODA projects are included. For more detailed information, refer to the Ministry of Internal Affairs and Communications website(http://www.soumu.go.jp/english/index.html).

57


2.2.3 The Ministry of Justice

Promotion of Training Programs, Research and Surveys in Collaboration withthe United Nations (ex-post evaluation)Evaluator: Research and Training Institute of the Ministry of Justice

1. Outline and objectives of programs [Program] Promotion of training, research and studies undertaken in cooperation with the UN[Basic objective] Ensure the effective operation of criminal law in developing countries.[Performance targets] Improvement of the operation of criminal law and implementation of international

training, seminars, etc. to promote international cooperation in the field of crimeprevention and treatment of criminals and the field of juvenile delinquency preventionand treatment of juvenile delinquents.

2. Outline of evaluation resultsThe Ministry implemented technical cooperation in the form of international training, seminar, etc. that met

the needs of recipient countries in the Asia-Pacific region. In view of the fact that this cooperation is evaluated ashaving contributed to cultivation of the knowledge and experience, etc. of people involved in criminal law in these countries, the Research and Training Institute, which is responsible for this policy, and those departmentsinvolved in budgeting are investigating measures to facilitate the continued provision and improvement of theseinternational training sessions, etc.

Notes:For more detailed information, refer to the Ministry of Justice website (http://www.moj.go.jp/ENGLISH/index.html)

2.2.4 The Ministry of Finance

(1) Action for Effective Assistance (ex-post evaluation)Evaluator: Ministry of Finance

1. Outline and objectives of policies The Ministry pursues greater focus and efficiency in ODA operations, clarifies priorities and conducts

appropriate evaluations of the outcomes of Japanese assistance, in order for Japan to provide effective assistanceto developing countries under current fiscal pressures and make effective use of its limited funding resources.

2. Outline of evaluation resultsBy participating in the Experts Meeting on Financial Cooperation and attending meetings for those ministries

and agencies involved in ODA operations, the Ministry has reinforced ODA policy formulation systems andprocesses, strengthened policy dialogues with recipient country governments, and enhanced the content ofCountry Assistance Programs. Further, in order to improve aid effectiveness, the Ministry has engaged in policyconsultations with Multilateral Development Banks (MDBs) and has been an active participant in related debatesat meetings of the OECD Development Assistance Committee (DAC). Specifically, involvement in the DACHigh Level Meeting of FY 2004 led to the signing of the Paris Declaration, which is a commitment to increasingthe effectiveness of development assistance.

58

(2) Infrastructure Development and Private-sector Assistance in Developing Countries(ex-post evaluation)

Evaluator: Ministry of Finance

1. Outline and objectives of policies In light of the view that assistance for infrastructure development and private-sector growth are important to

promote sustainable growth and poverty reduction in the developing countries as well as aid that makes a directcontribution to the social sector, the objective of this policy is to provide active support in these areas.

2. Outline of evaluation resultsOf the ODA loans extended during FY 2004, 23.5% was assigned to social services (water supply and

sewerage, education, health, etc.), 43.9% to the transport sector and 15.6% to power and gas infrastructure,thereby making a significant contribution to sustainable growth and poverty reduction in developing countries.

In addition, based on the view that private sector development and the establishment of a climate for investment are necessary to realize sustainable poverty reduction in Africa, the Ministry has conducted workinglevel talks with the African Development Bank, paving the way to agreement on the Enhanced Private-SectorAssistance for Africa (EPSA for Africa) initiative that will amount to up to US$1.2 billion in aid. C

hapter

2AnOverview

ofEvalu

ationResults

(3) Debt Relief (ex-post evaluation)Evaluator: Ministry of Finance

1. Outline and objectives of policies In light of the view that aid for poverty reduction and sustainable growth should be extended to developing

countries facing a temporary liquidity shortage in respect of external debt payments or debt sustainabilitydifficulties, the Ministry is adopting debt rescheduling and debt relief measures as agreed by the Paris Club (agroup of official creditors) for developing countries pursuing an IMF-supported adjustment and reform program.

The Ministry is also supporting the resolution of debt issues and poverty reduction measures by providingsignificant debt relief based on the Enhanced HIPC Initiative, a comprehensive approach to debt relief forHIPCs.

2. Outline of evaluation resultsAs a member of the Paris Club, Japan has adopted debt relief measures based on Paris Club agreements (12

agreements were reached in FY 2004).As a voluntary measure over and beyond the HIPC Initiative, Japan offers a full debt waver to HIPCs that

reach completion point (CP); five countries reached CP in FY 2004. Of the 27 countries for which HIPCInitiative assistance has been approved to date, Japan has contributed the largest amount (US$2.2 billion:approximately 25% of G7 contributions) of accumulated debt cancellations/commitments to debt cancellationmade by official bilateral creditors (approx. US$17.2 billion).

Using this measure, Japan has substantially reduced the economic burden on debtor countries, therebycontributing to the resolution of the debt problems facing the developing world.

59


(4) Governance Assistance (ex-post evaluation)Evaluator: Ministry of Finance

1. Outline and objectives of policies Private investment is of critical importance to development, and for developing countries to attract private

investors, it is critical that they develop legislation, establish a sound judiciary and competent administrativeagencies, and realize “good governance”.

2. Outline of evaluation resultsWhen Japan approved a third Poverty Reduction Support Credit (JPY2 billion, December 2004) for Vietnam,

the Vietnamese government was urged to incorporate into its program reforms that would contribute toimprovements in the management of public funds and the investment environment, and these reforms have beenimplemented. For example, the Policy and Human Resources Development Fund (World Bank) provided thefollowing support using contributions to MDBs.

Support for public financial management reform to the Cambodian Ministry of Economy and Finance(US$492,000, December 2004)Support for improvements to judicial services in Ecuador (US$775,000, December 2004)

This assistance allowed Japan to utilize its knowledge of financial and administrative reform and legislaturein developing countries, contributing to improvements in various systems.

(5) Addressing of Environmental Problems (ex-post evaluation)Evaluator: Ministry of Finance

1. Outline and objectives of policies The Ministry prioritizes environmental issues in the provision of developmental assistance in light of

growing international concern over global-scale environmental problems, such as ozone layer depletion andglobal warming where the damage and effects transcend national boundaries, and more immediate environmentalproblems, such as health-damaging water and/or soil pollution.

2. Outline of evaluation results(i) Action through the Global Environment Facility (GEF)

With interest in the earth’s environment on the rise, Japan is using the GEF to promote sustainabledevelopment in developing countries and efforts to protect and improve the global environment. Thissupport takes the form of funding and field action. Japan was an active participant in discussions thattook place at the GEF Council meetings held during FY 2004 (in May and November) concerning theapproval and evaluation of projects, effective utilization of the GEF Fund and on GEF operations, andmade a significant contribution to the decisions on GEF activities.

(ii) Action on greenhouse gas emissions reduction projectsThe Ministry is urging JBIC to step up its assistance for tackling environmental issues as a prerequisite

for sustainable growth in the developing world. As part of these efforts, JBIC established the JapanGreenhouse Gas Reduction Fund (in December 2004), which is being used to support projects targetingreductions in greenhouse gases in developing countries. JBIC has also concluded business cooperationagreements with host country governments and agencies and is exchanging information on relatedprojects, which has resulted in progress towards the achievement of targets.

60

(6) Citizen Participation in Development (ex-post evaluation)Evaluator: Ministry of Finance

1. Outline and objectives of policies The Ministry promotes extensive participation among all levels of society in development assistance work,

and strengthens coordination with domestic NGOs and local governments.

2. Outline of evaluation resultsThe following initiatives were undertaken during FY 2004.The Ministry of Finance has been holding meetings with NGOs on a quarterly basis since 1997, and in 2004,

discussions concerning the disclosure policies of the various multinational development banks, bilateral debtproblems involving tsunami-affected countries, Japan’s proposals regarding assistance for Africa, and the NamTheun 2 Dam in Laos, served to deepen understanding on both sides.

Through the Japan Social Development Fund (World Bank) and the Japan Fund for Poverty Reduction(ADB), which are mechanisms for providing direct assistance to the poor and building the capacity of localcommunities, NGOs and other groups, the Ministry extended priority assistance for civil society participation.Support extended via other funds is also being used to advance the cause for the participation of civilians indevelopment activities.

Since 2001, the Joint Japan-World Bank Graduate Scholarship Program has included an overseas studyframework for Japanese NGOs, which is contributing to the development of human resources within JapaneseNGOs.

Chapter

2AnOverview

ofEvalu

ationResults

(7) Gender Equality in Development (ex-post evaluation)Evaluator: Ministry of Finance

1. Outline and objectives of policies Efforts aimed at attaining equality between men and women in development and ensuring that both sexes

benefit from the results are a significant challenge for the international aid community, and the Ministry is supporting the MDBs and JBIC in their efforts to eliminate the gender divide in developing countries.

2. Outline of evaluation resultsThe Ministry has used subscriptions and contributions to the MDBs to provide support focused on

empowering women and has actively supported MDB efforts to attain equality between men and women indevelopment and to ensure that both genders reap the benefits of development assistance. It has also urged JBICto give due consideration to gender equality perspectives during the formation and implementation of assistanceprojects in a bid to eliminate the gender divide. The following initiatives were undertaken during FY 2004 andhave resulted in progress towards the achievement of these goals.

Cape Verde: Program to increase access to legal services for women (World Bank, Japan Policy andHuman Resources Development Fund: approved September 2004)India: Tamil Nadu Afforestation Project (II) (JBIC, March 2005)

(8) Public Relations Activities Regarding Development Assistance (ex-post evaluation)Evaluator: Ministry of Finance

1. Outline and objectives of policies The Ministry provides easy-to-follow information on Japan’s ODA operations using a variety of media, and

creates opportunities for Japanese citizens to have contact with ODA projects, and to enhance global publicrelations so as to improve public understanding of development assistance and raise the profile of Japanese aid.

61


2. Outline of evaluation resultsThe Ministry uses PR magazines and its official website to disseminate information on the status of projects

being undertaken in various countries to the general public. It has also conducted PR activities overseas, usingJapanese Overseas Diplomatic Missions to work on foreign media. Every year since 1993, the Ministry has beenproducing and distributing a pamphlet on the development assistance provided through the MDBs. Theseactivities are helping to raise the profile of Japanese aid.

(9) Peace-Building Assistance (ex-post evaluation)Evaluator: Ministry of Finance

1. Outline and objectives of policies Preventing frequent outbreaks of conflict and terrorism and establishing peace is becoming increasingly

important to stability and development in international community. Peace-building is a priority issue, and theMinistry is committed to implement the bilateral and multilateral aid necessary to provide seamless and flexibleassistance.

2. Outline of evaluation results(a) Reconstruction assistance for Iraq

At the International Conference on Reconstruction in Iraq (October 2003), Japan announced that it wouldprovide up to approximately US$5 billion in aid, i.e. approximately US$1.5 billion in grant aid (disbursed/committed in full as of May 2005) plus a maximum of US$3.5 billion, essentially in the form ofyen loans, to support demand for reconstruction in the medium term (to be used to fund project formationsurveys and other preparations for assistance primarily targeting physical infrastructure: power, water supply and sewerage, roads, infrastructure systems, etc.). In terms of debt relief, as Iraq’s largest creditorholding debts of around JPY 800 billion (including delinquency charges), Japan has contributed to thediscussions of the Paris Club and, in November 2004, Japan agreed to cancel 80% of Iraq’s debts; theExchange of Notes with the Government of Iraq is scheduled to take place in the near future. As this demonstrates, Japan is working steadily towards the achievement of the goals.

(b) Reconstruction assistance for AfghanistanDetermined not to allow Afghanistan to again become a breeding ground for terrorist activities followingthe simultaneous terrorist attacks of September 11, 2001, the international community has united behindpeace and reconstruction efforts, and Japan has pledged some US$900 million in aid to Afghanistan. DuringFY 2004, Japan’s Trust Fund was used to contribute to World Bank and ADB-supported programs thatincluded establishing the foundations for strengthening governance at the community level, contributing(via the World Bank) to the “National Solidarity Program” that supports small-scale projects undertaken bylocal communities, and extending support to the irrigation development project (via ADB) implemented innorthern Afghanistan, and Japan’s assistance has been highly commended.

(c) Reconstruction assistance for Sri Lanka Providing a balanced portfolio of assistance that reaches both conflict-affected areas in the north and eastand other underdeveloped regions is essential to establishing peace in Sri Lanka. Japanese ODA loan commitments to Sri Lanka reached approximately JPY27.9 billion in FY 2004, which was used to supportthe reestablishment of necessary infrastructure through post-conflict reconstruction and to improveproductivity and recover jobs by providing financial assistance to small companies whose productivity wasdamaged by the long-running conflict. These efforts are contributing to the peace process.

62

(10) Reconstruction Assistance for Indonesia Following the Sumatra-Andaman Earthquakeand Accompanying Tsunamis in the Indian Ocean (ex-post evaluation)

Evaluator: Ministry of Finance

1. Outline and objectives of policies The Ministry provides reconstruction assistance to areas affected by the Sumatra-Andaman earthquake and

accompanying tsunamis in the Indian Ocean.

2. Outline of evaluation resultsAs part of its reconstruction and rehabilitation efforts, Japan has provided US$40 million in aid to the World

Bank and the ADB via its Trust Funds (as of March 2004, $8.3 million had been committed to six projects viathe two funds), and agreed at the Paris Club (in March 2005) to allow countries hit by the tsunami that requestedsuch forbearance to suspend debt repayments until the end of 2005.

(11) Intellectual Assistance (ex-post evaluation)Evaluator: Ministry of Finance C

hapter

2AnOverview

ofEvalu

ationResults

1. Outline and objectives of policies To facilitate sustainable economic growth in developing countries, proper socioeconomic systems that are

commensurate with the developmental stage and economic structures of each country need to be designed and operated. Further, bearing in mind the policies and demand for assistance of these countries, Japan’s experiencein socioeconomic development and economic cooperation needs to be put to use in the development process anduse made of its superior technologies, knowledge, human resources and systems. From this perspective, theMinistry is working actively to support human resource development and promote international cooperation andexchange by offering its knowledge, experience and expertise on the fiscal and financial systems and policiesnecessary to socioeconomic development in the developing world.

2. Outline of evaluation resultsDuring FY 2004 the Ministry offered training and seminars for policy-makers and administrators in

developing countries with a view to developing individuals to lead the socioeconomic development process inthe future. The Ministry also participated in policy missions, provided policy recommendations, and dispatchedexperts to provide specialist advice to developing countries. In implementing these programs, the Ministry useddemand surveys, dispatched teams of investigators and interviewed local stakeholders so as to ascertain demandand conditions in the recipient countries in advance, and, upon completion, used questionnaires to solicit the opinions of participants on course content, etc., in an attempt to get feedback. Where necessary, appraisal missions were also dispatched with a view to ascertaining the post-training activities of participants and futuredemand for training. Moreover, efforts were made to provide follow-up with a view to making maximum use ofJapan-side training by inviting the foreign officials that participated in the training sessions held in this countryto make presentations at locally-held training sessions after they returned home and sending them on missions tocontribute to the evaluation of training results. These proactive efforts to promote international cooperation andexchange have resulted in high praise for curriculum and lecture content from the many participants.

Notes:For more detailed information, refer to the Ministry of Finance website(http://www.mof.go.jp/english/index.htm).

63


2.2.5 The Ministry of Education, Culture, Sports, Science and Technology

(1) Promotion of International Cooperation in Education that Makes Sense from a JapanesePerspective (ex-post evaluation)

Evaluator: Ministry of Education, Culture, Sports, Science and Technology

1. Outline and objectives of projectsEducation is one of the most important fields for poverty reduction in developing countries. Based on

discussions at the Committee for International Cooperation in Education (a private forum of the Minister), theMinistry aims to realize effective international cooperation on education that uses Japan’s experience and humanresources and to promote “internationalization form within” Japan.

2. Outline of evaluation results(a) Development of the Cooperation Base System

A Cooperation Base System has been developed as a national implementation system to support theutilization of Japan’s experience in education and the dispatch of teachers, which were also stated in thefinal report of the Forum for International Cooperation on Education as well as in the BEGIN initiativeannounced by Prime Minister Koizumi at the Kananaskis Summit.

Specifically, the Ministry organized Japan’s experience in education, first by compiling and analyzing pastexperiences in cooperation in the area of education in which Japan has a lot of experience and its mainstrengths in cooperation (i.e. science and math education, teacher training systems, education administrationand school management) and engaged in the development of a common cooperation model (e.g. concerningthe content of activities and teaching materials) which could be used for all cooperation projects.

The Ministry promoted the formation of groups for each cooperation sector in which Japan has lessexperience (e.g. school health care and environmental education) in order to organize Japan’s educationexperience, and expanded the provision of information through the dialogue process with developingcountries.

Using international forums, briefing sessions in Japan and the Ministry’s website (electronic archives),efforts have been made to disseminate these findings to a wide audience in order to give those involved ininternational cooperation in education the ability to access and utilize this information freely.

The steering committee has evaluated the Cooperation Base System construction project in the light of international trends in education cooperation and the assistance policies of the Japanese government, and isstudying ways to improve the system.

(b) Development and improvement of the system for teachers’ participation in internationalcooperation projectsThe number of teachers (147) seeking to participate in the JOCV’s “Participation System for IncumbentTeachers” exceeds the target number of participants (100) and, in fact, actual participants, i.e. thoseapplicants who have undergone the health checks and passed the final screening, numbered 64. Since this system was started in 2001, despite a drop in numbers year-on-year in FY 2002, the number of participantshas subsequently risen and the system is achieving certain results. The Ministry will continue to strengthenits information drive targeting local education boards, etc., so as to raise the baseline inventory ofincumbent teachers participating in the JOCV system and other overseas cooperation programs.

(c) Cooperation with UN Educational, Scientific and Cultural Organization (UNESCO)The Ministry has been providing assistance to adult literacy projects in the Asia Pacific region throughdonations to Japanese Funds-in-Trust and dispatches of experts. Primary enrollment and literacy rates in the

64

region both rose in the period from 1990 to 2000. It is not possible to confirm actual progress since thestatistics for FY 2004 have not yet been released, but the performance evaluation report compiled for thesame period by UNESCO and the Trust Fund Review Board verifies that the funds are operated effectivelyand that steady progress is being made on the projects, from which it can be concluded that Japanesecooperation is making reasonable headway.

Notes:This evaluation is based on the policy evaluation (FY 2004 performance evaluation).Non-ODA projects are included.

(2) Student Exchange Promotion Program (ex-post evaluation)Evaluator: Ministry of Education, Culture, Sports, Science and Technology

1. Outline and objectives of projectsWith a view to establishing friendly ties with foreign countries and of contributing to human resource

development, Japan has accepted government-sponsored students and provided study grants to those who areself-funded, based on the “Plan to Accept 100,000 Foreign Students” formulated in August 1983, and has workedto improve a range of facilities, including accommodation facilities for foreign students. As a result, foreignstudent numbers reached 109,508 in May 2003, thereby attaining the original target. The Ministry intends tocontinue promoting student exchanges in terms of both acceptance and dispatch and is seeking to ensure thequality of participants in this program.

Chapter

2AnOverview

ofEvalu

ationResults

2. Outline of evaluation resultsBy promoting various programs in an integrated manner, Japan has succeeded in accepting a total 117,302

foreign students (as of May 1, 2004) and continues to make favorable progress with this program. For the future,efforts are needed to ensure foreign student quality and to further promote exchanges in terms of both acceptanceand dispatch.

Feedback on the findings from this evaluation is to be provided to review panels and the steering committee,which are charged with maintaining current levels of achievement and of studying ways of addressing futurechallenges.


(3) International Cooperation by Universities and Human Resources Development for International Cooperation Activities (ex-post evaluation)

Evaluator: Ministry of Education, Culture, Sports, Science and Technology

1. Outline and objectives of projectsIn order to advance effective and efficient international development cooperation with the use of

“knowledge” that universities possess, and based on discussions at the Committee for International Cooperationin Education (a private forum of the Minister), this program has been designed to develop a foundation on whichuniversities as institutions can participate in international development activities. Another objective of theprogram is to develop and secure human resources for international development cooperation.

65


2. Outline of evaluation results(a) Construction of a database for university bodies and faculty members

During FY 2004, 265 universities and 3,560 faculty members were registered in the database, therebymaking steady progress towards the attainment of the goal “to register 300 universities and 3,000 facultymembers” (set for FY 2005).

(b) Development of a support centerIn response to the final report of the Committee for International Cooperation on Education released in July2002, the Ministry opened the International Development Cooperation Support Center in July 2003. Interms of achievements for FY 2004, the support center has been developed, tie-ups have been establishedand/or strengthened with seven aid organizations, 26 domestic and international university-affiliatedorganizations and five other partner organizations, thereby surpassing the goal “to establish and/orstrengthen links with five aid organizations, ten domestic and international university-affiliatedorganizations and five other partner organizations” (set for FY 2004). The support center is also intended toserve as a base for universities, as institutions, to participate in international development activities, and given the rise in participants in seminars on project commissions during FY 2004, it can be said thatfavorable progress is being made towards this goal.

(c) Human resource development With a view to promoting the involvement in international development cooperation activities of youngpeople specializing in development issues and developing human resources, during FY 2004 the Ministrydealt with 87 interns and job-seekers, which represents an increase of approximately 1.9 over the base year(FY 2002) and constitutes favorable progress in this area.

Maintaining the current direction of this program, the Ministry will continue to sustain and build on its linkswith domestic and international universities and aid organizations, and will analyze actual commissions soas to provide feedback to these organizations.


2.2.6 The Ministry of Health, Labour and Welfare

(1) Cooperation in International Organization ActivitiesCooperation with ILO’S Technical Assistance (ex-post evaluation)

Evaluator: Ministry of Health, Labour and Welfare

1. Outline and objectives of programs [Program 1] Contributing to gender-friendly employment development and the expansion of work/employmentopportunities for women in developing countries by providing financial assistance to technical cooperationprojects undertaken by the International Labor Organization (ILO)

Implementing the following projects through voluntary contributions to the ILO and the use of ILOspecialists:(1) Project in China which provides entrepreneurship training, supports potential small entrepreneurs through

micro-finance, etc. and creates job opportunities.(2) Project which provides vocational training and enhances awareness for women aiming at expanding

women’s employment opportunities in Cambodia and Vietnam.

66

[Program 2] Contributing to the development of a healthy working environment through seminars, etc. onimproving labor standards in developing countries

Making voluntary contributions to the ILO to provide the following seminar programs, research and surveyprograms, and training courses in Asia and the Pacific:(1) Regional seminars with tripartite participation (government, workers and employers) in order to disseminate

the concept of the ILO Declaration on Fundamental Principles and Rights at Work (Key labor standards: i.freedom of association and the right to collective bargaining; ii. the elimination of all forms of forced orcompulsory labor; iii. abolition of child labor; and iv. the elimination of discrimination in workplace).

(2) Labor Policy Fellowship projects which provide training opportunities related to labor and employmentpolicies of two or three countries in Asia, including Japan, to administrative officials in charge of labor andemployment policies in developing countries.

[Program 3] Contributing to increased vocational skill development in the Asia-Pacific region throughcooperation with the Asia Pacific Skill Development Program (APSDEP). The Ministry made voluntary donations (US$130,000 in FY 2004), assisted APSDEP project activities, andcarried out projects in Japan, including seminars using Japan’s experience, expertise and facilities.

Chapter

2AnOverview

ofEvalu

ationResults

(Note)APSDEP is a regional program in cooperation with the ILO. Its objectives include promotion of technicalcooperation mutually utilizing knowledge, experience, facilities, etc. in the vocational skill development sectorin the Asia-Pacific region, and promotion of the socioeconomic development of countries within the regionthrough improvement of vocational training and employment growth. APSDEP was established in 1978. Japan is making donations to APSDEP project activities and is implementing support projects such as holding seminarsthat utilize the experience, know-how, etc. possessed by Japan in the vocation.

[Program 4] Developing Japanese specialists in technical cooperationUndertaking international cooperation activities by providing hands-on training for Japanese trainees at the

sites of ILO-supported technical cooperation projects in the Asia-Pacific region.

2. Outline of evaluation resultsThese programs were carried out through the ILO and the APSDEP. In addition to the full use of extensive

networks, expertise and know-how accumulated by international organizations, the programs adopted a mechanism by which mutual cooperation of the members was encouraged, enabling a broad and efficientcontribution to the solution of the problems relating to employment and labor in the Asia and the Pacific,including countries that cannot be covered by bilateral cooperation.

The ILO conducts third-party evaluations, either mid-term or final depending on the duration of the plan,undertakes objective evaluations of the programs and is actively pursuing greater efficiency and effectiveness inimplementation. The above activities as a whole have gained a good reputation among the various governmentsas well as labor/management organizations.

APSDEP is also working to improve the effectiveness of its work and in FY 2004 held the Experts Meeting on Skills Development using Training Methods developed in the Region, which was based on needs voiced bymember countries.

Overall, these programs have been highly commended by the governments and labor-managementorganizations of the various countries, suggesting that the Ministry is contributing towards the achievement ofgoals, i.e. to cooperate with the activities of international organizations and promote health, labor and welfareadministration appropriate to the current age of globalization, and that progress has been made.

Notes:The policy evaluation outlined here was conducted in conformity with the Government Policy Evaluations Act(Law No. 86, 2001).

67


(2) Cooperation in International Organization ActivitiesAssistance to Asia-Pacific Economic Cooperation (APEC) Human ResourceDevelopment (Monitoring)


1. Outline and objectives of programs [Program] To contribute to increased vocational skill development in the Asia-Pacific region throughcooperation with APEC in the human resources development sector (1) Long-term training programs using local facilities owned by Japanese enterprises in the recipient countries

for basic vocational skill training of local residents (APEC Human Resources Development Seminar/SkillsTraining Program).

(2) International conferences of public and private sector representatives working in the field of humanresource development in the APEC countries for discussion on the issues relating to human resourcedevelopment in the region (APEC Forum on Human Resource Development).

(3) Training programs using private companies to provide necessary knowledge and skills for white-collarworkers in developing countries who are in leadership positions relating to production management ordistribution management (APEC White-Collar Training Program).

(4) Short-term training programs for the person in charge of human resources development in local enterprisesconcerning the methods and techniques of vocational training in a computerized work environment (APECIT Seminar).

2. Outline of evaluation resultsMonitoring was undertaken in FY 2005. There are plans to conduct a comprehensive evaluation in FY 2005

and FY 2006 (upon formulation of the Seventh Basic Plan for Human Resources Development).

Notes:The policy evaluation outlined here was conducted in conformity with the Government Policy Evaluations Act(Law No. 86, 2001).

(3) Contribution to International Society via the Promotion of International CooperationTechnical Assistance for Human Resources Development in Labor Sector (ex-postevaluation)


1. Outline and objectives of programs [Program 1] Developing human resources that can contribute to the building of sound industrial relations indeveloping countries

(1) Training project of personnel manager in developing countriesIn order to improve personnel and labor management skills in developing countries in the Asia region and topromote economic development in these countries, the Nippon Keidanren International Cooperation Centercarries out projects in which Japanese business corporations provide on-the-job training to middle-levelstaff who take leading roles in their organizations and are responsible for personnel and labor relationsmanagement in their countries. The project receives government subsidies.

68

(2) Support for ASEAN Project on Industrial RelationsThe objective of this project is to promote stable economic growth within the ASEAN region byestablishing stable industrial relations. It provides Japan’s experience to the ASEAN region and supports thedevelopment of industrial relations institutions in the new ASEAN member countries (Cambodia, Laos,Myanmar and Vietnam). As a 3-year plan started in FY 2002, the project is carried out in collaboration with the ASEAN Secretariat, and holds Japan-ASEAN Cooperation Committee meetings, conducts field studies,and holds national and regional seminars.The holding of seminars is commissioned to the ILO Association of Japan, Inc.

[Program 2] Cultivation of vocational training instructors in developing countriesIn order to address the shortage of skilled workers caused by industrialization of developing countries, this

program aims to develop and expand vocational training programs in developing countries through the Programfor Government-sponsored Foreign Students. This project accepts foreign students on an undergraduate course(4-year program) or a graduate course (2-year program) at the Polytechnic University, the only institution in Japan specializing in the training of vocational instructors. The students are trained in a broad range oftechniques and expertise required for vocational instructors, including specialized subjects, teaching techniquesand expertise for the operation of training/teaching facilities.

Chapter

2AnOverview

ofEvalu

ationResults

[Program 3] Promoting skills transfers to developing countries through acceptance of workers from developingcountries

The Ministry is implementing (i) the International Skills Development Training Scheme, which acceptsworkers currently employed in the private sector in developing countries, and trains them for futureprivate-sector leadership in their countries; (ii) the International Youth Vocational Training Scheme, whichaccepts young workers, the future skilled workforce in developing countries, and provides them with job trainingprimarily on advanced industrial technologies; (iii) the Guidance and Assistance Program for Foreign Trainees,which provides a variety of guidance and support to corporations accepting foreign trainees; and (iv) appropriateand effective promotion of Technical Intern Training Program.

2. Outline of evaluation resultsProviding support through the voluntary cooperation of Japanese and overseas labor and management

organizations for the development and cultivation of human resources that contribute to industrial relationsstabilization in developing countries in the Asia-Pacific region makes it possible to implement more practical andmeticulously designed projects that meet the needs of recipient countries. This kind of support has been highlyappreciated in the recipient countries. This approach also has the advantage that continuous assistance enablesthe creation of personnel networks and the acquisition of the latest information, which are essential for Japanesecompanies to expand their business overseas. In general terms, these programs are contributing to internationalsociety by promoting international cooperation through technical cooperation for human resource developmentin the labor sector, and to have made progress towards the achievements of this goal.

Notes:The policy evaluation outlined here was conducted in conformity with the Government Policy Evaluations

Act (Law No. 86, 2001).

69


2.2.7 The Ministry of Agriculture, Forestry and Fisheries

(1) International Cooperation for Food, Agriculture and Rural Areas (ex-post evaluation)Evaluator: Ministry of Agriculture, Forestry and Fisheries

1. Outline and objectives of policies There are approximately 850 million undernourished people in the world (2000-2002), 96% of whom are

concentrated in developing countries. This resulted in the announcement of two goals at the World Food Summitin 1996: to achieve world food security, and halve the world’s undernourished population by the year 2015.These goals were reaffirmed at the 2002 World Food Summit: Five Years Later, when individual countries wereurged to step up their efforts toward the goal of halving the world’s undernourished population.

As depletion and desertification of tropical forests caused by overgrazing and non-traditional slash-and-burnfarming becomes an increasingly serious global environmental issue, it is important to make active efforts torestore and conserve the agricultural land and grasslands that are being degraded in developing countries, and todisseminate and develop sustainable and highly-productive agricultural techniques.

Moreover, it is becoming increasingly necessary to bring ODA policies in line with Japan’s foreign anddomestic policies, and efforts are needed to make Japan’s position in the world, such as in WTO negotiations,more widely known and to promote understanding of Japan’s policies.

Considering the above, Japan, through the following initiatives, needs to actively promote internationalcooperation in food, agriculture and rural communities in line with the situation and needs of developingcountries so as to contribute to the resolution of starvation and poverty and other global-scale issues. (1) Basic surveys involving the collection of information and the verification of technologies and techniques

that are critical to on-target, effective and efficient implementation of technical cooperation in the field ofagriculture, forestry and fisheries.

(2) Placement of trainees within agrarian organizations and technology exchanges as a means of cultivatinghuman resources with a view to supporting self-reliant development in developing countries.

(3) Contribute funding to the various cooperation activities conducted by the Food and AgricultureOrganization of the United Nations (FAO) and the Consultative Group on International AgriculturalResearch (CGIAR) to extend cooperation utilizing the expertise of international organizations.

2. Outline of evaluation resultsInternational cooperation in food, agriculture and rural areas is intended to address the needs of developing

countries by contributing to reductions in starvation and poverty and to the preservation of the globalenvironment, and such initiatives are designed to promote understanding of Japanese agricultural policy. In thelight of these facts, a questionnaire was conducted targeting stakeholders within recipient country governmentsregarding “consistency with recipient country needs” and “degree of understanding of Japanese agriculturalpolicy”.(Questionnaire results are expressed in numerical terms: e.g. fully consistent with needs = 100%; totallyinconsistent with needs = 0%)

The survey respondents rated “consistency with recipient country needs” at 96.6 %, and “degree ofunderstanding of Japanese agricultural policy” at 97.4 %.

Consistency with recipient country needs: 96.6% (accomplishment level A)Degree of understanding of Japanese agricultural policy: 97.4% (accomplishment level A)(Accomplishment levels: A = over 90%; B = over 50%, below 90%; C = below 50%. As this policy used100% as the measure of full accomplishment, the resultant percentages correspond directly to policy accomplishment rates.)

Notes:Non-ODA projects are included.

70

(2) Forest Management (ex-post evaluation)Evaluator: Ministry of Agriculture, Forestry and Fisheries

1. Outline and objectives of policies The depletion and degradation of tropical forests and forest resources in northern Russia have become

critical issues in terms of global environmental conservation – the prevention of global warming, etc., thussustainable forest management must be tackled globally. Accordingly, this policy aims to promote sustainableforest management through international forestry cooperation initiatives, including technology transfers to thedeveloping countries in which such problems are occurring, human resource development, the implementation ofprojects intended to improve policy, holding of international conferences involving representatives fromgovernments and NGOs, etc., and financial contributions to FAO and the International Tropical TimberOrganization (ITTO).

2. Outline of evaluation resultsThe Ministry conducted a questionnaire survey targeting government stakeholders in the countries in which

various international forestry cooperation programs have been implemented. Project outcomes were rankedaccording to five levels.

The results of the survey, regarding contributions to sustainable forest management were expressed aspercentages (i.e. highest rankings assigned to all question categories = 100%). The resultant achievement ratewas 90%, or accomplishment level A (accomplishment levels: A = over 90%; B = over 50%, below 90%; C =below 50%. 100% is full accomplishment).

This questionnaire includes specific items of individual projects, responses to which are reflected in theimplementation reviews.

Chapter

2AnOverview

ofEvalu

ationResults

Notes:Non-ODA projects are included. Projects involving international forestry cooperation are evaluated as sub-indices of the “ForestManagement” policy.

(3) Management and Use of World Marine Resources (ex-post evaluation)Evaluator: Ministry of Agriculture, Forestry and Fisheries

1. Outline and objectives of policies To promote policies on the sustainable management and use of marine resources both within and beyond the

bounds of Japan’s Exclusive Economic Zone (EEZ) so as to secure a stable supply of marine products for thenation.

2. Outline of evaluation resultsIn FY 2004, efforts that included cooperating with international frameworks – consultations based on

bilateral and multilateral agreements on fishing in EEZ etc, and efforts to promote dialogue with other countriesconcerning fishing operations, to ensure Japan’s fishing quota while promoting appropriate conservation andmanagement of marine resources in high seas and foreign waters so as to secure a stable supply of marineproducts, yielded the following results:(a) The number of target species for resource management increased by one to 70 species after members of the

Commission for the Conservation of Southern Bluefin Tuna (CCSBT) reached consensus on national catchlimits, an issue that has remained untouched since the CCSBT agreement of 1988.

(b) Regarding the number of fishery agreements, the private pact made with Madagascar failed to functionappropriately when a Japanese fishing vessel was apprehended in Madagascar waters in January 2003;

71


failure to make any progress on negotiations aimed at resolving the issue led to the expiration of the 3-year pact, which became invalid in February of last year. Other agreements have all been maintained, however,meaning that the number has decreased by one year-on-year, to 48.

(c) Since 1988, Japan has employed voluntary controls in respect of the southern bluefin catch quota agreedupon in treaty settlement (CCSBT); however, with member countries having now reached officialagreement on national catch limits, Japan was able to secure the same quota as for the previous year.

Maintenance or increase in the number of fish species targeted for resource management by internationalfishery organizations: 70 (accomplishment level A)Maintenance or increase in the number of fishery agreements: 48 (accomplishment level C) Japanese catch quotas under respective treaties:

(1) Bluefin tuna (ICCAT): 3,427.25 tons(2) Southern bluefin tuna (CCSBT): 6,065 tons}

(Accomplishment levels: The ranking is A, where the number of fish species targeted for resourcemanagement and number of fishery agreements is maintained or has increased, and the ranking is C,where either parameter has decreased.)

Notes:Non-ODA projects are included.

2.2.8 The Ministry of Economy, Trade and Industry

(1) Cooperation through Development Studies (ex-ante evaluation)Evaluator: Ministry of Economy, Trade and Industry

1. Outline and objectives of programs To support policy formulation and development planning in developing countries that might lack experience

and expertise needed to devise efficient and effective development plans, and to use this as the basis for theformulation of effective technical cooperation programs for economic development in these countries.

2. Outline of evaluation resultsEfficient and effective building of industrial policies, institutions and development plans are essential for

developing countries to realize sound economic development. However, the developing countries might lack theknowledge and experience necessary for the formulation of such policies and plans. Accordingly, Japan’sassistance, implemented in the form of policy and technical recommendations based on Japan’s past experience,will be a very effective means of contributing to the economic development of these countries.

Furthermore, with a view to ensuring the effectiveness of policies and institutions formulated in this way, itis important to verify the feasibility and replicability of implementation systems tailored to the development ofinstitutions and, where necessary, to identify any potential problems.

This type of assistance, as Japan’s contribution to the international community, should be identified and promoted as national policy.

72

(2) Cooperation through Human Resources Development (ex-ante evaluation)Evaluator: Ministry of Economy, Trade and Industry

1. Outline and objectives of programs To support the establishment of foundations for economic progress in developing countries by assisting in

the development of human resources to engage in efforts to strengthen the industrial infrastructure, theliberalization of trade and investment, and economic integration.

2. Outline of evaluation resultsAmid ongoing economic globalization, Japan must urgently address efforts to improve the environment for

trade and investment in those developing countries with which it has established close ties and, specifically, givepriority to the development of human resources as a base for economic growth. This policy is of significantpublic interest in that the development of industrial infrastructure in developing countries contributes to a favorable environment for trade and investment in Japan. Moreover, government involvement is essential sincethis program helps developing countries achieve goals that they would be unable to achieve independentlythrough the provision of ODA.

Chapter

2AnOverview

ofEvalu

ationResults

(3) Technical Cooperation through Joint Research with Developing Countries(ex-ante evaluation)

Evaluator: Ministry of Economy, Trade and Industry

1. Outline and objectives of programs To solve technology development challenges that go beyond the research and development capabilities of

developing countries and develop the necessary capabilities in these countries by undertaking joint research thatdraws on Japanese research and development capabilities and promoting technical transfer.

2. Outline of evaluation resultsSince the research and development capabilities needed to resolve technology development challenges and to

achieve independent development, particularly those concerning energy and the environment, remain at lowlevels in developing countries, demand for technologies from Japan is consistently high, as it has a wealth ofexperience in these fields. At the same time, government involvement is essential for two reasons. Firstly, it isextremely difficult for profit-oriented private companies to execute this policy, and secondly, this needs to bepromoted as Japanese aid policy and understood to be part of Japan’s international contribution.

(4) Formulation of Financial Cooperation Projects (ex-ante evaluation)Evaluator: Ministry of Economy, Trade and Industry

1. Outline and objectives of programs This policy is primarily intended to increase the “visibility of Japanese aid” and to promote private-sector

involvement in economic cooperation. It includes rapid identification and formulation of Yen Loan projects thatutilize the business knowledge and experience of Japanese companies and identification and formulation of infrastructure development projects that will draw on private-sector capital in developing countries.

73


2. Outline of evaluation resultsThis policy is of significant public interest and requires government involvement because: (1) the outcomes

are not limited to specific companies but contribute to the resolution of global environmental problems throughthe implementation of environmental conservation projects, etc.; (2) contributes to the strategic and efficientimplementation of assistance by improving Japan’s ability to make proposals; and (3) contributes to increase the“visibility of Japanese aid” by drawing on Japanese knowledge and experience.

2.2.9 The Ministry of Land, Infrastructure and Transport

(1) Effect Analysis of Infrastructure Projects (ex-post evaluation)Evaluator: Ministry of Land, Infrastructure and Transport

1. Outline and objectives of projectsMany developing countries have insufficient knowledge of project formations, i.e. the expertise needed to

work out the specifics of individual projects based on national development plans, strategies and policies.To address this problem, The Ministry utilizes its experience, knowledge and expertise to provide assistance

for project formation to the governments of developing countries and is thereby contributing to the efficient andeffective development of infrastructure. The development of objective measurement and indication techniques isneeded to ensure that the outcomes can be explained in coherent terms to third parties during the projectformation process.

To this end, in FY 2004, the Ministry analyzed the effects of a rainwater drainage infrastructure projectimplemented in Dacca, Bangladesh, in order to produce the documents needed to research methods of analyzingproject outcomes. This project involved the development and construction of pump stations, sluice gates, leveesand drainage channels. The Japanese government provided Grant Aid for the pump stations and sluice gates;while the ADB was responsible for levees and drainage channels.

2. Outline of evaluation resultsThe rainwater drainage infrastructure project in Dacca was designed to avoid and/or mitigate damage caused

by flooding in the city.The survey involved interviewing local residents in municipal districts covered by the project and in districts

where the development of drainage infrastructure has been delayed as a means of systemizing project outcomes.According to the survey findings, it was confirmed that the project has resulted in reductions in floodingfrequency, in the number of dwellings flooded and in the duration of flooding. It has also shortened interruptionsto water, gas and electricity services. Further, the percentage of low-income families living in districts coveredby the project has dropped, while that of middle-income families has risen, suggesting that these flood controlmeasures have raised household incomes.

Notes:This policy evaluation is not based on the Government Policy Evaluations Act and is included for reference

purposes only.

74

(2) International Cooperation Evaluation Project (ex-post evaluation)Evaluator: Ministry of Land, Infrastructure and Transport

1. Outline and objectives of projectsThis project studies methods for appraising the outcomes of socioeconomic infrastructure development in

developing countries while conducting case studies, verifies the outcomes of such infrastructure development,and promotes effective future international cooperation projects.

2. Outline of evaluation resultsIn order to quantitatively evaluate the effect of the development of the project for port infrastructure in

Vietnam, the Ministry measured the development effect of individual projects by developing and utilizing an inter-regional input-output table in Vietnam. Furthermore, in order to assess the ripple effects on regionaleconomies and effects on the lives of local residents, such as employment creation, etc., the Ministry conductedsurveys on local people involved in transportation industries. These surveys confirmed beneficial effects of theimplementation of the project, such as a ripple effect for regional economies, and an increase in job opportunities,etc.

Chapter

2AnOverview

ofEvalu

ationResults

Notes:This policy evaluation is not based on the Government Policy Evaluations Act and is included for reference.For more detailed information, refer to the Ministry of Land, Infrastructure and Transport website(http://www.mlit.go.jp/english/index.html)

2.2.10 The Ministry of the Environment

(1) International Cooperation in Global Environmental Protection and Ensuring Coordination(ex-post evaluation)

Evaluator: Ministry of the Environment

1. Outline and objectives of programs To work towards compliance with the international framework on combating desertification and to study

methods for integrated conservation and management of natural resources in order to contribute to thedevelopment of an international framework on this issue.

2. Outline of evaluation resultsJapan has contributed to the promotion of international environmental policy on measures to address

desertification.

Notes:The policy evaluations of the Ministry of the Environment are based on performance evaluation methods and

cover “programs” comprising groups of projects that share the same objectives; the Ministry does not conductproject evaluations of ODA projects per se. Accordingly, the project evaluated above includes non-ODA projects,but to facilitate understanding, those sections relating to ODA projects have been extracted from the policyevaluation findings and are reported here.

75


(2) International Cooperation on Environmental Conservation in Developing Countries(ex-post evaluation)

Evaluator: Ministry of the Environment

1. Outline and objectives of programs To increase Japan’s intellectual contributions to international cooperation efforts and to build strategies for

this objective, for example by providing assistance to programs for sustainable development in developingcountries, with the aim of taking the initiative in the international community on actions toward sustainabledevelopment.

2. Outline of evaluation resultsAlthough Japan has received some praise for its contribution to environmental conservation in developing

regions, environmental issues in these regions remain serious, and there is a strong demand for Japan and otherdeveloped countries to provide technical and financial cooperation. Therefore, it is necessary for Japan tocontinue to actively develop international cooperation for developing regions.

Notes:The policy evaluations of the Ministry of the Environment are based on performance evaluation methods and

cover “programs” comprising groups of projects that share the same objectives; the Ministry does not conductproject evaluations of ODA projects per se. Accordingly, the above constitutes an overview and evaluation of apolicy that incorporates ODA projects and covers non-ODA projects as well.

For more detailed information, refer to the Ministry of the Environment website: (http://www.env.go.jp/en/)

76

2.3 Results of Evaluation by the Japan InternationalCooperation Agency (JICA)

2.3.1 Overview

JICA conducts evaluation to implement cooperation projects more effectively and efficiently. Evaluationresults are used to improve the quality of projects and secure accountability and to gain the support and understanding of the general public.

Since its reorganization into an independent administrative institution in 2003, there has been further demandfor JICA to ensure result-based project management and accountability. To this end, JICA has made increasingefforts to expand and strengthen its evaluation systems, to improve the quality of evaluation, and to enhance theobjectivity and transparency of evaluation results. As part of these efforts, JICA performs evaluations at four different stages during the project cycle – ex-ante, mid-term, terminal and ex-post – so as to judge as objectively as possible the relevance and effectiveness of its cooperation. JICA also advocates the use of external experts inthe evaluation process and secondary evaluations, which are performed by the Advisory Committee onEvaluation with a view to enhancing the transparency and quality of its evaluation work.

There are two categories of JICA evaluation: evaluations of individual projects and evaluations of programsconsisting of multiple projects.

Project-level evaluations cover individual projects at each stage of the project cycle, frompre-implementation to post-completion. The results of project-level evaluations are utilized to plan and manageindividual projects, to review the project plan, to judge the termination of the project, and to formulate and implement other similar projects.

Program-level evaluations summarize and analyze from a broad cross-sectional perspective the effectivenessand implementation-related issues of JICA assistance programs for a specific recipient country, sector ordevelopmental issue, or of specific forms of assistance, and draw lessons learned and make recommendations forfuture assistance programs. For example, JICA is implementing synthesis studies, comprehensive analysis ofevaluation results for similar projects related to specific issues and sectors, in order to analyze shared trends andissues, good practice, etc. and to identify lessons learned and recommendations that can be applied to futureassistance. These synthesis studies examine sectors in which JICA has worked most in the past and theprioritized issues in the international trends of assistance. For example, FY 2003 synthesis studies of theprimary/secondary education sector drew lessons that can be widely applied not only to primary/secondaryeducation, but also to other education and training. These included the organization of categories for the effectivedissemination of teacher training, cooperation with recipient government agencies and regions in order toconsolidate training, education sector evaluation method, etc.

In order to improve the quality of evaluations and strengthen feedback of evaluation results into projects,JICA introduced in FY 2003 the chief evaluator system and evaluation network, and developed a system forpromoting utilization of past evaluation results in the ex-ante evaluation stage. In FY 2004, JICA furtherincreased these efforts and launched a new initiative that involves gathering examples of best practice andaccumulating knowledge from evaluation findings. Furthermore, with a view to promoting the utilization ofevaluation results and achieving accountability, JICA is endeavoring to make reports available to the public andquickly post summaries of evaluation results on its website. JICA also holds evaluation seminars to providefeedback concerning major evaluation results, particularly of program evaluations, to the Japanese public andconcerned people in the recipient countries.

Chapter

2AnOverview

ofEvalu

ationResults

77


2.3.2 Program-Level Evaluations

(1) Thematic Evaluation “Communicable Disease Control in Africa”

Evaluation Advisors:Hiroshi Suzuki Professor, Niigata University Graduate School of Medical and Dental

SciencesYasuo Moritsugu Ex-Deputy Director, National Institute for Infectious DiseasesKoichi Miyoshi Professor, Graduate School for Asia Pacific Studies, Ritsumeikan Asia

Pacific UniversityHead Office:

Office of Evaluation, Evaluation Planning and Coordination Team, JICA Satoko Miwa, DirectorKazuaki Sato Muneyuki Kozu

Consultant:System Science Consultants, Inc.

Evaluation objectives With the recent spread of emerging and reemerging infectious diseases such as SARS and HIV/AIDS

provoking a renewed awareness of the importance of infectious disease control, measures to prevent HIV/AIDS, malaria and tuberculosis have become a priority issue in Japan’s program of health and medical assistance forAfrica. The Noguchi Memorial Institute for Medical Research (NMIMR) in Ghana, the Kenya Medical ResearchInstitute (KEMRI), and the Zambia University Teaching Hospital (UTH) have been selected as evaluation targetsfor Japan’s cooperation in the field of infectious disease control in Africa and their respective series ofcooperation activities are evaluated here. In the process of the evaluation study, the outcomes of research oninfectious diseases by the institutes and the ripple effects in the field of public health are examined in comparison with the positioning, functions and expected roles of the institutes in order to deriverecommendations for project planning and implementation.

Evaluation framework(1) Evaluation questions

How do the research institutes function, and what ripple effects have they brought to the field of publichealth in comparison with the positioning and functions of the institutes under the health sector of thetarget countries and neighboring countries through Japanese cooperation with representative researchinstitutes for communicable diseases control in Africa?How should the institutes be positioned and how should they fulfill their role in infectious diseases controlin the health sector of the target countries?

(2) Evaluation methods In organizing and analyzing the functions and roles of the three research institutes in infectious disease

control, the functions of similar research institutes in developed countries were analyzed and classified into three categories so as to conceptualize the general functions of these research institutes: “research”, “human resourcedevelopment” and “contributions to health services”. The field survey was conducted to verify the function andpositioning of the three research institutes from the three perspectives and to analyze the ways in which JICA’scooperation has strengthened these functions and the contributions it has made to infectious disease control.

78

Evaluation resultsIn Ghana, cooperation was initially extended by Japan to Ghana College of Health Services in 1968, Grant

Aid was later extended to the NMIMR in 1977 and in 1986 JICA started project-type technical cooperation,known as the Noguchi Memorial Institute Project. Since then, JICA has continuously cooperated with theNMIMR for the purpose of improving its research capacity in infectious disease and parasite control over a period of 28 years up to 2004 when this study was conducted. Currently, the West African Center forInternational Parasite Control Project is underway. Meanwhile, the Japanese government supported thedevelopment of facilities and equipment for the NMIMR through the execution of the Noguchi MemorialInstitute Power Grid Development Project in 1988 and the Noguchi Memorial Institute Rehabilitation and Extension Project in 1997.

In Kenya, KEMRI was established in 1982 with the completion of its main facilities through Grant Aidknown as the Kenya Medical Research Institute Construction Project. Using KEMRI as the base for its project-type technical cooperation, JICA has implemented seven projects targeting infectious disease control,starting with the Communicable Disease Research and Control Project and including the Infectious DiseaseControl Project and the International Parasite Control Project, both of which are now underway. In this regard,the government of Japan assisted in the development of KEMRI facilities and equipment, through Grant Aidtitled the Project for Improvement of the Kenya Medical Research Institute in the Republic of Kenya in 1997.

In Zambia, JICA launched project-type technical cooperation in infectious disease control in 1989, namelythe Infectious Disease Project, designating the UTH as an implementing organization, and the HIV/AIDS andTuberculosis Project is underway at the time of evaluation. The Virology Laboratory and TB Laboratory at UTHwere developed to provide a basis for project-related operations as part of the technical cooperation that has beenextended to UTH.

All three research institutes examined during this study have achieved results as centers of research,contributing to the development of health professionals both at home and in neighboring countries.

Though the positions as a research institute in infectious disease control of the NMIMR, the KEMRI, andVirology Laboratory and TB Laboratory of the UTH are all different, it was found that all these researchinstitutes have contributed to infectious disease control through research, human resource development, andcontribution to health services, which can be summarized as follows.

(1) Collaboration with the Ministry of Health and infectious disease control Despite different organizational positions as research institutes, they all contribute to infectious diseasecontrol of the governments by dispatching staff to provide technical advice to health programs andcommittees implemented by the Ministries of Health beyond the framework of the research institutes.At NMIMR, for example, which as a Ghana College-affiliated research institute falls under thejurisdiction of the Ministry of Education, a quality assurance system for state-run tuberculosismicroscopy centers is established with the cooperation of the Ministry of Health-supported nationaltuberculosis control program.

(2) Contribution as a reference laboratory and a collaborating centerAs shown in the major contribution made by the Virology Laboratory of the UTH to polio eradication inZambia as a reference laboratory, all three research institutes conduct tests and diagnosis which cannot be replaced by any other laboratories in the country or the surrounding countries as top referral. TheVirology Laboratory of the UTH is responsible for planning, implementing and monitoring the poliosurveillance system operated in cooperation with two government-run programs: the Expanded Programon Immunization and the polio eradication program.

Chapter

2AnOverview

ofEvalu

ationResults

79


(3) Human resource developmentAll three research institutes function as centers for human resource development, providing technicaltraining for medical personnel, not only inside the countries but also for the surrounding countries. Forexample, the NMIMR provides third-country training for African countries in collaboration with theWHO even after termination of JICA cooperation. As a base for parasite control, the NMIMR alsofunctions as a center for human resource development of the surrounding countries through third-countrytraining by the West African Center for International Parasite Control (WASIPAC).

The research facilities and equipment of these three institutes were built and upgraded with support fromJapan, which contributed to public health. Continuous implementation of technical cooperation has beencombined with improvement of facilities and equipment essential for research activities with Grant Aid and technical cooperation projects and, as a result, research capabilities of these research institutes have improved assynergistic effects. It was confirmed that as a result of technical transfer and long-term infrastructuredevelopment by Japan, these three research institutes conduct research activities in line with the health policy as a base of infectious disease control not only for the country but also for surrounding countries, and the activitiescontribute to improved of public health. The establishment of research infrastructure has made it possible tocarry out joint research activities with overseas research institutes, leading to further improvement of researchcapabilities. Moreover, acquisition of grants for research also strengthens the financial basis.

As observed at infectious disease research institutes in developed countries, such institutes do not simplyundertake research but provide a range of services, including human resources development, surveillance andreferral services, which lead to improvements in public health. It has been confirmed that in reinforcing thisframework (the function as research institutes), Japan’s cooperation has served to enhance the control ofinfectious diseases both within the respective countries and in the surrounding countries.

Major recommendations Japan has continuously provided technical cooperation to the NMIMR, the KEMRI, and Virology Laboratory

and TB Laboratory of the UTH over 20 to 30 years since their establishment. These research institutes havegrown to be leading research institutes in Africa.

The following list comprises recommendations on the direction of future cooperation.

(1) Strengthening relationship as development partnersFor the future, it is necessary to strengthen partnership relations through joint/commissioned researchwith the related institutes in Japan, while respecting the independence of research institutes on theAfrican side as much as possible.

(2) Cooperation in maintaining function as reference laboratories Against calls to enhance the functions of these institutes as reference laboratories for HIV/AIDS control,JICA is required to examine the necessity of assistance for these institutes to maintain the referencefunction and continuously contribute to infectious disease control.

(3) Utilization of research institutes as training centersWhen reviewing its assistance policy, JICA should consider to utilize the training capabilities of theseresearch institutes on other health projects implemented in Africa, and to consign third-country training tothem as the implementation agencies for South-South cooperation.

(4) Strengthening regional networksJICA positions the three research institutes as bases for infectious disease control in western, eastern andsouthern Africa, respectively. In the future, it is desirable to establish regional networks based on thethree research institutes, and to promote South-South cooperation including the dispatch of third-countryexperts.

80

The Kenya Medical Research Institute (KEMRI) A ward at the municipal health center of ZambiaUniversity Teaching Hospital (UTH)

(2) Thematic Evaluation “Volunteer Program (Japan Overseas Cooperation Volunteers)”

Chapter

2AnOverview

ofEvalu

ationResults

Evaluation Advisors:Hideo Kimura Professor, Graduate School of Arts and Sciences, the University of

TokyoAkiko Seto Principal of the Japan College of Foreign Languages, a non-profit

education foundationHead Office:

Office of Evaluation, Evaluation Planning and Coordination Team, JICA Satoko Miwa, DirectorKazuaki Sato Chihiro Saito Yoko Kaburagi Japan Overseas Cooperation Volunteers SecretariatRyuichi Nasu, Assistant to the director

Consultant:KRI International Corp.

Evaluation objectives JICA dispatches volunteers overseas under such programs as the Japan Overseas Cooperation Volunteers

(JOCV) Program, which marked its 40th anniversary in 2005. The JOCV Program is one of the pillars of JICA’sprograms as it aims to work with local people to meet the grassroots needs of developing countries and promoteand support the participation of Japanese citizens in international cooperation efforts.

Since its transformation into an independent administrative institution in FY 2003, JICA has made efforts todevelop evaluation methods suitable for the nature and characteristics of the JOCV Program, deciding toevaluate from three viewpoints: “contribution to social and economic development or reconstruction in thepartner country”, “promotion of friendly relationships and mutual understanding between Japan and the partnercountry”, and “sharing of volunteer experiences with society”. JICA officially introduced the evaluation into theJOCV Program in FY 2004.

This evaluation is complementary to the above program evaluation and comprises a comprehensiveevaluation conducted from the above-mentioned three perspectives of the outcomes of long-term cooperation(the past 10 years) in three countries: Malawi, Vanuatu and Honduras, through the JOCV Program as a typical example of Japanese volunteer programs.

81


A JOCV teaching a music class (Vanuatu) A JOCV teaching in elementary school (Honduras)

Evaluation framework(a) Evaluation questions

How has the dispatch of JOCV contributed to the three perspectives outlined above?How should the JOCV Program improve in order to generate greater effects?

(b) Evaluation methodsIn order to assess the outcomes of the JOCV Program in terms of the three aforementioned perspectives,this evaluation was conducted based on a document review, a questionnaire survey coveringex-volunteers, current volunteers and the host organizations, interviews with current volunteers,JICA-related parties, partner country governments, host organizations and beneficiaries, and observationsof volunteer activities.

DiscussionViewpoint 1: Contribution to social and economic development or reconstruction in the partner country

The survey results show that in the case study countries, plans for the JOCV Program are designedaccording to the development issues of the partner countries. The results also show that cooperation that meets local needs is implemented at a grassroots level. The characteristics of the JOCV Program—thatvolunteers work in local communities with local people—is appreciated by beneficiaries, host organizationsand other aid organizations.In their activities, many JOCVs try to actively communicate with local people, understand local customsand culture, and adapt to local society. Their cooperation activities have achieved certain results, althoughthe level of achievement varies depending on the host organization and individual volunteer. It was provedthat various types of dispatches and activities, such asgroup dispatch and formation of subcommitteesamong the volunteers engaged in similar activities,helped to achieve greater effects. Although there areoperational issues to be solved, ,it is important toimplement various types of volunteer dispatches andcombine a JOCV dispatch with other technicalcooperation projects in order to generate long-termeffects. It should be noted, however, that whileplanning various types of cooperation would beeffective, designing a dispatch plan that takesadvantage of being with local people, which is one ofthe JOCV Program’s characteristics, should not beoverlooked. In other words, an environment thatexploits the strength of volunteers should be

Interviewing farmers in a region whereJOCV is working (Malawi)

82

provided.In addition, it is necessary to address the problems referred to as inhibiting factors by both JOCVs and thepartner countries, such as language ability, the time-lag between a request for dispatch and actual dispatch,the timing of dispatch, the continuity of activities, and support systems by the JICA offices.

Viewpoint 2: Promotion of friendly relationships and mutual understanding between Japan and the partnercountryIn referring to the promotion of friendly relationships and mutual understanding between Japan and thepartner country, the concept of “friendly relationships and mutual understanding” is so wide-ranging that ithas not been clearly defined how to measure the degree of such promotion in past research and surveys. Thepurpose of this evaluation, therefore, was to examine what effects the JOCV Program has had in terms of friendly relationships and mutual understanding.It was found that regarding cultural exchange, which was considered a typical example of friendlyrelationships and mutual understanding, local people are not very familiar with general information onJapan such as Japanese culture and customs while some volunteers are actively trying to introduce Japaneseculture. Rather, it was found that Japanese values such as commitment to one’s job, punctuality and respectfor others had become known to local people and that these attitudes of volunteers led to the formation of local impressions about Japan and Japanese people.

Chapter

2AnOverview

ofEvalu

ationResults

It was also found that the JOCV field activities were based on relationships with local people. The activityprocess starts with the promotion of friendly relationships, followed by the development of understandingof the partner’s customs and values. Changes and development in the volunteers’ own values were alsoobserved. These are the effects and characteristics of the promotion of friendly relationships and mutualunderstanding in the JOCV Program.If the JOCV Program aims for the promotion of friendly relationships and mutual understanding, it is necessary to better clarify what mutual understanding is and what we expect from the program. Based onthe results of this evaluation, JICA’s strategy––what should be aimed for and what should be evaluated inthe JOCV Program in terms of the promotion of friendly relationships and mutual understanding––shouldbe thoroughly reviewed.

Viewpoint 3: Sharing of volunteer experiences with societyResults from this study reveal that awareness of the importance of activities for sharing JOCV experienceswith society is fairly high among ex-volunteers, and that many of them experience changes in terms ofmentality and humanity through participation in the JOCV Program. In addition, many of them havechanged their daily attitude, or communicate their experiences to share them with society. On the other hand,the survey results show that only a limited number of ex-volunteers engage in activities to contribute toJapanese and international societies.It is important to analyze what the JOCV Program expects from ex-volunteers and to explain to them beforeand during dispatch, referring to specific examples to raise their awareness. “Pursuit of career ininternational cooperation” and “use of local language” are not widely observed among the ex-volunteersand it may indicate that not all the ex-volunteers are interested, and in any case such opportunities are limited.

Major recommendations To design an effective JOCV Program, it is important to make the most of the characteristics of the JOCVProgram, i.e. accurate grasp of local needs by working with local people at grass-root level.–– It is important to implement the JOCV Program recognizing and maximizing the characteristics of the

JOCV Program: it can provide effective assistance at grass-roots level, promote poverty reduction, andidentify local needs, because volunteers live close to local people.

83


Group dispatch, dispatch to projects, and collaboration with other aid modalities are effective measures forextending the effects of the JOCV Program throughout the partner country. However, the following issuesshould be noted in implementing these approaches.–– It was found that approaches such as group dispatch, especially dispatch to projects, and collaboration

with other aid modalities, have been effective. However, it is also important to clarify the position andlimitations of the JOCV Program and make sure that individual volunteers are well informed of the roleto play. For the smooth operation of the JOCV Program, it is also effective to explain well to volunteersin advance and ensure their understanding about which abilities (specialty, language ability,communication ability, etc.) are expected, and to what extent, and who coordinates the whole project.

In order to enhance the promotion of friendly relationships, mutual understanding and sharing of JOCVexperiences with society, it is important to review again what is expected in the JOCV Program in thisregard. Then, it is important to communicate to volunteers before and during dispatch the necessity ofmutual understanding, friendly relationships, and activities for sharing JOCV experiences with society as well as their effects as presented in this evaluation.–– It is necessary to clarify the “promotion of friendly relationships and mutual understanding” and

“sharing of JOCV experiences with society”, and what is expected of the JOCV Program. In addition, itmight help newly dispatched volunteers in their activities to inform them of the issues the past volunteersconsidered important and behaviors and attitudes that have promoted the partners’ understanding.

2.3.3 Project-Level Evaluation (An Example of a Terminal Evaluation)

Country: CambodiaProject title: The Maternal and Child Health Project (Phase 2)Division in charge: Human Development Dept, Group 4Total cost: About 630 million yen (at the time of evaluation) Period of cooperation: April 2000 – March 2005

Partner country’s implementing organization: Ministry of Health, National Maternal and Child Health Center (NMCHC)Supporting organization in Japan: International Medical Center of Japan

Background and outline of cooperation Cambodia officially ended its 30-year civil war at the Paris Peace Conference in October 1991, and made its

first significant step towards democracy with the general election in 1993. In March 1992, Japan dispatchedmedical advisors to the Cambodian Ministry of Health to study the situation for three years. As a result, it was revealed that as well as medical facilities, human resources had been devastated and the training of medicalpractitioners was urgently required. Responding to the study results, Japan pledged to provide technicalcooperation in the field of maternal and child health, which had little support from other agencies, and build a national maternal and child health center with Grant Aid. In response, JICA provided project-type technicalcooperation for the purpose of improving the management and operation ability of the center, training activities,and examination//treatment standards between 1995 and 2000. The new center, which was built with Grant Aid,was opened in April 1997. Though the outcome of the first-phase technical cooperation was highly praised, itwas still necessary to enhance the clinical, training, and hospital management divisions. In particular, it was essential to improve the local maternal and child health service. Thus, the second-phase project, whose aim wasstrengthening human development for the improvement of maternal and child health, was implemented for fiveyears starting in April 2000.

84

Evaluation ResultsRelevance: Cambodia still has higher maternal death and infant mortality rates than its neighbors. In

particular, health/medical care workers at local medical institutions are in short supply. To develop highquality health/medical care workers is the primary issue of the strategic plan in the health/medical caresector and corresponds to the needs of Cambodia. The enhancement of health care service is a priority cooperation sector for Japanese cooperation with Cambodia, and is included in the priority cooperationsector of the JICA Country Program as well. As the target group of the project is women and children,and the medical fee exemption system for needy people has also been introduced to NMCHC, it benefitsthe vulnerable groups in society.

Effectiveness: Human resource development in health care largely depends on instructors, teachingmaterials, facilities, and training management, which were thus addressed comprehensively to increasethe achievement level of the project. The training that NMCHC conducted for doctors and midwivesfrom referral hospitals and health centers has been accredited as a national training course (MinimumPackage of Activities: MPA/Complimentary Package of Activities: CPA Training). The accumulatednumber of medical care service workers developed by the project from the first phase is 25 doctors and110 midwives working at referral hospitals, and 303 midwives working at health centers. Though theratio appears small compared to the total number of medical care workers in Cambodia, this is becausethe focus of the project is placed on developing human resources who can be the core or leaders to provide good quality medical service at medical institutions. Trainees are very satisfied with the courseand they gained adequate knowledge and skills by participating in the course. Though they face problems such as a shortage of manpower, equipment, and medicine at their work sites, they utilize theirknowledge and skills as much as they can.

Efficiency: As equipment was provided substantially during Phase 1, the provision during Phase 2 wasminimized and no delay in the timing of equipment provision was observed. Furthermore, thanks to theproper selection of equipment, the equipment is utilized effectively, and as a result of the activities toestablish a system for maintenance and management of the equipment, a high operation rate ismaintained. In terms of training cost, in addition to the burden borne by the national budget, advanceddonor coordination brought support from other donors such as UNICEF. As a result, the initial costborne by JICA, which was about 80% at the initial stage of the project, has decreased every year, and infact decreased to about 20% in the fourth year. Since the range of activities is wide, a relatively largenumber of counterparts received training, to which short-term experts were dispatched. However, byincorporating third-country training and third country experts, the cost was reduced. The timing for allinputs was appropriate and there was no major delay.

Impact: This project succeeded in developing medical care workers equipped with adequate knowledge andskills and contributed to achieving the overall goal (to improve the quality of maternal and child healthservice in Cambodia). If the working conditions and shortage of medical equipment, medicine, andmanpower at referral hospitals and health centers improve and the environment where trained medicalcare workers can demonstrate their learned skills is created, then it is possible to further increase thechance of achieving the overall goal. The Ministry of Health instructed hospitals/health centers tointroduce the medical care cost collection system that this project started and the system has spreadwidely throughout the country, showing the emergence of an institutional impact. Also, some of theformer trainees have become trainers for activities by NGOs and other donor agencies, showing theemergence of technical impact as well.

Sustainability: At NMCHC, where systems to plan, manage, evaluate hospital management, training, andequipment management were established, the development of the next generations’ human resources tolead in clinical care and training, as well as strengthening the management capacity for the programcalled Preventing Mother-to-Child Transmission (PMTCT) of HIV, are tasks for organizationalsustainability. Most of the equipment at NMCHC will become obsolete in the next five years and new

Chapter

2AnOverview

ofEvalu

ationResults

85


equipment must be purchased. Financial sustainability, however, can be maintained because of thestable financial source of the medical care cost collection system as well as increased expenditures ofthe national financial resource to NMCHC. As technology transfer to the counterparts was smoothlyconducted, technical sustainability is high. However, since guidance by Japanese experts will not beavailable after the completion of cooperation, they need to secure alternatives to acquire new technologyand information.

Contributory Factors Respect for the initiative and ownership of the partner government and implementing organization by

holding workshops during the planning stage and reflecting the opinions of local stakeholders in the project plan,as well as continuous support to address any concerns identified by the host country, contributed significantly tothe sustainability of project implementation.

Sector-wide approaches are gaining ground in Cambodia’s health sector, which means that the Ministry ofHealth and donor organizations are endeavoring to share information and coordinate their activities. Moreover,NMCHC is not only a member of the working group, its director is also the chairman and coordinator, and thishas had a positive impact on project effectiveness and efficiency by promoting understanding and increasing thenumber of organizations that are willing to cooperate.

Inhibitory FactorsThe performance indicators, numerical targets or indicators for measuring improvements in the quality of

counterparts and trainees were not clearly defined during planning and implementation, which made it difficultto respond to any changes in indicators or verify the extent to which the goals were achieved. This projectinitially assumed the task of training medical care workers at regional training centers (RTC) and referralhospitals in addition to the NMCHC. However, the training and supervising function of RTC and referralhospitals was so weak that the training was conducted only at NMCHC, which is the central level. This choiceseems reasonable. However, there is a lack of examination for a specific path to utilize trained personnel throughthis project to improve local-level health and medical care service.

Midwives’ training The National Mother and Child Health Center

86


Synthesis Study of Evaluation Results “Primary/Secondary Education/Science and Mathematics Sector”In the basic education sector, lessons from the past projects and cross-cutting analyses have been used in

planning and implementation of new projects.The Project for Strengthening Cluster-based Teacher Training and School Management in Vietnam and the

Strengthening Child-centered Approach in Myanmar Education have included training for school managerscoupled with teacher training in the activities. This action was based on the lesson that it is important topromote understanding and establish a system to encourage the participation of school managers andadministrators, which was obtained from the result of Synthesis Study on Evaluation in Science andMathematics Education Projects. Furthermore, in the Strengthening Child-centered Approach in MyanmarEducation and the Project for Development of Science and Mathematics Teaching for Primary and SecondaryEducation in Indonesia, the locus of responsibility for training costs has been clarified with the localstakeholders in an effort to ensure sustainability after the termination of cooperation. This was based on thelesson that it is necessary to reach agreement with local stakeholders on the apportioning of training costs inadvance to ensure project sustainability, which was lesson learned from the Strengthening of Mathematics andScience in Secondary Education in Kenya. In addition, there are other cases of using past lessons for projectplanning and management. For instance, in order to implement activities in accordance with the academicyear of local schools, the duration of a project was adjusted flexibly.

Synthesis Study of Evaluation Results “Information and Communications Technology”In the information technology sector, lessons from various similar projects in the past have been utilized

for subsequent projects.Based on the lesson learned from the Information Technology Human Resource Development Project in

the Philippines that cooperation projects in the IT sector should be short, intensive programs to avoid obsoletetechnologies during the project period because of the rapid pace of technological advance, the duration of theInformation Technology Upgrading Project in Jordan was shortened from five years to three. This project wasthe first case of outsourcing to the private sector in this field, and it enabled the short-term dispatch of expertswith the right knowledge and appropriate timing. This system actually responded to a lesson learned from theVietnam Information Technology Training, which explains that expert dispatch programs should be combinedwith short-term experts to respond to rapid technological advances and specialized technologies. This projectshed light on private outsourcing, which was utilized subsequently. Other lessons from past projects were applied to the Information Technology Upgrading Project in Jordan and Information Technology HumanResource Development Project in the Philippines. These lessons are the following: the first is the introductionof commission-based salaries for lecturers by courses that were charged. Equipment which can be procuredlocally, such as computers, needs to be obtained from the country. These efforts have allowed for a proper response to challenges such as obsolete technologies and equipment and the specialization of technologies,thus leading to improved quality of projects. Enhancing the system of training centers and charging fortraining programs have contributed to the sustainability of projects after the completion of cooperation.

Chapter

2AnOverview

ofEvalu

ationResults

87


2.4 Results of Evaluation by Japan Bank ofInternational Cooperation (JBIC)

2.4.1 Overview

JBIC undertakes ex-ante evaluations during the preparatory stages of projects and ex-post (project andthematic) evaluations at the completion stage. In FY 2004, with a view to enhancing the consistency of its evaluation system from ex-ante evaluation to ex-post evaluation, JBIC introduced, on a trial basis, mid-termreviews which are conducted five years after the conclusion of a loan agreement and focus on the relevance ofproject plans and the project’s effectiveness, as well as ex-post monitoring, which is conducted seven years afterproject completion and assesses the project’s effectiveness, impact and sustainability. JBIC also evaluates theMedium-Term Strategy for Overseas Economic Cooperation Operations, which determines the basic conceptsand direction of ODA loan assistance.

Ex-post evaluations verify, as quantitatively as possible, the efficiency, effectiveness, sustainability, etc. of projects, based on international standards and incorporating third-party perspectives. All projects are ex-postevaluated in order to ensure full accountability and to enhance the effectiveness and efficiency of ODA loanoperations. In order to publish evaluation results in a clear and objective way, a four-grade rating system is applied. All the evaluation results are posted on the JBIC website. There are two kinds of ex-post evaluation:program-level evaluation (thematic evaluation) of multiple projects with the same theme and project-levelevaluation (project evaluation) of individual projects.

The evaluation results, lessons learned and recommendations obtained from evaluation through cooperationwith related departments responsible for the review and project supervision of ODA loans and the offices ofpersonnel stationed in the recipient country are fed back directly and effectively by the Development Assistance Operations Evaluation Office of the Project Development Department. The information is also accumulatedwithin JBIC, and widely shared with the developing countries.

In order to further improve the evaluation system, JBIC established the Feedback Committee of Ex-postEvaluation of ODA Loan Projects, chaired by JBIC’s Executive Director in charge and with external expertsamong its members. Meetings of the Committee are held twice a year. The Committee discusses various issues,including the evaluation system and method, feedback activity, and evaluation capacity development. Theoutcome of discussion is reflected in the evaluation activities in an effort to improve evaluation and feedback.

88

2.4.2 Program-Level Evaluations

The Philippines Credit Support to Beneficiary of Agrarian ReformExternal Evaluator: Dr. Eulogio T. Castillo, PhD

(Professor, University of the Philippines Los Banos)Field Survey: October 2005

Outline and ObjectivesIn the Philippines, the Comprehensive Agrarian Reform Program

(CARP) was established under the Aquino administration with full-scaleimplementation beginning under the Ramos government, inaugurated in1992. There are two pillars to agrarian reform in the Philippines: landdistribution and support for beneficiary farmers. With approximately 70% ofthe 8.06 million hectares of land targeted for reform (which corresponds toaround 30% of the Philippine land mass) having now been transferred tofarmers, the importance of the latter pillar is gaining ground. In this project(Rural Assistance Support and Credit Program = RASCP), the successor of the Agrarian Reform Infrastructure Support Project (ARISP), in which smallscale irrigation systems were developed, the Department of Agrarian Reform(DAR) and the Land Bank of the Philippines (LBP) provide the 71 AgrarianReform Cooperatives (ARC; principal members are agrarian reformbeneficiaries) with technical assistance so that the ARC can arrange loansfrom the LBP. Moreover, the LBP extends loans to agricultural cooperativesincluding ARCs to support credit needs of agrarian reform beneficiaries andother group in rural areas. As with project evaluations, after the five DACcriteria were employed, a detailed analysis focusing on the effectiveness of technical and financial assistance for the ARC was conducted.

Chapter

2AnOverview

ofEvalu

ationResults

Rural Assistance Support and CreditProgram (evaluated project)

Agrarian Reform InfrastructureSupport Project (related project)

Loan Amount 10.799 billion yen 6.151 billion yenDisbursed Amount 10.717 billion yen 5.816 billion yenLoan Agreement March 1996 August 1995Executing Agency Department of Agrarian Reform, Land Bank of

the Philippines Department of Agrarian Reform

Evaluation Results(1) Project Evaluation based on the Five DAC Criteria

Using the same criteria as are employed for project evaluations, the following ratings were awarded to thisproject: Relevance: a; Effectiveness: a; Efficiency: a; and Sustainability: b; with an overall rating of A.

(2) Technical Assistance for the Agrarian Reform Cooperatives (ARC)The goal of the technical assistance was to strengthen the institutional capacity of infant ARC until when

they would meet the credit requirements set by the LBP. More specifically, these criteria include securing aminimum regular cooperative membership (60 members) and capital (30,000 Pesos) and holding of boardmeetings on a regular basis.

89


In this project, development advisors (NGOs, etc.) were deployed to strengthen the administrative structuresof the ARC and to assist the formation of ARCs. As a result, 48 of a total 71 ARC were able to obtain credit fromthe LBP. At the Jagna-Nagbalaye cooperative, for example, extensive work was undertaken to develop thefoundations for cooperative activities: rules were revised and developed for newly-launched projects, as aconsequence of which the cooperative succeeded in meeting credit requirements.

(3) Financial Assistance for the Agrarian Reform Cooperatives and their MembershipThe ARCs received financial assistance from the LBP for initiating various projects and for lending funds to

their members (to procure seeds, livestock, farm equipment, etc.). As shown in Graph 1, a pre and post projectcomparison of six cooperatives receiving both technical and financial assistance reveals that implementation ofthe RASCP provided a massive boost to cooperative activities. Two groups were subsequently comparedvis-a-vis income status of household level: a group that received financial assistance and a group that received noassistance. As Graph 2 demonstrates, the outcome of the financial assistance is confirmed by the fact thatfinancial expenses of the former equate to approximately half those of the latter. By contrast, after completion ofthe RASCP, ARC projects have tapered off and there are signs of a decline in repayment rates.

(4) Cooperatives (other than ARC)Under the RASCP, financial assistance via the LBP was also extended to agricultural cooperatives other than

ARCs. Eight of the beneficiary cooperatives were compared, revealing improvements resulting from projectimplementation (Graph 3).

(Numberof

projects)Pre-implementation

Mid-implementation (10,000 pesos)

6.3-foldincrease 4.7-fold

increase

Total projects(number)

Total investment(10,000 pesos)

Average ownedcapital

(10,000 pesos)

2.9-foldincrease

Cooperative farmerstargeted for financial

assistance(20 households)

On-farm incomeFinancial expense

IncomeOutlay Balance (Pesos)

Cooperative farmersnot targeted for

financial assistance(37 households)

Graph 1: Pre- and mid-term changes in 6 agrarian reform cooperatives targeted for assistance

Graph 2: Differences in average householdexpenses based on availability of financialassistance

Average ownedcapital

(10,000 pesos)

(Number ofprojects)

Pre-implementation

Mid-implementation (10,000pesos)

1.3-foldincrease

2.0-foldincrease

Total projects(number)

Graph 3: Changes in project numbers / capitalat 8 general cooperatives targeted for assistance

Feedback seminars were also attendedby ARC representatives

90

Feedback ResultsThe following opinions were put forward during feedback seminars held for project proponents.Various comments were heard from the cooperatives, which argued that LBP loan eligibility requirements

were too rigorous, that in order to avoid becoming dependent on credit it was necessary to increase capital and tobuild up savings, and that rapid expansion of cooperative activities risked inviting failure.

The Department of Agrarian Reform (DAR) pointed out that many ARC have managed to meet LBP crediteligibility requirements and this shows the success of the technical assistance component. In addition, the DARput forward a number of ideas geared toward enhancing project sustainability, including alliance of thecooperatives.

The LBP stressed the importance of financial assistance to farmers, and argued that instead of easing creditrequirements, priority be given to improving the capabilities of the cooperatives, pointing out that the ARCs which fulfilled LDP’s credit requirements have more sustainable projects.

Conclusions and RecommendationsThrough this project, the following framework for technical assistance was formulated. First, the DAR

supported the initial stage of capacity building of the ARCs. After the ARCs had met some targets formembership and capital, the LBP started financing and accounting-based support and strengthened institutionalcapabilities until the ARCs met LBP credit requirements. Nonetheless, there are many cooperatives throughoutthe Philippines that have yet to meet these requirements and there remains a pressing need for DAR and the LBPto provide ongoing technical assistance. In order to sustain the framework developed in this project, the budgetfor technical assistance must be appropriately distributed to both organizations.

Chapter

2AnOverview

ofEvalu

ationResults

JBIC’s OpinionThe LBP is committed to achieving the policy objective of supporting agrarian reform but is simultaneously

a financially independent organ. JBIC therefore recommends that expense requirements for technical assistance to the ARC be clearly identified and a support system established, e.g. to ensure that a portion of DAR budget is assigned to the LBP to cover the expense of technical assistance.

Dried copra in warehouse storage

91


92

ChinaAssistance for Environmental Improvement (Air/Water Quality)External Evaluator: Kyoto University Field Survey: January-November 2005

Outline and ObjectivesRapid economic development in China has led to exposed various environmental problems, including air and

water pollution. It has thus become necessary to extend environmental cooperation in the form of pollution abatement measures, the installation of sewerage treatment equipment, etc. JBIC has supported numerousenvironmental improvement projects in China. Although many of these projects are currently in progress, basedon heightened levels of domestic and international interest in the effectiveness of and necessity for ODA loans to China, there is now a pressing need to evaluate how effective such projects have been in improving theenvironment and what contribution they have made, if any, to enhancing/reforming environmental policies. Thisevaluation is designed to analyze the effectiveness of ODA loans to assist environmental improvement in Chinaand their role in enhancing environmental policies and systems with a view to determining the effects of theseprojects and considering future challenges, and, based on quantitative analysis, to estimate the effects on improvement of the environment through the policy reform of the Chinese government.

The 16 projects targeted for evaluation

Atmospheric changes in Guiyang pre- and post-project implementation

2000 2004

Photo provided by:Japanese expert team of the JICAChina-Japan Friendship Center for Environmental Protection Project

Evaluation Results(1) The Effectiveness of ODA Loans

Of the loans pledged to China during the fourth multi-year ODA loan period (1996-2000), 16 projectstargeting air and water pollution control in key regions were evaluated (approximately 160 billion yen). Thisevaluation revealed a 190,000-ton reduction in emissions of atmospheric pollutants (SO2) and a 340,000-tondecrease in water pollutants (COD*) (2003). In addition, the number of beneficiaries of the projects (estimated)is as follows: 3.95 million in the 10 cities covered by urban gas projects, 0.9 million in the 6 cities covered byregional heat supply projects, and 13 million in the 28 cities covered by sewerage projects.

(2) The Effectiveness of ODA Loans in Enhancing/Reforming Environmental Policies (i) Funding support to environmental investment planning promoted environmental pollutant control

projects and supported the introduction of total emission regulation in key regionsIn formulating its ninth five-year development plan (1996-2000; the 9.5 plan hereunder), the Chinese

government set three targets: (i) the introduction of total emission regulation for pollutants, (ii) pollutioncontrols in key regions, and (iii) increase in environmental investment (RMB450 billion, of which US$4billion was to be procured as foreign capital). To procure these funds, the government worked out theChina Trans-Century Green Project Plan, which lists environmental pollutant control projects in keyregions. The Japanese government, meanwhile, informed the government of China that it wouldemphasize environmental protection in its cooperation with China and pledged ODA loans to projectslisted in the Green Project Plan. These loans provided the underlying support, with the Chinesegovernment investing RMB360 billion in the environment and introducing total emission regulation.(The total amount of environmental ODA loan is equivalent to approximately 30% of the foreign capital,and also equivalent to some 7% of Chinese government investment).

Chapter

2AnOverview

ofEvalu

ationResults

(ii) Providing the foundations for the enhancement of environmental policy promoting the shift to anenergy-saving/resource-saving economic structure

Through ODA loan-funded factory pollution control projects, the Chinese government has confirmedthe effectiveness of energy-saving/resource-saving production technologies (cleaner productiontechnologies) in curbing environmental pollution without loss of corporate profits. In Benxi, Liaoning,for example, an environmental ODA loan was extended to support investment in cleaner productiontechnologies that served to increase corporate incentives to invest in environmental preservation and toadopt environmental conservation measures. A survey team was dispatched to Japan from Guiyang in conjunction with the implementation of an environmental model city project in that city to learn aboutadvanced systems. This contributed to the development of a recycling-oriented economic policypromoting efficient use of by-products and waste recycling in Guiyang. The decision was subsequentlymade to disseminate this policy on a nationwide basis.

(iii) Promoting the diffusion of systems and technologies relating to urban environmental infrastructuredevelopment

The development of sewage systems and other urban environmental infrastructure was supported byODA loans. This helped the Chinese government engage in eco-conscious urban development anddiffuse sewage disposal technologies. For example, the Gaobeidian sewage treatment plant in Beijinghas introduced state-of-the-art technologies and can boast the highest disposal capacity in China. Thosestaff who took part in Japan-based training programs have been active in China’s sewage industry. Thissewage treatment works has also established its own training center for sewage treatment technologies.

(3) Quantitative Analysis to Estimate the Effect of Policy Enhancement/ReformResults from a quantitative analysis of the effects of environmental policy enhancement/reform revealed a

* Chemical Oxygen Demand. An indicator of the amount of organic compounds in water; the larger the COD the greater the pollution by organic compounds.

93


sharp increase in SO2 emissions from 20.9 million tons in 1995 to 29.2 million tons in 2003; without theimprovements to environmental policy, however, the projection for 2003 was 31.4 million tons. In the result,desulfurization and coal substation effects were calculated to be 2.2 million tons. While COD emissions(industrial and residential) have declined from 22 million tons in 1995 to 14 million tons in 2003, the estimatefor the same year is 52 million tons; assuming that there is no improvement to environmental policy as it turnedout, the effects of the reform in the structure of industry, of environmental load reductions in productionprocesses, and of sewage treatment infrastructure are estimated to be 38 million tons.

Based on existing model research, controls on sulfur oxide emissions in China are expected to make acontribution to curbing the amount of deposition in Japan.

Effect of environmental improvements due to enhanced environmental policies and systems(SO2)

SO2 emissions (10,000 tons)

20.9 million tons

29.2 million tons

31.4 million tons

*

**

(2.2 million tons)

Effect of environmental improvementsdue to enhanced environmental policiesand systems (COD)

COD emissions (10,000 tons)

*

**

Effects of switching toindustrial structures,environmental improvementsin production processes,sewage infrastructure, etc.

22 million tons

14 million tons(38 million tons)

(52 million tons)

* Actual case** Estimate without environmental policy improvements

* Actual case** Estimate without environmental policy

improvements

Recommendations(1) In order to achieve drastic reductions in environmental loads, it is necessary to develop environmental

statistics that will enable an accurate diagnosis of environmental problems and set environmentaltargets from a preventive perspective.

(2) It is critical to consider policies that will give companies incentives to adopt environment measuresbased on the Chinese policy of promoting market economy. Moreover, it is important to strengthenincentives for environmental policy and to boost the enforcement system through companydevelopment of central and regional governments.

(3) In order to address the rapid progress of urbanization, it is necessary to analyze issues andcountermeasures toward realization of sustainable development of urban areas; such as considering the consistency of urban growth/environmental management plans with development of the urbanenvironmental infrastructure.

94

2.4.3 Project-Level Evaluations

ObjectivesThe objective of this project was to help increase number of qualified engineers by providing scholarships

for the study of science and engineering in Japanese universities to Malaysians who wish to study overseas, andthereby contribute to the economic development of Malaysia through promoting science and technology,together with enhancing the implementation of the Look East Policy.

Loan Amount/Disbursed Amount 5,493 million yen/5,317 million yenLoan Agreement May 1992 Terms and Conditions Interest rate 3.0%; Repayment period 25 years (grace period 7 years);

General untiedFinal Disbursement Date September 2002Executing Agency Yayasan Pelajaran MARA (YPM)

Evaluation Result In this project, a pre-departure program prior to study in Japan and an overseas study program at science and

technology departments of Japanese universities were implemented almost as planned. The project period was almost as planned, and the project cost was lower than planned.

Of the 310 students who studied in Japan, 279 students received degrees in such as electricity, electricalengineering, and mechanical engineering. Of the 243 project graduates whose careers were identified, 201 wereemployed in the private sector, primarily in the manufacturing and telecommunications industries (of these, 162were employed at Japanese-affiliated companies). Besides these, 3 became instructors and 37 continued theirstudies as graduate students. Of the 125 graduates whose job types were identified, 94 of them, or 75%, are employed in engineering positions.

The number of Malaysian students studying in Japan was in a downtrend in the latter half of the 1990s, butthis project offered the opportunity to study in Japan with public scholarships from 1995 to 1999. The results ofthe beneficiary survey shows that the graduates are playing a significant role in promoting science andtechnology in Malaysia and in the introduction of new technology from Japanese companies to Malaysia.

There are no problems in the technical capacity,operation and maintenance system, or financial status ofYPM.

As a lesson learned from this project, overseas studyprograms are more effective when the structure of thecooperation and coordination system between the executingagency and the universities that receive students isestablished, and consulting service with detailed responsesthat enables the said system is incorporated in the program.It is hoped to expand follow-up studies and to expand the pre-departure program facilities to meet the needs ofsubsequent projects.

MalaysiaHigher Education Loan Fund Project (HELP) External Evaluator: Takako Haraguchi (Global Link Management, Inc.) Field Survey: October 2004

Chapter

2AnOverview

ofEvalu

ationResults

95


Third-Party Evaluator’s OpinionThrough this project, the students learned and exemplified Japanese corporate ethics and ideology by

working at universities and companies, including Japanese-affiliated companies, and they contribute to economicdevelopment.

Third-Party Evaluator: Ms. Zubeidah Rawther (government-related company)Obtained a bachelor’s degree in economic analysis from the National University of Malaysia. Currently is senior manager of the marketing section of the UDA Holdings Berhad. Specializes in riskmanagement, etc.

Recommendation of JBIC study: introduce distance learningIn the JBIC study conducted during the project’s implementation, a recommendation was made to introduce

distance learning from Japan using a television conference system, and this was actualized. Distance learning has thebenefits of raising the level of education in Malaysia to the same level as in Japan because students can take the sameclasses in Malaysia, while simultaneously leading to cost-savings in the overseas study program.

Demonstration of distance learning

Factors in the Smooth Attainment of Degrees by StudentsOf the 279 students who received degrees, all of them received their degrees from science and technology

departments. Of these, 246 graduated in the standard 4 years, and nearly all of them acquired more than the requirednumber of credits. One of the factors in the smooth attainment of degrees by students was the fact that detailedsupport of the students was provided appropriately by the project consultant and the universities where students enrolled. In the beneficiary survey, there were respondents who said that they resolved academic issues by consultingwith their university lectures and daily life issues with the consultant.

A graduate (left) of this project who is employed as aninstructor in Malaysia in subsequent project(laboratory of YPM College)

A graduate of this project who works at a computer system development company

96

PakistanLocomotive Manufacturing Factory Project External Evaluator: Hajime Sonoda (IC Net Limited)Field Survey: September 2004

ObjectivesThis project was to progressively promote the domestic manufacture of locomotives by constructing a

locomotive manufacturing factory and transferring manufacturing technology to Pakistan, where railways play an important role in freight transport, and thereby contribute to economic development through stabilization ofrail transport.

Loan Amount/Disbursed Amount 9,760 million yen/9,670 million yenLoan Agreement February 1984 Terms and Conditions Interest rate 2.75%; Repayment period 30 years (grace period 10 years);

Partial untied Final Disbursement Date August 1994Executing Agency Ministry of Railways/Pakistan Railways C

hapter

2AnOverview

ofEvalu

ationResults

Evaluation Result In this project, construction of a locomotive manufacturing factory and technology transfer were conducted

almost as planned. The project period was much longer than planned due to the shortage of local currency andchanges in design. However, the project cost was lower than planned.

Under this project, production guidance was conducted, and a technological base for the factory wasestablished. Although 239 locomotives were planned to be manufactured in 12 years from the start of theoperation (1993), only 59 were manufactured, due to financial limitations of the executing agency. The share of rail against other means of transport also declined. Moreover, in line with the policy to promote domesticproduction the original plan was to attain a domestic locomotive production rate in Pakistan of 50% from thetenth year of operation and onward. However, the actual domestic production rate is as low as 33% because thestandardization of locomotive parts did not progressed as expected.

Locomotives manufactured at the factory constitute just over 10% of all diesel electric locomotives,transporting 20 million passengers a year in estimation. By manufacturing these locomotives domestically,approximately 1 billion rupees (24 million US dollars) of foreign currency was saved.

There are no problems in the technical capacity of the Pakistan Railway, and with regard to the operation andmaintenance system, reforms are progressing, includingformation of a public corporation and consignment ofoperations to private companies. Financially, the PakistanRailway is in deficit, but it is increasing income byraising fares and boosting the number of high-speedtrains. In order to increase the number of locomotivesmanufactured at this factory for the future, well-plannedallocation of the budget will be necessary.

One of the lessons learned from this project is that theimplementation method and size of the project should bedecided after the careful study of the financial capacity ofthe executing agency.

Third-Party Evaluator’s OpinionThe effect generated by this project is limited because of poor management of Pakistan Railways. However,

renewal of locomotives is relevant, and there is potential for an increase in rail transport through managerialimprovement.

97


Third-Party Evaluator: Ms. Rehana Sheikh (NGO)Obtained a master’s degree in public policy from Monash University. Presently holds the post of ProjectManager on the British Council/DFID Gender Equality Project. Specializes in public policy and rural development.

Relevance of the projectIn the transport sector in Pakistan, recovery of the share of

rail transport in long-distance, large-volume freight transport has been an important issue. However, as of 2003, the share of railways in passenger and freight transport declined to 10% and5%, respectively.

One reason is the shortage of funds to repair facilities, suchas tracks and locomotives, and that resulted in a decline in trainspeeds and the efficiency of freight transport in terms ofoperation management and waiting time. This project formspart of the progressive plan for domestic production of locomotives. However, at present, the plan itself is not beingactively promoted.

Pakistan Railways needs to conduct market studies focusedon the recovery of market share in long-distance, large-volumefreight transport and to promote construction and installation ofrailway infrastructure by double-tracking and introduction ofnew-type cars, while it steadily promotes administrative reforms such as the formation of a public corporation andallowance of partial entry of private companies into trainservice.

Changes in the Railway’s Share of Passenger andFreight Transport

Passenger Passenger FreightPassenger%50

40

30

20

10

01970 1975 1985 1990 1995 2000 2005 1980


JBIC has established a mechanism for feeding the lessons learned from ex-post evaluations of ODAloan-funded operations back into new projects and projects that are currently in progress. In specific terms, this involves reflecting the findings from evaluations of earlier, similar projects into the ex-ante evaluations andSpecial Assistance Facility (SAF) studies and linking them to improvements in implementation.

Findings from ex-post evaluations are used by JBIC and the developing countries to push for necessaryimprovements in operations, and the results of these efforts are then verified via ex-post monitoring surveysconducted seven years after the completion of a particular project. As a measure to support the efforts ofdeveloping countries to ensure the sustainability of ODA loan projects, a Special Assistance for ProjectSustainability (SAPS) study will be implemented as part of the SAF provision in the event of concerns over therealization of anticipated effects.

JBIC is currently implementing a SAPS in connection with the Yamuna Action Plan Project (India), forwhich an ex-post evaluation was conducted in FY 2004. The ex-post evaluation revealed that although thesewage disposal target has been reached, water quality in the River Yamuna has not improved. The SAPS willinclude an in-depth analysis of water quality and recommendations for improvement.

98

chapter 2: an overview of evaluation results · 2020-01-30 · chapter 2: an overview of evaluation...

Documents