1

Government of Lao People’s Democratic Republic

Executing Entity/Implementing Partner:

Ministry of Agriculture and Forestry, MAF

Vientiane, Lao PDR

Implementing Entity/Responsible Partner:

National Agriculture and Forestry Research Institute, NAFRI

United Nations Development Programme

M+E Manual for IRAS

Project ID:00076176 / ATLAS Award ID 60492

Improving the Resilience of the Agriculture Sector in Lao PDR to Climate

Change Impacts (IRAS Lao Project)

Prepared by Virgilio E. Cabezon, M&E Specialist

Project Contact : Mr Khamphone Mounlamai, Project Manager

Email Address : kphonemou@yahoo.com

October 2011

ສາທາລະນະລດັ ປະຊາທິປະໄຕ ປະຊາຊນົລາວ

Lao People's Democratic Republic

ອງົການສະຫະປະຊາຊາດເພ ື່ ອການພດັທະນາ

United Nations Development Programme

2

ACRONYMS

ALF Agriculture Livestock and Fisheries ALM Adaptation Learning Mechanism ANR Agriculture and Natural Resources Sectors APR Annual Progress Report CC Climate Change CCTA Climate Change Technology Adaptation CCTAM Climate Change Training Adaptation Module CPAP Country Strategy and Action Plan DAFO District Agriculture Forestry Office DDMC District Disaster Management Committee DFD Data flow diagram GEF Global Environment Facility GIS Geographic Information System GOL Government of Lao IRAS Improving the Resilience of the Agriculture Sector in Lao PDR to Climate Change Impacts M&E Monitoring and Evaluation MAF Ministry of Agriculture and Forestry NAFES National Agriculture Forestry Extension Office NAFRI National Agriculture Forestry and Research Institute OVIs Objectively Verifiable Indicators PAFO Provincial Agriculture Forestry Office PIR Project Implementation report PMTS Project Tracking and Monitoring System PRA Participatory Rapid Appraisal PSU Project Support Unit PTR Project Terminal Report QPR Quarterly Progress Report UNDP United Nations Development Programme

3

TABLE OF CONTENTS

I Background 5 1.1 Strategic Context of IRAS, the Project 5 1.2 Project Objective, Components, Location and Funding 5 II Monitoring and Evaluation System: An Introduction 8 2.1 Monitoring and Evaluation 8 2.2 Key Elements of Results-Based Monitoring 9 2.3 UNDP and GEF Focus on Tracking Results 10 III M&E System for IRAS 7 3.1 Objective of IRAS M&E System 7 3.2 IRAS M&E System 11 3.3 Steps in Designing M&E System 12 3.4 Project Tracking and Monitoring System 21 IV Baseline Data for IRAS Project 24 4.1 Objective and Use of Baseline Survey 24 4.2 Baseline Survey Procedures 24 V Learning and Knowledge Management 25 References 27 Annexes (In a separate volume/folder) Annex I-1 Project Organizational Structure Annex II-1 Revised Logical Framework Matrix: IRAS Annex III-1 Annual Work Plan Template Annex III-2 Sample Annual Work Plan Annex III-3 Quarterly Progress Report Template Annex III-4 Risk Log Template Annex III-5 Guidelines for Accomplishing Risk Log Annex III-6 Issue Log Template Annex III-7 Guidelines for Accomplishing Issue Log Annex III-8 Sample Issue Log Annex III-9 Annual Report Template Annex III-10 GEF Template Annex III-11 Field Visit Report Template Annex III-12 Project Final Report Template

4

Annex III-13 Initial Monitoring Communication Plan Annex III-14 Project Tracking and Monitoring System Annex III-15 M&E Information Matrix: IRAS Annex IV-1 Baseline Survey Questionnaire for Small Farmers Annex IV-2 Baseline Survey Questionnaire for GOL Ministries and Institutions Annex IV-3 Steps in Conducting Baseline Survey for Small Farmers Annex IV-4 Dummy Tables for Small Farmers Baseline Survey Annex IV-5 Dummy Tables for GOL Ministries/Institution Baseline Survey Annex IV-6 Instructions for Encoding the Answers of the Questionnaire: Baseline Survey for Small

Farmers Annex IV-7 Instruction for Encoding the Answers of the Questionnaire: Baseline Survey for GOL

Ministries and Institutions Annex IV-8 Worksheet for Baseline Survey for Small Farmers Annex IV-9 Worksheet for Baseline Survey for GOL Ministries and Institutions Annex V-1 Thematic Reporting Annex V-2 Monitoring Plan for DAFO and PAFO Annex VI-1 Power point Template for Informative Reporting

5

I BACKGROUND

1.1 Strategic Context of IRAS, the Project

The IRAS sits well within the context of the Lao PDR. Being a land locked country it is vulnerable to flooding and drought. This is evident by the extended amount of rainfall expended during the wet season resulting in the flooding of lowly elevated Provinces located in the southern and northern parts of the country. In the dry season, the observed temperature has become hotter resulting in prolonged dry season thus inhibiting the planting of crops and rearing of livestock and fisheries, the consequence of which is shortfalls in agricultural production and land productivity. Most Lao small farmers have fragile lands, and without protection from the extreme effects of flood and droughts these lands are will become unproductive. The challenge therefore for IRAS is to introduce and teach these small farmers with the use of climate-responsive farming innovations to reverse the further degradation of these fragile lands and consequently enhance land productivity.

1.2 Project Objective, Components, Location and Funding

1.2.1 Objective

The main objective of the project is to minimize food insecurity resulting from climate change impacts and to reduce the vulnerability of farmers to extreme flooding and drought events.1 The overall approach is to introduce adaptive techniques to small farmers that will allow them to diversify their farming operation and increase livelihood opportunities at the community. This will be accomplished by improving the enabling policy environment of the ANR sectors specifically dealing on food security coupled with the strengthening of the institutional capacities of GOL personnel to respond to CC phenomena and other natural and man-made disasters.

1.2.2 Components

Four (4) intertwined components comprise IRAS:

Component 1 (Outcome 1) on knowledge base information involves the underpinning of the information base for CC risks and vulnerability as a basis for better understanding of the impacts of CC to agriculture in the context of food security. Component 1 provides the improved data base for policy, planning and mainstreaming of CC adaptation schemes in the implementation of ANR sector projects. Three (3) outputs are envisaged under this component:

Output 1.1 – Reliable CC and vulnerability information generated

This output covers, among others, essential CC information, notably on drought and flood scenarios collected from regional and local institutions and analyzed further to permit their usefulness for macro

1 The project responds to the achievement of outcome 2 of UNDP Laos CPAP 2007-201, notably enhanced

ownership and capacity for pro-poor planning, implementation and harmonized aid coordination, and disaster management. The main output is increased capacity within the Government to prepare and respond to natural as well as man-made disasters at all levels.

6

and micro planning (2 Provincial pilot sites) as well. Foremost among this information are long-term series data on rainfall, flooding and droughts that occurred in Lao PDR. Vulnerability hazard maps including land-use maps are updated with the aid of GIS. The intent of the latter is to determine exactly where the hot spots are located and should there be interventions for food production, these areas can be avoided.

Output 1.2 – Long-term information system on floods and drought established at NAFRI

Output 1.2 essentially centralizes the data base for CC information at NAFRI. The intent is to have a one- stop shop of data base (time series data, digitized maps, CC studies, policies, etc) at NAFRI in a format that is easy to access and use by everybody, especially development GOL practitioners, including the academe.

Output 1.3 – Food security and land-use policies and other strategic framework reviewed as basis for improved CC adaptation guidelines

Output 1.3 is a compendium of all relevant policies and strategic frameworks related to CC adaptation in the context of food security. These policies are inventoried, reviewed and analyzed and should serve as good materials for formulating CC policy framework and CC adaptation guidelines for Provincial Government.

Component 2 (Outcome 2) on capacity building essentially focuses on improving the skills and competence of GOL personnel in integrating CC in the planning and formulation of policies, programmes and projects designed to eradicate widespread poverty in the rural areas. The key in component 2 is to sharpen the competence of GOL personnel in the setting up of regulatory frameworks, budgeting and planning and adaptation schemes consistent with sustainable and equitable use of natural resources, notably agricultural lands and the remaining forest and fishery resources. A single output is envisaged under this component:

Output 2.1 – Trained policy makers and agricultural officers and DDMC staff on CC risks and community-based CC adaptation

Output 2.1 is centrally on training the GOL staffs involved in policy and planning, extension officers and other staff at the district level. To make this output useful and effective, two (2) elements are built in the training design: (a) designed training curricula on policy analysis and evaluation, land-use planning with GIS application, CCTAMs, and disaster risk reduction measures; and (b) training approach that combines basic understanding and hands-on application, beginning from higher level officers down to the field level officers under a continuing cascading system.

Component 3 (Outcome 3) is on agricultural extension of CC resilient farming innovations combined with value-chain to increase the opportunities of the small farmers for income diversification. This component is related to component 2. The agricultural extension officers who have completed their skills’ enhancement on CCTAMs courses will implement the pathways to achieving an effective system of agricultural extension delivery to small farmers. Three (3) outputs are envisioned under this component:

7

Output 3.1 – ALF climate resilient farming technologies/options made available and adopted

Output 3.1 are ALF resilient technologies that have been validated and tested through on-farm trials. Indigenous technologies that are being applied by small farmers to mitigate the impacts of CC were integrated with improved technologies to improve production and yields of major staple food crops.

Output 3.2 - Farmers’ exposure to on farm, value chain innovations and off-farm experiences established and maintained effectively

Output 3.2 is the main pathway for adopting climate-responsive food production systems. Through this output, on-farm and off-farm trials are established and these will serve as the venue for training the small farmers to adopt these climate-responsive technologies. Besides essential farm implements and inputs provided, farmers’ field schools are also organized to increase the effectiveness of the demonstration centers. Successful farmers’ field school will be used as venue for cross visits of farmers and cross posting of farmer leaders who could serve as extension leaders to their co-farmers.

Output 3.3 - Small-scale irrigation facilities, protection measures and watershed

maintenance made available

Output 3.3 completes the process of agricultural extension delivery by providing small capital investments in the form of small-scale irrigation facilities and other infrastructures. Provision for small-scale irrigation facilities through rehabilitation of existing traditional weirs or use of water lift pumps could significantly improve cropping intensity. So as not to waste these investments, water users group are organized, and these groups could collect affordable water charges from members to use the fees for maintenance of the irrigation system.

Component 4 (Component 4) is the medium for sharing and exchanging lessons learned and success stories about the project interventions. Access to this medium through a user-friendly web portal established at NAFRI can increase the awareness of practitioners and small farmers on how to mitigate the impacts of CC change, including the adoption of climate-responsive food production systems. Two (2) outputs are envisaged under this component:

Output 4.1 - Knowledge information shared among local and global practitioners

Output 4.1 provides the off-line form of knowledge sharing, among others, are: (a) organizing international conference/workshops; (b) disseminating advocacy materials on CC adaptation; and (c) organizing media dialogues and forums at the national and provincial levels.

Output 4.2 - Knowledge networking initiative established at NAFRI Output 4.2 is a virtual market place for sharing and exchanging experiences on climate-responsive technologies for the benefit of all, especially the small farmers. It is a one-stop web portal at NAFRI to promote climate resilient farming innovations. Users to this portal can download or post their success stories and lessons learned with others across the Internet on mitigating the impacts of climate change to increase productivity.

8

1.2.3 Executing Agency and Funding The main executing agency is the MAF through NAFRI, the main agricultural and forestry research arm of the Ministry2. Funding for the project is mainly provided by GEF and UNDP, with counterpart funds from GOL. Total UNDP-GEF fund for the project is roughly $7.1 million to be disbursed over a period of 4 years, beginning in May 2011 until April 2015. Including the GOL counterpart fund of $5.2 million, total project cost is estimated at $12.3 million. 1.2.4 Location The project sites are located in the Provinces of Savannaketh and Xayaboury. Two (2) districts per Province were selected as the sites for project intervention.

II MONITORING AND EVALUATION SYSTEM: AN INTRODUCTION 2.1 Monitoring and Evaluation Monitoring and evaluation are “two distinct sets of organizational activities.” They are related but not identical.”3 A clear description where monitoring and evaluation play complementary roles is given below.

Monitoring Evaluation

Routine collection of information

Tracking project implementation progress

Measuring efficiency

Question? Is the project doing things right?

Analyzing information

Ex-post assessment of effectiveness and impact

Confirming project expectations

Measuring impacts

Question: Is the project doing the right thing?

Source: Alex and Byeriee, Monitoring and Evaluation, World Bank, 2000.

The complementary roles described above permit the postulation of the following:

Learning is a continuous process. Learning components generate knowledge about the good practices as follows:

The validity of assumed logic through which it is expected that inputs and activities would produce outputs, which will result into outcomes/impacts and ultimately increase the income of the beneficiaries, besides improved environmental and social safeguards.

What types of interventions are successful under what conditions?

2 The project implementing structure is given in the Draft Binder M&E Platform.

3 World Alliance for Citizens Participation (CIVICUS), by definition monitoring is the systematic collection and analysis of

information as the project progresses, while evaluation is the comparison of actual project impact against the agreed targets or strategic plan. Evaluation is focused with achievements rather than delivery of input. The discussion in this section draws heavily from the experiences of the National Agricultural Innovation Project (NAIP), World Bank 2000.

9

What are the improvements in project interventions for maximizing the outcomes? Given the above discussion, the M&E system is demystified below. It has three distinct, but complementary components

1. Performance monitoring process on the supply side, i.e. focus on activity wise input and output that should be supported by functional MIS

2. Evaluation process on the demand side response that requires baseline data, MTR, EOP surveys with a focus on outcomes. Has to be done by an independent institution

3. Learning and knowledge management to replicate best practices and/or highlight failures Source: National Agricultural Innovation Project, World Bank, 2000

2.2 Key Elements of Results-Based Monitoring

Results-based monitoring as opposed to the traditional mode of monitoring, i.e. activity-based has sharpened the tracking of project results owing to introduction of key elements as follows:

Key elements of results-based monitoring

Baseline data to describe the situation before the project intervention

Indicators for outcomes

Data collection on outputs and how and whether they contribute towards achievement of outcomes

Timeliness expressed such as at mid-term and end-term

More focus on perceptions of change among stakeholders

Systematic reporting with more qualitative and quantitative information on the progress towards outcomes

Done in conjunction with strategic partners

Capture information on success or failure of partnership strategy in achieving desired outcomes Source: National Agricultural Innovation Project, World Bank 2000

It is important to note that results are not a description of action but rather a “development change” attributable to the project. The achievement of change per se is not a result, i.e. if the causal link to the result is not established. Thus, for a better appreciation of the context of results-based monitoring, it is imperative to recall the terminologies being used in the construction of results framework as given below.

Results Framework Terminology Defined

Term Definition Relevance to Causal Chain Example Inputs Financial, human resources

and other materials mobilized to support activities of the project

Inputs are converted into activities by the implementing agency

UNDP and GEF parallel financing, staffing

Activities Action taken or work performed by which inputs are converted into specific outputs

Activities are meant to deliver outputs

Training, capacity building, construction activities

Outputs Project deliverables (on the supply side) expected to add value or potential users to

Outputs are designed to stimulate outcomes

Access to agricultural demonstration centers; capacity of implementing

10

stimulate development outcomes (demand side)

agency strengthened

Outcomes Expected or actual demand- side behavioural responses by the end-users and other stakeholders that demonstrate uptake, adoption and use of project outputs-thus validating the causal chain

Outcomes are measured before the end of project implementation in participating areas

Small farmers adopt intensive cropping systems or climate-responsive cropping systems

Impacts Positive or negative, primary and secondary long-term effects produced by a development intervention

Impacts are expected to begin after the end of implementation as an extension of the project’s causal chain (normally measured 5 years after end of project implementation)

Farmers income increased

Source: National Agricultural Innovation Project, World Bank, 2000

2.3 UNDP and GEF Focus on Tracking Results

UNDP and GEF, being the major funding institutions emphasize great attention in tracking results. In terms of tracking results, the distinction between these two (2) donors is that, GEF looks at the results only at the outcome level, including project development objective, while UNDP looks at the entire results framework4. The distinction is given below.

Results Framework (Logframe matrix) GEF UNDP

Outcome: Intermediate effects of outputs on clients

1

Outputs : products or services delivered by the project Activities: task personnel assigned to convert inputs to transform inputs to outputs Inputs: financial, human resources, and materials

Yes

No

No

No

Yes

Yes

Yes

Yes

1 Covers both project objective and components

4 Goal was excluded in the IRAS results framework. The project implementation period is relatively short to permit

measurement of long-term impacts. The indicator classified as impact in the IRAS logframe matrix given in Annex II-1 is considered as “emerging impact.”

11

III M&E SYSTEM FOR IRAS

3.1 Objective of IRAS M&E System The main objective of the M&E system is to facilitate the project management staff to:

Make sure that operational activities are planned and implemented on time and compatible with the project objective;

Establish learning environment and identify likely shortfall in the expected performance and share success endeavours;

Develop appropriate remedial actions;

Encourage the key stakeholders to initiate remedial actions, if needed; and

Establish the baseline data on outcomes as basis for mid-term and final evaluation of the project impact.

The above M&E duties are expected to happen at all three levels of the project management staff of IRAS as described below. Everyone has the responsibility to deliver the outcomes and outputs as designed and thus these duties are bound to occur.

Strategic planning Level – This refers to the top management, the Project Board, which manages the external environment of the project and oversees the implementation and attainment of project results;

Project planning level – This is the PSU and the Project Manager, with NAFRI as the lead institution. Their tasks is to make sure of the proper implementation of the activities under the 4 components;

Operation planning level - For IRAS, components 1, 2, and 4 are to be implemented by NAFRI, while component 3 is to be executed by NAFES. For both components, the DAFOs and PAFOs of two Provincial Governments (Savanaketh and Xayaboury) are designated to co-implement specific activities related to actual implementing level, among others, such as supervision and/or establishment of agricultural demonstration centers, identification of indigenous climate-responsive technologies, and supervision in the construction of small-scale infrastructures.

It should be noted that differences in terms of complexity, time element, number of people involved, area of operation and project components occur insofar as performing the above M&E tasks. For instance, in terms of complexity, it is extremely high at strategic planning level and low for operation planning level. In terms of time span for decision making, there is less time span as you move from operation to strategic planning level. The Project Board may review the project 4 times a year. The operation planning level may have to review progress of work weekly. The number of persons involved is much less at strategic planning level compared to operations level. Activities under each component are less compared to the project as a whole.

12

3.2 IRAS M&E System Results-based M&E system will be adopted for IRAS to monitor the processes employed, following UNDP and GEF norms and standards:

A correctly determined results framework obtained from clear definition of project objective, outcomes, outputs, and activities with measurable indicators, means of verification and key assumptions;

A well defined M&E strategy as regards information needs, tools and methodologies for data collection, analysis and reporting;

A comprehensive M&E plan with clear roles and responsibilities as regards data gathering and reporting; and

Continuing internal and external assessments 3.3 Steps in Designing M&E System 3.3.1 IRAS Results Framework

The results framework (logical framework matrix) for IRAS has been defined following UNDP’s standard guidelines.5 The outcomes were formulated for the project objective and the 4 components. OVIs were likewise described for each outcome and outputs of the 4 components. The results framework is given below.

Project objective and component outcome indicators – IRAS1

Description Outcome Indicators (OVIs) Use of Outcome Information Project objective Food insecurity resulting from climate change (CC) in Lao PDR minimized and vulnerability of farmers to extreme flooding and drought events reduced.

Food production of major crops, livestock and fisheries of targeted farmers/households in four (4) pilot districts increased by at least 25%

Perceived reduction in hunger among targeted beneficiaries in four (4) pilot districts measured by the number of months without rice reduced by xxx month

Per cent of targeted households below food threshold reduced by 5% (supplementary indicator)

Determine success of project and make recommendations for future intervention. For example, if the increase in food production (e.g. yield or rice) is less than 25% by the end of the project, components 2 and 3 and should be reviewed for efficiency.

Component 1 Knowledge-based on CC adaptation strengthened

At least a single CC adaptation policy framework drafted and adapted by 2014

Introduce adjustments to policy reform agenda

Review decision-making

5 IRAS results framework was reviewed in mid-August 2011 and for reason of consistency with UNDP’s SMART

guidelines, the results framework for IRAS was evolved. Decision to adopt the revised results framework, however, will have to get the approval of the Project Board.

13

CC adaptation compliance by two (2) Provinces (pilot sites) improved through the passage of resolutions and ordinances

structures and policy implementation

Component 2 CC adaptation capacities of GOL policy makers and agricultural officers strengthened

At least 30% of policy makers and agricultural officers at national level are using CC scenario planning to develop long-term adaptation strategies

At least 15% of agricultural extension officers are able to implement CCTA courses

At least 15% of DDMC staff are practicing disaster risk reduction measures in 4 pilot districts

Address capacity building issues and provide guidance on policy, agricultural extension methodologies and disaster risk reduction. For example, if the rate of using the acquired skills is less than the targets by the specified period, the activities must be adjusted as needed.

Component 3 Community-based CC adaptive farming systems’ technologies and off-farm opportunities increased

At least 30% of the targeted farmers have diversified their farming practices or used low sensitivity climate food production methods

Cropping intensity of land with improved small-scale irrigation facilities expanded by xxx%

Determine progress and address issues on small farmers’ ability to intensify their production through climate-responsive technologies and provide guidance on market linkage and value-adding innovations

Determine if interaction with small farmers is effective

Component 4

CC adaptation, monitoring, and learning (ALM) as a long term process widely leveraged

At least 2 GOL projects (proposed or pipeline) incorporating the IRAS best practices, methods and approaches are evident by 2014

Determine if knowledge sharing information is effective

Note: The targets are indicative, pending the determination of reliable baseline data. It is important that both the baseline data and targets should be carefully reviewed by the project management staff. In the spirit of “ownership” these values should be owned by those whose

performance would be monitored. 1 GEF as earlier mentioned would monitor only the outcome indicators described in this results framework.

Correspondingly, the arrangement for the results monitoring of the outputs under each component (outcome) which completes the results framework is detailed in the succeeding table given below. The reminder about targets and baseline data given at the end of the table is important to recognize specially by those whose performance would be monitored. We are referring to the GOL, notably the MAF and NAFRI whose ownership of the project is vested on these two institutions. Every effort must be ensured to achieve the targets, and if they are behind accomplishments by the end of the specified period, the activities should be reviewed as often as required. The project management should keep the results in view at all times and steer the project accordingly. Flexibility is the key in steering the project,

14

because the results are influenced by various stakeholders. This is the essence of results-based monitoring. Results Monitoring for Outcomes (component) and Outputs -IRAS

Component (Outcome)

OVIs Baseline

Targets Frequency and Reports

Data Collection Instrumen

ts

Responsibility for

data collection

Outcome 1 Knowledge-based on CC adaptation strengthened

Y1

Y2 Y3 Y4 Y5

Outcome 1

At least a single CC adaptation policy framework drafted and adopted by 2014

CC adaptation compliance by two (2) Provinces (pilot sites) improved through the passage of 2 resolutions and/or ordinances

0

0

1

2

Annual/Final report

-do-

APR/PTR GEF PIR

-do-

Sr. M&E consultant, PSU,

NAFRI

Provincial

M&E, PAFO

Reliable CC and vulnerability information generated

Output 1.1

At least one (1) CC scenario with GIS-assisted vulnerability hazard map analyzed by mid 2012 for application in ANR projects

At least four

0

0

0

1

4

1

Quarterly/An

nual report

-do-

-do-

QPR/APR

QPR/APR

QPR/APR

Sr. M&E consultant, PSU,

NAFRI

Provincial

M&E, DAFO

Provin

15

(4) district land-use plans overlay with CC risks prepared by mid 2012

At least one (1) codified/improved land-use guideline incorporating CC risks prepared by mid 2012

cial M&E, PAFO DAFO

Long-term information system on floods and drought established at NAFRI

Output 1.2

A functional data base established at NAFRI by end 2012

0

1

Quarterly/An

nual report

QPR/APR

Sr. M&E, consultant PSU,

NAFRI

Food security and land-use policies and other strategic framework reviewed as basis for improved CC adaptation guidelines

Output 1.3

xxx number of food security and land-use policies including other strategic framework relevant to CC adaptation were inventoried and reviewed

0

10

Quarterly/An

nual report

QPR/APR

Sr. M&E consultant, PSU,

NAFRI

16

Outcome 2 CC adaptation capacities of GOL policy makers and agricultural officers strengthened

Outcome 2

At least 30% of policy makers and agricultural officers at national level are using CC scenario planning to develop long-term adaptation strategies

At least 15% of agricultural extension officers at provincial/district level are able to implement CCTA courses

At least 15% of DDMC staff in 4 pilot districts are practicing disaster risk reduction measures

1

1

1

5%

5%

5%

10%

10

%

10

%

15%

15%

15%

Annual/Final

report

-do-

-do-

Perceptio

n survey/PRA APR/PTR

GEF PIR

-do-

-do-

Provincial

M&E, Sr.

M&E, PSU,

NAFRI

Provincial

M&E, Sr.

M&E, PSU, NAFE

S

Provincial

M&E, Sr.

M&E, PSU

Trained policy makers, agricultural officers, and DDMC staff on CC risks and

Output 2.1

At least three (3) training curricula developed by mid 2012 : (a) land-use planning; (b) food security policy analysis and evaluation; and (c) disaster risk reduction measures/ground practices

At least 6 training modules (CCTAMs) on community-based adaptation developed by mid

0

0

0

3

6

75

25

Quarterly/An

nual report

-do-

-do

QPR/APR

-do-

-do

Sr. M&E, PSU,

NAFRI

Sr. M&E, PSU, NAFE

S

Sr. M&E, Provin

cial M&E, DAFO, PAFO,NAFRI

, NAFE

S

17

2012

All identified policy makers, agricultural officers and DDMC staff covering xxx personnel completed training by mid 2013

Outcome 3 Community-based CC adaptive farming systems’ technologies and off-farm opportunities increased

At least 30% of the targeted farmers have diversified their farming practices or used low sensitivity climate food production methods

Cropping intensity of land with improved small-scale irrigation facilities expanded by xxx%

1

1

5%

10%

5%

15%

15%

Annual/Final

report

-do-

Perceptio

n survey/PRA APR/PTR

GEF PIR

PRA, APR/PTR

Provincial

M&E, DAFO, PAFO, NAFE

S

-do-

ALF climate resilient farming technologies/options made available and adopted

Output 3.1

At least 5 ALF climate resilient farming technologies/options made available and adopted by early 2013

0

5

Quarterly/Annual

report

Perception

survey/PRA

QPR/APR

Provincial

M&E, PAFO,DAFO NAFE

S

18

Farmers’ exposure to on farm, value chain innovations and off-farm experiences established and maintained effectively

Output 3.2

At least 20 demonstration centres on ALF farming systems equipped with essential farm implements have been established by mid 2013 and maintained by the targeted beneficiaries

At least two (2) value chain innovations (one per province developed and demonstrated

At least 400 farmers (100 farmers/pilot district) especially farmer leaders have

0

0

0

200

10

2

200

10

Quarterly/Annual

report

-do-

-do-

QPR/APR

QPR/APR

QPR/APR

Provincial

M&E, PAFO DAFO, NAFE

S

-do-

-do-

19

completed training by end 2013

Small-scale irrigation facilities, protection measures and watershed maintenance made available

Output 3.3

Xxx hectares of expanded irrigated land

Number of water users group organized and trained to manage effectively the irrigation facilities provided

1

1

5

2

15

2

Quarterly/Annual

report

Quarterly/Annual

QPR/APR

QPR/APR/PRA

Provincial

M&E, PAFO DAFO, NAFRI -do-

Outcome 4 CC adaptation, monitoring, and learning (ALM) as a long term process widely leveraged

At least 2 GOL projects (proposed or pipeline) incorporating the IRAS best practices, methods and approaches are evident by 2014

0

2

Annual/Final Report

APR/PTR GEF PIR

Sr. M&E, PSU,

NAFRI Knowledge

information shared among local and global practitioners

Output 4.1

At least one (1) international conference/workshop on the project knowledge organized by mid 2014

At least 2 advocacy materials on indigenous knowledge on CC adaptation, including lessons learned/best practices documented and published

0

0

0

1

2

3

Quarterly/An

nual

-do-

-do-

QPR/APR

-do-

-do-

Sr. M&E, PSU,

NAFRI

-do-

-do-

20

by mid 2014

At least 3 dialogues and media forums organized by 2014

Knowledge networking initiative established at NAFRI

Output 4.2

One stop web portal established at NAFRI

0

1

Quarterly/An

nual

QPR/APR

Sr. M&E, PSU,

NAFRI

1 To be determined by baseline survey

Note: The targets are indicative. The figures given in the yearly implementation period, e.g. % adoption/use, reflect a learning process and by the end of the target period, the cumulative accomplishment should be summed to get the overall performance. Thus it is important that the ownership of the project by GOL and project management staff should set sensible values for targets and baseline because their performance would be monitored according to these values.

3.3.2 Project Monitoring and Tracking System (PMTS)

The foundation for the project’s PMTS is the preceding results framework. The components of the PMTS are: (1) Monitoring templates to capture, among others, outcomes/outputs, input use, risks, lessons learned

and process variables (work plans). UNDP and GEF have systematic templates for capturing these variables with brief pro-forma to incorporate narrative description and analysis of information collected. In general, the frequency of monitoring the variable is also indicated in the template. The templates are given in Annexes III-1 to III-126. The use of a particular template in relation to monitoring the given variable is given below.

Stage of Activity Monitoring variable Template Report

Annual work plan Output, Activity Expenditure Annual work plan Annual Work Plan Report

6 Familiarization and use of the templates is mandatory part of the hands-on M&E training. Filling up the

UNDP/GEF monitoring templates is straightforward. Detailed instructions are given in the template. The second and most important is how to analyze and synthesize the data collected and translate them into the templates (forms for narrative and analytical descriptions) that can be useful for decision-making.

21

& Schedule Quarterly progress Outcome, Output, Activity,

Expenditure Risks Issues

Quarterly progress

ATLAS Risk log ATLAS Issue log

Quarterly Progress Report –cum annexes: Draft combined delivery report Annual work plan for the following quarter Project monitoring and communication plan for the following quarter Project risk log for the quarter Project issue log for the quarter

Annual progress Outcome, Output, Activity,

Expenditure Risks Issues Lessons learned

Annual progress GEF PIR ATLAS Risk log ATLAS Issue log ATLAS lessons Learned log

GEF Project Implementation Report Annual Progress Report-cum annexes Draft delivery combined report Annual Work plan for the following year Project monitoring and communication plan for the following year Project risk log for the following year Project Issue log for the following year Project lessons learned log for the following year GEF Project Implementation Report

Field visit Progress towards results, production of outputs, partnerships, key challenges

Field visit Field visit report-cum annexes: List of persons met Other annexes

Final progress Final outcome, output, activities completed, total expenditure, Lessons learned

Final progress Final Progress report – cum annexes Draft combined delivery report Project lessons learned log

(2) Collection of baseline data. As a rule, only the outcome variables are covered by the baseline data. Thus, baseline survey questionnaires are brief. The questions formulated are meant to capture the values of the variables identified in the outcomes.

(3) Communication with key stakeholders on any shortfalls and identified remedial actions and other important matters. A template on the initial communication plan is given in Annex III-13.

22

3.3.3 Roles and Responsibilities of the M&E Unit The project monitoring and evaluation unit is lodged at PSU. Concurrent monitoring is the responsibility of the senior M&E specialist who is likewise responsible for the day-to-day operations of the M&E. He/she will be supported by 2 M&E specialists, one per province, and detailed permanently at the concerned PAFO.7 Second, an independent entity will be tasked to carry out an outcome-focused evaluation at two stages: mid-term review and project completion.

Mid-term review/evaluation – The timing for the mid-term review is normally at the end of the second year of implementation. The mid-term evaluation will determine progress being made towards the achievement of outcomes and will identify courses of correction if needed. It will focus on the effectiveness, efficiency and timeliness of project implementation; will highlight issues requiring decisions and actions; and will present initial lessons learned about project design, implementation and management. Findings of this review will be incorporated as recommendations for enhanced implementation during the final half of the project’s term.

Final evaluation – Three months prior to the end of implementation, the final evaluation is normally recommended for commencement. The final evaluation will also look at impact and sustainability of results, including the contribution to capacity development and the achievement of global environmental goals. The final evaluation should also provide recommendations for follow-up activities.

The monitoring effort will lend support to project management in its day-to-day operations developing a PMTS, development of information network, and provide support to IRAS overall communication dissemination.8 Specific responsibilities for the Sr. M&E specialist include, among others:

Supervise the conduct the baseline survey for IRAS in coordination with two PAFOs;

Coordinate with two PAFOs for reporting processes;

Consolidate reports and contribute to the overall IRAS reporting requirements;

Provide guidance in monitoring and evaluation activities, specially to the two M&E specialists detailed at the PAFO;

Contribute to the development of IRAS management information system; and

Ensure gender-based oriented monitoring at national and provincial levels. 3.3.4 M&E Information Matrix

The M&E information matrix presented in Annex III-15 zeroed in on the indicators explaining, among others, the level of information needed and sources of information; whose responsibility, methods and frequency of data gathering. The sequencing and numbering of the indicators appearing in the first column of the matrix corresponds to the numbering of the project’s results framework (logframe

7 Only the Sr. M&E was hired during the preparation of the M&E Manual for IRAS, although this was already at the

latter stages of preparation. 8 Further details of the PTMS, including data flow diagram (DFD) is given in Annex III-14.

23

matrix). The second column explains the kind of data to be collected, i.e. the definition of the indicators, and the third column specifies the sources of information. The identified sources of information were explored at all levels. The fourth column indicates the means of data gathering, while the fifth column specifies the focal units/persons responsible for data collection and analysis. The last column enumerates the frequency of reporting. The important points to remember are as follows:

As you move to the lower level of objective hierarchy (e.g. activities), the information are straightforward and they are much easier to collect and conversely going up to the higher levels of hierarchy (e.g. goal/objectives), the types and sources of information become wider and thus the synthesis of the same are greater;

Getting and validating the data were identified through triangulation and PRAs. Triangulation allows getting information about the same topic in various ways and PRAs permit quick validation of the different sources of data; and

Least cost consideration was factored in as regards the use of data gathering and frequency of reporting. This was taken in consideration without necessarily sacrificing the quality of information. For instance, field observations and FGDs as means of data gathering compared to the more complex and expensive surveys were recommended.

The M&E information matrix is part of the M&E system. It provides a snapshot of what information will be collected and monitored for decision-making.

IV. BASELINE DATA FOR IRAS PROJECT

4.1 Objective and Use of Baseline Survey Monitoring for IRAS requires a periodic assessment of a situation over a period of roughly 4 years. The presence of a starting point for comparison facilitates the project management to assess what has changed over the specified period of time. Thus the need for information on the situation before project intervention has taken place is essential. This information is called the baseline data, the reference of base conditions where comparisons are made later. The baseline is the first critical assessment of performance indicators and is used as the origin by which project performance is monitored. For IRAS, the baseline data will only be collected for outcome variables. Comparing the baseline vis-à-vis achieved values requires a process, and this can be achieved in the following methods:

24

By comparing the situation before the project started (before situation) with situation after the project has completed its implementation (after situation). This involves understanding of the factors that affected the outcome;

By tracking changes inside the project area and comparing it with changes outside the project area (with and without project). This requires looking for a comparable area;

By comparing the difference between similar groups (inside the project area) and between a project group and a group outside the project influence (control group). This involves looking for a comparable group within the same area.

Any of the above methods can be used to assess the performance of the project. The key to the selection of a suitable method is cost consideration, plus availability and reliability of data to be used in the assessment of performance. The judgement is the discretion of the evaluator. 4.2 Baseline Survey Procedures 4.2.1 Baseline Questionnaire Two types of baseline questionnaires were drafted for IRAS: (i) baseline survey questionnaire for small farmers; and baseline survey questionnaire for GOL ministries and institutions. The draft questionnaires are given in Annexes IV-1 to IV-2. The baseline questionnaire for small farmers was meant to capture the outcome variables primarily for project objective and component 3, while the latter questionnaire was planned to capture the outcome variables mainly for component 2. Components 2 and 4 were partly covered in the baseline survey for GOL ministries and institutions. However, there are outcome indicators for these two components whose baseline values are zero and thus need not be established by baseline survey. The questions that would link to the determination of baseline values for the outcome variables are properly marked on both types of questionnaire. This system will permit easy reference to estimation of baseline values. 4.2.2 Procedures The detailed procedures for conducting the face-to-face interview for small farmers are given in Annex IV-3. The procedures outline, among others, the determination of sampling respondents, conduct of the face-to-face interview and processing the answers. The procedure for the GOL ministries and institution is straightforward. The respondents will be asked to fill up the two-page questionnaire. This will, however, require brief explanation from the project manager on what the project IRAS is all about to give a proper perspective for the respective heads of departments/agencies concerned to appreciate the purpose of the survey. For this purpose, it is advisable to have a prior meeting with all ministries/agencies to be covered by the baseline survey, before the designated interviewer can proceed to issue the questionnaire.

25

4.2.3 Processing the Answers to the Questionnaires Dummy tables were prepared to assist the national baseline consultants in processing the answers to the questionnaires. These are given in Annexes IV-4 to IV-5. Specific instructions given in Annexes IV-6 to IV-7 were likewise drafted on how to input the data in Excel worksheets (Annexes IV-8 to IV-9) to establish the data base.

V. LEARNING AND KNOWLEDGE MANAGEMENT

5.1 Opportunity to Critical Thinking Analyzing data to create new insights and understanding experiences in the project is all about critical thinking.9 M&E information is meaningless without critical thinking. To be useful, M&E information should be discussed during team meetings, shared with primary stakeholders and acted upon with dispatch if there are decisions to be made, especially if these are concerns relative to managing the project impact. Success stories and best practices are the most important information to share with the different stakeholders.10 Various opportunities will be explored to share the experiences of IRAS. These include: learning sharing workshops and/or conferences; linking with media to disseminate documentaries on project beneficiaries; and the establishment of a one-stop shop web portal dedicated on knowledge emporium. The details of these options are explained in the Draft Knowledge Management Plan. 5.2 Options for learning and Sharing The options that have been identified in the preceding section for learning and sharing are by no means new. IRAS need not design new protocols in order to make these options workable. IRAS can look at the activities of UNDP-GEF with respect to learning and sharing and can pattern from these tested options. On the establishment of a user-friendly web portal for knowledge emporium, there are good models within the region that can be replicated.11

9 A Draft Knowledge Management Plan for IRAS is available. For the time being, the draft plan is comprehensive. It

can be revisited as soon as value and impact stories become available. 10

Annex V-1 presents a suggested format for thematic reporting. 11

IRAS can study and pattern its web portal from the knowledge emporium web portal of the National Economic and Development Authority (NEDA), the premier government economic planning body in the Philippines.

26

REFERENCES 1. C.C.G.S Computing, 2011. Data Flow Diagrams 2. Monitoring and Evaluation Manual, 2007. National Agriculture Innovation Project 3. Monitoring and Evaluation Systems Manual, M&E Principles, 2005. Mekong Wetlands

Biodiversity Programme. 4. The Global Fund, 2008. Developing and M&E Plan 5. Serrat, Olivier. 2009. Improving Sector and Thematic Reporting 6. United Nations Development Programme, 2009. Handbook on Planning, Monitoring and

Evaluating for Development Results. 7. United States Agency for International Development, 2011. M&E Manual Data Flow