IAOD Evaluation Section, the Development Agenda (DA) and Development Oriented Activities

Julia Flores Marfetan, Senior Evaluator

2

1. Introduction to the Independent Evaluation Function

2. Services provided to the Development Agenda

3. Independent evaluation of the Development Agenda and development oriented activities

4. Types of Evaluations

5. Evaluation Approach for the Development Agenda and development oriented activities

IAOD Evaluation Section

Introduction to the Independent Evaluation Function

Evaluation DefinitionEvaluation is an objective, impartial

and systematic assessment to provide useful feedback about an activity, initiative, project, program,

strategy, policy, unit, organization or sector, whether financed from

regular budget or extra budgetary resources

4IAOD Evaluation Section

Evaluation in WIPO• Where did all start?

– Recently no real evaluation activity since 2004

– Evaluation was perceived as a program management function

What has IAOD and its Evaluation Section done since 2007?– The WIPO Evaluation Policy since 2007– The Evaluation Section was set-up by the DG in 2008 within IAOD with a dedicated Evaluation Team– Focusing the evaluation function in their core business “Independent Evaluations”– Independent Review of the PPR process– Validation of the 2008 PPR– Having dedicated evaluation team within the Evaluation Section – 2 post were created– Training to improve existing monitoring and evaluation frameworks– Development of self-evaluation guidelines for program staff– Improving the previous PPR and making it more performance oriented

5IAOD Evaluation Section

• What are the Evaluation Section future priorities?

– Undertake high quality independent evaluations of development oriented activities in form of country, thematic, strategic or program evaluations

– Develop WIPO Evaluation Strategy and Guidelines

– Contribute to enhance accountability, learning and transparency by delivering relevant information to our stakeholders specially Member States

6IAOD Evaluation Section

EVALUATION SECTION SERVICES

Services provided to the Development

Agenda (DA)

May help Member States in building their capacities to undertake evaluation exercises

Including impact evaluations depending on resources provided. This is important for

The country focus of the DA and DOA.

Design, manage and undertake independent evaluations of the

Development Agenda and development oriented activities

Other Support Services

CORE SERVICES: INDEPENDENT EVALUATIONS

Support for capacity building

Ensure that monitoring and performance management of

the development oriented activitiesis of good quality

Validation

8IAOD Evaluation Section

EVALUATION SECTION SERVICES

Independent Evaluation of Development Agenda (DA) and development oriented

activities (DOA)

• Independent evaluations contribute towards a UN system identifying ways to improve delivery of development oriented activities to the beneficiaries; overcoming weaknesses and building on strengths from a strong evidence base

• Independent evaluations assist Organizations to take a deep and detailed look at internal systems and development oriented activities, and examine what practical steps need to be taken to improve them

• According to the UNEG norms and standards the evaluation function and its products should be independent, credible, and useful for learning and accountability purposes

• It is central to the credibility and reliability of evaluation• By increasing independence in evaluation, we can decrease

certain types of bias including extreme conflict of interest • Evaluation quality without independence does not assure

credibility and does not protect learning

10IAOD Evaluation Section

Whz are independent evaluations necessary?

Assuring the Independence of Evaluations

Organizational Independence

The ES is part of IAOD and it performs its function independently from other

WIPO Management functions to ensure impartial and independent reporting

Behavioural Independence

The evaluation reports are based on evidence and stakeholders are consulted

at the various stages of the evaluation process. The budget for the evaluation is

managed solely by IAOD

Protection from outside interference

IAOD Evaluation Section is be responsible for designing and

executing the evaluation and evaluatorsresults will not be subject to

overruling or influence by any external authority

Avoidance of conflict of interestIAOD will assure that the evaluators undertaking the evaluation will in no manner have an official,

professional, personal or financial relation with any WIPO program and that he/she

does not have any current or previous involvement with the development oriented activities or the

entity being evaluated at a decision making level, or in a financial management or accounting role;

or seeking employment with the Organization.

11IAOD Evaluation Section

• Are we doing the right things? And are we doing them right?

• To know if we are achieving an effect (+/-) on the intended beneficiaries

• To contribute to learning • To support accountability and transparency to stakeholders

specially Member States.• To inform the CDIP, other stakeholders and management To

inform the planning, programming, budgeting, implementation and reporting cycle

• To contribute to the improvement of the performance of on-going and future activities and to the enhancement of institutional and development effectiveness.

Why do we need to evaluate the DA and DOA? (1)

12IAOD Evaluation Section

• To enhance the generation and use of value-added evaluative information for:

•Decision-making process concerning the improvement of present and future activities

•Policy formulation and review by Member States

•Management oversight by the Director General

13IAOD Evaluation Section

Why do we need to evaluate the DA and DOA? (2)

• Establishing a program theory: how the intervention will benefit the beneficiaries and the potentials for side-effects

• Setting up a monitoring framework which includes expected results targets, baselines, SMART KPIs including systems and data base for data collection

• Allow comparison: before and after the intervention• Plan the evaluation at the start of the intervention • Define the evaluation approach, methodology and

allocated a budget and time for it

14IAOD Evaluation Section

Requirements for evaluating the DA and DOA?

What will guide the evaluation of DA and DOA?

• The WIPO Evaluation Policy approved by the DG in 2007 guide the undertaking of the independent evaluations

• The Evaluation Policy lays a framework for the evaluation function of WIPO

• The Evaluation Policy provides a comprehensive framework for planning and conducting the evaluation following international best practice, taking due account of the Norms and Standards for Evaluation in the UN System adopted by the United Nations Evaluation Group (UNEG), and other international good practices.

• The Policy is applied to all independent evaluations

15IAOD Evaluation Section

Relevance

Are the Organization’s development oriented activities suited to the

priorities and policies of the target group?

Effectiveness

Are the development oriented achieving its

objectives?

Efficiency

Is the Organization for the implementation of its development oriented

activities using the least costly resources possible in order to achieve the desired

results?

Impact

What positive and negative changes are being produce by a development oriented

activities, directly or indirectly, intended or unintended?

Sustainability

Are the benefits of the Development Agenda likely to continue after the Organization’s funding has been

withdraw?

Coverage

which groups are included or excluded from the development oriented

activities, and the differential impact on those

included and excluded?

Coherence

Are other policies and programs affecting the implementation of the

Development Agenda?

Co-ordination

How are the development oriented activities harmonizing efforts with WIPO programs,

other aid agencies and alignment with country priorities

and systems?

DA and DOA

Are we doing the right things?

Are we doing them right?

Where is the Evidence?

16IAOD Evaluation Section

What are the evaluation criteria applied to the evaluation of DA and DOA?

• Evaluation will examine the processes and contextual factors to understand why, and to what extent, expected results have been achieved, and what impact they have had.

• Evaluation will recommend and suggest better ways of achieving expected results.

• Evaluation will identify good practices and lessons learned.

What other information will the evaluation deliver?

17IAOD Evaluation Section

Types of Evaluations

Independent Evaluations vs Costs

Country Level Evaluations

Program Evaluations

ThematicEvaluations

Systemic Evaluations

Organizational Assessments

Project LevelEvaluations only Above Sfr 1 mio

0

1

2

3

4

5

6

Mil

lio

n U

SD

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Su

gg

este

d %

fo

r ev

alu

atio

n

Size of Program Suggested % for Evaluation

19IAOD Evaluation Section

Summative EvaluationsFormative Evaluations

DA and DOA evaluation timing

Development Intervention Starts/ about

To start

Development Intervention

Is in the middle Of implementation

Development Intervention is completed

Time

Stage of theDevelopment Intervention

Type ofEvaluation

Ex-ante evaluation Mid-term evaluationEx-post or final

evaluation

DA and DOA

20IAOD Evaluation Section

Evaluation approach for the Development Agenda and

Development Oriented Activities

Utilization-focused evaluation approach

1. ToRs drafting

Participatory planning withKey stakeholders and partners

2. Evaluation process

Supporting the evaluation team with aResource or learning group

3. Draft findings/ Draft report

Consultation with key stakeholdersAnd workshops

4. Evaluation report

Dissemination to a wider public

5. Follow-up day

Assisting program managers with the Implementation of recommendations

22IAOD Evaluation Section