m&e framework and tools and development evaluation

78
KEM LEY | Principal investigator NHIM DALEN |Consultant BORAY BORALIN | Data Analyst UMAKANT SINGH | Advisor M&E Framework and Tools and Development Evaluation Professional Training

Upload: benjy

Post on 25-Feb-2016

45 views

Category:

Documents


7 download

DESCRIPTION

Professional Training. M&E Framework and Tools and Development Evaluation. 1. Introduction to the course. To intensify the M&E skills and expertise of researchers and improve the impact on general public and development. . - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: M&E Framework and Tools and Development Evaluation

KEM LEY | Principal investigatorNHIM DALEN |ConsultantBORAY BORALIN | Data AnalystUMAKANT SINGH | Advisor

M&E Framework and Tools and Development

Evaluation

Professional Training

Page 2: M&E Framework and Tools and Development Evaluation

1. Introduction to the course

To intensify the M&E skills and expertise of researchers and improve the impact on general public and development.

Main Objective

Specific Objectives

Expected Results

Impact

1. Building the capacity and skills of researchers on M&E system and development evaluation

2. Strengthening the capacity of researchers to be able to develop M&E framework and tools

3. Strengthening the capacity of researchers to be able to conduct program and project evaluation

4. Equipping researchers with M&E skills and expertise

1. Become familiar with concepts and practices of M&E2. Be able to develop M&E framework and Tools3. Be able to conduct program/project evaluation4. Equipped with M&E Skills and expertise

1. M&E Specialist2. Professional Research Consultant

Page 3: M&E Framework and Tools and Development Evaluation

1. Introduction to the course

M&E Framework and Tools DevelopmentModule 1: M&E Rapid AssessmentModule 2: M&E framework developmentModule 3: Monitoring tools developmentModule 4: M&E Tools Pilot and ReviewModule 5: Finalized M&E Framework and ToolsModule 6: Roll-out Plan and M&E Costed Capacity Plan

Development EvaluationModule 1: Objectives of EvaluationModule 2: Focus and ScopeModule 3: Select IndicatorsModule 4: Chose Study DesignModule 5: Data collection PlanModule 6: Data Enumerators TrainModule 7: Data Collection/Field WorkModule 8: Data processing and analysisModule 9: Data organization and interpretationModule 10: Evaluation Report Writing

Page 4: M&E Framework and Tools and Development Evaluation

OUTPUTS

PROCESS

INPUTS

OUTCOMES

Efficiency

Effectiveness

IMPACTS

OBJECTIVES

Input Monitoring

Process Monitoring

Outputs Monitoring

Outcomes Monitoring and/or

EvaluationImpact Monitoring

and/orEvaluation

Monitoring and Evaluation ?

Page 5: M&E Framework and Tools and Development Evaluation

II. M&E Framework and Tools Development

Roll out Plan

M&E Tools pilot and review and finalized tools Monitoring Tools Development

M&E Framework Development

M&E Rapid Assessment

Page 6: M&E Framework and Tools and Development Evaluation

Conceptual Framework

Results Framework

Logical Framework

Interaction of various factors

Logically links inputs, processes,

outputs, and outcomes

Logically linked program objectives

M&E Frameworks

Page 7: M&E Framework and Tools and Development Evaluation

Conceptual Framework

Community action and results for health and non health

Activities/services for communities

Systems

develop & manage

that they use to deliver

Commune Committee for Women

and Children

Community & Health Actors

Outputs

Health outcomes

Other outcomes

Impacts on health and

reduction of vulnerability

of OVC

Resulting in:

which in turn contribute to

that lead to

Page 8: M&E Framework and Tools and Development Evaluation

Result Framework

% of current school attendance among

double orphans and non orphans

aged 10-14

% of double orphans who

received education assistance and

scholarship;

# of OVC and community people

involved in parental

association and education for all

committee

# of school offering breakfast

% of double orphans whose

households received economic

support

# of OVC whose HH received

economic and food support

Page 9: M&E Framework and Tools and Development Evaluation

Narrative Summary

Objectively verifiable indicators

Means of Verification

Important assumptions

Overall Goal

Project Purpose

Outputs

Activities InputsPre-Conditions

Logical Framework

Page 10: M&E Framework and Tools and Development Evaluation

Type of Framewor

k

Brief Description

Program Management

Basis for Monitoring and

EvaluationConceptual Interaction of

various factorsDetermine which

factors the program will

influence

No. Can help to explain results

Results Logically linked program

objectives

Shows the causal relationship

between program objectives

Yes – at the objective level

Logic model Logically links inputs,

processes, outputs, and outcomes,

Shows the causal relationship

between inputs and the objectives

Yes – at all stages of the program from

inputs to process to outputs to outcomes/ objectives

M&E Frameworks

Page 11: M&E Framework and Tools and Development Evaluation

M&E Framework

Strategy1:

Objectives

Activity Domain

Core Indicators

Baseline Target Data Collection Methods

Responsible

Institution

Reference Indicator

Strategy2:

Goal: Strengthen the coordination, systems, coverage and quality, of services needed to mitigate the impact of HIV on the lives and futures of Cambodian children, while also addressing the underlying issues to vulnerable children.Impact Indicators: % of Birth Registration, Proportion of Current School attendance , stunt, underweight and wasted

Page 12: M&E Framework and Tools and Development Evaluation

M&E Tools Development

Select indicator standard Reporting Format Instruction Guide Data Flow and Management M&E Data Collectors Train Piloting and updating Roll out plan Data Base System Data Use Plan

Page 13: M&E Framework and Tools and Development Evaluation

Indicator Standards

A good Indicator should meet the following six standard;

The indicator is needed and useful The indicator has technical merit The indicator is fully defined Its feasible to measure the indicator The indicator has been field tested or used

operationally. The indicator set is coherence and

balanced ( relevant to indicator sets only)

Page 14: M&E Framework and Tools and Development Evaluation

Indicator Standards

STANDARD 1: THE INDICATOR IS NEEDED AND USEFUL Question 1: Is there evidence that this indicator is

needed at the appropriate level? Question 2: Which stakeholders need and would use the

information collected by this indicator? Question 3: How would information from this indicator be

used? Question 4: What effect would this information have on

planning and decision-making? Question 5: Is this information available from other

indicators and/or other sources? Question 6: Is this indicator harmonized with other

indicators?

Page 15: M&E Framework and Tools and Development Evaluation

Indicator Standards

STANDARD 2: THE INDICATOR HAS TECHNICAL MERIT

Question 1: Does the indicator have substantive merit or technically sound and significant or measure something significant and important within particular field

Question 2: Is the indicator reliable and valid?

Question 3: Has the indicator been peer reviewed?

Page 16: M&E Framework and Tools and Development Evaluation

Indicator Standards

STANDARD 3: THE INDICATOR IS FULLY DEFINED

Title and definition Purpose and rationale Method of measurement Data collection methodology Data collection frequency Data disaggregation Guidelines to interpret ad use data Strengths and weaknesses Challenges Relevant sources of additional information

Page 17: M&E Framework and Tools and Development Evaluation

Indicator Standards

STANDARD 4: IT IS FEASIBLE TO COLLECT AND ANALYSE DATA FOR THIS INDICATOR

Question 1: How well are they systems, tools and mechanisms that are required to collect, interpret and use data for this indicator functioning?

Question 2: How would this indicator be integrated into a national M&E framework and system?

Question 3: How what extend are the financial and human resources needed to measure this indicator available?

Question 4: What evidence exists that measuring this indicator is worth the cost?

Page 18: M&E Framework and Tools and Development Evaluation

Indicator Standards

STANDARD 5: THE INDICATOR HAS BEEN FIEL-TESTED OR USED OPERATIONALLY

Question 1: To what extend has the indicator been field-tested or used operationally?

Question 2: Is this indicator part of a system to review its performance in ongoing use?

Page 19: M&E Framework and Tools and Development Evaluation

Indicator Standards

STANDARD 6: THE INDICATOR SET IS COHERENCE AND BALANCED (Relevant to indicator sets only)

Question 1: Does the indicator set give and overall picture of the adequacy or otherwise of the response being measured?

Question 2: Does the indicator set have an appropriate balance of indicators across elements of the response?

Question 3: Does the indicator set over different M&E levels appropriately?

Question 4: Does the set contain an appropriate number of indicators?

Page 20: M&E Framework and Tools and Development Evaluation

Consistency or dependability of data and evaluation judgments, with reference to quality of the instruments, procedures and analysis used to collect and interpret evaluation data

Indication defines clearly what we should be measured. It defines the variables that help measure change within a given situation as well as describe the progress and impact.

The extent to which something is reliable and actually measures up to or make a correct claim. The process of cross-checking to ensure that the data obtained from one monitoring method are confirmed by the data obtained from a different method

INDICATOR PROTOCOLS

INDICATOR PROTOCOLSREQUIRES

• Definition• Measurement• Strengths• Limitations • Reliability• Precision• Validity• Objective• Owned• Accessible• Useful

Indicator Protocols

Page 21: M&E Framework and Tools and Development Evaluation

M&E Framework & Tools

M&E FRAMEWORK & TOOLS

DEVELOPMENT

M&E Rapid Assessment

M&E Framework

Development

Monitoring Tools

Development

M&E Tools Pilot and Review

Roll-out Plan and M&E Costed

Capacity Plan

Finalize M&E Framework and Tools

Page 22: M&E Framework and Tools and Development Evaluation

Instruction Guide What is instruction guide?

Instruction guide is a reference tool formulated tends to provide clear explanation on how to accurately complete the reporting format.

How to develop instruction guide? Identify purpose of the instruction guide State purpose of the reporting form Data sources Who prepare the report Frequency of reporting Reporting period Name of agency completing the report District Province Indicators

Page 23: M&E Framework and Tools and Development Evaluation

Instruction GuideIndicators: For example: Total number of OVC whose households received economic support (income

generation activities, livelihood support, regular cash transfer)

Write the total number of OVC whose households received economic support during the reporting period.

Definition: Economic support (IGAs and livelihood) has been defined as: Home gardening Animal husbandry Provision of agricultural seeds Small business development Money management training Emergency cash support Regular cash transfers Access to loan/microfinance Other

 Disaggregation: This data is disaggregated by gender. Write the total number of male OVC in the “Male”

column and the total number of female OVC in the “Female” column. Then write the total number of OVC (male + female) in the “Total” column.

Page 24: M&E Framework and Tools and Development Evaluation

Data Flow When mapping the flow of data, please

consider the following issues: Who will be responsible for data collection? Who will provide the data? Who will be responsible for supervision of data

collection? Who will be responsible for compiling and

aggregating data? How often are data collected, compiled, reported,

and analyzed? How are data sent from one level to the next? How is feedback on reported data provided?

Page 25: M&E Framework and Tools and Development Evaluation

Data FlowMinistry of Social Affairs, Veterans and Youth

Rehabilitation (MoSVY)(Child Welfare Department)

Youth Rehabilitation / Drug

Rehabilitation

Alternative Care

Centers

Provincial Department of Social Affairs, Veterans and Youth Rehabilitation (PoSVY)

DoSVY

Commune Council (via CDB)

Quarterly

Quarterly

Quarterly

Quarterly PoSVY Report on OVC

Provincial Department of Planning

Ministry of Planning

CCWC

POVCTF

Service Providers (NGOs)

Data flowFeedbackSupportive Supervision

NOVCTF

Village Council (via CBD)

Annual

Annual

Annual

Annual

Law Enforcemen

t (police, prison,

courts )

PHD

MoH

Page 26: M&E Framework and Tools and Development Evaluation

Identify R&R of Key Players

When developing role and responsibility of all key players involve in data collection, some important point that you should consider: What type indicator they need to collect and report? How many indicator they need to collect and report? How they collect those data (source of data –

registration book)? Which reporting form they use? How frequency that they should report – when? Who they should report to?

Page 27: M&E Framework and Tools and Development Evaluation

Source of data error Transposition—An example is when 39 is entered as 93.

Transposition errors are usually caused by typing mistakes. Copying errors—One example is when 1 is entered as 7;

another is when the number 0 is entered as the letter O. Coding errors—Putting in the wrong code. For example,

an interview subject circled 1 = Yes, but the coder copied 2 (which = No) during coding.

Routing errors—Routing errors result when a person filling out a form places the number in the wrong part or wrong order.

Consistency errors—Consistency errors occur when two or more responses on the same questionnaire are contradictory. For example, if the birth date and age are inconsistent.

Range errors—Range errors occur when a number lies outside the range of probable or possible values.

Page 28: M&E Framework and Tools and Development Evaluation

What to do when mistakes First, determine the source of the

error. If the error arises from a data coding or

entry error If the entry is unclear, missing, or

otherwise suspicious Once the source of the error is

identified, the data should be corrected if appropriate.

Page 29: M&E Framework and Tools and Development Evaluation

Points to consider when providing feedback Feedback should be constructive and not punitive Feedback should be useful to data collectors and

help them improve their work Errors should be pointed out and corrected The M&E supervisor should talk to the data

collector to find out the cause of the error so it can be prevented in the future

The M&E supervisor should discuss how data quality and reports can be improved in the future

Page 30: M&E Framework and Tools and Development Evaluation

Points to note when providing supportive feedback Provide both positive and negative feedback (e.g.

you do X very well but can improve Y) Provide feedback in a timely manner Help data collectors understand the problem so

they know how to correct it in the future Be helpful and collaborative

Page 31: M&E Framework and Tools and Development Evaluation

Why is it important to provide supportive feedback Builds relationship between data collectors and users at all

levels Important element of management and supervision Leads to greater appreciation of data Improves data quality Improves information use Improves service delivery and benefits the target

population and the community Improve program reporting- data collectors understand

trends in data and understand reasons behind numbers Incentivizes and motivates data collectors

Page 32: M&E Framework and Tools and Development Evaluation

Pilot M&E Tools Set criteria for selecting pilot

province Provide training on M&E reporting

tools to all data collectors Provide on the job training to all data

collectors

Page 33: M&E Framework and Tools and Development Evaluation

Pilot M&E tools review Objective:

Aim to take an in-dept look at the quality of the data that was collected during the pilot period and to assess the systemic factors that affect M&E performance and to gather direct input on the M&E tools and system.

Page 34: M&E Framework and Tools and Development Evaluation

Pilot M&E tools review Step in conducting the review:

Develop assessment tools▪ Data transmission, accuracy, processing and analysis

▪ Data transmission▪ Data accuracy▪ Data processing and analysis

▪ Data use▪ Some qualitative questions added

Provide training to assessment team Conduct assessment Conduct consultation meeting on the findings

Page 35: M&E Framework and Tools and Development Evaluation

Finalize Framework and Tools Key point affecting the finalization of

M&E framework and mechanics

Indicators▪ Does these indicators are feasible to collect?▪ Does these indicators are feasible to analyze and use?▪ Is there any evidence that financial and human

resources are available to allow an indicator to be measured and that the benefits of measuring the indicator are worth the costs?

A good indicator needs to be one that is feasible to measure with reasonable levels of resources and capacity.

Page 36: M&E Framework and Tools and Development Evaluation

Finalize Framework and ToolsThe situation may change meaning that an indicator needs to be changed, discarded or added.

M&E system mechanics Does the data collection tools are applicable? Does the reporting formats are applicable? Does the instruction guide (guideline) is user friendly?

Data management process How well functioning of the data flow of the system? Does existing human resource have an appropriate capacity to

manage the data flow? How clear the roles and responsibility of department or person

involved in M&E system? Does the frequency of data collection and reporting are appropriate

at each level?

Page 37: M&E Framework and Tools and Development Evaluation

Finalize Framework and Tools Revise M&E framework, with revised

indicator, M&E mechanics, and data management process

Conduct consultative meeting among M&E team and relevant stakeholders to finalize M&E framework and system

Get approval from top level of management (decision makers, policy makers).

Page 38: M&E Framework and Tools and Development Evaluation

Purpose of M&E/Data use

ShareData withPartners

Reporting/Accountability

ProgramImprovement

Page 39: M&E Framework and Tools and Development Evaluation

Data Analysis-, Interpretation and reportData Cleaning, entry, Processing,

Sampling Technique

Sample Size Calculation Objectives , Scope and Steps for Evaluation & Research

Development Evaluation

Page 40: M&E Framework and Tools and Development Evaluation

Reasons for Evaluation/Research

Royal Governme

nt of Cambodia

Development Partners and Civil Society

Threatened

Communities

• Unfair Compensation and worsen living condition• Loss of job• High Service cost for relocated site• There is no available legal, social and health services

Positive Impact ofdevelopment • Beautification• Development• Employment • GDP Growth• Economic Growth• Survive people

from Slum

Negative impact of development• Human Rights Violation• Inadequate housing rights• Unfair Compensation• Unfair development• Inequality of profits

distribution

Page 41: M&E Framework and Tools and Development Evaluation

Reasons for Evaluation/Research

Page 42: M&E Framework and Tools and Development Evaluation

Reasons for Evaluation/Research

Page 43: M&E Framework and Tools and Development Evaluation

ObjectivesFocus & Scope

Select IndicatorsChose Study design

Data Collection Plan

Data collection/Field WorkData Cleaning & Verification

Data Processing & AggregationData Analysing & Organization

Data Interpretation & Report

12

34

5

78

910

Data Enumerators Train

Data Use and Data Translation11

12

Steps for Evaluation and Research

6

Page 44: M&E Framework and Tools and Development Evaluation

Objectives

The overall objective of the program evaluation of HRTF is to assess the social economic impact of Cambodia Forced eviction in urban areas of Phnom Penh Municipality. The specific objectives of the program

evaluation is to know the status of economic, education, health, employment, food security and environment of threatened and relocated communities.

Page 45: M&E Framework and Tools and Development Evaluation

Scope and Focus Socio Econ omic Imp act

Relocated Househ old s

Economic St atu s

Ed ucat ion S tat us

Health St atu s

Emp loymen t

En viron men t

Th reatened Household

E con omic st at us

Ed ucat ion S tat us

Health

E mp loymen t

En viron men t

Poverty and

Quality of live

among relocated Househol

ds and threatene

d Househol

ds

Page 46: M&E Framework and Tools and Development Evaluation

Selected IndicatorsSelected Indicators Relocated

Households

Threatened Households

1. Percentage of Children drop out of school2. Percentage of households whose income below

poverty line 3. Percentage of households consumption4. Percentage of households with debt 5. Percentage of household access to registered

MFI6. Percentage of households with food shortage 7. Percentage of house members whose access to

health services in the past three months 8. Percentage of households have experienced

physical attack 9. Percentage of household have experienced

stigma and discrimination 10. Percentage of respondents have lost job due to

forced eviction

Page 47: M&E Framework and Tools and Development Evaluation

Study Design Qualitative and quantitative study design

(Cross Sectional Study) Household Survey (Cluster Sampling-Lot division) Key Informant Interview(KII)-Relevant Stakeholders Focus Group Discussion (FGD)-RS and TS HH Desk Study and Literature Review

▪ Cambodia Legal Frameworks▪ National and International Research Findings▪ NSDP and JMI 2009-2013, MoP▪ Pro-Poor Policy and National Safety Net Strategy, CoM▪ HRTF Baseline Survey 2010▪ HRTF Program and strategy documents▪ HRTF Strategic Plan 2011-2015 ▪ CCHR Survey on land and housing Issues 2011▪ Draft of National Housing Policy 2011▪ Country Report _Special reporters 2009, 2010, 2011▪ Others

Page 48: M&E Framework and Tools and Development Evaluation

Sample size Calculation

SDV Z Z2 p q e e2 n

99% 2.586.656

4 0.5 0.5 0.01 0.0001 16641

98% 2.335.428

9 0.5 0.5 0.02 0.0004 3393

95% 1.963.841

6 0.5 0.5 0.05 0.0025 384

90% 1.642.689

6 0.5 0.5 0.10 0.01 67

85% 1.442.073

6 0.5 0.5 0.15 0.0225 23

80% 1.281.638

4 0.5 0.5 0.20 0.04 10

Sample size (n) for Precision (e) of:Size of Population +/- 3% +/- 5% +/- 7% +/- 10%

500 a 222 145 83600 a 240 152 86700 a 255 158 88800 a 267 163 89900 a 277 166 90

1,000 a 286 169 912,000 714 333 185 953,000 811 353 191 974,000 870 364 194 985,000 909 370 196 98

6,000 938 375 197 987,000 959 378 198 998,000 976 381 199 999,000 989 383 200 99

10,000 1,000 385 200 99

15,000 1,034 390 201 9920,000 1,053 392 204 10025,000 1,064 394 204 10050,000 1,087 397 204 100

100,000 1,099 398 204 100

Over 100,000 1,111 400 204 100

Page 49: M&E Framework and Tools and Development Evaluation

Confidence and Precision

Confidence Level: The standard confidence level is 95%. This means you want to be 95% certain that your sample results are an accurate estimate of the population as a whole.

Precision: This is sometimes called sampling error or margin of error. We often see this when results from polls are reported.

Confidence Interval: We can say that we are 95% certain (this is the confidence level) that the true population's average salary is between 1,800 and 2,200 (this is the confidence interval).

Page 50: M&E Framework and Tools and Development Evaluation

Sample size Calculation

1 2

3 4

Population size

Sample size

Population Size

Sample Size

10 10 550 22620 19 600 23440 36 700 24850 44 800 26075 63 900 269

100 80 1,000 278150 108 1,200 291200 132 1,300 297250 152 1,500 306300 169 3,000 341350 184 6,000 361400 196 9,000 368450 207 50,000 381500 217 100,000

+385

N n= ----------

1+(N(e)2

2

2

eqpz

n

SDV Z Z2 p q e e2 n

99% 2.586.656

4 0.5 0.5 0.01 0.0001 16641

98% 2.335.428

9 0.5 0.5 0.02 0.0004 3393

95% 1.963.841

6 0.5 0.5 0.05 0.0025 384

90% 1.642.689

6 0.5 0.5 0.10 0.01 67

85% 1.442.073

6 0.5 0.5 0.15 0.0225 23

80% 1.281.638

4 0.5 0.5 0.20 0.04 10

Page 51: M&E Framework and Tools and Development Evaluation

Sample size Calculation

Population size

Sample size Population Size

Sample Size

10 10 550 22620 19 600 23440 36 700 24850 44 800 26075 63 900 269

100 80 1,000 278150 108 1,200 291200 132 1,300 297250 152 1,500 306300 169 3,000 341350 184 6,000 361400 196 9,000 368450 207 50,000 381500 217 100,000+ 385

Page 52: M&E Framework and Tools and Development Evaluation

Sample size Calculation N

n= ----------

1+(N(e)2

n: Sample SizeN: Population Studye: Level of precision

Yamane (1960) formula assumes a degree of variability (i.e. proportion) of 0.5 and a confidence level of 95%.

SDV Z Z2 p q e e2 n

99% 2.586.656

4 0.5 0.5 0.01 0.0001 16641

98% 2.335.428

9 0.5 0.5 0.02 0.0004 3393

95% 1.963.841

6 0.5 0.5 0.05 0.0025 384

90% 1.642.689

6 0.5 0.5 0.10 0.01 67

85% 1.442.073

6 0.5 0.5 0.15 0.0225 23

80% 1.281.638

4 0.5 0.5 0.20 0.04 10

Page 53: M&E Framework and Tools and Development Evaluation

Sample size Calculation

2

2

eqpz

n SDV Z Z2 p q e e2 n

99% 2.586.656

4 0.5 0.5 0.01 0.0001 16641

98% 2.335.428

9 0.5 0.5 0.02 0.0004 3393

95% 1.963.841

6 0.5 0.5 0.05 0.0025 384

90% 1.642.689

6 0.5 0.5 0.10 0.01 67

85% 1.442.073

6 0.5 0.5 0.15 0.0225 23

80% 1.281.638

4 0.5 0.5 0.20 0.04 10n= sample sizep = the approximate proportion you expect to find in the populationq = 1-pe = the level of precision you can tolerate (plus or minus 10%, etc.)z = the z-value from a table for the level of confidence you want

Page 54: M&E Framework and Tools and Development Evaluation

LQAS

LOT5= 19

LOT1= 19

LOT2= 19

LOT5= 19

LOT3= 19

LOT4= 191.

• Can be used locally• Can provide an accurate measure of

coverage ( benchmark)• Can be used for quality assurance• is a simple, low cost random

sampling methodology• Small sample• Meet the quality standards• Statistically determined sample size

LQAS = Lot Quality Assurance Sampling• Developed in the 1920’s• In 1980’s, method was adapted to measure health program coverage:

• Immunization• Malaria• Neonatal tetanus elimination• Leprosy elimination• Family planning,• HIV/AIDS prevention

• In Cambodia World Vision , CONCERN , ADRA, and other

Page 55: M&E Framework and Tools and Development Evaluation

Sample size for LQAS

where n= sample sizep = the approximate proportion you expect to find in the

populationq = 1-pe = the level of precision you can tolerate (plus or minus 10%,

etc.)z = the z-value from a table for the level of confidence you want

n = (1.96)2 (0.5 x 0.5) / (0.1) 2

n = (3.84) (0.25)/(0.01)

n = 96

2

2

eqpz

n

Page 56: M&E Framework and Tools and Development Evaluation

Sampling TechniquesNon Random Sampling

Purposive

Convenient

Snowball

Quota

Accidental

Cluster sampling is a multi-step way or we may want to take a stratified sample of farmers at various distances from a major city

you do not have a complete list of everyone in the population of interest

combinations of methods are used

we want to select 100 files from a population of 500?

Page 57: M&E Framework and Tools and Development Evaluation

Systematic Random Sampling

Name of Village Population

Cumulative population

Sampling Interval

Random number

Sample Size

A 510 510

B 750

C 910

D 570

E 800

F 750

G 600

H 450

K 530

L 900

Total 6770 385

Page 58: M&E Framework and Tools and Development Evaluation

LQASCommune 1: Pres Klang (Control Area)Name of ADP                      Number of Samples Seleced

Name of village Population Cumulative population

Sampling Interval

Random Number (0-5 month) (15-45yrs)

Mor Seth 914 914 169 105

5 5        274        443        612        781Okleng Por 769 1683  950

5 5        1119        1288        1457        1626Sromouve 643 2326  1795

4 4        1964        2133        2302Krang Doung 357 2683  2471 2 2        2640Anlong Svay 631 3214  2809

3 3        2978        3147

Total 19 19

Page 59: M&E Framework and Tools and Development Evaluation

Methods Source Advantage Disadvantage

1. Desk Study & Literature Review2. Population Base Survey3. Qualitative Data Collection3.1. Key Informant Interview-KII3.2. Focus group discussion-FGD3.3. Case Study 3.4. Best Practice3.5. Observation3.6. Self-administered questionnaires3.7. Exit Interview4. Routine Program Monitoring

Data collection Methods

Page 60: M&E Framework and Tools and Development Evaluation

Data Analyzing

Coding and Entry

• Analyzing

Editing

Checking

• DATA ORGANIZATION• DATA INTERPRETATION• REPORTING• DATA USE

Page 61: M&E Framework and Tools and Development Evaluation

Table

ChartGraphsDATA

DescriptionOpinion or

View

Data Interpretation & Organization

Page 62: M&E Framework and Tools and Development Evaluation

      Male Female TotalAge   n 131 91 222  5-9   13.0% 9.9% 11.7%  10-14   72.5% 69.2% 71.2%  15-18   14.5% 20.9% 17.1%        Current school attendant   76.7% 67.4% 72.9%Level of education n 133 102 223  Never attend school   29.5% 23.1% 26.9%  Primary school   65.2% 67.0% 65.9%  Secondary school   4.5% 9.9% 6.7%  High school   0.8% 0.0% 0.4%Type of education attended n 129 89 210  Formal education   43.4% 48.3% 45.4%  Non-formal education   11.6% 10.1% 11.0%  Both formal and non-formal education   39.5% 40.4% 39.9%Current living status n 132 93 225  Residential care   18.9% 17.2% 18.2%  Non-residential care   81.1% 82.8% 81.8%Status of children n 133 93 226  Orphan   26.9% 32.3% 29.1%  Street children   58.5% 55.9% 57.4%

 Children in conflict with the law   6.2% 2.2% 4.5%

 Chronically ill parent/caregiver during month of the last 12 months   23.1% 22.6% 22.9%

  Abused and exploited children   1.5% 2.2% 1.8%  Children addicted to drugs   0.8% 0.0% 0.4%  Children with physical disabilities   0.0% 1.1% 0.4%  Children infected by HIV   0.0% 0.0% 0.0%  Children living with poor HH   44.6% 38.7% 42.2%

Title does not say: what, when,

where

A mistake of using row and

column

No source of data

Footnote is needed

No total row

and column

Interpretation

Data Interpretation & Organization

Page 63: M&E Framework and Tools and Development Evaluation

Status of orphans and vulnerable in Kamreing and Battambang Province, Cambodia, 2010

Ref. Definition, MoSVY 2010, An orphan is a child who has lost one or both parents.A maternal orphan is a child whose mother has died. A paternal orphan is a child whose father has died. A double orphan is a child who has lost both parents.

Note: Types and Definition of OVC, MoSVY 2010

Male Female TotalType of orphan n 35 30 65  Maternal orphan   17% 17% 17%  Paternal orphan   46% 60% 52%  Double orphan   37% 23% 31%  Total   100% 100% 100%

Male Female TotalOverlap risk of children n 133 93 226  Once   45% 48% 47%  Double   50% 51% 50%  Triple   5% 1% 3%   Total   100% 100% 100%

Data Interpretation & Organization

Page 64: M&E Framework and Tools and Development Evaluation

Orphan Non-Orphan Orphan Non-OrphanMPK 2010 CDHS 2005

87.5%77.4% 76.0%

92.0%

Percentage of children aged 10-14 who currently attending school

Title does not say: what, when, where

Reference

Axis

Footnote is needed

Ordonez

Interpretation

Data Interpretation & Organization

Page 65: M&E Framework and Tools and Development Evaluation

Orphan Non-Orphan Orphan Non-OrphanMPK 2010 CDHS 2005

87.5%77.4% 76.0%

92.0%

Percentage of children aged 10-14 who currently attending school

Type of study

% o

f res

pond

ent

Comparison of school attendant among orphan and non-orphan aged 10-14 between MPK 2010 and CDHS 2005

MPK: Meatho Phum KohmaCDHS: Cambodia Demographic and Health Survey

Ref. End of project evaluation of MPK in 2010 in Battambang Province with two district (Battambang and Kamrieng).CDHS 2005, the nationwide study.

Data Interpretation & Organization

Page 66: M&E Framework and Tools and Development Evaluation

Poverty/work67%

To by my own16%

Mother/father coming here

11%Orphan

2%

DV, abuse and exploitation1% Other

3%

Main reason of being away from home

Data Interpretation & Organization

Page 67: M&E Framework and Tools and Development Evaluation

Education

Health care

Economic

Food and nutrition

Psychological

Other support54%

65%

22%37%

47%

16%

70%

70%

10%

50%70%

60%

Essential Service for OVC given by MPK compared to NPA Review 2008

MPK 2010 NPA Review 2008

Data Interpretation & Organization

Page 68: M&E Framework and Tools and Development Evaluation

KEM LEY | Principal investigatorNHIM DALEN |ConsultantBORAY BORALIN | Data AnalystUMAKANT SINGH | Advisor

Employment Rate

Poverty Line

Income per

capital

• 25% or 1/3 are under poverty line ( RKR, PVH and ST >40% (MoP 2010) • 12% food insecurity to 20% or 2,8 millions (CDRI 2008) • School drop out rate from 13% to 22% • Underweight:28%• Stunt : 40%• Wasted : 11% Source: CAS 2008 and CDHS 2010

• 23% or 3.5 m of young population• 72 of 100 people aged 15-24 are job seekers• 30,000 to 30,000 have entered job market but 67,000 new job created or 27% • Reason: Skill mismatch Source: ILO and CAMFEBA• Income per capita 285 in

1997 to 593US$ in 2007• More than 80% are farmers and 91% are living in rural areas and account for 48% of total poor • Benefits have not been equitably distributed• Gaps between rich and poor (the difference in share of consumption between the richest 20% of Cambodians and the poorest 25 & reveals a dramatic and widening gap in wealth)

Data InterpretationFragility of Cambodia Development

Page 69: M&E Framework and Tools and Development Evaluation

Employment Rate

Employment Rate

Agriculture

Sector

Industrial

Sectors

Service

Sector

• Income: 70% from self employment income, 27% from wage and salary, 2% from transfer received and 1% from other, • Labor Forces or working age (15-64) is 84% or 7.5 millions• Child under 18 is 41% and child labor (5-14) is 45%• Expenditure: 49%, food, 19% (House, water, electricity, 10% health and others

CSES 2009, MoP

30-40,000 seek job but absorption is 67,000 Job/27% or

Page 70: M&E Framework and Tools and Development Evaluation

Employment RateCambodia Population

13,395, 682 or young population Working Age Population (15-64 ) 84% or 7.5 millionsAdult working Population (64%)

including Old Age Working Population

Old Age Working Population(…)

23% of Working population

Children (5-17): 4.3 Millions or 35% 1.5 millions (5-14) were working

children or 45% Sources: ILO, 2009 CCA 2009 CDB, MoP 2009 CSES 2009 Good Governance and Social Accountability, TAF 2011, 8 Provinces

Average INCOME: 120 US$ per month per familyAverage Expenditure: 150 US$ per month per family

Page 71: M&E Framework and Tools and Development Evaluation

Increased Employment RateYoung Population

Increased employment Young PopulationAgriculture SectorSelf-EmploymentAccess to creditFarming System

market integrationReformed School Curriculum

Vocational Trainings

Industrial SectorService Sector

Domestic Workers

Retired and Old Age Population ManagementEarly Retired Population

Reduced Child LaborD&D –Practicing DecentralizationEmployment Young Population with CC, Village Councils and other lines offices of Ministries

USA, Singapore, ThailandEmployment and minimum wage policy and policy enforcement

108 NGO study: 40, 0000 or equal to all factory workers

NGO Sector-CARE, Plan Int..

Good Governance

Page 72: M&E Framework and Tools and Development Evaluation

Employed Population (15-64)

Page 73: M&E Framework and Tools and Development Evaluation

Employed Population (15-64)

Total employment workers (15-64) is 7.5 millions but Paid employee : 22.8% Self employed: 51.7% Unpaid family workers: 25.1% Employer: 0.3%

Page 74: M&E Framework and Tools and Development Evaluation

Items for Expenditure

Food Education

Motorcycle49%

Cell-Phone44%

Social Services

Housing/Water and Electricity

HealthLegal Services

TV 60%.

Page 75: M&E Framework and Tools and Development Evaluation

Selected items of durable goods owned by households

• Radio : 42.5%• TV :59.6%• Video tape/recorders/Players : 28.7%• Stereo : 13.5%• Cell phone : 43.8%• Satellite Dish :1%• Bicycle : 67.7%• Motorcycle :49%• Car : 3.8%• Jep/Van :1%• PC : 3.4%CSES 2009, MoP, RGC

Page 76: M&E Framework and Tools and Development Evaluation

The average monthly Household income per household and per capita

Household, Capita-US$

Cambodia 94 21 Phnom Penh 307 65Urban 134 54Rural 79 188% are negative income among formers_____________________________________________

_• Self employment income :70%• Wage and Salary :27%• Transfer received : 2%• Other : 1%CSES 2009, MoP, RGC

Page 77: M&E Framework and Tools and Development Evaluation

Description Cambodia UrbanRR

• Food 30$ or 49% 38$ 45.% 27$ or 52%

• Housing/Water/Ele 12$ or 19% 19$ or 23% 8$ or 15%

• Health 5$ or 7% 5$ or 5% 5$ or 9%

• Education 2$ or 2% 2$ or 3% 1$ or 1%

• Other 23% 24% 24%

Total 62$ 85$ 51$ CSES 2009, MoP, RGC

Consumption CompositionAverage monthly value in US per

capita

Page 78: M&E Framework and Tools and Development Evaluation

Conclusion

Average Monthly Income

(Rural Area) 18 US$

ExpenditureAverage monthly

expenditure per capitaIn Rural

Area51 US$

Average Monthly

Saving per capita

-33 US$

Landless Migrants Child LaborSchool Drop Out Sex Workers Fragile Population Others

Poverty

Social Insecuri

ty