EuropeAid

ENPI CROSS BORDER COOPERATION

SEMINAR ON MONITORING METHODOLOGY

5. Collecting and using relevant information / external monitoring

BRUXELLES, 9-10 SEPTEMBER 2010

EuropeAidProject / Programme Manager roles - issues

•Complexity of work and tasks (objectives, responsibility, accountability, performance measurement, bureaucracy) •Volume of work for Programme Managers – multiplicity of projects•How to increase time and resources allocated to supporting “management for results” and the use of monitoring systems?•How to enhance PM role as ‘informed dialogue partner’?

2

EuropeAidCollecting relevant information - issues

Formal: 3 types of Monitoring•Project progress reports (internal monitoring)•Official EC ROM reports or other external missions organized by the programme (external monitoring)•Field visits by Programme Manager (with / without ENPI CBC HQ representatives) and their own monitoring reports (Joint monitoring)

+Informal:•Project review meetings•Other sources – such as national statistics, special studies, media reports, etc.• Informal ongoing communication with stakeholders (i.e. during a conference / meeting)

3

EuropeAidOther information sources

Many other possible sources of information:•Ongoing informal contacts with project partners and other stakeholders•Other local development agencies and civil society groups •Other government agencies •Local media, internet, etc

4

EuropeAidProgramme Manager roles in project monitoring: the need for information

• Best to ‘triangulate’ sources and methods• Develop first hand knowledge of the projects

and establish effective working relationships with implementing partners

• Assess quality of internal monitoring arrangements and capacity development needs

• Assess content and quality of internal monitoring reports

• Collaborate with Monitoring teams and follow-up on recommendations

• Contribute to regular reviews of project progress

5

EuropeAidProgramme Manager roles in project monitoring

• Keep appropriate records of progress/results, issues and follow-up required

• Support timely disbursement of project funding, based on ongoing assessment of performance

• Facilitate effective information dissemination and communication

• Make timely decisions to solve problems and support implementation

• Comply with relevant instructions from ENPI CBC management

6

EuropeAidExternal Monitoring

• Typical External Monitoring are the missions organised through ROM contractors (see Module 2)• They are normally planned by the ROM external contractor

in agreement with the regional EC manager and the ENPI CBC HQ officers• In general they survey projects larger than 1 million euros

but on demand they can visit smaller projects• Outputs are the monitoring report, the BCS, containing

assessment of situation and recommendations• An alternative could be the development of external

monitoring organised by the programme management, following the main features and procedures of ROM to ensure consolidation and comparability of data

7

EuropeAidUsing external monitoring (EC ROM) - issues

Strengths• Independent assessment • Consistent methodology• Generates data that can be aggregated• Allows comparison of project quality over time, geographic zone, sector, etc• Provides often information where there is no other first hand information on the progress of the project;• Useful tool for use of findings, in particular at programme central level.

8

EuropeAidUsing external monitoring (EC ROM) - issues

Limitations• Currently only covers limited % of all projects• Rapid assessment – very limited time • Highly dependent on project documentation • Limited local participation/ownership• External Consultants often learn the most! • ROM not designed to give numerical data on performance;• Assessment of portfolio so far limited to “classic” project modality; • Limited scope for info on cross cutting issues

9

EuropeAidMaking the most of EC ROM

Programme Managers can:

•Participate in planning for EC ROM visits, and ensure local partners are actively engaged•Participate in EC ROM team meetings/site-visits•Review and access information from EC ROM reports•Contribute opinion to EC ROM reports, including feedback/response on EC ROM recommendations•Follow-up on EC ROM recommendations, as appropriate

10

EuropeAidData analysis - issues

•Data needs to be analysed to turn it into useful management information

•Some projects generate data that lends itself to some quantitative analysis – others do not

•Programme Managers can provide ideas/advice to implementing partners to improve their analysis – as well as contribute their own opinions as to what data may indicate about project performance

11

EuropeAidData analysis tips and tools

Options for quantitative analysis

•Planned vs actual

•Percentages / ratios

•Trends over time and comparison between periods

•Geographic variance

•Group variance

•Work norms and standards

12

EuropeAidInterpreting data

Indicators and resulting data/information help tell us what is happening, but not necessarily why

A project may not be achieving objectives and targets for a variety of reasons, including:o Unrealistic initial objectives/targetso Constraints outside project’s control; oro Poor project management

Reasons need to be understood if appropriate corrective action is to be taken

13

EuropeAidActivity D – analysis of BCS (In small groups)

• Read the common text of the BCS (separate handout) as basis to produce a monitoring reports• After reading the project profile provided (see handout),

prepare a question checklist you/others could use to conduct six-monthly field monitoring visits to this project • Limit to no more than 10 key questions (document on flip-

chart) Try to link the questions with the information you need • Identify key issues you want to include in your

implementation report update – including which sections of the report you would place this information in• Indicate the main sources of information/collection methods• Consider how you would help ensure the reliability/accuracy

of your findings/information (avoid bias)

14

EuropeAid

• After reading the project profile provided (see handout), prepare a question checklist you/others could use to conduct six-monthly field monitoring visits to this project

• Limit to no more than 10 key questions (document on flip-chart)

• Indicate the main sources of information/collection methods

• Briefly document/summarize on flip chart paper

• Consider how you would help ensure the reliability/accuracy of your findings/information (avoid bias)

15