youth entrepreneurship program (yep) monitoring...

1

Youth Entrepreneurship Program (YEP)

Monitoring, Evaluation and Learning (MEL) Framework

2

Contents

1. Introduction ......................................................................................... 3

1.1 Introduction to the Youth Entrepreneurship Program (YEP) ............................ 3

1.2 Problem and Solution Trees .................................................................. 4

Youth Entrepreneurship Program (YEP) Problem Tree ....................................... 4

Youth Entrepreneurship Program (YEP) Solution Tree ....................................... 5

1.3 Program Results Framework ................................................................. 6

2. Learning Agenda .................................................................................. 12

2.1 A strategic approach to learning ........................................................... 12

2.2 YEP’s Learning Vision ...................................................................... 12

2.3 YEP’s Learning Principles ................................................................... 12

Learning under the YEP: ........................................................................ 12

2.4 Key Learning Themes and Questions ...................................................... 13

2.5 Alignment with YBI’s 2014-17 Research and Learning Agenda ........................ 16

2.6 Alignment with the MIF’s learning agenda on youth entrepreneurship .............. 17

3. The Program MEL Cycle .......................................................................... 18

4. YEP Data Collection Plan ........................................................................ 19

5. MEL Products and Processes .................................................................... 23

6. MEL Tools .......................................................................................... 26

7. Key Stakeholders and plan for Communication of Findings ............................... 28

8. MEL capacity building for country project holders under the YEP ........................ 31

Bibliography: ............................................................................................ 34

Annex one: Mapping of YEP Learning Agenda against YBI 2014-2017 Research & Learning Agenda and MIF Learning Agenda on Youth Entrepreneurship ................................... 36

Annex two: MEL Capacity Diagnostic Tool Draft .................................................. 44

Annex three: YEP Learning Log Template .......................................................... 48

3

1. Introduction

1.1 Introduction to the Youth Entrepreneurship Program (YEP)

This document outlines the program’s MEL framework, which comprises the following five elements:

1.#Systema+c#MONITORING#against#program#and#project#logframes##

3.#Execu+on#and#publishing#of#RESEARCH#STUDIES#exploring#key#learning#themes#and#ques+ons###

2.#EVALUATION#of#(i)#YEP#overall,#and#(ii)#country#level#projects#under#Component#1,#and#(ii)#ASSESSMENT#of#capacity#building#

interven+ons#under#Component#2##

4.#APPLYING#knowledge#to#planning,#implementa+on#

and#decisionVmaking##

5.#Collabora+ng#and#SHARING#knowledge#across#the#region,#YBI#network#and#sector#to#influence#effec+ve#

policy#and#prac+ce##

This program is designed to increase the number of poor and low-‐income youth (aged 18-‐30) in Latin America and the Caribbean (LAC) successfully starting sustainable businesses and creating new jobs. Despite significant investments in this area, there remain systemic challenges for these young people in starting and growing businesses, such as lack of start-‐up financing and other essential non-‐financial support, including entrepreneurship training and mentoring. There is also a comparative lack of evidence about what works in terms of youth entrepreneurship support in LAC and a scarcity of tested and scalable models. Moreover, networking between key stakeholders engaged in youth entrepreneurship sector is weak This program aims to redress this by combining the experience, resources and network of the Inter-‐American Development Bank’s (IADB) Multilateral Investment Fund (MIF) in LAC, with those of the Youth Business International (YBI) network, a leading pioneer in entrepreneurship support for under-‐served youth, with initiatives spanning 38 countries. Through targeted country-‐level projects selected and implemented under a strategic learning agenda and rooted in cross-‐sector collaboration, the program will result in laying the foundations for a youth entrepreneurship ecosystem in selected countries across the Region and will generate significant knowledge and innovations for improved services and systems that may also be transferred to other regions. These activities will be facilitated and sustained through the establishment of a Regional Center on Youth Entrepreneurship, a knowledge and support Center, based on a network of relevant institutions. The Center will remain as a permanent legacy for the region, and a reference in LAC experience and good practice for other regions.

4

It should be noted that this framework is very much intended as a ‘living document’ and it will be subjected to consultation among all key regional stakeholders, and revised accordingly. Please see Section 7 of this document for the consultation plan. The MEL framework will subsequently be reviewed on an annual basis, and updated as needed.

1.2 Problem and Solution Trees1

Youth Entrepreneurship Program (YEP) Problem Tree

1 This explanation of the methodology draws heavily on http://evaluationtoolbox.net.au/index.php?option=com_content&view=article&id=28&Itemid=134

CAUSES&

EFFECTS&

1.#Inequality#is#stagnant# 3.#Young#people#are#socially#disenfranchised##

7.#Policy#is#not#rooted#in#evidence#6.#Under>investment#in#underserved#young#entrepreneurs##

4.#Young#people#are#insecure#about#future#prospects#and#lack#confidence#

2.#Unemployment#is#growing#

4.#The#knowledge#base#is#weak#re.#proven#and#scalable#entrepreneurship#support#models#for#under>served#youth#

PROBLEMS&

3.#CoordinaHon#across#key#stakeholders#engaged#in#LAC's#youth#entrepreneurship#sector#is#not#insHtuHonalised,#so#services#are#fragmented#

1.#Screening#and#selecHon#of#young#people#for#entrepreneurship#support#

is#not#rigorous#

5.#Youth#business#survival#rate#and#profitability#is#low#

1.#Young#people#lack#the#awareness,#knowledge#and/or#skills#to#start#a#business#

2.#LAC’s#bulging#youth#populaHon#does#not#have#access#to#high#quality#entrepreneurship#support#services##

3.#AllocaHon#of#resources#in#the#YE#sector#is#not#effecHve#or#efficient##

5.#Financial#insHtuHons#are#unwilling#to#lend#to#young#entrepreneurs##

6.#Cost>efficient#and#effecHve#approach#to#YE#support#are#not#clearly#idenHfied##

7.#Synergies#and#efficiencies#in#service#delivery#are#not#opHmised##

8.#Cross>sector#collaboraHon#is#not#systemaHc#4.#Youth#entrepreneurship#pracHHoners#have#limited#influence#on#public#policy#

5.#ExisHng#financial#mechanisms#do#not#unlock#finance#for#under>served#young#entrepreneurs#

6.#ExisHng#non>financial#support#is#not#equipping#young#entrepreneurs#for#sustainable#livelihoods#

7.#NGOs#have#limited#knowledge#and#experience#of#building#partnerships#with#the#private#sector#

8.#Scope#of#individual#youth#entrepreneurship#iniHaHves#is#o[en#geographically#and/or#themaHcally#restricted#

9.#Limited#delivery#of#follow>up##support#for#under>served#young#entrepreneurs#in#the#start>up#and#early#growth#stages#

10.#Provision#of#entrepreneur#training#does#not#meet#the#needs#of#young#people#

11.#Voice#of#youth#entrepreneurship#support#sector#is#weak#

12.#Monitoring,#evaluaHon#and#learning#not#systemaHc,#or#embedded#across#youth#entrepreneurships#support#operaHons#

13.#Limited#exploraHon#of#alternaHve#self>employment#opportuniHes#for#young#people#(eg.#corporate#value#chains)#

2.#Limited#awareness#of#entrepreneurship#as#a#

career#opHon#

The problem/solution tree methodology can be used to brainstorm different elements of a central challenge, and the causes/effects of these various problems – as such, it establishes the context in which an intervention is to take place.

Developing a problem tree involves writing causes/effects in a negative form (e.g. lack of knowledge, not enough money etc.)

Reversing the problem tree, by replacing negative statements with positive ones, creates a solution tree. A solution tree identifies means-‐impacts relationships as opposed to cause-‐effects. This provides some insights into the interventions that could be used to address the core challenge.

The IADB’s Multilateral Investment Fund (MIF) draws on this methodology to support the design of programs and projects.

The problem and solution tree developed for the YEP are presented below:

5

Youth Entrepreneurship Program (YEP) Solution Tree

[Note that YBI generally favours the Theory of Change approach to support programme/project design, and development of the corresponding MEL framework. A Theory of Change has not been created for the overall YEP, but country projects under Component 1 of the YEP will be designed using a combination of the problem/solution tree and Theory of Change approaches, as per the Country MEL Plan Template. More information on YBI’s use of the Theory of Change approach can be requested from [email protected]]

MEANS&

IMPACTS&

1.#Increase#in#incomes#reduces#inequality#

4.#A#dynamic#knowledge#based#on#proven#and#scalable#entrepreneurship#support#models#for#under>served#youth#is#created##

RESULTS&

3.#CoordinaAon#across#key#stakeholders#engaged#in#LAC's#youth#entrepreneurship#sector#is#insAtuAonalised##

1.#Screening#and#selecAon#of#young#people#for#entrepreneurship#support#

is#more#rigorous#

1.#Young#people#have#adequate#awareness,#knowledge#and/or#skills#to#start#a#business#

2.#LAC’s#bulging#youth#populaAon#has#access#to#high#quality#entrepreneurship#support#services##

3.#Investors#and#donors#are#able#to#idenAfy#and#invest#in#efficient#and#effecAve#youth#entrepreneurship#

4.#Financial#insAtuAons#are#willing#to#invest#in#young#entrepreneurs##

5.#Cost>efficient#and#effecAve#approach#to#YE#support#idenAfied##

6.##A#regional#center#of#excellence#and#naAonal#level#iniAaAves#idenAfy#and#draw#on#synergies#to#maximise#efficiency#and#effecAveness#of#YE#

support#

7.#Cross>sector#collaboraAon#is#embedded#in#youth#entrepreneurship#support#

5.##New#financial#mechanisms#unlock#finance#for#under>served#young#entrepreneurs#

6.#Non>financial#support#is#beTer#equipping#young#entrepreneurs#for#sustainable#livelihoods#

8.#Regional#infrastructure#developed#to#facilitate#collaboraAon#across#LAC#&#joined#up#delivery#of#support#services#to#YEs#

9.#Increased#delivery#of#follow>up##support#for#under>served#young#entrepreneurs#in#the#start>up#and#early#growth#stages#

10.#Entrepreneur#training#&#mentoring#rooted#in#best#pracAce#

11.#Voice#of#youth#entrepreneurship#support#sector#is#strengthened#

12.#Monitoring,#evaluaAon#and#learning#is#systemaAc#and##embedded#across#youth#entrepreneurships#support#operaAons#

13.#AlternaAve#self>employment#opportuniAes#for#young#people#(e.g.#corporate#value#chains)#are#explored#

2.#Awareness#of#entrepreneurship#as#a#

career#opAon#is#enhanced#

4.#Young#people#have#improved#future#prospects#and#increased#confidence#

3.#Young#people#are#included#2.#New#jobs#are#created#

5.#Youth#business#survival#rate#and#profitability#is#increased#

7.#NGOs#have#improved#knowledge#and#experience#of#building#partnerships#with#the#private#sector#

6

1.3 Program Results Framework

Impact Risks

The impact of the Program is to increase the numbers of poor and low-income youth in LAC successfully starting sustainable businesses and creating new jobs

Indicator 1 Baseline Inter Value

Inter Value

Target Country macroeconomic conditions are not stable Changes to the policy and/or regulatory environment increase constraints to starting a new business Young people are unable to access the support required to start and grow a business The quality of support services available to young entrepreneurs is weak

Number of youth-led businesses that survive for at least 1 year

0 7,268 16,205 19,740

Formula/Definition Month 0 Month 48 Month 72 Month 88

Sample of youth-led businesses still trading (operating) from the time they are launched or strengthened from a select number of country projects

Source: Follow up survey of sample of young entrepreneurs


Inter Value

Target

Number of jobs created 0 14,537 32,410 39,480


Sample of youth-led businesses surviving for at least 1 year creating jobs, including the founder entrepreneur in case of new businesses, assuming 1 out of 2 business creates one additional job



Inter Value

Target

Number of young entrepreneurs who increase their sales

0 5,815 12,964 15,792


Sample of those young people who start and/or grow a business that survives for at least 1 year from a select number of country projects, assuming 85% of these entrepreneurs are able to increase their sales.


Results Risks

The result is to improve the entrepreneurial capabilities of an increased number of young people


Inter Value

Target Cultural attitudes towards entrepreneurship become more negative

Number of new youth-led businesses created

0 5,192 11,575 14,100


7

through improved access to and quality of support services, laying the foundations for a stronger youth entrepreneurship ecosystem of selected countries across LAC

New formal or informal businesses started up, that are struggling to establish, formalize or scale

Source: YBI OMS or equivalent project monitoring system

Supply of mentors is weak or unreliable The enterprise lending environment slows Priorities of local partners and funders change Implementing partners do not see value in collaboration


Inter Value

Target

Number of existing youth-led businesses strengthened

0 5,192 11,575 14,100


Formal or informal existing businesses strengthened that are struggling to formalize or scale. To define when a business is considered strengthened, the NUYEP definition could be taken into account (NUYEP is the YBI's DfID funded program in Uganda). Please see note 1 above for the definition.

Source: YBI OMS or equivalent project monitoring system


Inter Value

Target

Number of young people who increase enterprise technical skills

0 28,035 44,397 58,032


Successful completion of formal training course and eligible for next stage of support, including those young people who choose not to continue with the program

Source: YBI program CRM


Inter Value

Target

Number of young people whose confidence levels have improved 6 months post intervention

0 21,805 34,531 45,136


A sample of young people from selected projects, including all program beneficiaries/participants whether they have started a business or not, i.e. entrepreneurs, employees of youth-led businesses and young people trained who go into decent employment, assuming 70% increase confidence

Source: Survey of sample of young people

8


Inter Value

Target

Number of implementing partners working collaboratively

0 1-3 2-4 6-10


Working collaboratively includes implementing partners putting in place partnership agreements across sectors in service delivery, knowledge development, resource mobilization among others and participating in joint or group working


Component 1 Risks

Design and Implementation of Youth Entrepreneurship Support Projects


Inter Value

Target No appropriate partners identified in countries where YBI has not already completed feasibility study/has a member. Demand from young people to start a business is low Lack of counterpart funding

Number of MIF-project funded implementing partners testing and/or scaling services for young entrepreneurs.

0 3-5 8-10 8-10


Implementing partners are YBI members subject to program criteria and YBI quality assurance; services include (1) youth outreach and recruitment; (2) screening and testing selection; 3) non-financial support services; (4) access to finance; and (5) alternative self-employment opportunities through micro-franchising and value chain insertion.



Inter Value

Target

Number of young people who receive support to start and/or grow a business

0 18,690 29,598 38,688


Support includes financial and/or non-financial services, or microfranchising or access to value chains; and is provided to help a young person start a new business or to strengthen an existing but struggling


9

business


Inter Value

Target

Number of young people provided with enterprise training to start and/or grow a business

0 31,150 49,330 64,480


Training is delivered in technical and life skills, to help young people decide if entrepreneurship is the right career choice for them; and prepare to start and grow a business.


Component 2 Risks

Capacity Building of Youth Entrepreneurship Support Organizations


Inter Value

Target Suitable and affordable local specialists with required skills not identified Needs of project holders and YBI are not aligned

Number of organizations receiving needs-based capacity building interventions to improve entrepreneur support services

0 3-5 5-7 8-10


Capacity building interventions are based on YBI global best practice and tailored specifically to LAC region; entrepreneur support services include youth outreach and recruitment; screening and selection; non-financial support services; access to finance; and micro-franchising and value chain insertion.



Inter Value

Target

Number of organizations receiving needs-based capacity building interventions to strengthen organizational delivery capability

0 2-4 4-6 8-10


10

Delivery capability areas include: technological infrastructure (YBI’s cloud-based Salesforce Operations Management System); communications; resource-raising and growth strategies, monitoring, evaluation and learning (MEL), partnership development


Component 3 Risks

Knowledge and Communication - Regional Center for Youth entrepreneurship


Inter Value

Target Program does not secure the required level of counterpart funding Appropriate and affordable location for Center not identified Appropriate Center staff are not identified

A Regional Center for Youth Entrepreneurship established, facilitating knowledge exchange among stakeholders and supporting a regional network of organizations

0 1 1 1


The Regional Center is new with an increasing number of full-time staff to manage the different functions of the Center, based on YBI’s global network structure, and with increasing financial sustainability



Inter Value

Target

Number of global capacity building tools tailored specifically for the LAC region

0 3-5 5-7 8-12


Global capacity building tools are primarily YBI’s suite of knowledge offerings in key entrepreneur service and organizational areas delivered and defined under Component 2 (above)



Inter Value

Target

Number of evaluations conducted contributing to the program learning agenda

0 5-10 20-40 59-81


11

Evaluations conducted include: 1 randomized impact evaluation 8 – 10 non-randomized studies including action learning approaches of projects (under Component 1) and 50 – 70 non-randomized studies including action learning approaches of capacity building interventions (under Component 2)



Inter Value

Target

Number of external knowledge products developed based on regional learning

0 6-8 16-20 20-25


Knowledge products include research publications, videos, offerings, website



Inter Value

Target

Number of Regional LAC Youth Entrepreneurship Summits

0 1 2 2


Youth Entrepreneurship Summits are conferences that convene regional partners and stakeholders for networking, knowledge sharing and skills exchange, based on YBI’s Global Summit model.


Note 1: A business is considered strengthened if at least one of the following criteria is met:

• Sales increase (–measured in US $; monthly/quarterly/biannual/annually – choose only one – at the data entry stage, convert to a monthly figure, and also ensure they are reporting on sales, not profit)

• Investment increase – start-up capital/new equipment acquired/new licence for franchising/assets/land/building/animals (emphasis on BUSINESS investment); timeframe is everything to date – measured in US $

• Additional business added to the existing business • New products/services • New branches/outlets New markets - local/national/regional/international

(Note that this definition has been adapted from a DfiD-funded program implemented by YBI and our member organisation in Uganda)

12

2. Learning Agenda

2.1 A strategic approach to learning In order to maximise the impact of the YEP, and ensure that lessons learned are captured and utilised, a strategic approach to learning is crucial. This is reflected in: • the program design, in that one of the three core components is dedicated to knowledge and

communication; and • the fact that the YEP team includes a MEL Specialist as one of the 3 core staff members. As such, we are developing this Learning Agenda at the outset, in consultation with key program stakeholders, to ensure that the YEP succeeds in systematically deriving and capturing learning throughout the program life cycle. This learning will then be disseminated back to practitioners and stakeholders across the Region to enhance the delivery of quality support to young entrepreneurs and help develop entrepreneurship ecosystems. Taking this strategic approach to learning will enable the YEP to provide insights into what works, does not work and why: I. for different beneficiary groups; II. in a range of geographical, economic and cultural contexts; and III. in respect of different support services: training, mentoring, and access to finance for young under-

served entrepreneurs as well as alternative self-employment opportunities for other young people.

2.2 YEP’s Learning Vision We believe that the YEP presents a unique opportunity to build up the evidence base around what works in supporting under-served young entrepreneurs in the LAC region, in alignment with YBI’s own vision and strategy to build up a global evidence base (see YBI’s 2014-17 Research and Learning Agenda: http://www.youthbusiness.org/wp-content/uploads/2014/04/YBIResearchandLearningAgenda.pdf). As such, we are committed to (i) instilling a strong learning culture and capabilities across program stakeholders, and (ii) complementing the measuring and reporting of program outputs, outcomes and impacts through the results framework with specific activities focused on generating and disseminating learning. This will help us ensure that lessons learned can feed back into the improved targeting of investments to generate enhanced efficiency and effectiveness of youth entrepreneurship support initiatives delivered through the YEP program itself and beyond.

2.3 YEP’s Learning Principles

Learning under the YEP: • Is integrated and embedded across all program activities to build insights throughout our learning

agenda;

• Is aligned with YBI’s Research and Learning Agenda and the MIF’s learning agenda on youth entrepreneurship, to maximise the transferability of learning in areas where knowledge is identified as lacking for both the network and wider sector;

• Is delivered through a range of methodologies/processes to ensure a proportionate and appropriate approach;

• Adheres to ethical principles, standards and guidelines;

• Involves beneficiaries and stakeholders as active participants wherever possible to ensure that MEL activities are meaningful and responsive.

13

2.4 Key Learning Themes and Questions Overall, the YEP’s learning agenda aims to respond to the overarching questions: • Which interventions and combinations of programs are more effective in enabling

the poor to start up and grow their own business? • Which type of intervention is more effective for whom, and to produce which

outcomes?” focusing specifically on youth? These questions will be explored through 5 Key Learning Themes, as follows:

The need for youth entrepreneurship support

Effective and innovative ways of supporting young entrepreneurs

Impacts of promoting youth entrepreneurship

Developing the youth entrepreneurship ecosystem

Effective MEL approaches for exploring these themes

Each theme is accompanied by a set of Key Learning Questions. Initially, almost 40 learning questions where identified. After a review and thanks to the experience gained during filed visits to YEP Members, the number of questions has been reduced, focalizing on what is really relevant at regional and country level (for the original list of questions please see annex 1). These learning questions will guide both the execution of every single project within the YEP and the development of knowledge product along the Program

.

LEARNING THEME ONE: THE NEED FOR YOUTH ENTREPRENEURSHIP SUPPORT

KEY LEARNING QUESTION RELEVANT AT COUNTRY LEVEL (WHICH ONES?) OR REGIONAL LEVEL?

To what extent do individual factors (e.g. lack of relevant skills/knowledge) versus systemic factors (e.g. cultural/institutional factors) affect the creation/growth of a business and its sustainability

Regional level, but particularly relevant for Argentina y Mexico

Do underserved young entrepreneurs need specific services compared with others? And if so, what do these look like?

Regional level, but particularly relevant for members that offer services for adults too

14

LEARNING THEME TWO: EFFECTIVE & INNOVATIVE WAYS OF SUPPORTING YOUNG ENTREPRENEURS


• What are the most effective and innovative ways of screening and selecting young entrepreneurs to support?

Regional level, but particularly relevant for Peru and Mexico

• To what extent does the existence of a quality business plan determine the success of a business? How should this business plan look like?

Regional level

• What are the most effective and innovative approaches to: • Training (to develop entrepreneurial, business and financial literacy skills) • Mentoring • Access to finance?

Regional level

What are the most effective and innovative ways to help young entrepreneurs build their networks?

Regional level

How can we most effectively and innovatively harness technology to support young entrepreneurs?

Particularly relevant for Chile and Brazil (e-learning)

How can you cost-effectively involve young people throughout the project cycle to ensure the relevance of and encourage buy in and commitment to the support offered? Who else should be involved beyond the young entrepreneur? (e.g. the wider family)

Regional level

What are the most efficient and effective delivery models (including partnership models) for supporting young entrepreneurs at scale, while maintaining quality?

Particularly relevant for Peru and Brazil (aim to become 2nd tier organizations)

LEARNING THEME THREE: IMPACTS OF PROMOTING ENTREPRENEURSHIP

KEY LEARNING QUESTION RELEVANT AT COUNTRY LEVEL (WHICH

ONES?) OR REGIONAL LEVEL? • To what extent does supporting young people to

start/grow businesses create jobs? • What kinds of jobs are created under which

circumstances?

Regional level (jobs created indicator included in all Logical Frameworks)

To what extent are the impacts of supporting young entrepreneurs sustainable, under what circumstances?

Regional level (sustainability indicator included in all Logical Frameworks)

To what extent do increased incomes resulting from young people starting/growing businesses reduce inequality?

Regional level (income/sales increasing indicator included in all Logical Frameworks)

LEARNING THEME FOUR: DEVELOPING THE YOUTH ENTREPRENEURSHIP ECOSYSTEM


• How do we enhance the environment to enable effective youth entrepreneurship?

• What kinds of activities are most effective in delivering which kinds of ecosystem-wide change?

Regional level

What are the most effective and innovative mechanisms for youth entrepreneurship practitioners to influence public policy, across different contexts?

Regional level, but particularly relevant for Argentina

How can we encourage financial institutions to lend to young entrepreneurs?

Regional level

What are the most effective and innovative models to promote synergies/partnerships and cross-sector

Regional level

15

collaboration across the youth entrepreneurship ecosystem? How can practitioners forge effective partnerships with the private sector, in terms of (i) resource raising, and (ii) integrating young entrepreneurs into corporate value chains?

Regional level

LEARNING THEME FIVE: EFFECTIVE MEL APPROACHES FOR EXPLORING THESE THEMES


What existing MEL approaches can be draw upon for exploring these learning themes and questions?

Regional level

What are the most effective MEL approaches for tracking the progress of young entrepreneurs during the cycle of support?

Regional level

• What are the most effective MEL approaches for tracking long-term impacts of supporting young entrepreneurs?

• And what are the timeframes involved?

Regional level

How can we incentivize stakeholders to implement MEL effectively?

Regional level

How can we capture the less tangible outcomes and impacts of helping young people start/grow their businesses?

Regional level

• What are the most effective and innovative approaches for sharing learning?

• How does evidence need to be packaged for different users?

Regional level

What are the most effective mechanisms for ensuring lessons learned feed back into enhanced support for young entrepreneurs?

Regional level

Based on YBI’s experience, in order to answer to the key questions above, quantitative and qualitative methodologies for data collection will be considered. Please refer to the MEL tool table in section 5 of this document. Each question will be answered across different contexts and youth segments, e.g. gender, livelihoods vs. dynamic entrepreneurs, rural vs. urban, particular ‘at risk’ groups.

The YEP Learning Agenda has been created through undertaking a literature review of existing evidence (as presented in the Bibliography to this MEL framework) in addition to a mapping exercise to ensure alignment with YBI’s Research and Learning Agenda 2014-17 and the MIF Learning Agenda on youth entrepreneurship (as outlined in sections 2.5 and 2.6 below, and detailed in Annex One). The Learning Agenda will be subjected to a regional stakeholder consultation, led by the YEP MEL Specialist in Q1 2015, after which this document will be revised, based on consultee feedback. In order to update this Learning Agenda, the following elements will be considered:

• prioritization of learning questions at both program and country level;

• which questions are more suitable for exploration at country Project level, and which lend themselves to an eco-system wide focus;

• the possibility of building a country project around responding to one of the key learning questions;

16

• which kinds of quantitative and qualitative methodologies could be utilized to explore which questions, drawing on YBI’s experience.

2.5 Alignment with YBI’s 2014-17 Research and Learning Agenda The YEP Learning Agenda is strongly aligned with YBI’s 2014-17 Research and Learning Agenda, which has key three themes, as follows: 1. Building the case for youth entrepreneurship

2. Demonstrating the value of the YBI network approach

3. Evaluating what works for underserved young entrepreneurs

YBI’s Research and Learning Agenda is grounded in our organisational Theory of Change, in particular the logic that a global network model maximises outcomes for underserved young entrepreneurs in communities around the world. Learning questions / hypotheses are based on testing the assumptions and knowledge gaps behind delivering impact according to this logic:

YBI Theory of Change

17

2.6 Alignment with the MIF’s learning agenda on youth entrepreneurship The YEP learning agenda was developed in line with the MIF’s learning agenda around youth entrepreneurship, as defined through a ‘knowledge gap matrix’, that is based on the lessons learned from its previous experiences and presents five key learning themes2:

• Innovation: Test innovative job training approaches for at-risk youth who often face additional stigma in the job market, especially if they come from violence-prone areas; (2) refine existing entrepreneurship models, test new screening and teaching approaches and explore new entrepreneurship alternatives for at-risk youth, such as microfranchising, youth cooperatives, and value chain approaches; and (3) promote financial services for young entrepreneurs and financial literacy for youth more broadly.

• Scale: Take a systemic approach to addressing youth employability by working with key public, private and civil society actors to scale up the most effective youth training and job placement models through the New Employment Opportunities for Youth Initiative.

• Gender: Create training programs that are relevant for both sexes by addressing the different needs and circumstances of both male and female youth.

• Youth Empowerment: Empower young people by involving them directly in the design and implementation of projects and create structures for youth leadership within projects.

• Results, Evaluation and Knowledge Sharing: Increase project impact, improve cost-effectiveness and guide decision-making about future programming by taking a comprehensive approach to monitoring and evaluation, using a set of common indicators, a mix of monitoring tools and both quantitative and qualitative evaluation methods

A detailed mapping of the alignment between the YEP learning agenda, YBI’s Research and Learning Agenda, and the MIF’s learning agenda on youth entrepreneurship is annexed to this document.

2 Please see http://www.fomin.org/en-us/HOME/Projects/Access-to-Markets/Youth for more details - note that the first of these learning themes relates more to the área of youth employment than youth entrepreneurship, hence it does not form a core part of the YEP Learning Agenda

18

3. The Program MEL Cycle The chart below outlines how the activities under each of the five elements of the MEL framework will be executed across the five-year timeframe of the YEP.

(*) Starting from march 2014

Closing workshop

Evaluate Capacity Development Trainings deployed

Element 2: Evaluation of projects under component 1&2

Develop approach & tools

Develop approach & tools

Annual research study Annual research study Annual research study Annual research study

Annual research pubblication Annual research pubblication Annual research pubblication Annual research pubblication

Year 1(*) Year 2 Year 3 Year 4 Year 5

Implement mid-term & process evaluations for selected projects

Ongoing monitoring

Element 1:Systematic Monitoring against Logframes

Element 3:Research studies

Element 4:Appliying knowledge to YEP programming

Element: 5Sharing knowledge generated through YEP

Implement endline evaluations for selected projects

Regional Youth Entrepreneurship Summit

Regional Youth Entrepreneurship Summit

Presentations at third party conferences and summits

19

4. YEP Data Collection Plan

Data will be collected with respect to • all indicators in the program results framework provided in section 1.3; • key learning themes and questions presented in section 2.4 • the ‘additionality’ achieved through taking a regional approach, rather than simply implementing a set

of individual country level projects • the Development Assistance Committee (DAC) criteria for evaluating development assistance, outlined

below

DAC EVALUATION CRITERIA

Relevance The extent to which the activity is suited to the priorities and policies of the target group, recipient and donor. Effectiveness A measure of the extent to which an activity attains its objectives. Efficiency Efficiency measures the outputs -‐-‐ qualitative and quantitative -‐-‐ in relation to the inputs. It is an economic term which signifies that the aid uses the least costly resources possible in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted. Impact The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended. This involves the main impacts and effects resulting from the activity on the local social, economic, environmental and other development indicators. The examination should be concerned with both intended and unintended results and must also include the positive and negative impact of external factors, such as changes in terms of trade and financial conditions. Sustainability Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable.

Source: http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm

20

Results Framework area

Information required

How will data be collected?/Source

Who will collect data?

When will data be collected?

Usage / purpose

Impact indicator 1

Number of youth-led businesses that survive for at least 1 year

Survey - data stored in OMS records; Final impact evaluation

YEP MEL Specialist; External evaluation consultant

End of the Program and one year after programme close

Program’s impact assessment

Impact indicator 2

Number of jobs created





Impact indicator 3

Number of young entrepreneurs who increase their sales





Result indicator 1

Number of new youth-led businesses created

Partners’ ME systems - data stored in OMS records; mid- term and final evaluation

Partner’s MEL Specialis;YEP MEL Specialist; External evaluation consultant

Annual basis Project monitoring, learning and reporting

Result indicator 2

Number of existing youth-led businesses strengthened




Result indicator 3

Number of young people who increase enterprise technical skills




Result indicator 4

Number of young people whose confidence levels have improved 6 months post intervention




Result indicator 5

Number of implementing partners working collaboratively



Annual basis Program monitoring and learning

Component indicator 1.1

Number of MIF-project funded implementing partners testing and/or scaling services for young entrepreneurs

Cooperation agreements with Partners records

YEP MEL Specialist

At agreement Progam monitoring


Number of young people who receive support to start and/or grow a business

Partners’ ME systems - data stored in OMS record; Support records - data stored in OMS records

Partner’s MEL Specialis;YEP MEL Specialist

Rolling basis, reported quarterly

Project monitoring, learning and reporting


Number of young people provided with enterprise training to start and/or grow a business

Partners’ ME systems - data stored in OMS record; Training records - data stored in OMS records

Partner’s MEL Specialis;YEP MEL Specialist


Project monitoring, learning and reporting


Number of organizations receiving needs-based capacity building interventions to improve entrepreneur support services

Capacity Building records - ata stored in OMS record

YEP MEL Specialist; CB Manager


Program monitoring and learning


Number of organizations receiving needs-based capacity building interventions to strengthen organizational delivery capability

Capacity Building records - ata stored in OMS record

YEP MEL Specialist; CB Team




A Regional Center for Youth Entrepreneurship established

Regional Center establishment documents

YEP MEL Specialist

Once established and annually for operations



Number of global capacity building tools tailored specifically for the LAC region

Tools produced YEP MEL Specialist; CB Team

Annually Program/project monitoring and learning


Number of evaluations conducted contributing to the program learning agenda

Products developed Partner’s MEL Specialis;YEP MEL Specialist

Annually Program/project monitoring, evaluation and learning


Number of external knowledge products developed based on regional learning

Products developed Partner’s MEL Specialis;YEP MEL Specialist



Number of Regional LAC Youth Entrepreneurship Summits

Assistance records YEP MEL Specialist


5. MEL Products and Processes

This section of the MEL framework outlines the different types of products and processes that will be generated and disseminated through the YEP.

Element 1: Systematic Monitoring against Program and Country Level Project Results Framework

Under Component 1, country project level Results Frameworks will be developed using the problem and solution tree approach presented in section 1.2 of this document. In addition to context/project specific indicators, a core set of indicators will be included in all project level Results Frameworks, which contribute to the wider program Results Framework, as follows:

• Number of young people who receive support to start and/or grow a business

• Number of young people provided with enterprise training to start and/or grow a business

• Number of new youth-led businesses created

• Number of businesses strengthened

• Number of young people who increase enterprise technical skills

Component 1 projects will use the YBI program Customer Relationship Management (CRM) system (which uses the Salesforce platform), to report and monitor progress against the Results Framework on a regular basis. Bi-annual progress will then be updated in to the PSR by the project representative in accordance with MIF obligations. The consolidation of each project’s progress in to the YBI program CRM will mean that the YEP team will have an over-arching programmatic view all country projects, and the regularity of reporting will give the team insight and ability to respond to project status. The YEP team will likewise track progress against Components 2 and 3 using the YBI program CRM so that the complete MEL framework requirements can be efficiently and effectively managed. On a bi-annual basis, this progress will be logged in the PSR.

Using the YBI programme CRM as the management tool for YEP will create transparency in 2 directions; out to key stakeholders, and in and between country projects. An appropriate level of insight will be given to all country project representatives of each other’s country project status in order to foster dialogue. In addition, the CRM will be used as the platform for delivering on a number of Component 3 aims. As such, the CRM will be used not just as a monitoring tool for Results Framework indicators, but also for sharing evaluation products and knowledge.

In Q1 2015, the YEP team will launch the Component 1 CRM reporting process for the first country projects that have been approved. For each additional project approved by MIF, the roll out of the related CRM reporting process will be timed with the launch of each project.

In Q2 2015, Component 2 and 3 reporting will be operationalized in the CRM, and in H2 the creation of a platform for collaboration and knowledge sharing between YEP country projects will be launched.

The YEP MEL specialist is actively working with a specialist from YBI’s technology team to deliver against this timeline.

Monitoring program and project level progress will enable us to determine how both the overall program and individual country-level/capacity building projects/elements of the Regional Centre are evolving, ascertain trends across projects, and identify and respond to any progress-related issues.

Element 2: Evaluation of (i) YEP overall, and (ii) country level projects under Component 1, and (ii) assessment of capacity building interventions under Component 2

At the overall program level, externally-conducted mid-term and endline evaluations will be commissioned by the MIF, while it is not considered necessary to conduct a program-level baseline evaluation, as the data will come from project level baselines. In addition, it is envisaged that a minimum of one impact evaluation using a Randomized Controlled Trial (RCT) methodology will be commissioned, either at project or program level.

Each of the 8-10 youth entrepreneurship projects to be delivered through Component 1 of the YEP will implement evaluations in line with an overall program level approach, which will combine quantitative and qualitative methods; for example, (i) sample-based surveys, (ii) Focus Group Discussions (FGDs), and (iii) individual interviews, for example, using the Life Histories method, and (iv) the Most Significant Change approach.

The 50-70 capacity building interventions to be delivered through Component 2 of the YEP will be evaluated (hereafter termed ‘assessment’ to avoid confusion with the more rigorous evaluation approach to be used for Component 1 projects) through a combination of process and outcome assessment, covering the following: (i) baselining of organizational capacity; (ii) process monitoring of different aspects of the project (e.g. did it meet stakeholder needs?), and (iii) assessment of the project’s impact on the organization. More extensive process assessments will be carried out for projects that involve piloting of new Offerings, and in cases where the context is considered to entail particular learning potential. The approach for conducting assessments of Component 2 projects is currently being scoped, in alignment with parallel processes at the wider organisational level, under the leadership of YBI’s Director of Network Strengthening, with technical input from YBI’s MEL team, including the YEP MEL Specialist. We expect the design work to be completed in Q1 2015, at which point more detail will be added to this section of the MEL framework.

Element 3: Execution of research studies exploring Key Learning Themes and Questions

At the program level, at least one original research study relating to the YEP Learning Agenda will be commissioned annually (except Year One) – these studies could relate to specific projects, or a cross-section of projects.

A scoping of potential research & learning partners at regional/country level (including universities) and their research and learning agendas will be conducted by the MEL specialist in Q1 2015. Results will be included in this MEL framework. In addition, an annual research publication will be developed (again, starting from Year Two), showcasing program and project level MEL activities from the past 12 months.

Element 4 of the MEL Framework: Applying knowledge to planning, implementation and decision-making

Strong emphasis will be placed on usage of data/findings generated through MEL processes from the outset, with all country projects required to consider this through completion of a ‘MEL Communications Plan’, outlining the key ways in which they will use data/findings, and how such data/findings should be presented to them, as part of their project MEL plans. Learning embedded in country project level reporting: Regular country project level reporting will include identification of lessons learned, in addition to reporting on Results Framework indicators. Use of learning logs: Furthermore, all country project holders will be required to complete learning logs (see Annex Three for the learning log template) on a quarterly basis, which will be used as the basis for bi-

annual project level learning and reflection sessions, facilitated by project MEL leads, with appropriate support from the YEP MEL Specialist. Learning logs are based on the Action Learning methodology, typically used for organisational and operational-level learning. The learning log template to be used by country project holders place a strong emphasis on how learning is applied, through specific questions around changes/improvements to project design and delivery as a result of lessons learned. It covers both the key learning questions that form the basis of the project Learning Agenda, as well as wider learning that has taken place among project stakeholders over the course of the preceding six months. Regional application of knowledge: The regional Youth Entrepreneurship Summits (modelled after YBI’s 2013 Global Summit) to be implemented through Component 3 of the YEP will include programme wider learning and reflection sessions, focused on key learning themes, while additional opportunities for sharing learning across country projects for internal application at the wider programme will be scoped as country projects come on board; options will include webinars and exchange visits. Element 5 of the MEL Framework: Collaborating and sharing knowledge across the region, YBI network and sector to influence effective policy and practice

The Strategy for Element 5 of the MEL framework we be developed as part of the process of strategy development for the Regional Center for Youth Entrepreneurship, following which more details will be added to this section of the MEL framework.

6. MEL Tools

The YEP MEL framework is intended as a live document. This section will document key MEL tools that are developed and implemented through the course of its execution.

Tool Name Tool Description Tool Purpose

Country project holder MEL Diagnostic (see Annex Two)

A tool to diagnose existing MEL capacity of country project holders, in order to identify gaps where capacity building is required

To provide a comprehensive picture of a member’s existing MEL capabilities and needs

Learning log template (see Annex Three)

A template to be used by country project holders, with specific questions around changes/improvements to project design and delivery as a result of lessons learned

To be used as the basis for project level learning and reflection sessions

Training Quality Evaluation Survey

A survey to be conducted at the end of each training session

To measure levels of participant satisfaction with training and to gather feedback and suggested improvements from participants

Focus Group Discussions

Focused discussion with a small group (usually 8 to 12 people) of participants with a moderator

To gather qualitative feedback around issues and to record attitudes, perceptions, and beliefs

Case Study A detailed descriptive narrative focused on a particular unit - a person, a site, a project. It often uses a combination of quantitative and qualitative data

To understand how different elements fit together and have produced the observed impacts

Contribution analysis

An approach for assessing causal questions and inferring causality in real-life program evaluations

To reduce uncertainty about the contribution one intervention is making to the observed results, through an increased understanding of why the observed results have occurred or not

Literature review An assessment of a body of research that addresses a research question

To identify what is already known about an area of study

Most Significant Change (MSC)

A collection of significant change (SC) stories emanating from the field level and systematic selection of the most significant of these stories by panels of designated stakeholders or staff

To have whole teams of people focused on project impact; to give a rich picture of the impact of a project and provide the basis for dialogue over key objectives.

Participatory Video

A process of supporting local communities to create videos in order to evaluate the impact of interventions that target them

To mobilise marginalised people and to help them implement their own forms of sustainable development based on local needs

Randomized evaluations

A type of impact evaluation that uses random assignment to allocate resources, run programs, or apply policies as part of the study design.

To determine whether a program has an impact, and more specifically, to quantify how large that impact is

Youth lead research

An approach whereby young people are directly involved in undertaking the research process i.e. designing and implementing, data collection tools, data entry and analysis, and production of research outputs.

To empowering young researchers and to cover findings that and implementing, data collection tools, data

Social Return on Investment (SROI)

SROI is an approach to understanding and managing the value of the social, economic and environmental outcomes created by an activity or an organisation. It is based on a set of principles that are applied within a framework.

To understand and manage the value of the social, economic and environmental outcomes created by an activity or an organization; to give people a voice in resource allocation decisions

Use of YBI’s Contexts Framework

YBI has developed a Contexts Framework through understanding ‘what works where, for who’ and why’, implemented through a workshop approach

A workshop approach for using the framework to diagnose operating context

Entrepreneurial Inclusion Index (EII)

Alianza Emprendedora has designed this tool, which is implemented in the form of a game and a quiz.

To monitor the less tangible aspects of entrepreneurship

(Semi-structured) Interviews

Directed interviews to selected people or groups To probe and follow up topics of interest in depth

Surveys Systematic collection of information from a sample of population

To determine the impact, results and/or outcome of a project/program

Direct observation

Observation of specified site, using a detailed observation form

To determine the impact, results and/or outcome of a project/program

7. Key Stakeholders and plan for Communication of Findings In order to effectively execute the fourth and fifth elements of the YEP’s MEL framework (namely (iv) applying knowledge to planning, implementation and decision-making; and (v) collaborating and sharing knowledge across the region, YBI network, and sector, to influence effective policy and practice), it is important to identify the key stakeholders who will use findings generated from elements 1-3, and map out how findings will be disseminated to them. Stakeholder consultations will be organized on a regular basis. Such consultations will not be considered as a single act, but as a series of opportunities to create and share knowledge about understandings, opinions, expectation around youth entrepreneurship in the region. Each stakeholder consultation process will take into account who needs to be consulted, for what purpose and over which topics. Every contact with stakeholders will start with an explanation of why they will be consulted and how the information collected will be taken into account. Depending on the kind of stakeholder (partners, mentors, public and private sector, NGOs , donors, policy makers, think tank, academic, among others), different methods of consultation may be adopted. These could include, among others: one-on-one meetings, focus groups, semi structured interviews, questionnaires, discussion forum, workshops. The consultations will be gender-inclusive, on the basis that men and women can have differing views and needs.

The table below will be used to map out key MEL stakeholders, the learning questions they are interested in (cross-referenced with the table in section 2.4 above), how they are likely to use the findings from YEP’s MEL activities, and how findings will be disseminated to them:

Key Stakeholders

Learning Questions they are principally interested in

What knowledge they can contribute

How will they use findings?

How will findings be disseminated to them?

Public bodies supporting young entrepreneurs

• What are the benefits of investing in youth/ the risks of not investing in youth?

• What are the economic impacts of promoting youth entrepreneurship – at the individual, household, community and wider macro levels?

• To what extent does supporting young people to start/grow businesses create jobs?

• To what extent do increased incomes resulting from young people starting/growing businesses reduce inequality?

• How do the impacts of promoting youth entrepreneurship compare with those of other labour market interventions?

• Policy making • Youth

mobilization at national and local level

• To provide baseline level data

• To provide statistical data

• To design entrepreneurship/employment support policies that more effectively meet the needs of young people

• To design youth entrepreneurship/ employment support programs

• To better allocate funds

Multi-stakeholder workshops, conferences,

Civil society organisations supporting young entrepreneurs (including YBI’s members)

• What are the most effective MEL approaches for tracking the progress of young entrepreneurs during the cycle of support?


• What are the most effective and innovative ways of screening and selecting young entrepreneurs to support?

• Is psychometric testing an effective screening approach?

• What are the most effective and innovative models to promote synergies/partnerships across the youth entrepreneurship ecosystem?

• What are the most effective and innovative approaches to: • Training (to develop

entrepreneurial, business and financial literacy skills)

• Mentoring • Access to finance?

• Experience from the field

• Experience in mobilizing young entrepreneurs

• Networking

• To strengthen their MEL system

• For project design and implementation

• To build partnership • To influence public

policies • To access to funds

seminars, Regional Entrepreneurship summits, publication of case studies documenting lessons learned, regular reporting evaluation studies, newsletters, press conferences.

Mentors • Do young entrepreneurs need specific services compared with others?

• And if so, what do these look like?

• What are the most effective and innovative approaches to mentoring?

• Experience from the field

• One to one relation with young entrepreneurs

• Emotional aspect of mentoring

• New approaches, techniques and tools for mentoring

Private institutions (microcredit institutions, entreprises...)

• How can we encourage financial institutions to lend to young entrepreneurs?

• Do young entrepreneurs need specific services compared with others?

• And if so, what do these look like?

• Technologies for development

• Skilled human capital development

• Marketing

• To explore new markets

• To offer new products and/or services

• To be part of a wider entrepreneurship ecosystem

• To stimulate innovation

Donors financing YE’ship programs

• What are the most effective MEL approaches for tracking the progress of young entrepreneurs during the cycle of support?


• What youth entrepreneurship support approaches do not work, and why?

• What are the most efficient and effective delivery models (including partnership models) for supporting young entrepreneurs at scale, while maintaining quality?

• Global partnerships creation

• Technical assistance

• For program design • To better allocate

funds • To build and share

knowledge

Universities • What are the most effective and innovative ways to enhance the voice of young entrepreneurs and youth entrepreneurship practitioners?

• To what extent do individual (e.g. lack of relevant skills/knowledge) versus systemic factors (e.g. cultural/institutional factors) play a role?

• What are the most effective and innovative ways to raise awareness of entrepreneurship as a possibility for young people?

• How do the impacts of promoting youth entrepreneurship compare with those of other labour market interventions?

• Technologies for development

• Training • How to produce

high level research and studies

• To share knowledge • To develop

innovative academic offers

Young entrepreneurs

• What are the most effective and innovative ways to enhance the voice of young entrepreneurs and youth entrepreneurship practitioners?

• What are the most effective and innovative mechanisms for youth entrepreneurship practitioners to influence public policy, across different contexts?

• To what extent does the existence of a quality business plan determine the success of a business?

• Experience on the field

• Provide stories of success

• To create/ strengthen a business

• To expand network of contacts

• To access to funds

8. MEL capacity building for country project holders under the YEP

Why is it important to build capacity in MEL under the YEP? Having a strong MEL system in place will help country project holders to be more efficient and effective, by helping them to understand ‘what works’ and what doesn’t, and why, as well as allowing them to share these insights with wider YEP stakeholders, and indeed the wider YBI community and youth entrepreneurship sector, through Component 5 of the YEP MEL Framework, thereby contributing to building up the global knowledge base on this highly under-evidenced sector. Furthermore, enhanced ability to demonstrate impacts will make only support reporting to the MIF under the YEP, but will make country project holders more attractive to funders more widely, thereby helping them to leverage counterpart funding, as well as more able to influence policy at national, regional and indeed international levels. Capacity Assessment MEL capacity building interventions under the YEP will commence with a MEL needs assessment, to comprise part of the wider capacity assessment undertaken during proposal development missions (see Annex Two below). Development of project MEL systems

Part A: In person support: Country project MEL systems will be built using the Theory of Change (ToC) approach, as used by YBI’s MEL team. Project level ToCs will be mapped out during proposal development missions, based on the problem/solution trees that will form the output of brainstorming sessions to shape the project approach. A project level Results Frameworks (RFs) will then be developed, based on the project ToC, using the RF template provided by the MIF. The YEP MEL Specialist will be present during proposal development missions to lead this process, including the provision of technical support to definition of indicators, ensuring project level RFs are aligned to the overall programme RF etc. Part B: Follow up remote support: Following the initial proposal development mission, the YEP MEL Specialist will provide remote support as required in order to finalise the project level ToC and RF, as well as providing support to the development of the country project level MEL plan, which outlines

(i) the Project Learning Agenda (drawing on the overall program level Learning Agenda (LA) – the YEP MEL Specialist will guide the process of selecting/adapting priority learning questions for exploration through the project and highlighting how their investigation will contribute to the overall programme level LA;

(ii) a MEL Communications Plan, outlining the key stakeholders interested in MEL data/findings, their key areas of interest, how they will use the data/findings, and how data/findings will be disseminated to them;

(iii) the Project MEL cycle, outlining how this intersects with the overall project cycle; (iv) a Data Collection Plan, outlining how data will be collected with respect to all RF indicators

and key learning questions included in the project LA, and how the project will consider the DAC evaluation criteria (namely relevance, efficiency, effectiveness, impact and sustainability)

as well as presenting the project level Problem/Solution Trees, Theory of Change and Results Framework. Implementation of capacity building Support provided by YEP MEL Specialist The YEP MEL Specialist will be the lead provider of MEL capacity building to country project holders under the YEP; indeed, this is a core function of his role. He will be further backstopped by YBI’s MEL Advisors, through regular sharing of best practice tools, processes etc. Capacity building interventions will be planned and delivered in response to capacity gaps identified through the MEL capacity assessment. In addition to ongoing support with respect to project montoring and reporting, MEL capacity building under the YEP will also cover support to the design and implementation of research and evaluation studies, addressing key learning questions under the project LA. Thus far, YBI’s

MEL team has identified the following types of studies that they can support YBI members to run (see also 2.2 on this). Each study uses a recognised technique to evaluate outcomes or impacts of the member’s work. However, we very much welcome country project holders to suggest their own methodology rather than choosing one of these options, and the YEP MEL Specialist will endeavour to seek appropriate specialist support under 2.2 where he does not have direct expertise himself: Most significant change approach: The most significant change (MSC) technique is a form of participatory monitoring and evaluation. MSC focuses on monitoring the results of a project by involving a variety of stakeholders in creating stories of changes. The YEP MEL Specialist is due to be trained in this methodology in early 2015, following which he will able to roll out the training to country project holders. A typical support package would include training on how to use the MSC technique; support and assistance to implement the MSC technique as part of the project MEL system; and support and assistance in documenting the outputs (stories and case study reports) from the exercise. Measuring Value for Money (VfM): VfM is all about understanding the relationship between costs and benefits of a programme. YEP’s MEL Specialist (who has in turn be trained by YBI’s MEL Specialists) can provide country project holders with an introduction to VfM and support to design a system for monitoring and analysing Value for Money provided by their project, using a range of techniques (for example basic VfM planning, cost effectiveness analysis, cost utility analysis, social cost benefit analysis, multi-criteria appraisal). Context Analysis workshop: Over the past three years, YBI has been working on a global research project to develop a framework for analysing ‘what kind of youth entrepreneurship support interventions work where, and why?’ This has involved the design of a workshop approach for utilising the framework in order to undertake an analysis of the local context and, in doing so, understand how project design and delivery needs to be shaped in order to maximise impact. Workshops would be facilitated by the YEP MEL Specialist. Support provided by other technical specialists YBI’s MEL team could also provide support to undertake the following types of studies through our increasing network of research and learning partners: Participatory Video Evaluation: YBI’s MEL team has become increasingly interested in partnering with Insightshare (see www.insightshare.org) , specialists in Participatory Video evaluation, to design and implement studies utilising this exciting methodology for involving a group or community in evaluating the outcomes/impacts of interventions in which they are participating, through creating ‘stories of changes.’ A typical support package would include:

• a seven day training in PVE, delivered by YBI’s partner Insightshare, comprising: (i) 3 days of

workshop-based learning activities; (ii) 3 days of practising the method (including editing) with up to 24 participants, recording 3 video stories of change; and (iii) 1 day for screening, evaluation and close up.

• the following outputs: (i) a short report on the training, (ii) photos, and basic post production of the videos, and (iii) 1 master DVD and 2 further copies of the DVD.

Youth led research study: YBI has formed partnerships with various consultants specialised in youth-led qualitative research, which involves training up young people as qualitative researchers to conduct studies with their peers. Using this innovative approach means members can benefit from the insights gained through using young people’s unique perspective, while it also provides young people with additional skills and training. A typical support package would include the provision of 7-15 days consultancy support and training for country project holder staff and the young people identified to lead the study would receive 7-15 days of consultancy support and training to get a small-scale project off the ground, including the training a group of young people in qualitative research methods, support to their work on their own ‘live’ research project, as well as research design and management; focus group moderation and facilitation training; and support with analysis of findings. Rigorous quantitative evaluation: The YEP MEL Specialists can guide country project holders to the most appropriate Latin America-based researchers specialising in rigorous quantitative evaluation methodologies, such as randomised impact evaluation, and facilitate introductions to support co-design of research/evaluation projects of this nature.

Peer to peer capacity building

Where particular capacity gaps are identified among country project holders in areas in which other project holders possess expertise, a peer to peer capacity building approach will be employed, utilizing a mixture of in-person and remote support, as per other types of capacity building. Peer to peer capacity building in MEL will also be a feature during the regional Youth Entrepreneurship Summits (modeled after YBI’s 2013 Global Summit) to be implemented under Component 3 of the YEP.

Bibliography: • Kevin Hempel, Nathan Fiala. Measuring success of Youth Livelihood Interventions - A practical Guide to

Monitoring and Evaluation. 2012, The International Bank for Reconstruction and Development/The World Bank (.pdf)

This guide addresses the monitoring and evaluation of youth livelihood interventions, with a specific focus on impact evaluation and a case study from the Northern Uganda Social Action Fund.

• Larry Dershem Tamar Dagargulia Lilly Saganelidze Stephanie Roels. NGO Network Analysis Handbook: how to measure and map linkages between NGOs. 2011, Save The Children (.pdf)

The aim of this handbook is to help NGO staff accomplish the following objectives: a) understand what network analysis is, b) how are network data collected, c) how to create visual maps of the network, and d) how to analyze the network data for program/project development or evaluation.

• David Card Pablo Ibarrara ́n Juan Miguel Villa. Building in an Evaluation Component for Active Labor Market Programs: A Practitioner’s Guide. 2011, IZA (.pdf)

This guide outlines the main evaluation challenges associated with Active Labour Market Programs and shows how to obtain rigorous impact estimates using two leading evaluation approaches: the randomized design and the difference in differences.

• Effective Capacity Building in Nonprofit Organizations. 2001, Venture Philanthropy Partners (.pdf)

This document is a tool for measuring an organization’s capacity level. The report also shares the lessons learned from nonprofits that have engaged in successful capacity building efforts.

• Steve Cumming. 2013-2014 Youth Think Tank Report: Engaging Young People. 2014, The Master Card Foundation (.pdf)

This document presents a method for seeking more meaningful ways to engage young people in dialogue about decision-making, program design, evaluation and policy-making.

• McKenzie, D. and C. Woodruff. What Are We Learning from Business Training and Entrepreneurship Evaluations around the Developing World. 2012, The World Bank (.pdf)

This paper undertakes a critical review of entrepreneurship evaluations, synthesizing the emerging lessons and understanding the limitations of the existing research and the areas in which more work is needed. • Valerie Stetson, Communicating and Reporting on an Evaluation. 2008, Catholic Relief Service and

American Red Cross (.pdf) This guide provides readers with information that helps private voluntary organization staff facilitate learning among individuals, groups, and organizations by communicating and reporting evaluation processes and findings more effectively. • El compromiso con los stakeholders. Manual para la practica de las relaciones con los grupos de interés.

2005, AccountAbility, United Nations Environment Programme, Stakeholder Research Associates Canada Inc. (.pdf)

This document provides a best practice guide to stakeholder engagement, with the broader aim of promoting the use of stakeholder engagement worldwide as a way of advancing sustainable development goals. • Give Youth a Chance. An agenda for action. 2012, FOMIN (.pdf) This report analyzes important lessons learned during the 18 years of experience FOMIN have funding over 120 youth employment and entrepreneurship training projects around the region.

Key YBI documents (please see www.youthbusiness.org):

• YBI 2014-17 Strategic Plan: http://youthbusiness.org/wp-content/uploads/2014/04/YBI_StrategicPlan_2014-17.pdf

• YBI Research and Learning Agenda 2014-17: http://www.youthbusiness.org/wp-content/uploads/2014/04/YBIResearchandLearningAgenda.pdf

• The YBI Network approach: How three YBI network members are supporting young entrepreneurs around the world, with lessons for best practice: http://www.youthbusiness.org/wp-content/uploads/2012/08/TheYBINetworkapproach.pdf

• Youth Entrepreneurship: a Contexts Framework: http://www.youthbusiness.org/wp-content/uploads/2012/08/ContextsConsultation.pdf

• Closing the gap – how non-financial support is increasing access to capital for young entrepreneurs: http://www.youthbusiness.org/wp-content/uploads/2012/08/ClosingTheGap.pdf

• Recommendations for action: How governments, businesses and civil society organisations can help young people get started in business: http://www.youthbusiness.org/wp-content/uploads/2012/08/RecommendationsforAction.pdf

• Beyond Collateral - How to increase access capital for young entrepreneurs: http://www.youthbusiness.org/wp-content/uploads/2012/08/BeyondCollateral.pdf

Annex one: Mapping of YEP Learning Agenda against YBI 2014-2017 Research & Learning Agenda and MIF Learning Agenda on Youth

Entrepreneurship

Comparing Key Learning Themes

Building the case for youth entrepreneurship

The need for youth entrepreneurship support

Innovation

Evaluating what works where and why for under-

served young entrepreneurs

Effective ways of supporting young entrepreneurs

Scale

Demonstrating the value of YBI’s network approach

Livelihoods impacts of promoting youth entrepreneurship

Gender

Social and psycho-social impacts of promoting youth entrepreneurship

Empowerment

Developing the youth entrepreneurship ecosystem

M&E

Comparing Key Learning Questions – Need for youth entrepreneurship support

What are the challenges facing youth globally? Is the situation escalating / static / improving?

In what ways are young people wanting to start/grow businesses ‘under-served’, and why, across different contexts and youth segments?

What are the differences at the gender level when starting a business? (Focus on different sectors? Have different growth expectations?_

What are the benefits of investing in youth? What are the risks of not investing in youth? What kind of economic and social opportunities do young people need to fulfil their potential?

What are the key barriers to young people starting businesses? To what extent do individual (e.g. lack of relevant skills/knowledge) versus systemic factors (e.g. cultural/institutional factors) play a role?

What activities/services/tools effectively support women to remove barriers (practices and cultures) to their employment/self employment? How to engage men/women in trades that are non-traditional to their gender, and how is this accomplished sustainably? (is there evidence of this being successful?)

Do young people face additional constraints? If so, to what extent? What are these constraints, and which constraints have the most effect? What are other significant discriminatory factors? Do young entrepreneurs need specific services compared with others?

Comparing Key Learning Questions – Effective ways of supporting young entrepreneurs

What are the most effective ways to raise awareness of entrepreneurship as a possibility for young people, across different contexts and youth segments?

How to raise awareness among vulnerable young people to increase the use of ICT (information access?) related to education, employment and entrepreneurship?

What are the most effective ways of screening and selecting young people for entrepreneurship support across different contexts and youth segments?

Do psychometric tests work when applied to young people (across different cultures/ages/degress of vulnerability)? To what extent is a quality business plan necessary to determine / predict the success of the company?

What quality assurance standards and mechanisms are most relevant to youth entrepreneurship practitioners? How should YBI’s quality

assurance processes be adapted to different member models and capacities? What is the Return on Investment for (i) members and (ii) the network team of undertaking quality assurance, and how do we maximise it?

What is the value of a shared approach to technology as a network? How can YBI most effectively use technology across different contexts to support i) the network as a whole; ii) individual members; and iii) young entrepreneurs? Which network processes are most effective to identify i) which of the technology options available are most strategic to adopt; and ii) how they should be implemented?

What works where, why, and for whom? What is the Return on Investment of YBI Offerings for (i) members and (ii) the network team, and how do we maximise it? What level and type of central coordination is necessary to sustain effective communities of practice? How widely should YBI communities of practice

What are the most effective training/mentoring/finance models to support young entrepreneurs across different contexts and youth segments?

What mentoring methodologies are most effective for different segments of youth? What are the most effective platforms to provide mentoring? How necessary is access to finance for the success of a venture? What financing products are effective for young people? How can you involve young people cost-effectively throughout the project cycle way (beyond that as beneficiaries, partners and leaders) to improve the relevance of the

be opened up for participation? What kinds of community activities are most effective in delivering which kinds of change?

project and its appropriation by the beneficiaries? (e.g. the alumni as mentors? in designing curriculum, etc.). What kind of activities should be planned to achieve this? "

Is a holistic approach to entrepreneurship support more effective than single service interventions? If so, why, and what combination of support is most effective where, why and for whom?

What does an effective integrated package of support to young entrepreneurs look like across different contexts and youth segments? What are the similarities and differences?

What is the combination (training modules, elements of financial and non-financial) that is more efficient to prepare the entrepreneur? Are there variables to be taken into account especially for methodological, such as age, education level, vulnerability, rural vs. urban, opportunity vs. necessity entrepreneurs etc. How to increase access to ICT by young people? How you can use social networks and ICT to complement training in entrepreneurship? And to provide training and support? What are the networks that are more effective? What educational and socio-economic levels should be targeted?

Is it good practice to use the platform of the project to provide training not related to the project (sex education, domestic violence, etc..) What are effective ways of working with the family of the beneficiary?

Is locally embedded service delivery the best option, and if so, what are the most effective models? Does being part of a network enable local service providers to scale and improve services and improve outcomes for

What are the most efficient and effective delivery models (including partnership models) for supporting young entrepreneurs at scale, across different contexts and youth segments?

Are there successful models of entrepreneurship for young people working to scale? How can we develop effective methodologies to scale youth entrepreneurship and enhance the quality of services offered? What kind of alliances can be most effective?

underserved young entrepreneurs? If so, how? Which in-country models best deliver sustainable and scalable quality services to under-served young entrepreneurs, under which circumstances? Does expanding the YBI network necessarily have to take place through adding new members or scaling existing members, or are there other strategies that are more efficient and achieve an equivalent impact, for example strengthening in-country ecosystems?

Comparing Key Learning Questions – Livelihoods Impacts

What are the economic and social impacts of entrepreneurship, and are they sustainable? How do these impacts compare with those of other labour market interventions?

What kind of youth entrepreneurship support initiatives are most effective in enhancing business profitability, and/or sustainability? To what extent does supporting young people to start/grow businesses create jobs? What kinds of jobs are created under which circumstances? To what extent are livelihoods gains from helping young people start/grow businesses sustainable, under what circumstances? To what extent do increased incomes resulting from young people starting/growing businesses reduce inequality?

Comparing Key Learning Questions – Social and Psychosocial impacts

What are the economic and social impacts of entrepreneurship, and are they sustainable? How do these impacts compare with those of other labour market interventions? (REPEATED)

Is the promotion of youth entrepreneurship an effective response to social disenfranchisement of youth, and if so, what are the causal pathways? What are the psycho-social impacts of promoting youth entrepreneurship, and how do they differ by context, and across youth segments? How can we capture the less tangible outcomes and impacts of helping young people start/grow their businesses?

Comparing Key Learning Questions – MEL for youth entrepreneurship programmes

What are minimum standards for data quality and how do they align with network quality assurance? What are the most useful mechanisms for sharing data, in order to meet network and external needs? What is the Return on Investment for (i) members and (ii) the network team of deriving and applying quality data?

How can we capture the less tangible outcomes and impacts of helping young people start/grow their businesses?

What M & E instruments on employment/entrepreneurship issues have been effective? How long should we continue to measure an entrepreneurship / employment initiative after completion of the program to see if it was successful or not? Are there effective tools to track the success of ventures / jobs created? What are the best practices MIF (as supervisor and financier) can implement to ensure efficient monitoring of projects? What incentives should apply to the Implementing Agencies?

How does the YBI network What should be a common minimum

most effectively (i) attract and (ii) allocate resources in order to build an evidence-based network? How can YBI most effectively design and package evidence to increase its usefulness to, and uptake by, target audiences? How is technology best deployed to build and share evidence?

set of indicators of employment / entrepreneurship that is relevant? How do we effectively integrate the results of studies into a project? How do we effectively mainstream gender issues in market assessments of the projects? When is a study of gender needs warranted?

Comparing Key Learning Questions – Developing the youth entrepreneurship ecosystem

How can we encourage financial institutions to lend to young entrepreneurs?

What is the value of collaboration within and across sectors? What the most effective models of collaboration?

What is the most effective way to promote synergies/partnerships across the youth entrepreneurship ecosystem? How we can embed cross-sector collaboration across youth entrepreneurship initiatives? How can service delivery NGOs operating in this arena forge effective partnerships with the private sector, in terms of (i) resource raising, and (ii) integrating young entrepreneurs into corporate value chains?

What needs to be in place to enable effective youth entrepreneurship? What are the most effective ways to operate in different institutional and cultural environments? How do we enhance the enabling environment? Does a global network approach increase influence and impact, and if so, how?

What are the most effective network processes and activities to amplify local and regional voices at the global level? How do we best leverage members’ skills, experiences and resources in this area for network value? How do we identify, recruit and make the best use of ambassadors to ensure that they add value to the YBI brand?

How can we enhance the voice of young entrepreneurs and youth entrepreneurs practitioners?

Annex Two: MEL Capacity Diagnostic Tool Draft

This diagnostic tool is intended to provide a comprehensive picture of a country project holder’s existing MEL capabilities and needs, to assist in scoping a MEL capacity building project under Component 2 of the YEP.

It should take approximately 2 hours to complete. It is designed to be completed by the YEP MEL Specialist during a proposal development mission. Before undertaking the diagnostic with the country project holder, it would be useful to consult:

• The YBI member organisation, to see if they have any data collection tools, M&E manuals and/or evaluation reports and any other background documents that can be shared in advance, to maximise the effectiveness of the time spent ‘on the ground’

• The member’s Relationship Manager and any other members of the YBI Network Team who have recently visited/had contact with the member, in case they are aware of any relevant documents, so we are not requesting information we already have available

Name of potential country project holder

Staff members interviewed (Names and job titles)

Visit date

Location on the YBI server where any corresponding documentation (e.g. data collection tools/evaluation reports) are stored

1. What does MEL mean to them?

(Guided discussion)

1.1 How do they define MEL?

1.2 Do they think it’s important? Why? Why not?

1.3 Do they have a theory of change? (Attach if documented )

1.4 How do they staff their MEL function?

1.5 MEL Funding (What kind of budget do they have available for MEL? And how is the budget structured - is it a standalone budget, or built into project budgets?)

2. What data do they currently collect and how (Gather copies of data collection tools and snapshots/blank copies of data storage forms/systems, where possible]

Data collection area (e.g. project impacts/beneficiary registration)

Indicators data is collected on

How is data collected?

When is data collected?

Who collects data?

3. Software used for MEL [Add details of software used for data collection, entry, processing and analysis] 4. Users of MEL data

User of MEL data/findings

What? What for? How are data/findings communicated to them?

Issues?

Beneficiaries

Project staff

Donors

Policymakers

5. Evaluation

5.1 Their experience so far

5.2 Future plans/aspirations:

6. Key MEL topics

(Do they consider these issues, and if so, how?)

6.1 Attribution (How do they determine causal links between their interventions and expected/observed outcomes/impacts? Do they take into account external factors?)

6.2 Gender (Do they collect gender-disaggregated data? In what other ways do they consider gender?)

6.3 Long-term outcomes (Do they track their programme participants once they have graduated from the programme? If so, how, and what metrics do they capture? Note this is covered briefly under section 2, but there is space here for a more detailed discussion )

6.4 Context (Do they consider the importance of context in designing programmes and/or interpreting the data they collect? If so, how?)

7. MEL Best Practice

7.1 What best practices would they like to share across the YBI network? (e.g. data collection tools, processes, evaluation studies)

Wider youth entrepreneurship sector

Media

Others? (Specify)

7.2 What best practices would they like to receive from across the YBI network? (e.g. data collection tools, processes, evaluation studies)

Annex Three: YEP Learning Log Template

YEP learning log

Project name and country

Period: (MM-MM/YY)

Actions and Reflections

What were the key achievements of the past four months?

What actions were delivered i) as planned? ii) in addition to

plans?

What planned actions were not delivered?

What worked well over the past four months? Why?

What didn’t work well over the past four months? Why?

What was the most significant barrier/challenge? Was it expected? (How) was it dealt with?

What key issues have not been dealt with? Why? Was the time spent on actions proportionate to results?

What threats/risks have been identified for planning over the next four months?

What opportunities have been identified for planning over the next four months?

Learning and planning

Have you made any positive changes over the past four months as a result of what you learned? If so, what? (Eg. redesigned a process; stopped doing something inefficient or ineffective)

What will you plan to do differently over the next four months as a result of your learning?

What insights were gained into your Key Learning Questions? (The questions identified in your project MEL plan)

[Insert KLQ One here]

[Report insights here]

[Insert KLQ Two here]


[Insert KLQ Three here]


(Within realistic resource constraints) what actions will you plan to deliver in the next four months to address your Key Learning Questions? Have any additional Key Learning Questions emerged over the course of the past four months?

youth entrepreneurship program (yep) monitoring...

Documents