Making TVET Providers Accountable: Ten Years of Hard Lessons at MCC

Ryan Moore & Marcel RicouMillennium Challenge Corporation

Three Learning Takeaways

• MCC’s experience in TVET has been underwhelming, but hopefully we’re doing better

• Focus on accountability to improve TVET service delivery

• Explore ways to make service providers more directly accountable to employers

Introduction to MCC

What makes us unique?• Selectivity• Minimum economic

rate of return• Strict 5-year clock• Focus on results• Country-led

solutions and implementation

4

US Government Development Agency with a mission to

Reduce poverty through economic growth.

Sectors Where MCC Works

Energy 17%

Transport 31%

Water 10%

Financial Services 1%

Agriculture 18%

Health, Education & Community

Development17%

Governance 6%

March 2018 | www.mcc.gov

• Investments selected based on an assessment of the greatest constraints to inclusive economic growth

• Investments address policy, institutional and social constraints to growth

5

Countries Where MCC Works

6

MCC currently works in 24 countries. MCC has 15 active compact and threshold programs, and 9 programs under development across Africa, Latin America and Asia.

• We evaluate every project• Independent researchers• Timed to allow results to

emerge (after closeout)• Learning agenda influences

program design• Impact evaluations where

appropriate• All evaluations and data

posted to our catalogue

data.mcc.gov/evaluations7

How we do Evaluations

Growth, Employment, and TVET

MCCat TVET

Why MCC invests in TVET and Workforce Development

The private sector lacks the skilled workforce needed to grow and compete.

Our investments focus on delivering skills so that:

Participants get jobs.Firms have access to in-demand skills.

10

Our TVET Investments to date

11

Global & Historical Context

12

Less than 1/3 of evaluations of TVET programs show jobs and income impacts. Short term job training offers opportunities, but most programs fail to deliver…

… but, TVET can deliver on its promise for promoting employment and growth.

What we’velearned so far…

El Salvador Investments:Strengthen 20 Technical 2ry Schools & one Post-2ry Institute

• Infrastructure, equipment, new programs, teacher training – including bringing in trainers with experience

• Competency Based Training/ Assessment

• Scholarship program

Short-term for vulnerable populations• Classroom-based courses offered

based on user requests, no labor mkt data available

• Median trainee: 30 yr old woman, rural, ~8 yrs of education

14

El Salvador

15

Results:• Secondary Schools

• Increased enrollment but no effect on graduation, progression, placement or one year income

• Experienced teachers replaced with traditional civil servants

• Technical Institute• Existing autonomous governance,

CBT&A = perceived quality• Offered tracks had mixed results on

the labor market• Non-formal training

• Higher self-reported employment and income, varied by sector

• Desire for more practical training

MongoliaInvested in:• National governance – New Gov TVET

Agency• Facility upgrades / new equipment• Introduced NQF and CBT&A• Teacher and manager training• Twinning with international partners

Relevant results:• Impact evaluation of new equipment• Performance evaluation = some policy

reforms maintained; perceptions of quality improved; unconfirmed anecdotes of improved placement

• Both studies limited - but, no evidence of impact

16

Namibia

17

Invested in:• Built/renovated & equipped Community

Skills Dev. Centers• Management training

Relevant Results:• 85% program completion. Positive

perceptions.• After 1 year, only 40% employed; only

13% in field of study. (Female results worse)

• Many in temp. employment/dissatisfied

Namibia

18

Invested in:• Established national payroll levy with

linked industry councils• Scholarships for training

Relevant Results:• Payroll levy collected, but disbursements

slow• Industry councils were

ineffective/underdeveloped• Increased enrollment/completion from

scholarships, but no employment/wage impact

Achieved a LOT of our targeted outputs

19

Did we achieve our objectives?

20

“Increase income generation through the provision of improved technical training programs.”

-El Salvador Education and Training Activity “Alleviate skills constraints to enterprise growth and productivity and to ensure that … [graduates] acquire competencies that are needed for gainful employment and income-earning opportunities. ”

-Namibia Education project

“Improve the quality of TVET education, its relevance to employer needs, and to increase employment and incomes.”

-Mongolia Vocational Education Project

21

Did we achieve our objectives?

22

First Generation Takeaways

Dynamic labor market needs

Simply doing more status quo in public system delivered weak results

Hands-on, practical learning is key

Need to address gender disparities in both recruitment and outcomes Judge success by hard data AND feed that analysis into governance

The Challenge of Service Delivery in TVET

Accountability in TVET Provision

24

A Short-er Route of Accountability

25

A New Results Framework

27

28

29

What we’redoing differently…

and we hope better!

Quick Reminder…

31

The Second GenerationWith Georgia, we put the private sector at the heart of the project. Competitive grant program that required local

business partners w/ co-investment Policy development / technical assistance

32

Interim Evaluation Findings: Competitive grants demonstrated value of

private sector; built trust. Strong industry engagement: 50% leverage Strong perceptions of training Public provider governance = sustainability risk Low women’s participation (only 14%)

The Second Generation of MCC/TVETIn Morocco, we identified a form of PPP called Institut de GestionDeleguee: Sectoral training centers run

by industry associations Partial government subsidy

33

91% job placement compared to status quo of ~50%

Results-based Financing (RBF)• Simple concept: make payment

contingent on specific, measurable results.• In Morocco, we’re piloting a program which pays providers for job placement

and job retention (instead of paying only for training completion)

MCC’s Principles for TVET DeliveryGovernment’s role re-envisioned

• Towards a funder and regulator of TVET

34

Practical and private-sector driven content• Learning and assessment driven by the private sector, including as evaluators• Trainers = industry practitioners• Training = practical and in the work place

Autonomous management of service delivery• Independent board of directors; majority private sector• Private sector staffing model (not civil servants)

Diversified Financing and Services. Generation of resources from: • Sale of in-service or consulting services to firms• Donations• Student Fees

Lessons Learned

Start with the big picture – is TVET the right solution?

Lesson 1:

A demonstrated skills gap should be a precondition for investment in TVET

37

Keep your eye on the ball

Lesson 2:

TVET should have two primary goals: Placing graduates in jobs with

improved incomes, and Providing the private sector with

in-demand skills

39

The missing link – Employer Accountability

Lesson 3:

Focus first on short-route accountability:Especially between employers and service providers

41

National Governance Matters

Lesson 4, Part 1:

Don’t forget to strengthen Long-Route accountability…

By strengthening national systems, especially:

Data for decision making like tracer studies.

Quality assurance tools like national qualifications frameworks and competency-based approaches.

43

Innovate the Long RouteLesson 4, Part 2:

…Especially when you can’t address the Short Route.

44

Challenge is our Middle Name

Lesson 5:

TVET interventions alone are unlikely to resolve pre-existing inequalities in the labor force, but TVET has a role to play

46

Conclusion

47

- Global track record isn’t great, and MCC did no better - Key challenge is service delivery – especially in making TVET accountable to employers

- Our lessons are built around a new results framework- Emerging toolkit to improve MCC practice- We’ll need to keep learning with 2nd Gen evaluations

48

So what?Together, by focusing on enhanced service delivery accountability

we can realize TVET’s promise for growth and employment.

For more information, visit:mcc.gov/sectors/sector/education

data.mcc.gov/evaluations49

Help us improve the GYEO Summit by filling out the survey for this session. Surveys can also be found

on the Summit Mobile App.