7/31/2019 Flor Unintended Outcomes

1/14

UNINTENDED OUTCOMES:

MINING HIDDEN GEMS (OR DUDS) IN

EDUCATIONAL EVALUATION

Alexander G. Flor

University of the PhilippinesOPEN UNIVERSITY

7/31/2019 Flor Unintended Outcomes

2/14

EVALUATION PRAXIS

Our teaching/theorizing on educational evaluationis sterile without practice

Educational projects M&E provides an ideal

prospect to practice, do research, improve teaching Dominant education sector reform evaluation

paradigms is based on theories of change (TOC)

Logical framework or project design matrix

Pre-identified M&E points

Outcome and output indicators based on the verticaland horizontal logic of the project design

7/31/2019 Flor Unintended Outcomes

3/14

EVALUATION PARAMETERS

Parameters that we set

by design in education

evaluation provide us

with focus necessary inobjective, disciplined

inquiry

Rubrics

Results based indicators

At times, parameters

serve as horse blinders

7/31/2019 Flor Unintended Outcomes

4/14

IMPACT EVALUATION

Ex-post regulatory

evaluations

Final project evaluations

Require us to removethese blinders at some

point in the process to

explore the unintended

outcome Prescribe the use of

mixed methods research

7/31/2019 Flor Unintended Outcomes

5/14

UNINTENDED OUTCOMES

Mined in 3 M&E projects conducted in 2011 in 3Asian subregions representing 3 education

subsectors and 3 development agencies.

PROJECT AGENCY SUBSECTOR SUBREGION

Fiji Citizenship Education Project UNDP Basic Education Pacific Islands

INO: Maritime Education and Training JICA Higher Education Southeast Asia

Nepal Strengthening National

Capacity on EFA (Literacy NFE)

UNESCO Nonformal

Education

South Asia

7/31/2019 Flor Unintended Outcomes

6/14

UNDP Fiji In School Citizenship

Education Project (CEP)

7/31/2019 Flor Unintended Outcomes

7/14

OUTCOMES

Success Indicators.

CEP contribute to: MDG 2 (universal primary education);

MDG 4 (gender equality);

MDG 7 (environmental sustainability); and

MDG 8 (global partnership for development) Unintended. These outcomes included:

Increased capacity of teachers to design, develop & utilizeinstructional materials;

Increased capacities to facilitate learner centered sessions& participatory teaching methods;

Interdenominational or interreligious collaboration;

an educational model that effectively addresses the needfor integration.

7/31/2019 Flor Unintended Outcomes

8/14

JICA-Indonesia

Maritime

Education &

Training Project

STCW PLUS

ALPHA

MET Delivery

Systems

Improvement

STCW 2010

eLearning &

Networking

7/31/2019 Flor Unintended Outcomes

9/14

Outcomes of MET Improvement

Strategic presence in the World Fleet

Prominence in the White List through

curriculum improvement

Indonesia becomes the number 1 supplier of

Merchant marines

Deck and Engine Officers

Captains

Indonesia becomes a Global Maritime Power

7/31/2019 Flor Unintended Outcomes

10/14

Unintended Economic Outcomes

Current Opportunity Cost

From Class 3 to 2: USD 3,960,000/year

From Class 2 to 1: USD 8,400,000/year

Educational Expenditure Savings

eLearning is much cheaper than F2F learning

Reduced overhead, manpower costs

Projected Contribution to GDP

USD 300 Million/ year

7/31/2019 Flor Unintended Outcomes

11/14

UNESCO Nepal

CAPEFA (NFE)

NFE as one of4 priorities inNational EFAAction Plan

Components Assessment of

CLCs

Piloting anNFE-MIS

PilotingLiteracyModules inhousehold

surveys

7/31/2019 Flor Unintended Outcomes

12/14

INTENDED OUTCOMES

Increased national ownership and partnership

building

Strengthened institutional capacities

Improved organizational capacity

Improved quality

Accelerated knowledge generation forcapacity development

7/31/2019 Flor Unintended Outcomes

13/14

UNINTENDED OUTCOMES

increased awareness and interest on NFEprograms among communities thatparticipated in the pilot literacy surveys

increased commitment and sense of

fulfillment among project stakeholders whoparticipated in the pilot literacy surveys

increased number of training resources andcontent

CapEFA validated the importance and needfor an NFE-MIS among policy makers,program planners and implementors andrenewed their commitment to establish one

7/31/2019 Flor Unintended Outcomes

14/14

EPILOGUE

It is not difficult to uncover/mine unintendedimpacts through mixed methods researchbecause usually these info are the first to be

volunteered by your key informants. However, discovering unintended outcomes do

not always paint a rosy picture

Nevertheless, our predisposition for focused or

disciplined inquiry must always be tempered byour curiosity for attendant phenomena ineducational evaluations.