PROGRESS EVALUATION OF
THE UNICEF EDUCATION IN
EMERGENCIES AND
POST-CRISIS TRANSITION
PROGRAMME (EEPCT):
Liberia Case Study
April 2011
Cover page
EVALUATION REPORT
EVALUATION OFFICE
March 2011
PROGRESS EVALUATION OF
THE UNICEF EDUCATION IN
EMERGENCIES AND
POST-CRISIS TRANSITION
PROGRAMME (EEPCT):
Liberia Case Study
EVALUATION REPORT
EVALUATION OFFICE
March 2011
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 1
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 1
Progress Evaluation of UNICEF’s Education in Emergencies and Post-Crisis Transition Programme: Liberia Case Study © United Nations Children‘s Fund, New York, 2010 United Nations Children‘s Fund Three United Nations Plaza New York, New York 10017 March 2011 The purpose of the evaluation reports produced by the UNICEF Evaluation Office is to assess the situation, facilitate the exchange of knowledge and perspectives among UNICEF staff and to propose measures to address the concerns raised. The content of the report do not necessarily reflect the policies or views of UNICEF. The text has not been edited to official publication standards and UNICEF accepts no responsibility for error. The designations in this publication do not imply an opinion on legal status of any country or territory, or of its authorities, or the delimitation of frontiers. All photographs in the evaluation report are the copyright of UNICEF © UNICEF/2010/Columbia Group for Children in Adversity For further information, please contact: Evaluation Office United Nations Children‘s Fund Three United Nations Plaza New York, New York 10017 [email protected]
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 2
PREFACE
The purpose of this evaluation was to identify and assess progress of the Education in Emergencies and Post-
Crisis Transition (EEPCT) Programme and to enable systematic reflection towards improving programme
results. The EEPCT Programme was examined at global, regional and country levels through quantitative and
qualitative methods that combined comprehensive coverage with in-depth analysis.
The EEPCT Programme began in 2006 as a four year (later extended to five), US $201 million dollar
partnership between UNICEF and the Government of the Netherlands. The EEPCT Programme aims to ―put
education in emergency and post-crisis transition countries on a viable path of sustainable progress toward
quality basic education for all.‖ EEPCT funds support UNICEF education programming in 39 countries and
territories and are also used to advance the global agenda for education in crisis-affected contexts.
The Evaluation Office commissioned this independent progress evaluation in June 2010. The global
evaluation was overseen by a Reference Group led by internal and external technical experts in evaluation,
education, and emergencies. The evaluation was conducted by Columbia Group for Children in Adversity,
associated with Columbia University. The independent team of consultants was led by Neil Boothby and
Peter Buckland. The evaluation was managed by Silvia De Giuli (Evaluation Specialist), and Ashley Wax
(Evaluation Specialist).
The evaluation methodology included: extensive document review; six in-depth country case studies in
Angola, Colombia, Côte d‘Ivoire, Liberia, the Philippines, and Sri Lanka; extensive interviews; on-line surveys
for key programme staff; comprehensive focus group discussions; review and analysis of data collected
against the programme Logical Framework.
This report presents in-depth analyses and results of the progress of the EEPCT programme in Liberia. Carl
Triplehorn and Evelyn Kandaka lead the case study with support from research associate, Natasha Rothchild,
and national researchers, Willie Benson, Mabel Flumo and Francis T. Wayne.
Special thanks to UNICEF staff across the organization and to the Reference Group who both participated
actively and provided substantive comments on emerging issues and interim reports. We would like to
acknowledge, in particular, the support of Susan Durston, Chief of Education, and Jordan Naidoo Senior
Advisor, Education Section. Likewise, we appreciate the efforts and support by the country office, including
Stella Kaabwe. Genuine thanks to the Government of the Netherlands, European Commission and other
partners who have supported education in emergencies and post-crisis transition, as well as evidence-based
decision-making.
Samuel Bickel Officer-in-Charge Evaluation Office UNICEF New York Headquarters
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 3
TABLE OF CONTENTS
PREFACE .............................................................................................................................................. 2
ACRONYMS .......................................................................................................................................... 5
EXECUTIVE SUMMARY ....................................................................................................................... 7
The Way Forward: Recommendations ................................................................................................ 11
1.0 INTRODUCTION ........................................................................................................................... 14
1.1 Overview ....................................................................................................................................... 14
1.2 Country Context ............................................................................................................................ 15
1.3 Educational Context ...................................................................................................................... 15
1.4 EEPCT Programme ...................................................................................................................... 17
2. METHODOLOGY .......................................................................................................................... 19
2.1 Purpose and methods ................................................................................................................... 19
2.2 Field-site and sample selection ..................................................................................................... 20
2.3 Limitations ..................................................................................................................................... 20
3. EVALUATION FINDINGS ............................................................................................................. 21
3.1 Overview: Results Analysis ........................................................................................................... 21
3.2 Goal One: Improved quality of education response in emergencies and post-crisis transition
countries .............................................................................................................................................. 21
3.3 Goal Two: Increased resilience of education sector services delivery in chronic crises, arrested
development, and deteriorating contexts ............................................................................................ 24
3.4 Goal Three: Increased education sector contributions to better Prediction, Prevention and
Preparedness for emergencies due to natural disaster and conflict ................................................... 27
3.5 Goal Four: Evidence-based policies, efficient operational strategies and fit-for-purpose financing
instruments for education in emergencies and post-crises. ................................................................ 27
3.6 DAC Criteria .................................................................................................................................. 30
3.6 Focus Groups Discussion Outcomes ............................................................................................ 33
3.7 Cross-cutting Issues ...................................................................................................................... 35
3.8 Operational issues and management issues ................................................................................ 37
3.9 Partnerships .................................................................................................................................. 38
4.0 THE WAY FORWARD ................................................................................................................... 38
4.1 Lessons learned and conclusions ................................................................................................. 38
4.2 Recommendations ......................................................................................................................... 41
REFERENCES .................................................................................................................................... 44
ANNEXES............................................................................................................................................ 48
Annex 1: Methodology ......................................................................................................................... 48
Annex 2: Daily Schedule ..................................................................................................................... 54
Annex 3. Interviews Conducted: National, Sub National and Community Levels ............................... 60
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 4
Annex 4: Liberia EEPCT Evaluation Framework................................................................................ 61
Annex 5: Evaluation Tools ................................................................................................................... 65
EE-DAC Score Card .................................................................................................................... 65
Adequacy Survey Checklist ......................................................................................................... 84
Programme Design ...................................................................................................................... 84
Programme Implementation ........................................................................................................ 84
Programme Learning ................................................................................................................... 84
Financial ...................................................................................................................................... 85
Best Practices/Standards ............................................................................................................ 85
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 5
ACRONYMS
ALP Accelerated Learning Programme
BEGE Basic Education and Gender Equality
CEO County Education Officer
CFS Child Friendly School
CGCA Columbia Group for Children in Adversity
CPC Country Programme for Children
CRC Convention on the Rights of the Child
CO UNICEF Country Offices
DEO District Education Officer
ECD Early Childhood Development
ECD/DAC Organization for Economic Co-Operation and Development‘s Development Assistance
Criteria
EC/EU European Community /European Union
EPF Education Pooled Fund
EEPCT Education in Emergencies and Post-Crisis Transition programme
EFA Education for All
EMIS Education Management Systems
EPDF Education Programme Development Fund
ESP Education Sector Plan
FAWE Forum for African Women Educationalists
FGD Focus Group Discussion
FTI Fast Track Initiative
GER Gross Enrolment Ratio
IASC Inter-Agency Standing Committee
IDP Internally Displaced Person
INEE Inter-Agency Network for Education in Emergencies
IRC The International Rescue Committee
LAB4LAB Learning Along Borders for Living Across Boundaries
LACE Liberia Agency for Community Empowerment
LET Liberian Educational Trust
LCO UNICEF Liberia Country Office
LPERP Liberia Primary Education Recovery Programme
NGO Non-Governmental Organization
MDGs Millennium Development Goals
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 6
MOE Liberian Ministry of Education
MOF Liberian Ministry of Finance
MSEE INEE Minimum Standards for Education in Emergencies, Chronic Crises and Reconstruction
NER Net Enrolment Ratio
NPSCE National Primary School Certificate Exam
OECD Organisation for Economic Co-operation and Development
OSI Open Society Institute
PTA Parent Teachers Association
PRES Programme Review and Evaluability Study
PREV Progress Evaluation of the UNICEF Education in Emergencies and Post-Crisis Transition
Programme
RTTI Rural Teacher Training Institute
SC/UK Save the Children/United Kingdom
SWAp Sector Wide Approach
TOT Training of Trainers
UN United Nations
UNDP United Nations Development Programme
UNESCO United Nations Educational, Scientific and Cultural Organization
UNGEI United Nations Girls Education Initiative
UNHCR United Nations High Commissioner for Refugees
UNICEF United Nations Children's Fund
UNMIL United Nations Mission in Liberia
UNOPS United Nations Office for Project Services
USAID U.S. Agency for International Development
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 7
EXECUTIVE SUMMARY
The Progress Evaluation of the UNICEF Education in Emergencies and Post-Crisis Transition Programme is
an appraisal of the implementation of UNICEF‘s ‗EEPCT: Education in Emergencies and Post-Crisis
Transition Programme‘ (EEPCT). Started in 2006, EEPCT is a four-year, $201 million dollar partnership
between UNICEF, the Dutch Government and the European Union. EEPCT currently supports programming
in 39 countries, as well as global initiatives such as the Inter-Agency Education Clusters and the Inter-Agency
Network for Education in Emergencies. The EEPCT programme aims to ‗put education in emergency and
post-crisis transition countries on a viable path of sustainable progress towards quality basic education for all‘.
The evaluation was global, reviewing EEPCT implementation in 39 countries. However, it also included six
country case studies to support the global analysis. UNICEF Evaluation Office chose Liberia as one of the
case studies. This report examines how EEPCT funds have been used within UNICEF‘s programmes in
Liberia, and discusses ways forward.
Liberia Background
Liberia‘s violent internal conflict ended in 2003. The 14-year conflict devastated the country‘s infrastructure
and caused economic collapse. A significant portion of the population was internally displaced or became
refugees. Seven years later, the country is in the process of recovery and development. To ensure children‘s
access to education and to support a prolonged peace, UNICEF implemented the EEPCT programme.
The Liberian education system was shattered during the country's 14-year war, with the destruction of
schools, loss of staff, and disruption of children‘s education. Schools are currently being built, and teachers
trained and educational materials distributed, but recovery is not keeping pace with public-school enrolment;
between the 2005/2006 and 2008/9 school years, this saw a 48% increase, 33% of which was female.1 This
increase in enrolment was aided by the enforcement of the 2001 Education Act, which made primary school
education compulsory, and the 2006 abolition of school fees.2
The UNICEF Liberia Country Office was active throughout the war and reconstruction period. Within the
education sector, UNICEF responded through its Basic Education and Gender Equality (BEGE) programme.
The office collaborates closely with the Ministry of Education (MoE) and its programmes are closely aligned
with those of the MoE‘s three-year Liberian Primary Education Recovery Programme (LPERP).
The UNICEF country office began implementation of the EEPCT programme in 2007. Liberia is one of 39
countries participating in this US $201 million dollar partnership between UNICEF, the Government of the
Netherlands and the European Union/European Community. The office received a total of US $24,180,000
between 2007 and 2010, making it the largest beneficiary of the 39 programme countries.
In Liberia, the EEPCT programme is implemented through two different avenues, one directly implemented by
UNICEF and the other through the Liberian Education Pooled Fund (EPF). The programmes directly
supported by UNICEF include the continuation and expansion of the Accelerated Learning Programme (ALP),
building the capacity of the District Education Offices, printing supplementary readers, training of teachers and
building a Learning Along Borders for Living Across Boundaries (LAB4LAB) School. Additionally, UNICEF
provided technical assistance to two national school censuses and to development of policies on abolition of
school fees, and supported the creation of long-term educational planning documents such as the Education
Sector Plan.
1 Department of Research and Planning, Status of Education System in Liberia (2005/2006-2008/2009). Liberian Ministry of Education, 2010. 2 Scott, Aldophus, ‘Never to late to learn, 2008. [Access on September 21, 2010 from http://www.unicef.org/infobycountry/liberia_42354.html]
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 8
UNICEF also supported the creation of the US $16.25 million Liberian Education Pooled Fund, which in turn
supported three initiatives: (1) the rehabilitation of the Rural Teacher Training Institutes (RTTIs) through
renovation of teachers housing, logistical support and residential furniture; (2) the purchase and distribution of
1.2 million textbooks; and (3) the construction of 40 primary schools.3
In total, the UNICEF office has implemented roughly nine programmes as part of the EEPCT programme:
educational materials, instructional support, infrastructure support, technical support, Child Friendly Schools,
ALP, LAB4LAB/Talent Academies, EMIS and EPF.
Approach and Methodology
At the country level, the evaluation focused on EEPCT implementation as it relates to beneficiary and
education systems outcomes, as well as the quality and innovation involved in implementation. The evaluation
examined how the following EEPCT programme goals were implemented:
1. Improved quality of education response in emergencies and post-crisis transition countries.
2. Increased resilience of education in chronic crises, arrested development and deteriorating contexts.
3. Increased education-sector contribution to better Prediction, Prevention and Preparedness for
emergencies due to natural disasters and conflict.
4. Evidence-based policies, efficient operational strategies and fit-for-purpose financing instruments for
education in emergencies and post-crisis situations.
The country case studies were designed both to fit into the larger global assessment of the EEPCT
programme and to act as stand-alone studies of EEPCT implementation and outcomes in different national
contexts. This report represents a detailed description and analysis of EEPCT‘s operations, lessons learned
and discussion of ways forward in Liberia. The global report synthesizes findings on country, regional and
global levels.
The Liberian evaluation drew on both quantitative and qualitative methods. Data sources included document
reviews, interviews with key informants, surveys, and field visits; there were also focus-group discussions with
children, youth, parents, and educators. The above information was used to identify and link results in terms of
approaches, processes and outputs.
The evaluation faced limitations due to the timing, logistics, and the decentralization of information between
UNICEF and the MoE. The primary impact of these limitations was to decrease the numbers of schools visited
and the number of beneficiaries consulted. However, some of the challenges were mitigated by ensuring the
communities knew the time and needs of PREV team visits. The team also added an extra research-staff
member from the Columbia Group for Children in Adversity (CGCA), which carried out the evaluation.
Evaluation Findings
EEPCT has significantly improved the UNICEF country office‘s educational response and has already had a
significant impact on the education of Liberia‘s children.
Despite this overall verdict, there are both positive and negative findings from the evaluation. These are set
out below, and are explained in more detail in the main body of the report.
3 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one programme, one process. UNICEF: Monrovia. 2009. Pg.
26.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 9
(1) EEPCT progress was enhanced through the UNICEF Country Office‘s adherence to existing programme
plans.
The strength of EEPCT in Liberia is its integration, where possible, into pre-negotiated plans with the MoE.
The LCO specifically states that it was ‗not implementing EEPCT as a country programme but as a BEGE
supported by other donors as well.‘4 The LCO‘s adherence to these plans is illustrated through its integration
of the LAB4LAB initiative into the pre-existing plans for CFS and its choice not to implement the Talent
Academy because it did not fit within the existing plan.
(2) Proper planning and monitoring did not occur, due to EEPCT being an HQ-driven initiative.
Throughout the evaluation, LCO emphasized that ‗EEPCT was never meant to be a Liberia Country Office
Programme‘ as it was not part of the 2008-2012 Liberia Country Programme. EEPCT was viewed as an HQ-
driven initiative with limited involvement from the Country or Regional Offices.5
This was exemplified by the lack of clarity in the development and dissemination of EEPCT documentation
and understanding of how the programme would be integrated into UNICEF planning structures. As an
example, EEPCT funding came after the approval of the 2008-2012 UNICEF Liberia‘s Country Programme for
Children (CPC) which had been aligned with the MoE‘s LPERP and Education Sector Plan (ESP). The initial
focus on the regional LAB4LAB and Talent Academies initiatives was additionally confusing as it was not clear
whether EEPCT could support programmes outside these two initiatives.
The promotion of EEPCT was also cited as lacking awareness of the realities of field implementation. COs are
judged according to the implementation of the core programmes of Child Survival and Development (CSD),
BEGE and Child Protection, which are derived from government plans. Deviation from these programmes,
such as EEPCT, could affect how offices are evaluated.6
(3) The UNICEF Country Office was able to attract donor support by combining standard UNICEF
programming with LAB4LAB innovation.
The Country Office‘s combination of the innovative LAB4LAB programme with UNICEF‘s standard CFS model
helped leverage support from a significant donor. In 2010, the Japanese government contributed US $8.5
million to support the ‗construction of six LAB4LAB schools, 24 new MoE standard primary schools and the
renovation/expansion of 60 others to increase schooling for 22,500 children.‘7 The office stated that part of the
attraction for the Japanese was the application of CFS principles in this innovative new way.8 School
construction was seen as an entry point to incorporate CFS principles into standard practice.
(4) UNICEF and the Liberian government did not have sufficient staff and technical expertise to implement
the breadth and complexity of EEPCT programmes.
Capacity assessments of COs and implementing organizations should be conducted before programmes are
initiated. As an example, the Country Office stated that in implementing the EPF it ‗lacks enough technical
staff to be able to effectively support the larger macro-processes.‘9 The lack of a monitoring and evaluation
officer in the office has equally affected programme implementation. UNICEF implementing partners, such as
the MoE, should be subject to equal scrutiny. The constant reference to the limited capacity of the government
possibly should have led to more modest interventions with greater monitoring and technical support, as was
4 Email, 16 August 2010. 5 UNICEF, EEPCT Comments on Draft Report, Liberia, 2010. 6 Email, 16 August 2010. 7 UNICEF, The Project for support to Child-Friendly Schools Development, Project Proposal submitted to the Government of Japan, Ministry of Affairs,
2010. Pg. 9. 8Interview, New York, 17 November, 2010. 9 UNICEF,UNICEF Liberia: Annual Report 2007, Government of Liberia –UNICEF Programme of Cooperation. Monrovia. 2007.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 10
seen by the additional support provided in the textbook distribution and the finalization of school renovation.
As an example, OSI supported the textbook distribution and will conduct an evaluation of the process within
2010.
(5) Programme transitions should be gradual, with benchmarks to ensure relevance and sustainability.
If programme responsibilities are to be sustainable, there is a need to work within the timeframe and
capacities of beneficiary communities and organizations. For example, the revitalization of the CEO (County
Education Officers) Complexes should have included a longer transition period. As it stands, it was assumed
that the government would supply fuel to CEOs to conduct school monitoring, which it could not do. Equally,
emergency programmes, such as ALP, should have definitive points for review and adjustment. The Country
Office is hoping to begin developing best practices for programme transitions by conducting an ALP
evaluation and an assessment of out of school children in 2011. Results from these studies will inform the
MoE‘s decision regarding the ALP. With regard to the LAB4LAB initiative, UNICEF will support the first year
of the school‘s operation to determine its operating costs.
(6) Monitoring of programmes does not draw upon communities and is poorly supported through programme
transitions.
Program monitoring is the responsibility of all parties involved in program implementation, including children.
Programme planning should include sufficient long-term resources to develop a culture of monitoring and
evaluation internally but also within the MoE and communities which they serve. Children and communities
monitored textbook distribution and maintenance through the implementation of a textbook policy; similarly,
communities were mobilized to monitor ALP programmes in their communities. Building upon these examples,
there should be further community involvement in school construction and supply distribution. Communities
should be involved in the selection of contractors for school construction and should be informed about
upcoming supply distributions via radio transmissions, while materials received should be signed for by a
student, parent and the school principal.
(7) Programmes lack components such as assessments, baselines and strategies to link improvements to
policy change through advocacy.
Few of the EEPCT-funded programmes were based on situational assessments and baselines; will hinder
evaluation the impact of these programmes and advocacy for change. For example, the lack of a situational
assessment and baseline for the LAB4LAB/CFS programme will hinder evaluation and thus the
communication of the programme‘s benefits to communities and government. Advocacy strategies also
need to be included in the development of targeted products. For example, while a documentation exercise for
the EPF was undertaken, the target audience for this product was unclear.
(8) Decentralization supports the implementation of other interventions.
The EEPCT‘s capacity building of the CEO Centres was described by the Country Offices as part of their
government capacity building initiatives. The evaluation team would identify it as supporting the resilience of
the education system. This support is in line with MoE‘s plans for decentralisation, which it hopes will increase
service efficiency, transparency, accountability and responsiveness.10
EEPCT‘s support for the rehabilitation
of CEO complexes and provision of motorbikes was an initial step complemented by training, data collection
and production of the National School Census. The training of 92 DEOs and 14 CEOs to perform the census
built MoE capacity and created the opportunity for links between the community level (schools), the county
level (CEOs and DEOs) and the National MoE.11
This capacity is important because it improves monitoring
10 World Bank. Issues in program design: education and decentralization. Decentralization Thematic Group. [ Accessed on September 18, 2010 at http://www.ciesin.columbia.edu/decentralization/English/Issues/Education.html] 11 UNICEF. UNICEF Liberia: Annual Report 2009. Government of Liberia-UNICEF Programme of Cooperation. Monrovia. 2010.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 11
and evaluation at the community level, including more efficient and timely data collection and improved
programme management.
(9) Significant participation of children and communities was lacking within EEPCT programming.
A significant portion of EEPCT work was ‗upstream‘ in supporting the MoE. This strong upstream approach
did not bring communities and children into the decision making and dialogue. By incorporating children‘s
voices into programme design and implementation, the efficiency and quality of EEPCT programming could
be enhanced. Children have valuable insights and energy to contribute in post-crisis situations. At both the
community and national level, children can help advocate change on issues such as gender-based violence in
schools. This should be an intentional area of focus in moving forward with CFS schools.
(10) Emergency Education needs a structured approach that takes into account natural disasters and conflict.
With the on-going tensions in Guinea, UNICEF and the MoE would benefit from a more structured approach to
the implementation of emergency education responses. The Country Office chose not to implement any
Disaster Risk Reduction Programmes as part of the EEPCT programme. Conflict in surrounding countries and
the potential resurgence of conflict overshadows the risk of natural disasters in the country. When asked
about disaster preparedness, one community-level key informant expressed that children should be taught to
flee if war came again.12
As mentioned above, Liberia also faces minor threats of natural disasters such as
floods and storms.13
The Way Forward: Recommendations
Government/MoE:
Evaluate the relevance the ALP programme. This can be supported by UNICEF‘s ALP evaluation and
the UNESCO/UNICEF assessment of out-of-school children.
Establish an emergency education/preparedness section in collaboration with the IASC Education
Cluster Leads (UNICEF and Save the Children) based upon the FTI‘s recommendation.
Remain engaged with the implementation of CFS to identify best practices to adopt as national policy.
Establish, and identify resources for, a monitoring plan for CEOs.
UNICEF Headquarters:
Lead the development of programme learning workshops around specific emergency education
programme components, such as ALP. The results of these workshops could then be developed into
a programme plan to make promising practices into best practices through thorough assessment,
baselines and evaluations.
Reinforce the application of the programme cycle at the global, regional and country level through
inclusion in workshops, review of proposals and newly developed programme documents. This
should emphasize the importance of tying evaluations to advocacy.
12 Interview. Liberia. 1 August 2010. 13Preventionweb. Liberia profile (Accessed September 21, 2010 from (http://www.preventionweb.net/english/countries/statistics/risk.php?iso=lbr) 2010.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 12
UNICEF HQ should lead a lessons learned review of the EPF to identify when such a strategy should
be applied but also the implications for the CO regarding staffing and technical support. If Pooled
Funds become a common intervention UNICEF should develop in-house capacity to provide technical
support.
Adapt the CFS model to be used in emergencies.
Identify resources to continue the LAB4LAB and Talent Academy programmes to an established
transition point. Both of these initiatives are high level, regional/global initiatives. UNICEF is under
some obligation to see them through to commonly identified transition point. This would include
updating the Global Framework of LAB4LAB and Talent Academies in collaboration with the Regional
Offices and COs. Such a process would include a sociological and political survey, survey,
identification of interventions and indicators, and decisions regarding how LAB4LAB/Talent
Academies will proceed as a regional/global endeavour.
Regional Offices
Identify and provide the support needed by EEPCT countries through existing personnel, short term
contracts or consultancies. Participate actively in the development of long-term LAB4LAB and Talent
Academy planning.
UNICEF Liberia Country Office:
Review opportunities for children‘s participation in decision making and programme implementation.
Specifically, children‘s involvement should be integrated into the expansion of the CFS programme.
Initiate a long-term implementation and transition plans for the Ganta LAB4LAB School. The LCO
needs to develop a plan for how continued costs will be covered based upon the first year operational
costs (including electricity, computer maintenance, radio license, etc). New partners, such as Search
for Common Ground/Talking Drum to lead radio programming, should be identified as part of this
planning process.
Support the MOE to make informed decisions regarding the future of the ALP including exploration of
options such as youth/adult literacy programming and early childhood education.
Review the implementation of CFS programme to include situational assessments as well as baseline
and evaluation strategies that will provide data which can be used for advocacy.
Review the staffing that is needed to ensure sufficient monitoring and transitioning of programmes.
Possible areas of focus include the clustering of activities around (1) ALP – conducting the ALP
Evaluation, Out of School Children Assessment and coordination with the government – and (2)
CFS/LAB4LAB programme – implementation of the LAB4LAB School, coordination of additional
schools with Japanese funding and the roll-out of the CFS programme. These positions are important
for the continuation for the programme after EEPCT.
Identify and address missing technical support that has resulted from the absence of a monitoring and
evaluation officer. Short-term consultants could be hired to support assessments, baseline and the
development/application of indicators.
Review UNICEF‘s strategy to support the Ministry of Education to monitor schools, distribution and
data collection.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 13
Develop a multi-year CFS advocacy strategy to accompany the 2008-2012 Education sector plan.
The CFS programme should be a product of a solid assessment, and development of indicators and a
programme baseline. Moreover, this should be coordinated with the development of messages and
products for targeted audiences.
Document the ALP programme including programme structure, guidelines, monitoring structures and
government involvement.
Plan a dissemination and advocacy strategy for the ALP Evaluation and the UNICEF/UNESCO
assessment. Messages should be targeted for communities, the Liberian government, UNICEF and
the international community.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 14
1.0 INTRODUCTION
1.1 Overview
UNICEF‘s Education in Emergencies and Post-Crisis Transition Programme (EEPCT) began in 2006 as a
four- (later extended to five-) year, US $201 million dollar partnership between UNICEF and the Government
of the Netherlands with €4 million (about $5.8 million) additional support contributed by the European
Commission. The EEPCT programme is intended as a strategic intervention in support of the Millennium
Development Goals (MGDs) and the Education For All (EFA) movement, and both increases institutional
capacity and provides direct programme support. EEPCT works in 39 countries, and also supports global
initiatives such as the Inter-Agency Standing Committee‘s Education Cluster and the Inter-Agency Network for
Education in Emergencies (INEE).
Monitoring of the EEPCT‘s implementation was included within the project‘s design. In October 2009, UNICEF
commissioned an independent programme review and evaluability Study (PRES) to assess EEPCT‘s
performance so far. The evaluability component looked at a number of priority areas for future evaluation,
including: access to quality education; education system development; disaster risk reduction (DRR); and
conflict-related impacts—and offered guidance on how these components, as well as EEPCT as a whole,
could be evaluated. This progress evaluation addresses the PRES findings and recommendations. It
examines the implementation of the EEPCT programme to date, globally and in more detail in six case-study
countries. The evaluation‘s objectives include:
1. Taking stock of the first three years of implementation to determine, as systematically and objectively
as possible, the EEPCT Programme‗s relevance/appropriateness, effectiveness, efficiency,
coherence/coordination, and, as far as possible, impact and sustainability in relation to its objectives;
2. Evaluating both the intermediate results achieved and the processes set in motion, to assess
EEPCT‘s added value for the education sector in emergencies and post-crisis transitions, as well as
UNICEF‘s specific contribution to the programme;
3. Gathering relevant and applicable lessons learned on education interventions in emergencies,
transition, and fragility-affected contexts; and,
4. Making recommendations for future programming and for future decision-making by UNICEF
headquarters, regional and country offices, and relevant stakeholders.
The six countries to be examined as case studies of EEPCT implementation were chosen by UNICEF‘s
Evaluation Office. This report examines EEPCT implementation in Liberia, included because it is the largest
beneficiary of EEPCT funding and also has an active fit-for-purpose financing mechanism. Findings of this
case study also contributed to the overall report on implementation of the EEPCT globally.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 15
1.2 Country Context
Liberia‘s violent internal conflict ended in 2003. The
14-year conflict devastated the country‘s
infrastructure and caused economic collapse. A
significant portion of the population was internally
displaced or became refugees. Liberia is one of the
poorest countries in Africa;14
84% of the population
live on less than US$1.25 a day, and 85% of the
population work in the informal sector.15
Of children
under the age of five, 26% are underweight, and 36%
do not use an improved water source.16
Liberia ranks
169 out of the 182 countries in the UN Human
Development Index.17
In 2008, Liberia initiated the
2008-2011 Poverty Reduction Strategy (PRS) to align
itself with the MDGs. Data about the situation in
Liberia is limited but it is clear that only a few of the
MDGs will be met.18
The extended war has made Liberia a challenging
environment in which to implement programmes.
The war had a significant impact on infrastructure
such as power, telephone and internet – and building
maintenance. Additionally, there are too few qualified
Liberian staff and contractors for reconstruction.
Liberia‘s limited road system was especially affected as
bridges were destroyed and roads fell into disrepair. All of this has made communication, travel and provision
of programme support difficult.
1.3 Educational Context
The war shattered The Liberian education system; 48% of schools were either destroyed, or damaged enough
to not be fully useable.19
This was coupled with the looting of roofing, school furniture and educational
materials, and the destruction of the two operational Rural Teacher Training Institutes (RTTIs) in Bong and
Maryland Counties.
14 Government of the Republic of Liberia. 2008 National population and housing census: preliminary results, Institute of Statistics and Geo-Information Services, 2008.pg.2. 15 Department of State, Background Note on Liberia, 2010. U.S. government. [Accessed by on September 18, 2010 at http://www.state.gov/r/pa/ei/bgn/6618.htm] 16 UNDP, Human Development Report, 2010. [Accessed online September 18, 2010 at http://hdrstats.undp.org/en/countries/data_sheets/cty_ds_LBR.html] 17 UNDP. Human Development Report 2009 – Liberia, 2009. [Accessed online September 18, 2010. At http://hdrstats.undp.org/en/countries/country_fact_sheets/cty_fs_LBR.html] 18 Republic of Liberia, Liberia’s progress towards the Millennium Development Goals 2008, Monrovia. 2008. 19 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010.
Figure 1: Map of Liberia
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 16
During the war, when and where security permitted, the Liberian Ministry of Education (MoE), UN, NGOs and
donors sought to maintain the education system. At the same time, UN agencies and NGOs also established
education programmes in the refugee camps along the border in Guinea and Côte d‘Ivoire. Students who fled
to urban Montserrado County or the refugee camps in Guinea and Côte d‘Ivoire were able maintain their
education and integrate into the Liberian school system upon their return.
Replacement and rehabilitation of damaged and destroyed schools continues, but slowly. Between the
2006/2007 and 2008/2009 school years, the number of operational schools increased from 865 to 1,282.21
The MoE reports that 59% of schools still require repair or reconstruction, and only 15% have water.22
There
is a lack of qualified teachers; during the war, both teachers
and educational administrators were killed, forced to leave the
profession, migrated to urban areas, or sought refuge in
another countries. In areas and times of stability, volunteer
teachers without formal training or experience entered the
classrooms. These teachers continue to be the foundation of
Liberia‘s education system, in which only 40% of all teachers
are considered qualified.23
The war also affected the ratio of
male to female teachers, with only 11% of the workforce now
women.24
However, between the 2005/2006 and 2008/9 school years,
Liberia saw an increase in schooling at all levels. Table 1
shows this increase across the public school system for this
period. Overall, there has been a 48% increase in public school
enrolment, of which 33% was among girls.25
This increase was
aided by the School Law ratified in 2002, which abolished school fees for primary schools in 2006 and made
schooling at this level compulsory.26
However, the education system is severely stressed and cannot
adequately accommodate this influx of new students. In addition, some children remain out of school as they
are unable to pay unofficial school support ‗fees‘, as well as the costs of school supplies and uniforms.
Overage students are common in primary school classrooms as a consequence of the war coupled with a
culture of late enrolment of children. According to the MoE, primary school enrolment is only for children ages
6-11; but 63% of primary school students in the 2007/2008 school year were over 11.27
Class sizes are
reported to exceed 60 students.28
Yet improvements are being made, as , according to the Fast Track
Initiative (FTI): ‗growth in gross enrolment in primary school has decreased to 89% (85% girls and 93% boys)
from 155% in 2006 and net enrolment ratios are 42%29 up from 25% in 2006.‘
30 Secondary gross enrolment is
33%, with 29% girls and 37% boys.31
A USAID assessment of youth in 2008 found that 85% of those sampled wanted to further their education and
build skills. The greatest request (39%) was for vocational skills, while 27% requested literacy and numeracy
20 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010. 21 Ibid. 22 Ibid. 23 Ibid. 24 Ibid. 25 Ibid. 26UNICEF, EEPCT Comments on Draft Report, 2010. 27 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010. 28 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010. 29 UNICEF, EEPCT Comments on Draft Report, 2010. 30 Fast Track Initiative. Recovery of the Education Sector in Liberia. 2010. [Access on September 18, 2010 from http://www.educationfasttrack.org/newsroom/focus-on/fti-annual-report/recovery-of-the-primary-education-sector-in-liberia/] 31 UNICEF, Self assessment, 2010.
Table 1: Liberia Public Enrolment Data between 2005/6 and 2008/9
20
Total number of student in
public schools
% increase between
2005/6 and 2008/9
Pre-primary 335,600
69%
Primary 340,869 31%
Accelerated Learning Programme
60,405 55%
Junior High 46,176 36%
Senior High 20,335 94%
Table 1
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 17
skills.32
Children whose education was interrupted during the crisis have now grown up illiterate. Although
progress has been made, Liberia is unlikely to achieve the MDG and EFA goal of universal primary
education.33
UNICEF‘s programmes are closely aligned with those of the MoE‘s Liberian Primary Education Recovery
Programme (LPERP). The LPERP is a three-year programme focused on the rehabilitation of primary
education. This programme, which began in 2007, has been the framework for educational reconstruction.
During its first year, Liberia was admitted to the EFA/Fast Track Initiative (EFA/FTI) Partnership but was not
granted financial support. However, with the assistance of the Education Programme Development Fund
(EPDF), the Ministry has been preparing a ten-year Education Sector Plan (ESP) based on an exhaustive
Country Status Report (CSR) and consultation process that included all education stakeholders. This
consultative development has allowed for building consensus between the government and donors.
The ESP outlines the short-, medium-, and long-term government strategies for the education sector and is
accompanied by a three-year implementation plan. A sector policy letter lays out the key government
priorities, main policy reforms and target indicators to be met over the ten-year period of the ESP. This letter
has been endorsed by both the MoE and the Ministry of Finance.
In terms of government spending on education, it is estimated that since 2004, average real spending on
education has increased by 26 percent per year.35
This increase is similar to other areas of government
expenditure. In 2010, Liberia qualified for US $40 million-worth of funding from the EFA/FTI Catalytic Fund.36
1.4 EEPCT Programme
Since 2007, UNICEF-Liberia has received US$24.2 million in EEPCT support (Table 2). In Liberia, the EEPCT
programme is implemented through two different
avenues. First, it is directly implemented within UNICEF
Education Section‘s Basic Education and Gender
Equality (BEGE) programme. The second avenue is
through support of the MOE‘s Liberia Education Pooled
Fund (EPF).
The primary EEPCT programme implemented within the
BEGE is the Accelerated Learning Programme (ALP).
Starting in 1998, ALP was an ‗emergency‘ intervention to
enable over-age children to re-enter or finish primary
school in three years instead of six. With support from
UNICEF, the MoE implements the programme by
providing learning, recreational, and teaching materials,
furniture and refresher courses for the local teachers.
UNICEF is the MoE‘s largest supporter for the ALP. In
2009, the UNICEF-supported ALP was expanded from
11 to all 15 counties in the country.37
In 2010, 30,78538
students were supported by UNICEF within the government‘s ALP programme, which accounted for 41% of
the estimated 76,000 students nationally.39
32 Coyne, Geoff, et al, Accelerated Learning Program for Positive Living and United Service: Mid-term Evaluation, USAID. 2008. 33 Republic of Liberia, Liberia’s progress towards the Millennium Development Goals 2008. 2008. 34 UNICEF, Self-assessment, 2010. 35 World Bank and Government of Liberia, Public Expenditure Management and Financial Accountability Review (PEMFAR), 2009. 36Schmidt, Caroline and Aleesha Taylor. Liberia’s Education Pooled Fund: a case for private foundation engagement in post-conflict education
recovery. Open Society Institute. 2010. Pg 2. 37 UNICEF, UNICEF Liberia: Annual Report 2009, Government of Liberia-UNICEF Programme of Cooperation, Monrovia, 2010. 38 Ibid.
Table 2: Liberia EEPCT Support34
Year
Receiv
ed
BEGE
Support
Education
Pooled
Fund
(EPF)
Total
EEPCT
Support
2007 2,000,000 2,000,000
2008 1,500,000 12,000,000 13,500,000
2009 1,500,000
2,500,000 1,000,000
2010 2,980,000 3,200,000 6,180,000
TOTAL 8,980,000 15,200,000 24,180,000
Table 2
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 18
EEPCT also enhanced the operational capacity of the County Education Offices. The training of the County
Education Officers (CEOs), District Education Officers (DEOs) and enumerators supported the development
of the national Education Management Information System (EMIS) to produce the 2007/2008 and 2008/9
Census Reports.40,41
The data from this report directly supported the implementation of LPERP.
With support from the EEPCT programme, the UNICEF country office chose to participate in the regional
Learning Along Borders for Living Across Boundaries (LAB4LAB) programme. The LAB4LAB programme
seeks to use education regionally as a means of addressing developmental challenges to peace posed
throughout the Mano River countries of Guinea, Côte d‘Ivoire and Liberia. Built on land donated by the
community, the LCO initiated construction of the LAB4LAB school in Ganta, Nimba County, in 2009. The
school contains advanced features such as solar panels, computers and a radio transmitter to facilitate cross-
border communications. The construction of the school has been challenging due to security issues, poor
roads and a long rainy season.
In addition to the above efforts, EEPCT has supported infrastructure development through contributing
consultants to support the design of the pooled fund (2007), development of the -sector-wide approach
(2007), the LPERP implementation plans (2008), the Education Sector Plan (ESP) (2009), and the School Fee
Abolishment Policy (2009). In the case of the ESP, the EEPCT funded an additional consultant in coordinating
inputs from partners for the application for the Catalytic Fund.
EEPCT has also funded the training of 250 school teachers, and helped buy over 1 million textbooks and
readers for grades one, two and three and 20,000 school chairs. Most recently EEPCT funds were used to
support supported the MOE to establish a Child-Friendly Schools programme (CFS). To maintain program
coherence, the LCO integrated the LAB4LAB School under its existing BEGE plans for CFS.42
However, the largest portion of EEPCT support in Liberia has gone to the creation and initial funding of the
EPF (the pooled fund), which is the second avenue for educational support for the LPERP. which was
established with EEPCT and Open Society Institute (OSI) funding as well as UNICEF technical support. The
EPF is intended to assist development of a ‗fit-for-purpose‘ financing system, to implement an education
reform programme while cultivating national capacity (in line with EEPCT Goal Four). However, according to
the 2009 Consolidated Report to the Government of the Netherlands, UNICEF Headquarters saw the EPF as
a programme while the UNICEF country office felt it was another funding mechanism.43
The MoE and Ministry of Finance oversee the EPF, assisted by an advisory board composed of the fund‘s
donors, UNICEF and the Open Society Institute. The EPF was designed with short-term technical assistance
from UNICEF Headquarters working with the MoE from November 2007 until the launch of the EPF on May
22, 2008. Headquarters assistance was also provided to operationalise the Fund through a sector-wide
approach.44
Since its inception, the EPF has supported three initiatives: the re-establishment of the three Rural Teacher
Training Institutes (RTTIs); the distribution of 1.2 million textbooks; and the construction of 40 schools
throughout the country.
39 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010. 40 UNICEF, Progress Report for UNICEF’s Education in Emergencies and Post-Crisis Transition Programme. Monrovia. 2009. 41 Ibid. 42 Interview, New York, 16 November 2010. 43 UNICEF, EEPCT Comments on Draft Report, 2010. 44 Ibid.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 19
2. METHODOLOGY
The methodology used by the evaluation is described briefly in this section, and in more detail in Annex 1.
The evaluation team consisted of two co-leaders and a biostatistician, a finance administrator, and a
programme officer. Besides the six case-study countries, the co-leaders and support staff were responsible for
implementation of global evaluation activities, as well as final data analysis, synthesis and report writing. The
Liberia case study team consisted of two co-team leaders, a Columbia University Research Associate and
three national researchers.
2.1 Purpose and methods
This evaluation is a progress evaluation, or PREV; its purpose is to determine the impact of EEPCT so far,
identify lessons learned and make recommendations for strengthening the programme in the years to come.
Within this context, the evaluation seeks to achieve three inter-related objectives:
Provide an outcome-indicative impact analysis of the EEPCT Programme (2006-2009)
Examine the programme against DAC evaluation criteria as applied to education in emergencies, transitions
and fragility
Provide evidence-based conclusions, lessons learned and recommendations.
EEPCT was also examined with reference to five cross-cutting issues. These included ender; rights-based
programming; disaster risk reduction; and sensitivity to conflict and fragility. The fifth was accountability,
monitoring, evaluation and learning.
At the country level, the evaluation examined how EEPCT‘s goals were translated and adapted to country
contexts and why. In addition, the PREV sought to determine the extent to which global and regional
strategies and outputs are supporting change in the six case-study countries. The country case studies were
designed to both fit into the larger global assessment of the EEPCT programme and to act as stand-alone
studies of EEPCT in different national contexts. The research plan allowed for six to eight weeks per country.
A GCA-Inter-Agency Network for Emergency Education (INEE) Memo of Understanding also enabled close
collaboration with education agencies and their staff in each country where a case study was conducted.
The evaluation used both quantitative and qualitative methods, using the results of one method to support the
informed development of others. This approach enabled both in-depth exploration of methods and innovation
as well as the collection of comparable data across countries. A literature review ranged from government
documents to publications produced by local and international NGOs to academic literature. The evaluation
also reviewed financial allocations relative to the four programme goals and the cost-effectiveness of EEPCT.
The evaluation included individual interviews with relevant UNICEF staff, government officials, NGO staff and
other key stakeholders at the community, regional and global levels. In case-study countries, the team
consulted key stakeholders; these were identified in consultation with the UNICEF country offices, and
included key government and civil-society actors and others who were most knowledgeable of the education
sector in general and the EEPCT programme in particular.
The research team employed participatory methods with beneficiaries—children, parents, educators and local
communities members—to ensure an upstream flow of findings and recommendations. Field visits also
documented the results at the country level of global and regional contributions (INEE Minimum Standards for
Education in Emergencies, Chronic Crisis and Early Reconstruction, Education Cluster coordination and
Regional Office technical support). There were also focus-group discussions that used different tools to
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 20
capture participants‘ perceptions of EEPCT; these tools included the DAC score card, Child-Friendly Schools
checklists, and participative ranking methodology (PRM). These are discussed in more depth in annex 1.
2.2 Field-site and sample selection
Field-visit sites in Liberia were selected both at random and through planned sampling processes. After
consultations with the UNICEF country office and the national research team, Monterrado and Nimba
Counties were chosen for field visits, as the majority of the country‘s students are enrolled in these two
counties. The country office was also consulted on the EEPCT programme components.
When these were identified, the locations of programme sites were determined. Because the textbook
programme that was administered as part of the LPERP and supported by the EPF was nationalized,
comparison groups of public schools not benefitting from the programme were not possible. As the
accelerated learning programme (ALP) was implemented in all counties but not all schools, a random
selection of schools was made from a list of all UNICEF-supported ALP programmes. Sites from this selection
were then chosen based on accessibility and distance.
The sample of stakeholders was selected at national and sub-national level, including MoE staff, government
officials, county and district education officers, and international and national NGO staff. At community level,
the stakeholders included students, teachers and parents. Fuller details of the focus-group participants may
be found in annex 1.
2.3 Limitations
The evaluation team took multiple steps to ensure the methods used in Liberia were consistent with those of
the research teams in the other case-study countries. To achieve this, there were several components
common to each country, including an introductory PowerPoint presentation for stakeholder consultations,
user guides for each tool and instructions for all data collection methods; central training; and a consistent
approach to data entry, including creation of a singular entry form, and to supervision of research teams
during the fieldwork.
However, the lack of global clarity on multiple terms within the sector limited a theory based approach to this
evaluation. This limitation includes the meaning of terms (emergency, post-crisis, transition, quality education,
resilient education system). For these reasons, the CGCA created blogs that sought to clarify these terms.
Despite this, development of definitions, terminology and concepts continued to be difficult for focus-group
participants at the community level, due to differences in education, insufficient English-language abilities or
lack of similar words within vernacular language. Also, no programme baseline was established, which made
it impossible to report on percentage changes in goal-area outcomes. The evaluation team addressed some
of these concerns through the use of retrospective baselines.
In addition to the above, several other factors had an impact on the evaluation at field level in Liberia. Few key
informants or focus-group participants knew of EEPCT. There were also some practical problems; for
example, the evaluation coincided with school holidays, making it harder to gather participants from all cohorts
for focus groups at community level (this was mitigated by contacting regional officials and community
members prior to the evaluation visit). Also, UNICEF did not hold all the information regarding programme
sites, locations and participants, so it had to be obtained from different parties at the MoE, slowing the
planning process. Finally, in one country, the county and district education officers were at a workshop abroad
at the time of the evaluation team‘s mission.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 21
3. EVALUATION FINDINGS
3.1 Overview: Results Analysis
The progress of EEPCT implementation was estimated both overall, and through the individual programmes
against three of the four EEPCT goals (in the case of Goal Three, the UNICEF office was not implementing
any activities). The evaluation found that UNICEF Liberia‘s EEPCT programme was having a significant
impact on reconstruction of Liberia‘s education system. This section presents the evaluation‘s conclusions on
EEPCT in Liberia in detail – first by the four EEPCT goals, and then with reference to overarching areas
(including operations and management, and performance on cross-cutting issues such as gender).
As will be seen, EECPT is having an attributable impact under Goal One (to increase Liberian children‘s
access to quality education). Since its inception, EEPCT has increased the number of textbooks in schools
nationwide; trained 250 teachers to C-level certification; conducted CFS training; supported the completion of
19 schools, including the LAB4LAB School; and provided important technical support to the MoE, such as
supporting the drafting the School-Fee Abolition Policy.
Contrary to what is reported in the EEPCT global donor report, the UNICEF Liberia country office did not see
its programme as contributing to Goal 2, as ‗Liberia does not have chronic emergencies, no arrested
development or deteriorating contexts.‘ 45
All of the programmes attributed to resilience were carried out
under Goal 3. The resilience of the education system was supported through the continuation and expansion
of the ALP to provide direct support to 30,785 students.46
The regional LAB4LAB regional initiative in which
the country office participated is another means of supporting resiliency and peace within Liberia as well as in
the Mano River Region.
The EECPT programme supported two national school census reports and better planning and programme
implementation within UNICEF, the MoE and other education stakeholders. The EEPCT has also supported
the development and continuation of the Liberian Education Pooled Fund. This fund‘s dual purposes are to
support government capacity building and to provide programme support. Overall, the programme has fallen
short of these goals due to an over-estimation of the capacity of the government to implement large projects.
Donors who have conducted external reviews of the fund have decided not to fund it.47
The research team found that while EEPCT has made strides in several of the goal areas, there are still
certain limitations to its work in Liberia. The following sections provide a detailed description and analysis of
the particular goal areas in which EEPCT funds and capacity have been invested.
3.2 Goal One: Improved quality of education response in emergencies and post-crisis
transition countries
The UNICEF country office was able to provide information for four out of the five indicators for Goal 1. One
indicator, regarding primary completion rate, could not be calculated due to lack of information.
UNICEF in Liberia has achieved the EEPCT goal of 80% of the children being enrolled if the indicator is
applied to Gross Enrolment Ratio (GER). The indicator is not achieved if applied to Net Enrolment Ratio
(NER).48
EEPCT contributed directly to the increased number through the construction of the LAB4LAB
School, and more than 800 permanent learning spaces were created under the EFP.
45 UNICEF, EEPCT Comments on Draft Report, 2010 46 UNICEF. UNICEF Liberia: Annual Report 2009. Government of Liberia-UNICEF Programme of Cooperation. Monrovia. 2010. 47 Interview. 5 August, 2010. 48
UNICEF, [Results Framework Response], 2010.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 22
Training and educational materials
In 2007, UNICEF distributed 140,000 Grade One and ALP supplementary learners‘ kits.49
In 2008, the EPF
supported the procurement and distribution of 1.2 million English, Math and General Studies textbooks with
teacher‘s guides, and trained principals and teachers on systems for their retention and use. According to
UNICEF, this distribution has reduced the student-to-textbook ratio from 27:1 prior to the programme to 2:1.50
However, as school was not in session during the evaluation, this programme was assessed through literature
review, focus groups and observation. For this reason, the evaluation team was not able identify teachers
trained, observe classes or accurately measure the student-to-textbook ratio. A key informant within the MoE
stated that the textbooks had been procured but could not attest to their distribution as no end-user monitoring
system was in place.51
Other informants reported problems with the capacity of county and district education
officers to monitor the equitable distribution of texts. The Open Society Institute contributed one person to
support the procurement and distribution of the 1.2 million textbooks;52
but it is doubtful this was sufficient.
Assessing impact of the textbooks through the focus-group discussions was difficult as participants did not
distinguish between the textbooks themselves and their effect on the students that used them. However, four
out of the seven discussions conducted with children and youth cited the positive aspects of textbooks by
saying that they were less afraid, their attendance was improved and that the books enabled them to teach
other students. An important finding was that the textbooks were a sustainable input since they were
distributed with training and copies of a Textbook Policy and Instructional Material Management Handbook.
Parents, in one out of the two discussions conducted with them, stated the importance of being involved in the
distribution of the textbooks and signing for them during distribution. However, parents also reported this
process as a weakness, since they were monetarily responsible if the books were damaged or not returned.
Three out of five groups of youth also cited this as a weakness.
Infrastructure
Observation, literature review, key-informant interviews and focus groups were used to evaluate the progress
of this programme. The EPF supported the MoE in constructing 40 schools across the country. This project
was divided between two contactors at the time of evaluation data collection. Only 18 of the schools, which
were all contracted to one contractor, had been completed. The remainder were all between 40-45%
completed due to contractual issues between the MoE53
and the other contractor regarding overheads. The
completion of these schools awaited the MoE‘s selection of an alternative contractor.
The evaluation team visited four out of the 18 completed schools. The school buildings were examined to
evaluate whether or not selected criteria from the UNICEF CFS Manual were met.54
Two of the schools met
four of the 12 selected criteria; the other two met only three criteria. However, the building design was outside
UNICEF‘s control, as MoE designs and standards were used. The UNICEF country office was clear that these
schools were not planned to be ‗Child Friendly‘.
The team also visited one of the five rehabilitated County Education Office (CEO) complexes. EEPCT‘s
revitalization of these complexes assisted the government‘s process of decentralization. The rehabilitation
included, where appropriate, warehouses, seminar rooms with kitchens, wells with water towers and
generators/housing for electricity. This support enabled the complexes to act as hubs for the DEOs and their
49 UNICEF, UNICEF Liberia: G3001 Netherlands, UNICEF Liberia Country Office: Monrovia, 2007. 50 UNICEF. Progress Evaluation of the Education in Emergencies and Post Crisis Programme (PREV). Liberia: UNICEF. 2009. 51 Interview. 12 July, 2010. Liberia. 52 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. pg 16 53 Interview. Monrovia, Liberia. 27 July, 2010. 54 UNICEF, Child Friendly Schools Manual. New York. 2009. Criteria included the following: These criteria included the following: whether or not the school appeared child friendly; if classrooms were easily exited in emergency; all classrooms have windows; separate spaces are provided for teachers and administrators; teachers and administrator location allows for monitoring students; availability of water on school grounds; separate latrines for boys and girls; locks on latrine doors; appropriate ratio of latrines to boy and girl students; separate latrines for teachers; separate space with soap and water for hand washing; and presence of a disaster risk reduction plan.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 23
monitoring. The complexes were also reported to decrease operational costs.55
USAID and the EU have
recognized their importance and have started to provide additional support.56
In the two sub-national
government focus-group discussions conducted, one ranked the change in monitoring and evaluation as
much better (5) while the other ranked it as better (4). The latter group explained that the lower ranking was
partly due to the fact that while motorcycles had been provided, the Ministry decreased the amount of fuel and
maintenance provided, curtailing their monitoring availability.
Under the LPERP, the EEPCT programme combined resources with other partners, including USAID, and
planned to assist the re-establishment of the three Rural Teacher Training Institutes (RTTIs) in Webo, Zorzor
and Kakata (the RTTI in Webo was eventually not supported due to delays in construction). However, the EPF
contributions to the RTTIs were not clear and the team were not able to verify them. According to the UNICEF
2008 donor report, classroom and library furniture and equipment were provided. 57
However, according to
Schmidt‘s report on the EPF, the support included ‗transport, cookers, residential furniture and other
requirements‘58
and the procurement of ‗three buses, three pick‐ups and three jeeps.‘59
Additional learning
materials were provided under the second tranche of EEPCT funding. As school was not in session, the team
could not confirm this support or identify provided teaching and learning materials. However, a USAID mid-
term evaluation noted that these items were non-existent or in poor condition.60
The sustainability of the RTTIs was not considered. From the literature review, the MoE had assumed that the
recurrent costs of US $2 million per year would be covered by project partners; however, the MoE request for
support was turned down by the EPF. USAID and other organizations have come forward to assist the
Government in supporting these institutes. However, USAID‘s mid-term evaluation of the programme found
the RTTIs underused and unable to attract qualified trainees. It also highlighted the lack of female trainees
(only 44 out of 478 candidates in 2009).61
Child-Friendly Schools Programme (CFS)
There has been training on the CFS Framework at the LAB4LAB School, for its 16 teachers as well as those
from the surrounding area. There was also later training for 55 trainers-of-trainers. In 2010, UNICEF held
sensitization workshops to support full implementation in 2011.62
This concept of implementing CFSs from the
start of construction onwards has attracted the attention of the Japanese government and has garnered
support of US $8.5 million to take the concept forward. The proposal will support ‗construction of six LAB4LAB
schools and 24 new MoE standard primary schools, and the renovation/expansion of 60 others to increase
schooling for 22,500 children.‘63
However, no baselines and indicators have been developed to monitor
programme implementation or assist in evaluation of project outcomes and impacts.
Coordination
In relation to coordination, the EEPCT programme established an Education Development Partners
Secretariat, to coordinate donor support to the sector within the sector-wide approach. The Secretariat hosted
bi-weekly partners‘ meetings, ensuring the alignment of partner support to government education priorities.
The Secretariat organized the first Education Sector Review in June 2009, and was instrumental in
55 UNICEF, EEPCT Comments on Draft Report, 2010. 56 UNICEF. Self-assessment. Monrovia. 2010. 57 UNICEF, Progress Report for UNICEF’s Education in Emergencies and Post-Crisis Transition Programme. Monrovia. 2009. pg. 12. 58 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg.
44. 59 Ibid. 47. 60 Coyne, G., et al. Accelerated Learning Program for Positive Living and United Service: Mid-term Evaluation, USAID, 2008. pg.9 61 Coyne, G., et al. Accelerated Learning Program for Positive Living and United Service: Mid-term Evaluation, USAID. 62 UNICEF. Utilization of Donor Funds and Expenditures –EEPCT. Monrovia. 2010. 63 UNICEF, The Project for support to Child-Friendly Schools Development, Project Proposal submitted to the Government of Japan, Ministry of Affairs, 2010. Pg. 9.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 24
coordinating partners‘ inputs and appraisal of the 10-year ESP for Liberia. This supported the successful US
$40 million EFA/FTI submission in 2010.64
3.3 Goal Two: Increased resilience of education sector services delivery in chronic
crises, arrested development, and deteriorating contexts
As stated above, the UNICEF country office did not categorize its programmes under Goal Two as Liberia was
not felt to be a country in chronic crisis, arrested
developed or deterioration.65
(Resilience will be
reviewed later in this report with reference to the DAC
Scorecard and Results Logframe.) Some of the four
indicators for Goal Two were not applicable. However,
information was provided for one of them – the
percentage of the national budget dedicated to
education, as a sign of resilience. The data showed
an almost doubling of this percentage from 7.9% to 14
% between 2007-2009. EEPCT can claim some
attribution to this rise due through the formation of the
EPF. The office‘s results framework states that
education remains a high priority.
Accelerated Learning Programme (ALP)
EEPCT directly supports the access of 8% of the country‘s children to primary education and half of the
country‘s total ALP students supported by the government.66
According to the evaluation of the program
conducted by UNICEF prior to the start of EEPCT, the ALP program has many strong features. As with ALP
programmes in other countries, the MoE has strong ownership of the programme, having developed the
curriculum, teaching manuals and structure. Selection of communities for the programme is at county level,
while PTAs are responsible for running the actual classes. MoE Master Trainers provide pre-service and in-
service training to ALP teachers. Monetary incentives are provided for the ALP teachers and the principals to
promote the operation of the programme.67
Since its inception, the ALP programme has benefited more than 60,000 children.68
This large number can be
attributed to UNICEF‘s 10 years of support to the programme, which has continued via EEPCT funds since
2007. From the 2005/2006 to the 2008/2009 school years, the number of ALP students increased by 55%,
compared to a 33% increase in overall primary school enrolment.69
However, this rise coincides with key
informants reporting that the ‗ALP system was being abused‘ due to the large number of under and overage
children in the programme.70,71
The 2007/8 school census shows that parents register their children late for
primary school (see Chart 1), which should start at six years old.72
This late enrolment overlaps with the ALP
64 UNICEF, EEPCT Comments on Draft Report, 2010. 65 UNICEF, EEPCT Comments on Draft Report, 2010. . 66 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010. 67 Nicholson, Susan. Assessment of the Accelerated Learning Programme in Liberia Implemented by UNICEF. 2007. 68Wayne, Trokon. and vonHahmann, Gail. Achieving quality in accelerated learning: Student assessment in Liberia" Paper presented at the annual meeting of the 53rd Annual Conference of the Comparative and International Education Society, Francis Marion Hotel, Charleston, South Carolina, 2009 . 69 Department of Planning, Research & Development, Status of Education System in Liberia (2005/2006-2008/2009), Liberian Ministry of Education. 2010. 70 Ministry of Education. Accelerated Learning Program Policy Guidelines. Government of the Republic of Liberia. Government of the Republic of Liberia. 2008. 71 Interview. Monrovia, Liberia. 27 July, 2010. 72 Ministry of Education. Accelerated Learning Program Policy Guidelines. Government of the Republic of Liberia. Government of the Republic of Liberia. 2008.
0
2
4
6
8
10
12
14
2007 2008 2009 2010
% of nationalBudget
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 25
programme, which starts at age eight. Additionally, the census shows a substantially older overage population
(see Chart 2). The programme was not designed for this breadth of learners.73
The upcoming ALP evaluation in 2011 will examine why such a significant number of children are enrolled in
ALP. Younger children may enrol to avoid or prepare for taking primary-school entrance exams, but reasons
for older students are less clear. An evaluation of USAID‘s similar ALP+ programme, which also includes
vocational training, highlighted the programme‘s shift from serving those whose school has been interrupted to
serving as an expansion of, or alternative to, formal education.74
This is a potentiality for the MOE
programme.
The efficacy of the ALP programme is difficult to determine as data is available on enrolment and
matriculation, but not for student retention. No government systems are in place to track ALP students‘ entry
and progress in the primary or secondary education systems. This is a global challenge regarding ALP
programmes.75
The 2007 MOE/UNICEF ALP Assessment highlights this and cites informants who note that
ALP students who passed the National Primary School Certificate Exam (NPSCE) were even more likely to go
to grade 7, were in grade 12 or had reached university.76
Also, the programme is completely supported by donor funds – which raises issues of sustainability.77
Support from the MoE has been uncertain; the previous administration favoured phase-out, but the current
73 Ministry of Education. Accelerated Learning Program Policy Guidelines. Government of the Republic of Liberia. Government of the Republic of Liberia. 2008. 74 Coyne, G., et al. Accelerated Learning Program for Positive Living and United Service: Mid-term Evaluation. USAID. 2008. 75 Johannessen, Eva Marion, A study of former TEP students in Angola and Burundi, NRC, 2005. 76 Nicholson, Susan. Assessment of the Accelerated Learning Programme in Liberia implemented by UNICEF. 2007 77 UNICEF, EEPCT Comments on Draft Report, 2010.
Chart 2: Distribution of Liberian Students in ALP by Age
Chart 1: Distribution of Liberian Students by Age
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 26
administration would like to maintain the programme. Responsibility for UNICEF-supported ALP schools was
due to be transferred to the MoE in 2010 as an EPF initiative. UNICEF will continue to provide technical
support such as joint monitoring visits, facilitating refresher training workshops and the planned ALP
evaluation in 2011.78
LAB4LAB
The LAB4LAB Liberia component is one part of a regional peace and development initiative that is one of the
innovations of the EEPCT programme, and was conceptually linked to the CFS initiative.79
A global
framework was developed for implementation of the LAB4LAB programme in 2008.80
In Liberia, LAB4LAB
schools were viewed as model CFSs. The two-year framework articulated the programmes implementation
into three components: design, construction and equipment; programming and outreach activities; and
capacity-building and development. UNICEF Headquarters has provided limited leadership in the
implementation of the LAB4LAB schools, aside from support for construction and conducting one regional
workshop.
The LAB4LAB School in Ganta, Nimba County has largely been completed. At the time of the evaluation
team‘s field work, registration for the impending school year was underway. However, the LAB4LAB School is
a pilot that cost more than US $1,000,00081
, not including operating costs. None of the contractors who
tendered for the contract had prior comparable construction experience. Nevertheless, lessons have been
learned and future UNICEF-built LAB4LABs will be scaled down for faster and cheaper construction.82
To assess the LAB4LAB School, the evaluation team conducted focus-group discussions and key informant
interviews and completed a school construction checklist. As with the other constructed schools, the team
applied the School Assessment Checklist that it had designed with CFS criteria. The school met 11 out of the
12 criteria (92%) on the checklist. This is notable as none of the six standardized government schools
constructed under the EPF that were visited met more than four of the 12 criteria (see section on EPF, above).
The only component lacking in the LAB4LAB School was an emergency preparedness plan, which has not
been a focus of the MoE or UNICEF in Liberia. The initial LAB4LAB design is being adapted for future
MoE/UNICEF construction and could serve as a source of advocacy for improved government school design.
However, based on key informant interviews, the roles that the MoE, UNICEF and the community will play in
sustaining the LAB4LAB School are unclear or not fully disseminated. Significant support will be required to
maintain the school, radio equipment, computers, programming and teachers. On a broader scale, the
programme has developed minimal regional connections. In 2009, the Côte d‘Ivoire Country Office hosted a
regional workshop on the implementation of LAB4LAB Schools and Talent Academies.83
However, this event
did not promote discussion on how the LAB4LAB programme would be organized regionally, how the peace
agenda would be developed, or the refinement of a long-term plan.84
Key informant interviews revealed that
there is minimal to no communication with LAB4LAB projects in Guinea or Côte d‘Ivoire.85
The regional impact of LAB4LAB is unrealized. No baselines or assessments were conducted to guide and
measure the programme‘s short and long-term outcomes, nor was the goal of peace operationalised in such a
way as to point to specific, measurable outcomes. As post-election tensions in Guinea again threaten the area
with potential refugees,86
the ability of the school to support peace in such a situation is unclear.
78 UNICEF, EEPCT Comments on Draft Report, 2010. 79 UNICEF, The Project for support to Child-Friendly Schools Development, Project Proposal submitted to the Government of Japan, Ministry of Affairs, 2010. 80 UNICEF, [Global Framework of LAB4LAB and Talent Academies –Draft], New York, 2008 81 UNICEF. Self Assessment. Monrovia. 2010. 82 Interview. Liberia. 1 August, 2010. 83 UNICEF Report of the workshop on the Project “Learning along borders for living across boundaries” and “Talent Academies”. Abidjan. 2009. 84UNICEF Report of the workshop on the Project “Learning along borders for living across boundaries” and “Talent Academies”. Abidjan. 2009. 85 Interview. Liberia. 7 August, 2009. 86 Aghailas, Tamba, Guineas’ post-election unrest threatens fragile peace in Liberia, Voice of Liberia, 2010. [Accessed 17 November, 2010 from http://www.voiceofliberia.org/guineaelectioncrisis.htm]
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 27
Talent Academies
An assessment for the Talent Academies in 2009 included possible strategies for implementation. As noted by
the consultant undertaking the assessment, this was ‗virgin territory‘ for UNICEF.87
Despite a positive
assessment, the UNICEF country office decided not to implement the Talent Academies as the programme
did not fit within their existing BEGE plan.88
The office also concluded that it did not have the capacity to
implement an additional innovative programme; limited technical support was provided by HQ to guide this
activity.
3.4 Goal Three: Increased education sector contributions to better Prediction,
Prevention and Preparedness for emergencies due to natural disaster and conflict
The office does not implement any programmes which contribute to Goal Three. However, it did provide
information for two of the four indicators under Goal 3 in the Results Framework. More specifically, the office
cited that a few activities focused on peace, including an unnamed UN programme for youth and its own
LAB4LAB programme. For the final indicator concerning an increase in international educational assistance,
Liberia had qualified for the $40 million grant from the EFA/FTI Catalytic Fund; and donors such as USSAID
and EU, had committed themselves to extending their programmes.
The two indicators that were not addressed were the adoption of a Disaster Risk Reduction Strategy and
Education for Conflict Prevention. Although the office positioned the LAB4LAB programme under Goal 2,
neither this nor the UN youth employment programmes were cited as strategies to address potential threats to
peace and stability.
3.5 Goal Four: Evidence-based policies, efficient operational strategies and fit-for-
purpose financing instruments for education in emergencies and post-crises.
LCO provided information on all seven of the indicators under Goal 4. Regarding the first two indicators,
coordination increased within the period of EEPCT implementation with the continuation of the Education
Development Partners Group (EDPG) and formation of the Education Cluster. The EDPG was central to
formulation and implementation of the LERP and the ESP as well as the successful application to the IMF
Catalytic Fund. EEPCT can claim some credit for this due to the formation and on-going support of the pooled
fund, the EPF. The second indicator of having a credible education sector plan was fulfilled through the
coordination highlighted above. EEPCT support was vital to Liberia‘s achievement of the third indicator – to
increase the number of countries using EMIS and systematic data collection annually. Since 2007, EEPCT
has supported National School Censuses annually and School Mapping since 2009.
The last two indicators relate to the existence of a pooled fund and its continuing support. UNICEF Liberia is
the only Country Office that has initiated a pooled fund with EEPCT funding. Regarding the indicator of
increased international funding for education, the EPF has not attracted any new donors other than the EC.
However, Liberia has qualified for a $40 million grant from the EFA/FTI Catalytic Fund. Aside from this,
USAID and EU extended their programmes past 2010.89
Assessments, baselines and evaluations are not consistently applied across EEPCT programmes. This may
be partly due to the absence of a Monitoring and Evaluation Officer in the UNICEF Country Office for more
than eight months of EEPCT implementation. However, the lack of baselines precedes this, as no regional or
87 UNICEF. Talent Academy Report: assessment of the talents, potentials, & preferences of out-of-school young people in urban communities in Liberia. Monrovia. 2009. 88 UNICEF, UNICEF, EEPCT Comments on Draft Report, 2010. 89 UNICEF, [Results Framework Response], 2010.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 28
local baselines were conducted for the LAB4LAB programme. In 2009, a process was begun to document the
EPF.90
At the same time, an audit of the EPF was conducted.91
The EEPCT programme also helped the MoE
to develop guidelines for the ALP.92
This uneven application of monitoring has affected the office‘s ability to measure the effect and impact of their
programmes and identify areas for improvement. Meanwhile, six focus groups at the national, sub-national,
and community levels were asked about their perceptions of change in monitoring and evaluation of the
education system. Half reported it to be the same as prior to the implementation of the EEPCT programme,
two ranked it as better, and one ranked it as much better (see Chart 4). This indicates that the participants
are aware of monitoring and evaluation and that there is room for its improvement.
The UNICEF Country Office drew upon comparable programmes to implement EEPCT. Global and regional
workshops seem to be the best means of disseminating information about programmes. EEPCT helped key
staff members participate in many of these workshops, including the aforementioned LAB4LAB workshop in
Abidjan in 2009. In the case of the CFS programme, the government officials who attended the workshop
were motivated by its content and interested in bringing the concepts back to Liberia. By posting the ALP
Evaluation on the UNICEF website, the office sought to share its experience with other UNICEF offices and
implementing organizations. Document sharing can be an effective means of conveying information and
lessons learned.
EMIS
EEPCT‘s support of the MoE‘s Educational
Management Information System (EMIS) assisted
internal and external educational programming and
planning. As noted earlier, two National School
Censuses were completed for the 2007/2008 and
2008/2009 school years. The three focus-group
discussions at the national and regional level all cited
the importance of the census data. The improving
data collection processes contributed to the
Education Sector Plan which was approved for the
FTI/Catalytic Funding in 2010.
To support wider dissemination and use of the
statistics, a MoU between the MoE and the National Statistics Office was signed to link census data with the
Liberian Institute of Statistics and Geo-Information Services (LISGIS). This linkage will improve planning on
issues such as teacher deployment and distribution of teaching and learning materials.93
School mapping and
micro-planning will assist allocation of new schools and resources by the size of the school and the number of
children within its catchment area.94
Liberia Education Pooled Fund (EPF)
The EPF has already been described briefly (see section 1.4, above). It is a US $16.25 million dollar pooled
fund managed by the MoE and Ministry of Finance (MoF).95
The EPF emerged when Liberia was refused
90 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009 91 Ernst and Young and MGI MONBO & Company. Draft Audit report on the Project financial statements of the Education Pooled fund of the Ministry of Education pooled Fund of the Ministry of Education Liberian Primary Education Recovery Program (L-PERP). 2009 92 Ministry of Education. Accelerated Learning Program Policy Guidelines. Government of the Republic of Liberia. Government of the Republic of Liberia. 2008. 93 UNICEF, Self-assessment, 2010 94 UNICEF, Self-assessment, 2010. 95Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg 28.
Chart 4
Change in Monitoring and Evaluation
No change
Better
MuchBetter
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 29
support from the EFA/FTI Catalytic Fund in 2007.96
As an alternative, the EPF was set up with EFA/FTI‘s
encouragement. In May 2007 UNICEF agreed to take the lead in the EPF‘s implementation.
The EPF was established using EEPCT funds between June 2007 and May 2008 in consultation with the
LCO, UNICEF HQ, the MoE, the MoF, Open Society Institute (OSI) and the World Bank.97
Due to the limited
capacity of the government, the fund is under dual management; the MoE is the lead organization and
oversees the planning and implementation of EPF activities, but fiscal management and procurement is
overseen by the MoF. 98
The partnership is intended to build the capacity of the MoE so that it can manage
funds directly in the future. The pooled fund began with US $12 million of support from EEPCT and US $4.25
million from OSI. In March 2010, UNICEF contributed an additional US$3.2 million to the fund, enabling OSI
to release its remaining US $750,000 and complete the foundation‘s US $5 million commitment.99
However, the concept of an education pooled fund pre-dates EEPCT and, unlike other initiatives such as
LAB4LAB and the Talent Academies, is driven and owned by the country office. The EPF is unique for several
reasons. From the start, it brought together a wide variety of education actors to inform the fund‘s
development and dispersal of funds.100
One key informant reported that an advantage of the EPF was that it
could move faster and was more flexible than other mechanisms.101
Another informant stated that the EPF
was the best forum for discussion of educational issues in Liberia.102
The fund has also been noted as an
example of public-private partnerships103
and gives the government access to multi-year funding.
The implementation of the EPF was challenging for the UNICEF Country Office as it did not have the in-house
capacity to establish and develop the Fund. UNICEF HQ provided multiple technical support visits, but a
temporary post might have been more useful in providing sustained support. UNICEF Liberia‘s 2007 Progress
Report stated that the ‗UNICEF Liberia education programme lacks enough technical staff to be able to
effectively support the larger macro-processes which have impact on the MDGs and at the same time pilot
viable solutions to inform policy at the national level.‘104
As stated earlier, the limited capacity of the MoE was also a challenge.105
The Country Office has stated that
this was ‗most clearly illustrated by the need for technical support for OSI to support the textbook procurement
and distribution'.106
Regarding school construction, the MoE did not ‗have the capacity to supervise school
construction projects of such magnitude‘.107
This lack of capacity was not limited to the MoE, as the MoF also
requires significant outside technical support to financially manage the fund.108
The EPF is viewed by some key informants as the most poorly managed of Liberia‘s different sectoral
funds.109
Unlike other funds, the EPF is co-managed by the MoF‘s Project Financial Management Unit
(PFMU), which is supported by the World Bank. This does not include a strategy for capacity building – which
is essential, given the high turnover of staff within the Ministry. Nonetheless, the presence of the EPF is
96 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg 28. 97 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg vii. 98 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg. vii 99 Schmidt, Caroline and Aleesha Taylor. Liberia’s Education Pooled Fund: a case for private foundation engagement in post-conflict education recovery. Open Society Institute. 2010. Pg 3 100 Interview, Liberia, 4 August 2010. 101 Interview, Liberia, 5 August 2010. 102 Interview, Liberia, 4 August, 2010. 103 Schmidt, Caroline and Aleesha Taylor. Liberia’s Education Pooled Fund: a case for private foundation engagement in post-conflict education
recovery. Open Society Institute. 2010. 104UNICEF, UNICEF Liberia: Annual Report 2007, Government of Liberia –UNICEF Programme of Cooperation, Monrovia, 2007. 105 UNICEF, EEPCT Comments on Draft Report, 2010. 106 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg 16 107 UNICEF, EEPCT Comments on Draft Report, 2010. 108 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. Pg 30. 109 Interview, Liberia. 5 August 2010.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 30
believed to have contributed to the successful application to the Fast Track Initiative/Catalytic Fund.110
To
date, the EPF has dispersed three tranches of funds, as shown in Table 4.
The sustainability of the Fund is questionable. The 2010 plans will exhaust much of the remaining EPF
resources. Key informants stated that some donors cannot or have not chosen to contribute to the Fund due
to accountability and reporting issues. In 2009, the EU chose not to contribute; based upon an unshared
capacity assessment of the Fund. Rather a separate fund was created which received the newly awarded US
$40 million contribution from the FTI/Catalytic Fund.112
3.6 DAC Criteria
The analysis below was done by measuring stakeholders‘ perceptions of EEPCT‘s performance against the
OECD/DAC criteria. The charts in the following sections show the ratings given broken down by different
levels of stakeholder – community, sub-national and national.
Relevance/Appropriateness (average 4.5)
Relevance/Appropriateness was assessed through an
analysis of access to education, quality of education, and
girls‘ enrolment. Overall, the stakeholders reported
improvement in these areas (an average score of 4.5).
Chart 5 illustrates the stakeholders‘ perceived changes in
these areas since the beginning of EEPCT. There is no
statistically significant difference in mean rankings of the
different levels of stakeholders. These perceptions are
corroborated by Liberia‘s overall increase in Gross
Enrolment Ratio (GER).113
110 Interview, Liberia, 5 August 2010. 111 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009. 112 Interview. Liberia. 5 August 2010. 113 UNICEF [Results Framework Response], 2010.
Education Pooled Fund Disbursements111
Date Tranche Supported Activities
2008
September $1,507,901
Operationalising three Rural Teacher Training Institutes (RTTIs)
including vehicles, rehabilitation and refurbishment of teachers
residences
November $5,510,829
Procurement of 1.2 million textbooks for English, Math, General
Science and Social Studies with teachers‘ manuals (for public
schools only)
2009 March $5,238,268 Construction of 40 six-room Primary Schools in 15 counties
Chart 5
Table 4
4.6667 4.6667 4.1667
0
1
2
3
4
5
CommunityEducators
Subnational National
Relevance
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 31
Overall, out of eight focus-group discussions conducted with children, four stated that that their feelings
towards school were much better while the remaining four ranked it as better. Parents were difficult to
assemble, but the one group of 12 organized ranked their confidence in the quality of education as much
better. Out of the eight groups with teachers, four ranked the education as much better, while the remaining
groups ranked it better.
Among the five groups with educators that discussed retention rates in school, four ranked the change in girls‘
retention as much better while one ranked it as better. Moreover, two out of five FGDs with girls and female
youth ranked their attendance as much better. These perceptions are confirmed by comparison of enrolment
rates from the 2006/2007 with those from the 2008/2009 National School Censuses.
Of five focus groups with teachers that ranked their ability to teach, four ranked this as much better while the
fifth ranked the change as better (overall mean rank 4.8). Teacher training and the ALP were two issues that
featured prominently in the discussions. Recommendations varied amongst the groups, with common themes
arising around the issues of teacher training and programme sustainability.
Effectiveness (average 3.75)
Key outcomes that were considered to evaluate the
effectiveness of the programme were the education
system‘s ability to recover from emergencies, government
preparedness and increased early-warning knowledge.
Overall, stakeholders reported improvement in effectiveness
(3.75). Chart 6 shows the changes in the stakeholders‘
perceptions. There was no statistically significant difference
between the mean rankings of three levels. However, the
programmes were viewed as more effective at the national
level. Of the six focus groups at the three levels, two
reported no change in government preparedness and early
warning, one reported it as better, and one as much better while one
didn‘t know.
As Liberia is still emerging from the conflict, the recovery of the education system is seen as a key means of
improving safety and security. The evaluation team found that perceptions of safety and security are a
concern on school premises. Of the five FGDs conducted with children that discussed safety and security
issues, three ranked their sense of security at school as the same or no change since the programme began
while one ranked it as better and one ranked it as much better. This varied from youth perception of the levels
of violence. Of the four youth FGDs, two gave no response while one ranked it as better and one ranked it as
much better.
Efficiency (average 4.4)
Stakeholders also reported improvement in efficiency
(with an average score of 4.4). Key outcomes are
considered under programme performance compared
to similar programmes and child safety in schools.
Chart 7 illustrates the assessment by the national, sub-
national, and community stakeholder groups of the
change in these criteria since the programme began.
4
4.8 4.6667
0
1
2
3
4
5
CommunityEducators
Subnational National
Efficiency
3.5 3.75 4
0
1
2
3
4
5
CommunityEducators
Subnational National
Effectiveness
Chart 7
Chart 6
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 32
Coherence and coordination (average 3.75)
Coherence and coordination were assessed through
implementation of the INEE Minimum Standards. In
all other case-study countries, the Education Cluster
support for the MoE coordination role was also
measured as a means to assess coherence and
coordination. However, the conflict and recovery in
Liberia pre-dates that of the Education Cluster, so
the question was not asked. Overall, stakeholders
reported improvement in coherence and
coordination (an average score reported 3.75). Chart
8 illustrates the participants‘ assessments of these
criteria. Of the three FGDs conducted with government
officials, two reported they didn‘t know when asked about the change in implementation of the Minimum
Standards of the Inter-Agency Network for Education in Emergencies (INEE), while one group didn‘t answer
this question.
Sustainability (average 3.8)
Sustainability was assessed based on the sector‘s
perceived ability to respond to another emergency,
its capacity to support education in emergencies
and monitoring and evaluation. Overall,
stakeholders reported improvement in sustainability
(with an average score of 3.8). Chart 9 illustrates the
focus group participants‘ assessment of these
criteria.
Of the six discussion groups conducted, three
ranked monitoring and evaluation of the education
system as the same or no change since the time the
programme began, while two groups ranked it as
better and one group ranked it as much better.
Summary
Across all levels and categories there was an overall average score of 4.4. The lowest overall average ranking
within the five categories was recorded for both the inter-related issues of effectiveness and
coherence/coordination (3.75). Relevance and appropriateness received the highest overall average rank
(4.8). Community level stakeholders‘ lowest scores were on both coherence/coordination and effectiveness
(3.5), while relevance received the highest rank (4.6). Sub-national stakeholders ranked efficiency highest
(4.8), and coherence/coordination and effectiveness lowest (3.75). At the national level, efficiency was ranked
highest (4.6), and sustainability was ranked lowest (3.66).
3.5 3.75 4
0
1
2
3
4
5
CommunityEducators
Subnational National
Coherence and Coordination
3.7 4.1
3.7
0
1
2
3
4
5
CommunityEducators
Subnational National
Sustainability and Connectedness
Chart 8
Chart 9
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 33
3.6 Focus Groups Discussion Outcomes
The focus-group discussion tool was used in at the
community level with children (ages 9-13), youth
(ages 14-18), educators and parents. Each group
was asked a set of questions which measured the
intended outcomes of the EEPCT programme at the
particular beneficiary level. Participants were asked
to provide a group ranking of 1 to 5, similar to the
assessment against DAC criteria described in the
section above.
Overall, community-level stakeholders ,including
children, youth, parents and educators, reported
improvements against the various education-related criteria since the EEPCT programme began (see Chart
10).
0 1 2 3 4 5
Feelings about school
Attendance
Girls' participation
Boys' participation
Sense of safety and security
Ability to protect self if another emergency
Amount students help each other
Students' involvement in problem solving
Use of fighting to resolve interpersonaldifferences
Students' Average Ranking of Change
All Students
Youth
Children
0
1
2
3
4
5
Overall Ranking of FGDs
Overallaverage
Chart 10
Chart 11
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 34
Children‘s outcomes were measured through change in the following criteria: feelings about school;
attendance; girls‘ participation; boys‘ participation; sense of safety and security; ability to protect themselves in
the event of another emergency; the amount that students help each other; students‘ involvement in problem
solving at school; and the use of fighting to resolve interpersonal differences.
The overall average of five groups of children was four out of five (referring to ‗better‘). The five focus-group
discussions with children were conducted at three different schools in two different counties. The mean
rankings for the three schools were 4.13, 3.93 and 3.88, which all reflected perceptions of improvement since
the programme began. There was no statistically significant difference between the mean rankings of the
children at the three schools (alpha = .05 and p-value = 0.733).
The overall average ranking of four groups of youth was 4.41. Like the groups of children, the four youth-
group discussions were conducted at three different schools in the two counties visited. The mean rankings of
youth at the three schools were 4.50, 4.44 and 4.29, which indicate perceptions of positive change since the
programme began. There is no statistically significant difference between the means of the three schools
(alpha = .05 and p-value=0.819). (See Chart 11)
Educator‘s outcomes were measured through change in the following criteria since the programme began:
ability to teach; provision of teaching and learning materials; quality of training; interaction between children of
different groups; reporting and monitoring procedures; quality of education at the school; retention rate for
girls; ability to respond to future emergencies; amount that students help each other; students‘ involvement in
problem solving; the use of fighting to resolve interpersonal differences; and feelings about school. With these
criteria in mind, the overall average ranking of the five groups of educators was 4.32 (referring to a ranking of
‗better‘). (See Chart 12)
Finally, parents‘ outcomes were measured through the following criteria: community involvement in school
emergency planning; the community‘s ability to address safety and abuse in the school; confidence in the
quality of education; community ownership of school construction; the amount that students help each other;
students‘ involvement in problem solving; the use of fighting to revolve interpersonal differences; and feelings
about school. As parents were largely unavailable during the working day, the evaluation team was able to
conduct only one focus-group discussion with parents. However, the average ranking of all these criteria was
positive (4.38).
0 1 2 3 4 5
Ability to teach
Quality of training
Reporting and monitoring procedures
Girls' retention rate
Amount students help each other
Use of fighting to resolve interpersonal…
Educators' Average Ranking of Change
Chart 12
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 35
3.7 Cross-cutting Issues
Rights-Based Approach
EEPCT has supported UNICEF‘s upstream work with the MoE to achieve the Right to Education and
principles set forth in the Universal Declaration of Human Rights. Liberia ratified the Convention on the Rights
of the Child (CRC) in 1993 and has submitted four progress reports to the Committee on the Rights of the
Child. With the entry of a new government in 2006, the Education Law of 2001, which abolished primary
school fees, was applied; also, school was made compulsory.114
However, obstacles still remain to its
enforcement, including protecting schools from charging additional fees and demanding birth certificates. The
ALP guidelines cite the Right to Education as a founding principle and include a component on Peace and
Human Rights. However, the evaluation was not able to confirm how this has been implemented. Overall,
children‘s participation is not strong within EEPCT. Key decisions such as textbook selection and distribution
did not involve children, and they are not involved in major EEPCT components such as the EPF.
Gender
Gender is a central component of UNICEF‘s
BEGE Programme, which includes EEPCT. As
part of this, UNICEF supports the Gender
Education Unit within the Ministry of Education. No
specific gender initiatives were undertaken under
EEPCT, but the EEPCT-supported programme did
provide data regarding the gender disparities
according to age, educational level and
geography.115
This data improved programme
planning and implementation.
ALP also has a significant gender component.
Girls often do not complete their education;
however, participation in ALP is presently near
gender parity. This is positive, but census data is required to understand its significance.
UNICEF staffs were aware of the gender disparity between male and female students and teachers. The
importance of female teachers was viewed as ‗debateable.‘116
This contradicts other sources that state that
their presence can affect the enrolment, attendance and retention of girls and female youth in school.117
This
lack of awareness could be attributed to a lack of a functioning UNGEI forum. UNICEF reported that a forum
had launched in January 2009, but that it was not functional due to capacity constraints in MoE‘s Girls‘
Education Unit.118
No documentation was seen regarding this, and the UNGEI website cites that no
partnership had been formed.119
Disaster Risk Reduction (DRR)
No information was provided on DRR programmes or emergency education programmes within the country.
The LCO chose not to implement any DRR programmes as part of the EEPCT programme. Conflict in
surrounding countries and the potential resurgence of conflict overshadows the risk of natural disasters in the
country. When asked about disaster preparedness, one community-level key informant said that children
114 UNICEF, EEPCT Comments on Draft Report, 2010. 115 Department of Research and Planning, Status of Education System in Liberia (2005/2006-2008/2009). Liberian Ministry of Education, 2010. 116 UNICEF, EEPCT Comments on Draft Report, 2010. 117 UNESCO, The Impact of Women Teachers on Girls Education, Paris, 2006. (Accessed on November 18, 2010 from Link: http://unesdoc.unesco.org/images/0014/001459/145990e.pdf 118 UNICEF, EEPCT Comments on Draft Report, 2010. 119 UNEGI, Liberia: background, 2010. (Accessed on October 2, 2010 from http://www.ungei.org/infobycountry/liberia.html)
Perceptions of Government Preparedness
No change
Better
Much better
Don't Know
Chart 3
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 36
should be taught to flee if war came again.120
Aside from conflict, Liberia faces minor threats such as floods
and storms. With this in mind, Liberia did not provide data for any of the Goal Three indicators in the Revised
Logframe. The majority of focus groups did not have knowledge of DRR activities or believed they had not
changed with the implementation of EEPCT. The weakness of the MoE‘s preparedness and response was
noted in the 2010 FTI/Catalytic Fund Approval.121
Sensitivity to Conflict and Fragility
EEPCT support has been largely been used to stabilize the education system. ‗Peace‘ programming exists in
a small way within the ALP programming.122
Peace is stated as a key component of the LAB4LAB
programme, but it is unclear how this aim is to be measured. As the programme lacks any clear indicators, it is
difficult to determine whether LAB4LAB‘s peace component is being achieved or how it will be connected to
other programmes in Liberia or within the region.
Accountability, Monitoring and Evaluation
The global instrument for monitoring the implementation of EEPCT
is the Revised Logframe (right). The UNICEF Country Office stated
that one of the weaknesses of the EEPCT programme was that it
had to create data retrospectively.123
It was able to provide data for
14 out of the 20 indicators; most of the information provided was
qualitative and typically did not cover more than one year. The office
presented information on the two EEPCT goals relating to the
change in the number of children both in and out of school. School
Census data was cited earlier; however, the number of children out
of school was not calculated.
The office‘s donor reports in 2007, 2008, and 2009 show an
increasing clarity on EEPCT aims and goals. However, EEPCT
remains unmentioned within the CO Annual Reports.
Monitoring of programme implementation occurred at different levels within EEPCT programmes. Monitoring
was most intense in programmes directly overseen by UNICEF, such as the onsite monitoring of the
construction of the LAB4LAB School. Most of the monitoring for programmes such as ALP and distribution of
learning materials was done by the MoE. Equally, the MoE was responsible for monitoring its own support of
the RTTIs, textbook distribution and school construction. The MoE has limited capacity to conduct this level of
monitoring and this in turn impairs the UNICEF Country Office‘s capacity to show the progress and quality of
programming being supported.
Due to the lack of assessments, no indicators were created nor baselines conducted for EEPCT programmes.
The PREV sought to establish retro-active baselines within the research methodology but found this
challenging as the programme is nationalized and it is difficult to disaggregate EEPCT and non-EEPCT
programmes.
Within the EEPCT timeframe, the single monitoring initiative was the documentation of the EPF.124
12 Interview. 1 August 2010. 121 World Bank. Appraisal of the 2010-2020 Education Sector Plan. 2010. 122 MOE. Accelerated Learning Program Policy Guidelines. Government of the Republic of Liberia, 2008. 123 UNICEF, [Results Framework Response], 2010. 124 Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one process. UNICEF: Monrovia. 2009.
Goal LCO
Response
Total Number
of Indicators
Goal 1 4 5
Goal 2 1 4
Goal 3 2 4
Goal 4 7 7
14 20
C
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 37
3.8 Operational issues and management issues
Understanding of the programme cycle
The lack of baseline data was a challenge in gauging the progress of EEPCT. For example, no regional or
country assessments had been conducted for the LAB4LAB programme, hindering the development of
indicators and baselines. The Country Office illustrated the achievement of this objective by stating:
―completion of the [LAB4LAB] school has brought hope for more development in Nimba County and in Ganta
city. As the school was being built and when it was completed, some people thought it was a college or
university.‖ The positive spin offs, it was stated, might not be felt immediately, but other counties now also
want such model schools if money becomes available. At any one time there were more than 500 workers on
site at the school. These were mostly young former refugees whose job prospects were limited before school
construction began. The total landscape of the school‘s neighbourhood has changed since construction. Many
people have built houses close to the school so their children can attend it. When it did open its doors,
children could not stop washing their hands with soap at break time. However, while this qualitative
information is important in evaluating the programme‘s effectiveness, it also needs to be accompanied by
quantitative indicators and data.
Staffing and technical support
The Country Office is managing a large education programme with upstream and downstream components.
The BEGE programme is presently being implemented by two international and two national staff members.
This staffing level is not sufficient to oversee the direct implementation of programmes, the EPF and the
capacity building of the MoE. Review of EEPCT‘s expenditures shows a significant amount of monitoring of
the LAB4LAB School but no equivalent monitoring of the rehabilitation of the CEO/DEO complexes, ALP, and
projects implemented under the EPF including the government school construction or textbook distribution.125
Sustainability and Exit Strategy
The country office argued that the sustainability of EEPCT programmes is due to their efforts and not due to
the EEPCT global design as COs were not involved in its inception.126
The planning for sustainability was
hindered by the fact that the EEPCT funding arrived late during the initial year and is only distributed annually.
Certain programmes supported under EECPT do not have clear exit or continuation strategies. This requires
immediate consideration, as programmes such as the LAB4LAB and the ALP are scheduled to be transferred
to the MoE in 2010. While the ALP was not intended to be sustainable, the system of programme delivery is a
national resource that could be used for other education programme purposes.
The Country Office emphasized that LAB4LAB was a multi-country programme and therefore linked with other
countries with regard to sustainability. Sustaining of the planned regional LAB4LAB network has not been
costed. At the national level, however, the office will provide support for the first year of LAB4LAB operation to
determine what is required technically and financially to maintain the completed school. This information can
be used for later governmental budgeting.
EEPCT has provided support to the MoE to enhance the EMIS and create the EPF. While there are no plans
to phase out this support, no benchmarks have been identified to mark the transition from direct technical
assistance to capacity building.
125 UNICEF, Utilization of Donor Funds – EEPCT Liberia, 2010. 126 UNICEF. Self-assessment. Monrovia. 2010.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 38
3.9 Partnerships
The MoE is the currently the Country Office‘s sole implementing partner for the EEPCT programme. UNICEF
also coordinates with the MoE‘s other bilateral and multilateral education development partners as well as
national and international NGOs in the implementation of the LPERP, the development of national
frameworks, the development of the ESP, and support of the EPF. These partners include UNESCO, UNHCR,
UNDP, WFP, UNMIL, USAID, EC/EU, WB, OSI, IRC, LET, SC/UK, OXFAM, FAWE, and ActionAID.
The focus on working with and through the MoE potentially leaves a gap in building capacity from the
community level, specifically through Parent-Teacher Associations. As an example, the ALP programme is
overseen at the PTA level.
4.0 THE WAY FORWARD
4.1 Lessons learned and conclusions
EEPCT has significantly improved the Liberia Country Office‘s educational response and has already had a
significant impact on the education of Liberia‘s children. The LCO has implemented nine programmes as part
of the EEPCT programme: educational materials, instructional support, infrastructure support, and technical
support, Child-Friendly Schools, ALP, LAB4LAB/Talent Academies, EMIS and EPF.
EEPCT progress was enhanced through the UNICEF Country Office‘s adherence to existing programme
plans.
The strength of EEPCT in Liberia is its integration, where possible, into plans pre-negotiated with the MoE.
The Country Office specifically stated that it was ‗not implementing EEPCT as a country programme but as a
BEGE supported by other donors as well.‘127
The office‘s adherence to these plans is illustrated through its
integration of the LAB4LAB initiative into the pre-existing plans for CFS and its choice not to implement the
Talent Academy because it did not fit within the existing plan.
Proper planning and monitoring did not occur, due to EEPCT being an HQ-driven initiative.
Throughout the evaluation, the Country Office emphasized that ‗EEPCT was never meant to be a Liberia
Country Office Programme‘ as it was not part of the 2008-2012 Liberia Country Programme. EEPCT was
viewed as an HQ-driven initiative with limited involvement from the Country or Regional Offices.128
This was exemplified by the lack of clarity in the development and dissemination of EEPCT documentation
and understanding of how the programme would be integrated into UNICEF planning structures. As an
example, EEPCT funding came after the approval of the 2008-2012 UNICEF Liberia Country Programme for
Children (CPC), which had been aligned with the MoE‘s LPERP and Education Sector Plan (ESP). The initial
focus on the regional LAB4LAB and Talent Academies Initiatives was additionally confusing as it was not clear
whether EEPCT could support programmes outside these two initiatives. The promotion of EEPCT was also
cited by the country office as lacking awareness of the realities of field implementation. COs are judged
according to the implementation of the core programmes of Child Survival and Development (CSD), BEGE
and Child Protection, which are derived from government plans.129
The UNICEF Country Office was able to attract donor support by combining standard UNICEF programming
with LAB4LAB innovation.
127 Email, 16 August 2010. 128 UNICEF, EEPCT Comments on Draft Report, Liberia, 2010. 129 Email, 16 August 2010.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 39
Overview - Lessons learned and conclusions:
(1) EEPCT progress was enhanced through the Country Office’s adherence to existing programme plans.
(2) Proper planning and monitoring did not occur, due to EEPCT being an HQ-driven initiative. (3) The Country Office was able to attract donor support by combining standard UNICEF
programming with LAB4LAB innovation. (4) UNICEF and the Liberian government did not have sufficient staff and technical expertise to
implement the breadth and complexity of EEPCT programmes. (5) Programme transitions should be conducted gradual, with benchmarks to ensure relevance and
sustainability. (6) Monitoring of programmes does not draw upon communities and is poorly supported through
programme transitions. (7) Programmes lack components such as assessments, baselines and plans to link improvements
to policy change through advocacy. (8) Decentralization supports the implementation of other interventions. (9) Significant participation of children and communities was lacking within EEPCT programming. (10) Emergency Education needs a structured approach.
The Country Office‘s combination of the innovative LAB4LAB programme with UNICEF‘s standard CFS model
helped leverage support from a significant donor. In 2010, the Japanese government contributed US $8.5
million to support the ‗construction of six LAB4LAB schools, 24 new MoE standard primary schools and the
renovation/expansion of 60 others to increase schooling for 22,500 children.‘130
The LCO stated that part of
the attraction for the Japanese was the application of CFS principles in this innovative new way.131
School
construction was seen as an entry point to incorporate CFS principles into standard practice.
UNICEF and the Liberian government did not have sufficient staff and technical expertise to implement the
breadth and complexity of EEPCT programmes.
Capacity assessments of COs and implementing organizations should be conducted before programmes are
initiated. As an example, the Country Office stated that in implementing the EPF it ‗lacks enough technical
staff to be able to effectively support the larger macro-processes.‘132
The lack of a monitoring and evaluation
officer in the office has had an equal effect on programme implementation. UNICEF implementing partners,
such as the MoE, should be subject to equal scrutiny in this respect. The constant reference to the limited
capacity of the government suggests that there should have been more modest interventions with greater
monitoring and technical support, as was seen by the additional support that was provided in the textbook
distribution and the finalization of school renovation. As an example, OSI assisted the textbook distribution
and was due to conduct an evaluation of the process during 2010.
Programme transitions should be gradual, with benchmarks to ensure relevance and sustainability
130 UNICEF, The Project for support to Child-Friendly Schools Development, Project Proposal submitted to the Government of Japan, Ministry of
Affairs, 2010. Pg. 9. 131Interview, New York, 17 November, 2010. 132 UNICEF,UNICEF Liberia: Annual Report 2007, Government of Liberia –UNICEF Programme of Cooperation. Monrovia. 2007.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 40
If programme responsibilities are to be sustainable, there is a need to work within the timeframe and
capacities of beneficiary communities and organizations. For example, the revitalization of the CEO (County
Education Officers) Complexes should have included a longer transition period. As it stands, it was assumed
that the government would supply fuel to CEOs for school monitoring, which it could not do. Equally,
emergency programmes, such as ALP, should have definitive points for review and adjustment. The Country
Office is hoping to begin developing best practices for programme transitions by conducting an ALP
evaluation and an assessment of out-of-school children in 2011. Results from these studies will inform the
MoE‘s decision regarding the ALP. With regard to the LAB4LAB initiative, UNICEF will support the first year
of the school‘s operation to determine its operating costs.
Monitoring of programmes does not draw upon communities and is poorly supported through programme
transitions.
Programme monitoring is the responsibility of all parties involved in programme implementation, including
children. Programme planning should include sufficient long-term resources to develop a culture of monitoring
and evaluation internally but also within the MoE and communities which they serve. Children and
communities monitored textbook distribution and maintenance through the implementation of a textbook
policy; similarly, communities were mobilized to monitor ALP programmes in their communities. Building upon
these examples, there should be further community involvement in school construction and supply distribution.
Communities should be involved in the selection of contractors for school construction and should be informed
about upcoming supply distributions via radio transmissions, while materials received should be signed for by
a student, parent and the school principal.
Programmes lack components such as assessments, baselines and strategies to link improvements to policy
change through advocacy.
Few of the EEPCT-funded programmes were based on situational assessments and baselines; this will hinder
evaluation of these programmes and advocacy for change. For example, the lack of a situational assessment
and baseline for the LAB4LAB/CFS programme will hinder evaluation and thus the communication of the
programme‘s benefits to communities and government. Advocacy strategies also need to be included in the
development of targeted products. For example, while a documentation exercise for the EPF was undertaken,
the target audience for this product was unclear.
Decentralization supports the implementation of other interventions.
The EEPCT‘s capacity building of the CEO Centres was not envisioned as a programme and was described
by the Country Offices as part of their government capacity building initiatives. However, the evaluation team
would identify it as supporting the resilience of the education system. This support is in line with MoE‘s plans
for decentralisation, which it hopes will increase service efficiency, transparency, accountability and
responsiveness.133
EEPCT‘s support for the rehabilitation of CEO complexes and provision of motorbikes was
an initial step complemented by training, data collection and production of the National School Census. The
training of 92 DEOs and 14 CEOs to perform the census built MoE capacity and created the opportunity for
links between the community level (schools), the county level (CEOs and DEOs) and the National MoE.134
This capacity is important because it improves monitoring and evaluation at the community level, including
more efficient and timely data collection and improved programme management.
Significant participation of children and communities was lacking within EEPCT programming.
133 World Bank. Issues in program design: education and decentralization. Decentralization Thematic Group. [ Accessed on September 18, 2010 at http://www.ciesin.columbia.edu/decentralization/English/Issues/Education.html] 134 UNICEF. UNICEF Liberia: Annual Report 2009. Government of Liberia-UNICEF Programme of Cooperation. Monrovia. 2010.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 41
A significant portion of EEPCT work was ‗upstream‘ in supporting the MoE. This strong upstream approach
did not bring communities and children into the decision-making and dialogue. By incorporating children‘s
voices into programme design and implementation, the efficiency and quality of EEPCT programming could
be enhanced. Children have valuable insights and energy to contribute in post-crisis situations. At both the
community and national level, children can help advocate change on issues such as gender-based violence in
schools. This should be an intentional area of focus in moving forward with CFS schools.
Emergency Education needs a structured approach that takes into account natural disaster and conflict.
With the on-going tensions in Guinea, UNICEF and the MoE would benefit from a more structured approach to
the implementation of emergency education responses. The Country Office chose not to implement any
Disaster Risk Reduction programmes as part of the EEPCT programme. Conflict in surrounding countries and
the potential resurgence of conflict overshadows the risk of natural disasters in the country. When asked
about disaster preparedness, one community-level key informant expressed that children should be taught to
flee if war came again.135
As mentioned above, Liberia also faces minor threats of natural disasters such as
floods and storms.136
4.2 Recommendations
Government/MoE:
RECOMMENDATION: Evaluate the relevance the ALP programme. This can be supported by UNICEF‘s
ALP evaluation and the UNESCO/UNICEF assessment of out-of-school children.
RECOMMENDATION: Establish an emergency education/preparedness section in collaboration with
the IASC Education Cluster Leads (UNICEF and Save the Children) based upon the FTI’s
recommendation.
RECOMMENDATION: Remain engaged with the implementation of CFS to identify best practices to
adopt as national policy.
RECOMMENDATION: Establish, and identify resources for, a monitoring plan for CEOs.
UNICEF Headquarters:
RECOMMENDATION: Lead the development of programme learning workshops around specific
emergency education programme components, such as ALP. The results of these workshops could then
be developed into a programme plan to make promising practices into best practices through thorough
assessment, baselines and evaluations.
RECOMMENDATION: Reinforce the application of the programme cycle at the global, regional and
country level through inclusion in workshops, review of proposals and newly developed programme
documents. This should emphasize the importance of tying evaluations to advocacy as well as training on
how to develop advocacy strategies.
RECOMMENDATION: UNICEF HQ should lead a lessons-learned review of the EPF to identify when
such a strategy should be applied, but also the implications for the CO regarding staffing and
technical support. If Pooled Funds become a common intervention, UNICEF should develop in-house
capacity to provide technical support.
135 Interview. Liberia. 1 August 2010. 136Preventionweb. Liberia profile (Accessed September 21, 2010 from (http://www.preventionweb.net/english/countries/statistics/risk.php?iso=lbr) 2010.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 42
RECOMMENDATION: Adapt the CFS model to be used in emergencies.
RECOMMENDATION: Identify resources to continue the LAB4LAB and Talent Academy programmes
to an established transition point. Both of these initiatives are high level, regional/global initiatives.
UNICEF is under some obligation to see them through to commonly identified transition points. This would
include updating the Global Framework of LAB4LAB and Talent Academies in collaboration with the Regional
Offices and COs. Such a process would include a sociological and political survey, identification of
interventions and indicators, and decisions regarding how LAB4LAB/Talent Academies will proceed as a
regional/global endeavour.
Regional Offices:
RECOMMENDATION: Identify and provide the support needed by EEPCT countries through existing
personnel, short-term contracts or consultancies. Participate actively in the development of long-term
LAB4LAB and Talent Academy planning.
RECOMMENDATION: Participate actively in the development of long-term LAB4LAB and Talent
Academy planning.
UNICEF Liberia Country Office:
RECOMMENDATION: Review opportunities for children’s participation in decision-making and
programme implementation. Specifically, children‘s involvement should be integrated into the expansion of
the CFS programme.
RECOMMENDATION: Initiate long-term implementation and transition plans for the Ganta LAB4LAB
school. The Country Office needs to develop a plan for covering continuing costs based upon the first-year
operational costs (including electricity, computer maintenance, radio licence, etc.). New partners, such as
Search for Common Ground/Talking Drum to lead radio programming, should be identified as part of this
planning.
RECOMMENDATION: Review the implementation of the CFS programme to include situational
assessments as well as baseline and evaluation strategies that will provide data which can be used
for advocacy.
RECOMMENDATION: Review the staffing that is needed to ensure sufficient monitoring and transition
of programmes. Possible areas of focus include the clustering of activities around (1) ALP – conducting the
ALP Evaluation, Out-of-School Children Assessment and coordination with the government – and (2)
CFS/LAB4LAB programme – implementation of the LAB4LAB School, coordination of additional schools with
Japanese funding and the roll-out of the CFS programme. These positions are important for the continuation
for the programme after EEPCT.
RECOMMENDATION: Identify and address missing technical support that has resulted from the
absence of a monitoring and evaluation officer. Short-term consultants could be hired to support
assessments, baseline and the development/application of indicators.
RECOMMENDATION: Review UNICEF’s strategy to support the Ministry of Education to monitor
schools, distribution and data collection.
RECOMMENDATION: Develop a multi-year CFS advocacy strategy to accompany the 2008-2012
Education sector plans. The CFS programme should be a product of a solid assessment, and include
development of indicators and a programme baseline. Moreover, this should be coordinated with the
development of messages and products for targeted audiences.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 43
RECOMMENDATION: Document the ALP programme, including programme structure, guidelines,
monitoring structures, and government involvement.
RECOMMENDATION: Plan a dissemination and advocacy strategy for the ALP Evaluation and the
UNICEF/UNESCO assessment. Messages should be targeted on communities, the Liberian government,
UNICEF and the international community.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 44
REFERENCES
Coyne, Geoff, et al., Accelerated Learning Programme for Positive Living and United Service: Mid-term
Evaluation, USAID, 2008.
Collaborativeye. Decentralization : Collaboration, Resilience, and NO PROFIT. 2010. [Accessed on
September 18 from http://www.collaborativeye.com/collaboration_journal/decentralization-collaboration-
resilience-and-no-profit.html]
Davies, Lynne. Understanding education‘s role in fragility: synthesis of four situational analyses of education
and fragility – Afghanistan, Bosnia-Herzegovina, Cambodia, Liberia, University of Birmingham, 2010.
Department of State, Background Note on Liberia. U.S. government. 2010. [Accessed by on September 18,
2010 at http://www.state.gov/r/pa/ei/bgn/6618.htm]
Department of Research and Planning, Status of Education System in Liberia (2005/2006-2008/2009).
Liberian Ministry of Education, 2010.
Ernst and Young and MGI MONBO & Company, Draft Audit report on the Project financial statements of the
Education Pooled fund of the Ministry of Education pooled Fund of the Ministry of Education Liberian Primary
Education Recovery Program (L-PERP), 2009.
Fast Track Initiative. Recovery of the Education Sector in Liberia. 2010. [Access on September 18, 2010 from
http://www.educationfasttrack.org/newsroom/focus-on/fti-annual-report/recovery-of-the-primary-education-
sector-in-liberia/]
Government of Liberia/UNICEF Liberia. Education Programme 2007 Annual Work Plan. 2007.
Nkutu, Anne; Bang, Trude and Dorothy Tooman. Protecting children‘s right to education: evaluation of NRC‘s
Accelerated Education Program in Liberia. Nordic Consulting Group. 2009.
Johannessen, Eva Marion, A study of former TEP students in Angola and Burundi, NRC, 2005.
Kaabwe, Stella. Note for the Record – Subject: Liberia Education Pooled Fund Replenishment.
UNICEF: Monrovia. 2009.
Liberia Institute of Statistics and Geo-information Services. 2008 Population and housing census: final results.
Monrovia. 2009.
Liberia Institute of Statistics and Geo-information Services. Liberia Demographic and Health Survey 2007.
Monrovia. 2008.
MOE. Accelerated Learning Program Policy Guidelines. Government of the Republic of Liberia. 2008.
MOE, County and District Education Officers, Republic of Liberia, n.d.
MOE. Concept for the improvement and upgrading of the Education Management System (EMIS).
Department of Planning, Research and Development. Government of the Republic of Liberia. 2008.
MOE, Education Supplies for Montserrado County, n.d.
MOE, Education Supplies for Schools in Grand Gedeh, n.d.
MOE, Education Supplies for Nimba, n.d.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 45
MOE. Liberian Primary Education Recovery Program prepared for Fast Track Initiative. Government of the
Republic of Liberia. 2007.
MOE, Kakata, Margibi County ALP Teachers Listing, n.d.
MOE, List of MOE/UNICEF supported schools in Montserroda County, n.d.
MOE, List of MOE/UNICEF supported schools in Nimba County, n.d.
MOE. List of MOE/UNICEF supported schools in Sinoe County, n.d.
MOE, Listing of ALP Schools and teachers in Bomi County, n.d.
MOE, Listing of ALP Schools and teachers-Grand Bassa County, n.d.
MOE, Listing of ALP Teachers from Bong County, n.d.
MOE. Maryland County ALP Schools and teachers per district, n.d.
MOE. National Policy on Girl‘s Education-simplified version. Government of the Republic of Liberia. 2008.
MOE. MOU for the Liberia Education Pooled Fund Documents. Government of the Republic of Liberia. 2008.
MOE. National Policy on Girl‘s Education-simplified version. Republic of Liberia. 2008.
MOE, [School rehabilitation lists for UNOPS and LACE], n.d.
MOE, A system in transition: the 2007/08 National School Census Report. Government of the Republic of
Liberia. 2008.
MOE, [List of MOE/UNICEF Supported Schools and ALP teacher in] Yarnee District, Rivercess County
MOE and UNICEF. Liberian Enrolment Initiative: enhancing enrolment and completion of education for girls,
[n.d.]
Nicholson, Susan. Assessment of the Accelerated Learning Programme in Liberia Implemented by
UNICEF. 2007.
Osher, David, et al. UNICEF Child Friendly Schools Programming: Global Evaluation Final Report. American
Institutes for Research: Washington, D.C. 2009.
Preventionweb. Liberia profile. [Accessed September 21, 2010 from
http://www.preventionweb.net/english/countries/statistics/risk.php?iso=lbr]. 2010.
Republic of Liberia, Liberia‘s progress towards the Millennium Development Goals 2008, Monrovia. 2008.
Rodriquez, Luis, et al. Mid-term assessment of Liberia Teacher Training Program.
USAID: Washington, D.C. 2009.
Save the Children. Rewrite the Future: Improving access and quality of education for children affected by
conflict in Grand Gedeh and Bong County. 2010.
Schmidt, Caroline. The Education Pooled Fund in the Republic of Liberia: one mechanism, one program, one
process. UNICEF: Monrovia. 2009.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 46
Schmidt, Caroline and Aleesha Taylor. Liberia‘s Education Pooled Fund: a case for private foundation
engagement in post-conflict education recovery. Open Society Institute. 2010.
Scott, Aldophus, ‗Never too late to learn, 2008. [Access on September 21, 2010 from
http://www.unicef.org/infobycountry/liberia_42354.html]
UN. MDGinfo. [Accessed on September 21 at http://www.devinfo.info/mdginfo2010/]. 2010.
UNEGI, Liberia: background, 2010. [Accessed on October 2, 2010 from
http://www.ungei.org/infobycountry/liberia.html]
UNDP, Human Development Report. 2009. [Accessed online September 18, 2010 at
http://hdrstats.undp.org/en/countries/data_sheets/cty_ds_LBR.html]
UNDP. Human Development Report 2009 – Liberia. [Accessed online September 18, 2010 at
http://hdrstats.undp.org/en/countries/country_fact_sheets/cty_fs_LBR.html]
UNESCO, The Impact of Women Teachers on Girls Education, Paris, 2006. [Accessed on November 18,
2010 from Link: http://unesdoc.unesco.org/images/0014/001459/145990e.pdf]
UNESCO, Basic Learning Initiative, Accessed on January 10,
http://www.unesco.org/education/blm/chap1_en.php.
UNICEF. Child Friendly Schools Manual. New York. 2009.
UNICEF. Education in Emergencies: a resource took kit. UNICEF Regional Office for South Asia. 2006.
UNICEF, Education in Emergencies and Post-Crisis Transition: consolidated 2009 progress report to the
Government of the Netherlands and the European Union, 2010.
UNICEF. Education in Emergencies and Post-Conflict Transition: a synthesis of main finds from evaluations
from 2004-2009. Evaluation Office Working Paper. UNICEF: New York. 2009.
UNICEF, [Global Framework of LAB4LAB and Talent Academies –Draft], New York, 2008.
UNICEF. Netherland‘s support to education in post conflict transition in Liberia. UNICEF: Liberia. 2010.
UNICEF, The Project for support to Child-Friendly Schools Development, Project Proposal submitted to the
Government of Japan, Ministry of Affairs, 2010.
UNICEF, Proposal on Education in Emergencies and Post-Crisis Transitions (presented to the Ministry of
Foreign Affairs, Government of the Netherlands), 2006.
UNICEF. Report of the workshop of the Project ―Learning along borders for living across boundaries‘ and
‗Talent academies‘.2009.
UNICEF, [Results Framework Response], 2010.
UNICEF. Self-assessment. Monrovia. 2010.
UNICEF. Talent Academy Report: assessment of the talents, potentials, & preferences of out-of-school young
people in urban communities in Liberia. Monrovia. 2009.
UNICEF, UNICEF Liberia: G3001 Netherlands, UNICEF Liberia Country Office: Monrovia, 2007.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 47
UNICEF, UNICEF Liberia: Annual Report 2007, Government of Liberia –UNICEF Programme of Cooperation.
Monrovia. 2007.
UNICEF. UNICEF Liberia: Annual Report 2008. Government of Liberia –UNICEF Programme of Cooperation.
Monrovia. 2008.
UNICEF. UNICEF Liberia: Annual Report 2009. Government of Liberia-UNICEF Programme of Cooperation.
Monrovia. 2010.
UNICEF, Utilization of Donor Funds –EEPCT Liberia. 2010.
Wayne, Trokon and von Hahmann, Gail. Achieving quality in accelerated learning: Student assessment in
Liberia: paper presented at the annual meeting of the 53rd Annual Conference of the Comparative and
International Education Society, Francis Marion Hotel, Charleston, South Carolina. 2009 .
World Bank. Issues in program design: education and decentralization. Decentralization Thematic Group.
[Accessed on September 18, 2010 at
http://www.ciesin.columbia.edu/decentralization/English/Issues/Education.html]
World Bank, Appraisal of the 2010-2020 Education Sector Plan, 2010. [Accessed on September 18, 2010 at
http://www.educationfasttrack.org/media/library/Country_Documents/Liberia/ESP_Appraisal_Endorsement_R
eport.Final-Fe-.27-2010.pdf]
World Bank. Data. 2010 [Accessed September 24, 2010 from http://data.worldbank.org/country/liberia]
World Bank and Government of Liberia, Public Expenditure Management and Financial Accountability
Review, 2009.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 48
ANNEXES
Annex 1: Methodology
Purpose
The purpose of the PREV is to determine the outcome and indicative impact of EEPCT and draw on lessons
learned and recommendations that will be useful for strengthening the programme‘s on-going practices and
policies in the years to come. Within this context, the PREV seeks to achieve three inter-related objectives:
Provide an outcome-indicative impact analysis of the EEPCT Programme (2006-2009)
Examine DAC evaluation criteria as applied to education in emergencies, transitions and fragility
Provide evidence-based conclusions, lessons learned and recommendations.
Approach
At the country level, the PREV‘s focus was on programme implementation, results and learning as they relate
to beneficiary and education systems outcomes, with attention to the incorporation of standards, and the level
of quality and innovation involved in implementation. It examined how the goals were translated and adapted
to country contexts and why. In addition, the PREV sought to determine the extent to which global and
regional strategies and outputs are supporting change at the country level by documenting global and regional
inputs in the six case study countries.
The country case studies were designed to both fit into the larger global assessment of the EEPCT
programme and to act as stand-alone studies of EEPCT implementation and outcomes in different national
contexts. This report represents a detailed description and analysis of EEPCT‘s operations, impact, lessons
learned and ways forward in Liberia, while the global report will synthesize findings on country, regional and
global levels.
Evaluation Methodology
The evaluation employed a sequential mixed-methods approach, drawing on both quantitative and qualitative
methods, combining comprehensive coverage with in-depth analysis. This approach strengthens validity
through triangulation, as well as by using the results of one method to support the informed development of
others. These methodologies allow for in-depth exploration of methods and innovation as well as the collection
of comparable data across countries.
Cross Cutting Issues
The PREV also examined five cross-cutting issues from perspectives that would affect EEPCT‘s ability to
meet its goals. These include:
Gender: Are equity, equality, and empowerment taken into account in EEPCT programmes? Do
gender elements of programming include the needs of girls and women and boys and men?
A rights-based approach to programming: Is a rights-based approach a cornerstone for all
programming? Does EEPCT help increase participation and equitable inclusion?
Disaster Risk Reduction: Does EEPCT help minimize vulnerabilities and disaster risks throughout a
society? Does it take measures to avoid (prevention) or to limit (mitigation and preparedness) the
adverse impacts of hazards, within the broad context of sustainable development?
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 49
Sensitivity to conflict and fragility: Have conflict and fragility-sensitive approaches been implemented
when possible and can these be institutionalized?
Accountability, monitoring, evaluation and learning: To what degree have best practices in monitoring,
evaluation and knowledge transfer been adopted in implementing the programme?
Data Collection and Analysis
Six countries were identified for the field research component and were used as case studies for the PREV.
The research plan allowed for six to eight weeks per country. A GCA-Inter-Agency Network for Emergency
Education (INEE) Memo of Understanding also enabled close collaboration with education agencies and their
staff in each country where a case study was conducted.
The evaluation employed a sequential mixed-methods approach - that is, it drew on both quantitative and
qualitative methods - to combine more comprehensive coverage with in-depth analysis. The approach aimed
to: strengthen validity through triangulation; enable the use of results of one method to help develop the
instrument of the other; extend the comprehensiveness of the findings and generate new insights. The
research team employed participatory methods in its qualitative data collection with beneficiaries—children,
parents, educators and local communities members—to ensure an upstream flow of findings and
recommendations. Careful attention was paid to structuring these participatory approaches to ensure they
were transparent, systematic and replicable. Field visits also documented the extent to which global and
regional contributions (INEE Minimum Standards for Education in Emergencies, Chronic Crisis and Early
Reconstruction, Education Cluster coordination and Regional Office technical support) contributed to results at
the country level.
Data Collection Tools
Data sources included document reviews, interviews with key informants, surveys, field visits and FGD that
were conducted with children, youth, parents, and educators. The above information, collected in each
instance according to the programming priorities of EEPCT, was used to identify and link results in terms of
approaches, processes and outputs.
Primary and Secondary Literature Review. This body of literature ranged from government documents to
publications produced by local and international NGOs to academic literature. The PREV Team also reviewed
financial allocations relative to the four programme goals and the cost-effectiveness of EEPCT.
EEPCT Logframe This instrument is used for tracking EECPT indicators over time per its four goals. This
instrument is also intended to provide data to the achievement of EEPCT comprehensive aim and goal.
UNICEF Self-Assessments. As part of the global analysis, COs assessed their implementation of EEPCT by
describing the local context, the specific aims and goals that were established for EEPCT. Cos also assessed
the extent to which these aims and goals are being achieved and, importantly, the reasons that they are not.
Adequacy Survey: An adequacy survey was used to examine whether programmes launched at global,
regional and country levels completed what they set out to do in terms of the activities, trainings, material
development and services planned in order to achieve specific objectives or results. The adequacy survey
focused on the key tasks, inputs and outputs, proposed in the EEPCT plans, and sought to verify the extent to
which UNICEF and its implementing partners met their ―input and output‖ commitments.
Individual Interviews: One-on-one interviews were conducted with relevant UNICEF staff, government officials,
NGO staff and other key stakeholders at the community, regional and global levels. Identification of ―key
informants‖ was done in consultation with UNICEF at all levels. Interview questions were guided by the
information garnered from the document reviews and the issues highlighted in the PRES.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 50
Key Stakeholders Consultations: Stakeholders were identified at the global, regional and country levels. In
case study countries, the PREV evaluation team worked with the UNICEF CO to identify key government (e.g.
ministries of education, planning, child development and gender, etc.), civil society (e.g. international NGOs,
national NGOs, academic advisors, etc.), donors (e.g. EC, ECHO, DFID, etc.) and others who were most
knowledgeable of the education sector in general and the EEPCT programme in particular.
Such FGD were conducted using the following tools:
Emergency Education - DAC Score Card (EE-DAC): The tool was used at different levels (national, sub-
national and community) within Liberia to quantifiably capture perceptions of the obtainment of EEPCT Goals
based around the five OECD/DAC criteria (Relevance/Appropriateness, Effectiveness, Efficiency, Coherence
and Coordination and Impact and Sustainability).
It ensured consistent data collection at different levels (national, sub-national and community) within Côte
d‘Ivoire. The interviewees used the score ranging from 1 to 5 for the assessment based on the DAC criteria,
with 1 indicating ―much worse‖, 5 indicating ―much better‖, and 3 indicating ―the same/no change‖. Following
the rating, the interviewee provided reasons positive (+) or negative (-) scoring. Their perceptions on
recommendations for strengthening EEPCT were also enlisted. The Score Card approach allows quantitative
findings to emerge as well as qualitative explanations regarding these findings.
Participative Ranking Methodology (PRM): PRM is a participatory approach to data collection in which a
group of knowledgeable participants are guided in responding to a specific question or set of questions. This
methodology promotes an engaged and participatory process that rapidly highlights key findings while
providing the opportunity for deeper analysis as resources permit. The PRM is particularly effective at
engaging children, youth, parents and educators ensuring inclusivity (male and female). . Collected in a
structured manner, responses were consolidated and used to inform this formative evaluation.
Focus Group Discussion (FGD): The focus group discussions provided the opportunity to gather responses on
changes of specific areas of implementation in both qualitative and quantitative manners. These tools were
used specifically at the community level with children, youth, parents and, where available, educators.
Child Friendly Schools Criteria Checklists: Child Friendly School (CFS) checklists were used to evaluate
schools fulfilment of selected CFS criteria. 137
The tool was developed by the global PREV team for broad use
in case study countries, and not solely intended for programmes implementing CFS, to assess how well
EEPCT-assisted schools complied with CFS criteria. In Liberia, the tool was used to evaluate both the
LAB4LAB School as well as EPF supported schools to gauge consistency of education construction.
Field Site Selection
Field visit sites in Liberia were selected through both random and purposive sampling processes. Based on
consultations with LCO and the national research team, Monterrado and Nimba Counties were chosen for
field site visits due to the fact that the majority of the country‘s students are enrolled in these two counties.
The LCO was consulted for clarity on the EEPCT programme components. When these were identified, the
locations of programme sites were determined. Because the textbook programme that was administered as
part of the LPERP and supported by the EPF was nationalized, comparison groups of public schools not
benefitting from the programme were not possible. As the ALP was implemented in all counties, but not all
schools, a list of all UNICEF-supported ALP programmes was obtained from the MOE. With these lists,
schools were randomly selected using a random number generator. Sites were then selected based on
137 UNICEF, Child Friendly Schools Manual. New York. 2009. Criteria included the following: These criteria included the following: whether or not the school appeared child friendly; if classrooms were easily exited in emergency; all classrooms have windows; separate spaces are provided for teachers and administrators; teachers and administrator location allows for monitoring students; availability of water on school grounds; separate latrines for boys and girls; locks on latrine doors; appropriate ratio of latrines to boy and girl students; separate latrines for teachers; separate space with soap and water for hand washing; and presence of a disaster risk reduction plan.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 51
accessibility and distance. Due to time and accessibility constraints, the evaluation sought to visit programme
rich sites.
Sample Selection
The categories of stakeholders for the purpose of sampling were selected at three different levels as follows:
The National Level (Government Officials, MOE, and international and national NGO Staff)
The Sub National Level: (Government Officials, CEOs and DEO, and international and national NGO staff)
The Community Level (Students – separated by gender and age, teachers – mixed gender groups of
teachers/educators, and parents –separated by gender when possible). FGD participation can be viewed in
Table 3:
Focus Group Discussion Participation
FGD
Participants
Total %
National Sub-National Community
Government 3 6 18 0 24 13.9
INGOs & NGOs 2 4 3 0 7 4
Teachers 8 0 0 43 43 24.9
Students 11 0 0 81 81 46.8
Parents 2 0 0 18 18 10.4
Total 26 10 21 142 173 100
Data Analysis
Quantitative data analysis was performed on the adequacy survey and EE-DAC Score Card. Data was
entered in Excel by national researchers on a daily basis and analysed using SAS. The participatory ranking
exercise and FGDs all yielded data that was primarily quantitative.
Analysis of qualitative data collected from key stakeholder interviews, FGDs and PRM exercises drew on
direct quotes to exemplify themes found during data collection and was summarized in a memo format, which
looked for emergent categories and how they related to the broader research. Themes and commonalities
within and across categories were identified. The literature review and interviews were used to add depth and
explanation to the themes that emerged, as well as to develop and identify emergent categories.
Evaluation Team
The overall management of the CGCA Evaluation Team consisted of two evaluation co-leaders, a
biostatistician, finance administrator, and programme officer. In addition to overseeing and supporting the
work of evaluation teams in the six case study countries, the co-leaders and support staff were responsible for
implementation of global evaluation activities, as well as final data analysis, synthesis and report writing.
Table 3
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 52
The Liberia case study team consisted of two co-team leaders, a Columbia University Research Associate
and three national researchers. Regular communication between the case study and global teams was
maintained and weekly updates were provided to the CGCA management team to ensure that the evaluation
process ran smoothly and to address any questions or difficulties as they arose.
Limitations
The evaluation team took multiple steps to ensure the methods used in Liberia were consistent with those
utilized by research teams in the other case study countries. To achieve this, several materials and plans were
made to mirror approaches and actions in each country, including:
The development of an introductory PowerPoint to be used at stakeholder consultations delineating the goals,
purpose, design and tools used in evaluation;
The creation of user guides for each tool and instructions for all data collection methods;
A central training was conducted, followed by parallel trainings in each country for the full field
research teams;
A consistent approach developed to data entry, including creation of a singular entry form, and
supervision of research teams during the field work stage;
Bi-weekly calls between research teams and the evaluation leaders;
The lack of global clarity and common understanding on multiple terms within the sector limits a theory based
approach to the EEPCT programme itself and, by extension, this evaluation. This limitation includes the
meaning of terms within the programme‘s name (emergency, post-crisis, transition) and within its goals
(quality education, resilient education system). For these reasons, the CGCA created blogs that seek to
provide a platform to clarify these terms. Despite the development of definitions, terminology and concepts
within the tools continued to be difficult for FGD participants at the community level to comprehend due to
differences in education level, insufficient English language abilities or lack of similar words within vernacular
language.
LCO did not establish a programme baseline, which made reporting on percent change in goal area outcomes
impossible. The evaluation team addressed some of these concerns through the use of retrospective
baselines to determine programme outcome related findings. Retrospective baselines helped to facilitate
plausible outcome determinations in the absence proper baseline and monitoring procedures.
In addition to the above overall framework limitations, the following had an impact in conducting the evaluation
at field level in Liberia:
1. Timing of the evaluation: The evaluation coincided with school holidays which created challenges for
gathering participants from all cohorts for FGDs at the community level. This was mitigated by
contacting regional officials and community members prior to the evaluation visit. However, far fewer
FGDs were run with parents, teachers and regional educators than with youth.
2. Knowledge of EEPCT: Few key informants or focus group discussion members knew of EEPCT.
This hindered the application of the PREV tools.
3. Decentralization of information: While the MOE is UNICEF‘s only implementation partner at present,
UNICEF did not hold all information regarding programme sites, locations and participants. Therefore,
it was necessary to obtain this information directly from different relevant parties at the MOE which
slowed the planning process.
4. Availability of CEOs and DEOs: In the case of Nimba County, the County Education Officer (CEO)
and District Education Officers (DEOs) were not present at the time of the PREV team‘s mission due
a workshop in another county. While a point person was appointed by the CEO to assist the team in
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 53
coordination, the decentralization of information at the county and district levels limited the PREV
team‘s ability to contact schools for site visits.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 54
Annex 2: Daily Schedule
Date Time Description Status Responsibility
Ongoing Literature Review Completed
6 July 9:30am Introductory meeting with UNICEF CO and
Education Team
Completed Evaluation team
7 July Training on EE-DAC Scorecard
Training on local calendar
Completed Natasha
8 July
9:00am
Interview 1 with Former MOE
Interview 2 with MOE
Interview 3 with MOE
Completed Evaluation team
9 July
10:30am-
12:30pm
1:00pm-
2:30pm
3:30pm
Training on PRM
Meeting with DEOs and assistants from
Montserrado, Grand Gedeh and Nimba
counties
PRM exercise with test group of
neighborhood children
Completed Evaluation team
12 July
10:00am
2:00pm
4:00pm
Preparation for launch
Launch of PREV at MOE
Interview 4 with MOE
Completed Evelyn, Natasha
Evelyn, Natasha
Evaluation team
13 July
10.00a.m.
12:30pm
2:00pm
4:00 pm
EE-DAC Scorecard with national government
personnel
Data entry
EE-DAC Scorecard with national level NGOs
EEPCT Programme overview with UNICEF
Completed Evaluation team
Evaluation team
Natasha
14 July
10:00am
11:00 am
2:00pm
4:00pm
4:30pm
Data entry
EE- DAC Scorecard with UNICEF
Interview with MOE
Interview with former MOE
Retrieve additional documents from UNICEF
Incomplete
Complete
Evaluation team
Natasha
15 July 10:00 am
11:00am
Field Work Montserrado County
Interview 5 with CEO, Montserrado County
Completed Evaluation team
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 55
2:00pm
EE-DAC Scorecard with Montserrado County
CEO and District Education Officers
Field Site 1: Montserrado County
School 1, Todee District
Field Test tools
School Construction Checklist – School 1,
Montserrado County
16 July
10:00am
2:00pm
Evening
Field Site 1: Montserrado County
School 1, Todee District,
Field test tools
Field Site 2: Montserrado County
School 2, Careysburg District,
Data entry
Completed
Evaluation team
19 July
9:30am
Evening
Data entry De Giuli
Sylvia De Guili (Evaluation Specialist, NYC)
arrives in Monrovia
Completed Evaluation team
20 July
10:00am-
3:00pm
9:00pm
Data entry
Silvia meets with Natasha re scheduling
Completed Evaluation team
21 July
9:30 a.m. PREV Liberia team meeting with Silvia at
UNICEF
Completed Evaluation team
23 July
9:30am
11:00am
Evening
Interview 6 with UN
Interview 7 with World Bank
Carl Triplehorn arrives
Completed
Completed
Evelyn, Natasha
24 July 8:00am Interview 8 with UNICEF Regional Rep
Meeting with PREV Liberia Team
Review revised schedule
Completed Carl, Natasha
Evaluation team
25 July
3:00pm –
5:00pm
5:00pm
Team meeting
Complete team training
Evaluation team
Evaluation team
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 56
26 July
11:00am
Liberian Independence Day
Meeting with Silvia re schedule
Literature review
Completed Carl, Natasha
27 July
9:00am -
6:00pm
Evening
Field work in Montserrado and Bomi
Counties
School construction checklist:
School 2, Montserrado County
School 3, Bomi County
School 4, Bomi County
Interview 9 with UNICEF
Meeting with Isabel Crowley
Meeting with Silvia re inception report
Completed
Completed
Completed
Completed
Evaluation team
Carl
Carl
Natasha, Carl
28 July
9:00am -
6:00pm
Evening
Interview 10 with LACE
Interview 11 with Architecture for Humanity
Completed
Completed
Evelyn, Carl
Carl
29 July
9:00am -
6:00pm
Evening
Field site 2: Montserrado County
School 2, Careysburg District
Interview 12 with Principal
PRM/FGD
Data entry
Completed Evaluation team
9:00am Interview 13 with Save the Children Completed Carl
30 July
9:00am -
6:00pm
Evening
Field Site 3: Montserrado County
School 3, Left Bank District
Interview 14 with Principal
PRM/FGD
Data entry
Completed Evaluation team
10:00am
11:00am
Interview 15 with former MOE
Interview 16 with PFMU
Completed Carl and Evelyn
31 July 2:00pm Interview 17 with MOE Completed Carl, Natasha
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 57
7:00pm
Interview 18 with DEO, Nimba County
Nimba County Planning
Interview 19 with UNICEF
1 August
9:00am
3:00pm
Evening
Depart for Nimba County
Field Site 4: Nimba County
Programme Focus: CFS
FGD – Teachers
Key informant interview 20 with LAB4LAB
school
Data entry
Completed Evaluation Team
Evening Preparation for field work Evaluation Team
2 August
8:30a.m.
10:00am.
Visit LAB4LAB School
Field Site 4: Nimba County
School 4, Sanniquellie District
Interview 21 with Principals
Conduct PRM/FGD with:
Children (9 – 13 years) boys
Children (9 – 13 years) girls
Youth (14 – 18 years) girls
Youth (14 – 18 years) boys
Total 4FGDs
Data entry
Completed Evaluation team
Montserrado County NGOs FGD –EE-DAC Incomplete Evelyn
3 August
09.00a.m.
– 06.00pm
Field Site 5: Nimba County
School 5, Bain-Gar District
Interview 22 with Principals
Conduct PRM/FGD with:
Children (9 – 13 years) girls
Teachers –mixed
Parents - mixed
Completed Evaluation team
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 58
Evening
Field Site 6: Nimba County
School 6, Baing-Gar District
Interview 23 with Principals
Conduct PRM/FGD with:
Teachers –mixed
Total 4 FGDs
Data entry
1:00pm
EE-DAC FGD with Nimba County CEO and
DEOS – Gbanga, Bong County
Completed Carl
4 August
9:00am -
6:00pm
Evening
Field Site 7: Nimba County
School 7, Saclepea District
Interview 24 with Principal
PRM/FGD with:
Children (9-13) boys
Youth (14-18) girls
Parents - mixed
Teachers
Total 3 FDG
Data entry
Completed Evaluation team
9:30am
2:00pm
Interview 25 with MOE
Interview 26 with EU representative
Interview 27 with USAID
Interview 28 with UNICEF
Completed Carl, Evelyn
5 August
9:00am -
6:00pm
Field Site 8: Nimba County
School 8, Zoe-Geh District
Interview 29 with Principal
PRM/FGD with:
Children (9-13) - girls
Completed Evaluation team
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 59
Evening
Youth (14-18) - boys
Teachers - men
Total 3 FDG
Data entry
8-12:00
12 –
Evening
Logistics
Data entry
Complete Carl, Evelyn
6 August
9:00am
12:00noon
2:00pm
Evening
EE-DAC FGD - Nimba county NGOs
Field Site 9: Nimba County
School 9, Sanniquellie District
Interview 30 with Principal
PRM/FGD with:
Teachers - mixed
Field Site 10: Nimba County
School 10, Sanniquellie District
Interview 31 with Principal
PRM/FGD with teachers
Data entry
Completed Evaluation team
11:00am Debriefing at UNICEF Completed Carl. Evelyn
7 August
8:00am
1:00pm
4:00pm
Return to Monrovia
Visit Kakata RTTI, Bong County
Team meeting and debriefing
Completed
Evaluation Team
8 August 6:00am Depart Monrovia to Côte d‘Ivoire Completed Carl, Natasha
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 60
Annex 3. Interviews Conducted: National, Sub National and Community Levels
Liberia Key Informant Interviews
Level Number of Interviews
Regional 1
National 20
Sub-national 2
Community 5
Total 28
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 61
Annex 4: Liberia EEPCT Evaluation Framework
Liberia EEPCT Programme
Year Activity Location of
Implementation
Goal 1: Improved quality education response in emergencies and post-conflict countries
Educational
materials and
instructional
support
2007 Trained 250 primary school teachers to receive C level
certification
Magribi, Grand
Bassa, Bong,
Nimba
2008 Dispersed support through pooled fund to purchase 1.2
million in English, Math, General Science and Social
Studies with respective teachers' guides, Textbook Policy
and Instructional Material Management Handbook and
teacher training
All government
schools
2009 Supported printing of 125,000 consolidated of
supplementary readers for grades 2 and 3.
All government
schools
Infrastructure
2007 Renovated Montserrado County Education Office Montserrado
2008 Renovated 2 DEO complexes Montserrado,
Bomi
2008 Constructed 1 DEO/CO office complex Gbarpolu
2008 Constructed 2 warehouses Bomi,
Montserrado
2008 Purchased and distributed 20,000 arm chairs 10 Counties
2008 Dispersed support through the Education Pooled Fund to
support to revitalization of three Rural Teacher Training
Institutes (RTTIs) including logistical support, building and
furnishing of teachers' housing, vehicles, cookers,
residential furniture.
2009 Constructed CEO office complex and warehouse Grand Bassa
2009 Constructed generator house, wells, and water towers for
CEO offices
Monteserrdo,
Bong, Bomi,
Gbarplu
2009 Repaired and installed two 25 KVA generators Bomi, Gbarpolu
2009 Furnished CEO office complex Gbarpolu
2009 Dispersed support through the Education Pooled Fund to All counties
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 62
support construction of 40 schools
Technical
Support
2007 Hired consultant to provide technical assistance for
development of sector wide approach
National
2008 Hired consultant to provide technical assistance for the
LPERP implementation plans
National
2009 Assisted MOE to develop 10-year Education Sector Plan
(ESP)
National
2009 Hired coordinator for Education Development Partners
Meetings
National
2009 Hired consultant to support government to write School
Abolish Policy
National
Child Friendly
Schools
2009 Trained 55 Trainer of Trainers for the Child Friendly
School Programme
Gbarpolu
2010 Trained an estimated 500 teachers and principals in Child
Friendly School approach
National
Goal 2: Increased resilience of education sector services delivery in chronic crises, arrested development
and deteriorating contexts
Goal 3: Increased education contribution to better prediction and preparedness for emergencies due to
natural disasters and conflict
ALP 2007 Procured 140,000 learners kits for ALP and Grade 1 National
2007 Procured and distributed recreational kits for ALP
2007 Hired 2 logisticians for end-user monitoring of ALP supplies
2007 Trained 994 teachers in ALP refresher course Montserrado,
Bomi, Bong, Lofa,
Maryland, Nimba,
Grand Gedeh
2008 Supported XXXXX student in ALP programmes including
payment of teachers, learning materials and monitoring
11 Counties
2008 Procured and distributed 66,125 learners kits to ALP and
Grade 1 students
11 Counties
2008 Hired logisticians to monitoring to monitor distribution of
learners kits
2008 Provided 948 teachers initial and refresher training 11 Counties
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 63
2008 Provided 276,000 copies of 3 types of Grade 1 and
Readers and ALP Level 1 in kits
11 Counties
2008 Developed, printed and disseminated ALP Policy
Guidelines
National-level
2009 Expanded ALP to four new counties Grand Bassa,
Rivercess,
Margibi, Sinoe
2009 Funded printing and distribution of 10,000 ALP Policy
Guidelines
National-level
2009 Supported 30,785 ALP students ( paying teacher, providing
materials, and monitoring)
11 Counties
2009 Printed and distributed 450 sets of ALP Manuals for a
Levels 1, 2, 3
11 Counties
LAB4LAB 2008 Paid for national focal point for LAB4LAB Ganta, Nimba
County
2008 Obtained title deed for LAB4LAB Ganta, Nimba
County
2008 Supplied transportation (2008-2010) Ganta, Nimba
County
2008 Supported staff DSA(2008-2010) Ganta, Nimba
County
2008 Awarded construction contract for LAB4LAB Ganta, Nimba
County
2009 Started construction of LAB4LAB Ganta, Nimba
County
2010 Opened LAB4LAB Ganta, Nimba
County
Talent
Academies
2009 Conducted an assessment for Talent Academies look a report
Goal 4: Evidence-based policies, efficient operational strategies and fit-for-purpose financing instruments
for education in emergencies and post-crisis situations.
EMIS
2007 Supported EMIS Technical Manager to develop tools for the
collection of educational data
Country-wide
2007 Trained enumerators in revised education tools Country-wide
2007 Conducted census
2008 Completed, published and distributed 15,000 2007-8
National School Census
Country-wide
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 64
2008 Supported EMIS Technical Manager to continue EMIS
assistance
Country-wide
2008 Trained enumerators on educational tools Country-wide
2009 Funded 2008-2009 census Country-wide
2009 Provided technical and financial support for first phase in 3
counties of national school mapping and micro-planning
Gbarpolu, Bomi,
Grand Cape
Mount
Education
Pooled
Fund
2007 Supported the Deptuy Director from PARMO to provide
technical support
Country-wide
2008 Support the Deputy Director from PARMO to provide
technical support
Country-wide
2008 Established Liberia Education Pooled Fund Country-wide
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 65
Annex 5: Evaluation Tools
EE-DAC Score Card
The EE-DAC Score Card included in the Inception Report, is one of the tools to be used in the country case studies undertaken in the EEPCT evaluation. It is based on EEPCT Goals consisting of (1) Improved quality of education response in emergencies and post crisis transition, (2) Increased resilience of education service delivery, (3) Increased education sector contribution to better prediction, prevention and preparedness for emergencies caused by natural disaster and conflict and (4) Evidenced-based policies, efficient operational strategies and fit-for-purpose financing instruments which are reflected in the form of evaluation questions. The OECD / Development Assistance Criteria (DAC) is applied for the assessment of the achievement of such Objectives as defined below: 1. Relevance / Appropriateness
Whether project is in line with local needs and priorities (as well as donor policies). Appropriateness is the tailoring of humanitarian activities to local needs, increasing ownership, accountability and cost-effectiveness accordingly
2. Effectiveness The extent to which an activity achieves its purpose or whether this can be expected to happen on the basis of outcomes. Implicit within the criterion is timeliness
3. Efficiency The outcomes- qualitative and quantitative – as a result of the inputs. This generally requires comparing alternative approaches to achieving an output, to see whether the most efficient approach has been used
4. Coherence and Coordination The need to assess security, developmental, trade and military policies as well as humanitarian policies, to ensure that there is consistency and, in particular, that all policies take into account humanitarian and human-rights considerations
5. Sustainability / Connectedness
Connectedness refers to the need to ensure that activities of a short term emergency nature are carried out in a context that takes longer term and interconnected problems into account.
The tool is to be used at Focus Group Discussions (FGDs) with the different stakeholder categories to assess EEPCT based on key selected elements stated as evaluation questions in the format provided. The different stakeholder groups will analyze in their own respective groups and come up with a collective ranking using the scores 1 to 5, with 1 being the lowest and 5 being the highest score for the achievement of the specific objectives. A score of 8 can be assigned if respondents collectively agree they do not know the answer, or a 9 if they choose not to respond at all. Following the scoring, the interviewees / focus group participants would be asked the reasons for their assessment with a (+) or (-) indicative of positive or negative reasoning. This would be followed up by qualitative recommendations or comments for improvement of programme success / delivery as perceived by the stakeholder groups. The format appearing in the following page should be used for the assessment and the process steps are outlined below:
Identify the different stakeholder categories at the different levels as follows and organize 3 different meetings at each level lasting 1 hour per group.
i. National level – Policy / decisions makers on education: Government, UN and NGOs
ii. Sub National / District level – Education Supervisors-Implementers: Government, UN and NGOs
iii. Community level – Teacher-Programme staff (only) The above meetings will bring together Government, UN, and INGO-National NGOs and educators-
programme staff who would for their own groups, discuss the questions and agree on a collective ranking (1-5, 8 or 9 for each) and indicate the score by a ―‖ on the respective box in the format.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 66
Facilitate discussions in the groups to probe into the causal factors / reasons (+ positive or - negative) for the ranking and to come up with recommendations for improvement to be followed up. Ask groups to site 3 reasons for their rankings. Write down answers exactly as stated (do not interpret or summarize comments).
The role of the facilitators (2 facilitators per meeting—1 to facilitate and 1 to support and record) is
considered very significant in moderating / focusing the discussions and maintaining objectivity. They should be prepared in advance as follows:
- Make it clear to the participants that this evaluation is a learning process so that it lends itself for capacity building and self-assessment.
- Be familiar with the detailed programme activities related to EEPCT so that the group facilitation could be done effectively.
- It may be helpful to display on flip chart papers the four Goals and sub components so that when the assessment is done the participants could refer to them or else could be distributed as handouts.
- Be familiar with the definitions of DAC to be able to explain to the participants.
- It would be useful to display the definitions of DAC for reference by the participants or else could be distributed as handout.
- Have the formats for each group prepared on large sheets of paper as working document which could be self documented by the participants.
- If time permit group presentations for plenary consensus may be done.
It is envisioned that this tool will be used for field studies and key informant interviews at the global and regional levels. In each group scenario, a participatory approach will be adopted using the EE-DAC Score Card as a framework to collect the necessary qualitative data based on stakeholder perceptions. The scoring component of the score card will yield quantitative results as well. The EE-DAC Score Card forms filled by the different groups should be later consolidated into a single format with the summary of reasons and recommendations for each level. The consolidated formats for each level in the different countries can be taken up for comparative analysis for assessing the status in the overall analysis. An excel data entry programme will be used to enter data. Data should be entered daily. The initial pilot should be shared with NY team to verify data is being consistently collected and coded. Final results will be sent to NY as soon as completed. Data analysis results will be returned to the field team within 2 weeks to be incorporated into country case studies. Data will also be examined across countries for the synthesis report.
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 67
Questions
Score Reasons for the Score
( + Positive / - Negative)
Key Recommendations /
Strategic Focus (based on gaps
identified) 1
Much Worse
2 Worse
3 Same
4 Better
5 Much Better
8 Don’t Know
9 No
Resp
Relevance/Appropriateness
1. Has access to education in emergencies or post crisis transition changed since the programme began?
-
2. Has the quality of education in emergencies or post crisis transition changed since the Programme began?
-
3. Has girls‘ enrolment in schools changed since the Programme began?
-
Effectiveness
4. Have education system‘s ability to recover from emergencies changed since the Programme began?
-
5. Has government preparedness and early warning knowledge changed since the Programme began?
-
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 68
Efficiency
6. How well does the Programme perform activities compared to other similar programmes?
-
7. Has child safety in schools changed since the Programme began?
-
Coherence and Coordination
8. Has implementation of INEE minimum standards changed since the Programme began?
-
9. Has education cluster support of the Ministry of Education coordination role changed since the Programme began?
-
Sustainability/Connectedness
10. Has education sector‘s ability to prepare for and respond to emergencies changed since the Programme began?
-
11. Has government capacity to support education in emergencies and-or post crisis transitions changed since the Programme began?
-
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 69
12. Has monitoring and evaluation of the education system changed since the Programme began
-
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 70
FGD DATA COLLECTION FORM (CHILDREN, YOUTH)
Question: What are the strengths of the programme?
Date: Group: Children / Youth
Community: Gender: Girls / Boys
Moderator: Number of Children in Group:
Note taker: Age Range:
Key Strengths Identified:
Free list: Rank Order:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
COMMENTS:
(Write down what the children say using their exact words.)
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 71
FGD DATA COLLECTION FORM (CHILDREN, YOUTH)
Question: What are the weaknesses of the programme?
Date: Group: Children / Youth
Community: Gender: Girls / Boys
Moderator: Number of Children in Group:
Note taker: Age Range:
Key Weaknesses Identified:
Free list: Rank Order:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
COMMENTS:
(Write down what the children say using their exact words.)
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 72
FGD DATA COLLECTION FORM (PARENTS, EDUCATORS)
Question: What are the strengths of the programme?
Date: Group: Parents / Educators
Community: Gender: Men / Women / Mixed
Moderator: Number of Participants:
Note taker:
Key Strengths Identified:
Free list: Rank Order:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
COMMENTS:
(Write down what the participants say using their exact words.)
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 73
FGD DATA COLLECTION FORM (PARENTS, EDUCATORS)
Question: What are the weaknesses of the programme?
Date: Group: Parents / Educators
Community: Gender: Men / Women / Mixed
Moderator: Number of Participants:
Note taker:
Key Weaknesses Identified:
Free list: Rank Order:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
COMMENTS:
(Write down what the participants say using their exact words.)
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 74
FGD DATA COLLECTION FORM (PARENTS, EDUCATORS)
Question: What makes an education system resilient?
Date: Group: Parents / Educators
Community: Gender: Men / Women / Mixed
Moderator: Number of Participants:
Note taker:
Key Qualities Identified:
Free list: Rank Order:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
COMMENTS:
(Write down what the participants say using their exact words.)
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 75
FGD DATA COLLECTION FORM (YOUTH)
Question: What makes an education system resilient?
Date: Group: Youth
Community: Gender: Girls / Boys
Moderator: Number of Children in Group:
Note taker: Age Range:
Key Qualities Identified:
Free list: Rank Order:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
COMMENTS:
(Write down what the youth say using their exact words.)
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 76
FGD DATA COLLECTION FORM (CHILDREN)
Date: Group: Children
Community: Gender: Girls / Boys
Moderator: Number of Children in Group:
Note taker: Age Range:
Note establish a common time reference for both programme and control groups which could be “since the emergency” or another
commonly shared event.
Much
worse
Worse No
change
Better Much
Better
Don‘t
know
N/A-
NR
For Children 1 2 3 4 5 8 9
How much have your feelings about school changed since the
programme began, if at all?
How much has your attendance changed, if at all?
How much has girls‘ participation changed, if at all?
How much has boys‘ participation changed, if at all?
How much has your sense of safety and security changed, if at all?
How much has your ability to protect yourself if there is another
emergency changed, if at all?
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 77
How much has the amount students help each other changed, if at all?
How much has students‘ involvement in problem solving at school
changed, if at all?
How much has the use of fighting to resolve interpersonal differences
changed, if at all?
COMMENTS:
(Write down what the children say using their exact words.)
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 78
FGD DATA COLLECTION FORM (YOUTH)
Date: Group: Youth
Community: Gender: Girls / Boys
Moderator: Number of Youth in Group:
Note taker: Age Range:
Note establish a common time reference for both programme and control groups which could be “since the emergency” or another
commonly shared event.
Much
worse
Worse No
change
Better Much
Better
Don‘t
know
N/A-
NR
Youth 1 2 3 4 5 8 9
How much have your feelings about school changed since the
programme began, if at all?
How much has your attendance changed, if at all?
How much has girls‘ participation changed, if at all?
How much has boys‘ participation changed, if at all?
How much has your sense of safety and security changed, if at all?
How much has your ability to protect yourself if there is another
emergency changed, if at all?
How much has the amount students help each other changed, if at all?
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 79
How much has students‘ involvement in problem solving at school
changed, if at all?
How much has the use of fighting to resolve interpersonal differences
changed, if at all?
COMMENTS:
(Write down what the youth say using their exact words.)
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 80
FGD DATA COLLECTION FORM (EDUCATORS)
Date: Group: Educators
Community: Gender: Men / Women / Mixed
Moderator: Number of Participants:
Note taker:
Note establish a common time reference for both programme and control groups which could be “since the emergency” or another
commonly shared event.
Much
worse
Worse No
change
Better Much
better
Don‘t
know
N/A-
NR
Educators 1 2 3 4 5 8 9
How much has your ability to teach changed, if at all?
How has the provision of teaching and learning materials changed, if at all?
How has the quality of the training changed, if at all?
How has the interaction between children and youth of different groups changed, if at all?
How have reporting and monitoring procedures changed, if at all?
How has the quality of education at the school changed, if at all?
How has the retention rate for girls changed, if at all?
How has the usefulness of the revised Teaching Kits changed, if at all?
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 81
How has the usefulness of the revised School in a Box changed, if at all?
How has the usefulness of the revised Early Childhood Kits changed, if at all?
How has the school‘s ability to respond to future emergencies changed, if at all?
How much has the amount students help each other changed, if at all?
How much has students‘ involvement in problem solving at school changed, if at all?
How much has the use of fighting to resolve interpersonal differences changed, if at all?
How much have your feelings about school changed since the programme began, if at all?
COMMENTS:
(Write down what the participants say using their exact words
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 82
FGD DATA COLLECTION FORM
(PARENTS/SCHOOL MANAGEMENT COMMITTEES/ PARENT TEACHERS ASSOCIATIONS)
Date: Group:
School Management Committees/Parent Teacher
Associations
Community: Gender: Men / Women / Mixed
Moderator: Number of Participants in Group:
Note taker:
Note establish a common time reference for both programme and control groups which could be “since the emergency” or another
commonly shared event.
Much
worse
Worse No
change
Better Much
Better
Don‘t
know
N/A-
NR
Parents/ School Management Committees/Parent Teachers Associations 1 2 3 4 5 8 9
How has community involvement in school emergency planning changed, if at all?
How has the community‘s ability to address safety and abuse in the school changed, if at
all?
How your confidence in the quality of education at the school changed, if at all?
How has community ownership over school construction changed, if at all?
How much has the amount students help each other changed, if at all?
How much has students‘ involvement in problem solving at school changed, if at all?
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 83
How much has the use of fighting to resolve interpersonal differences changed, if at all?
How much have your feelings about school changed since the programme began, if at
all?
COMMENTS:
(Write down what the participants say using their exact words.)
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 84
Adequacy Survey Checklist
These questions should be completed in each country. The research team may use various sources to
answer them, should confirm answers with programme staff whenever possible, and should cite sources
of information when possible.
Programme Design
1. Before the programme began, was there a situational assessment?
2. Was there a baseline assessment?
3. Was gender considered when planning and implementing programmes? How so?
4. Are gender-related indicators included in the monitoring plan?
5. Were programme indicators and evaluations structured to look only at outputs or also impact?
6. Does the programme have a plan for discontinuation, phase-out or handover?
Programme Implementation
1. Did replenishment kits arrive within programme established time frames? 2. Did educators receive a minimum of quarterly training? 3. Can youth enrolled in ALP programmes pass a basic literacy and numeracy test? 4. Can children/schools/educational institutions demonstrate knowledge of an emergency response
plan for their school or educational institutions? 5. Does the country have radio programming and/or other remote educational programmes? 6. Does the country demonstrate a direct contribution to the EFA/MDG indicators? 7. Has technical support has been provided?
a. What?
b. By whom? - HQ:
- RO:
- Other:
Programme Learning
1. Has the government adopted the CFS model as a government programme?
2. Were good practice reports/trainings issued to partners at the country level?
3. For each programme implemented in this country, have evaluations been done? List dates of
evaluations for each programme.
4. Were the results of the evaluation shared? How? With whom?
5. Have the results and recommendations of the evaluations been integrated into programming?
How so?
6. How is programme progress and learning from the field level shared with regional and country
level? Is this sharing useful and productive?
7. Do donor or government initiatives reference EEPCT?
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 85
Financial
1. Is the EEPCT programme delineated in financial and/or programme documents?
2. Has there been an increase in government financial support for EEPCT programmes?
3. Are donor funds transferred to the field office as per project-established time frames?
4. Are CAF funds transferred to field offices per project-established time frames?
5. How much of the 2009 allocation of funds was spent (allocation v. expenditures)?
6. Has EEPCT support reached an appropriate number of beneficiaries, given programme costs
(needs vs. coverage)?
Best Practices/Standards
1. Do partner agreements reference INEE minimum standards?
2. Do peace education programmes use UNHCR/INEE/UNESCO programme materials?
3. Has the country achieved compliance with the Minimum Operating Security Standards?
4. Do government preparedness plans incorporate UNICEF methods and approaches?
5. Was a SWOT analysis done in country?
a. If so, did the programme address at least one SWOT recommendation?
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 86
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 87
PROGRESS EVALUATION OF THE UNICEF EEPCT: Liberia Case Study 88