literacy landscape assessment guidebook · 2020. 11. 19. · literacy landscape assessment...
Post on 25-Jan-2021
5 Views
Preview:
TRANSCRIPT
-
Literacy Landscape Assessment Guidebook
NOVEMBER 2019
This paper was made possible by the support of the American people through the United States Agency for International Development (USAID). The paper was prepared for
USAID’s Building Evidence and Supporting Innovation to Improve Primary Grade Reading Assistance for the Office of Education (E3/ED), University Research Co., LLC, Contract
No. AID-OAA-M-14-00001, MOBIS#: GS-10F-0182T.
-
On the cover: Photo caption: Left – Students in Cambodia; Right – Students in the Philippines
Photo: Left Credit – Global Partnership for Education;
Right Credit: The Enabling Writers Project /University of San Jose-Recoletos
-
Literacy Landscape Assessment Guidebook
November 2019
Prepared by: Hank Healey, Emily Morris, and Emily Kochetkova
With support of: Jennifer Bowser Gerst, Reading within Reach
-
RIGHTS AND PERMISSIONS
Except as otherwise noted, Literacy Landscape Assessment Guidebook is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0), https://creativecommons.org/licenses/by-nc-nd/4.0/.
Under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, you are free to copy and redistribute the material in any medium or format under the following conditions:
Attribution: You must give appropriate credit, provide a link to the licensed material and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. A suggested citation appears at the bottom of this page, and please also note: This document was developed by Reading within Reach, through the support of the U.S. Agency for International Development (USAID) and licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Non-Commercial: You may not use the material for commercial purposes.
NoDerivatives: If you remix, transform, or build upon the material, you may not distribute the modified material.
Third-Party Content: USAID does not necessarily own each component of the content contained within the work. USAID therefore does not warrant that the use of any third party-owned individual component or part contained in the work will not infringe on the rights of those third parties. The risk of claims resulting from such infringement rests solely with you. If you wish to reuse a component of the work, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright owner. Examples of components can include, but are not limited to, tables, figures or images.
Disclaimer: The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development.
Suggested citation: Healey, Hank, Emily Morris, and Emily Kochetkova. Literacy Landscape Assessment Toolkit. Washington, D.C.: USAID (2019). Prepared by University Research
Co., LLC (URC) under the Reading within Reach (REACH) initiative for USAID’s Building Evidence and Supporting Innovation to Improve Primary Grade Assistance for the Office of
Education (E3/ED).
-
Acknowledgments
This resource was developed by Reading within Reach (REACH), which is supported by the U.S. Agency for International Development (USAID). REACH works to
facilitate the development, dissemination and application of evidence-based resources, research and practices to increase the impact, scale and sustainability of
early grade reading (EGR) and literacy programs. As its Secretariat, REACH also provides support to develop and sustain the Global Reading Network (GRN), an
interactive hub and resource repository to connect and mobilize stakeholders interested in early grade reading.
The authors wish to thank each of the individuals who helped conceptualize, advise and review the production of this toolkit. Thank you to Jennifer Gerst of the
Reading within Reach (REACH) initiative at University Research Co., LLC (URC) for her tireless leadership. We are grateful to Melissa Chiappetta, Rebecca Rhodes,
Eirini Gouleta, Elena Walls, and Nate Haight of USAID for their constructive guidance and leadership.
Thank you to URC staff and consultants, Stephanie Kenner, Alison Pflepsen, Merle Schwartz, Amy Pallangyo, and Andy Smart for their conceptual and visual
guidance. The authors also thank the GRN, Basic Education Coalition (BEC), Education in Conflict and Crises Network (ECCN) and Rapid Education and Risk
Analysis (RERA) team members for their formative ideas, careful review, and real examples. These include: Cambridge Education (Sabine Kube-Barth), Catholic
Relief Services (Aminata Jalloh), Chemonics (Jordene Hale and Jennifer Swift-Morgan), Creative Associates (Susan Hirsch-Ayari), EDC (Rachel Cristina, Simon
James, and Daniel Lavan), FHI 360 (Julia Frazier, Nathalie Louge, Emily Miksic, and Guitelle Nicoleau), International Rescue Committee (Shahida Maheen), MSI
(Gaelle Simon), Room to Read (Christabel Pinto), RERA Toolkit team (Jim Rogan, Gwen Hean, and Ashley Henderson), RTI International (Luis Crouch), School to
School International (Mark Lynd), SIL-LEAD (Paul Frank), Another Option (Dee Bennet and Rebecca Martinez), and Ash Hartwell.
-
List of Acronyms CBO Community-Based Organization DDL Development Data Library DEC Development Experience Clearinghouse DFID Department for International Development E3/ED Bureau for Economic Growth, Education, and Environment, Office of Education EO Education Officer EGR Early Grade Reading EGRA Early Grade Reading Assessment EMIS Education Management Information Systems ESA Education Sector Analyses FGD Focus Group Discussion GER Gross Enrollment Ratio HR Human Resources IRB Institutional Review Board KII Key Informant Interview KM Knowledge Management LLA Literacy Landscape Assessment LO Learning Objective M&E Monitoring and Evaluation MERL Monitoring, Evaluation, Research, and Learning MOF Ministry of Finance NER Net Enrollment Ratio NGO Non-Governmental Organization PISA Programme for International Student Assessment PTA Parent Teacher Association REACH Reading within Reach SACMEQ The Southern and Eastern Africa Consortium for Monitoring Educational Quality SMC School Management Committee SRGBV School-related Gender-based Violence TE Teacher Education (pre-service teacher education) TLM Teaching and Learning Materials TPD Teacher Professional Development (in-service teacher training) TTC Teacher Training Colleges TWG Technical Working Group USAID United States Agency for International Development
-
List of Figures, Boxes, and Tables
Figures Figure 1: USAID Reading MATTERS Framework: Key Elements of a Reading Program 17 Figure 2: Summary of the Four Phases of the LLA 20 Figure 3: Literacy Domains 21 Figure 4: Interconnected Data Source Contexts 22 Figure 5: Core Principles of the LLA 26 Figure 6: Equity Analyses 27 Figure 7: System Map Example 57 Figure 8: Distinctions Between Quantitative and Qualitative Approaches 66 Figure 9: Distinctions Between Survey, Interview, and FGD Methods (Tools) 67 Figure 10: Mechanisms Driving Learning Coherence 99 Figure 11: Date Use for Improved Learning 101 Figure 12: Lines of Accountability 102 Figure 13: Assessing a Program’s Viability in Another Context 117
Boxes Box 1: LLA Objectives 19 Box 2: LLA Design Document Template 37 Box 3: Data Triangulation 55 Box 4: Education System Mapping (Systemigram) Exercise: Sample Agenda 61 Box 5: Conducting a Language Mapping Exercise to Answer Critical Questions 64 Box 6: Qualitative Sampling (Interviews, FGDs, Observations 69 Box 7: Site Planning: Collaborating, Learning, and Adapting to the Context 69 Box 8: LLA Primary Data Collection Tools 1-11 (Example) 70 Box 9: Translation Tips 71 Box 10: The Role of the Facilitator 74 Box 11: Ethical Issues Checklist for Primary Data Collection Activities 75 Box 12: Nodes (Categories) of Enablers, Barriers, and Recommendations 82 Box 13: Stakeholder Meeting to Refine and Revise Recommendations: Sample Agenda 88 Box 14: LLA Final Report Outline (Example) 93
Tables Table 1: LLA Journey Map (Phases and Steps) 24
Table 2: LLA Objective, Example Entry 31
Table 3: LLA Parameters 31
-
Table 4: LLA Team Tasks and Deliverables 33
Table 5: LLA Team Roles and Qualifications 34
Table 6: TWG Members 39
Table 7: Key Documents and Resources 41
Table 8: Education Systems Landscape 42
Table 9: Background Education Statistics 44
Table 10: Inventory of Literacy Initiatives 46
Table 11: Stakeholder Analysis 48
Table 12: Guiding Questions Outline for Education Systems (Context 1) and
Learning Environment (Context 2) 57
Table 13: Guiding Questions Outline for Home and Community (Context 3) 59
Table 14: Primary Data Collection Planning Table for Contexts 2 and 3 (Context 3 Example) 63
Table 15: Pilot Planning Table 72
Table 16: Recommendations Prioritization Table (Example from Context 3) 90
Table 17: Context 1 Standards 103
Table 18: EGR and Literacy Initiatives Costs 115
Table 19: Analytical Framework Approaches for Qualitative Data
(FGDs and Key Informant Interviews) 120
Table 20: Supporting Resources Across Literacy Domains 122
-
9
Contents Acknowledgments 5
List of Acronyms 6
List of Figures, Boxes, and Tables 7
Figures 7
Boxes 7
Tables 7
Introduction 15
Background 15
The Strategic Objective of the LLA and the Reading MATTERS Framework 16
Figure 1: USAID Reading MATTERS Framework: Key Elements of a Reading Program 17
Research Questions 18
The LLA Process: Using the Toolkit 18
Box 1: LLA Objectives ........................................................................................................................................................................................... 19
LLA Mapped Elements: ........................................................................................................................................................................................ 20
Figure 2: Summary of the Four Phases of the LLA 20
Literacy Domains .............................................................................................................................................................................................. 21
Figure 3: Literacy Domains 21
Figure 4: Interconnected Data Source Contexts 22
Table 1: LLA Journey Map: Phases and Steps .................................................................................................................................................. 24
Principles of the LLA 26
-
10
Figure 5: Core Principles of the LLA 26
Figure 6: Equity Analyses 27
Key Terms 28
PHASE ONE: Planning and Procurement 30
Step 1: Define the objective(s) of the LLA 30
Table 3: LLA Parameters ...................................................................................................................................................................................... 31
Step 3: Develop a Scope of Work to procure an LLA 33
Table 4: LLA Team Tasks and Deliverables ........................................................................................................................................................... 33
Step 4: Select and train the LLA Team 34
Step 4a: Select the LLA Team ............................................................................................................................................................................... 34
Step 4b: Facilitate LLA training for LLA Team ...................................................................................................................................................... 35
PHASE TWO: Design and Background Data 36
Step 1: Develop the LLA Design 36
Box 2: LLA Design Document Template ............................................................................................................................................................... 37
Step 2: Form a Technical Working Group (TWG) 38
Table 6: TWG Members ...................................................................................................................................................................................... 39
Step 3: List key resources, record education landscape information and background data, and inventory literacy initiatives 39
Step 3a: Identify key documents and resources .................................................................................................................................................. 39
Table 7: Key Documents and Resources .............................................................................................................................................................. 41
Rapid Education and Risk Analysis (RERA) 41
Education Sector Analysis 41
Step 3b: Record the Education Systems Landscape ............................................................................................................................................ 42
-
11
Table 8: Education Systems Landscape................................................................................................................................................................ 42
Step 3c: Record the background education statistics .......................................................................................................................................... 43
Table 9: Background Education Statistics ............................................................................................................................................................ 44
Step 3d: Conduct an inventory of all the literacy programs (government, NGO, CBO, etc.) .............................................................................. 46
Table 10: Inventory of Literacy Initiatives ............................................................................................................................................................ 46
Example of entry into Template 10: Inventory of Literacy Initiatives in Workbook Phase Two. ......................................................................... 46
Step 4: Conduct an initial stakeholder analysis 47
Table 11: Stakeholder Analysis ............................................................................................................................................................................ 47
Step 5: Convene a stakeholder design workshop 49
PHASE THREE: Implementation of Data Collection 54
Box 3: Data Triangulation ..................................................................................................................................................................................... 55
Step 1: Answer the Guiding Questions for all Contexts (columns 1a-1d) with existing data 56
Table 12: Guiding Questions Outline for Education Systems (Contexts 1) and Learning Environment (Context 2) ........................................... 57
Step 2: Map out the education system through a stakeholder meeting (Context 1 only) 60
Box 4: Education System Mapping (Systemigram) Exercise: Sample Agenda ..................................................................................................... 61
Figure 7: System Map Example. This figure depicts what the results of a mapping exercise might look like. 62
Figure 7: System Map Example 62
Step 3: Plan the primary data collection 63
Table 14: Primary Data Collection Planning Table for Contexts 2 and 3 (Context 3 Example) ............................................................................ 63
Step 3a. Determine which Guiding Questions will be asked ............................................................................................................................... 63
Box 5: Conducting a Language Mapping Exercise to Answer Critical Questions ................................................................................................. 64
Step 3b. Determine the methodological approach(es), tools (methods), and sample sizes ............................................................................... 65
file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724008file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724008file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724008file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724012file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724012file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724012file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724014file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724014file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724014file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724018file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724018file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724018
-
12
Figure 8: Distinctions Between Quantitative and Qualitative Approaches 66
Figure 9: Distinctions Between Survey, Interview, and FGD Methods (Tools) 67
Box 7: Site Planning: Collaborating, Learning, and Adapting ............................................................................................................................... 69
Box 6: Qualitative Sampling (Interviews, FGDs, Observations) ........................................................................................................................... 69
Step 4: Create data collection tools (survey, interview, and focus group discussion) 70
Step 4a. Draft the data collection tools ............................................................................................................................................................... 70
Box 8: LLA Primary Data Collection Tools 1-11 (Example) ................................................................................................................................... 70
Step 4b. Translate and pilot the tools .................................................................................................................................................................. 71
Box 9: Translation Tips ......................................................................................................................................................................................... 71
Table 15: Pilot Planning Table .............................................................................................................................................................................. 72
Step 4c. Train the facilitators and standardize how the data will be recorded ................................................................................................... 73
Box 10: The Role of the Facilitator ....................................................................................................................................................................... 74
Box 11: Ethical Issues Checklist for Primary Data Collection Activities ............................................................................................................... 75
PHASE FOUR: Analyzing, Reporting, and Disseminating 79
Step 1: Analyze and synthesize findings 80
Analyze the primary and secondary data ............................................................................................................................................................ 81
Step 1a. Analyze the extent, enablers, and barriers (Guiding Question Tables columns 4a-6)........................................................................... 81
Box 12: Nodes (Categories) of Enablers, Barriers, and Recommendations ......................................................................................................... 82
Step 1b. Synthesize findings and enablers and barriers across literacy domains (bottom of Guiding Questions Tables) .................................. 84
Step 1c. Visualize Context 1 data in system maps to compare with Phase 2 - Step 5 “ideal maps” ................................................................... 85
Step 2: Formulate preliminary recommendations from Guiding Question Tables, column 7a) ......................................................................... 86
Step 3: Conduct a stakeholder meeting to revise and refine the preliminary recommendations (Guiding Question Tables column 7b) ........ 87
file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724022file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724022file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724022file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724023file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724023file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724023file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724028file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724028file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724028file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724031file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724031file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724031file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724032file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724032file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724032
-
13
Box 13: Stakeholder Meeting to Refine and Revise Recommendations: Sample Agenda .................................................................................. 88
Step 4: Conduct a TWG meeting to prioritize the recommendations ................................................................................................................. 89
Step 4a. Conduct a prioritization analysis ............................................................................................................................................................ 89
Table 16: Recommendations Prioritization Table (Example from Context 3) ..................................................................................................... 90
Step 4b. Determine a final list of recommendations ........................................................................................................................................... 92
Step 5: Write a final LLA report 92
Box 14: LLA Final Report Outline (Example) ........................................................................................................................................................ 93
Step 6: Disseminate findings 95
ANNEXES—LLA RESOURCES, TOOLS, AND EVIDENCE 96
Annex A: The Learning Coherent System 96
Mechanisms that Drive Coherence 97
Protocols .............................................................................................................................................................................................................. 98
Systems Maps ...................................................................................................................................................................................................... 99
Figure 10: Mechanisms Driving Learning Coherence 99
Figure 11: Data Use for Improved Learning 101
Figure 12: Lines of Accountability 102
Table 17: Context 1 Standards ........................................................................................................................................................................... 103
Annex B: Key Terms and Concepts 106
Annex C: Host Country Government Capacity and Commitment (Context 1) 114
Table 18: EGR and Literacy Initiatives Costs ...................................................................................................................................................... 115
Figure 13: Assessing a Program’s Viability in Another Context 117
Annex D: Guidance for Qualitative Data Collection (FGDs and Interviews) 119
file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724042file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724042file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724042
-
14
Table 19: Analytical Framework Approaches for Qualitative Data (FGDs and Key Informant Interviews) ....................................................... 120
Annex E: Supporting Resources by Literacy domain 122
Table 20: Supporting Resources Across Literacy Domains ................................................................................................................................ 122
Annex F: Evidence 128
Annex G: Language Mapping Experience and Resources 129
Afghanistan .................................................................................................................................................................................................... 129
Ghana ............................................................................................................................................................................................................. 130
Mozambique .................................................................................................................................................................................................. 131
References 132
-
15
Introduction
Background
In the USAID Education Policy launched in November 2018, the United States Agency for International Development (USAID) set a priority of
ensuring that “children and youth gain literacy, numeracy, and social-emotional skills that are foundational to future learning and success”1. As
of 2018, USAID has supported early grade reading and literacy programs in 45 countries through a range of activities, such as teaching and
learning materials development, teacher education and professional development, government education management and information
systems strengthening, and community and family engagement in literacy learning2. Evaluation and research on progress made to date shows
that while reading improvements are observable, there are also notable areas where further progress could be made to ensure all children can
read and comprehend3. For instance, in many countries, evidence shows that children are learning to decode text but cannot actually
comprehend what they decode4.
The Literacy Landscape Assessment Toolkit (LLA)
The Literacy Landscape Assessment Toolkit (LLA) recognizes that country context and environment matter in the success of literacy
programs, and their ability to be scaled and sustained.
• The toolkit was developed as an exercise for capturing and analyzing the different landscapes and contextual factors that enable literacy
acquisition for all children, as well as factors that may be hindering desired progress.
• It can be adapted to the various countries or contexts where USAID works, including in conflict or crisis-affected areas, as well as to other
learning outcomes of interest, such as numeracy.
• The LLA outlines practical steps in gathering, analyzing, and using contextual data germane to literacy acquisition in the primary grades (or
equivalent) to better inform the design and adaptation of reading interventions.
1 USAID. USAID Education Policy. (Washington, D.C.: USAID, 2018), 4 2 USAID. USAID Education Policy. (Washington, D.C.: USAID, 2018), 4 3 Thomas Alvares de Azevedo et al. Synthesis of Findings and Lessons Learned from USAID-funded Evaluations (Washington, D.C.: Management Systems International, 2018). 4 Young-Suk Grace Kim et al. The Landscape Report on Early Grade Literacy (Washington, D.C.: USAID, 2016).
-
16
The Strategic Objective of the LLA and the Reading MATTERS Framework
The strategic objective of the LLA is to provide USAID missions and teams, host-country governments, implementing partners, and other key
stakeholders with critical information and data about the literacy and early grade reading landscape. This data and information are intended to
assist in designing, planning, implementing, or adapting effective literacy efforts and programming in USAID-recipient countries, and to ensure
that the literacy needs of all children can be addressed at scale. The LLA Toolkit is based on existing evidence (see Annex) and reflects USAID’s
Reading MATTERS Framework. The LLA incorporates data collected per other assessments, such as Education Sector Analyses (ESA), Rapid
Education and Risk Analysis (RERA), and Early Grade Reading Assessments (EGRA).
The USAID Reading MATTERS conceptual framework, as shown in Figure 1 below, includes the key elements of a reading program.5 This
assessment exercise can: identify literacy domains in the landscape that enable or inhibit literacy acquisition and progress, indicate the extent to
which elements of the system work together in a learning coherent manner, and gauge government capacity and commitment for necessary
reforms. This assessment process relies on key data collection stages: a) secondary data analysis of existing documents and evidence, and b)
primary data collection through interviews and focus group discussions (FGD), as well as surveys as needed. The final analysis of these data is
meant to inform decisions on where USAID programs might best leverage resources, such as Basic Education Funds, and build on strengths of
the host country education system to help governments and key stakeholders ensure that all children gain and continue to gain the literacy skills
that are foundational to future learning and success.
5 For more information on the Reading MATTERS Framework, see https://www.edu-links.org/resources/usaid-reading-matters-conceptual-framework
https://www.edu-links.org/resources/usaid-reading-matters-conceptual-framework
-
17
Figure 1: USAID Reading MATTERS Framework: Key Elements of a Reading Program
Figure 1: USAID Reading Matters Framework: Key Elements of
a Reading Program. This framework takes into consideration
systems theory6 and asserts that many programmatic domains
of reading programs must be functioning individually and in
alignment with one another for the goal of all children reading
to be realized. Research and adaptation are critical to ensuring
this system is relevant to the contexts. The outer circle
indicates that reading programs must build on host-country
government capacity and commitment in order to be scaled
and sustained.
6 Lant Pritchett. “Creating Education Systems Coherent for Learning Outcomes: Making the Transition from Schooling to Learning.” RISE Working Paper.
Oxford, United Kingdom: Research on Improving Systems of Education, December 2015.
-
18
Research Questions
The research questions guiding the LLA are generally as follows:
➢ Which aspects of the education system enable and ensure quality teaching and learning for all children in primary school? Which aspects could be strengthened to ensure learners are attaining minimum grade-level proficiency in reading at the end of primary school?
➢ How are early grade literacy skills being taught in the learning environment? What are the system enablers and barriers for learners attaining minimum grade level proficiency in reading at the end of primary school?
➢ How are early grade literacy skills being developed and furthered in the home and community environment?
The LLA Process: Using the Toolkit
The LLA Toolkit is comprised of two documents:
1) this LLA Guidebook, which describes the LLA process and outlines the phases and steps involved, and
2) a separate LLA Workbook (multiple segments) with templates in which to document findings per
detailed instructions. The two documents are to be used in tandem.
The LLA Toolkit contains
multiple tools and activities
that can be applied in part or as
a whole to suit the needs, time,
and budget of its users.
-
19
The LLA is designed for use in any country, region, or lower-level jurisdiction (i.e., province, district, or county) where early grade literacy data7
exists or will soon be collected, and can be adapted to several contexts, including in crisis or conflict areas.8
The LLA is designed for use towards two main objectives, as explained below:
Box 1: LLA Objectives
Objectives Assessment and Examples Timing
Inform the design of a new program
Revealing what already exists to enable the system to function well with regards to building proficiency in literacy, as well as identifying key elemental areas that can be strengthened.
Example: a USAID mission is designing a new program and is considering what components at the system, classroom, and community levels to focus on in the program.
Prior to procuring a new reading program.
Inform the adaptation or scaling of an existing program(s)
Providing contextual information and insight into why results and outcomes may have occurred or to document major changes (policy, political, etc.) in the literacy environment.
Example 1: A reading program is not producing the intended results, and the mission and/or implementer wants to adapt the program for greater impact, or a mission wants to assess the enablers and barriers in an environment before bringing a program to scale.
Example 2: A significant education policy or systemic change occurs, and a mission wants to understand the implications across the literacy landscape.
During implementation of existing programs and/or after a major shift in the country’s education landscape has occurred.
7 Literacy data can include any national or representative data collected per the designated regions covered by the toolkit. Data can be collected through
national assessment tools, early grade reading assessments (EGRA), or other literacy assessments conducted within the last 5 years. If there is no early grade reading data for the region or population of learners and teachers targeted by the LLA exercise, this data will need to be collected before or concurrent to the LLA exercise. Literacy data is critical as it provides an overarching picture of whether or not countries are able to meet their literacy objectives and to what extent system reform is needed.
8 Per the USAID Rapid Education and Risk Analysis (RERA) Toolkit (USAID, 2019), crisis-affected includes areas affected by natural disasters and health
epidemics and also can include climate vulnerabilities and crime and violence activity. Conflict-affected includes areas affected by active armed conflict or post conflict (see Annex B for more definitions). If a RERA has been conducted, the LLA should build on information already collected as will be indicated throughout the toolkit.
-
20
LLA Mapped Elements:
➢ Phases (4), each with steps and templates
➢ Programmatic Domains for Reading & Literacy (10)
➢ Contexts (3)
Phases: The LLA is divided into four phases summarized in Figure 2 and outlined below in Table 1, The LLA Journey Map with accompanying
steps.
Figure 2: Summary of the Four Phases of the LLA
Figure 2: Summary of the Four Phases of the LLA. The phases of the LLA inform the design of a new program or the adaptation or scaling
of an existing program or programs.
Phase 1
(4-8 weeks)Planning &
ProcurementIdentifies the parameters of the LLA and team.
Phase 2
(6-8 weeks)
Design & Background
Data
Includes the background info needed and stakeholder consultation to promote host-country ownership and
implementing partner involvement.
Phase 3
(10-20 weeks)Implementation of Data
CollectionGathering primary and secondary data.
Phase 4
(10-20 weeks)
Analyzing, Reporting, &
Disseminating
Ensures key objectives and questions proposed in design have been answered.
-
21
Literacy Domains
There are also seven programmatic literacy domains based on the Reading MATTERS Framework.
For the LLA, one additional political economy domain has been added: Host Country Government
Capacity and Commitment.
Figure 3: Literacy Domains
Programmatic Literacy Domains (Not Sequential)
Domain Detail
Standards Evidence-based standards, norms, and policies
Texts High-quality teaching and learning materials
Teachers Effective teachers and classroom instruction
Mentors Effective coaches and mentors
Administrators Quality administrative support and supervision
Regular Assessments Assessments inform instruction
Host Country Government Capacity and
Commitment Government financing for early grade literacy
Extra Practice Extra practice and support outside of school
Figure 3: Literacy Domains. The phases of the LLA are mapped to these literacy domains, which are not sequential.
-
22
Figure 4: Interconnected Data Source Contexts
Figure 4: Interconnected Data Source Contexts. The LLA builds on secondary data from existing reports, documents, and sources, and also
includes primary data collection.
CONTEXT ONE, Education Systems: The elements of the system in place to facilitate literacy learning.
CONTEXT TWO, Learning Environment (formal and non-formal): The teaching and learning infrastructure, practices, beliefs, and attitudes
around literacy learning.
CONTEXT THREE, Home and Community: The infrastructure, practices, beliefs, and attitudes around literacy learning outside of the learning
environment (how literacy acquisition is supported at home and in the community). Context three corresponds to the Extra Practice tile from the
Reading MATTERS Framework. As revealed in the research, the key literacy domains that influence reading and literacy acquisition in the Home
and Community Context are: 1) availability of text and literacy programs; 2) familial demographics and cultural contexts; and 3) caregiver
engagement in reading and literacy acquisition.
Context 1: Education Systems
Context 3: Home and
Community
Context 2: Learning
Environment
-
23
Each LLA Context is critical to supporting children’s literacy acquisition, but the stakeholders, factors considered, and documents analyzed will
differ across these Contexts. Completing the exercises for all three Contexts will provide the most comprehensive understanding of the
landscape. While the Education Systems Context should be completed in its entirety, the LLA is flexible in that, within the Learning Environment
and Home and Community Contexts, teams will decide which of the literacy domains to analyze for their particular context.
-
24
Table 1: LLA Journey Map: Phases and Steps NOTE FOR USER: Create a local team workspace for this toolkit. Each Phase and Step of the LLA below references Excel-based Workbook Resources in the form of fillable templates, sample agendas or forms. Once the LLA Team sets up a workspace, for ease of use the team may wish to ensure the templates referenced below are hyperlinked to the corresponding Workbooks and that at each icon in the LLA Guidebook labelled “Back to Journey Map” hyperlinks to Table 1 below.
Step Workbook Resource
LLA WORKBOOK
Phase 1
Planning and Procurement (4-8 weeks)
Step 1: Define the objective(s) of the LLA Template 2: LLA Objectives
Step 2: Set the parameters of the LLA Template 3: LLA Parameters
Step 3: Develop a Scope of Work to procure an LLA
Template 4: LLA Scope of Work
Step 4: Select and train the LLA Team Template 5: LLA Team Roles and Qualifications
LLA WORKBOOK
Phase 2
Design and Background Data (6-8 weeks)
Step 1: Develop the LLA design Template Box 2: Design Outline
Step 2: Form a Technical Working Group (TWG) Template 6: TWG Members
Step 3: List key resources, record education landscape information and background data, and inventory literacy initiatives
Template 7: Key Documents and Resources Template 8: Education Systems Landscape Template 9: Background Education Statistics Template 10: Inventory of Literacy Initiatives
Step 4: Conduct an initial stakeholder analysis Template 11: Stakeholder Analysis
Step 5: Convene a stakeholder design workshop Annex A
LLA WORKBOOK
Phase 3
Implementation of Data Collection (10-20 weeks)
Step 1: Answer the Guiding Questions for all Contexts (columns 1a-1d) with secondary data
Templates 12: Guiding Questions Outline for Education Systems (Context 1) and Learning Environment (Context 2) Template 13: Guiding Questions Outline for Home and Community (Context 3)
Step 2: Map out the education system through a stakeholder meeting (Context 1 only)
Template Box 4: Education System Mapping Sample Agenda
Step 3: Plan the primary data collection Template 14: Primary Data Collection Planning Table for Contexts 2 and 3 (Context 3 Example)
Step 4: Create data collection tools (survey, interview, and focus group discussion)
Template 15: Pilot Planning Table
Step 5: Collect and record the primary data in the Guiding Questions Templates
Templates 12A, 12B and 13, Data Collection Section
LLA WORKBOOK
Phase 4
Analyzing, Reporting, and Disseminating (10-20 weeks)
Step 1: Analyze and synthesize findings Templates 12A, 12B and 13, Analysis Section
Step 2: Formulate preliminary recommendations
Refer to Templates 12A, 12B and 13
Step 3: Conduct a stakeholder meeting to revise and refine the preliminary recommendations
Template Box 10: Stakeholder Meeting to Refine and Revise Recommendations: Sample Agenda AND Refer to Template 12 and 13
Step 4: Conduct a TWG meeting to prioritize the recommendations
Template 16: Final Recommend
Step 5: Write a final LLA report Template Box 13: Final Report Outline
Step 6: Disseminate findings
https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%201.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%201.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%201.xlsx?web=1https://www.urc-chs.com/sites/default/files/urc-grn-lla-1.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-1.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-1.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-1.xlsxhttps://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%202.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%202.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%202.xlsx?web=1https://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-annex.pdfhttps://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%203.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%203.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%203.xlsx?web=1https://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%204.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%204.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%204.xlsx?web=1https://www.urc-chs.com/sites/default/files/urc-grn-lla-4.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-4.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-4.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-4.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-4.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-4.xlsx
-
25
The LLA steps are designed to be conducted both remotely (from anywhere) and in-country. The total time required will vary by context (i.e., size of population and crises or conflict environment) and defined parameters (i.e., number of literacy domains covered in Phase 3, whole country or regions, etc.). The timeframes for each phase (estimated above in Table 1, the LLA Journey Map) will vary depending on the parameters set in Phase 1. In countries where multiple LLAs must be prepared (i.e., in Tanzania for mainland and Zanzibar, which have separate Ministries of Education), Phase 1 may take longer than the estimate provided. Likewise, in situations affected by conflicts and/or crises the timeline and process will have to be adapted. The implementation of the LLA is facilitated by a team of experts and advised by a Technical Working Group (TWG). These experts and
members of the TWG are identified by the USAID Mission per criteria described in Phase 1.
-
26
Principles of the LLA
The principles undergirding the LLA reflect the principles articulated within USAID’s 2018 Education Policy as well as USAID’s commitment to
collaborating, learning, and adapting (CLA).
Figure 5: Core Principles of the LLA
Figure 5: Core Principles of the LLA. The LLA is based on three core principles.
The goal:
All children reading
•The LLA exists to uncover and illuminate factors that can hinder or support children’s literacy acquisition, and to provide information to guide governments, donors, and implementers in removing hindrances and strengthening supports so that all children, regardless of background and demographics, are reading and continue doing so over time.
Host country ownership
•It is critical for key government stakeholders to be integrally involved in each phase of the LLA so that recommendations are developed collaboratively.
•This means that all units and levels of government that are assessed by the LLA should be involved in work-planning, data collection, and dissemination of findings.
Focus on inclusion and
equity
•Access to resources and the learning needs of girls and boys--children with disabilities; indigenous children; children who speak minority languages; children in remote and rural areas; children living in crisis-affected or conflict-affected areas; or children living in vulnerable circumstances are not always adequately considered in policy and practice.
•The LLA explicitly considers these groups across spaces and reading domains.
-
27
Figure 6: Equity Analyses
Figure 6: Equity Analyses. There are a number of factors that must be considered when conducting equity analyses. If equity analyses have been conducted per the RERA, all tables and tools should be adapted to reflect the major equity issues revealed. 9
9 USAID’s Education Policy states that marginalized children and vulnerable groups include “girls, children affected by or emerging from armed conflict or humanitarian crises, children with disabilities, children in remote or rural areas (including those who lack access to safe water and sanitation), religious or ethnic minorities, indigenous individuals, orphans and children affected by HIV/AIDS, child laborers, married adolescents, and victims of trafficking” (USAID, 2018, p. 46).
Language issues
Disability
Gender
Minority and/or indigenous groups
Crisis/conflict contexts
ICT (Access)
Equity Analyses
-
28
Key Terms
Here we provide definitions of a few key terms; Annex B contains a full list of LLA terms and their definitions.
LITERACY: “Literacy is defined as the ability to read and write to gain and produce meaning in context. Early grade reading or literacy refers to instruction that takes place in primary school grades 1-3”10. LEARNING COHERENT SYSTEM: An education system that is learning coherent is one in which there is alignment between all aspects, or literacy domains. That is, the teaching and learning materials, the teacher education and teacher professional development courses and programs, the coaching practice, and student assessment are all aligned with and support the achievement of the learning objectives outlined in the curriculum. (More detail can be found in Annex A.) READING MATTERS STANDARDS: A government’s evidence-based policies and norms related to literacy, as reflected in their curricula, syllabi, and other guiding frameworks. LLA STANDARDS: The evidence-based ideal (based on learning coherence and all children reading) used in all literacy domains to measure the extent to which something is happening in the education system or learning environment. LEARNING ENVIRONMENT: The learning environment constitutes the formal and informal spaces where teaching and learning take place. In the LLA, the learning environment includes the teaching and learning infrastructure as well as educators’ practices, beliefs, and attitudes around literacy learning.
COACHING/MENTORING: “Teacher educators commonly define coaching as ‘onsite, job-embedded, sustained professional development for teachers’ (Bean, 2014)11. Through coaching, teachers receive continuous support to help them acquire and master new knowledge and skills to improve their instruction and, ultimately, student achievement”12.
10 Young-Suk Grace Kim et al. The Landscape Report on Early Grade Literacy (Washington, D.C.: USAID, 2016). 11 Rita M. Bean. The Power of Coaching: Improving Early Grade Reading Instruction in Developing Countries: Final Report. Report produced for USAID
(Silver Spring, MD: Aguirre Division of JBS International, Inc., 2014).
12 Alison Pflepsen, Coaching in Early Grade Reading Programs, resource developed by REACH and the GRN (Chevy Chase, MD: URC, 2019).
-
29
ASSESSMENT: “Assessment is an essential component of effective instruction as it provides information about students’ achievement levels and reasons for the performance level. There are multiple types and purposes of student assessments such as screening (to identify some students who might be at risk for literacy failure and require further diagnostic assessment), diagnostic (to identify specific areas of strengths and weaknesses), formative and progress monitoring (to reveal whether students are learning adequately), and summative (overall achievement level)”13.
13 Kim et al. Landscape Report, 2016.
-
30
PHASE ONE
Back to Journey Map
PHASE ONE: Planning and Procurement
Step 1: Define the objective(s) of the LLA
Purpose: To identity why the LLA is being conducted and to develop consensus as to how it will be used by the USAID Mission.
Who leads: USAID Mission, with input from selected host-country education representatives.
☐ Look at Box 1 in the introduction and decide which of the two objectives the LLA will be addressing; include country-specific information.
☐ Complete Template 2 in the Workbook Phase One. An example follows.
Overview
The Planning and Procurement process consists of four steps that lay the groundwork for the LLA. This section of the
Guidebook provides details for each step, the first three of which are to be carried out by USAID Mission staff in advance
of procuring a team to lead the LLA.
☐ Step 1: Define the objective(s) of the LLA
☐ Step 2: Set the parameters of the LLA
☐ Step 3: Develop a Scope of Work to procure an LLA
☐ Step 4: Select and train the LLA Team
-
31
PHASE ONE
Back to Journey Map
Table 2: LLA Objective, Example Entry
Objective Assessment and Examples Timing
Inform the design of a new program
e.g., to see how the Ministry’s current education system is operating in regards to literacy and to understand how the new policy changes of local language (L1) instruction are influencing teaching and learning. Also, to understand home and community reading and literacy practices for informing the new literacy program design.
After EGRA baseline is complete and before project design is complete
Example of entry for Template 2: LLA Objectives in Workbook Phase One.
Step 2: Set the parameters of the LLA Purpose: To set the parameters of LLA and what geographical areas, grade levels, and other considerations will be covered. Who leads: USAID Mission, with input from selected host-country education representatives.
☐ Complete the LLA parameters table in Template 3 of the Workbook Phase One. Top level questions are listed here below and in more detail in the Workbook.
Table 3: LLA Parameters
Questions
1. What contextual assessments have been conducted to date and how will this LLA exercise expand these analyses? (e.g. Education Sector Analyses, Rapid Education and Risk Analysis (RERA), and Early Grade Reading Assessments (EGRA)).
2. How often will the LLA be implemented?
3. Where will the LLA be implemented?
4. Which grades/classes within pre-primary/primary (or equivalent) will the LLA cover?
5. Which modalities will the LLA cover (formal or nonformal education)?
-
32 Back to
Journey Map
6. Which kinds of learning institutions will the LLA cover? Who operates these schools/centers?
7. What populations and key issues will be considered in this LLA process?
8. Is this country/region currently in a crises/conflict affected area?
9. Which Contexts will be completed?
-
33
PHASE ONE
Back to Journey Map
Step 3: Develop a Scope of Work to procure an LLA
Purpose: To procure a team that can undertake the technical and operational tasks required for conducting the LLA.
Who leads: USAID Mission.
When: After the objectives and parameters of the LLA have been determined.
☐ Modify the Scope of Work Template 4 found in the Workbook Phase One, adding details about the objectives, parameters, and the country or regional context. The overall length of an LLA, as mentioned, can vary and is likely to require at least 6 months and as many as 12 months.
☐ Modify the time frame according to the parameters set, keeping in mind things that will be outside of the LLA team’s control, such as contract approvals, availability of stakeholders for meetings and interviews, timing of data collection in relation to school calendar, etc. USAID Mission personnel will determine the approach for procurement.
Table 4: LLA Team Tasks and Deliverables
Phase Deliverables Time Frame
Phase 1: Form LLA Team ▪ LLA Team contracted 2-4 weeks
Phase 2: Design and Background Data
▪ Steps 1 and 2 completed: LLA Design and Implementation Plan; final report outline (both finalized with TWG)
▪ Steps 3 and 4 completed: landscape, background information, and key resources; stakeholder analysis
6-8 weeks
Phase 3: Implementation of Data Collection
▪ Steps 1 through 3 completed: Secondary data analysis and plans for primary data collection across Contexts
▪ Steps 4 and 5 completed: Primary data collected and entered into Guiding Questions tables across Contexts
10-20 weeks
Phase 4: Analyzing, Reporting, and Disseminating
▪ Steps 1 and 2 completed: Data analysis and preliminary recommendations
▪ Steps 3 and 4 completed: Refine and prioritize recommendations
▪ Steps 5 and 6 completed: Final report and dissemination
10-20 weeks
Summary of information going into Template 4: LLA Scope of Work in Workbook Phase One.
-
34
PHASE ONE
Back to Journey Map
Step 4: Select and train the LLA Team
Step 4a: Select the LLA Team
Purpose: To select a team with the requisite experience, skills, and expertise to carry out the work of the LLA. Who leads: USAID Mission.
☐ Review proposals or applications for executing the work per the defined the LLA Team roles in the Scope of Work (Table 5).
☐ Select the most suitable team. Contracting options could include a standalone contract or an addition (subcontract) to an existing implementation contract. USAID may also need to develop a Memorandum of Understanding (MOU) with the government that outlines the timeframe and local counterpart commitment to the LLA. Table 5: LLA Team Roles and Qualifications
Position Role Qualifications
USAID Mission Lead With the Ministry/Department of Education co-lead, oversee the LLA process and facilitate communication with government and other donor stakeholders.
Experience working in the education sector and managing assessments or evaluations.
Host-country Government Education Lead
With the USAID co-lead, oversee the LLA process and facilitate communication with government and other donor stakeholders.
High-level experience and knowledge of the education system.
LLA Team Lead Coordinate all steps and activities in the LLA process; manage timeline; provide quality control; communicate with USAID; oversee synthesis of reporting.
Demonstrated experience managing mixed methods research projects.
Country-Level Education Advisors
Work closely alongside international experts on the LLA team, providing contextual insight and technical knowledge related to various lines of inquiry, contributing to LLA outputs, and facilitating connections as needed.
A deep internal understanding of the country’s education context and the technical expertise needed to collaborate with international counterparts on the LLA team (i.e., literacy, equity, M&E, etc.).
Education Systems Expert Lead all activities under Context 1. Demonstrated experience leading system analysis efforts, with emphasis on financial and cost-analysis expertise.
-
35 Back to
Journey Map
PHASE ONE
Literacy/Teaching and Learning Expert
Lead all activities under Contexts 2 and 3. Demonstrated expertise in literacy and research and assessment in teaching and learning in developing country contexts.
Research and CLA Expert Oversee and manage the primary data collection and analysis activities.
Demonstrated expertise leading all aspects of quantitative and qualitative research (research design and methodologies, tools development, data collection, analysis, and writing) in developing country contexts.
Equity (Social and Gender Inclusion) Expert(s), ideally with Experience in Crises and Conflict Environments
Support all Contexts by contributing to secondary data analysis and primary data collection and analysis related to equity analyses. Depending on the size of the country and team, you may need multiple experts that specialize in disability, gender, ICT, crises and conflict, etc.
Demonstrated experience leading equity research and analysis related to education. If RERA has been conducted in this environment, ideally a person familiar with the RERA data.
Summary of LLA Team Roles and Qualifications to be entered with identified team members into Template 5 in Workbook Phase One.
Step 4b: Facilitate LLA training for LLA Team
Purpose: To provide training for the selected LLA Team, including the USAID Mission Lead and host-country Government Education Lead. Who leads: Individuals with LLA expertise.
☐ Convene LLA Team and USAID and host-country Government Leads for a 4-day training.
-
36
PHASE TWO
Back to Journey Map
PHASE TWO: Design and Background Data
Step 1: Develop the LLA Design
Purpose: To create a document that encompasses all of the different components of the LLA plan, including the background, objectives, data collection tools, and plans for the report. This document can be submitted to USAID Mission staff, host-country government, and other stakeholders for approval. Who leads: LLA team, who then shares with Technical Working Group (see Step 2 below) and USAID Mission for feedback.
Overview Phase 2 of the LLA involves four main steps that guide the design of the LLA. These steps are listed sequentially but can occur in the order that makes the most sense for the LLA team. Phase 2 takes approximately 6-8 weeks.
☐ Step 1: Develop the LLA Design
☐ Step 2: Form a Technical Working Group (TWG)
☐ Step 3: List key resources, record education landscape information and background data, and inventory literacy initiatives
☐ Step 4: Conduct an initial stakeholder analysis
-
37
PHASE TWO
Back to Journey Map
☐ Complete the Box 2: LLA Design Document, found in the Workbook. It integrates the templates and tools from the Workbook as indicated implementing in Box 2 below. This document will evolve throughout Phase One and Phase Two as these steps are completed.
Box 2: LLA Design Document Template
Table of Contents, List of Tables, List of Figures, Acronyms, etc.
LLA purpose and objectives • Write up the LLA objectives (Template 2) and the intended audience. If the LLA is part of a larger activity (i.e., part of an existing effort), describe how this fits into these larger efforts.
• State how this LLA aligns with USAID’s Reading MATTERS framework.
Background • Describe the context in which the LLA will take place, including the regions, grade levels, etc. per the parameters (Template 3, Workbook Phase One).
• Include critical information on the education systems landscape (Template 8) and education statistics (Template 9) for the above context. Integrate data from Education Sector Analyses, Rapid Education and Risk Analysis (RERA), Early Grade Reading Assessments (EGRA) and other contextual assessments conducted.
• Include any information on key literacy initiatives in the landscape that need to be considered (Template 11). • Describe the LLA team and their different roles (in Annex D the actual team’s staffing and LoE will be designated).
Methodology for data collection • Describe which literacy domains in the three Contexts will be covered by the LLA exercise (provide an outline of the key literacy domains as in Templates 13 and 14 in Phase Three).
• Describe how the secondary data collection will be conducted and name key documents and resources (Template 10).
• If applicable, describe how educational system mapping will be conducted (systemigram described in Step 2 in Phase 2).
• Outline stakeholder participations, including what steps they will be included in and who these stakeholders are (Template 12, Phase Three).
• Explain the proposed process for collecting primary data and which tools will be used (Template 15, Phase 3). Note that this plan will evolve and may be different in the final report.
• Summarize the steps to be taken to recruit and train data collection staff for primary data collection. • Identify any known limitations to the LLA design, including staffing limitations, logistical issues, political or social
constraints, etc.
Data analysis approach • Describe how the data will be analyzed (including disaggregation, triangulation, and coding and sorting).
-
38
PHASE TWO
Back to Journey Map
Work plan • Create a work plan (e.g., Gantt Chart) with details on when, where, and how deliverables and activities will be carried out. Ensure all human resources on the LLA team are reflected in the work plan.
Recommendations and final report • Include information on how the recommendations will be devised, including the stakeholder analysis and TWG prioritization.
• Describe the parameters of the final report and how it will be disseminated.
Annex A LLA Scope of Work (Template 4, Phase One)
Annex B LLA Team Roles and Qualifications (Template 5, Phase One)
Annex C Data Collection Tools (KII, Survey, Interviews, or FGDs) and IRB documents (as applicable)
Box 2: LLA document to be completed located in Workbook.
Step 2: Form a Technical Working Group (TWG)
Purpose: To identify a group of international and host-country experts (including government representatives) that ultimately advise the LLA exercise and: a) provide feedback on the LLA Design Document (Step 1 above); b) determine the list of final recommendations (Phase 3, Step 4); c) provide feedback on the Final Report Outline and Final Report (Phase 3, Step 5) ; and d) ensure on-going participation, collaboration, and ownership of the LLA process by the host-country. Who leads: LLA Team Lead with host-country government partners and in consultation with USAID Mission.
☐ Identify and coordinate with existing education sector TWGs, Local Education Groups, Steering Committees, etc. to either form a new TWG to provide oversight for the LLA, or to orient an existing TWG with the LLA.
TWG members should include education sector stakeholders from government, parastatal, non-government, and civil society institutions/organizations that support education and literacy efforts in the country/region. Example participants might include ministry officials, representatives from non-governmental organizations (NGO) and community-based organizations (CBOs), teacher unions, parent alliances (parent teacher associations, school management committees), teachers and head teachers, a book publisher representative, and other private sector and donor representatives as appropriate. If other assessments have been conducted, such as Education Sector Analyses, Rapid Education and Risk Analysis (RERA), and Early Grade Reading Assessments (EGRA), representatives from these assessment teams should be included in the
-
39
PHASE TWO
Back to Journey Map
TWG. While robust participation is ideal, the number of participants for this can vary greatly depending on stakeholder interest and time during the course of the LLA, and the culture of other TWGs in the country.
☐ List TWG members in Template 6 of the Workbook, following the example provided below.
Table 6: TWG Members
Name Affiliation Description of Role
e.g., Mr. Ezekiel Stanley MoE, Standards and Curriculum
Director of Department; oversees changes and revisions to standards and curriculum.
e.g., Ms. Mariam Ali MoE, Teacher Training Institute
Director of TTI; oversees hiring and professional development of TTI faculty; manages teams that revise changes to TTI curricula
Example of entries for Template 6: TWG Members in Workbook Phase Two.
Step 3: List key resources, record education landscape information and background data, and inventory
literacy initiatives
In this step, users collect data that help inform the Design Document and provide the overall context of the country or region where the LLA will be conducted. This data will also be used in the background section of the Final Report.
Step 3a: Identify key documents and resources
Purpose: Identify documents and data sources critical to answer the guiding questions in each Context (for Part Two). This table will serve as the basis for the Reference list in the final report and should be updated throughout the LLA process as the data analysis is conducted. This is included in the Planning and Design stage as it is important to include in the Design Document so the TWG and Government partners can add to this list. Who leads: LLA Team Lead and host-country education representatives.
-
40 Back to
Journey Map
When: After the parameters have been set and the team formed in Steps 2 and 3 and before any in-depth document analysis has commenced. Steps 5a-c can be completed simultaneously.
☐ Identify and complete the Key Documents and Resources table (example next page) in the Workbook Phase 2, Template 7.
☐ Keep updating the template throughout the process, as each can be consolidated with the others and serve as the references section in the final report.
-
41
PHASE TWO
Back to Journey Map
Table 7: Key Documents and Resources
a. Title of Resource b. Source/Links c. Year d. Author e. Information
Country Context
Rapid Education and Risk Analysis (RERA)
Education Sector Analysis
Education Systems, Context 1
e.g., Education Law Kenya Basic Education Act 2013 Parliament Passed into law in 2013; revised in 2017
e.g., National Education Plan
Learning Environment, Context 2
e.g., Academic research and studies Implementing large-scale instructional technology in Kenya: Changing instructional practice and developing accountability in a National Education System
2017 Piper, B., Oyanga, A. Mejia, J. and Pouezevara, S.
Impact of the National Tablets Program to increase the number of classroom observations done by coaches and to improve student learning outcomes.
e.g., EGRA reports, national exam results*
Kenya Tusome Midline EGRA Report
2017 MSI, Inc. Results of midline evaluation of USAID-and DFID- funded EGR program
Home and Community, Context 3
e.g., Academic research and studies
e.g., School Management Committee (SMC) and Parent-Teacher Association (PTA) policies, reports, and strategies
* As described in the introduction section, literacy assessment data are critical to this LLA, and if there are no data for the country or region where the LLA is being conducted, this
should be prioritized in the LLA Design.
Example of entries for Template 7: Key Documents and Resources in Workbook Phase Two.
http://kenyalaw.org/lex/rest/db/kenyalex/Kenya/Legislation/English/Acts%20and%20Regulations/B/Basic%20Education%20Act%20No.%2014%20of%202013/docs/BasicEducationActNo14of2013.pdfhttp://shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-and%22HYPERLINK%20%22http:/shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-andhttp://shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-and%22HYPERLINK%20%22http:/shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-andhttp://shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-and%22HYPERLINK%20%22http:/shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-andhttp://shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-and%22HYPERLINK%20%22http:/shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-andhttp://shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-and%22HYPERLINK%20%22http:/shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-andhttp://shared.rti.org/content/tusome-external-evaluation-midline-reporthttp://shared.rti.org/content/tusome-external-evaluation-midline-report
-
42
PHASE TWO
Back to Journey Map
Step 3b: Record the Education Systems Landscape
Purpose: The Education System Landscape table documents the basic structure of the education system and how it functions. The table provides important contextual information about the system’s basic structure, the level of decentralization, and how it is financed. This contextual information will aid in understanding the system’s assumptions about literacy learning in the primary grades as well as how the system is organized. This information will help inform the background sections of the Design Document and Final Report. While Context 1 Guiding Questions provide a deep dive into aspects of the system, as they align to Reading MATTERS literacy domains, this general information is important for guiding the refinement of the different questions in the Guiding Questions tables and providing overarching data before data collection commences. Most of this information can likely be gathered based on country-level experts’ knowledge, the overarching Education Policy, and the education authorities’ websites. Integrate relevant ESA and RERA data if previously collected. Who leads: LLA Team Lead and Country-Level Education Advisors.
☐ Complete the full Education Systems Landscape Table in Workbook Phase Two (Template 8). Modify as needed. An example is provided below.
Table 8: Education Systems Landscape
Education System Questions Details Source (Link)
1. Basic structure of the system: Number of years and official entry age for each level.
e.g., 12 years of basic education, 2-6-4 (2 pre-primary, 6 primary, 4 lower secondary). Upper secondary and tertiary are beyond the basic education system. Entry age for pre-primary is 4; grade 1 is 6.
Education Policy
2. Early grade literacy is established during which grades per the national standards and curriculum (i.e., preschool to grade 2 or kindergarten-grade 1, etc.)? What literacy instructional approach is used? What languages are used for instruction, and in what grades?
•Early grade reading starts in pre-primary 1 and extends through Standard 4 (therefore six years). Pre-primary taught in Kiswahili with English as a subject (and Arabic as a language taught in religious schools). •Pre-literacy starts with syllables (agglutinated language) and phonics (name of letters, syllable sounds, and shape).
Education Policy. National curriculum and review by SIL report
-
43 Back to
Journey Map
PHASE TWO
3. Teacher education (pre-service). What division is responsible for pre-service teacher education?
•National teacher training institute provides pre-service for all government schools. Non-formal learning center provided by Department of Adult and Non-formal Education.
Education Policy and MoE website
4. Professional development (in-service). What division is responsible for administering in-service education?
•Teacher Training Centers in all of the decentralized district offices. Non-formal learning center in-service training largely provided by NGOs in collaborate with Department of Adult and Non-formal Education.
Education Policy and MoE website
5. Assessment. In what grades is literacy assessed annually? What literacy constructs are tested? What assessment tools are used?
School administered: Grade 2 Centrally administered: Grade 4 Regional assessment (cross-5 countries): Grade 5
Education Policy. National Exam Board’s website and strategy
6. Decentralization. Describe the degree to which the education system is decentralized. At what levels of the system are particular decisions made?
Central Ministry oversees the policy, pre-service teacher training, curriculum and materials developed, and examinations and assessments (as well as inspections and monitoring). The district education offices oversee in-service education training, mentoring and coaching.
Education Strategy and Education Policy
Example of entries for Template 8: Education Systems Landscape in Workbook Phase Two.
Step 3c: Record the background education statistics
Purpose: The Background Education Statistics table below consolidates the data on critical education and related indicators and contextualizes the education system. Again, this information will help inform the background sections of the Design Document and Final Report. Who leads: LLA Team Lead and Country-Level Education Advisors.
☐ Complete the full Background Education Statistics Table in the Workbook Phase Two (Template 9). Users should complete the table to the extent that data exists. If data has been collected per the RERA or an ESA, integrate the data into the table. Many countries will not have disaggregated data on language, disabilities, and gender, but gaps will provide a picture of the extent to which the data is gender and disability-responsive. An example follows.
-
44
PHASE TWO
Back to Journey Map
Accompanying tools/resources: Data can be accessed from national education reports and datasets as well as the Demographic Health Survey
and international agency datasets at http://databank.worldbank.org, http://data.uis.unesco.org/, https://knoema.com, and
http://www.who.int/gho/countries/en/.
Table 9: Background Education Statistics
Education Statistic (National) Data (Year) Source (Link) Education Statistic Data (Year) Source (Link)
Pre-primary education
Pre-primary gross enrollment (or intake) ratio (GER), by sex and disability
Total: 112.5% Male: 113.5% Female:111.5% (2018)
http://uis.unesco.org/country/GH
Pre-primary school net enrollment ratio (NER) by sex
Total: 72.2% Male: 73.8% Female: 70.6% (2018)
http://uis.unesco.org/country/GH
Pupil teacher ratio for pre-primary (policy and actual national average)
32.8 (2016 avg.) https://www.indexmundi.com/facts/ghana/pupil-teacher-ratio
Gender parity indices for pre-primary GER or NER
1 (2016) https://tradingeconomics.com/ghana/school-life-expectancy-pre-primary-gender-parity-index-gpi-wb-data.html
Percent of pre-primary students enrolled in government schools, by sex and disability
Total: 72.8%
(2015) https://ourworldindata.org/pre-primary-education
Actual (average) entry ages in pre-primary
3 (2016) https://ourworldindata.org/pre-primary-education
Primary education
Percent of primary school students who have attended pre-primary, by sex and disability
Percent of primary school students who have attended government-run pre-primary, by sex and disability
Primary school gross enrollment ratio (GER), by sex and disability
Primary school net enrollment ratio (NER), by sex and disability
Gender parity indices for primary GER or NER
Primary dropout rates, by sex and disability (if data)
Survival to last grade of primary (%), by sex and disability
Primary to secondary school transition rate, by sex and disability
http://databank.worldbank.org/http://data.uis.unesco.org/https://knoema.com/http://www.who.int/gho/countries/en/http://uis.unesco.org/country/GHhttp://uis.unesco.org/country/GHhttp://uis.unesco.org/country/GHhttps://www.indexmundi.com/facts/ghana/pupil-teacher-ratiohttps://www.indexmundi.com/facts/ghana/pupil-teacher-ratiohttps://www.indexmundi.com/facts/ghana/pupil-teacher-ratiohttps://tradingeconomics.com/ghana/school-life-expectancy-pre-primary-gender-parity-index-gpi-wb-data.htmlhttps://tradingeconomics.com/ghana/school-life-expectancy-pre-primary-gender-parity-index-gpi-wb-data.htmlhttps://tradingeconomics.com/ghana/school-life-expectancy-pre-primary-gender-parity-index-gpi-wb-data.htmlhttps://ourworldindata.org/pre-primary-educationhttps://ourworldindata.org/pre-primary-educationhttps://ourworldindata.org/pre-primary-educationhttps://ourworldindata.org/pre-primary-education
-
45 Back to
Journey Map
Repetition rates for primary school, by sex and disability
Reading data for primary (i.e., % of children reading at grade level per EGRA), by sex and disability
Enrollment trends by type of disability (and gender if data)
# of students (or %) with disabilities enrolled in primary
Pupil textbook ratio for primary (policy and actual national average)
Pupil teacher ratio for primary (policy and actual national average)
Actual (average) entry ages in primary
Available data on school-related gender-based violence (SRGBV)
Out-of-school children (and adults)
Percent of primary school age children not in school, by sex and disability
National literacy/literacy rate, by sex (and age group) and disability (useful for the Context 3 context)
Education Financing
Government expenditure on education as % of GDP
Government expenditure on education as % of total government expenditure
% of education budget spent on salaries, non-personnel recurrent expenditures (i.e., textbooks) and capital expenditures
Government expenditure per student for pre-primary
Government expenditure per student for primary
Example of entries for Template 9: Background Education Statistics Table in Workbook Phase Two.
-
46
PHASE TWO
Back to Journey Map
PHASE TWO
Step 3d: Conduct an inventory of all the literacy programs (government, NGO, CBO, etc.)
Purpose: To identify the key early grade reading and literacy initiatives (preschool to primary) being implemented in the country/region that
directly align with the literacy domains in the Reading MATTERS framework.
Who leads: LLA Team Lead and Country-Level Education Advisors.
☐ Using Template 10 in Workbook Phase Two, include key programs in the country or region that directly align with the programmatic literacy domains in the Reading MATTERS framework. Initiatives beyond these literacy domains, such as infrastructure development of schools, do not need to be captured in this inventory; nor do programs that are very small-scale and/or do not have demonstrated results.
Table 10: Inventory of Literacy Initiatives
Example of entry into Template 10: Inventory of Literacy Initiatives in Workbook Phase Two.
Agency/ Organization
Initiative Name (Years)
Initiative Type & MATTERS Literacy domain
Target Audience
Donors/ Partners
Literacy Focus Area(s)/ Activities
LLA Context(s)
Government
e.g., National Literacy Programme 2010-present
Curriculum & TLM (evidence-based standards, norms and policies)
All grade 1-3 children
USAID, UNICEF
Reaches all government schools nationally and guides grade-level literacy instruction. Emphasizes phonemic awareness. Delivers in 5 languages depending on region.
1, 2
USAID & Implementing Partners
Multilateral (e.g. UNICEF, etc.)
Non-state schools & education providers
-
47 Back to
Journey Map
Step 4: Conduct an initial stakeholder analysis
The host-country government is critical to the LLA. At least two host-country experts are part of the
LLA Team, and host-country government representatives are part of the TWG. In addition to the
host-country government representatives from central and decentralized offices, representatives
from non-governmental organizations (NGO), community-based organizations (CBOs), teacher
unions, parent alliances (parent teacher associations, school management committees), and
teachers and head teachers are key stakeholders. Where stakeholders have been identified in an
ESA or RERA, integrate these stakeholders into the analysis below.
Purpose: To identify key actors and stakeholders across the three Contexts, and to ensure that
these stakeholders are included in the stakeholder worksho
top related