Africa knows! Conference, 10th December 2020Winny Cherotich - PAL NetworkVirginia Ngindiru – Zizi Afrique FoundationArmando Ali – Facilidade ICDS

Adaptation of solutions from South Asia: Experiences of improving learning outcomes in Africa

WHO ARE WE? How did it all start?

It started with an idea, a resolve to understand the ‘invisible problem’ and act on it.

Children are in school but are not Learning

ORIGINS: INDIA 2005‘from south to south’

THE ASERAPPROACH INSPIRED SIMILAR CITIZEN-LED

ASSESMENTS TO START…

ACTION: Teaching at the Right Level (TaRL)

Set Simple and Achievable Goals

Simple Assessments to gauge learning levels and form groups

Grouping and teaching children according to their learning levels

Using level-appropriate teaching learning methodologies and materials

Tracking children’s progress based on periodic assessments and enabling movement across groups

The TaRL movement

India:Combined

Activities for MaximisedLearning (CAMaL)

Mozambique:Wiixuutta

Nithweelaka –

Learning while Playing

Pakistan:Learning for

Access (ChaloParho Barho –Lets Read and

Grow

Kenya:Accelerated

Learning Program

Mexico:Jugando con las Matematicas –

Playing with Mathematics

Botswana:Teaching at the

Right Level (TaRL)

“Every Child Can”

Nigeria:

LEARNigeriaRemedial

Program (LRP)

Senegal:

KepparuJangandoo -

Under the shadow of Jangandoo

Tanzania:

Jifunze – Learn

Uganda:

Action for Learning Initiative

Wiixuutta NithweelakaAccelerated Learning

Program

Focus: Kenya and Mozambique programs

Accelerated Learning Program - Kenya

• 3 counties: Bungoma, Turkana and Tana River

• Grades 3-5

• 6,800 and 7,000 learners have participated in literacy and numeracy camps respectively. (2018-20)

Wiixuutta Nithweelaka - Mozambique

• 6 schools in Larde district

• Grade 3-5

• 814 pupils (2019)

Camp 1 (10-days)

Camp 2 (10-days)

Camp 3 (10-days)

Camp 1 (15-days)

Camp 2 (15-days)

Camp 3 (15-days)

Literacy assessment tool - Kenya

Numeracy Assessment tool - Mozambique

Literacy Results

Accelerated Learning Program Wiixuutta Nithweelaka

42.12

26.70

14.59

7.96 8.62

11.32

31.89

29.43

8.68

18.68

0.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

40.00

45.00

Not even letters Can read letters butnot words

Can read words Can readparagraphs

Can read simplestories

WN Literacy progress (%)

Base line After 50 days

27%

21%

25%28%

0%2%

6%

14%

18%

59%

0%

10%

20%

30%

40%

50%

60%

70%

Hawezi Silabi Maneno Aya Hadithi

ALP Literacy progress

Baseline After 30 days

Numeracy Results

Accelerated Learning Program Wiixuutta Nithweelaka

54.56

33.50

40.96

18.91

65.07

37.81

63.72

45.11

0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

Can do addition Can do Subtration Can do Multiplication Can do Division

WN numeracy levels (%)

Base line After 50 days

5%

38%

41%

12%

3%

0%

3%

8%

11%

17%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Addition subtraction Multiplication Division Application

ALP Numeracy progress

baseline After 30 days

Partnerships that mattered in adapting successful programs

• Partnership with like-minded civil society organizations

– Pratham - a learning partner to the PAL Network

– country level, county/local level partners ensure successful implementation of the program

• Partnerships with parents and the school communities

– Training teachers and community facilitators

– Parental engagement where evidence was packaged and shared

• Partnerships with government

– Program advisory group (National and County levels)

Lessons from implementation

• Teacher capacity - Adoption of these pedagogical practices will only be effective if teacher capacity is improved.

• Role of parents - We do believe that the quality of service delivery in education depends, on one hand, on the extent to which parents are involved in the school management and their capacity to hold the head teachers and teachers accountable.

• Importance of school governance to boost and sustain learning outcomes.

The flow of learning assessment methodologies; lessons from the citizen-led

assessment approach in Africa

Africa Knows PresentationAmos Kaburu & Zaida Mgalla

11th December 2020

Introduction

• Measuring learning outcomes has long been associated withthe global North and flow linearly to the Global South.

• According to Brookings Institute (2016), assessmentmethodologies, approaches, and tools in the global south haveborrowed mainly from the global North, where systems arepresumably developed.

• In other cases, national systems are relied upon to measurelearning and determine program effectiveness where learninginterventions happen.

Evolution and uptake of Citizen Led Assessments

• Non-state actors since 2005 have repackaged the socialaccountability frameworks in education through measuring learning

• Moving from the extreme continuum of proxi indicators of learningcivil society organizations in the global south have increased theirparticipation in monitoring EFA goals and now the SDGs.

• The Citizen–Led Assessment (CLA) approach was born in India in2005 when Pratham, one of India's largest NGOs, designed aninnovative approach to assessing all children's foundational readingand numeracy abilities, regardless of their schooling status. Thisassessment is the Annual Status of Education Report (ASER) inIndia.

About the global mapping study

• Over the past 15 years, the ASER tools and approaches have beenborrowed and adapted by many countries across the Global South.

• CLAs evolved organically from this approach, spurred by the interestof citizens who understood the importance of obtaining reliabledata on children's foundational learning that could build awarenessand inform policy and practice.

• After more than four years of anecdotal conversations about thespread of the CLA approach, PAL Network undertook a mappingstudy in 2018 to establish where the 'ASER' testing tools are beingused, by whom and for what purpose.

Spread of CLAs

Countries where citizen-led assessment (CLA) tools have been used

Findings from the global mapping study

An analysis of the mapping reveals three distinct uses for the CLAs in Africa. Assessment for accountability • Republic of Guinea Bissau in West Africa adapted the Mali and Senegal and

aligned with the national curriculum. • No program attached and was just a one off- Not popular Assessment for action (Program Design) • Tied to specific interventions especially programs such as Teaching at the Right

Level as the first step to select children to join the remedial programs. • Common Côte d'Ivoire, Ghana, Kenya, Madagascar, Nigeria, Uganda and

Zambia. • The assessments were adapted from the neighbouring PAL Network countries.Program Evaluation• Commonly used for evaluating effectiveness of learning interventions • The programs utilised CLAs at the periodic points of baseline, midline and

program endlines.

Findings from the global mapping study

Project and Program Evaluation

4projects

Girls’ Education Challenge

Project Evaluations

Countries | Kenya, Sierra Leone,

Somalia, Uganda

5projects

USDA McGovern-Dole ‘Food

for Education’ Program

Evaluations

Countries | Burkina Faso,

Côte d'Ivoire, Kenya, Mali,

Senegal

Teaching at the Right

Level

Countries | Côte d'Ivoire,

Ghana, Kenya, Madagascar,

Nigeria, Uganda, Zambia

Learning Intervention

Programs in crisis and

emergency settings

Countries | Niger, Nigeria,

7projects

2projects

3projects

5projects

Other learning

intervention program

Countries | Burundi, Kenya,,

Tanzania

The Earth Institute –

Millennium Village

Projects

Countries | Ghana, Kenya,

Nigeria, Tanzania, Uganda

Learning Intervention Program with Assessment

The flow of the CLAs was both through adoption and adaptation • The findings reveal a two-pathway flow of the citizen led assessments in Africa.• Taken on the "as was basis"- adoption and adjustment of design and frameworks – adaptationAdoption pathway• This was common in the first instance when CLAs first reached Africa in late 2000s.• Common in intervention programs especially design and evaluation because of short program

duration• Follows language and curriculum similarity therefore using assessments from Kenya, Uganda,

Tanzania, Nigeria and Mozambique.A case of adaptation• The adaptation pattern involves adjusting the tests to suit the contexts in most cases aligning with

the national curricula- in contexts where tools did not originally exist• Adaptation was long through the steps of developing the frameworks, setting up test development

teams, trialing and field pilots before the use• In other countries where programs used CLAs for program evaluations, adjustments to the

assessments were carried out that increased the number of assessment items and increaseddifficulty.

• However, the domains were not adjusted.• Similarly, the assessments did not go beyond literacy and numeracy. For instance, the Girls

Education Challenge that used the CLAs in Kenya developed versions that were pegged on gradefive, yet they retained the tests' framework where the domains of reading for fluency andcomprehension were the focus.

Adoption/Adaptation decision Rule

1) Context: The flow of the methodologies shows a horizontal move aroundgeographies and languages: Adoption in similar while adaption in differenteducation systems and languages of instruction

2) CLAS were adopted and adapted where there was an immediate need: The flowof the assessment was needs based: assessments for accountability were lesscommon, program evaluations had more uptake while program design weremoderately common- all informed by purpose and effort needed.

3) CLAS were considered cost-effective: This was evident in in-country uptake, whereusers felt that the over ten steps needed in designing and implementing CLAsmade it difficult for them especially for small intervention programs

4) Time at hand: Program design and evaluation favoured ready made tools(adoption) while accountability went for adaptation because of time at hand

Conclusions

1) CLAs are an acceptable public good in Africa playing two critical roles- social accountability and resolving the learning crisis.

2) The CLAs utility is limited in adoption gravitating the use towards adaptation to meet the complex needs

3) CLAs cannot fully meet the assessments needs in Africa meaning that seeding common assessments will increase comparability in the emerging needs of monitoring SDG4.

Measuring SDG 4: Experiences from designing

and implementing an international citizen-led

assessment in Africa

Dr. Mo Adefeso-Olateju

Muhammad Usman

Oyindamola AdegbouePanel B12

Contents

About Citizen-Led Assessments

Why ICAN?

What is ICAN?

ICAN 2019: Large-scale household-based implementation

ICAN and Consensus Building

Conclusion

Citizen-Led Assessment (CLA) approach innovated in India in 2005

Adopted and adapted in other Global South contexts

People’s Action for Learning (PAL) Network formalised in 2015, now comprising members in 14 countries across 3 continents

Nigeria

Nepal Tanzania Pakistan

Mexico

CLA approach is relevant for the Global South

Core features of CLAs:

Conducted in households to include all children irrespective of schooling status

Implemented orally and one-on-one as many children cannot read

Cover foundational learning content taught in early primary classes

Administer simple-to-use tools, processes and produce easy-to-understand data to

ensure wider engagement

Ensure collaboration with local stakeholders to create awareness and fuel local action

Global goal for education: SDG 4

SDG 4 focuses on ensuring “inclusive and equitable quality education and lifelong

learning opportunities for all.”

Within Target 4.1, the first indicator (SDG 4.1.1) tracks the “proportion of children and

young people achieving at least a minimum proficiency level in (i) reading and (ii)

mathematics, by sex”

(a) in Grade 2 or 3

Robust and regular data on learning outcomes are needed over time and across countries

Limited relevance of existing data to track progress towards SDG 4.1.1 for grades 2 or 3

Critical to measure learning early so that corrective measures can be implemented

ICAN - A comparable assessment for the Global South

The minimum proficiency

level descriptor for

numeracy under SDG 4.1.1

for class 2 or 3 requires

students to demonstrate

skills in number sense and

computation, shape

recognition and spatial

orientation.

NUMBER KNOWLEDGE

Counting, comparing

Number recognition

Operations (without and with carry-over, borrow and remainder)

Real world problems

50% number of objects

GEOMETRY

Position directionand

Shapes figuresand

MEASUREMENT

Length capacityand

Time calendarand

DATA DISPLAYRetrieving simpleinformation

50%

Overview of domains and tasks in the ICAN assessment tool ICAN contextual questionnaires

Sample tasks from the ICAN assessment tool

ICAN assessment tool is available in 11 languages on the PAL Network website

ICAN 2019: Large-scale household-based implementation

Proof of concept - feasibility of using common tools across different country contexts

ICAN 2019 retained all core features of the CLA architecture

Translation, training, data collection procedures based on PAL Network’s Data Quality

Standards Framework (DQSF) in all participating countries

Implementation by PAL member organisations in collaboration with local partners

Conducted in 13 countries

60 randomly sampled rural communities in 1 district per country

District not an outlier in terms of learning outcomes

Administered in randomly sampled households

to children in the age-group of 5-16 years

Each child assessed orally, one-on-one

ICAN 2019: sampled districts and reach

Building Consensus amidst Varying National Priorities

Sampling

Defining ‘numeracy’ for early grades

Contextual indicators

What to assess Enablers of Consensus Building

Working groups

Project Management Teams

Language

Conclusion. . .

ICAN plays a unique role in generating evidence on Learning

Outcomes in the Global south and informs on SDG target 4.1.1

ICAN is methodologically rigorous and offers opportunity to

generate data on foundational learning.

ICAN is an offering from the global south, to the global south.

The Potentials of Knowledge Adaptation and Creation in Africa: Experience from PAL Network Citizen Led Assessment

A paper Presented at Africa Knows Conference on 11th December 2020

By: Zaida Mgalla, Executive Director, Uwezo Tanzania

Outline

• Introduction

• Knowledge description

• Method of knowledge adaptation

• Outcomes

• Conclusion

Introduction

• The purpose of this paper is to share and describe the -Potentials of

Knowledge Adaptation and Creation in Africa based on the

experience of implementing Citizen led Assessment on literacy and

numeracy among children aged 5-16 years, as implemented by PAL

Network member organizations in Africa.

• In this paper knowledge is considered in its acquisition through observation, exchange and sharing experiences that adds value by benefiting continuous learning and adaptation to improve practice.

• CLAs

Introduction…..

• The formal education provision in Africa is also guided by international commitments and agenda such as Sustainable Development Goals- SDG 4.

• Many African countries have complied to achieve SDG4 ensuring access to school and quality and equitable learning for all.

• Many countries in Africa conduct various types of assessment and final grade examinations to measure success in learning, e.g. the final Primary School Leaving Examination ( PLSE).

•

• Most of these National exams are multiple choice oriented and focusing on measuring children's mastery of curriculum content than competencies that shows actual ability of the child.

• Multiple choice exams, can give a loophole for some children to guess correct answers and achieve high pass rate.

• E.g. children passing the PSLE and selected to join Secondary school while they cannot read, write or do simple arithmetic operations.

Learning Adaptation process

• In the struggle to find the best way of assessing actual ability of children in Literacy and Numeracy, a team of education experts from Kenya, Uganda and Tanzania visited Prathamorganization in India in 2008, to learn about the Citizen Led Assessment approach that was designed to assess children aged 5-16 years in foundational reading and numeracy abilities, regardless of their schooling status. (ASER, 2008, Uwezo East Africa Assessment reports, 2013, 2015).

Learning adaptation……

• The visit to India was triggered by the fact that many children were in school but their learning levels of foundational literacy and numeracy skills were very law,

• The experts from East Africa were able to learn from what Pratham education experts have gone through, or are going through to address similar assessment challenges.

Learning adaptation……

• Back to East Africa- Uwezo East Africa was established in Tanzania, Kenya and Uganda based on the new knowledge gained on the innovative CLA in assessing basic literacy and numeracy skills.

• CLA are household based

• Conducted by Citizens

• Assess literacy and Numeracy skills for children age 5-16

• Simple assessment tools

• Cost effective

• Partnership with CSOs/NGOs/ & Government

Uwezo Establishment in E/Africa

Adaptation process and Methods

• Uwezo assessment were designed to fit in the education philosophy and policy context,language of instruction in school, and curriculum expectations in each country.

• The knowledge gained from Pratham was extended to Uwezo East Africa teamincluding- through cross country learning visits of Uwezo staff and partners forfurther knowledge creation.

• Followed by development and documentation of Uwezo assessment goals, principles, assessment standards, tools and process,

• Language choice for assessment, English (in all countries), local languages in Uganda and Kiswahili in Tanzania and Kenya.

• Partnership building with Local NGOs and citizen volunteers involvement

• Training model for capacity building of key partners and implementers, data management systems and tools, and systematic monitoring of assessment activities.

Literacy tests

The CLA data at glance

• East Africa: Some Assessment findings:

• On average only 3 out of 10 children in grade 3 can pass Uwezo literacy and numeracy tests of grade 2 level in East Africa. (Uwezo 2015, Jones et al. 2014; Uwezo 2017).

• About 30% of class 7 pupils cannot pass a grade 2 literacy and numeracy test.

• Large proportions of children aged 9-13 years had not acquired Grade 2 literacy competencies in any of the languages assessed: 31% in Kenya, 46% in Tanzania and 70% in Uganda (Uwezo 2017, 14). Even larger proportions were unable to perform Grade 2 level numerical tasks.

• There is high inequality in access and learning across the countries, by Regions, and districts, family poverty status .

Adapting CLA and its spread in Africa

• In 2015, Peoples' Action on Learning (PAL) Network wasestablished to coordinate all countries that are implementingCitizen Led Assessment and related actions to improvelearning outcomes.

• Within few years, PAL network facilitated the hosting of manyorganizations from African countries (Ghana, Nigeria,Cameroon, Malawi, Mozambique, Mali, Senegal andBotswana) in East Africa for knowledge sharing and capacitybuilding to adapt and conduct CLA.

West Africa

2012Senegal

MaliNigeria

2015 2011

In Wolof, ‘Jàngandoo’ translates to ‘learn together’ – the large-

scale, citizen-led effort to assess learning levels in Senegal

LEARNigeria (Let’s Assess, Engage and Report Nigeria) is a program

of The Education Partnership Centre based in Lagos, Nigeria.

‘Bɛɛkunko’ means ‘the concern of everyone’ in Bamanakan – an independent mechanism for the assessment of learning in Mali

Adapting CLA and spread in Africa…

• CLA in those new countries was also adjusted to fit their context andlanguage e.g Portuguese in Mozambique, English and local languages inNigeria

Outcomes

• CLA knowledge and skills was well adapted and contextualized in Africaand enabled to build synergies among the implementing organisations andrespective governments

• CLA has gained tremendous popularity and recognition as a source ofindependent data generated by citizens.

Adapting CLA and spread in Africa..

• Outcomes…..

• CLA became a 1st household based assessment experience in Africaand enabled to capture attention on learning outcomes of out ofschool children and the inequality in education

• These connections within and outside Africa bred contextualassessments, new knowledge, collaboration, and coherence,resulting in a community of practice.

• We have developed regional hubs that provide space to continueexperience sharing, learning and creating new knowledge from eachother

Conclusions

• Systematic knowledge adaptation and connections within and outside Africa bred contextual citizen led assessments, new knowledge, collaboration, and coherence, resulting in a South-South community of practice.

• Africa has potentials for knowledge adaptation and creation of new knowledge that fits the African context and interest

•

• When done right, knowledge adaptation and exchange can build the capacity, confidence, and conviction of individuals and groups to act and improve practices

• The CLA data in many countries had been a workup call on learning crisis and

influence policy decision and practice to improve access to quality education for all

children

www.palnetwork.org

THANK YOU!