evaluation of unicef girls’ education project phase 3 (gep3)€¦ · for advice on the...

Tom Pellens, Rachel Outhred, Zara Majeed, Andrej Kveder, Michele Binci, Johanna Wallin, Fatimah Kelleher, Adrian Beavis and Shefali Rai

Draft Baseline Technical Report prepared by EDOREN on behalf of UNICEF GEP3

Evaluation of UNICEF Girls’ Education Project Phase 3 (GEP3)

September 2016

EvaluationoftheGirl’sEducationProject3

DisclaimerEDORENisaconsortiumofleadingorganisationsininternationaldevelopmentandeducation:OxfordPolicyManagement(OPM),andtheInstituteofDevelopmentStudies(IDS)attheUniversityofSussex,andissupportedbyUKAid.EDORENcannotbeheldresponsibleforerrorsoranyconsequencesarisingfromtheuseofinformationcontainedinthisreport.AnyviewsandopinionsexpresseddonotnecessarilyreflectthoseofOPM,IDSandEDORENoranyothercontributingorganisation.

EDOREN No2,16MafemiCrescent Tel +2348107278718EducationData,Research&EvaluationinNigeria Utako Tel +2348176678243 Abuja,Nigeria Email [email protected] Website www.edoren.org©EDOREN i

Acknowledgements

Weareverygratefultothegovernmentdepartmentsandstaffinthefiveprojectstates,aswellasatthenationallevel,whofacilitatedthebaselineresearchandsharedtheirperspectiveswiththeevaluationteam;inparticular,StateMinistriesofEducation(SMoEs),StateUniversalBasicEducationBoards(SUBEBS),StateAgenciesforMassEducation(SAMEs),CollegesofEducation,andtheGenderUnitoftheFederalMinistryofEducation(FME).

WewouldliketothanktheUKDepartmentforInternationalDevelopment(DFID),whichcommissionedthisindependentevaluation–particularlyDFIDEducationAdviserMsLauraBrannellyandtheDFIDEvaluationAdvisersforNigeria,MsKristenHopkinsandMrLaurieThompson,forusefuldiscussionsandadviceontheevaluationdesignandimplementation.

Ourthanksalsogototheimplementingorganisation,theUnitedNationsChildren’sFund(UNICEF):TerryDurnnian(SeniorEducationSpecialist),CharlesAvelino(EducationSpecialist)andAlassaneOuedraogo(MonitoringandEvaluationOfficer)gavemuchoftheirtimetotheevaluationteaminordertoparticipateintheevaluationdesignandtoshareinformationontheGirlsEducationProjectPhase3(GEP3)programme.SpecialthanksareduetotheGEP3stateteams–inparticular,thestateprojectcoordinators,andUNICEFstaffintheGEP3states,forassistanceinaccessinginformationandarrangingmeetingswithkeyinformants.Inaddition,wethanktheReadingandNumeracyActivity(RANA)implementationteamandmonitoringandevaluation(M&E)supportstaffforcoordinatingtheRANAroll-outwiththeevaluationteamandprovidinginformationontheRANAproject.

ForadviceonthedevelopmentoftheHausapupillearningassessmentwearegratefultoProfessorMalamiBuba,fromSokotoStateUniversity.WearealsogratefultoDrDavidJohnsonoftheDepartmentofEducationatUniversityofOxford,whodesignedtheoriginalTeacherDevelopmentNeedsAssessments(TDNAs)forGEP,whichservedasabasisforthedevelopmentoftheteachercompetencyandskillsassessment,aswellastoAdrianBeavis,forhisworkonpre-literacyitemdevelopmentandteachermotivationinstrumentdevelopment.

ThisevaluationisbeingcarriedoutwithfullindependencebytheEDOREN(EducationData,ResearchandEvaluationinNigeria)project.TheEDORENteamresponsibleforthedesignoftheevaluationandtheimplementationandanalysisofthebaselinedataconsistedof:TomPellens(TeamLeaderandEvaluationSpecialist);RachelOuthred(ProjectManager,SeniorEducationSpecialist,QualitativeDesignLeadandLeadforInstrumentDevelopment);ZaraMajeed(InstrumentDevelopmentandQuantitativeAnalysisSupport);AndrejKveder(SurveyTechnicalLead);MicheleBinci(Econometrician);FatimahKelleher(QualitativeLead);JohannaWallin(QualitativeResearcher);VictorSteenbergen(QualitativeResearcher);andShefaliRai(Researcher).Thecoreteamwassupportedbyagroupofspecialistadvisers:AdrianBeavis,DavidMegill,JusteNitiema,OladeleAkogun,StuartCameronandSouroviDe.QualityassuranceofthetechnicalreportwascarriedoutbyMonazzaAslam.Thekeyinformantinterviews(KIIs)forthebaselinetheoryofchangeassessmentwereconductedwiththeassistanceofDrHafsatLawalKontagora.


DisclaimerEDORENisaconsortiumofleadingorganisationsininternationaldevelopmentandeducation:OxfordPolicyManagement(OPM),andtheInstituteofDevelopmentStudies(IDS)attheUniversityofSussex,andissupportedbyUKAid.EDORENcannotbeheldresponsibleforerrorsoranyconsequencesarisingfromtheuseofinformationcontainedinthisreport.AnyviewsandopinionsexpresseddonotnecessarilyreflectthoseofOPM,IDSandEDORENoranyothercontributingorganisation.

EDOREN No2,16MafemiCrescent Tel +2348107278718EducationData,Research&EvaluationinNigeria Utako Tel +2348176678243 Abuja,Nigeria Email [email protected] Website www.edoren.org©EDOREN ii

ThequantitativebaselinefieldworkwasconductedbytheOxfordPolicyManagement(OPM)Nigeriaofficesurveyteam,whichincluded:FemiAdegoke(SurveyManager);ArogendadeEkundayo(FieldManager);BabatundeAkano(DataManager);andBukolaAdekolu(SurveyAdministrator).Thefieldworkwascompletedsuccessfullythankstothehardworkofthefieldenumerators.Ourspecialthanksthereforegoto:GodwinAdams,KogentDauda,FahadLawal,MuhammadKassim,LadidiLilianBulus,GloriaOlisenekwu,RachealLinusAvindia,MatthewOmaye,DanielOmokagboAyala,ArmayauSallau,MiriamUkemenam,MercyAbalaku,GabrielPanshakPandukur,EuniceAtajiri,MohammedLawanKagu,MusaAhmedMutallab,SandraWilson,CelestineAninna,AbdulhamidAbubakar,OluwaseyiMoses,LukmanAbubakarAbdullahi,DanlamiUseniNakoto,PreciousChrysanctus,JosephineMalum,GloriaObiageliUtti,IyaboAbulganiyu,ElizabethBello,HajaraNakkudu,MariamBako,AbubakarShehu,RaliyaSuliaman,JummaiChabo,AnnahJoseph,VeronicMichael,GraceBala,RuthAbbahJonah,AgathaTobias,BridgetBakar,OluwaseunTemtopeBalogun,HenriettaHabu,JanetJammaiJulius,MercyMusa,LilianNakoto,RuthVincent,CharityAndrew,TabithaChaamba,FatimaHamza,RuthAgono,BlessingDavo,AishaMusa,FalaluMa’sudMuhammed,SalomeSanda,MarleeyahSuleiman,AgathaLeo,ZainabLawal,AishaAbu,PaulSusanGaladima,HassanaGarba,SaratuTebu,SarayaDaudaLoya,NazeerMuhamman,HabibaMuhammed,HassanuHashim,LarabaIliyaandRotmwaIshayaBangson.

ThequalitativefieldworkwasconductedbyateamofnationalqualitativeresearchersworkingwiththeEDORENqualitativeresearchteam,andwascompletedduetotheirdedicationandgreatefforts.InthisregardwewouldliketothankYusufIbrahim,DrAdamuBabikkoi,JiddereMusaKaibo,HalimaSanni,DrAmminuDukku,NafinatuAbdullahi,FeliciaYakubuandMadaraAdamu.

Needlesstosay,duringthefieldworkstateeducationofficers,localgovernmentofficers,communityleaders,integratedQur’anicschool(IQS)proprietors,headteachers,teachers,pupilsandparentsplayedacrucialrolebygivingtheirtimesoastoallowtheimpactevaluationbaselineresearchtobeconducted.Wewouldliketoextendabigthankyoutothemall.

Forfurtherinformation,contacttheGEP3EvaluationProjectManager,DrRachelOuthred([email protected]),ortheEDORENCountryoffice([email protected]).ThecontactpersonatUNICEFisTerryDurnnian([email protected]),andtheresponsibleDFIDadviserisLauraBrannelly([email protected]).


©EDOREN iii

Executivesummary

Thisreportpresentsthefindingsofquantitativeandqualitativebaselinedatacollectionundertakenby

EducationData,ResearchandEvaluation inNigeria (EDOREN)aspartofamulti-yearevaluationof the

GirlsEducationProjectPhase3 (GEP3).This is the full technical reportof thebaselineevaluation,whichdescribes themethodology for the evaluation and the full set of quantitative and qualitative findings indetail.Itiscomplementedbyasynthesisdocumentthatsummarisesandcollatesthebaselineevidenceforalargeraudience.GEP3 is an eight-year project (2012–20) that seeks to improve school access, retention and learning

outcomesforgirlsinfivenorthernNigerianstates.ItismanagedbytheUNChildren’sFund(UNICEF)andfundedbytheUKDepartmentforInternationalDevelopment(DFID).Between2014and2017theprojectispiloting a series of interventions in primary schools and Integrated Qur’anic schools (IQS). Those mosteffective in improvingeducationoutcomes for girlswill be scaledup.Akey focusof theevaluation is toinformdecisionsrelatedtothescale-upofGEP3’sinterventions.Thescopeoftheevaluationinvolves:1. ahigh-levelexplicationandexaminationofGEP3’stheoryofchange(ToC);2. animpactevaluationofGEP3’searlylearningintervention;and3. aperformanceevaluationofGEP3’ssupporttoIQSs.

Thebaselineevaluationhasthreecoregoals:first,tocapturetheeducationsituation,andperceptionsofthissituation,atthestartofthere-designedGEP3,sothatchangescanbemeasuredduringfollow-updatacollection rounds, andproject attributionor contribution canbe assessed; second, to answer evaluationquestions about the relevance of the project; and third, to provide evidence to informGEP3’s ToC andprojectimplementation.

ExaminationofGEP3’sToC

Methodology

ThebaselineassessmentofGEP3’sToCfocusesontwoevaluationquestions:

• HowplausibleisGEP3’sToCinthecontextoftheGEP3states?• HowappropriateareGEP3’sinterventionsintermsoftheimplementationstrategy?Theplausibility of the ToC is examinedby reviewing stakeholders’ understandingof intendedoutcomes,interrogating the logic of the outcome chain, and identifying factors that have a key bearing on theachievement of the stated outcomes. GEP3’s implementation strategy is assessed on three dimensions:stakeholderinvolvement, implementationcapacityandtheextenttowhichtheprojectfocusesonunder-served groups. This assessment is based on key informant interviews (KIIs) conducted largelywithGEP3state-levelstakeholdersinAugust2015.KIswerepurposivelyselected,withthesupportoftheGEP3stateteams,basedontheirknowledgeoftheproject.

Keyfindings

The plausibility of GEP3’s ToC is supported by its coherent logic, synergies across interventions, and

stakeholders’commonunderstandingofitsmainobjectives–althoughtherearecaveatshere.Synergiesbetween the project’s interventions could enhance its impact, but also present risks, in thatimplementation difficulties on one dimension of the project could undermine its performance on otherfronts. A second key caveat is thatwhile stakeholders understandGEP3’smain objectives, the project’soperational design is not always understood. For instance, at the time of the interviews government


©EDOREN iv

stakeholdersinvolvedinteachertrainingdidnotyetunderstandwhatcapacitiesneedtobedeveloped,inwhatways, andwhen. This couldmake it difficult for them to assessGEP3’s performanceor be held toaccountfortheresultsachieved.

Stakeholdersnoted that theproject’sobjectiveof increasinggirls’enrolment is feasible,particularly in

primaryeducation,butthatimprovementsinretentionandlearningoutcomeswillbehardertoachieve.

Stakeholderswereoftheviewthatenrolmentdrivesandcashtransfersarebotheffectivestrategiesthataddressimportantbarrierstogirls’accesstoeducation,andthattheycomplementoneanother.Retentionis held to bemore challenging as it is shaped by supply-side factors, such as the presence of sufficientteachers,thequalityofteaching,andthequalityoftheschoolenvironment.Stakeholderswerelesslikelyto recognise learning as a key expected project outcome. When they did, they noted that learningoutcomesaremoredifficult to improve thanaccess,owing to the lowqualityof teaching,particularly inrural areas, and insufficient investment in the supply of quality education. There was wide consensusamongstKIs that tackling thequalityof teachingandteachers iscentral to improving learningoutcomes.ThisisconsistentwiththefocusonthisvariableinGEP3’sToC.

Awiderangeofstakeholdersendorsedthe importanceof theearly learning intervention.Stakeholderssupporttheemphasisonliteracyduringearlygradesasafoundationalskill.However,thereisnoconsensusamong stakeholders about the extent towhich the use of amother tongue is a necessary condition toimprovelearningoutcomes.ThebaselinefindingsoftheearlylearningevaluationconfirmthatHausaisthelanguage of the immediate environment in Katsina and Zamfara, but they also highlight that Hausaknowledgeamongteachersis low.Thisneedstobetakenintoaccountinthedesignoftheearlylearningintervention.

GEP3’s interventions to support school governance are considered to be largely promising, although

thereare some risks related to the targetedempowermentof School-BasedManagementCommittees

(SBMCs).SBMCempowermentisbothapivotalintermediaryoutcomeinGEP3’sToC,aswellasoneofitsmostprecariouslinks.TheexpectationsplacedonSBMCsintermsofrolesandresponsibilitiesareveryhigheventhoughtheseorganisationsareoftenstartingfromaverylowbase,particularlyinIQSs.Atthestatelevel, High-Level Women Advocate (HiLWA) members engage with decision-makers and schoolcommunitiesandhavethepotentialforindirectinfluence.TheGirlsEducationSteeringCommittee(GESC)alsohasthepotentialtosupportgovernance,totheextentthatitisactive,sufficientlylocallyownedandaddresses key challenges. GEP3’s support to the educationmanagement information system (EMIS) andAnnualSchoolCensus(ASC)willplausiblyimprovedatareliability,althoughthiswillnotnecessarilyleadtodatabeingusedinthepolicy-makingprocess.

The findings point to some specific risks to the scale-up process. The scale-up process is reliant ongovernmentfunding.However,thereissignificantuncertaintyabouttheextenttowhichthisfundingwillbeforthcoming.ThisisdespitecertainsupportivemeasuresbeingintroducedaspartoftherecentdesignofGEP3,notablyanincreasedemphasisonhigh-leveladvocacyandafocusonamoremanageablesetofoutcomes. The scale-up of GEP3’s support to IQSs is particularly uncertain because the institutionalmandate over IQSs is unclear, the number ofwell-established IQSs available for scale-up is limited, andthere are supply-side constraints, particularly with regards to facilitators. One further constraint is thatstakeholdersdonothaveaclearunderstandingofhowscale-upisexpectedtotakeplace.

Overall,thebaselinefindingspointtoanumberofriskstothecausallinksintheToC.Criticalassumptionsrelatedtothemanagementandresourcingoftheeducationsystemarehighlyuncertain,inparticularthereleaseofgovernmentfunding,school investmentandhumanresourcecapacityremainingonaparwithincreasedenrolment,effectivemonitoringattheschoolandinterventionlevels,andSBMCs’abilitytoplaythewide-rangingrolethatisbeingsoughtaspartoftheproject.

Baseline findings on implementation capacity are mixed. The project makes an appreciable effort toinvolve a variety of stakeholders. However, their involvement in operational planning is uneven across


©EDOREN v

interventions. Government capacity building is embedded in project design, but an effective system isneeded to identify and address themost important capacity development needs.Monitoring capacity isrightly emphasised in GEP3’s re-design but its operationalisation has yet to advance. With regards toequity,severalGEP3interventionsaredesignedtobeequityenhancing,butitisunclearwhetherthemostvulnerablehouseholdsandgroupsareabletobenefitfromandactivelyparticipateinGEP3interventions.Notably,GEP3isnolongeroperatinginLocalGovernmentAreas(LGAs)withthehighestgendergapinallstates.

ImpactevaluationofGEP3’searlylearningintervention

GEP3’searlylearninginterventionaimstoimprovetheearlylearningskillsofchildreninprimaryGrades

1to3(Primary1(P1)–Primary3(P3))inthemothertongue,whilealsopreparingchildrentolearnwithEnglish as a language of instruction by the time they transition to Grade 4. A key measure of theintervention’ssuccesswillbe improved literacyskills.The interventionwillbe implementedoverathree-yearperiod (2016–2018) insixLGAs inZamfaraandKatsina (threeperstate).The interventionhas threekeycomponentsatschoolandcommunitylevel:theprovisionofapackageofHausa-mediumteachingandlearning materials to schools; early grade professional development for teachers and head teachers(includingmonthlyschoolvisits);andasetofcommunityawarenessandengagementactivitiestosupportearlygradeliteracy.

Methodology

The evaluation of the early learning intervention uses a theory-based approach and is designed as a

clusteredrandomisedcontrolledtrial(RCT),stratifiedbyLGAandtypeofschool(primaryschoolvs.IQS),and randomised at the school level. The intervention’s ToC was used as a framework to formulate theevaluationquestions.TheRCTdesignallowstheevaluationteamtomeasuretheattributableimpactoftheearly learninginterventiononlearningoutcomesbycomparingoutcomechangesinatreatmentgroupofschoolswith those ina control group that is statistically similaronaverage.TheRCTdesign is combinedwith the overarching theory-based evaluation approach to measure not only changes in final outcomevariablesbutalsonetchangesinintermediaryoutcomevariablesalongtheassumedcausalchain.Thiswillallowtheevaluationteamtounpackhowchangetakesplace.

Inorder tomeasure changes inoutcomesapanel surveywasdesigned fordata collectionatbaseline,

midlineandendline.Datawillbecollectedateachofthesethreestagesinasampleof120publicprimaryschoolsand120IQSsdrawnfromthesixinterventionLGAsacrossKatsinaandZamfara.Halfofthepublicprimary schools and IQSs form part of the treatment group, while the other half serves as the controlgroup. Baseline data collection was conducted in October–November 2015 during the first term of the2015–2016 school yearbefore the startof implementationof theearly learning intervention.Within thesample schools, male and female pupils and teachers were randomly sampled to form part of a panelsurvey.Sevendatacollection instrumentswereadministeredwithineachschool:pupilEnglishandHausaliteracy assessments, pupil and teacher questionnaires, a teacher knowledge and skills assessment, ateacherclassroomobservationandaheadteacherquestionnaire.Theliteracyassessmentswerecarefullydesignedandpilotedtoensurethatitemdifficultymatchedpupils’ability.

Analysisofthebaselinedataindicatedthatrandomisationhadworkedtocreatecomparabletreatment

and control groups. We assessed whether the randomisation had achieved its intended purpose bychecking whether key outcome variables and school-, teacher- and pupil-level characteristics differedbetweenthetreatmentandcontrolgroupsatbaseline.Thegreatmajorityofvariablesinvestigateddidnotshowanystatisticallysignificantdifferencebetweenthetwogroups.

Keyfindings


©EDOREN vi

Thebaselinefindingshighlightthattheearlylearninginterventionwillbeimplementedinachallenging

context.Thevastmajorityofschoolsarelocatedinruralareasandhavepoorinfrastructure.Whilealargershare of IQSs than public primary schools have access to drinking water and electricity, they have lessrooms on average and have less access to books and functional toilets for girls. Teachers in IQSs arepredominantlymale. The baseline also found that 40% of IQSs only have one teacher who teaches theintegrated curriculum subjects, which raises questions about the applicability of the school-level peermentoringapproachintheseIQSs.

Head teachers make some effort to address teacher attendance, but are less likely to take action to

improvethequalityofinstruction.Almost90%ofheadteachersinpublicprimaryschoolsreportedtakingactiononteacherattendance.ThecorrespondingfigureforIQSswasjustover50%.Overhalfoftheheadteachers interviewedhadnotobserveda single lessonduring theprevious school term.Similarly,halfofheadteachersdonothavemeetingswithteachersormeetthemlessthanonceamonth.

Anotableshareofearlygrade teachersdonot teach languages.Roughlyhalfof teachers reported thattheyteachonlyonesubject,ofwhichjustover40%teachasubjectotherthanHausaorEnglish.GiventhattheReadingandNumeracyActivity (RANA)willemphasiseearlygradereading, thispoolof teachersmaynot be suitable candidates for the intervention. This needs to be taken into account when selectingteachersfortraining.

Theteacherssurveyeddemonstratedverylimitedknowledgeandskillsinrespectofmostofthedomains

covered by the teacher assessments. Less than 3% of teachers were able to display competence inidentifying lowperformers,evidencing judgementsanddiagnosingpupilperformance, interpretingwordsandphrases,andwritingskills.AlthoughallteachersreportedthattheyspeakHausa, lessthan40%wereabletodisplaycompetenceinprimaryGrade1and2-levelHausa.ThisissignificantforaninterventionthatfocusesonteachinginHausaasteachers’subjectknowledgehasimportantimplicationsforthequalityofteaching.Teachers’verypoorskills in identifying lowperformersanddiagnosingpupils’performancearelikely to present key hurdles to improving teaching quality given the wide recognition in the educationliteraturethatchildrenlearnbestwhenteachingistargetedtowhatthechildisreadytolearn.Therearesomeminorvariationsinperformanceacrossdifferentgroupsofteachers,butknowledgeandskillslevelsareconsistentlylowwithinallgroups.

Thesefindingsregardingteachers’knowledgeandskillshaveafewkeyimplicationsfortheearlylearning

intervention.Theverylowlevelsofknowledgeandskillsamongstteachersatbaselineindicatesthatthereis substantial scope for improvement in this area. However, it also presents challenges, in that there isoftenalackofbasicfoundationalskillsthattheprojectcanbuildon.Thefindingsshouldserveasausefulinput to the RANA implementation team as they highlight some of the key areas of weakness that theintervention would need to address. They also provide an indication of the scale of the challengesconfrontingtheproject,whichwillhaveimplicationsforimplementationdecisions–forinstancerelatedtothefrequencyandcontentoftraining,andthefocusoftheschoolvisits.Classroom observations indicate that the quality of early grade instruction is low across all groups of

teachers.Theextentofpupil-centred learningobservedatbaselinewas low,aswas theextent towhichteacherslinkthelessontopreviouslearningandlearningobjectives.Incontrast,timeontaskwashigh,atanaverageof96%ofatotallesson.However,thisfindingshouldbeinterpretedwithcautionasitislikelythatthepresenceofobserversintheclassroomledtoanincreaseintimeontask.Thebaselinealsosoughttomeasuregender-sensitiveteachingpracticesusingclassroomobservationdata,butextremecomplianceeffectswereobservedacrossall items,raisingquestionsaboutthevalidityandreliabilityofthemeasure.Teachers’ responses to the assessments indicate that teachers are aware of objectives to target girls inclass,astheystatethatitisimportanttofocusongirls,buttheirresponsesalsopointtothepersistenceofdeeplyingrainedgenderbiases.


©EDOREN vii

Theavailabilityanduseof teachingand learningmaterialswasobserved tobevery low. This suggeststhatthedistributionoflearningmaterialsbytheRANAinterventioncouldfillanimportantgap.However,itwillbevital formaterialstobepitchedatanappropriate level, taking intoaccountthevery lowlevelsofknowledgeamongstbothteachersandpupils.Hausa-basedteachingisprevalentintheearlygrades.Hausawasusedinallclassesobservedatbaseline.In roughly half of classes, a second languagewas used in addition to Hausa. However, as noted above,manyteacherslackcompetenceinbasicHausa.The baseline sought to assess teacher motivation and attendance as these variables are likely to

influence the extent to which improved knowledge and skills amongst teachers translate into better

teaching.Teachermotivationwas relativelyhighon threedimensions: theeffortput into,andperceivedimportance, of teachers’ work; their enjoyment of teaching; and interaction between teachers. Scoresrelated to pressure andwork-related tension and teachers’ perceived self-efficacywere low. Therewaslimited variation across teachers. Absenteeism was higher in IQSs than in public primary schools, withfacilitators reporting that theywere absent for an averageof eight days over theprevious 60days. Thecorrespondingfigureforpublicschoolswasthreedays.ThebaselinefindingsindicatethatinbothEnglishandHausa,veryfewpupilshavelevelsofknowledge

that are appropriate for their grade. In both subjects, the vast majority of P2 pupils assessed haveknowledgeand skills expectedofpre-school children (pre-literacy skills). Furthermore, inboth cases, thepeak of the distribution of performance falls well below the cut-off point between pre-literacy andemergingliteracy.Thissuggeststhatsubstantialeffortwouldbeneededtoachieveasignificantincreaseintheshareofpupilsmovingfrompre-literacytoemergingliteracy.Pupils’knowledgeofphonic isparticularly low.Thepsychometricanalysis foundthat itemsthat requireknowledgeofphonicsrankasthemostdifficultitemsinboththeHausaandEnglishassessments.Correctlysoundingoutlettersandidentifyingsimilarsoundswasmoredifficultforpupilsthanwritingorreadingfullpassages.Learningoutcomesdiffer by age and gender.Older pupils in P2 performbetter than younger ones andboys perform better than girls. Gender differences in performance are small in the younger years, butincreaseoncegirlsreachpuberty(around12yearsofage).Regressionanalysis indicatesthatpupilcharacteristics,socio-economicbackgroundandschooltypeare

associatedwithlearningoutcomes,butteachercharacteristicsarenot.Thelatterfindingmayreflectthefactthatcompetencylevelsareverylowacrossallteachers.Itislikelythatastrongercorrelationwouldbedetected if there were greater variation in competency levels across teachers, and if their ability toinfluencepupils’learningoutcomeswashigher.Thisalsoimpliesthatiftheearlylearninginterventionleadsto substantial improvements in teachers’ knowledge and skills we should see a more significant linkbetweenteachercharacteristicsandlearningoutcomesatendline.

EvaluationofGEP3’ssupporttoIQSs

GEP3’s support to IQSs (IQSS) seeks to improve education outcomes at these schools, with the aim ofprovidinganacceptablealternativeformofqualitybasiceducationforgirls.Theinterventionseekstoraiselearning outcomes in basic literacy and numeracy for pupils at IQSs (especially girls), improve retentionamong girls, and (to a lesser extent) increase girls’ enrolment. The project targets registered Islamiyya,Qur’anicTsangayaEducation(IQTE)centresthatofferanintegratedcurriculumandinwhichatleast40%ofpupilsaregirls.TheseIQSslargelyoperateascommunity-basedinitiatives,butarewillingtobuildlinkswithgovernmentforthepurposesofmonitoringandtechnicalsupport.GEP3’sIQSSinvolvestheprovisionof training and mentoring of IQS facilitators, training for head teachers, the distribution of classroom


©EDOREN viii

teachingandlearningmaterials,capacitybuildingforCentre-BasedManagementCommittees(CBMCs)andtheprovisionofmini-grants.

Methodology

Theevaluationapproachdrawsontheprinciplesofcontributionanalysisandreliesonastrongmixof

quantitative and qualitative methods. It does not make use of a comparison group to make causalinferences but rather seeks to make credible causal claims about the intervention’s contribution toeducationoutcomesbyverifyingthechainofexpectedresultsandassumptionsasperacredibleToC,aswell as assessing alternative explanations for the outcomes observed. The evaluation will assess threecontributionclaims:• GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs;• GEP3’sIQSScontributestoanimproved,girl-friendlyenvironmentwithintheschools;and• more effective teaching of formal subjects and an improved, girl-friendly environment contribute to

improvedlearninglevels,particularlyamonggirls.

ThequantitativecomponentoftheevaluationconsistsofrepresentativesamplesurveysamongGEP3IQSsinthe12GEP3LGAsinBauchiandNiger.Thebaselinesurveytookplaceinasampleof60IQSs,stratifiedbyLGA.Itinvolvedtheadministrationofthesameseveninstrumentsusedfortheearlylearninginterventionevaluation, as well as a pupil numeracy assessment and CBMC questionnaire. The baseline qualitativeresearchtookplaceinsixIQSsthatwerepurposivelysampledusingtypicalandextremecasesamplingandthat were also included in the quantitative survey. The same set of IQSs will be visited at midline andendline.

Keyfindings

ContributionClaim1:GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs

The landscape of IQSs is complex. Variations in school structures, pupil–teacher ratios, facilitators’qualifications,schoolleadershipandmanagement,andthenumberandhoursforwhichintegratedsubjectsaretaughtperschoolsuggestthattheoutcomesofinterveningintheseschoolsarelikelytodifferfromoneanother. Some IQSsmaybemobile,whichwouldhave implications for continued teachingand learning,particularly for girlswho are unlikely tomovewith theMallam.1 The extent of integration varies acrossIQSs,andisloweronaverageinBauchithaninNiger.Acrossthetwostates,headteachersreportedthatanaverageof threehoursperweek is spentonteaching the integratedcurriculum,which iswellbelowtheeight hours recommended by official guidelines. A quarter of IQSs only have one facilitator, which willconstrainamentoringprocessbasedonschool-basedpeerinteraction.Onecrucialsupportivefactoristhatintegrationdoesappeartohavegainedacceptanceamongparentsandthecommunity.Facilitators’knowledgeandskillsinkeydomainsassociatedwitheffectiveteachingareverylow.Only2%of facilitators were able to display competence in identifying low performers. Less than 1% hadcompetence in writing skills and none of the facilitators assessed were competent in evidencingjudgementsanddiagnosingpupilperformance.Only34%offacilitatorswereabletodisplaycompetenceinGrade1and2-levelHausa.Mostfacilitatorsdisplayalackofunderstandingofwhatstepstheycantaketoimprovepupils’performance.Aswiththeearly learning intervention,thesefindings indicatethatthere is

1ThetermMallamcanhavedifferentinterpretations.WeconsidertheMallamtobetheheadofthereligiousschool,responsibleforreligiouseducation.


©EDOREN ix

tremendous scope for improvement in teachers’ performance, but also that it will be crucial for theinterventiontobecarefullytargetedtotheverylowlevelsofskillsandknowledgeamongstfacilitators.Facilitators scoredverypoorlyona composite indexof effective classroompractices. Inparticular, thedepthofpupil-centredlearningwasfoundtobelowandtherewaslittleeffortamongfacilitatorstolinkthelesson to previous learning and learning objectives. Time on task was generally high: in 65% of lessonsobservedpupilsspent100%ofthelessonontask.However,thismayhavebeeninfluencedbythepresenceofobservers.Thereismixedevidenceonfacilitators’attitudesandclassroompracticestowardsgirls.Thefacilitatorsinthesixcasestudyschoolsexpressedpositiveattitudestowardsgirls’education,althoughtheseresponsesarepronetosocialdesirabilitybias.Indeed,bothqualitativeandquantitativedatapointtothepersistenceofgenderbiases–forinstancerelatedtogirls’abilities.Thefindingsindicatethatfacilitatorsareawareofand sometimes practice gender-sensitive techniques, such as actively engaging both girls and boys.However, thesepractices seem tobeperformed in a tokenisticway—in linewithdevelopmentprojects’sensitisationefforts—andgender-biasedclassroompracticescontinuetotakeplaceinIQSs.Aswiththeearlylearningintervention,thebaselineevaluationsoughttoassessfacilitators’motivation

and attendance. On average, facilitators think that their role is important and they enjoy working asteachers, but they have poor perceptions of their teaching efficacy. The qualitative research found thatfacilitatorsfeelintrinsicallymotivatedbywhattheydo,andthatschoolstakeholders,includingpupils,thinkthatfacilitatorsasgenerallydedicated.However,italsoindicatesthatthefactthatmostfacilitatorsarenotpaidmakesitdifficulttoattractqualifiedfacilitatorsandholdthemaccountable.Thequantitativefindingsconfirmthataverysmallshareoffacilitatorsarepaidasalaryorastipend(3%inBauchi,33%inNiger)andpointtoapositivelinkbetweenreceivingremunerationandteachermotivation.Roughly75%offacilitatorsreported that they had been absent at least once in the previous threemonths. Furthermore, the casestudy findings indicate that few IQSshave set timetables, and formal subjects are taughtasandwhenafacilitatorisavailable.Theavailabilityanduseofteachingandlearningmaterialsisverylimited.AlmostnoHausamaterialsareavailable,althoughitisimportanttonotethatHausaisnotthemothertongueofallpupils.InBauchi,93%of sampledchildren reported speakingHausaathomebut the corresponding figure inNiger is just54%,with43%speakingNupe.SchoolleadershipinIQSsiscomplex,whichraisessomechallengingquestionsaboutwhopreciselyshouldbe targeted by the pedagogical leadership component of the IQS intervention. IQSs have a variety ofdifferent leadership roles (Mallam, head teachers, proprietor). These may be held by one person ordifferentpeopleandthewayinwhichresponsibilitiesaresplitacrosstheseindividualsvariesacrossIQSs.Theprofessionalandacademicqualificationsofheadteachers inIQSsaregenerally low,and,onaverage,below those of facilitators. This raises questions about head teachers’ technical capacity to exercisepedagogical leadership over facilitators. The appointment of a head teacher is not necessarily based onability and qualifications but has to dowith social statuswithin the community and perceptions aroundleadershipmoregenerally.

ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentinIQSs

CBMCs have some potential to bring about improvements in the learning environment in IQSs. Themanagement of IQSs is seen as collective, without clearly defined and delegated roles, which providesopportunitiesforcommunity-basedmanagementstructureslikeCBMCs.Thebaselinefindingsalsoindicatethatmost CBMCs are established and active, as indicated by regularmeetings andmonitoring activities.CBMCs have considerable community representation, although women are under-represented and veryfewmembersarechildren.This indicatesthatatpresentCBMCsareunlikelytoprovideaforuminwhichgirls’voicesandneedscanbeheard. Inall IQSs included in thequalitativestudytheMallamholdsakeypositionintheCBMC,suchaschairperson.ThisraisesquestionsabouttheextenttowhichCBMCscanplayaneffectiveroleinholdingschoolleaderstoaccount.


©EDOREN x

CBMCsdomakeanefforttoraiseresourcesfromthecommunity,butthisappearstoofferlimitedscope

inregardtobridgingresourcegapsinIQSs.ThequalitativeresearchindicatesthatCBMCsfeelthatalackof fundshinders theirability to ‘solve’problemsat IQSs.Theyalsonote thatwhileparentsarewilling tocontributeresources,theyoftenhavelimitedcapacitytodoso.ThesefindingshighlightthecaseforCBMCtrainingtogobeyondanarrowfocusoncommunityresourcemobilisationtoencompassthemobilisationofresourcesfromalternativesources.

CBMCmembersseemtolargelyunderstandtheirrolesandresponsibilitiesbutoftenlackthecapacityto

enact them. Key areas in which CBMCs are under-performing are school planning and financialmanagement.CBMCmembersdomakeaneffort tomonitor thequalityof the teachingand learning, inparticular pupils’ and facilitators’ attendance. CBMC members assert that the main constraint thatunderminestheirperformanceisalackoffunds,particularlytoimproveinfrastructureandpayfacilitators.

Most IQSs do not offer a girl-friendly school environment at present. The vast majority face majordeficiencies inphysical infrastructure.Only3%of the schoolshaveaccess toawater source,20%of theIQSsdonothaveaphysicalclassroomstructureforthestudents,andlessthan25%havefunctioningtoiletsforpupils,withonly8%havingfunctioningtoiletsforgirls.Only3%ofIQSshaveamothers’group,teacher–student association or pupil groupwhere students can discuss their concerns. Gender-biased classroompracticesandattitudesprevail,whichcanaffectgirls’learningopportunitiesandself-confidence.However,there appears to be an openness among school leaders to try and address these issues. ResourcesmobilisedbyCBMCsareinvestedinschoolsandgirl-friendlyinvestmentsareconsidered,buttheamountsraisedappearinsufficientgiventhepoorconditionoftheschoolenvironment.

Theextentofgovernmentengagementwith IQSs is limited.Themajorityof IQSs in thesamplehadnotbeen visited by a government official during the previous term. Government officials say that they facechallenges in reaching all the schools on a regular basis. Communities generally view support from thegovernmentnegatively,consideringgovernmentactorstobeunreliable.

ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlyschool

environmentcontributetoimprovedlearninglevels,particularlyamonggirls

Thebaselineevaluationidentifiessomekeyfindingsthatarerelevanttothiscontributionclaim.Girlsandboys are both equally likely to attend IQSs. However, both face challenges in regard to attending andremaining fully engaged with the learning process because of out-of-school responsibilities, which arelinked to household poverty. A large share of childrenwho study in IQSs attend another school aswell.There were significant cross-state differences, with 72% of pupils in Bauchi reporting attending otherschools,ascomparedtoonly13%pupilsinNiger.InBauchi,mostofthesechildrenwereattendingapublicprimary school (94%), as compared to 55% in Niger. This indicates that interventions at local primaryschoolsmayconstituteanalternativeexplanationforanyimprovementsinlearningoutcomesatIQSs.

Pupils’ learningoutcomesarevery low. InbothHausaandEnglishover90%ofGrade2pupilsassesseddisplayed literacy levels associatedwith pre-school pupils. Furthermore,most of these pupils are a longway from the cut-off for emerging literacy skills (those associated with the P1 curriculum). Numeracyoutcomeswerebetter,with69%ofpupilsdisplayingemergingnumeracyskills,comparedto21%withpre-numeracyskills.However,only11%ofpupilshadnumeracyskillsassociatedwiththeP2curriculum.Boysandgirlsbothperformpoorly,althoughgirls’performancedeclines,comparedtothatofboys,ataroundpuberty(roughly12yearsofage).Continuedbarrierstogirls’accessto,andretentionin,schoolingexist.Attitudestowardsgirls’educationarenotalwayssupportive,althoughthisappearstobechanging.Changingattitudesalone,however,maynot be sufficient to bring about change in behaviour as poverty is often cited as a critical reason whyparentsdonotsend theirchild toanyschool,publicprimaryor IQS.Bothboysandgirlsoftenengage insomeformofincome-generatingactivity,whichparentstendtoviewasafinancialandpracticalnecessity.


©EDOREN xi

Thisadverselyaffectsschoolattendance.Earlymarriageisalsocitedasakeyreasonforlowenrolmentandretentionamonggirls.

Recommendations

ThebaselinefindingspointtocertainkeyrecommendationsforGEP3.ThebaselineassessmentofGEP3’sToC highlights that continued political engagement is required to ensure that government funds aremobilisedforprojectscale-up.Theprojectshouldspecifyandcommunicateitsoperationalobjectivesandstrategies to government andother implementing partners, strengthen itsmonitoring of assumptions inthe ToC, and ensure that monitoring information is used to facilitate learning and accountability. Inparticular,emphasisshouldbeplacedonmonitoringGEP3’ssupporttoSBMCs/CBMCs,giventheirpivotalroleintheproject’sToC.

Baseline findingson theearly learning interventionhighlight that theRANA interventionwillneed tobecarefully tailored to thevery low levelsofknowledgeandskillsamongst teachers toensure that trainingcontentandmaterialsarepitchedattherightlevel,andthattrainingissufficientlyintensivetofillthelargegapsinteachers’subject,pedagogicalandcurriculumknowledge.Teachercapacitydevelopmentalsoneedsto incorporate actions to change teachers’ awareness of their own potential for influencing learningoutcomes,andtheirunderstandingofhowbesttosupportpupils’learning.Thepeermentoringapproachneedstobeadaptedfor the largenumberof IQSsthatonlyhaveoneteacherwho iseligible forsupportunder the intervention. RANA also needs to sufficiently strengthen the capacity of IQS governmentstakeholders to ensure that they effectively monitor and provide support supervision. Finally, thecomplexityoftheIQSleadershipmodelindicatesthatspecialattentionneedstobegiventoensuringthatappropriateindividualsatIQSsaretargetedfortheprogramme’spedagogicalleadershiptraining.

ThefindingsontheIQSSinterventionhighlighttheneedfortheprojecttostrengthenitsmonitoringoftheintervention. The IQS context is diverse, flexible and evolving. In order to adapt the intervention to thiscontext,quicklearningandfeedbackbasedonmonitoringdataisneeded.Theselectionoffacilitators,headteachersandCBMCmembersfortrainingrequirescloseattention,verificationandmonitoring.FacilitatortrainingandmentoringneedstobecarefullyadaptedtotheverylowlevelsoffacilitatorcompetencyandtheIQScontext.Similarly,learningandteachingmaterialsneedtobetailoredtotheverylowlevelsofskillsandknowledgeamongstbothfacilitatorsandpupils,andtothelanguageoftheusers(particularlyinNiger,where Nupe is widely spoken). Training and mentoring on gender-sensitive class practices needs to gobeyond a focus on gender-sensitive teaching practices to try and tackle entrenched gender biases, forinstancerelatedtogirls’abilitytolearn.Womenandgirls’participationinshapingeducationandtheschoolenvironmentneedsfurther investigation:atpresent,bothareunder-representedonCBMCs,which limitsthe scope for thesebodies toact as forums forwomenandgirls’ participation in the sector. Finally, thebaseline findings highlight the need for GEP3 to promote the mobilisation of resources beyond thecommunityandtoadvocateforfacilitatorstobeadequatelyremuneratedbythegovernment.


©EDOREN xii

Tableofcontents

Acknowledgements i

Executivesummary iiiExaminationofGEP3’sToC iiiImpactevaluationofGEP3’searlylearningintervention vEvaluationofGEP3’ssupporttoIQSs viiRecommendations xi

Listoffigures,tablesandboxes xiv

Listofabbreviations xviii

1 Introduction 11.1 Background 11.2 Objectivesofthebaseline 11.3 Organisationofthereport 2

2 BaselineassessmentofGEP3’sTheoryofChange 32.1 Objectiveofthisreport 32.2 Methodology 32.3 PlausibilityofGEP3’sToC 42.4 AppropriatenessofGEP3’simplementationstrategy 35

3 Baselineevaluation–Earlylearningintervention 433.1 GEP3’searlylearningintervention 433.2 Methodology 493.3 Comparisonofbaselinecharacteristicsbetweeninterventionandcontrolschools 873.4 Analysisofthedata 89

4 BaselineoftheIQSSevaluation 1374.1 GEP3’sIQSS 1374.2 Methodology 1414.3 PresentationoftheupdatedToC 1674.4 Analysisofthedata–ContributionClaim1:GEP3’ssupporttoIQSscontributestomore

effectiveteachingofformalsubjectsinIQSs 1774.5 Analysisofthedata–ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-

friendlylearningenvironment 2214.6 Analysisofthedata-ContributionClaim3:Moreeffectiveteachingofformalsubjectsandan

improved,girl-friendlyschoolenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls 248

Bibliography 270

AnnexA GEP3TheoryofChangeDiagram 273

AnnexB BaselineTheoryofChangeassessment 274B.1 Focusofkeyinformantinterviewsperstate 274B.2 OverviewofkeyinformantsinterviewedforthebaselineTheoryofChangeassessment 275

AnnexC Mapsofsampledschools 280


©EDOREN xiii

AnnexD CalculationofWeightsandFinitePopulationCorrectionFactorsforGEP3School,TeacherandPupilData 284

D.1 CalculationofGEP3Weights 284D.2 FinitePopulationCorrectionFactorsfortheGEP3Analysis 286

AnnexE EthicalReview 288

AnnexF TeachermotivationsubscalesforGEP3 290F.1 Distributionofthefiveteachermotivationsubscales 290

AnnexG Randomisationchecksoftreatmentandcontrolgroup 292G.1 SchoolLevelBalance 292G.2 PupilLevelBalance 297

AnnexH QualitativeEvaluationMatrixforSupportforIQS 301

AnnexI Evaluationgovernance,managementandindependence 302I.1 Governanceandmanagement 302I.2 Stakeholderinvolvementinframeworkdevelopment 303

AnnexJ RobustnessChecksonAgeforMainRegressionModel 304ExclusionofAgeVariable 305

AnnexK GEP3AssetIndex 308

AnnexL EarlyLearningDescriptiveStatistics 314

AnnexM IQSchoolSupport(IQSS)DescriptiveStatistics 326

AnnexN CBMCIQSSDescriptiveStatistics 342


©EDOREN xiv

Listoffigures,tablesandboxes

Figure1: DiagramdepictingToCfortheearlylearningintervention......................................................46Figure2: TimelineofdatacollectionandtheRANAintervention...........................................................52Figure3: Schoolentryprotocol...............................................................................................................58Figure4: Teachersampling......................................................................................................................59Figure5: Pupilselection–Headteacher’soffice.....................................................................................60Figure6: Pupilselection–classroom......................................................................................................61Figure7: Instrumentdevelopmentprocess............................................................................................62Figure8: GEP3surveyteamstructure.....................................................................................................67Figure9: Teacherknowledgemodel.......................................................................................................78Figure10: Theorisedcausalpathway(takenfromtheTDP).....................................................................82Figure11: Schoolsizebasedonaveragenumberofteachers,byschooltype.........................................91Figure12: Prevalenceofheadteachermanagementactions...................................................................92Figure13: Actionstakenandagegroups..................................................................................................93Figure14: SchoolstructureinIQSs............................................................................................................95Figure15: InfrastructureinKatsinaandZamfara......................................................................................96Figure16: Teacherqualificationandeducationbyschooltype................................................................99Figure17: Percentageofteachersachievingwithinthelower,middleandupperbandsofcompetence

acrosstheteacherknowledgeandskillssubscales................................................................102Figure18: Percentageofobservedteachersengagingindifferentkindsofteachertalk.......................107Figure19: Percentageofobservedteachersengagingindifferenttypesofteacheraction...................107Figure20: Percentageofclassroomswherepupilsengagedindifferenttypesofpupilaction..............108Figure21: Percentageofteachersscoringzeroinpedagogical,curriculumandsubjectknowledge.....110Figure22: Meanpupil-centredteachingscorebyyearsofexperienceasateacher..............................112Figure23: Percentageoftimeontask.....................................................................................................112Figure24: PercentageofclassroomsusingEnglish,HausaandArabic...................................................114Figure25: FrequencyofHausauseinclass.............................................................................................114Figure26: Comparisonofmotivationsubscales......................................................................................116Figure27: Motivationbyagegroups.......................................................................................................116Figure28: DistributionofagegroupsinIQSsandpublicprimaryschools.............................................119Figure29: HWItertilecategorisationbystate.........................................................................................120Figure30: Categoriesoffactorsthatinfluencelearningoutcomes........................................................121Figure31: DistributionofHausaliteracyproficiency..............................................................................123Figure32: DistributionofEnglishliteracyproficiency.............................................................................125Figure33: Hausaliteracybygenderandschooltype..............................................................................127Figure34: Hausameanscalescorebyage(95%confidenceinterval)....................................................128Figure35: MeanHausascalescorebyageandgender(95%confidenceinterval).................................128Figure36: Hausabywealth.....................................................................................................................129Figure37: InterventionlogicdiagramoftheIQSS...................................................................................141Figure38: TimelineforIQSSactivitiesanddatacollectioninGEP3pilotIQSs........................................146Figure39: Teachersampling....................................................................................................................152Figure40: Flowchartofdatacollectionrelatingtofacilitators...............................................................156Figure41: DiagramdepictingToCoftheIQSS.........................................................................................169Figure42: Causalpackageandunderlyingassumptionsof‘moreeffectiveteaching’............................172Figure43: Causalpackageandunderlyingassumptionsof‘improved,girl-friendlyschoolenvironment’...

.............................................................................................................................................174Figure44: Causalpackageandunderlyingassumptionsof‘improvedlearningoutcomes’....................176Figure45: Numberofintegratedsubjectstaughtperfacilitator............................................................181Figure46: Facilitatortrainingcontent,bystate......................................................................................182Figure47: Religioussubjectstaughtinschools,bystate........................................................................184Figure48: Percentageoffacilitatorsachievingwithinthelower,middleandupperbandsofcompetence

acrosstheteacherknowledgeandskillssubscales................................................................185


©EDOREN xv

Figure49: Percentageofobservedfacilitatorsengagingindifferentteachertalkactivities..................189Figure50: Percentageofobservedclassroomswherepupilsengagedinvarioustypesofpupilaction190Figure51: Numberoflanguagesusedbythefacilitatorduringonelesson............................................191Figure52: PercentageoffacilitatorsusingHausa,Arabic,Englishandotherlanguagesinclass............191Figure53: Percentageoffacilitatorsscoringzeroinpedagogical,curriculumandsubjectknowledge..193Figure54: Depthofpupil-centredteaching,bystate..............................................................................194Figure55: Percentageoftimeontaskinobservedlessons....................................................................195Figure56: Lengthoflessonsobserved,bystate......................................................................................195Figure57: MotivationsubscalesforIQSfacilitators................................................................................196Figure58: Availabilityanduseofresourcesinobservedlessons............................................................199Figure59: Contentofheadteachertraining,bystate............................................................................200Figure60: Frequencyofheadteachermeetingswithteachers,bytrainingstatusofheadteacher......201Figure61: Frequencyofheadteachermeetingswithteachers,bystate...............................................202Figure62: Availabilityofschoolrecords,bystate...................................................................................203Figure63: AveragenumberofmembersinCBMCs.................................................................................223Figure64: ContentofCBMCtraining.......................................................................................................224Figure65: ActiontakenbyCBMCstoaddresspupildropout,bystate...................................................226Figure66: AmountmobilisedbyCBMCs.................................................................................................227Figure67: ReceiptofGEPschoolgrant,bystate.....................................................................................227Figure68: Infrastructurerepairneeds,bystate......................................................................................230Figure69: Investmentcategories,bystate.............................................................................................231Figure70: Sourcesofmonitoringvisitstoschools,bystate...................................................................232Figure71: Ageofpupilsbygender..........................................................................................................250Figure72: Languagesspokenathome,bystate.....................................................................................251Figure73: Languageofinstruction,bystate...........................................................................................252Figure74: DistributionofHausaliteracyproficiency..............................................................................254Figure75: Englishliteracyproficiencydistribution.................................................................................255Figure76: Distributionofnumeracyproficiency.....................................................................................257Figure77: MeanHausascalescorebyageandgender(95%confidenceinterval).................................258Figure78: MeanEnglishscalescorebyageandgender(95%confidenceinterval)...............................258Figure79: Meannumeracyscalescorebyageandgender(95%confidenceinterval)...........................259Figure80: MeanEnglishscalescores,bystate........................................................................................259Figure81: MeanHausascalescores,bystate.........................................................................................260Figure82: Meannumeracyscalescores,bystate...................................................................................260Figure83: GEP3evaluationgovernancearrangements..........................................................................302Figure1: DiagramdepictingToCfortheearlylearningintervention......................................................46Figure2: TimelineofdatacollectionandtheRANAintervention...........................................................52Figure3: Schoolentryprotocol...............................................................................................................58Figure4: Teachersampling......................................................................................................................59Figure5: Pupilselection–Headteacher’soffice.....................................................................................60Figure6: Pupilselection–classroom......................................................................................................61Figure7: Instrumentdevelopmentprocess............................................................................................62Figure8: GEP3surveyteamstructure.....................................................................................................67Figure9: Teacherknowledgemodel.......................................................................................................78Figure10: Theorisedcausalpathway(takenfromtheTDP).....................................................................82Figure11: Schoolsizebasedonaveragenumberofteachers,byschooltype.........................................91Figure12: Prevalenceofheadteachermanagementactions...................................................................92Figure13: Actionstakenandagegroups..................................................................................................93Figure14: SchoolstructureinIQSs............................................................................................................95Figure15: InfrastructureinKatsinaandZamfara......................................................................................96Figure16: Teacherqualificationandeducationbyschooltype................................................................99


©EDOREN xvi

Figure17: Percentageofteachersachievingwithinthelower,middleandupperbandsofcompetenceacrosstheteacherknowledgeandskillssubscales................................................................102

Figure18: Percentageofobservedteachersengagingindifferentkindsofteachertalk.......................107Figure19: Percentageofobservedteachersengagingindifferenttypesofteacheraction...................107Figure20: Percentageofclassroomswherepupilsengagedindifferenttypesofpupilaction..............108Figure21: Percentageofteachersscoringzeroinpedagogical,curriculumandsubjectknowledge.....110Figure22: Meanpupil-centredteachingscorebyyearsofexperienceasateacher..............................112Figure23: Percentageoftimeontask.....................................................................................................112Figure24: PercentageofclassroomsusingEnglish,HausaandArabic...................................................114Figure25: FrequencyofHausauseinclass.............................................................................................114Figure26: Comparisonofmotivationsubscales......................................................................................116Figure27: Motivationbyagegroups.......................................................................................................116Figure28: DistributionofagegroupsinIQSsandpublicprimaryschools.............................................119Figure29: HWItertilecategorisationbystate.........................................................................................120Figure30: Categoriesoffactorsthatinfluencelearningoutcomes........................................................121Figure31: DistributionofHausaliteracyproficiency..............................................................................123Figure32: DistributionofEnglishliteracyproficiency.............................................................................125Figure33: Hausaliteracybygenderandschooltype..............................................................................127Figure34: Hausameanscalescorebyage(95%confidenceinterval)....................................................128Figure35: MeanHausascalescorebyageandgender(95%confidenceinterval).................................128Figure36: Hausabywealth.....................................................................................................................129Figure37: InterventionlogicdiagramoftheIQSS...................................................................................141Figure38: TimelineforIQSSactivitiesanddatacollectioninGEP3pilotIQSs........................................146Figure39: Teachersampling....................................................................................................................152Figure40: Flowchartofdatacollectionrelatingtofacilitators...............................................................156Figure41: DiagramdepictingToCoftheIQSS.........................................................................................169Figure42: Causalpackageandunderlyingassumptionsof‘moreeffectiveteaching’............................172Figure43: Causalpackageandunderlyingassumptionsof‘improved,girl-friendlyschoolenvironment’...

.............................................................................................................................................174Figure44: Causalpackageandunderlyingassumptionsof‘improvedlearningoutcomes’....................176Figure45: Numberofintegratedsubjectstaughtperfacilitator............................................................181Figure46: Facilitatortrainingcontent,bystate......................................................................................182Figure47: Religioussubjectstaughtinschools,bystate........................................................................184Figure48: Percentageoffacilitatorsachievingwithinthelower,middleandupperbandsofcompetence

acrosstheteacherknowledgeandskillssubscales................................................................185Figure49: Percentageofobservedfacilitatorsengagingindifferentteachertalkactivities..................189Figure50: Percentageofobservedclassroomswherepupilsengagedinvarioustypesofpupilaction190Figure51: Numberoflanguagesusedbythefacilitatorduringonelesson............................................191Figure52: PercentageoffacilitatorsusingHausa,Arabic,Englishandotherlanguagesinclass............191Figure53: Percentageoffacilitatorsscoringzeroinpedagogical,curriculumandsubjectknowledge..193Figure54: Depthofpupil-centredteaching,bystate..............................................................................194Figure55: Percentageoftimeontaskinobservedlessons....................................................................195Figure56: Lengthoflessonsobserved,bystate......................................................................................195Figure57: MotivationsubscalesforIQSfacilitators................................................................................196Figure58: Availabilityanduseofresourcesinobservedlessons............................................................199Figure59: Contentofheadteachertraining,bystate............................................................................200Figure60: Frequencyofheadteachermeetingswithteachers,bytrainingstatusofheadteacher......201Figure61: Frequencyofheadteachermeetingswithteachers,bystate...............................................202Figure62: Availabilityofschoolrecords,bystate...................................................................................203Figure63: AveragenumberofmembersinCBMCs.................................................................................223Figure64: ContentofCBMCtraining.......................................................................................................224Figure65: ActiontakenbyCBMCstoaddresspupildropout,bystate...................................................226Figure66: AmountmobilisedbyCBMCs.................................................................................................227


©EDOREN xvii

Figure67: ReceiptofGEPschoolgrant,bystate.....................................................................................227Figure68: Infrastructurerepairneeds,bystate......................................................................................230Figure69: Investmentcategories,bystate.............................................................................................231Figure70: Sourcesofmonitoringvisitstoschools,bystate...................................................................232Figure71: Ageofpupilsbygender..........................................................................................................250Figure72: Languagesspokenathome,bystate.....................................................................................251Figure73: Languageofinstruction,bystate...........................................................................................252Figure74: DistributionofHausaliteracyproficiency..............................................................................254Figure75: Englishliteracyproficiencydistribution.................................................................................255Figure76: Distributionofnumeracyproficiency.....................................................................................257Figure77: MeanHausascalescorebyageandgender(95%confidenceinterval).................................258Figure78: MeanEnglishscalescorebyageandgender(95%confidenceinterval)...............................258Figure79: Meannumeracyscalescorebyageandgender(95%confidenceinterval)...........................259Figure80: MeanEnglishscalescores,bystate........................................................................................259Figure81: MeanHausascalescores,bystate.........................................................................................260Figure82: Meannumeracyscalescores,bystate...................................................................................260Figure83: GEP3evaluationgovernancearrangements..........................................................................302Box1: RANA’sintermediateresultsandsub-components..................................................................43Box2: RANAmonitoringtools.............................................................................................................48Box3: EvaluationquestionsforevaluationofGEP3’searlylearningintervention.............................49Box4: P2ortheequivalentlevel.........................................................................................................54Box5: P1–P3orequivalentlevel.........................................................................................................54Box6: EstimationofMDEs..................................................................................................................56Box7: Explanatoryfactorsincludedinthepupilscorrelationmodel................................................130Box8: EvaluationquestionsforevaluationofGEP3’ssupporttoIQSs.............................................142Box9: KeystepsincontributionanalysisappliedtotheevaluationofGEP3’sIQSS.........................145Box10: EstimationofMDEs................................................................................................................149


©EDOREN xviii

Listofabbreviations

ASC AnnualSchoolCensus

BSAME BauchiSAME

CAPI Computer-assistedpersonalinterview

CBMC Centre-BasedManagementCommittee

CS CompositeSurvey

CSO Civilsocietyorganisation

DAC DevelopmentAssistanceCommittee

DFID DepartmentofInternationalDevelopment(UK)

ECCDE Earlychildcaredevelopmentandeducation

EDOREN Education,Data,ResearchandEvaluationinNigeria

EGRA/EGMA EarlyGradeReadingAssessment/EarlyGradeMathsAssessment

EMIS Educationmanagementinformationsystem

ERC EthicsReviewCommittee

ESSPIN EducationSectorSupportProgrammeinNigeria

FGD Focusgroupdiscussion

FME FederalMinistryofEducation

FOMWAN FederationofMuslimWomen’sAssociationsofNigeria

FTTSS FemaleTeacherTraineeScholarshipScheme

G4G GirlsforGirls

GEP3 Girl’sEducationProjectPhase3

GESCs Girls’EducationSteeringCommittees

GPE GlobalPartnershipforEducation

GPI GenderParityIndex

HiLWA High-LevelWomen’sAdvocates

HWI HouseholdWealthIndex

ICC Intra-clustercorrelation

IQQS IQSsupport

IQS IntegratedQur’anicschool

IQTEs Islamiyya,Qur’anicTsangayaEducationCentres

ITC Inter-temporalcorrelation

JSS JuniorsecondarySchool

KI Keyinformant

KII Keyinformantinterview

LGA LocalGovernmentArea

LGEA LocalGovernmentEducationArea


©EDOREN xix

M&E Monitoringandevaluation

MA Mothers’Association

MDAs Ministries,departmentsandagencies

MDE Minimumdetectableeffect

MTSS Medium-TermSectorStrategy

NCE NationalCertificateinEducation

NGN NigerianNaira

NHREC NationalHealthResearchEthicsCommittee(ofNigeria)

NIPEP NigeriaPartnershipforEducationProject

OECD OrganisationforEconomicCo-operationandDevelopment

OPM OxfordPolicyManagement

P2 Primarygrade2

PEA Politicaleconomyanalysis

PIU ProgrammeImplementationUnit

QCO Qualitativeclassroomobservation

RANA ReadingandNumeracyActivity

RARA ReadingandAccessResearchActivity

RCT Randomisedcontroltrial

RLP RANALiteracyPackage

SAME StateAgencyforMassEducation

SAVI StateAccountabilityandVoiceInitiative

SBMC School-BasedManagementCommittee

SESOP StateEducationSectorOperationalPlan

SMoE StateMinistryofEducation

SUBEB StateUniversalBasicEducationBoard

SPARC StatePartnershipforAccountability,ResponsivenessandCapability

SSCE SeniorSecondaryCertificateExamination

SWAGS StrengtheningWomenandGirls’Spaces

TDNA TeacherDevelopmentNeedsAssessment

TDP TeacherDevelopmentProgramme

TEGIN TransformingEducationforGirlsinNigeria

ToC Theoryofchange

TPD Teacherpracticediscussion

UBEC UniversalBasicEducationCommission

UNICEF UnitedNationsChildren’sFund

WCDP WholeCentreDevelopmentPlan

WSDP WholeSchoolDevelopmentPlan


©EDOREN 1

1 Introduction

1.1 Background

ThisreporthasbeenpreparedbyEducationData,ResearchandEvaluationinNigeria(EDOREN).EDORENiscontractedbytheUKDepartmentforInternationalDevelopmentinNigeria(DFIDNigeria)toevaluatetheGirlsEducationProjectPhase3(GEP3),inlinewiththeGEP3EvaluationFramework(EDOREN2015).

GEP3isaneight-yearproject(2012–20)thatseekstoensurethatmoregirlscompletebasic

educationandacquireskillsforlifeandlivelihoodinfivenorthernNigerianstates:Katsina,Sokoto,

Bauchi,NigerandZamfara.ItismanagedbyUNICEF,inpartnershipwithfederalandstategovernments,andisfundedbyUKAidDFID.Theprojectaimstoimproveaccess,retentionandlearningoutcomesforgirls.Between2014and2017,aseriesofinterventionsarebeingpilotedinpublicprimaryschoolsandintegratedQur’anicschools(IQSs).Thosemosteffectiveinimprovingeducationoutcomesforgirlswillbescaledup.During2014–2015,EDOREN,incollaborationwithUNICEFandDFIDNigeria,developedanevaluationframework,inparallelwithGEP3’sredesign.Theevaluationframeworkdefinedtheobjectives,scope,questions,designandpartnerresponsibilitiesoftheevaluationactivities,withafocusontheperiod2014–2017,toinformdecision-makingregardingscale-up.

ThescopeofEDOREN’sevaluationworkconsistsofthreesetsofevaluationquestionsandpieces

ofanalysis:

1. ahigh-levelexplicationandexaminationofGEP3’stheoryofchange(ToC);

2. animpactevaluationofGEP3’searlylearningintervention;and

3. aperformanceevaluationofGEP3’ssupporttoIQSs.

1.2 Objectivesofthebaseline

Theobjectivesofthebaselineanalysisare:

• tocapturetheeducationsituation,andperceptionsofthissituation,atthestartoftheredesignedGEP3project,withafocusonspecificinterventions,sothatchangescanbemeasuredduringfollow-updatacollectionrounds,andprojectattributionorcontributioncanbeassessed;

• toanswerspecificevaluationquestionsabouttherelevanceoftheproject;and• toprovideevidencetoinformtheGEP3ToCandprojectimplementation.

InlinewiththescopeofEDOREN’sevaluationworkthebaselineanalysisisorganisedintothreeparts:i)abaselineassessmentofGEP3’sToC,drawingmainlyonKIIswithstate-levelstakeholdersofGEP3;(ii)analysisofthequantitativebaselineresultsofinterventionandcontrolschoolsinLocalGovernmentAreas(LGAs)inwhichGEP3’searlylearninginterventionisbeingimplemented;and(iii)analysisofthequantitativeandqualitativebaselineresultsforIQSsthatareeligibleforGEP3support.


©EDOREN 2

1.3 Organisationofthereport

Thebaselinereportisorganisedintotwovolumes:asynthesisevaluationreportandatechnicalreport.Thesynthesisevaluationreportconsolidatesthemainfindingsandsupportingevidencecollectedthroughvariousevaluationmethods.Itisdesignedtobeaccessibletoallreaders.ThepresenttechnicalreportoftheGEP3’sbaselineanalysis,whichcomplementsthesynthesisbaselinereport,coversthemethodologyandtechnicalanalysisindetail.Itisintendedforthoseinterestedinthedesign,methods,detailedstatisticalresults,anddetailedqualitativeanalysisoftheevaluation.ItisstructuredaccordingtothethreecomponentsofEDOREN’sevaluationactivities:

Chapter2presentsthebaselineassessmentoftheGEP3ToC.Afterasummaryofthemethodologyused,itexaminestheplausibilityofGEP3’sToC.NextitassessestheappropriatenessofGEP3’simplementationstrategy.

Chapter3describesthemethodologyandbaselineresultsofGEP3’searlylearninginterventionevaluation.ItstartswithanoverviewoftheGEP3earlylearningintervention.Next,theevaluationandbaselinedatacollectionmethodologyispresented.Thisisfollowedbyacomparisonofbaselinecharacteristicsbetweeninterventionandcontrolschools,tocheckforbalanceaspartoftherandomisedcontroltrial(RCT)design.Thefollowingsectiondetailstheanalysisofthebaselinedata.

Chapter4providesapresentationofthemethodologyandbaselineresultsoftheevaluationofGEP3’ssupporttoIQSs.AnoverviewofGEP3’ssupporttoIQSsisfirstprovided,afterwhichtheevaluationdesignanddatacollectionmethodsarediscussed.Next,theanalysisofthequantitativeandqualitativedatacollectionispresentedaccordingtothecontributionclaimsthatareformulatedbasedontheinterventions’ToC.

Theannexesofthisreportprovideadditionaldetailedmethodologicalandanalyticalinformation.


©EDOREN 3

2 BaselineassessmentofGEP3’sTheoryofChange

2.1 Objectiveofthisreport

ThisreportpresentsthebaselineassessmentoftheGEP3ToC,inlinewiththeGEP3evaluationframework.ThereportfocusesontwomainevaluationquestionsaspartoftheevaluationofGEP3’srelevance:2

• HowplausibleisGEP3’sToCinthecontextoftheGEP3states?• HowappropriateareGEP3’sinterventions,intermsofimplementationstrategy,giventhe

contextoftheGEP3states?

AsdiscussedintheGEP3evaluationframework,afirststepintheevaluationprocessistofurtherexplicateandexaminetheToC–inparticularbyinterrogatingcausalpathwaysandinfluencingfactorsthatenableGEP3interventionstobringaboutintendedoutcomes.GEP3’sToCdiagramisincludedinAnnexA.3

TheassessmentpaysparticularattentiontoidentifyingfactorsthatarelikelytoinfluencewhetherGEP3achievesitsoutputsandoutcomes.Thisallowsimplementingpartnerstopayattentiontothesefactorsduringprojectimplementation,inordertoimprovetheproject’slikelihoodforsuccessandforM&EtoassessandaccountforhowGEP3isachievingitsoutcomes.

2.2 Methodology

TheassessmentofGEP3’sToCisbasedonKIIsthatwereundertakenmostlywithstate-levelstakeholdersofGEP3.Anumberofnational-levelstakeholdersoractorsworkinginsectorsrelevanttoGEP3interventionswerealsointerviewed.Furthermore,theassessmentmakesuseofadocumentreview,althoughanextensivereviewofsecondarydatasourcesorliteraturehasnotbeencarriedout.

KIswerepurposivelyselectedtocovergovernmentstaffinmanagementpositionsintheeducationministries,departmentsandagencies(MDAs)thatconstituteGEP3’spartnership,aswellrepresentativesofgovernmentornon-governmentagenciesinvolvedinGEP3implementation.Furthermore,somenon-governmentinformantsthatcouldprovidefurtherinsightsaboutspecificinterventionswereselected.4KIIswerespreadacrossGEP3interventions,puttinganemphasisondifferentinterventionsindifferentGEP3states.AnnexBprovidesanoverviewofthefocusthatwasplacedontheinterventionsineachstate.ToidentifythepotentialKIs,alonglistofKIswasdraftedincollaborationwithGEP3stateteams,whichweresubsequentlydiscussedwitheachoftheStateProjectCoordinatorsduringacall.BasedonthisdiscussionandKIavailability,afinallistwasselectedbytheevaluationteam.

Fieldworktookplacefrom10Augustuntil27August2015,withtworesearchersspendingapproximatelytwodaysineachoftheGEP3statecapitals(Katsina,Gusau,Sokoto,Minnaand

2Theevaluationframeworkidentifiedanumberguidingsub-questions.SeeAnnexA.3ThediagramispartofUNICEF’s(2015a)ToCdocumentfortheproject.4Non-governmentinformantswereselectedbasedontherichandconcreteknowledgeitwasassumedtheycouldprovideaboutaspecificintervention.Theywereselectedbytheresearchteamincoordinationwith,andsometimesattheproposalof,GEP3stateteamsandDFID.


©EDOREN 4

Bauchi).Theresearchteamconducted,onaverage,11face-to-faceKIIsperstate.KIIswithnational-levelexpertswereconductedinAbujaorbyphone.ThelistofpeopleinterviewedaspartoftheKIIsispresentedinAnnexB.Theresearchteamconductedatwo-dayparticipatorymethodsreviewbeforethestartoftheinterviewperiod.TheteammetwithDFIDandUNICEFforadebriefaftertheinterviewperiod(inAbujaon28August).

Interviewsfollowedsemi-structuredinterviewguidesthatwereadaptedtotheinterviewedstakeholderandtheinterventionstobecoveredineachstate.Whiletheinterviewguidesensuredthatpre-setbasiclinesofinquirywerepursued,thesemi-structurednatureoftheinterviewsallowedforparticularsubjectstobefurtherexploredandexpanded,inlinewiththeexploratorynatureofthebaselineToCassessment.

Followingthemethodologypresentedintheevaluationframework,themainlinesofinquiryconcerned:

• theeducationcontextofthestate;• thestakeholders’understandingofprojectand/orinterventionobjectivesandpathwaysto

change;• factorsinfluencingtheoutcomesoftheprojectand/oritsspecificinterventions;• thestakeholders’understandingofimplementationstrategies,andinputsandprocessesusedas

partofthisimplementation.

Interviewswereaudiorecordedafterreceivingconsentfromtheintervieweesandinterviewnotesweretranscribedwiththesupportoftheaudiorecordings.DatawereorganisedandanalysedbytheresearchteamusingNvivoqualitativedatamanagementsoftware.

Themethodologyhadcertainlimitations:

• Becauseoftheexploratorynatureoftheinquiry,aprioritywasgiventobreadthoverdepth,byinterviewingalargenumberofstakeholdersinordertoallowtheinputofmultipleperspectives.Furthermore,stakeholdermeetingswerealsousedtodiscusstheevaluationplans.Thishaslimitedin-depthexaminationofspecificinterventionsandcausalpathways.

• DuetotheGEP3redesignandthe2015elections,someKIswerenewtoGEP3’sredesignedinterventionsandtheirimplementationinspecificstates.5Furthermore,someinterventions,suchastheearlylearninginterventionandGirlsforGirls(G4G)groups,hadyettobeoperationalised.Thisalsolimitedthedepthofthedatacollection.

• Duetotimeconstraints,theinterviewstargetedstate-levelornational-levelstakeholders.Therefore,itwasnotpossibletocompleteandtriangulatetheinformationobtainedwithstakeholdersatlocalgovernmentlevel,althoughtheseindividualsareimportantstakeholdersintheimplementationoftheGEP3interventions.

2.3 PlausibilityofGEP3’sToC

ThischapterexaminestheplausibilityofGEP3’sToCbyreviewingstakeholders’commonunderstandingofitsintendedoutcomes,byinterrogatingthelogicofitsoutcomechain,andby

5ThreeoutoffiveoftheGEP3statecoordinatorshadrecentlystartedworkingintheirspecificstate,althoughallofthemhadexperienceofworkingaspartofGEP3inotherstates.TheDirectorofPlanning,ResearchandStatisticsinKatsinahadonlybeeninhispositionfortwomonths.HewasassistedintheinterviewbytheGEP3deskofficer.TheKIfortheCollegeofEducationinZamfarahadonlybeentheGEP3focalpersonforonemonth.


©EDOREN 5

makingmoreexplicitfactorsthatarenecessaryfor,ormayhinder,theachievementoftheoutcomes.Thechapterfirstprovidesabird’s-eyeassessmentofGEP3’sultimateoutcomes,itslogicandmainproposedinfluencingfactors.Next,adetaileddiscussionofdifferentinterventionsperGEP3outputispresented.ThechapterendswithanassessmentoftheplausibilityofGEP3’spilot-to-scaleapproach.

2.3.1 Generalassessment

ThekeyprojectstakeholdersaregenerallywellawareoftheenvisionedultimateoutcomeofGEP3,intermsofimprovinggirls’education.Higherlevelgovernmentpartners6,UNICEFfieldofficersandGEP3stateteamswereaskedaboutwhatthemainresultsarethatGEP3intendstoachievebytheendoftheproject.Increasedgirls’schoolenrolmentwaswellrecognisedasamainintendedoutcome,andwascommonlylinkedtothetargetofgetting1millionadditionalgirlsintoschool.TheKIsrecognisethatGEP3’sultimatesuccessisalsodependentongirls’retentionandimproved

learning.Theseoutcomeswerementionedlessfrequently,particularlyamonggovernmentKIs,whichsuggeststhatGEP3isforemostassociatedwithimprovinggirls’accesstoschool,representedbyitstargetofgetting1millionadditionalgirlsintoschool.

TheUNICEFKIsinthefieldofficesexpectthatincreasedgirls’enrolmentwillbeeasiertoachieve

comparedtoretentionorlearningbecausethelatterrelyoneducationsupply-sidefactors,suchassupplyofqualifiedteachers,whichisrecognisedasakeychallenge(alsobygovernmentKIs,seebelow).Consequently,teachercapacitydevelopmentiswidelysupportedasakeyareaofintervention,notonlybecauseoftherecognitionofcurrentteachercapacitybeinganimportantchallengebutalsobecauseteachingqualityisconsideredcentraltoimprovinglearningoutcomes.Furthermore,communityownershipthroughtheworkoftheSchool-BasedManagementCommittees(SBMCs)iscommonlyheraldedasthesolutiontodemand-sideandsupply-sideconstraintsandtogovernanceproblems.GEP3’semphasis,sincetheredesign,onteachingquality

andcommunityempowermentthroughSBMCsisthereforewellalignedwithstakeholders’

prioritiesandtheirunderstandingofhowthequalityofeducationcanbeimproved.Aswillbediscussedinthenextsections,theeffectivenessofthisstrategyisdependentonawiderangeofassumptions.

Whilegender-sensitiveteachingpractices,theinvolvementofMothers’Associations(MAs)andinvestmentinthegirl-friendlinessofschoolenvironmentsarerecognisedbyKIsaswaystoimprovethequalityofeducation,particularlyforgirls,theKIIsprovidelittleevidenceabouthowthiswillbeoperationalised,whichintheendwilldeterminehowplausibleitisthatthisoutcomewilloccur.

WhileKIshavearelativelygoodknowledgeofGEP3’soverallstrategy,andappreciatethefactthattheredesignhasimprovedtheproject’sfocus,itsoperationaldesignislesscommonlyunderstood.Attributesofintervention-specificoutputsandoutcomes—suchastheprioritytargetgroups,thetiming,andthenatureoftheoutcome—areoftennotcleartoKIs.Thismaybepartlyduetotheredesignandthefactthatsomeinterventionsstillneedtobeoperationallydefinedandcommunicated.Whenoutputandoutcomeattributesarenotclear,performancewillbedifficulttomeasureandassess.

TheintermediateoutcomesofGEP3’sdifferentinterventionsarehighlyinterlinkedandsynergetic,

whichsupportstheassumptioninGEP3’sToCthatcombinedinterventionswillsupportthemost

6Thequestionwasaskedatdirectorandhigherlevels(permanentsecretaryorsecretary)intheStateMinistriesofEducation(SMoEs)andStateUniversalBasicEducationBoards(SUBEBs).


©EDOREN 6

vulnerablegirlsmoreeffectively.Forexample,thecashtransferprogrammebuildsonthesensitisationconductedbytheenrolmentdrivesandthecapacityoftheSBMCstosupportthecashtransferenrolmentandmonitoring.Theoutcomesofteachercapacitybuildingdependonthesupportivesupervisionbytheheadteacherandtheschoolenvironmentbeingimprovedviamini-grantsandcommunityinvolvement.Thesesynergiespotentiallyreinforcetheresultsoftheinterventions,althoughtheyalsoconstituteariskofinterventionproblemsspreadingacrossthe

programme.Inthisregard,theSBMCempowermentinterventioniskeysincemostofGEP3’sschool-levelinterventionsassumefunctioningSBMCs.ThefactthatsupporttoSBMCswasmaintainedinallGEP3focusschoolsaspartoftheprojectredesignisthereforeimportant.7AsrecognisedintheGEP3ToCdocument,theToCalsoincludesapotentially‘negative’relationshipbetweenincreasedaccessanddeterioratingteachingandlearningquality.Totheextentthatsupply-sideinvestmentdoes

notkeepupwithincreasedenrolment,teachingandlearningqualitymaybenegativelyaffected.KIsindicatethatthisriskismaterialising(seebelow).Finally,itisworthnotingthatthecompletepackageofinterventionsisnotimplementedinallGEP3states.Therefore,combinedinterventionscanstrengthenresultsinthosecontextsforwhichthedifferentinterventionsareplanned.

WithregardstoGEP3’ssupporttoIQSs,KIsaresupportiveofintegrationandGEP3’scontributiontothis.TheKIsworkingwithIQSsexpectGEP3tocontributetogirls’enrolment,retentionandmainstreamingintheformaleducationsystem.ImprovedlearningoutcomesarementionedbysomeKIs.ItwillbeusefultoclarifytowhatextentmainstreamingisanimpactthatGEP3considersas

partofitsscope.IfmainstreamingisconsideredtofallwithinthescopeofGEP3,thiswouldrequirepayingattentiontobarrierstomainstreaming.Povertyismentionedasonesuchbarrier.

‘Therearemanyparentsthathavetheinterestathearttosendchildrentoschool,buttheymaynotbeabletocaterforsendingtheirchildrentosecondaryschool.PovertyisoneofthefactorsthatmotivatesparentstosendtheirchildrentoQur’anicschoolbecauseitisfree.TheseparentsmaystillbetoopoortofacilitatemainstreamingoncethechildrenhavepassedthroughBasicorPost-Basic.’(GovernmentKI,Bauchi)

Totheextentthatpubliceducation,inparticularsecondaryeducation,isnotfullyfreeofcharge,povertymayinhibitatransitionfromIQSstothepubliceducationsystem8.SomeKIsclaimtherearenoinstitutionalbarrierstomainstreaming,andsuggestthatIQSpupilsareabletotransitiontoaformalschoolwithoutadministrativehindrance.Nonetheless,itrequiresfurtherinvestigationtodeterminehoweasyitisforpupilstodosoinpractice.

ThefactorsthatinfluenceGEP3’sToCarediscussedinmoredetailinthenextsections.Inthissection,wehighlightthemainfactors:

• KIsconfirmthefindingsofEDOREN’spoliticaleconomyanalysis(PEA)thatacentralfactoraffectingthesuccessoftheprojectisstategovernmentsfulfillingtheircommitmentintermsof

financialcontributionstotheproject,especiallythereleaseoffundsbythestateexecutivegovernor.WhiletheGEP3ToCacknowledgesthisandhasincludedinterventionstomitigatethisriskfactor,theKIsindicatethisremainsanimportantconcern.AreviewbytheStatePartnershipforAccountability,ResponsivenessandCapability(SPARC)andtheStateAccountabilityandVoiceInitiativeSAVI(2015)confirmsthatthepublicfinancialmanagementsituationintwooutofthree

7AccordingtotheGEP3workplanrefresherSBMCtrainingisplannedinallfiveGEP3states.8WhilepublicbasiceducationisofficiallyfreeinNigeria,parent–teacherassociationlevies,thecostofschooluniforms,writingmaterialsandotheradhoccostsaremajorcausesofnon-enrolment,absenteeismanddropout(HumphreysandCrawfurd,2014)


©EDOREN 7

GEP3statesexaminedisstillpoor.9Furthermore,someoftherecommendationsofEDOREN’sPEA(2014)remainunaddressed.StategovernmentexpenditureonGEP3interventionscanstillnotbesystematicallyaccessed,anditisnecessarytorelyonadhocrequeststogovernmentcounterpartstoprovideinformationonexpendituresonGEP3activities.Thelevelofresourceprovisionisnotexplicitlylinkedtotheachievementofspecifictargets.However,positiveelementsarethatthenewlyelectedgovernorsseemtoprioritiseeducationintheirdiscourseandhaveundertakensomestepstosupporteducationpolicy-making.GEP3’sfocusonanarrowersetofinterventionsfollowingitsredesignhasallowedformoreeffectivemanagementandfollow-upofstatecommitments.Furthermore,UNICEF,togetherwithstatepartners,hasconductedhigh-leveladvocacyinrelationtothestategovernors,whichismentionedbyKIsasimportantinordertoimprovethelikelihoodoffundingrelease.

• KIsagreethatteachingqualityiscentraltoimprovinglearningoutcomes.Therefore,teacher-relatedsupportingfactors,suchasnumberofteachers,deploymentandtransfersofteachers,andmotivationofteachersareconsideredkeytothesuccessoftheprogramme.Monitoringofteachersishighlightedasbeingespeciallyimportant.

• Asmentionedabove,theroleofthecommunityinthedemand,supplyandgovernancesideoftheschoolsystemisanotherfactoremphasisedbyalargenumberofKIs.ThecommunityisexpectedtodirectlycontributetoschoolinputsandtotheschoolenvironmentinlinewiththesloganoftheUniversalBasicEducationCommission(UBEC)‘Educationforallistheresponsibilityofall’.ThroughtheSBMCs,thecommunityisalsoassumedtotakeownershipofschooldevelopmentandoversight.Hence,theKIsconfirmthekeyassumptioninGEP3’sToCthattheeffectivenessoftheSBMC/Centre-BasedManagementCommittees(CBMCs)hasanimpactonGEP3’soutcomes.Aswillbediscussedbelow,thispresentstwomainrisks.First,thehighexpectationswithregardtoSBMCsmaynotmatchtheircapacity,asthecapacitybuildingprocessisexpectedtobegradualandSBMCsoftenstartfromalowinitialcapacitylevel.Second,communityownershipandresourcemobilisationmaybepromotedasasubstitutefor,ratherthanacomplementto,astategovernmentthatisaccountableforprovidingandresourcingaqualityeducationsystem.

• EffectivemonitoringishighlightedbyseveralKIsaskeytothesuccessoftheproject.ThiscorrespondstobothmonitoringandoversightofschoolactivitiesbytheSBMC,headteacherorlocal/stategovernmentofficers,aswellasmonitoringofGEP3projectimplementation.SomeKIslinkthistoaneedforinterventionstohaveastrongerresultorientation:holdingpeopletoaccountforresults.KIsacknowledgethatthemonitoringsystemsareweak,althoughinvestmentshavebeenmadeinthepasttostrengthenmonitoringandqualitycontrol.10Therefore,GEP3rightlyemphasisesmonitoringinitsToC.However,effectivemonitoringrequiresacommonunderstandingoftheobjectives,rolesandresponsibilities,andapproachofthemonitoring,andsufficientresourcesforimplementation.

• InthecaseofIQSs,additionalkeyinfluencingfactorsapply.KIsworkinginthefieldofintegratedQur’aniceducationarguethatthebuy-inoftheproprietorforintegrationandprojectinterventionisvitalinorderforinterventionstoleadtoresults.However,thebaselinefindingsoftheIQSsupport(IQSS)evaluationindicatethatschoolleadershipinIQSsiscomplex.ThemanagementoftheIQSisnotanindividualresponsibilitywithclearlydefinedanddelegatedresponsibilities.Therefore,itwillbeimportanttotakeintoaccountthevoicesofkeycommunity

9SPARCandSAVI(2015),‘StateoftheStates–PoliticalEconomy,ProspectsforReformandFiscalConditions’,unpublishedPowerPointpresentation,13July2015.10InSokoto,schoolqualityassuranceofficershavebeenscreenedandtrainedwithsupportoftheNorthernEducationInitiativeproject.InZamfara,accordingtoaKI,theSUBEBrecentlysuppliedqualityassuranceofficerswithnewmotorcyclesandfuelallowances.InBauchi,theSUBEBexpandeditszonalofficesfromthreeto10toensuretherearemorequalityassuranceunitsclosertotheschools.


©EDOREN 8

figures,giventheinfluencetheymayhaveonthesuccessofintegration.Furthermore,theselectionofIQSsthatactuallyandcontinuouslyofferformaleducationtogirlsisaprerequisiteinordertoimprovegirls’education.WhilethisisforeseenintheeligibilitycriteriausedbyGEP3,11thebaselinedatacollectionindicatesthatinpracticenotallGEP3IQSsareintegratedorhavegirlsenrolled.ThequalitativeresearchalsohighlightsthatsomeIQSsmaystillbemobile(whichneedsconfirmationatmidline).GiventhatgirlsareunlikelytofollowtheMallamwhentheschoolmoves,theireducationwillpotentiallybeinterrupted.12

Furthermore,theprojectoutcomesmaybeinfluencedbyawidevarietyofcontextual,non-project-relatedfactorsspecifictoeachstate.Thepresenceofotherprojectscontributingtothesameoutcomesisonesuchfactor.Theseprojectsvaryfromstatetostate.Forexample,theNigerianPartnershipforEducationProject(NIPEP)aimstosupportsimilarinterventionsasGEP3inKatsinaandSokoto.Thiscancreatesynergiesbetweenprojects,butmayalsoconflateoutcomes.Similarly,theTeacherDevelopmentProgramme(TDP)issupportingin-serviceteachertraininginKatsina,ZamfaraandNiger.13TotheextentthatTDPprovidessupporttoteachersfromGEP3focusschoolssynergiescanbecreated,whileatthesametimeitwillbedifficulttoattributechangesineducationoutcomestoGEP3alone.

GovernmentpoliciesandstructuresmayalsoleadtovariationinGEP3outcomesfromstatetostate.Forexample,NigerStatehashadapolicyofimplementingWardDevelopmentProjects,todecentraliseinvestmentinbasicsocialservices–including,amongotherthings,ineducation.14Intermsofgovernmentstructures,statesmayhavespecificMDAstosupportgirls’educationortheintegrationofbasiceducationintheQur’anicschools.Forexample,theZamfaragovernmenthasestablishedtheFemaleEducationBoard,whichismanagingboardingschoolstargetedatgirls.ThepreviousgovernmentinBauchicreatedtheDirectorateofTsangayaEducation,whichprovidedallowancestoproprietors.SuchinstitutionaldifferencesmayinfluenceGEP3’sresultsindifferentways.SuchinstitutionsmaycomplementGEP3interventionandsupportGEP3achievingitsresults.Thepaymentofproprietorallowancesisonesuchexample.Ontheotherhand,totheextentthattheseinstitutionscompeteforthesamestateresourcestheymayinfluencethecapacityofGEP3’sdirectgovernmentpartnerstoimplementinterventions.ThenationalandstateelectionsthatwereheldinMarchandApril2015respectivelymayaltergovernmentpoliciesandstructures.Atthetimeoftheinterviews,itwasyettobeseenwhichGEP3stateswouldchangetheireducationpolicy,structuresorbudgetprioritisationasaresultoftheelections.

Finally,GEP3’soutcomeswillbeaffectedbyabroadersetofsocioeconomic,demographic,cultural

andpoliticalfactors.InsecurityandconflictwerementionedinBauchi,ZamfaraandKatsinaashavingaffectedschooling.InBauchi,thishasresultedinthestateeducationsystemhavingtocopewithaninflowofinternallydisplacedpeople.InZamfaraandKatsina,truancyamongbothpupilsandfemaleteachersoccursduetofearsregardingsecurity,especiallyintheoutskirtsofthetwostates.KIsinNigerpointedoutthat,giventhesizeofNigerState,15improvingschoolinfrastructureandaccessibilityisproportionallymoredifficultcomparedtosmallerstates.

11IQSsareeligibleforGEP3supportduringthepilotphasewhen40%ofpupilsaregirls.12ThetermMallamcanhavedifferentinterpretations.WeconsidertheMallambeingtheheadofthereligiousschool,responsibleforthereligiouseducation.13Initsfirstphase,from2014,TDPoperatesinJigawa,ZamfaraandKatsina.ThereareplanstoextendittoKano,KadunaandNigerstatesinlate2016.14Thepolicyawards100,000NigerianNaira(NGN)permonthforeducationdevelopmentatwardlevel.15NigerStateisthelargeststateinNigeriaintermsofsurfacearea.Onaverage,Nigerhasfourprimaryschoolsper100km2,comparedtonineper100km2inKatsina.


©EDOREN 9

2.3.2 Output1:Increasedenrolmentandretentionofgirlsinbasiceducation

UnderOutput1GEP3seekstoincreasegirls’enrolmentandretentioninbasiceducationbyaddressingbarrierswhichaffectdemand.Threeinterventionsthatcontributetothisresultaresupported(seedetailsbelow):

• enrolmentdrives:annualcampaignstoidentifyout-of-schoolgirlsandtoencourageschoolenrolmentandretention;

• acashtransferprogramme:unconditionalcashtransfersprovidedtofemalecaregiversofallgirlsbetweentheagesofsixand15livinginselectedprimaryschoolcatchmentcommunities;

• G4Ggroups:girls’groupsestablishedinprimaryschoolsandIQSsundertheguidanceofamatronselectedfromtheschool’sMAorfromamongthefemaleteachers.

Inaddition,theprojectintendstoinfluencesupplyinordertomeetincreaseddemand,withthesupportofhigh-levelpoliticalengagement.

Whilethisoutputisformulatedtocoverbasiceducation,theemphasisoftheinterventionshasshiftedtoprimaryschooleducationaftertheGEP3redesign.Enrolmentdrivesdocoverthewholerangeofbasiceducation,includingearlychildcaredevelopmentandeducation(ECCDE)andjuniorsecondaryschool(JSS)education,butthecashtransferprogramme—eventhoughitcoversgirlsbeyondprimaryschoolage—isorganisedaroundaselectionofprimaryschools.TheG4Ggroupswillsimilarlybeoperationalisedinprimaryschools.Hence,resultsaremorelikelytobeachievedin

primaryeducationratherthanacrosstheentirebasiceducationcycle.

2.3.2.1 Enrolmentdrives

Enrolmentdrivecampaignsareannualeventsthathavebroadacceptanceandsupportamongstakeholders.TheyaremeanttobeledbySBMCsandMAs,withthesupportofcommunityleadersandreligiousleaders.AllKIsperceivethesedrivesasaneffectivemechanismforincreasing

enrolment,byraisingparentalawarenessabouteducation.Thisappreciationfollowsfromtheirmultidimensionalnatureintermsofinvolvingawiderangeofstakeholdersatdifferentlevels(state,localgovernmentandcommunity),usingmultiplecommunicationstrategies,andmultiplemessagesthatcanspeaktoadiverseaudience.Whilethecampaignscoverbasiceducationingeneral,theinclusionofgirlsandotherdisadvantagedgroupsisemphasised.16Thefactthattheenrolmentdrivesareabletopenetratethecommunities,withtheactiveinvolvementofthecommunitiesthemselves,ishighlightedasakeyfeatureinregardtotheireffectiveness.

Theinterventionisbasedontheassumptionthatdemandforeducationistheresultofpeople’sknowledgeaboutthebenefitsofeducation,andtheextenttowhichtheyhavetherightattitudesandinterestswithregardstoeducation.Theexpectationisthattheprovisionofinformationwillinfluencetheirawareness,attitudesandbehaviour.InthediversecontextoftheGEP3states,thelevelofawarenessof,andattitudestowards,girls’educationvaries.SeveralKIspointoutthattheculturalacceptanceof‘Western’educationandtheappreciationofthevalueofeducationhaveimprovedconsiderably.Theseattitudesvaryregionallyandbetweenruralandurbanareas.17Therefore,asawarenessandattitudeschange,enrolmentdriveswillneedtobecomemore

16 KIs pointed out that last year’s enrolment drive emphasised the enrolment of children from ethnic minorities andphysicallydisadvantagedchildren.17 In Bauchi, KIs highlighted the difference between the Northern, Central and Southern region. Acceptance of, and aninterestin,formaleducationishigherintheSouthernregion,whichtraditionallyhasbeeninfluencedbyChristianmissions,andtheCentralregional,whichisinfluencedbytheBauchiurbancentre.


©EDOREN 10

contextualised–targetingtherightmessagetotherightpeople.SomeKIsindicatedthatthisisalreadytakingplace,intheformofintensifyingcampaignsinareasoflowenrolment.

Changesinawarenessandattitudesdonotnecessarilyresultinchangesinbehaviour.Inparticular,povertyorotherconstraintsmayinhibitactualchangesinbehaviour.GEP3acknowledgesthisinitsdesignbypilotingacashtransferprogrammetotacklethefinancialbarrierposedbypoverty.Asawarenessandattitudeschange,thefinancialbarriermaybecomerelativelymoreimportantand

theneedtoexpandaninterventiontargetingthisbarriermayincrease.However,weassessthattheimplementationofacashtransferprogrammeacrossGEP3statesatscaleisunlikely(seebelow).

KIsacknowledgethattheobjectiveoftheenrolmentdrivesisnotonlytoincreaseenrolmentbutalsotoretaingirlsinschool,whichisconsideredtobethemoredifficultobjectivetoachieve.Retentionwillbeinfluenced,amongotherthings,byparents’andgirls’perceptionsregardingthequalityofeducation,whichwilldependonschoolsbeingabletomeettheincreaseddemandwithadequate

facilitiesandteachers.TheGEP3ToChasidentifiedasakeyassumptionthatgovernmentcansupplyprimaryschoolandteacherstomeetincreasingdemand.InallstatesKIsperceivedthatincreasedenrolmentisnotsufficientlyaccommodatedintermsofkeyresourcessuchasteachersandclassrooms,althoughsomeseethisasaproblemthatismainlymaterialisinginurbanareas.

‘However,aweaknessofenrolmentdriveisthattheyoverenrol.ThisistheoneareawheresomeSBMCmemberscomplain.Therearemorechildrenenrolledthanthefacilities,infrastructures,materialsorteachersprovided.Insomeschoolsonlyoneteacher,theheadteacherorArabicteacher.Theremaybeover200pupilswithjusttwoclasses;theinfrastructureisjustnotthere.Parentsarecomplaining:youasksustosendchildrentoschoolbutwherearetheteachers,wherearetheclasses?’(Non-governmentKI,Zamfara)

‘Ineducationallyadvancedcommunitiestheschoolsbecomeoverstretched.(…)InBauchinowtheclassesareoverstretched;thereisaclassthathas150pupils,180even200pupilsinoneclassbothintheprimaryandjuniorsecondary.InsideBauchi,Toro,Azare,andothersthatareintheurbanareasmostoftheschoolsareoverstretched.’(GovernmentKI,Bauchi)

Thismaycreateabacklashinregardtoparents’andgirls’responses,aswellasinregardtoSBMCs’effortsinrelationtoenrolmentdrives.Themostseriouschallengeliesinsufficientteachersupplyinruralareasandinfrastructuresupplyinurbanareas.Furthermore,astheenrolmentdrivesexpandtheirreachonthedemandside—forexample,bytargetingECCDEandtheenrolmentofdisabledchildren—theriskofthesupplysidenotmeetinghigherdemandincreases.

Besidestheresponseonthesupplyside,KIshighlightedthefollowingotherkeyfactorsthatwillinfluencewhetherenrolmentdrivesachievetheirintendedresults:

• TheroleoftheSBMCsandMAsareconsideredcentraltothesuccessoftheenrolmentdrives.Theirroleisnotonlyseenintermsofchannellingthecampaignmessagesdirectlyintothecommunities,butalsointermsoffacilitatingsupply-sideresponsestotheincreaseddemandbymobilisingcommunityresources,signallingsupply-sidebottleneckstolocalorstategovernmenteducationofficialsand/orpotentiallyholdinggovernmenttoaccounttoresolvesuchbottlenecks.Aswillbediscussedbelow,themultiplerolesthatSBMCsareexpectedtotakeonmaynotbeinlinewiththeiremergingandgradualcapacitybuildingandempowermentprocess,particularlyinruralareas.Hence,theirfunctionalityandcapacityisakeyassumptionforsuccessfulenrolmentdrives.GEP3hasincludedSBMCcapacitybuildingaspartofitsdesignandToC,whichstrengthenstheplausibilityofthisassumptionbeingmet.Nonetheless,the


©EDOREN 11

assumptionwillrequireclosemonitoring,whichisforeseeninGEP3’sdesign,viatheSBMCeffectivenessmonitoring.

• KIsconfirmthepositivecontributionthattraditionalleadersandfemalerolemodelsmakeinregardtogirls’enrolment.Traditionalleadersareconsideredcredibleopinionleaderswhocanaddresssocio-culturalnorms,whilefemalerolemodelsareassumedtoemanatethebenefitsofgirls’education.KIsvalueGEP3-supportedactionsthatstrengthenthisfactor,suchasincorporatingtraditionalleadersinzonalmeetings,thecontributionoftheHigh-LevelWomenAdvocates(HiLWA)groupandtheexpansionoffemaleSBMC/MAmembersintheorientationmeetingoftheenrolmentdrives.

Intermsofcontextual,non-GEP3-relatedfactorsthatwillinfluencetheimpactoftheenrolmentdrives,thesecuritysituationisariskfactor.KIsindicatethatinBauchi,theBokoHaraminsurgencyloweredschoolenrolmentin2014insomeareaswhereattacksoccurred.Enrolmentisalsoaffectedbytheimmigrationofinternallydisplacedpersonsfromneighbouringstates.ThesecuritysituationwasalsoidentifiedasaninfluencingfactorinZamfaraandKatsina.Finally,itisimportanttorecognisethathighfertilityinnorthernNigeriawillincreasetheschoolagepopulationandsubsequentlymaycontributetoincreasedenrolment.

2.3.2.2 Cashtransferprogramme

TheunconditionalcashtransferprogrammethatisbeingpilotedinNigerandSokotoaimstoreducethepoverty-relatedbarrierthatispreventinggirls’enrolmentandretention.KIshaveacommon

understandingofthisobjective:theygenerallyperceivetheprogrammeasaneffectivestrategy

andindicatethathighinterestintheprogrammeinthetargetedcommunitiescanbecountedondespiteinitialmistrustamongsomeparentsabouttheintervention’sintentions,whichresultedinsomeeligiblegirlsnotbeingregistered.TheSokotoandNigerPIUsarguethatthehighcommunityinterestisalsodemonstratedbythedemandfortheprogrammethatisnowcomingfromnon-eligiblecommunities.

ThePIUsarewellinformedaboutthekeyfeaturesofthecashtransferprogramme,suchasthegeographicalcategoricaltargetingofthebeneficiariesanditsunconditionality.Theyareespeciallysupportiveoftheunconditionalnatureofthecashtransferbecauseconditionalitycouldunderminetheincome-smootheningeffectbydenyingtransferstothosewhofailtosatisfyconditions,suchasschoolattendance–potentiallyinthosetimeswhenthetransferisneededmost(forexample,whenthechildfallsill).Theeffectivenessoftheunconditionalnatureofthecashtransferis,however,contestedbyotherKIs,whofavoursomeformofconditionalitybasedonthegirlscomingtoschool,ontheassumptionthatthiswillincentiviseattendanceasfamilieswouldlosethecashtransferwhenacertainattendancethresholdisnotmet.WhiletheKIsconfirmthatthecashtransferfocusesonalleviatingthepovertybarriertogirls’education,theyalsoacknowledgetheimportanceof

sensitisationandtheroleoftheSBMCsandheadteachersinfollowinguponattendance.

‘Itisthesensitisationthatmatters,sothatpeopleunderstandthateventhoughthereisnoconditiontheysendtheirchildrentoschool(…).TheinvolvementofSBMCandtraditionalleadersinthecashtransferprogrammehelpedalotinthesensitisationofthecommunities.’(Non-governmentKI,Sokoto)


©EDOREN 12

Furthermore,thePIUsclaimthatthenon-paymentofseveraltransfersduringthefirstyearofimplementation18didnotsubstantiallyaffectschoolparticipation,which,ifthisholdstrue,indicatesthatfactorsotherthanincomemaybeinfluencingtheparticipationdecisionandmay,therefore,influencetheachievementofoutcomes.However,nodatawereprovidedtosupportthisclaimandUNICEFhasindicatedthatthePIUshavenotdocumentedenrolmentandschoolattendanceofcashtransferbeneficiaries.

KIsrecognisethesynergybetweenthecashtransferprogrammeandtheenrolmentdrives.Thelatterreinforcesthesensitisationaroundthecashtransferprogramme.InSokoto,thecashtransferprogrammewaslaunchedintheweekoftheenrolmentdrive.Inaddition,theon-the-spotschoolenrolmentatthetimeofthecashtransferpaymentisalsomentionedasaneffectivefeaturecontributingtogirls’enrolment.

TheKIshighlightedthefollowingkeyfactorsthatinfluencetheoutcomesofthecashtransferprogramme:

• Similartoenrolmentdrives,thecommitmentandcapacityofSBMCmemberstosupporteffectivesensitisationandattendancemonitoringarehighlightedasimportantcontributionstothesuccessofthecashtransferprogramme.Inaddition,SBMCsareconsideredtoprovidesupply-sideinputstoincreasedenrolmentduetothecashtransferprogramme.Inturn,thecashtransferprogrammemayinfluencetheroleoftheSBMCsbystrengtheningtheirfunctionintheschoolandcommunity(withanassociatedriskofunderminingitiftheycannottransparentlyandadequatelyimplementthatrole).19

• Again,similartoenrolmentdrives,supply-sidefactorsliketheavailabilityandattendanceofteachersareconsideredkeytomotivatingparentstosendtheirchildrentoschool.

• Communitycommunicationandsensitisationregardingtheeducationalobjectiveofthecashtransferisalsoseenasessential,giventheunconditionaldesign.20Thismaychangehowparentsperceiveeducation.However,thisislikelytobeaffectedbyparents’perceptionsofthequalityofsupply-sidefactorsandthecredibilityofthepeopleinvolvedinthesensitisation.ThelatterislinkedagaintotheinvolvementofSBMCsandtraditionalleaders.

Twointerrelated,centralinfluencingfactorsthatGEP3haslessinfluenceonarethebroaderlegalandpolicyframeworkwithregardstosocialprotectioninGEP3states,aswellaspoliticalleadershipchangesandsubsequentpolicyprioritychanges.Currently,nosocialprotectionlegalframeworkisinplaceinGEP3statesthatcanprovidecontinuityandstabilitytoacashtransferprogramme.Thismeansthatthepolicypurposeofacashtransferprogrammemayshiftbecauseofchangesinstate-levelleadershiporfundingprioritiesatfederalanddonorlevel.AccordingtoKIs,thediscontinuationoftheeducationcashtransferpilotsinBauchiandKatsinawasinfluencedbychangesinpoliticalleadershipandare-prioritisationofthepurposeofcashtransfers:topovertyalleviationratherthaneducation.InNigerandSokoto,thefirstyearofoperationsoftheGEP3cashtransferpilothasbeenaffectedbytheelectionsandthetransitioninpoliticalleadership.Asrecognisedby18 At the time of the interview, only one payment out of four was transferred in Sokoto, while two payments wereoutstandinginNiger.19TherolesandresponsibilitiesoftheSBMCintheGEP3cashtransferprogrammeinclude:1)conductingmobilisationandsensitisationactivities;2)assistinghouseholdswithappeals,complaintsandchangemanagementprocedures;3)communicatingdetailsofthepaymentdaytobeneficiaryhouseholds;4)beingpresentonthepaymentdaystoassistcommunication,identityverificationandsubmissionofappeals,complaintsandchangemanagementrequests;and5)ifinstructed,conductinghomevisitsofgirlsthatarenotenrolledornotattendingregularly(GEP3-CTPOperationalManual).20Benhassineetal.(2015)foundevidence,basedonarandomisedcontroltrialinMorocco,thata‘labelled’cashtransfer,whichmakestheeducationobjectiveofthecashtransferclear,isenoughtoimproveenrolment,withnoneedforexplicitconditions.


©EDOREN 13

theGEP3ToCandconfirmedbytheKIs,statefundingsupportforcashtransfersdependsonthepolicyprioritiesofthenewgovernors,thefederalgovernmentmechanismanddonorpriorities.

2.3.2.3 G4G

AccordingtotheGEP3ToC,G4Gsseektoempowerschoolgirlsandtoaddresstheirretentionbyprovidingthemwithaspaceinwhichtomeetandaddressissueswhichaffecttheirschoolparticipation.Thisisdoneundertheguidanceoffemaleteachers/matronswhocanactasrolemodels.TheG4GsaretobeestablishedundertheumbrellaoftheSBMC/CBMC,toensuresustainability,undertheassumptionthatSBMCs/CBMCstakeownershipofthegroupsandpromotetheirimplementationbeyondtheterminationofexternalsupport.

ItisplannedthattheG4Ginterventionwillbeimplementedin2016–2017,andsoitisyettobeoperationalised.Itwouldstrengthentheplausibilityoftheintervention’sToCtoclearlyspecifyandagreeontheintervention’sobjectives,itstargetpopulationandtheconceptualisationofgirls’

empowerment.Withregardtotheobjectives,atthetimeoftheKIIsitwasnotclearwhethertheinterventionaimstoachievetheretentionofin-schoolgirlsor/andthe(re)enrolmentofout-of-schoolgirls.TheGEP3ToCdocument(UNICEF2015a)presentsG4Gasaninterventionthatseekstoaddressgirls’retention,whiletheGEP3SBMCstrategypaper(UNICEF,2015g)introducesitspurposeasbeingtocreateaplatformfortheempowermentofgirlstoexercisetheirrighttoenrolandremaininschool.21Closelyrelatedisthespecificationofthetargetpopulation.Itneedstobeclearwhetherout-of-schoolgirlsareadirecttargetpopulationofG4G(andhenceinvitedtobemembersoftheG4G),anindirecttargetpopulation(reachedthroughin-schoolG4Gmembers),ornotatargetgroupatall.TheGEP3ToCdocumentandGEP3SBMCstrategypaperseemtosuggestthatout-of-schoolgirlsareatargetpopulation,indicatingthatG4Gwill‘developanactionplanforcommunitysensitisationand[promote]theenrolmentofout-of-schoolgirls’andthatmembersofG4Ggroupswill‘serveaspeermobilisers,identifyingtheirpeerswhoareneverenrolledordroppedoutofschool’.22Furthermore,theageofthetargetpopulationwillrequireconsiderationasbarrierstoeducationvaryaccordingtothegirls’lifecycle.

Finally,girls’empowermentisamultidimensionalconcept,rangingfrommoretangibleconditionslikeaccesstoresourcestomoreintangible,subjectivevalues,feelingsandperceptions–bothattheindividualandthestructurallevel.Insum,thefocus,scope,conceptualfoundationandstrategyoftheinterventionneedstobeclarifiedtoenhancetheplausibilityofitsToC.

SomefactorsthatKIsmentionedasinfluencingthelikelysuccessofG4Gare:

• TheschoolneedstohavefemaleteachersorMAmemberstoactasmatronsandrolemodelsforthegirls.KIsconfirmthatthedeploymentoffemaleteacherstoruralschoolsischallenging.TheStrengtheningWomenandGirls’Spaces(SWAGS)projectdealtwiththischallengebyliaisingwiththeEducationSecretariesatthelocalgovernmentlevelinordertotransferfemaleteacherstoalimitednumberofpilotschools.However,itseemslessfeasibletoimplementthisprocessatscaleacrossallruralschools.

• Lowqualityteachingandunqualifiedteachersattheschoolmaypotentiallyaffectgirls’retention.EmpiricaltestingwouldberequiredtodeterminetheextenttowhichaG4Gcouldcounterthenegativeeffectonretentionofalowqualityteachingenvironment.

21Aspartofthereviewofthisdocument,UNICEFspecifiedthattheobjectiveoftheG4Gistheretentionofgirlsenrolledinprimaryschools.22Aspartofthereviewofthisdocument,UNICEFindicatedthatout-of-schoolgirlsarenotatargetpopulationoftheintervention.


©EDOREN 14

• HeadteachersneedtobeconvincedabouttheprogrammesothattheirleadershipcanfacilitatethefunctioningoftheG4G.Equally,thebuy-inby,andfunctioningof,theSBMC/MA,undertheumbrellaofwhichtheG4Gwillbeestablished,isrequired.Parents–particularlyofout-of-schoolgirls–shouldalsobeconvincedaboutthisapproach.

• AKIquestionedhowgirlswillbemotivatedtoparticipateinG4Gmeetings.Vocationalactivitiesaresuggestedasameanstomotivategirls.However,totheextentthatsuchvocationalactivitiesrequireresourcestooperate,resourcemobilisationwillneedtobeincorporatedaspartoftheG4Gactivities,tomakethemsustainable.23

Animportantcontextualfactorthatwillinfluencegirls’participationinG4Gmeetingsistheaccessibilityandsecurityoftheschool.ItcanbeexpectedthattheoutcomesofG4Gwillvarybetweenmoreruralandurbanareas,giventhedifferencesinschoolaccessibility.

2.3.3 Output2:Improvedcapacityofteacherstodelivereffectivelearningforgirls

InterventionsunderGEP3’sOutput2seektoensurethat,oncegirlsareenrolled,improvedteachingqualityandschoolmanagementresultinbetterlearningoutcomes.Particularemphasisisputonearlygradeinstructionandlearning,becauseearlyacquisitionofliteracyandnumeracyskillsisconsideredfoundationalforfuturelearning.Thefollowinginterventionsareimplementedwiththeaimofachievingthisoutcome:

• Teachercapacitydevelopment:trainingandmentoringofprimaryschoolandIQSteachers,with,atthecore,a1.5-yearcycleofmonthlycluster-leveltraining/mentoringmeetingsinitiatedbyfive-dayinductionworkshop;

• Headteachercapacitydevelopment:cycleofthree-daytrainingsessionspertermoveratwo-yearperiodforprimaryschoolsandIQSheadteachers;

• Earlylearninginterventions:earlygradeteacherin-serviceprofessionaldevelopmentinprimaryschoolsandIQSs,includingcluster-leveltraining,weeklyschool-basedpeer-mentoring,on-sitemonthlymonitoring,headteachertraining,andteachingandlearningmaterialdistribution.

TheoveralllogicandobjectivesofthisoutputarewidelyunderstoodandsupportedbyKIs.Lowlearningoutcomesandinadequateteachingqualityarecommonlyrecognisedaskeyproblems.AwiderangeofKIsargued,inlinewithGEP3’sToC,thattacklingthequalityofteachingandteachersiscentraltoimprovinglearningoutcomes,whileacknowledgingthatthisneedstobesupportedbyanimprovementinthebroaderlearningenvironment.Thecapacityoftheheadteacherisrecognisedasanimportantsupportingfactor.Furthermore,KIsvalidatedtheimportanceofearlygradelearningasameansofacquiringbasicliteracyandnumeracy,andpavingthewayforsuccessfullearningoutcomesinlatergrades.

Beforepresentinganintervention-leveldiscussionbelow,itisimportanttohighlightthatingeneralmonitoringandsupervisionareconsideredkeysupportingfactorstotranslatethecapacity

buildingofteachersandheadteachersintoschool-levelbehaviourchange,andultimately

improvedlearningoutcomes.ThisisrecognisedinGEP3’sToC,whichsupportsitsplausibility.SomeKIsexpressedtheirappreciationforGEP3’sincreasedemphasisontheissuesinceitsredesign.

‘Weneedbettermonitoring.Whenyoutrainfacilitators,givethemsupportandmotivatethem,itisbettertosuperviseormonitorthemtoseeifworkisbeingdoneornot.Monitoringand

23ResourcemobilisationisindicatedintheTransformingEducationforGirlsinNigeria(TEGIN)Girls’ClubManualascontributingtotheeffectiverunningandsustainabilityofagirls’club.


©EDOREN 15

supervisionisveryimportant.Sometimesmonitoringhappens;especiallythosethatarepaidNGN7,500,theysupervise.’(Non-governmentKI,Sokoto)

‘GEP3introducedmonitoring.Aftertrainingtheteachersaremonitoredintheirschools;howaretheyimpartingtheknowledgethattheyacquiredduringtraining?Thiswillhelpalottogetthelearningoutcomes.Whathappensintrainingisdifferentfromwhathappensintheclassroom.’(GovernmentKI,Katsina)

TheplausibilityofGEP3’sToCwithrespecttomonitoringcanbestrengthenedbybuildingacommon

understandingamongstakeholdersabouttheobjectivesofthedifferenttypesofmonitoring

activities.DifferenttypesofmonitoringandsupervisionarepresentinGEP3’sToCindifferentways–eachwiththeirownlogicandobjectives.Thesetypesofmonitoringandsupervisionarecurrentlynotwelldistinguished,commonlyunderstoodorspelledout.SeveralKIspointouttheimportanceofSBMCmembersmonitoringteachers,butKIsdonothaveacommonunderstandingaboutthefocusandscopeofthismonitoring.Forexample,itremainsambiguouswhetherSBMCmembersaremandatedtomonitorteachingpracticeandoutcomes,ratherthantofocusonmonitoringteacherattendance.24Headteachermonitoringiscommonlyunderstoodasincludingpedagogicalsupervisionandmentoring,butitisnotclearhowthiswillworkforIQSproprietorswhomaynothaveapedagogicalbackgroundorinterest.Trainersofteachersandheadteachersaremeanttoconductfollow-upmonitoringthatissupposedtoincludeamentoringelement.However,howthiswillbeoperationalisedremainsvague.Also,localandstategovernmenteducationstaffhaveamonitoringfunction,whichisgenerallyunder-resourced.Howtheirmonitoringfunctionalignswiththebefore-mentionedmonitoringrolesisalsounclear.

Likemanyinterventions,monitoringandsupervisioncanbeeffectivewhentherightconditionsaremet.Forexample,KIsconfirmthatthemonitoringpotentialofaCBMCiscircumscribedbythewillingnessoftheIQSproprietortosupportsuchmonitoring.Thesupervisoryleadershipofaheadteacherisconditionedbyhis/herowncapability,credibilityandpowertoact.ManyKIshavehighexpectationsofthemonitoringroleofSBMCsbutalsoacknowledgetheirstillemergingcapacity.Suchpotentialconstraintsneedgreaterrecognitionandreview,inordertobeabletoadaptschoolmonitoringtotheactualconditionsintheschools.

2.3.3.1 Teachercapacitydevelopment

Asmentionedabove,thereisawideconsensusamongtheKIsthatdevelopingteachercapacityin

primaryschoolsandIQSsishighlyrelevant,andthisiswellacceptedasanobjectiveoftheproject.Theimportanceofemphasisinggender-sensitiveteachingpracticesaspartofGEP3,inordertoimproveoutcomesforgirls,isacknowledged,althoughoneKIhighlightedthatitismorerelevantinupperprimarygrades,whengenderstereotypingbecomesmorepronounced.OtherelementsthatseveralKIsemphasiseasameanstoimprovinglearningoutcomesparticularlyforgirlsare:1)increasingtheirinterestinschoolingbypromotingthebenefitsofeducationviafemaleteacherrolemodels,careertalksorvocationalskillstraining;and2)improvingthegirl-friendlinessoftheschoolenvironmentbyensuringappropriatewaterandsanitationandsecurity.

Inordertofullyassesshowplausibleitisthatthisteachercapacitydevelopmentwillleadtoresults,itisimportanttoconsidertheattributesofitsapproachandintendedoutcomes,suchaswhatcapacitiesneedtobedeveloped,whosecapacitiestheseareandhowandwhenthesewillbe24AccordingtoUNICEF,SBMCsaremeanttoonlymonitorteacherattendance,notpedagogicalpractices,whichistheresponsibilityofheadteachersandteachertrainers.However,theGEP3ToCdocument(UNICEF2015a)indicatesthat‘they[SBMCs]willalsoexerciseoversightovertheperformanceoftheschoolandthequalityoftheeducationprovided(…)’.


©EDOREN 16

developed.TheGEP3StrategyPaperonSchool-basedTeacherDevelopment(UNICEF2015h)outlinessomeoftheseelements,suchasafocusonpedagogicalandcontentknowledge.However,KIsaregenerallynotwellawareofsuchattributesoftheredesignedteachercapacitydevelopment

approach.EvengovernmentstakeholdersinvolvedintheIQSfacilitatortrainingscheduledforearlySeptember2015hadonlyavagueunderstandingofthefeaturesofthistraining.GovernmentKIsinBauchiinvolvedintheschool-basedteacherdevelopmentdidnotknowwhattheredesignedschool-basedteacherdevelopmentapproachwouldlooklike.

WhiletheGEP3school-basedteacherdevelopmentstrategypaperandGEP3ToCpostulateasafirstassumptionthatwhereteacherdevelopmentiswelltargetedandrelevantteachers’performancewillimprove,whatthistargetingentailsremainsunclearinthestrategydocuments.TargetingspecificgroupsofteachersrarelycameupduringtheKIIsascriticalinordertoimproveteachingorlearning,althoughonceprobedKIsdosupporttargetingearlygradeteachers.Thismaybeduetothefactthatthecapacityneedsarecommontomostteachers,or,asoneKIpointedout,thatteachertraininghastraditionallybeenaboutallteachers‘fairly’benefittingfromtraining,ratherthanimplementingtrainingwithapurposeandresultinmind,suchastrainingthoseteachersthatmaycontributemosttoimprovingpupilslearningoutcomes.SeveralKIsacknowledgethatinthepast,theselectionoftrainingparticipantshasresultedinunqualifiedpeoplebeinginvitedandsuchselectionlackedmonitoring.25Otherspointoutthatthe‘right’or‘trainable’teachersneedtobetrainedbutwhattheattributesofsuchrightandtrainableteachersareremainsundefined.Therefore,itwouldincreasetheplausibilityoftheToCasregardsachievingitsoutcomesiftheapproachtotargetingwaselaboratedon,intermsofwhichteachersitwouldbebesttotrainandwithwhatresultinmind.

Ingeneral,theKIsinterviewedonthetopicfavouredachild-centred,activelearningapproachtoteaching,withafocusonbasicliteracyandnumeracy.Thecommunicationandlanguageskillsofteachersarealsomentionedasanelementtoincludeinteachertraining.Monitoringand

supervisoryfollow-upvisitsareconsideredkeytoachievingresults.

Aclusterapproachforfollow-upteacherinteractionisvaluedasameansforcontinuouslearningandsharingofexperiences.However,KIsquestionwhetherresourceswillbeavailablefortraineesandmentorstoattendregularmeetings,andwhetherthementorswillbesufficientlypreparedtoeffectivelyfacilitatethemeetings.Aone-offtrainingapproachisconsideredtobeinsufficienttoachievechangesinteachingpractices,whichtheGEP3teacherdevelopmentapproachhastakenintoaccountbyprogrammingfollow-upmentoring.

TheGEP3ToCindicatesthatIQSfacilitatorcapacitydevelopmentwillbeadaptedtoteachingandlearningintheIQSenvironment.TherearelikelytobelargevariationsinthebackgroundsoftheIQSfacilitatorsandtheIQSsinwhichtheyareteaching.Somefacilitatorsareteachersintheformaleducationsystemwhileothersarecommunitymemberswithlittleinthewayofteachingqualifications.26Teachinghours,availabilityofmaterials,teachingfacilities,pupilcharacteristics,27typeofIQS28andproprietorsupportarelikelytovaryconsiderably.Therefore,thecapacitybuilding

25ThiswasconfirmedbytheIQSSbaselinesurvey,whichfoundthataroundone-fifthoftrainedfacilitatorsthatwerefoundinIQSseligibleforthesurveywerenotteachingattheschool.26 InBauchi,KIs indicated that facilitatorsworking inSUBEB-managed IQSsaregenerally selected fromteachersunderaLGEA contract. In Sokoto, facilitators are identified from the IQS community or neighbouring community, with SAMEcarryingoutanexaminationofthefacilitatorpresentedbytheproprietor.27ReadingandAccessResearchActivity(RARA)researchamongIQSsinBauchiandSokotohasfoundthatIQSpupilsbelongtoawiderangeofagecategories,includingadultlearners.28Qur’anicschoolsvaryconsiderablyintermsoftheQur’aniceducationprovided,thetypeofpupilstheyattract,andhowthepupilsrelatetotheirtraditionalreligiousteacher,themallam.Forexample,IQSsmaybeboardingschoolsthatattract


©EDOREN 17

approachdoesnotonlyneedtobeadaptedtotheIQScontext,butalsoneedstotakeintoaccount

largevariationsinthatcontext.

‘Thepedagogy,approach,andthemethodaredifferentinnon-formalcomparedtoformal.UNICEFtrainingusedtojoinfacilitatorandteachertraining,butinrealityitisdifferent.Thesefacilitatorswouldjustgo,hearandlearn,buttheycannotgototheirlevelanduseitbecausethestrategyisdifferenttotally.’(GovernmentKI,Katsina)

TheGEP3IQSstrategypaperjustifiablyemphasisestheimportanceofmentoringinthecapacitybuildingprocess.Thismaymakeitpossiblefortheneededtailoredapproachtobeadopted.However,KIsacknowledgethattheeffectivenessofsuchmentoringassumesadequateresourcesformentormeetingstobeheld,sufficientmentorcapacityandfacilitatorsbeingsufficientlymotivatedtobementored.Asmentionedabove,italsoneedstobeclarifiedwhatsuchmentoringwillactuallyentail,andwhowillbeinvolvedandinwhatrespect.

SomeotherkeyfactorshighlightedbyKIsasaffectingtheresultsofteachercapacitydevelopmentonteachingandlearning,areasfollows:

• Theinitialskillsandknowledgeofteachersarelikelytoaffecttowhatextenttheteachersareabletotranslategainedknowledgeandskillsintopractice.SeveralKIsclaimthatsometeacherswhoareselectedfortrainingare‘untrainable’becausetheydonothavetheminimumunderstandingneededtocomprehendthetraining.ThebaselinefindingsinbothprimaryschoolsaswellasIQSsdemonstratethatteachershaveverylimitedknowledgeandskillsinthemajorityoftheareasrequiredtofunctioneffectivelyasateacher.Inparticular,teachersdisplaylowcompetencyinHausawritingandcomprehension.TheGEP3ToCacknowledgesthistosomeextent,bystatingthattheselectionofteacherswill,amongotherthings,bebasedonaconceptualunderstandingofteaching.Hence,teachertrainingpedagogyandmaterialswillneedtobeappropriateforverylowinitialteacherskillsandknowledgelevels.

• TeachercommitmentandmotivationisanotherinfluencingfactorhighlightedbyseveralKIs,whichislinkedtoteacherattendance.Lowcommitmentandmotivationisassociatedwithlowandirregularremuneration,poorlivingandsocialamenitiesinruralareas,andapoorteachingenvironment.AccordingtoKIs,primaryteachersarenotalwaysremuneratedontimeinallGEP3states29andtheirsalaryisconsideredtoolowtoincentivisedeploymenttoruralareaswhereamenitiesandschoolfacilitiesarepoor.Furthermore,severalKIspointedtoteacherappointmentanddeploymentbeingpoliticised,whichalsoaffectscommitment.

• RemunerationappearstobeparticularlyproblematicforIQSfacilitators.IQSfacilitatorsaregenerallypaidamonthlyallowance,whichisnotonaparwiththestandardofNGN7,500permonthinanyoftheGEP3states.30TheallowanceseemstovarybetweenNGN3,000andNGN5,000,andtheactualpaymentoftheallowancedependsontheavailabilityoffunding.ExperiencefromtheEducationSectorSupportProgrammeinNigeria(ESSPIN)indicatesthatthenon-paymentofallowanceseventuallyresultsinfacilitatorsstoppingteaching.ThebaselinefindingsoftheIQSSevaluationconfirmthataverysmallshareoffacilitatorsarepaid,witha

mostlyitinerantboysandthatfocusonrecitationoftheQur’an,whileothersmayoperateasadayschooltargetingboysandgirlsfromthecommunity.29 Informationabouttheregularityof thepaymentof teachers’pay ismixedandseeminglycontradictory. InBauchi, thepayment of civil servantswas not regular in 2015. In Katsina and Zamfara, a SPARC/SAVI political economy assessmentdatedJuly2015indicatesthatthegovernmentowesuptosixmonths’ofsalariestoworkers.However,KIsinthesestatesindicatethatnosalariesareowedtoteachers.30 It isnotclearwherethisstandardcomesfrom.OneKI indicatedthatthis istheamountrecommendedintheNationalBenchmark,buttheNationalBenchmarkforIntegratingBasicEducationintoQur’anicSchoolsinNigeriarecommendsNGN10,000.


©EDOREN 18

positiveassociationbetweenreceivingremunerationandteachermotivation.TheGEP3ToCassumptionthatgovernmentwillpayteachersregularlyandappropriatelytosupportmotivationinregardtoeffectiveperformanceisthereforeunlikelytoholdinthecaseofIQSs.

• Teacherattendanceandmotivationis,however,alsolinkedtosupervision,monitoringand

accountability,which,accordingtosomeKIs,isamoresignificantfactorinfluencingteacherbehaviourthanremuneration.TheseKIsmakethecomparisonwithprivateschoolsinGEP3states,whereteacherpayisgenerallylowerthaninpublicschoolsbutteachingandlearningoutcomesareperceivedtobebetterbecauseteachersareheldtoaccountinrelationtodeliveringeffectiveteaching,andareadequatelysupervised.31ImprovementofteachermonitoringandsupervisionisincorporatedwithinGEP3’sToCviaimprovementofheadteachercapacityandSBMCgovernance.However,asdiscussedabove,thelikelihoodofitactingasastrongsupportivefactorinGEP3’sToCcanbestrengthenedbyclarifyingitsobjectivesandrecognisingitlimitationswhencertainconditionsarenotinplace.ThequalitativeresearchfindingsoftheIQSSevaluationsuggeststhataccountabilityrelationshipsareweakenedinthecaseofIQSfacilitatorsbecauseofalackofremuneration.

• Accesstoteachingandlearningmaterialsandtheschoolenvironmentarelikelytoaffecttheextenttowhichimprovedteacherknowledgeandskillscantranslateintoeffectiveteachingandlearningoutcomes.MostofthesefactorsareincorporatedintoGEP3’sdesignthroughdifferentinterventions.Itisnotensured,though,thatthesedifferentinteractingfactorssupporteachotheratthesametimeattheindividualschoollevel.

• Pupilattendanceandmotivationareinfluencingfactorsinregardtoimprovinglearningoutcomes.32Output1interventionsaremeanttocontributetogirls’schoolattendance.Eventhoughthecashtransferprogrammeisunconditional,KIsbelievethatgirls’attendanceispositivelyinfluencedbecauseofSBMCs’follow-uponnon-attendanceandthesensitisationimplementedaspartoftheprogramme.Girls’motivationhasalreadybeenbrieflydiscussedabove.

• InIQSs,additionalfactorsneedtobeconsidered.Theattitudeoftheproprietorandthecommunitytowardsintegrationwillinfluencetheextenttowhichabetterskilledfacilitatorisabletoputimprovedteachingskillsintopractice.Asthequotebelowemphasises,thisrequiresbothcontinuoussensitisationoftheproprietorandbuildinghis/hertrust.Iftheproprietordecidestodiscontinueteachingoftheintegratedcurriculumwithintheschool,theopportunitytoimprovelearningoutcomeswithintheschoolwillbeterminated.‘Howdoestheproprietoraffecttheteachingandlearning?Heistheowneroftheschool,soifhedoesn’tacceptnobodywillevengotoschool.(…)Yousensitisethemandcreateawarenessandgivethemthatsupporttotheschool.Someofthemdon’thavetoiletorwaterpointanditisveryimportantfortheschools.Youstillgivethemthestipendmonthly.Itisbecauseofthesupportthatiswhytheintegrationistakingplace.Ifthereisnosupportnothingwillhappen.Theyalsoneedtotrustthatyouarenotgoingtotaketheirschoolfromthem.’(GovernmentKI,Katsina)

Furthermore,theactualavailabilityoffacilitatorstotrainandmentorisanobviousbutimportantassumptionoftheToC.ThiswillvarybyIQSandstate.KIsinallstatesotherthanSokotoindicatethatonaverageonefacilitatorteachesformalsubjectsineachIQS,whichislessthanthenumberassumedintheGEP3logframe.AccordingtobaselinedatafromBauchi,Niger,KatsinaandZamfara

31Itshouldbenotedthatnon-religiousprivateschoolsaremorelikelytobelocatedinurbanareas,whichmaybeanotherexplanatoryfactorfordifferencesineducationoutcomesbetweenpublicandprivateschools.32 Other factors, like the child’s health, cognitive ability and socio-economic background, can also influence learningoutcomes.Suchfactorsarenotdiscussedhere.


©EDOREN 19

mostIQSshaveonetotwofacilitators.33AnotherIQS-specificfactoristhatpupilsmayattendboth

anIQSandapublicprimaryschool.ThiswasparticularlyobservedinthebaselineIQSsampleinBauchi,where72%ofpupilsreportedattendinganotherschool.ThiscouldcontributetothedifferencesinlearningoutcomesthathavebeenobservedbetweenpupilsofIQSsandpublicprimaryschools(USAID,2013).34However,thisdifferenceisnotwellunderstood.KIsspeculatethatitcouldbeexplainedbythelevelofsupervision,thedegreeofinteractionbetweenpupilsandproprietors,thematurityofthepupils,orpupils’motivation.

Theabovefactorsareembeddedintheinstitutionalandpolitical-economycontextofeachstate,whichdetermineteacherrecruitment,deploymentandremunerationpolicies,fundingdecisions,

andinstitutionalrolesandresponsibilities.Forexample,Katsina’srestrictivecivilservantrecruitmentpolicyinthepasthasaffectedteachersupplyinthestate,whichislikelytoinfluencethepoolofteachercandidatestobetrainedandtheschools’pupil–teacherratio.Teacherdeploymentmayalsohavebeenaffectedbythe2015elections.Giventhatteacherdeploymenthasbeensubjecttopoliticalinfluenceinthepast,politicallyinspiredteachertransfersmaytakeplace,whichagainmayaffecttheteachersthataretrained.Institutionalarrangementsmayalsoaffectteachingandlearningoutcomes.TheGEP3IQSsaremanagedmostlybySAME,exceptinBauchiwheremostGEP3IQSsfallunderSUBEB.Thecapacityofbothinstitutionsdiffer,amongotherthingsintermsofaccesstofundingandtrainedteachers,aswellasintheirexperiencewithintegrationandthenon-formalsector.Thisaffectsthecontextandconditionsinwhichtrainedfacilitatorsoperateandmentoringhappens,whichcouldinfluenceresults.35

2.3.3.2 Headteachercapacitydevelopment

AccordingtoGEP3’sToC,headteachercapacitydevelopmentaimstoimprovethecapacityofheadteacherstosupporteffectiveteachingandschoolmanagement,contributingtoimprovedlearningintheclassroomandagender-sensitiveschool.Atthetimeoftheinterviews,thisinterventionwasyettobeoperationalised.Therefore,theattributesofthisinterventionanditsoutcomes(what,who,how…)wereyettobedefinedbasedonanassessmentofheadteachercapacityneeds.

Animportantattributethatrequiresclarificationistheexpectedfocusofthecapacitybuilding.KIsemphasisethedifferentrolesexpectedoftheheadteacher,suchasschoolmanagement,pedagogicalleadership,andsupervisionofcurriculumdelivery.Holdingteachersaccountableisalsomentionedasimportant,butitremainsundefinedhowandwhythisshouldtakeplace,aswellashowtheheadteacherisincentivisedtoholdteachersaccountable.Leavingthesefactorsundefinedlowerstheplausibilityasregardstheinterventionachievingresultsbecausethecapacitydevelopmentmaynotsufficientlytargetthekeymechanismsthroughwhichheadteacherscancontributetoeffectiveteachingandlearning.GEP3isscheduledtoconductaparticipatoryreviewofitscurrentapproachtoheadteachertraining.

TheToCofthisinterventioninthecontextofIQSsrequiresfurtherclarification.TheGEP3ToCsuggeststhattheinterventiontargetsheadteachertrainingingovernmentschools,whiletheIQSstrategypapersetstargetsforheadteachertraininginIQSs.KIsworkingintheIQSsectorindicate

33InBauchiandNiger25%ofIQSshaveonlyoneteacherofnon-religioussubjects,while37%havetwo.InKatsinaandZamfara,40%ofIQSonlyhaveasingleteachertoteachintegratedcurriculumsubjects.34RARAresearch in Islamiyya,Qur’anicTsangayaEducationcentres (IQTEs) foundthat IQTE learnershave relativelyhighlearningoutcomescomparedtoearlygradeprimaryschoollearners.35 RARA research (USAID, 2014) explored the association between learning outcomes and the agencies supporting theIQTEs. Higher performance was observed in IQTEs supported by SAME compared to SUBEB. However, it was notdemonstratedifsuchdifferencesweresignificant.Thecausalroleofthisinstitutionalfactorisalsouncertain.


©EDOREN 20

thataheadteacherpositionthatisseparatefromtheproprietororMallamrarelyexistsandthattheIQSproprietortakesontheroleofheadteacher.TheroleoftheproprietorvariesstronglyfromoneIQStoanother:focusingonschooladministrationinsomewhilebeingstronglyinvolvedintheteachinginothers.Hence,capacitydevelopmentofproprietorswillneedtobetailoredand/or

flexibletotheIQScontextandalignedwiththecapabilities,interestsandroutinepracticesoftheproprietors,inordertoobtaintheirbuy-in.

Severalinfluencingfactorsmentionedaspartoftheteachercapacitydevelopmentinterventionequallyaffecttheresultsoftheheadteachercapacitydevelopment.TwofactorsareworthhighlightingwithregardstothefunctioningoftheheadteachersinthecontextofGEP3states.First,KIsemphasisethatheadteacherappointmentisnotnecessarilytiedtoqualificationsandcanbe

subjecttopoliticalallegiance.Thismayaffecttheircompetencyandmotivationtocarryouttheposition,theirlikelihoodoftransferattimesofpoliticaltransitionandtheircredibilityinregardtoholdingteacherstoaccount.

‘Theappointmentofheadteachersissometimesbasedonpoliticaldemands.Politiciansdemandthatanallyismadeheadteacher.Youmayfindthattheheadteacherhaslowerqualificationsthantheteachers.Thismakesheadteacherslosetheircapacitytolead.’(GovernmentKI,Zamfara)

Second,theirabilitytoinfluenceteacherappointmentorbudgetallocationtoimprovethelearning

environmentiscircumscribedbythewiderpublicserviceandfinancialmanagementsysteminthe

state.TheEDORENliteraturereviewofbasiceducationinNigeria36indicatesthatheadteachershaveverylittlepowerandfewresourcestoaffectchange.Therefore,itisanuncertainassumptionthatheadteachershavethecapabilitytomakeschools,forexample,moregirl-friendly(inthewaythatKIsmostlyinterpretagirl-friendlyenvironment):thatis,providingseparatetoilets,installingwaterpointsandappointingfemaleteachers.GEP3doesaddressthisassumptionbystrengtheningtheSBMCs’capacitytomobiliseresourceswhichcansupportheadteachersinregardtobringingaboutchange.ThisassumesstrongcoordinationandalignmentofinterestsbetweenheadteachersandSBMCs.

2.3.3.3 Earlylearningintervention

TheearlylearninginterventionhassetasitsmainoutcometheimprovementofHausaliteracyinGrades1to3.37ThemaincausalassumptionunderlyingitsToCisthatliteracylearningoutcomes,particularlyinthemothertongue(whichisassumedtobeHausaformostpupils),willimproveinearlygradesifteachingimprovesintermsof:a)ateachingprocessthatemphasisesliteracyasafoundationalskillandhasafocusonactivelearningandtimeontask;andb)theuseofthepupils’mothertongueastheprimarymediumofinstruction.

Asmentionedabove,awiderangeofKIsendorsetheimportanceofearlygradelearning,whichprovidessupportfortheweightthatthisinterventionhasreceivedintheredesignedGEP3programme.Furthermore,KIssupporttheemphasisonliteracyduringearlygradesasafoundationalskill.AswasstatedbyoneoftheKIs:‘oncepupilsareabletoreadandspeaklearningwillgoeasily’.ThiswasalsoconfirmedinanIQSsetting.OneKIinvolvedinIQSteachingstatedthat‘thefirststepweareofferingistoshowthemhowtoread,howtowrite’.

36HumphreysandCrawfurd(2014).37HausaisthemainlanguagespokeninnorthernNigeria.


©EDOREN 21

Withregardstotheuseofthemothertongueastheprimarymediumofinstructioninearlygrades,thereisnooverallconsensusamongKIstowhatextentthisisanecessaryconditionforimproving

learningoutcomes.SeveralKIsargueinfavour–thatis,tostrengthentheuseofthemothertongueinearlygradesbasedonthefollowingarguments:thepupilwillbetterunderstandconceptswhentheyareexplainedinthemothertongue;communicationintheclassisenhanced;thetransitionbetweenhomeandschoolissmoothened;andtheexposuretophoneticsandphonologyinthemothertonguefacilitatestheirunderstandinginotherlanguageslikeEnglish.OtherKIsseetheuseofthemothertongueaspotentiallyusefulbutdonotattributelowlearningoutcomestolanguageorconsideritanecessaryconditiontoimprovelearningoutcomes,emphasisingthatotherelementsinthelearningenvironmentaremoreinfluential,suchasteachercompetencies.Comparisonismadewithprivateschools,whereEnglishisusedthroughoutandlearningoutcomesareperceivedtobebetter.

Questionsarealsoraisedaboutthedesirabilityandacceptanceofusingthemothertongueinearly

grades.SomeKIsdoubtthattheuseofthemothertonguewillbeacceptedinmetropolitanareasorareaswithhigherliteracylevels.Theyalsorefertothepotentialunintendedoutcomethatwhenmothertongueinstructionismorestronglyenforced,moreeducated,wealthyparentsmaysendtheirchildrentoprivateschools,reinforcingthedichotomybetweenprivateandpublicschools.Also,parentsmaynotsupporttheuseofthemothertonguesincetheymayperceivetheuseofEnglishasaproxyforthequalityofeducation.Furthermore,someKIssuggestthattheuseofthemothertongueinearlygradesmayresultinitcontinuingtobeusedinupperprimarygrades.Finally,someKIsarguethatHausaisnotthemothertongueofallpupils,andHausadialectsvaryfromstatetostate.However,thebaselinedatainKatsinaandZamfara,wheretheearlylearninginterventionispiloted,confirmthatHausaisthelanguageoftheimmediateenvironment,whichvalidatestheToCassumptionforthisintervention.Also,merelybecausetheyspeakalanguagedoesnotmeanthatteachersareliterateinit,orthattheyareabletouseteachingandlearningmaterialsinthatlanguage,eveniftheyareavailable.Thebaselinedatapointtothisreality.WhileallteacherssurveyedreportbeingabletospeakHausa,Hausaliteracylevelsamongteachersarelow.

‘Theimpedimentisthatsometeachersarenotversedinusingthelanguageforinstruction.Thereisadifferenceinspeakingalanguageandusingit.TherearedifferenttermsinHausafromstatetostate.TeachinginHausaisdifficult.’(Non-governmentKI,Katsina)

ImplementingtheearlylearninginIQSswillrequirecarefuldesign.Asmentionedbefore,thecapacitybuildingapproachneedstobeadaptedtotheIQScontextandshouldtakeintoaccountlargevariationsinthatcontext.WhatconstitutesearlygradesfromtheperspectiveoftheRANAteachingandlearningmethodologyneedstobeassessedagainsthowgradeprogressionisorganisedinIQSs.Othervariationsoftheorganisationoftheselevelsarelikelytoexist.Furthermore,themothertongueorlanguageofimmediatecommunityistherecommendedmediumofinstructioninBasic,whileEnglishandthelanguageoftheimmediateenvironmentisrecommendedinPost-Basic.TotheextentthattheRANAapproachincludesgrade-specificfeaturesthatarenotwelladaptedtotheIQScontexttheresultsoftheinterventionmaybeaffected.Also,theRANAschool-basedpeer-mentoringwillneedtobeadaptedtothecontextofahighpercentageofIQSsonlyhavingasinglefacilitator,asthebaselinedemonstrates.Finally,thedesignoftheinterventionwillneedtosuitthewaytheproprietorhasorganisedthedeliveryofthecurriculumintheIQS.

2.3.4 Output3:Improvedgovernancetostrengthengirls’education

AccordingtoGEP3’sToC,interventionsunderthisoutputaimtoinfluencekeyareasofgovernanceinstatebasiceducationsystemswhichaffectgirls’basiceducationinthelongterm.Theoutput


©EDOREN 22

addressesbothschool-levelgovernancethroughtheempowermentofSBMCs/CBMCsaswellasmostlystate-levelgovernancebysupportinginterventionsthatinfluenceplanning,programmingandfundingforgirls’education.ThemaininterventionsunderOutput3are:38

• capacitybuildingSBMCs/CBMCs:trainingandfollow-upmonitoringofSBMC/CBMCmembersfrompublicprimaryschoolsandIQSs,includingtheprovisionofschoolgrantsmanagedbySBMC/CBMC;

• supporttotheAnnualSchoolCensus(ASC)andtheeducationmanagementinformationsystem(EMIS):trainingandtechnicalsupporttostate-levelEMISstaffinASCplanning,dataentry,dataverification/cleaning,databasemanagementanddataanalysis;

• supporttotheGirls’EducationSteeringCommittees(GESCs):coordinationsupportforcontinuationoftheGESCsineachstateandatnationallevel;and

• supporttoHiLWA:supporttotheadvocacygroupHiLWAatthestateleveltoengagewithdecision-makerstoinfluenceactionstoimprovewomen’sandgirls’participationintheeducationsector.

2.3.4.1 EmpowermentofSBMCs/CBMCs

HighexpectationsregardingSBMCs.GovernmentKIsconsiderSBMCsakeyinstrumenttopromote

communityownershipofschools,withtheexpectationthatstrengthenedcommunityownershipwillalleviatedemand,supplyandgovernanceproblemsattheschoollevel.GovernmentKIshighlightedthatcommunitiesshouldnotrelysolelyonthegovernmenttoensureschooldevelopment.TheyseeSBMCsasameanstoshareresponsibilityforeducationwiththecommunity.Non-governmentKIsalsoseeSBMCs‘inthelimelight’,asoneintervieweeputit.TheyappreciateSBMCsasbeingawaytostrengthencommunityownershipoftheschool,highlightingtheirroleinschoolmonitoring,sensitisingparentsandschoolmanagement,observingthatSBMCsare‘inthedriver’sseatwhenitcomestowhattheschoolreallyneeds’,‘domorethangovernmenttomaintainschools’,and‘dothemonitoringthatpeoplefromtheministryareunabletodo’.

GiventhepivotalrolethatSBMCsareassignedintheschoolsysteminGEP3states,andthefactthatalmostallGEP3-supportedinterventionsrelystronglyontheircontribution,itishighlyrelevanttocontinuestrengtheningtheircapacityandsupporttheirempowermentasameanstoimproveschoolgovernanceaspartoftheredesignedGEP3.

WhiletheroleofSBMCsinschool-levelgovernanceisbroadlysupportedbyKIs,theirroleinthegovernanceofthewiderstateeducationsystemremainstobeclarified:inparticular,theirrolein

regardtoholdingstateandlocalgovernmentstoaccountforservicedelivery.ThisremainedlargelyunaddressedbygovernmentKIs,althoughsomeappreciatedSBMCsasbeinganefficient,directchanneloftheinformationneededforstate-levelplanningandaction.Non-governmentKIsperceiveSBMCsasagrassrootsvoicethatcanputpressureonthegovernmenttofulfilitscommitments.GEP3doesnotexpectoraimtosupportagovernanceroleforSBMCsbeyondtheschoollevel.39Thisassumesthatbyholdingactorsattheschoollevel(e.g.headteachersandteachers)toaccountSBMCscanimprovetheperformanceoftheschool.However,thisislikelytohaveitslimitationsbecauseseveralaspectsofschools’performance(suchashumanresourcemanagement,provisionoflearningmaterials,schoolinfrastructure)dependonstateandlocalgovernmentaction.TheremunerationoftheIQSfacilitatorsisacaseinpoint.TheIQSSbaselinedataindicatethatlocal38Output3alsoincludesadvocacyfortheFemaleTeacherTraineeScholarshipScheme(FTTSS).Thisinterventionhasnotbeenthefocusofthisassessmentbecauseitisbeingphasedout.39ThiswasconfirmedbyUNICEFaspartoftheirreviewofafirstdraftofthisreport.


©EDOREN 23

stakeholdersconsideritdifficulttoholdunpaidfacilitatorstoaccount.Hence,theCBMCs’effectivenessinregardtoimprovingfacilitatorperformancedependsongovernmentactionthattheCBMCmayormaynotbeabletoinfluence.

TheempowermentofSBMCsisbothapivotalintermediaryoutcomeinGEP3’sToCaswellasone

ofitsmostprecariouslinks.ThereisagapbetweenthehighexpectationswithregardtoSBMCs,intermsoftheirmanyrolesandresponsibilities,40andthecapacitybuildingandempowerment

processthatwillbegradualandthatstartsfromanoftenlowinitialcapacitylevel,particularlyinruralareas.KIspointoutthat,particularlyinruralareas,manySBMCmembershavelittleeducationandstilllackanunderstandingoftheirrolesandresponsibilities.TheplannedtrainingandmentoringismeanttoaddressthisbutKIsunderlinethatsuchtrainingandmentoringneedstoberegularandcontinuousinorderforittobuildthenecessarySBMCcapacity.TheyalsoindicatethatonlyalimitednumberofSBMCmembersreceivetrainingandthatthecapacitybuiltisnotnecessarilycascadeddowntoallmembers.Monitoringisagainhighlightedasimportant,whichGEP3isrightlyemphasisingviaitsSBMCEffectivenessMonitoringprocess.Giventheimportanceofmentoringandsupportivesupervision,itwouldbevaluabletopayattentiontothisaspectintheSBMCEffectivenessMonitoring.

‘TheSBMCshavebeendoingalotofjobs.Theyarewellmotivated.Theproblemis,especiallyintheruralareas,gettingtheSBMCstofullygrasptheirjobs.Someofthemwhentheygettheinitialtraining,theygettheSBMCsupport,thatwillbetheendofit.Theyhardlythinkofmotivatingtogetsomeinternalsupport.Theyaretherewaitingforthenextsupporttocome.ThereisneedformoretrainingbecausemostoftheSBMCsarenotfullyaware,especiallyintheruralareas.Butintheurban/semi-urbanareasyoufindSBMCsdoingsomewonderfuljobs.Peoplecomingingivingmoralandfinancialsupport,evenbuildingsomeblocksinsomeschools.Gradually,itismovingtowardssomeoftheruralareas.’(Non-governmentKI,Zamfara)

Ownershipisgeneratedthroughparticipationandactionthataddressestheneedsofthecommunity.Itisfacilitatedthroughnetworksthattransferknowledgeandideas,andthatgeneratesocialcapital.41Therefore,theactiveparticipationofSBMCsininterventionssuchasenrolment

drives,thecashtransferschemeandgrantmanagementcancontributetoownershipbuildingtotheextentthatitsbuildsonandnurturesbottom-upenthusiasmforaction,targetsissuesthatcommunitieswanttoseeaddressed,andtakesintoaccountlocalcapacityconstraints.StateSBMC

structures,trainingandparticipationincoordinationmeetings(e.g.forenrolmentdrives)offer

opportunitiesforSBMCtobuildsocialcapital.

TheestablishmentofMAsaspartoftheSBMCisappreciatedbytheKIs.TheyemphasisetheeffectiveroleofMAsatthehouseholdlevelinregardtosensitisingindividualsaboutgirls’enrolmentduringenrolmentdrivesandfollowingupontruancy.WhiletheinterviewsdidnotspecificallyinquireabouttheroleofwomeninSBMCs,thefactthatKIsmainlysituateMAs’andwomen’srolesintermsofsensitisingathouseholdandcommunitylevelsignalsaneedforfurtherinquiryregardingtheirroleatschoollevel–andinparticularregardingtheirabilitytoinfluencea

40 Some of the SBMC responsibilities mentioned by KIs are: supporting enrolment drives; supporting cash transferinterventionprocesses(registration,payment,attendancefollowup);supportG4G;supportingschoolmaintenance/repair;resource mobilisation for school investment; mobilising additional resources to motivate and support teachers; schoolgrantmanagement;developingandimplementingWholeSchoolDevelopmentPlans(WSDPs);keepingrecordsonWSDPsandresourcesmobilised/spent;teacherandpupilattendancemonitoring;curriculumimplementationmonitoring;holdingteachers accountable for performance; follow-up on pupil absenteeism; overall school oversight; maintaining therelationshipbetweenthecommunityandtheschool;andholdinggovernmenttoaccountforservicedelivery.41BurnsandWorsley(2015).


©EDOREN 24

girl-friendlylearningenvironment,throughwholeschooldevelopmentplanningand/orholdingheadteachersandteachersaccountableforimplementinggender-sensitiveschoolpractices.

Channellingfinancialschoolsupport,suchastheprovisionoftheGEP3grants,throughtheSBMCsisconsideredbyseveralKIsasanimportantfactorcontributingtoSBMCempowerment.TheyseeitasawaytomakeSBMCsfeelresponsibleandaccountablefortheiractions,andtheyexpectittotriggeradditionalresourcemobilisationwithinthecommunity.

‘Thefinancialsupportencouragesschoolsespeciallyintheruralareas,becauseifyouaskthemtousetheirownmoneytheywillstartaskingquestions.Butifyougivethemthatinitialsupport(thegrant)itwillpropthemup.Unfortunatelysomecommunitieswhenyougivethemthegrantstheyjustsitdownifthereisnomentoringfollow-uptofindoutwhattheyhavedonewiththegrants.Sincethegrantcamein,whatcontributionoftheirownhavetheymadeintheirowntimetotheschoolsbecausethegrantissupposedtoencouragethemtocontributetotheschool.Tobuildupschool–communityrelationsandparticipationofthecommunityinrunningtheschool,financialsupportincombinationwithtrainingandmentoringneedtoworktogether.’(Non-governmentKI,Zamfara)

However,thisdependson:theSBMCseffectivelyaccessingandusingthefinancialsupportfortheschool;communitiesbeingawareofthesupportprovided;SBMCshavingtheabilitytomanagethefinancialsupport;andthefinancialsupport,itsuseandresultsbeingtransparentlymonitored.Furthermore,thebaselinefindingsoftheIQSSevaluationindicatethatwhileCBMCsinIQSsundertaketheeffortofmobilisingadditionalresources,povertylimitstheirabilitytodoso.GEP3grantsseemtohavebeenthemainsourceoffinancialsupportchannelledthroughSBMCs,althoughfederalgovernmentself-helpgrants(providedviaUBEC)arementionedasapotentialalternativesource.ThesearecurrentlychannelledthroughtheLocalGovernmentEducationArea(LGEA).TotheextentthatchannellingfinancialsupportthroughSBMCsisindeedanimportantcontributoryfactorinregardtotheireffectiveness,itwillbeimportanttoclearlydefinetherolesandresponsibilitiesofSBMCsandLGEAsinregardtoschoolfinance.

OtherkeyfactorshighlightedbyKIsthataffecttheresultsofSBMCcapacitydevelopmentandempowerment,areasfollows:

• ThecompositionoftheSBMCs.SBMCmembersneedtobeproperlyselectedfromwithinthe

communitysothattheycanadequatelyengendercommunityownership.Itisarguedthattheinclusionofwell-respectedcommunitymembersincreasesanSBMC’scredibilityandpowertoinfluencebothschoolandcommunity.Thepresenceofwell-educatedmemberswhoareawareoftheeducationsystemcontributestotheSBMCseffectivelytakingontheirmultipleroles.Thisismoreplausibleinurbanthaninruralareas.Effectiverepresentationofwomencanbea

problemduetosocio-culturalnormsinhibitingjointmale-femalemeetings.TotheextentthatMAsareinvolvedinallSBMCfunctions,femalevoicecouldbechannelledviathisstructure.GEP3intendstobringgirls’voicesintoschoolgovernancebylinkingG4GgroupstoSBMCs.However,thisobjectiveneedstobemadeexplicitintheinterventiondesignbecausethisobjectivewasnotincludedinpastexperienceslikeSWAGS.Finally,theassignmentofpositionswithintheSBMCneedstosupportkeepingteachersandheadteachersatarm’slength.KIsmentionedthatitisbestthatteachersorheadteachers,whoformpartoftheSBMCs,nottakeontheroleofchairmanortreasurerasthiscouldunderminetheSBMC’sschoolmonitoringfunctionandcommunityownershipofinvestmentintheschool.

• SBMCacceptanceandsupportintheschoolandcommunity.KIsindicatethatheadteachersandteachersgenerallyaccepttherolethattheSBMCplaysintheschoolsinceitismeanttobea


©EDOREN 25

partnerinschooldevelopment.However,totheextentthatSBMCstakeanauthoritativeapproachtoschoolmonitoringandlackcredibility,thisacceptancemayfade.Asmentionedabove,acceptancebythecommunitydependsonthewaySBMCmembersareselectedandhowfartheyupholdtransparency.OneKIpointedoutthatthepoliticalaffiliationofsomeoftheSBMCmemberscanaffectthesupportfor,andfunctioningof,theSBMC.

• Accesstotheschool.Thedistancebetweentheschoolanditscommunities,andaccessibilitybyroad,influenceshowactiveSBMCmemberscanbeintheschool.KIsconfirmthatitislikelythatharder-to-reachcommunitiesmaybelessrepresentedintheSBMC.Thispointstoatrade-offbetweentheeffectivenessofanSBMCandequitablerepresentationontheSBMC.Inaddition,thesecuritysituationaffectsthefunctioningofSBMCs.

• Institutionalisationofgovernmentsupport.TheestablishmentofSBMCsisanationalpolicy,whichhasbeendomesticatedtodifferentdegreesatstatelevel.Forexample,accordingtooneKI,NigerhaslegalisedtheestablishmentofastateSBMCassociation,whilethisisnotyetthecaseinSokoto.Thedegreethatgovernmentsupportisinstitutionallyembeddedand‘goesbeyondpaper’,asoneKIputit,contributestohowcredibleSBMCsandtheirstateassociationsareinperformingtheirroles.However,thecreationofaninstitutionalenablingenvironmentmaystifleownershiptotheextentthatitimposesactionsnotwantedbytheschoolcommunity,insteadofenablingcommunityself-organisedaction.

CBMCsinIQSsfaceamorechallengingsituationthantheircounterpartsinformalschools.Theircapacitybuildingandempowermentthereforeneedstobemetwithevenmorerealisticexpectationsinregardtograduallyimprovingresults.Ingeneral,KIsconsiderthatCBMCsareabletotakeupsimilarrolesandresponsibilitiesasSBMCsbutthisissubjecttomorefactorsthatneedtobeaccountedfor.Thekeyfactorsmentionedareasfollows:

• TheCBMCneedstobeactuallyestablished.WhileinthecaseofformalschoolsitisplausibletoassumethatSBMCshavebeenestablished,giventheirlegalinstitutionalisationandrelativelylongtrackrecordofsupport,thisisnotthecaseforCBMCs.TheGEP3baselinedatacollectionindicatesthatactualintegrationinGEP3cannotbeassumed:hencetheestablishmentoftheCBMCcanalsonotbeassumed.TheestablishmentofCBMCsisaprerequisiteinorderfortheircapacitytobebuilt.

• TheCBMCneedstobeacceptedbytheproprietor.SinceCBMCsoperateinaprivateschoolenvironment,theiracceptancebytheproprietoriskey.‘SometimestheproprietorsarenothappyastheyseetheCBMCsestablishedencroachingontheirpowersandreducingtheirauthorityintheschools,thoughnotinallcases.Thereisaneedforrealorientationbecauseit’samisunderstandingonthepartoftheproprietors.Ifthisiswellexplainedtothem,theywillgivetheirsupport.’(Non-governmentKI,Sokoto)

• Theproprietor’sacceptanceislikelytodependontheaddedvaluethattheCBMCcanbringtotheschool,particularlyintermsofresourcemobilisation,theproprietor’sacceptanceofintegrationingeneral,andthewaythattheproprietorisinvolvedandhasavoiceinitsfunctioning.OneKIwasscepticalabouttheCBMCs’resourcemobilisationcapacitybeyondthetermoftheGEP3grantsinceoneofthemaindriversforparentstosendtheirchildrentoIQSsispoverty.ThisisimportantbecausetheCBMCs’resourcemobilisationfunctionisrelativelymoreimportantthanisthecaseforSBMCssinceIQSshavelessaccesstogovernmentfunding.

• Theacceptanceofintegrationbythecommunity.TheCBMCcanbebothanimportantsensitisingforceforintegrationwithinthecommunityaswellasbeingaffectedbylowacceptanceofintegrationbythecommunity.Ifthereislowacceptanceofintegrationamong


©EDOREN 26

parentsitwillbedifficulttomobiliseresourcesfromthemorhavethemparticipateasCBMCmembers.

• EffectiveCBMCmonitoringdependsontheroleoftheproprietorintheCBMCandonsupply-

sideconditionsbeingmet.SeveralKIsindicatedthattheproprietormaybethechairmanoftheCBMC.ThiswillinfluencethecommunityoversightfunctioninrespectoftheIQSsince‘howcanyoumonitoryourself?’–asoneKIquestioned.Inaddition,CBMCmonitoringisunlikelytobeeffectivewhenfacilitatorsarenotpaidandlearninginputsarenotprovidedbecauseitisdifficulttoholdfacilitatorstoaccountinsuchacontext.

• ThetypeofIQSinvolvedwillaffectthefunctionalityoftheCBMC.IftheIQSisofanomadicnature,thelinkwiththecommunityinwhichittemporarilyresidesmaybeweak,whichwillaffecttheestablishmentoftheCBMC.Similarly,ahighpercentageofboardingpupilsthatcomefromoutsideofthecommunitymayaffectthislink.KIsdidnotseethelatterasaproblem,arguingthatpupilsfromoutsideofthecommunityaregenerallywellacceptedin,andintegratedinto,thecommunity.

2.3.4.2 State-levelinterventionssupportinggovernanceinthestatebasiceducation

system

GEP3supportsseveralinterventionsabovetheschoollevel—mostlyatthestatelevel—toinfluenceorinformpolicy-making,planning,programmingandbudgetingaroundgirls’education.HiLWAseekstoengagewithdecision-makersandschoolcommunities,withtheaimofinfluencingactionstoimprovewomen’sandgirls’participationintheeducationsector.TheGESCispromotedasaforumforcoordination,supervision,advocacyandpoliticalengagementamongabroadgroupofstakeholdersofgirls’education.ThecapacitydevelopmentofstateEMISteamsissupportedtoimprovetheirabilitytoprovidereliableASCdataforGEP3outcomereportingandstateplanning.Inaddition,GEP3teamsandUNICEFstaffprovideassistanceandlobbystateandfederalgovernments.

HiLWA

AprerequisitestepinHiLWA’sToCisthatHiLWAneedstoberecognisedandthatHiLWAisabletoconnectwithstakeholdersatthestateandcommunitylevel.BasedonKIIsinZamfaraandBauchi,thevisibilityandrecognitionofHiLWAseemstovarydependingonwhenHiLWAwasestablishedandwasabletobecomeactive.Forexample,inZamfarastakeholdersaremoreawareofHiLWAcomparedtoBauchiwhereHiLWAbecameactivelateron,in2014.HiLWA’sdegreeofactivityhasbeeninfluencedbyGEP3’sredesignandbytheelections.

IntermsofHiLWA’sabilitytoconnectwithstakeholders,itsinvolvementinenrolmentdrivesand

engagementwithSBMCstructuresandinitiatives42offersanimportantlinktotheschool

communities.ItsengagementwithSBMCspresentsanimportantopportunitybecauseitallowsHiLWAtoconnectwithschoolcommunitiesandMAsbeyondtheannualenrolmentdrives.Thispotentiallyallowsittomorefrequentlyexerciseitrolemodelfunction,aswellastogathercommunity-basedevidenceoneducationissueswithwhichitcanapproachstate-leveldecision-makers.

TheKIsindicatethatHiLWAmembersareinapositiontoconnectwithhigh-leveldecision-makers,althoughHiLWAmembersstatedthatgettinganaudiencewiththesedecision-makersisachallenge.HiLWAmembersvisitedthegovernorinSokotoandthegovernor’swifeinZamfaraandBauchi.InBauchiHiLWAmetwiththeheadofthecivilservicetodiscussgreaterwomen’sparticipationin42HiLWAmembersinZamfaraindicatedthattheyparticipatedinSBMCworkshops.


©EDOREN 27

governmentfunctionsattheopportunemomentwhenthegovernmentmadethedecisiontorecruitadditionalteachers.Suchopportunisticactions,i.e.beingonthelookoutforopportunespacesforactionandstakeholderstoinfluence,isagoodapproachinadynamicandcomplexpolicyenvironment.Nonetheless,someimportantpolicyprocesses,likeperiodicplanningandbudgeting,followastructuredcycle.ItisnotclearhowHiLWAaimstobeinvolvedinsuchprocessesbeyondindirectlyliaisingwithstakeholdersthathaveaninfluenceonsuchprocesses.

BeyondtheprerequisitefirststepofstakeholdersbeingawareofandexposedtoHiLWA,factorsthataffectHiLWA’sToCandwhetheritachievesitsresults,areasfollows:

• HiLWA’scredibilityinregardtobeingabletohaveinfluenceandactingasrolemodels.Thisdependsonthecredibilityofitsmembers.TheKIsindicatethatthemembershavehighcredibilitygiventheirformerhigh-levelpositions.43HiLWA’scredibilitywillalsobeinfluencedbytheextenttowhichstakeholdersunderstandHiLWA’sroleandstructure.SomeKIswonderedwhetherHiLWAisanon-governmentalorganisation(NGO)oraproject.Transparencyinterms

ofroleandstructureavoidspotentialconfusion,whichmaydamageHiLWA’scredibility.• HiLWA’scapacitytoinfluence.ThispartlydependsonHiLWA’scredibilityandtheopportunities

forinfluencethatitseizes,butalsoontheextenttowhichHiLWAmembersareinapositiontodirectlyhaveinfluence.KIsindicatedthatitisdifficultforwomenthatarestillinhigh-levelpositionstobecomeHiLWAmembersbecauseofthetimecommitmentrequiredandbecauseHiLWAdoesnotofferadirectbenefit.ThecasesofBauchiandZamfaraindicatethatmemberswhohavebeeninhigh-levelpositionareoftenretiredornotintheirpositionanymore.Therewillbeatrade-offbetweenseekingmemberswithactiveengagementinHiLWA,whichrequiresatimeinvestment,andaimingtoincludememberswithahighinfluencingcapacitybecauseoftheircurrentposition(butwhowillhavelimitedtime).

• ThemotivationoftheHiLWAmembers.KIssuggestthatHiLWAmembersarewellmotivatedtobepartofHiLWA.Motivationwillbeimportantbecause,asoneHiLWAmemberargued,resultswillbe‘aquestionofkeeping[continuing]topush’.Theirmotivationislikelytodependon‘somethingcomingoutofit’,andonmembersfeelingsupportedandrecognised.

• AvailableresourcestosupportHiLWAactivities.ItseemsthatGEP3isseizingtheopportunitytocarryHiLWAmembersalongthroughsynergieswithotherinterventions,suchasenrolmentdrivesorSBMCworkshops.Thisisanefficientuseofresources.However,oneHiLWAmemberraisedresourcelimitationsinregardtotheirlogisticsandoperations(e.g.transportation)asachallenge,andindicatedthattheyareusingtheirpersonalfunding.

• Politicalandinstitutionalenvironment.ThetransitionperiodinrelationtotheelectionshasaffectedHiLWA’sscopeofwork:inparticular,itsgovernmentadvocacyactivities.Inaddition,theinstitutionalenvironmentwillcircumscribeHiLWA’seffectivenessininfluencingwomenparticipation.Forexample,theBauchiHiLWAmembersprovidedtheexampleofthefactthatBauchihasaquota-basedcivilservantemploymentsystemacrossitsregions.Hence,regionswithquotasthatarefilledupwillhavefeweropportunitiestoincreasewomen’semployment.Hence,theexistinginstitutionalisedemploymentpracticesmaylimitHiLWA’sroomformanoeuvreunlesssuchinstitutionsthemselvesarechanged.Finally,thereisariskthatHiLWAwillitselfbeaffectedbypoliticalaffiliation,whichmayundermineitscredibility.

GESC

SimilartoHiLWA,theprerequisitesfortheGESCtosupporteducationgovernanceisforstakeholders43Forexample,inZamfaramembershipincludestheformercommissioner,aformermemberofparliament,andthewifeofaformergovernor.InBauchi,theHiLWAchairanddeputychairareformerpermanentsecretaries.


©EDOREN 28

tobeawareofitsexistence,tobeinvitedand,participateintheGESCmeetings,andhaveacommonunderstandingofitsmandate.MostKIsareindeedawareoftheGESCatstateorfederallevel,44althoughsomenon-governmentKIsthataresupposedtobeGESCmembershadnotheardofit.ThemandateoftheGESCisgenerallyunderstoodasexercisingaGEP3oversightfunction,andactingasanadvocatefor,andcoordinating,actionstoimprovegirls’education.TheextenttowhichtheGESCisunderstoodashavingapoliticalengagementmandateislessclear,particularlyinregardtodiscussingandadvocatingaboutfundingchallenges.

InallstatesKIsconfirmthattheGESCisestablishedinthestateandhasbeenactiveinthepast.DuetotheprojectredesignandelectionstheGESChasmetinfrequentlyintherecentpast.45Inseveralstates,GESCseemstohavebeenreactivated,withmeetingsscheduledinAugustandSeptember2015.TherewasnouniformunderstandingamongKIsaboutwhethertheGESChasanannualworkplanbutsomeKIsindicatedthatthiswillbepartofthereactivationoftheGESC.

InBauchi,theGESCformedalobbysubcommitteeduringthepastgovernmentwiththeobjectiveoftakinguppoliticalengagementonfundingissues.Suchanexplicitpoliticalengagementfunctionwaslesspronouncedinotherstates.Inonestate,oneKIarguedthatfundingissueswerenotdiscussedinthecommittee,whileanotherKIindicatedthatstakeholdersarefreetovoicefundingconcerns.

‘ThecommitteehasthePTAchairman,theSBMCchairperson,CSOs[civilsocietyorganisations],womengroups.So,thereareindefinitepeoplethatarealwaysfreetovoiceoutissuesofbudget.Itisnotonlygovernmentofficialssothereisnowayyoucandenythem.Budgetsarediscussedatthecommitteemeetingbecausemembersofvariousagenciesarepresentandmaywantcertaininformationandclarifications.’(Non-governmentKI,Bauchi)

GiventhatreleaseoffundsbystategovernmentshasbeenoneofthemainchallengesofGEP3,the

plausibilityofGESCcontributingtoGEP3’soutcomeswouldbestrengthenedoncemonitoringand

politicalengagementaroundthisissueismoreexplicitlyincorporatedintoitsmandateandis

plannedfor.

TheToCforGESCseemstobethatifawiderangeofstakeholdersperiodicallymeet,discussandcoordinateactionstosupportgirls’education,thenthiswillinfluencegovernmentplanningandbudgetingdecisions,becausestakeholdersaremadeawareoftheissuesandareinapositionofinfluencetoact.Themechanismforactioncouldbethatonceanissueisraised,stakeholdersfeelpeerpressuretoact(externalmotivation),orthatstakeholdershavesufficientownershipthattheyfeelintrinsicallymotivatedtoact.SomeKIs,however,questiongovernmentownershipoftheGESC,indicatingthatitis‘alien’driven,orthattheGEP3teamiskeytomakingthemeetinghappen.LackofgovernmentownershipislikelytoaffectGESC’seffectiveness.AnotherKIpointedoutthattheGESCcouldtriggerademonstrationeffectbyshowcasingideasabouthowtoimprovegirls’educationthatcanbepickedupbythewidergroup.

AkeyfactorthataffectstheGESCToCiswhoisinvitedtobecomeamemberofthecommittee.Themembershipshouldallowforinfluenceandeffectivecoordinationofactions.Sincethekeycentreofpowerforfundingisthegovernor,theextenttowhichmembershaveaccessto,andinfluenceover,thegovernormattersforGESCeffectiveness.Thereislikelytobeatrade-offbetweenhavingahigh-levelpoliticalmembership(likecommissioners)thatpotentiallyhavehighinfluence,butwhose

44TheKIIsdidnotinquireaboutthefederal-levelGESCduringstate-levelinterviews.OnlyoneKIbroughtuptheLGA-levelGESC,buttheinterviewsdidnotexplicitlyinquireaboutthis.45TheinterviewteamwasnotabletoverifywhetherGESCmeetingshavebeentakingplaceandmembershavebeenparticipating,basedonmeetingminutes.


©EDOREN 29

commitmenttoattendingregularmeetingsisuncertain,versusamoretechnical,administrativemembershipthathaveamorecertaincommitmenttothemeetingsbutmayhavelessinfluence.Atthispoint,theredoesnotseemtobeacommonunderstandingamongKIsaboutwhoissupposedtochairtheGESC:thecommissionerorthepermanentsecretaryoftheMinistryofEducation.

SeveralKIsappreciatedthemulti-sectorandmulti-stakeholdercharacteroftheGESC.Forexample,oneKIpointedoutthathavingadirectorofaradiocooperationintheGESChelpstogetsupportatthetimeofenrolmentdrives.Finally,theinfluentialroleoftraditionalleaderswasraisedinseveralinterviews.

Besideawareness,degreeofactivity,ownershipandmembershipoftheGESC,otherfactorsthatarelikelytoinfluenceitsToCare:

• Thepoliticalenvironment.Asmentionedbefore,theelectionsandtransitioningovernmenthasaffectedthefunctioningoftheGESC.Furthermore,politicalconflictsmayaffectthefunctioningofthecommittee,whichisaparticularriskwhenitsmembershaveamorepoliticalprofile.OneKIsuggestedthatinthepastpoliticalmisunderstandingsbetweenthegovernorandthecommissionerhaveaffectedGESCactivitiesinZamfara.

• Fundingavailability.TheeventualtranslationofGESCactionsintoactualinfluenceonplanningandbudgetsassumesthatfundingisavailable.SeveralKIsconfirmedthatthegovernorhasdiscretionaryfundingavailablethatrequiresadvocacytoaccess.However,thisenvelopewilldependontheoverallbudgetenvelopethatisavailableviafederaltransferssincemoststateshavelittleinternallygeneratedrevenue.

SupporttotheASC

ThecapacitydevelopmentofstateEMISteamsaimstoimprovetheirabilitytoprovidereliableandregularASCdataforGEP3outcomereportingandstateplanning.AfirststepinthechangepathwayforEMIScapacitydevelopmentisforstakeholderstohaveacommonunderstandingofthecapacityneeds.TheKIsconfirmsomeofthecapacityneedsthatarepresentedintheGEP3strategypaper

onEMIS:inparticular,relatedtocoordination,enumeration,dataverification,andaccessanduseofdatabaseandentrysoftware.TheEMISteamsseemtohaveinvestedstronglyinthedataverificationsystems,whichisanecessaryconditionforthereliabilityofthedata.Itwillbeimportantforthis

systemtobewellinstitutionalisedandresourced,toensuresustainabledataquality.

Whilestrengthenedcapacityinthegenerationofreliabledataisanecessaryfirststep,atalaterstagecapacitytobetterfacilitatetheaccesstoandusageofdataneedstobeconsideredsincetheusageofthedataisrequiredfortheASCtocontributetoplanning,demonstratingresultsorotherpurposes.Ingeneral,KIstakeitasagiventhatonceEMISdatareliabilityisadequate,datacanbeaccessedandwillbeusedinpolicy-makingprocesses.However,thisisnotnecessarilythecase.Theinterfacebetweenknowledgeandpolicyisinfluencedbyfactorssuchasthepoliticalcontext,theinterestoftheactorsinvolvedindataproductionandpolicy-making,andtheimplementationofintermediaryfunctionstostrengthentheknowledge-policyinterface.46

AccesstotheASCdataisaconditionforusageofdata.Government-levelaccessisfacilitatedby

havingStateMinistryofEducation(SMoE)andSUBEBteamscollaborateonthestateEMISunitsandbyhavingavarietyofstakeholdersrepresentedonthestateEMIScommittees.TheinvolvementofawiderangeofstakeholdersontheEMIScommitteemaycontributetotheownership,andsubsequentusage,ofthedata.Accessbynon-governmentactors,whichcan46Jonesetal.(2013).


©EDOREN 30

contributetosustainabledemandfordataandmonitoringoftheeducationsector,officiallyreliesonformaldatarequeststothePermanentSecretaryoftheSMoE.Thisprocessdoesnotseemtoworkeffectively:EDORENsentaformalrequestforthe2013ASCreportanddatatoallGEP3SMoEs47attheendofSeptember2015andnoresponsewasreceived.TheGEP3strategypaperonEMIScapacitybuilding(UNICEF2015c)indicatesthatallannualASCreportswillbepublishedandprintedandcopiesplacedinthepublicdomain.AccordingtooneKI,ASCdataarealsodisseminatedtothelocalgovernments,andthemediaareinvolvedindisseminatingthemtoawideraudience.48TheextenttowhichASCdataareaccessibleattheschoollevelisnotclear,butKIsindicatethatschoolskeepacopyoftheASCquestionnaire,whichgivesthemprimaryaccesstothedata.However,thisdoesnotallowthemtocomparetheirschool’ssituationwiththatofotherschools.Oneoftheshort-termobjectivesofGEP3supporttoEMIS,accordingtotheGEP3StrategyPaper(UNICEF2015c),istoensurethatEMISdatafeedbacktoLGAsandschoolstoinformplansinsupportofgirls’educationandmonitoringprogress.

KIsindicatethatASCdataareusedforplanningpurposes,suchasStateEducationSectorOperationalPlans(SESOPs).OnegovernmentKIalsopointedoutthatthedatacouldbeusefulfortargetingenrolmentdrivestoLGAswiththehighestgirl’senrolmentgaps.OtherusagesmentionedbytheKIsareplanningandmonitoringfordonor-supportedprojects,assessmentsbyspecialcommittees49orforresearchpurposes.GEP3directlyinfluencestheuseofEMISdatabyincludingthedatainitsdonorreporting.

TheGEP3StrategyPaper(UNICEF2015c)alsoreferstotheEMISdatabeingmadeavailableforinclusioninthepreparationofthestateMedium-TermSectorStrategy(MTSS)andAnnualSectorPerformanceReview.Ingeneral,nospecificactionsseemtobeforeseenaspartofGEP3’sstrategytoactivelyfacilitatetheuseofEMISdata.Whilethisisconsistentwiththefactthatthedatafirstneedtobeimprovedintermsofreliabilityandtimeliness,beforeemphasisinguse,andKIIsseemtoindicatethatusageistakingplace,itwouldbeworthexaminingwhetherthedemandsideofthe

EMISneedsstrengtheningintheGEP3ToCinorderachieveitseffectiveuseinevidence-based

planning,resourceallocationandpolicyorientationtoimprovegirls’education.

GEP3seekstoimprovegirls’representationineducationdatabyincludingIQSdatainEMIS.ThisisplausibleforthoseIQSsthatarewellalignedwiththeformaleducationsystemviaSUBEBs(theseIQSsarealreadyincludedintheEMIS),butdoesnotseemplausibleatthismomentforthemore

informalIQSsthatgenerallyfallunderSAME’sresponsibility.WhileUNICEFhasclarifiedthatintegratingIQSdataintoEMISdoesnotnecessarymeanintegratingthesedatawithformalschooldata,basicrequirementsforqualityeducationdatacollectionarenotyetinplace.AreliableIQSlistingwouldberequiredasastartingpoint,beforedatacollectioninmostIQSscantakeplace–particularlythosemanagedbySAME.TheGEP3baselineexperienceindicatesthatinformationontheexistenceofIQSsandtheirintegrationstatusisincomplete.Therealsoseemtobefluctuationsinintegrationstatus,whichwouldrequirecontinuousupdatingofthislisting.Furthermore,ifenrolmentdataareadesiredindicator(whichislikely)acensuswouldhavetocapturethewidevarietyofenrolmentsituations50acrossavarietyofgradecategorisationspresentinIQSs.Also,ifthe

47TherequestwasaddressedtothePermanentSecretary,withacopysenttotheDirectorofPlanningResearchandStatistics.LetterdeliverywasconfirmedbyDHL.48TheZamfaraEMISunitindicatedthatmediaannouncementswereaired,tointroducetheASCbeforeandduringtheconductingoftheASC.Equally,duringdisseminationtheyinvitethemediatobroadcastabouttheASC.49TheZamfaraEMISunitprovidedtheexampleofaSecondarySchoolAssessmentCommitteethatwasassignedtolookatthesecondaryschooleducationinthestateandtoadviseonhowitcouldbeimproved,byrequestingtheASCdata.50RARAresearchonIQSsencounteredawidevarietyofenrolmentsituations:pupilsjustenrolledinQur’anicclasses,pupilsenrolledinQur'anicclassesandintegratedclasses,pupilsenrolledinQur’anicclasses,integratedclassesandformal


©EDOREN 31

dataneedtobeusedtomeasureadditionalaccesstoformaleducationthedatacollectionneedstotakeintoaccountpossibledoublecountingwiththeformaleducationsectorwhenpupilsattendbothformalandnon-formalschools.KIspointoutthatdatacollectionformscouldnotbesimplytransferredfromtheformalschoolASCsincetheywouldneedtobeadjustedtothegradeclassificationandcontextofIQSs.51GiventhecapacityconstraintsfacedbySAMEitisunlikelythattheycanimplementthisrigorouslyatthemoment.

FactorsthatmayaffectthelikelihoodofGEP3supporttotheASCresultinginimprovedproductionanduseofreliableASCdataareasfollows:

• ResourceavailabilityforASCandEMISunitsforsustainableoperationsandcontinuedcapacitybuilding.Staffcapacitiesarestillbeingbuiltandfacilities,suchascomputers,arelimited.KIsconfirmedthatreleaseofgovernmentfundsforASChasbeendifficult.

• MotivationoftheEMISunitstaff.Dataqualitydependsonthemeticulousimplementationofdatacollection,verificationandmanagementprotocols.Wellmotivatedstaffareimportant,andthisinturndependsonseveralfactors,includingappreciationofworkdone,thefacilitiesavailabletodotheworkandtheremunerationthattheyreceive.Duringtheinterviews,EMISstaffvoicedconcernsaboutthelackofmodernhardware/softwaretocarryouttheworkanddelaysinpayment,whichtheyindicateddiscouragesstaff.

• Qualityofthedataentryandmanagementsoftware.EMISstaffhighlightedthesoftwareusedasoneofthemajorchallengestotheirworkinthepast.ThestaffarepositiveaboutthesoftwarerecentlyintroducedbyUNICEF,althoughitsintroductioncreateddelays.

• Anenablingschoolenvironmentandsystem.Poorqualityrecord-keepingatschoollevelwilltranslateintopoorqualitydatabeingrecordedintheASCquestionnaires.WhiletrainedenumeratorsareresponsiblefortheASCdatacollection,theyrelyontheheadteachersandtheschoolrecordstoprovidesomeofthedata,suchaspupildata.Hence,oneEMISKIpointedoutthatthoseattheschoollevelconcernedwithrecord-keepingneedtobetrainedandsensitisedtokeepqualityrecords.TheKIsalsomentionedthatshiftsintheacademiccalendarcancreateplanningproblemsfortheASC.Additionally,insecurityorclimateincidencesmayinhibitaccesstotheschools.

• Interestofgovernmentstaffandpolicy-makersinusingEMISdatainpolicy-makingprocesses.EMISdatamaybeignoredinpolicy-makingprocessesevenwhenthedataarereliableandtimely.OneKIpointedoutthatpolicy-makersneedtobesensitisedontheuseofEMISdatainplanninganddecision-making.

2.3.5 Pilot-to-scale-upapproach

Inordertoreachlargenumbersofgirls,GEP3followsapilot-to-scale-upapproach.GEP3’sToCindicatesthatakeyassumptionunderlyingtheoverallprojectapproachisthatstategovernmentsarewillingandabletoimplementlarge-scalechangesforgirls’education,oncerelevantandcompellingevidenceofthevalueofnewapproachesisclear.TheGEP3logframeconcretisesthescale-upambitionbyformulatingtargetsforprimaryschoolsandIQSstobereachedduringscale-upandadecreasingratioofprojectfinancingcomingfromdonorfundingcomparedtothestategovernmentcontribution.

schools,andpupilsjustenrolledinintegratedclassesinIQSs.Pupils’attendanceisalsonotassumedtobeasregularasinformalschools,withpupilsnotattendingforextendedperiodsoftimewithoutbeingconsidereddropouts.51Forexample,teachinginanIQSmaynottakeplaceintraditionalclassrooms,whichwouldrequireadjustinghowinfrastructureiscapturedintheASCform.


©EDOREN 32

Abasicprerequisiteforthepilot-to-scale-upassumptiontoholdisforstategovernmentstobeawareoftheexpectationtoscale-up.WhileKIsaregenerallyawareofthescale-upobjective,there

was,atthetimeoftheinterviews,noclearunderstandingaboutwhatscale-upwouldlooklike,

andonlylimitedplanninghappeningforit.AsoneKIputit,‘thediscussionofscale-upisafloat’.GovernmentKIscouldnotprovideinformationabouthowmanyschoolsareexpectedtobereachedduringscale-up,despitetargetsbeingincludedinthelogframe.OnlyfewKIshadconcreteideasabouthowscale-upcouldbestbeimplemented.Forexample,onegovernmentKIsuggestedthatinterventionsshouldfirstbeexpandedtomoreLGAs,withlimitedschoolsbeingtargetedperLGA(asisthecaseduringthepilotphase)because‘othercommunitiescanwitnessinterventionsasapilot’.AnotherKIarguedforadditionalsupporttoberesult-based,makingthecasethatadditionalSBMCsupportshouldbelinkedtoresults.

StateGEP3teamsandUNICEFstaffinterviewedrecognisedthatplanningforscale-upisimportant

andneedstostartwellaheadof2017,takingintoaccountthetimelinesofplanningandbudgetingprocesses.Scale-upplans,targetsandbudgetsneedtobeincorporatedintostateplanningprocesses,suchasSESOPsand/orMTSSs.SomeKIsindicatethatscale-upplanningisalreadytakingplaceandisbeingincorporatedintothereviewoftheMTSS.Similarly,scale-upneedstobetakenintoaccountinannualstatebudgetprocesses.52OneKIarguedthatthefoundationsofscale-upneedtobeintegratedintothe2016budget,ratherthanscale-upfundingbeingincludedonlyinthe2017budget,whenscale-upisscheduledtostart.Forexample,investmentsintheeducationsupplysideshouldbeintensifiedin2016,soastobeabletoaccommodatethescale-upofGEP3interventionsfrom2017onwards.

‘IfGEP3wouldwanttoscaleupin2017,theprojectwouldneedtoengageinApril2016whentheaggregatestateresourceenvelopestarttobedrafted.Itisgoodtobe‘around’whenthishappens.AtminimumGEP3shouldgetinvolvedwhenannualbudgetcallcircularisissuedtosectors.SectorbudgetsshouldbefinishedbySeptember.However,budgetcyclerarelyhappensasplanned.’(Non-governmentKI,National)

InlinewithEDOREN’sPEAstudy(2014),KIsconfirm,however,thatincorporatingscale-upprojectionsinplanningandbudgetprocessesisnotasufficientconditionforfundingbeingavailableforscale-up,sincethereleaseoffundsdoesnotnecessarilyfollowbudgets.Nonetheless,whilenotasufficientcondition,integratingscale-upinsectorplansandbudgetsisasupportingconditionandmakesscale-upmoreplausible.

FundingavailabilityisconsideredbymostKIstobethemainfactorconditioningGEP3scale-up.Atthetimeoftheinterviews,KIshadnoclearunderstandingabouthowthescale-upwouldbefunded.SomegovernmentKIsarguethatfundingwillbeavailablefromthestatebudget,andsupporttheirclaimbymakingreferencetointerventionsalreadybeingscaleduporsustainedwithstatesupport,suchasenrolmentdrivesorFTTSS.53ThechangeingovernmentthathastakenplacehasalsomadesomeKIsoptimisticthatfundingforscale-upwillbeavailable,asgovernorshaveexpressedaninterestineducation.However,otherKIsexpresseddoubtsaboutfundingthatreliesstronglyonstategovernment.Theyforeseethatstatebudgetswillbeconstrainedasincreasingfundingneedswillemergewithagrowingschoolpopulation,whilestateresourcesremainleanand

52OneKIpointedoutthatforscale-upfundingtobeincludedinthe2017budget,theprojectwouldneedtoengagewiththebudgetprocessinApril2016,whentheaggregatestateresourceenvelopestartstobedrafted.Anothercriticalmomentofengagementiswhentheannualbudgetcallcircularisissuedtothesectorministries.SectorbudgetsaretobecompletedbytheSeptemberbeforethebudgetyear.53 Government partners in Katsina and Niger indicate that the FTTSS has continuedwith state support, despite GEP3’sfinancialsupportbeingstopped.


©EDOREN 33

highlydependentonfederalsubvention.

‘Letmespeakonanoptimisticnote.Wemaynotsaythatit[thescale-up]isnotimplementableandmaynotbepositive.Itispositivebecauseitisagoodintention.However,ifyoulookatglobalprojections,particularlyforoilwhichisthemainstayofoureconomyinNiger,itisnottoogoodfor2016upto2017.However,onanotherpositivelight,Nigerstatehasbeencollaboratingwithotherinstitutionstoimproveagriculturevaluechain.Ifwedothatatleastthestate’sGDPwillvastlyincreaseandthenourcapacitytoimplementprogrammeswillalsoimprove.”(GovernmentKI,Niger)

Somepotentialfundingfromnon-statesourcesforscale-upareproposed,suchasfundingfromtheGlobalPartnershipforEducation(GPE)inSokotoandKatsina,orUBEC-matchedgrantfunding.SomeKIsexpectalternativenon-governmentfundingsourcestobemobilised:forexample,mobilisationofcommunityresourcesviaSBMCs,contributionsbyphilanthropists,orspecialstatefunds,liketheSokotoEducationDevelopmentFund.

MostKIsseethereleaseoffundsasbeingdependentonthepoliticalwillofthegovernor.Factorsthatinfluencethispoliticalwillare(accordingtoKIs):high-leveladvocacy,governmentownershipofandinvolvementintheproject,alignmentbetweentheprojectandgovernmentpolicies/structures,andevidenceabouttheresultsoftheinterventions.However,afewKIsdidnotbuyintothisargument,arguingthatbudgetreleaseisnotcausedbylackofpoliticalwillbutsimplybecause

resourcesarenotavailable.

‘Thedebateaboutwhybudgetsarenotreleasedisthewrongdebate.Youcannotimplementifyoudon’thavetheresourcesinthefirstplace.Theonlysolutionistodoproperbudgeting.’(Non-governmentKI,National)

WiththesupportoftheUNICEFfieldofficeeducationteams,GEP3isengaginginhigh-leveladvocacytoinfluenceplanningandbudgetingforgirls’educationatthestatelevel.Therecentelectionsrequirereinvestinginadvocacyinrelationtothegovernor,whichalsooffersan

opportunity,sincestateexecutivesaresettingouttheirpolicypriorities.Atthetimeoftheinterviews,UNICEF,togetherwithGEP3stakeholders,hadpaidcourtesyvisitsinSokoto,NigerandBauchi.KIsindicatethattheroleofexternalactors(suchasdonors),traditionalrulers,andSBMCscanbeparticularlyeffectiveforadvocacy.GESCandHiLWAcanalsoactasadvocacyplatforms(seeabove).TheKIssuggestthatpolicy-makersareresponsivetoevidenceintheirdecision-making.Inthisregard,GEP3’sToCisconfrontedwithachallengeintermsofthetimingofevidencebecoming

availableversusthetimingregardingwhentheevidencemaybeneeded.Asdiscussedabove,planningandbudgetingforscale-upwillneedtostartbefore2017,whilecertainitemsofevidence,liketheEDORENevaluationresults,willonlybecomeavailablein2017.

Theplausibilityofscale-upvariesdependingontheinterventionsinvolvedandthestatecontextualfactorsrelatedtotheseinterventions.Scale-upofteacherandSBMCcapacitybuildinginterventionsarerelativelymoreplausiblebecausetheneedfortheseinterventionsiswellrecognisedandacceptedascentraltoimprovedschooling.Thereisaninstitutionalandlegalframeworkthatsupportstheinterventions,andKIscanpointtospecificfundingsourcesthatcouldfinancescale-up–thatis,UBECfundingandGPE.Statecontextualfactorsneedtobetakenintoaccountaswell.Forexample,scale-upoftheGEP3earlylearninginterventioninKatsinawilldepend


©EDOREN 34

onthestate’scommitmenttoasimilarJollyPhonicsearlylearningintervention.54

Similarly,enrolmentdrivesarewellacceptedandsupported.Sinceenrolmentdrivesalreadycovertheentirestate,scale-upislikelytoexpressitselfinthewayenrolmentdriversareimplemented,ratherthanexpandingtheirreachtomoreschoolcommunities.Twofactorsmayinfluencegovernmentcontinuingwithstate-wideenrolmentdrives:first,ifthegovernmentcannotinvestsufficientlyinteachersandotherschoolsupply-sideelementstomeettheincreaseddemandcreatedthroughtheenrolmentdrives,governmentmayscaledowntheintensityoftheenrolmentdrivesorfocusthemwheresupplycanmeetdemand;second,theneedforenrolmentdrivesmaydecreaseinareaswhereawarenessaboutgirls’educationissufficientlyraisedandthenumberofout-of-schoolgirlsislow.

Totheextentthatthecashtransferprogrammecannotbesustainedbyotherdonorprojects(liketheGPE),thecashtransferprogrammeisalessplausiblecandidateforscale-upbecauseitiscapital

intensive,isnotwellinstitutionalised,andisseenasadonorpriority.Cashtransferswouldrequireconsiderablecapitalexpenditurefromthestate,iffundedthroughthestatebudget.Atitscurrentscale,thecashtransferscorrespondtoapproximately13percentoftheaverage2009–2013actualannualcapitalexpenditureinNiger,55whilethe10,700girlscurrentlybenefittingrepresentaround4percentofthefemaleprimaryschoolpopulation,whichisonlypartofthecashtransfertargetpopulation.56Evenalimitedexpansionofthetargetpopulationwouldrequireaconsiderableproportionofthestatecapitalexpenditures.Statecapitalbudgetsarehighlydependentonfederalallocations,andarelesslikelytobereleasedcomparedtorecurrentexpenditures.57Sincefederalallocationsaredependentonoilrevenues,thegovernment-fundedscale-upofthecashtransferwouldessentiallybeafunctionoftheoilprices,asoneKIputit.Lowoilpricesthereforemakethescale-upofthecashtransferprogrammelessplausible.

‘Theinterventionthatisgoingtobealittlebitdifficulttoscaleupisthecashtransferprogrammebecauseitiscapitalintensive.ThestatemadeacommitmentatthecommencementoftheprojectofcounterfundingofNGN21,000,000to12,500students.Untiltodaythismoneyhasnotbeenreleased.Thisisbecauseofthelimitedresourcesatthedisposalofgovernment.Ifgovernmenthasnotbeenabletoredeemitscommitmentforthefirstphaseuptillnow,itwillbeveryunwiseforonetotalkaboutscale-upofthatproject.’(GovernmentKI,Niger)

Inaddition,comparedtoSBMCsandteachercapacitydevelopment,cashtransfersarenotbasedonanestablishedinstitutionalorpolicyframework,whichmakesthemmoresensitivetoadhocpolicychanges.AccordingtoKIs,acombinationofthesefactorshascontributedtothecashtransferprogrammesinKatsinaandBauchibeinghalted.Finally,oneKIalsoquestionedwhetherscalingupdemand-sideinterventionslikecashtransfersmakessensewhensupply-sideinvestmentscannotkeepup.

Thescale-uptheIQSsupportisparticularlyuncertain.WhileKIsaregenerallysupportiveofthe

54JollyPhonicsiscurrentlyimplementinganearlylearningintervention,withthesupportoftheSUBEB,acrossZamfara.ItstartsimplementationinKatsinainthe2015–2016academicyear.55Weestimatethecapitalexpenditurecostofthecashtransfer(i.e.thecostofthebeneficiarytransferpayments)atNGN214millionperyearforonestate(NGN5,000xfourquartersx10,700beneficiaries).Theaverageactualannualcapitalexpenditureintheperiod2008–2013wasapproximatelyNGN1.7billion(source:SPARCPFMdatabase).56Inthe2014–15schoolyear275,359girlswereenrolledinprimaryschoolinNiger(ASC2014–2015).Thecashtransferinterventiontargetsallgirlsbetweenfiveand16yearsofage.57TheSPARCPFMdatabasedemonstratesthatintheperiod2009–2013theratiobetweenactualcapitalexpendituretobudgetfluctuatedbetween3%and89%,withanaverageof48%,whiletheratiobetweenactualrecurrentexpendituretobudgetonlyfluctuatedslightlyarounda92percentaveragefigure.


©EDOREN 35

integrationofbasiceducationinQur’anicschools,theinterviewsindicatethatscale-upislikelytobeaffectedbytheinstitutionalcontextofintegration,SAME’saccesstoresources,thenumberofwell-establishedIQSsavailableforscale-up,andsupply-sideconstraints–particularlywithregardtofacilitators.Withregardtotheinstitutionalcontext,KIsdonothaveacommonunderstandingabout

whichinstitutionisresponsibleforintegration.SomeKIsarguethatSUBEBisresponsible,giventheirmandatetoprovideuniversalbasiceducationtoallchildren,whileothersconsiderSAMEtoberesponsiblebecauseofitsmandateregardingnon-formaleducation.ThiscreatesadispersedlandscapeofIQSsmanagedbydifferentagencies58thatischaracterisedbylackofcoordination.Thiswillmakescale-upinstitutionallyandpoliticallycomplex.

‘DifferentactorsworkintheIQSsector:SAME,ArabicandIslamicEducationBoardandSUBEB.Allhaveanelementofthisactivitygoingoninschools.Thereshouldbeacomingtogethersoyoucanscale-upwithinalltheseschools.Thereisaneedforacollaborativepartnership.’(GovernmentKI,Sokoto)

Furthermore,SAME,whichmanagesthelargestnumberofIQSs,hasrelativelylimitedanduncertainaccesstofunding.ThemainfederalUBECfundingsourceisnotaccessibletoSAME.Furthermore,SAMEKIsindicatethattheirstatefundingisuncertainfromoneyeartoanother.Hence,scale-upfundingforIQSsupportisthereforehighlyunclearifscale-upismeanttorelyonSAME-managedIQS.

Inaddition,theGEP3logframehassetthetargetforIQSscale-upat500IQSsperstate,butatthetimeoftheinterviewthisnumberoftargetIQSsdoesnotseemtobeinplaceinallstates,atleastwhenthecurrentIQSeligibilitycriteriaaremaintained.59Evenwherethenumbersareavailableonpaper,itisuncertainwhethertheIQSsarewellintegratedinpractice.Selectingandsustainablyfundingfacilitatorsatscaleinlinewithagreedstandards(inparticularly,astipendofNGN7,500)alsoseemsunlikely.KIsindicatethatitisalreadychallengingtoselectqualifiedfacilitatorsinagreementwiththeproprietoratlimitedscale;andpaymentoffacilitatorstipendshasbeenhighlydependentonuncertainbudgetreleases.

2.4 AppropriatenessofGEP3’simplementationstrategy

BesidestheplausibilityofitsToC,therelevanceofaprojectisafunctionoftheappropriatenessofitsimplementationstrategyforthecontextinwhichitoperates.AnappropriateimplementationstrategyisaprerequisiteforthecausalpathwaysintheToCtooccur.Aspartofthisassessment,weconsiderthefollowingdimensionsofimplementationappropriateness:

• Involvementofstakeholders,particularlygovernmentstaff,intheimplementationprocess;• Capacitytoimplementtheprojectinaqualitymanner;and• Equityinthereachofthetargetpopulation.

ThisassessmenthasbeenconstrainedbytheoperationalchangesaspartoftheprojectredesignatthetimeoftheKIIs.Theoperationalisationofsomeinterventions,suchastheearlylearning

58Furthermore,insomestatesadditionalagencieshavebeencreatedthatalsoplayaroleinintegration,liketheIslamicEducationBoardinSokoto,andtheOfficeoftheDirectorGeneralforTsangayainBauchi.59InBauchi,SUBEBsupports526centresregisteredforintegration,whileBauchiSAME(BASAME)manages175centres.MostofthesecentresareoftheTsangayatype,withattendancemostlybyboys(source:SUBEBandBASAME).InNiger,SAMEmanages450IQSs,whileSUBEBmanagesalimitednumberofmodelIQTEs(source:SAMEandSUBEB).InKatsinaSAMEsupportsaround600IQSs,someofwhichtargetAlmajiri,whicharemostlyitinerantboys(source:SAME).InSokoto,SAMEhasabout630IQSs,whileSUBEBmanagesninecentres(source:SAME)


©EDOREN 36

interventions,hadyettobedefined.TheGEP3stateteamshadrecentlybeenrestructured,whichaffectedtheirunderstandingoftheoperationaldetailsoftheGEP3strategyintheirrespectivestates.

2.4.1 Stakeholderinvolvement

Stakeholderinvolvementinprojectdesign,decision-making,planningandimplementationcontributestoownership,whichinturnincreasesthelikelihoodofsoundoperationalperformanceandsustainabilityofaproject.GEP3’soperationalplansupportsstakeholderownershipbyemphasisingimplementationwithandthroughstateeducationpartners.Furthermore,byinvolvingstateactorstheircapacitiescanbebuilt.

Overall,KIsvaluethefactthatGEP3operatesfromwithinstategovernments,whichisconsideredtocontributetogovernmentinvolvementandcoordinationbetweentheGEP3stateteamandgovernmentstaff.

‘GEP3isdesignedinaverygoodformtobeproperlyimplementedwithcooperationofthestategovernment.Itisimportantthatitissupportedbygovernment.WithGEPoperatingundergovernmentroofyouarecarryinggovernmentalong.YoucaneasilygetaholdofPermSec.Governmentcapacityintermsofplanningisalsobuilt,forexample,creatingworkplans.’(Non-governmentKI,Sokoto)

GovernmentpartnerswereinvolvedintheGEP3projectredesignandtheGESCisdesignedasaspacetocontinuetheirengagementinprojectplanning,deliberationanddecision-making.FocalpersonsanddeskofficersatpartnerMDAsareappointedtocoordinateandsuperviseimplementation.Furthermore,thefocalpersonisasignatoryontheprojectaccount.Hence,itisforeseenthatstructuresandsystemsenablegovernmentinvolvement.

Projectownershipandimplementation,aswellastheextenttowhichstructuresandsystemsareactuallyoperational,areinfluencednotonlybywhetherbutalsobyhowgovernmentpartnersareinvolved.Governmentinvolvementistosomeextentexternallyratherthaninternallydriven,withKIsqualifyingGEP3’sgovernmentinvolvementas‘carryinggovernmentalong’,andstatingthat‘involvementhappenstotheextentthatleadershipissensitised’.Also,asmentionedabove,KIsquestiongovernmentownershipoftheGESC.Operationalisationofstructuresandsystemsthat

facilitategovernmentinvolvementareinfluencedbychangesinstaffingandthepoliticalcontext.Forexample,therestructuringoftheGEP3stateteamsaspartoftheredesignandnewgovernmentappointmentsasaresultofthe2015electionshaveinfluencedtheactivitiesoftheGESC.Theelectionsalsoaffectedtheoperationalisationofthecashtransferprogrammeimplementationunitsasgovernmentoperationalsupportwasnotprovidedasplannedduetothetransitioningovernment.

GovernmentinvolvementandownershipislikelytovaryacrossGEP3interventions.Theenrolment

drivescancountonbroadstakeholderengagementandenthusiasm,alongtrackrecordofgovernmentimplementation,andanannualprocessofplanningandlearningfrompastaction.KIshave,overall,agoodunderstandingoftheoperationalprocessesoftheenrolmentdrives.

‘TheUNICEFapproachofinvolvingallstakeholdersfromgrassrootsleveltotoptopartakeinenrolmentdrivehasshowntremendouswillingnessofalltosupportthisintervention.’(GovernmentKI,Sokoto)


©EDOREN 37

Similarly,theoperationalprocessesoftheASCandcashtransferprogramme,supportedbyjointSUBEB–SMoEimplementationunits,arecommonlyunderstood.Whileexternalconsultantshaveprovidedsupportduringdesignandimplementation,governmentstaffhavebeencloselyinvolvedbothinthedesignandinleadingtheimplementation.

Thedesignandplanningofotherinterventionsislesscoordinatedwithgovernmentpartners.TheoperationalisationofGEP3’ssupporttoIQSslacksclarityandcoordination.KIsworkingwithIQSswerenotveryawareoftheplanningofIQSfacilitatortrainingatthetimeoftheinterviews,eventhoughthiswasscheduledforwithinonemonthoftheinterviews.ThebaselinesurveysuggeststhatGEP3-focusIQSsarenotwellidentified.60InBauchi,stakeholdersdonothaveacommonunderstandingofhowmanyGEP3-focusIQSaremanagedbySUBEBandhowmanybytheBauchiStateAgencyforMassEducation(BASAME).Weakmonitoringandinstitutionaluncertaintyabouttherolesandresponsibilitiesforintegrationarelikelytobefactorscontributingtothissituation.TheprovisionofgrantsisanotherinterventionthatKIsareuncertainabout,assomedonotseemtoknowwhethergrantswillstillbeprovided.Also,withregardstoSchool-basedTeacherDevelopment,severalKIsinBauchiwerenotawarethatanewapproachisconsidered,ortheyindicatedthattheywerewaitingforDFIDtoprovidethemwitharedesignedapproach.

‘LastyearinOctoberwewereinvitedtoaone-daystakeholders’meetinghereinBauchiwithtwoconsultantsfromDFIDandtalkedabouttheredesignedGEP3andthefocusoftheredesign.Theconsultantstoldusthatschool-basedteacherdevelopmentwillnolongeranswertothatnamebecauseofotherelementsofteachertraining;sowearenowwaitingforDFIDtogiveustherevisedortheredesignedapproachwhichisyettobeavailabletous.’(GovernmentKI,Bauchi)

Atthetimeoftheinterviews,KIsinKatsinaandZamfarahadlittleinformationaboutthe

implementationoftheearlylearninginterventionorG4G,beyondthefactsuchinterventionswereplannedfor.Governmentinvolvementintheimplementationoftheearlylearninginterventionwillbeparticularlyimportantbecauseasimilarintervention,JollyPhonics,hasalreadybeenrolledoutacrossZamfaraandislikelytoberolledoutacrossKatsinain2016.

KIsconsidertheinvolvementofabroadrangeofstakeholdersatdifferentlevels(community,local,

stateandnationallevel)astrengthofGEP3’simplementationstrategy,inparticularthe

participationofSBMCsandMAs.TheempowermentofSBMCstatestructuresisapromisingmechanismforfacilitatingcontinuousandgraduallearning,andforprovidingvoiceandasenseofself-efficacytotheSBMCsandMAs.NGOsandCSOsarealsoinvolvedinGEP3implementation–inparticularinSBMCcapacitybuildingandmonitoring.However,KIIsindicatethatatlocalandstateleveltheirinvolvementisproject-driven,focusingmostlyonservicedeliveryandperiodicrepresentationinmeetingswheninvited,ratherthanstructurallystrengtheningthevoiceofcivilsocietyingirls’education.

Governmentinvolvementismeanttostrengthengovernmentcapacity.Thisisparticularlyrelevant

inthecaseofmonitoring.AlthoughGEP3’smonitoringfunctioninvolvesbothgovernmentaswellasnon-governmentpartners,itsrelianceongovernmentcapacityissubstantial.Thisisarisksincegovernmentmonitoringcapacityisconsideredweak(seebelow).Nonetheless,government

involvementinGEP3monitoringisrightlyconsideredimportant.Notonlydoesitofferanopportunitytobuildmonitoringcapacity,italsoenablesgovernmentstafftodirectlyobservethe

performanceoftheinterventions.Inorderforthistotranslateintolearningandimprovement,thespacesandprocessesneedtobeinplaceforgovernmentstafftoreflectonandaccountforresults.6020%oftheGEP3-focusIQSscontactedaspartofthebaselinesurveywerefoundtobenon-integrated,non-existingornottohavegirlsaspupils.


©EDOREN 38

Itseemsthatforenrolmentdrivestheannualstatecoordinationmeetingsoffersuchaspace.KIsindicatethatthefutureheadteacherandSBMCtrainingwillalsobeadaptedbasedonassessmentsandmonitoringresults.Thisagainoffersanopportunityforinvolvinggovernmentstaffinmonitoring,aswellasinjointreflectiononresults.

2.4.2 Governmentcapacity

Workingthroughgovernmentsystemsofferstheopportunitytobuildcapacitythroughaction,aspartofGEP3’sobjectivestoimprovegovernanceandachievesustainabilityandscale.However,italsoexposestheprojecttooperationalrisksrelatedtoweaknessesininstitutional,organisationalandindividualcapacitieswithingovernment.

‘Aweaknessforworkingthroughgovernmentmightbeinthebureaucraticnaturewhereyoucannotjettisonbureaucracy.Thestrengthofworkingthroughgovernmentisthatyouarebuildingtheinstitutionalandhumancapacity,whichleadstosustainability.’(Non-governmentKI,Niger)

KIsindicatethatindividualcapacityintermsofpeople’sskillsandknowledgeisweakeratcommunityandlocallevelcomparedtostatelevel.Asmentionedbefore,someSBMCmembersareilliterate,whichneedstobetakenintoaccountinthetasksrequiredfromthem.KIsalsosignalthatlocal-leveldeskofficersdonothavetherequiredcapacityyettocarryoutalltasksrequiredofthem,althoughsomearguethatthisisnotduetoskilllevelsbutratherbecauseoftheresourcesavailable.Regularfollow-upaspartofcapacitybuildingishighlightedbymanyKIsaskeybutthisisespeciallyaffectedbychallengesinresourceavailability.Aneffectivesystemofidentifyingwhereindividualcapacityneedsarehighestisthereforeessentialinordertobeabletoprioritiselimitedresources.TheSBMCEffectivenessMonitoringofferssuchasystematSBMClevel,totheextentthatitcapturesallrolesandresponsibilitiesrequiredoftheSBMC,suchastheirroleinthecashtransferprogramme.GEP3’sOperationalPlanalsoforeseestheestablishmentofacapacitydevelopmentreportingsystemthataimstoidentifycapacitydevelopmentneedsatLGEAlevel.Atthetimeoftheinterviews,theextenttowhichthiswasdevelopedwasnotyetclear.

IndividualcapacitybuildingismorelikelytobesuccessfulatstatelevelsinceGEP3isabletomore

directlyprovidesupportandmobilisespecifictechnicalexpertise.Forexample,theinterviewedmembersofthestatecashtransferPIUwerepositiveaboutthecapacitybuildingprovidedbytheexternalconsultantsduringtheinterventiondesign.The2015GEP3AnnualReviewpointstoimprovementinstateEMIScapacityduetomorepurposefultechnicalinputs,theinvolvementoftheGEP3M&Eleadandexternalpartnerships.TeacherFacilitatorsandMasterTrainersimplementingteacherdevelopmentaresourcedfromgovernmentandtheCollegesofEducation.KIsindicatethatthisisastrengthbecauseitensuresthattheyhaverelevantexperience.However,theTDPbaselineevaluation(Deetal.,2015)indicatesthatalthoughTeacherFacilitatorshavestrongbackgroundsinclassroomteachingandschooladministrationtheyhavethesamepedagogicallimitationsintheirskillsasaredemonstratedbyteachersthemselves.

Inadditiontoknowledgeandskills,individualcapacityisinfluencedbythemotivationoftheimplementationstaffandtheirretentionaftercapacityhasbeenbuilt.SomeKIsstatethatdelaysinpaymentstogovernmentstaffinvolvedinGEP3interventionsdiscouragestheirparticipationin

interventionimplementation.Inaddition,thetransferofgovernmentstaffoncetheircapacitieshavebeenbuiltaffectstheirabilitytoactasaneffectiveprojectresource.AccordingtooneKI,UNICEFhasinsistedthatthosethathavebeentrainedshouldnotbetransferredindiscriminately.Nonetheless,itremainsachallenge,inparticularatatimeofleadershipchange.


©EDOREN 39

‘Lettherebemotivationasstaffgetdirectpaymentimmediatelyworkisdone.Lettherenotbedelayinpayment.CurrentlytherearedelaysofthreemonthsforpeoplegoingfromASCenumeratoruptodataentryofficers.Thatdiscourages.’(GovernmentKI,Niger)

‘Anotherproblemalsoisthatpeoplecomeandgo.Thiscreatesaproblemwhenyoubuildcapacity.(…)Sothetrainingtheyweregivenforsometimetobeabletodeliver,attheendofthedayheispostedelsewhere.’(Non-governmentKI,Bauchi)

Governmentorganisationalcapacityvariesacrossinterventions.ThecashtransferschemeandEMISaremanagedbyspecialunits,jointlystaffedbySUBEBandSMoEs,whichfacilitatescoordination.Theseunitshavereceivedintensiveexternalsupporttoimproveproceduresandsystems.61TheKIIsindicatethattheseunitsarewellawareoftheirrolesandresponsibilities.Althoughtheyarestillconfrontedbychallengesinthereleaseoffunds,whichaffectstheiroperation,theirorganisationalcapacityislikelytosupporteffectiveimplementation.

‘InBauchiandKatsinatheydonothaveoperationalmanualsintermsofimplementationofcashtransferprogramme.TheSokotoProgrammeImplementationUnithasmanualsintermsofrecord-keeping,payment,mobilisationandsensitisation.Wehaveworkingdocumentsthatguideallactivities.InKatsina,oncetheconsultantlefttheydidnothaveanydocument.’(GovernmentKI,Sokoto)

However,organisationalcapacityatlocalgovernmentlevelmaynotbeonaparwithstatelevel

capacity,whichposesarisktoimplementation.

‘TheSBMCsneedmoretrainingbecausesomearenotliterate.Therewasonlyonetraining,andyoushouldcontinuouslyre-trainthem.Identifyingthebeneficiarieshadsomechallenges.So,theyhadtofallbackontheSBMCstogofromhousetohouseinordertoconfirmthebeneficiaries.So,theyneedtrainingontechnicalitiesofhowtheygoaboutobtaininginformation.’(GovernmentKI,Sokoto)

Similarly,enrolmentdrivesfollowwell-understoodprocedures,whichareperiodicallyreviewedandsubjecttosharingofexperience.Amulti-tiercoordinationprocessinvolvingawiderangeofstakeholderssupportstheirimplementation.TheearlylearninginterventionandG4Gwillbeimplementedwiththesupportofnon-governmentactors.Thisislikelytosupporttheorganisationalcapacityforimplementationduringthepilot,andismeanttostrengthengovernmentorganisationalcapacityforscale-up.OnepointofattentionisthattheirimplementationtakesplaceinthecontextofsimilarinterventionshavingbeenconductedinthepastorbeingcarriedoutatthesametimeinKatsinaandZamfara.Ifoperationalproceduresdiffersubstantiallyfrompastexperience,thismayresultinconfusionamonggovernmentstaff.

Monitoringandfollow-upsupportsupervisionareconsideredbymostKIsaskeyoperational

processesaspartofprojectimplementation.ItisthereforerelevantthattheGEP3redesignhasemphasisedtheproject’smonitoringfunctionandhaskeptM&EofficersaspartoftheGEP3stateteams.GiventhatthenewGEP3stateteamshadrecentlystartedoperationsatthetimeoftheKIIs,itwasnotyetclearhowthiswouldbeoperationalised.GEP3partlyreliesongovernmentcapacitytomonitorGEP3interventions.Whilestategovernments,withthesupportofpartners,haveinvestedinqualityassuranceunitsatstate,zonalandLGAlevel,KIspointtoinadequateandinfrequent

61KIshighlightthat incontrasttocashtransferschemes introducedinBauchiandKatsina inthepast,thePIUs inSokotoandNigerhaveaccesstomanualsandprocedurestosupporttheiroperations.ThePIUsalsoappreciatetheuser-friendlysupportsoftware.Similarly,theinterviewedEMISmembersvaluethenewlyintroducedASCdatasoftware.


©EDOREN 40

implementationofinterventionmonitoringandfollow-upaftertraining.Themainreasonforthisthatwashighlightedisthatresourcesarenotavailableorarenotreleasedformonitoringactivities.Otherreasonsmentionedare:thequalificationsofthemonitors(particularlyatlocalgovernmentlevel),accessibilityoftheschoolsandpoliticalmeddling.TheKIIsindicatethatmonitoringisparticularlyweakinIQSsbecauseSAMEanditslocalgovernmentstructureshaverelativelylimitedaccesstoresourcestoconductthemonitoring.TheGEP3baselinesurveyconfirmsthatintegrationwithinGEP3-focusIQSsisnotwellmonitoredandthatIQSsreceivefewexternalmonitoringvisits.

Finally,GEP3’simplementationislikelytobeinfluencedbyinstitutionalcapacityconstraints.Amajorissueisthelackofclarityregardingrolesandresponsibilitiesfortheintegrationofformal

educationinQur’anicschoolsbetweenSAMEandSUBEB.Also,thelackofapolicyframeworkforcashtransferschemesmayaffecttheimplementationofthisintervention,particularlyintimesofleadershiptransition.However,SBMCsupport,theASCandenrolmentdrivesarewellanchoredinpolicyandstatestrategy.Theearlylearningintervention,G4GandHiLWAareproject-driven,sotheirembeddednessingovernmentinstitutionalstructureshasyettobeestablishedanddemonstrated.

2.4.3 Equity

EquityineducationisaconsiderationthatisattheheartofGEP3.Theprojectintendstoreducethedisparitiesbetweengirlsandboysineducation.VulnerabilitywasaconsiderationintheselectionofthesixGEP3LGAswithineachstateatthestartofGEP3,usingthegendergapasatargetingcriterion.62Error!Referencesourcenotfound.presentstheaverageGenderParityIndex(GPI)amongGEP3LGAsandintheentireGEP3states.Furthermore,itdetailshowthesixGEP3LGAsperstatearedistributedacrosstheGPIquintiles.ThefirstquintileincludestheLGAswiththe20%lowestGPI,whilethefifthquintilerepresentstheLGAswiththe20%highestGPIwhenallLGAsinthestatearerankedbytheirGPI.InBauchiandNiger,GEP3ismostlyoperatinginLGAsthataremost

vulnerableintermsofagendergap,whileinKatsina,ZamfaraandSokotoGEP3isimplementedin

LGAswithrelativelylowergenderdisparitiescomparedtotherestofthestateLGAs.Theevaluationteamdoesnothaveaccesstotherawschool-levelASCdatatoreviewtowhatextentthemostvulnerableschools,intermsofgendergap,areincludedamongtheGEP3focusschools.Theinitialschoolselectiondidtargetmostlyruralschools,whichcanbeexpectedtobemorevulnerableintermsofgenderdisparityasbarrierstogirl’seducationaregenerallyhigher.

Table1: GPI–averagesandquintiledistribution,perLGA

GEP3

states

AverageGPIa NumberofGEP3LGAsperGPIquintile

GEP3LGAs AllLGAs Q1 Q2 Q3 Q4 Q5

Katsina 0.78 0.75 -- 1 3 2 --

Zamfara 0.56 0.55 -- -- 1 3 2

Sokoto 0.65 0.61 -- 2 1 -- 3

Niger 0.69 0.78 2 2 1 1 --

Bauchi 0.75 0.85 2 2 1 1 --

Source:ASC2014–2015aAverageofLGA-levelGPI,calculatedbydividinggrossprimaryschoolenrolmentofgirlsbygrossprimaryschoolenrolmentofboysintheLGA

62Cocco-Klein,S.(2014)NoteonGEP3targetingandimplementationstrategy.UNICEF.


©EDOREN 41

SeveralGEP3interventionsaredesignedtobeequityenhancingbeyondtheoverallgenderequity

focus:thatis,ashavinganemphasisonreachingthemostvulnerablepopulations.Thecashtransferprogrammetargetsschoolswiththehighestproportionofout-of-schoolgirls.ThesupporttoIQSsismeanttoexpandaccesstoqualitybasiceducationformarginalisedchildreninrurallocations,since,accordingtotheGEP3valueformoneystrategypaper,Qur’anicschoolsareoftenlocatedincommunitiesthatareunder-served.

Thedegreetowhichthemostvulnerablegroupswillactuallybereachedwillbeinfluencedbythe

operationalisationoftheinterventionstrategies.AccordingtotheinterviewedrepresentativesofthecashtransferPIU,thecashtransferprogrammehasreachedthetargetedpopulation.Empiricalverificationisrequiredastowhetherthemostvulnerablehouseholdsareabletoreceivethetransferasplanned.OneKIpointedoutthatcashtransferprogrammesinNigeriaarecurrentlynotrights-based:beneficiariesdonotseeitastheirrighttoreceivethetransferandgenerallydonotcomplainiftheydonotdoso.Thismaybeparticularlythecaseforthemostvulnerablegroups.

WithregardtoIQSs,theIQSselectioncriteriaincludedtherequirementthat40%ofthepupilsreceivingtheintegratedcurriculumaregirls.ThisshouldensureGEP3supportbenefitsgirls.However,asindicatedbefore,baselinedatacollectionsuggeststhatthisselectioncriterionisnotalwaysmet.

Theenrolmentdriveimplementationstrategyisadaptedfromyeartoyearbasedonlearningandstakeholderfeedback.Equityconsiderationsseemtobepartofthisadaptation.InZamfara,oneKIpointedoutthatinthe2014enrolmentdrivespecificattentiontoethnicminoritiesanddisabledchildrenwasincluded.InBauchi,the2014enrolmentdrivetargetedLGAswiththelowestenrolment.Localgovernmentcoordinationmeetingshaveexpandedtheleveloffemalerepresentation.SomeKIsacknowledgetherisk,though,thatenrolmentdrivescouldbereachingcommunitiesclosetotheschool,orwhereSBMCmembersreside,morethanremotecommunitieswheretheymaybeparticularlyneeded.

Theremotenessofcommunitiesandtheschoolislikelytoaffectwhichcommunitymembersare

representedontheSBMC.OneKIpointedoutthatthiswillparticularlyplayaroleduringtherainyseason,whenrepresentativesfromsomecommunitiesmaynotbeabletoattendSBMCmeetings.AspointedoutbyanotherKI,remoteschoolsaredisadvantagedsincetraineementorsmayprovidementoringatclusterlevelratherthanvisitindividualschools.ThisisparticularlyburdensomeforSBMCmembersofschoolsthatarelocatedataremotedistancefromtheclusterpoint.

HumphreysandCrawfurd(2014)pointoutthatequityconsiderationsneedtobeaddressedwhenSBMCsareempoweredandexpectedtomobiliseresourcesfortheschool.Theyindicatethatsystemsneedtobeputinplacetomakesurethatpoorcommunitiesthatareunabletomobiliseresourcesarenotfurtherdisadvantagedbyreceivingnofinancialsupport.

TheKIIsraisedtwoequityconsiderationswithregardstoteacherdevelopment.First,severalKIspointedtotheriskofincreasingthedisparitiesbetweenruralandurbanschoolsbyreinforcingmother-tongueinstruction.Bettereducatedparents,mostlyinurbanareas,arelikelytodemandinstructioninEnglish.Totheextentthatmothertongueinstructionisreinforcedinsomeschools—likelytoberuralschools—thismayexacerbatethedifferencebetweenurbanschoolsteachinginEnglishandattractingthechildrenofeducatedparentsversusruralschoolsteachinginthemothertongueandattractingthechildrenofless-educatedparents.Second,KIsindicatedthattheteachertrainingmaterialsaremostlyinEnglish.Thismaydisadvantageteacherswithlowercapacity–particularlyIQSfacilitators.


©EDOREN 42


©EDOREN 43

3 Baselineevaluation–Earlylearningintervention

3.1 GEP3’searlylearningintervention

3.1.1 Objectivesandexpectedresults

GEP3’searlylearninginterventionaimstoimprovetheearlylearningskillsofchildreninthe

primarygradesonetothree(P1–P3)inthemothertongue,whilealsopreparingchildrentolearnwithEnglishasalanguageofinstructionbythetimetheytransitiontogradefour.Akeymeasurementofsuccesswillbeimprovedliteracyskills.63

Tothisend,UNICEFhascontractedFHI360toimplementtheRANAproject.Theproject’sprimaryfocusisonimprovingreadingoutcomesforgirlsandboysingradesonetothree.Improvementinnumeracywillbeindirectlytargetedbyintegratingnumeracyintoreadinginterventions.64TheRANAprojecthasidentifiedthreeintermediateresults,eachofwhichhastwosub-components(seeBox1).

Box1:RANA’sintermediateresultsandsub-components

1. IntermediateResult1:Readingandnumeracyinstructionimproved1.1. Relevantmaterialsdevelopedanddistributed1.2. Teacherin-serviceprofessionaldevelopmentimproved

2. IntermediateResult2:Increasedengagementofcommunitiesinreadingactivities2.1. Communityawarenessincreased2.2. Schoolcommunitiesengaged

3. IntermediateResult3:Sustainablereadingdeliverysystemsimproved3.1. Readingpolicyimproved3.2. Interventionssustainablydeveloped

3.1.2 Interventionstrategy

RANAwillbeimplementedoverathree-yearperiodinsixLGAsthatareamongGEP3’spilotLGAs

inZamfaraandKatsina(threeLGAsperstate).TheRANALGAshavebeenpurposefullyselectedamongtheGEP3LGAsinviewoftheiraccessibilitytothestatecapital.Theinterventiontargets120publicprimaryschoolsand80IQSsthatareincludedinGEP3’spilotschoollists(60primaryschoolsand40IQSsoutof,respectively,210GEP3primaryschoolsand200GEP3IQSsperstate).Table2presentsthedistributionofthetargetschoolsacrossthesixLGAs.Noscale-uptargetshavebeensetbeyondthethree-yearpilotperiod.

63UNICEF(2015f)StrategyPaperonEarlyLearning(literacyandnumeracy)RANA.64Numeracyinterventionsareplannedtobeginonlyafterthe2016/2017academicyear(RANAQuarterlyReportSeptember–December2015).


©EDOREN 44

Table2: NumberofRANApilotschoolsperGEP3LGAandtypeofschool

TypeofschoolKatsinastate Zamfarastate

Batsari Kankia Rimi Bungudu K.Namoda Tsafe

Publicprimaryschools

21 17 22 23 19 18

IQSs 14 12 14 18 12 10

Source:Owncalculations,basedonFHI360reports

TheRANAprojectwillconsistofthefollowingcoreactivities:65

1. Provisionofapackageofteachingandlearningmaterials

• RANAwilldevelopanddistributetheRANALiteracyPackage(RLP):animprovedHausa-basedliteracycurriculum,andteachingandlearningmaterialsforP1,P2andP3inHausa(trainer’sguide,teacher’sguide,scriptedlessons,pupil’sworkbookandreadaloudbooks).

• Gender66andnumeracythemesareintegratedintothereadingcontent,exercisesandmaterials.

2. Teacherin-serviceprofessionaldevelopment

• P1–P3teachersandheadteachers67willbetrainedinHausa-basedliteracyinstructionusingtheRLP,andinteacherprofessionalisation(focusingontopicssuchastimeontask,lessonplanning,andeffectivepreparationandutilisationofmaterials).

• Trainingwilltakeplaceatthelevelofaclusterofschools,ledbyaMasterTrainer.Intotal,20clustershavebeenformedineachstate,witheachclusterconsistingofthreetosixschools.AllclustersinKatsinahaveamixedcompositionofpublicprimaryschoolsandIQSs,whilefourclustersinZamfaraonlyconsistofpublicprimaryschools.

• Peer-mentoring:oneLeadTeacherperschoolfacilitatesweeklymeetingsamongteacherswithintheschool,toshareideas,discussconcernsaboutRLPmaterials,andreviewtheforthcomingweek’slessonsandexercises.68

• MasterTrainerswillconducton-sitemonthlysupervisorysupportandmonitoringvisits.SUBEBand/orLGEAstaffwillconducttwice-monthlyvisits.

3. Communityawarenessandengagementactivities

• RANAstateteams,assistedbyFederationofMuslimWomen’sAssociationsofNigeria(FOMWAN),willsensitisecommunitiesandotherstakeholdersinregardtotheproject,inordertoobtaintheirsupportforearlygradeliteracy.

• SchoolcommunitieswillbeengagedthroughtheSBMC/CBMC,witheachoneappointingaLiteracyChampiontocoordinateliteracyactivitiesintheirschooland,asappropriate,withinthelocalcluster.LiteracyChampionswillbeinvitedtoannualtrainingsessionstolearnaboutactivitiestosupportliteracyacquisition.

65RANAQuarterlyReportOctober–December2015.66Genderisintegratedbyusinggender-sensitivecharacters,illustrationsandadjectives,aswellasthroughteachersensitisation(source:communicationwithFHI360).67InIQSsalltheteacherswhoteachnon-religioussubjectsareinvitedfortraining.IftheIQSdoesnothaveaspecificheadteacherpositiontheproprietorwillbetrained(source:communicationwithFHI360).68Forschoolswithonlyoneortwoteachersinearlygradesthisschool-basedsystemofteachersupportwillneedtobeadjusted,possiblybycreatingmini-clustersofgeographicallyproximatesmallschoolswithinthelargerclusters.


©EDOREN 45

• RANAwillprovideaCommunityLiteracyActionGuide,asetoflow-costresources,materialsandexercises,andwillpromotethecreationofcommunity-ledLiteracyActionPlans.

4. State-levelengagementandpolicyimprovementactivities

• State-leveleducationstakeholders,liketheSMoE,SUBEBandSAME,willbeengagedtoencouragestateownershipandsustainabilityoftheproject.

• RANAwilladvocateforachangeinpoliciesofimmediateconcerntotheproject,suchaspoliciesthatsupportearlygradeliteracyor(re-)deploymentofteachers.

TheRLPandteacherin-serviceprofessionaldevelopmentwillberolledouttogradesP1andP2duringthesecondandthirdtermsofthe2015/2016academicyear.AllteachersteachinginP1andP2willparticipateinthetraining.Teachersof96targetedschoolsweretrainedinFebruary2016duringaPhase1roll-out,whiletheteachersintheremaining104schoolsweretrainedinApril2016(Phase2roll-out).P3willbecoveredfromthestartofthe2016/2017academicyear,whichwillinvolvedistributingP3-tailoredmaterialsandinvolvingP3specificteachersinprofessionaldevelopment.TheinterventionpackagewillbesimilarforpublicprimaryschoolsandIQSs.TheRANApilotprojectisscheduledtoendinAugust2018.

RANAisimplementedbyaconsortiumoforganisations.FHI360isresponsibleforprogrammemanagementandtechnicalquality.FOMWANcarriesoutcommunityengagementandmobilisation.TheAchievingHealthNigeriaInitiativeisresponsibleforteachertraining,curriculumandmaterialsdevelopment,andforM&E.RANAeducationfieldteamsaretobeembeddedwithinthestatesandlocaleducationauthorities,whoaremeanttoleadtheinterventions.

RANAwilltakeplaceagainstthebackdropoftheotherGEP3interventionsbeingimplemented(butonlyinpublicprimaryschools69),suchasannualenrolmentdrives,SBMCtrainingandgrantprovisionfrom2015onwards,headteachertrainingin2016,and,potentially,thepilotingofG4GinasubsetofGEP3schools.

3.1.3 InterventionToC

69InKatsinaandZamfaraGEP3willonlyinterveneinthe40RANAIQSsineachstate,outoftheinitial200IQSsselectedforGEP3support.


©EDOREN 1

Figure1visualisestheToCoftheearlylearningintervention.Themaincausalassumption

underlyingtheToCisthatliteracylearningoutcomesofgirlsandboys,particularlyintheHausa

mothertongue,willimproveinearlygradesifteachingpracticeimprovesthroughtheuseof

improvedteachingandlearningmaterials(useofRLP)andthepresenceofmoreknowledgeable,

skilledandgender-sensitiveteachers.Numeracywillbeinfluencedindirectlybyincorporatingnumeracythemesintothereadingcontent,exercisesandmaterials.


©EDOREN 2

Figure1: DiagramdepictingToCfortheearlylearningintervention

Source:Authors


©EDOREN 3

Thecentralassumptiondiscussedaboveisconditionalonschoolstakeholders(suchasparents,communityleaders,IQSproprietors,headteachers,teachersandgovernmentstaff)beingsupportiveofanincreasedemphasisonmothertongueinstructionandliteracyacquisition;andontheteachersadheringtouseofthemothertongueduringinstruction.Tothisend,theprojectwillengagethecommunity,championliteracyinthecommunities,trainheadteachersandadvocateforgovernmenttoimplementenablingpublicpolicies.OtherfactorsthatwillinfluenceimprovedHausa-basedteachingpracticesareregularpupilattendanceandtheclassandschoolenvironmentsbeingconduciveforlearning.ThefocusonHausaassumesthatitisthemothertongueandthelanguagethatpupilsknowandunderstandbest.

Improvementofteacherknowledgeandskillsthroughin-serviceteacherdevelopmentisassumedtobecentraltomoreeffectiveteachingintheearlygrades.70ThisassumesthattheimprovedHausaliteracycurriculum,thematerials,thepedagogicalmethodologyandthein-servicetrainingandmentoringapproacharewelltargetedtotheneedsandcapacitylevelofteachers,aswellastotheirteachingenvironmentinpublicprimaryschoolsandIQSs.Tooptimiseteacherlearning,theprojectenvisionsateacherdevelopmentapproachbasedonprogressiveprofessionaldevelopmentandschool-basedpeer-mentoring,complementedbysupportivesupervisionandmonitoringbyMasterTrainersandgovernmentstaff.ThesuccessofthisapproachdependsonteachersbeingliterateinHausa,weeklypeermeetingsbeingfeasiblegiventhecontextoftheschool(forexample,teachersbeingabletomeetregularlywithotherteachersgiventhesizeoftheschooloritslocation),LeadTeachershavingthecapacity,motivationandavailabilitytoleadthemeetings,andMasterTrainersandgovernmentstaffhavingtheresources,competenciesandincentivestoconducteffectivesupportivesupervision.Throughgender-sensitivetrainingthataddressesequityintheclassroom,teachersareexpectedtotargetgirlsmorespecificallyinclass,whichismeanttoincreasethelikelihoodofgirlsbenefitingoftheeducationprovided.

Inorderforteachertrainingtoresultinimprovedknowledge,skillsandteachingpractices,teachersareassumedtobemotivatedtolearnandtranslatenewknowledgeandskillsintopractice.Peer-to-peerinteraction,supportivesupervisionandotherincentives(e.g.trainingcertificates)aremeanttocontributetothis.Theextenttowhichteachermotivationtranslatesintoimprovedteachingmayalsobeinfluencedbyteachers’remunerationandtheirworkingconditions,amongotherfactors.Furthermore,actualimprovedteacherefficacyandlearningoutcomesmayimproveteachers’perceivedefficacy,whichinturnmayincreasetheirmotivation.

ImprovedteachingandlearningisassumedtobefacilitatedbythedistributionandusageofrelevantteachingandlearningmaterialsinHausa.Thisrequiresthattheteachershavethepedagogicalknowledgeandskillstoeffectivelyusethematerialsduringteaching.Furthermore,itassumesthatthematerialsarewellalignedwiththecurriculumandwiththecompetencylevelsoftheteachersandpupils.

Whilethecentralobjectiveoftheearlylearninginterventionistoimprovepupils’literacyand,toalesserdegree,numeracyskills,itisassumedthatthiswillfacilitatetheacquisitionofEnglishasasecondlanguage,andthetransitiontoEnglishinlatergrades.Furthermore,improvedlearningintheearlygradesisexpectedtocontributetohigherretentionratesbecausechildrenwhoperformwellearlyonareassumedtobemoremotivatedandsupportedbytheirparentstoremaininschoollonger,alinkthatmaybeparticularlyimportantforgirls,whofaceahigherriskofdroppingout(UNICEF,2014d).Apotentialfeedbackloopexistsbetweenincreasedretentionandteachingquality70Wedistinguishbetweenthreetypesofknowledge:subjectknowledge,curriculumknowledgeandpedagogicalknowledge.SeeSection3.2.9.2.


©EDOREN 4

inearlygrades.Thisfeedbackloopcanbenegativeorpositive.Ifhigherretentionisnotaccompaniedbyincreasedteachingandschoolresources,itcanleadtohigherpupil-to-teacherratios,whichmaynegativelyaffectteachingquality.Ontheotherhand,teachersmaybeincreasinglymotivatedbythehigherretentionoftheirstudents,whichmaypositivelyinfluenceteachingquality.

3.1.4 PlannedresearchandM&E

InthissectionwedescriberesearchandM&Eactivitiesthatarecomplementarytotheevaluationandthatcouldbeofusefortheinterpretationoftheevaluationfindings.

• PlannedM&EbyRANA

RANAwillconductmonitoringandformativeevaluationbasedonthemonthlyschoolsupportvisitsthatwillbeconductedateachschoolbyMasterTrainers,sometimesincollaborationwithSchoolSupportOfficers.Dimensionsthatwillbemonitoredduringschoolsupportvisitswillincludefidelityofimplementation(properuseofRANA-scriptedlessons),availabilityoflearningmaterials,pupillearning(readingandmathematics)71,basictimeuse,andgenderequity.Datawillbecollectedelectronicallyusingtablets.Box2presentsthesixdatacollectiontoolsthatwillbeusedbytheMasterTrainersandSchoolSupportOfficers.

Box2:RANAmonitoringtools

1. HeadTeacherMeetingGuide:collectsinformationonhowtheRANAprogrammeisbeingimplementedintheschool,whatspecificchallengesarebeingfaced,andwhatisworkingwell.

2. LeadTeacherMeetingGuide:collectsdataonandreviewsweeklyclassobservationswithLeadTeacher.

3. ClassroomObservationTool:capturesdataonactivitiesincludedintheteachers’scriptedlessonsthatteachersdeliverintheclassroom;observationofthetimetakenforeachactivity;anddataontheclassroomenvironment,includingteacherconfidenceandpreparedness,andequalparticipationofboysandgirls.

4. TeacherMeetingGuideandFeedbackTool:facilitatesdiscussionofobservedresultsfromclassroomobservationwiththeteacherandcapturesdataonteacherpracticesthatcannoteasilybeobserved(includinggenderpractices).

5. EGRA:pupilreadingassessmentcoveringlettersoundknowledge(P1andP2),letterwriting(P1),syllableandshortwordreading(P1andP2),listeningcomprehension(P1andP2)andoralreadingfluency(P2).

6. EGMA:pupilmathematicsassessmentcoveringnumberidentification(P1)andquantitydiscrimination(P2).

TheRANAprojecthasestablishedaresultsframework,whichincludesoutcome,outputandprocessindicators.

• PlannedresearchbyUNICEF

71 Master Trainers and School Support Officers will observe classrooms, will conduct early grade reading assessments(EGRA) and early grade mathematics assessments (EGMA) with a small number of students (three to five studentsrandomly selected from each observed class), andwill hold formativemeetingswith teachers, head teachers, and leadteachers.


©EDOREN 5

UNICEFisplanningresearchintoparents’attitudesregardingthelanguageofeducationandlearning.Thisresearchisscheduledfor2017/2018.Thetermsofreferenceforthisstudyhaveyettobespecified.

3.2 Methodology

Thissectiondescribesthemethodologythatguidedthedesign,implementationandanalysisofthebaseline.Itstartswithapresentationoftheevaluationquestionsanddesignasaframeworkforthebaseline’smethodologicalchoices.Subsequently,specificmethodologicalissuesarediscussed:thesamplingstrategy,therandomisationprocess,theinstrumentdesign,thesurveyfieldworkandthedataqualityassurance.Thesectionendswithadescriptionofethicalandinclusionissues,andmethodologicallimitations.

3.2.1 Evaluationquestions

TheevaluationofGEP3’searlylearninginterventiontakesatheory-basedapproach.Theintervention’sToCwasusedasaframeworkforformulatingtheevaluationquestions.Theevaluationquestionsinterrogateawiderangeofthecause–effectassumptionsunderlyingdifferentstepsintheToC,inordertobetterunderstandhowchangemaycomeabout.Theevaluationquestionssubsequentlyguidedtheoverallevaluationdesignandbaselinemethodology.TheevaluationquestionsarepresentedinBox3.

Box3:EvaluationquestionsforevaluationofGEP3’searlylearningintervention

1. TowhatextentdoestheearlylearninginterventionimproveHausaliteracyandEnglishlanguagelearningoutcomesamonggirlsandboysintheearlygradesinprimaryschoolsandIQSs?Towhatextentdoestheearlylearninginterventionreducethegapbetweenthelearningoutcomesofthelowestperformingpupilsandtheexpectedlearningoutcomes,asexpressedinthecurriculum?

2. Towhatextentdoesteachers’knowledgeofliteracyandlanguageacquisitioninearlygradesimproveasaresultoftheintervention?

3. Towhatextentdoteachers’skillsinearlygrade,gender-sensitiveinstructionimproveasaresultoftheintervention?

4. Towhatextentandhowdoteachersadjustandchangetheirclassroompracticesasaresultoftheintervention?

5. AremorereadingandlearningmaterialsinHausausedintheclassroomduetotheintervention?Dotheycontributetomoreeffectiveteachingandlearning?

6. Towhatextentdoestheearlylearninginterventionimprovepupilretention,especiallyretentionofgirls?

Inadditiontotheintervention’sToC,thechoiceoftheabovequestionshasbeenbasedonthefollowingadditionalconsiderationsandcontextualfactors:

• InlinewithGEP3’semphasisonqualityoflearningandDFID’scurrentfocusonlearningwithintheeducationsector(DFID,2013),theGEP3evaluationprioritisesmeasuringchangesinlearningoutcomesoverchangesinaccesstoeducation.WhiletheRANAprojectemphasisesreadingasoneaspectofpupil’slearningoutcomes,theevaluationquestionsadoptliteracyasamore


©EDOREN 6

comprehensiveconstructtobeexamined,inlinewiththeGEP3strategypaperontheearlylearningintervention.Duetothefactthattheinterventiononlyindirectlytargetsnumeracy,andbecausebudgetconstraintslimitedthenumberofassessmentsthatcouldbeincludedintheevaluation,numeracyisnotprioritisedintheevaluationquestions.

• Fromanequityperspectiveitisimportanttounderstandiflearningistakingplaceforall,andwhetherGEP3isimprovingthelearningoutcomeofthepupilswhoarecurrentlyfallingbelowexpectedlevels,asexpressedinthecurriculum.Hence,theevaluationquestionsincludeafocusonexaminingthelearningoutcomesofthelowestperformingpupils.

• GiventhefactthattheearlylearninginterventionisofthestrategicinterestfortheprojectasawholeandthelimitedevidencebasefortheinterventionthereisstrongagreementamongGEP3managementandDFIDNigeriathattheevaluationshouldemphasisedeterminingtheeffectsoftheinterventiononteachingandlearningoutcomes,andshouldquantifytheattributableimpactoftheinterventionontheseoutcomes.

• Theevaluationfocusesonthelinkagebetweenteacherknowledgeandskills,teacherclassroompracticesandlearning,andtheactualuseoflanguageinthisteaching/learningnexus.TheevaluationtherebyrespondstothehighneedforsuchevidenceidentifiedinHumphreysandCrawfurd(2014).ResearchplannedunderGEP3willassesscommunityengagementactivitiesandparents’attitudestowardthelanguageofinstruction.

3.2.2 Evaluationdesign

Toanswerthequestionsregardingwhetherchangesinlearning,teachingandteacherknowledgeandskillsareattributabletotheearlyintervention,theevaluationisdesignedasaclusteredRCT,stratifiedbyLGAsandtypeofschool(primaryschoolvs.IQS).72ForeachtypeofschoolanequalsizedstudysampleofschoolswasrandomlyselectedfromamongallGEP3schoolsineachofthesixGEP3LGAsinwhichRANAisbeingimplemented.73Subsequently,halfoftheschoolstudysampleineachLGAwasrandomlyassignedtoatreatmentgroupandtheotherhalftoacontrolgroup.TheselectedstudytreatmentschoolswereincludedamongthelargergroupsofschoolstargetedbyFHI360toreceivetheRANAintervention,whilethecontrolschoolswerenotselectedforinterventionimplementation.Thedetailsofthesamplingstrategy,samplesizesandrandomisationprocedurearediscussedinthenextsections.

Theaimoftherandomselectionoftreatmentandcontrolschoolsistoconstructtwogroupsofschoolsthatarestatisticallyequivalentonaverageatthestartoftheevaluation(i.e.atbaseline),intermsofobservableandunobservablefactors.Iftherandomisationsucceedsincreatingsuchequivalentgroups,anydifferencesobservedinoutcomevariablesatalaterstageoftheinterventioncanbeattributedtotheintervention(Section3.3presentsthedataonthebalancebetweenthetreatmentandcontrolgroup).WeoptedtoconducttherandomisationattheschoollevelasthisisalsotheunitatwhichmostofRANA’sactivitiesareimplemented,74andbecauseschoolsoffersufficientunitsofrandomisationtoachievethedesiredstatisticalpower.

WechoseaRCTdesignbecause:a)theevaluationquestionsemphasisethequantificationofattributableimpactandfocusonarelativelywell-delineatedcausalchainbetweenteacher

72Theclusteredaspectoftherandomisationreferstothefactthattherandomisationtakesplaceataclusterorgrouplevelinsteadofatanindividuallevel.Inthiscasethecluster/groupistheschool.73Thereareapproximately35GEP3primaryschoolsand33IQSsperLGA.74Someactivitieswilltakeplaceatthelevelofaclusterofschools(i.e.cluster-basedtraining),whichwillrequireparticularattention,asdiscussedbelow.


©EDOREN 7

development,teachingpracticeandlearningoutcomes;b)thedesignisstrongintermsofminimisingselectionbiasandthereforeidentifyingattributableimpact;c)sufficientsampleunitsareavailabletogeneratestatisticallysignificantfindings;d)baselinedataarenotstrictlyrequiredtoassessimpact,whichprovidesflexibilityatendlineinregardtoevaluatingimpactonoutcomesforwhichnobaselinedataareavailable;ande)weconsidereditfeasibletoconstructandmaintainavalidcontrolgroup.Whiletheidentificationandmaintenanceofavalidcontrolgroupisaconcernandisexposedtorisks75,thefactthatRANAispilotedwithinasubsetofidentifiableGEP3schoolsfacilitatedtheconstructionofsampleframesfromwhichtreatmentandcontrolschoolscouldbeselected.Furthermore,sincethecontrolgroupisselectedfromamongGEP3schools(i.e.fromamongschoolsinwhichRANAwouldnotbepiloted),GEP3shouldhaverelativelygoodawarenessof,andcontrolover,whattakesplacesinsuchschools.Thislowerstheriskofthecontrolgroupbeingcontaminated.

TheRCTdesigniscombinedwithanoverarchingtheory-basedevaluationapproach.Theintervention’sToCisusedtoselecttheoutcomesofinterestforwhichchangeswillbecomparedbetweentreatmentandcontrolschools.Theseoutcomescoverintermediaryandfinaloutcomesalongtheassumedcausalchain,whichwillmakeitpossibletounpackhowchangetakesplace.

Aseriesofquantitativemethodsareusedwithinthesampledschoolstocollectdataonoutcomesandotherexplanatoryvariables–inparticulartheadministrationofpupilandteachertests,structuredteacherclassroomobservationsandasurveyofpupils,teachersandheadteachersusingclosed-ended,structuredquestionnaires.Duetobudgetconstraints,EDORENwillnotimplementschool-orcommunity-levelqualitativeresearchduringthe2015–2017evaluationperiod.76Theevaluatorsareawarethatthisisalimitationofthedesign.However,thismaychangeinthe2017–2020scale-upperiod,duringwhichqualitativeresearchcouldbeimplementedforin-depthexplanationofthequantitativefindingsatmidline.ThequantitativemeasurementalongtheToCdoesprovidearichdatasetthatunpackstheToC.Furthermore,qualitativedatageneratedthroughanewroundofstate-levelstakeholderinterviews(scheduledforthefirstquarterof2017)willenrichtheunderstandingofthecontextinwhichRANAtakesplace,andwillprovideinsightsaboutimplementationchallengesandinterventionperformance.Inaddition,RANAM&Edatacancomplementquantitativesurveydata.

Thedatacollectionmethodsare/willbeappliedatbaseline,midlineandendline.BaselinedatacollectiontookplaceinOctober–November2015inthefirsttermofthe2015–2016schoolyearbeforethestartofRANAimplementation(seeError!Referencesourcenotfound.).Thedetailsofthebaselinesurveyimplementationarepresentedintherestofsection3.2.MidlinedatacollectionisscheduledforMay–June2017,whichisthefifthschooltermofRANAimplementation.Thisisbeforetheinterventionisscheduledtofinish.ThetimingofthemidlinehasbeenchosentoallowforevaluationfindingstoinformGEP3scale-updecision-makingin2017.EndlinedatacollectionisnotincludedinError!Referencesourcenotfound.becausetheresources,scopeandtimingofendlinedatacollectionhaveyettobedecided,althoughitstimingispreliminarilysetfor2019.Whileallevaluationquestionscanbeansweredatmidline,endlinedatacollectionwouldofferanswersaboutlonger-termeffects.

75Schoollistshavebeenunreliableinthepastandarealmostcompletelynon-existentforIQSs.Stakeholdershavestressed,andpastexperiencehasdemonstrated,thatitisdifficulttocoordinatewithgovernmentpartnersinordertopreventcontrolschoolsfrombeingaffectedbytheintervention(‘contamination’).ThesecuritysituationinnorthernNigeriaalsoincreasesthedegreeofunpredictability.76Initially,EDORENhadplannedsemi-structured,qualitativeclassroomobservationsandin-depth,unstructuredteacherinterviewstocomplementstructured,quantitativeclassroomobservations.


©EDOREN 8

Figure2: TimelineofdatacollectionandtheRANAintervention

Apanelapproachisusedtosamplepupilsfortheevaluation.Thesamepupilsthatarerandomlyselectedatbaselinewillbere-interviewedandre-assessedduringthemidlinesurvey,bothininterventionandcontrolschools.Apanelapproachischosenbecauseitismorecost-effectiveintermsofrequiredsamplesizethanacross-sectionsurvey.Italsoallowsustoefficientlymeasureandcomparetheretentionofpupilsinthesampledcohortsintheinterventionandcontrolgroups.Furthermore,itdoesnotcausecomparabilityproblemswhenbaselineandmidlinesurveystakeplaceduringdifferenttermsintheschoolyear,sincebothtreatmentandcontrolgroupswillonaveragehavegonethroughthesameschoolcyclebetweenbaselineandmidline,andarethereforecomparable.Thedisadvantageofapanelapproachisthatacertaindegreeofattritionofpupilsfromthesamplecanbeexpected.Sincetheperiodbetweenbaselineandmidlineisnotverylong,wethinkthattheattritionlevelwillbeacceptableandwehavetakenthisintoaccountindeterminingthesamplesize.77

Atbaseline,pupilsarerandomlysampledfromamongallpupilsinP2grade(orequivalentinIQSs)whentheyareinthefirsttermofthe2015–2016schoolyear.ThepanelsurveyapproachwillenableustomeasuretheimpactoftheearlylearninginterventiononimprovinglearningoutcomesbetweenthestartofP2andtheendofP3forthecohortofpupils.TheevaluationwillnotbeabletomeasuretheimpactofapupilbeingexposedtotheentirethreeyearsofRANA,fromP1toP3.P2pupilswerechosenasthetargetpopulationatbaselineforthefollowingreasons:

• BecauseofresourceandlogisticalreasonsthedatacollectionfortheevaluationoftheearlylearninginterventionislinkedtothedatacollectionforevaluationoftheIQSS.Becausethisevaluationusesacross-sectionalsurveyP1pupilsintheirfirstschooltermarenotanappropriatestudypopulationbecausetheywouldnothavebeenexposedtoanyteachingatbaselinenoratmidline.

• ThemeasurementofliteracyatthestartofP1—beforebeingexposedtoanyteaching—wouldlikelydemonstratestrongflooreffectsandofferlimitedvaluableinformationinregardtoobservingtherealdifferencesbetweeninterventionandcontrolschoolpupilabilities.

• SinceliteracylevelsatP1wouldlikelytobeverylimitedinthetreatmentandcontrolschoolsitwouldbeimpossibletoverifywhetherthetreatmentandcontrolgroupsarebalancedonthismainoutcomeofinterest.

77Ifanendlinesurveyisconductedin2019inthestudyschoolsitishighlylikelythatalargepercentageofthepupilpanelmayhavedroppedoutortransferredschoolinthefour-yearperiodbetweenbaselineandendline.Ifsufficientresourcesareavailablepupilscanbetrackedtobesurveyedoutsideoftheirformerschools.Iftheresourcestodothisarenotavailable,thesizeoftheoriginalpanelofpupilswhoarestillatthesampledschoolmaynotbesufficienttodrawstatisticallysignificantresults.Insuchacase,weproposetotakeanewcross-sectionalsampleatP2and/oranuppergrade.

Schoolyear2015–2016


Schoolyear2017–2018 2018

Sep–Dec Jan–Mar Apr–Jun Sep–Dec Jan–Mar Apr–Jun Sep–Dec Jan–Mar Apr–Jun Aug

Baseline Midline

Phase1 Phase2RANAroll-out

inP1–P2

RANAroll-outin3


©EDOREN 9

Duetothephasedroll-outofRANAinP1–P2intheperiodFebruary–May2016,thepupilsin96oftheschools(Phase1)willhavearoundtwotothreemonthsofadditionalexposuretoRANAinterventionscomparedtotheother104schools(Phase2).Giventhattheschoolsareknownthisinformationcanbetakenintoaccountduringanalysis.

Teacherandheadteachersurveyswillalsotakeapanelsurveyapproach.Thesameteachersandheadteacherswillbesurveyedatbaselineandmidline.Althoughsomeattritioncanbeexpected,weanticipatethatattritionlevelsatmidlinewillbelow.AspartofRANAimplementation,agreementissoughttokeepthetrainedteachersinthelowerprimarygradesandforteachersnottobetransferredtonon-projectschools.78

3.2.3 Samplingstrategyandsampleofthebaselinesurvey

3.2.3.1 Universe

ThestudyuniverseforthebaselineoftheearlylearninginterventionevaluationismadeupentirelyofpublicprimaryschoolsandIQSsselectedforGEP3interventioninthesixLGAsinZamfaraandKatsinawhereRANAisimplemented.SincetheimplementingorganisationpurposefullyselectedthreeRANALGAsoutofthesixGEP3LGAsineachstate,thestudyuniversedoesnotcorrespondtotheentireGEP3schoolpopulation.

TheeligibilityofschoolsforGEP3supportrequiresthataschoolfulfilscertaincriteriasetbytheproject.ThisisparticularlyimportantwhendefiningtheuniversefortheIQSs.AllIQSsincludedintheGEP3needtohavestartedintegration(thatis,non-religious,formalcoresubjects,suchaslanguageandmathematics,aretaughtattheschool),shouldteachbothboysandgirls79andshouldberegisteredwithagovernmentagency.

Thedefinitionoftheuniversealsoextendstoteachersandpupils.ThetargetpopulationofpupilsneedtobeenrolledinP2andneedtostudynon-religioussubjects.Inpublicprimaryschoolstheidentificationofthepupilsisrelativelystraightforward;however,inIQSamoredetaileddefinitionofthepupiluniverseiswarrantedduetonon-standardcompositionandorganisationoftheIQS.Inordertoachievecoherentlydefinedpopulationuniversesacrosstwotypesofschools,itwasnecessarytofurtheroperationalisetheP2equivalentlevelforIQSs(seeBox4).

78BasedonEDOREN’sexperiencewithTDPandESSPINsurveysinNigeria,thisissueofobtainingagreementregardingthenon-transferoftrainedteachersneedstobeapproachedwithcaution.Inpractice,teachertransferisdifficulttocontrol.Therefore,wewilltakeintoaccountattritionintheteachersamplewhendeterminingthesamplesize.79Aminimumof40%ofpupilsaregirls.


©EDOREN 10

Box4:P2ortheequivalentlevel

InIQSsweareinterestedinobtaininginformationforpupilsthatareatthesamelevelasP2pupilsinpublicprimaryschools:thatis,pupilsthatareatthe‘equivalentlevel’toP2.Forthepurposeofoursurvey,suchpupilshavebeendefinedas‘pupilsthathavebeenlearningnon-religioussubjectsatprimaryschoollevelatthisschoolformorethanoneyear,butforlessthantwoyears’.Inaddition,theymusthavestartedatthesampledIQSatthelowestprimarylevel.Thesetwoconditionstogetherensurethatthechildrenhavehadexposuretomorethanoneyearofformalschooling,butlessthantwoyears.ThismakesthemcomparabletoP2pupilsinpublicprimaryschools.Theclassesthatsuchpupilsareplacedinarethereforethe‘equivalentlevel’classes,whicharewhatweareinterestedin.ThesemaybecalledStage1,Stage2,BasicLiteracy,Post-basicliteracy,orsomeothernameuniquetotheparticularIQS.Similarlytothepupils,theuniverseofteachersconsideredfortheearlylearninginterventionconsistsofteachersteachingatthedesignatedschoolsandteachingpupilsenrolledinGrades1to3orequivalent(seeBox5).

Box5:P1–P3orequivalentlevel

InIQSsweareinterestedinobtaininginformationforteachersthatteachpupilsthatareatthesamelevelasP1–P3pupilsinpublicprimaryschools:thatis,pupilsthatareatthe‘equivalentlevel’asP1–P3.Forthepurposeofthesurvey,suchpupilshavebeendefinedas‘pupilsthathavebeenlearningnon-religioussubjectsatprimaryschoollevelatthisschoolforlessthanfouryears’.Inaddition,theymusthavestartedatthesampledIQSatthelowestprimarylevel.Thesetwoconditionstogetherensurethatthechildrenhavehadexposuretolessthanfouryearsofformalschooling.ThismakesthemcomparabletoP1–P3pupilsinpublicprimaryschools.Theclassesthatsuchpupilsareplacedinarethereforethe‘equivalentlevel’classes,whicharewhatweareinterestedin.ThesemaybecalledStage1,Stage2,BasicLiteracy,Post-basicliteracy,orsomeothernameuniquetotheparticularIQS.

3.2.3.2 Samplingframe

TheschoolsamplingframeconsistsofallGEP3schoolsintheselectedLGAsofthetwodesignatedstates.ThesamplingframewasconstructedbasedonGEP3schoollistsmaintainedbyUNICEF.Thetotalsamplingframeforthebaselineoftheearlylearninginterventionconsistsof410schools(210publicand200IQSs)insixLGAsintwostates.AlloftheschoolsinthesamplingframewereassumedtobeGEP3compliantandthuseligibleforinclusionintheintervention.

However,duringthefieldworkitbecameapparentthatthequalityofthesamplingframeofIQSswaspoor.TheQur’anicschoolsweremeanttohavestartedintegration–teachingnon-religioussubjects.Duemainlytonon-integration(schoolsnotteachingnon-religioussubjects)manyoftheIQSsincludedinthesamplingframewerefoundtobeineligibletobesurveyed.Thisrequiredthesamplingofreplacementsschools(seebelow).


©EDOREN 11

3.2.3.3 Samplingsizeandsamplingparameters

Ineachstate,60primaryschoolsand60IQSsweresampledforstudypurposes,creatingatotalsamplesizeof240schoolsacrossthetwostates;halfofwhichwereassignedtothetreatmentgroupandtheotherhalftothecontrolgroup.Therefore,weoversampledtheproportionofIQSsinthestudy(50%ofschools)comparedtotheproportionofIQSsinwhichtheearlylearninginterventionwillbeimplemented(40%ofschools),toensureasufficientlylargeIQSsamplesize.Weexpecttosurveythesamesampleofschoolsatbaselineandmidline,andweexpectthelikelihoodofattritionwithintheschoolsampletobelow.

Ineachschool,thetargetsamplesizeforpupilsissixgirlsandsixboysfromP2ortheequivalentgrade,resultinginatotaltargetedsamplesizeof2,880pupils(1,440girlsand1,440boys).Thesamplesizeiscalculatedtoallowfordisaggregatedanalysisforgirlsandboys.ThetargetedsamplesizeforteachersisthreeteachersforeachpublicprimaryschoolandtwoteachersineachIQS.ThenumberofteachersislowerinIQSsbecauseweanticipatedonlytwofacilitatorsonaverageperIQS.Hence,thetotaltargetedteachersamplesizeis360primaryschoolteachersand240IQSfacilitators.Asdiscussedbefore,weexpectthesamplesizeforpupilsandteacherstobeloweratthemidlinesurveyduetoattritionofthepanel.Table3presentsthesamplesizesforallschoolsandinstrumentsimplementedwithineachschool.

Table3: Sampleparametersfortheearlylearningintervention

Publicprimaryschools IQSs

States Katsina,Zamfara Katsina,Zamfara

Numberofschoolsperstate 60 60

Totalnumberofschools 120 120

Instrumentsperschool

Headteacherinterview 1 1

Teacherinterview† 3 2

Teachercompetencytest 3 2

Classroomobservation 3 2

Pupilinterview* 6girls,6boys 6girls,6boys

EnglishliteracyassessmentforP2 6girls,6boys 6girls,6boys

HausaliteracyassessmentforP2 6girls,6boys 6girls,6boys†Thesameteacherswillbeinterviewed,testedandobserveddeliveringlessons*Thesamepupilswillbeinterviewedandtested

Thesamplesizewasdeterminedbyevaluatingtheminimallevelofchangeintwokeyoutcomeindicators(thelearningoutcomeandtheteachingknowledgeandskillsoutcome)thatcanbedetectedwithstatisticalconfidence.Thisisreferredtoastheminimumdetectableeffect(MDE).Inarandomisedcounterfactualdesign,theMDEcorrespondstothedifferenceinthemeanoutcomeestimatebetweentheinterventionandcontrolgroups(singledifference)orthedifferencebetweenthebaselineandfollow-upestimatesoftheoutcomefortheinterventionandcontrolgroups(difference-in-differenceestimate).Box6presentstheMDEsforthelearningoutcomeindicatorandteacherknowledgeandskillsindicatorusingadifference-in-differenceestimatebasedonthepupilandteachersamplesizes.ThedetailsofthestatisticalpowerprinciplesusedaregivenintheGEP3EvaluationFramework.


©EDOREN 12

Box6:EstimationofMDEs

MDEsforpupillearningoutcomesInlinewiththeGEP3logframe,themainliteracyindicatorisdefinedasthepercentageofgirlsthatachievebasicliteracy(specifiedasHausaliteracy).Usedindeterminingtherequiredsamplesize,theMDEisafunctionofthebaselinevalueoftheoutcomeindicator.Atthetimeofsamplesizecalculationwedidnotknowthebaselinevaluesofthelearningoutcomeindicators,butlearningoutcomesinnorthernNigeriaaregenerallyverylow.WethereforeestimatetheMDEusingplausibleandmaximumscenarios.Furthermore,weassumeanintra-clustercorrelation(ICC)of0.3,arelativelyhighinter-temporalcorrelation(ITC)of0.8,giventhepanelapproach,80%power,anda95%confidencelevel.Wetakeintoaccount30%attritionofsampledpupilsbetweenbaselineandmidline.UsingaplausiblescenariothatthebaselinepercentageofpupilsthatachievebasicHausaliteracyisaslowas10%,weestimatethata3.8percentagepointnetincreasewouldbestatisticallydetectablewhenthedataofprimaryschoolsandIQSs,girlsandboys,areanalysedjointly.WhenprimaryschoolsandIQSsareanalysedseparately,butcombiningthedataofgirlsandboys,weestimatethatthisMDEwouldbe5.3percentagepoints.TheMDEestimatefordisaggregatedanalysisofgirlsandboysforeachtypeofschoolequals5.9percentagepoints.ThelargestMDEforagivensamplesizewouldbeproducedundertheunlikelyscenarioofabaselinevalueof50%ofpupilsachievingHausaliteracy.Insuchacase,theMDEamountsto6.3percentagepointsforcombined‘primaryschoolandIQS,girlsandboys’analysis,8.9percentagepointsforseparateprimaryschool/IQSanalysisbutcombininggirlsandboys,and9.8percentagepointsforafullydisaggregatedanalysis.Bycontrollingforsomecovariatesthatinfluencelearningoutcomes,forexampletheageofthepupil,weexpecttoreducethevarianceoftheestimatesaspartoftheanalysis.Thiswillallowustodetectsmallereffectsizesforagivenpowerandsamplesize.MDEsforknowledgeandskillsoutcomesTheteacherknowledgeandskillsoutcomeindicatoristhepercentageofteachersdemonstratingminimumteachingknowledgeandskills.The2013TDNAassessedteachercompetenciesforprimaryschoolsandIQSteachersinKatsinaandZamfara.80Whileresultsvariedfromstatetostate,a10%baselinevalueseemsplausible(andispotentiallyevenonthehighside).WeassumeanITCof0.8,anICCof0.3,80%power,anda95%confidencelevel.Wetakeintoaccount25%attritionofsampledteachersbetweenbaselineandmidline.Takingintoaccountthesamplesizeandaboveparametersweestimatethatwewillminimallydetectanetincreaseof4.7percentagepointsbetweenbaselineandmidline,assumingabaselinevalueof10%andjointlyanalysingprimaryschoolandIQSdata.TheMDEwillbehigherwithdisaggregatedanalysis,i.e.approximately6.4to6.7percentagepointsforprimaryschoolsandIQSsrespectively.Assuminga50%baselinevaluescenario,whichreturnsthehighestpossibleMDEforagivensamplesize,theMDEwouldbe7.8percentagepointsinthecaseofaggregatedanalysisofprimaryschoolsandIQSsandapproximately11percentagepointsinthecaseofdisaggregatedanalysis.InlinewiththeGEP3logframe,weanticipatetheincreaseinteachercompetenciestoberelativelylargesinceteacherswillbethedirectbeneficiariesofintenseteachertrainingandmentoring.

3.2.3.4 Samplingdesign

80Johnson,D.andHsieh,P.J(2014)


©EDOREN 13

Themainsamplingmethodforschoolswasasingle-stagestratifiedsystematicrandomdraw,whileteachersandpupilsweredrawnthroughatwo-stagestratifiedsystematicsample.Measurementswereperformedbothatschoolaswellasatthepupilandteacherlevels,asdescribedingreaterdetailintheinstrumentssection(seeSection3.2.5).

Twotypesofschools(publicandIQS)wereusedasexplicitstrata.Thesamplingdrawwasfullyseparateforeachtypeofschool.SecondaryexplicitstrataweredefinedbytheLGA.AnequalnumberofschoolswasdrawnfromeachLGA.Withineachexplicitstratumanimplicitstratificationwasbasedontypeoflocation(urban,rural)andwithineachtypeoflocationonEuclidiandistancefromAbuja.ThedistancefromAbujawasusedasaproxyforthegeographicaldispersionofthesamplewithineachLGA.ThedistancetoAbujawascalculatedusingGPScoordinatesprovidedwithinthesamplingframe.81AbujawaschosenasanarbitrarylocationthatwasfarenoughawaytoallowforgeographicaldispersionofEuclidiandistanceswithintheLGAs.

Thesystematicrandomsamplingusedarandomstartandarealsamplingstep.Implicitstratificationwasachievedbysortingwitheachexplicitstratum.

IneachLGA20publicand20IQSswererandomlydrawn.Inaddition,twoschoolspertypeperLGAweredrawnasinitialreplacementschools,toprovideapoolofschoolsthatcouldbesurveyedincaseanyoftheschoolsinthemainsamplewerefoundtobeineligibleorinaccessibleduringthefieldworkperiod.Therefore,thesamplingsetuphadprovisiontoselect22schoolsofeachtypeinanygivenLGAinZamfaraandKatsina.

Thereplacementschoolsweremarkedafterthesamplehadbeendrawn.AuniformlydistributedrandomvariablewasusedtosorttheschoolswithineachtypeandLGAandthelasttwoschoolsinthissequenceweredesignatedasreplacementschools.Similarly,therandomvariablewasusedtodesignatethereplacementsequenceamongthedesignatedreplacementschools.

DuringfieldworkmanyoftheIQSswerefoundtobeineligible.Thereasonforineligibilitywasmainlynon-integrationorpoorintegrationofschools.Forthispurpose,additionalreplacementschoolshadtobesampledduringthefieldwork.Thedrawingofthereplacementschoolsmostlymirroredthemainsamplingapproach.InsomeLGAsthenumberofeligibleGEP3IQSsbecamequitelimitedduetohighproportionsofineligibility.Inthesecasesalloftheavailablereplacementswerereleasedtothefieldforthesurveyteamtoselectreplacementsinanorderthatcorrespondedtoapre-assignedrandomsequenceofrelease.

MapsofthesampledschoolsincludedinthestudyarepresentedinAnnexB.

Schoolentry

Thesamplingoftheteachersandpupilswascarriedoutbythesurveyteamsattheselectedschools.Therefore,schoolentrywasimportantinestablishingarapportandestablishingthecredibilityofthedatacollectionatschools.Italsoimprintedtheimportanceofrigorousselectionofalltherespondentsatschoolsandwastheoccasionforpresentingtotheheadteacherthesamplingprocedurestobeusedinpupilandteacherssamplingapproaches.Figure3presentstheschoolentryprotocolthatwasfollowedbythesurveyteams.

81GPScoordinateswereonlyavailableinthesampleframefortheearlylearninginterventionbaseline(KatsinaandZamfara).TheywerenotavailableinthesampleframefortheIQSSbaseline(NigerandBauchi)


©EDOREN 14

Figure3: Schoolentryprotocol

07:30Arrivalattheschool

Isschoolopen?

Presentthewrittenconsentformtotheheadteacherandseek

consenttoproceed

NO ContactStateCoordinator

ConsentisGiven NO Contact

StateCoordinator

YES

Sampleteachers

Samplepupils

Proceedwithinterviews

YES

Introduceyourselftotheheadteacherandpresenttheauthorisationletter

Samplingofteachers

Afterconsentwasgivenbytheheadteachertoperformthestudyattheselectedschooltheteamproceededwiththesamplingofteachers.ThesamplingofteachersfollowedtheprocessvisualisedinFigure4.

InordertoensurethatonlyeligiblesampledteacherswereincludedintheGEP3baselinestudy,thesupervisorexplainedtotheheadteacherexactlywhichteacherswereeligibleforinclusioninthesurvey.TheeligibilitycriteriaaredescribedinmoredetailinSection3.2.3.4.

Thesamplingofteacherswasconductedontabletsusingcomputer-assistedpersonalinterview(CAPI)software.ACAPIformwasusedtoinputalltheeligibleteacherspresentintheschoolontheday.TheCAPIselectionformselectedtherelevantteachers.Ingeneral,theheadteacherwasnotconsideredtobeeligibleforteachersampling,eveniftheheadteacherteachesherself/himself.IncaseswheretheIQSdidnothaveanadequatenumberofteachersteachingnon-religioussubjects,theheadteachercouldalsobeincludedintheteachersurvey.


©EDOREN 15

Figure4: Teachersampling

TalkwiththeHeadTeacher

Public

Askforteacherattendanceregistersforthedayofthe

interviewforteachersteachinginP1-P3

YES

Check:• Itisforthecurrentday• Itcoversteachersofthecorrectgrade• Itcoversteachersteachingnon-religious

subjects

Areanyattendancerecords

available?NO

RequesttheHeadTeachertoidentifyallteachers(includingvolunteers)teachinggrades

P1-P3

Arerecordscomplete?

YES

EnterthenamesofalltheteachersinthelistsornamedbytheheadteacherintoasamplingForm(CAPI)

NO

Identifythemissinggroupsofteachersormissingnamesofindividualteachers(includeanyvolunteerteachers)

IstheschoolpublicorIQS? IQS

AretheclassestaughtingradesP1-P6?

YES


interviewforallteachersteachingP2orequivalentpupils

NO


available?

YES

RequesttheHeadTeachertoidentifyallteachers(includingvolunteers)teachingP2or

equivalentpupils

NO

Foreachteacherspecifywhetherteacherteachesreligiousornon-

religioussubjects

CAPIselectsteachersrandomly:• 3teachersinpublicschools• 2teachersinIQS

Inviteselectedteacherfortheinterview

Istheteachereligibleandhasgiven

consent?

Administertheinterview,classroomobservationandtestto

theteacher

EnterthenamesofallselectedteachersonSchoolCompletion

Form

YESNO

Samplethereplacementteachers


©EDOREN 16

Samplingofpupils

Thesamplingofpupilswascarriedoutintwodistinctphases:

1. Consultationwiththeschoolrecordsandtheheadteacher,whichmostcommonlytookplaceintheheadteacher’soffice.

2. Physicalselectionofpupils,whichmostcommonlytookplaceintheclasswherethepupilsweretaught.Ifthepupilswerespreadacrossmultiplestrandsofthesamegrade,allthechildrenforthatthegradeweregatheredtogetherandtheselectionwasperformedonthefullcontingentofpupils.

ThefirststageofthepupilsamplingfollowedtheprocesssetoutinthediagraminFigure5.

Figure5: Pupilselection–Headteacher’soffice

TalkwiththeHeadTeacher

Askforpupilattendanceregistersforthedayoftheinterview

YES

Check:• Itisforthecurrentday• Itcoversthecorrectgrade(P2orequivalent)• Itcoversallarmsofthegrade• Itcoversbothboysandgirls

Areanyattendancerecordsavailable? NO

RequesttheHeadTeachertoidentifyallpupilsattendingP2orequivalentgrade

Arerecordscomplete?

YES

Proceedtotheclassroomwiththelistandusetheteachertohelpidentifyall

thepupilsonthelist

NOIdentifythemissinggroupsofpupilsormissingnamesofindividualpupils

ThepupilswhowereeligibleforassessmentinthisstudywerethoseatP2orequivalentlevel.Itwasimportantforthesupervisortomakesuretheheadteacherthoroughlyunderstoodwhichpupilsweresampled.

Oncetheeligiblepupilswereidentifiedthesecondstageofthepupilselectiontookplaceinclass.Girlsandboyswererandomlyselectedbygirlsandboysseparatelydrawingchitsfromanopaquebag.TheprotocolforthisprocessisdescribedinFigure6.Insmallerschoolstheteamswerenotabletofind12pupilsattendingP2orequivalentgrade.Inthisinstance,allthepupilsthatwereavailableinschoolonthedaywereinterviewed.


©EDOREN 17

Figure6: Pupilselection–classroom

Line-upalleligiblepupils–makesureyouhaveallchildrenlined-upifmorethanonestrandofthe

gradeistaught.Splitthemaccordingtogender

Organize2opaquebagsofchits,1forgirlsand1forboys

Eachbagshouldcontain:• 6markedchits• (Numberofboys/girls–6)un-

markedchits

Eachpupilpicksupachitfromthebag

Separateselectedchildreninonepartoftheclassroomandalerttheteacherthatthepupilshavebeen

selected

Inviteselectedchildforthetest

Isthechildeligibleandhas

givenconsent?

Arrivetotheclassroomandexplainselectionprocesstothe

teacher

YES

Administerthetesttothepupil

YES

Prepareforsamplingofreplacementpupils.

Eachbagshouldcontain:• (Numberofreplacedboys/

girls)ofmarkedchits• (Numberofboys/girlsnot

selected–numberofreplacementsneeded)un-markedchits

EnterthenamesofallselectedchildrenonSchoolCompletion

Form

NO

Haveyouexhaustedallinitially

selectedpupils(12)?

NO

3.2.3.5 Weighting

Samplingweightswereconstructedtoreflectthecomplexsamplingdesign.Theweightswereestimatedforalllevelsofanalysisandreflectthemulti-levelnatureoftheschooldata,withpupilsandteachersnestedwithinschools.Theestimatedweightsarepopulationweights,whichsumuptothetotalpopulationsizeasdefinedbytheuniverseandthesamplingframe.Theweightsoftreatmentandcontrolgroupswereestimatedseparatelyandeachsumuptothetotalpopulationlevel.Duetotheveryhighsamplingfractionrate(i.e.thelargeproportionoftheeligibleschoolssampled),finitepopulationcorrectionfactorswerealsoestimatedtoadjustthevarianceestimates.AdetailednoteonweightingcanbefoundinAnnexC.

3.2.4 Randomisationprocess

Randomisationwasperformedonthesampledschoolsinadoubleblindedfashion:neithertheschoolnorthesurveyorganisationwereawareoftherandomassignment.


©EDOREN 18

Astratifiedrandomisationapproachwaschosenoverasimplerandomisationinordertoensureabalancebetweentreatmentandcontrolschoolsonkeycharacteristics.RandomassignmentwascarriedoutforeachLGAandtypeofschool.Furthermore,thedistanceoftheschoolfromtheLGAcapitalwasusedforstratificationtoensurebalancebetweentreatmentandcontrolgroups.AscanbeseenfromthemapsinAnnexB,thetreatmentandcontrolschoolsaresimilarlyspreadoutgeographically,whichindicatesthatthegeographicalstratificationworked.

FivestrataonthedistancefromtheLGAcapitalweredefinedbasedontheEuclidiandistancebetweeneachschoolwithintheLGAandtheLGAcapital.Theschoolsweregroupedintofivegroups,accordingtotheestimatedquintilesatthenationallevel.QuintileestimationatthenationallevelensuredthatthedefinitionandthemeaningofeachofthequintileswasequivalentacrossdifferentLGAs.Forthisreason,notallquintileswerepresentineachLGA.Forexample,inasmallerLGAthemostdistantquintilegroupmaybemissing.

Stratifiedassignmentrequiresthatthesamplingunitsarerandomlyallocatedtotreatmentandcontrolschoolswithineachstratum.Itisthereforerequiredthatanevennumberofschoolsbeallocatedtoeachstratum.Inordertoachievethis,asimpleadjustmentprocesswasrun,wherebyschoolsweresystematicallyshiftedbetweentwoneighbouringdistancestratauntilallthestratainalltheLGAshadanevennumberofunits.

Thefinalrandomallocationwasperformedintwophases.Inthefirstphaseeachschoolwasallocatedtooneofthetwodesignatedgroups.Inthesecondphaseeachofthegroupswaslabelledasthetreatmentandcontrolgroup,basedontherandomdraw.

3.2.5 Instrumentdesignandpreparation

3.2.5.1 Instrumentdevelopmentprocess

Theprocessofinstrumentdevelopmentwasbasedonmultipleiterationsandfieldtesting,tocontextualisetheinstrumentsbyreferencetotherealityoftheGEP3schools.ThiswasparticularlyimportantforIQSssincethesearenotformalschoolsettings.ThekeystepsinthisprocessaredepictedinFigure7below.

Figure7: Instrumentdevelopmentprocess

Deskresearch:AdeskreviewoftheexistingevidenceonIQSswascarriedout,toguidethedraftingofmeasurementtoolsandsamplingprotocolswithinthisunder-researchedcontext.Inaddition,we

Deskresearch Pre-piloting Draftinstruments Piloting Finalisationof

paperversions

TranslationCAPIprogrammingCAPIpilotFinalCAPI

instruments


©EDOREN 19

reviewedgoodpracticesforeducationmeasurementandexistingeducationsurveyinstrumentsusedinNigeria.

Pre-pilotingandfieldresearch:BasedontheGEP3EvaluationFramework,andinformedbythedeskresearch,asetofexploratorytoolswasdevelopedandtrialledinthefield.InDecember2014preliminaryinstrumentsforinterviewswithheadteachers,CBMCs,pupils,andteachersweretestedinsixschools(IQSsandprimaryschools)inNigerState.Furthermore,thefieldvisitsprovidedabetterunderstandingoftheIQSsystem,thedivisionofrolesandresponsibilitiesbetweenmultipleactors,andtheapproachtobeusedforsensitivequestions,suchasthoserelatingtoschoolexpenditureandfinancing.GroupdynamicsinSBMC/CBMCfocusgroupswerealsoexploredanddocumented.

Draftinstruments:Theinformationgatheredduringthepre-pilotingandfieldresearchwasusedtodevelopdraftinstrumentsappropriatefortheGEP3baselineandapplicabletotheparticularcontextofIQSs.Theinstrumentsdevelopedwere:(i)inlinewiththekeyindicatorsidentifiedintheGEP3ToCandEvaluationFramework;(ii)sensitivetowardsthecontextofIQSs,includingtheschoolstructure,termdates,andterminology;and(iii)sensitivetowardstheculturalcontextofNigeria,andnorthernNigeriainparticular.Atotalofsevenquantitativeinstrumentsweredevelopedforthebaselineoftheearlylearninginterventionevaluation(seeSection3.2.5.2below).

Pilotingexercise:ThedraftinstrumentswerethenpilotedinonepublicprimaryschoolandthreeIQSsinJune2015.Thepilotingexercisehelpedhighlightadditionalchangestoimprovetheinstruments,suchasthefinalisationofresponsecategoriesforclosedresponsequestions,andthelanguagetobeusedtocommunicatewiththeIQSheadteachers.Inaddition,specialattentionwaspaidtotheinteractionwithchildren,todevelopmodulesthataremostlikelytoproducereliableresults.ThefeedbackfromthelocalresearcherswasinvaluableinthisphasebecausetheyhadadditionalinsightsintotheIQSsystemandthecontextofnorthernNigeria.

Thepilotingexercisewasalsousedtotestsamplingprotocolsforchildrenandteachers.Theresearchteamtrialledseveralsamplingmethods,notingthestrengthsandweaknessesofeachmethod.Thisexercisehelpeddevelopfinalsamplingprotocolswhichminimisedtheriskofsamplecontaminationandrespondentanxiety.Thelatterwasespeciallyimportantgiventhenatureoftheinteractionwiththechildren.

Finalisationofinstruments:Theinstrumentswerefinalisedbasedonthepilotingexerciseandadditionalfeedbackfromthelocalresearchers.

Translation:AfterfinalisationtheinstrumentsweretranslatedintoHausa,thelanguagespokenbythemajorityofthepopulationinthefiveGEP3states.ThetranslationwasqualitycheckedbytheEDORENNigeriaofficetoensureitsappropriateness.

CAPIprogramming:ThetranslatedandfinalisedinstrumentswereprogrammedforuseinCAPIontabletsintwolanguages,EnglishandHausa.Theprogrammedquestionnaireswereextensivelycheckedforcoherencewiththepaperversionsbeforebeingtakentothefield.

CAPIpilot:ThefinalCAPIquestionnaireswerepilotedinsixschoolsinAugust2015inNigerstateinordertooptimisethestructureoftheCAPIquestionnaireandtheinterviewflow.Thepilotalsomadeitpossibletofurthertestproperschoolentryandfieldtimemanagement.


©EDOREN 20

3.2.5.2 Overviewoftheinstruments

Seveninstrumentsweredevelopedtocollectquantitativedataateachearlylearninginterventionschoolandeachcontrolschool/IQS(seeTable4Error!Referencesourcenotfound.).

Table4: Overviewofthedatacollectioninstruments

Instrument Summary

Pupilquestionnaire

Thepupilquestionnaireprecedesthepupilassessments.Itchecksforpupildisabilityandcollectsdataonpupilhouseholdassets.Inadditionthepupilquestionnairecollectsdataonpupilbackgroundcharacteristics,suchasgender,yearsinschool,ifthepupilattendsanyotherschooletc.Pupilsidentifiedashavingadisabilityarenotgivenassessmentitemsthatrequirethatparticularability,inordertorespondmeaningfully(forexample,apupilwithanidentifiedinabilitytoseeisnotaskedtocompletethereadingitems).

PupilEnglishLiteracyassessmentforP2

TheEnglishLiteracyassessmentcontains13items,witheachitembeingmadeupofseveralsub-items.WhiletheEnglishLiteracyassessmentforGEP3istargetedatadifferentproficiencylevelthantheESSPINassessments,bothESSPINandGEP3havebeencalibratedontothesameEnglishLiteracyscale(theEDORENscale)throughlinkitems.ThismeansrobustcomparisonsbetweenESSPINCompositeSurvey(CS)resultsandGEP3evaluationresultscanbemade.Theassessmenttestsarangeofliteracyknowledgeandskillsacrossthepre-literacy,emergingandbasicliteracyranges.Knowledgeandskillsincludeletterrecognition,phonologicalknowledge,printconcepts,oralliteracy,verbalcomprehension,initialsoundsandletters,readinghighfrequencywords,verbalandwrittengrammar,writinghighfrequencywords,readingfluency,copyingandspellinghighfrequencywords.

PupilHausaLiteracyassessmentforP2

TheHausaLiteracyassessmentisdesignedtotestthesameliteracyknowledgeandskillsastheEnglishLiteracyassessment.Itemsarenotmerelytranslated,butratherparallelitemsaredevelopedtotestsimilarconceptswhenappliedtotheHausalanguage.

Teacherquestionnaire

Theteacherquestionnairecollectsdataonteachers’backgroundcharacteristics,absenteeism,trainingandremuneration.Italsoincludesquestionsonmeetingsandsupervisionfromtheheadteacherandamoduleonteacherperceptionsandattitudes,whichwillbeusedtodevelopameasureofteachermotivation.

Teacherknowledgeandskillsassessment

Theteacherknowledgeandskillsassessmentisdividedintothreesections,collectivelycomprising30items,includingmultiplechoice,shortresponseandlongresponseitems.ThisincludesmarkingpupilresponsestoHausaliteracyquestions,indicatinggradelevelsforitemsbasedoncurriculum,developingananswersheetforareadingtestaimedatGrade2pupils,answeringa


©EDOREN 21

3.2.6 Surveyfieldwork

3.2.6.1 Surveyimplementationandmanagement

ThesurveyfieldworkwasimplementedbyOPMNigeriabetween19October2015and20November2015.Itwasprecededbyathree-weektrainingperiodforsurveystaff(seebelow).

ThesurveystaffwererecruitedbyOPMNigeria.Statesurveycoordinatorsandqualityassurancemonitorswereselectedfromapoolofhighlyexperiencedcandidates.TheselectionwasmainlybasedontheirexperiencewithothereducationsurveysinnorthernNigeria.82Otherstaffwereselectedthroughcompetitiverecruitment,whichincludedcanvassingthepotentialcandidates’CVsandface-to-faceinterviews.SpecialattentionwasgiventoexperienceinconductingsurveysinNigeria,withadditionalemphasisgiventoanyeducationresearchexperience.Theselectionprocessalsotookintoconsiderationthegenderbalanceoftheteams,andespeciallytheirlocallanguageproficiency.AllrecruitedinterviewerswereproficientinbothspokenandreadHausa,whichwasthemainlanguageusedforsurveyadministration.OtherlocallanguagerequirementswerealsotakenintoconsiderationwhenselectingteammembersasinsomeareasHausaisnotthemainlanguagethatisspoken.Inselectinginterviewers,specialattentionwasgiventotheknowledgeofArabic,Fulfulde,NupeandGbagyilanguageproficiencyastheselanguagesareusedinthesurveyedareas.

ThesurveywasadministeredusingCAPI.Datawerecollectedusingultra-portablecomputertabletsandtheCSProsoftwareplatform.

ThesurveywassupervisedbytheEDORENSurveyTechnicalLeadthroughoutthesurveyimplementationanddatacleaningprocess.TheEDORENSeniorEducationSpecialistandQuestionnaireDevelopmentExpertattendedthetraining.TheEvaluationTeamLeaderwasinvolvedinanydecisionthataffectedthesurveydesign:forexample,thereplacementofsampledIQSsduetochallengeswiththeIQSsampleframe.

82Statesurveycoordinatorsandqualityassuranceofficerswereinvolved,amongothers,indatacollectionaspartoftheTDPevaluation,ESSPINCS2andChildDevelopmentGrantProgrammeevaluation.SeveralofthemhadparticipatedintheGEP3instrumentpre-testing.

readingcomprehensionquestion,andmakingjudgementsaboutpupils’writing,includinggrammar,spellingandadditionalsupportneeded.

Teacherclassroomobservation

Theclassroomobservationrecordsthefrequencywithwhichtheenumeratorobservesspecificteachertalk,teacherlanguageuse,teacheractionandpupilaction.Theenumeratoralsorecordsteacheractionatthestartofthelesson,actionattheendofthelesson,theresourcesusedandsubjectobserved.

Headteacherquestionnaire

Theheadteacherquestionnaireincludesseveralsectionsrelatingtobackgroundinformationontheschool,basicinformationabouttheschool,schoolmanagementandmonitoring,attendanceandenrolment,attitudestowardsintegrationforIQSsandschoolinfrastructure.


©EDOREN 22

3.2.6.2 Surveytraining

Toensurethatallsurveyteamscoulddotheirworkeffectivelywhilstmaintaininghighqualitystandardsallsurveyfieldstaffwereintenselytrainedbetween28Septemberand17October2015inAbuja,includingpracticefieldvisits.Thesurveytrainingwasdividedintotwoparts:thefirstwasforthestatesurveycoordinatorsandqualityassuranceofficers,whichlastedfrom28to30September2015,whilethesecondwasfortheenumerators,from02to17October2015.Thelatterwasfurthersplitintotwobatchessoastoenabletheenumeratorstohaveadeepunderstandingofthesurveyinstrumentsandprotocols.Attheonsetofthetrainingtherewasnodistinctionbetweentheroleofthesupervisororthatoftheenumerators.Alltrainingparticipantsweretrainedasequals.Uponcompletionofthetraining,themostexperiencedandbestperformingenumeratorsweredesignatedasteamsupervisors.Specialsupervisorbriefingsessionswereheldwiththeminordertostrengthentheirunderstandingoftheirrolesandresponsibilities.

Adetailedtrainingmanualwaspreparedforthetraining.Duringthetrainingthemanualwasupdatedbasedonthefeedbackreceivedandclarificationsmadeduringthetraining.

Insummary,themainsubjectscoveredbythetrainingsyllabuswerethefollowing:

• OverviewofGEP3:GEP3’sobjectives,implementationstates,itsduration,anditsrelationshipwithEDORENandUNICEFwaspresentedtotheparticipants,forthemtoreinforcetheirunderstandingoftheprogramme.

• Surveyprotocoloverview:thesurveyandschoolselectionprotocolswereexplainedindetailandincludedin-depthexplanationsofthedifferentinterventiontypesandschooltypesunderGEP3,inorderforthefieldteamstothoroughlyunderstandthecontextoftheirdatacollectionenvironment.

• Communication(reportinglineandfeedback):theflowofcommunicationbetweenstatecoordinators,qualityassuranceofficers,teamsupervisors,andenumeratorswasdiscussedintermsofreportingandroutinefeedback.

• Familiarisationwithsurveyinstruments:participantswereexposedtoeachofthesurveyinstruments,includingthemainquestionnaireandCAPI.

• Mentoringandongoingtraining:thiswasfacilitatedforstatecoordinatorsandqualityassuranceofficers,forthemtogainanunderstandingofhowtoprovidementoringandsupporttointerviewersduringfieldwork,exceptinthecaseofmajorissuesthatwouldexceedtheirknowledgeandauthority.

• Fieldworkmonitoringforms:Inordertomanagefieldworkeffectively,aswellastomonitoractivitiesconducted,monitoringformsweresharedwiththefieldteam.Theseincludedtheschoolcompletionforms,monitoringactivityformsanddailyreporttemplates.

• Qualityassuranceprinciples:Informationaboutfieldethicsandprotocolsthatguaranteedataqualityweresharedwiththefieldteam.

Thetrainingusedseveraldifferenttrainingapproachestoachievethemaximumtrainingeffect,suchasdetailedinstrumentreview,smallgroupworkandmockinterviews.Enumeratorswerealsoexposedtoreal-lifefieldexperienceearlyoninthetraining.Thefirstvisitwasorganisedwithinthefirstfivedaysoftraining,andwasfollowedupbytwomorefieldvisitsduringthetraining.Earlyexposuretoreal-lifefieldsituationscontextualisesthetopicsandapproachesdiscussedinclassandhelpsenumeratorstobettergraspconcepts.ThetrainingwasconductedexclusivelyusingCAPIversionsofthequestionnaires.Enumeratorswerealsotaughtaboutotherfunctionalitiesrelatedto


©EDOREN 23

computer-assisteddatacollection,suchashowtosynchronisefilescontainingfilledquestionnairestothecloudandotherITanddatamanagement-relatedissues.

3.2.6.3 Fieldworkorganisation

Afterbeingtrained,thesurveyfieldteamwasoriginallystructuredasfollows:

• fourteamscomprisedofonesupervisorandfourenumerators;• onefieldcoordinatorperstate(statesurveycoordinator);• onequalityassurancestaffmemberperstate;and• OPMNigeriacoreteammemberswerepresentinthefieldtoprovidesupport,supervision,and

mentoring.

However,basedonthespecificneedsandparticularitiesofeachstate,theteamstructurewasprogressivelymodified.Asaresult,oneteamfromBauchiandonefromNiger(seethebaselinesurveyoftheIQSSevaluation)thatwerefacingahighrateofschoolineligibilitywerereallocatedtoZamfaraandKatsinarespectively.Thus,bythethirdweekoffieldworkZamfaraandKatsinawereoperatingwithfiveteamseach.Eachteamwasalsoaccompaniedbyadriver.

TheSurveyManagerandFieldManagerwereresponsiblefortheoverallmanagementoftheGEP3datacollection.TheEDORENSurveyTechnicalLeadprovidedtechnicalsupportandoversight.Statesurveycoordinatorsfocusedonthestatelogistics,schoolpre-schedulingandarrangementoftheschoolvisitswhilethequalitycontrolteamwastaskedwithoverseeingprotocoladherenceandefficiencyofdatacollection.

Figure8: GEP3surveyteamstructure

3.2.6.4 Unitssurveyed

Usingtheeligibilitycriteriamentionedpreviously,theteamsurveyed1,389pupilsand299teachersfrom120publicprimaryschoolsinKatsinaandZamfara,withonlyonereplacementinZamfaraofa


©EDOREN 24

schoolthatdidnothaveaP2class(SeeTable5).83Morethan100%oftheschoolsamplesizewasachieved,whilethepercentageofsampledpupilsandteachersfromthesurveyedschoolsequals96%and84%,respectively.Additionally,inZamfaraandKatsina1,264pupilsand178teachersweresurveyed(88%and74%,respectively,ofthetargetedsamplesize)in120IQSs.84Therelativelylowerpercentageofteacherssurveyedcomparedtothetargetedsamplesizewasduetothefactthatsomeschoolsdidnothavethetargetednumberofteachers,whichwasparticularlythecaseintheIQSsinKatsina.

22%oftheoriginallysampledIQSshadtobereplaced,mainlyduetotheIQSsnotbeingconsideredeligibleforthesurveygiventhestudyuniverse.85ReplacementsweremostlyduetotheIQSsnotteachingnon-religioussubjectsyet/anymore,ortheirbeingrecentlyintegrated.86Asexplainedabove,areplacementprocessinlinewiththesampledesign—avoidingtheintroductionofbiasinthisprocess—wasplannedfor,todealwiththechallengeswiththesampleframe.OnlyoneIQSproprietorrefusedtobepartofthesurveyandhadtobereplacedaccordingly.

Table5: Summaryofthesurveyachievement

Publicprimaryschools IQSsKatsina Zamfara Katsina Zamfara

Schools

Targetsampleschools 60 60 60 60

Totalschoolsurveyed 60 60 65 63

Originallysampledschoolssurveyed 60 59 50 44

Replacementschoolssurveyed 0 1 15 19

Totalschoolscontacted 60 61 90 87

Contactedschoolthatwerenotsurveyed–reasons

Recentlyintegrated 8 1

Non-integrated 3 20

Non-existing 3 0

Nogirls 0 1

Other 1 11 2

Percentageofsampleschoolssurveyed 100% 100% 110% 102%

83ItwasreportedthattheschoolenrolledpupilsonlyonceeverytwoyearssotherewerenocurrentP2pupilswhentheteamvisitedtheschool.84EightextraIQSsweresurveyedbutthesehavenotbeenincludedinthestudybecausethenumberofstudytreatmentschoolshadbeenagreedwiththeimplementingpartner.85ForanIQStobeconsideredeligible,ortoqualify,forthesurveyitwasrequiredthat:1)theschoolmusthaveaP2equivalent;2)itmusthaveteachersofnon-religioussubjectteachingP1–P3;3)itmusthavebeenintegratedforapproximatelyayear;and4)itmusthavefemalepupils(wedidnotimposethecriterionthattheIQShadtohave40%girlsenrolledbecausethiswashardtoverifyandmayhavebeentoorestrictivetoattainthesamplesize).86ThelengthofintegrationintheIQSswasparticularlyanissueinBatsariLGAinKatsina,whereapproximately80%ofschoolsintegratedin2015.Inthiscasetheeligibilitycriteriaofone-yearintegrationwasappliedmoreflexiblybysurveyingtheIQSsthathadactuallystartedintegrationinJanuary2015–hencehaving10monthsofintegration.


©EDOREN 25

Pupils

Targetsamplepupils 720 720 720 720

Percentageoftargetpupilssurveyed 96% 97% 84% 92%

Teachers

Targetsampleteachers 180 180 120 120

Percentageoftargetteacherssurveyed 83% 83% 59% 89%

Source:Fieldworkreports

3.2.6.5 Interactionwith,andinterferenceby,externalbodies

InteractionbetweenUNICEFandGEP3teamsinthestatesandthesurveyorganisationwasestablishedveryearlyintheprocess.TheGEP3state-levelofficialsparticipatedintheenumeratortrainingfortwodaysinordertosharetheirexperienceaswellastheirexpectationsregardingtheprotocolandconduct.Duringthetrainingperiodstatecoordinatorsestablishedarapport.Ineachstate,thestatesurveycoordinatorsharedthesurveyworkplanwiththeUNICEFfocalpersonandotherfieldmonitoringpersonnel,suchaslocaleducationauthorities,SUBEBs,andSAMEs.Thisensuredthattherelevantgovernmentofficersagreedtoandsupportedthedatacollection,andintroducedthesurveyteamtotheschools.ThesurveyteamreceivedformallettersofintroductionfromtheLGEAandUNICEF.

Insomecases,representativesofgovernmentagencieswerefoundtohavevisitednon-eligibleIQSsbeforethestatesurveycoordinatordid,toupdatetheschoolrecordsand/ortoputinplaceteachersofnon-religioussubjects.ThiswasparticularlyexperiencedinBunguduLGAinZamfara.Interferencefromeducationofficerswasalsofacedbythefieldteam.Inoneoftheschools,forexample,theeducationofficertriedtoinfluencetheteamtosurveyaschoolthatwasfoundtobeotherwisenoteligible.Anotherexample,occurredinanLGAwheretheeducationofficertriedtomakesurethataschoolwassurveyedeventhoughitwasanadulteducationschool.

3.2.6.6 Fieldworkchallenges

Themostrelevantchallengesfacedduringfieldworkaresummarisedbelow:

• TeachinghoursatIQSs:IQSsdonotusuallyfollowtheirtimetablesstrictly.Inaddition,someclassesaretaughtearlyinthemorningandsomelateatnight.Furthermore,mostoftheIQSsinthesurveyedstatesoperateforjusttwohours.Thisrequiredthesurveyteamstomakedetailedenquiresregardingtheday’splanandfortheteamstosetoutveryearlyorlate,oftenadministratingtheheadteacher,teacherandCBMCinterviewbeforethearrivalofthepupilsforsampling.

• Distanceanddifficultterrain:AlargenumberoftheIQSsarelocatedfarawayfromthenearestmetropolitanareasotheteamshadtosetoutveryearlyinordertogettotheschoolattherighttime.Moreover,mostofthecommunitiesdidnothaveaplacewheretheteamcouldspendthenight,whichmeanttheyhadtoreturnlateandhadtostartmovingtoanotherschoolthefollowingdayasearlyaspossible.

• Pre-appointment:SchedulingappointmentswiththeIQSswithoutaphysicalpresencewasalmostimpossible,asmostofthecontactnumberswereusuallynotreachableorwereincorrect.


©EDOREN 26

• Harvestseason:Insomecommunities,pupilsdonotattendschoolduringharvestseasonbecausetheyareengagedattheirparents’farms.

• IneligibleIQSs:Asdiscussed,anumberofIQSswereconsideredineligibleforthesurvey.Thesurveyteamidentifiedsomecasesinwhichproprietorswerealteringdocumentstopretendthattheirschoolswereeligible,whentheywerenot.Sometimes,anentirecommunitywasinvolvedinthismanipulation.

• Pupilsampling:InIQSstherewereinstanceswhereteacherstriedtoaddpupilstothesurveywhowereinothergradesorwhoattendedpublicprimaryschoolsfortheirnon-religiouscurriculumandwereonlytakingreligiousclassesattheIQS.

• Changeinschoolname:Duringschoolvisitsitwasobservedthatsomeschoolshadanamethatdifferedfromwhatwascapturedinthesampleframe,eitherduetoachangeintheschoolnameorwronglyspeltnames.

3.2.7 Dataqualityassurance

3.2.7.1 QualityassurancewithinaCAPIquestionnaire

Computer-assisteddatacollectiongenerallyimprovesthequalityofdatacollectedinasurveybyprovidingconsistentdatalinkage,automatedrouting/skips,andconsistencycheckswhichcanprovideinstantfeedbacktothefieldworkers.CAPIfurthermorereducesentryerrorsandfieldworkers’potentialtocheat.Decreasedinterviewtimealsoincreasestheefficiencyoffieldwork.Immediateavailabilityofdataallowsbetterorganisationandclosermonitoringoffieldwork,andenablesdatamanagerstoprovidetimelyfeedbacktothefieldteams,whichhelpstopreventfutureerrors.

Consistentdatalinkagewasmaintainedthroughoutthedatacollectionbyencodingthedatastructureintotheinstrumentsandhardcodedlistsofidentifiers.Thisisparticularlyimportantincomplexsurveyswithmultipleinstruments,suchasthisone.Thisallowsforsoundmappingandlinkingofdifferentlevelsofobservationsorcases,aswellasearlyidentificationofmissingcases.

Automatedskipsandroutingbehaviour,aspartoftheCAPIquestionnaire,ensuredthatonlycorrectquestionswereadministeredandthatnoanswersweremissing.Interviewerswereabletofullyfocusontheinteractionaspectsoftheinterviewandthuswereabletoestablishacloserbondwiththerespondent.Automatedskippingpatternsincreaseinterviewingspeedandallowmorecomplexskippingpatternsthatcansourceinformationfromallpartsofthequestionnaireinordertoadministerveryspecificsetsofquestionstailoredtoveryspecificcases.

Dataconsistencycheckswerebuiltdirectlyintothesurveyinstrumentstoprovidein-interviewfeedbacktothefieldworkers,allowingthemtoaddressinconsistenciesastheyaroseduringinterviews.Consistencychecksrangedfromsimplequestionspecificchecks,likerangechecks,tocomplexcheckscomparinginformationacrossdifferentsections.

CAPIreducessurveyerrorsandeliminateserrorsduetodataentry.TheuseofanelectronicversionofthequestionnaireintheGEP3baselinesurveymeansthatdatawereenteredduringtheinterviewitself,thusmakingseparatedataentryobsolete.TheCAPIquestionnairestrictlyenforcedtheprescribedquestionnaireroutinganddatavalidationforallresponsestosurveyquestions.Theinterviewersthusonlyposedthequestionswhichweremeanttobeaskedandcodedtheresponse


©EDOREN 27

onapre-definedscale.Duetothefactthattherewasonlyoneentry(insteadofthethreeinpaper-basedsurveys)thepossibilityofkeyingerrorswasgreatlyreduced.

CAPIalsoensuredtimelyfeedbackfromthefieldtothecentraldatamanagementteam.Thecompletedinterviewfilesweretransmittedfromthefieldviainternetdaily,ifpossible,andwerecentrallyexportedtostatisticalsoftwarepackages.ThesurveyanddatamanagementteaminAbujathushadtheabilitytocloselymonitorfieldworkandtoprovidetimelyfeedbacktofieldworkers,thuspreventingfutureerrorsfromhappening.Automatedroutingandpre-codedanswersincreasedtheinterviewingspeed,reducingrespondentfatigueandtheoveralltimerequiredforthefieldwork.

3.2.7.2 Qualityassuranceinthefield

Aqualityassuranceteam,oneineachstate,wasconstitutedtoperformin-fieldmonitoringandspotchecksonthedatacollectionteam.Duringthedatacollection,thisteamvisited138selectedschoolswheretheycarriedoutmini-surveysthroughliveobservationsandalsoconducted79phone-inbackcheckswiththeheadteacher(Table6).

Table6: Observationsfromearlylearningintervention

STATE/LGALiveobservations Backchecks

IQSs Publicprimaryschools IQSs Publicprimary

schools

Katsina

Batsari 1 4 5 12

Kankia 11 10 6 5

Rimi 9 12 8 7

Total 21 26 19 24

Zamfara

Bungudu 7 13 3 4

KauraNamoda 14 8 1 0

Tsafe 13 13 2 4

Total 34 34 6 8

Theliveobservationsassessedthewayinwhichthesurveyinstrumentswereimplementedinthefield.Sincealltheinformationwasenteredintoaqualityassurancedashboard,thesurveyandfieldmanagercouldmonitoronacontinuousbasistheperformanceoftheteam,theprogressmade,andthechallengesfacedattheschoollevel(i.e.noregister,pupils’arrivaltime,etc.).Therelevantinformationwasalsopassedtothestatesurveycoordinatorsandsupervisorsofthefieldteams,inorderprovidefeedbacktothefieldteamandtotakecorrectivemeasuresalmostimmediately.

Inaddition,afterthefieldteamhadleftaschool,thequalityassuranceteamphonedrandomselectedheadteachersandteachersandundertookashortquestionnaire.Thiswastocheckwhethertheinterviewswerecarriedoutproperly,withrespecttotherespondents,andinaccordancewithbothqualityandethicalstandards.


©EDOREN 28

Detailedbelowisasummaryoftheoversightfunctionsthatthequalityassuranceofficerscarriedoutduringthefieldexercise:

• ensuredsamplingandallprocedureswerestrictlyfollowed;• conductedspotchecksforteamsandensuredtheywerewheretheyplannedtobe;• observedongoinginterviewsandaddressedissuesthatrequiredattention;• ensuredthattherewasacleardistinction,amongtheteam,betweentheperiodinwhichthe

programmewasintroducedtotheschoolauthorityandtheactualtimetheschoolbegantoteachnon-religioussubjectsalongsidereligioussubjects;

• performedrandombackchecksvisitstoschoolswherethesurveyhadalreadybeencompleted;and

• gavecontinuousfeedback/refreshertrainingwithstateteamswherechallengeswereencountered.

Thequalityassuranceteamreportedtheirfindingstothestatesurveycoordinators,aswellastothesurveyanddatamanagementteaminAbuja.Togetherwiththestatesurveycoordinatorstheydebriefedthefieldteamsontheirfindingsandbroughtattentiontotheirfindingsinordertoimprovethequalityofthedatacollection.

Italsoneedstobestressedthatsomeofthequalityassurancefunctionsofboththestatesurveycoordinatorsandindependentmonitorswashamperedbytheunanticipatednecessityofactivelyanddiligentlyscreeningtheschools,especiallyIQSs,foreligibility.Thescreeningprocessmostoftenhadtobedoneinpersonandontheground,asthepoortelephoneconnectionsinthemoreremoteareasoftenpreventedcontact.

3.2.7.3 Qualityassurancepost-datacollection

TheCAPIdatacollectionapproachrequiresonlyveryminorpost-datacollectionprocessingandediting.CAPIenforcedverystrictvalidationrulesatinterviewtime,whichwereembeddedintheelectronicsurveyquestionnairesthemselves.Theroutingofthequestionnaireswasalsostrictlyenforced,i.e.itwasmandatoryforallquestionstobefilledandtheinterviewercouldnotproceedwiththeinterviewunlesstheinformationprovidedcorrespondedtoallvalidationrulesimposedonit.

Thepost-datacollectionprocessingdid,however,ensuretheconsistencyofalltheidentifyinginformationforeachinstrument.Asmultipleinstancesofthesameinstrumentwerefieldedateachschoolandtheirwithin-schoolidentificationwasassignedmanuallybytheinterviewers,theconsistencyoftheseidentificationelementshadtobeconsolidated.Fieldworkmonitoringformsanddailyfieldreports,aswellinterviewtimedata,wereusedtoreconcileanyinconsistencies.Theschool-levelidentifierswerepre-loadedandhardcodedintothequestionnairesthemselvesinordertoavoidduplicationormis-assignedIDs.

Allthedatawerealsocheckedforcompletionagainstsecondarysources,suchassurveymanagementandsamplecompletionforms.Thisprocessensuredthatallofthecollectedinformationwascontainedinthefinaliseddatacompilation.

Theactualcompilationofseparatedatarecordswasachievedduringtheactualdatacollectionprocess.ThedataweretransmittedfromthefielddailyandthedatamanagersattheOPMNigeria


©EDOREN 29

centralofficeinAbujacompiledtherecordsintoworkingdatabases.Afterthecompletionofthedatacollection,allofthecollecteddatawerecollatedinanalysis-leveldatafiles.Thedatabasewasformattedandreshapedaccordingtothespecificationsforanalysis.

3.2.8 Ethicsandinclusion

Anyevaluationmustabidebyrelevantprofessionalandethicalguidelinesandcodesofconductforindividualevaluators.Evaluatorsmustbemindfulofgenderroles,ethnicity,ability,age,sexualorientation,languageandotherdifferenceswhendesigningandcarryingouttheevaluation(OrganisationforEconomicCo-operationandDevelopment)(OECD)DevelopmentAssistanceCommittee(DAC)Standards,2010).DFIDhasdrawnupitsownprinciplesforensuringhighstandardsofethicsinresearchandevaluations,andthesewereadheredtoaspartoftheGEP3evaluation(DFID,2011).Theseinclude:

i. researchersandevaluatorsareresponsibleforidentifyingtheneedforandsecuringnecessaryethicalapprovalforthestudytheyareundertaking;

ii. researchandevaluationmustberelevantandofhighquality,withaclearvalue;iii. researchersandevaluatorsshouldavoidharmtoparticipantsinstudies;iv. participationinresearchandevaluationmustbevoluntaryandfreefromanyexternal

pressure;v. theconfidentialityofinformation,privacyandanonymityofparticipantsmustbeensured;vi. culturalsensitivitiesmustberespected;vii. UKhumanrightslawmustbeadheredto;viii. allresearchandevaluationresultsmustbepublishedandcommunicated;ix. researchandevaluationmustbeindependentofthoseimplementingtheprogramme;andx. thereshouldbeaparticularemphasisonparticipationbywomenandothersociallyexcluded

groups.

Theseethicalprinciplesareimportant,andrunthroughtheevaluationdesign.Oneareaofparticularimportanceforthisevaluationwasensuringthatchildrenwereaskedage-appropriatequestionsandthatappropriateconsentwassought.UNICEFguidelinesonconductingresearchwithchildrenwerefollowed.87Theevaluationpaidparticularattentiontothemother-tonguelanguagewheninterviewingchildrenofayoungageinordertoguaranteechildren’sunderstandingofthedatacollectionprocess.Furthermore,datacollectionwassensitivetochildren’sdisabilities,imposingspecificprocessesfortheinclusionofdisabledchildrenwhilebeingsensitivetotheirdisability.SurveyprotocolswereapprovedbyOPM’sEthicalReviewCommittee(ERC)(seeAnnexD)andaresearchethicsrequestwassubmittedtotheNationalResearchEthicsCommitteeinNigeria(NHREC).

3.2.8.1 Permits,consent,confidentialityanddatasets

Noincentivesweregiventorespondentsforparticipationinthestudy.Eachschoolthatwaspartofthequantitativebaselinesurveyreceivedasmalleducationalgift(e.g.aninflatableglobe)andparticipantsinthelearningassessmentsreceivedrefreshments.

87Grahametal.(2013)EthicalResearchInvolvingChildren.Florence:UNICEFOfficeofResearch–Innocenti.


©EDOREN 30

Consentwassoughtfromboththeheadteacherandindividualsparticipatinginthestudy.Descriptionsofthestudypurpose,therequestedinvolvementoftheparticipantandtheexpectedtimefortheadministrationofeachinstrumentwasprovidedtoeachparticipantpriortoseekingconsent.

Allpersonaldatacollectedaspartofthissurveyareonlyavailabletoauthorisedindividualsforanalyticalpurposesandarehandledusingdataprotectionbestpractices.Eachrespondenthasbeenassignedauniqueidentifier,whichwasusedintheanalysisofthedata.Allcleaned,anonymisedanddocumentedbaselinedatasetswillbemadepublic(subjecttoDFIDapproval)toenablenationalresearchers,researchstudentsandothereducationstakeholderstoaccessandusetheevaluationdatatoconductadditionalanalysisandresearch.

NHRECexemption

AnapplicationoutliningtheGEP3EvaluationEthicalStandards,informationonpermits,consent,confidentialityanddatasets,theprovisionofallparticipantinformationforms,consentformsandinstrumentintroductions,protocolsforworkingwithchildrenandspecificprotocolsfortheparticipationofchildrenwithdisabilitiesinthesurveywassubmittedtotheNHRECon29September2015.On07October2015anoticeofresearchexemptionwasprovidedtoEDOREN.TheexemptionletterisprovidedinAnnexD.

OPMethicscommitteeapproval

ApartialapplicationwassubmittedtotheOPMERCinMay2015forthequantitativeresearchcomponentoftheevaluation.ThissubmissionwasconsideredbytheERCon15May2015.Arequesttore-submitwithadditionaldocumentationwasmadetotheevaluationteam.AfterthefinalisedtoolswereprovidedtotheERC,ethicalapprovalwasgivenon28September2015.PleaseseeAnnexDfortheapprovalletter.

3.2.8.2 Specificprotocolsfortheparticipationofchildrenwithdisabilitiesinthesurvey

Enumeratorswereinstructedduringthetrainingandintheenumeratormanualtomakeanefforttomakeallthepupilsfeelateaseandtohelpthemunderstandthatthiswasnotanexam.Enumeratorswereencouragedtoaskquestionsaboutthepupils’livesordayiftheywereveryshy.Enumeratorsweretolookforaquietandsomewhatisolatedplacetoadministerthetests,avoidingtheheadteacher’sofficebecausethepupilmightbeshyinsuchalocation.

Ifthepupilwasidentifiedashavingadisabilitythatwouldnotaffecttheirabilitytotakethetestenumeratorswereinstructedtomakesuretheywereabletogettothetestlocation.Forexample,apupilwhocannotwalkmaystillbeperfectlyabletotakethetest,butmayhaveneededtobeassisted–byusingawheelchairorliftingthem–toreachtheplacewherethetestwasadministered.

Severalquestionswereincludedinthepupilquestionnairetocheckifthechildhadanydisabilitiesthatcouldimpedetheirabilitytoundertaketheassessment.Theseincludedaskingifthechildcouldhear,speak,writeandsee.Ifasampledchildhadanydisabilitywhichwouldnotaffecttheirabilitytotakethetest,theywerenotexcluded.Ifthepupilcouldnothearenumeratorswereinstructedtoendtheinterview.Inallothercases,enumeratorscontinuedwiththeinterview,andCAPIautomaticallyskippedanyquestionsthatthechildwouldhavebeenphysicallyunabletoanswer.


©EDOREN 31

3.2.8.3 Sensitivitytogenderandequity

Sensitivitytogenderandequitywithintheevaluationdesigncanbedividedintofivemaincategories:

• evaluationfocus;• nobiasinthemeasurement;• detectingdisaggregatedoutcomesbygroup;• seekingexplanationsforobserveddifferencesinoutcomes;and• focusingonpupilsatthelowendofthelearningdistribution.

Evaluationfocus:Allaspectsoftheevaluationincludeafocusongenderandmarginalisation.Inaddition,thefocusonIQSsispartlybecauseIQSSismeanttobeaninclusivestrategytoreachouttomarginalisedchildreninrurallocations.AlargenumberofchildreninnorthernNigeria,particularlyfromruralareas,donotattendformalschools,butdoattendacentreofferingQur’aniceducation.Therefore,comparisonsofdifferencesbyschooltypeareaprimaryfocusinevaluatingtheextenttowhichGEP3hascontributedtoimprovedteachingandlearninginIQSs.

Nobiasinthemeasurement:Genderbiascanaffectlearningassessments,andhappenswhenanitemisinterpreteddifferentlybytest-takersand,thus,‘whatwearetesting’isdifferentforgirlsandforboys.Throughtheuseofpsychometricanalysis,thisbiascanbeidentifiedanditemsthatarebiasedinthiswaycanberemoved.ItisessentialfortheGEP3evaluationsthatatestoflearningoutcomesdoesnotdiscriminatebetweengirlsandboys–meaningthatresultsbetweengenderscanbereliablycompared.Inotherwords,throughtheuseofpsychometrictechniques,wecanensurethataboyoragirlwiththesamelevelofliteracy(withallotherthingsbeingequal)willreceivethesamescoreonascaleoflearning.ARaschanalysisoftestitemsusedinotherEDORENevaluationswasundertakenin2014(McGrane,2014)andfoundthattheitemsdonotdiscriminateonthebasisofgender.Newitemsunderwentthesameanalysis.

Detectingdisaggregatedoutcomesbygroup:Inordertoanalyseoutcomesbysocialgroup,dataonpupil,schoolandcommunitycharacteristics(suchasgender,socioeconomicstatus,geographiclocation,etc.)mustbecollectedandlinkedtooutcomedata.Theremustthenbesufficientstatisticalpowertodisaggregateoutcomesbygroup.TheGEP3evaluationcollectedcontextualdataatthepupil,schoolanddistrictlevelswithsufficientstatisticalpowertoanalyselearningoutcomesbygender,whichisofprimaryimportanceinevaluatingtheextenttowhichGEP3hasimprovedlearningoutcomes,especiallyforgirls.

Explainingobserveddifferencesinoutcomes:GirlsinnorthernNigeriafaceamultitudeofbarrierstoachievingliteracy,languageandnumeracyskills.Anydifferencesinlearningoutcomeswereanalysedusingtheapproachesoutlinedabove.However,thereasonsforthesedifferencescannotbeobservedpurelybasedonoutcomedata.Somefactorsareeasilyobservable,suchasaccesstospecificresources.Ifidentifiedinadvance,differencesininputstoeducationbasedongendercanbeinvestigatedthroughthequantitativesurveyandtheimpactonlearningoutcomesbygroupcanbeanalysed.However,otherfactorsarelatentorcanonlybeobservedbygainingacontextualunderstandingofcommunitiesandschools.

Focusingonpupilsatthelowendofthelearningdistribution:Inordertoensureanequityfocusismaintainedtheevaluationofchangesinlearningoutcomeswillbeviewedthroughanequitylens.Purelycomparingmeanincreasesinscalescoresatbaselineandfollow-upsurveyscanobscurefrom


©EDOREN 32

viewthelearningofthemostmarginalisedchildren.Theanalysesofincreasesinlearningoutcomeswillalsoincludeananalysisofthegapbetweenthelowestperformingpupilsandtheexpectationsofthecurriculum.Theanalysiswillinclude:(1)thepercentageofpupilsachievingbelow-curriculumexpectations;and(2)thegapbetweentheactualskillsbeingachievedbypupilsatthelowerendofthedistributionandtheskillsexpectedbythecurriculum.

3.2.9 Outcomemeasurementandconstructs

3.2.9.1 Pupilassessmentinstruments

EnglishliteracyandHausaliteracyassessmentswereconstructedfollowingthefivestepssetoutbelow,includingclarifyingconstructs,testtargeting,administration,psychometricanalysis,drawingbenchmarksandsecondarydataanalysis.

Clarifyingconstructs

Thefirststepinundertakingalearningmeasurementexerciseistodefineexactlywhatistobemeasured.Definingexactlywhatitisthatweexpectchildrentoknowandbeabletodoisattheheartofthemeasurementoflearning.Literacyandreadingarequitedifferentconstructs,asarenumeracyandmathematics.Atitssimplest,tobeliteratemeanstobeabletoobservesymbolsorsignsfromapageandtoascertainmeaningsthatarestandardisedtosomeextent.Italsomeansbeingabletoproducetextbywritingthesamesymbolsorsignsinordertorecordmeaning.So,eveninitsmostbasicofforms,literacyisalotbroaderthanreading.Internationalgold-standardassessmentprogrammes,suchasTrendsinInternationalMathematicsandScienceStudy(TIMSS)andProgressinInternationalReadingLiteracyStudy(PIRLS),goevenfurther,incorporatinganelementofsocialcontextintotheirunderstandingsofliteracy.Similarly,theOECD’sdefinitionofnumeracyisbroaderthanbeingabletooperatemathematicalequations–rather,ithingesonthe‘abilitytoaccess,use,interpret,andcommunicatemathematicalinformationandideas,toengageinandmanagearangeofsituationsinlife’(OECD,2014).Thisdefinitionisbroaderthanunderstandingmathematicalconceptsbutalsoincludesbeingabletoapplytheminavarietyofknownandunknownsituations.

TheconstructstobemeasuredthroughthelearningachievementtestsintheGEP3evaluationareEnglishliteracy/language,Hausaliteracy/languageandnumeracy.TheGEP3evaluationusesthebroaderconstructsofliteracyandnumeracy,ratherthanreadingandmathematics,asdiscussionswithstakeholdershavehighlightedthattheprogrammeexpectschildrentoimprovelearninginareasthatfitintothesebroaderareasofliteracyandnumeracy–suchastounderstandtexts,usereadingtounderstandtheworld,drawinferencesandcommunicateinwriting,usemoneyineverydaylife,andreadaclock.

Targeting

Theflawedassumptionthattestedchildrenareabletoreadorwritealreadyhasbeenavoided.TestitemsweredesignedorselectedfromexistingEDORENteststomeasureskilllevelsbelow,atandabovetheskillsassumedtohavebeenreachedgiventhegradethechildisattending.

Themajorweaknessindatameasuringliteracyandnumeracyinlowandmiddle-incomecountriesisthatassessmentsmeasureskillsatlevelsthataretoohighformostofthechildrentakingthetests.ThiswasobservedinthecaseofMali,whereover80%ofstudentsinGrade2couldnotreadasingle


©EDOREN 33

wordinfournationallanguages,whileintheNigerianstateofSokoto81%ofthestudentscouldnotreadfullwords(GlobalEducationMonitoring,2012).Theseflooreffectsarenotproblematicifthegoaloftheassessmentistoestablishnationallearninglevels.Forevaluationpurposesitwouldnotbepossibletodetectgrowthinliteracyornumeracyunlessthosechildrenpreviouslyoutofrangeofthetestscomeintotheabilitylevelmeasuredbythetest.Forthosechildrenwhodonotcomeintorange,butwhoseliteracyornumeracylevelsdoimprove,theimpactoftheinterventionwillbeunderestimated.Therefore,theGEP3assessmenttoolsweredesignedtoensureitemdifficultymatchedpupils’ability.Theleveloftheassessmentandpupils’abilitieswereassessedduringthepilotingandadjustedbasedontheresults.

Astheassessmentsusedfortheevaluationareadministeredtoapanelofpupils,theassessmentsneedtobeabletocoverawidevariationinpupilknowledge(fromthelowestperformingpupilsatbaseline,tothehighestperformingpupilsatmidline–thefollowingschoolyear).Thisplacesagreatrequirementonthetesttocrediblymeasureknowledgeoveralargerangeofdifficulty,withoutmakingthetesttoolongforyoungpupils.ThiswillbealleviatedthroughtheuseofRasch(itemresponsetheory)modelling.Thetestswilluselinkitemstolinkthedifficultyofthebaselinetesttothemidlinetestandtoplacetheresultsontothesamescale(metric).TheEDORENmetricsteamhasalsoplacedthislearningontothesamemetricashasbeenusedforESSPINEnglishLiteracyandNumeracy.

Administration

LearningachievementtestsusedaspartoftheGEP3evaluationuseasetofassessmentitemsorquestionstoassesschildren’sliteracyand/orlanguageandnumeracyknowledgeandskills,basedonaone-on-oneindividualoralinteractionbetweenachildandanenumerator.

Psychometricanalysis

TheuseofRaschmodelling(IRT)increasestheamountofanalysisrequiredasmoresophisticatedtechniquesareused,ratherthanaddingupatotalnumberofitemscorrectinthetestandconvertingthenumberintoapercentagescore.Inthisregardtheevaluationteamfollowedthefollowingsteps:thefirststepwastotestthepsychometricpropertiesoftheitemstoensuretheywereusefulmeasuresofwhatpupilsknow.Thesecondstepwastoremoveanyitemsthatdidnotperformwellandwouldbiastheresultsiftheywerecountedintheanalysis.Inathirdsteptheteamrankedtheitemsaccordingtodifficulty.Thiswasdonebyapsychometrician,usingfit-for-purposesoftware.Thesoftwarethenalsorankedpupilsaccordingtotheirabilityandplacedthepupilsandtheitemsontothesamemetric.Thisisaprobabilitymodelaspupilsareplacedonthescaleaccordingtotheprobabilityofapupilansweringthecorrespondingitemcorrectly.

Proficiencybands

Informeddecisionswerethenmaderegardingthestandardsoflearning.Forexample,whatskillsdoweexpectpupilstohavetoconsiderthem‘literate’?Thisdefinesthelearningstandardacrossthemetricandcreatesaproficiencyband.Thepercentageofchildrenthatfallwithinthisrangeisthencalculated.

TheseproficiencybandsweredrawnthroughabenchmarkingworkshopundertakenwithTDP,GEP3,DEEPENandESSPINstakeholdersinNovember2015fortheEnglishliteracyandnumeracyassessments.FortheHausaassessment,thecut-offpointwasdrawntoreflecttheEnglishliteracyproficiencyrangesasmuchaspossible.inotherwords,theskillsthatfallwithineachproficiency


©EDOREN 34

rangeforHausaareassimilaraspossibletotheskillswithinthesameproficiencyrangeforEnglishliteracy.

Secondaryanalysis

Theprimaryanalysisproducestwovariablesforeachlearningconstruct,perpupil.Thefirstisascalescore.Thescalescoreisaprecisemeasureofwhere,alongtheachievementscale,thepupilsits.Thescalescoreisusefulforfine-grainedanalysislikemulti-levelmodelling,regressionsandcorrelations.Thesecondvariableiswhichproficiencybandthepupilfallswithin.Thisvariableisusefulfordescribingwhatlearningpupilshaveachievedandhowwhatchildrenactuallyknowandcandohaschanged,overthecourseoftime.

3.2.9.2 Teacherknowledgeandskillstheoreticalmodel

Teacherknowledgeandskillstest

AdaptedfromLeach(2002)andCogill’s(2008)modelsofpedagogy,theknowledgemodelsetoutinthefigurebelowconceptualisestheknowledgethatcontributestoobservableclassroompractices.Withinthemodel,therearethreetypesofknowledgethatateacherdrawsonwithinclassroompractice.Curriculumknowledgereferstoknowledgeofwhatshouldbetaughttoagroupofstudents,knowledgeofthenationalsyllabus,understandingoftheschoolandgrade-levelplanningdocumentsandknowledgeofexaminationscontent.Pedagogicknowledgereferstoknowledgeofthelearnersinthesetting,knowledgeofhowtoprovidetheconditionsthatenablepupilstounderstandandtheselectionoflearningandassessmentmaterials.Subjectknowledgereferstoknowledgeoftheessentialquestionsofthesubject,thenetworksofconcepts,thetheoreticalframeworkandmethodsofinquiry.

Figure9: Teacherknowledgemodel

Fusedknowledgethatcontributestoobservablepractices

Curriculumknowledge

Pedagogicknowledge

Subjectknowledge


©EDOREN 35

Testconstruction

Theteacherknowledgeandskillstestwasdividedintothreesections,collectivelycomprising30items,includingmultiplechoice,shortresponseandlongresponseitems.InsectiononeteacherswereaskedtomarkpupilresponsestoHausaliteracyquestionsandtoindicatethegradelevelatwhichtheanswershouldbeknownbypupils,asdefinedbythecurriculum.InsectiontwoteacherswereaskedtofillinmissinginformationonananswersheetforareadingtestaimedatGrade2pupils.Inaddition,teacherswereprovidedwithtwonewspaperarticlesandaskedtofillinmissinginformation,includingbasiccomprehensionquestionsandquestionsthatrequiredtheinterpretationofwordsandphrases.Sectionthreeaskedteacherstoidentifypoorandgoodpupilwork,andtoreviewpupilworkinordertomakejudgementsaboutpupils’writing,includingtheorganisationofthewriting,theuseofgrammar,punctuation,spelling,thepupils’abilitytoself-correctandreflectontheirwritingandthepupils’abilitytoformlettersandusespacesbetweenwords.Teacherswerethenaskedtodescribehowthatmightsupportthepupilsinimprovingtheirwriting.

Teacherresponsestotheitemsweremarkedandcross-checkedusingamarkingmatrixandscoringguide.Themarkingandcross-checkswerecarriedoutbytwosenioreducationspecialists.OnespecialistisapreviousdirectorinNigeria’seducationsystemandtheotherisaninternationaleducationspecialist.

Teacherresponseswereusedtodevelopeightareasofteacherknowledgeandskills:

1. syllabusknowledge;

2. abilitytoidentifylowperformers;

3. abilitytoevidencejudgementsanddiagnose;

4. abilitytobuildonstudentknowledge;

5. teacherwritingskills;

6. teacherGrade2Hausaknowledge;

7. teachercomprehensionskills;and

8. interpretingwordsandphrases.

Dataanalysisandsubscaleanalysis

Themainpurposeofthedataqualityandsubscaleanalysiswastoassessthequalityofthedataonteacherknowledgeandskillswithregardtovalidityandreliability,andtoestablishtwothresholdpointsforeachsubscale.

Thefirst,andlower,thresholdpointismeanttorepresentarudimentarylevelofachievement.Thismeansthatthereappearstobeevidenceofsomeabilitytounderstandandperformsomeoftheteachingtasksmeasuredbythesubscale.Thesecond,andhigher,thresholdismeanttorepresentacompetentlevelofachievement.Thismeansthatthereisevidenceofabilitytoconsistentlyunderstandandperformteachingtasksmeasuredbythesubscale.

ThereliabilityofthescaleswasexaminedusingCronbach’salpha.

Highlevelsofmissingdatawereobserved.Theamountofmissingdataisamajorconcern.First,becauseitmayindicatethatmanyoftheteacherswhoparticipatedintheprogrammewerenotabletodomanyofthetasksaskedofthem.Thesetaskswereintendedtorepresentthecoreelementsof


©EDOREN 36

whatteachersshouldknowandbeabletodoinordertoteachereffectively.Sotheconcernis,here,thattheteachersmaynotbeveryskilled.Thismaysuggestthatthetestwas‘toodifficult’.However,thetestwasdesignedaroundnotionsofwhataminimallycompetentteachershouldbeabletoknowanddo.Itisprobably,therefore,unhelpfultobeoverlyconcernedaboutthetest’sdifficulty.Therealissueisthecompetencyleveloftheteachers.

Thesecondconcernisthattherangeofanalyseswaslimitedduetothemissingdata,andhencethecapacitytodrawempiricallystrongconclusionswasimpeded.

Duetothehighdemandplacedononeassessment(tomeasuretherangeofknowledgeandskillsrequiredtodeterminethecompetenceofateacherineightkeyareas)somesubscaleshadfewitemsfromwhichtodraw.Alimitednumberofitemslimitedthereliabilityofascale.Inthecaseof(1)syllabusknowledgeand(2)abilitytobuildonpupilknowledge,thenumberofitemsavailableonwhichtobuildthescale(threeandfourrespectively)werenotsufficienttoyieldreliableresults.Itwasdecidedthatinordertoutilisethegathereddata,theanalysiswouldfocusonprovidingexamplesofitems,descriptionsofthekindsoferrorsteachersmadeandinformationonthepercentageofteacherswhorespondedcorrectlytospecificitems.

Withinthecontextofexploratoryresearchongroups(asopposedtoadiagnosticinstrument)aCronbach’salphaofover0.5–0.6isoftenarguedtobeacceptable(SeeSuhretal.,2009andYuandWatkins,2011).Basedonthis,theremainingsixsubscalesachievedacceptablelevelsofreliability.

Table7: Teacherknowledgeandskillsproficiencybands

SubscalesLowband

Noevidenceofskill

MiddlebandEvidenceof

rudimentaryskill

UpperbandEvidenceofcompetence

Abilitytoidentifylowperformers 0–2 2.5–7 Greaterthan7

Abilitytoevidencejudgementsanddiagnose

0–2 2.5–7.5 Greaterthan7.5andabove

Teacherwritingskills 0–2 2.5–3.5 Greaterthan3.5

TeacherGrade2Hausaknowledge

0–2 2.5–3.5 Greaterthan3.5

Teachercomprehensionskills 0–2 2.5–3 Greaterthan3

Interpretingwordsandphrases 0–1 1.5–2 2.5andabove

Teachermotivationscales

TheTeacherMotivationscalesweredrawnfromapreviousstudyundertakenbyEDOREN–theTDP.88Inthisstudy‘motivation’wasunderstoodas:

88ThissectionistakenfromaninternalOPMdocumentpreparedaspartoftheTDP.ThedocumentistitledNotesonmeasuringteachermotivationfortheTeacherDevelopmentProgrammebaselinesurveyandissignedanddatedSC[StuartCameron]20.8.2014.ThereferencelistprovidedinthisdocumentisincludedintheBibliographyforconvenience,intheeventthatfurtherinformationissought.


©EDOREN 37

…amixtureofperceivedefficacyandeffort/importanceattachedtoteachingwork;butisalsomeasuredlessdirectlythroughinterest/enjoymentandpressure/tensionitems.

Accordingtothisdefinitionamotivatedteacherisonewho:

• seesthemselvesaseffectiveandasmakinganeffort;• seestheirworkasimportant;• isinterestedinandenjoystheirwork;and• managesworkpressureandtension.

Theteachermotivationscalesareunderpinnedbyatheoryofactiontoexplainhowaprogrammeoperatestoincreaselevelsofteachermotivation.TheitemsincludedineachsubscalecanbefoundinAnnexE.

Thetheoryofactionisdepictedasacausalchainin


©EDOREN 38

Figure10.Accordingtothistheory,theGEP3interventionwillleadto:

• betterpedagogicandcontentknowledgeandskills,andbetterteacher-to-teacherinteraction(subscale6in


©EDOREN 39

• Figure10),whichwillinturn,leadto:• betteractualteacherefficacy,whichcanbemeasuredastheextenttowhichlearningoutcomes

improve,whichwillleadto:• increasedperceivedefficacy(subscales1and2in


©EDOREN 40

• Figure10)–thatis,theteacherswill(accurately)perceivethattheyhavebecomemoreeffective,whichwillinturnleadto:

• greaterinterestandenjoyment(subscale3),moreeffortbeingmadeandmoreimportanceattachedtotheirwork(subscale4),andlesspressureortensionexperiencedregardingtheirwork(subscale5).

‘Efficacy’canbeseenasperceivedrelativeinfluence,whichcanbedividedintotwoaspects:

• howimportantisteachingeffortvisàvispupilbackground(subscale1)?• howimportantisteachingeffortvisàvisschoolinfrastructureandprofessionalsupport(subscale

2)?


©EDOREN 41

Figure10: Theorisedcausalpathway(takenfromtheTDP)

Interestinand

enjoymentofteachingSubscale3

Teacher-to-teacher

interactionSubscale6

>Efficacyasateacher

>

Perceivedefficacyasateacher

Subscales1and2

EffortandimportanceSubscale4

Pressureandwork-relatedtension

Subscale5

Dataquality

Thequalityofadatasetcanbeinfluencedbytheamountofmissingdata.Therewasverylittlemissingdata:onaverage,only1.2%ofdataonthemotivationitemswasmissing.

Itisimportanttoconsiderthedistributionofvaluesforeachvariable.Highlyskeweddata–datathatpileupatoneendofthedistribution–arenotwellsuitedtocertainstatisticalprocedures,includingsomeofthoseplannedtobeundertakenintheanalysisoftheteachermotivationitems.Mostofthedatawerefoundtobehighlyskewed.Whileitisimpossibletoknowforsure,thesedistributionsaresuggestiveofcomplianceeffectsinatleastsomeofthesedata.Thus,cautionisrecommendedintheiruseandinterpretation.

Procedureswereexploredtotransformdatatoimprovetheirdistribution;however,theseprocedureswerenotusedastheydidnotimprovethedistributionmarkedly,andbecausesuchtransformationcanaddconsiderablecomplexitytotheinterpretationandreportingoffindings.

Analysis

ThedatawereanalysedusingexploratoryfactoranalysisinthecomputerapplicationSPSSStatistics.Theanalysiswasundertakenafterreversecodingnegativelywordeditems.Principalaxisfactoring,withorthogonalrotation(Varimax)wasused.Thisisaconservativeapproach,andonethatassiststheinterpretationofresults.ThereliabilityofscaleswasassessedusingCronbach’salpha.

Thevalidityofthescales–theextenttowhichtheyaremeasuringwhattheypurporttomeasure–seemslikelytobeacceptablegiventhattheyarederivedfromandare(moreorless)consistentwith


©EDOREN 42

thetheoreticalaccountof‘teachermotivation’thatshapedthedevelopmentoftheitems.However,itisprobablyadvisabletotreatthevalidityofthescaleswithsomecaution.

Thefactoranalysissuggeststhatfivescalesareretrievablefromthesedata.Table8showsthesubscalenameandthenumberofthesubscaleasusedinthepathdiagram(


©EDOREN 43

Figure10),alongwiththereliabilityofeachofthese‘new’subscales.89

Table8: Summaryofresultsofthefactoranalysis

Subscalenumberandnameusedinthepathdiagram Reliability

1:Importanceofteachingeffortvs.pupilbackground(perceivedefficacy)0.592:Importanceofteachingeffortvs.schoolinfrastructureandprofessional

support(perceivedefficacy)

3:Interestinandenjoymentofteaching 0.53

4:Effortandimportance 0.62

5:Pressureandwork-relatedtension 0.58

6:Teacher-to-teacherinteraction 0.72

Theresultsofthefactoranalysis,whilenotstrong,permitaninvestigationofthetheoryunderpinningthenotionof‘teachermotivation’,althoughanyfindingsarecarefullyphrasedbecausethesubscalesneedfurtherrefinement(asharperfocus).Theevidenceofcomplianceeffectsinsomeofthedataalsoneedstobeconsidered.

Allfiveelementsofthetheorydepictedinthepathdiagramhaveasubscalethatworkswellenoughtobeused.Here‘wellenough’meansthatthereisevidencethatthesubscaleisplausiblyuni-dimensionalandhasalevelofreliabilitythatisdefensible.

Thecorrelationsbetweenthesubscaleswereinvestigated.Sixofthepairsofcorrelationsareverylow(whichisdesirablegiventhattherotationusedinthefactoranalysiswasorthogonal).Theremainingfourpairshaveamoderatelystrongcorrelation,whichislikelytobeareflectionofthereliabilityofthesesubscales.

AnnexEprovidesasetofgraphsshowingthedistributionofvaluesforeachofthesubscales.

3.2.10 Limitationsandriskstothemethodology

Variouslimitationsandrisksneedtobetakenintoaccountintheabove-presentedmethodology:someofthesecanbeaddressedbycarefulimplementationoftheevaluation,whileothersconstituteriskstothemethodology.Table9reviewsthelimitationsandrisks.

Table9: Limitationsof,andrisksto,theevaluationmethodologyoftheearlylearningintervention

89Cronbach’salphaisusedasthemeasureofreliability.

Limitations/risks Whythisislimitingandwhatwehavedonetoaddressthis

Interventionnotprovidedasplanned

ItisassumedthatthesampledtreatmentschoolswillreceivetheRANAintervention,whilethecontrolschoolswillnot.Furthermore,thedesignassumesthatallprimaryschoolteachersinearlygradesandallIQSteachers


©EDOREN 44

teachingtheintegratedcurriculuminearlygradeswillbetrained.Iftheinterventionisnotconductedasplanned—forexample,sometreatmentschoolsorteachersarenot,orareonlypartially,exposedtotheintervention,orcontrolschoolsareincludedintheintervention—theabilitytoestimateimpactandcomparabilitybetweentreatmentandcontrolschoolsmaybereduced.Themitigationofthisriskofpartialcompliance,highlightedintheGEP3EvaluationFramework,requiresclosecoordinationbetweenimplementersandevaluators.Theimplementer,FHI360,haschangedtheinterventionroll-outtoschoolscomparedtoGEP3’sstrategypaper,whichwasinitiallygoingtoinvalidatetheevaluationdesign.Througheffectivecoordinationtheimplementerwaseventuallyabletoalignimplementationroll-outwiththeinitialplan.Nonetheless,adegreeofphasedroll-outisstillimplemented,whichwillresultinaslightnon-randomvariationwithinthetreatmentgroup.Giventhatschoolsthatformpartofthetwophasesofroll-outareclearlyidentifiedthiscanbetakenintoaccountintheanalysis.Withregardstoteachertraining,RANAhasconfirmedthatallearlygradeteachersintreatmentschoolswillbetrainedandmentored.Nonetheless,thebaselineexperiencehasdemonstratedthat,particularlyinIQSs,peoplethatarenotactuallyteachingintheschoolparticipateintraining.Therefore,itwillrequireclosemonitoringtotraintheactualtargetgroup.

Attrition

Sampledschools,teachersorpupilsmaydropoutofthesurveypanelbeforemidline,thuscreatingaproblemofmissingdata.Acertaindegreeofattritionistakenintoaccountinthesamplesizecalculation.GiventhatallstudyschoolsareGEP3schoolsweassumeverylimitedattritionattheschoollevel.OneIQSproprietorincludedinthetreatmentgrouphasindicatedthathemaystopintegrationevenbeforetheinterventionstarts.GovernmentandGEP3staffareconsultingwiththeproprietortomaintainhisinterest.Non-integrationwouldmeanthattheschoolfallsoutsidethetargetpopulation,whichwouldwarrantreplacementoftheschool.Attritionmayposearisktothecomparabilityoftheinterventionandcontrolgroupsifattritionissystematicallydifferentinbothgroups.Ifthecompetenceoftheteacherswhodropoutofthecontrolsampledifferstothatoftheteacherswhodropoutoftheinterventionsample,biasisintroducedinthecomparisonbetweenthetwosamples.Suchattritionishardtocontrolfor,butiftheattritionrateishighasampleofteachersthathavedroppedoutcouldbetrackedoutsideofschoolandsurveyed,iftheresourcestodosoareavailable.

Contamination

ContaminationreferstootherinterventionsaffectingtheRANAoutcomes,whichmaycreateaconfoundingfactorinthemeasurementoftheimpact.Thiscreatesaproblemtotheextentthatsuchcontaminationaffectstheinterventionandcontrolschoolsdifferently.GEP3willimplementotherinterventionsamongpublicprimaryschoolsinbothinterventionandcontrolgroups.Itisimportantthattheprojectdoesnotbiasinterventionimplementationtowardoneofthegroups.WeassumethattheriskinthisregardislowbecausethisisunderGEP3’scontrol.TheTDPandtheJollyPhonicsprojectarealsoprovidingteachertraininginpublicprimaryschoolsinKatsinaandZamfara.WhilesomedegreeofoverlapwithGEP3schoolsispossible,thereiscurrentlynoindicationthatitaffectsinterventionandcontrolgroupsinadifferentdegree,suchascouldcreateabiasinimpactmeasurement.ContaminationofthecontrolgroupmayalsohappenwhenRANAisimplementedincontrolschools.TheidentitiesofcontrolandinterventionschoolswerecommunicatedtotheRANAimplementingagency,andthiswastakenintoaccountintheimplementationroll-out.


©EDOREN 45

Spillovers

Spilloversoccurwhenpupilsorteachersfromsampledcontrolschoolsareaffectedbytheintervention,whichwouldcompromisethecontrolgroupasagoodcounterfactual.Spillovercouldhappenifinterventionteachersshareknowledge,skillsormaterialswithcontrolteachers.SincecontrolteacherswillnotbeincludedintheRANAclusters,nodirectinterventionspillovercantakeplace.BecauseoverlapbetweenGEP3andTDPinterventionswillbeavoidedinKatsinaandZamfara,wecanassumethattheRANAinterventionandcontrolteacherswillalsonotbeconnectedthroughTDPcommunitiesofpractice.However,theriskwillremainthatcontrolteacherswillbeincontactwithRANAinterventionteachersduetothegeographicalproximityoftheschools.Similarly,theheadteachersofcontrolschoolsmaypickupideasfromRANAiftheyconnectwithRANAinterventionschools.SuchexposuretoRANAwillbeindirect,whichwilldiluteitsimpact.Furthermore,baselinedatademonstratethatpupilsinIQSsmayalsoattendpublicprimaryschools.TotheextentthatpupilsfromcontrolIQSsattendclassesinpublicprimaryschoolsthatarepartofthetreatmentgroup,spillovercouldhappen.Whiletheriskinthisregardislikelytobesmall,acertainamountofriskremains.Atmidline,exposuretoRANAcanbesurveyedwithinthecontrolgrouptoassessthisissue.

Timingoftheimpactmeasurement

Impactwillbemeasuredatmidlineandendline.Midlineisscheduledforthethirdtermofthe2016–2017schoolyear,inordertoallowfindingstoinformscale-updecision-making.ThiswillexposeteachersandpupilstothreetofourtermsofRANAintervention.Giventhatteachercapacitydevelopmentconsistsofintensetrainingandmentoringthisperiodshouldallowforchangesinteacheroutcomestomaterialise.Sincepupillearningoutcomeschangemoreslowlylongerexposurewouldoffermoretimeforimpacttomaterialise.However,thishastobebalancedwiththetimingoftheevidenceneeds.Usingbaselinedata,weestimatethata3percentagepointnetincreasewouldbestatisticallydetectablewhenthedataofprimaryschoolsandIQSs,girlsandboys,areanalysedjointly.

Externalvalidity

Externalvalidityreferstotherepresentativenessofthefindingsbeyondthesurveysample.ThefindingswillberepresentativeforGEP3schoolswithintheLGAsselectedfortheRANAintervention.BypurposivelyselectingtheLGAsbasedonproximitytherepresentativenessforGEP3schoolsasawholeintherespectivestatesmaybeaffected.However,thetheory-basedapproachenablesustounpickhowandunderwhatconditionsimpactonlearningoutcomesoccurs,whichallowsformoregeneralisablefindings.

Evaluation-driveneffects

Beingpartofanevaluationcanchangebehaviourindependentoftheintervention.Forexample,teachersthatarepartofapanelsurveymayexertgreatereffortthantheywouldiftheywerenotselectedforinterviewing.Ifsucheffectsarecommontobothinterventionandcontrolgroups,theydonotaffectcomparabilitybuttheymayaffectthegeneralisabilityofthefindingstoothersituations.Sincethesurveyswillbeone-offeventsthattakeplacetwiceintwoyears,wedonotexpectastrongimpactonteacherorpupilbehaviour.TherandomisedevaluationdesignhasinfluencedtheformationoftheRANAschoolclusters,sincetheimplementingpartnerhadtoincluderandomlyselectedtreatmentschoolsintheclusters,whichmayhaveinfluencedtheproximityradiusofthecluster.TheextentofthispotentialeffectdependsontheproximityoftheGEP3schoolsintheRANALGAs.ByfocusingtheinterventiononalimitednumberofLGAsandstratifyingthesamplebyLGAs,theconcentrationofschoolsisenhanced.

Mixedmethods Amixedquantitativeandqualitativemethodsapproachoffersstrongaddedvalueintermsofdefiningandexplainingtheimpactofanintervention.Byusing


©EDOREN 46

atheory-basedapproachandaddingstate-levelqualitativedatacollectiontothequantitativemethods,theexplanatorypoweroftheevaluationisenhanced.However,theevaluationteamisnotabletocomplementquantitativedatacollectionatschoollevelwithqualitativeresearchatthesamelevel,whichwillconstraintherichnessoftheunderstandingoftheimpact.TheGEP3-ledqualitativeresearchmayprovideanalternativesourceofinformation,andqualitativeresearchcanstillbeconsideredduringthe2017–2020evaluationperiod.

Surveyresponseachievement

Asdiscussedabove,thesurveyteamsucceededinsurveyingahighpercentageoftheintendedschoolandpupilsample.Thepercentageofteacherssurveyedwasrelativelylower–thatis,around80%.Thiswasnotduetoalowresponseratebutratherbecauseinsomecases,particularlyinIQSs,theschoolonlyhadoneortwoteachers.WhilethismayaffecttheintendedMDEsize,weestimatethatthiswillnotjeopardiseavalidimpactjudgementsinceitisexpectedthattheincreaseinteachercompetencieswillberelativelylarge(a30%increaseaccordingtotheGEP3logframe).

Flooreffectsinmeasurement

Highlevelsofmissingdatawereobservedintheteacherassessment.Theamountofmissingdataisamajorconcern.First,becauseitmayindicatethatmanyoftheteacherswhoparticipatedintheprogrammewerenotabletodomanyofthetasksaskedofthem.Thesetaskswereintendedtorepresentthecorebasicelementsofwhatteachersshouldknowandbeabletodoinordertoteacheffectivelyataminimumlevel.Thustheconcernisthattheteachersmaynotbeveryskilled.Whilethissuggeststhatthetestwas‘toodifficult’,thetestwasdesignedaroundnotionsofwhataminimallycompetentteachershouldbeabletoknowanddo.Itisprobably,therefore,unhelpfultobeoverlyconcernedaboutthetest’sdifficulty.Theissuehereisthecompetencyleveloftheteachers.However,theseflooreffectslimittherangeofanalysesabletobeundertakenandlimittheevaluationteam’scapacitytodrawempiricallystrongconclusions.

Socialdesirabilitybias

Anumberofitemsregardingattitudestowardsgirls’educationwereincludedintheteacherquestionnaire.Whilethereliabilityoftheresultswasacceptable,theveryhighproportionsofteachersrespondingpositivelytostatementsregardingtheimportanceofgirls’educationrevealedacomplianceeffect.Thedistributionofresponsesacrossthestronglydisagree,disagree,agreeandstronglyagreeLikertscaleindicatesthatteachersareconformingtoperceivedcorrectresponses,creatingasocialdesirabilitybiasinthemeasurement.Giventhesecomplianceeffects,thedataonteachers’attitudestowardsgirl’seducationwerenotabletobeusedintheanalysis.Thislimitationhighlightstheimportanceofobservationdataandqualitativemethodstoprobesocialnormsandgenderattitudes,particularlyinanevaluationofaprogrammedescribedasaninterventiontargetedatgirls.

Biasinsampling

Aspartofthesamplingprocessbiasesmayoccurwhenunitsareselectedthatdonotformpartofthetargetpopulationorwhensomeunitsofthetargetpopulationarelesslikelytobeselectedthanothers.Ashasbeendiscussed,insomeschools,stakeholderstriedtoinfluencepupilsamplingorschooleligibility.ThiswasparticularlyariskinIQSsgiventheirnon-standardorganisationandthepoorqualityofthesampleframe.Totheextentpossiblethisriskwasmitigatedthroughthedefinitionandimplementationofrigorousandsystematicsamplingproceduresunderclosesupervision.


©EDOREN 47

3.3 Comparisonofbaselinecharacteristicsbetweeninterventionandcontrolschools

ThereliabilityofthequantitativeestimationmodelemployedfortheimpactevaluationoftheGEP3earlylearninginterventionreliesontherobustnessoftheRCTthatwasdesignedandimplementedtoidentifytreatmentandacontrolgroups.RCTisanexperimentaldesignthatensuresnosystematicdifferencesexistbetweentreatmentandcontrolobservationsandthus,givenrandomnesshastrulybeenachieved,addressestheissueofselectionbias.ArobustRCTisthereforecriticaltoachieveunbiasedestimatesofintervention(earlylearning)impact.

Forthisreason,wehaverunaseriesofcheckstodeterminewhetherornottherandomisationprotocolwasfollowedcorrectlyandifitachievedtheexpectedresults.Inparticular,ifthesamplingunitsofinterestwereassignedtotreatmentorcontrolgroupsrandomlyandthesamplingstrategyrandomisationprotocolwasmaintainedduringthesurveyfieldworkwewouldexpecttoseenosystematicdifferencesbetweenthetwogroups.Thisentailsthatnostatisticallysignificantdifferencesshouldemergebetweensampletreatmentandcontrolgroupsforarangeofcharacteristicsrelatedtotheobservationsonwhichtherandomisationchecksareperformed.

3.3.1 Categoriesandvariables

TherandomisationcheckswereperformedforthemainRCTsampleandfordifferentsub-samplesofinterest.Atthesametime,checkswererunatschool,teacherandpupillevels.SeparatecheckswereperformedforpublicprimaryschoolsandIQSs.Datawithwhichtoperformtherandomisationcheckswerederivedfromtheheadteacher,teacherandpupildatasets,whichwereproducedwithinformationfromtheirrespectivequestionnaires.Thesetofvariablesonwhichthecheckswererunisspecifictoeachcategoryofinterestandencompassesdifferentthematicareasofinterest.Balancecheckswerealsoperformedonkeyoutcomevariables,includingHausaandEnglishscaledscoresandteacherknowledge.Whilstmultiplecheckiterationsweretestedacrossdifferentcombinationsofvariables,wepresentinthetablesbelowaselectionofthekeyoutcomesandrelatedcharacteristicsatschool,teacherandpupillevel.Thewholesetofresults,includingchecksonsamplesandsub-samplesofinterest,isincludedinAnnexF.

3.3.2 Balanceachieved

Attheschoollevel,dataderivedfromheadteacherswereusedtocheckthebalancebetweentreatmentandcontrolgroupsacrossarangeofschoolcharacteristics,includinggender-specificfactorssuchasthenumberoftoiletsexclusivelyforgirls,boys’andgirls’enrolmentrates,pupilandteachergenderratios(boystogirlsandfemaletomaleteachers),aswellaspupilclassroomratiosandotherschool-specificaspects,includingtheoccurrenceofmajorschoolrepairworks.Asshowninthetablebelow,thelatteristheonlyvariableforwhichwedetectsomesignificantdifferencebetweentreatmentandcontrolgroupswhenlookingatthewholesampleofIQSsandprimaryschools.Alltheothervariablespertainingtothevariouscategoriesofinterestdiscussedaboveshownosignofstatisticallysignificantdifferencesbetweentreatmentandcontrolgroups.Thefullsetofschool-levelvariablesforwhichrandomisationprotocolchecks,andthecorrespondingfindings,areoutlinedinAnnexF.


©EDOREN 48

Table10: Balancecheckofschool-levelcharacteristics,publicprimaryschoolsandIQSs

VariablesMean

treatmentgroupMean

controlgroup TotalN+

Headteacherrateofabsenteeism 60% 70% 240

Headteacherobservedalesson 50% 50% 225

Schoolneedsrepairs 90%** 100% 239

Schoolhasseparatefunctionaltoiletforgirls 30% 20% 239

Notes:**,*indicatethatthetreatmentandcontrolmeansaresignificantlydifferentat5%and10%.+Nistotalsample(treatmentandcontrol).

WhenseparatelyinvestigatingIQSsandpublicprimaryschoolsintheRCTsample,theweaklysignificantdifferenceinschoolrepairsisonlyfoundinIQSs,whilstnosignificantdifferenceisdetectedamongsttreatmentandcontrolpublicprimaryschools.Thefactthatthegreatmajorityofschool-levelvariablesarefoundtobebalancedbetweentreatmentandcontrolgroups,andthefactthatthesignificantdifferenceinschoolrepairsisweakandlimitedtoasub-groupoftheschoolsample,indicatesthattherandomisationprotocolhasgenerallyachievedasatisfactorybalancebetweentreatmentandcontrolgroupsattheschoollevel.

Thisisfurtherconfirmedbythebalancediagnosisperformedonteachercharacteristics.Acrossteachers’demographic,qualificationandprofessionalcharacteristicsnostatisticallysignificantdifferenceisdetectedbetweentreatmentandcontrolgroups.Giventhattherandomisedassignmentoftheearlylearninginterventionunderevaluationwascarriedoutattheschoollevel(theclusterlevelofourRCTdesign),thesebalancediagnosticresultsareparticularlyimportantandgiveusconfidenceintherobustnessoftheestimatesofprogrammeimpactthatwilleventuallyemergefromthecomparisonoftreatmentandcontrolgroups.Ofparticularrelevanceisalsothefactthatabalanceisfoundbetweentreatmentandcontrolgroupsacrossteacherknowledgeandskillsindicators,asshowninthetablebelowforaselectionoftheseintermediateoutcomeindicators.Theonlyweaksignificance(10%)isdetectedforteachercomprehensionskills,whichdisappearswheninvestigatingthebalanceoftreatmentandcontrolgroupsinIQSsandpublicprimaryschoolsseparately.

Table11: Balancecheckofteacher-leveloutcomesandcharacteristics,publicprimaryschoolsandIQSs

VariablesMean

treatmentgroup

Meancontrolgroup TotalN+

Abilitytoidentifylowperformers 0.4 0.5 463

Teacherwritingskills 0.2 0.2 463

TeacherHausaknowledge 2.9 2.8 463

Teachercomprehensionskills 2.1* 2.3 463

Notes:**,*indicatethatthetreatmentandcontrolmeansaresignificantlydifferentat5%and10%.+Nistotalsampleacrosstreatmentandcontrol.


©EDOREN 49

Additionalbalancingtestsbetweentreatmentandcontrolgroupswerealsocarriedoutforpupillearningoutcomesandpupilcharacteristics,withtheinclusionofvariablesconcerningchildren’shouseholds’factors.Inthiscaseaswell,thegreatmajorityofcharacteristicsinvestigateddonotshowanystatisticallysignificantdifferencebetweenthetwogroups.ThetwolearningoutcomeindicatorsofHausaandEnglishscaledscoresshownostatisticallysignificantdifferencebetweentreatmentandcontrolgroups.Intheoverallsampleofallpupils,onlyfortwovariablesaresomedifferencesinmeansdetectedacrosstheoverallpupilsample,bothreferringtoownershipofassets(chairsandmobilephones).Whendisaggregatingthesamplebetweenmaleandfemalepupils,itispossibletonotethatfurtherdifferencesemerge,althoughonlyforaminorityofvariables.Boththemagnitudeandthedegreeofstatisticalsignificanceinanyofthesedifferencesarelowandshouldnotrepresentaconcern.Smallandstatisticallyweakdifferencesinalimitednumberofvariablesaretobeexpectedandareconsiderablyreducedintheoverallsampleofpupils.Aselectionofvariablesispresentedinthetablebelow,andtherestarepresentedinAnnexF.

Table12: Balancecheckofpupil-leveloutcomesandcharacteristics,publicprimaryschoolsandIQSs

Variables Meantreatmentgroup

Meancontrolgroup TotalN+

Hausaliteracyscaledscore 507.341 509.568 2649

Englishliteracyscaledscore 354.374 356.946 2649

PupilspeaksHausaathome 99% 99% 2623

Pupilhouseholdhasmobilephone 90%** 95% 2580

Notes:**,*indicatethatthetreatmentandcontrolmeansaresignificantlydifferentat5%and10%.+Nistotalsampleacrosstreatmentandcontrol.

3.4 Analysisofthedata

TheanalysisofdatafortheevaluationoftheRANAearlylearninginterventionprogrammeisprovidedbelow.TheanalysisiscategorisedintofourmainsectionsandfollowstheprogrammeToC:

• Theanalysisbeginswithdescriptionsofthecontextwithinwhichteachingandlearningtakesplace,includingschoolleadershipandmanagement,IQSintegration,schoolinfrastructureandresourcesandthegendersensitivityoftheschoolingenvironment.

• Thesecondsectionreportsonteachercharacteristics,knowledge,practicesandmotivations.Itbeginsbydescribingtheteacherswithinthesettingandthendescribeswhattheyknowandwhattheydointheirlessons,fromaneducationalperspective.Theanalysisthenfocusesontheknowledge,skillsandpracticesofteachersfollowingtheinterventionToCandthereforeincludestheToCintermediaryoutcomes,teacherknowledgeinliteracyandlanguageintheearlygrades,improvedteacherinstructionskillsintermsofactivelearningandincreasedtimeontask,theuseofHausa-basedteachingandlearningmaterials,andimprovedHausa-basedteachingintheearlygrades.Inaddition,wediscussteachermotivationandattitudesasimportantassumptionsunderlyingthecausalchain.


©EDOREN 50

• Thethirdsectionrelatestopupilsanddescribesthepupilswithinthesettingandtheircurrentlevelsoflearning.

• Theanalysisconcludeswithaninvestigationintotherelationshipsbetweenthecharacteristicswithintheschoolsetting,thecharacteristicsofteachersandteachingandlearning,andtheobservedHausaliteracyskillsofpupils.

Thebaselineanalysisofquantitativedataprovidesabenchmarkthatisausefulcomparisonformidlineandeventuallyendlinedata,andwillbeusedforassessingtheimpactoftheprogrammeunderevaluation,inthiscasetheGEP3earlylearningintervention.Atthesametime,thebaselinealsooffersthepossibilityofundertakingadescriptiveanalysisofassociationsofbaselineindicatorsacrosscategoriesofinterests.Thissectionprovidesasummaryanalysisofthemostrelevantandinterestingdescriptiveassociationsbetweenindicatorsandcategoriesofinterestthatemergedfromtheanalysisofbaselinedata.Itisimportanttobearinmindthattheseassociationsshouldnotbeusedformakingstatisticalclaimsregardingcausalinference,especiallygiventhefactthattherelativelysmallsamplesizesusedinouranalysis(e.g.teacher-andschool-levelsamples),andthefurtheranalysisofsub-samplesacrosscategories(e.g.femalepupilsinprimaryschools)isboundtoyieldmeanestimateswithlargeconfidenceintervals.Thedescriptivedifferencesdiscussedinthissectionshouldthereforeonlybetreatedasasummarypictureofourbaselinesample(weightedfortheextrapolationoftheresultstothepopulationsofschools,teachersandpupilsfromwhichitisdrawn).Thevisualanalysisintheformofbarchartsandlinegraphspresentedalongsidethetextserveadescriptivepurpose,tofacilitatethenarrativeontheresults.Alargersetofdescriptivestatistics,includingmeanestimates,standarderrorsandnumberofobservationsforeachvariableisincludedinAnnexK.Attheendofthissectionwealsopresentaregressionmodelthatfocusesoncorrelationsbetweenourmainlearningoutcomevariablesandtheirmaininfluencingfactors.Thiscorrelationanalysisallowsustomakemorestatisticallyrobustclaimsregardingtherelationshipbetweentheselectedexplanatoryandoutcomevariables,asisexplainedinmoredetailintherelevantparagraph.ThesebaselinefindingscanbeusedtoshedlightontherobustnessofGEP3’sToCassumptions,withafocusontheDACcriteriaofthe‘relevance’oftheprogramme’sdesign.Inparticular,thedescriptiveandregressionresultspresentedinthissectioncanhelptoinvestigatewhetherassumptionsregardingteachers’knowledgeandpractices,schoolenvironmentandpupils’characteristicsareconfirmedbythedata.However,thisstilldoesnotamounttoacausalanalysisofimpact,whichwillbecarriedoutinthenextstagesoftheevaluationthroughtheproposedRCTapproach.

3.4.1 Teachingcontext

Teachingcontextincludestheschoolleadershipandmanagementoftheschool,IQSintegration,schoolinfrastructureandresources,andthegendersensitivityoftheschoolingenvironment.

3.4.1.1 Schoollocationandsize

Thegreatmajorityofschoolsinourearlylearningsamplearelocatedinruralareas,withonlyaround14%ofschoolslocatedinurbanareas.TheproportionsaresimilarwhenwelookseparatelyatIQSsandprimaryschools,withthelatterslightlymorerepresentedinruralareas.

Theaverageschoolinthesamplehasaroundeightteachersintotal,thoughinthiscasethereisacleardifferencebetweenthetwotypesofschools,withpublicprimaryschoolsconsiderablylargerthanIQSs.Whenlookingatthesub-sampleofteacherswhoteachintegratedsubjects,theaverage


©EDOREN 51

numberacrossschoolsdropstoaroundsixduetothesmallnumber(aroundtwo)ofintegratedteachersinIQSsonaverage.Similarly,theaveragenumberofteachersperschoolacrossthewholeearlylearningsampledropsfurtherwhenlookingatteachersteachingearlygrades.Inthiscase,theaveragenumberisreducedbyasmallernumberofearlygradeteachersinpublicprimaryschoolswhencomparedtothewholeteacherroaster–whereas,inIQSs,practicallyallteachersengagedinintegratedsubjectsarefoundtobeteachingearlygradeclasses.Thegraphbelowhelpssummarisethiscomparison,whichprovidesanintroductoryideaoftheschoolsize.

Figure11: Schoolsizebasedonaveragenumberofteachers,byschooltype

Animportantpointtonoteisthatover40%ofIQSsintheearlylearningsamplereportedlyhaveonlyonesingleteacherofintegratedsubjects,withasimilarproportionofteachersteachingintegratedsubjectsinearlygrades.Theproportionofpublicprimaryschoolsthathaveonlyoneteacherofintegratedsubjects,ontheotherhand,ismuchlower,ataround3%.Giventhefocusoftheinterventiononteacherpeer-to-peerexchangesandcollaboration,thelargenumberofschoolswithonlyoneteacherisofconcern.Inparticular,thisfindingseemstosuggestthatschool-levelpeer-mentoringisnotapplicabletoalargeproportionofIQSs.Sincepeer-to-peerisanimportantassumptionunderpinningtheeffectivenessofthetrainingenvisaged,itisrecommendedthattheprogrammeidentifiesalternativewaysofensuringthatskillsandknowledgeacquiredbyteachersthroughtheearlylearninginterventionaresharedamongstteacherspossiblyfromdifferentbutnearbyschoolswithonlyasinglefacilitator.

Thissectioncontinueswithadescriptionoftheenvironmentinwhichteachingandlearningtakesplace.Thisincludesadiscussionofrelevantinformationatourdisposalonleadershipandmanagement,schoolinfrastructureandexternalmonitoring.ItalsoincludesashortsectiononIQSs’integration.Thesearefactorsthatmayinfluencetheteachingpracticesintheschools.

3.4.1.2 Schoolleadershipandmanagement

Informationwascollectedattheschoollevel,throughtheheadteachersurvey,onarangeoffactorsthatcanbeassociatedwithschoolleadershipandmanagement.Withinourearlylearningsampleofschoolstheoverallpicturethatemergesfromthedescriptiveanalysisseemstoshowarelatively

0 2 4 6 8 10 12

Numberofteachersinearlygrades

Numberofteachersofintegratedsubjects

Numberofteachersintotal

PublicSchools IQSchools AllSchools


©EDOREN 52

positivesituationwhenitcomestotakingactiononteacherandpupilattendance.Asshowninthegraphbelow,around70%ofheadteachersreporttakingactiontoimproveteacherattendance,andthisproportionrisestoover90%whenitcomestotakingactiontoimprovepupils’attendance.Lesspositiveistheproportionofheadteacherswhohaveobservedalesson,whichisjustunder50%.Also,thepercentageofheadteacherswhohavenomeetingswithteachersorwhomeetthemlessthanonceamonth,includingindividualorgroupmeetings,isaround50%.

Figure12: Prevalenceofheadteachermanagementactions

Whilstgenderdoesnotseemtobeafactorinthelevelofschoolleadershipandmanagementrecorded,theheadteacherageappearstohavearelationshipwiththis.Asshowninthegraphbelow,olderheadteachersarefoundtobemoreproactivethanyoungeronesonteachermanagement:amongstheadteachersolderthan50,over50%observedalessonandalmost70%tooksomeformofactiontoimproveteacherattendance;inbothcases,theproportionoftheyoungestgroupofteachers(under25yearsofage)observingalessonortakingactionismuchlower,atjustunder20%.Thistrenddoesnotapplytoimprovingpupils’attendance,forwhichpracticallyallteachersacrossthedifferentagegroupsarereportedtobetakingsomeformofaction.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Observedalesson Tookactiontoimproveteacherattendance Tookactiontoimprovepupilattendance


©EDOREN 53

Figure13: Actionstakenandagegroups

Cleardifferenceswithregardtoobservingteachersandtakingactiontoimproveteacherattendancearealsonoticeablewhencomparingstatesandschooltypes,withagainnonoticeabledifferencedetectedregardingactiontakentoimprovepupils’attendance.Whilstalmost60%ofheadteachersobservedalessonand75%tookactiontoimproveteacherattendanceinZamfara,theequivalentproportionsinKatsinaare34%and67%.EvenmoreaccentuatedisthedifferencebetweenpublicprimaryschoolsandIQSs:only53%ofheadteachersinIQSstakeactiontoimproveteacherattendance,whichcomparestoalmost90%ofheadteacherstakingactioninpublicprimaryschools.Itseemsthereforethatthewillingnessandabilityofheadteacherstoleadandimproveonteachingintheirschools,whichisanassumptionoftheprogramme’sToC,mightbeanissueinQur’anicschools.ThismayalsobeduetothedifferenttypeofleadershipstructureinIQSs,whereheadteachersarenotalwaysthesamepersonastheschool’sprivateproprietor.Althoughthesizeforthesub-sampleofheadteachersinIQSsistoosmalltoprovidestatisticalrobustness,theindicationthatseemstoemergeisthatwhentheheadteachersarenotalsotheproprietortheyarelesslikelytotakeactiontoimproveteacherattendance90.Itwillbeparticularlyimportantfortheearlylearninginterventiontoensurethattherelevantchainofteachingresponsibilitywithineachschoolisclearlyunderstoodanditsinfluencemaximisedwhenrunningthetraining.

Interestingly,thepersonalcharacteristicsoftheheadteacheralsoseemtoplayaroleintheactiontakentoimprovetheattendanceofteachers.Havingsomeformofqualification,oratleastaSeniorSecondaryCertificateExamination(SSCE)academicdegree,isassociatedwithahigherprobabilityoftakingactiontoimproveteacherattendance.Atthesametime,andevenmoreimportantlyinthecontextofGEP3,headteacherswhohaveattendedsomeformoftrainingarereportedlymorelikelytotakeactiontoimproveteacherattendance:theproportionofnon-trainedteacherswhotakeactionis63%,whilsttheproportionoftrainedteacherswhotakeactionis74%.

90Amongstthe26IQSheadteachersobservedwhowerereportedlyalsoschoolproprietors,18(69%)declaredthattheytookactiontoimproveteacherattendance,whilsttheproportionisapproximately50%forthenon-proprietorheadteachers.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Observedalesson Takeactiontoimproveteacherattendance

Under25 25to34 35to49 Over50


©EDOREN 54

3.4.1.3 Externalmonitoring

Aspartoftheearlylearningintervention,regularmonitoringvisitsbymastertrainersandgovernmentstaffareplannedfor.Itisthereforeinterestingtoexamineatbaselinewhethertheschoolsreceivedanymonitoringvisitsfromgovernmentofficialsorotherexternalagentsduringthelastschoolyear.Accordingtoheadteacheraccounts,almost80%ofschoolsreceivedamonitoringvisit.Amongsttheschoolsthatreceivedthevisits,theaveragenumberofvisitperschoolisaroundeight.Visitswereperformedbyarangeofgovernmentagenciesandschoolswerevisitedbymultiplevisitingofficers.Aroundtwo-thirdsofschoolswerevisitedbyanLGEAofficer,withthetwoothermostprevelantcategoriesofvisitorsbeingSUBEBofficersandrepresentativesfromNGOsandotherdonoragencies.

ThereappearstobeacleardifferencebetweenIQSsandpublicprimaryschoolsinthelevelofexternalmonitoringthattheyreceive.Practicallyallpublicprimaryschoolswerefoundtoreceivesomemonitoringvisits,whileonly58%ofIQSsreportedhavingreceived(anytypeof)monitoringvisitduringthelastschoolyear.Moreover,amongstthoseschoolsthathavereceivedavisit,thenumberofvisitsagainvariesconsiderablybetweenpublicprimaryschoolsandIQSs.Thelatterreportedhavingreceivedjustoverthreevisitsonaverage,whilstfortheformerthefigureisover11visits.Over50%oftheIQSsreceivedoneortwomonitoringvisits.ItwillthereforebeimportanttopayparticularattentiontomonitoringpatternsinIQSssoastoensurethatvisitsbyGEP3trainersandotherstakeholdersoccurasfrequentlyasforpublicprimaryschools.

3.4.1.4 IQSintegration

Theearlylearninginterventionwillbeimplementedin80IQSsinKatsinaandZamfara.TheteachingconditionsinIQSsdiffersignificantlyfromthoseinpublicprimaryschools.Asmentionedabove,thenumberofteachersteachingintegratedsubjectsatalllevelsintheIQSsincludedinourearlylearningsampleisjustovertwoonaverage,whichisconsiderably(butnotsurprisingly)lowerthanthenumberinpublicprimaryschools,atjustover10perschool.ThefractionoffemaleteachersamongstthoseteachingintegratedsubjectsismuchlowerinIQSs,whereonly4%ofnon-religioussubjectteachersarewomen(comparedto14%inpublicprimaryschools).

Acrossall120IQSsinourearlylearningsample,wefoundthat72%ofschoolsarestructuredlikeapublicprimaryschool,16%arestructuredintostagesand12%haveadifferentstructure.Astagereferstoagroupinginwhichpupilsaretaughtthepublicprimaryschoolsyllabuscorrespondingtotheirrelevantgrade.AmongstthegroupofIQSsthatarestructuredintostages,thegreatmajorityreporthavingthreestages,whichcorrespondtothreelevels–ascomparedtothepublicprimaryschoolstructure,whichhassixlevels.Thegraphbelowshowsthedistributionofthesetwoschoolstructuretypes(i.e.publicprimaryschoolsstructurevs.otherstagecomposition)inIQSs.


©EDOREN 55

Figure14: SchoolstructureinIQSs

WhencomparingIQSsinKatsinaandZamfara,itisnotablethatalargerproportionofschoolsarestructuredlikepublicprimaryschoolsinKatsinathaninZamfara.TheaveragenumberofyearsthatIQSshavebeenintegratedishigherinZamfarathaninKatsina.Whilsttheheadteachercharacteristics,includinggenderandroleasaproprietor,donotseemtohaveaclearassociationwithintegration,itisthelocationoftheschoolthatagainappearstoberelevant.Alargerproportionofschoolsarestructuredlikepublicprimaryschoolsinurbanareaswhencomparedtoruralareas,andschoolsinurbanareashavebeenintegratedforlonger.

3.4.1.5 Schoolinfrastructureandresources

Infocusingonschoolinfrastructuralstatusitisclearthatalmostallschools(95%)areinneedofrepairs,withonlyaslightlylargerproportionofIQSs(97%)inneedofrepairscomparedtopublicprimaryschools(92%).Locationdoesnotseemtomakeanydifference,withschoolsinurbanandruralareasofbothKatsinaandZamfaraequallyinneedofschoolrepairs,althoughsubstantialdifferencescanbedetectedbetweenstatesacrossotherinfrastructuralvariables(seeFigure15).Asexpected,alargerproportionofschoolsinurbanareashaveaccesstowatersourcesandelectricity.RelativelymoreschoolsinZamfarareporthavingaccesstowater,electricityandfunctionaltoiletsforgirls.ThegraphbelowshowshowtheinfrastructurallevelofschoolsinZamfaraappearstobebetteraccordingtothreekeyaspects.

0%

10%

20%

30%

40%

50%

60%

70%

80%

IQschools

Publicschoolstructure Schoolhas2stages Schoolhas3stages Schoolhas1stage


©EDOREN 56

Figure15: InfrastructureinKatsinaandZamfara

WhilsttheproportionofIQSsandpublicprimaryschoolsthathaveaccesstoasourceofdrinkingwaterissimilar,thereisacleardifferencewhenitcomestoelectricity,with37%ofIQSshavingelectricity,comparedtoonly9%ofpublicprimaryschools.However,onaverage,itwouldseemthatpublicprimaryschoolsarebetterresourcedthanIQSs.Forexample,publicprimaryschoolswerefoundtohaveconsiderablymoreroomsthanIQSs,aswellasmorefunctioningtoilets.Atthesametime,whilstalmost20%ofpublicprimaryschoolshaveacollectionofbooks,thispercentagedropsto4%amongstIQSs.Finally,over90%ofpublicprimaryschoolshaveaplaygroundorsportsarea,comparedtounder20%ofIQSs.

Schoolconditionsformapartofthebroadlearningenvironmentinwhichpupilsfindthemselves.ThefactthatpublicprimaryschoolsappeartobeonaveragebetterresourcedthanIQSsisafactorthatwillhavetobetakenintoaccountintheimpactestimationanalysisatendline.Specifically,school-relateddifferencesindescriptiveandexplanatoryfactorswillhavetobediscountedfromtheimpactanalysistoisolatetheeffectsoftheearlylearninginitiativeonpupillearning.Atthesametime,theprogrammeimplementationshouldbeawareofthedifferencesbetweenpublicprimaryschoolsandIQSsinregardtothecollectionofbooksintheschools,forinstance,asthesereflectdifferentschoolenvironmentsthataremoreorlessconducivetolearning.

3.4.1.6 Girl-friendliness

BecauseofGEP3’sfocusongirls’educationwehavecollecteddataonsomemeasuresoftenassociatedwiththegirl-friendlinessoftheschoolenvironment.However,itischallengingtotrytoachieveareliablemeasureofthegirl-friendlinessofschools,asdataongender-sensitiveschool-levelfactorsaredifficulttodefineandcollect.Theinformationthatwasobtainedonthisforourearlylearningsamplecannotthereforebeconsideredexhaustive.Overall,30%ofschoolswerefoundtohaveseparatefunctionaltoiletsforgirls.Thisaverageproportionisclearlydrivenbyfacilitiesfoundinpublicprimaryschools,as48%ofthesehaveseparatetoiletscomparedtoonly7%ofIQSs.Oneinterestingaspecttoreportisthatthreeoutofthefiveschoolsinourearlylearningsamplethathaveafemaleheadteacherhaveafunctioningseparatetoiletforgirls.Theproportionismuchlowerfor

0%

10%

20%

30%

40%

50%

60%

Electricity Schoolhasasourceofdrinkingwater Schoolhasseparatefunctionaltoiletforgirls

Katsina Zamfara


©EDOREN 57

therestoftheschools,whichhaveamaleheadteacher(29%).Althoughofinterestasanecdotalevidence,theverysmallandunbalancedsampledoesnotallowforanyrobustdescriptiveanalysisofthistrend.

SomedifferencesarealsoobservedwhencomparingKatsinawithZamfara,withschoolsinKatsinashowingahigherratioofgirlstoboysinP1toP3andoverall(around0.8)thanZamfara(around0.6)91.Interestingly,whilsttheaverageratioofgirlstoboysinP1toP3wasrecordedas0.73,thegenderratioreflectingthosepresentonthedayofthesurveyteam’svisittoP2classeswas1.8onaverage.Thismaypointtowardsabiasduetoknowledgeoftheanticipatedpresenceofsurveyorsintheschool.Whilsttheproportionoffemalepupilsisrelativelyhigh,asalreadydiscussedtheproportionoffemaleteachersinoursampledschoolswasfoundtobelowandthisiscorroboratedbythesub-sampleofteachersspecificallyteachingintegratedsubjects,forwhichtheproportionoffemaleteachersisaround10%.

3.4.2 Teachercharacteristics,knowledge,practiceandmotivations

Thissectionreportsonteachercharacteristics,knowledge,practicesandmotivations.Itbeginsbydescribingtheteacherswithinthesettingandthendescribeswhattheyknowandwhattheydointheirlessons.Thebaselinefindingsrelatedtotheknowledge,skillsandpracticesofteachersarereportedon.FindingsrelatedtothefollowingkeyintermediaryoutcomesintheToCarediscussed-teacherknowledgeinliteracyandlanguageintheearlygrades,improvedinstructionskillsinactivelearningandincreasedtimeontask,theuseofHausa-basedteachingandlearningmaterials,andHausa-basedteachingintheearlygrades.Teachermotivationandattitudesarealsodiscussedasthesehaveaninfluenceonthequalityofteaching.

3.4.2.1 Teachercharacteristics

Age,sexandlanguage

Thetypicalteacherteachingintheearlygradesisa36-year-oldmalewhospeaksHausaandEnglish.Morespecifically,thegreatmajorityofteachers(85%)intheearlylearningsamplearemale.Theratiochangesdependingonthetypeofschool:IQSfacilitatorsarenearlyallmale,withonly3%offacilitatorsbeingfemale,whileacrosspublicprimaryschoolsabout20%ofteachersarefemale.Interestingly,therealsoseemstobeadifferencebetweenthetwostates,withalargerproportionoffemaleteachersinKatsinathaninZamfara.92Notsurprisingly,alargerproportionoffemaleteachersarealsofoundinurbanareasascomparedtoruralareas.

Theaverageageofmaleteachersisslightlyhigherthanfemaleteachers,whichisalsodetectablewhenlookingattheagegroupdifferentiation.Whilst13%ofmaleteachersareabove50yearsofage,only1%offemaleteachersbelongtothatolderagegroup.Acrossallteachers,over80%belongstotheagegroupspanningfrom25to50years,withonly6%youngerthan25.WhencomparingIQSsandpublicprimaryschoolsthough,theproportionofyoung(below25)teachersisconsiderablyhigherinIQSs(11%)thaninpublicprimaryschools(4%).Conversely,thereisahigherproportionofolder(above50)teachersinpublicprimaryschools(13%)thaninIQSs(9%).

91TheASCdataconfirmthehigherratioofgirlstoboysinKatsinacomparedtoZamfara.TheratioinP1–3equals0.78inKatsinaand0.54inZamfara.92Thisisinlinewiththe2014–2015ASCdata,whichreportthat23%ofprimaryschoolteachersinKatsinaarefemale,comparedto15%inZamfara.


©EDOREN 58

Whenitcomestolanguage,allteachersindicatethattheyspeakHausa,thegreatmajority(85%)alsostatethattheyspeakEnglish,whilstonlyaroundone-fifthofteachersspeakArabic93.Notsurprisingly,arelativelylargerproportionofteachersspeakArabicinIQSsthaninpublicprimaryschools,asArabiccanbeassociatedwiththestudyoftheQur’an.Atthesametime,whilstaround90%ofteachersclaimtospeakEnglishinpublicprimaryschools,thepercentagedropstoaround70%inIQSs.ThereisnosizeabledifferencebetweenteachersinKatsinaandZamfarawhenitcomestolanguagesspoken.

Classestaught

Allsurveyedteachers(excludingheadteachers)wereteachingP1toP3orequivalentearlylevelatthetimeofthebaselinestudy.ThisisafeatureofthesamplingapproachtakenandisnotrepresentativeofteachingresponsibilitiesinGEP3schools.Thefollowinganalysisprovidesdescriptionsofearlyyearsteacherswithinoursample.Aminorityofthem(around30%)alsotaughtP4toP6classes,thusbeinginvolvedinbothupperandlowerlevelsofteaching.ThiswasparticularlyprominentinKatsina,where55%ofteacherstaughtfromP4toP6,comparedtoonly17%inZamfara.Atthesametime,whilst41%ofteachersinpublicprimaryschoolswerefoundtobeteachingatbothupperandlowerlevels,thisproportiondropsto5%inIQSs.

Onaverageacrosspublicprimaryschoolsjustunder50%ofteachersarefoundtobeteachingonlyonenon-religioussubjectandaround20%ofteachersteachmorethantwonon-religioussubjects.ThereisacleardifferencewithIQSs,wheretheproportionofteachersthatonlyteachonenon-religioussubjectisalmost60%andonly10%teachmorethantwonon-religioussubjects.Interestingly,theproportionofteachersteachingmultiplenon-religioussubjectsisconsiderablyhigherinKatsinathaninZamfara,wherealmost60%ofteachersonlyteachonenon-religioussubject,comparedtojustover30%inKatsina.Finally,whenspecificallylookingatIQSs,thedatashowthatamongsttheintegratedsubjects,HausaandMathsaretaughtby50%ofteachers,Englishbyjustover40%,whilstSocialStudiesandBasicScienceareonlytaughtby5%and2%ofIQSteachers,respectively.Thiscomparesnegativelytopublicprimaryschools,where30%ofteachersarefoundtoteachSocialStudiesandaround25percentteachBasicScience.

ItisimportanttonotethatthisinformationisderivedfromtheteacherquestionnairewhichwasadministeredtooursampleofpublicprimaryschoolandIQSteachers.However,informationcollectedfromheadteachersonsubjectstaughtintheirschoolsshowsadifferentpictureattheschoollevel,withover90%ofschoolsreportedlyteachingMathsandHausa,over90%ofschoolsteachingEnglishandjustover50%ofschoolsteachingSocialStudiesandBasicScience.Thedifferencebetweenthetwotypesofschoolsisalsomarkedinthiscase,withmostpublicprimaryschoolsreportedlyteachingeverycoresubjectandonlyaround10%ofIQSsteachingSocialStudiesandBasicScience.TheproportionofIQSsteachingMathsandHausaisover90%andover70%arereportedlyteachingEnglish.ThedivergenttrendbetweenpublicprimaryschoolsandIQSsisconsistentacrossthetwosourcesofinformation.

Teachingexperienceandqualifications

Thegreatmajorityofteachers(includinginthiscaseheadteacherswithteachingresponsibilities)arereportedtohavesomeformofprofessionalqualification.Inparticular,70%ofteachershaveat

93Informationonlanguagewascollectedfromteachersonlybutnotfromheadteacherswhoalsoperformteachingduties.Similarly,informationonteachinginearlygradeswascollectedfromthesamesampleofteachersbutnotteachingheadteachers.


©EDOREN 59

leastaGrade2qualificationorequivalent,whilst54%alsopossessaNationalCertificateinEducation(NCE).Whenlookingatthegenderdifferentiation,almost80%offemaleteachershaveanNCE,comparedtoonly50%ofmaleteachers,thelatterformingaround85%oftheearlylearningsample,asmentionedabove.MostoftheNCEholdersarefoundinpublicprimaryschools,with70%ofteachersinpublicprimaryschoolshavinganNCE,comparedtoonly22%ofteachersinIQSs.Overall,thedifferencebetweenschooltypesisslightlyhigherwhenlookingatanytypeofqualification,as86%ofpublicprimaryschoolteachershavesomeformofprofessionalqualification,comparedto36%inIQSs.Furthermore,while64%ofteachersinKatsinahaveanNCE,only48%inZamfarado.AlargerproportionofteachersinurbanareashaveanNCEqualificationthanthoseinruralareas.

Whenitcomestogeneraleducation,over70%ofteachersacrossthetwoschooltypeshavetakentheSSCEorequivalent,around15%havealoweracademicdegree(primaryorJSS)andonly2%haveauniversitybachelor’sdegree(ahigherproportionhaveadiplomacertificate).Thelatterareonlymen,butamongstthesmallsub-sampleoffemaleteachers,over93%haveanSSCEdegree.Theredoesnotappeartobeagreatdifferenceinacademiceducationbetweenthetwotypesofschoolsorthetwostates.Eventheproportionofteacherswithareligiouseducationissimilarataround20%inbothpublicprimaryschoolsandIQSs.

Figure16: Teacherqualificationandeducationbyschooltype

Amoreobservablepatternofdiversificationacrossschoolsandstatesisobservableforteachers’experience.MoreexperiencedteachersarefoundinpublicprimaryschoolsandinZamfara.Thisisevidentfromboththeindicatorscoveringthetotalnumberofyearsofexperienceandthosespecificallyfocusingonexperienceinthesameschoolinwhichteachersweresurveyed.Acrossthewholesample,theaveragenumberofyearsofexperienceis10,with89%ofteacherswithatleasttwoyearsofexperienceinanyschoolandalloftheteacherswithatleasttwoyearsofexperienceintheschoolwheretheyweresurveyed.Thisindicatesagenerallyhighlyexperiencedpoolofteachersinourbaselinesample,withnonoticeabledifferencebetweengenders.TeachersinpublicprimaryschoolsaremoreexperiencedthanthoseinIQSs,withtheaveragenumberofyearsasateacherinanyschool(includingtheschoolwheretheyweresurveyed)beingover11yearsinpublicprimaryschoolsandundereightyearsinIQSs.

0%

10%

20%

30%

40%

50%

60%

70%

80%

TeacherhasaGrade2orequivalent

TeacherhasanNCE HighestqualificationisSSCE

HighestqualificationisBA/BSc

Teacherqualifiedinreligiouseducation

IQschools Publicschools


©EDOREN 60

Justover40%ofteacherswerefoundtohaveattendedsomeformoftraininginthelasttwoyears.Theproportionisslightlyhigherformen(43%)thanwomen(37%),forteachersinKatsina(44%)comparedtothoseinZamfara(41%),andishigherinIQSs(46%)thaninpublicprimaryschools(41%).Interestingly,teacherswhohaveundertakentrainingarealsodisproportionatelyrepresentedamongstthoseteacherswhoreceivesomeformofpayment:41%comparedto35%.Interestingly,alargerproportionofteachersaretrainedinruralareas,yetalthoughthisfigureisarelativeoneitmayalsoreflectthedisproportionatelylargesub-sampleofruralschoolsinourearlylearningsample.Whatemergesclearlyfromthedataisthatteacherswithsomeprofessionalqualificationaremorelikelytohavereceivedtraining,asalmosthalfofteacherswithaprofessionalqualificationhaveattendedtraining,comparedtojustover30%ofnon-qualifiedteachers.Whenthetypeoftrainingreceivedamongstthesub-sampleoftrainedteacherswasinvestigatedthemajorityofthetrainingwasreportedtobeoncurriculumsubjectsandliteracyandnumeracy,witharound30%ofteachersreportedlytrainedonschoolleadershipandmanagementandplanning.Onlyaround20%ofteachersweretrainedspecificallyonHausa.Almosthalfofthetraining(47%)wasconductedbyUNICEFaspartofGEP,withtheotherorganisersbeinggovernmentpartnerassociations(e.g.SUBEBsandLGEAs).Theassumptionthattrainingisrelevantandtargetedisacriticaloneandthefactthatonly20%ofteacherstrainedinthepastattendedHausacoursesmaybeanindicationthatthereismoreinterestinotherteachingandmanagement-relatedsubjects.However,itcouldalsoindicatethatHausacourseswerenotoffered.Inanycase,giventhefocusoftheearlylearninginterventiononHausaliteracyteachingandlearningoutcomesitwillbeimportanttoensureastrongfocusontrainingspecificallyonHausaforearlylearningtraining.

3.4.2.2 Whatteachersknowanddo

Thissectionofthereportdescribesteachers’knowledgeandskillsandpractices,ascollectedthroughthreeprimarysources:ateacherassessment,aclassroomobservationandateacherquestionnaire.Teachers’knowledgeandskillsreferstoteachers’proficienciesasperthesubscalesdrawnfromtheteacherassessmentandaqualitativereviewofteachers’responsestoopenendedquestionswithintheassessment.Teacherpracticesreferstothepracticesofteachersintheclassroom,asobservedthroughclassroomobservation.Teacherpracticeswerecategorisedintothreecategories:teachertalk,teacheractionandpupilaction.ThesectionthenreportsonthebaselinevaluesofrelatedintermediaryoutcomesintheearlylearninginterventionToC.

Whatdoteachersknow?

Theteacherknowledgeandskillstestwasdividedintothreesections,collectivelycomprising30items,includingmultiplechoice,shortresponseandlongresponseitems.Insectiononeteacherswereaskedtomarkpupils’responsestoHausaliteracyquestionsandtoindicatethegradelevelatwhichtheanswershouldbeknownbypupils,asdefinedbythecurriculum.InsectiontwoteacherswereaskedtofillinmissinginformationinordertoprepareananswersheetforareadingtestaimedatGrade2pupils.Teacherswereprovidedwithtwonewspaperarticlesandaskedtofillinmissinginformation,includingbasiccomprehensionquestionsandquestionsthatrequiretheinterpretationofwordsandphrases.Sectionthreeaskedteacherstoidentifypoorandgoodpupils’workandtoreviewpupils’workinordertomakejudgementsaboutpupils’writing,includingtheorganisationofthewriting,theuseofgrammar,punctuation,spelling,thepupils’abilitytoself-correctandreflectontheirwritingandthepupils’abilitytoformlettersandusespacesbetweenwords.Teacherswerethenaskedtodescribehowtheymightsupportthepupilsinimprovingtheirwriting.


©EDOREN 61

Definingteachercompetence

Thepercentageofteacherswhodemonstrateminimumteachingknowledgeinliteracyandlanguageismadeupofthesixsubscalesdevelopedfromtheteacherassessment.

Thresholdsweredefinedforeachsubscaletodifferentiatebetweenteacherproficiencylevels.Twocut-offpointsweredefinedforeachscaletocreatethreeproficiencylevelsperscale.Theproficiencylevelsincludelowband–noevidenceofskill,middleband–evidenceofrudimentaryskill,andupperband–evidenceofcompetence.

AscanbeseeninFigure17,inthemajorityofareastherewasnoevidencethatteachershavetheknowledgeandskillsrequiredtoteacheffectively.Rudimentarylevelsofskillswereobservedinasmallproportionoftheassessedteachersinidentifyinglowperformers,evidencingjudgementsanddiagnosingwritingskillsandinterpretingwordsandphrases.Lessthan3%ofteacherswerefoundtobecompetentinidentifyinglowperformersandinterpretingwordsandphrases.AgreaterproportionofteacherswereabletodisplayknowledgeandskillsinGrade2Hausaandincomprehensionthaninotherareasofteacherknowledgeandskills.However,inabsoluteterms,theshareofteacherswhowereabletodisplaycompetenceinthisareawasfairlylow,atjust40%.Furthermore,itisimportanttostressthatthelevelsofHausaassessedwerebenchmarkedtoGrade1-and2-levelHausaknowledge.Itemstestedincludedbasicgrammarandtheinitiallettersofeverydayobjectsandanimals.

Thefindingthatover60%ofteacherswereunabletodisplaycompetenceinGrade1-and2-levelHausaishighlysignificantforanearlylearninginterventionthatfocusesonteachinginHausa.ThisisrecognisedintheearlylearningToC:oneoftheassumptionsintheToCisthatteachersareliterateinHausa.Theeducationliteratureindicatesthattwokeydeterminantsofteachers’abilitytoraisepupillearningaretheirbehaviouralskills(teachingstyleandbehaviouralskills)andtheirowncompetence(Dundaretal.2014).Thisimpliesthatteacherswholackcompetenceinearlygrade-levelHausaareunlikelytobeabletoteachthissubjecttopupils.Inlinewiththisfinding,thebaselinefindingshighlightthattheRANAinterventionwillneedtofocusnotjustonpedagogicalandcurriculumknowledge,butalsoonsubjectknowledge,includingofP1andP2-levelHausa.

Anothernotablefindingisthatnoteachersintheearlylearningschoolsarecompetentinevidencingjudgementsanddiagnosingpupils’workorwriting.Thereisaverystrongandlong-standingperspectiveineducationalresearchthatastudentlearnsbestwhenteachingistargetedtowhats/heisreadytolearn.PsychologistLevVygotskyproposedthis90yearsago(knownasthe‘zoneofproximaldevelopment’,Vygotsky1997).Thisconcepthasbeenmainstreamedineducationaltheoryandpracticesincethe1970sanditiswidelyrecognisedthatteachersshouldtargetteachingbasedonreliableevidenceofwhatstudentsknowandarereadytolearn(Griffin2014;Masters2013p.15;AndersonandScamporlino2013;CentreforEducationStatisticsandEvaluation2015).

TheAmericanpsychologistDavidPaulAusubelhasstated:‘IfIhadtoreduceallofeducationalpsychologytojustoneprinciple,Iwouldsaythis:themostimportantsinglefactorinfluencinglearningiswhatthelearneralreadyknows.Ascertainthisandteachhimaccordingly.’(CitedinMasters2013).

Theinabilityofthevastmajorityofteachersinthesampletoidentifylowperformers,evidenceteacherjudgementsconcerningpupilperformanceanddiagnosethenextstepsofteachingpresentsseverechallengestoimprovingpupillearninginKatsinaandZamfara.


©EDOREN 62

Figure17: Percentageofteachersachievingwithinthelower,middleandupperbandsofcompetenceacrosstheteacherknowledgeandskillssubscales

Thewritingskillsofteachersarealsoanareathatraisesconcernsregardingtheirabilitytoincreasepupillearningoutcomesatscale.Whilenoteacherswerefoundtobecompetentinwriting,justover2%displayedarudimentarylevelofwritingskills.Inordertoinvestigatetheextenttowhichteachers’knowledgeandskillsareassociated,ananalysiswasundertakenofthecorrelationbetweenthesubscalesdevelopedfromtheteacherknowledgeandskillsassessment.Thetablebelowshowsthecorrelations(Pearson’s)betweenthesubscales.

Table13: Correlationbetweensubscalesofteachers’knowledgeandskills

Syllabu

sKno

wledg

e

Abilityto

iden

tifylowperform

ers

Abilityto

evide

ncejudg

emen

tsand

diagno

se

Teache

rwritingskills

Teache

rHau

sakno

wledg

e

Teache

rcom

preh

ension

skills

Interpretin

gwordsand

phrases

‘Syllabusknowledge’

PearsonCorrelation

1

Sig.(two-tailed)

N 2935

‘Abilitytoidentifylowperformers’

Correlation .051 1

Sig.(two-tailed) .218

Abilitytoidentifylowperformers

Abilitytoevidence

judgementsanddiagnose

Teacherwritingskills

TeacherGrade2Hausa

Knowledge

Teachercomprehension

skills

Interpretingwordsandphrases

UpperBand 2.3 0 0 39.9 33.6 2.0

MiddleBand 6.2 .2 2.1 23.1 15.3 2.9

LowerBand 91.5 99.8 97.9 37.0 51.0 95.2

0102030405060708090

100

Percen

t


©EDOREN 63

‘Abilitytoevidencejudgementsanddiagnose’

Correlation .224** .188** 1

Sig.(two-tailed) .000 .000

‘Abilitytobuildonpupilknowledge’

Correlation -.103* .083* .415**

Sig.(two-tailed) .013 .043 .000

‘Teacherwritingskills’Correlation .272** .527** .476** 1

Sig.(two-tailed) .000 .000 .000

‘TeacherHausaknowledge’

Correlation .219** .273** .171** .214** 1

Sig.(two-tailed) .000 .000 .000 .000

‘Teachercomprehensionskills’

Correlation .068** .246** .134** .227** .228** 1

Sig.(two-tailed) .000 .000 .001 .000 .000 1

‘Interpretingwordsandphrases’

Correlation .159** .141** .157** .528** .124** .225**

Sig.(two-tailed) .000 .001 .000 .000 .000 .000

*Correlationissignificantatthe0.05level(two-tailed),and**thecorrelationissignificantatthe0.01level(two-tailed).

Therearefourcorrelationsinthistablethatarenoteworthy(showninred):

• ‘Abilitytobuildonpupilknowledge’with‘Abilitytoevidencejudgementsanddiagnose’(r=0.415)

• ‘Teacherwritingskills’with‘Abilitytoidentifylowperformers’(r=0.527)

• ‘Teacherwritingskills’with‘Abilitytoevidencejudgementsanddiagnose’(r=0.476)

• ‘Interpretingwordsandphrases’with‘Teacherwritingskills’(r=0.528)

Thesecorrelationssuggestthatteachers’writingskillsinparticularmaybethekeydimensioninfluencingteachers’performanceinthetest,andperhapsalsotheperformanceoftheteachersinthesetasksintheirclassrooms.Inotherwords,theliteracylevelsoftheteachersappeartobeakeyissue:iftheselevelsarelowtheylimitperformanceacrossarangeoftheareasteachersneedtobecompetentintoimprovepupillearning.

Qualitativereviewofteacherresponses

Severalthemesemergedfromaqualitativereviewofincorrectteacherresponsestoitemsintheteacherknowledgeandskillsassessment.Examplesofresponsesfromtwosectionsoftheteacherassessmentareprovidedwithinthissectionofthereport,thatrepresenttheemergingthemes.Onesectionoftheteacherassessmentprovidedalistofpupilnameswithinaclass,thegenderofeachchildandthenumberofwordseachchildcanreadperminute.Teacherswereaskedto:(1)describetheachievementofgirlsandboysintheclass;(2)identifywhichpupilsneedthemostassistanceandsupport;and(3)provideexamplesofwaysinwhichateachercouldsupportpupilstoincreasetheirabilitytoreadandunderstandtext.Thestimulusmaterial(listofnamesandreadingproficiency)describedaclasswhereabouthalfofthepupilsscoredzeroandtherewasasmalldifference


©EDOREN 64

betweenboys’andgirls’achievement,withboysscoringslightlybetterthangirlsonaverage.Teacherswerealsoprovidedwithexamplesofpupils’writingandwereaskedtodescribewhattheteachershouldfocusontoimprovethepupil’swriting.

Teachersgenerallyperceivedpupileffortasthecauseoflowperformanceandthereforeperceivedincreasingpupil(andtosomeextentteacher)effortasthebestresponsetolowperformance.EarlylearningexampleAprovidesanexampleofateacherresponsethatfocusesonincreasingeffortandpayingattentioninclass,ratherthandisplayingknowledgeregardingtheerrorsinthepupil’sworkandhowateachermightmeaningfullyimprovethepupil’swriting.

EarlylearningexampleA:Studenteffort

Secondly,extremelylowlevelsofteachercompetencewereobserved.Thiswasevidencedinteacherresponsesthatclearlyhighlightedtheinabilityoftherespondingteachertodifferentiatebetweensubjects.InearlylearningexampleBtheteacherproposesusingstonestohelpthepupilscount,inordertoincreasepupils’abilitytoreadandunderstandtext.

EarlylearningexampleB:Differentiatingbetweensubjects

Thirdly,teacherresponsesreflecteddevelopmentprogrammeaimsandobjectivesinsuperficialorunevidencedways.ExamplesoftheseresponsesincludeearlylearningexamplesCandD,inwhichteacherresponsesstatetheimportanceoffocusingongirlsintheclasswithoutanysupportingstatementslinkingthisimportancetotheidentificationofgirlsaslowperformersintheclass.InexampleD,theteacher(incorrectly)statesthattheperformanceofboysandgirlsisequal,butthenstatesthatgirlsneedmoresupportandassistancefromteachers.

EarlylearningexampleC:Assistinggirls


©EDOREN 65

EarlylearningexampleD:Assistinggirls

Whilesometeachersrespondedwith‘politicallycorrect’statementsregardingassistinggirlsintheclassroom,otherresponsespointedtogenderbiases:perceivinggirlsasnaturallylesssuitedtothedemandsofeducationthanboys,ormoresuitedtootherroleswithinsociety.EarlylearningexampleEreflectsoneteacher’sperceptionregardingtheinherentlackofcapacityofgirls.

EarlylearningexampleE:Girls’educationandGenderroles

Thesethemeswerereflectedacrosstheearlylearningteacherassessmentresponses.Collectively,theysuggestthatteachersdonotknowhowtodrivepupillearning,whichincludesteachersbeingunabletoascertainwhichactivitiesarebestsuitedtoimprovereading.Attimesthisincludes


©EDOREN 66

proposingactivitiestoincreasenumeracy,ratherthanreadingskills.Thefocusoneffortwithinthiscontextislikelyareflectionofthelowlevelsofcompetenceamongstteachers–ifteacherslackboththeknowledgeandskillstoimprovepupillearningtheyhavefewexplanationsforlowperformanceandfewavailableoptionstoimprovepupillearning,beyond‘tryingharder’.

Responsesstatingthatteachersshouldfocusongirlsintheclasswerenotsupportedbyjustificationsforsuchafocus,didnotreflecttheneedsofgirlsintheclassroomandinsomeinstanceswereunderpinnedwithresponsesregardingthelimitedcapacityorsocietalrolesofgirls.Thisindicatesthatmanyteacherslackeitherthewilland/ortheknowledgeandskillstoimprovegirls’learningintheclassroom.

Overall,thesefindingsstarklyhighlighttheverylowlevelsofteachers’knowledgeandskillsinmostofthedomainsthatarerelevanttoearlygradeteaching.Ofparticularsignificancearethefindingsthat60%ofteachersarethemselvesnotfullyproficientinGrade1and2levelHausa,andthatanoverwhelmingmajorityareunabletoassesspupils’levelsoflearning,whichiscriticaltoeffectiveteaching.ThishasafewimplicationsforGEP3.First,ithighlightssomeofthekeyissuesthattheearlylearninginterventionwouldneedtoaddressthroughanappropriatecombinationofteachertrainingandcarefullytailoredteachingandlearningmaterials.Second,itprovidesanindicationofthescaleofthechallengeconfrontingtheprogramme,whichcouldfeedintodesigndecisionsrelatedtothefrequencyoftrainingandthelevelatwhichitispitched.Finally,itsuggeststhatbaselinelevelsofteacherknowledgeandskillscouldunderminetheprogramme’simpact,andthatthismeritsregularmonitoringbytheprogrammeimplementationteam.

Whatdoteachersdointheirlessons?

Thepracticesofteachersintheclassroomwerecategorised,onthebasisoftheclassroomobservations,intothreegroups:teachertalk,teacheractionandpupilaction.

Analysisoftheclassroomobservationdataonteachertalkindicatesthatteachersmakesgreateruseofrote-basedapproachesthanofpupil-centredapproaches.Ofthedifferenttypesofteachertalkthatwereobserved,threeareconsideredtobepupil-centred:askingorrespondingtoanopenquestion,assistingingroupwork,andusingachild’snameinclass.AshighlightedinFigure18,onlyaminorityofteachersengagedinthesetypesofteachertalk.In10%ofclassroomsteacherswerenotpresentatsomestageduringtheongoingclassroomobservation.Thisdoesnotincludeteacherswhofinishedtheclassearly.


©EDOREN 67

Figure18: Percentageofobservedteachersengagingindifferentkindsofteachertalk

Teachers’actionsdidincludetheuseofsomepupil-centredapproaches.Inparticular,mostteachersmovedamongstpupils.However,veryfewusedatextbookorothermaterials.

Figure19: Percentageofobservedteachersengagingindifferenttypesofteacheraction

Pupilactioninclassroomsisalsoausefulindicatorofthepedogogicalapproachbeingundertakenbytheteacher.Highlevelsofnon-roteactivitiyareusuallyanindicationthatatleastsomepupil-centredteachingandlearningintakingplace.Thefollowingtypesofpupilactionfallintothiscategory:groupdiscussion,grouporworkinpairs,respondingtoanopenquestion,askingtheteacheraquestion,usingatextbook,readingaloud,anddoingindividualwork.Thisisincontrasttochanting,listening,andrespondingtoclosedquestions.AsshowninFigure20,theuseofrote-basedapproacheswas

100.0

68.1

19.5

45.3 42.6

60.8

10.00.0

20.0

40.0

60.0

80.0

100.0

Instructs Remindstheclasstopayattention

Assistsgroupdiscussion

Usesachild'sname

Asksorrespondstoanopenquestion

Asksorrespondstoaclosedquestion

Teacherisnotpresent

98.7

11.7

72.0

90.2

16.6 19.1

0

10

20

30

40

50

60

70

80

90

100

Writesorreadsfromtheblackboard

Givesdictation Demonstrateshowtodosomething(includingusingtheblackboard)

Movesamongthepupils

Usesatextbook Usesmaterials


©EDOREN 68

moreprevalentthantheuseofnon-roteactivities.Inparticular,groupdiscussion,groupworkandinstancesofchildrenaskingtheteacheraquestionwereobservedinaverysmallminorityofclassrooms.

Figure20: Percentageofclassroomswherepupilsengagedindifferenttypesofpupilaction

3.4.2.3 Teachers’knowledge,skillsandpracticesaspertheToC

ThemaincausalassumptionunderlyingtheToCfortheRANAinterventionisthatliteracylearningoutcomes,particularlyintheHausamothertongue,willimproveinearlygradesifteachingpracticesimprovethroughtheuseofimprovedteachingandlearningmaterialsandthepresenceofmoreknowledgeableandskilledteachers.AsrepresentedintheToCdiagraminSection4.1.3,thefirstcausalchainstatesthatthetrainingandmentoringofleadteachersandheadteacherswillleadtoanincreaseinteachers’knowledgeinliteracyandlanguageacquisitionintheearlygradesandanimprovementinteachers’instructionskills,includingtheuseofgender-sensitiveteachingpractices.Thiswill,inturn,leadtoanimprovementinHausa-basedliteracyteachingintheearlygrades.ThesecondcausalchainstatesthatthedistributionoftheRANALiteracyPackwillleadtoanincreaseduseofHausa-basedteachingandlearningmaterials,whichwillalsoimprovetheHausa-basedliteracyteachingintheearlygrades.Thissubsectionreportsontheintermediaryoutcomesforthesetwocausalchains.

Teacherknowledgeinliteracyandlanguageacquisitionintheearlygrades

AsdiscussedinSection3.2.9,teachersdrawonthreetypesofknowledgewithinclassroompractice.Subjectknowledgereferstoknowledgeoftheessentialquestionsofthesubject,thenetworksofconcepts,theoreticalframeworkandmethodsofinquiry.Pedagogicalknowledgereferstoknowledgeofthelearnersinthesetting,knowledgeofhowtoprovidetheconditionsthatenablepupilstounderstandcontent,andtheselectionoflearningandassessmentmaterials.Curriculumknowledgereferstoknowledgeaboutwhatshouldbetaughttoagroupofstudents,knowledgeofthenationalsyllabus,understandingoftheschoolandgrade-levelplanningdocumentsandknowledgeofthecontentofexaminations.

56.7

19.0 20.2

43.2

61.1

8.3 2.8

60.2

38.2

60.2

89.4

0102030405060708090

100


©EDOREN 69

Basedonthismodel,threecompositeindiceswerecreatedfromthedata–oneforeachoftheseknowledgetypes.Thesubjectknowledgeofteachersismeasuredthroughacompositeindexofteachers’writingskills,teachers’Hausaskills,teachers’comprehensionskillsandteachers’abilitytointerpretwordsandphrases.Thepedagogicalknowledgeofteachersismeasuredthroughacompositeindexofteachers’abilitytoidentifylowperformersandevidencejudgementsanddiagnoselearninggaps.Thecurriculumknowledgeofteachersismeasuredthroughthesyllabusknowledgesubscaledevelopedfromtheteacherknowledgeandskillsassessment.Theabsolutevaluesofteachers’scoresontheseindicesarenotinherentlyinformative,butratherprovidebaselinevaluestoassessimprovementsovertime.However,differencesinscoresbetweengroupsareofinterestatbaseline.Theshareofteacherswhoscoredzeroontheseindicesisalsoworthnoting.

Levelsofpedagogicalknowledgeatbaselinewereextremelylow,withover90%ofteachersscoringzeroontherelevantcompositeindex(seeFigure21).Thisindicatesthatthevastmajorityofteacherswereunabletodemonstrateproficiencyinknowledgeofthelearnersinthesetting,knowledgeofhowtoprovidetheconditionsthatenablepupilstounderstand,andintheselectionofappropriatelearningandteachingmaterials(pedagogicalknowledge).Therewereveryfewdifferencesinlevelsofpedagogicalknowledgeacrossdifferentgroupsofteachers.Teacherswithaprofessionaleducationqualificationdemonstratedslightlyhigherlevelsofpedagogicalknowledgethanthosewithoutone,althoughthedifferencewassmall.Therewerenoclearpatternsbyschooltypeorstate,andteacherswhohadpreviouslyattendedGEP3/UNICEFtrainingdemonstratedsimilarlevelsofknowledgetothosewhohadnot.

Levelsofsubjectknowledgewerealsolow,withsomevariationacrossgroupsofteachers.Overall,justoverone-fifthofteacherswereunabletodemonstrateanyproficiencyintheessentialquestions,conceptsandmethodsofenquiryinliteracyandlanguage(subjectknowledge).TeachersinpublicprimaryschoolsdemonstratedslightlybettersubjectknowledgethanthoseinIQSs.TeachersinKatsinadidslightlybetterthanthoseinZamfara,andteacherswithaprofessionaleducationqualificationperformedslightlybetterthanthosewithoutone.TherewerenocleardifferencesinthesubjectknowledgeofteachersbystateorbyattendanceatGEP/UNICEFtraining.


©EDOREN 70

Figure21: Percentageofteachersscoringzeroinpedagogical,curriculumandsubjectknowledge

Curriculumknowledgelevelswere,similarly,lowacrossallgroups,withalargeminorityofteachers(43%)unabletodemonstrateproficiencyinknowingwhatshouldbetaughttoagroupofstudents.Teacherswithaprofessionaleducationqualificationdemonstratedslightlyhigherlevelsofcurriculumknowledgethanthosewithoutone,andteachersinZamfarademonstratedslightlyhigherlevelsofcurriculumknowledgethanteachersinKatsina.Therewerenocleardifferencesbyschooltype.

Improvedinstructionskillsinactivelearningandincreasedtimeontask

AspertheRANAToCitisproposedthatanincreaseinteachers’knowledgeinliteracyandlanguageacquisitionintheearlygradesandanimprovementinteachers’instructionskills,includingtheuseofgender-sensitiveteachingpractices,willleadtoanimprovementinHausa-basedliteracyteachingintheearlygrades.

Beyondmeasuringtheproportionofgirlsandboyswithintheclassroomsetting,themeasurementofgendersensitivityisextremelycomplexandoftencannotmeetreliabilitycriteria.Forexample,withinthefacilitatorquestionnaireseveralitemsweredevelopedtomeasureattitudestowardsgirls.Acrossallitemsextremecomplianceeffectswereobserved,significantlycallingintoquestionthevalidityandreliabilityofthemeasure.Therefore,gender-sensitiveteachingcouldnotbemeasuredthroughclassroomobservations.Inlinewiththisconclusion,alternativeapproachestoassessinggender-sensitiveteachingwillbeexploredatmidline.Forthepurposesoftheevaluation,theRANAinterventions’impactongender-sensitiveteachingcanbeassesseddespitethelackofbaselinedataonthisvariableastheRCTwillallowustocomparemidlinedatabetweentreatmentandcontrolschools.BecauseofbudgetaryconsiderationsqualitativeresearchhasnotbeenincludedintheELevaluation.However,thedifficultyofcompilingarobustmeasureofgender-sensitiveteachingfrombasedonquantitativedatacollectionmethodshashighlightedtheneedtoconsiderincludingqualitativemethods.Thiswillrequirefurtherdiscussion.

0 10 20 30 40 50 60 70 80 90 100

Percentofteachersscoringzero

PercentofteachersscoringzeroPedagogicalknowledge 91.5Curriculumknowledge 43.4Subjectknowledge 22.2


©EDOREN 71

AcompositeindexwasdevelopedinordertomeasurechangesintheToCintermediaryoutcome–teachers’instructionskillsinearlygrades–frombaselinetomidlinetoendline.Scoresonthecompositeindexarebasedontheextentofpupil-centredlearningactivitiesobservedintheclassroom,observationsoftheteacherlinkingthelessontopreviouslearningandlearningobjectives,andtimeontaskinclass.

1. Themeasureoftheextentofpupil-centredlearningisbasedonwhetherteacherswereobservedengaginginthefollowingpractices:assistingingroupdiscussion,usingachild’sname,askingorrespondingtoanopenquestion,movingamongthepupils,usingavailablematerials,engagingingroupdiscussion,facilitatinggroupworkorworkinpairs,facilitatingopenpupilquestions,facilitatingpupilreadinginclassandrespondingtoopenquestions.

2. Observationsoftheteacherlinkingthelessontopreviouslearningandlearningobjectivesincorporatedobservationsoftheteachertalkingaboutthepreviouslesson,outliningtheobjectivesoftheobservedlesson,revisitingtheobjectivesofthelessonattheendofthesessionandsummarisingthelesson.

3. Timeontaskwasmeasuredbythepercentageoftimethepupilswereengagedduringlessonobservations.Theclassroomobservationtoolrecordedarangeofpupilbehavioursatthree-minuteintervals.Ifpupilswereengagedinanyactivityrelatedtolearning,thiswascategorisedastimeontask.

Overall,teachers’performanceonthecompositeindexwaslowacrossallgroups.Publicprimaryschoolteachersreceivedslightlyhigherscoresinoverallteacherpracticesandtherewerenocleardifferencesbyschooltypeorstate.Therewerealsonoobservablepatternsconnectingteacherpracticesandteachermotivationorperceivedteacherefficacy.Thisindicatesthatteachers’perceivedself-efficacyiseithernotrepresentativeoftheiractualcompetenciesorthelevelsofcompetencyinteacherpracticesaresolowthatdifferencescannotbedetected.ThisisaninterestingfindinginlightoftheToCassumptionthatteachermotivationinfluencesthetranslationofteacherknowledgeandskillsintoteachingpractices.Itwillbepossibletotestatendlinewhetherimprovementsinteachers’pedagogicalcompetencyandteachingpracticesduetotheearlylearninginterventionwillalsochangetherelationshipwiththeirmotivation.Endlinefindingsonpotentialchangesinthisrelationshipfromthebaselinewillthusbeofparticularinterest.

Overallscoresinpupil-centredteachingwerelow,withslightlyhigherlevelsofpupil-centredteachingobservedinKatsina,inpublicprimaryschoolsandforfemaleteachers.Oneobservationofinterestisthatwhileonaverageslightlyhigherlevelsofpupil-centredteachingwereobservedasteachers’experiencelevelsincreased,thelevelsofpupil-centredteachingdipslightlyataround10yearsofexperienceandthendivergesignificantly,withbothhigherandlowerlevelsofpupil-centredteachingoccurringafteraround20yearsofteachingexperience.


©EDOREN 72

Figure22: Meanpupil-centredteachingscorebyyearsofexperienceasateacher

Timeontaskwasmeasuredbythepercentageoftimethatpupilswereengagedduringlessonobservations.Thiswasclassifiedasthetotaltimeduringwhichpupilswereperforminganyactionrelatedtolearning(forthedifferenttypesofactionsthatwererecorded,seeFigure20).Theclassroomobservationtoolrecordedpupilactionsatthree-minuteintervalsduringthelesson.Thesedatawereusedtoestimatetimeontask.Onaverage,pupilswerefoundtobeengagedfor96%oflessontime.Thevastmajorityofclassrooms(roughly85%)werefoundtobeontaskforover90%ofthelesson(seeFigure23).However,thesefindingsshouldbeinterpretedcarefullyasitislikelythatthepresenceofobserversintheclassroomincreasedthepercentageofon-tasktimeinlessons.

Figure23: Percentageoftimeontask

0

5

10

15

20

25

0 5 10 15 20 25 30 35

MeanPu

pil-C

entred

TeachingSc

ore

Years'ExperienceasaTeacher

Mean

Linear(Mean)

.2 1.0 1.2 3.19.7

84.9

0

10

20

30

40

50

60

70

80

90

100

Percentageofclassrooms

Lessthan50 50-59.9 60 - 69.9 70-79.9 80-89.9 90-100


©EDOREN 73

UseofHausa-basedteachingandlearningmaterials

ThesecondcausalchainintheToCstatesthatthedistributionoftheRANALiteracyPackwillleadtoanincreaseduseofHausa-basedteachingandlearningmaterials,whichwillalsoimproveHausa-basedliteracyteachingintheearlygrades.However,aspertheprogrammeassumptionsoutlinedinSection3.1.3,increasesinHausa-basedteachingandlearningarebasedontheassumptionthatthematerialsarealignedwiththecompetencylevelsofteachers.Therefore,itisproposedthattheknowledgeofteachers,theinstructionalskillsofteachersandthepresenceofappropriateHausa-basedteachingandlearningmaterialswillcollectivelyincreaseHausa-basedteachingintheearlygrades.

InordertoexploretheavailabilityanduseofHausa-basedteachingandlearningmaterials,datawerecollectedduringlessonobservationsontheHausamaterialsavailableforboththesubjecttaughtinthelessonobservedandothersubjects,andtheuseofHausamaterialswithinthelesson.MaterialsinHausawereobservedbeingusedin2.4%oftheobservedlessons.MaterialsinHausawereavailableforthesubjectobserved,butwerenotusedinafurther1%ofobservedlessons.Inabsoluteterms,Hausaresourceswereavailablein72ofthe2,235lessonsobserved,andwereusedin53ofthose72lessons.Interestingly,almost18%ofteachersindicatedthattheyhavetheHausamaterialstheyneedtodotheirjobsintheteacherquestionnaire.Thisindicatesthatapossible14%ofteacherseitherdonotseeaneedforHausamaterialsorhaveaccesstomaterialsthatwerenotobservedorusedduringlessonobservations.

ThekeyindicationthatemergesfromthesebaselinefindingsisthataccesstoanduseofHausamaterialiscompletelyinadequate.ThedistributionofapackageofteachingandlearningmaterialsaspartofRANAisthereforehighlyrelevantintermsofprovidingaccesstosuchmaterials.InlightofthelowHausaliteracylevelsoftheteachersitwillbeimportantforthematerialstobewelladaptedtothelowteachercompetencylevels,andtoensurethatpeer/supervisorysupportisregularlyavailabletofacilitatetheiruse.

Hausa-basedteachinginearlygrades

Section0discussesteachers’knowledgeinliteracyandlanguageacquisitionintheearlygradesandteachers’instructionskillsaspertheToC.TheToCproposesthatincreasesinteachers’knowledgeandinstructionskillsandincreasesintheuseofHausa-basedteachingandlearningmaterialswillleadtoanimprovementinHausa-basedliteracyteachingintheearlygrades.

AcompositeindexwasdevelopedinordertomeasurechangesinHausa-basedteachingintheearlygrades.ThemeasuresincludedinthecompositeindexweredevisedusingthepercentageoftimetheteacherspokeHausainclassandteachers’Hausaskills.Typically,teachersinIQSsscoredhigheronthiscompositeindexthanteacherswithinprimaryschools,teachersinKatsinascoredhigherthanthoseinZamfara,andteacherswhodonotspeakEnglishandthosewhodonotspeakArabicscoredhigherthanteacherswhodospeaktheseotherlanguages.

InallclassroomstheuseofHausawasobservedonatleastoneoccasionandinapproximatelyhalfoftheclassesasecondlanguagewasusedinadditiontoHausa.


©EDOREN 74

Figure24: PercentageofclassroomsusingEnglish,HausaandArabic

ThefrequencyofHausausewasdocumentedduringclassroomobservationsandameasureofthepercentageoftimetheteacherspokeHausawascalculated.TherewerenolessonobservationswhereHausawasusedmorethan60%ofthetime.Inaboutthree-quartersofobservedlessons,Hausawasspokenbytheteacherbetween30%and60%ofthetime.

Figure25: FrequencyofHausauseinclass

TeacherswhohavebeenpreviouslytrainedinteachinginHausatypicallyspokeHausainclassforaslightlyhigherproportionofclasstime,asdidteachersinKatsina.MaleteachersalsospokeHausamorefrequently.Nodifferencewasobservedbetweenteacherswhodoanddonotholdaprofessionaleducationqualification.Teachersover50typicallyspokeHausamoreinobservedlessonsthantheiryoungercolleagues(upto10%more)andteachersinIQSstypicallyspokeHausa

0

10

20

30

40

50

60

70

80

90

100

English Hausa Arabic

IQS PS

.0

5.0

10.0

15.0

20.0

25.0

30.0

ShareoftimethatHausawasusedduringlesson

Shareoflesson

sobserved

Lessthan10

10-19.99

20-29.99

30-39.99

40-49.99

50-59.99

60andover


©EDOREN 75

forapproximately5%moreoftheobservedlesson.OtherlanguagesspokenbytheteacherinclassroomobservationsincludedArabicandEnglish.

3.4.2.4 Teachermotivationandattendance

Teachermotivation

Teachermotivationisanimportantaspecttoconsiderasitissupposedlyrelatedtoteachers’willingnessandabilitytoeffectivelyimpartknowledgetopupils.Thisisalsowhyourcompositeindexofmotivationisincludedinourregressionmodelthatlooksintostatisticallysignificantcorrelationsbetweenlearningandrelevantexplanatoryvariables.MotivationisthereforeseeninouranalysisasapotentialinfluencingfactorthatcanhelpexplainhowpupilsachievetherecordedlevelsofliteracyinHausaandEnglish.Duetosamplesizelimitations,andthealreadydiscussedskeweddistributionofteacherknowledgeandpedagogyscales,itisnotfeasibletoundertakethesametypeofanalysisoftheassociationandcorrelationbetweenteachermotivationandpupilknowledge.However,theinclusionofbothsetsofinformationinourregressionmodelallowsustocontrolforanydetectedsimultaneouseffectsofknowledgeandmotivationonlearning.

Teachermotivationisinvestigatedthroughtheuseofarangeofmotivationscaledscores,whichformthebasisfortheconstructionofacompositemotivationindex.Thesubscalescoverthefollowingareas:

• perceivedteacherefficacy;• interestin,andenjoymentof,teaching;• effortputinto,andperceivedimportanceof,teaching;• pressureandwork-relatedtension;and• teacher-to-teacherinteraction.

Theanalysisoftheindicationsemergingfromacomparisonofthedifferentmotivationsubscalesshowsthat,onaverage,thesurveyedteachers(wehave474observationswithdataavailableforthemotivationanalysis)scorehighestoneffortandlowestonperceivedteacherefficacy.Atthesametime,interestinandenjoymentofteachingisobservedtoberelativelyhighwhencomparedtopressureandwork-relatedtension.Thisseemstosuggestthatoverallteachersconsidertheirroletobeimportantandenjoyworkingasteachers,whilstbeingconsciousofthefacttheyarelimitedintheirabilitytocontributetopupils’learning.Itisalsointerestingtonotethattheteacher-to-teacherinteractionscoreseemstoindicatearelativelyhighlevelofcollaborationamongstteachers,whichcouldhelpthedevelopmentofspillovereffectswithinschoolsofanyteacher-specificintervention,includingteachertraining.

Thechartbelowpresentsavisualanalysisofthedifferencesacrossmotivationsubscales.


©EDOREN 76

Figure26: Comparisonofmotivationsubscales

Theoverallteachermotivationindexwasconstructedasacompositemeasureofallteachermotivationsubscales,withequalweightingforallsubscales.Whilsttheaveragemotivationindexvalueacrossthewholeteachersampleisnotparticularlyinformative,itisanalyticallyinterestingtolookatthedifferenceinmotivationscoresacrosscategoriesofinterestforouranalysis.Whenitcomestogender,itappearsthatfemaleteachersareslightlymoremotivatedthantheirmalecounterparts,althoughoursub-sampleoffemaleteachersisquitesmall,asalreadydiscussed.Intermsofage,thereseemstobelittledifferenceinmotivationacrossagegroups,althougholderteachers(i.e.over50)tendtobeslightlylessmotivatedthanyoungerteachers.

Figure27: Motivationbyagegroups

1.99

3.57

3.73

2.19

3.47

0 0.5 1 1.5 2 2.5 3 3.5 4

Perceivedteacherefficacy

Interestandenjoymentofteaching

Effortandimportance

Pressureandworkrelatedtension

Teacher-to-teacherinteraction

2.91

2.92

2.93

2.94

2.95

2.96

2.97

2.98

2.99

3

3.01

TeachermotivationcompositeindexUnder25 25to34 35to49 Over50


©EDOREN 77

Generallyspeaking,themotivationindexshowsarelativelyhomogeneoussituationacrosssomeofthekeycategoriesofinterestfortheanalysis.Urbanandruralmotivationisverysimilar,withteachersinurbanschoolsonlyslightlymoremotivatedthantheirruralcounterparts.NodifferencesareobservedbetweenteachersinKatsinaandZamfaraorteachersinIQSsorpublicprimaryschools.

Ascouldprobablybeexpected,receivingasalaryhelpsmotivation,thoughagainthedifferencewithteacherswhodonotreceiveanypaymentisverysmallandthestandarderroroftheestimatesdonotallowforanystatisticallysignificantinterpretationofdivergingpatterns.Evenwhenfurtherinvestigatingthetypeofpaymentnotrendinregardtointerestseemstoemerge,asteacherswhoreceivetheirsalaryalwaysinfullandontimearefoundtohaveonlymarginallyhighermotivationthantherestoftheteachers.Similarly,havingaqualificationorahigheracademicdegreeisassociatedwithonlyslightlyhigherestimatesofoverallmotivation,butagainwithinastatisticalmarginoferror.

Interestingly,havingattendedtrainingdoesnotseemtohaveanydiscernibleeffectonteachers’motivation.Asalreadydiscussedabove,oneoftheintermediaryoutcomeassumptionsoftheearlylearninginterventionisthatateacherwouldbemotivatedtoimprovetheirknowledgeandskills,thusattendingandmakingthemostoutofthetrainingopportunities.However,thelackofadescriptiveassociationbetweenprevioustrainingandmotivationseemstosuggestthatthiscannotbetakenforgranted.Theendlinewilloffertheopportunitytoassesswhetherastrongercorrelationbetweentrainingandmotivationwillemergeoncetheearlylearninginterventionhasbeenimplemented.

Teacherattendance

Ourbaselinedataonabsenteeismshowsthat60%ofteachersindicatedhavingbeenabsentatleastonceinthelastthreemonths.Thegreatmajorityofteachersidentifytheirownillnessortheillnessofafamilymemberasthemainreasonfortheirabsence,withsocialorreligiousobligationsandweatherconditionsalsoemergingasdeterminingfactors.Theestimatedaveragenumberofdaystheywereabsentinthesameperiodofthreemonthsisjustunderfivedays.Gender-specificanalysisshowsthatmaleteacherswereabsentforslightlylongerthanfemaleteachers,with4.9daysofabsencecomparedto4.6days94,withthesesmalldifferencespointingtowardsanegligiblegenderinequality.Itisinterestingtonotethatwhilstyoungerteachersweremorelikelytobeabsentatleastonceduringthepreviousthreemonths,olderteachers(over50yearsofage)wereabsentforlonger,justover6.2days.

InZamfarateacherswerebothmorelikelytobeabsent(67%comparedto57%)andtendedtobeabsentforlonger(5.8dayscomparedto3.5)thaninKatsina.Atthesametime,absenteeismisfoundtobemoreprevalentinIQSsthaninprimaryschools;inthiscase,thedifferenceintheaveragenumberofdaysateacherwasabsentforisparticularlynoticeable,withIQSteachersabsentforapproximatelyeightdaysonaverage,comparedtojustoverthreedaysinprimaryschools.However,attendancedataonIQSsneedtointerpretedwithcautionastheIQSteachingscheduleisnotasstructuredasinprimaryschools,whichleadstoamoreflexibleteacherattendanceinIQSsandcouldpartiallyexplainthedifferenceintheabsenteeismrates.Giventhefactthatteachersattendingschoolandclassesisindeedanassumptionunderlyingthesuccessfulimplementationoftheearlylearninginterventionoutputs,thedetecteddifferencebetweenpublicprimaryschoolsandIQSsneedstobetakenintoconsideration.Inparticular,itseemsadvisablefortheimplementerstodevise94Thesefiguresaccountforthefactthatpartoftheteachersamplehasbeenabsentfor‘zerodays’inthelastthreemonths.


©EDOREN 78

trainingschedulesthathelptheprogrammetoreachthelargestpossibleaudienceofteachersinIQSs.Atthesametime,itwillbeimportanttoincentivisetrainedteachersandleadteacherstoregularlyattendtheirclassessoastoensurethattheirimprovedknowledgeandteachingpracticesarepassedontopupilsinschool.

Notsurprisingly,whenlookingatpaymentstatus,teacherswhodonotreceivesomeformofsalaryorstipendaremorelikelytobeabsentandtobeabsentforlongerperiods.WhilstthelikelihoodofbeingabsentishigherforteacherswhodonotgetpaidinbothIQSsandpublicprimaryschools,thelongerlengthofabsenteeismforunpaidteachersseemstobedrivenbypublicprimaryschoolteachersasinIQSsbothpaidandunpaidteachersarefoundtobeabsentforsimilarlylongperiods,asdiscussedabove.Absenteeismisalsomoremarkedamongstteacherswithnoprofessionalqualification.However,whenlookingattheassociationbetweenattendingatrainingcourseandteacherattendance,theredoesnotseemtobeanydiscernibletrend.Theproportionofabsentteachersandthelengthoftheiraverageabsenteeismareverysimilarbetweenthetwocategories(whicharealmostequallysplitinourearlylearningsample).

3.4.3 Pupilsinearlygrades

Thisthirdsectiondescribesthepupilswithinthesettingbeingdiscussedandsummarisestheircurrentlevelsoflearning.

3.4.3.1 Pupils’backgroundcharacteristics

Theearlylearningsampleofpupilsthatformsthebasisforourevaluationisequallydividedbetweenmaleandfemalepupils,withtheformeronlyslightlymoreprevalent.Theaveragepupilageisjustovernineyears,withsurveyedIQSpupilsolderonaveragethanpublicprimaryschoolpupils.However,alargenumberofobservationsinourearlylearningsampledidnotreportage:around35%ofoursampleofpupilsdidnotprovideinformationonage.Theagegroupcategorisationreportedinthegraphbelowshowsthatthekeydifferencebetweenthetwotypesofschoolinregardtopupilagedistributioniscentredaroundthelowerandupperendoftheagedistribution.Amongthosewhoreportedage,nopupilsinpublicprimaryschoolswerereportedtobeovertheageof16,comparedtosome5%ofpupilsinIQSswhowereaged16ormore;atthesametime,alargerproportionofveryyoung(belowsevenyearsofage)pupilsisfoundinpublicprimaryschoolswhencomparedtoIQSs.

Thechartbelowshowsthisdifferentdistributionofpupilagegroupsbyschooltype,whendiscountingtheproportionofpupilsinoursampleforwhomageinformationwasnotpresent.


©EDOREN 79

Figure28: DistributionofagegroupsinIQSsandpublicprimaryschools

WhilstallpupilswerefoundtospeakHausaathome,practicallynochildspeaksEnglishoranyotherlocallanguageathome.Around98%ofoursamplewasattendingP2oranequivalentlevelatthetimeofthesurvey,whichisinlinewiththesamplingstrategyadoptedfortheearlylearningevaluation.Furtherdetailsonpupilschoolattendancecannotbeinvestigatedrobustlyduetoalargenumberofmissingvaluesandissuesrelatedtohowinformationwasgatheredonthisvariable.95

ThedivisionofthesamplebetweenIQSsandpublicprimaryschoolsisunbalancedtowardsthelatter,with60%attendingpublicprimaryschoolsand40%attendingIQSs.Whilstover50%ofthesampledchildrenwerefoundtobecurrentlyattendinganotherschoolotherthantheoneinwhichtheyweresurveyed,anadditional11%ofpupilswerefoundtoalsohavebeenattendingotherschoolsinthepastwheremaths,readingandwritingweretaught.Onapositivenote,only3%ofpupilsreportedfacingsomedifficultyingettingtoschool.

Whilsttherural/urbandifferentiationshowsaclearimbalance,withover80%ofpupilsattendingschoolsinruralareas,thepupilsampleisalmostequallydividedbetweenthestatesofKatsinaandZamfara,withonlyafewmorepupilsattendingschoolsinZamfara.Adescriptiveanalysisofpupils’socioeconomicconditionsindicatesthatmostpupilsareinthemiddlerangeofthepovertyspectrum.ThelatterisinvestigatedthroughtheuseofaHouseholdWealthIndex(HWI)(assetindex)thatweconstructedfortheearlylearningsample,asdescribedinAnnexJ.Inparticular,whenlookingatthewealthtertiledifferentiation,theaverageacrossthesampleissetaroundthesecondtertile.

Mostpupilbackgroundcharacteristicsareconsistentacrossourmainanalyticalcategoriesofinterest.Thereare,forinstance,nodetectablegenderdifferencesacrossthemainpupilcharacteristics,withboysonlyalittlebitolder(justovernineyearsofage)onaveragethangirls(just

95Ontheonehand,thepresenceofpupilsduringthesurveywascalculatedbyreferencetoP2pupilsandwasbasedonmanualcountingduringlessonobservations.Ontheotherhand,thenumberofpupilsnormallyintheclassesissolelybasedonteacherreporting.Inaddition,thereareveryfewschoolsforwhichbothattendanceandenrolmentrecordswereavailable.

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

IQSsPublicschools

Below7 7-10years 11-15years Above16


©EDOREN 80

undernineyears)inthesampleofpupilsthatreportedageinformation.However,amarkeddifferenceisobservedbetweenthesocioeconomicstatusofthesub-samplesofpupilsbelongingtothetwostatesandthetwotypesofschoolwhichformpartoftheearlylearningsample.Specifically,pupilslivinginKatsinaandattendingIQSswerefoundtobepoorerthanthoselivinginZamfaraandattendingprimaryschools.ThegraphbelowshowsthatthelesspoorconditiononaverageofpupilsinZamfaraisdrivenbyadisproportionatelylargerpercentageofpupilsbelongingtothehighest(third)tertileoftheHWI.Theimportanceofpupilwealthstatuswillemergeasanimportantfactoraccountingfortheirlearningperformanceinourregressionanalysis.

Figure29: HWItertilecategorisationbystate

3.4.3.2 Pupillearningoutcomes

Learningoutcomesaretheresultofacomplexandmultivariateschoolsystem,andofteacherandstudentfactors,whichareinfluencedbysocialandeconomicconsiderations.Thewaysinwhichthesefactorsinteractarecomplexandcanneverbefullymappedoraccountedforinananalysis.

Significantly,oneofthemostimportantpredictorsoflearningoutcomesiswhatthepupilsbringwiththemintotheclassroom.Innateability,socialandculturalfactors,aswellashousehold-specificfactors,significantlypredictpupilperformanceinmoststandardisedassessments(Hartas2011,OuthredandBeavis2012,OuthredandBeavis2013,Mayer,1997,DahlandLochner2005,VanderBer2015).

Learningoutcomemeasurement

TheHausaliteracyassessmentisdesignedtotestthesameliteracyknowledgeandskillsastheEnglishliteracyassessment.Itemsarenotmerelytranslated,butratherparallelitemsaredevelopedtotestsimilarconceptswhenappliedtotheHausalanguage.TheEnglishliteracyassessmentcontains13items,witheachitembeingmadeupofseveralsub-items.

Theassessmenttestsarangeofliteracyknowledgeandskillsacrossthepre-literacy,emergingandbasicliteracyranges.Knowledgeandskillsincludeletterrecognition,phonologicalknowledge,

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Katsina Zamfara

Lowtertile Mediumtertile Hightertile


©EDOREN 81

printconcepts,oralliteracy,verbalcomprehension,initialsoundsandletters,readinghighfrequencywords,verbalandwrittengrammar,writinghighfrequencywords,readingfluency,copyingandspellinghighfrequencywords.

EnglishliteracyandHausaliteracyassessmentswereconstructedfollowingfivesteps,includingclarifyingconstructs,testtargeting,administration,psychometricanalysis,drawingbenchmarksandsecondarydataanalysis.

InthecaseofnorthernNigeria,pupillearninglevelshavebeenfoundtobelowinpreviousstudies,includingpreviousGEP3learningassessments,ESSPINCSs,andtheTDPbaselinesurvey.Itisimportanttoseektounderstandtheextenttowhichlevelsoflearningarepredictedbypupil-,teacher-andschool-levelfactorsinordertoidentifypolicy-andprogramme-levelinputstoimprovelearningoutcomes.

Figure30: Categoriesoffactorsthatinfluencelearningoutcomes

Figure30summarisesthefactorsthatarethoughttoaffectpupils’learningoutcomes.ThissectionofthereportdescribespupillearningoutcomesanddescribesthelearningachievementofGrade2pupilswithinthesampledpublicprimaryschoolsandIQSsinKatsinaandZamfara.ThefollowingsectionsdescribethelearninglevelsofpupilsinHausaandEnglishliteracyandinvestigatethefactorsassociatedwithpupillearninglevels.Theanalysisaimstoprovideaclearerpictureoftheextenttowhichtheobservedlearningoutcomescanbeexplainedbyvariousfactors(e.g.pupils’background,teachermotivationandschoolresources)andtheextenttowhichcontributingfactorsareamenabletopolicyandprogrammeintervention.

Approachtoanalysingdataonlearningoutcomes

TheanalysisofdataonlearningoutcomespresentedinthisreportuseestimatesofpupilachievementbasedonRaschmodelling.Thisapproachallowsforvalidcomparisonstobedrawnacrosslearningassessmentsadministeredtodifferentgradesandoverdifferentyears.Assuch,ascaledscoreisthemathematicaltransformationofapupil’srawscoresinordertoreportherscoreonacontinuumconsistentlyovertheyears,andacrossdifferentversionsoftheassessment(BangladeshDirectorateofPrimaryEducation,2013;Outhred,2015).Raschanalysisalsoallowsfortestdifficultyandpupilabilitytobereportedindependentlyonthesamescale.Inaddition,ascaledscoreof,say,500willmeanthesameatendlinein2017asitdidatbaseline


©EDOREN 82

in2014.

Rawscoresarefrequentlyusedtocalculatepupillearningperformancebyanalysingthepercentageoftestitemsthatthepupilansweredcorrectlyorthepercentageofpupilsansweringoneormoreitemscorrectly.Althoughthisisarguablytheeasiestwayofgradingtests,fromastatisticalpointofviewrawscoreshaveseveraldrawbacks(Outhred,2015).First,whencalculatingapercentagescore,eachassessmentitemisgivenanequalweight,whichimplicitlyassumesthateveryitemisequallyindicativeofthelevelofknowledgeorskillsthatthepupilpossesses.Inreality,thisislikelynottobethecase.Theuseofrawscoresalsolimitstheextenttowhichassessmentresultscanbecomparedovertimeandacrosslocations.Finally,rawscoresconfoundthedifficultyleveloftheassessmentwiththeabilityorknowledgelevelofthepupils.Itisnotpossibletoseparateoutthesetwocomponents,whichjointlydeterminethepercentageofquestionsastudentanswerscorrectly.Bearinginmindtheserelativemeritsofscaledversusrawscores,scaledscoreshavebeenusedfortheanalysisofdataonlearningoutcomesinthisreport.

Hausaliteracylearningoutcomes

ThefindingsoftheHausaliteracyassessmentindicatethatthemajorityofpupilshaveyettoacquiretheknowledgeandskillsinHausaliteracyexpectedbytheNigeriancurriculum.ItisexpectedthatpupilswhoarebeginningP2willhavemasteredtheP1curriculumandwillbereadytoreceivetheP2curriculum.However,theresultsindicatethatonly5.3%ofpupilsareperformingwithintheexpectedrange.Afurther2.6%ofpupilsareabletodemonstrateemergingliteracyskills,while92%ofpupilsareunabletodemonstrateliteracyskillsinHausabeyondpre-literacyskills.

Table14: Hausaproficiencyleveldescriptions

Proficiencyrange Descriptionoftheknowledgeandskillsofpupilsachievingwithinthisrange

Pre-literacyHausaPupilswhoachievedwithinthepre-literacyrangewereabletodemonstratesomeofthefollowingskills:knowledgeofprintconcepts,identifytheinitialletterinhis/hernameandwritetheinitialletterinhis/hername.

EmergingHausaliteracy

Inadditiontotheskillsabove,pupilsachievingwithinthisrangewereabletodemonstrateatleastsomeoftheknowledgeandskillswithintherangeexpectedbytheP1curriculum.Pupilsachievingwithinthisrangewereableto:soundoutlettersounds,spellsomehighfrequencywords,andreadashortpassagewithlimitedaccuracy.

BasicHausaliteracy

Inadditiontotheskillsabove,pupilsachievingwithinthisrangewereabletodemonstrateatleastsomeoftheknowledgeandskillswithintherangeexpectedbytheP2curriculum.Pupilsachievingwithinthisrangewereableto:identifysimilarsounds,readhighfrequencywords,spellhighfrequencywordswithaccuracy,copyasentence,soundoutlettersounds,andreadashortpassage.

Figure31representsthedistributionofperformanceinHausaliteracy.TheyaxisrepresentstheHausascalescorederivedfromthepsychometricanalysisofpupilperformanceintheassessment(formoreinformationseeSection3.2.9.1).Ascanbeseeninthefigure,thepeakofthedistributionfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,indicatingthat


©EDOREN 83

substantialeffortwouldberequiredtoachievelargegainsinthepercentageofpupilsmovingfrompre-literacytoemergingliteracy.

Figure31: DistributionofHausaliteracyproficiency

Englishliteracylearningoutcomes

ThepurposeoftheEnglishliteracyassessmentwithintheevaluationoftheGEP3earlylearninginterventionistoestablishabaselineofEnglishproficiencyinordertoexploretherelationshipbetweenimprovedmother-tongueinstructionandtransitionintoLanguage2(L2)inthelateryearsofprimaryschool.EnglishliteracyscoreswerelinkedtotheESSPINEnglishmetricthroughlinkitems.ThisenablesthedirectcomparisonofresultswiththeESSPINCSfindingsinsixNigerianstates.

TheresultsoftheEnglishliteracyassessmentindicatethatthevastmajorityofpupilshaveyettoacquireeitheremergingorbasicEnglishliteracyskills.Under1%ofpupilsdemonstratedatleastsomeoftheskillsthatfallwithinthebasicEnglishproficiencyrangeand3.3%ofpupilsdemonstratedatleastsomeoftheskillsthatfallwithintheemergingliteracyrange.However,96%ofpupilscouldnotdemonstrateliteracyskillsinEnglishbeyondpre-literacyskills.TheassessmentfindingsalsoindicatethatpupilproficiencyinHausaaftermorethanayearofschoolingisnotsignificantlyhigherthanpupilproficiencyinEnglish.

0 1000 2000 3000 4000 5000 6000 7000

263.23

344.18

391.83

432.47

468.25

499.18

526.67

551.65

574.30

594.73

612.97

629.03

643.14

655.91

668.05

680.30

693.62

709.37

730.13

759.89

799.60

848.81

934.78

2.6%ofpupilsdemonstratedknowledgeandskillswithintherangeexpectedbytheP1curriculum.

Thislevelofproficiencycanbedescribedasemergingliteracy.

Over92%ofpupilsdemonstratedpre-literacyHausaskills.Childrenachievingwithinthepre-literacyproficiency

rangedonotyethaveemergingliteracyskills.

5.3%ofthepupilsdemonstratedGrade

2Hausaliteracyskills.Thislevelofproficiencycanbedescribedasbasicliteracy.


©EDOREN 84

Table15describestheknowledgeandskillsofpupilsachievingwithineachoftheproficiencyranges.TheseproficiencyrangesarealsodirectlycomparabletotheESSPINCSproficiencyranges.

Table15: Englishproficiencyleveldescriptions


Pre-literacyEnglish

Pupilswhoachievedwithinthepre-literacyrangewereabletodemonstratesomeofthefollowingskills:knowledgeofprintconcepts,understandingandrespondingverballywithagrammaticallycorrectsentencetoasimplequestionabouttheirage,understandingandrespondingverballywithagrammaticallycorrectsentencetoasimplequestionabouttheirname,andsayingtheinitiallettersoffamiliarobjectandanimalnamesandwords.

EmergingEnglishliteracy

Inadditiontotheskillsabove,pupilsachievingwithinthisrangewereabletodemonstrateatleastsomeoftheknowledgeandskillswithintherangeexpectedbytheP1curriculum.Pupilsachievingwithinthisrangewereableto:verballycomposeashortgrammaticallycorrectsentenceinthecontinuouspresenttenseinresponsetoaquestionaboutapicture,listentoashortpassageandrememberspecificdetailstorespondverballytoaquestion,andtocopywordsthatwereclearlyshapedandcorrectlyorientated,withanunderstandingofspaceandfullstops.

BasicEnglishliteracy

Inadditiontotheskillsabove,pupilsachievingwithinthisrangewereabletodemonstrateatleastsomeoftheknowledgeandskillswithintherangeexpectedbytheP2curriculum.Pupilsachievingwithinthisrangewereableto:usephonicknowledgetosaytheinitialsoundsoffamiliaranimalnames;listentoashortpassageandrememberspecificdetailssoastorespondverballytoaquestion(one-wordanswerswereacceptable;useknowledgeofcommoninflectionsinspellings;displayknowledgeofplurals;writetheanswertoaquestion;usephonicknowledge(andawareness)toreadupperandlowercaseletters;spellsimplehighfrequencywordsaccurately;readhighfrequencywordsandphonicallydecodabletwo-syllableandthree-syllablewordsthatincludecommondiagraphsandadjacentconsonants(e.g.,black)insimplesentences;understandandrespondinwritingwithagrammaticallycorrectsentencetoasimplequestionaboutthepositionofaneverydayitem;listentotwosentencesandrespondverballytoaquestionwithagrammaticallyaccuratesentence;independentlyreadformeaningashorttextwitharangeofsentencestructures,highfrequencywords,andtwo-syllableandthree-syllablewordsthatincludecommondiagraphsandadjacentconsonants;verballycomposeashortgrammaticallycorrectsentenceinthecontinuouspresenttenseinresponsetoaquestionaboutapicture;copywordsthatareclearlyshapedandcorrectlyorientated,withanunderstandingofspaceandfullstops;useappropriateintonationwhenreadingtextswitharangeofsentencestructures,highfrequencywords,andtwo-syllableandthree-syllablewordsthatincludecommondiagraphsandadjacentconsonants;useknowledgeofcommoninflectionsinspellings,includingplurals,towritetheanswertoaquestion;readarangeofsimplesentenceswithhighfrequencywords,phonicallydecodabletwo-syllableandthree-syllablewordsthatincludecommondiagraphsandadjacentconsonants(e.g.,black)independently;rememberspecificdetailsfromashortsimpletextreadtorespondverballytoaquestion;andreadasimplesentenceformeaningandcompleteamissingwordusingthecorrectspelling.


©EDOREN 85

Figure32depictsthedistributionofperformanceacrosstheEnglishliteracyscale.TheyaxisrepresentstheEnglishscalescorederivedfromthepsychometricanalysisofpupilperformanceintheassessment(formoreinformationseeSection3.2.9.1).Ascanbeseen,thecuspofthedistributionfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,suggestingthatsignificanteffortwouldbeneededtoattainlargeadvancesintheshareofpupilsachievingemergingliteracy.

Figure32: DistributionofEnglishliteracyproficiency

3.4.3.3 Discussion

AnanalysisofpupilresponsestoHausaliteracyitemsandthedifficultyofvariousitemsprovidesaninsightintothelearningtrajectoriesofpupilswithinthecontextinwhichRANAoperates.TheanalysisfindsthatitemsthatrequireknowledgeofphonicsrankasthemostdifficultitemsinbothHausaandEnglish.Correctlysoundingoutlettersandidentifyingsimilarsoundsweremoredifficultforpupilsthanwritingorreadingfullpassages.Thisindicatesthatcurrentlypupilswithinthecontextwhohaveachievedbasicliteracyaredoingsowithoutsignificantexposuretophonicsknowledge.

0 2000 4000 6000 8000 10000 12000

202.77

203.56

305.91

337.10

338.50

359.65

361.28

382.01

407.18

411.74

436.34

453.62

460.94

482.66

500.60

513.99

525.58

534.65

551.84

575.49

599.06

624.90

Over96%ofpupilsdemonstratedpre-literacyEnglishskills.Childrenachievingwithinthepre-literacyproficiency


.5%ofthepupilsdemonstratedgradetwoEnglishLiteracyskills.Thislevelof

proficiencycanbedescribedasbasicliteracy..




©EDOREN 86

Thereisevidencethatinmanysub-SaharanAfricancontexts,the‘look-and-say’methodisusedmoreregularlythanthephonicsapproach.Adu-Yeboah’sstudyfoundthatteachersandteachereducators‘tendedtogravitatetowardsthe“look-and-say”approach’andthatpedagogywasprimarilytransition-based(p.29).Thestudyconcludedthatwhilethecurriculumoftenmakestheassumptionthatteachersareskilledintheuseofbothlook-and-sayandphonicsinstruction,eventeachereducatorsthemselvesarenotequippedwiththeknowledgeandskillsrequiredtoteachphonics(Adu-Yeboah,p.70).

Asystematicreviewoftheresearchliteratureontheuseofphonicsfoundthatsystematicphonicsteachingisassociatedwithbetterprogressinreadingaccuracy.However,therewasweakandnosignificanteffectforreadingcomprehension(Torgeson,BrooksandHall,2006).Ifreadingistobeunderstoodasbeingabletounderstandandinteractwithwrittentext,readingfluencyisnotanadequateindicatorofreadingandcomprehensionasanindicatorofliteracyachievementemergesasbeingofimportance.Whiletheuseofphonicsinstructionhasadvantages,ifweunderstandliteracyasbeingabletounderstandandinteractwithwrittentext,readingfluencyisabridgetoliteracy,ratherthanameasureofliteracyitself.

Inaddition,itisimportanttokeepinmindthattheresultsofstudiesinvestigatingthelinkbetweenphonologicalawarenessandreadingaccuracyandcomprehensionprimarilycomefromwithineducationsystemsthatpossesshighlyliterateandqualifiedteachers.Astheresultsoftheteacherassessmentsdiscussedabovestarklyhighlight,theseconditionsdonotholdinthecontextinwhichGEP3isbeingimplemented.Thisraisesquestionsabouttheadditionalresourcesofsupportthatwouldneedtobeprovidedinorderforaphonics-basedapproachtobeeffectivelyappliedinthiscontext.

3.4.3.4 Factorsassociatedwithimprovedlearning

WhenlookingatthemeanHausaandEnglishliteracyscaledscoreacrossthewholesampleofpupils,itispossibletonotesomedescriptivetrendsthatareworthmentioningatthisbaselinestageoftheevaluation,soastoassesswhatfactorsareassociatedwithlearningperformanceandhowthiswillchangeovertime.Interestingly,HausaandEnglishshowaverysimilarpatternacrossmostcategoriesofinterest,thoughwithdifferentscalescoremagnitude,thusimplyingacommonliteracytrendinthetwosubjects.

Themaindescriptiveindicationsofinterestarethefollowing:

• Malestudentsarefoundtoachievehigheraveragescoresthanfemalepupils.Interestinglythough,whenlookingatthedifferencebetweenpupilsinIQSsandpublicprimaryschools,itisnotedthatwhilstmalepupilsperformbetterthanfemalepupilsinIQSs,thetrendisnotmaintainedinprimaryschools,withthetwogendersperformingatpracticallythesamelevel(slightlybetterlearningoutcomesforfemalepupils),asshowninthegraphbelow.


©EDOREN 87

Figure33: Hausaliteracybygenderandschooltype

• Bywayofapartialexplanationofthegendertrendobservedabove,itisimportanttohighlightagainthatpupilsincludedinoursampleofIQSsareconsiderablyolder(almostfouryearsofdifferenceonaverage)thanpupilsinthesampledpublicprimaryschools.Althoughageisaffectedbythemissingobservationissuediscussedabove,thenumberofpupilsforwhichagedataaremissingissimilarforbothIQSsandprimaryschools.Amongstpupilsforwhichwehaveageinformation(thegreatmajorityofourearlylearningsample),olderchildrenperformbetterthanyoungerones,withchildrenover10achievingconsiderablyhigherscoresthanyoungerpupils.

• PupilsinKatsinaperformslightlybetterthanthoseinZamfara,butthisdifferenceisverysmallandnotstatisticallysignificant,withbothstatesshowingscoresaroundthe500pointmarkforHausaandthe350pointmarkforEnglish.

• Asshowninthegraphbelow,olderchildrenperformbetterthanyoungerones,withthoseover16achievingconsiderablyhigherscoresthanpupilsaged11to15andsevento10.Pupilsundertheageofsevenarefoundtoperformworsethantherest.Consideringthatnopupilisagedover16inoursampledpublicprimaryschoolsandthatthepupilsover16intheIQSsscoreonaverage687intheHausascaledscore,thekeyroleofageisundeniable.Atrendemergeswhenreviewingtheachievementofpupilsagainstageandageandgender.AscanbeseeninFigure34,typicallythosepupilswhoaremovingbeyondpre-literacyskillsareintheirlateteens.Thisindicatesthatachievementispossiblyassociatedwithcognitivedevelopmentmoresothanyearsinschool,asthedataindicatetheaverageyearsofschoolingisnotsubstantiallydifferentbetweentheolderandyoungerchildreninthesample.

0 100 200 300 400 500 600 700

Hausaliteracyscalescoreinpublicschools

HausaliteracyscalescoreinIQschools

Female Male


©EDOREN 88

Figure34: Hausameanscalescorebyage(95%confidenceinterval)

• WhendisaggregatingliteracyachievementbyageandgenderatrendemergesacrossbothHausa

andEnglish.Genderdifferencesinperformancearesmallintheyoungeryears,howeveroncegirlsreachthepointinthelifecycleassociatedwithpuberty(around12yearsofage)gendergapsinperformanceincrease.Withinthesample,girlsovertheageof16performedmorepoorlythanboththeiryoungercounterpartsandtheirmalepeers.

Figure35: MeanHausascalescorebyageandgender(95%confidenceinterval)

• Asdiscussedabove,pupilsinIQSsachievedahigherscoreonaveragethanthoseinpublicprimaryschools,whichmaybedrivenbythedifferenceinagesbetweenthetwopupilgroups.Itisworthreportingthisdifference,givenitsmagnitude,aswellasthesignificanceofitscorrelationwithlearninginourregressionanalysis,asdiscussedbelow.

450.0000

500.0000

550.0000

600.0000

650.0000

700.0000

750.0000

800.0000

850.0000

4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

300.0000

400.0000

500.0000

600.0000

700.0000

800.0000

900.0000

4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Boy Girl

Cutpointbetweenpre-literacyandemergingliteracy


©EDOREN 89

• Pupilsattendingschoolslocatedinurbanareasarefoundtoachievehigherscoresthanthoseinruralareas,thoughitisimportanttobearinmindthattheurbansampleisconsiderablysmaller.

• Relativelymorewell-offpupilsperformedbetterthanpoorerones,withaclearincreasingpatterndetectableacrossourconstructedwealthindextertile.Thegraphbelowshowshowthejumpfromthemedium(second)tothehigh(third)tertileisparticularlyaccentuatedforbothHausaandEnglish.Thisisconfirmedbyourrelatedregressionanalysis,whichshowsthat,evenwhencontrollingfortheroleplayedbyotherrelevantinfluencingfactors,thecorrelationbetweenliteracyandsocioeconomicstatusisparticularlysignificantforthetopendofthewealthdistribution.

Figure36: Hausabywealth

3.4.4 Analysisofrelationships

Theconsiderablesizeofthepupilsampleallowsustofurtherinvestigatetherobustnessofsomeofthedescriptiveassociationsdescribedabovebyconstructingaregressionmodelspecification.Thelattercanbehelpfulinordertoinvestigatethestatisticalsignificanceofcorrelationsbetweenouroutcomeindicatorsofinterestatthepupillevel:namelypupils’learningoutcomes(i.e.HausascaledscoresandEnglishscaledscores)andarangeofinfluencingfactorsthatcanhelpexplaintheirlearningperformance.Teacher-levelanalysisislimitedinthisrespectbyarelativesmallsampleanditisthereforenotfeasibletobuildarobustmodelthatfocusesonteachers’correlationsatbaseline.Wehave,however,embeddedinformationonteachersubjectknowledgeandpedagogicalpracticesaswellastheirdegreeofmotivationinourpupilcorrelationmodel.

InBox7wepresentthecategoriesofexplanatoryfactorsthatareincludedinthepupilcorrelationmodel.Theinclusionofthesedifferentexplanatoryfactorsinourregressionspecificationwasinformedbytheanalyticalcategoriesillustratedinourconceptualframeworkoffactorsinfluencinglearningoutcomesandbytheavailabilityandqualityofdatacollectedattheschool,teacherandpupillevel.Thechoiceoffactorsthatwecontrolforinourregressionspecificationsisimportantasitdetermineshowmuchtheeffectofeachindividualvariable(asindicatedbythemagnitudeofits

470

480

490

500

510

520

530

Lowtertile Mediumtertile Hightertile


©EDOREN 90

coefficientandthesignificancelevelreflectedbyitsp-value)detectedcanbeconsideredrobusttootherpotentiallyconfoundingfactors.

Forthisreasonwehavedecidedtousetheavailableinformationonageinourmainregressionspecification,giventhecriticalinfluenceofthisdemographiccharacteristiconlearning.Asdiscussedabove,though,agedataarenotavailableforallpupils;therefore,thesamplesizeforourmainregressionis1,613observations.Althoughthisstillrepresentsasizeablesamplefortheanalysis,thereisariskofbiasintheregressionestimatesduetotheexclusionofpupilsforwhichthereisnoinformationonage.Inordertodeterminewhetherthisisinfactanissue,wehaveundertakentwodifferentrobustnesstests.Ontheonehandwehaveimputedtheagegroupdistributionofpupilsforwhichinformationismissingonthebasisofthedistributionofthesamplewithageinformation,separatelyforIQSsandprimaryschools.Ontheotherhand,wehavealsobuiltaregressionspecificationwithoutanyagevariable,butinsteadincludingavariableonwhetherthepupilisaboarderornotinstead.Thisvariablewaschosenaspupilswhoareboardersareolderthannon-boarders.Asshownbytheregressionresultstheindicationemergingfromourmainregressionmodel,whichincludestheoriginalagevariable,isconfirmedbythesetwoalternativespecifications.Hence,thisgivesusconfidencethatagecanbecontrolledforasoneofthekeyfactorsinfluencinglearning.

Theinclusionofpupil-,teacher-andschool-levelfactorsentailsthattheeffectofeachvariableisisolatedfromtheconfoundingfactorsthatwecontrolforatthoseanalyticallevels.Atthesametime,wehavealsobuiltaschool-levelfixedeffectsmodel,whichallowsustofocusontheinfluenceofpupil-levelvariables,whilstcontrollingforallschool-levelconfounders(observableandunobservable).

Box7:Explanatoryfactorsincludedinthepupilscorrelationmodel

• Pupilpersonalcharacteristics,includinggenderandagegroup• Pupilsocioeconomicstatus,asdefinedbytheHWItertilecategorisation• Pupilschoolinformation,includingwhetherpupilsattendpublicprimaryschoolsorIQSs,

whethertheyattendotherschoolsinparallelandwhethertheycanwrite• Locationinformation,includingwhethertheschoolisinKatsinaorZamfaraandwhetheritis

locatedinanurbanorruralarea• Schoolinfrastructure96,includingwhethertheschoolhasseparatetoiletsforgirlsandwhetherit

hasaccesstoasourceofdrinkingwater• Teacheraveragemotivation,asdefinedinouroverallmotivationindexillustratedabove,inthe

schoolattendedbythepupil• Teacheraveragesubjectandpedagogicalknowledge,asdefinedintherelevantindexes

illustratedinthesectionabove,intheschoolattendedbythepupil:• teacherknowledge1:teachersubjectknowledge;• teacherknowledge2:teachersyllabusandcurriculumknowledge;• teacherpedagogy1:teacherpedagogicalknowledge;and

96Othervariablesforschoolcontextinformation,includingforinstanceelectricityorpupil/teacher,teacher/classroom,pupil/classroomandpupilandteachergenderratioswerenotincludedastheywouldhavereducedtheregressionsamplesize.Dataonthefactorsaboveandotherschoolaspectswereinfactcollectedonlyfromalimitednumberofschools.Informationonclasssizewasalsocollectedaspartofthelessonobservationexercisethroughmanualcountingineachschool.Whenincludingthisvariableinthemodelnosignificantcorrelationisdetectedandtheotherexplanatoryvariablesapproximatelymaintaintheirdirection,magnitudeandsignificance.Inanycase,thisalternativeclasssizevariablehasnotbeenincludedinthecorrelationmodelspecificationasitisnotrepresentativeoftheaverageclasssizesinourearlylearningschools.


©EDOREN 91

• teacherpedagogy2:mother-tongueteachinginearlygrades.

ThetablesbelowreporttheresultsobtainedwithourmainregressionmodelsonfactorsinfluencingHausascaledscoresandEnglishscaledscores,respectively,inourearlylearningpupilsample.Thetablesreportthecoefficientestimatedforeachexplanatoryvariable,theirassociatedstandarderror,t-statisticsandp-values,whichprovideanindicationofthelevelofsignificanceinthecorrelationbetweeneachexplanatoryvariableandtheoutcomevariableofinterest,eitherHausaorEnglishscaledscores.

Table16: MainfactorsinfluencingHausascaledscores

Variables Coeff. Stderr T-stat P-value

Gender(pupilisfemale) -9.212 6.294 -1.464 0.145

Agedfrom7to10 23.376 6.828 3.423 0.001***

Agedfrom11to15 114.695 8.856 12.951 0.000***

Agedover16 178.553 20.394 8.755 0.000***PupilinsecondHWItertile 13.769 7.181 1.917 0.056*

Pupilinthird(top)HWItertile 37.849 6.713 5.638 0.000***

Publicprimaryschool(notIQS) -57.353 11.026 -5.201 0.000***Schoolhasgirltoilets 8.479 7.486 1.133 0.259

Schoolhaswater 8.971 7.026 1.277 0.203

Pupilattendsotherschool -5.628 6.903 -0.815 0.416Schoolinruralareas -32.710 11.559 -2.830 0.005**

SchoolinKatsina 3.438 6.683 0.514 0.607

Teachermotivation 12.749 16.729 0.762 0.447

Teacherknowledge1 25.998 14.323 1.815 0.071*Teacherknowledge2 2.246 4.721 0.476 0.635

Teacherpedagogy1 -3.103 24.650 -0.126 0.900

Teacherpedagogy2 0.414 0.364 1.137 0.257constant 446.139 57.053 7.820 0.000***

Numberofobservations:1,613

R-squared0.339

Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.

Table17: MainfactorsinfluencingEnglishscalescores

VariableNames Coeff. Stderr T-stat P-value

Gender(pupilisfemale) -6.752 5.706 -1.183 0.238

Agedfrom7to10 18.687 6.733 2.775 0.006**


©EDOREN 92


R-squared:0.332


Thefirstclearindicationemergingfromtheresultsisthatteacher-andschool-levelfactorsdonotappeartobesignificantlycorrelatedwithliteracylearningoutcomes.Inparticular,noneoftheteachermotivation,knowledgeorpedagogyindexesarefoundtobesignificantlyassociatedwitheitherHausaorEnglishscores.Previousmodeliterationsthatincludedeachindexseparatelyalsoconfirmthatnocorrelationisdetected.Thelackofadetectablecorrelationbetweenteacherknowledgeandskillsandlearningoutcomesseemstobeinlinewithwhatemergedfromthedescriptiveandteacherassessmentanalysespresentedabove.Inparticular,theverylowcompetencylevelsrecordedforteachersinearlylearningschoolscanhelpexplainwhytheircurrentsubjectknowledgeandteachingpracticesarenotinfluencingpupils’learningoutcomes.

Atthesametime,schoolcontextvariables,suchaspresenceofseparatetoiletsforgirlsandpresenceofasourceofdrinkingwaterintheschool,alsodonotshowanysignificantcorrelationwithHausa.Whilstthestatelocationoftheschool,ineitherKatsinaorZamfara,doesnothaveanyassociationwithliteracyoutcomes,thelevelofurbanisationisshowntobecorrelatedwithlearning.PupilsattendingschoolsinruralareasarefoundtoachievelowerHausaandEnglishscoresthanthoseinurbanareas,acorrelationthatissignificantinstatisticaltermsandslightlystrongerforHausathanEnglishintermsofmagnitude.Finally,animportantfindingofourcorrelationanalysis,whichseemstoalsoconfirminthiscasetheindicationsofthedescriptiveassociationsdiscussedintheprevioussections,isthatpupilsinpublicprimaryschoolsappeartoachievelowerliteracyscoresinbothlanguagestestedthanpupilsinIQSs.Thenegativecorrelationbetweenliteracyandpublicprimaryschoolsislarger(sizeofthecoefficient)forHausathanEnglishscores.Whilstthedescriptivestatisticsalsopointedtowardsadifferenceinagebetweenthegroupsofpupils,whichcouldpartially

Agedfrom11to15 82.448 9.185 8.977 0.000***

Agedover16 126.046 14.208 8.871 0.000***PupilinsecondHWItertile 16.665 5.759 2.894 0.004**


Publicprimaryschool(notIQS) -45.609 8.715 -5.234 0.000***

Schoolhasgirltoilets 13.246 6.512 2.034 0.043**Schoolhaswater 7.182 5.873 1.223 0.223

Pupilattendsotherschool 3.371 6.704 0.503 0.616

Schoolinruralareas -28.869 9.372 -3.080 0.002**SchoolinKatsina 4.834 5.710 0.847 0.398


Teacherknowledge1 15.479 12.668 1.222 0.223Teacherknowledge2 -0.142 3.710 -0.038 0.970


Teacherpedagogy2 0.192 0.350 0.550 0.583

constant 278.692 53.062 5.252 0.000***


©EDOREN 93

explainthisdivergence(forthemajorityforwhichagedataareavailable),theresultsofourregressionanalysisareimportantastheyareobtainedbycontrollingforthepotentialroleplayedbyotherinfluencingfactors,suchasage.Inotherwords,thestatisticallysignificantbetterperformanceofIQSpupilsisconfirmedalsowhendiscountingforthepotentiallyconfoundingeffectofpupils’age.Importantly,thisfindingisconfirmedinbothrobustnesschecksperformedwithalternativeregressionspecificationsthatmakeuseofthewholesamplebydealingwiththemissingageinformation.Theotherclearindicationthatemergesfromourregressionmodelspecificationsisthatpupilindividual-levelexplanatoryfactorsaresignificantlycorrelatedwiththeirlearningoutcomes.Notsurprisingly,agecomesoutassignificantlycorrelatedwithlearning,witholderpupilsdoingbetterthanyoungerones.TheageliteracytrajectoryisprominentforbothHausaandEnglish,aspupilsdoincreasinglybetterastheygetolder.Themagnitudeofthecorrelationisdisproportionatelylargerforthesecondandespeciallythirdagegroup,thusindicatingthattherelationshipbetweengettingolderandachievingbetterliteracyoutcomesisnon-linear.Thisisalsoconfirmedwhenimputingageandusingthewholesampleofpupils97.

Genderdoesnotappeartobesignificantlycorrelatedwithliteracyinourmainmodel.Theotherpupil-levelcharacteristicthatemergesascrucialforliteracyistheirhouseholdsocioeconomicstatus.AsshownbytheHWItertilecategorisation,relativelywealthierpupilsarefoundtoperformbetterthantheirpoorercounterparts.ForbothHausaandEnglish,thedifferenceinlearningperformanceisparticularlyaccentuatedinmagnitudetermswhenmovingfromthemiddletothehighesttertile.Thisseemstoindicatethatwhilstwealthisanimportantfactor,thereexistsaneconomicthresholdbelowwhichimprovementsinsocioeconomicstatusarenotsufficienttoaffectlearning.Atthesametime,giventhegenerallylowlevelofliteracyamongstthesurveyedpupils,itseemsreasonabletopresumethatwealthisnotsufficienttoachievegoodlearningoutcomes.Thelackofcorrelationbetween(verylow)teacherknowledgeandpedagogyandlearningmayhelpexplainthis.Asmentionedbefore,ifteacherknowledgeandpedagogyimproveasaresultoftheGEP3interventionandpupillearninginthetreatmentgroupisalsofoundtobebetterthanthecontrolatmidline,wewouldexpecttoseeanincreasedcorrelationbetweenthesetwofactors.

Theanalysisisextendedfurtherbyusingaclusterfixedeffectsmodel(theschoolbeingourcluster),whichenablesustocontrolforallschool-levelinfluencingfactorsatonce,whichinturnallowsustoisolatetheeffectonouroutcomevariablesofinterest(i.e.HausaandEnglishscaledscores)oftheremainingnon-schoolinfluencingfactors.Theseresultsarepresentedinthetablesbelow(Table18andTable19).

Table18: Hausaschoolfixedeffectsmodel


Gender(pupilisfemale) -13.161 6.266 -2.100 0.037**

Agedfrom7to10 15.989 5.891 2.714 0.007**

Agedfrom11to15 83.130 10.829 7.677 0.000***Agedover16 133.901 23.743 5.640 0.000***

PupilinHWItertile -0.598 7.513 -0.080 0.937

97Intheregressionspecificationthatusesboardingasanalternativeexplanatoryvariable,pupilswhoareboarders(olderonaverage)arefoundtoperformbetterthantheothers,especiallyinHausa,forwhichthesignificanceofthecorrelationishigher.


©EDOREN 94

PupilinthirdHWItertile 20.247 6.027 3.360 0.001***



R-squared:0.606


Table19: Englishschoolfixedeffectsmodel


Gender(pupilisfemale) -10.366 5.698 -1.819 0.070*Agedfrom7to10 15.998 7.250 2.207 0.028**

Agedfrom11to15 61.565 10.618 5.798 0.000***

Agedover16 96.922 17.944 5.401 0.000***

PupilinHWItertile 9.296 5.917 1.571 0.118PupilinthirdHWItertile 14.186 5.770 2.458 0.015**



R-squared:0.509


Themainindicationsonindividual-levelinfluencingfactorsareconfirmedbythefixedeffectsmodel,includingtheimportanceofageandsocioeconomicstatus.Inaddition,whencontrollingforallotherschool-levelfactors,genderemergesasbeingsignificantlycorrelatedwithlearning:malepupilsperformbetterthantheirfemalecounterparts.Thesignificantimportanceofgenderisalsoinlinewiththeresultsofourrobustnesscheckmodel.Thefactthatgenderemergesassignificantlycorrelatedwithliteracywhencontrollingforallpossibleschool-levelcharacteristics,whichincludeteachersaswellastheteachingenvironment,indicatesthatbeingamaleorafemalepupilhasaneffectonlearningthatwaspartiallydisguisedbyotherfactorsaffectingbothgroups.Itisnotpossibletopindownwhatalltheseschoollevelfactorsactuallyare,astheyincludebothobservableandunobservableschoolandteachercharacteristics.Itis,however,reasonabletoassumethatthesewouldincludeaspectsthataffectbothboysandgirls,eitherequallyordifferentially.Hence,thespecificinfluenceofgenderonlearningbecomesmoreapparentwhencontrollingforthishostofconfoundingfactors.Itisimportanttopointoutthatthemagnitudeaswellasthelevelofstatisticalsignificanceofthecorrelationbetweengenderandliteracyareinanycasealsolowinthefixedeffectsmodel,at5%significanceforHausa,andespeciallyforEnglishatonly10%significance.

Whilsttherelevanceofdemographicandhousehold-levelfactorsonpupilperformanceisnotsurprising,thelackofaneffectonlearningofteacherknowledgeandpedagogicalpracticesneedstobecarefullyinterpreted.Inparticular,thismaywellreflectthefactthatcompetencylevelsareverylowacrossallteachers.Itislikelythatastrongercorrelationwouldbedetectedifthereweregreatervariationincompetencylevelsacrossteachers,andifteachers’abilitytoinfluencepupils’learning


©EDOREN 95

outcomesweregreater.ThisalsoimpliesthatiftheELinterventiondoesleadtosubstantialimprovementsinteachers’knowledgeandskills,weshouldseeamoresignificantlinkbetweenteachercharacteristicsandlearningoutcomesatendline.

3.4.5 Pre-analysisplanforimpactmeasurementatmidline

ThedescriptiveandcorrelationanalysespresentedabovepresentbaselinebenchmarkvaluesfortheGEP3impactevaluationbyfocusingonthebaselinesituationoftwomainareas:ontheonehand,thebaselineanalysisdrawsasummarypictureofthemainteachers’andpupils’knowledgeandskills,aswellastheirbackgroundcharacteristics;ontheotherhand,theyprovideaninsightintosignificantassociationsandcorrelationsbetweenoutcomeindicatorsandcategoriesofinterest,whichshedlightonthefactorsthatarefoundtoinfluencepupils’learning,includingtheroleofteacher-levelintermediateoutcomes.

Atmidline,therewillbeafirstassessmentoftheimpactofGEP3ontheearlylearninginterventionsampleunderevaluation,whichwillbeinformedbybothsetsofbaselineinformation.QuantitativedatawillbeagaincollectedonHausaandEnglishoutcomes,aswellasteachersubjectknowledgeandpractices.Twosetsofanalysisareplanned:

1. Impactanalysis:quantitativemeasurementoftheimpactoftheGEP3earlylearninginterventiononpupillearningoutcomes(i.e.HausaandEnglishscaledscores),aswellasteachers’knowledgeandskills,bycomparingtheoutcomelevelsoftreatmentobservations(i.e.pupilsandteachers)withtheoutcomelevelofcontrolobservations.Disaggregatedanalysisbygirlsandboyswillbeconducteddependingonthestatisticalpoweravailable,whichinturnwilldependonthesizeofthedisaggregatedsub-samples.

2. Correlationanalysis:analysisoffactorsinfluencinglearningoutcomesaftertheGEP3interventionhastakenplaceandteachers’knowledgeandpracticesaresupposedtohaveimprovedaccordingly.ThisanalysiswillbedirectlycomparabletotheregressionanalysisperformedatbaselinetoassesswhetherthesignificanceofthecorrelationshavechangedasaresultofGEP3.

IftheimpactanalysisshowsthatthereisasignificantdifferencebetweentreatmentandcontrolgroupsattributabletotheGEP3earlylearningintervention,thecorrelationanalysisofinfluencingfactorswillhelpdeterminewhetherthishasalsoaffectedtheroleplayedbyschool-levelfactorsdirectlyinregardtolearning.Ifteachers’knowledgeandpedagogyimprovefromthelowbenchmarklevelsmeasuredatbaseline,itseemsreasonabletoexpectthattheirinfluenceonHausaandEnglishscaledscoreswillalsoincrease.ThiswouldindeedbeinlinewiththeassumptionsbehindtheprogrammeandevaluationToC.Thisdemonstratesthecrucialrelevanceofthebaselineanalysis,withrespecttothemidlineimpactanalysis.

Theeconometricmodelenvisagedformidlinewillalsotakeadvantageoftheadditionalparametersandmodellingoptionsthatwillbeavailableatthatstage.IfthecomparisonbetweentreatmentandcontrolgroupsshowsapositiveandsignificantdifferenceinoutcomeindicatorsattributabletoGEP3itwillbeimportanttoaccountforwhichofthetwogroupspupilsandteachersbelongtoinourregressionspecification,withtheuseofadummyvariable.Thiswillallowustomaintainouroverallsamplesizeintact,whilstdiscountingthedirectinfluenceofinterventionimpactfromtheothercorrelationsinthemodel.Finally,itwillalsobepossibletoexploitthelongitudinalstructureofoursampleinourregressionanalysissoastocontrolfortime-invariantobservedandunobserved


©EDOREN 96

factors,andtofurtherisolatetheeffectofeachexplanatoryvariablewiththeoutcomeindicatorsofinterest.Inordertoassesstherelevanceofthisapproach,theresultsofapanelregressionanalysiswillbecomparedtothoseofarepeatedcross-sectionalsetting.Anydifferenceinthesignificanceofthecorrelationscapturedbythetwomodelscanthenbeattributedtotheexistenceoftime-invariantunobservablecharacteristicscontrolledforinthepanelfixedeffectsmodel.Thisrangeofeconometrictechniqueswillhelpenrichourmidlinequantitativeanalysis.


©EDOREN 97

4 BaselineoftheIQSSevaluation

4.1 GEP3’sIQSS

4.1.1 Objectivesandexpectedresults

GEP3’sstrategyforintegratedQur’aniceducationfocusesontheimprovementofeducationinIQSs,withtheaimofprovidinganacceptablealternativeformofqualitybasiceducationforgirls.98Thefinaloutcomesexpectedare:improvedlearningoutcomesinbasicliteracyandnumeracy(especiallyforgirls),improvedretentionofgirls,and,toalesserextent,increasedenrolmentofgirls.

Atschool-communitylevel,theIQSSfocuses,inthefirstinstance,onstrengtheningthequalityofteachingand,secondly,onimprovingthelearningenvironment,especiallyforgirls,withincreasedcommunityparticipationthroughCBMCs.Atthesupra-schoollevelGEP3seekstoenhancethecapacityofthelocalgovernmentauthoritytosupporteffectivemanagementandimprovecoordinationanddialoguethatcontributestoplanning,coordinationandpolicy.

4.1.2 Interventionstrategy

GEP3targetsregisteredIQTEsthatimplementanintegratedcurriculum(becomingIQSs)andlargelyoperateascommunity-basedinitiatives,butthatarewillingtobuildlinkswithgovernmentforthepurposesofmonitoringandtechnicalsupport.Atleast40%ofthestudentsenrolledshouldbegirlsfortheIQStobesupportedbyGEP3.

Duringthe2015–2017period200IQSsperstateacrossthesixGEP3LGAsinNiger,BauchiandSokotoarereceivingafullschool-levelsupportpackage.Inallstates,exceptBauchi,theIQSsthatarebeingsupportedduringthisperiodwerealreadytargetedduringGEP3’sinitialyears.99Duringthescale-upphase(2017–2020)theprojectseekstoexpandtoanadditional7,500IQSs,ofwhich4,500willreceiveGEP3financialsupporttoimplementthesupportpackage.

TheIQSSpackageatschoolcommunitylevelconsistsofthefollowing:

1. TrainingandmentoringofIQSfacilitators100

GEP3supportscentre-orcluster-basedtrainingthataimstostrengthenfacilitators’classroompracticesandtoensurefacilitatorsbecomemoreeffectiveteachersofcoresubjects.TwofacilitatorswhoteachcorecurriculumsubjectsaretargetedperIQS.Atthecoreofthetrainingaremonthlycluster-leveltraining/mentoringmeetingsbetweenagroupofIQSfacilitatorsandaTeacherFacilitator,whoactsastrainerandmentor.101DuringthemeetingsatacentrallocationintheLGAfacilitatorsreviewthecurriculumforthefollowingmonth,receiveguidanceonit,watchdemonstrationlessons,andhavethechancetogivefeedback.ExperiencedTeacher

98UNICEF(2015b).AftertheGEP3re-designin2014,thefocusofGEP3supportshiftedfromsupportingtheintegrationofQur’anicschoolstoensuringthatQur’anicschools,oncetheyhaveagreedtointegration,areabletoimplementtheintegratedcurriculumatalevelofqualitysuchthatpupilsactuallylearn.99InBauchi,awholenewlistofIQSshasbeenselected,andthereforetheseIQSsmayreceiveGEP3supportforthefirsttime.100TheapproachtothetrainingandmentoringofIQSfacilitatorsisbasedontheapproachandpedagogymaterialsusedbytheTDP.101TeacherFacilitatorsareselectedfromSAMEs,SUBEBsandLGEAs.


©EDOREN 98

DevelopmentTeammembersobservethementoringsessions.Beforethestartofthe1.5-yearcycleofmonthlymeetingstheIQSfacilitatorsattendajointfive-dayinductionworkshop,inwhichtheyareintroducedtobasicpedagogicalconceptsandtheusageoftheteachertrainingmodules.Thetrainingcoversbothpedagogyandsubjectmatter(English,mathematicsandsciences).ThecontentisbasedonthecontentdevelopedbytheTDPforprimaryschoolteachertraining.ThemanualisslightlyadaptedtosuittheIQScontextandistranslatedintoHausa.

AfirstcohortofoneIQSfacilitatorperIQSattendedtheinductionworkshopinAugust/September2015.AsecondfacilitatorfromeachIQSwillstartthetrainingandmentoringcycleintheJuly–September2016quarter.

2. Provisionofapackageofclassroomteachingandlearningmaterials

GEP3doesnotaimtodevelopnewclassroommaterialsforteachersorlearners.102Incoordinationwiththestates,locallyavailableteachingandlearningaidsforlowerprimarygradeswillbeprocuredanddistributedtoeachIQS.Currently,thefollowingmaterialshavebeenidentified:Hausaalphabetchart,Hausaconsonantandvowelflashcards,Hausaalphabettextbook,aHausaalphabethandwritingworkbook,andaHausanumericaltextbook.103

ThematerialsareplannedtobedeliveredtotheIQSsbyApril–May2016.

3. Trainingofheadteachers

AccordingtotheGEP3StrategyPaperonIQSs(UNICEF2015b),headteachertrainingismeanttoimproveheadteachers’centre-basedmanagementandpedagogicalskills.Theactualcontentofthetrainingisstillunderdevelopment.104ItisnotyetclearwhowillbetargetedforthetrainingasnotallIQSshaveaheadteacherposition.GEP3estimatethathalfoftheIQSsmayhaveapersonwhoplaystheroleofheadteacherandwhocouldbetrained.Thetrainingapproachislikelytoconsistofconductingtrainingsessionsofthreedayspertermoveratwo-yearperiod.105

ThetrainingwasoriginallyplannedtostartbetweenAprilandJune2016buttheactualstartdatewilldependonthetimingofthefinalisationofthetrainingmaterials.

4. CapacitybuildingofCBMCs

ThecapacitybuildingapproachforCBMCsfollowsthetrainingapproachforSBMCs,aspromotedbyUBEC,withsupportfromESSPIN.Itconstitutesacombinationofaninitialmulti-daycluster-leveltraining,withfollow-upmentoringvisits,atleastonceaterm,bythetrainerstotheCBMCsoveraperiodofnineto12months.DuringtheinitialtrainingCBMCmembersaretrainedonthreekeysubjects:rolesandresponsibilitiesoftheCBMC,financialmanagementandgender-responsivewholecentredevelopmentplanning.106Acascadetrainingapproachisusedwithnationalmastertrainerstrainingstate-leveltrainingteams.Thestatetrainers—sourced

102 This is an adjustment compared to the GEP3 Strategy Paper on IQSs (UNICEF 2015b), in which the development ofsimplifiedinstructionalmaterialsandlessonplanningguidesforfacilitatorswasplanned.103Morespecifically,itisplannedforeachIQStoreceivefiveHausaalphabetchats,sixHausaconsonantandvowelflashcards,fiveHausaalphabettextbooks,20Hausaalphabethandwritingworkbooks,andfiveHausanumericaltextbooks(CommunicationwithUNICEFd.d.20March2016).104FirstdraftsoftheheadteachertrainingguideandhandbookareexpectedbytheendofMarch2016.TheirdevelopmentisledbyESSPIN(CommunicationwithUNICEFd.d.20March2016).105Thetrainingapproachisstillunderdiscussion(CommunicationwithUNICEFd.d.20March2016).106AccordingtotheGEP3StrategyPaperonSBMCs(UNICEF2015g)therolesandresponsibilitiestrainingwillenablenewmemberstounderstandtheSBMCspolicy,guidelinesontheestablishmentofCBMCsandactivitiesexpectedtobecarriedoutbyCBMCs.Thewholeschooldevelopmenttrainingbuildsthecapacityofmemberstoeffectivelyparticipateinevaluatingthestrengthsandweaknessesoftheirschool,toidentifyimprovementprioritiesandtomobiliseresourcestosupportimplementation.ThefinancialmanagementtraininggivesCBMCsmembersbasicskillstoensureschoolresourcesareeffectivelyusedtoachieveimprovementobjectives.


©EDOREN 99

fromSAMEs,SUBEBs,LGAsandCSOs—conductthecluster-leveltrainingandfollow-upmentoringvisits.GovernmentofficialsandCSOstaffaregroupedintopairstoconductthetrainingandthefollow-upmentoring.ThetrainingtargetsfourmembersofeachCBMC:thechairandsecretaryareexpectedtoattend,plustwoothermembersselectedbythecommunityfromtheCBMCmembership.

Theinitialcluster-leveltrainingisscheduledtotakeplaceinJuly–September2016.

5. Provisionofmini-grants

GEP3providesgrantstosupportCBMCstoimplementpriorityactivitiesidentifiedintheWholeCentreDevelopmentPlan(WCDP)thatdirectlycontributetotheenrolmentorretentionofgirlsintheIQS.107CBMCsareeligibleforGEP3grantsafterattendingtheCBMCtraininganddevelopingaWCDP.AgrantcanbeprovidedtoaCBMCatmosttwiceoveratwo-yearperiod.108CBMCgrantproposalwillbejointlyapprovedbythestategovernmentandUNICEFfollowingarecommendationbytheLGA.109110Grantamountsareplannedtobeanequivalentof$1,000perIQS.111

Grantsareplannedtobeprovidedfromthebeginningofthe2016–2017schoolyear.

Atsupra-schoollevelGEP3conductsadvocacywithSAMEsandSUBEBsinregardtopayingfacilitatorsalariesand/orotherincentives,andprovidingtextbookssuppliedbyUBEC.Government-levelsupportisfocusedonenablingtheinvolvementofSAMEandSUBEBstaffatstateandLGAlevelinthetrainingoffacilitatorsandCBMCmembers,CBMCeffectivenessmonitoringandgrantprovision.CBMCeffectivenessmonitoringwillbeconductedtermly(threetimesayear),startinginOctober–December2016,andwillbecarriedoutbyCBMCtrainersfromSAMEsinallIQSs.Inaddition,independentenumeratorsmanagedbystateGEP3teamswillconductmonitoringinasampleofIQSs.112

Table20presentsthetargetresultsthatGEP3aimstoachieveduringthepilotperiodandscale-upperiod.

107AccordingtotheGEP3StrategyPaperonSBMCsasecondaryobjectiveoftheGEP3grantistoencouragetheprovisionofgrantsbygovernmenttosupportimplementationofWSDPs.108Asecondgrantiscontingentuponsatisfactoryimplementationoftheplannedactivitiesandaccountingforfundsreceived(CommunicationwithUNICEFd.d.29March2016).109ThiscanbethemandateofSAMEsand/orSUBEBs.Thisiscurrentlynotclearandislikelytobespecifiedinthegrantguidelines.110Grantproposalsandapprovalswillbebasedonagreedgrantguidelines,whichareunderdevelopment(CommunicationwithUNICEFd.d.20March2016).111Thegrantamountisbeingdiscussedinordertoharmonisetheamountacrossprojects(amongothers,toharmonisewiththeGPEproject).112GEP3stateteamsandTeacherFacilitatorsmayalsoconductlessonobservationsinordertoprovidefeedbacktofacilitatorclustermeetings.Thisisstillunderdiscussion(CommunicationwithUNICEFd.d.20March2016).


©EDOREN 100

Table20: GEP3targetresultsforIQSS

2015–2017 2017–2020a

Numberoffacilitatorstrained 600 15,000

Numberoftrainedfacilitatorswhoattendedatleast80%ofmentoringsessions

360 5,400

Numberofheadteacherstrained 300 3,750

NumberofIQSswithfunctioningCBMC 420 3,600

Source:GEP3logframea2017–2020targetsonlyincludescale-upschools,notoriginalpilotschools

InNigerandSokotothetargetedIQSsareregisteredwiththeSAME,whichistheleadagencycoordinatingtheimplementationofthesupport.InBauchi,191GEP3IQSsaremanagedbytheSUBEB.TheremainingnineIQSsfallunderBASAME.

4.1.3 Interventionlogic

Figure37presentstheinterventionlogicofGEP3’ssupporttoIQSs,withafocusatthecommunityandschoollevel.WeusedamoredetailedToCasaframeworkforformulatingtheevaluationquestionsduringtheevaluationdesign(seeGEP3’sEvaluationFramework,2015).TheToCwillbediscussedindetailinSection4.2asitformsanintegralpartofthecontributionanalysisapproachthatisusedfortheevaluationoftheIQSS.

ThemainlogicofGEP3’ssupporttoIQSsisthatgirls’learningcanimproveinIQSswhentheIQScanprovidequalitybasiceducation,whichrequiresthatfacilitatorsteachmoreeffectivelyandthattheschoolenvironmentimprovesandbecomesmoregirl-friendly.EffectiveteachingisexpectedtoimprovebyGEP3supportingfacilitatortrainingandmentoring,headteachertrainingandthedistributionofteachingandlearningmaterials.CBMCtrainingandtheprovisionofmini-grantsareexpectedtocontributetoanimprovementintheschoolenvironment.Atthegovernmentlevel,GEP3’sstateadvocacyandLG(E)Acapacitysupportareassumedtocontributetotheimprovementofmonitoringandsupportsupervisionbygovernmentstaffaswellassustainedgovernmentfinancialsupport,whichisexpectedtocontributetotheintermediaryoutcomespresentedinFigure37.


©EDOREN 101

Figure37: InterventionlogicdiagramoftheIQSS

4.1.4 PlannedresearchandM&E

InthissectionwedescriberesearchandM&Eactivitiesthatarecomplementarytotheevaluationandthatcouldbeofuseintheinterpretationoftheevaluationfindings.

• CBMCeffectivenessmonitoring

Fromthelastquarterof2016(October–December)GEP3willsupporttheimplementationoftermlyCBMCeffectivenessmonitoring.ThemonitoringwillbeconductedbyLGAandstateSocialMobilisationOfficers.Inaddition,independentenumeratorsmanagedbytheGEP3stateteamswillcarryoutvalidationdatacollectioninasampleofIQSs.MonitoringwilluseatooldevelopedforSBMCeffectivenessmonitoring.ThetoolcapturesdataonCBMCfunctionality,113theparticipationofwomenandchildren,thepromotionofinclusiveeducationandinformationonenrolmentandaverteddrop-outsduetoCBMCaction.114

• Reviews

IntheperiodApril–June2017GEP3hasplannedareviewofIQSteachercapacitydevelopmentworkinordertoprovidelessonsandevidenceforscale-up.Inthesameperiodanassessmentoftheuseandperformanceofthegrantsisscheduled.

4.2 Methodology

Thissectiondescribesthemethodologythatguidedthedesignandimplementationofthebaselineevaluation,andanalysisofthebaselinedata.Itstartswithapresentationoftheevaluationquestionsandevaluationdesignasaframeworkforthebaselineevaluation’smethodologicalchoices.Subsequently,specificmethodologicalissuesarediscussedwithregardstothequantitativeandqualitativedatacollection.Thesectionendswithadescriptionofethicalandinclusionissues,andmethodologicallimitations.

113Thiscoversinformationoncommunitycommunication,resourcemobilisationandutilisation,WCDPandcommunityparticipation.114SBMCMonitoringToolforSocialMobilisationOfficers,versionOctober2015.


©EDOREN 102

4.2.1 Evaluationquestions

TheToCofGEP3’ssupporttoIQSswasusedasaframeworktoformulatetheevaluationquestions.Theevaluationquestionsinterrogateawiderangeofthecause–effectassumptionsunderlyingdifferentstepsintheToCinordertobetterunderstandhowchangemaycomeabout.Theevaluationquestionssubsequentlyguidedtheoverallevaluationdesignandbaselinemethodology.TheevaluationquestionsarepresentedinBox8.

Box8:EvaluationquestionsforevaluationofGEP3’ssupporttoIQSs

1. HowwellhasGEP3teachercapacitydevelopment(trainingandmentoring)contributedtoimprovedteachers’knowledgeandskills,andmoreeffectiveteachingintheclassroom?

2. HowwellhasGEP3teachercapacitydevelopmentcontributedtoanimprovementingender-sensitiveteaching?

3. HowwellhasGEP3headteachercapacitydevelopment(trainingandmentoring)contributedtoimprovedpedagogicalleadershipandschoolmanagement?

4. HowwellhasGEP3CBMCcapacitydevelopmentcontributedtoimprovedschoolmanagementandincreasedmobilisationofresourcesforschoolinvestment?

5. HowwellhaveCBMCsbeenabletoadequatelymanagemini-grantsandinvesttheseresourcesintheimprovementofagirl-friendlyschoolenvironment?

6. Howfarhaveteachingandlearningmaterialssuppliedthroughtheinterventionbeenperceivedbyteachersandheadteachersasappropriateandwelltargeted?Havetheybeenusedformoreeffectiveteaching?

7. Towhatextenthavepupilliteracyandnumeracyskills,especiallyofgirls,improvedinGEP3-supportedIQSs?HowhasGEP3contributedtosuchanimprovement?TowhatextentdoestheIQSSinterventioncontributetoreducingthegapbetweenlearningoutcomesandexpectedlearningoutcomes,asexpressedinthecurriculum?

8. Amongcommunitymembersandleaderswhataretheattitudesto,andwhatisthelevelofacceptanceof,integratingformalsubjectsintoQur’aniceducation?Howhavetheseattitudesandhowhasthislevelofacceptancechangedduringtheintervention?Why?Howhaveattitudeschangedinregardtogirlsreceivingformaleducation?

9. WhatunintendedconsequencesdoesIQSShaveforteachers,headteachers,pupilsandproprietorswithintheIQSs,aswellasforthebroaderschoolcommunity?

Inadditiontotheintervention’sToC,thechoiceoftheabovequestionshasbeenbasedonthefollowingadditionalconsiderationsandcontextualfactors:

• AspartofGEP3’sredesignthefocusofIQSShasshiftedfromimprovingaccesstoeducationtoimprovingthequalityofeducationinIQSs.Theevaluationquestionsthereforeemphasiseteachingandlearningoutcomesaswellastheteachingandlearningenvironment.

• FromanequityperspectiveitisimportanttounderstandiflearningistakingplaceforallandwhetherGEP3iscontributingtoimprovingthelearningoutcomeofthepupilswhoarecurrentlyfallingbelowexpectedlevels,asexpressedinthecurriculum.Hence,theevaluationquestionsincludeafocusonexaminingthelearningoutcomesofthelowestperformingpupils.


©EDOREN 103

• TheIQSSconsistsofseveralcomponentsandisimplementedinacontext—theQur’aniceducationsystem—thatisdiverseandnotwellunderstood.115Integrationtakesplacewithinaninstitutional,socialandculturalenvironmentthatisunevenlysupportive,withmultiplestakeholders.ThismakestheIQSShighlycomplex,withmultiplefactorsinfluencingthecourseoftheintervention.GiventhiscomplexitytheevaluationquestionsaimtoachieveanunderstandingofhowIQSScontributestointendedchangeswithintheIQScontext,ratherthantoquantifytheattributableeffectsoftheintervention.

• Theinterventioncomplexityintroducesadegreeofuncertaintyregardingtheoutcomesthatcanbeexpected.TheevaluationquestionsthereforepayattentiontounintendedconsequencesoftheIQSS.

4.2.2 Evaluationdesign

MostoftheevaluationquestionspresentedintheprevioussectionrequireinvestigationofthecausallinkbetweentheIQSSandchangesinoutcomevariables.Weproposetheapplicationofanevaluationapproachthatdrawsontheprinciplesofcontributionanalysis.Thisisapragmatictheory-basedevaluationapproachthatassessescausalpathwayswithintheToC,reportswhethertheintendedchangesoccurredornot,andidentifiesthemaincontributionstosuchchanges(DelahaisandToulemonde,2012;Mayne,2012).Inaddition,weproposelookingatunintendedchangestakingplacewithintheIQSsandtheirbroadercontextduetotheGEP3intervention.Weconsiderthisimportantbecause,unliketherelativelywell-researchedpublicprimaryschoolsystem,littleisknownaboutthesystemdynamicsofIQSs.

TheevaluationoftheGEP3’ssupporttoIQSsdoesnotmakeuseofacomparisongrouptomakecausalinferencebecauseofthenatureoftheintervention,itscontextandtheresourcesavailable.IQSScomprisesadiversesetofsupportactivitiesthattakeplaceinacontextthatisdiverseandnotfullyunderstood.ThiswarrantsastrongevaluationemphasisonhowtheinterventioncontributestochangewithintheIQS,ratherthanwhetherchangeisattributabletotheoverallpackageofIQSS.Furthermore,itwasdifficulttoconstructacomparisongroup.SincetheIQSSisprovidedtoallGEP3pilotIQSsintherespectivetargetedstates,acomparisongroupwouldneedtobeselectedfromIQSsthatarenottargetedbyGEP3.NolistofIQSssimilartoGEP3IQSswasavailablefortheconstructionofacomprehensive,unbiasedsampleframefromwhichacomparisongroupcouldbeselected.Therewasalsolittleguaranteethatavalidcomparisongroupcouldbemaintained.Finally,resourceswerenotavailabletoconstructcomparisongroupsforalloftheinterventionsunderevaluation.Prioritywasgiventotheearlylearningintervention.

Attheheartofcontributionanalysisistheaimofbeingabletomakecrediblecausalclaimsaboutthecontributionaprogrammeismakingtoobservedoutcomesinacomplexenvironmentwherethereareanumberofotherinfluencingfactorsatwork(Mayne,2012).Criticaltocontributionanalysisisthedevelopmentofawellthought-outandcredibleToCthatmakesexplicitbothhowtheintervention(orinterventioncomponent,suchasteachertraining)contributestoagivenchange(thisisoftenreferredtoasthecausalmechanism),aswellastheconditionsandinfluencingfactorsnecessaryforthiscontributiontomaterialise.Contributionanalysisalsoinvolvestheassessmentofalternativeexplanationsfortheobservedchanges.Thecredibilityofthecausalinferencedependsontheextenttowhichalternativeexplanationscanbedisprovedandinfluencingfactorsareaccountedfor.Therefore,theevaluationapproachdoesnotseektoquantifythenetattributableeffectofIQSS,

115Forexample,howisQur’aniceducationinterconnectedwiththeformaleducationsystem?


©EDOREN 104

butrathertoassesswhetherandhowtheinterventionismakingadifferenceincombinationwithotherinfluencingfactors.

Thecontributionanalysisapproachfollowsasetofsixmethodologicalsteps,whicharepresentedanddiscussedinBox9inthecontextoftheIQSSintervention.Thesestepsleadtoananalysisbasedontheformulationofacontributionstory:thatis,acompellingcaseaboutthecontributionbeingmadebytheintervention.Thecontributionstoryissystematicallyorganisedintoseveralcontributionclaims,whichassessacausalchaingeneratedbytheinterventioninconjunctionwithselectedinfluencingfactors(referredtoasacausalpackage).AsexplainedinBox9andintherepresentationoftheToCoftheIQSSinSection4.2wehaveidentifiedthreecontributionclaimsthatwillbetheframeworkofbuildingthecontributionstory.

Amixed-methodsapproachcombiningquantitativeandqualitativedatacollectionisappliedtomeasurechangealongtheToC,toassesscausallinkagesandtolookintounintendedoutcomes.QuantitativedatacollectionconsistsofrepresentativesamplesurveysamongGEP3IQSsinthesixGEP3LGAsinbothBauchiandNiger,116involvingtheadministrationofpupilandteachertests,structuredteacherclassroomobservationsandclosed-ended,structuredquestionnairesamongpupils,teachers,headteachersandCBMCmembers.Baseline,midlineandendlinesurveyswillbeconductedinacohortofIQSssampledatbaseline.ThequalitativedatacollectioninvolvesKIIs,focusgroupdiscussions(FGDs),qualitativeclassroomobservationandunstructuredteacherpraxisdiscussionsinpurposivelysampledcasestudyIQSs.ThesamecasestudyIQSswillbevisitedatbaseline,midlineandendline.Thecommunity-/school-levelqualitativeresearchiscomplementedwithandinformedbythefindingsofthestate-levelstakeholderinterviews.

Duringthe2015–2017pilotperiodthefocusoftheanalysisisonGEP3’scontributiontoachievingintermediaryoutcomes,suchaschangesinteacherknowledgeandskills,teachingpracticesandschoolenvironment.Learningoutcomeswillnotbemeasuredatmidlinebecauseofmethodologicalreasonsrelatedtotheuseofacross-sectionalsamplesurvey,whichisrequiredbecausetheevaluationdesigndoesnotincludeacomparisongroup.117Across-sectionalsamplesurveyhastwoimportantconsequencesthatconstrainmeasurementoflearningoutcomesatmidline.First,thiswouldrequiremeasurementinthesameschooltermasbaseline(i.e.October–November2017),whichisnotappropriatetimingfromtheperspectiveofevidenceneeds.Second,weestimatethatwiththegivensamplesize—whichiscircumscribedbytheavailablebudget—onlyaneffectsizeofamagnitudethatisunlikelytooccurin1.5yearswouldbeabletobedetectedwithstatisticalconfidence.Comparedtoapanel,across-sectionalsamplerequiresarelativelylargesamplesizetomeasureagivenMDEsize.Therefore,theeffectonlearningoutcomeswillbethesubjectoftheend-of-projectevaluation,meaningtheevaluationquestionregardingimprovementinlearningoutcomeswillnotbeansweredatmidlinebutatendline.Whileendlinedatacollectionhasnotyetbeenformalised,thereisanagreementthatthiswilltakeplace.StakeholdersindicatedthatevidenceontheeffectivenessofGEP3intermsofimprovingteachingqualityandtheschoolenvironmentwillprovidevaluableinformationforscale-updecision-makingatmidline.Itcanbeassumedthatifthequalityofteachingandschoolenvironmentsimproves,andotherfactorsthatmaynegativelyaffect116ThefullpackageofIQSSisalsoimplementedinSokoto.Duetobudgetconstraintsthesurveycanonlybeadministeredintwostates.NigerandBauchihavebeenpurposivelyselectedbythestakeholders,takingintoaccountbudgetconsiderations.117 A panel approach is not feasible because itwould not be possible to separate out the effect of ageing and years ofschooling from the intervention effect sincewe are not comparing the intervention groupwith a comparison group ofpupils.Thecontributionofsuchalternativeexplanationsfortheimprovementoflearningoutcomeswouldbeimpossibletoassess without a comparison group. In a cross-sectional survey, alternative explanations related to a pupil’s cognitivedevelopmentdonotneedtobeaccountedfor.


©EDOREN 105

learningoutcomesdonottakeplace,learningoutcomeswillimprove.Thisassumptionwillbeverifiedbyendline.

Box9: KeystepsincontributionanalysisappliedtotheevaluationofGEP3’sIQSS

1. Setoutthecause–effectissuestobeaddressedTheevaluationquestionscircumscribethecausalissue.ThemainfocuswillbeonthecausallinkbetweenGEP3facilitatorcapacitydevelopment,teachers’knowledgeandpedagogicalskills,effectiveteacherpracticesandlearningoutcomesinthecontextofIQSs.GiventherelativelylargenumberofcausallinksintheToC,thebreadthoftheanalysiswillneedtobebalancedwithitsdepth,andthereforewewillnotbeabletoinvestigateeverycausallinktothesamedegreeofdepth.2. DevelopapostulatedToCAfirstversionoftheToCwasincludedintheGEP3evaluationframeworkandsupportedtheformulationoftheevaluationquestions.AspartofthebaselinetheToCisfurtherdeveloped(seeSection4.2).3. GatherevidenceontheToCThroughamixed-methodsdatacollectionstrategyatdifferentpointsintime(baseline,midlineandendline)wewillcollectdataonchangestakingplaceintheoutcomevariablesidentifiedintheToC,aswellasassessthestrengthsandweaknessesofthelinksintheToC.ViaaquantitativesurveyinasampleofIQSsinBauchiandNigerwequantifychangesalongtheToC,aswellasotherinfluencingfactors.QualitativeresearchinanumberofcasestudyIQSswillprovideinformationtoallowustotriangulatethequantitativedataandassessthecontributionofGEP3supportamongdifferentfactorsresultinginthechangesobserved.Thisquantitativeandqualitativeschool-leveldatawillbecomplementedwithinformationfromamidlineroundofstakeholderinterviewsatstatelevel.Finally,theGEP3monitoringandresearchwillprovideanothersourceofinformation,inparticularallowingustoassesshowtheIQSSwasimplemented.4. AssembleandassessthecontributionstoryThequantitativeandqualitativeevidencewillbeanalysedtostatisticallyidentifythechangesthathavetakenplace,assessthestrengthsandweaknessesofthecausalmechanismsandexploreinfluencingfactors,consideringalternativeexplanations.TheanalysiswilleitherconfirmthepostulatedToCorwillsuggestrevisionswheretherealityappearsnottomatchtheToC.Basedontriangulationofevidencefrommixed-methodsdatacollectionacontribution‘story’willbesetout,whichwillexplainwhyitisreasonabletoassumethattheprogrammeactionshavecontributedtotheobservedoutcomes.Thecontributionstorywillbeorganisedaccordingtothefollowingthreecontributionclaims:

a. GEP3’ssupporttoIQSscontributestomoreeffectiveteachingofformalsubjectsinIQS;b. GEP3’ssupporttoIQSscontributestoanimproved,girl-friendlyschoolenvironmentinIQSs;

andc. moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlyschoolenvironment

contributetoimprovedlearninglevels,particularlyamonggirls.

ThesethreecontributionclaimscoveralltheevaluationquestionsthatrequireinvestigationofthecausallinksbetweentheIQSSandchangesinoutcomevariables.5. SeekadditionalevidencewhereneededTofurtherenhancethecredibilityofthecontributionclaimssetoutinthe‘story’,additionalevidencecanbesoughtout.Credibilitywillbetestedbyexposingtheanalysistopeerreview.Triangulationofdatasourceswillstrengthenthecontributionclaims(whichisthereasonwe


©EDOREN 106

ThequantitativeandqualitativebaselinedatacollectiontookplaceinOctober–November2015duringthefirsttermofthe2015–2016schoolyear.Atthattime,thefive-dayinductionworkshopaspartofthetrainingofthefirstIQSfacilitatorcohorthadalreadytakenplace(seeFigure38).Forlogisticalreasonsthebaselinecouldnottakeplaceearlier.Wedonotexpectthistosignificantlyaffectthevalidityofthebaselinedatasincethecluster-levelmonthlymeetingsareconsideredthecoreofthecapacitydevelopmentinterventionandhadnotyetstartedatthetimeofthebaseline.Furthermore,whileteachingknowledgeandskillsmayhavebeenaffectedtoalimitedextent,itishighlyunlikelythatpupillearningoutcomeswouldalreadyhavebeeninfluenced.Thedetailsofthebaselinesurveyimplementationarepresentedinthenextsection.MidlinedatacollectionisplannedinMay–June2017.BythattimeallIQSSactivitieswillhavehadayearofimplementation.EndlinedatacollectionisnotincludedinFigure38becausetheresources,scopeandtimingforthisdatacollectionhaveyettobedecided,althoughitstimingispreliminarilysetfor2019.Asdiscussedabove,thereisanagreementthatendlinedatacollectionwilltakeplace,whichwillbeimportantinordertoassessthecontributionoftheIQSStolearningoutcomes.

Figure38: TimelineforIQSSactivitiesanddatacollectioninGEP3pilotIQSs Schoolyear

2015–2016Schoolyear2016–2017


Sep–Dec Jan–Mar April–Jun

Sep–Dec Jan–Mar Apr–Jun Sep–Dec Jan–Mar

IQSS • Facilitatortraining

• Materials

• Headteachertraining • CBMCtraining

• Mini-grants

Datacollection

• Quant.survey

• Qual.casestudies

Baseline Midline

Thepupiltestsandsurveywereadministeredtoacross-sectionofpupilsatbaselineandwillbeagainatendline.Thismeansthatpupilswillbesimilarlysampledatbothsurveymoments.Testoutcomeswillrepresentthelevelanddistributionoflearningamongpupils,girlsandboys,beforetheIQSSandaftertheIQSS.Differentpupilswillbetestedinbothsurveyrounds(seeaboveformethodologicalreasonsfortheuseofacross-sectionalsamplesurvey),whichcontrastswiththepanelsurveyapproachproposedfortheearlylearninginterventionevaluation.Pupiltestswillnotbeadministeredatmidlinebecauseweestimatethattheminimumchangethatcanstatisticallybedetectedwithconfidencegiventhefeasiblesamplesizeistoolargecomparedtowhatcanbeexpectedbymidline(seeabove).Teacherassessmentwillbeconductedatmidline.Hence,changes

proposearigorousmixed-methodsdatacollection).Wehavenotplannedforadditionalevidencegenerationduringthe2015–2017period,duetobudgetconstraints,butthiscanbereassessedaftermidline,withthepotentialforittobeincludedinthe2017–2020evaluationperiod.6. StrengthenthecontributionstoryThefinalproductofthecontributionanalysiswillbeacontributionstorythatgoesbeyondthedescriptionofchangesobservedandalsomakesanassessmentofthecontributionofGEP3supporttosuchchange,whilealsotakingintoaccountotherinfluencingfactors.

Statestakeholderinterviews

Statestakeholderinterviews


©EDOREN 107

inthisimportantintermediaryoutcomewillallowustoassessthedifferencethattheIQSSismakinginthequalityofeducationintheIQSs.

AtbaselineandendlinepupilsarerandomlysampledamongallpupilsinP2gradeoritsequivalentlevelinIQSs,whentheyareinthefirsttermoftheschoolyear.Pupilsneedtohavehadatleastaroundoneyearofnon-religiouseducationinthesampledIQSstobeincludedinthetargetpopulation,inordertoensurethattheyhavebeenexposedtoteachingthatisthesubjectoftheIQSS.

Teacherandheadteachersurveyswilltakeapanelsurveyapproach.Thesameteachersandheadteacherswillbesurveyedatbaselineandmidline.Althoughsomeattritioncanbeexpectedweanticipatethatattritionlevelsatmidlinewillbeacceptable,andwillbemoderatedbythefactthattheteacherandheadteacherdevelopmentsupportwillincentiviseagainstattrition.Teacherattritionismorelikelytobeaproblematendlinein2019.Therefore,across-sectionalsampleofteacherswouldneedtobeconsideredatendline.

TheCBMCquestionnairewillbeadministeredamongaselectionofthreetofourcurrentCBMCmembers,including,atleast,theCBMCchairandmembersthathavebeentrainedthroughGEP3support.TheCBMCchairwillbethemainrespondent.TheinterviewwillbeconductedinagroupsettingbecauseinterviewpilotsdemonstratedthatwithmultipleCBMCmemberspresentvaluablecomplementaryinformationisobtained.WhileatbaselinethesurveyallowsforadegreeofflexibilityregardingwhichCBMCmembersarepresentbesidestheCBMCchair,118atmidlinetrainedmemberswillbeinvited.

4.2.3 Methodologyofquantitativedatacollection

4.2.3.1 Samplingstrategyandsampleofthebaselinesurvey

ThesamplingstrategyforquantitativebaselinedatacollectionoftheIQSSevaluationisalmostequivalenttothestrategyusedforthebaselineoftheearlylearninginterventionevaluation.ThischapterwillonlyoutlinetheelementsofthestrategythataredifferentoruniquetotheIQSSintervention.

Universe

ThestudyuniverseforthebaselineoftheIQSSinterventionevaluationconsistsofallIQSsselectedforGEP3interventioninthe12LGAsinBauchiandNigerwhereIQSSwillbeimplemented.SinceGEP3isalsoimplementingIQSSinSokoto,thestudyuniversedoesnotfullymatchGEP3’sinterventionuniverse.DuetobudgetconstraintsNigerandBauchiwerepurposefullyselectedtobecoveredbytheevaluation.

TheinclusionofschoolsintoGEP3requiresthatanIQSfulfilscertaincriteriasetbytheproject.AllIQSsincludedintheGEP3projectneedtohavestartedintegration,shouldteachbothboysandgirlsandberegisteredwithagovernmentagency.119

118 During the CBMC interview the secretary must be present for the interview to take place because he or she is animportantsourceofinformation.Inaddition,atleastonewomanrepresentativeandtheCBMCtreasurerareinvitedtotheinterview,wheneverpossible.Thevice-chaircanreplacethechairasmainrespondent.119Aminimumof40%ofpupilsaregirls.


©EDOREN 108

Thedefinitionoftheuniversealsoextendstoteachersandpupils.ThetargetpopulationofpupilsneedtobeenrolledinaP2equivalentgradeandstudynon-religioussubjects.Becauseofthenon-standardcompositionandorganisationoftheintegratedcurriculuminIQSs,aspecificdefinitionofthepupiluniverseiswarrantedinordertoachieveacoherentlydefinedpopulationuniverse(seeBox4inthemethodologicalchapteroftheearlylearninginterventionevaluation).

Similartothepupils,theuniverseofteachersconsideredaspartoftheIQSSinterventionconsistsofteachersteachingatthedesignatedschoolsandteachingpupilsenrolledinGrades1to3orequivalent(seeBox5inthemethodologicalchapteroftheearlylearninginterventionevaluation).

Samplingframe

ThesamplingframeofschoolsincludesallIQSsincludedintheGEP3projectinthetwoselectedstates.ThesamplingframewasconstructedbasedonGEP3schoollistsmaintainedbyUNICEF.ThetotalsamplingframeforthebaselineoftheIQSSinterventionconsistsof400IQSsin12LGAsintwostates.AlloftheIQSsinthesamplingframewereassumedtobeGEP3compliantandthuseligibleforinclusionintheintervention.

BecausetheevaluationdesignoftheIQSSisbasedonacross-sectionalsamplesurveyofpupils,withoutacontrolgroup,itiskeythatthesampledpupilshavebeenexposedtoapproximatelyoneyearofteachingoftheintegratedcurriculum.Itisoflittleuse,inevaluatingtheintervention,tocomparelearningoutcomesofpupilsthathavenotbeenexposedtotheteachingoftheintegratedcurriculumbeforeandaftertheIQSS.

DuringthefieldworkitbecameapparentthatthequalityofthesamplingframeofIQSswaspoor.TheQur’anicschoolsweremeanttohavestartedintegrationbyteachingnon-religioussubjects.Duemainlytonon-integration(schoolsnotteachingnon-religioussubjects)orrecentintegrationmanyoftheIQSsincludedinthesamplingframewerefoundtobeineligibleforbeingsurveyed.Thisrequiredthesamplingofreplacementsschools.

Samplesizeandsamplingparameters

Thesamplesizeisinfluencedbybudgetandsurveydesignconsiderations(i.e.cross-sectionvs.panelsurvey).Withintheseparametersthesamplesizeisintendedtomaximisethestatisticalpowerofthemeasurementofthechangeinthelearningoutcomeindicatorbetweenbaselineandthefollow-upsurvey(andtominimisetheMDE).

Ineachstate,30IQSsweresampledforstudypurposes,creatingatotalsamplesizeof60IQSsacrossthetwostates.Weexpecttosurveythesamesampleofschoolsatbaselineandmidline,andthelikelihoodofattritionwithintheschoolsampleisexpectedtobelow.Ineachschool,thetargetsamplesizeforpupilsissixgirlsandsixboys,resultinginatotaltargetedsamplesizeof720pupilsacrossthe60IQSs(360girlsand360boys).Thesamplesizesarecalculatedtoallowfordisaggregatedanalysisforgirlsandboysatendline.Giventhecross-sectionalnatureofthepupilsampling,thissamplesizewillremainthesameatendlineasatbaseline.

ThetargetedsamplesizeforteachersistwofacilitatorsperIQS,andtherefore120IQSfacilitatorsintotal.Weexpectthesamplesizeoftheteacherstobeloweratthemidlinesurveyduetopossibleattritionofthepanel.

Table21presentsthesamplesizesforallIQSsandinstrumentsimplementedwithineachIQS.


©EDOREN 109

Table21: SampleparametersfortheIQSSintervention

Bauchi Niger

NumberofIQSsperstate 30 30

InstrumentsperIQS Headteacherinterview 1 1

Teacherinterview† 2 2

Teachercompetencytest 2 2

Classroomobservation 2 2

CBMCinterview 1 1

Pupilinterview* 6girls,6boys 6girls,6boys

EnglishliteracyassessmentforP2 6girls,6boys 6girls,6boys

HausaliteracyassessmentforP2 6girls,6boys 6girls,6boys

NumeracyassessmentforP2 6girls,6boys 6girls,6boys†Thesameteacherswillbeinterviewed,testedandobserveddeliveringlessons*Thesamepupilswillbeinterviewedandtested

Thesamplesize,amongotherfactors,willdeterminetheminimalchangeinanoutcomeindicatorthatcanbedetectedwithstatisticalconfidencebetweenbaselineandafollow-upmeasurement.Aswasexplainedearlier,thisisreferredtoastheMDE.InthecaseoftheIQSSevaluationtheMDEisbasedonasingledifferencebetweenfollow-upmeasurementandthecorrespondingbaselinevalues.InBox10wesummarisetheMDEsforthelearningoutcomeindicatorandteachers’knowledgeandskillsindicatorgiventhepupilandteachersamplesizes.ThedetailsofthestatisticalpowerprinciplesusedarepresentedintheGEP3EvaluationFramework(EDOREN,2015).

Box10:EstimationofMDEs

MDEsforpupillearningoutcomesInlinewiththeGEP3logframe,themainlearningoutcomeindicatorisdefinedasthepercentageofgirlsandboysthatachievebasicliteracyandnumeracy.Inadditiontothesamplesize,theMDEisafunctionofthebaselinevalueoftheoutcomeindicator.Wedonotknowthebaselinevaluesofthelearningoutcomeindicatorsatthispoint,but,aswasdiscussedpreviously,learningoutcomesinnorthernNigeriaaregenerallyverylow.However,learningoutcomedatamostlycorrespondtothesituationinpublicprimaryschoolsandfewdataexistaboutlearningoutcomesinIQSs.WethereforeestimatetheMDEsusingplausibleandmaximumscenarios.WeassumeanICCof0.3,asimilarITC,80%power,anda95%confidencelevel.Usingaplausiblescenariothatthebaselinepercentageofpupilsthatachievebasicliteracyornumeracyisaslowas10%,theMDEisestimatedat6.8percentagepointswhengirls’andboys’dataareanalysedtogether,and7.3percentagepointswhenanalysedseparately.ThelargestMDEforagivensamplesizewouldbeproducedwhenthebaselinevalueis50%.Insuchacase,theMDEamountsto11.4percentagepointsforacombinedgirlsandboysanalysisand12.2percentagepointsforaseparateanalysis.Wedonotexpectalargemagnitudeofeffectsinlearningoutcomesbetweenbaselineandamidlinein2017,butconsideritreasonablebetweenbaselineandanendlinein2019,whichiswhymidlinepupiltestingisnotproposedaspartofmidlinedatacollection.Ifalowbaselinevalueisassumed,suchas10%,theMDEisinlinewithan8percentagepointincreasebyendline,astargetedintheGEP3logframe.Bycontrollingforsomecovariatesthatinfluencelearningoutcomes,


©EDOREN 110

Samplingdesign

ThemainsamplingmethodfortheIQSswasasingle-stagestratifiedsystematicrandomdraw,whileteachersandpupilsweredrawnthroughatwo-stagestratifiedsystematicsample.ExplicitstrataweredefinedbytheLGA.AnequalnumberofschoolswasdrawnfromeachLGA.Withineachexplicitstratumanimplicitstratificationwasbasedontypeoflocation(urban,rural).Thesystematicrandomsampleusedarandomstartandarealsamplingstep.Implicitstratificationwasachievedbysortingwitheachexplicitstratum.

IneachLGAfiveIQSsweredrawn.Inaddition,oneIQSperLGAwasdrawnasaninitialreplacementIQS,toprovideapoolofIQSsthatcouldbesurveyedincaseanyoftheIQSsinthemainsamplewerefoundtobeineligibleorinaccessibleduringthefieldworkperiod.Therefore,thesamplingsetuphadprovisiontoselectsixschoolsinanygivenLGAinNigerandBauchi.

Thereplacementschoolsweremarkedafterthesamplehadbeendrawn.AuniformlydistributedrandomvariablewasusedtosorttheIQSswithineachLGAandthelastIQSinthissequencewasdesignatedasareplacementIQS.

DuringthefieldworkmanyoftheIQSswerefoundtobeineligible.Thereasonforineligibilitywasmainlynon-orrecentintegrationofschools.Forthispurpose,additionalreplacementschoolshadtobesampledduringthefieldwork.Thedrawingofthereplacementschoolsmostlymirroredthemainsamplingapproach.InsomeLGAsthenumberofeligibleGEP3IQSsbecamequitelimitedduetohighproportionsofineligibility.Inthesecasesalloftheavailablereplacementswerereleasedtothefieldforthesurveyteamtoselectreplacementsinanorderthatcorrespondedtoapre-assignedrandomsequenceofrelease.

Specifically,arandomvariablewasusedtodesignatethereplacementsequenceamongthedesignatedreplacementIQSs.

forexampleageofthepupil,weexpecttoreducethevarianceoftheestimatesaspartoftheanalysis.Thismayallowustodetectsmallereffectsizesforagivenpowerandsamplesize.

MDEforknowledgeandskillsoutcomesTheteacherknowledgeandskillsoutcomeindicatoristhepercentageofteachersdemonstratingminimumknowledgeandteachingskills.The2013TDNAassessedteachers’professionalknowledgeandskillsforIQSteachersinsomeGEP3states.Whiletheresultsvariedfromstatetostate,a10%baselinevalueseemsplausible.Betweenthebaselineandmidlineapanelsurveyapproachwillbeused,whichresultsinahighITC.WeassumeanITCof0.8,anICCof0.3,80%power,anda95%confidencelevel.Wetakeintoaccount25%attritionofsampledteachersbetweenbaselineandmidline.TakingintoaccounttheaboveparametersweestimateanMDEof5.1percentagepointsbetweenbaselineandmidline,assumingabaselinevalueof10%.TheMDEwillbehigherbetweenbaselineandendlineifacross-sectionalsampleisdrawn(assuming0.3ITCandnoattrition)–thatis,8.5percentagepoints.Assuminga50%baselinevaluescenario,whichreturnsthehighestpossibleMDEforagivensamplesize,theMDEforapanelsurveyapproachbetweenbaselineandmidlinewouldbe8.4percentagepoints,and14.2percentagepointswhenselectingacross-sectionalsampleatendline.InlinewiththeGEP3logframe,weanticipatethatIQSSwillhaveaconsiderableeffectonteacherknowledgeandskillssinceteacherswillbethedirectbeneficiariesofintenseteachertrainingandmentoring.TheGEP3logframetargetsa30percentagepointincreaseinthepercentageofteachersdemonstratingminimumteachingcompetencies.


©EDOREN 111

AnnexCprovidesmapswithanoverviewofthesampledIQSsinthedifferentLGAsinBauchiandNiger.

WithinIQSs,samplingofpupilsandteacherslargelymirroredtheapproachdescribedintheearlylearninginterventionsectionofthisreport–pleaseseeforreferenceSection3.2.3.4.

ThesamplingofIQSfacilitatorsemployedaslightmodificationduetothemodalityoftheintervention.AtleastonefacilitatorfromeachGEP3IQSinNigerandBauchiwastrainedinSeptember2015.Thesetrainedfacilitators,iffoundatthecentre,wereselectedwiththeprobabilityof1asoneofthetwopotentialrespondentsinthebaselinesurvey.Figure39illustratesthesamplingflow.

TheCBMCquestionnairewasadministeredtothechairpersonorvice-chairpersonoftheCBMC,inthepresenceofuptothreeotherCBMCmembers.ThesecretaryoftheCBMChadtobepresentfortheinterviewtotakeplace.Inaddition,atleastonewomanrepresentativeandtheCBMCtreasurerwasalsotobepresentfortheinterview,wheneverpossible.

Weighting

Samplingweightswereconstructedtoreflectthecomplexsamplingdesign.Theweightswereestimatedforalllevelsofanalysisandreflectthemulti-levelnatureoftheschooldata,withpupilsandteachersnestedwithinschools.Theestimatedweightsarepopulationweights,whichsumuptothetotalpopulationsize,asdefinedbytheuniverseandthesamplingframe.Duetotheveryhighsamplingfractionrate(i.e.thelargeproportionoftheeligibleschoolssampled),finitepopulationcorrectionfactorswerealsoestimatedtoadjustthevarianceestimates.AdetailednoteonthecalculationoftheweightscanbefoundinAnnexD.


©EDOREN 112

Figure39: TeachersamplingTalkwiththeHeadTeacher

Public


interviewforteachersteachinginP1-P3

YES

Check:• Itisforthecurrentday• Itcoversteachersofthecorrectgrade• Itcoversteachersteachingnon-religious

subjects


available?NO

RequesttheHeadTeachertoidentifyallteachers(includingvolunteers)teachinggrades

P1-P3

Arerecordscomplete?

YES

EnterthenamesofalltheteachersinthelistsornamedbytheheadteacherintoasamplingForm(CAPI)

NO

Identifythemissinggroupsofteachersormissingnamesofindividualteachers(includeanyvolunteerteachers)

IstheschoolpublicorIQS? IQS

AretheclassestaughtingradesP1-P6?

YES


interviewforallteachersteachingP1-P3orequivalentpupils

NO


available?

YES

RequesttheHeadTeachertoidentifyallteachers(includingvolunteers)teachingP1-P3or

equivalentpupils

NO

Foreachteacherspecifywhetherteacherteachesreligiousornon-

religioussubjects

CAPIselectsteachersrandomly:• 3teachersinpublicschools• 2teachersinIQS

Inviteselectedteacherfortheinterview

Istheteachereligibleandhasgiven

consent?

Administertheinterview,classroomobservationandtestto

theteacher

EnterthenamesofallselectedteachersonSchoolCompletion

Form

YESNO

Samplethereplacementteachers

Whichinterventiondoesthe

schoolbelongto?

EARLYLEARING

IQSSUPPORT

RequesttheHeadTeachertoidentifytheteacherorfacilitatorthathasbeentrainedbyUNICEF

GEPtrainer.

Howmanyteachersor

facilitatorshavebeentrained?

ONE TWO


©EDOREN 113

4.2.3.2 Instrumentdesignandpreparation

NineinstrumentsweredevelopedtocollectquantitativedataateachsampledIQS.Theinstrumentdevelopmentprocesswasthesameastheonedescribedinthemethodologicalsectionoftheearlylearninginterventionevaluation(seeSection3.2.5).TheinstrumentsforthebaselineoftheIQSSevaluationweresimilartotheonesusedforIQSsaspartofthebaselineoftheearlylearninginterventionevaluation,inordertoallowforcomparabilityandtoachieveefficiencygains.However,twoadditionalinstrumentsweredevelopedtocoveradditionalevaluationquestionsandelementsintheToC:apupilnumeracyassessmentandaCBMCquestionnaire.TheinstrumentsarepresentedinTable22.

Table22: Overviewoftheinstruments

Instrument Summary

Pupilquestionnaire

Thepupilquestionnaireprecedesthepupilassessments.Itaimstocapturebackgroundbasicbackgroundinformation,includinggender,numberofyearsatschool,ifthepupilattendsotherschoolsetc.Italsoincludeschecksforpupildisabilityandcollectsdataonpupilhouseholdassets.Pupilsidentifiedashavingadisabilityarenotgivenassessmentitemsthatrequirethatparticularabilityinordertorespondmeaningfully(forexample,apupilwithanidentifiedinabilitytoseeisnotaskedtocompletethereadingitems).

PupilEnglishliteracyassessmentforP2

TheEnglishliteracyassessmentcontains13items,witheachitembeingmadeupofseveralsub-items.Whiletheassessmentistargetedatadifferentproficiencyrange,theEnglishliteracyassessmentislinkedtotheESSPINCSassessmentsandisprojectedontotheEDORENEnglishliteracyscale.Therefore,robustcomparisonsbetweenESSPINlearningoutcomeresultsandGEP3evaluationlearningoutcomeresultscanbeundertaken.Theassessmenttestsarangeofliteracyknowledgeandskillsacrossthepre-literacy,emergingandbasicliteracyranges.Knowledgeandskillsincludeletterrecognition,phonologicalknowledge,printconcepts,oralliteracy,verbalcomprehension,initialsoundsandletters,readinghighfrequencywords,verbalandwrittengrammar,writinghighfrequencywords,readingfluency,andcopyingandspellinghighfrequencywords.

PupilHausaliteracyassessmentforP2

TheHausaliteracyassessmentisdesignedtotestthesameliteracyknowledgeandskillsastheEnglishliteracyassessment.Itemsarenotmerelytranslated,butratherparallelitemsaredevelopedtotestsimilarconceptswhenappliedtotheHausalanguage.

Pupilnumeracyassessment

Thenumeracyassessmentcontains16items,witheachitembeingmadeupofseveralsub-items.Whiletheassessmentistargetedatadifferentproficiencyrange,thenumeracyassessmentislinkedtotheESSPINCSassessmentsandisprojectedontotheEDORENnumeracyscale.Therefore,robustcomparisonsbetweenESSPINlearningoutcomeresultsandGEP3evaluationlearningoutcomeresultscanbeundertaken.Theassessmenttestsarangeofnumeracyknowledgeandskillsacrossthepre-numeracy,emergingandbasicnumeracyranges,includingnumber


©EDOREN 114

4.2.3.3 Surveyfieldwork

ThesurveyfieldworkimplementationfortheIQSSbaselinesurveyfollowedthesameprotocolsandproceduresthatwerefollowedintheearlylearninginterventionbaselinesurvey.OnlytopicsspecifictotheIQSSinterventionarediscussedinthissection.

Unitssurveyed

Usingtheeligibilitycriteriamentionedabove,theteamsurveyed576pupilsand96teachersfrom60IQSsinBauchiandNiger(SeeTable23).Thetargetedsamplesizeof60IQSswasachieved,whilethepercentageofsampledpupilsandteachersfromtheseschoolsthatweresurveyedequals80%and80%respectively.Inaround20%oftheIQSsonlyhalforlessofthetargetnumberof12pupilscouldbesurveyedbecauseonlyalimitednumberofpupilswereeligibleforthesurvey.Thebelow-targetachievementofteacherssurveyedwasduetothefactthattheschoolsdidnothavethetargetednumberofteachers.

concepts,additionandsubtraction,measurement,moneysums,multiplication,division,volumeandcapacity.

Teacherquestionnaire

Theteacherquestionnairecollectsdataonteachers’backgroundcharacteristics,absenteeism,trainingandremuneration.Italsoincludesquestionsonmeetingsandsupervisionfromtheheadteacherandamoduleonteacherperceptionsandattitudes,whichwillbeusedtodevelopameasureofteachermotivation.

Teacherknowledgeandskillsassessment

Theteacherknowledgeandskillsassessmentisdividedintothreesections,collectivelycomprising30items,includingmultiplechoice,shortresponseandlongresponseitems.ThisincludesmarkingpupilresponsestoHausaliteracyquestions,indicatinggradelevelsforitemsbasedonthecurriculum,developingananswersheetforareadingtestaimedatGrade2pupils,answeringareadingcomprehensionquestion,andmakingjudgementsaboutpupils’writing,includinggrammar,spellingandadditionalsupportneeded.

Teacherclassroomobservation

Theclassroomobservationrecordsthefrequencywithwhichtheenumeratorobservesspecificteachertalk,teacherlanguageuse,teacheractionandpupilaction.Theenumeratoralsorecordsteacheractionatthestartofthelesson,actionattheendofthelesson,resourcesusedandsubjectobserved.

Headteacherquestionnaire

Theheadteacherquestionnaireincludesseveralsections,includingbackgroundinformationontheschool,basicinformationabouttheschool,schoolmanagementandmonitoring,attendanceandenrolment,attitudestowardsintegrationforIQSsandschoolinfrastructure.

CBMCquestionnaire

TheCBMCquestionnairewasadministeredtothechairmanoftheCBMCand,wherepossiblewithothermembersoftheCBMCpresenttoverifyresponses.Thequestionnaireincludesitemscoveringgeneralinformation,CBMCactivities,schoolmonitoringandimprovement,andCBMCfinancingandfundraising.


©EDOREN 115

35%oftheoriginallysampledIQSshadtobereplaced,mainlyduetotheIQSnotbeingconsideredeligibleforthesurveygiventhestudyuniverse.120Replacementsweremostlyduetothecentrenotteachingnon-religioussubjectsyet/anymore,oritbeingrecentlyintegrated.121Asexplainedabove,areplacementprocessinlinewiththesampledesign—avoidingtheintroductionofbiasinthisprocess—wasplannedfor,todealwiththechallengeswiththesampleframe.

Table23: Summaryofsurveyachievement

Bauchi Niger

Schools

Targetsampleschools 30 30Totalschoolsurveyed 30 30

Originallysampledschoolssurveyed 16 23Replacementschoolssurveyed 14 7

Totalschoolscontacted 57 48Contactedschoolthatwerenotsurveyed–reasons Recentlyintegrated 14 1

Non-integrated 8 15 Non-existing 4 1 Nogirls 0 1

Other 1 0Percentageofsampleschoolssurveyed 100% 100%Pupils

Targetsamplepupils 360 360

Percentageoftargetpupilssurveyed 82% 78%

Teachers

Targetsampleteachers 120 120

Percentageoftargetteacherssurveyed 77% 83%

Source:Fieldworkreports

StatusoftrainedfacilitatorsinIQSSintervention

InBauchiandNigerstatesthenamesof70-UNICEFtrainedfacilitators(60fromthemainsampleofschools,withadditional10replacements)from12LGAswerereceived,tobeincludedinthesurvey.Withthisinformation,thesurveyfieldteamwasabletomatchandcompleteinterviewswithi)17facilitatorswhowerepartofthelist,andii)twofacilitatorswhosenamesdidnotmatchwiththelistbutwhoparticipatedinthetrainingunderadifferentname.Intheschoolsthatwereconfirmedtobe

120ForanIQStobeconsideredeligibleorqualifiedforthesurvey,itwasrequiredthat:1)theschoolmusthaveaP2equivalentclass;2)itmusthaveteachersofnon-religioussubjectteachingP1–P3;3)itmusthavebeenintegratedforapproximatelyayear;and4)itmusthavefemalepupils.121ThelengthofintegrationintheIQSswasparticularlyanissueinBatsariLGAinKatsina,whereapproximately80%ofschoolsintegratedin2015.Inthiscasetheeligibilitycriteriaofone-yearintegrationwasappliedmoreflexiblybysurveyingtheIQSthathadactuallystartedintegrationinJanuary2015–hencehaving10monthsofintegration.


©EDOREN 116

eligible,insixcasesthefacilitatorswerenotknown,inoneschoolthefacilitatornolongertaughtthere,andintwocasesthelistedpersonswereconfirmedtohaveattendedthetrainingbutwerereportednottobeteaching(seeFigure40).In13eligibleschoolsthefieldteamswerenotabletofindthelistedfacilitators.Therefore,atotalof48sampledfacilitatorswerenotfound.

Figure40: Flowchartofdatacollectionrelatingtofacilitators

2StatesBauchiNiger

12LGAs5schoolsineach

70facilitatorNamesprovided

Fielded?

NO

3schoolsNotattempted

YES

67schoolsfielded

Schooleligible?

YES

41schoolseligible

YES

17interviewed2interviewedunderassumedname_____________________________19* (18%)

Facilitatorsfound?

Knownreasons1Nolongerteaches6personsnotknownatschool2knownbutnotteaching_____________________________9* (9%)

Unknownreasons13Notfoundatschool_____________________________13* (12%)

NO

NO

Schoolsvisited105

(100%)

Schoolseligible19Facilitatornamenotprovided_____________________________19* (18%)

Schoolsineligible19Facilitatornamenotprovided_____________________________19** (18%)

38Replacement

schools

Schoolsineligible26schoolsineligible_____________________________26** (25%)

NO

4.2.4 Methodologyofqualitativedatacollection

Thissectiondiscussesthemethodologyofthequalitativedatacollection,includingtheoverarchingmethodologicalapproachused,howrigourwasachieved,thesamplingstrategyandgeneralisability,structuredandunstructuredmethodologies,andinstrumentdesign.Finally,thefieldworkandanalyticalapproachesaredescribed.

4.2.4.1 Methodologicalapproach

Thequalitativeresearch,similartothequantitativeresearch,aimstocollectdataatthreepointsintime:2015,2017and2019(with2019tobefurtherdetermined).Themethodologyforthequalitativeresearchusesthefollowingthreemaintoolstocollectdata,structurefindingsanddrawconclusionsagainsttheoverallobjectivesofthecontributionanalysis:

1. TheprincipalorganisingstructurefortheevaluationmethodologyistheToCoftheIQSSinterventions,whichisusedtoexaminethecausallinkbetweentheinterventionoutputsandoutcomesinthecontextofIQSs.TheToCwillinformtheevaluationasawhole,butisparticularlyimportantforthequalitativeresearchbecauseitpermitsstrongergeneralisation.

2. TheevaluationmatrixturnstheToCoftheIQSSintoasetofhypotheses,assumptions,andareasofinterest,onwhichtheinstrumentationisbased.

3. DatafromasmallnumberofIQScasesisusedtoassessifandhowtheexpectedchainofeventspositedbytheToCmayoccur,validatetheinfluencingfactorsandassessalternative


©EDOREN 117

explanations.AnIQScaseconsistsoftheIQSwithinitsschoolcommunity–hence,includingbothschool-levelactorssuchasproprietor,headteacher,teachers,CBMCandpupils,aswellascommunity-levelactors,suchasparents,communityleadersandgovernmentofficials.

Theaimoftheinitialqualitativebaselineresearchistoexaminethehypothesesandassumptionswithkeystakeholders,andcontextualfactorsthatarelikelytoaffectimplementationandoutcomes,andtoassesstherelevanceoftheplannedactivitiesforbettereducationoutcomes.Asaresult,thequalitativebaseline,togetherwiththequantitativeresearch,examinesthebaselinesituationoftheToC(confirmatorymethodology)andrefines,developsnewhypothesesfor,andenrichesandcontextualises,theToC(exploratorymethodology).ThisprocessallowstheEDORENteamtoprovideanevidence-basedcritiqueoftheToC,identifyingpotentialweaknessesandproposingwaysofstrengtheningtheprogramme’sToC.

4.2.4.2 Rigour

Amajormethodologicalchallengeinqualitativeresearchisthedefinitionandachievementof‘rigour’,particularly,asinthiscase,whentheresearchmethodologyneedstobeopentotheidentificationofnewhypotheses,causesandunexpectedimpact.Therefore,themethodologyshouldcontainanemergentdimensionnotfullyprescribedattheoutset.Moreover,qualitativeresearchisoftenaccusedofbeingopentoresearchbiasoranecdotalimpressions,impossibletoreproduceanddifficulttogeneralise(MaysandPope,1995).However,rigourinqualitativeresearchcanbeachievedinthefollowingways.First,sampling‘isacoredesignissuebecausethepurposefulsamplewilldeterminewhatyoulearnabout’(Patton,2015):throughacarefullychosensamplingstrategy,themostrelevantsamplecanbeapproximated.Inaddition,rigourcanbeensuredthroughtheadequacyofqualitativeinstrumentsthatbestreflecttheToC,fieldworkthatinvolvescarefullytrainedandwell-managedresearchers,andqualityassurancethroughouteachresearchstage.Lastly,analysisbiascanbeminimisedbyrelyingonaconfirmatoryanalysisapproachthatutilisesqualitativedatabasedonasetofpre-existinghypotheses.Together,thesemeasuresensurerigourinqualitativeresearchthrough‘systematicandself-consciousresearchdesign,datacollection,interpretationandcommunication’(MaysandPope,1995).Howthequalitativeresearchdesignaddressestheseissuesisdiscussedintherelevantsectionsbelow.

4.2.4.3 Samplingstrategyandgeneralisability

Aswithmostqualitativeresearch,thechosenapproachtosamplingforthequalitativecomponentoftheIQSSevaluationwasdesignedtogenerateresponsesfromsmallnumbersofindividualsandgroupsthatarerepresentative(thoughnotstatistically)ofgroupsrelevanttoGEP3’sIQSS,andwhichallowsomeidentificationofheterogeneouscontributions.

Specifically,theresearchemployspurposive‘typicalcasesampling’and‘extremecasesampling’.Thisisnotdesignedtoproduceresultsthataregeneralisableinthesamesenseasquantitativedata.Rather,thegeneralisabilityofthequalitativeresearchresultsderivesfromtheextenttowhichtheyareembeddedinaToCthathassomevalidityinawidercontext.ThisformofsamplingallowsexplorationofwhattheIQSSisdoinginatypicalcase,butalsoperformancein‘higherperforming’and‘lowerperforming’cases,whichhelpsexplainthereasonsbehindhigherandlowerlevelsofcontributionacrosscontexts.

Forthepurposesofthisevaluation‘lowerperforming’caseswereinterpretedasschoolsandcommunitieslikelytobemostresistanttochangeinincreasingtheeducationoutcomesforgirls


©EDOREN 118

(lowerfemaleenrolment,negativeattitudestowardgirls’educationandlowerlevelsofacceptanceoftheintegrationprocess).‘Higherperforming’caseswereinterpretedasschoolsandcommunitieslikelytobelessresistanttochangeinincreasingtheeducationoutcomesforgirls(higherfemaleenrolment,positiveattitudestowardgirls’educationandhighlevelsofacceptanceoftheintegrationprocess).

Thequalitativestudysampleaimstoreflectthe‘typicalcase’andthe‘extremecase’ofthequantitativedatacollectionsample.ThisinitiallyreliedonaUNICEFlistofIQSsthatwerereportedtomeetcertaineligibilitycriteria,suchashavingasufficientpercentageofgirlsastheirpupilsandteachingnon-religioussubjects.However,asthequantitativedatacollectionprecededthequalitativestudy,itbecameclearthatmanyoftheIQSsincludedinthesamplingframewerefoundtobeineligibleforbeingsurveyed(seeSection4.2.3).Forthatreason,toensuretheeligibilityofallsampleschools,thestudysamplewasrestrictedtothoseschoolsthathadbeenpreviouslyinspectedandwereconfirmedasmeetingUNICEFeligibilitycriteria.

SixschoolswerepurposefullysampledamongpreviouslyinspectedGEP3IQSsinBauchiandNiger.Foreachstate,alowerperforming,typicalperformingandhigherperformingschoolwereidentified.Thesamplesizewasdeterminedbyresourceconstraintsandaneedtoahaveaminimalspreadoftypicalandextremecasesacrossthetwostates.

AchallengetoidentifyingthetypicalandextremeIQScasesisthatIQSsarenotcurrentlyincorporatedintothestates’EMISdatabases.Assuch,itwasimpossibletosampledirectlybasedonIQSperformance.Instead,thesampledesignreliedonatwo-stagedapproach.

First,threeLGAsperstateweresampledineachcasecategoryusingaveragepublicprimaryschoolperformanceongirls’educationineachGEP3LGAasaproximateindicatorforIQSperformanceongirls’education.Basedontheevaluationmatrix,andinlinewiththebestavailableEMISdata,thefollowingindicatorswereselected:

• Girls’access:TheGPI,ascalculatedbytheaverageshareofgirlsversusboysenrolledinschool.122

• Girls’retention:Anapproximatedtransitionrate,ascalculatedbyaschool’sshareofgirls’enrolmentinGrade6(endofprimaryschool)dividedbyitsshareofgirls’enrolmentinGrade1.123

• Girls’educationquality:Acompositeindexofthepupil-to-qualified-teacherrate,shareofqualifiedfemaleteachersandpupil–classroomratio.124

Fromthis,anoverallcompositeindexwasgeneratedthatrankedtheLGAsfromhighesttolowestperforming,basedonanequalweightingacrosseachofthethreeindicators.LGAswerethensampledtobeasclosetothetop,middleandbottomofthelistaspossible.

Secondly,IQSswereselectedwithineachLGAbasedonthefollowingcriteria:

122Thiswasusedintheabsenceofbettervariables,suchasthegender-disaggregatednetenrolmentrate.123Thismeasureisanimperfectapproximationandisaffectedbyshiftingenrolmentsizeovertime;atruetransitionratewouldbecalculatedbasedonaspecificcohort’sprogressionfromGrade1toGrade6.124Inabsenceofdirectmeasuresofpupilperformance(e.g.testscores),thesethreevariablestogetherprovideaweak,butcommonlyusedandindicative,measureofschoolqualityandwealth.


©EDOREN 119

• Frominspectionvisits,IQSselectionusedtwoextracharacteristics:variationinyearssinceintegration,andthenumberofnon-religioussubjectfacilitators.Aslonger-runningschoolsandmorefacilitatorswereconsideredtobe‘better’IQSs,theywereselectedforthehigherperformingLGAs(withthereverseusedforselectinglowerperformingIQSs).

• IQSlocationwassampledtoincorporateaspreadofurban,peri-urbanandruralschools,withlessurbanareasgenerallyperceivedtoleadtoalowerqualityIQS.Thesamplingofmoreremoteschoolswasbalancedwiththeaimofcompletingasmanyinstrumentsaspossiblewithinthepossibleinterviewingtimeframe,soastoreturnbacktoeachstatecapitalbeforedark.

• Languagealsoplayedapartinsampling:areaswithadominantminoritylanguage(e.g.NuperatherthanHausa)requiredtwo-waytranslationfromoneteammember–overusecouldleadtofatigueandreducethequalityofinterviewresponses.Assuch,onlyoneoutofthesixIQSswassampledasanon-Hausaspeakingschool.

• Securityfactorsweretakenintoconsideration:oneLGAinBauchiwasswappedatlatenoticeinresponsetorumoursregardingBokoHaramactivity.

Basedontheaboveprocess,twoIQSswereselected(firstandsecondchoice)foreachofthesixLGAsinNigerandBauchi(12intotal).ThecategorisationofthesampledIQSswasdiscussedwiththestateGEP3teamandresponsiblegovernmentofficials,whovalidatedqualitativelythecategorisationoftheIQSsinhigherperforming,typicalandlowerperforming.Forbothstates,theteamsmanagedtovisittwooftheirfirstchoiceIQSs,while,duetounresponsivenessorunavailability,theyhadtovisitonesecondchoiceIQS.Noadditionaladjustmentsweremadetothesample.

AteachIQSparents,pupils(girlsandboys)andCBMCmemberswereinvitedtoparticipateinFGDs.TheselectionwascarriedoutwiththesupportoftheIQSproprietororMallam,whichwasimportantinordertoestablishagoodrapportforthequalitativeresearch,butmayhavebroughtaboutselectionbias.InadditiontotheIQSs,thequalitativeresearchalsoundertookKIIswithLGAIQSofficers.ThesewereselectedtomatchtheLGAsofthesampledschools.

4.2.4.4 Qualitativeinstruments

Thequalitativeresearchprimarilymakesuseoffourresearchtechniquesorinstruments:aqualitativeclassroomobservation(QCO),anunstructuredteacherpracticediscussions(TPDs),KIIsandFGDs.


©EDOREN 120

Table24: Instrumentsbyparticipantgroup

InstrumentProp

rietor/Owne

r/M

allams

Headteache

rs

Facilitator/Teacher

Pupils

CBMCmem

bers

Parents

Commun

itylead

er

Localgov

ernm

entIQS

officer

QCO �

TPDs �

KIIs � � � �

FGD � � �

ThetoolsfortheKIIsandFGDsweredevelopedinordertocaptureinformationonthecoreareastoprobe,asoutlinedinthequalitativeevaluationmatrix,whicharederivedfromtheunderlyingassumptionsintheToC.

Thestrengthsofcombiningquantitativeandqualitativemethodsinevaluationcanalsoapplyattheclassroomlevel,intheformofclassroomobservations.Theclassroomobservationsundertakenaspartofthequantitativesurveyprovidedataonthedegreetowhichteacherpracticesarechanging,whiletheuseofQCOscombinedwiththeTPDsaremeanttoanswerthe‘how’and‘why’questions(FasseandKolodner,2000;1993;Merriam,1988;Yin,1984).TheQCOissemi-structuredandwasconductedbyaneducationalist.Theinstrumentlistssomeofthecoreareasinwhichtheprogrammeaimstochangeteacherbehaviourandpedagogyandtheresearchertookdetailednotesundereachsectionduringthecourseoftheclassroomobservation.Aftertheobservationhadtakenplace,theresearcherundertooktheTPDwiththeteacher.Thediscussionfollowedthestructureofthelessonobservationbutotherwiseremainedunstructured.Thediscussionfocusedonprobingthepedagogicalapproachoftheteacherandengagingtheteacherinadiscussionabouthowandwhytheteacheremploysspecificmethodswithintheclassroom.Specificattentionwasgiventounderstandinghowandwhytheteacherimplementsgender-sensitiveclassroompractices.AtmidlinethiswillincludereflectingontheextenttowhichthetrainingprovidedthroughGEP3haschangedtheteacher’spraxis125andwillprovidedataonalternativeexplanations.

UNICEFstatesthat‘“Girlfriendly”schoolsareschoolsthat[girls]wanttoattend…[they]providechildren–girlsinparticular–withasafe,nurturingandgender-sensitivelearningenvironment’.126Parents,pupilsandrepresentativesfromtheCBMCprovidedivergentpointsofviewregardingtheextenttowhichgirlswanttoattendschool,andwhy(orwhynot).Thecasestudiesseektoconstructastrongstoryregardingthelearningenvironmentandtoidentifyfactorsthatareperceivedtocontributetochanges.

DiscussionguideswereusedfortheFGDs,tailoredtowardtheareasthatweretobeprobed,andmakinguseofage-appropriatelanguage.Semi-structuredinterviewguideswereused.Thesewere125Praxisreferstothewaysinwhichindividualsengagewithandreflectonasituation,asboththinkersandactors.Praxisincorporatesvariouscomponents– includingthetheoretical (guidingrules), theproductive (applicationofguidingrules),andthepractical(howtoproduceundertheguidingrules). 126www.unicef.org/mdg/mali_59595.html.


©EDOREN 121

organisedaroundthecoreareastoprobe,thusensuringadegreeofstandardisationwhileatthesametimeallowingthenationalandEDORENqualitativeresearchersenoughflexibilitytopickuponinterestingthemesandemergingtopicsandconcerns.

KIsarepeoplewithin-depthknowledgeandunderstandingofparticularsubjects.Inthecaseofthisresearch,theKIsaretheproprietor/owner/MallamsoftheIQS,headteachers,communityleadersandthegovernmentIQSofficerattheLGAlevel.127WhilethechoiceofKIswaslargelystandardised,thedesignwasflexibleenoughtoalsoincorporateotherKIsshouldtheneedhaveariseninthefield.ThiswasparticularlyimportantfortheIQSSasthestructureswithinIQSsarenotsetandstandardisedlikeinpublicprimaryschools.Forexample,theinterviewwiththeLGAIQSofficerwasaddedafterthestate-levelstakeholderinterviewshadshowntheimportancethatstakeholdersattachedtomonitoringandsupervisionoftheeffectivenessoftheIQSS.

4.2.4.5 Fieldwork

Implementationandmanagementofthefieldwork

ThefieldworkwasimplementedbyEDORENbetween03and14November2015.Itwasprecededbyafour-daytrainingperiodforresearchers(seebelow).

ThequalitativeresearcherswererecruitedbyEDOREN,andinitially11memberswereinvitedfortrainingfromapoolofhighlyexperiencedcandidates.Selectionwasmainlybasedoncandidates’CVsandtheirexperiencewithotherqualitativeeducationresearchinnorthernNigeria.128Theselectionprocessalsotookintoconsiderationthegenderbalanceoftheteams,andespeciallytheirlocallanguageproficiency.AllmemberswereproficientinbothspokenandreadHausa,whichwasthemainlanguageusedforinterviews.OtherlocallanguagerequirementswerealsotakenintoconsiderationwhenselectingteammembersasinsomeareasHausaisnotthemainlanguagespoken.IntheselectionofinterviewersspecialattentionwasgiventoArabic,Fulfude,NupeandGbagyilanguageproficiencysincetheselanguagesarespokeninthestudyarea.

Twoteamswereestablishedforthefieldwork,eachteamcoveringonestate.Onewasledbyanexternalconsultant(anexperteducationist),whiletheotherwasjoint-ledbyaneducationistandaqualitativeresearchspecialist.Thesestateteamleadssupervisedallinterviews,carriedouttheQCOs,oversawthedatatranscriptionandensuredqualityassuranceateachstageoftheresearch.

Eachteamleadselectedfourresearchersfromtheoverall11memberstrainedtoholdthepositionsofleadinterviewer,focusgroupfacilitator,note-takerandtranslator.Thisselectionwasdonehalf-waythroughtraining,andwasbasedoneachindividual’sengagementandexpertise,suitabilityforeachofthefourpositions,thegenderbalanceoftheteamsandspecificlanguagerequirements.

Training

Toensurethatallsurveyteamscoulddotheirworkeffectivelyandinaccordancewithhighqualitystandards,allqualitativeresearcherstrainedintensivelyfrom31Octoberto02November2015inAbuja.Theinstrumentswerepilotedattheendofenumeratortraining.

127TheseIQSofficerswereLGArepresentativesfortheSAME.128Qualitativeresearcherswereinvolved,amongothers,indatacollectionandanalysisfortheEDORENThematicResearchonTeacherManagement,andonaGEP3assessmentoftheFTTSS.


©EDOREN 122

Thisexercisefirstlyincludedanumberofinteractiveexercisestodeepenresearchers’understandingoftheGEP3IQSintervention,theToCandtheevaluationframework.Next,thetrainingmovedtoprudentprocessesforqualitativeresearch,includingdatacaptureandtranslationprotocols,transcriptionproceduresandresearchethics.

Thefinaltwodayswerespentextensivelyengagingwitheachoftheeightinstruments.Thiswasdoneinorderforresearcherstopracticeessentialinterviewingtechniques,suchasprobing,whileconcurrentlyrefiningtheflowandphrasingofeachoftheinstruments.Thetrainingusedseveraldifferenttrainingapproachestoachievethemaximumtrainingeffect,suchassmallgroupwork,mockinterviewsandgrouppresentationsoffindingsandreflections.

Fieldworkorganisation

Foreachofthetwostates,theresearchteamcomprisedof:

• astateteamleader:ashasbeensaid,thestateteamleadssupervisedallinterviews,carriedouttheQCOs,oversawthedatatranscriptionandensuredqualityassuranceateachstageoftheresearch.Oneoftheteamswasjoint-ledbyaneducationistandaqualitativeresearchspecialist;

• oneLeadInterviewer,responsibleforconductingtheKIIs;• onefocusgroupfacilitator,responsibleforconductingFGDswithCBMCs,pupilsandparents;• onetranslator,responsibleforaidingtheteamlead/sinconductingtheQCOsandTPDs;and• onenote-taker,responsiblefortakingnotesduringFGDsand(timepermitting)KIIs,aswellas

lookingafteraudio-andvideo-recordingequipment.

Atthebeginningofthefieldworkeachmemberwasgivenexplicitguidanceandtrainingontheirparticulartask.Forthetranslator,forinstance,thismeantrepeatedexercisesontwo-waysimultaneoustranslation.Fortheleadinterviewerandfocusgroupfacilitator,thismeantthestateteamleaderofferedguidanceonelementssuchasprobingorcreatingasafespacetoinciteresponses.

Whilemembersprimarilyhadresponsibilityfortheirdesignatedrole,inpracticetheyalsotookonadditionalresponsibilities.Forinstance,duetothelargeamountofinterviewstakingplace,allmembersweremaderesponsibleforapartofthetranscription,whichwasthenqualityreviewedbythedesignatednote-taker.Similarly,whenmultipleinterviewstookplaceatthesametime,othermemberscouldstepintofunctionastranslator.However,allKIIswereperformedbytheleadinterviewerorstateteamlead,whileallFGDswereperformedbytheFGDfacilitatororstateteamlead.

Fieldworkchallenges

Someofthemostpertinentchallengesfacedduringfieldworkarelistedbelow:

• TeachinghoursatIQSs:IQSsusuallydonotfollowtheirtimetablesstrictly.Inaddition,someclassesaretaughtearlyinthemorningandsomelateatnight.Furthermore,mostoftheIQSsoperateforjusttwohours.Thisrequiredteamstomakedetailedenquiresregardingtheday’splan,andtosetoutveryearlyorlate,oftenadministratingtheheadteacher,teacherandCBMCinterviewtoolsbeforethearrivalofthepupils.


©EDOREN 123

• Holiday:SomeIQSswerenotinsession,althoughtherewasnomentionofthisintheirtimetables.ThismeantthatteachersandpupilshadtocometoschoolinanarrangedmannerinordertocarryouttheKIIs,FGDsandclassroomobservations.

• Distanceanddifficultterrain:ManyIQSsareremotelylocated,withoutgoodroadaccess,whichrequiredlongtraveltimestoreachthem.Thismadeitmoredifficultfortheteamstocompletealltheinstrumentswithinschoolsandtoreturntotheirbasebeforedarknessfell.Combinedwiththedifficultteachinghours,thismeantattimesthattheclassroomobservationshadtoberescheduledtoanunusualtime,orthattheresearchershadtocomebackforseveraldays,tocompleteallinstruments.

• Pre-appointment:SchedulingappointmentswiththeIQSswithoutaphysicalpresencewasverydifficult,asmanyIQSslieoutsideofthereachofmobilecommunicationnetworks.

4.2.4.6 Qualityassurance

Inordertobestensuretherigourofthequalitativeresearchthestateteamleadsprovidedqualityassuranceateachstageoftheresearch.Thiswasfirstensuredatthetrainingstagebycarefullypreparingeachofthepotentialteammembersonqualitativeresearchmethodsandbyselectingonlythemostappropriateindividualsforeachoftheteamroles.Thetrainingstressedunbiaseddeliveryofinstruments,andemphasisedbuildingteammembers’capacityandunderstandingofthetoolsandstudyasawhole.

Rigourandmitigationofbiasinthequalitativefieldworkwasachievedthroughinvolvementofdifferentindividualsinthefieldteams,allowingteammemberstosupporteachother,discussandquestionfindings.Teamleadswerepresentatalltimesduringfieldwork,tooffersupportandaddresstechnicalqueriesinthefield.TeamleadsalsosatinonthemajorityofinterviewsandFGDs,timepermitting,toprovideadditionalguidanceandprobingwherenecessary.Additionalqualityassurancewasachievedthroughdailyteambriefsanddebriefs.Inthedebriefstheteamwoulddiscusskeyfindingsfromtheday,linkfindingstotheevaluationmatrix(whilstnotingfindingsthatweredivergentfromthematrix)andcomparefieldobservationalnotestohelpsituateanalysis.Thedebriefingsalsoallowedtheteamtoaddresslogisticalandtechnicalqueriesthatmayhavearisenduringtheday.AstheteamreturnedtoeachIQSformorethanonedayitwaspossibletofurtherexploreontheseconddayqueriesthataroseoutofthefindingsfromdayone.Briefingswereheldeachmorningbeforetheteamsetout,afterhavingreviewedfindingsandtranscriptsfromthedaybefore,toidentifyquestionsneedingfurtherexploration,andgivetechnicalpointersforthedayahead.

Qualityassuranceoftranscriptionwasensuredbyallowinganinitialtranscripttobewrittenupbyoneindividualandthenverifiedinpartsbyasecondindividual.129Thisback-translationwasusedtoensurestandardisationoftranslationfromHausatoEnglish.Anydiscrepancieswerediscussedamongstteammembers.Transcriptionandback-translationtookplaceduringthefieldwork,andsoalsoprovidedimmediatelessonsintermsoffacilitatinginterviewingortheneedforadditionalprobing.Suchlessonswerethendiscussedduringthedailybriefinganddebriefingmeetings.Theteamleadalsoreviewedallcompletedtranscripts,togiveguidance(particularlypriortogoingtothenextschool),thusallowingforreflexivityinthefield,andstrengtheningrigour.Teamsalsokeptrecordsoftheiractivities,andmadeadditionalnoteswherepossible,sothattheycouldbelinkedtotranscriptsandanalysis.Uponreturningfromthefieldtheteamleadreviewedtranscripts,compared129Thismeantthatoneresearcherwouldtranslate/transcribeawholetranscript.Asecondresearcherwouldthenlistentopartsoftheaudiofromthattranscriptandtranslate/transcribethatpart.Thetwopartswerethencompared.


©EDOREN 124

themtofieldnotesandconsultedwiththeprimarytranslatorincaseswheretherewasanyconfusion,toclarifythattranscriptionwascomplete.Thefactthatthetranscriptionswerecarriedoutverbatimhelpedmitigatebiasatthestageofdatacapturing.

4.2.4.7 Analyticalapproach

Theanalyticalapproachtothequalitativedatausedappliedthematicanalysis.Theselectedprincipalapproach—‘confirmatoryanalysis’—aimstoconfirmasetofpre-existinghypotheses(assessingthecontributionclaim)andgeneratecodesfromthehypothesestobeappliedtothedata.Thisisincontrasttoexploratoryanalysis,whichderiveshypothesesfromthedatacollected(Guestetal.,2012).Atbaseline,exploratoryanalysiswasusedasaninitialanalyticaltechnique,toensurethatthequalitativecomponentisresponsivetonewinformation(challengestothecontributionclaim).AsitwasimportanttotesttheassumptionsoftheToC,exploratoryanalysiswasusedtofirstgroupthedata,afterwhich,confirmatoryanalysiswasusedtovalidatethisinformation.

Appliedthematicanalysisrequiresresearcherstointerpretdataanddoesnotrelyoncountingwordsorphrases,butratheridentifiesanddescribesimplicitandexplicitideasthatareorganisedintothemes.Tostrengthentherigourofthematicanalysisandmitigateresearcherbiastheanalysiswascarriedoutinseveralstages.First,familiarisationwiththedatawasachievedthroughdailydebriefsinthefieldandadayinAbujawithbothstateteams,consistingofpresentationsanddiscussionsoftheinitialfindingsandkeythemes.Second,onerandomlysampledschoolfromeachstatewasselectedandusedforinitialinductivebrush-coding.130ThiswasdoneusingNvivosoftware.Alltranscriptswerecodeddirectlybasedonwhatarosefromthedata,andtheinitialcodingstructurewasdiscussedamongstthethreeteamleadstomitigatesingle-researcherbias.Theremainingtranscriptswerethereaftercodedintothisstructure,whilstallowingfornewcodestoemergeduringtheanalysis.Third,theinitialbrush-codeswereanalysedandsynthesisedintothematicnodes.Attimesthismeantgroupingnodestogether,andattimesitmeantre-codingbasedonthisinitialanalysis.

Thesetofhypothesesinthequalitativeevaluationmatrixprovidedaninitialsetofthemestobeusedtodevelopthethematicnodes.Eachpieceofdatawasconsideredinlightofthecontextfromwhichitcame(forinstance,theknowledgethatthepersoncitedislikelytohaveaboutthesubject,theincentivestheymayhavetorespondinparticularwaysandcorroborationbyotherqualitativesources).Thestrengthofthematicnodeswasfurtherdeterminedbasedonthenumberofreferencesandsourcesmakingupthenode,toensuretriangulationoffindings.Asaverbatimtranscriptionwasmadefindingswerealsoanalysedinlightofwhatandhowquestionshadbeenasked.Researchersthenassessedthebalanceofthesegroupsandwhethertheconclusionssupportedtheinitialhypotheses.FindingswereconsideredbothwithIQSsasseparatecases,andthroughcommonthemesacrossthesixIQSs.Theanalysiswasundertakenbythesameresearcherswhoconductedthefieldwork,toensurethaterrorsofinterpretationwereminimised,andfindingswerediscussedamongsttheteamateachstageofanalysis.

130Inductivebrush-codingreferstodevelopinginitialcodesastheyarisefromthedata,regardlessofwhetheritsitswithintheevaluationmatrix.Thisisdonetoensurethatanalysisisdrawnfromthedatacollected,andnotlimitedorbiasedbasedonwhatwasexpectedtobefoundpriortofieldwork.


©EDOREN 125

4.2.5 Ethicsandinclusion

TheethicsproceduresforthequalitativecomponentoftheIQSSevaluationweredescribedinthediscussionoftheethicsapprovalprocessgiveninSection3.2.8.On07October2015anoticeofResearchExemptionwasprovidedtoEDOREN.TheExemptionLetterisprovidedinAnnexE.Inaddition,afterthefinalisedtoolswereprovidedtotheOPMEthicCommittee,ethicalapprovalwasgivenon28September2015(seeAnnexE).

4.2.6 Outcomemeasurementandconstructs

Thesamemeasurementandtoolsforpupillearningoutcomes,teachermotivationandteacherknowledgeandskillsareusedintheIQSSevaluationasareusedintheearlylearninginterventionevaluation.AdescriptionofthesemeasurementsandconstructsisgiveninSection3.2.9.

4.2.7 Limitationsandriskstothemethodology

Differentlimitationsandrisksintheabove-presentedmethodologyneedtobetakenintoaccount.Someofthesecanbeaddressedbycarefulimplementationoftheevaluation,whileothersconstituterealriskstothemethodology.Table25reviewsthelimitationsandrisks.

Table25: LimitationsofandriskstotheevaluationmethodologyfortheIQSS

Limitations/risks Whythisislimitingandwhatwehavedonetoaddressthis

PlausibilityoftheToCanddepthofanalysis

Theevaluationdesigndrawscausalinferencesbyverifyingtheintervention’sToCagainstevidenceandaccountingforalternativeexplanations.ThevalidityofthisdesigndependsonaplausibleToC,theunderstandingofthecausalmechanismsatplay,andtheidentificationandverificationofalternativeinfluencingfactors.WithinthecomplexanddiverseenvironmentofreligiouseducationinnorthernNigeriathisischallenging,particularlywhencarryingouttheevaluationundertemporalandresourceconstraints.TotakeonsuchanambitioustaskwehaveconductedathoroughexplicationoftheToCoftheGEP3IQSS(seesection4.3).Furthermore,weconcentratetheevaluationinthefirstinstanceontheinterrogationofhowGEP3hascontributedtotheimprovementofteachingandlearning,whichallowsamorefocusedanalysistobeundertaken.

Neteffectmeasurement

Whilepre-andpost-measurementofoutcomesduringbaselineandfollow-upsurveysallowsthequantificationofchangesininterventionoutcomes,themethodologydoesnotforeseethemeasurementofthenetattributableeffectofGEP3’sIQSSsincechangeisnotmeasuredinacomparisongroup.Judgementaboutcausalimpactwill,rather,bebasedonareasoned,crediblestoryofhowIQSShascontributedtothechangeinoutcomevariables.

TheresultsofthequantitativesurveywillberepresentativeofthetargetpopulationsintheGEP3pilotschoolsinBauchiandNiger

ThestudyuniverseofthequantitativesurveyconsistsoftheGEP3pilotIQSsinBauchiandNiger.TheseIQSsarenotthemselvesrepresentativeoftheentireQur’anicschoolpopulationintheGEP3states.ThisisbecauseGEP3pilotIQSshavebeenselectedbasedoncertaincharacteristicsandtheGEP3LGAsmaynotfullyrepresentallLGAsinthestate.Itisimportanttobearthisinmindinregardtotheexternalvalidityofthefindings.Dueattentionispaidtounderstandingthecontextandinfluencingfactorsthatcontributetotheoutcomesobserved.ThisallowsustoassesswhetherthefindingscanbereliablygeneralisedtothestateasawholeandtoawidergroupofIQSs.


©EDOREN 126

Resourcesforendlinesurveystilltobedecided

Itisplannedtomeasurechangesinlearningoutcomesbycomparingthebaselineandendlinesituation.Atthispointtheevaluationresourcesforthe2017–2020periodhavenotyetbeendetermined.Thisposesariskfortheimplementationoftheendlinesurvey.However,thereisagreementbetweenthemainstakeholdersthatanendlineevaluationofIQSSwilltakeplace.

Compositionofcross-sectionofIQSpupilschangesbetweenbaselineandfollow-upsurveys

TheIQSSmayaffectthecharacteristicsofthepupilpopulationintheIQSsstudied,andthereforebaselineandfollow-upsurveysmayrepresentdifferentpupilpopulationsintermsofunderlyingcharacteristics.Totheextentthatsuchachangingcompositionaffectslearningoutcomes,itmayprovideanalternativeexplanationforchangesinlearningoutcomes.Wewillassessthishypothesis,butitwillnotbepossibletoquantifytheeffctofchangesinthecompositionofthepupilbodyatIQSsonlearningoutcomes.

Attritionofthesample

BothquantitativeandqualitativeresearcharebasedonalongitudinalsampleofIQSs.ItispossiblethatsampledIQSswillcloseorwillrefusetobepartoftheevaluationafterbaseline,whichwouldresultinalossofdata.WehadinitiallyassumedthattheIQSssupportedbyGEP3hadbeenselectedbasedonthedegreeoftheirintegrationandconnectionwiththeformaleducationsystem.ThebaselinedatacollectionhasdemonstratedthatGEP3IQSsarenotnecessarilywellintegratedandintegrationisnotalwaysmaintained.Theriskofattritionisthereforehigherthanwasinitiallyexpected.Nonetheless,non-integratedorrecentlyintegratedGEP3IQSswereexcludedfromthesampleframeduringbaseline.WecanthereforeassumethatthesampledcohortofIQSsarerelativelystableschools.Sincethequantitativesurveyfollowsacross-sectionalapproachthereisnoriskofpupilattrition.Teacherandheadteacherattritionispossibleandthusteacherattritionistakenintoaccountinthesamplesizecalculation.

Surveyresponseachievement

Asdiscussedabove,thesurveyteamsucceededinsurveyingahighpercentageoftheintendedschools,althoughaconsiderablenumberofreplacementshadtobesurveyed.Thepercentageofpupilsandteacherssurveyedwasaround80%,whichislowerthanexpected.Thiswasnotduetoalowresponseratebutratherbecausetheschoolsinvolveddidnothavemoreeligiblepupilsorteachers.Thelowerthanexpectedpupilsamplesizemayaffecttheprecisionofthemeasurementofchangesinlearningoutcomes.However,asthedegreeofintegrationincreasesovertime,theintendedsamplesizeofpupilsismorelikelytobeachievedatendline(giventheuseofacross-sectionalsample).Inthecaseofteachers,whichfollowsapanelapproach,weestimatethatlessprecisionwillnotjeopardiseavalidjudgementofchangesinceitisexpectedthattheincreaseinteachercompetencieswillberelativelylarge(a30%increaseaccordingtotheGEP3logframe).Ingeneral,thesubstantialsmallerschoolsamplesizecomparedtotheearlylearningevaluationlimitstheuseofregressionmodellingtoexaminerelationshipsbetweenvariables,andthereforethequantitativeanalysiswillbebasedmoreondescriptiveanalysis,whichwillbecomplementedwiththequalitativefindings.

Flooreffectsinmeasurement

Highlevelsofmissingdatawereobservedintheteacherassessments.Theamountofmissingdataisamajorconcernbecauseitmayindicatethatmanyoftheteacherswhoparticipatedintheprogrammewerenotabletocarryoutmanyofthetasksaskedofthem.Thesetaskswereintendedtorepresentthecorebasicelementsofwhatteachersshouldknowandbeabletodoinordertoteacheffectivelyataminimumlevel,sotheconcernisthattheteachersmaynotbeveryskilled.Whilethissuggeststhatthetestwas‘toodifficult’,thetestwasdesignedaroundnotionsofwhataminimallycompetentteachershouldbeabletoknowanddo.Itisprobably,therefore,unhelpfultobeoverlyconcernedaboutthetest’sdifficulty.Therealissueisthecompetencyleveloftheteachers.


©EDOREN 127

4.3 PresentationoftheupdatedToC

TheinterventionToChasanimportantmethodologicalroleintheevaluationofGEP3’sIQSS.Whilecausalinferenceinthecaseoftheevaluationoftheearlylearninginterventionisderivedfromcounterfactualcomparisonusingacontrolgroup,contributionanalysisinferscausalityfroma

However,theseflooreffectsdolimittherangeofanalysesabletobeundertakenandlimittheevaluationteam’scapacitytodrawempiricallystrongconclusions.

Socialdesirabilitybias

Anumberofitemsregardingattitudestowardsgirls’educationwereincludedintheteacherquestionnaire.Whilethereliabilityoftheresultswasacceptable,theveryhighproportionsofteachersrespondingpositivelytostatementsregardingtheimportanceofgirls’educationrevealedacomplianceeffect.Thedistributionofresponsesacrossthestronglydisagree,disagree,agreeandstronglyagreeLikertscaleindicatesthatteachersareconformingtoperceivedcorrectresponses,creatingasocialdesirabilitybiasinthemeasurement.Giventhesecomplianceeffects,thedataonteachers’attitudestowardsgirls’educationwerenotabletobeusedintheanalysis.Thislimitationhighlightstheimportanceofobservationdataandqualitativemethodstoprobesocialnormsandgenderattitudes,particularlyinanevaluationofaprogrammedescribedasaninterventiontargetedatgirls.

Biasinsamplingofthequantitativesurvey

Aspartofthesamplingprocessbiasesmayoccurwhenunitsareselectedthatdonotformpartofthetargetpopulation,orwhensomeunitsofthetargetpopulationarelesslikelytobeselectedthanothers.Insomeschoolsstakeholderstriedtoinfluencepupilsamplingorschooleligibility.ThiswasparticularlyariskinIQSs,giventheirnon-standardorganisationandthepoorqualityofthesampleframe.Totheextentpossiblethisriskwasmitigatedthroughthedefinitionandimplementationofrigorousandsystematicsamplingproceduresunderclosesupervision.

Inferencebeyondtheselectedqualitativecasestudysitesislimited

Whilethecontributionwillbeexaminedatmultiplelevels,thefindingsofthequalitativeresearchwillreflecttheparticularcommunitiesandschoolsselected.Incontrasttothequantitativedesign,thequalitativeresearchdoesnotaimtofacilitatesample-to-populationinference,butrathertogatherin-depthunderstandingofhowtheIQSSToCunfoldswithinaspecificcontext.Purposefulsamplinghasbeencarefullyconsideredtoensurein-depthlearningaboutthephenomenaofintegrationofformalsubjectsinQur’anicschools.Nevertheless,thereremainsariskthatthelearningwillbeaffectedbythechoiceofcommunitiesandschools.

Giventhenon-representativenatureofthequalitativeselectionofcommunitiesandschoolstheinformationprovidedisindicative

Thequalitativecomponentoftheevaluationoffersnuancedfirst-personaccountsofpeople’sperspectivesandexperiencesoftheactivities,withoutclaimingthattheseaccountsarerepresentativeofothersimilarcommunities’andschools’experiences.However,whenconsideredtogetherwiththerepresentativequantitativeresults,thequalitativefindingsprovideinterestingperspectivesonunderlyingissuesandfactorsthatcandeterminethecontributionofGEP3’sIQSStoimprovinglearningoutcomes,especiallyforgirls.

Thenumberofcasestudysitesislimitedduetoresourceconstraintsfortheevaluation

InordertoassesshowGEP3’sIQSScontributestoimprovedlearningoutcomes,andhowthiscontributionmaydifferacrossheterogeneouscontexts,casestudysitesthatcollectivelyaccountforarangeofcontextsshouldbeincluded.Giventheresourceconstraintsoftheevaluationonlysixsuchsiteshavebeenincluded.Ideally,nineor12casestudysitesacrosstwostateswouldprovideagreaterdegreeofgeneralisabilityasthedatageneratedwouldbebasedonagreaternumberofpossiblecontextualsituations.


©EDOREN 128

reasonedToC.Therefore,inthissectionwepresentanin-depthdiscussionoftheToCoftheIQSS,startingfromtheToCincludedintheGEP3EvaluationFrameworkandupdatedbasedoninsightsfromthebaselinedatacollection.SincetheoverallToCresultchainpresentedintheEvaluationFrameworkwaswell-founded,littleupdatinginthisregardhasbeenrequired.Thereviewmostlyfocusesonfurtherexplainingthemechanismandassumptionsunderpinningtheresultchain.


©EDOREN 129

Figure41visualisestheToCofGEP3’sIQSS.InthefollowingsubsectionswewilldiscusstheToCaccordingtothreecausalpackagesthatmakeupthisToC.Eachcausalpackageconsistsofcausalfactorsinterlinkedviacausalmechanisms,plustheirunderlyingassumptions.Acausalpackageisassumedtobesufficientforgeneratingkeyintermediaryorfinaloutcomesontheconditionthatthecausalchainandassumptionshold.ForeachcausalpackagerelatedtoGEP3’sIQSStheremaybealternative,non-GEP3relatedexplanationsfortheoutcomeoccurring,whichwillrequireexaminationaspartoftheevaluation.

Eachcausalpackagepresentedbelowcorrespondstothecontributionclaimspresentedinthemethodologicalsection.131

131(Mayne,2012).Thecausalpackageisdefinedasbeingsufficienttocauseanoutcome,whileitsindividualelements,inparticulartheinterventionelements,arenecessarypartsforthepackagetomakeadifferencetotheoutcome.


©EDOREN 130

Figure41: DiagramdepictingToCoftheIQSS

4.3.1 GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs

4.3.1.1. Causallinkagesandmechanisms

GEP3’sstrategyforintegratedQur’aniceducationaimstoprovideaccesstoqualityeducationinIQSs,particularlyforgirls,byimprovingeffective,gender-sensitiveteachingoftheformalsubjectsincludedintheharmonisedintegratedcurriculum.MoreeffectiveteachingisconsideredakeyintermediaryoutcomeoftheIQSS.Weconceptualiseeffectiveteachingasacombinationofthefollowingattributes:1)demonstratedteachingcompetency,drawingonthreetypesofknowledge(subjectknowledge,pedagogicalknowledgeandcurriculumknowledge);2)useofapupil-centredlearningapproach;3)timeontask;4)effectiveuseofteachingandlearningmaterials;and5)noobservablegenderbiasduringteachingpractice.

ThecorecausalpathwaytoachievemoreeffectiveteachingisbyimprovingIQSfacilitators’knowledgeandskillsthroughtrainingandmentoring.Thetrainingismeanttoimprovetheirknowledgeandskillsintermsofthecoresubjectsoftheharmonisedintegratedcurriculum,pedagogy,andtheintegrationprocessinQur’aniceducation.Thetrainingisalsomeanttoimpartabetterunderstandingofgender,equityandgender-sensitiveteachingmethods.Thementoring


©EDOREN 131

processisassumedtoplayanimportantroleinfacilitatorseffectivelyacquiringthisknowledgeandskills,andputtingitintopractice,byactivatingthefollowingmechanisms:

• peerinteractionthatencouragesfacilitatorstodiscussacquiredknowledgeandskills,shareexperiencesandprovidefeedbackbasedonacollaborativerelationshipamongpeers;

• school-basedsupportthatallowsfacilitatorstoaccessandrefreshtheirknowledgeandunderstandingatanytimewhilstusingtheprovidedmaterials;and

• continuousface-to-face,practicalsupportbyanexperiencedmentorbasedonalong-term,personalrelationshipthatreinforcesthelearninganddeliveryofnewlyacquiredknowledgeandskillswithconfidence.

Mentoringisalsoassumedtoincreaseteachermotivation.Whileincreasedteachermotivationisnotadirectintendedobjectiveoftheintervention,itisassumedtobeakeycontributingfactortomoreeffectiveteaching.Theunderlyingtheoryisthatpeerinteractionandongoingsupportincreasethefacilitators’perceivedteachingefficacy,self-confidenceandself-esteem,whichresultsinincreasedmotivation,asexpressedin(i)moreinterestandenjoymentintheirwork,(ii)moreeffortbeingmade,(iii)moreimportanceattachedtotheirwork,and(iv)lesspressureandtensionexperiencedinrelationtotheirwork.132Thisinturnsenhancesthelikelihoodoffacilitatorsactuallyapplyingtheirimprovedknowledgeandskills,andspendingincreasedtimeontaskonactivitiesthatbenefitstudentlearning.

ThedistributionofHausateachingandlearningmaterialscontributestomoreeffectiveteachingasmaterialsprovideguidancefortheteacherduringtheteachingprocessandfacilitatetheknowledgetransferbetweenteachersandpupils.Thisrequiresteacherstohavethepedagogicalknowledgeandskillstoeffectivelyusethematerialsduringteachingpractice.Pupilworkbooksandnotebooksalsoallowpupilstopracticeandallowteacherstoassesspupilsinwriting.TheprovisionoflearningmaterialsmayalsoensuretheinclusivenessofintegratedQur’aniceducation.However,itmustbenotedthatpovertyhasbeenidentifiedasoneofthekeybarrierstoaccessingpublicprimaryschoolandtotheextentthattheintegrationprocessforcesparentstoinvestinlearningmaterials,comparedtothesituationwithoutintegration,parents’inabilitytodosomaycausepupilstodropout.

Headteachertrainingseekstosupportamoreeffectiveteachingprocessinseveralways.First,thetrainingisassumedtoincreasepedagogicalleadershipintheIQSbyprovidingtheheadteacherwithknowledgeandskillsintermsofpedagogyandleadership.This,inturn,willprovidethefacilitatorswithschool-based,continuousguidanceinhowtobetterdeliveryinstruction.Second,leadershipcaninspireandmotivatethefacilitator,whichcontributestoimprovedattendanceanddemonstratesitselfinincreasedeffortandenjoymentofwork.Third,thetrainingalsoaimstoimprovetheheadteacher’sschoolmanagementpractices,suchasteacherattendancemonitoringandclassscheduling,whichinturncancontributetoincreasedfacilitatorattendanceandtimeontask.Improvedschoolmanagementcanalsoincreasetheresourcesavailableforfacilitatorremuneration,whichcanaffecttheirattendance,andcansubsequentlycontributetomoreeffectiveteaching.

Animproved,moregirl-friendlyschoolenvironmentandimprovedmonitoringandsupportsupervisionbylocalgovernmenteducationstaffisassumedtocontributetomoreeffectiveteaching.Abetterschoolenvironment,intermsofinfrastructure,classroomconditions,safetyandrelationshipsbetweendifferentactorsintheschool,canimproveteachers’andpupils’comfortand

132ThistheoryisbasedonthetheoryofmotivationusedbytheTDPinNigeria.


©EDOREN 132

feelingofwellbeingintheschool.Itcanalsoleadtoincreasedinstructionaltimebycreatingthephysicalconditionsrequiredforteaching(e.g.light).Girl-friendlyschoolfeatures,suchasthepresenceoffemaleteachersorfemaleparticipationinschoolmanagement,cancontributetoanincreaseingender-sensitiveteachingpracticesbycreatingaclimatethatismoresupportiveofgirls.Improvedgovernmentmonitoringandsupportsupervisioncancontributetobetterschoolmanagementandcanmotivatefacilitatorsandproprietorstocontinueprovidingtheintegratedcurriculumasitsignalsgovernmentsupportforintegration.Furthermore,sustainedfinancialgovernmentsupportcansupportfacilitatorremuneration,whichwillcontributetofacilitators’motivationandattendance.

Whilemoreeffectiveteachingisassumedtocontributetoimprovedlearningoutcomes,improvementinlearningoutcomescaninturncontributetoanincreaseinfacilitators’perceivedteachingefficacy,whichisexpectedtostrengthentheirmotivation.Anincreaseingirls’enrolmentandretentionmayalsoaffectteachingpractices:first,itmaycontributepositivelybycreatingamoregirl-friendlyschoolenvironment,whichinturncansupportgender-sensitiveteachingpractices;second,itmayhaveanegativeeffectbyincreasingpupil–classroomandpupil–teacherratios.

4.3.1.2. Assumptions

Assumptionsaredefinedhereasinfluencingconditionsthatmightenablecausallinkagesandmechanismstowork,ormayimpedethoselinkagesandmechanism.KeyassumptionsunderlyingthecausalchainofContributionClaim1arepresentedinFigure42.

Implementationassumptionsarenotincludedinthediagram.Itisassumedthatthetrainingandmentoringareimplementedasplannedandcoverallthecomponentsofthetrainingcurriculuminaqualitymanner,andthattheteachingandlearningmaterialsaredistributed.Theseareimportantassumptions,inparticularwithregardstothementoringsinceGEP3’scapacitydevelopmentapproachforfacilitatorsstronglyemphasisescontinuous,school-basedfollow-upbyexperiencedmentorscomplementedwithpeerinteraction.


©EDOREN 133

Figure42: Causalpackageandunderlyingassumptionsof‘moreeffectiveteaching’

4.3.1.3. Alternativeexplanations

Weconsidertwoalternative,non-GEP3-relatedexplanationsthatcouldalsobesufficientfortheintendedchangeinteachingintheIQSstobeobserved:

• Facilitatorsbuildcapacityinotherways.IffacilitatorsintheGEP3IQSsparticipateinothertrainingorcapacitybuildingprocessesduringtheGEP3projectperiod,teachingcouldbecomemoreeffectiveregardlessofGEP3.Suchcapacitybuildingmaytakeplacewhenfacilitatorsarealsoteachersinpublicprimaryschools.Furthermore,ifnewfacilitatorswhohaveahigherteachingcompetencylevelareemployed,teachingmayalsobecomemoreeffective.

• Facilitatorsaremotivatedinotherways.IffacilitatorremunerationorworkingconditionsimprovewithoutanyinfluencebyGEP3interventions,facilitatorsmaybecomemoremotivatedandteachmoreeffectivelyregardlessofGEP3.

4.3.2 GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentinIQSs

4.3.2.1 Causallinkagesandmechanisms

MainlythroughitssupporttoCBMCsGEP3seekstoimprovetheschoolenvironmentoftheIQSandmakeitmoregirl-friendly.Weconsidertheschoolenvironmenttoencompassthephysicalenvironment,intermsofinfrastructureandclassroomconditions,itssecurityandperceivedsafety,andtheschool’sorganisationandinstitutionalculture,intermsofleadership,organisationalprocesses,relationshipsbetweenschoolstakeholders,andtraditions.Theschoolenvironmentaffectsthelearningexperienceofthepupils,aswellastheteachingexperienceofthefacilitators.


©EDOREN 134

GEP3seekstocontributetothegirl-friendlinessoftheschoolenvironmentintheIQS,promotinginvestmentininfrastructureandresourcesthatimprovetheschoolexperienceofgirlsandaschoolorganisationthatsupportsgenderequity.

Thecorecausalpathwaytoimprovingtheschoolenvironmentisbyimprovingschoolmanagementandsupportingthemobilisationofmoreresources.Additionalresourcesthataremobilisedcanbeinvestedincriticalinputsthataddressbarrierstogirls’educationandthatareprioritisedinschooldevelopmentplans.Betterschoolmanagementisassumedtoimprovetheschoolenvironmentbecause,throughthedevelopmentofWCDPs,strengthsandweaknessesintheschoolenvironmentareassessed,prioritiesforimprovementidentified,andresourcemobilisationactionsoutlined.ThroughtheactiveparticipationoftheCBMC—whichismeanttorepresentthedifferentschoolstakeholders—theprioritiesandpreferencesofvariousstakeholdersaretakenintoaccount:inparticularthepreferencesofwomenandgirls.Itisassumedthatwomenandgirls’participationintheschoolmanagementbettersignalsgirls’needsandstrengthensthedemandforgirl-friendlyschoolinputs.Furthermore,betterschoolmanagemententailsmoreadequatefinancialmanagementandrecord-keeping,whichsupportsplanningandtransparency,andhencemoreeffectiveuseofresourcesprovidedbythecommunityorotherexternaldonors.Giventheoftenlow-resourcesettingoftheIQSsresourcemobilisationisanecessaryconditiontotranslatedevelopmentplansintoinvestments.

ThroughafunctionalCBMCcommunityrepresentativesareinvolvedintheIQS.ThisisassumedtoincreasecommunitysupportforintegratedQur’aniceducation,particularlyforgirls,becauseitmakescommunitymembersmoreconsciousoftheintegrationandempowersthemtohaveasayinitsimplementation.Thiscontributestoincreaseddemandforgirls’education.Inaddition,ifthecommunitybecomesmoreinvolvedinthemanagementanddecision-makingoftheIQSthenitisassumedthattheywillbemorelikelytosupportitfinanciallyorwithotherresourcesbecausetherewillbecommunityownership,oraperceptionofownership,overtheIQS.Finally,communitymembersareabletobetterholdschoolleadersandteachersaccountableforthequalityoftheschoolenvironmentandattendancesince,throughtheirinvolvement,theyaremoreawareofwhatoccursintheschool,andagainbecauseoftheirstrongersenseofschoolownership.

Themainmeanstoachievetheaboveintermediaryoutcomesisthetraining,mentoringandmonitoringoftheCBMCmembers.Thetrainingaimstoimprovetheirunderstandingoftheirrolesandresponsibilities,whichcanenableschoolmanagementandresourcemobilisationbecausemanagementandsupervisorytasksandfunctionswillbebetterdefined(e.g.teacherattendancemonitoring,planning,andfundraising).BytrainingCBMCmembersonwholecentredevelopmentplanningtheirskillsinschoolplanningaremeanttoimprove.Theinclusionofgenderinthetrainingcurriculumaimstocreateawarenessoftheissue,whichissubsequentlyassumedtoresultinmoregender-responsivewholecentredevelopmentplanning.TrainingonresourcemobilisationandfinancialmanagementismeanttomakeCBMCmembersmoreabletomobiliseandmanageresources.Theperiodicmentoringvisitscanreinforceandrefreshthelearningfromthetrainingworkshops,providingpracticalfeedbacktailoredtothecontextofeachIQS.Itislesscleartowhatextentthementoringvisitswillre-enforcegendersensitisationtoensurethatCBMCmembersfollowuponandmonitorthegirl-friendlinessoftheschoolenvironment.Monitoringcansignaldeficiencies,basedonwhichactioncanbetaken,andcanincreaseaccountabilitybydirectingattentiontotheachievementofresults.GEP3willsupportgovernmentstafftocarryouttermlyCBMCeffectivenessmonitoring,providingatoolthatismeanttoimproveCBMCs’monitoringcapacity.

BeyondtheroleoftheCBMC,theheadteachertrainingseekstoincreasetheheadteachers’skillsinschoolmanagement,whichisalsoassumedtocontributetomoreeffectiveschoolmanagement.


©EDOREN 135

Themini-grantsdirectlycontributetoanincreaseinschoolresourcesand,giventhattheyaretargetedatgirl-friendlyinvestments,canimprovethegirl-friendlinessoftheschoolenvironment.ThegrantscanalsoworkasanincentiveforCBMCmemberstodeveloptheircapacity,asmini-grantsareprovidedontheconditionthatCBMCmembershaveparticipatedinthetraining.Furthermore,themanagementofthegrantcanimproveoverallcommunity-basedschoolmanagementthrougha‘learning-by-doing’capacitybuildingapproachfortheCBMC,whichismeanttomanagetheresources.

Gender-sensitiveclasspracticesandtheincreasedpresenceofgirlsintheIQS,duetoincreasedenrolmentandretention,canstrengthenaschoolclimatethatsignalsanopennesstogirlpupilsandcancreateasocialenvironmentwheregirlsfeelwelcomeamongtheirpeers.

4.3.2.2 Assumptions

Figure43presentstheassumptionsabouttheconditionsthatneedtobeinplaceforIQSSinterventionstoactuallyresultinanimproved,moregirl-friendlyschoolenvironment.

Figure43: Causalpackageandunderlyingassumptionsof‘improved,girl-friendlyschoolenvironment’


©EDOREN 136

4.3.2.3 Alternativeexplanations

Weconsiderthefollowingalternative,non-GEP3-relatedexplanationsthatcouldimprovetheschoolenvironmentandmakeitmoregirl-friendly:

• CBMCmembersandschoolleadershipimprovetheircapacitythroughothermeans.SchoolmanagementandresourcemobilisationcanimprovebynewstakeholdersbecominginvolvedintheIQS,regardlessofGEP3support.Forexample,aphilanthropistmaytakeaninterestintheIQSortheIQSmaystartbenefittingfromtheinfluenceorcompetenciesofahighlyeducatedor/andrespectedcommunitymember.ItisnotonlydifficulttoidentifyallsuchinfluencesbutitwillbealsohardtoseparateoutGEP3’sinfluenceinsuchinstancessinceCBMCtrainingpromotessuchinvolvement.SomeCBMCmembersmayalsobemembersoftheSBMCintheneighbouringpublicprimaryschool,andmaybuildtheircapacitythroughtheirparticipationinpublicprimaryschoolmanagement.

4.3.3 Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlylearningenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls

4.3.3.1 Causallinkagesandmechanisms

TheprincipalfinaloutcomethatGEP3aimstoachievewithitsIQSSisforpupilslearningoutcomestoimprove,particularlyforgirls.Moreeffectiveteachingisconsideredacriticalcontributingfactor,whichwillbesupportedbyanimproved,moregirl-friendlyschoolenvironment.Asthesetwofactorsimproveandresultinbetterqualityeducationgirls’retentionisalsoassumedtoincrease.Inaddition,girls’enrolmentisexpectedtoincreasebecauseofanincreaseddemandforgirls’educationasaresultofmorecommunityinvolvementandsupportforIQSs.Girls’retentionandenrolmentarenotfurtherdiscussedbelow,astheyarenotthefocusofthisevaluation.However,itisimportanttonotethatincreasedretentionandenrolmentcaninturninfluenceeffectiveteachingandtheschoolenvironment,aswasdiscussedintheprevioussections.

Internationalstudiesindicatethatwhatteachersknow,whattheydoandhowmuchtheycareaccountsformorevarianceinpupilachievementthananyotherpolicy-amenablevariable(Hattie2003).Severalmechanismsareatplaythatmaycausemoreeffectiveteachingtoleadtoimprovedlearningoutcomesforgirls.First,thecombinationofimprovedsubjectknowledgeandpedagogicalskills,supportedbyfit-for-purposeteachingandlearningmaterials,canenablemorecompetentandequippedfacilitatorstobettersupportpupils’learningachievementintheclassroom.Byteachinginamoregender-sensitivemannertheteacher’sinteractionwiththegirlpupilimproves,whichcanimprovethequalityofthegirl’seducationinparticularandprovideadditionalmotivationforhertolearn.Second,student-centredteachingmethods—acompetencythatGEP3trainingintendstostrengthen—encourageschildren,especiallygirls,toattendschools,whichincreasesthepupil’sactualinstructionaltimeandcan,therefore,increaselearning.Third,effectiveteachingalsoentailsfacilitatorsattendingandteachingwheninschool(asaresultofbetterschoolmanagement).This,again,canincreasetheactualinstructionaltime,andhencepositivelyinfluencelearningoutcomes.

Totheextentthattheschoolenvironmentismoreconducivetogirlslearning,learningachievementisfurtherstrengthened.Again,thecausalmechanismshereareseveral.First,improvedphysicalconditionsunderwhichteachingandlearningtakeplace(e.g.lessovercrowdedclassroomsorteachinginashadedlocation)enhancethedegreeofsupportthatteacherscanprovideandtheattentionthatpupilscangivetolearning.Second,thephysicalconditionsinfluencetheinstructionaltimeasprotectionagainstrain,orlightduringeveningclasses,enableclassesto


©EDOREN 137

takeplace.Third,asafe,comfortableandenjoyableschoolenvironmentaffectspupils’experienceofschooling,whichislikelytoleadtomorefrequentattendanceandgreaterengagementintheschoolingenvironment.Totheextentthattheschoolenvironmentismoregirl-friendly,girls’schoolexperienceisparticularlyenhanced.

4.3.3.2 Assumptions

Figure44presentstheassumptionswithregardstotheconditionsthatneedtobeinplaceformoreeffectiveteachingandanimproved,girl-friendlyschoolenvironmenttoactuallyresultinimprovedlearningoutcomes,particularlyforgirls.Wedonotfocushereonassumptionsrelatedtoretentionbutmanyofthepresentedassumptionsarerelevanttoachievingtheretentionofgirls.

Therearemanyfactorsthat,inturn,influencewhethertheconditionshold.Thedifferentassumptionsarealsointerdependent.Pupils–inparticular,girls–actuallyattendingclassisinfluencedbyexternalfactors,suchastheiroutsideschoolactivitiesandresponsibilities,publicprimaryschoolattendance,economicconstraints,schoolaccessibility,andtheattitudesofthepupilsandtheirsocialenvironment.Girlsareassumedtobeinterestedinacquiringaformaleducation,whichwilldependontheiraspirationsandthevaluetheyseeinformaleducation–given,inparticular,theexpectationofearlymarriage,whichisanimportantbarriertogirls’retention.

Figure44: Causalpackageandunderlyingassumptionsof‘improvedlearningoutcomes’

4.3.3.3 Alternativeexplanations

Weconsiderthefollowingalternative,non-GEP3-relatedexplanationsthatcouldalsobesufficienttoexplainimprovedlearningoutcomes:


©EDOREN 138

• Improvedaccessandqualityeducationinotherschools.ThebaselinedatademonstratethatIQSsarenottheonlyalternativetoaccessingformaleducation.PupilsareenrolledinbothIQSsandpublicprimaryschools.Theextentthatpupilsattendthetwoschoolsdependsonmanyinteractingfactors,suchasaccessibilityofthepublicprimaryschool,therelativecostofattendingpublicprimaryschoolsandpovertyoftheparents,aswellasculturalbeliefsandsocialnormswithregardstoformaleducationandgirls’accesstothis.Iftheseunderlyingfactorschangepublicprimaryschoolattendancemayincreaseandlearningoutcomesmayimprove,regardlessoftheintegrationofformaleducationintheQur’anicschools.Thequalityofeducationalsoplaysarole.Accesstopublicprimaryschooldoesnotnecessarilyresultinimprovedlearningoutcomes.TotheextentthatthequalityofeducationinthepublicprimaryschoolsthatIQSpupilsareattendingbecomesbetter,theirlearningoutcomescanimprove.Furthermore,inlessremoteareaspupilsmayattendvocationalorothertraining,whichmayprovidesomeformofbasiceducationormotivatepupilstolearn.Finally,inmoreurbanareasformalprivateschoolsmayalsoattractpupils,whichcanaffecttheirlearningoutcomes.

• Changesinthecompositionofthepupilpopulation.Overtime,theIQSmayattractadifferentpupilpopulation:forexample,youngerpupilsstarttheintegratedcurriculumormorepupilswithpreviouseducationexperienceareattractedtotheIQS.TotheextentthattheirlearningoutcomelevelsaredifferentfromthelearninglevelsofthebaselinepopulationchangesintheaveragelearningoutcomeintheIQSmaybeobserved,regardlessofimprovedteachingoranimprovedschoolenvironment.

4.4 Analysisofthedata–ContributionClaim1:GEP3’ssupporttoIQSscontributestomoreeffectiveteachingofformalsubjectsinIQSs

Thenextthreesubsections(Sections4.4–4.6)presentthebaselineanalysisofthequantitativesurveydatagatheredin60IQSsandthequalitativecasestudyfindingsinsixIQSsinNigerandBauchi.TheevidencehasbeenstructuredaroundthethreecontributionclaimspresentedintheIQSSToC.Thesebaselinefindingswillfunctionasthereferencepointtowhichthemidlineandendlinedatawillbecomparedinordertoassesstheimpactcontributionoftheprogramme.

Thequantitativeanalysispresentsadescriptiveanalysisofassociationsofbaselineindicatorsacrosscategoriesofinterest.GiventhatthesamplesizeoftheIQSSsurveyisrelativelysmallanddataaredisaggregatedfurtheracrossvariouscategories(bystate,byownership,etc.)theaveragespresentedhavelargeconfidenceintervals.Hence,itisimportanttoemphasisethattheseassociationsshouldnotbeusedtomakestatisticalclaimsofcausalinference.Rather,theyshouldonlybetreatedasasummaryofthebaselinesample.Thefigurespresentedalongsidethetextserveadescriptivepurpose,tofacilitatethenarrativeregardingtheresults.Alargersetofdescriptivestatistics,includingmeanestimates,standarderrorsandnumberofobservationsforeachvariableonaverageandbystate,isincludedinAnnexM.

Inthissectionwepresentthebaselinedatarelatedtoteachers,teachers’knowledgeandteachingpracticesinIQSs.Inaddition,wediscussothercomponentsoftheIQSSToCthatareassumedtosupportmoreeffectiveteaching:thatis,pedagogicalandschoolleadership,andaccesstoteachingandlearningmaterials.Thequantitativedataanalysisispresentedfirst,followedbytheanalysisofthequalitativecasestudyfindings.


©EDOREN 139

4.4.1 Quantitativeanalysis

ThequantitativeanalysisbeginswithadescriptionofthecontextsinwhichtheteachingofformalsubjectstakeplacewithinIQSsandprovidesadescriptionofwhatfacilitatorsknowanddowithintheirlessons.TheanalysisthenturnstoreportingonthequantitativeoutcomesasperthefirstcontributionclaimoftheGEP3IQSS.TheToCfortheIQSSprogrammeindicatesthattheteachingoffacilitatorswillbecomemoreeffectiveasaresultof:(1)theincreasedmotivationoffacilitators;(2)improvedsubjectknowledgeandpedagogicalskillsoffacilitators;(3)improvedskillsingender-sensitiveclasspractices;(4)theavailabilityofteachingandlearningmaterials;and(5)increasedheadteacherpedagogicalleadership.Thissectionoutlinestheevidenceatbaselineoneachoftheseoutcomes.

4.4.1.1 TheIQScontext

Overviewofschoolstructure

IQSstakevariousformsinBauchiandNiger,rangingfromthosethatareorganisedatasinglelevel,whereallthechildrenareinthesame‘class’studyingunderasinglefacilitator,tothosewheretheschoolisstructuredexactlylikeapublicprimaryschool,withchildreninclassescalledPrimary1,Primary2andsoon.InthesampleofIQSsamajorityoftheschoolswereorganisedinamannersimilartopublicprimaryschools.However,therewereotherschoolstructuresaswell.Table26mapsthevariousIQSstructuresencounteredagainsttheircorrespondingprimaryschoolclasses.

Table26: MappingIQSstructurestoprimaryschoolclasses

Type1 Type21 Type3 Type4 Type5

P1 Awal

Class1/Stage1/Aji/Tsangaya

Class1/Tsangaya1/Aji1/Stage1

Class1/Stage1/Grade1P2 Sani

P3 Salisu Class2/Stage2/Grade2P4 -2 Class2/

Tsangaya2/Aji2/Stage2

P5 - -

P6 - -

62% 5% 8% 15% 10%

Notes:1Awal,SaniandSalisuareArabicforfirst,secondandthird,respectively.2‘-‘indicatesthatacorrespondingleveldidnotexistattheIQS.Forexample,Type2schoolsdidnotprovideupperprimaryclasses.

BothType1andType2IQSshaveonelevelcorrespondingtoeachprimaryschoolclass,thedifferencebeingthatType1useIQSsthepublicprimaryschoolclassificationofPrimary1andPrimary2whileType2IQSsusealocalcategorisation.67%ofthesampledschoolshadone‘level’correspondingtoeachprimarygradeclass.ItisworthnotingthatType2schoolsonlycateredtothelowerprimarygrades,anddidnothavelevelscorrespondingtoP4–P6.AninterestingdifferenceisthatinNigeralargershareoftheIQSswereType1orType2,whileinBauchiarelativelylargershareofIQSswereType3.Inaddition,therewerenoType2IQSsinBauchi.Thisdifferencealsoholdsintherural–urbandisaggregation:alargerproportionofruralIQSshadonelevelperpublicprimaryschoolclass(Type1orType2)whileurbanIQSsweremorelikelytobeoneoftheremainingthreetypes.FortheIQSSsamplearelativelylargershareofIQSsinBauchiwereurban,comparedtoNiger.Asaresult,moststate-leveldisaggregationsdiscussedinthissectionandinwhatfollowsalsohold


©EDOREN 140

trueforurban–ruraldisaggregations,whereNigerisdrivingtheresultsassociatedwithrurality.Forschooltype,thissuggeststhatruralschools,andschoolsinNiger,weremoreoftenstructuredlikeformalpublicprimaryschools.

Type3schoolsarethosewhereallthechildrenstudyinthesameclass,whichcouldbeduetoashortageofspace,studentsorteachers.Thiswastheleastcommontypeofschoolingstructureencountered.Type4schoolsareschoolswhereeachlevelintheIQScorrespondstothreeprimaryschoollevels.Hencetheschoolhaspupilsattwo‘levels’.ThesearemostcommonlyreferredtoasStage1andStage2,butthisisnotalwaysthecase.Thelastcategory,Type5,includesschoolswhereeachlevelcorrespondstotwoprimaryschoollevels.Throughoutthisreport,wherewerefertoP1–P3‘level’orP2orequivalent‘level’theschoolinquestionmaybeorganisedaccordingtooneofthefivetypesofstructuresdiscussedabove.

Schoolshadanaverageof6.6teachersofbothintegratedandQur’anicsubjects,andthisdidnotvarymarkedlyacrossstates.Ofthese,anaverageof2.78teachersperschoolwereteachersofintegratedsubjects(i.e.facilitators),while2.22facilitatorsperschooltaughtattheP1–P3level.25%ofIQSshadonlyonefacilitatorforgradesP1toP6,while36.7%hadonlytwo.Whenfocusingonlyonearlygrades(P1–P3),aroundone-thirdoftheschoolshadonlyoneteacherofintegratedsubjects.Thishasimplicationsfortheteacher–teacherinteractionassumedaspartoftheIQSSToC(seeSection4.1.3),sinceschool-basedpeerinteractionwillnotbepossibleforoveraquarterofthetrainedfacilitators.

Thereareimportantdifferencesinthenumberandgenderofteachersofintegratedsubjects,acrossstates.Theseteachersarereferredtoasfacilitatorsthroughoutthisreport.Fromhereonwardstheanalysiswillfocusonlyonfacilitatorsthatwereteachingregularclassesofsubjectsintheintegratedcurriculum.Table27showsthatNigerhasmoreteachers,andmorefemaleteachers.However,overalltherearefewfemalefacilitatorsinIQSs.

Table27: FacilitatorsinIQSs,bystate

Variable Bauchi Niger

Totalnumberofteachers1 6.66 6.5

Numberoffacilitatorsatalllevels 2.39 3.18Fractionoffacilitatorsthatarefemale 0.03 0.16

NumberoffacilitatorsP1–P3 1.93 2.5

FractionoffacilitatorsthatarefemaleP1–P3 0 0.07

Note:1Referstobothteachersofreligioussubjectsandintegratedsubjects

SchoolownershipforIQSsisdifferenttoownershipofpublicprimaryschools.IQSscanbeheadedbyaheadteacher,whoisresponsiblefortheday-to-dayfunctioningandmanagementoftheschool.Theschoolowneriscalledtheproprietor,andmaybethesamepersonastheheadteacherormaybeadifferentperson.Inone-thirdoftheIQSstheheadteacherwasalsotheproprietor,whilefortheremainingschoolstheseroleswerefilledbydifferentindividuals.Thedifferentschoolownershipstructuresledtointerestingtrendsinthedata.

Schoolswheretheheadteacherandproprietorroleswereseparatedhadmorefacilitatorsonaverage,andhadmorefemalefacilitators.Theyalsohadbetterinfrastructure,asindicatedbythepresenceofmoreroomsoverall,moreroomsbeingusedforclasses,morefunctioningtoiletsfor


©EDOREN 141

pupils,andthegreaterlikelihoodofhavinganelectricityconnection.Thishintstowardsthefactthatschoolswheretheheadteacherandproprietorrolesareseparatedareperhapsbetterestablishedasschools.Thisnotionissupportedbythefactthatsuchschoolshadbeenintegratedforlongerthanschoolswheretheheadteacherandproprietorwerethesameperson.SchoolswheretheserolesareseparatedaremorecommoninNigerthaninBauchi.

Asmentioned,allthefacilitatorssurveyedtaughtintegratedsubjectsattheP1–P3levelatthetimeofthebaselinestudy.Aminorityofthem(around12%)alsotaughtP4toP6classes,thusbeinginvolvedinteachingatbothupperandlowerlevelsoftheschool.ThiswasparticularlyprominentinBauchi.Whenfacilitatorswereaskedtospecifythesubjectstheywerecurrentlyteaching,eachfacilitatorreportedteachinganaverageoftwointegratedsubjects.Inlinewithoursamplingstrategy,amajorityofthesurveyedfacilitatorswereteachersofmathematicsorEnglish,with65%offacilitatorsteachingmathematicsand71%ofthefacilitatorsteachingEnglishinthecurrentterm.Surveyedfacilitatorsalsotaughtotherintegratedsubjects:32%ofthefacilitatorsreportedteachingHausa,24%reportedteachingsocialstudiesand11%reportedteachingbasicscience.Thisreflectsthefactthatallschoolswerenotteachingallfivecoreintegratedsubjects:English,mathematics,Hausa,scienceandsocialstudies.Findingsfromtheschool-levelheadteacherinterviewconfirmthatalittleover10%ofsurveyedschoolswereteachingallfiveintegratedsubjects.ThistrendisbeingdrivenbytheschoolsinNiger,ofwhichanoticeablylargerproportionwereteachingallfiveintegratedsubjectscomparedtoschoolsinBauchi.Althougharound70%ofsurveyedschoolstaughtHausaasasubject,thefocusonEnglishandmathematicsteachingwasclearinthedata,with90%oftheschoolsteachingbothofthesesubjects.Thevariedadoptionoftheintegratedcurriculumdeservesattentionduringimplementation,totheextentthatGEP3seekstoimprovetheeffectiveteachingofallcoresubjects.133Forschoolsthatarenotalreadyteachingmultipleintegratedsubjects,additionalsupportmayberequiredforfacilitators,aswellaspupils,tocopewithadditionalsubjects.

Whenlookingattheentirerangeofintegratedsubjectsbeingtaughtbysampledfacilitators(whichincludesphysicalandhealtheducation,creativearts,andcomputerscience,inadditiontothefivecoreintegratedsubjects),weseethatslightlymorethanathirdoftheIQSfacilitatorswereteachingonlyoneofthesubjects,whileanotherthirdwereteachingtwosubjects(seeFigure45).Therearesharpintra-statedifferences:inNiger,onlyaboutaquarterofthefacilitatorswereteachingasinglesubject,whileabouthalfwereteachingtwosubjects.ThistrendwasreversedforBauchi,whereapproximatelyhalfthefacilitatorstaughtonlyasingleintegratedsubject.ThisisinlinewiththefactthatmoreschoolsinNigertaughtmultiplesubjects,whileschoolsinBauchitaughtonlyoneortwo.

133TheGEP3StrategyPaperforIQSs(UNICEF,2015b)indicatesthatGEP3willsupportthedevelopmentofatrainingpackageandimplementationprogrammetoimprovethecapacityofIQSteachers/facilitatorstobeeffectiveteachersofcoresubjects.Thestrategydoesnotspecifywhetherthisreferstoallcoresubjects.TheTrainingManualforIQTETeachersandFacilitatorscoversallcoresubjects.However,theoutcomesstatedintheStrategyPaperfocusontheimprovementofpupils’literacyandnumeracyskills,whichindicatesthatliteracyandmathematicsmaybethecoresubjectstheeffectiveteachingofwhichtheprojectwantstoemphasise.


©EDOREN 142

Figure45: Numberofintegratedsubjectstaughtperfacilitator

FacilitatorsinIQSs

ThetypicalfacilitatorinIQSSschoolsismale,around32yearsofageandspeaksHausa.Femalefacilitators,thoughfewinnumber,areconcentratedinschoolsinNiger.Thereseemstobemobilityacrossschools,asindicatedbytheaveragenumberofyearsasateacher(aroundsevenyears)versustheaverageyearsofteachingatthecurrentschool(around3.5years).Alittleoverhalfthefacilitatorshadbeenatthecurrentschoolformorethantwoyears,whichislikelyrelatedtotherecentintegrationofmanyIQSs.WecollectedinformationonteacherturnoverattheP1toP3levelforasmallsampleoftheschoolsandfoundthatalthoughforamajorityoftheschools(abouttwo-thirds)nofacilitatorsfromP1–P3hadlefttheschoolduringthelastoneyear,inschoolswhereteacherturnoverwaspresenttheturnoverrateswerehigh.Asashareoftotalfacilitatorsattheearlygrades,anaverageof80%offacilitatorslefttheschoolinthepreviousyear.Thisamountstoanaverageof2.1facilitatorsleavingtheIQSduringthepreviousyearinIQSswithapositiveturnoverrate.ThishighlightstheneedtoincorporatetheriskofteacherturnoverintothedesignofteachertraininginitiativesbecauseiftrainedteachersleavetheIQSpupilsmaynotbeabletobenefitfromtheinvestmentinteachertraining.

FacilitatorsinBauchiareolder,withnoticeablymoreyearsofteachingexperiencethanfacilitatorsinNiger.ThenumberofyearsspentatthecurrentschoolisalsogreaterforBauchifacilitators.

AlmostallofthefacilitatorsreportbeingabletospeakHausa,andmostofthemreportbeingabletospeakEnglish.InNiger,Nupeisacommonlanguage,with39%offacilitatorsreportedlyspeakingthelanguage.InBauchi,facilitatorsgenerallydonotspeakathirdlanguage,althoughArabicandFulfuldearespokenbyaminority(21%and13%,respectively).Thiscorrespondsclosely,atthestatelevel,tothelanguagesspokenathomebypupils.

Aroundhalfofthefacilitatorshavesomedegreeofprofessionaleducationalqualification.Offacilitatorswhohadaformofprofessionalqualification,around80%hadanNCE,while15%hadaGrade2certificate.TwiceasmanyteachersinBauchihadsomeformofprofessionalqualificationcomparedtoteachersinNiger.Only20%oftheNigerteachersdeclaredthattheyhadanNCE,comparedto51%oftheBauchiteachers.Hence,thefacilitatortrainingneedstobeadaptedto

35.00%33.00%

21.00%

8.00%

2.00%

0%

5%

10%

15%

20%

25%

30%

35%

40%

1 2 3 4 5


©EDOREN 143

teachersthathavenoprofessionaleducationalqualificationinteaching,particularlyinNiger.Intermsofacademicqualifications,asmallshareofthesamplewasonlyeducateduptotheprimarylevel,whileamajorityhadanSSCE.Aroundone-fifthoftheteachersalsohaveareligiouseducationqualificationofsomeform.AconsiderablylargershareoftheBauchiteachershadareligiouseducationqualification,comparedtoteachersinNiger(33%versus11%).

Ofthetotalfacilitatorsinterviewed,around43%hadattendedsomeformoftrainingduringthelasttwoyearswhilebeingemployedatthisschooloranotherschool.Facilitatorsattendedaroundsixdaysoftrainingonaverageduringthelasttwoyears.MorefacilitatorsinNigerattendedtraining,butthereporteddurationoftrainingwaslongerforBauchi.Olderfacilitatorsweremorelikelytohaveattendedatrainingcoursethanyoungerones.Inaddition,facilitatorswithsomedegreeofprofessionalqualificationweretrainedmoreoftenandforlongerthanthosewithout.Thissuggeststhatdifferencesinteacheroutcomesseenacrossprofessionalqualificationcategoriescouldpartlystemfromthefactthatthesearetheteachersthathavebenefiteddisproportionatelymorefromtraining.

Therearealsointerestingdifferencesinthecontentofthefacilitatortraining.Thoughtheactualpercentagesareperhapsnotasinformative,itisinterestingtoseethatfacilitatorsinBauchireportedbeingtrainedinHausateachingandintegrationactivitiessignificantlymoreoftenascomparedtoteachersinNiger.Perhapsthesourceofthetrainingcanexplainsomeofthesedifferences–inBauchifacilitatorsreportedbeingtrainedbySUBEB25%ofthetime,whilethiswasthesourceoftrainingreportedbyonly4%oftrainedfacilitatorsfromNiger.TrainingbyGEPandUNICEFwerereportedat77%and67%inBauchiandNiger,respectively.Surprisingly,therewerenoreportsofanytrainingbySAMEineitherofthestates.

Figure46: Facilitatortrainingcontent,bystate

Schoolandclasssize

Schoolsize,asindicatedbyclassroomsinuse,varies,withanaverageoftworoomsobservedbeingusedforclassesonthedayoftheschoolsurvey.Themeanconcealstherealitythataround20%ofIQSshadnoclassroomstructure.Thisfactisalsosupportedbythequalitativeresearch.Anadditional30%oftheschoolshadonlyoneclassroom.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Trainedoncommunityinvolvement

Trainedonintegrationactivities

Trainedonschoolleadership

Trainedoncurriculumsubjects

TrainedonteachinginHausa

Trainedonliteracyandnumeracy

Trainedonteachingmethods

Bauchi Niger


©EDOREN 144

Pupilenrolmentdatawerecollectedinordertodevelopanarrativeregardingthepupil–teacherratioandthepupil–classroomratio.Adetailedanalysisofthesedataisnotpossiblesinceonlyaverysmallshare(about30%)oftheIQSshadsomesortofwrittenenrolmentrecordforthecurrentschoolyear.Thispresentedamajorhindranceinthecollectionofreliabledataonpupilnumbers.Despitethelimiteddataavailableonpupilenrolment,itisclearthatthenumberofpupilsstudyingintegratedsubjectsinIQSsvariesgreatly.Figuresrangefromasfewas20toasmanyas400childrenintheschool,withcross-statevariationbeingaslargeaswithin-statevariation.Ofthese,amajorityofpupilswereenrolledintheearlygrades,i.e.P1–P3level.AlthoughthegenderratiodoesnotseemtopointtowardsadisproportionatelyhighernumberofboysinIQSs,averylimitedsamplesizeforenrolmentdatadoesnotallowustomakestrongerclaimsaboutthegenderratiointheseschools,anditsimplications.Overall,theratioofgirlstoboyssuggeststhatIQSsdonothavemoreboysthangirls.Thisinferenceissupportedbythe(limited)dataonpupilenrolment,andmoresobythemanualcountingofP2pupilspresentonthedayofthevisit.Interestingly,theratioofgirlstoboyswashigherinBauchithaninNiger,andincaseswheretheheadteacherwastheproprietor.Thoughthiscouldpartiallybebiasedduetotheschools’knowledgeofthesurveyandthefocusoftheprogrammeongettinggirlsintoschool,thedifferencesarenoteworthy.Thisalsoindicatestheneedtoinculcaterecord-keepingpracticesinIQSsaspartofschoolmanagementpractices.

Theclassroomobservationdatasuggestameanpupil–teacherratioof45andamedianof40.Therangeisquitevaried,fromfourto183childrenbeingtaughtbyoneteacher.Aqualitativeindicationofthefindingthatthepupil–teacherratioishighinIQSscomesfromthefactthat94%ofsampledfacilitatorsagreedwiththestatement‘Therearetoomanypupilsinmyclassroom’.HenceitappearsthatfacilitatorsinIQSsareteachinginvariedandchallengingcontextsintheIQSSschools.

IQSintegration

IQSsinthesamplehavebeenestablishedforapproximately25yearsonaverage,withBauchischoolshavingbeenestablishedforalmosttwiceasmanyyearsasschoolsinNiger.Theschoolshavebeenintegratedfor2.5yearsonaverage,withschoolsinBauchihavingbeenintegratedforaboutoneyearlessthanschoolsinNiger(twoyearsversusthreeyears,onaverage).

Integration,asmeasuredbyschoolsteachingtheentirepackageoffivecoreintegratedsubjects,isfoundtobegreaterinschoolsinNigerthaninBauchi.Inthelatter,only3%oftheschoolswereteachingallfivecoreintegratedsubjects,whileinNiger,20%oftheschoolsweredoingso.SchoolsinNigerwerealsospendingmorehoursperweekonteachingintegratedsubjects.InBauchi,studentsstudiednon-religioussubjectsfor2.2hoursonaverage,whileinNigeritwas3.8hours–asignificantdifferenceandanindicatorthatschoolsinNigeraremoredeeplyintegrated.Thereisoneoutlier–oneschoolintheNigersamplereportedlyprovides17hoursofintegratedsubjectteachingperweek(allotherschoolsacrossbothstateshadintegratedsubjectteachingfor11hoursorlessperweek).Byremovingthisoutlierfromthesample,theaverageforNigerfallsto3.3.Althoughlower,thisstillsupportsthefactthatschoolsinNigerareteachingmoreintegratedsubjectsforalongerdurationthanschoolsinBauchi.

ItisnotstraightforwardtoassumethatsinceNigerschoolsarespendingmorehoursteachingintegratedsubjectstheymustbeswitchingawayfromreligioussubjects.Thechartbelowshowstherangeofreligioussubjectstaughtinschools,bystate.ForallbutonesubjectalargershareofschoolsinNigerwereteachingeachsubjectmentionedbelow.


©EDOREN 145

Figure47: Religioussubjectstaughtinschools,bystate

BauchischoolsseemclosertothetraditionalstructureofIQSs,wherechildrenstayonasboarders.43%oftheschoolsinBauchihadboarders,comparedtoonly27%oftheschoolsinNiger.Inaddition,schoolsinBauchiweremostlyfreeforboarders,whereasaround25%oftheschoolsinNigerchargedboarders.

ThefindingsonthedegreeofintegrationsuggestthatIQSsinNigerarebetterintegrated,asevidencedbyteachingofmoreintegratedsubjects,andmorehoursspentonteachingformalsubjects.Bauchischoolshavebeenintegratedforfeweryears,whichperhapsexplainsthelesserfocusonteachingintegratedsubjects.Thevariationinthedegreeofintegrationmayresultinvariationsintheimprovementofeducationqualitythattheprojectwillachieve.Inschoolswithlimitedhoursspentonteachingformalsubjectsandonlyshortexperienceofteachingintegratedsubjectsthepotentialimpactonlearningmaybeless.However,asmentioned,mostIQSsprovidemathematicsandliteracyclasses,whichcorrespondstotheIQSSinterventionsfocusonnumeracyandliteracylearningoutcomes.

4.4.1.2 Whatfacilitatorsknow

Thissectionofthereportdescribestheknowledge,skillsandpracticesoffacilitators,ascollectedthroughthreeprimarysources:afacilitatorassessment,aclassroomobservationandafacilitatorquestionnaire.Facilitatorknowledgeandskillsreferstoproficienciesasperthesubscalesdrawnfromthefacilitatorassessmentandaqualitativereviewoffacilitatorresponsestoopenendedquestionswithintheassessment.Facilitatorpracticesreferstothepracticeoffacilitatorsintheclassroom,asobservedthroughtheclassroomobservation.Facilitatorpracticeswerecategorisedintothreecategories:teachertalk,teacheractionandpupilaction.

Definingteachercompetence

Thepercentageofteacherswhodemonstrateminimumteachingknowledgeinliteracyandlanguageismadeupofthesixsubscalesdevelopedfromtheteacherassessment.

Thresholdsweredefinedforeachsubscaletodifferentiatebetweenteacherproficiencylevels.

60%

47%

57%

40%

63%

53%

60%

50%

70%

23%

70%

63%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Arabic

Tajweed

Quran

Sira

Hadith

Fiqh

Shareofsc

hools

Niger Bauchi


©EDOREN 146

Twocut-offpointsweredefinedforeachscale,tocreatethreeproficiencylevelsperscale.Theproficiencylevelsare:lowband–noevidenceofskill;middleband–evidenceofrudimentaryskill;andupperband–evidenceofcompetence.

Facilitators’knowledgeandskills

Thefacilitatorassessmentwasdividedintothreesections,collectivelycomprising30items,includingmultiplechoice,shortresponseandlongresponseitems.

• InsectiononefacilitatorswereaskedtomarkpupilresponsestoHausaliteracyquestionsandtoindicatethegradelevelatwhichtheanswershouldbeknownbypupils,asdefinedbythecurriculum.

• InsectiontwofacilitatorswereaskedtofillinmissinginformationinordertoprepareananswersheetforareadingtestaimedatGrade2pupils.Facilitatorswereprovidedwithtwonewspaperarticlesandaskedtofillinmissinginformation;thisincludedansweringbasiccomprehensionquestionsandquestionsthatrequiredtheinterpretationofwordsandphrases.

• Sectionthreeaskedfacilitatorstoidentifypoorandgoodpupils’workandtoreviewpupils’workinordertomakejudgementsaboutpupils’writing,includingtheorganisationofthewriting,theuseofgrammar,punctuation,spelling,thepupil’sabilitytoself-correctandreflectontheirwritingandthepupil’sabilitytoformlettersandusespacesbetweenwords.Facilitatorswerethenaskedtodescribehowtheymightsupportthepupilinimprovingtheirwriting.

AscanbeseeninFigure48thevastmajorityoffacilitatorsdidnotdisplayevidenceofcompetenceinthesixdomainscoveredbythefacilitatorassessments.Noneofthefacilitatorsassessedwereabletoshowcompetenceinevidencingjudgementsanddiagnosingpupils’work.Similarly,lessthan1%hadeffectivewritingskills.Lessthan3%offacilitatorswerefoundtobecompetentinidentifyinglowperformers,andininterpretingwordsandphrases.Rudimentarylevelsofskillswereobservedinaverysmallproportionoftheassessedfacilitatorsinthesefourdomains.

Figure48: Percentageoffacilitatorsachievingwithinthelower,middleandupperbandsofcompetenceacrosstheteacherknowledgeandskillssubscales

0%

20%

40%

60%

80%

100%

Abilitytoidentifylowperformers

Abilitytoevidence

judgementsanddiagnose

Teacherwritingskills

TeacherGrade2Hausa

knowledge

Teachercomprehensio

nskills

Interpretingwordsandphrases

Upperband 1.9 0 .9 34.0 25.4 2.8Middleband 6.5 .9 .9 27.2 18.1 4.2Lowerband 91.6 99.1 98.1 38.8 56.5 92.9


©EDOREN 147

Thefindingthatnoneofthefacilitatorsassessedwerecompetentinevidencingjudgementsanddiagnosingpupils’workorwritingissignificant.Ashighlightedintheearlylearningevaluationsectionofthisreport,thereisaverystrongtraditionineducationalresearchthatastudentlearnsbestwhenteachingistargetedtowhats/heisreadytolearn.Facilitators’inabilitytoidentifylowperformersandevidencetheirjudgementsconcerningpupilperformancepresentschallengesinregardtoimprovingpupillearning.

AgreaterproportionoffacilitatorswereabletodisplayknowledgeandskillsinGrade2Hausaandincomprehensionthanotherareasoffacilitatorknowledgeandskills.However,itisimportanttostressthatthelevelsofHausaassessedwerebenchmarkedtoGrade1-and2-levelHausaknowledge.Itemstestedincludedbasicgrammarandtheinitiallettersofeverydayobjectsandanimals.Thefindingthatover60%offacilitatorswerethemselvesunabletodisplaycompetenceinGrade2-levelHausaraisesseriousquestionsaboutfacilitators’abilitytoraisepupils’learninglevelsinthisarea.

Inordertoinvestigatetheextenttowhichfacilitators’knowledgeandskillsareassociated,ananalysisofthecorrelationbetweenthesubscalesdevelopedfromthefacilitatorknowledgeandskillsassessmentwasundertaken.Thetablebelowshowsthecorrelations(Pearson’s)betweenthesubscales.

Table28: Correlationsbetweensubscalesoffacilitators’knowledgeandskills

Syllabu

skno

wledge

Abilityto

iden

tifylowperform

ers

Abilityto

evide

ncejudgem

entsand

diagno

se

Teache

rwritingskills

Teache

rHau

sakno

wledge

Teache

rcom

preh

ensio

nskills

Interpretin

gwordsand

phrases

‘Syllabusknowledge’

PearsonCorrelation

1

Sig.(two-tailed)

N 2935

‘Abilitytoidentifylowperformers’

Correlation .051 1

Sig.(two-tailed) .218

‘Abilitytoevidencejudgementsanddiagnose’

Correlation .224** .188** 1

Sig.(two-tailed) .000 .000

‘Abilitytobuildonpupilknowledge’

Correlation -.103* .083* .415**

Sig.(two-tailed) .013 .043 .000

‘Teacherwritingskills’Correlation .272** .527** .476** 1

Sig.(two-tailed) .000 .000 .000


©EDOREN 148

‘TeacherHausaknowledge’Correlation .219** .273** .171** .214** 1

Sig.(two-tailed) .000 .000 .000 .000

‘Teachercomprehensionskills’

Correlation .068** .246** .134** .227** .228** 1

Sig.(two-tailed) .000 .000 .001 .000 .000

‘Interpretingwordsandphrases’

Correlation .159** .141** .157** .528** .124** .225**

Sig.(two-tailed) .000 .001 .000 .000 .000 .000

*Correlationissignificantatthe0.05level(two-tailed),and**thecorrelationissignificantatthe0.01level(two-tailed).

Therearefourcorrelationsinthistablethatarenoteworthy(showninred):

• ‘Abilitytobuildonpupilknowledge’with‘Abilitytoevidencejudgementsanddiagnose’(r=0.415)

• ‘Facilitatorwritingskills’with‘Abilitytoidentifylowperformers’(r=0.527)• ‘Facilitatorwritingskills’with‘Abilitytoevidencejudgementsanddiagnose’(r=0.476)• ‘Interpretingwordsandphrases’with‘Facilitatorwritingskills’(r=0.528)

Thesecorrelationssuggestthatfacilitators’writingskillsinparticularmaybethekeydimensioninfluencingfacilitators’performanceinthetest.Inotherwords,theliteracylevelsofthefacilitatorsappeartobeakeyissue.Ifthislevelislow,itlimitsperformanceacrossarangeoftheareasfacilitatorsneedtobecompetentintoimprovepupillearning.

Overall,thesefindingsstarklyhighlightthattherearelargegapsinfacilitators’knowledgeandskillsonalldomainsassociatedwitheffectiveteaching,includingbasicHausaliteracy.AsnotedinSection3.4,effectiveandwell-designedtrainingprogrammescanfillanyknowledgeandskillsgaps,includingthoseinteachers’ownsubjectknowledge.ThechallengeforGEP3willbetoensurethatitsfacilitatortraininginterventionisdesignedandimplementedinamannerthatallowsittoaddresstheseverylargedeficienciesinteachers’competencies.Thereisalsoaneedfortheprogrammetoberealisticaboutwhatcanbeachievedintheshorttimeframecoveredbyitspilotphase.

Qualitativereviewoffacilitatorresponses

Thefacilitatorknowledgeandskillsassessmentincludedaqualitativereviewoffacilitators’responsestoincorrectanswersbypupils.Anitemwithinthefacilitatorknowledgeandskillsassessmentprovidedalistofpupilnameswithinaclass,thegenderofeachchildandthenumberofwordseachchildcouldreadperminute.Facilitatorswereaskedtofirstdescribetheachievementofgirlsandboysintheclass,identifywhichpupilsneededthemostassistanceandsupport,andprovidewaysinwhichafacilitatorcouldsupportpupilstoincreasetheirabilitytoreadandunderstandtext.Withintheitem,abouthalfofthepupilsscoredzeroandtherewasasmalldifferencebetweenboys’andgirls’achievement,withboysscoringslightlybetterthangirls.Facilitatorswerealsoprovidedwithexamplesofpupils’writingandaskedtowritewhatthefacilitatorshouldfocusontoimprovethepupils’handwriting.

Thereviewfoundthatfacilitatorsgenerallyperceivedeffortasthecauseoflowperformanceandheldthatincreasingpupileffortwasthebestresponsetolowperformance.IQSSExamplesAandBprovideexamplesoffacilitatorresponsesthatfocusonincreasingeffortandpayingattentioninclass,ratherthandisplayingknowledgeregardingtheerrorsinthepupil’sworkandhowafacilitator


©EDOREN 149

mightmeaningfullyimprovethepupil’swriting.Thefocusoneffortwithinthiscontextisalikelyreflectionofthelowlevelsofcompetenceamongstfacilitators–thefactthattheylackboththeknowledgeandskillstoimprovepupillearningleavesthemwithfewexplanationsforlowperformance.

IQSSExampleA:Facilitatoreffort

SupportforIQSExampleB:Teachereffort

Responsesalsoreflecteddevelopmentprogrammeaimsandobjectivesinshallowways.Responsesincludedstatementsabouttheimportanceoffocusingongirlsintheclass,withoutanysupportingstatementsidentifyinglowperformersintheclassorsuggestinghowgirls(andboys)canbesupportedbythefacilitator.

Someresponsespointedtodeeplyingrainedgenderbiases,withperceptionsthatgirlsarenaturallylesssuitedtothedemandsofeducationthanboys,ormoresuitedtootherroleswithinsociety(seeIQSSExampleC).Inanumberofexamples,facilitatorsstatethatgirlsneedmoreassistancebecausetheyhave‘anaturalweakness’.

IQSSExampleC:Girls’educationandgenderroles


©EDOREN 150

4.4.1.3 Whatfacilitatorsdointheirlessons

Thepracticeoffacilitatorsintheclassroomwascategorisedonthebasisoftheclassroomobservationsintothreecategories:teachertalk,teacheractionandpupilaction.Thebaselinefindingsonteachertalkindicatethatteachersaremorelikelytouserote-basedapproachesthanpupil-centredapproaches,althoughthereissomeuseofthelatter.Ofthedifferenttypesofteachertalkthatwereobserved,threeareconsideredpupil-centred:askingorrespondingtoanopenquestion,assistingingroupwork,andusingachild’snameinclass.AsseeninFigure49,overhalfoffacilitatorsusedachild’snameandalmost70%askedorrespondedtoanopenquestion.However,lessthan10%assistedingroupdiscussions.Inabout10%ofclassroomsfacilitatorswerenotpresentatsomepointduringclass.Thisdoesnotincludelessonsthatwereendedearly.

Figure49: Percentageofobservedfacilitatorsengagingindifferentteachertalkactivities

AsnotedinSection3,pupilactioninclassroomsisalsoausefulindicatorofthepedogogicalapproachbeingundertakenbytheteacher.Ofparticularinterestisthelevelofnon-roteactivity,whichisindicativeofthedegreeofpupil-centredteachingthatistakingplace.Thefollowingtypesofpupilactionfallintothiscategory:groupdiscussion,grouporworkinpairs,respondingtoanopen

0

10

20

30

40

50

60

70

80

90

100

Instructs Remindstheclasstopayattention

Assistsgroupdiscussion

Usesachild'sname

Asksorrespondstoanopenquestion

Asksorrespondstoaclosedquestion

Teacherisnotpresent


©EDOREN 151

question,askingtheteacheraquestion,usingatextbook,readingaloud,anddoingindividualwork.Thisisincontrasttochanting,listening,andrespondingtoclosedquestions.Figure50highlightsthatwhilefacilitatorswereengaginginsomenon-roteactivities,theuseofmostoftheseapproacheswaslimited.Inparticular,groupdiscussionandinstancesofchildrenaskingtheteacheraquestionwereobservedinlessthan10%ofclassrooms.Groupworkwasalsoveryuncommon.Individualworkandreadingaloudweremorecommonandinstancesofpupilsrespondingtoopenquestionswereobservedinamajorityofclassrooms.

Figure50: Percentageofobservedclassroomswherepupilsengagedinvarioustypesofpupilaction

Patternsoflanguageusevariedacrossthetwostates.InBauchi,themajorityoffacilitators(72%)wereobservedusingjustonelanguageduringthelessonobservation.Afurther24%wereobservedusingtwolanguages(seeFigure51).InNiger,25%offacilitatorsusedjustonelanguageduringthelesson,while60%usedtwolanguages.Asmallminorityoffacilitatorsinbothstatesusedthreeormorelanguagesduringalesson.Teachersinurbanschoolswereslightlymorelikelytousejustonelanguageduringthelesson(54%)thanthoseinruralschools(46%).ThemajorityoffacilitatorsinbothstateswereobservedusingHausaatleastonceduringthelesson(Figure52).InBauchi,allfacilitatorsusedHausaatleastonce,and24%usedEnglish.AsmallminorityusedArabic(2%)oranotherlanguage(6%).InNiger,72%usedHausa,59%usedEnglish,and48%usedanotherlanguage.Theclassroomobservationtooldidnotrecordwhattheseotherlanguageswere,althoughitislikelythatNupefeaturesheavilyheregiventhatitisspokenbyanotableshareoffacilitatorsandpupils.

0102030405060708090

100


©EDOREN 152

Figure51: Numberoflanguagesusedbythefacilitatorduringonelesson

Figure52: PercentageoffacilitatorsusingHausa,Arabic,Englishandotherlanguagesinclass

4.4.1.4 Facilitators’knowledge,skillsandpracticesaspertheToC

AsdiscussedinSection3.2.9,teachersdrawonthreetypesofknowledgewithinclassroompractice.Thebaselinedatahavebeenusedtocreatecompositeindicesforeachoftheseknowledgetypes.

• Subjectknowledgereferstoknowingtheessentialquestionsofthesubject,thenetworksofconcepts,theoreticalframeworkandmethodsofinquiry.Thisismeasuredthroughacompositeindexofteachers’writingskills,Hausaskills,comprehensionskillsandtheirabilitytointerpretwordsandphrases.

0%

10%

20%

30%

40%

50%

60%

70%

80%

Teacherusedonelanguageduringlesson

Teacherusedtwolanguagesduringlesson

Teacherusedthreelanguagesduringlesson

Teacherusedfourormorelanguagesduringlesson

Bauchi Niger

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Hausa Arabic English Other

Bauchi Niger


©EDOREN 153

• Pedagogicalknowledgereferstoknowledgeoflearnersinthesetting,knowledgeofhowtoprovidetheconditionsthatenablepupilstounderstandandtheselectionoflearningandassessmentmaterials.Thisismeasuredusingacompositeindexthatdrawsonteachers’abilitytoidentifylowperformersandevidencejudgementsanddiagnoselearninggaps.

• Curriculumknowledgereferstoknowingwhatshouldbetaughttoagroupofstudents,knowledgeofthenationalsyllabus,understandingoftheschool-andgrade-levelplanningdocumentsandknowledgeofthecontentofexaminations.Thisismeasuredthroughthesyllabusknowledgesubscaledevelopedfromtheteacherknowledgeandskillsassessment.

Theabsolutevaluesofthecompositeindices’scoresarenotinherentlyinformative,butratherprovidebaselinevaluesforcomparisonatmidlineandendline.However,differencesinscoresbetweengroupsareofinterestatbaselinetoinformtheprogramme.Theshareoffacilitatorswhoscoredzeroontheseindicesisalsoworthnoting.

Withregardstosubjectknowledge,justoverone-fourthoffacilitatorsscoredzeroonthecompositeindex.FacilitatorsinBauchihadslightlyhigherlevelsofsubjectknowledgethanthoseinNiger.Thosefacilitatorstrainedonintegrationactivitiesalsohadslightlyhighersubjectknowledge,asdidthosewithaprofessionaleducationqualification.Whilenoclearpatternsemergedoverthefacilitatoragespectrum,facilitatorsovertheageof50hadslightlylowerscoresinsubjectknowledge.Interestingly,asfacilitators’perceivedself-efficacyincreased,theirsubjectknowledgedecreased.Thismayindicatethatasfacilitatorknowledgeincreases,facilitators’self-perceptionsbecomemorerealistic,oritmaysimplyindicatethatfacilitatorsarenotparticularlygoodjudgesoftheirownknowledgeandskills.Conversely,facilitators’subjectknowledgeispositivelycorrelatedwithfacilitators’interestin,andenjoymentof,teaching.

Facilitatorpedagogicalknowledgelevelswereextremelylowacrossallfacilitatorgroups.Overall,91%offacilitatorsscoredzeroonthecompositeindex.FacilitatorsinBauchihadslightlyhigherlevelsofpedagogicalknowledgethanthoseinNiger.Therewerenoclearpatternsasregardspedagogicalknowledgeandfacilitatorcharacteristics,mostlikelyduetotheverylowscoresontheindexacrossallgroups.Aswithsubjectknowledge,therelationshipwithperceivedself-efficacyindicatesthatfacilitatorsmaynotbeverygoodjudgesoftheirownknowledgeandskillsasmanyfacilitatorswhoreportedhigherlevelsofperceivedteacherefficacywereunabletodemonstrateanypedagogicalknowledgeatall(scoringzeroforthismeasure).Facilitatorswhoreportedhigherlevelsofteacher-to-teacherinteractiondisplayedslightlyhigherlevelsofpedagogicalknowledge.However,allobserveddifferencesweresmallduetothelowlevelsofpedagogicalknowledgeobservedacrossthegroups.

Justover40%offacilitatorsscoredzeroontheindexforcurriculumknowledge.Therewerenodifferencesinfacilitatorcurriculumknowledgebystate.Therewerealsofewdifferencesinknowledgebyfacilitatorcharacteristics.


©EDOREN 154

Figure53: Percentageoffacilitatorsscoringzeroinpedagogical,curriculumandsubjectknowledge

Referringtothethreetypesofknowledgethatteachersdrawonintheclassroom:thevastmajorityoffacilitatorswereunabletodemonstrateproficiencyinknowingthelearnersinthesetting;theywereunabletodemonstrateknowledgeofhowtoprovidetheconditionsthatenablepupilstounderstand;andtheywereunabletodemonstrateproficiencyintheselectionofappropriatelearningandteachingmaterials(pedagogicalknowledge).Alargeminorityoffacilitators(about41%)wereunabletodemonstrateanyproficiencyinknowingwhatshouldbetaughttoagroupofstudents(curriculumknowledge),andaboutaquarterofthefacilitatorswereunabletodemonstrateanyproficiencyintheessentialquestions,conceptsandmethodsofenquiryinliteracyandlanguage(subjectknowledge).

AcompositeindexwasalsodevelopedtomeasurechangesintheToCintermediaryoutcome:improvedteacherpracticesandgendersensitivity.Theindexisbasedonthedepthofpupil-centredlearningactivitiesobservedintheclassroom,observationsofthefacilitatorlinkingthelessontopreviouslearningandlearningobjectives,andtimeontaskinclass.

Themeasureofthedepthofpupil-centredlearningwasbasedonclassroomobservationsrelatedtofacilitatorandpupilbehaviour.Inparticular,itwasbasedonwhetherornotfacilitatorsassistedgroupdiscussion,usedachild’sname,askedorrespondedtoanopenquestion,movedamongpupils,usedavailablematerials,engagedingroupdiscussion,orfacilitatedgrouporworkinpairs;andonwhetherpupilsrespondedtoanopenquestion,askedthefacilitatoraquestion,usedatextbook,readaloud,orcarriedoutindividualwork.Inlinewiththedataontheseconstituentvariables(seeFigure48andFigure49),facilitators’performanceonthisindexwasrelativelypoor,withmostfacilitatorsscoringinthebottomhalfoftherangeofpossiblevalues(seeFigure54).

0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0

Facilitatorsscoringzero(%)

Facilitatorsscoringzero(%)Curriculumknowledge 41.4Pedagogicalknowledge 90.6Subjectknowledge 26.1


©EDOREN 155

Figure54: Depthofpupil-centredteaching,bystate

Facilitators’scoresonthesecondcomponentoftheindex(linkingthelessontopreviouslearningandlearningobjectives)incorporatedobservationsofthefacilitatortalkingaboutthepreviouslesson,outliningtheobjectivesoftheobservedlesson,revisitingtheobjectivesofthelessonattheendofthesessionandsummarisingthelesson.Roughly20%offacilitatorsdidnotcarryoutanyoftheseactivitiesatthestartorendofthelesson,while8%carriedoutallfour.FacilitatorsfromBauchiperformedbetteronthiscomponentoftheindexthanthosefromNiger.

Timeontaskwasmeasuredbythepercentageoftimethatpupilswereengagedduringlessonobservations.Thiswasclassifiedasthetotaltimeduringwhichpupilswereperforminganyactionrelatedtolearning(forthedifferenttypesofactionsthatwererecordedseeFigure50).Theclassroomobservationtoolrecordedpupilactionsatthree-minuteintervalsduringthelesson.Thesedatawereusedtoestimatetimeontask.Withinthemajorityoflessonsobserved(64.5%),childrenspent100%ofthelessononactivitiesrelatedtolearning(seeFigure55).However,thesefindingsshouldbeinterpretedcarefullyasitislikelythatthepresenceofobserversintheclassroomincreasedthepercentageofon-tasktimeinlessons.Theaveragedurationofthelessonsobservedwas24minutes,althoughthisvariedacrossthetwostates(seeFigure56).

0%

10%

20%

30%

40%

50%

60%

0 0-5 5-10 10-15 15-20 above20

Bauchi Niger


©EDOREN 156

Figure55: Percentageoftimeontaskinobservedlessons

Figure56: Lengthoflessonsobserved,bystate

ThequantitativecomponentoftheIQSSevaluationalsosoughttoassesstheprevalenceofgender-sensitivepractices.Beyondmeasuringtheproportionofgirlsandboyswithintheclassroomsetting,themeasurementofgendersensitivityisextremelycomplexandoftencannotmeetreliabilitycriteria.Withinthefacilitatorquestionnaireseveralitemsweredeveloptomeasureattitudestowardsgirls.Acrossallitemsextremecomplianceeffectswereobserved,significantlycallingintoquestionthevalidityandreliabilityofthemeasure.Therefore,changesinthegendersensitivityoffacilitatorclassroompracticesarebestdetectedthroughthequalitativeresearch.

Overall,scoresonthecompositeindexofeffectiveteacherpracticewerelowacrossallgroupsoffacilitators.Facilitatorswhometregularly(onceaweekormore)withtheheadteacherdemonstratedlowerlevelsofeffectiveteacherpracticesthanthosewhodidnot.Facilitatorsin

.0

20.0

40.0

60.0

80.0

100.0

Percentofclassrooms

Percen

toftim

eon

task

Lessthan50%

50-59.9

60-69.9

70-79.9

80-89.9

90-99

100percent

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Lessthan12minutes 12-20minutes 21-29minutes 30-36minutes

Bauchi Niger


©EDOREN 157

BauchidemonstratedslightlyhigherlevelsofeffectiveteacherpracticethanthoseinNiger.Therewerenoclearpatternsbyfacilitatorcharacteristics,exceptthatfacilitatorswhowereover50yearsoldperformedworsethantheiryoungercolleagues.

4.4.1.5 Facilitatormotivationandattendance

Facilitatormotivationwasinvestigatedthroughtheuseofarangeofmotivationscaledscores,whichformacompositemotivationindex.Thesubscalescoverthefollowingareas:

• perceivedteacherefficacy;• interestin,andenjoymentof,teaching;• effortputinto,andperceivedimportanceof,teaching;• pressureandwork-relatedtension;and• teacher-to-teacherinteraction.

Acomparisonofthedifferentmotivationsubscalesrevealsthat,onaverage,thesurveyedfacilitatorsscorehighestoneffortandlowestonperceivedteacherefficacy(Figure57).Motivationsubscalesforinterestin,andenjoymentof,teachingandeffortputinto,andperceivedimportanceof,teachingarequitehigh,whereassubscalesmeasuringpressureandwork-relatedtensionandperceivedefficacyappearquitelow.Thisseemstosuggestthatfacilitatorsconsidertheirroletobeimportantandthattheyenjoyworkingasteachers.However,thereseemstobeanindicationthatthefacilitatorsfeeltheirabilitytoimpactpupiloutcomesissomewhatlimited.Itisalsointerestingtonotethattheteacher-to-teacherinteractionscoreseemstoindicatearelativelyhighlevelofcollaborationamongstfacilitators.Thiscouldhelpstrengthentheeffectsoffacilitator-specificinterventionsattheschoollevel.Forexample,iffacilitatorssharetheirlearningfromtrainingwithoneanother,theeffectoffacilitatortrainingwillbemultipliedatthatparticularschool.However,itisworthnotingthattheteacher-to-teacherinteractionheremayimplyinteractionwithteachersofreligioussubjectsaswell,aswehavealreadyestablishedthat25%ofIQSsonlyhaveonefacilitator.

Thegraphbelowpresentsavisualanalysisofthedifferencesacrossmotivationsubscales.

Figure57: MotivationsubscalesforIQSfacilitators

1.9

3.64

3.8

1.98

3.5

0 0.5 1 1.5 2 2.5 3 3.5 4

Perceivedteacherefficacy

Interestandenjoymentofteaching

Effortandimportance

Pressureandwork-relatedtension

Teacher-to-teacherinteraction


©EDOREN 158

Theoverallteachermotivationindexwasconstructedasacompositemeasureofallteachermotivationsubscales,withequalweightingforallsubscales.Whilsttheaveragemotivationindexvalueacrossthewholefacilitatorsampleisnotparticularlyinformative,itisanalyticallyinterestingtolookatthedifferenceinmotivationscoresacrosscategoriesofinterestforouranalysis.Variablessuchasageandprofessionalqualificationarenotassociatedwithdifferencesinmotivationindexscores.Ruralfacilitatorsareonlyslightlymoremotivatedthantheirurbancounterparts.Animportantcorrelationisthatfacilitatorsthatmeetwithheadteachersindividuallyaremoremotivatedthanthosethatdonot,andthatgroupmeetingswithheadteachersdonotseemtobeanimportantdeterminingfactorofmotivation.

Facilitatorsthatreceiveasalaryorstipendfortheirservicesandfacilitatorsthathaveattendedatrainingsessioninthelasttwoyearsappeartobemoremotivated,asindicatedbyahighervalueforthecompositemotivateindex.Thisisnotentirelysurprising.Itis,however,interestingtonotethatfacilitatorsinBauchiweremoremotivatedthanfacilitatorsinNiger.ThisisnoteworthysincemorefacilitatorsinNigerarepaidthanarepaidinBauchi,andfacilitatorsinNigeraremorelikelytobetrained.Thissuggeststhatthereareotherconfoundingvariablesassociatedwithmotivationbeyondremunerationandtraining.

Facilitatorremuneration

FacilitatorsinNigeraremorelikelytobepaid.Inaddition,facilitatorsinschoolswheretheheadteacherandproprietorareseparatearemorelikelytobepaidteachers.Thisisinlinewiththeideathatinplaceswherethesetworolesareseparate,schoolsareoperatingmoreformally.Thoughremunerationisaninterestingavenuetoexplore,practically,averysmallshareofthefacilitatorsinthesamplewerepaideitherasalaryorastipend:only3%ofthesampleinBauchiand33%ofthesampleinNiger.Hencetheclaimsthatcanbemadebasedonthesedataarequitelimited.

Facilitatorattendanceandtransfers

Around75%ofthefacilitatorswereabsentatleastonceduringthelastthreemonths,withanaveragedurationofabsenteeismofapproximatelysevendays.FacilitatorsinBauchiwerefarmorelikelytobeabsent,andforlonger,thanfacilitatorsinNiger.Absenteeismrateswerelowestforteachersintheyoungestagebracketandhighestamongthe25–34agebracket.

Thegreatmajorityoffacilitatorsidentifiedtransportissues,theirownillnessortheillnessofafamilymemberasthemainreasonfortheabsence.Socialorreligiousobligationssuchasweddingsandfuneralsalsoemergedasdeterminingfactors.Theheadteacherdatarevealalmostidenticalreportedreasonsforteacherabsenteeism.Theaveragenumberofdaysheadteacherswereabsentinthesameperiodofthreemonthsisrelativelylow,ataboutsixdays.

Therewerenomarkeddifferencesintheabsenteeismratesforfacilitatorsbypaymentstatus.However,facilitatorsthatwereunpaidwereabsentforlongerthanthosethatwerepaid.Facilitatorsthatreceivednotrainingweremorelikelytobeabsent,andtobeabsentforlonger,thantrainedfacilitators.Aswehavediscussedabove,trainedfacilitatorswerealsomoremotivated.

Aroundaquarterofthesampledheadteacherswhodidnottakeanyactiontoimproveteacherattendancereportedthatteacherabsenteeismwasnotaproblemintheirschool.Whenexploredfurther,thedatarevealthatmostoftheseresponsescomefromtheBauchisample,where37%oftheheadteacherssaidthatteacherattendancewasnotaproblemintheirschools.Incontrast,only10%oftheheadteachersinNigermadesimilarclaims.ThisfindingiscontradictorytothefindingthatfacilitatorsinBauchiaremorelikelytobeabsent,andareabsentformoredays,thanteachers


©EDOREN 159

inNiger.ThismaybeexplainedbytheIQSsinBauchibeingmoretraditionalandmoreinformallystructured,andthereforefacilitatorattendancebeingmoreflexiblyorganisedandflexibleattendancebeingmoreaccepted.ThissuggeststhatadditionalschoolmanagementandleadershipsupportmaybeneededforIQSstransitioningfrominformal,flexiblemodelsofoperatingtoaschoolstructurewithmoreformalmonitoringandmanagement.

Record-keepingintheIQSSschoolswasquitelimitedandaround40%ofthesamplehadnoformofwrittenrecords.Thisaffectedseveralvariablesofanalysis,includingfacilitatorturnover.Awrittenteacherledgerorattendancerecordwasinexistenceinonly12%ofthesampledschools.Hencethefindingsbasedonsuchdataneedtoapproachedcriticallysincethereareincentivestomisreportthisinformationattheschoollevel.Throughthelimitedrecalldataavailable,facilitatorturnoverisestimatedtobearound20%asashareoftotalfacilitatorsinschool.Ratesofturnoverwerehigherinruralareasandininstanceswheretheheadteacherwastheproprietor.

4.4.1.6 Availabilityofteachingandlearningmaterials

Theavailabilityofteachingandlearningmaterialswaslimited–77%oftheclassroomsobservedhadnosuchresourcesavailable.Themostcommonlyavailableresourceswerethosemadebyhandbyfacilitators,followedbytextbooks(see


©EDOREN 160

Figure58).In78%ofobservedlessons,allavailableresourceswereusedduringthelesson.Thefactthatthisdidnotapplytoallofthelessonsobservedmaysimplysuggestthatsomeoftheresourcesavailablewerenotappropriateforthatparticularlesson.However,anotherpotentialinterpretationisthatfacilitatorsarenotalwaysawareofhowtomakethebestpossibleuseofavailableresources,andmayneedtoreceivesometraininginthisarea.Overall,thefindingssuggestthatthedistributionofteachingandlearningresourcescouldfillanimportantgap,providedthatthesearecarefullytailoredtoprevailinglevelsofknowledgeandskillsamongstfacilitatorsandpupils.


©EDOREN 161

Figure58: Availabilityanduseofresourcesinobservedlessons

4.4.1.7 Pedagogicalleadership

Attheschoollevel,informationwascollectedonheadteacherleadershipandmanagementthroughtheheadteachersurveyinstrument.Oneofthequestionsofinterestwastounderstandwhoattheschoollevelfulfilstheheadteacherrole:i.e.whoisresponsibleforday-to-daymanagementandsupervisionofteachers.Ourpre-pilotingexperiencesuggestedthatthispersonisnotalwaysreferredtoasthe‘headteacher’.ThedataconfirmtheideathatalthoughinmostIQSstheheadteacheriscalledassuch,insomecasestheheadteacherandproprietorarethesameperson,inwhichcasethepersonfulfillingtheheadteacherrolemaybereferredtoastheproprietor.Inone-thirdoftheIQSSsample,theheadteacherwasalsotheproprietor,andthiswasmorecommonforschoolsinBauchi,asalreadydiscussed.

Headteacherswerealmostalwaysmaleandweregenerallyolderthanteachers,withanaverageageof40years.However,thereareimportantdifferencesbetweentheaverageheadteachersinBauchiversusthoseinNiger.TheheadteachersintheBauchisampleareabout10yearsolder,onaverage,thanheadteachersintheNigersample.AroundhalfoftheNigersampleofheadteachersfallwithintheagebracketof25–34years,whiletheBauchisamplewasconcentratedinthe35–49yearsagebracket.HeadteachersinBauchihad10moreyearsofworkexperienceatthecurrentschoolcomparedtoheadteachersinNiger(ameanoffiveyears,versus15),inadditiontohavingmoreyearsofteachingexperience.ThoughheadteachersacrossbothstateshadsimilarlikelihoodsofhavinganNCE,headteachersinNigerhadhigheracademicqualifications,asindicatedbymorethanhalfofthemhavinganSSCEorhigher,comparedtoonly25%oftheBauchiheadteachers.Inaddition,agreatershareofBauchiheadteachershadareligiouseducation(over55%,comparedtolessthan20%inNiger).ItisnoteworthythatasmallershareofheadteachershadanNCE(23%)comparedtofacilitators(35%).ThisdifferenceisparticularlylargeinBauchi,where51%offacilitatorshadanNCE,comparedto27%ofheadteachers.Thismayaffecthowfacilitatorsperceivethecapabilityofheadteachersinregardtotakingonapedagogicalleadershiprole.

Incaseswheretheheadteacherdoubledastheproprietorthemeanagewashigher.Infact,45%oftheheadteachersthatwerealsoproprietorsoftheirschoolwereovertheageof50,whilethiswas

0.0

5.0

10.0

15.0

20.0

25.0

Video Audio

Toolsorobjectsfromthelocal

environment

Handmade MaterialsinHausa

Poster,chartorpictures Textbook

Resourcesused 0.0 0 3.8 19.9 1.0 1.9 8.8

Presentbutnotused 0.0 0 4.8 4.7 .9 1.9 1.9

Percen

tofo

bservedlesson

s


©EDOREN 162

thecaseforonly15%oftheheadteachersthatwerenotproprietors.Suchheadteacherswerealsomoreexperiencedasheadteachersandteachers,andtheyhadloweracademicqualifications;asmallerproportionhadanNCE.Theschoolsoftheseheadteacherswerealsoestablishedearlier,buthadfeweryearsofintegration.Thesetrendsparallelthetrendsfoundacrossthetwostates–whichisunsurprisingsinceBauchihadmoreheadteachersthatwerealsoproprietorsthanNiger.ThisisinlinewithotherevidencediscussedsofarwhichsuggeststhatBauchischoolsareclosertothetraditionalQur’anicschoolstructure,andlessintegrated,thanschoolsinNiger.

AlargershareofheadteachersinNiger(50%)hadreceivedsomesortoftraininginthelasttwoyearsthanheadteachersinBauchi(37%).TheheadteachersinBauchithatweretrained,however,weretrainedformoredays.Thecontentofthistraining,asreportedbytheheadteachers,alsovariedgreatly.ItappearsthattheheadteachersinBauchiwerereceivingmoretrainingonintegration,schoolmanagementandleadership,andgeneralteachingpracticesthanheadteachersinNiger.ThoughbothstatesreportedtrainingbyGEPandUNICEF,alargershareofheadteachersinBauchireportedtrainingconductedbySUBEBsandtheLGEAs,whichsuggestsgreaterinteractionwithgovernmentbodies.ThisisalsosupportedbythefindingthatagreatershareofschoolsinBauchireportedmonitoringvisitsfromagovernmentbodyduringthelastschoolyear(44%,versus33%).

Figure59: Contentofheadteachertraining,bystate

Headteachersthathadsomesortofprofessionaleducationalqualificationweremorelikelytobetrainedthanthosewithout,andthisdifferenceislargerthantheBauchi/Nigerdifference.SimilartrendsappearwhendisaggregatingthesamplebetweenheadteachersthathaveanSSCEandthosethatdonot.ThisdifferenceispartlydrivenbythefactthatmorequalifiedheadteachersarefoundintheNigersample.However,thiscouldalsoindicatethatinthepast,betterqualifiedheadteachersweremorelikelytohavebeenselectedfortraining.

Thesurveyinstrumentsmeasuredpedagogicalleadershipthrougharangeofvariables,suchasheadteacherlessonobservations,headteachermeetingswithteachersingroupsorindividually,andactionstakentoimproveteacherandpupilattendance.Throughoutthesepedagogicalleadershipvariables,thereisastrongassociationbetweenheadteachersbeingtrainedandimprovedleadershipandmanagementpractices.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Communityinvolvement

Integrationactivities

Developinginstructionalmaterials

Managementandplanning

Schoolleadership

Extracurriculars Curriculumsubjects

TeachinginHausa

Literacyandnumeracy

Teachingmethods

Bauchi Niger


©EDOREN 163

In40%ofthecases,theheadteacherobservedlessonsduringthelastterm.Thecross-statedifferencesuggeststhatagreatershareofheadteachersinNigerareobservingteacherlessons.Thisisvalidatedbythedatafromtheteacherquestionnaire:over80%oftheteachersinNigerreportedtheheadteacherobservingtheirlesson,whilethiswasonly65%forBauchi.However,noneoftheheadteachersinNigerkeptanywrittenrecordsoftheirobservations.Therearenostrongassociationsbetweenlessonobservationsandtheseparationoftheheadteacherfromtheproprietor,orbyagecategories.Trainedheadteachers,however,weremorelikelytocarryoutlessonobservations,andweremorelikelytohaveobservationrecords.

Inaddition,63%oftheheadteachersreportedtakingactiontoimproveteacherattendance,whilethiswas93%forpupilattendance.Therewerenonoticeabledifferencesacrossstatesorownershipstatusofschools(i.e.whethertheschoolisownedbytheheadteacherornot).Aroundaquarteroftheheadteachersreportedthattheydidnotneedtotakeanyactionsinceteacherabsencewasnotaproblemintheirschool.ThiswasgreaterforheadteachersfromBauchi,wherejustover35%oftheheadteachersreportedthis,comparedtoonly10%oftheheadteachersinNiger.Agreatershareoftrainedheadteachersreportedtakingactiontoimproveattendanceofteachersandpupilsintheirschools.Actionstakentoimproveteacherattendanceincludeddiscussingreasonsforabsencewithteachersandencouragingteacherstoattendschool.

Anotherindicatorofschoolleadershipmeasuredbythesurveywashowoftenheadteachersmetwithteachers,individuallyorinagroup,todiscussteachingandperformance.Thisincludedbothformalandinformalsessionsheldduringthelastcompleteterm(ApriluntilJuly,atthetimeofthesurvey).Onlyabout13%ofheadteachersdidnotmeetwithteachersatallduringthelastterm.ThiswasalotmorecommoninschoolsinBauchithaninNiger.Inaddition,individualmeetingswerelesscommoncomparedtogroupmeetings,acrossbothstates.Thechartbelowsuggeststhattrainedheadteacherswerefarlesslikelytonevermeetwithteachers,andweremeetingmorefrequentlywithteachers,thanuntrainedheadteachers.Similartrendsappearforgroupmeetingswithteacherswhendisaggregatedbyheadteachertrainingstatus.

Figure60: Frequencyofheadteachermeetingswithteachers,bytrainingstatusofheadteacher

3%

16%

3%

41%

38%

21%

8%

8%

37%

25%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Meetsteachersonceaweekormore

Meetsteachersmorethanonceamonth

Meetsteachersonceamonth

Meetsteacherslessthanonceamonth

Nevermeetsteachers

Training Notraining


©EDOREN 164

HeadteachersinNigermetmorefrequentlywithteachersindividually(andingroups)comparedtotheheadteachersinBauchi,asshownbelow.Thoughinbothcasesaround30%oftheheadteachersreportingnevermeetingwithteachers,headteachersinNigermetmoreoften.

Figure61: Frequencyofheadteachermeetingswithteachers,bystate

Thecontentofmeetingswithteacherswaslargelyconcernedwithpupilabsenteeismandprovidingsupporttoindividualstudent’sneeds,inadditiontoteachingpracticesandpedagogy.TheonlynoticeabledifferenceacrossstatesisthatinNigerteachingpracticesandpedagogyarediscussedfarmorethaninBauchi,whileinBauchithefocusisontheneedsofindividualstudents.

Sofar,thedatasuggestthattrainedheadteachersarefulfillingrolesofschoolleadershipandmanagementbetter,andthatheadteachersinNigerareperformingbetterthanheadteachersinBauchiacrossvariouscategories.However,intermsofformalrecord-keeping,suchasmaintainingupdatedattendanceandpupilenrolmentregisters,andwrittenlessonobservationrecords,headteachersinBauchiareperformingbetter.Thisdifferencemaybeassociatedwiththevariedcontentofheadteachertrainingacrossthetwostates,orwithgreatergovernmentoversightinBauchi.ThoughBauchiisperformingslightlybetterinthisregard,therearestillgainstobemadeacrossallIQSssincetheoveralllevelofrecord-keepingisquitelow.ThisfurtherhighlightstheneedforheadteachersinIQSstobetrainedin,andsupportedon,schoolmanagementandmonitoringactivities.

3%

10%

7%

47%

33%

19%

15%

4%

30%

31%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Onceaweekormore

Morethanonceamonth

Onceamonth

Lessthanonceamonth

Never

Niger Bauchi


©EDOREN 165

Figure62: Availabilityofschoolrecords,bystate

4.4.2 Conclusionanddiscussion

ThediscussionsofarrevealsthatintegrationactiviteshavebeenimplementedinschoolsacrossBauchiandNigertovaryingdegrees.Moreoever,thelandscapeofIQSsiscomplex,andnouniformmodelseemstobepresent.Varyingschoolstructures,disparitiesinownership,aswellasdifferencesinthenumberandhoursofintegratedsubjectstaughtperschool,suggestthattheoutcomesofpupilsattendingtheseschoolsarelikelytodiffergreatlyfromoneanother.Onaverage,however,schoolsinNigerseemtomoreintegrated,withgreaterheadteacherpedagogicalleadershipandtraining,thanschoolsinBauchi.

Levelsoffacilitatorknowledgewereextremelylow,particularlyinregardtopedogogicalknowledge.Effectiveclassroompracticescoreswerealsolowforallfacilitatorgroups.Therewereveryfewresourcesavailabletobeusedbyfacilitators,withhand-maderesourcesbeingusedmorethananyotherresourcetype.Theanalysisalsofindsthatwhereresourcesareavailable,theyaregenerallyused.

Evidencesuggeststhatincreasingtheknowledgeandskillsoftheteacherwithintheclassroomandexposingchildrentoreadingmaterialsearlyinlifearetwoofthestrongestpolicy-amenablepredictorsoflearningachievement.Sowhilethoselowbaselinelevelsareconcerningandlimittheanalysistheyalsoprovideopportunitiesfortheintervention.However,expectationsshouldremainrealisticastherearealsosignificantbarrierstosubstantiallyraisingtheknowledgeandskillsoffacilitatorsatscale,asverylowlevelsofknowledgeandskillsareobservedatbaseline.

4.4.3 Qualitativecasestudyanalysis

ToassessContributionClaim1,‘GEP3’sIQSScontributestomoreeffectiveteachingoftheintegratedcurriculuminIQSs’,thequalitativestudyaimedtoexplorethequestion‘willfacilitatortrainingandmentoring,accesstofit-for-purposeteachingandlearningmaterials,andimprovedheadteacherpedagogicalleadershipandsupportleadtomoreeffective,gender-sensitiveteachingoftheintegratedcurriculum?”

0%5%

10%15%20%25%30%35%40%45%50%

AttendancerecordexistsforP2forcurrent

year

Updatedattendance

recordexistsforP2forcurrent

year

EnrolmentrecordexistsforP2forcurrent

year

Writtenrecordsofmeetings

kept

Teacherattendancerecordexists

TimetableprovidedforP2

Nowrittenrecordsprovided

Bauchi Niger


©EDOREN 166

Thesectionsbelowdiscussthequalitativefindingsatbaselinerelatingtothecausalpackagethatcontributestoeffectiveteachingofformalsubjects.TheanalysisdrawsondatafromthreeschoolsinBauchiState(lowerperforming,typical,higherperforming)andthreeschoolsinNigerstate(lowerperforming,typical,higherperforming).

4.4.3.1 Perceivedbenefitsandchallengesoftheintegratedcurriculum

Akeyassumptionbehindcontributionclaim1-thatfacilitatortrainingandmentoring,accesstofit-for-purposeteachingandlearningmaterials,andimprovedheadteacherpedagogicalleadershipandsupportwillleadtomoreeffectiveteachingofformalsubjects-isthatIQSstakeholders(proprietors,teachers,parents,pupilsandothercommunitymembers)willbesupportiveoftheintroductionoftheintegratedcurriculumintheQur’anicschool.Thiswilldependonthebenefitsandchallengesthatthestakeholdersperceiveinregardtointegration.Ifstakeholdersdonotacceptintegration,thiscouldposeabarrierinregardtotrainedfacilitatorsturningtheirimprovedknowledgeandskillsintoimprovedpractices.

Thesixschoolssampledforthequalitativestudyoperateindiversecontexts.Asaresult,theydisplaydifferentlevelsofawarenessof,andattitudestowards,formaleducation.ItisworthnotingthatschoolswillhaveacceptedintegrationandafocusupongirlsasaprerequisiteforanyGEP3support.Asaresult,GEP3IQSstendtodisplaypositiveattitudestowardsintegrationandgirls’education,eitherintrinsicallyorduetoresponsebias.

Theintegratedcurriculumappearstohavelargelygainedacceptanceinthecasestudycommunities.Whilststakeholdersacknowledgedtheinitialchallengesinwinningoverthecommunity,withsomeparentswithdrawingtheirchildrenfromtheschool,theintegratedQur’aniceducationnowseemstobeanintegralpartoftheschoolsinwhichitissituated.InBauchi—whereintegrationhasbeenmorerecent—itsembeddednesswithincommunitiesseemsmorefragile.IntheBauchitypicalIQS,afacilitatorreferredtosomemembersofthecommunitywhostilldonotsupportformaleducation,andmocktheIQSfacilitatorsasaresult.Overall,however,respondentshighlightedthegradualacceptanceofthevalueof‘Westerneducation’,describingthisasdueto‘enlightenment’and‘havingunderstoodtheworldinwhichwelive’.

‘Previously,manypeoplewerelookingatitasaformofdiversion[fromtheirtraditionalways],butgraduallyparentsthemselvesbecameawareofthesystemandstartedsendingintheirchildrentotheschool,particularlywhentheystartedrealisingthatpracticallychildrenaregettingeducatedbothways–theQur’anicandwestern.”(Mallam/headteacher,NigerhigherperformingIQS)

Thisshowsarecognitionthatbothreligiousandnon-religiouseducationareimportant,andthatneithertypeofeducationissufficientinisolation.Respondentsdescribedthetwotypesofeducationascomplementingeachother,enablingchildrentosucceedinallaspectsoflife:‘traditional’and‘now’,aswellasintheafterlife.AstheMallamfromtheBauchitypicalIQSexplained:‘Thereisonechildren’sIslamicbookcalled“NuruDalala”.ItteachesthatamanisrequiredtohaveIslamicandformaleducation.Withoutthetwo,itwillbelikeabirdflyingononewing,whichisimpossible.Soweneedtheintegrationofbothsothateverythinginlifecanbeachieved.’

Twoseparatecasestudyschoolsandvariousrespondentsusedtheexampleofthebirdwithonewingtoexemplifytheperceivedvalueofintegration.Somerespondentsdescribedthevalueof‘Western’educationasopeningupopportunitiesforemploymentandexperience.Inisolation,however,itwassuggestedthatthiseducationcanchangeachildsothattheybecometoomuchlike


©EDOREN 167

the‘Englishman’andlosetheirculturalrootsandvalues.Theseresponsespointtoatendencytoviewformaleducationas‘Western’education,whichresultsinongoinghesitationamongsomecommunitymemberstowardstheintroductionofformaleducation.Languageplaysanimportantrolehere:‘Boko’isusedinterchangeablytomeanWesterneducationandalsotorefertoformaleducationwhenneeded.134

Communitiesperceivethattheacceptanceofformaleducationhasincreased.Thisisowingtoacombinationof‘outsiderscomingandexplaining’(intheformofsensitisationcampaigns)andpeopleseeingthepositiveoutcomesofreceivingformaleducation.Seeingthatpeoplehaveendedupinhigher-statuspositionswhilstremaining‘devoutMuslims’isarguablyimportant.ThemainreasongivenfortheimprovementinacceptanceistheawarenessandunderstandingthatformaleducationcancomplementIslamicteachingswithoutde-valuingmoralandtraditionalaspects.Additionally,theperceptionisthattheneedformoreformalskills—suchascounting(forthemarketandotherincome-generatingactivities)andreadingletters(forprivacy,andthereforenotrequiringthehelpofneighbours)—isincreasing.

‘Iamverymuchcomfortablewiththeintegrationprocessinvolvingbothboysandgirlsbecauseteachingtheformalsubjectsisveryimportant.Forinstance,theexampleIcitedaboutbanking.Ifyouarenoteducatedintheformalsubjects,youhavetocallanotherpersontowriteforyouwhowillknowallyoursecretsandprivacybutifyouareeducated,youwilldoalltheseforyourself.Ifyouareaskedtofillinaparticularform,anditisnoticedthatyoudonotknowhowtofillit,youbecomeembarrassed.Ifyouareeducated,[thereare]nosuchchallengesonyourpart.”(Mallam/headteacher,NigerlowerperformingIQS)

Respondentsdescribededucatingachildinformalsubjectsasbenefittingbothafamilyandacommunityasawhole.Educatingachildensuresthatparentsandothersareabletoreceivehelpfromtheirchildreninreadinglettersandinothertasksrequiringcertainlevelsofformaleducation.Moreover,childrencantakeupemploymentinsectorssuchasteachingorinhospitals,thushelpingthecommunitytodevelop.

Keystakeholdersinallsixcasesstressedtheimportanceofgradualintegration.ItisessentialtobeconsiderateofthefearsthatparentshaveoftheIQSsresemblingpublicprimaryschoolstoocloselytherefore,negativelyaffectingreligiousteaching.Therearestillchallengesforintegration135andthoughKIs,includingparents,refertothebenefitsofthecombinationofthetwo,themajorityofstudyIQSsstillplacegreateremphasisonreligiouseducation.

‘Iftheschoolisrelocated,theparentswillcomplainsincetheyareyettounderstandtheintegratedsystem.Theywillfeeltheschoolhasbeenconvertedtoaformalschoolinsteadofthereligiousschoolthatitusedtobe;andthatwillleadtowithdrawalofchildren.Nowweareintegratingtheformaleducationgraduallytoavoidthat.’(CBMC,Bauchitypicalperformingschool)

However,casestudycommunitiesdonotseeintegratingtheformalcurriculumintoQur’anicschoolsastheonlywaytocombinereligiousandformaleducation.Inseveralcases,childrenareattendingbothprimaryschool(inthemornings)andtheIQS(intheafternoon):‘likenowthe

134‘Boko’isaHausawordmeaningboth‘education’and‘secular’.ItisalsousedtodescribetheRomanscript,andcanalsobetranslatedinto‘trick’or‘manoeuver’(HippocrenePracticalDictionary:Hausa-EnglishEnglish-Hausa).Itispossiblethat,unlessanothertermcanbeusedtopopularlydescribeformaleducation(withanarrativeofmodern,northernNigerianeducationatitsheart),theongoingtensionarounda‘loss’ofcultureandreligionwillremainachallenge.135Exploredthroughoutthequalitativeanalysis.


©EDOREN 168

majorityofpupilsherestillattendtheconventionalprimaryschoolinthecommunity,theygototheprimaryschoolandstillcometotheIslamiyya’(CBMC,BauchihigherperformingIQS).136ThisseemslessthecaseinIQSsthatareremotelylocated.InthecaseofoneofthemoreremoteruralIQSsinBauchi,communitieshadnoaccesstoaformalprimaryschoolatallandpupilsinterviewedindicatedthatfewattendedpublicprimaryschool.137ItwillbeimportantatmidlinetoexplorefurthertheperceivedbenefitsofintegratedQur’aniceducationinacontextinwhichpublicprimaryschoolingisreadilyavailable.

4.4.3.2 Facilitatorandheadteacherknowledge,skillsandteachingpractice

TheGEP3ToCarguesthattrainingandmentoringoffacilitatorswillcontributetotheirimprovedsubjectknowledgeandpedagogicalskills.Anunderlyingassumptionisthatthecapacitydevelopmentprocesstargetstheneedsofthefacilitatorsandheadteachers.Therefore,inthissection,weprovideaqualitativebaselineassessmentofcurrentknowledge,skillandpractice,whichinfluencetrainingneeds.

Facilitators’andheadteachers’backgroundandqualifications

TheleveloffacilitatorqualificationsvariessignificantlyacrossthesixIQScasestudies.InthehigherperformingIQSsinbothstates,facilitatorshavetheNCE;whereasinthelowerperformingIQSsfacilitatorshaveonlyjust,orhavenearly,completedsecondaryschool.InboththehigherperformingIQScases,thefacilitatorsarealsoteachinginpublicprimaryschools.Inallcases,facilitatorsarefromthelocalcommunity.

Thequalificationandteachingexperienceofheadteachersdifferssignificantlyacrossschools,andtheextenttowhichheadteachershavesufficientsubjectandpedagogicalknowledgetotakeontheroleappearstobedependentonwhethertheMallamhaspriorqualificationsornot.ThelevelofeducationamongstheadteachersrangesfromnoformaleducationtoholdingtheNCE138.InoneBauchischool,theheadteacher,whowasalsotheIQSproprietor,wasauniversitygraduate.Overall,thereisalackofclaritywithinthemajorityofIQSsregardingtherolesandresponsibilitiesofeachactor.SeveralrespondentsinitiallystatedthattheIQShasthreeseparaterolesofproprietor,Mallamandheadteacher,allheldbydifferentpeople.However,furtherprobingshowedthatthereisoftenonepersonwhoplaysallthreeroles.Insomeschools,theproprietoroftheschool—whoisalsooftentheMallamoftheQur’anicschool—appearstotakeontheroleoftheheadteacherastheIslamiyya/Tsangayaintegrates.Wherethereisaseparateproprietoroftheschool,theheadteacheroftheIQStendstobetheMallamoftheQur’anicschool.Thesub-section‘Schoolmanagement–actors,rolesandresponsibilities’willfurtherexplorethechallengesaroundsystematicallyidentifyingappropriateheadteacherstotakeonpedagogicalleadership.

Facilitatorsandheadteacherswhohavehigherlevelsofqualificationsfeelmoreconfidentteachingtheintegratedcurriculum.ThiswasnotrestrictedtofacilitatorswiththeNCE,butfacilitatorswhohadalevelofformaleducationbeyondsecondaryschool(suchasnon-teachingdegrees).FacilitatorswiththeNCEalsotendtoteachinpublicprimaryschools.IncaseswherefacilitatorsdonothavetheNCE,thereseemstobeacleargapinteacherknowledge,intheareaofsubjectandcurriculumknowledge—withteachersbeingunawareofwhatisrequiredtobetaught.

136Seefurtherdiscussionaroundpublicprimaryschools/IQSsandgirls’enrolment/retentioninthediscussionaroundContributionClaim3.137ItisimportanttonotethatduetosecurityconsiderationsthequalitativeresearchteamonlyvisitedrelativelyaccessibleIQSs.Hence,thestudywillnotbeabletorevealthedynamicsinremoteareas.138TheNCEisanA-Levelgradecourseintendedforsecondaryschoolleaversandtheirequivalents.


©EDOREN 169

Whilstfacilitatorsreportthattheyhavehadsometrainingonmethodsforteaching,orthattheyusetextbookstoplanlessons,theycanonlyteachwhattheythemselvesknow.

Facilitators’subjectknowledgeinthetypicalandlowerperformingIQSsisweak.Facilitatorsareunabletoexplainwhichsubjectsitisimportanttoteach,orwhytheyteachwhattheyteach.Moreover,duringtheQCOsitwasclearthatfacilitatorsthemselves,attimes,didnothavebasicliteracyskills,beyondrecitingthealphabet.Manyfacilitatorswereteachingatthelevelthattheywerethemselvestaught.Inthosecaseswheretheirowneducationwaslimited,theywereonlyabletopassonbasicreadingandnumeracyskillstotheirstudents.Facilitatorswhoalsoteachinpublicprimaryschools(mostlyfoundinthetwohigherperformingcases)oftenhavetheirsubjectspecialisations,whichtheythenalsoteachintheIQS.Intheotherfourcases,facilitatorseitherhavenotreceivedfurthereducationorhavenon-teachingdegrees,suchasaccountingorplumbing.ManyTeacherFacilitatorsarethusteachingsubjectswithouteitheradvancedsubjectknowledge,oranypedagogicaltrainingonimpartingthatknowledgetostudents.

‘Well,themajorthingsoffocusthatweteachthemareissueslikespellingsofbasicthingswhichtheycaneasilylearntodoeveniftheydidn’tseethebasicthingspractically.Iftheyknowhowtospellthethingsandtheyseethempracticallylaterthatwouldgivethemanideaofthemeaningofthatthing.’(Niger,lowerperformingIQS)

However,communitymembersoftenperceivefacilitators’subjectknowledgetobehigh.AsGloveretal.(1990)discuss,perceptioncanbedependentonthereferenceframeofarespondent.Thatis,arespondent’sperceptionofthesubjectknowledgeofteacherswillbedependentontheirownlevelofknowledge.ParentsintheIQScommunitiesoftenlackanyformaleducationandthoughtheyperceiveteacherstoteachwelltheyexpresshavingdifficultiesfollowingattimes:‘theyareteachingthemverywell.Youknowsometimes,itisdifficulttounderstand’(ParentsFGD,BauchihigherperformingIQS).Evenincaseswherefacilitatorsthemselvesdonothavesubjectspecialisationbeyondsecondaryschool,respondentsbelieveteachershaveasubjectfocus.TheperceptionofknowledgeisthushighlyinfluencedbytherelativelackofformaleducationwithintheIQScommunities.

Pedagogicalskillsandteacherpractices

InmanyoftheIQSsvisited(withtheexceptionoftheNigerhigherperformingIQS),whilepedagogywasnotstrongamongthefacilitators,facilitatorsdisplayedsomeunderstandingoftheimportanceoflessonplanningandstructure,withattemptsatdeliveringlessonsinsomekindofsystematicmanner.Forthemostpart,facilitatorswhowereabletoexplainhowandwhytheypreparedorexecutedalessoninacertainwaydidsoonasurfacelevel.Whilesomefacilitatorsidentifiedpedagogyasimportantforeffectiveteaching,therewaslessunderstandingofwhatitentailed:‘ifyoushowanythingwithstudents,ordoanyteachingthatthechildrencan’tunderstand,youmustteachverywell.Sothatifyouareteaching,thestudentswillunderstandtheteaching.’(Headteacher,NigertypicalIQS)

Facilitators’abilitytomakeuseofeffectivemethodsisrelatedtothedepthofsubjectknowledgethattheyhave139.IntheQCO,facilitatorsdemonstratedsomeknowledgeofeffectiveteachingmethods,suchasstatingthelessonobjective,askingquestionsaboutandreviewingthepreviouslesson,andusingparticipatorymethods,suchasgroupwork.DuringtheTPDs,facilitatorsalsoreferredtotheimportanceofchild-centredteachingandofactivelyincludingchildreninthelesson.However,whilstfacilitatorsdisplayedawarenessofteachingandpedagogicalmethods,inmostcases139AdediwuraandTayo(2007).


©EDOREN 170

facilitatorsthemselvesdidnotseemtohavesufficientsubjectknowledge.Itseemsthatsomefacilitatorshaveknowledgeofpedagogicaltechniquesandmethodswithoutadeepenedknowledgeofsubjectmatter.Thus,theyoftenlacktheabilitytoteacheffectively.

‘Weweretaughthowtopreparelessonplansandhowtouseflashcards.Backthen,thewayweusedflashcardisquitedifferentfromhowweuseitnow.Backthen,weholdflashcardlikethis(forlonger)tothepupilsbutnowweweretaughtthatfromthemeaningofflash,wearesupposedtoallowthechildrentohavejustaglanceandwecoverthecard.Andweweretaughtthatmostoftheknowledgeofourlessonshouldcomefromthepupils,theyshouldsaymorewhileasateacherIammorelikeaguardian.’(TPD,BauchilowerperformingIQS)

Themostcommonlyusedteachingmethodsarerepetition,recitalandaskingtheclassquestions.Thoughvarioustechniquesareused,lessonstructuringseemsadhocandverybrief,withfacilitatorsmainlyemployingthemethodstheyhavebeentoldtouseorwhattheyhaveexperiencedorobservedinpublicprimaryschools.Itseemsthatteachersdonotalwayshaveadeeperunderstandingofthevalueofcertainmethods,andarenotsurewhentouseaparticularmethod.OneofthefacilitatorsintheBauchihigherperformingIQSexplainedwhyhethoughtrepetitionwasanimportantteachingmethod:‘Iwantthemtounderstandbetter.IfIdonotrepeatitseverally,theywillnotlearnandifIrepeatjustonceortwice,mostofthemwillforgetitbytomorrowbutnowithasbecomepartofthem,likeasong.Sometimesyoufindthechildrensinging“Monday”andsoon.’AnotherobservationfromtheQCOswasthefacilitators’inabilitytosetmeaningfulclassworkandtomovearoundtheclassroominordertocheckandmarkpupils’progress.InallthreeschoolsinBauchi—eventhehigherperformingone—thefacilitatorsreferredtopupilsneedingtowriteintheirbooks,andmadeoverturesregardingcorrectingclasswork.Nevertheless,inpracticethisbehaviourwasnominal:itwasoftendonerightattheendoftheclass,withnorealtimegiventoexplainingcorrections.Asnotallstudentshadnotebooks,thepracticewasalsoinequitable.

ItisclearthatthelevelofeffectiveteachingvariessignificantlyacrossIQSs.Amongstthesixcasestudyschools,theNigerhigherperformingIQSstoodout.ThemajorityoffacilitatorshadanNCEandalsotaughtatapublicprimaryschool.Facilitatorsdisplayedahigherlevelofsubjectmatterknowledge,andconfidenceinusingpedagogicaltechniquesandeffectiveteachingmethods.IntheotherIQSs,facilitatorswhohavereceivedanyformoftrainingdodisplayacertainlevelofknowledgeregardingteachingmethods,butthisdoesnotseemtobecomplementedwithsufficientsubjectknowledge,whichaffectstheeffectivenessofthesemethods.Ingeneralthough,inallschoolssampled(andparticularlyinfiveofthesix)thereisacriticalneedforongoingteachertrainingandmentoringthatstrengthenbothpedagogicalskillsaswellassubjectknowledge.

Effectiveteachingoftheintegratedcurriculumwillalsodependontheinstructionaltimeallottedtoteachingofformalsubjects.Instructionaltimevariesbetweenlower,typicalandhigherperformingIQSs.Observedlessonslastedbetween10and45minutes,withthemostcommontimespentteachingbeing10–20minutes.Classesobservedincludedmaths,computerscience,English,andHausa.Thestudymayhaveinfluencedtheobserveddurationsinceclasstimeswerescheduledinordertoallowforobservation.However,pupils,facilitatorsandotherstakeholdersreportedthatlessonsnormallylastaround20minutes,andthatabouttwosubjectsaretaughtperday.Thisdidvarybetweencases,withhigherperformingIQSsreportingthatclassesnormallylastaboutanhour.ThequalitativedataarenotclearonhowfrequentlyallIQSsteachformalsubjects.ThoughsomeIQSsreferredtoformalsubjectsbeingtaughtfivetimesaweek,othersspokeoftwotothreetimesaweek.Still,apartfromintheNigerhigherperformingIQS(teachingfivetimesperweek),therewereinconsistencieswithinthecasesthemselvesregardinghowoftenanIQSteachesformalsubjects.


©EDOREN 171

Thereare,however,someindicationsthatclassesdonottakeplaceregularly,owingtofacilitatorabsenteeismandweatherconcerns.

Facilitatortrainingandsatisfactionwithacquiredskills

AstherehavebeenpriorroundsofinterventionsunderGEP,thequalitativedatacapturesomeperceptionsregardingGEP3’straining,andsatisfactionwithacquiredskills–evenatbaseline.TheIQScasesallreporthavingreceivedsomeformoftraining,althoughthebaselinedataarenotfullyabletocapturetheextentofthis.Moreover,itisnotalwayspossibletodistinguishGEPtrainingfromotherformsofsupportandtraining–somethingthatwillbeimportanttoconsideratmidline.

FacilitatorsinallIQScasesexpressedaninterestinreceivingtraining.GEPtrainingalreadyreceivedhas,accordingtorespondents,focusedonsensitisationandtheimportanceofformaleducationthatisinclusiveofgirls.InsomecasesitseemedthatfacilitatorshadreceivedtrainingaspartoftheCBMC,ratherthanreceivingfacilitator-specifictraining.Somefacilitatorsalsoreferredtohavingreceivedtrainingonhowtouseteachingandlearningmaterials,andtheimportanceofmother-tongueinstruction.Perceptionsaroundtrainingarethatyouaretaught‘tools’withwhichtoteachandvaluesrelatingtotheimportanceofinclusiveinstructionandformaleducation.

‘Iwastaughtonteachingaidsandhowtorelatebetterwiththepupilsbyputtingonasmilingfaceandalsoplayingwiththem,andalsonotusecanetobeatthem.’(TPD,BauchilowerperformingIQS)

Facilitators,however,refertoneedingtraininginformaleducation–tofurthertheirownknowledgeinordertobeabletoteach.Asdiscussedabove,theabilityoffacilitatorstousetheskillsacquiredintrainingeffectivelywillvarydependingontheirinitialknowledgebase.Iffacilitatorsdonothavesufficientsubjectknowledgetomakeuseoftheteachingmethodsthiswilllikelylimittheextenttowhichtrainingincreasestheeffectivenessofteaching,asteachersdonotknowwhenandwhydifferentteachingmethodologiesareappropriate.Thisisakintolearninghowtouseallthetoolsinatoolbox,withoutknowingforwhatpurpose,andwhen,touseeachone.

Itisimportantthattrainingiscontextuallydrivenandtargetedtothelevelofknowledgeoffacilitatorsandheadteachers.TherangeofqualificationsandknowledgeofformaleducationacrossthesixcasesislikelytohaveimplicationsforseveralGEP3outputs.ItfurtherhasimplicationsfortheabilityofaheadteachertotakeonapedagogicalleadershippositionwithintheIQS,asdiscussedbelow.

Inschoolswithmorethantwofacilitators,oneneedstoconsiderhowknowledgecanbedisseminatedandembeddedbeyondthetwofacilitatorstargetedforGEP3training.IntheBauchitypicalschool,forexample,theMallam/proprietorhadhimselfattendedthefacilitatortrainingandhadbeensharinghisknowledgewiththetwofacilitatorsunderhisremit.However,thelikelihoodofthishappeningconsistentlycannotbereliedon,asitisnotoftentheMallam/proprietorwhoattendsthetraining.IntheBauchihigherperformingschool,theinterviewswiththefacilitatorsdemonstratedthat,whileonefacilitatorhadattendedGEP3training,theotherwasnotevenawareofit.Notonlymaythisdampentheimpactofthetraining,itmayalsocauseresentment.Furthermore,socialrelationships(e.g.betweenyoungerandolderteachers)mayaffecthowmuchtransferofknowledgeactuallytakesplace.


©EDOREN 172

4.4.3.3 Gender-sensitiveteaching

GEP3seekstoimprovefacilitators’skillsingender-sensitiveclasspracticesinorderforteachingtobecomemoreeffective,particularlyforgirls.Thisassumesthatfacilitatorsareamenabletohelpingtoimprovegirls’education.Facilitators’perceptionsof,andattitudestowards,theeducationofgirlsandboyswilllikelyinfluencethis.

Perceptionsof,andattitudestowards,girls’andboys’education

Facilitatorsexpresspositiveattitudestowardsgirls’educationinthesixcaseIQSs.However,therelativehigherimportancetheyattachtoboys’comparedtogirls’educationpointstopersistingbiases.Asbrieflymentionedabove,therehavebeensensitisationcampaignsandtrainingaboutgirls’education140.Still,subtletiesemergefromthevariousdiscussionsinformingthequalitativecases.Forexample,althoughallfacilitatorsperceivegirls’educationasimportant,somefacilitatorsthinkthatgirlsarelessablethanboysare,andthatgirlsdonotrequireasmucheducationasboys.Thus,oneneedstoconsiderpositiveattitudestowardsgirls’educationasrelativetoattitudestowardsboys’educationandabilities.

Inallbutonecase(NigerhigherperformingIQS)facilitatorsperceivedboystobemoreintelligentthangirls.Somerespondentsconsideredthisdifferencetobesomethingthatcame‘fromGod’(Niger,typicalIQS).Furtherprobingonwhatwasmeantby‘intelligence’wasneeded,asrespondentssometimesusedthewordinterchangeablywith‘performance’.Itisnotalwaysthecasethatfacilitatorsconsidergirlsasinnatelyless‘able’thanboys.Somefacilitatorsareawareofthegenderedcontextinwhichtheyteach—bothinstitutionallyandculturally.

‘Thegirlsarenotasbrilliantastheboys.

Whydoyouthinktheboysareabletoperformbetter?

Because,backtheninthiscommunity,girlsarenotusuallyenrolledinschool.Youfindoutthatifparenthasfivegirlsandaboy,theywillgiveoutthatboytobeenrolledinschoolandleaveoutthegirls…theboyshavestayedlonger,thegirlsjoinedrecently.’

(TPD,BauchihigherperformingIQS)

InpupilFGDs,boyssaidthatformalsubjectsaremoreimportantforboysthantheyareforgirls.Ratherthansayinggirlswerenotasabletolearnformalsubjects,thereasonprovidedwasthatboysaremorelikelytoneedtheskillstaughtinformalsubjects,suchasbusiness,whereasgirlsaretobemarried(seefurtherdiscussiononearlymarriageandenrolmentinSection3.2).

Thoughgirlsthemselvessaiditwasimportantforthemtolearnformalsubjects,therewasadifferenceinaspirationsbetweenboysandgirlsinsomeschools.Aspirationscanbeagoodindicationofattitudesinacommunitysincetheyshowthegeneralframeofreferencepossessedbygirls.Inallschools,boysaspiredtobecomedoctors,teachersandbusinesspersons,ortojointhemilitary.Manygirls,whenasked,tendedtoshrugtheirshouldersandsaytheywantedtoget

140Researchbiasmayhaveaffectedresponsessincetheresearchteamwasassociatedwiththe‘GirlsEducationProject’.Assuch,oneneedstoconsidertheseexpressionsofpositiveattitudescarefully.Thelackofpriorstudiesandexperiencesfromthecasestudyschoolsalsomeantthattheresearchteamexercisedahighlevelofinitialcautioninregardtonotoffendingrespondents,whichpreventedmoredirectpersonalprobingaroundgenderattitudes insomecases. ‘True’attitudesandperceptionsofgirlsarethusdifficulttopindownatthisstage.Formidline,questioningmethodscanbedevisedbasedonthegreaterfamiliaritywenowhave,inordertoprobemoredeeply,whilststillbeingcautiousofsensitivities.


©EDOREN 173

married.Interestingly,thiswasnothomogenousacrossIQSs,butappearedtobelinkedtothelevelofschoolingavailable,andhowlongintegrationhadbeeninplace.Forexample,intheBauchilowerperformingschool(withnoformalschoolclosebyandonlyonefacilitator)girls’aspirationswerelowest.IntheNigertypicalIQS,thecommunityhasmadeitcompulsoryforallchildrentocompletesecondaryschoolbeforemarriage.Here,girlswantedtobecometeachers,doctors,businesswomenandmechanics.Thisisnottosaythatgirlswillbeabletocontinueontotheirpreferredprofessions.Inseveralcaseschools,somegirlsspokeoftheirdesiretogoontobecomeprofessionals,suchasmidwivesanddoctors,butwerealsoawarethatearlymarriagewilllikelytakethemoutofschoolandwillthusbeabarriertotherealisationoftheseaspirations.

Inclusiveinstruction

Theresearchpointedtovariousformsofgenderbiasinclassroomorganisationandinstruction.Insomeclassrooms,girlssatatthebackoftheclassroomwhilstboyssatatthefront,whileinothersgirlsandboyssatondifferentsides.Reasonsforthisseparationincluded‘Islamicteachings’,andassertionsthatthissegregationisessentialinorderforparentstoevenconsidersendingtheirchildrentotheIQS.

‘BecauseinIslam,itisnotgoodforthegirlstobeinfrontandtheboystoseetheirbacks.SothatistheshariaoftheIslamandthatiswhywedoit.’(TPD,BauchitypicalIQS)

Thisindicatesthatregardlessoftheacceptanceofanintegratedcurriculumwithintheschool,traditionalQur’anicschoolingpracticeswithinthegender-biasedclassroomarrangementprevailinthemajorityofthesampleschools,andattitudesinthisareaappeartoremainquiteentrenched.Inthoseschoolswherethegirlswereattheback,theirinclusionwithintheclasswasclearlymoredifficulttomaintain,althoughforthosegirlswhoweredeterminedtoparticipateitwasnotadeterrent.Theobservationsalsoindicatedthatboysseatedatthefrontweremoreeasilyabletoengage,andautomaticallyreceivedmoreofthefacilitator’sattention.Thereisanunderstandingamongstsomefacilitatorsthatsittingatthefrontoftheclassroomispreferential.Asonefacilitatorstated:‘evenintheformalschool…theyoungonesinfrontandtheolderonesatthebackbecauseoftheeaseofcopyingnotes.Ifyoumixthemup,boththeyoungonesandtheolderones,theyoungonesatthebackwillbeunabletoseethewritingsontheboard.’(TPD,BauchihigherperformingIQS)

Whilefacilitatorsseemtodemonstrategendersensitivityintheirinteractionwithpupils,genderedexpectationsaboutclassroombehaviourstillneedsfurtherattention.DuringtheQCOs,facilitatorswouldactivelyaskbothboysandgirlsquestions.ThepupilFGDsconfirmedtheseobservationsandbothboysandgirlssaidthatfacilitatorsallowanyonetoanswerquestions.However,respondentsoftenperceivegirlstobeshyerandquieterthanboysare,describinggirlsas‘lessinterested’or‘lessmotivated’.Thoughfacilitatorsreportthattheyalwaysactivelytrytoengagethesestudents,somegirlssaidthatteachershaveneveraskedthemaquestion,andthatgirlshaveneveransweredaquestioninclass.InsomeIQSsboyssaidthatgirlsneveranswerquestionsorpayattentioninclass,astheyareonlyinterestedinmarriage.InoneoftheseIQSs,girls,however,reportedthattheMallamgetsangryiftheyspeaktoomuchinclassandthatthefemalefacilitatorpreferstheboystoanswer.

Somefacilitatorsclaimthattheyfocusmoreongirlsinschoolsincegirlswillbethereforashortertimeand,assuch,needtolearnmoreinlesstime.Interestingly,thefactthatgirlsarelesslikelytocontinueontofurthereducation,orevencompletesecondaryschool,hasvariousoutcomesasregardsinclusiveinstruction.


©EDOREN 174

‘Fornow,wewantthegirlstolearnmorebecausetheydon’tgotoschoolforlong.Theyoftengetmarriedandsometimesrelocateafterthemarriagebut,fortheboys,theyarealwayswithusandcanfurthertheirstudiesanytime.Butforagirl,onceshehasgrownshestartsthinkingofgettingmarriedsotheboysstaylongerinschoolandsomeofthem(boys)wanttobecometeacherstoo,thatiswhyweareconcentratingonthegirls.’(TPD,BauchitypicalIQS)

Somegirlshavesimilarperceptions:

‘Thegirlsshowmoreinterestbecauseherethegirlisnotallowedtolearnmuch.Andwehavebeentoldtoacquireknowledgeforourownsakeandthesakeofourchildrenbecausesomeday,youmightendupteachingothersoratleastteachingyourownchildren.’(GirlsFGD,BauchitypicalIQS)

Similarly,differingperceptionsoftheabilitiesofboysandgirlsaffectthenatureofinstruction.However,insomecases,thisdoesnotleadtoformsofinstructionthatdisfavourgirls,butrathertoeffortstocombatperceiveddifferencesinperformancelevels.

‘Butifyourememberduringthegroupworkactivitywecombinedboththeboysandthegirls.Itisnotlikewefindthegirlsdisgusting,orwebelittlethem.But,fromourownunderstanding,theboyslearnfasterthanthegirlssothatiswhyduringgroupactivitywetrytomixthemupinordertoshareideas.ButifIgroupthemaccordingtotheirgender,andIgivethemworktodo,therewouldbechallengesandthechallengeisthattheboysperformbettersothatiswhywemixthemtoshareideas?’(TPD,BauchilowerperformingIQS)

Therearealsosomegenderedelementstopupilassessment.Somefacilitatorsdescribepittingboysandgirlsagainsteachotherthroughperformancecompetitions,bygroupingpupils:whichevergroupwins‘getspampered’–‘thiswillnaturallymaketheothergroupmoreserious’(CBMC,Bauchihigherperformingcase).

WithintheQCOstheseoftenconflictedperspectivesregardinggirls’educationcamethroughinthewayfacilitatorswouldtrytoincludegirlspracticallyaspartoftheclass.InthetwoBauchischoolswheregirlssatattheback,thefacilitatorswouldfluctuatebetweenfocusingontheboyswhowereclosertothem,tothensuddenlymakinganefforttoengagewiththegirlswhowereattheback.IntheBauchitypicalschoolandtheNigerhigherperformingschoolgirlsweremorelikelytoraisetheirhandsandensuretheirownengagement(therebyalsobeingmorelikelytobecalledtotheboardtocompleteasumetc.),whileinthelowerperformingschoolthegirlswerelargelysilent.

Somefacilitatorstriedtoredressanygenderimbalancesinparticipationbycallingonlyongirlstoanswercertainquestions.However,itisimportanttoconsiderthattheresearchteamobservingtheclassmayhaveinfluencedthisbehaviour.Itisworthconsideringthepitfallsofsegregatingboysandgirlsasregardstheirresponsestoquestionsinclass(e.g.whenafacilitatorasks‘onlytheboys’tostandupandanswer,andthen‘onlythegirls’tostandupandanswer),asthiscanservetoentrenchdifference.

Overall,thequalitativeresearchfoundthatgenderbiasesaffectclassroomorganisationandpracticesduringtheteachingoftheintegratedcurriculum,despitethedisplayofgender-sensitivetechniques,duetoentrenchedgenderedattitudesandexpectationsaboutthebenefitsofeducationforgirls,andculturalnormsabouthowgirlsandboysrelate.Itis,therefore,importanttodelivergender-sensitivetechniquesintheclassroomwiththesupportofincreasinglytransformedgender-sensitiveattitudesamongstthefacilitatorsthemselves.


©EDOREN 175

4.4.3.4 Facilitatormotivation

WhilefacilitatormotivationisnotadirectintendedobjectiveofGEP3’sIQSS,weassumeitisanimportantcontributingfactortomoreeffectiveteaching.Weconsidermotivationtodemonstrateitselfin,amongotherthings,theeffortthefacilitatorsmakeintermsofattendance,punctualityandtimededicatedtothework.Wefirstdiscusstheseaspectsandthecontextinwhichtheyoccur.Next,wepresentfindingsonsomeimportantfactorsaffectingmotivation,suchasremuneration,jobsatisfactionandsocialstatusattachedtoafacilitatorrole.

Ingeneral,thequalitativefindingsfromthesixIQSsindicatethatissuesofremuneration,socialstatusandcommunityrolesandrelationsshapefacilitators’motivation.TheIQScontextprovidesadifferentsettingforfacilitatormotivationascomparedtopublicprimaryschools.Sincefacilitatorsareoftenfromthecommunitiesthemselvestheyareoftenaskedbythetraditionalleader,ortheMallam,totakeuptheroleofschoolfacilitators.

Attendance,punctualityandinstructionaltime

Communitystakeholders(includingpupils)perceivefacilitatorstobededicatedandpunctual.Thequalitativestudyexploredreasonsforlossofinstructionaltime,andwhilstrespondentsbroughtuplackoffacilitatorsasakeyreasonforinsufficientinstructionaltimefacilitatorabsenteeismwasnotseenasakeyreason,butratherthelackoffacilitatorsinIQSs.

‘Theteachersarealwayspunctual,youseethatourteacherwiththeredcap,therewasatimehewashavingeyeproblembutthatdidnotstophimfromcomingtoteachus,heworeglassesandcame.Heisalwaysaroundtoteachmaths.HewaseventakentothehospitalinBauchitogettreatment.’(BoysFGD,BauchitypicalIQS)

FewoftheIQScasesseemtohavesettimetables,withformalsubjectstaughtasandwhenafacilitatorisavailable.Whilstrespondentsmentionedthatformalteachingtakesplaceforcertainperiodsoftime,forexamplefortwohourstwiceaweek,inpracticeteachingappearstohappeninamoreadhocway,withaclassfor20minutes‘betweenthehoursof4and6pm’.Thetimetableforformalteachingalsoappearslessstructuredthanthatforreligiousteaching.Thismakesitdifficulttogetauniformunderstandingofhowandwhenformalsubjectsaretaught,whichmakesitchallengingtomonitorformalsubjectclasses.

‘TheQur’aneverydayintheeveningaround4pm,exceptThursdays,whiletheformalsubjectsareusuallyonWednesdaysandSaturdays.Intheeventthattheformalteacherisunabletocome,wecontinuewiththeQur’an.TheformalteacherteachesfortwohoursandteachesEnglish,writing,reading,Hausaandmathematics.Heteachesoneortwosubjectsperday.’(Mallam,BauchilowerperformingIQS)

TheFGDswithpupilscalledattentiontotheroleoftheclassmonitor,whoteachesyoungerstudentsifthefacilitatorisnotpresent.ThiswascommoninseveraloftheIQSs,wherepupilssaidthattheyoften‘runtheclasses’.DuringtheQCOintheNigertypicalIQS,thefacilitatorstartedtheEnglishclassbyhavingchildrensingthealphabetonebyone.Hethenlefttheclassroomonandoffandsatoutsideorinthedoorwayoftheclassroom.Assuch,facilitatorsmaybepresentandstartaclassontime,butitisnotnecessarilythecasethattheyarepresentintheclassroomtoensureeffectiveinstructionaltime.

Out-of-schoolactivitiesandtheweatherfurtherdisruptinstructionaltime.ManychildrenattendingIQSshaveresponsibilitieshelpingonthefarmorathomewithincome-generating


©EDOREN 176

activities,suchashawking(seefurtherdiscussioninthesection‘Qualitativecasestudyanalysis’inrelationtoContributionClaim3).Thisadverselyaffectsinstructionaltime,asparentsarelesslikelytosendtheirchildrentoschoolifittakesuptoomuchofthechildren’stime.Asfarmworkisseasonal,theattendanceofpupilsissignificantlyworseduringharvest.141Similarly,incontextswheretheIQSislocatedinafarmingcommunity,thefacilitatorsarealsoinvolvedinagriculturalworkandthereweresomeindicationsthatfacilitatorswouldbelesslikelyto‘runschool’duringthistime.Additionally,manypupilsattendpublicprimaryschool,thusrestrictingIQSinstructionaltimetoafternoonsandevenings,whenQur’anicteachingsalsooccur.142

Someofthequalitativecaseschoolsdonothaveaphysicalschoolstructure,butteachoutside.Thismeansthatinstructionaltimeseverelydecreasesduringtherainyseason:‘itisonlyduringrainyseasonthatweareafraidtocomebecausewehavenoshadeandthewindsarestrong.Wedonotevencomeregularly’(BoysFGD,BauchitypicalIQS).Additionally,thehotsunintheabsenceofaroof(asintheBauchitypicalIQS)meansthatthepresenceofshadecouldalsodeterminethetiminganddurationofaclass.Evenincaseswherethereisaphysicalschoolstructure,floodingduringtherainyseasoncanpreventschoolsfromopeningorchildrenfromattending,asIQSsareoftendifficulttoaccess.Thequalitativeresearchalsoexploredsecurityasareasonforlossofinstructionaltime.Securityconcernswerelesspressingthaninitiallyassumed,althoughthismaybeduetotheinfluenceofsecurityconsiderationsinsampling.

Remuneration,jobsatisfactionandbeingvaluedbyothers

ThemajorityofstakeholdersinIQScommunitiesreportthatfacilitatorsarenotpaidfortheirwork,apartfromtokensgatheredbythecommunity.Facilitatorsreceiveanamountconsideredinsufficienttoliveon.

‘Infact,I/youcancallthemvoluntarybutbecausetheyareonlygivensomepaltryassistancecomparedtotheirneeds.Despitethefactthatmajorityofthemarewomentheyalsohavetheirownuniquedailyneedsforsurvival,whatwegivethemcannotbeequatedtoasalary,honestly.Itisjustasmallallowanceandtheyarenotworkingelsewhere.’(CBMCFGD,NigerhigherperformingIQS)

IQSstakeholdersacknowledgethatthereisaninherentinconsistency,withfacilitatorsexpectedtodeliverlessonsatthestandardofteachers,whilefendingforthemselvesfinancially.Thereappearstobeanadditionalgenderedcomponenttothisroleconflict,143wherebymalefacilitatorsarerequiredtoprovideforafamily,aswellastohaveacertainsocialstandinginordertomarry.Thus,manyfacilitatorsareinvolvedinotherincome-generatingactivities,suchasfarmingorteachinginthepublicprimaryschool,orareengagedinothersocialdevelopmentprogrammingactivities.ThishasimplicationsforinstructionaltimeandtheamountofeffortputintoteachingattheIQS.Asanexample,thequalitativeteamfoundthattryingtoarrangetheQCOandTPDaccordingtotheclassschedulefortheBauchilowerperformingschoolwasdifficult,asthe(sole)facilitatorinthatschoolwasengagedinanotheragency-sponsoredvaccinationdrivejustoutsideofthecommunity.Itis

141Theagriculturalcycleandfarmworkaffectbothboysandgirls.Whileboysaremorelikelytobeengagedinfarmlabouractivitiesatplantingandharvesttime,suchaslandpreparationandthreshing,girlsandwomenareoftenresponsibleforoff-farmandpost-harvestactivitiesrelatedtotheyieldsproduced,suchascropprocessing,storage,andfoodpreservationandprocessing.142TheonlyschoolwhereitwasnotpossibletoattendbothIQSandprimaryschoolwasintheBauchilowerperforming,wheretheresimplywasnoprimaryschoolavailablenearby.Laterinthereportwediscussissuesaroundpovertyandresidualresistancetoformalschoolingasabarriertoattendingpublicprimaryschool.143GetzelsandGuba(1957).


©EDOREN 177

importanttoconsiderthecaptureofthefeweducatedindividualswithinacommunitybyotherprogrammeswithinthewiderissueofinsufficientfacilitatorremuneration.

Bynotpayingfacilitators,theexpectationsplacedonafacilitator,andthedegreetowhichacommunitycanholdfacilitatorstoaccount,mightdecrease.Asisevidentfromthequalitativefindings,thisisapressingconcerninthecommunities:

‘Theteachersarenotpaidsalarysoweacceptwhatevertheyofferratherthantoallowthechildrentobeleftlikethatwithoutlearning.’(ParentsFGD,BauchitypicalIQS)

‘Whenteachersarenotpaidgoodsalaryitmakesthemnonchalantandthatdiscouragesthechild.’(ParentsFGD,BauchitypicalIQS)

Yet,remunerationclearlyisnotthemainmotivationforfacilitatorsteachingatanIQS.InallsixIQSs,facilitatorsweremembersofthelocalcommunity.Stakeholdersperceivedthistobethereasonwhytheyagreetoteachvoluntarily.ItisalsoworthnotingthatseveralofthefacilitatorshadpreviouslybeenpupilsundertheQur’anictutelageoftheMallamorhispredecessor(thiswasthecaseinfiveofthesixIQSs),possiblycreatingcontextsinwhichasenseofresponsibilitytothecommunityorevendutytoarespectedformerteacherareatplay.IntheBauchitypicalschool,thefacilitatorreferredtobeingcalledbyhisteacher(theMallam)tocomeandteachinhisschoolinorderto‘helpteachouryoungerones’oncehehadcompletedhishighschooldiploma.

FinancialconstraintsmakeitdifficultforIQSstoattractqualifiedfacilitators.InmanyoftheIQScommunitiesthemajorityofpeoplelackaformaleducationbeyondsecondaryschool.Facilitatorsarethus,‘childrentaughtbythiscommunity’,andinmanycasestheirsubjectknowledgeislow(asdiscussedabove).

‘Youknowgettingateacherfromoutsidethevillagethatisfarfromtheroadisdifficult.Wemustgetmoneyforfuelorbicycles.Wedonothavemoney.Sowegivelittlemoneytotrytogetsomeoneandtheyappreciatebecausetheyarealsoteachingtheiryoungerones.’(CBMCFGD,NigertypicalIQS)

Nevertheless,inspiteofthelackofremunerationfacilitatorsinIQSsreportfeelingintrinsicallymotivatedbywhattheydo:‘educationisimportantparticularlyforthegirlchildbecauseitwillhelpthemgreatlyinlife.ThatiswhyIwanttogiveitmybestasacontributiontoseetheyprogressinlife.’(Femalefacilitator,NigerhigherperformingIQS).Facilitatorscitetheirmotivationascomingfromwantingtohelpchildrenandtohelpthecommunitytodevelop,beinginterestedinteachingandthebeliefthatarewardwillcomefrom‘God’.

‘WegetourrewardfromthecommunityandfromGodinheaven.Thecommunityrespectandvalueus;whereverwegoboththeparentsandthepupilsrespectusandarehappytoseeus.’(Facilitator,BauchitypicalIQS)

Therearecontrastingaccountsoftheperceivedsocialstatusofbeingafacilitator,andthislinkstoissuesofremuneration.Thoughcommunitymembersrefertotheimmenseimportanceoftheteachingprofessionandhowtheywillberewardedinheavenforthissacrifice,facilitatorsreportnotfeelingvaluedduetotheirlowpay,withexamplesgivenofhavingbeendeniedagirl’shandinmarriage.Thefinancialchallengesmeanthatfacilitatorsstruggleto‘moveupalevel’inlife,ascomparedtotheirpeers:‘IhavetaughtforalmostfouryearsbutoneofmyfriendswhomIschooledtogetherwithinAzareisnowworkingwiththeairport.Heownsahouseandacar.Asforme,only


©EDOREN 178

thatmotorcyclethatyouareseeingoutside.Thedifferenceisclear.[IfIwas]notateacherIwouldhavepassedthislevel.’(TPDBauchihigherperformingIQS)

Thesamefacilitatorspokeoftherewardforateacherasbeing‘onlyinheaven’,aresponsethat—whencoupledwithhisexperienceofhavingalowstatusandbelow-parpayincomparisontopeers—indicatesapotentialchallengetothecurrentgoodwillandmotivationoffacilitatorswhobelieveintrinsicallyintheeducationalpurposeoftheirwork.Genderedexpectationsofmenwithinthesocietyasbreadwinnersplaysacriticalrolehere(asexemplifiedbyfacilitators’inabilitytosecureamarriage),andsuchpressuresinapredominantlymaleprofessioncannotbeunderestimatedwhenitcomesnotonlytothemotivationoffacilitators,butalsotoretentionwithintheprofession.Thereportentitled‘BeingaManinNigeria:PerceptionsandRealities’(2015)identifiesthesocialnormthatviewsmenas‘solebreadwinners’inNigeriansocietyasengenderingfeelingsoffailurewhenitisnotpossibletoliveuptothoseexpectations144.Meanwhile,Kelleheretal.(2011)identifiedconsistentlyinadequateteacherremunerationinmanycountriesas‘oneofthecorereasonswhymanymeneitherchoosenottoentertheprofessionorendupleavingitinthelongrun”145.

TheGEP3ToCassumesthatmotivationwillincreasewherefacilitatorsareabletoaccessallthetechnicalproficiencies,guidance,support,andmaterialsneededinordertobeagoodteacher.However,suchmotivation,anditstranslationintoteachingeffort,isinfluencedbytheverybasicandpracticalunderpinningofanIQSfacilitator’seconomicneedsandresponsibilities,andbyhowthesemayconflictwiththewell-intentionedcommitmenttoteachingtheintegratedcurriculumintheabsenceofadequateremunerationandthestatusthatthisismorelikelytobringwithit.Therefore,adequateremunerationisanimportantcontextualfactorthatislikelytoinfluencetheimpactpathwaytomoreeffectiveteaching.

4.4.3.5 Pedagogicalleadership

OneofthekeyintendedoutcomesofGEP3isenhancedschoolleadershipandmanagement.Theprogramme’sToCnotesthat,throughcapacitydevelopment,headteacherswillbeabletoleadschoolsandfacilitatorsmoreeffectively.Thissectionwillfocusprimarilyonpedagogicalleadership,whilstContributionClaim2discussesschoolmanagementmorebroadly.

Schoolleadershippositions(proprietor,Mallam,headteacherandCBMCmembership)arenotnecessarilyseparateroleswithintheIQScontext.AsdiscussedinthesectiononContributionClaim2,thereisalackofclaritywithinmostcommunitiesregardingwhohaswhatrole,andwhatthatroleentails.Whenasked,somerespondentsconsideredthateachIQShaseachmanagementposition.However,whenprobedfurtherthemostcommonscenarioseemstobeonepersontakingonallthreerolesofproprietor,Mallamandheadteacher.Inmanycases,thispersonalsoholdsaprominentpositionontheCBMC,suchaschairpersonorsecretary.Consequently,whiletherolesmayallexistwithinasingleIQS,thisdoesnotnecessarilymeanthattherearedifferentpeopleholdingeachrole.WhileapersontakingupseveralrolesinanIQSdoesnotnecessarilyaffectGEP3’sschoolleadershipandmanagement,lackofclarityabouttheserolescanmakeitdifficulttoidentifywhoseleadershipcapacitiestobuild.Differentactorsmaybeinvolvedintheimplementationofsimilarroles,andthemostqualifiedpersonmaynotleadontherole,forexample,ofpedagogicalleader(seebelow).

144VoicesforChange(2015)BeingaManinNigeria:PerceptionsandRealities,DFID.145Kelleheretal.(2011)WomenandtheTeachingProfession:ExploringtheFeminisationDebate,UNESCOandtheCommonwealthSecretariat.


©EDOREN 179

TheremaybesomevarianceinassigningrolesandresponsibilitieswhenitcomestodifferenttypesofIQS(bothpriortoandafterintegrating).AsAntoninis(2014)outlines,thereare,broadly,twotypesofreligiouseducationinstitutionsinnorthernNigeria:Qur’anicreligiouseducationinstitutionsandIslamiyyareligiouseducationinstitutions.Qur’anicinstitutionsencompassbothmakarantaallo(‘schooloftheslate’)andTsangaya.146Inbothinstitutions,theMallam(teacher)isinchargeoftheschool,andtimetablesareadhoc,ontheinitiativeoftheMallam.147Assuch,childrenareattimesabletoattendpublicprimaryschoolaswell.148TheTsangayaelementmeansthattheinstitutionhasamobileelement,inwhichtheMallammoveswithhispupils‘inthebeliefthatanitinerantlifeisessentialforthemtofullyconcentrateontheirstudy’.149SincetheMallaminTsangayasis‘incharge’ofthelocationoftheschool,beitinhishouseoronthemove,heislikelytoautomaticallybetheowneroftheschool.IslamiyyainstitutionsgobeyondmemorisingtheQur’anandincludeavarietyofIslamicsubjects,150suchasArabic,Islamichistoryetc.Islamiyyastakeamorestructured,stationaryformthanMakarantaallosandTsangayas,andtendtohavesetinstructionaltimes.151MallamsdonotnecessarilyownIslamiyyas.

MobilityoftheTsangayaschoolsislikelytointerruptteachingoftheintegratedcurriculumandmayaffectthegenderinclusivenessoftheschool.InoneTsangayaIQSinBauchi,theheadteacher(Mallam)saidthattheywerenotmobileanymore,duetothesecuritysituationinBauchi,whiletheboysintheFGDreferredtohavingbeen‘onthemove’andmadeclearthatfurthermobilitywasplannedforthefuture.TheMallamwouldthenmoveonlywithboys.Itisunclearatthisstagewhethertheformalsubjects’facilitatorwouldcontinuetoteachinthecommunityintheabsenceoftheMallamandtheboys.However,thereisahighlikelihoodthatthiswouldinterrupttheteachingoftheintegratedcurriculum,andthatgirlsmaybecomedisconnectedfromtheschool–thereforeaffectingtheinclusiveinstructionofthegirls.

ThecasestudyIQSswerereferredtoasTsangayainBauchiandIslamiyyainNiger,althoughallschoolsappearedtodisplaycharacteristicsclosertoIslamiyya.ThequalitativestudyfoundTsangayasreferredtoasschoolswithamobileelement,incontrasttothestationaryIslamiyyathatfollowamoreformalstructureconcerningtimetablesandteachingapproach.However,inpractice,therewerecomeinconsistencieswiththisterminology,particularlyinBauchi,wherealltheschoolswerecalledTsangayasbutalldisplayedslightlydifferentcharacteristics.Furthermore,inconversationrespondentsfrequentlyusedTsangayasinterchangeablywithMakarantaAllo.AllthreeBauchiTsangayasappearedtohave,onaspectrum,similarcharacteristicstoIslamiyyas,althoughnotalloftheBauchiIQScaseshadphysicalinfrastructure.Thehigherperformingschoolhadasolidphysicalstructure,moregirlsattendingschoolthanboys,andalargenumberofQur’anicteacherscoveringawiderangeofIslamicsubjects.OftheothertwoBauchiIQSs,onlyinthecaseofthelowerperformingschoolwastheresomeindicationthatitwasstillmobile,althoughthiswasinconclusive.InNigerstate,allthreeschoolswereIslamiyyas.Theschoolshadseparateformalstructuresinallbutonecase(NigerhigherperformingIQS),wheretheschoolwaslocatedinthehouseoftheproprietor.

RolesandresponsibilitiesvarysignificantlyfromoneIQStoanother.Thecommunitiesconsiderthepeoplewhofoundedtheschool,andwhoownandrunit,tobetheproprietors.IncaseswherephilanthropistsstartedtheIQS,orwherethecommunitydidsothroughdonations,theperceptionisthatschoolsare‘owned’bythecommunityanditisthuslessclearwhowouldbeactingas

146Antoninis(2014).147Ibid.148Ibid.149Antoninis(2014:83).150Ibid.151Ibid.


©EDOREN 180

proprietor.WhentheschoolhastransitionedfromaQur’anicreligiousinstitutionintobecominganIQS,theMallamislikelytobetheactingproprietor.TheMallamisinchargeofreligiouseducation.InallthesixIQSvisitedtheMallamisalsotheheadteacher,although,asinthecaseofNigertypicalIQS,he/shemaynotbethehighestreligiousauthorityinthecommunity.Inseveralcases,theMallam/headteacheralsoholdsaprominentpositionontheCBMC,suchaschairperson.Additionally,eachcommunityhasacommunityleader,which,inallsixcases,wasseparatefromtheabove-mentionedroles.Still,thecommunityleaderisnormallyabletoinfluenceschoolmanagementandisoftenpartoftheCBMC.

Theheadteacherroleismainlyconsidereda‘schoolleader’role.Asdiscussedabove,thequalificationsandteachingexperiencethattheMallamhassignificantlydiffersacrossthesixcases,rangingfromhavingreceivednoformaleducationtohavinganNCE.Theabilityofheadteacherstoprovidepedagogicalsupervisionandmentoring,andtheplausibilityofthis,isthusquestionableincaseswhereheadteachersdonothavepriorsubjectknowledgeorteachingexperience.

‘AllinallIspentabout18yearsinsearchofknowledgebutthroughQur’aniceducation,noformaleducationatall.But,Iwastaughta,b,c,dbyafriend,andnowIcanreadinHausa.IcanevenreadHausanewspaper.ButIhavenotgonetoschoolintermsofformal.’(Headteacher/Mallam,BauchilowerperformingIQS)

However,mostheadteachersinthequalitativestudyappearedconfidentintheirabilities.Assenioreducatorswithintheirestablishmentstheyviewedthemselvesashavingtheabilitytoinstructtheirfacilitators.ThenotableexceptiontothiswastheBauchilowerperformingcase.Headteachersfrequentlyconsultwithfacilitators,andgiveadviceonhowtoteachandinteractwithpupils.Advicerangesfromadviceonhowchildrenshouldsitintheclassroom,tohowtouseparticipatorymethods,dependingonthecapabilityoftheheadteacher.Pedagogicalleadershipseemstobesharedamongstmanystakeholders,withCBMCmembersguidingheadteachers,headteachersadvisingfacilitators,andproprietorsandcommunityleadersweighingin.Itwilllikelybeachallengetopinpointwhototrain,astheassignedheadteachersareoftennotthemostqualifiedintermsofformaleducation.152

‘He[theMallam]givesmeadvicebecausesometimeswhenIamteachinghecomesinandsitsdownjustthewayyoudidtoday,ifheseesanythingwrongthatconcernsthechildren,afterthelessonhewillaskandsaywhyareyourthingsdonedifferentlyfromours(Arabicstudies),doitthiswayandthenhewillgivemeadvice.Youseethisproprietor[Mallam]hasformaleducationtoo.Hedidnotgettofinishbuttheywerethefirsttoattendformalschoolinthiscommunity.Hecompletedhisprimaryschool,gottosecondaryschool,whichwastheonlyschool[backthen],buthedroppedoutofschoolinGrade4.Heisalwayshappywithwhatwedo,sometimesifmyattentionisneededelsewhereorurgently,hetakesoverthelesson.LikeifIamteachingalphabetsorsimplesentenceshewilljusttakearulerorstickandcontinuesthelesson.’(TDP,BauchihigherperformingIQS).

Asdiscussedabove,theappointmentofMallamsasheadteachersisnotnecessarilybasedonabilityandqualificationsbuthastodowithsocialstatuswithinthecommunityandperceptionsaroundleadershipmoregenerally.Inordertodiscussthevariousrolesandresponsibilitiesfurther,weneedtoclarifywhateachmanagementroleentails,andthustoidentifyneedsforcapacity

152Thehigherperformingcase inNigersignificantlydiffers fromtheother IQSs,aspreviouslydiscussed(seeannex).ThisIQShasaproprietorwithanNCE,andshehasclearpedagogicalleadership.ThereisaseparateMallam/headteacherwhodoesnothaveanNCE,andseveralfacilitatorswithNCEs–theheadteacheristhustheleastqualified;however,theremaybeagenderedcomponenttothis.


©EDOREN 181

developmentandtraining.Theability,perceivedcredibility,andpowertoenacttheseresponsibilitieswouldthusvarydependingonwhethertheheadteacherisresponsibleformonitoringwhetherteachinghappens,orwhethertheheadteacherisresponsibleforthedeliveryofqualityteaching.Pedagogicalleadershiprequiresknowledgearoundformaleducationandmethodsofinstruction,whilenormativeleadershipalignsmorecloselywithideasofvision,valuesandmoralleadership.153WithintheIQScontext,itmaynotbeplausibletoassignthesameindividualtheseresponsibilities.Mallamswillhaveahighsocialstandingandcredibilityintermsof‘whatisrightandwrong’morally,andthusholdanimportantpositionintermsofinfluencingthecommunityconcerningintegrationandgirls’education.Assuch,Mallamsmaybeassignedpositionsbasedonperceivedpoliticalandsocialvalue,ratherthanteachingknowledge:‘ThisschoolhasbeenaQur’anicschoolfor10yearsnow,butwhenwedecidedtointegrateformalsubjectswedecidedtoallowthesameMallamtocontinuetoteachthechildreninsteadofbringingsomeoneelsetoteach.’(CBMCFGD,NigertypicalIQS)

ForGEP3,thevariancebetweenthecapacityoftheheadteacher/Mallamintermsofconfidenceandqualificationsraisesthequestionofwhotoinviteforpedagogicalleadershiptraining.Itislikelythattheprogrammewillneedtotaketimetoaddressthisonanindividual(IQS)basis.Theapplicationofcertaincriteriaandcontextualissues–suchasplacingafacilitatorabovehischildhoodMallamwhenitcomestopedagogicalleadership–needscarefulconsideration.SimplyaskingtheIQStonominateandsendsomeonebasedonthetitleofheadteacher/Mallammaynotyieldpositiveresultswithoutprovidinganexplanationoftheexpectationsofthisroleintermsoftechnicalcapacity,andnotjustseniority/status.

Itmayalsobethatthereisnoclear‘candidate’forheadteacher/pedagogicalleader,asseeninsomeoftheIQScasesvisited,wherenooneinthecommunityhadanyformaleducationbeyondsecondaryschool.ThelowerperformingschoolinBauchiinparticularexemplifiesthis.InthisIQS,theMallamclearlydeferredtotheIQSfacilitatorinmatterspertainingtoformallearning,believinghimselfunabletoadvisehisfacilitator(achildhoodfriend,ofasimilarage,andwithwhomhehadmoreofapeerrelationship),becausehehadnotgonethroughformalschooling,whereasthefacilitatorhad.ItwasclearthattheIQSfacilitatorwasverymuchguidinganddirecting,himself.Astheonlyfacilitator,hecouldnotbeidentifiedasaheadteachertoprovidepedagogicalleadershiptoothers.Incontrast,theBauchitypicalcaseshowedtheclearestevidenceofasuccessfulpedagogicalrelationship,wheretheMallamhadhighereducationalqualificationsthanbothhisfacilitators(hewasauniversitygraduate)andwasalreadytrainingthetwoIQSfacilitatorsinpedagogicaltechniquesfromtheGEP3training.

Consequently,thecomplexrelationshipsandthechallengesinidentifyingclearheadteacher/facilitatorrelationshipsmayhaveimplicationsfortheGEP3ToCpathwaythroughwhichformalcurriculumpedagogicalleadershipandmentoringwilltakeplace.

4.4.3.6 Availabilityofteachingandlearningmaterials

GEP3supporttoIQSwillincludethedistributionofteachingandlearningmaterials,meanttocontributetoteachingthatismoreeffective.Below,wefirstdiscussthebaselineobservationoftheavailabilityanduseofteachingandlearningmaterialsinthecasestudyIQSs.Next,wediscusstheperceptionsofIQSrespondentsabouttheappropriatenessofthematerials.

153NSCL(2003).


©EDOREN 182

Availabilityanduseofteachingandlearningmaterials

ThecurrentavailabilityanduseofteachingmaterialsvariessignificantlybetweenthesixcasestudyIQSs.TheNigerhigherperformingIQShasaccesstoawidearrayofmaterials,includingexercisebooks,textbooks(forallsubjectstaught),variousteachingaids(flashcards,countingblocks)andevenfunctioningcomputers.OtherIQSshaveonetextbookfromwhichthefacilitatorsextractclassexercisesandrepeatlessons.InotherIQSstherearenotextbooksavailablebutfacilitatorslookthroughbooksinthepublicprimaryschooltoplanslessons.OneIQSmadeuseofstones,sticksandbottletopstocountinordertoallowchildrentovisualisemaths.AllIQSsvisitedhadablackboard;insomecasesthiswastheonlymeansofenablingchildrentoreadandwrite.

DuringQCOsitbecameclearthatmanychildrenlackbasicmaterials,suchasnotebooksandpens.Thisrequiresthefacilitatorstoassesshomeworkverbally.

‘Ithappensalot.Youfindapupilcomingtotheclasswithoutpensorbooks,soweusuallytellthemtotelltheirparentstobuythempensandbooks,andiftheycontinuetocometoschoolwithoutwritingmaterialswejustleavethem,thereisnothingwecando.’(TPD,BauchihigherperformingIQS)

AssomeIQSsvisitedhavenoabilitytoprovidepupilswithlearningmaterials,thisresponsibilityfallsontheparents.Thiscouldhaveanegativeimpactoninclusiveinstruction.Severalrespondentsconsidertheinabilityofparentstopurchaselearningmaterialsasareasonwhychildrenwillnotattendschool.

Perceptionsofteachingmaterialsfortheintegratedcurriculum

SomeIQSshavereceivedacurriculumforteachingHausa,andsomeflashcardsandotherteachingaids.AmongstthoseIQSsthathavereceivedsomematerials,respondentsperceivematerialstobeappropriateandcapableofaidinglearningamongpupils.Itwaschallengingtoprobethisissueofteachingandlearningmaterials,asatthisstagefewIQSshaveanymaterialsandassuchthequestionwasinterpretedas‘whatdowewant’.Hence,respondentsmainlyciteditemsneededbytheschool,rangingfromkettlesandmats,toexercisebooksandpencils.

Respondentsreferredtochallengeswithregardtotheuseofmother-tongueinstructioninthecommunitiessampled.Somerespondentspointedtothelackofmother-tonguematerials–inparticular,teachingandlearningmaterialsinHausaandtheHausacurriculum.However,inotherIQSs,Hausawasnotthemothertongueforall.Forexample,intheNigertypicalcase,thecommunitymainlyspokeNupe,withbarelyanyHausaspoken.InanotherIQS,althoughthemajorityofstudyparticipantsspokeHausa,therewerereferencestopupilswhodidnot.However,itcannotbeclaimedthatthequalitativecasestudiesarerepresentativenessofthewiderpupilpopulation.

‘It[training]wasgood,butourproblemhere,thosepeoplearoundheredon’tunderstandHausa.WeunderstandEnglish154andNupethatisourarea.IhaveonlyreceivedcurriculumforHausa.’(Headteacher/Mallam,NigertypicalIQS)

‘ThereasonwhywelikeHausa,seenowyouaskedthisquestionwithHausaandshedoesnotunderstandtheHausaright?ThatiswhywewanttolearnitverywellbeforetheystarttoteachusEnglish.’(FGDgirls,BauchilowerperformingIQS)

154Theheadteacher,whospokealevelofPidginEnglishandstruggledtocommunicatewiththeresearchteaminEnglish,saidthis,andthewidercommunitydidnotspeakmuchEnglish.


©EDOREN 183

‘WealsowishtheteachersuseMiya,theindigenouslanguage,toteachtheMiyamaswhodonotunderstandHausaandEnglishtopickup.’(ParentsFGD,BauchitypicalIQS)

Atpresent,thecompleteabsenceofqualityliteracyandnumeracylearningmaterialsinsomeschoolspresentsaclearbaseline.Theextenttowhichprovisionoffit-for-purposeteachingandlearningmaterialsisabletocontributetoeffectiveteachingisagainhighlycontextdependent,astheneedsofIQSsvarysignificantly.Moreover,teachingandlearningmaterialsneedtobefit-for-purpose,notonlyinrelationtocontent,butalsoinrelationtotheskillleveloffacilitators–assomefacilitatorsonlyhavebasicliteracyandnumeracyskills.

4.5 Analysisofthedata–ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlylearningenvironment155

Inthissectionwepresentthebaselinedataonschoolmanagement,resourcemobilisation,andcommunitysupportforintegration.Inaddition,wediscussothercomponentsoftheIQSSToCthatareassumedtosupportmoreeffectiveteaching:thatis,thedegreeofgovernmentsupportandthegeneralschoolconditions.Thequantitativedataanalysisispresentedfirst,followedbytheanalysisofthequalitativecasestudyfindings.

4.5.1 Quantitativedata

4.5.1.1 Schoolmanagement

Roleoftheproprietor

Ofthe60schoolsintheIQSSsample,over90%hadaproprietor.ThissuggeststhatinmostofthecasestheIQSwasinfactownedbyanindividual,ratherthanthecommunity.Intheremainingcases,theIQSwascommunityowned,exceptinonecasewhereitwasownedbythegovernment.Almostalloftheproprietorsweremale.FemaleproprietorsonlyexistedintheNigersub-sample,whichisinlinewiththetrendsinfacilitators’genderobservedacrossthetwostates.

Inone-thirdofthesampledschoolstheheadteacherwasalsotheproprietor.Insuchschools,inadditiontoowningtheschool,theproprietorwastheoneresponsibleforday-to-dayschoolmanagement.ThisoperatingstructurewasmorecommonforschoolsinBauchithaninNiger.Asalreadydiscussed,schoolsinBauchiwereoperatinginawaythatwasclosertothe‘traditional’modelofanIQS.Thisfindingfurthersupportsthatnotion.

TheestablishmentoftheCBMC

CBMCswereonlyavailableforinterviewin52outofthe60IQSSschoolssurveyed.Insteadof30schoolsineachstatehavingaCBMC,28schoolsinNigerand24schoolsinBauchihadaCBMC.TheremainingschoolshadnotformedaCBMCyet.

ThereareinterestingdifferencesbetweenschoolsthathaveaCBMCandthosethatdonot.AlthoughthesamplesizeforschoolswithoutaCBMCisquitesmall,whichsuggeststhatweshouldbecarefulaboutextrapolatingthisevidence,thissectionpresentsabriefsummaryofthedifferences.SchoolswithaCBMCareapproximatelytwoyearsolder,morelikelytoteachallfiveintegratedsubjects,andspendmorehoursperweekteachingtheintegratedcurriculum.Suchschoolsarealsomorelikelyto

155Thisrelatestolinks4,5,6,7and10intheToC.


©EDOREN 184

haveseparatedtheroleofheadteacherandproprietor.ThissuggeststhatCBMCsweremoreprevalentinschoolsthathad‘deeper’integration,andnotnecessarilyjustmoreyearsofintegration.

InsomecasestheCBMCwasformedbeforetheyearinwhichtheQur’anicschoolintegrated,andthuscouldhavefunctionedasoneofthedriversofintegrationactivitiesattheschoollevel,whileinothercasestheCBMCwasformedafterintegration,possiblyasanoutcomeofbeingintegrated.Intheoverallsample,theCBMCwasformedbeforeintegrationin37%ofthecases.AlmosthalfoftheschoolsinNigerhadaCBMCbeforeintegration,whilethiswastrueforonlyaquarteroftheschoolsinBauchi.AlookattheyearssinceCBMCformationsuggeststhatinapproximately70%ofthecaseswheretheCBMCwasformedbeforeintegration,theCBMCwasformedtheyearbeforeintegration.

Onaverage,schoolshadhadaCBMConlywithinthelastthreeyears.SchoolsinNigerandschoolswheretheheadteacherwasnottheproprietor,hadhadaCBMCforlongerthanschoolsinBauchiandschoolswheretheheadteacherwasalsotheproprietor.

CBMCmembership

Onaverage,CBMCshad14members,withaquarterofthembeingfemale.Therepresentationofchildren,definedasanyoneundertheageof15years,wasverylow,asonly17%ofCBMCshadachildmember(andagirlchildmember).Asashareoftotalmembers,only2%ofalltheCBMCmemberswerechildren.ChildmembershipwashigherinBauchithaninNiger.InCBMCswithachildmember,therewereapproximatelytwochildmembersonthecommittee.Overall,however,therepresentationofchildrenwasquitesmallinabsolutenumbers,whichposesachallengeinregardhowtovoicetheconcernsof,andtorepresent,children–especiallygirls–inthemanagementoftheschool.

87%ofallCBMCshadatleastonefemalemember.BauchiCBMCsnotonlyhadmoremembers,buthadagreatershareoffemalemembers,byalmost10percentagepoints.Thisisinterestingtonote,giventhatfemalerepresentationinteachingandschoolownershipisextremelylimitedinBauchi.Thissuggeststhatalthoughtherearebarrierstofemalesworkingformallyinschools,femaleinvolvementwithintheschoolsinBauchiisnotnecessarilylesserascomparedtoNiger.However,membershipdoesnotimplyparticipation,sincethefemalememberattendanceduringthelastCBMCmeetingwasmuchhigherinNiger(48%)thaninBauchi(26%).Itisalsoencouragingtonotethatin80%ofthecases,theCBMCshadawrittenrecordofCBMCmembership,whichwasusedasevidencewhengatheringinformationonmembership.AgreatershareofCBMCsinBauchimaintainedtheseformalrecordsthaninNiger.

InabouthalfoftheCBMCs,theCBMCwaschairedbytheproprietororheadteacher.Intheremainingcases,theCBMCwaschairedbyaparent,religiousleaderorcommunitymember,wherethesewereseparatefromtheheadteacherandproprietor.SuchCBMCsweremorecommoninNigerthaninBauchi.HeadteachersandproprietorswereusuallymembersofCBMCs,eveniftheywerenotthechairs.Inabout80%ofthecasestheheadteacherortheproprietorwerepartoftheCBMC.OthermembersoftheCBMC,andtheirparticipationintheCBMC,aredepictedinFigure63below.


©EDOREN 185

Figure63: AveragenumberofmembersinCBMCs

ThissuggeststhatthereisconsiderablerepresentationofparentsandcommunitymemberswithintheCBMC,asintended.Inaddition,communitymembersfillingthechairorvice-chairpositionwithintheCBMCscouldsuggestthattheyhavepowerwithintheschoolsetting.ThenumbersindicatethatthereweremoreparentsintheCBMCchairandvice-chairroleinNigerthaninBauchi.

CBMCmeetingsandtrainings

Amajority(90%)oftheCBMCsreportedmeetingduringthelastyear,withanaverageoffourmeetingsperyear.Two-thirdsoftheCBMCswereabletoprovidesomesortofevidenceoftheirmeetings.Thisevidencerevealsthattheattendanceratewas41%ofthemembersatthelastCBMCmeeting.InschoolswheretheCBMCwasestablishedafterintegration,CBMCsweremeetingmoreregularly,andhadgreaterattendanceandgreaterfemaleattendanceatmeetings.BauchiCBMCsalsoappeartobemeetingalmosttwicethestipulatedamountofthreemeetingsayear.InNiger,thoughthemeetingswerelessfrequentcomparedtoBauchi,theattendanceratewashigher(46%inNiger,versus36%inBauchi).

Onaverage,approximately60%ofCBMCshadreceivedsomesortoftrainingduringthelasttwoyears.Similarlytotheprevalenceofteacherandheadteachertraining,agreatershareoftheCBMCsinNigerreceivedsomesortoftrainingcomparedtotheCBMCsinBauchi.InNiger,70%oftheCBMCshadreceivedtrainingoverthepasttwoyears,whileinBauchithiswasonlyslightlyabove50%.Ineachstatethetraininglastedanaverageoffourdays,thoughonaveragethenumberoftrainingsessionsattendedbyCBMCmembersinNigerwasslightlyhigherthanthenumberoftrainingsessionsattendedbyCBMCmembersinBauchi.AsreportedbytheCBMCchair,thetrainingcontent,asummaryofwhichispresentedinFigure64below,didnotvarygreatlyacrossstates.

0

2

4

6

8

10

12

14

16

TotalmembersofCBMC

TotalfemalesinCBMC

TotalchildreninCBMC

NumberofteachersinCBMC

NumberofparentsinCBMC

Numberofoldorcurrentpupilsin

CBMC

Numberofreligiousleadersin

CBMC


©EDOREN 186

Figure64: ContentofCBMCtraining

MostoftheCBMCmembersreportedbeingtrainedonCBMCresponsibilities,wholeschooldevelopmentplanningandcommunitysensitisationaboutgirls’education.ThemostcommonsourceoftrainingdeliveredinboththestateswasGEPandUNICEF,ataround65%.InbothstatestherewerealsoreportsofCBMCsbeingtrainedbyLGEAsandSUBEBs,thoughthiswasgreaterforBauchithanNiger.AverysmallshareinNigerreportedtrainingbySAME.

4.5.1.2 Schoolmanagementpractice

AkeyroleoftheCBMCistodeveloptheWCDP,andtomonitorprogressagainstit.ThiswasalsoindicatedasbeingakeyareaonwhichCBMCsweretrained(Figure64).Overall,onlyabout20%oftheCBMCshadcompletedaWCDP,andanother20%wereintheprocessofdevelopingaWCDP.OftheCBMCsthathadacompleteplanavailableorwereintheprocessofdevelopingaplanonlytwo-thirdshadawrittenformoftheplanavailable,andone-thirdcouldprovidesomekindofwrittenevidencetodemonstrateCBMCinvolvementindevelopingtheWCDP.Intheremaining60%ofCBMCsnoWCDPexisted.Differencesbetweenstatesseemminor,whileproportionallymoreIQSslocatedinruralareashadacompletedWCDPcomparedtoIQSslocatedinurbanareas.However,smallsamplesizesleadustocautionagainstanyhardconclusionsaboutsuchdisaggregatedanalysis.ThemainactionpointmentionedintheWCDPwasthebuildingofnewinfrastructure.

TheevidenceonfinancialmanagementbyCBMCmembersislimited.WeattemptedtoverifyanyclaimsmadebytheCBMCmembersbyaskingforsupportingevidence.Only27%oftheCBMCsreportedusingacashoraccountsbook,ofwhichlessthanhalfhadanupdatedcashbookavailable.QuitealotofCBMCs(about70%ofthesample)usedabankaccounttomanagefunds.Inmostofthesecasestheaccountwasinthenameoftheschool,followedbyitbeinginthenameofaCBMCmember(inapproximately27%ofthecases).Inlessthan3%ofthecasestheaccountwasintheheadteacher’sname.Only40%oftheCBMCsindicatedthattheykeptrecordsofbankdepositsandwithdrawals,ofwhichonlyhalfcouldshowanyrecordsfromthelastyear.Theuseofacashbookorbankaccountwasmorecommoninurbanareas,perhapsduetoeasieraccesstofinancialservices.IncaseswheretheCBMCwaschairedbyacommunitymembertheCBMCsweretwiceaslikelytouseabankaccounttostorefunds,ascomparedtoCBMCsthatwerechairedbytheproprietororheadteacher.ThisevidencesuggeststhatCBMCsneedfurthertrainingandsupporttoimprovetheir

0% 10% 20% 30% 40% 50% 60% 70% 80%

CBMCresponsibilities

Enrolmentdrives

Communitysensitisationaboutgirls'education

Wholecentredevelopment planning

Communitymobilisationforschoolplanning

Resourcemobilisation

Financialmanagement/accounts

Communitysensitisationaboutboys'education

Lessonobservationsandteachingmethods

Schoolenvironment

Schoolintegrationtechniques


©EDOREN 187

financialmanagementpractices,whichisunsurprisingsinceonlyaboutaquarteroftrainedCBMCsreportedbeingtrainedonfinancialmanagement(Figure64).

CBMCmemberswerequiteextensivelyinvolvedinschoolmonitoringactivities.AlmostallCBMCchairsindicatedhavingvisitedtheschooltomonitoritsperformanceduringthelastterm,withtheaveragenumberofreportedvisitsbeingseven.Thefrequencyofmonitoringishigherwhenthechairistheheadteacher,butingenerallargedeviationsarenotobservedwhenthechairisacommunitymember.Furthermore,over80%ofCBMCsreportedlymonitoredpupilandteacherattendanceandtookactiontoimprovepupilandteacherattendanceduringthelastterm.Thisholdstruewhendisaggregatingbystate,trainingstatus,andbytheCBMCbeingchairedbyacommunitymember.Inaddition,oftheCBMCsthathadaWCDPorwereintheprocessofdevelopingone85%reportedmonitoringactivitiesagainsttheWCDP.ItisimportanttopointoutthatinalmostallthecasesthevisitsoftheCBMCmemberstoschoolswerenotrecordedinthevisitors’bookoranyotherlog.Hence,thesevisitscouldnotbeverifiedbyevidence.

TherewasgreatervariationinthemonitoringactivitiesofCBMCmembersregardingqualityofteaching.ItappearsthattrainedCBMCmembersmonitorteachingqualitymoreoftenincaseswheretheCBMCchairistheheadteacherorproprietorthanwhenthechairisacommunitymember.In92%ofthecasesinwhichtheheadteacherorproprietorwasthechairorco-chairteachingqualitywasmonitored.Thiswasonlyhappening82%ofthetimeincaseswheretheCBMCwaschairedbyacommunitymember.MonitoringofteacherqualitywasmoreprevalentinNigercomparedtoBauchi,despiteBauchihavingmoreCBMCschairedbytheheadteacherorproprietor–perhapsbecausemoretraininghadtakenplaceinNiger.

TheCBMCmembersreportedmonitoringteacherattendancebyvisitingtheschoolandclassesduringlessons.Actionstakentoimproveteacherattendanceincludeddiscussingreasonsforabsenteeismwithteachersandmotivatingthemtobepresentregularly.Forpupilattendance,CBMCmembersreporteddiscussingreasonsforabsenteeismwiththepupilsandparents,andencouragingparentstosendtheirchildrentoschool.Itisinterestingtonotethatsecurity-andtransport-relatedissueswerenotmentionedatallduringanyofthediscussionsaroundabsenteeism.ActionstakenbyCBMCmemberstoaddresspupildropoutlargelyentailedahighlevelofinvolvementwiththecommunity,asindicatedbythedirectcommunicationwithparentsandmeetingswithcommunitymembers.


©EDOREN 188

Figure65: ActiontakenbyCBMCstoaddresspupildropout,bystate

4.5.1.3 Resourcemobilisation

CBMCsaremeanttocontributetothemobilisationofresourcesforschoolimprovementandtheimplementationofschoolactivities.Thedatashowthat65%oftheCBMCsreportedmakingeffortstomobilisecashtosupportschoolimprovementduringthelastschoolyear,ofwhichover80%successfullymobilisedfunds.InruralIQSsproportionallymoreCBMCstriedtomobilisecashcomparedtourbanIQSs.IQSschairedbycommunitymemberswerelesslikelytomakeeffortstomobilisefunds,butequallylikelytosuccessfullymobilisefundsamongtheCBMCsthatmadeaneffort.ThereseemtobelittledifferencesinsuccessfulcashmobilisationdependingonwhoistheCBMCchair.CBMCsinBauchiappearsomewhatmoresuccessfulinmobilisingcashthenthoseinNiger,thoughthesamplesizeforthisisquitesmall.AlloftheCBMCsthatweresuccessfulinmobilisingcashreportedmobilisingcashfromthecommunity,withaverysmallsharereferringtocontributionsfromthegovernment,NGOsandotherexternalagencies.

TheamountmobilisedbyCBMCsduringthelastyearvariedgreatlyacrossschools.Theabsenceofwrittenrecordsforthismoduleofthesurveymadeitchallengingtocollectcomplete,accurateinformationattheschoollevel.50%oftheCBMCsthatwereabletomobilisefundsraisedlessthanNGN16,500duringtheyear.TherestraisedbetweenNGN20,000andNGN50,000.Therearetwooutliersinthisdataset,wheretheCBMCraisedNGN151,000andNGN285,000duringthelastschoolyear.ThemedianamountraisedwasNGN20,000.Thoughresourcemobilisationfromthecommunityisproductive,whencomparedagainstthemagnitudeofthefinancialcontributionofschoolgrants(seebelow)itisclearthatthefundsmobilisedbytheCBMCsarenotasignificantpooloffundingforschoolimprovementactivities.Thissuggeststhatthereisstillroomtoaugmentcommunityinvolvementwithgovernmentsupport.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Communitysensitisationbygoing

doortodoor

CBMCorganisedmeetingswith

communitymembers

Directcommunicationwithparents

Pupilattendancemonitoring

Pupilcounselling

Bauchi Niger


©EDOREN 189

Figure66: AmountmobilisedbyCBMCs

Inadditiontoresourcesraisedthroughactivefundraising,around20%oftheCBMCsalsoreceivedfundsfromvarioussources.ThemostcommonsourcesoffundsreceivedwereUNICEF,SAMEorcommunitymembers.ThequestionnairealsoaskedCBMCmembersdirectlyaboutthereceiptofaGEPorUNICEFschoolgrant.Thefigurebelowshowsthatabout40%oftheCBMCsinBauchireportedhavingreceivedaGEPschoolgrantduringthelastschoolyear,comparedtojustunder30%oftheCBMCsinNiger.TheamountsthatwerereportedlyreceivedbyCBMCsweremostlyNGN150,000orNGN120,000;theformerwasmostlyrecordedinNiger,whilethelatterwasrecordedinBauchi.ThemaximumamountreportedwasNGN285,000,whiletheminimumamountwasNGN6,500.Thisincludesfundsreceivedfromallsources,thoughthelargestsharestemsfromUNICEForGEPgrants.

Figure67: ReceiptofGEPschoolgrant,bystate

0

50000

100000

150000

200000

250000

300000Am

ount(N

aira)

42%

28%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Bauchi Niger


©EDOREN 190

Overall,CBMCsinBauchiraisedmorefundingthanCBMCsinNiger.TheshareoffundsraisedfromthecommunityasashareoftotalfundsraisedwasalsohigherinBauchi.ThissuggeststhattheBauchiCBMCsweremoresuccessfulinmobilisingresourcesfromthecommunity.However,CBMCsinNigerhadaccesstomorefundingoverallthanCBMCsinBauchi,afteraccountingforfundsreceived.Secondly,almost30%oftheCBMCshadaccesstonofundsduringthelastyear,eitherduetonoeffortsbeingmadebytheCBMCtoraisefundsorduetounsuccessfulefforts,coupledwithnofundsreceivedfromanysource.ThissuggeststhatasevereresourcegapexistsinalargeshareofIQSs.Finally,around40%oftheCBMCsalsomobilisednon-cashresources,whichcameexclusivelyfromthecommunity,andweremorefrequentlyutilisedinBauchithaninNiger.

4.5.1.4 Communityinvolvementandsupportfortheschool

TheevidencediscussedsofarsuggeststhatthereiscommunitysupportforIQSs.ThisisexhibitedbycommunitymembersbeingpartoftheCBMCandchairingtheCBMC,aswellascommunitymembersdonatingmoneyandotherresourcesforschoolimprovementactivitiesthroughtheCBMC.Furthermore,CBMCmembersarereportedlyextensivelyinvolvedinschoolmonitoringactivities.

TheCBMCsseemtoactivelyreachouttothecommunity.Over80%statethattheyhaveraisedawarenessaboutthevalueofnon-religiouseducationinthecommunityduringthelastschoolyear,60%ofwhichreportedlyorganisedmeetingswithcommunitymembers,39%haddirectcommunicationwithparentsand26%organisedmeetingswithreligiousandtraditionalleaders.Aroundthree-quartersofCBMCsindicatedthattheyhaveraisedawarenessaboutthevalueofnon-religiouseducationspecificallyforgirlsinthecommunity.

CommunityfinancialsupportwasgreatestinschoolswheretheCBMCwaschairedbyacommunitymemberwhowasnottheheadteacherorproprietorattheparticularschool.SuchCBMCsmobilisedagreateramountfromthecommunity(anaverageofNGN61,000,comparedtoNGN21,000),mobilisedalargershareoftotalfundsfromthecommunity(100%versus97%),andwererecipientsofmorefunds(averageofNGN154,000,versusNGN88,000).Theywerealsomorelikelytohaveaccesstofundsduringthelastyear,andweremorelikelytohaveutilisednon-cashresourcesraisedfromthecommunity.

However,morecommunityinvolvementwithintheCBMCisnotalwayscorrelatedwithimprovedactivitiesattheschoollevel.Forexample,whentheproprietororheadteacheristhechairthereseemstobeahigherlikelihoodoftheCBMCmeeting,aswellasahigherattendancerateatthemeetings.Inaddition,althoughCBMCsareatleastaslikelytomonitorteacherandpupilattendanceiftheyarechairedbyacommunitymember,CBMCschairedbytheheadteacherorproprietoraremorelikelytotakeactiontoimprovepupilandteacherattendance.Whenaskedaboutthereasonforinactivity,CBMCchairslargelyreportedthatteacherandpupilattendanceisnotaproblematthisschool.AlthoughtheCBMCschairedbyacommunitymemberdonotappeartofaceinstitutionalchallengesinbeinginvolvedwithschoolmanagementactivities(asindicatedbythefrequencyofmonitoringactivities)theydoappearlessequippedtotranslatemonitoringintoaction.BuildingcapacityinthisregardshouldbeapotentialfeatureofCBMCtrainingundertheprogramme.

4.5.1.5 Schoolenvironment

Schoolenvironment

Theschoolenvironmentplaysanimportantroleinthelearningexperienceofthepupils,aswellastheteachingexperienceofthefacilitators.Weassesstheschoolenvironmentforchildrenalongthe


©EDOREN 191

linesof‘soft’infrastructure,whichincludesgroupsforpupilstointeractwithoneanother,alibrary,andaplayground,aswellas‘hard’infrastructure,whichreferstophysicalinfrastructure.

TheIQSsinoursamplearealmostuniversallydeficientintermsofsoftinfrastructure.Only3%oftheschoolshadalibrary,definedas‘acollectionofbooksotherthancoursebooks’,withnosharpcross-statedifferences.Onlyaboutone-thirdoftheschoolshadaplayground.Whereschoolshadaplayground,girlsalmostalwayshadaccesstoit.InNigerandinurbanareasplaygroundswerelesscommon.Only3%oftheschools,whichamountstoonlytwoschoolsinthesample,hadgroupswherechildrencouldcometogetherandsharetheirconcerns.Thesegroupstooktheformofamothergroupandateacher–studentassociation.GiventhelowmembershipofchildreninCBMCsthisraisesagainthequestionofhowchildren’svoicescanberepresentedindecisionstoimprovetheschoolenvironment.

Intermsofphysicalinfrastructure,schoolswerealsoinapoorcondition.Accordingtothesurveydata90%oftheschoolsare‘inneedofmajorrepairs’,withalloftheschoolsinBauchiinneedofrepairsand80%oftheschoolsinNiger.Only40%oftheschoolshadanelectricityconnection,withmoreschoolsinNigerhavingaconnectionthanschoolsinBauchi,despiteBauchihavingahigherrepresentationofurbanschoolswhereelectricityconnectionismorelikelytobeavailable.Theshareofschoolsconnectedtothegridthathadelectricityonthedayofthesurveywasevenlower,at30%.Accesstodrinkingwaterwaslimitedaswell,withonly30%ofschoolshavingsomesortofwatersource.Oftheschoolsthathadawatersource,around90%hadwateravailablefromthesourceonthedayofthevisit.Acrossbothstates,therewasanaverageoftworoomsbeingusedforclassesonthedayofvisit.However,around20%oftheIQSshadnoclassroomstructureandanadditional30%oftheschoolsonlyhadoneclassroom.TheschoolsoperatingwithoutanyformalclassroominfrastructurewereconcentratedinBauchi,whereasalargeshareofschoolsinNiger(about50%)onlyhadoneclassroom.

Lessthan25%oftheschoolshadafunctioningtoiletforpupils,whileonly8%oftheschoolshadaseparatefunctioningtoiletforgirls.However,alltheschoolsthathadmorethanonefunctioningtoiletforpupilshadaseparatetoiletallocatedforfemalepupils,whichsuggeststhatwhenthereisinfrastructureavailable,schoolmanagementisawareofthisneed.22%oftheschoolshadmadesecurityarrangements,suchasafenceorboundarywall.AlargershareoftheschoolsinNiger(30%)hadafenceorothersecurityarrangementscomparedtoonly10%oftheschoolsinBauchi.

Thissuggeststhattheinfrastructuresituationisquitepoorinboththestates,andchildrendonothaveaccesstobasicinfrastructuresuchasfunctionaltoiletsanddrinkingwaterattheschool.TheinfrastructurerepairneedsofschoolsareshowninFigure68.ThissuggeststhatthetwostateshavesimilarneedsbutthatmoreIQSsinBauchiareinneedofrepairs.WhileIQSslocatedinurbanareashavebetteraccesstoelectricityandwaterandmoreIQSshavefunctioningtoilets,theinfrastructureisgenerallyingreaterneedofrepair.InIQSswheretheproprietoristheheadteacher,theinfrastructuresituationisrelativelyworseoff.Theneedforschoolrepairswasalsoexpressedbyfacilitators.Inthefacilitatorinterviews,over50%ofthefacilitatorsagreedwiththestatementthat‘Itistoodifficulttoteachinthisschoolbecausethebuildingisinapoorcondition’.Giventhatabout30%ofCBMCsreportedhavingnoaccesstofundsinthelastschoolyear,thelikelihoodofrapidinfrastructureimprovementsislimitedunlessexternalsupportisprovided.Schoolsalsoneedadditionaltrainingontheimportanceofsoftinfrastructure.ThesecouldbeareasthattheGEP3interventioncouldfocusonattheschoollevelastheschoolenvironment(bothsoftandhardinfrastructure)influencespupillearning.


©EDOREN 192

Figure68: Infrastructurerepairneeds,bystate

Investmentactivities

CBMCsspentanaverageof80%ofthetotalfundsraisedduringthelastschoolyearonschoolimprovementactivities.UtilisationoffundswashigheriftheheadteacherorproprietorwasthechairoftheCBMCversusifacommunitymemberheldthisposition.FundsutilisationwasalsohigherinBauchithanNiger,thoughthetotalvalueoffundsavailabletoschoolsinBauchiwaslower.Unsurprisingly,schoolinfrastructureinBauchischoolswaspoorerascomparedtoschoolsinNiger,asalltheschoolsinBauchiweremarkedasbeinginneedofmajorrepairs.Overall,thephysicalconditionofschoolsinNigerwasnotsignificantlysuperior.Thiscouldbeduetothefactthattheactualamountoffundsavailableattheschoollevelisnotsubstantialandisdividedacrosstoomanyinvestmentcategories.Thefigurebelowsuggeststhatthismaybeaplausibleexplanation.ThetypeofinvestmentsthatweremostfrequentlyreportedbytheCBMCareinfrastructureworks,suchastheconstructionofnewbuildings,renovationofthebuildingorconstructionofatoiletandwaterfacilities.Secondinorderistheprocurementofteachingandlearningmaterials.Itisnoteworthythatdespitefacilitatorsreceivinglittleremunerationandthefactthatoftenonlyafewfacilitatorsareavailable,fundsarerarelyinvestedinfacilitatorrecruitmentorremuneration.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Windows

Rooforceiling

Classroomwalls

Fenceorboundarywall

Floor

Furniture

Watersource

Toilets

Niger Bauchi


©EDOREN 193

Figure69: Investmentcategories,bystate

Girl-friendlinessoftheenvironment

Thediscussionoftheschoolenvironmentsofarpresentsstrong,consistentevidencethattheenvironmentinIQSsisnotchild-friendlyorgirl-friendly.Schoolshaveasevereshortageofhygienefacilitiesforpupils,andespeciallyforgirls.Inaddition,bothgirlsandboysarebarelyrepresentedintheCBMC.TherepresentationofchildandgirlchildmembersintheCBMCwasnegligible,asonly17%ofCBMCshadachildmember(andagirlchildmember),andchildrenmadeuponlyaround2%oftotalmembership.87%ofCBMCshadatleastonefemalemember.AlthoughtheattendancerateatthelastCBMCmeetingoffemalememberswasslightlylowerthantheoverallattendancerateofCBMCmembers(35%forfemalemembers,versus41%overall),forNigerthefemalememberattendanceratewasonaparwiththeoverallattendancerate.ForBauchi,thefemaleattendanceratewasmuchlowerthanthestateaverage.ThissuggeststhatthereisvariedparticipationoffemalemembersinCBMCfunctionsacrossthetwostates.

Therearealmostnoschoolswithinteractivegroupsforpupilstodiscusstheirproblemsandexperiences.ThisindicatesthatifGEP3wantstopilotG4GgroupsinBauchiorNigertosupportgirls’retention,groupsforpupilswillneedtobeestablished.Inaddition,thereisrelativelylowfemalerepresentationintermsofteachers,headteachers,andproprietors,thoughtheabilityoftheinterventiontoinfluencethelattermaybelimited.

Despitealloftheabove,itappearsthatgirlsarenolesslikelytobeattendinganIQScomparedtoboys.Thiswasindicatedduringthemanualcountingofpupilspresentonthedayofthevisit,aswellasduringthelessonobservationsandenrolmentregisters,thoughpoorenrolmentandattendancedatamakesitdifficulttoobtainthesecomparisonsacrossallthesampledschools.Inaddition,thefactthataround90%oftheCBMCshadatleastonefemalememberandfemaleattendanceratesatCBMCmeetingswerehigherthantheoverallaveragesuggeststhatculturalperceptionsaboutfemaleinvolvementintheseactivitiesareperhapsnotextremelyrestrictive,andthisholdstruemoresoinNigerthaninBauchi.HencethereiscapacityforgainstobemadeandtoincreasetheparticipationandrolesofwomeninthesebodiesandanyothergroupsthatareformedattheschoollevelundertheGEP3.

0%5%

10%15%20%25%30%35%40%

Bauchi Niger


©EDOREN 194

4.5.1.6 Governmentmonitoring

BauchiIQSsseemtohavegreaterinteractionwithvariousgovernmentbodies.TwiceasmanyheadteachersinBauchireportedtrainingconductedbySUBEBandtheLGEAthanheadteachersfromNiger.ThisholdstruefortrainingreceivedbyCBMCmembersaswell.Moreover,agreatershareofschoolsinBauchi(44%)reportedmonitoringvisitsduringthelastterm,comparedtoschoolsinNiger,whereonly33%oftheschoolsreportedmonitoringvisits.ThisdoesnotseemtobeassociatedwiththefactthatBauchicountsmoreurbanIQSsbecauseurbanIQSsinthesampleareaslikelytohavereceivedamonitoringvisitasruralIQSs.ThetotalnumberofmonitoringvisitswasalsogreaterinBauchithanNiger,withanaverageof4.2visitslastyear,comparedto3.1visitsinNiger.Thesourcesofmonitoringvisitswerecomparableacrossstates,withLGEAandSUBEBofficialsmonitoringschoolsinbothstatesandSAMEonlyvisitingschoolsinNiger.

Figure70: Sourcesofmonitoringvisitstoschools,bystate

4.5.1.7 Conclusionanddiscussion

TheevidencesuggeststhatexposuretoGEP’seffortsofimprovingschoolmanagementandgirl-friendlinessisvariedacrosstheIQSswithinandacrossstates.Thebaselinelevelandconditionofschoolinfrastructuresuggeststhereisroomforimprovementinthewaytheseschoolsattempttoattractgirlsandretainthem.Lackofresourcesappearstobeakeycorrelateofthis.However,itisencouragingtonotethatfinancialmanagementisformalisedtoalargeextentintheseschools,withCBMCsusingabankaccountinthenameoftheschooltomanageresources.Additionaltargetedtrainingonfinancialmanagementandresourceutilisationwillservetoimprovetheschoolmanagementaswellasthestateofgirl-friendlinessintheseschools.


Thequalitativecasestudiesexplorethequestion:‘Willimprovedschoolmanagement,increasedcommunityinvolvement,andadditionalresourcescontributetoanimprovedlearningexperienceforchildren,particularlyforgirls?’ThisanalysisaidsinthediscussionofContributionClaim2oftheGEP3ToC:GEP3’sIQSScontributestoanimproved,girl-friendlylearningenvironment.Thus,the

0%

10%

20%

30%

40%

50%

60%

70%

80%

LGEA SAME SUBEB MinistryofEducation

UBEC TrainersfromSBMC/CBMC

training

NGOs,donors,otherexternal

agencies

Don'tknow

Bauchi Niger


©EDOREN 195

qualitativestudyexploresthemesaroundleadership,managementandresponsibility,aswellasperceptionsaroundwhatconstitutesagirl-friendlyschoolenvironment.ThroughFGDsandKIIs,thediscussionsalsoconsideredsocialrelationswithinthecommunities,bothbetweenschoolmanagementstakeholders,andbetweenpupils(inparticulargirls)andschoolleadership,toanalysetheextenttowhichschoolplanningprocessesactivelyconsidergirls’voices.

4.5.2.1 Schoolmanagement–actors,rolesandresponsibilities

OneofthekeyoutputsofGEP3isenhancedschoolleadershipandmanagement.Thequalitativefindingsshowthatvariousstakeholdersareinvolvedintheintegrationprocess.CommunityKIsidentifycommunityleaders,proprietors,imams,headteachers,teachers,parentsandtheCBMCasbeingessentialforsuccessfulintegration.Asdiscussedinthesection‘Schoolleadership:Roleofcommunityleadersandstakeholdersinmanagementonpedagogicalleadership’,itisimportanttogainabetterunderstandingofhowschoolleadersandmanagementunderstandandenacttheirrolesandresponsibilities.

Schoolleadership:Roleofcommunityleadersandstakeholdersinmanagement

IntheIQScontext,thedelegationofresponsibilitiesforintegrationandrunningtheschoolisnotclear.Instead,stakeholdersperceiveseveralactors–whoalsoperceivethemselves–tobepartofthesuccessfulrunningoftheschool,inmanycasesmeaningthatmanagementbecomescollective.Thisinturncreatessomeconfusionaroundwhoisinchargeofwhat,whoseapprovalisneededand,assuch,‘whohasthelastword’whenitcomestodecision-making.Asinthediscussionaroundpedagogicalleadership,ingrainedsocialrelationsinthecommunitiesattimesmeansthatthemostqualifiedpersonisnotalwaystheonewhoisresponsiblefordecision-makingandtasks.

Schoolleadershipdoesnotonlyconsistofthoseactivelyinvolvedwiththeschool,butincludesstakeholderssuchascommunityleadersandimams.TheembeddednessofIQSswithinpre-existingcommunitystructuresmeansthatthesestakeholdersoftenhaveinfluenceovertheschool’svision,andholdrespectedleadershippositionsintermsofguidingcommunityacceptanceandthesensitisationprocessonissuessuchasintegration.Assuch,theattitudesofthesekeyfigureshaveahighdegreeofinfluenceovercommunities’acceptanceofintegration.InallsixIQSsvisited,communityleadersshowedgeneralacceptanceofintegration,andwereinmanycasesactivelyinvolvedinschoolmanagementthroughpositionsontheCBMC.

‘They[theCBMC]usuallyconsultmeonissuesconcerningtheschoolandtheeducationofthechildren.Theyareawarethatoncealeaderacceptssomething,hecanconvinceothermembersofthecommunityandtheideawillhavewideracceptability.’(Communityleader,BauchilowerperformingIQS).

InNigerlowerperformingIQS,thequalitativeteamwasunabletointerviewthecommunityleaderduetointernalpoliticalrivalriesbetweentheMallam(theheadteacher)andthecommunityleader.TheMallamhadrecentlycomeoutontopinadisagreement,andthusaskedtheteamnottointerviewthecommunityleaderasthismightnegativelyaffecttheintegrationprocess.Itwasuncleartowhatextentthiswouldhavebeenthecase;however,itgavesomeinsightintotheimportanceofunderstandinginternalpowerrelationsforsuccessfulinterventions.ItwasclearthatinthiscasetheMallamhadahighersocialstandingamongstcommunitymembersandassuchhadbeenabletoproceedwithintegrationregardlessofsupport(orlackthereof)fromthecommunityleader.However,inotherIQScontextstheattitudesofcommunityleadersappeartohavesignificantimplicationsforthesuccessoftheintegrationprocess.


©EDOREN 196

Respondentsmainlyunderstandtheroleoftheheadteacherinschoolmanagementasaleadershipresponsibilitythatinvolvesadvocatingforintegrationandgirls’education.Headteacherscarryoutsomebasicfunctionstofulfiltheseresponsibilities,throughspeakingtothecommunity156andencouragingparentstosendtheirchildrentoschool.Inthismanner,teacherssaytheyreporttotheheadteacherwhentheyhaveissueswithpupilattendance,andtheheadteachercommunicatesthistotherestofthecommunity.

‘Onissueslikepupilsabsenteeismorlatecomingwecomplaintohimandhegiveshisfullsupporttoaddressthematterbyannouncinginthemosqueduringprayers,orhesendsamessagetothecommunityleadertoaddresstheparentsandletthemknowthattheteachersarecomplainingaboutpupilsattendanceandbythatmeansweseechanges.’(TPD,BauchihigherperformingIQS)

‘Everysocietyhasitscultureandtradition,sothiscommunityhasaculturethatdoesnottoleratetheintegrationofformaleducationwiththeQuranic.Butatthemomentthereareseriesofawarenesscampaignbythevillageheadandmyselfviathemosqueandweareenjoyingtotalacceptancenow.’(Mallam,BauchitypicalIQS).

HeadteachersdoholdauthorityinmanyIQSsandarethemainpeopletowhomteachersreport.ThestrengthoftherolerelatestothefactthattheheadteacherisoftenalsotheMallam,andattimesalsotheproprietor,andthusholdsahighstatuspositionthatisnotnecessarilyassociatedwith,orlimitedto,‘headteacher’.Therespectandtrustgiventothisinstitutionwithinthecommunity–someofwhichappeartohavebeenestablishedover100yearsago–seemstobeacriticalsuccessfactorintheintegrationprocess,withthepositionoftheMallam(oftenahereditaryrole)beinghighlyimportant.

Headteacherstendtohaveprimaryresponsibilityforpupildiscipline,withteachersreportinganymisbehaviourtotheMallam(headteacher)forhimto‘sanction’pupils.Theheadteacherisalsoresponsibleforencouragingfacilitatorstocometoschool,andisoftenthepersonwhohasinitiallyaskedamemberofthecommunitytotakeontheteacher/facilitatorrole.

Insomecases,157themajorityofthefacilitatorswerealsoformerpupilsoftheMallam.Thus,theheadteacherhassomeinfluenceovermonitoringfacilitatorattendanceandmanagingfacilitators.Asdiscussed,pedagogicalleadership,thelevelofqualificationsandfacilitatorexperienceamongstheadteachersvariessignificantlyacrossIQSs.Assuch,thelevelofleadershiponacademicachievementandfacilitatorguidance(fromapedagogicalstandpointforsecularlearning)isgenerallyweak.Still,amoreinformalprocessofmentoringappearstobetakingplaceinsomeIQSs158.InviewofsomeoftherelationshipsthatMallams/headteachershavewithfacilitatorswhoareex-pupils,theirpositionasrespectedQur’anicteachersmeansthatfacilitatorsconsidertheiroverallguidancetobeimportantandworthlisteningto.

InsomeIQSstheheadteacheristhemainpersonresponsiblefordeliveryoftheformalcurriculum.Thus,theheadteacheriseffectivelysupposedtomonitorthemselves,intermsofattendanceandperformance.Thesub-sectionbelowonCBMCs’capacitydevelopmentfurtherdiscussesthelackofclearstructurearoundmonitoring.

156ThisisincaseswheretheheadteacheristheMallam.Understandingsof‘headteacherresponsibilities’arethuscollapsedwiththoseoftheMallam.157BauchihigherperformingandtypicalIQS,NigerlowerperformingandtypicalIQS.158ObservedinBauchihighandtypicalperformingcases,andNigerlowperformingcase.


©EDOREN 197

Whiletheresponsibilitiesofkeycommunityfiguresareattimessymbolic,theyoftendoholdpositionsontheCBMC,andthusactivelyformpartoftheschoolmanagement.TrainingforCBMCsmayhavelessimpactinIQSswherecommunityleadersarepartoftheCBMCbuthavenotbeentrainedbyGEP3.Aspreviouslydiscussed,individualswithexperienceofformaleducationand/orschoolmanagementmaynotalwaysholdpositionsofimportance.TheheadteacherinNigerlowperformingIQS,forexample,takesontheresponsibilitiesofpedagogicalleadershipandchairstheCBMC,inspiteofnothavinganyformaleducation.Furthermore,religiousleadershipappearstoconferhighersocialstandingthanformaleducationalexperience.Thismaylimittheoverallcontributionofsecularheadteachertrainingtoimprovedschoolmanagement.TheimportanceofexistingsocialstructuresandnormswithintheIQScommunitiesmeanthatGEP3willneedtoconsiderhowtoensurethatallrelevantstakeholdersareincluded,andfortrainingtoespeciallytargetthosewhoholddecision-makingauthoritywithincommunities.

CBMCs:Characteristicsandfunctionality

AlltheIQSsvisitedhaveanestablishedCBMC.MostschoolscreatedCBMCsbecauseoftheintegrationprocess.DuringtrainingstakeholdersinthecommunitywereinstructedtostartaCBMCtosupportintegrationandgirls’education.IntheNigerlowerperformingIQS,CBMCmemberssaythattheyhavenotreceivedtraining.Instead,thecurrentCBMCsecretary,whohaschildreninthelocalpublicprimaryschoolandhadseenhowtheSBMCwasworkingthere,initiatedtheCBMC.Assuch,therewasaspillovereffectfromtheimplementationandsupportofthepublicprimarySBMCtotheIQSinthisparticularcommunity.

‘ThereasonhowtheCBMCcameaboutwasaftertheschoolwasbuiltitwasbasicallyQur’anicschool.Thentherewasnothinglikecommittee,itwasjusttheproprietorthatwasinchargeoftheschool.Wecalledontheproprietoranddiscussedwithhim,whichwashowtheCBMCcameabout.Youknowineverythingthereisneedforhavinganassociation,inschoolifthereisassociationandchallengesariseswewillknowhowtogoaboutit,tosolveitbutifthereisnoassociationthingsmightnotmovesmoothly.SoforthereweappointedtheChairman,Secretary,Treasurer,etc.Weare12membersinCBMC.ThereasonwhyweestablishedthisCBMC,firstly,istosolveanychallengesthatariseintheschool.ThisisthefirstreasonwhyweestablishedtheCBMC;itwasn’texistingintheinitialstage.’(CBMC,NigerlowerperformingIQS)

However,thefunctioningoftheCBMCvariessignificantlyacrossschools.InNigerlowerperformingandtypicalIQSs,thoughtheCBMCmeetsweeklyitisunclearwhatthepurposeofthesemeetingsis.Overall,theobjectivesandroleoftheCBMCarevague(‘encouragetheintegrationprocesses’and‘planthefutureofthisschool’),especiallyintermsofplanningtowardsachievingtheseobjectives.Still,thisprovidesastartingpointforcapacitybuilding,withCBMCs’attitudestowardintegrationappearingtobemostlypositive.

CBMCmemberstendtobeamixtureofkeystakeholderswithinthecommunityandparents.Insomecases,facilitatorsalsositontheCBMC.InallIQSsincludedinthequalitativestudytheMallamholdsakeypositionintheCBMC,suchaschairperson.ThisisimportantsincetheMallamisalsooftentheproprietorand,assuch,isactivelyinvolvedinbothresourcemobilisationandmonitoring.ThereisthusaquestionaroundpotentialconflictsofinterestwithintheIQSstructure,and,furthermore,theextenttowhichmonitoringcanbeeffectivewhenrolesandresponsibilitiestendtobelargelyoverlapping.Forexample,thepresenceoftheMallam/headteacher/proprietorwithinaCBMCcouldbeproblematicincaseswherethatindividualisquitedominant,aswasobservedintheBauchitypicalIQSFGD.Decision-making–suchasregardingtheuseoffunds–couldeasilybe


©EDOREN 198

heavilyinfluencedbysuchanindividual.Accountabilityandtransparencycanthereforebecomeissues.

ThemaincriteriaforbeingamemberoftheCBMCis‘forthepersontoberesponsibleand(ableto)adviseandimpactpositivelywhentheneedarises’(CBMC,BauchihigherperformingIQS).Thisdoesnotmeanthatonehastoholdapositionofpowerwithinthecommunity,onlythatoneiswillingtomaketheeffortandconsiderthevaluesoftheCBMC.OnedoesnothavetohavechildrenintheIQS,butoneneedstounderstandtheimportanceofeducationandagreewiththeprogramme(GEP/GEP3).

‘Itsometimeshappenthatforyoutobeamember,itmustbeseenthatyouarecapableandhavebeenputtingineffortstowardsimprovingtheprogramme.Itcouldbethatyourlovefortheprogrammeisobvious.Thatway,youcanbeco-optedintoit.’(CBMC,BauchilowerperformingIQS)

CBMCsmostlyconsiderwomentobeanimportantresourceoftheCBMCastheyhavethepotentialtoconvincemotherstosendtheirchildrentoschoolandtoadvocatetheobjectivesoftheCBMCtowomeninthecommunities.NotalloftheCBMCsvisitedhavefemalemembers,andinthosethatdothemembershipappearstobehighlydisproportionate,withoneortwomembersoutof12–17beingfemaleinthelowerperformingandtypicalperformingIQSsinBauchi,andinthelowerperformingschoolinNiger159.WhilstinthehigherperformingIQSsinbothstateswomencurrentlyholdaroundathirdofmembershippositions,theNigertypicalIQSdoesnothaveanyfemalemembersatpresent.Assuch,therepresentationofwomenonCBMCsvariessignificantlyacrossIQSs.CBMCmembersdescribedchallengesingettingwomentojointheCBMC.Forexample,inNigerlowerperformingIQSwomenaregenerallynotactivelyapartofthepublicsphereoractivitiesinthecommunityaftertheygetmarried.Thus,thecommitteehadrecentlylostafemalememberasshehadgottenmarried.ThismeansthatthereiscurrentlyonlyonefemalememberontheCBMC,andsheisthefemalefacilitatoroftheschool.Theyarenotsurehowtoconvinceanotherwomantojoinoncethisfacilitatorgetsmarried.Thus,genderrolesandresponsibilitiesoutsideofschoolmanagementaffecttheextenttowhichwomenformanactivepartoftheCBMC.Thisisinspiteofrecognisingfemalemembersasbeingimportantinordertoachievetheobjectivesoftheintegration.ThismayprovetobeachallengefortheadvocacyaspectoftheCBMC,anditwillbeimportanttoconsiderhowtoreachwomenincommunitieswheretherearenowomenbeingtrainedbyGEP(eitherthroughfacilitatortrainingorthroughbeingmembersoftheCBMC).

ThetypeofIQSmayfurtheraffectthefunctionalityoftheCBMC.InallthequalitativecasestheIQSsarenotmobile,andappeartobeactivelyengrainedinthecommunity.CBMCsconsiderthisanimportantaspectofwhytheyareabletoenacttheirresponsibilities,sincecommunitymembersperceivethemtoberepresentativeofthecommunityinwhichtheyaresituated.WhereanIQSismobile,thismayaffecttheextenttowhichaCBMCcanbeofuse.ItisalsoimportanttonotethatifanIQSismobile,girls’learningwilllikelyceasecompletely(asgirlsdonottravelwiththeMallam)unlessitisarrangedthattheIQSfacilitatorremainsbehindandcontinuesteachingthegirlsandremainingboyswhodidnotmove.Inthatinstance,theIQSfacilitatorwouldlikelybeonher/hisownwithouttheheadteacher/Mallam.TheMallamnotbeingpresentcouldweakentheCBMC’seffectiveness.

159ItwasdifficulttoestablishtheexactmembershipandgenderdistributionofCBMCsasschoolsapartfromNigerhighperformingIQSdidnotkeepamembersregistry.


©EDOREN 199

CBMC:Perceptionsofmaintasks,responsibilitiesandcapacityconstraints

InthesixIQScases,CBMCmembersgenerallyunderstandtheirroleandresponsibilities,buthaveweakcapacitytoenactthese.ThequalitativeresearchfoundthatCBMCsseetheirrolesasbroad–coveringawiderangeofresponsibilities,fromresourcemobilisationtoadvocacyoftheimportanceofeducation.CBMCsmainlyseetheirroleasinvolvingidentifyingwhychildrenarenotenrolledandlearning,andto‘solve’theseproblems.However,CBMCsarguethattheylackthefundstoseethesesolutionsthroughandstate‘wecandobetter’.Moreover,CBMCsappearwillingtotakeonnewskillsandtasks.SeveraltimesduringFGDs,CBMCmemberswouldsay‘wewilldothismorenowsinceyouhaveaskedusit’(NigerlowerperformingIQs),inresponseto,forexample,questionsonwhethertheymeetwithparents.CBMCsandotherstakeholdersperceivethemainresponsibilitiesoftheCBMCtobe:

• increasingawarenessoftheimportanceofeducation;• ensuringattendanceofpupilsandteachersthroughproblem-solvingandmonitoring;• resolvingproblemstogetherwithcommunitystakeholders;and• improvingschoolinfrastructure.

AmainresponsibilityoftheCBMCisincreasingtheawarenessoftheimportanceofeducationforbothboysandgirls.SincetheCBMCsvisitedaspartofthequalitativestudyallhadkeycommunitystakeholdersasmembers,inmanyregardstheyoperateas‘minivillagecouncils’.CBMCsthusdiscussallproblemsrelatedtoeducation(includingreligiouseducation).Sincecommunityandreligiousleadersarekeyincallingmeetingswiththecommunityandinadvocatingfortheintegratedcurriculumandgirls’education,theinvolvementofthesefiguresandtheiracceptanceofproposedchangesisessentialfortheCBMCtobeabletofulfilitsrole.

‘Wevisitanddiscusswiththem,tellingthemthatifweallowthechildrentogetenrolledittranslatedintosomethingmeaningforthewholecommunityinthefuture,tellingthemthatthebenefitsmaynotbeimmediatebutperhapsinthefuture.Assuch,theyshouldhelpthechildrenbyallowingthemtobeenrolledsothattheycanreapboththeworldlybenefitsandthebenefitsinthehereafter.’(CBMCNiger,lowerperformingIQS)

CBMCsgenerallyfeelthattheyaredoingagoodjob,butbelievetheycoulddobetteriftheyfacedfewercapacityconstraints.Respondentsseelackoffundstoimproveinfrastructureandtopayfacilitatorsasthemainchallenges.Theperceptionislessofparentsbeingunwillingtocontributefunds,butmorethatparentshavelimitedfundstogive.ThefactthatoneofthekeyreasonswhyparentssendtheirchildrentoIQSsappearstobeduetopovertyandaninabilitytoaffordtosendchildrentothepublicprimaryschoolsupportsthis.ThesefindingssuggestthatresourcemobilisationbeyondtheGEP3grantsmaybevolatileandvariable,especiallyincaseswhereCBMCsrelyonphilanthropy/richcommunitymembersinthecity,orevenpoliticalphilanthropy.

CBMCsreportbeingawareofthechallengesconcerningnotpayingfacilitators,sincethismayaffectbothfacilitatorandpupilattendance.Theyarealsoawareofthedifficultyofmobilisingresourcesfromparents.However,theylackthecapacitytosolvethesechallenges.BeingeitherprivatelyrunbyaproprietororbythecommunitytheseIQSsrelymoreonresourcemobilisationthanpublicprimaryschools160.Byrelyingoncontributionsfromcommunitymemberswhomaynothavefundstogive,thereisariskthattheworkofCBMCswilldiminishas‘externalresources’runout.

160Haq and Islam (2005).


©EDOREN 200

‘Concerningraisingoffund,weusetotellthechildrentobring100Nairawhenevertheyresumefromsallahbreak.Butjustfew,about10pupilswereabletopaysinceweresumedfromthelastsallahbreakwhichwasseveralmonthsago.’(CBMC,BauchitypicalIQS)

IntheNigerhigherperformingIQStheCBMCasksparentstocontributeinanywaytheycan:‘Wehavemadecontributions.Eachofuswasrequestedtobringeitherwood,nailsorzinc,whichwedid.(Girls’FGD).Thesenon-monetarycontributions,whilstimportant,arenotsufficientasregardspayingfacilitators,whichrespondentsperceivetobethemainfinancialissue:

‘Ourmajorproblemismoney;wedonothavesufficientfundstosupporttheschoolthewaywewantto.Thereisevennomoneytogivetheteachersinordertomotivatethem.Theyjustvolunteer.’(CBMC,BauchihigherperformingIQS)

CBMCsfurtherconsiderpromotingfacilitatorattendanceandperformancetobepartoftheirresponsibilities.However,asCBMCmembersmonitorfacilitatorperformancewithouthavingpriorteachingexperienceandlackofguidance,monitoringoftenbecomessimplysymbolic.AsoneCBMCmemberstated:‘Wealsocarryoutvisits(schoolvisits),althoughwehavenolessongoalsatthattime’(CBMC,BauchitypicalIQS).Nevertheless,stakeholdersconsidermonitoringbyCBMCstobeuseful,perceivingthatitmotivatespupilsandparentsbydemonstratingthatcommunityleadersandkeystakeholderstakeeducationseriously.

CBMCsinallschoolsbelievethattheywouldbebetterabletomonitorfacilitators,andholdthemtoaccount,iftheywereabletopaythemasalary.ThiswouldalsoenabletheIQStoattractqualifiedfacilitators.Thisisparticularlychallengingwhentherearenotenoughqualifiedfacilitatorswithinthecommunity161,creatingtheneedfortheCBMCtoincentivisepeoplefromoutsidethecommunitytoteachattheIQS,oratleasttoreimbursetheirtravelcosts.

‘Wealsotalkaboutteachers.Wewantmoreteachersbroughtwhocanteachformalsubject.AsfortheArabic,thereisnoproblem.Fortheformalsubjectteacheritisnotconsistent.Sometimeshewillcomeandsomeothertime,hewillnotcome.Ifwehavepeoplewhocandevotetheirtimetous,itwillguaranteebetterquality.’(CBMC,BauchilowerperformingIQS)

Thus,CBMCsallstatethatoneoftheirmaintasksisraisingfundsforthefacilitators.However,thereisanimplicitconflicthereasCBMCsacknowledgethatparentshavelimitedcapacitytocontributefunds.CBMCsseethisassomethingthatsupportfrom‘externals’mustsolve.

However,CBMCsdonotnecessarilyinvestfundsexclusivelyforformalcurriculumfacilitators.InBauchihigherperformingIQS,formalfacilitatorscomefromthepublicprimaryschoolandare,assuch,paidsalariesfortheirregulargovernmentdayjobs,althoughtheystillworkvoluntarilyintheIQS.TheCBMChencemobilisesresourcesinordertopaytheArabicteachersbutnottheformalfacilitators:

‘Youknowthereare10teachersforIslamicstudiesinthisschoolandwedonatesomemoneyduringtheCBMCmeetingtosupporttheseteachers…becausetheydon’thavegovernmentjobs.’(TPD,BauchihigherperformingIQS)

AnotherkeyresponsibilityofCBMCsisdealingwithpupilabsenteeism.CBMCsperceivetheissueofpupilabsenteeismtobemultifaceted,althoughtheyseeitasrootedinparentsnotunderstandingtheimportanceofformaleducation.CBMCsallperceivetheirresponsibilitiesasincludingexplaining

161By‘qualified’ismeantsomeonewhocanreadandwrite,orsomeonewhohassomeyearsofformaleducation.


©EDOREN 201

thepurposeofformaleducationtoparents,andtakingthetimetositdownwithparentstomakesuretheyunderstandthis.CBMCsinmanyoftheIQSsvisitedhencegofromhousetohousetocontactparents.IncaseswhereaCBMChasfemalememberstheywillgoandspeaktomothers.

‘They[theCBMC]arereallyhelpingusinthattheyencourageourparentstoalwayssendustoschoolandnotsendusonerrandsthatcouldpreventusfromcomingtoschool.’(Girls,BauchihigherperformingIQS)

MethodsusedbyCBMCstoensurethatchildrenattendschoolalsoincludewardheadsmobilisingpupilstocometoschool(inNigerhigherperformingIQS),youthmonitoringgroupsbeingformedtopromoteattendance(BauchihigherperformingIQS),andCBMCmembersspeakingtoparentsofchildrenwhoareoftenabsent(NigerlowerperformingIQS).InNigertypicalIQS,theCBMChasdecidedtomakecomingtoschoolcompulsory.Whenparentsdonotcomply,theCBMChastheauthoritytophysicallyfighttheparentsandbringchildrentoschool:‘Itiscompulsorytocomply.Suchparents,wetalktothemandwecanevenforceonsuchparentsbecausewefightthem.’(CBMC,NigertypicalIQS)

However,respondentsperceivepupilabsenteeismtobeaconsequenceofpoverty.Parentsoftenrequirechildrentoworkorlookafteryoungersiblings,andthustheybearanopportunitycostifthesendtheirchildrentoschool.Tosolvethis,theNigertypicalIQSseestheschoolmuchasacommunitydaycarecentre,whereallchildrengo,sothatparentscanstillgotothefarm.Thismaynegativelyaffectlearning,aschildrenofallages(includingbabiesandtoddlers)arepresent:‘Whenschoolishappeningallthechildrenofthecommunityareinschool.’(CBMC,NigertypicalIQS)

Inordertomotivatepupils,CBMCsinbothhigherperformingIQSs‘testpupils’andprovidegiftsforthosewhoperformbest.Rewardingpupilsisseenasawayofencouragingparentstosendtheirchildrentoschool.TheNigerlowerperformingIQSaskschildrentocomeuponstageduringcommunitygatheringsandrecitethe‘a,b,c’,withtheideathatthiswillmakeparentsproudandwillmotivatethemtosendtheirchildrentoschool.InNigerlowerperformingIQSamainissueidentifiedwasthelocationoftheIQS.Pupilshadtocrossaroadwithheavytraffictoaccesstheschool,whichpreventedparentsfromlettingtheirchildrenattendtheIQS.MembersoftheCBMCthereforestandandstopthetraffictohelppupilscrossbeforeandafterschool,inordertoincreaseattendance.Inmostcases,CBMCsthusactas‘problemsolvers’,identifyingreasonsforpupilabsenteeismandsolvingthem.

Lastly,CBMCsperceivethemselvesasbeingresponsibleforschoolinfrastructureandimprovingthelearningenvironment162.AllIQSshavemadeeffortstoimprovethephysicalschoolenvironment,eitherthroughconstructingaschoolorthroughpurchasingmats,kettlesetc.However,insomecasestherearequestionsaboutwhetherCBMCsusefundsinthebestpossiblemanner.IntheBauchitypicalperformingIQS,theCBMCusedresourcestopurchaseatruck-loadofsandtoleveltheground,inordertocombatrainfallerosionthathadcreatedslopes,leadingtopupilssittinguncomfortably.However,thesandhadonceagaineroded,resultinginareturnoftheunevenslope.TheCBMChadusednofundstoerectaroofofsomesort,leavingpupilsunprotectedfromtherainandthesun.ThisraisesquestionsregardingwhatinformsCBMCs’decisionsregardingspendingfunds,andthusraisesquestionabouttheirdecision-makingcapacity.

162SeefurtherdiscussioninSection2.2.2ongenderconsiderationsinschoolmanagement.


©EDOREN 202

CBMCs:Capacitydevelopment

Asdiscussedabove,CBMCmembersarewillingandabletotakeonnewskills,knowledgeandwaysofworking.However,thesituationsCBMCsfaceinIQSs,intermsofthesocioeconomiccontextandlevelofqualifications(aswellasnavigatingengrainedsocialnorms),meansthattheyfacearangeofpracticalchallengeswhenitcomestomobilisingresourceseffectively.Thesecontextualconstraintsmayneedtobeconsideredinordertotargetcapacitybuildinginresourcemobilisation.EvaluatingtheefficiencyandeffectivenessofaCBMCthusneedstotakeintoaccountthecontextwithinwhichitoperates.Inthissense,CBMCscannotbeeasilycomparedwithSBMCsinpublicprimaryschools.Moreover,CBMCstendtobehighlyembeddedintheschoolstructure:theydealwithaspectsofreligiouseducationandgeneralcommunitywellbeing,inadditiontoaspectsofintegration.Becauseofthisbroaderremit,CBMCswilllikelybeengagednotonlyindiscussingandmanagingIQSmatters,butalsoablendofbothQuranicandsecularlearning,aswellascommunityissuesexternaltoeducationmatters.

InsomeIQSsvisited,CBMCshadreceivedtrainingongendersensitisationandtheimportanceofformaleducation.SomeCBMCs(forexampleNigertypicalIQS)referredtohavingreceivedtrainingonhowtospendallocatedfunds.ThequalitativedatashowthatCBMCsimplementactivitiesbasedonwhattrainingcontenttheyhavereceived,butthelackofresourcesandclarityaroundrolesandresponsibilitiesofvariousstakeholderslimitstheenactmentoftasksandresponsibilities.Asthequotebelowexemplifies,theperceptionsaroundtrainingreceivedwerethatifCBMCsstartedimplementingintegrationactivities,supportwaslikelytocomefurtherdowntheline.

‘TheytaughtusalotIcanrememberonlyfew.FirsttheytoldustheimportanceofformaleducationinourQur‘anicstudyandtobeintunewiththeworld.Ifthemelodychanges,thedanceshouldalsochange.WetheMallamsshouldalsogiveoursupportbyallowingchildrentogotoschool,andweagreed.Theysaiditisverylikelyattheendthegovernmentwillgivetheirsupport.SowedecidedtofindteacherstoteachtheseformalsubjectssincewearenoteducatedthoughIamnotamongtheuneducatedbecauseIameducatedbutsinceIfoundmyselfteachingtheQur’anicknowledge,IaccepteditbutIteachHausaalso.IgotanEnglishteacherbutIhadwishedthegovernmentcouldtaketheresponsibility.ThenIgotMallamAnas,amathematicsteacher,sinceIdonotnowknowhowtocalculate…Theysaidtheparentsshouldsupport…Theydosupportaccordingtotheirstrengthbuttheschooldoesnotreceiveanyfunds,itjust10Nairainaweek.Theyalsosaidthechildrenshouldbecaredforandnottobebeaten,nowyoudon’tseemyteacherscarrycane.’(CBMC(Mallam),BauchitypicalIQS)

CBMCcapacitywillremainacriticalissueforGEP3.TheextenttowhichGEP3planstofocusonfurtherCBMCcapacitybuildingbeyondwhathasalreadyoccurredcouldbecriticalifthisaspectoftheToCistofollowitsdesiredcausalpath.CBMCsappeartoneedfurthertraining,goingbeyondtheirdutiesinregardtosensitisationanddevelopingmoretheircapacitiesinmonitoringandensuringtheactivesupportofgirls’educationinIQSs.Thiscanincludetrainingonhowtoengagewiththefacilitatorsinregardtotheirattitudesandteachingbehaviours,andagreatersenseofaccountabilityonthepartoftheCBMCtoensureagirl-friendlylearningenvironment.ThechallengestheLGEAfaceinmonitoringalloftheIQSalsomeansthatfollow-throughonCBMCeffectivenessisdifficult,witheachIQSoperatinginwhatappeartobelargelyunstructuredandcontextuallydrivencircumstances.


©EDOREN 203

4.5.2.2 Attitudesandinvolvementofparents

Thequalitativecasestudiesfoundparentstobesupportiveofintegrationandtheassociatedeffortsbyschoolleadersandmanagement.TheperceptionsarethattheprovisionoftrainingtoCBMCmembersandmini-grantstotheIQSscontributestoincreasedcommunityinvolvementinandsupportfortheIQSs.ItisworthnotingthattherewasaselectionbiasintheparentsspokentosincetheMallamwouldgatheragroupofparentstotakepartintheFGD.163Moreover,insomeIQSsthequalitativeteamspoketonomothers,asitwouldhavebeeninappropriatewithintheculturalcontext.Incontrast,intheNigerhigherperformingIQStheteamonlyspoketomotherssincefatherswereawayonmarketdayduringthequalitativevisit.Fromthevariousdiscussionswithschoolleadersandmanagement,aswellaspupils,itisclearthatIQSsrelyoncommunityinvolvementandsupportinordertofunction.Asdiscussed,theIQSs’embeddednesswithinsocialstructuresmeansthatsupportfromtheMallamandotherreligiousauthorities,aswellasfromcommunityleaders,seemsvitalforsuccessfulintegration.Theactivesupportoftheserespectedindividuals,inturn,helpstremendouslyingarneringsupportfromparents.

‘Iamtheowneroftheschoolanditbearsmyname,butIdon’tclaimownership.Iamastakeholderandthecommunityleader.CBMCandothermembersofthecommunityareallstakeholdersthatensureasmoothrunningoftheschool.’(Mallam,BauchihigherperformingIQS)

‘Wemeettheteacherstotellthemourworries,astheyalsotellustheirsandtogetherwerubmindsforsolution.’(Parents,BauchitypicalIQS)

CommunicationbetweenparentsandtheIQSseemstofunctionwell.IncaseswheretheCBMCorheadteacherhaveissueswithpupilabsenteeismorresourcestheycallparentstomeetingstoeitherdiscusstheimportanceofeducationorfortheCBMCtorequestfunds.InseveralIQSsparentsreporthavingbeeninvolvedindecidinghowtospendgrantsreceived.Assuch,thereseemstobeahighlevelofcommunicationbetweenschoolsandcommunities.Thiscommunicationhappensthroughpre-arrangedmeetingsbetweenCBMCsandparents,throughmeetingsatthemosquebetweenheadteachersandparents,aswellasthroughone-to-onediscussionsbetweenheadteachersorCBMCsandparents.Parentsreporthavingnoproblemsinspeakingtoheadteachers.Thisappearstoberelatedtothefactthattheheadteacherisoftenareligiousauthority.Itappearscommonforparentstoaskaboutchildren’sschoolingafterprayerhasfinished.Theimportanceofthemosqueasameetingpointmeansthatcommunitymembersfrequentlyinteractwitheachother.Thiscreatesopportunitiesforparentstoengagewithschoolleadersandmanagement.

Moreover,CBMCsactivelyinvolveparentsinordertodealwithpupilabsenteeism.ParentsinallIQSsreportthattheCBMCand/orheadteacherhasaskedthemtomonitorwhethertheirchildrengotoschoolbyaskingchildrenwhattheylearnwhentheygethome,andbywalkingchildrentoschooland/orcheckinginwithfacilitatorstoseethatchildrenattendschool.ParentsinallFGDsreportregularlyaskingfacilitatorsortheMallamiftheirchildrenattend,andattimesaskingtheirchildrenwhattheyhavelearntinschool.Moreover,intheBauchihigherperformingIQStheCBMChasstarteda‘vigilantegroup’toensurethatpupilscanwalksafelytoschoolandtomonitorattendance.

163Atmidline,itwillbeimportanttostrategisehowbesttoavoidthisselectionbias.ThiscouldpotentiallybedonebyapproachingtheCBMCmorebroadlyinordertoseekparticipantsfortheparentFGD–thiswouldtakethedecision-makingoutofthehandsofoneindividual.However,thismaystillbesubjecttoselectionbias,so,ifpossible,analternativecouldbetogatherthewholecommunityaspartofanintroductorysession,andaskforvolunteerstoraisetheirhandsfromthere.IfIQSshaveintroducedanattendancelistatthetime,itmayalsobepossibletorandomlysamplebothpupilsandparentsthroughtheselists.


©EDOREN 204

Pupilabsenteeismmightbeanindirectindicatorofcommunitysupportforschools,andCBMCsandfacilitatorsreportthemainissueinthisregardbeingparentsnotunderstandingthevalueofeducation.However,theIQScontextiscomplex.ThoughchildrenmaynotregularlyattendformalclassesintheIQS,thisdoesnotmeantheydonotattendpublicprimaryschools.Moreover,theperceptionisthatsocioeconomicreasons(asdiscussed)arethemainreasonwhychildrendonotattendschool,andpupilabsenteeismdoesnotnecessarilymeanthatparentsdonotsupporttheIQS.

‘Wedoaskthem(children)questionsonceinawhileaboutwhattheyarelearningintheschool,accordingtoourlevelofunderstanding.Thereisasmallgirlthatweobserveddoingwellinmaths.Sometimesweevenchecktheirbooks.Asparents,weindividuallysendandmakesurethatthekidscometoschooltolearn.Wemeetanddiscusswiththeteachersandthenadviseappropriatelyaboutthewayforwardasfarastheschoolisconcerned.’(Parents,BauchilowerperformingIQS)

However,notallparentsareequallyinvolvedintheactivitiesoftheIQS.Someparentsactivelyengageinschoolactivities,andformpartoftheCBMC,whilst‘somedon’tcare’(Parents,NigerlowerperformingIQS).Althoughparentsmaybeinvolvedindecisions,andactivelyengagewithdiscussions,thatdoesnotnecessarilymeanthattheyhavedecision-makingpower.SincetheMallamoftenhashighstatus,throughbeingtheproprietor,theheadteacher,aCBMCmemberandareligiousauthority,hisdecision-makingauthoritytendstobestrong:‘Ourparticipationislimited.Youknowhedecidesbecauseheistheimamandiswellrespected.’(Parents,NigerlowerperformingIQS).Furthermore,oneschoolinBauchi(lowerperforming)haspupilswhoattendfromoutsidethecommunity,whocomefromasfarawayasNigerStateandstayforyears.Itisnotclearwhoisresponsiblefortheattendance,andformonitoringthelearning,ofthosepupilswhoseparentsarenotpresentinthecommunity.

Moreover,theinvolvementofmenandwomendifferssignificantlydependingontheIQScontext.AsdiscussedinSection0,mencurrentlymakeupthemajorityofCBMCmembershipinallschoolsvisited.Incaseswherewomenarepresenttheytendtohavefewerformalpositions,andonlyinonecase(NigerhigherperformingIQS)didawomanholdanamedposition(treasurer).IntheNigerlowerperformingandtypicalIQS,onlymenattendmeetingswhentheIQScallsparentstomeet:‘No,they(women)don’tcometothemeetings;weinthevillagearestrictwhenitcomestothis.Wekeepthemathome.’(Niger,lowerperformingIQS).Consequently,inthiscasewomenareonlyinvolvedthroughtheactiveengagementoftheonefemaleCBMCmemberbut,asdiscussedinSection0,thismightnolongerbepossibleoncethewomenmarry.Inmostcases,womenarethusreliantontheinformationtheyreceivethroughtheirhusbandsandothermeninthefamily.

InNigerhigherperformingIQSwomenaremoreinvolved,andevenmakeupasignificantnumberofCBMCmembers.Theproprietoroftheschoolisfemale,andisperceivedtobean‘eldersister,ormother’tothecommunity.SincetheIQSisinherhouse,itisindeedeasierformotherstobeactivelyinvolvedintheIQS.Moreover,thisIQSalsorunsformaleducationclassesformarriedwomenandthuswomenarepresentintheschool,ascomparedtootherIQSswherewomenaremoreconfinedtotheprivatesphereofthehousehold.

‘Actually,itisthisgoodthingshestarted.Thatmadeustoknowheranditisbecauseofthetrustthathasbeenestablishedbetweenusandthetrainingourchildrengetthatwehavegivenherourchildrentotrainandeducate.Ourchildrencomeheretobetaughtandtheyaretaughtgoodbehaviours.Weareveryhappy,wetrusteachotherandwehavebeenlivingpeacefully.Wemothersseeherasaneldersister,someasamotherandwehavebecomelikeonefamilynow.’(Niger,higherperformingIQS)


©EDOREN 205

4.5.2.3 Perceivedcommunitysupport

AcrossallsixIQSvisitedparentsandkeycommunityfiguresreportprovidingsomeformofsupporttotheschool.Thequalitativeresearchfoundthattheperceptionsaroundsupportarenotlimitedtofinancialcontributions,butencompassmoralsupportandtrust.

‘Thewidercommunityistrying.Thepeopleappreciatemyeffortsandcontributionstowardstheeducationoftheirkids,byprayingformeandpattingmeontheback.Ialsoappreciatetheircontributionsandconcerntowardstheschool.’(Mallam,Bauchihigh-performingIQS)

ParentsinallcasesreportprovidingsomesortoffinancialsupporttotheIQS,mainlythroughcontributingtowardspayingfacilitators‘atoken’.Theyareawareofnotbeingabletocontributeasmuchasrequired,butattributethistopovertyratherthantoalackofwillingness.Parentshighlightthattheysupporttheschoolsinotherways,eitherthroughmonitoringtheattendanceofchildren,providingadvice,orthroughcarryingoutcommunityworksthatbenefittheschool.

4.5.2.4 LGEAsupportandmonitoring

Governmentauthoritiesand‘externals’areperceivedtobethemainactorsresponsibleforprovidingfurthersupporttotheIQS.

‘Yes,they(theLGEA)cametoseeus;theymetevenwiththecommunityleader.Thatiswhywecounselledtheteacherstheothertime,forthemtobepatient.Thevisitorswenttothedistrictheadandreiteratedthesamedetails.Andemphasisedthatperhapsinthefuturewhoeverisinvolvedmaybeemployedtobepaidfully.Thatisthewayitis…’(CBMC,NigerlowerperformingIQS)

RespondentsreportthatLGEAofficialsor‘government’rarelyornevervisittheIQS(apartfrominNigerhigherperformingIQS).TheLGEAofficersinterviewedaspartofthequalitativeanalysisconfirmedthis.Theyperceivethechallengeofattemptingtovisit30schoolsandmoreonaregularbasistobeunmanageable,giventhedistancesconcerned,thelackoftransportationinmanycases,andtheirworkloads.Furthermore,morefundsaresaidtohavebeenpromisedfromtheLGEAbutnotreceived.TheCBMCinNigerlowerperformingIQSsaystheyhaveneverreceivedsupport.BycontrasttherespectiveLGEAofficersaidthatallschoolshavereceivedgrants.TheCBMCinthiscasedoesnothaveabankaccount.Generally,communitiesviewsupportfromthegovernmentnegatively,consideringgovernmentactorstobeunreliable164.Still,theviewisthatthegovernmentshouldbeprovidingfacilitators:ifanIQSdoesallthatisaskedofit,thenthisshouldberecognisedthroughfinancialsupportorsupportoffacilitators.Iftheperceptionisthatnomajorsupportistocome,thismayhaveimplicationsforcommunityattitudestowardsintegration.

LGEAofficers’engagementwithcommunities–whenitoccurs–appearstofollowasimilarformula.Theheadteacher/Mallam/proprietoristheprimaryentrypointforengagementwiththeIQS,theCBMC,andthecommunityatlarge.OfficersinBauchispokeofreceivingmonthlyreportsfromtheIQS.Itappearsthattheseoccurthroughactualvisitstotheschoolbytheofficerhimself,whowouldthencompileareportbasedondiscussionsandmeetingswithstakeholdersinthecommunity.Giventhechallengesfacedinreachingalltheschoolsonaregularbasis,thismakestheregularityofthesereportsandtheircontributiontoefficientlymonitoringandsupportingtheIQSasignificant

164NigerhigherperformingIQSisanexceptionhere,reportinghavingseveralvisitsfromtheLGAandhaving‘governmentmembers’aspartoftheirCBMC.


©EDOREN 206

challenge.TheeffectivenessofwhattakesplacewhenLGEAofficialsmakethesevisitsisalsoaconcern,asarticulatedhere:

‘Undermyrole,thechallengesIhavefoundisthoseschoolsthatwereselectedarehardtoreach.Wheneverthereisanimportantissue,wewanttodiscuss,wehavetopreparetime,butIcannotreachthemontime.Thentohavecommunicationwiththem–theydonothavenetworkservice–Ihavetotrek,ortakevehiclefrommyLGAdowntothecommunity.ThenwheneverIorganiseameetingwiththecommunitymemberstodiscussanimportantissueabouteducation,mostoftheparentsorcommunitymembersdon’tliketoattend,thinking[that]thegovernmentisgivingsomething,andwedon’twanttogiveitout,[but]theyalwayswanttogetsomethingwhenevertheyarecalledtoameeting.’(BauchiLGEAofficer,typicalcase

LGEAofficersalsoseetheirroleasprovidingsupervisionandadviceonteachingandmanagementpractices,andofferinginstructionandcorrectionbasedontheirobservations:

‘Youknow,whenwevisit,andwefindoutthattheyarenotcarryingouttheactivitiessuccessfully,wewilladvisethem,orstillre-trainthem.Maybeinthemethodofteachingtheyarenotgoingstraightforward,ortheyareleavingsometacticsormethods,eitherinevaluation,eitherinmethodology…iftheyleavesome[thingout],wewillremindthem.Wewilltellthemthatisthewaytheyshoulddoit”.(LGEAofficer,Bauchitypicalcase)

4.5.2.5 Schoolexperiencefor(girls)children

ThequalitativeresearchinthesixIQSsacrosssixdistrictsoffersinsightsintosomeoftheexperiencesofgirlchildren,andhighlightssomeoftheconstraintsthatgirlsmayfaceinattendinganIQS.ThisisbynomeansrepresentativeofthechallengesandexperiencesofallgirlsattendingIQSsintheGEP3states,butprovideshintsastoareaswheregenderconsiderationsareimportantforprogramming.

TheFGDswithpupils(boysandgirlsseparately)askedchildrentodescribetheirtypicaldayinschool,andtodiscussthewaysinwhichtheyinteractedwiththeirclassmatesandfacilitators.Bothboysandgirlssaythattheenvironmentisfriendlyforallchildren,andthatsincetheyteachthemtorelatewelltoeachothertherearefewproblems.

However,duringFGDsandinQCOs,girls(inallIQSsapartfromtheNigerhigherperformingIQS)weresignificantlyquieterthanboyswere.Girlsoftenwhisperedanswersorhidbehindtheirhijabs.Whilstboyswouldimmediatelyhighlighttheirknowledge,girlswouldgive‘yesandno’answersandwouldtakealotlongertowarmupandsharetheirperspectives.

Intheclassroomsvisited,girlssitattheback,orboysandgirlssitonseparatesidesoftheclassroom.Neitherboysnorgirlsperceivethistobeanissuesince‘theMallamhastoldusthisisthewayhewantsustosit’(Girls,NigerlowerperformingIQS).Whenasked,girlssaytheywouldliketositinthefrontbutthatthisis‘notthewayitisdone’.IntheBauchitypicalIQS,theschoolislocatedoutsideandchildrensitonaslope,withchildrenfurtherawayfromtheblackboardsittingatlowerslevelsoftheslope.Inthiscase,thepracticeofseatinggirlsatthebackoftheclassdirectlyunderminestheirabilitytoseetheblackboard:‘Wherethegirlssit,waterhaserodedtheplace’(TPD).

Pupilsbroughtuptheseissuesofshynessandseatingarrangementsasperceiveddifferencesbetweenboysandgirls,especiallyintheFGDswithboys.Boysinterpretedgirls’shynessasasignthatboysaremoreintelligentthangirlsare–becauseboysanswerquestionsinschoolmoreoften


©EDOREN 207

thangirlsdo.Boysperceivegirlstobedistractedand‘notinterestedinschool,butonlythinkofmarriage’(Boys,NigerlowerperformingIQS).

“Betweentheboysandthegirlswhoexpressesmorefreedominclass?

Theboysarefreer.

Idonotknowhowtheyfeel.

Why?

Becausetheyareshy

Theyarenotsupposedtobehaveliketheboysdo.’

(Boys,BauchihigherperformingIQS)

Insomeofthequalitativecases,girlsperceivethefacilitatortobemoreconsiderateofboysandtogetangryifgirlsanswertoomanyquestionsinclass(NigerlowerperformingIQS).Theperceptionthatgirlsare‘shyer’maydirectlyrelatetotheattitudesoffacilitatorstowardsboysandgirls.IntheNigerhigherperformingIQSgirlswereasactiveandvocalinclassasboyswere,andconsideredthemselvesintheFGDtobemoreintelligentthanboys.IntheotherIQSs,girlswouldquietlygiggleinresponseandboyswouldanswerclearlythattheyweredefinitelymoreintelligent.

ThecaseoftheNigerhigherperformingIQShighlightstheeffectofthepresenceofastrongfemalerolemodel.Inthiscase,girlssaidthatofcoursegirlsweremoreintelligentbecause‘lookatMallama’(Niger,higherperformingIQS).ThisIQShasafemaleproprietorwhoisastrongcharacterinthecommunityandwhorunstheschoolandmanagesfacilitators.Sheisalsothemostqualifiedintermsofformaleducation,andspeaksfluentEnglish.Thisclearlycreatedasenseofconfidence,notonlyamongstthegirls,butalsoamongstmothers,believingthattheirdaughterscouldgoontostudyandbecome‘likeMallama’(Parents,NigerhigherperformingIQS).Thepresenceofastrongfemalerolemodelthusseemsimportant.

However,thefactthatanIQShasfemalefacilitatorsdoesnotnecessarilymeanthatgirlswilldisplayahigherlevelofconfidence.ThetwoIQSsvisitedinNiger(lowerandhigherperforming),whichhadfemalefacilitators,showedthemostevidentdifferences,bothinfacilitators’attitudestowardsgirlsandgirls’agencyintheclassroom.Thus,itmaybemoreimportantthatthefacilitatorhaspositiveattitudestowardsgirlsandthatsheisalsoempoweredwithintheschool,thanthatthefacilitatorisfemale.

AllIQSsvisitedusesomecorporalpunishment.Pupilsperceive‘flogging’asthemainthingtheydislikeaboutschoolandthatdetersthemfromattendingschool.ThelevelofopennessaroundcorporalpunishmentvariessignificantlybetweenIQScases.Insomeschools,facilitatorsandstakeholdersreportneverusingcorporalpunishment,whilstchildrensaytheyarefloggedorbeatenwhentheymisbehaveoraretooloud(NigerlowerperformingIQS).Inothercases,facilitatorsareopenabouttheuseofcorporalpunishmentandconsideritanappropriatewayofdiscipliningpupils.Intheseschools,CBMCssaidthatoneoftheirresponsibilitiesistospeaktofacilitatorsaboutbetterwaysofdiscipliningchildren(Bauchi,typicalIQS).IntheBauchitypicalIQS,theMallam/headteacherreportedthattheuseofthecanenolongertookplace.However,themathsfacilitatoropenlyspokeofusingit.Thisindicatesthatwhiletheunderstandingattheleadershiplevelisthere,thepracticehasyettobecurbed.Thereasonsforthislackoftranslationintopracticeareunclear.Inthehigher


©EDOREN 208

performingIQS,facilitatorssaytheydonotusecorporalpunishmentunlesstheyreallyhaveto:‘Ihatedoingit,beatingthemisthelastresortIfallbackto’(TPD,NigerhigherperformingIQS).Still,somepupilsspeakofbeatingsiftheyspeakHausainschoolwhentheyshouldbespeakingEnglish,oriftheymaketoomuchnoise.Interestingly,allpupilsconsiderpublicprimaryschoolsaslessofasafespace:‘Nobodycanesusherebutattheformalschooltheycaneusandthatdiscourageusfromattendinglessonsthere’(BoysFGD,BauchihigherperformingIQS).Itwillbeinterestingtoexplorefurtheratmidlinealternativemeansofdiscipliningchildren.Severalrespondentsmentionlearningintrainingthattheyarenottobeatchildrensinceitwilldeterthemfromattendingschool.Thisappearstohavebeenactedupontosomeextent.

InmostIQSsvisited,girlslackthespacetoplay.Theperceptionsarethatboyshavemorefreedomtoplay,thoughgirlsexpressawishtoplaythewayboysdo.Oneaspectrelatedtothisissecurity.SomeschoolsinBauchicomplainaboutthelackofafencearoundtheIQS,andissuesofboysdisturbinganddistractinggirlsfromreading(CBMC,BauchihigherperformingIQS).Thisissuefurtherhasthepotentialtoaffectinstructionaltime.Thoughrespondentssaygirlsendschoolatthesametimeasboys,theperceptionisthatitisdangerousforgirlstoattendlatesessions,andthisisresultinginlessinstructionaltime.

Bothboysandgirlsreportbeinguncomfortableinschool,andsomeIQSslackaccesstofacilitiessuchasrunningwater,toiletsandshade.InbothhigherperformingIQSstherearebathroomfacilitiesavailableforbothboysandgirls.However,thesearenotconsideredsufficientandpupilssaythatboysoftengointhebushwhilstgirls‘gobackhometoeasethemselves’(Boys,BauchihigherperformingIQS).InBauchilowerperformingandtypicalIQSsrespondentssaythatgirlscanusethefacilitiesintheMallam’shouse.However,therearecontrastingaccountsofwhetherornotthistakesplaceinpractice.

‘Thechallengesthatboysandgirlsfaceinthisschoolisthatduringlessonsachildmaywanttoeasehimselfbutyouknowforboysitiseasierforthemtofindwheretoeasethemselves,butforagirlshecannoteaseherselfanywhere.Sheneedsagoodplacewhichwedon’thave.’(TPD,BauchitypicalIQS)

InNigerlowerperformingandtypicalIQSsgirlsdonothaveanybathroomfacilitiesbutreturnhometoeasethemselves.Thereisarecognitionofthisasanissue,sincewhengirlsgohometheyrarelycomebacktoschool,leadingtogirlsmissinglargepartsofclasses.

4.5.2.6 Genderconsiderationsinschoolmanagement

‘Sogirlsshouldhaveagoodenvironmentwheretheycaneasethemselvesandalsoagoodplacetoseat.’(TPD,BauchitypicalIQS)

Thereisevidencetosuggestthatlargedifferencesinschools’cultureshaveaninfluenceongirls’educationalperformanceandtheiragencyindemandinggenderequality165.Ensuringgenderresponsivenessacrosscross-schoolsystems,structuresandprocessesseekstoaddressinstitutionalculturesinamannerthatwillpromotegirls’education.Thisincludesnotonlyteachingandthecurriculum,butalsocommunication,managementstructures,andpremises.166Untherhalteretal.(2014)identifyseparatetoiletfacilitiesforboysandgirls,andeffortstochangetheinstitutional

165Untherhalteretal.(2014).166Atthill,CandJha,J(2009)TheGenderResponsiveSchool:AnActionGuide,CommonwealthSecretariat.


©EDOREN 209

culturethroughadvocacyandcapacitybuilding,tobeimportantmeansofcreatingagirl-friendlyschool.

Onecanthusgroupaspectsofgirl-friendlylearningenvironmentsintotwobroadcategories:institutionalandinfrastructural.Institutionalcultures–acceptedtraditionsandbehaviourswithinaschoolcontext–arecriticalinthetransferofgendernormsthatcaneitherbenefitornegativelyaffectgirls’education.Infrastructuralrealitiesrefertotherecognitionofgirls’specificneeds(includingbiologicalneed),andinparticularissuesrelatedtosafetyandacceptedsocialnorms.

IntheIQSsvisited,schoolleaderswerewillingtoreflectonandconsidergenderequityinschoolplanning.CBMCs,asmentioned,saythatoneoftheirmainresponsibilitiesistoincreaseawarenessoftheimportanceofformalschoolingforbothboysandgirls.InallIQSsinthequalitativesample,CBMCsaytheyactivelyworktosensitiseparentsandtoconvincethemtosendtheirchildrentoschool.SeveralCBMCsconsidermotherstobethemainopponentsofthischange.Theynotetheneedtobepatient,andtoallowthischangetooccur–whenmotherswillrealisethevalueofeducationforallchildren.Theroleofmothersasabarriertogirls’educationcriticallyrelatestotheeconomicrealitiesoftheirincome-generatingandpovertyalleviationactivities.

‘ThemainwayCBMCisassistinginthewomenaspectisthatwehavesomechallengeswithsomeparents.Somechildrenmaydesiretocometoschool,buttheirmotherswillhinderthembysendingthemonvariouserrands.Whatwedoisgofromhousetohousetosensitiseourfellowwomenontheimportanceofeducation.’(CBMC,BauchilowerperformingIQS)

SomeCBMCs,suchastheCBMCforBauchihigherperformingIQS,alsohighlighttheimportanceoffemalerolemodels.TheideahereisthatifwomenareencouragedtobecomemembersoftheCBMC,andgirlsseetheirmothersplayinganactiveroleontheCBMC,thiswillencouragethemtostayinschool.Inspiteofengrainedgenderdifferencesinallthecommunitiesvisited,respondentsreportthatchangeshaveoccurred:‘Letmetellyou,seethesewomen?Beforeweonlygreetcasually,butlookatthemnow.Amongus.Interactingwithus.Thiskindofthingwasnotpossiblebefore.’(CBMC,BauchihigherperformingIQS)

Thereisanawarenessthatinstitutionalculturesmayaffectgirls’learningoutcomes:forexample,thetraditionofhavinggirlssitatthebackoftheclass.IndiscussionwithfacilitatorsandCBMCssomeparticipantsreportedthatseatingarrangementsinclassmaybeunfavourableforgirls.However,thereisafearofmixingboysandgirlsintheclassroomforseveralreasons.OneisthatitissimplynotwhatIslamicteachingssay.Secondly,itisfearedthattherewillbeinappropriaterelationsbetweenboysandgirlsiftheyareseatedtogether.Ifgirlssitatthefrontoftheclassroomthiswouldfurtherallowboystowatchthemfrombehind.

‘WhyIfeeltherewouldbeproblemisbecauseboysandgirlsrelationship.Aboywilltouchagirljusttolookforhertrouble,sothatiswhyitisgoodtoseparatetheboysfromthegirlswithdifferentseatingspaceandwithouttouchingeachother.Soeveniftheteacherisnotaroundtheysitinthatposition.Everygirlwithherspaceandeveryboywithhisspace.’(CBMC,BauchitypicalIQS)

IntheNigertypicalIQS,theschoolhasrespondedtothisissuebyconstructingtworoomsinthenewschoolbuilding,oneforgirlsandoneforboys.Byseparatingboysandgirlsinthismanner,girlswillbeabletositupfront,closertotheboard,whilststillnotmixingwiththeboys.Theperceptionisthatthiswouldsolvetheissue.However,thismayhaveseveralimplications.Forexample,thefacilitatormaynotgiveasmuchattentionorefforttoteachingthegirls.Still,itmayhaveapositiveimpact,sincegirlsmightperceiveaseparateclassroomtobeasaferspaceforlearning.Thiswillneed


©EDOREN 210

furtherexplorationatmidline–ifthecommunitydoesinfactcompletethebuildingandimplementthenewsegregatedteaching.

CBMCsappeartohaveconsideredtheattendanceandretentionofgirlswhenallocatingresources.AllIQSsreportthattheyarewillingtoinvestcommunityresourcesinschoolinginfrastructurethatbenefitsgirls.CBMCsperceiveakeychallengeforgirls’attendancetobelackofuniforms.SeveraloftheIQSsinNigereitherprovide(NigerhigherperformingIQS)orhavestartedtoprovide(NigertypicalandlowerperformingIQSs)schooluniforms(Hijabs)togirls.However,inNigertypicalIQS,thefundsonlysustainedabout15pupilsreceivinguniforms,andthustheCBMCdecidedthatitwasbettertoallowchildrentowearanythingtoschool,sothatthechildrenwhodidnotreceiveuniformswouldnotfeelbad.Girlsagreethatuniformswouldmakethemfeelencouragedtocometoschool.

Otherperceivedneedsforcreatingagirl-friendlyenvironmentarebuildingtoiletsandprovidingwaterinschool–recognisingthatthesefacilities(inparticular,toilets)aremoreessentialforgirlsthanforboys.SomeIQSshavealsoinvestedinkettles(‘teapots’)167.Girlsreportkettles/’teapots’asbeingsomethingtheyneedinordertogotothetoilet:‘ifoneaskspermissionfromtheteacher,ifshehasnothingwithher,shewillhavetogolikethat’(Girls,BauchihigherperformingIQS)–meaningthatshewillhavetogowithoutawatercontainer.However,thoughIQSsmayallocateresourcestoinvestinfacilitiesandmeansthatcouldimprovethelearningenvironmentforgirlsitwillbeessentialatmidlinetoexploretheextenttowhichtheseareactivelyreservedforandusedbygirls.Forexample,communitiesalsousekettles/’teapots’inablutionspriortoprayer,sothereisapotentialthatboyswillhavepriorityusageofkettles/’teapots’andthatnonewillbeavailableforgirlstouse.

Givensafetyconcerns,distancetoschoolisparticularlyimportantforgirls.Somecommunitiesreportchallengeswithboyscomingtotheschoolandharassingthegirls.Inonecommunity,theCBMChasstartedavigilantegroupofparentsthatmonitortheschoolandevenwalkgirlshome.Still,thequalitativeteamobservedboysaskingafacilitatortodeliveramessagetotheschoolproprietor:‘nomatterhowmuchvigilantetheMallambringstoguidetheschoolwewillstillseethosegirls’(Bauchi,higherperformingIQS).Thisunderlinesthechallengesofensuringsafetyforgirls.

4.6 Analysisofthedata-ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlyschoolenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls

ThissectionfocusesonthebaselinedataofpupilslearningoutcomesinIQSs,andfactorsthatinfluencelearningoutcomes.Wewillpayparticularattentiontolearningoutcomesforgirls.Thequalitativefindingsalsodiscussfactorsthatinfluenceretention,asecondaryoutcomeintheToC.168Thequantitativedataanalysisispresentedfirst,followedbytheanalysisofthequalitativecasestudyfindings.

167Whatisreferedtoas‘kettles’,isabasic‘teapot’.Thesearefilledwithwaterforgirlstousethemwhentheygotothebathroom.Theyarealsousedforablutionpriortoprayer.168TheenrolmentdataavailableintheIQSwerenotofsufficientqualitytoquantitativelyreportonretention.


©EDOREN 211

4.6.1 Quantitativeanalysis

4.6.1.1 PupilsinIQSs

NumberofpupilsinIQSs

DatacollectiononschoolenrolmentsuggeststhatthegenderratioinIQSsisquitebalanced.BothgirlsandboysseemtobeattendingIQSs,andthisholdstrueacrossbothstates.Giventhelownumberofschoolsthathadformalenrolmentrecords(only30%),thesedatacouldonlybecollectedforasmallshareofthesample,hencethisfindingcannotbetakenasrepresentativeofthesampleasawhole.Rather,itshouldbeviewedasadescriptionofthestateofsomeschoolsandnotbeusedtogeneralisefindings.Thelackofformalrecord-keeping,aswellasthefluidorganisationandtimingofclasses,provedtobearealchallengeinregardtoobtainingformalinformationonschooloperationswithinIQSs.Thepupilgenderratio,i.e.theratioofenrolledgirlstoboys,wasfoundtobeslightlyhigherthanonefortheschoolsasawhole,andinearlygradeenrolments.ThissuggeststhatthereareatleastasmanygirlsenrolledinIQSsasthereareboys.Thisholdsacrossbothstates,withfiguresbeingslightlymoreinfavourofgirlsinBauchi.However,giventhesamplesizeissuesinthisindicator,wecannotreportabsolutevaluesorsignificancevalues.

Toaddressthedatalimitationimposedbyalackofenrolmentrecords,wealsocollecteddataonpupilattendanceonthedayofthevisit,throughamanualcountingexerciseofallthepupilsstudyingintegratedsubjectsinP2ortheequivalentlevelonthedayofthevisit.Thesedataexistforalmostalltheschools,andshowthatjustasmanygirlsasboyswereattendingintegratedsubjectclassesinIQSsonthedayofthevisit.Thisisaninterestingfinding,whichsupportstargetingIQSsasameansofeducatinggirls.However,thereisthecontinuedriskthattheattendanceonthedayofthesurveyisnotrepresentativeofwhathappensattheschoolonadailybasissincetherewasanincentiveforschoolstomisrepresentthisinformation.Effortsweremadetomitigatethisrisk,butitisnotpossibletoaddressthiscompletely.ItisalsointerestingtonotethatthegenderratiowasslightlyhigherinBauchiandincaseswheretheproprietordoubledastheheadteacher,aswellaswheretheheadteacherwastrained.

Thenumberofpupilsregisteredforintegratedsubjectclassesrangefromasfewas20toasmanyas400childrenatalllevelsintheschool.Ofthis,themajorityareattheearlygradelevel,i.e.P1–P3level.Acrossstates,aslightlyhighershareofNigerschoolshadenrolmentabove100pupilscomparedtoschoolsinBauchi.

Itwasnotpossibletocalculatetheteacher–pupilratioforschoolsasawholegiventhelackofqualitydata.Instead,othersourcesofevidencehavebeenusedtoestimatethis.Anindicationofclasssizecomesfromthedataonlessonobservations.Theclassroomobservationdatasuggestsameanpupilteacherratioof45andamedianof40.Thisrangeisalsoquitevaried,withfromfourto147childrenbeingtaughtbyoneteacher,withnonoticeabledifferencesinthepupil–teacherratioacrossstates.ThismeansthatclasssizeisincrediblyvariedacrossIQSsinbothstates.TheattendanceofboysandgirlsinthelessonsobservedalsosuggeststhattherearealmostequalnumbersofboysandgirlsstudyingintegratedsubjectsinIQSs,withslightlymoregirlsthanboysinP2lessonsobservedonthedayofsurvey.

Pupilcharacteristics

Atotalof576pupilsweresurveyed,ofwhich296werefromBauchiand280werefromNiger.Thesamplingstrategytargetedanequalproportionofgirlsandboys.Thiswasfairlywellachieved:48%


©EDOREN 212

ofpupilssurveyedweregirls.ThereissomevariationbetweenBauchiandNiger,with44%femalepupilsintheformerand52%inthelatter.AlmostallpupilssurveyedwereenrolledinP2oranequivalentlevel.

TheaverageageofaP2-levelpupilinanIQSis8.3years.ChildreninBauchiwereolder,onaverage,comparedtochildreninNiger,byhalfayear,withameanageof8.5years.Itisimportanttopointoutthattheseareself-reportedagesofthechildren,anddonotexistfortheentiresampleas32%ofthechildrendidnotknowtheirage.FortheremainingchildrenthereportedagehasbeenplottedseparatelyforboysandgirlsinFigure71below.Thisshowsthepercentageofboys(andgirls)thatareofaparticularage,asashareoftotalboys(andgirls)inthesample.Thisindicatesthatthepatternofparticipationintheearlyyearsissomewhatdifferentforboysandgirls.BoysinP2equivalentareyoungerthangirls,onaverage.Thissuggestsearlierentryintoschoolingforboysovergirls.Thiscouldalsosuggestthatgirlsarebeingheldbackmoreoften.Thisinformationcannotbeconfirmedusingrepeaterdata,however,duetoalackofformalrecordsattheschoollevel.

Figure71: Ageofpupilsbygender

Amajorityofthepupils(75%)speakHausaathome,followedbyNupe(22%)andFulfude(2%).NoneofthepupilsreportedspeakingEnglishathome.Thecross-statedifferencesbylanguagespokenathomearedepictedinthefigurebelow.AlloftheNupespeakersareconcentratedinNiger,whileFulfudeandKanuriarespokenbypupilsinBauchi.

0%

5%

10%

15%

20%

25%

30%

35%

4 5 6 7 8 9 10 11 12 13 14

Percen

tageofp

upils

Girls Boys


©EDOREN 213

Figure72: Languagesspokenathome,bystate

Itisinterestingtocomparethistothelanguagesspokenbythefacilitatorsandthelanguageinwhichteachingtakesplaceatparticularschools.Accordingtotheinterviewswithfacilitators,thefacilitatorsinBauchiandNigerspeakalloftheabove-mentionedlanguages.Forexample,39%ofthefacilitatorsinNigerreportedbeingabletospeakNupe,while13%offacilitatorsinBauchireportedbeingabletospeakFulfulde.Thusitappearsthatatthestatelevel,pupillanguagecorrespondswellwiththelanguagethatfacilitatorsreportbeingabletospeak.

Thesurveyfindingsindicatethatteachingwithinschoolsistakingplaceinavarietyoflanguages.However,teachingappearsnotalwaystobetakingplaceinthemothertongueofthepupils:inparticularwhenitconcernsminoritylanguages,suchasNupe,FulfuldeandKanuri.Accordingtotheinformationcollectedduringheadteacherinterviews,oneofthreelanguagesisusedtoteachmathematicsinschoolsattheP1–P3level.Figure73showsthatalthoughNupeisusedinschoolsinNiger,theshareofNupelanguageuseislessthantheshareofpupilsthatspeakNupe.Whencomparingthelanguagepupilsreportedspeakingathomewiththelanguageofinstructionreportedbytheheadteacherduringinterviewwefindthatinabout82%ofthecasespupilswerebeingtaughtintheirmothertongue,andthiswasexclusivelyeitherHausaorNupe.Intherestofthecases,therewasamismatchbetweenthepupils’mothertongueandthelanguageofinstruction.ThissuggeststhatchildrenthatspeakNupesufferfromthegreatestlinguisticdisadvantageattheschoollevel,intermsoflanguageofinstructionandthenumberofchildrenaffected.Inaddition,nativespeakersofotherminoritylanguages,suchasFulfudeandKanuri,alsosufferadisadvantage,thoughthenumberofchildrenaffectedissmallerinabsoluteterms.

3%

93%

3% 0% 1%1%

54%

0%

43%

2%

Fulfulde Hausa Kanuri Nupe Others

Bauchi Niger


©EDOREN 214

Figure73: Languageofinstruction,bystate

Almostallofthepupilsinthesamplecouldhear,see,speakandwrite.Onlyonepupilwasvisuallyimpaired,andfourwereunabletowrite.Allfiveoftheseweremalepupils.Thiscouldperhapsindicatethatparentsarenotsendingdisabledchildrentoschool,thoughthereisnowaytoverifythisindication.

Alookathouseholdassetownership,asreportedbychildren,suggeststhatchildreninIQSsacrossthetwostatescomefromhouseholdsthathavebasicassets,likemats,chairs,bedsormattresses,morefrequentlythanfromhouseholdswithluxuryassets,likecomputers,cameras,airconditioningandTV.Thisisnotsurprisinggiventhatpovertyisprevalentintheseareas.Cross-statedifferencessuggestthatchildreninBauchicomefromhousesthathaveslightlybetterassetownershipthanchildreninNiger,asdepictedbygreaterownershipofbasicassetsaswellasluxuryassets.However,theshareofthepopulationthatownsluxuryassetsissolowoverallthatthisdifferenceisunlikelytotranslateintosubstantialdifferencesacrossthetwosamples.Forexample,only8%oftheBauchipupilsamplereportedowningeitheracomputerorcamera,whilethisis4%and5%,respectively,inNiger.

Pupils’schoolexperience

PupilsintheIQSsamplewereaskediftheycurrentlyattendedanotherschool.Itissurprisingtoseethat40%ofthechildrenreportedcurrentlyattendinganotherschoolinadditiontotheschoolatwhichtheyweresurveyed.Thisisnotrestrictedtoschoolsthatteachintegratedsubjects.Thepercentageofpupilsattendinganotherschooldoesnotchangebypupilgender.Thecross-statedifferenceissignificant,however:72%ofpupilsinBauchireportedcurrentlyattendinganotherschool,comparedtoonly13%ofpupilsinNiger.InthecaseofBauchithisotherschoolmostlyreferredtoapublicprimaryschool,whileinNigerthiswasonlythecaseforhalfthepupilsattendinganotherschool–hencetheotherschoolcouldalsorefertoanotherreligiousschools,avocationaltrainingschooloranyotherprivateschool.Thecross-statedifferenceispartydrivenbytheurban–ruraldifference:about70%ofthestudentsinurbanareaswereattendinganotherschool,comparedtoaround35%ofthestudentsinaruralarea.Thefactthatpupilsareattendingotherschools,particularlyinBauchi,isimportanttonotebecauseitmayaffectthetimethattheyhaveavailabletoattendtheintegratedcurriculumclasses,anditindicatesthatasubstantialshareofIQSpupils

50%

100%

0%

67%

66%

17%

MathematicsistaughtinEnglish

MathematicsistaughtinHausa

MathematicsistaughtinNupe

PercentageofschoolsNiger Bauchi


©EDOREN 215

haveaccesstoformaleducationthroughachannelotherthantheGEP-supportedIQSs.Thisislikelytohaveimplicationsfortheimprovementsinlearningattributabletotheprogramme,aspupillearningofintegratedsubjectswillalsobeinfluencedbytheirattendanceatotherschoolsthatteachintegratedsubjectsduringthecourseoftheevaluation.

Onlyaverysmallpercentageofpupilsreporteddifficultiesingettingtoschool.Lessthan4%indicatedthattheyexperiencedifficultiesingettingtoschool,andthisdidnotvaryacrossstateorpupilgender.Ofthosewhoreporteddifficulty,themajorityreportedtrafficanddangerincrossingroadsasthemaindifficulty.Longdistancestogettoschoolandhazardousterrainthroughwhichtotravelweretheonlyotherdifficultiesmentioned,bylessthan20%ofthesamplethatreportedadifficulty.ThissuggeststhattheIQSsareaccessibleandwelllocatedinthecommunities.

Asmallshare(only6%)ofthesampledpupilswereboardersattheIQSs.Asexpected,boysmadeupalargershareofthisthangirls.12%oftheboysandonly1%ofthegirlswereboardingstudents.AgreatershareofpupilsinBauchi(10%)wereboarderscomparedtochildreninNiger(3%).Thisisinlinewiththeschool-leveldata,whichsuggestthatalargershareofschoolsinBauchiwereboardingschoolsandwereclosertothetraditionalIQSoperationalstructure.

4.6.1.2 Studentlearningoutcomes

Learningoutcomemeasurement

TheHausaliteracyassessmentisdesignedtotestthesameliteracyknowledgeandskillsastheEnglishliteracyassessment.Itemsarenotmerelytranslated,butratherparallelitemsaredevelopedtotestsimilarconceptswhenappliedtotheHausalanguage.TheEnglishliteracyassessmentcontains13items,witheachitembeingmadeupofseveralsub-items.Theassessmenttestsarangeofliteracyknowledgeandskillsacrossthepre-literacy,emergingandbasicliteracyranges.Knowledgeandskillsincludedletterrecognition,phonologicalknowledge,printconcepts,oralliteracy,verbalcomprehension,initialsoundsandletters,readinghighfrequencywords,verbalandwrittengrammar,writinghighfrequencywords,readingfluency,andcopyingandspellinghighfrequencywords.EnglishliteracyandHausaliteracyassessmentswereconstructedfollowingfivesteps,includingclarifyingconstructs,testtargeting,administration,psychometricanalysis,drawingbenchmarksandsecondarydataanalysis.

Asstatedearlierinthisreport,learningoutcomesaretheresultofacomplexandmultivariateschoolsystem,andteacherandstudentfactors,whichareinfluencedbysocialandeconomicconsiderations.Thewaysinwhichthesefactorsinteractarecomplexandcanneverbefullymappedoraccountedforinananalysis.Therefore,studentlearningoutcomesneedtobeinterpretedwithcare.Forexample,oneofthemostimportantpredictorsoflearningoutcomesiswhatthepupilsbringwiththemintotheclassroom.Intellectual,socialandculturalfactors,aswellashousehold-specificfactors,significantlypredictpupilperformanceinmoststandardisedassessments(Hartas2011,OuthredandBeavis2012,OuthredandBeavis2013,Mayer,1997,DahlandLochner2005,VanderBer2015).Therefore,whilecomparisonsoflearningoutcomesbetweenstudentsfromdifferentschoolingsystemscananswerimportantequityquestions,cautionneedstobeexercisedwhendrawingcausalinferences.

ThissectionofthereportdescribesthelearningachievementofP2pupilswithinthesampledIQSsinBauchiandNiger.Thefollowingsectionsdescribekeyteacher-andschool-levelindicatorsonwhich


©EDOREN 216

datawerecollected,andinvestigatetheextenttowhichfactorsbelongingtothesedifferentanalyticalcategoriesareassociatedwiththeobservedlearningoutcomes.

Hausaliteracylearningoutcomes

ThefindingsoftheHausaliteracyassessmentsuggestthatthemajorityofpupilshaveyettoacquirebasicliteracyinHausa.Only2.5%ofthepupilsinNigerandBauchiwereabletocompletesomeofthetaskswithinthebasicliteracyrange.Afurther1.8%ofpupilswereabletodemonstrateemergingliteracyskills,whilethevastmajorityofpupilswerefoundtohaveonlypre-literacyHausaskills(pleaserefertoTable14inSection3.4.3fordescriptionsofeachoftheHausaproficiencyranges).

Figure74representsthedistributionoftheperformanceinHausaliteracy.TheaxisrepresentstheHausascalescorederivedfromthepsychometricanalysisofpupilperformanceontheassessment.Ascanbeseeninthefigure,thepeakofthedistributionfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,indicatingthatsubstantialeffortwouldberequiredtoattainasignificantincreaseintheshareofpupilsmovingfrompre-literacytoemergingliteracy.

Figure74: DistributionofHausaliteracyproficiency

Englishliteracylearningoutcomes

TheresultsoftheEnglishliteracyassessmentsuggestthatpupilproficiencyinEnglishafterayearofschoolingisslightlyhigherthanpupilproficiencyinHausa,although,again,alargemajorityofpupilshaveyettoacquireanyknowledgeorskillsbeyondpre-literacy.Ofthepupilsinthesample,3.4%wereabletodemonstratesomeoftheknowledgeandskillsthatfallwithintheemerging

0 500 1000 1500 2000 2500

263.23344.18391.83432.47468.25499.18526.67551.65574.30594.73612.97629.03643.14655.91668.05680.30693.62709.37730.13759.89799.60848.81

2.5%ofthepupilsdemonstratedGrade2Hausaliteracyskills.Thislevelof

proficiencycanbedescribedasbasicliteracy.



Over95.7%ofpupilsdemonstratedpre-literacyHausaskills.Childrenachievingwithinthepre-literacyproficiency



©EDOREN 217

literacyrangeand2.4%wereabletodemonstratesomeoftheskillsthatfallwithinthebasicliteracyrange.Theremaining94.1%wereonlyabletodisplaypre-literacyskills(pleaserefertoTable15inSection3.4.3fordescriptionsofeachoftheEnglishproficiencyranges).

ThefigurebelowdepictsthedistributionofperformanceacrosstheEnglishliteracyscale.TheaxisrepresentstheEnglishscalescorederivedfromthepsychometricanalysisofpupilperformanceontheassessment.AswithpupilperformanceinHausa,thecuspofthedistributionfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,suggestingthatsignificanteffortwouldbeneededtoachievelargeadvancesintheshareofpupilsachievingemergingorbasicliteracy.

Figure75: Englishliteracyproficiencydistribution

Numeracylearningoutcomes

Pupils’performanceonthenumeracyassessmentwasconsiderablybetterthanthatontheHausaandEnglishliteracyassessments.Intotal10.6%ofpupilsdemonstratedP2-levelnumeracyskills.Afurther68.9%demonstratedknowledgeandskillswithintherangeexpectedbytheP1curriculum.Theremaining20.5%hadnumeracyskillsexpectedatpre-schoollevel.Table29describestheknowledgeandskillsofpupilsachievingwithineachoftheproficiencyranges.TheseproficiencyrangesarealsodirectlycomparabletotheESSPINCSproficiencyranges.

0 500 1000 1500 2000 2500 3000202.78

203.56

305.91

337.10

359.37

381.54

407.18

411.74

432.60

453.62

482.66

495.07

502.02

516.29

534.65

550.54

563.77

587.17

611.49

657.78

Over94.1%ofpupilsdemonstratedpre-literacyEnglishskills.Childrenachievingwithinthepre-literacyproficiency


2.4%ofthepupilsdemonstratedGrade2EnglishLiteracyskills.Thislevelof

proficiencycanbedescribedasbasicliteracy.

3.4%ofpupilsdemonstratedknowledgeandskillswithintherangedescribedasemergingliteracy.


©EDOREN 218

Table29: Numeracyproficiencyleveldescriptions


Pre-numeracy

Childrenachievingatpre-schoollevelareabletodemonstrateknowledgeandskillinatleastsomeofthetasksthatareconsideredtobewithintherangeofpre-numeracyproficiency.Theseskillsincludebeingabletocomparethelengthoftwostraightlines,usenon-standardunitsofmeasuretocomparethecapacityofthreecontainers,andcountto10.

Emergingnumeracy

ChildrenachievingatemergingnumeracylevelareabletodemonstrateknowledgeandskillsinatleastsomeofthetasksthatareconsideredtobewithintherangeofGrade1proficiency.Theseskillsincludebeingabletorecogniseandcompleteasequenceofthreetwo-digitnumbersthataremultiplesoffive;subtractaone-digitnumberfromatwo-digitnumbercrossingthe10boundary,involvingmoney;subtractaone-digitnumberfromatwo-digitnumberfromoneto19;readananalogueclocktothehour;usenumberknowledgetocompleteasequenceofthreenumbersbelow10;understandunitfractions(1/3)andusethistofindfractionsofacommon2Dshape;understandunitfractions(¼)andusethistofindfractionsofacommon2Dshape;addtwothree-digitnumbers;usenon-standardunitsofmeasuretomeasurelength;chooseastrategytoaddathree-digitnumberandatwo-digitnumbercrossingthe10boundary,involvingmoney;comparethelengthoftwostraightlines;readananalogueclocktothehour;addtwothree-digitnumbersverticallythataremultiplesoffive,involvingmoney;chooseastrategytoaddathree-digitnumberandatwo-digitnumbercrossingthe10boundary,involvingmoney;recallthetwotimestable;subtractaone-digitnumberfromatwo-digitnumberfromoneto19;understandunitfractions(1/4)andusethistofindthefractionsofacircleandasquare;andaddtwotwo-digitnumbersthataremultiplesoffive.

Basicnumeracy

ChildrenachievingatGrade2(basicnumeracy)areabletodemonstrateknowledgeandskillsinatleastsomeofthetasksthatareconsideredtobewithintherangeofGrade2proficiency.Theseskillsincludebeingabletousenon-standardunitsofmeasuretocomparethecapacityofthreecontainers;subtractatwo-digitnumberfromathree-digitnumber(bothmultiplesoffive)crossingthe10sboundary,involvingmoney;readananalogueclocktothehourandhalfhour;recogniseandcompleteasequenceofthreenumbersthataremultiplesof50andlessthan200,andcompleteasequenceofthreetwo-digitnumbersthataremultiplesoffive;subtractatwo-digitnumberfromatwo-digitnumber;addtwothree-digitnumbersthataremultiplesoffive,involvingmoney;usestandardunitsofmeasuretomeasurethelengthofasmallobject(cm);subtractaone-digitnumberfromatwo-digitnumberfromoneto19crossingthe10boundary;namecommon2Dshapes;addtwothree-digitnumberscrossingthe10boundary;subtracttwothree-digitnumbers;addandsubtractlengthordetermineareasize;chooseastrategytoaddathree-digitnumberandatwo-digitnumbercrossingthe10sboundary,involvingmoney;extendcountingpast800andcountin10s;addtwothree-digitnumberscrossingthe10sboundary;extendcountingpast1,000andcountin100s;andidentifyandcountthefacesoffamiliar3Dshapes.

Ascanbeseeninthefigurebelow,thecuspofthedistributionfornumeracyachievementfallsjustabovethecut-offpointbetweenpre-numeracyandemergingnumeracy.Thisindicatesthatlargegainsinthepercentageofpupilsfallingwithintheemergingnumeracyrangecouldbeachievedas


©EDOREN 219

manychildrenareonthecuspofachievingemergingnumeracy.Gainsinthepercentageofpupilsachievingbasicnumeracywillbemorechallengingandwillrequiremoreeffort.

Figure76: Distributionofnumeracyproficiency

Quantitativedataonassociatedfactors

Theanalysisoffactorsassociatedwithachievementhighlightsthattheverylowlevelsoflearningintheinterventionschoolsarelargelyconsistentacrossgroups,althoughboysdoperformslightlybetterthangirls.AscanbeseeninTable30,girlsareover-representedinthepre-literacyandpre-numeracyproficiencyrangesandunder-representedinthebasicliteracyandnumeracyproficiencyranges.

Table30: Learningoutcomesbygender

0 200 400 600 800 1000 1200 1400 1600 1800 2000

185.51

322.29

397.32

452.08

490.81

513.70

529.90

543.92

557.76

572.69

589.46

608.00

628.10

650.58

677.49

715.34

Hausa English Numeracy

Girls Boys Girls Boys Girls Boys

Pre-literacy/numeracyskills 97.5% 94.0% 96.2% 92.2% 22.4% 18.5%

Emergingskills 0.4% 3.0% 2.0% 4.8% 70.7% 67.5%

Basicskills 2.0% 2.9% 1.8% 3.1% 7.0% 14.0%

20.5%ofpupilsdemonstratedpre-numeracyskills.Childrenachievingwithinthepre-numeracyproficiency

rangedonotyethaveemergingnumeracyskills.


Thislevelofproficiencycanbedescribedasemergingnumeracy.

10.6%ofthepupilsdemonstratedP2numeracyskills.Thislevelofproficiencycanbedescribedasbasicnumeracy.

.


©EDOREN 220

Figure77showsthatallagegroupstypicallyachievewithintheHausapre-literacyproficiencyrange,andthattherearenosubstantialdifferencesbetweentheachievementofboysandgirlswithineachagelevel,withtheexceptionoffourandfive-year-olds.

Figure77: MeanHausascalescorebyageandgender(95%confidenceinterval)

Incontrast,inthecaseofEnglishliteracy(Figure78)andnumeracy(Figure79)theperformancegapbetweenboysandgirlswidensatthestageinthelifecycletypicallyassociatedwithpuberty(around12yearsofage).

Figure78: MeanEnglishscalescorebyageandgender(95%confidenceinterval)

0

100

200

300

400

500

600

700

4 5 6 7 8 9 10 11 12 13 14

Hausa

Boys Girls

0

50

100

150

200

250

300

350

400

450

500

4 5 6 7 8 9 10 11 12 13 14

Boys Girls

Cut-offpointbetweenpre-literacyandemergingliteracy

Cut-offpointbetweenpre-literacyandemergingliteracy


©EDOREN 221

Figure79: Meannumeracyscalescorebyageandgender(95%confidenceinterval)

Acrossthethreeassessments,pupilsinBauchiperformedslightlybetterthanpupilsinNiger,asrepresentedinFigure80,Figure81andFigure82.ThismaybeexplainedbythelargerlanguagediversityinNigerorthefactthatalargershareofpupilsinBauchiattendpublicprimaryschoolsatthesametimeasattendingtheIQS.

Figure80: MeanEnglishscalescores,bystate

0

100

200

300

400

500

600

4 5 6 7 8 9 10 11 12 13 14

Boys Girls

200

250

300

350

400

450

500

Bauchimean Nigermean

Cut-offpointbetweenpre-numeracyandemergingnumeracy

Cut-offpointbetweenemergingnumeracyandbasicnumeracy


©EDOREN 222

Figure81: MeanHausascalescores,bystate

Figure82: Meannumeracyscalescores,bystate

Overall,thebaselinefindingsindicatethatpupils’learningoutcomesinGEP3IQSsareverylow.Thismeansthatthereistremendousscopeofimprovementbutalsohighlightsthatinterventionswillneedtobecarefullytailoredsothattheyareappropriatelytargetedtoexistinglearninglevels.Furthermore,asnotedabove,inbothHausaandEnglish,thebulkofpupilshavepre-literacyskillsandmanyarealongwayfromthecut-offpointforemergingliteracy.Thissuggeststhatachievinglargegainsintheshareofpupilswhohaveemergingliteracyskillsorhigherislikelytorequireconsiderableeffort.

GirlsandboysappeartobeattendingIQSsequally.Atanyparticularlevelgirlsareslightlyolderthanboys,whichsuggeststhattheyareenteringschoollater.Pupilsalsocomefromvariedbackgrounds,intermsoflanguagespokenathomeandassetownership,andthisiscorrelatedwithpupillearningoutcomes.

200

250

300

350

400

450

500

550

600

650


300

350

400

450

500

550



©EDOREN 223


ContributionClaim3bringstogethertheprevioustwoclaimsaroundeffectiveteachingandagirl-friendlyschoolenvironmenttoconsiderhowthesemaycontributetoimprovedlearningoutcomes,particularlyforgirls.Thequalitativecasestudiesprovidemorein-depthinsightintofactorsthatinfluenceeffectiveteachingandanimprovedschoolenvironmentresultinginimprovedlearning.Wefirstexaminepupil-andschool-levelcharacteristicsthatmayaffectpupils’learning.Next,wediscusskeybarrierstogirls’schoolattendanceandretention.

4.6.2.1 Pupilandschool-levelcharacteristics

Pupils’attitudesandaspirations

Pupil’saspirationsandmotivationarelikelytoaffecttheextenttowhichtheyattendschoolandputeffortintolearning.This,inturn,islikelytoaffecttheextenttowhicheffectiveteachingandanimproved,girl-friendlyschoolenvironmentdocontributetobetterlearningoutcomesforgirls.

Thereappearstobeadifferenceinhowboysandgirlsviewthevalueofeducationfortheirfuturelife.Inallschools,pupilsconsiderformalsubjectsimportantforbothboysandgirls.Still,inmanyIQScommunities,stakeholders(includingboys)perceivegirlsasbeingtoofocusedonmarriage,andtheybelievethatthisaffectstheirattendanceandinterestinschool.

‘Thechallengesaretheeagernesstomarry.Themomentagirlturns12allshecanthinkofismarriage,youmostlyfindhere.Thosegirlsyousaw169areallpreparingtogetmarried.Soourmainchallengehereisearlymarriage.’(TPD,BauchihigherperformingIQS)

Pupilsthemselvesrecognisethatinmanycasesgirlswillgetmarriedatanearlyageandthatthiswillinfluencewhatgirlscanchoosetodowiththeireducation.Somegirlsperceiveformaleducationasimportantinordertoteachtheirchildren,andtobeabletodobasiccalculationsandreadletters.Inmostcases,girlsdoaspiretobecomeprofessionallyactive,wishingtobecometeachersorbusinesswomen(sellingriceetc.).However,thisisnotaconsistentfindingacrosstheIQSs.IntheNigerlowerperformingIQS,manygirlswishtobemarriedwhenfinishingschool,whereasboysaspiretoworkinthegovernment.Incontrasttoperceptionsofgirlsasonlybeinginterestedinmarriage,somegirlsclearlyindicatefrustrationattheideaofleavingschoolearlierthanboys,inordertomarry.

ThemanagementandfacilitatorsinIQSsperceivegirls’interestinformalsubjectstohaveincreased.Inthepastgirlsoftenleftschoolwhentheformalsubjectsstarted.However,throughtheactivitiesofCBMCsandfacilitators(suchasencouraginggirlswithgiftsandbiscuits),girlsarenowmorelikelytoattendclasses.

Pupils’participationineducationbeyondIQS

PupilsoftenattendbothanIQSaswellasapublicprimaryschool.Therefore,thereislikelytobesomespillovereffectofinterventionsatlocalprimaryschoolsonoutcomesattheIQS.SeveralofthecasestudyIQSshavenearbypublicprimaryschools.SomegirlsattendboththeIQSandthepublicprimaryschool,attendingpublicprimaryschoolinthemorningandtheIQSintheafternoonorevening.Inthiscase,thequalityofteachingatthepublicprimaryschoolhasadirectbearingongirls’learningoutcomes.Therearealsolikelytobesomeindirectspillovereffectsforgirlswhodo169ReferstothegirlsparticipatingintheFGD.


©EDOREN 224

notattendpublicprimaryschools.Inparticular,someIQSfacilitatorsalsoteachatthepublicprimaryschool(asintheBauchihigherperformingIQS),orareinfluencedandpeer-assistedpedagogicallybyfriendswhoteachattheprimaryschool(aswiththeBauchitypicalIQS).ThiscreatesalikelihoodofsomespilloverfrominterventionsattheprimaryschooltooutcomesattheIQS.ItalsomeansthatIQSsthathaveapublicprimaryschoolnearbyarelikelytobeatanadvantagecomparedtothosethatdonot.

ThemainreasonforpupilsattendingonlytheIQSappearstobealackoffunds(seebelow),althoughparticipantsstatethatsomegirlsdonottoattendpublicprimaryschoolduetoparentsnot‘valuingeducationforgirls’.Incaseswhereparentsmaynotwishtosendgirlstothepublicprimaryschool(eitherforculturaloreconomicreasons),theIQSoffersanopportunityforgirlstobeexposedtoformaleducation.However,thereissomeriskthat,followingtheintroductionofsecularteachingattheIQS,parentsmayrefrainfromsendingtheirdaughterstopublicprimaryschoolsandsendthemtotheIQSonly,inordertosavemoney.

Thebroaderpointhereisthattheevidencepointstotwodiscreteissuesthatunderminegirls’accesstoschooling:thecostofschoolingandsomepersistingresistanceamongstparentstosecularschoolingforgirls.Thesetwoissuesinteracttounderminegirls’accesstoschooling.ItwillbeimportantforGEP3tounderstandtheinteractionbetweenthesetwoissues,andtoengagewithbothintandem.

InseveralcasespupilsalsoattendvocationaltrainingoutsidetheIQS/publicprimaryschool.InBauchitypicalIQS,participantsrefertoan‘empowermentcentre’forgirlsandwomen,wheretheylearnsewingandsoap-making.Thereareafewreferencestothiscentrealsoteachingliteracy.Thesetypesofalternativelearninginterventionsandsourcesofsupportmightaffectlearningoutcomespositively,bysupplementingchildren’sknowledge.

TypesofIQS

ThecasestudyIQSsseemtoresideinafixedlocation,althoughsomeuncertaintyremainsaboutthemobilityofoneIQS.TheIQSsvisitedstatedthattheywerenolongermobile.InNiger,allschoolswereestablishedIslamiyyasandwerefixedinonelocation.InBauchi,schoolswerecalledTsangayasbuttwoatleastseemedtobeoperatingasIslamiyyas.ThelowerperformingIQSstillhadcharacteristicsofaTsangaya,albeitbeingintransitiontowardsmoreIslamiyyafeatures.Inthatschoolatleast,thereweresomeaccountsbypupilsthattheboysmightbemoving–althoughitisunclearwhetherthisisthecase.Reasonsgivenfornotmovingincludedthesecuritysituationinthestate.Assuch,itisunclearwhethertheTsangayamaymoveagainifthesecuritysituationstabilised.

IfanIQSwastomovethiscouldhaveadirectimpactonlearningoutcomes.InthecaseofboysmovingwiththeMallamforseveralmonthsthismayresultinnotreceivinganyformaleducationduringthistime–unlesstheMallamwasalsoteachingformalsubjectsanddecidedtodosoduringthemobility.Forgirls,sincetheywouldnotbemoving,itwoulddependonwhethertheteachingofformalsubjectscontinuedintheabsenceoftheMallamandmalepupils.AsthenomadiccultureofcertainTsangayasmayhavesignificantimplicationsforlearningoutcomesandtheretentionof,inparticular,girls,itisimportanttoexplorethisfurtheratmidline.AspartoftheimplementationofGEP3goingforward,itmaybenecessarytoconsiderhowthisinformationcanbeverifiedandtrackedaspartofthemonitoringprocessattheLGAlevel.


©EDOREN 225

4.6.2.2 Barrierstoattendanceandretention(includinggendereddifferences)

Attitudestowardsgirls’education

AttendanceandretentionofgirlsinintegratedQur’aniceducationremainsachallenge.AcrosstheIQSsstakeholdersreferredto‘allchildrenattendingschool’170whilstsimultaneouslystatingthatsomeparentspulledchildren(inparticulargirls)outofschoolwhentheintegratedsubjectswereintroduced.Similarly,notallgirlsattendformalclasses:someleaveafterreligioussubjectshavefinished(Bauchi,higherperformingIQS).

AllIQSsvisitedfacechallengeswithregardtoretaininggirlsinschool,duetoearlymarriageorotherculturalconstraints(seebelow).Theyalsofacechallengeswithregardtotheattendanceofbothboysandgirlsbecauseofout-of-schoolresponsibilities(suchaschoresorwork).Otherchallengesaretherelativelackofunderstandingoftheimportanceofformalsubjects,andconcernsabouttheinfrastructureandfacilitiesatIQSs(discussedinSection4.4.1.2).Theformerpossiblyrelatestotheperceptionthatthesesubjectsarenotrelevanttothelivesofpeopleinthecommunities.Forboys,thisrelatestoadearthofjobopportunitiesthatrequireaformaleducation.Forgirls,thisrelatestothefactthatwomentypicallydonotenterformalworkoutsideofthehomeaftermarriageowingtoculturalpracticesofseclusionandtherealitiesoftheirreproductiveresponsibilities.

Therearealsopersistingbarriersrelatedspecificallytoattitudestowardsgirls’education.Alackofawarenessoftheimportanceofeducation,inparticularforgirls,waswidelycitedasareasonforcontinuedchallengesinregardtoattendance.CBMCsinallIQScasesspokeofthisissue:‘Youknowthisisavillage,peopledon’tvalueformaleducationforgirls.Theybelievesincethey(mothers)werenoteducated,whyshouldtheirchildren(girls)beeducated?’(CBMC,BauchitypicalIQS).Inaddition,whilstparticipantsconsiderededucatingthe‘girlchild’asimportant,theperceptionisthatitismoreimportantinmostcasestoeducatethe‘boychild’.

‘Wepreferthatthemanhasmoreknowledgethanthegirlbecausebyhernatureagirlissupposedtoacquireknowledgefromtheman.Everydayoftheman,evenreligionspokeonit,ifthewomanseeksoneknowledgethenthemanseekstwoknowledge.Themanshouldteachhiswifeoneandherknowledgeisincreased.Thenthemangoestoseekfurtherknowledgeandteachesthewoman.Thatispartofthelaw.Yousee,knowledgebestowsononeresponsibilities.Ifthewomanismorelearnedthantheman[is]thenthereisaproblem.Evenamongsiblings,thefemaleknowsmorethantheman,thereisproblem.’(Parents,BauchihigherperformingIQS)

Thereweresomedifferencesinattitudesbetweenruralandurbanareas.InBauchi,thehighertolowerperformingcasesfollowedtheperi-urbantoremoteruraldivideclosely.Inthelowerperforming(rural)case,anabsenceofwomenattheparents’FGDpossiblyindicatedmothers’lackofengagementwiththeIQSprocess.Thisinturncorrelateswithwomen’slowereducationalandliteracylevelsinruralcommunitiesoverall.

However,stakeholdersperceivedattitudestowardsgirls’educationtobechanging(seeSection4.4.1.2).Whilstpreviouslytherewouldbesignificantchallengesingettinggirlstoattendschool,acceptanceofintegrationhasbecomemorewidespreadinallcases.Moreover,stakeholdersagreedthatmorecould,andneededtobe,donetoincreaseawarenessoftheimportanceofeducation.Asparentshavenotyetexperiencedthepositiveconsequencesofallowingtheirchildrentoattend

170WhileKIsperceiveparticipationinIQSstobehigh,thismayattimesbeduetochildrenattendingreligiousclasses,anddoesnotnecessarilyindicateattendanceduringformaleducation.


©EDOREN 226

school,thereisstillawaytogo.CBMCssawthemselvesassettinganexamplebysendingtheirchildrentostudy,evenbeyondsecondarylevel,toencourageotherparentstodothesame.Overall,parentsandcommunitiesdoappeartobewillingtoreflectonandreconsidertheroles,responsibilitiesandcapabilitiesofgirls,andattitudestowardsgirls’educationappeartobeslowlychanging.

Capacityconstraints

EventhoughCBMCactivitiesandwidercontextualchangesappeartobepositivelyinfluencingattitudestowardseducation,includingeducatinggirls,changesinattitudesdonotnecessarilyleadtochangesinbehaviour.Thequalitativeresearchindicatesthatfinancialcapacityconstraintshaveacriticalbearingonattendanceandretention.

‘Wedidnothavetheopportunityofformaleducationproperlyandourknowledgeofreligionthewayitshouldbeimpacted.Wetakethesetwoasveryimportantbutthebarriersarethatwehaveplentychildrenandthereisnohelpfromanywhere.Forinstance,ifonehas10–15childrenintheschoolandeachchildweretopay#50,itwillbedifficultfortheparenttomeetup.Thesamethinggoesforbothprimaryandsecondary.Theresultisthatyouwillsendsometoschoolandleavesomebehindaccordingtoyourfinancialstrength.Atthesametime,wewantofallofthemtogo.Ihavesoninprivatesecondaryschoolandthebooksaresoexpensive.Thesearethekindofthingsthatcouldhindereducation.Wewantthegovernmenttostepinandhelp.’(Parents,BauchihigherperformingIQS)

Respondentscitepovertyasakeyreasonwhyparentsdonotsendtheirchildrentoschool(publicprimaryorIQS).Costsofeducationplayasignificantroleindeterminingwhetherparentssendtheirchildrentopublicprimaryschools.IQSsvisiteddonotchargeashigh(ifany)schoolfeesaspublicprimaryschoolsdo,andinmostcasesdonotrequireuniformseither.Schoolfeesareoftennon-compulsoryorareawayfortheCBMCtomobiliseresourcestofeedbackintotheschoolandtoofferteachersasymbolic‘tokenofgratitude’.However,eveninIQSsthathavelowfeesornofees,povertyremainsabarriertoaccessforsomechildren,owingtotheopportunitycostofschooling.Thislinkstotherolethatchildren(andparticularlygirls)playinsupportingthefamily’slivelihoodactivities.

SchoolmanagementinallsixIQSsbelievefeesdiscourageparentsfromsendingtheirchildrentoschool.Feesmayaffectboysandgirlsdifferently.

‘Becauseofpovertyaroundhereitcanbeaseriousissue.Itcandiscouragetheparentsandwewillendupwithlowattendance.IwillpreferbothboysandgirlstogotoschoolbutincasethereissuchasituationIthinktheboywillbesenttoschoolbecausetheboyisinapositiontoassistothers.’(Mallam,BauchilowerperformingIQS)

Parentsappeartobemorelikelytoprioritiseboysinregardtogoingtoschool,althoughthereweresomereferencestoparentspreferringtosendgirlstoschoolsincetheyaremorelikelytostaybehindandhelpthecommunityifeducated.Inordertoencourageparentstosendalltheirchildrentoschool,someIQSschargeparentsasetamountregardlessofhowmanychildrentheysend.Additionally,schoolfeescanaffectattendance,aschildrenfeelreluctanttocometoschoolduetoembarrassmentatnotbeingabletopaythefee(Parents,BauchitypicalIQS).

CBMCandschoolleadersinallsixcasestudyschoolsfeeltheycannotinsistonpayment,sinceitwillresultinparentssendingthechildrenaway.Thisincludesaskingparentstopayforschooluniformsorrequiringchildrentobringpens.Respondentsbelievethattheresourceconstraintsof


©EDOREN 227

thecommunitylimitthecapacityofschoolmanagementtoinvestinIQSs,whichinturnmayaffectthequalityofeducationprovidedandthecapacitytoretaingirls.

‘Wewouldwanttoaddmoresubjectsbutthechallengewehaveistheteachersthatteachthechildren,wecan’taffordtopaythem.Initiallythechildrenpaid10Nairaeveryonceinaweek.Wenoticedthatmostchildrendonotcometoschoolagainbecausetheirparentscouldnotaffordit.Sowestopped.’(CBMC,NigerlowerperformingIQS)

Theissueoffees,therefore,alongwiththewideropportunitycostofschoolingrelatedtochildwork,indicatesthatimprovedteachingandanimprovedschoolenvironmentwilllikelynotbeenoughtoaddressthesemoredeep-rootedchallenges.Evenifthequalityofeducationimproves(andaqualitysupplyisthereforeprovided),theinabilityofparentstoreconcilethepossiblecostimplicationscouldremainprohibitive,unlessthevalueofeducationfromaninstrumentalistperspective(whatitcantangiblyleadto)isalsoeffectivelycommunicated,withdemonstrableresultsthatcanoffsetthosecosts.171

Ontheothersideofthisargument,assumingthatretentionispartlyaffectedbyperceptionsaroundthevalueandqualityofeducation,itwillbeimportantforschoolstobeabletoteacheffectivelywhenthedemandforeducationincreasesincommunities.DuringQCOs,researchersobservedovercrowdedclassroomsinsomeIQSs,withsomepupilssatoutsidetheclassroom.CBMC’slimitedabilitytomobilisesufficientresourcescouldhaveanegativeimpactonthequalityofteaching,andthuspotentiallyretention.Yet,someCBMCsfeelthatparentsdonotprioritiseeducationwhenallocatingresourcesandthatasattitudeschange,sowillspending.

‘Fornow,itisbecausetheyhavenotreallyenjoyededucationwhycostcanaffectthem.Whentheybegintoseethebenefits,theywillratherspendmoneytheyuseinbuyingsemira172inpreparationfortheirmarriageonsendingthem[theirchildren]toschool.Becausetheplateswillbebrokeneventually,buttheknowledgewillalwaysbethereandwilldiewiththeowner.’(CBMC,BauchilowerperformingIQS)

Outsideschoolactivitiesandresponsibilities

Thequalitativeresearchexploredhowboysandgirlsspendtheirtimeoutsideofschool.IntheNigercases,theteamspoketoboysandgirlsestimatedtobebetweentheagesofsevenand13.InBauchiStatetheywerebetweentheagesofeightand15years.AcrossthesixcasestudyIQSs,boysandgirlswereallresponsibleforincome-generatingactivities.Childrenwouldengageinincome-generatingactivities,suchasfarmingorlookingaftercattle(boys),orhawkingandsellingproduceinthemarket(mainlygirls).Additionally,parentsconsiderchildrenhelpingwithresponsibilitiesinthehousehold(likelookingafteryoungersiblingsorfetchingwater)tobeanecessarypartofincome-generatingactivities.Girlsgenerallyhadmoreresponsibilitiesforhouseholdchores.

Nocleardifferencesemergedfromthequalitativedatainregardtothetimingofpupils’dailyroutines.PupilsinallsixIQSsreferredtowakingupandeitherstudyingArabicorcompletingchores,

171Convincingparentsthateducationoffersavaluethatfitswiththeirneedsisstillonlyonepartoftheprocess.Iftheyareunabletopracticallyapplylearninginamannerthatleadstothoseincurredcostsbeingmitigatedorovercome,oriftheopportunitiesarenotthereinthefirstplace(likejobs),thevaluewillnotbeseen.Forexample,ifmothersaretoldthatkeepingtheirdaughtersinschool(andawayfromhawking)tolearnnumeracyandliteracywillmakearealdifferencetotheirdaughtersbeingabletoruntheirownbusinessesmoreprofitably,theywouldneedtoseethatdemonstratedquitequicklyinordertobeconvinced,otherwisetheywillperceivethattheyarejustincurringlossesbyremovingthemfromtheirincome-generatingroles.172Traditionalplates.


©EDOREN 228

afterwhichtheywouldattendeitherpublicprimaryschoolorwork.Intheafternoonandevening,childrenreportedeitherworkingorattendingtheIQS.Thereafter,pupilsdohomeworkinsomeIQSs(NigerhigherperformingIQS,NigertypicalIQS)andvariouschores.Homeworkappearstobedominatedbyreligiousstudies.ChildreninallIQSsreportedhavingsometimetoplay,althoughnotasmuchastheywouldwant,whereasparentsconsiderchildrenplayingtoomuchtohaveanegativeeffectonattendanceandpupil’smotivationtolearn.

Out-of-schoolresponsibilitiesaffectchildren’sschoolattendance.Schoolstakeholdersattributethistotheperceivedlackofcapacityofparentstoprioritisechildren’seducation,whichtheyassociatewithalackofunderstandingofthevalueofeducation.173Parents,however,seeitasafinancialandpracticalnecessity.

‘Thereismajorchallengeforthegirlsbecausemostoftheirparentsmakethingsforsale,likebeanscake,soyabeancake,riceandsoon,andtheygivethegirlstogosell.Thisaffectsthechildren’sattendancebecauseeithertheycometoschoolverylate,almostintheclosinghours,ortheymissschool.’(TPD,BauchihigherperformingIQS)

InthreeoftheIQSsCBMCshaveactivelyinstigatedthatpupilshavetodropallchoresandworktheminuteitistimeforschool(NigertypicalIQS,NigerhigherperformingIQS,andBauchihigherperformingIQS).Thismeansthatparentsarehesitantaboutextendingtheinstructionaltime,since‘theyneedustobehomefordomesticjob’(Boys,BauchihigherperformingIQS).Accordingtoonestakeholder,possiblesolutionsaretoprovideparentswithfarmimplements,suchasploughsorfertilisers,toaidparentswhilsttheirchildrenareinschool(Mallam,NigerlowerperformingIQS).Aslongasparentsrequirechildrentoassistinincome-generatingactivities,directlyorindirectly,thisislikelytohaveanegativeeffectonattendanceandenrolment.Asparents’attitudestowardsformaleducationbecomemorefavourable,itwillbecomemoreimportanttoaddressbarriersrelatedtotheopportunitycostofschooling.

Respondentsperceivethatmothersarelesspositiveaboutgirls’educationthanfathers.Participantsstatethatwomenattimeshindergirlsfromattendingschool.Thismighthavebeenthoughttobeacaseofisolatedexperienceifitwasonlyreferencedonce;however,fiveoutofsixIQSsmentionedthisoccurring.

‘Therearechallenges.Youknowyouwomenareclosertothegirlchild.Theysaythatknowledgeforachildliesatthefootofthemother.Wemenarehardlyathome.Yousendyourdaughtertoschoolandgoouttofindfoodforthefamily.Thenthemotherwillnottalkherintodoingotherthingsinsteadofgoingtoschool.Wearefacingsuchchallenges.’(Parents,BauchihigherperformingIQS)

Pupilssharetheviewof(mainlymale)stakeholdersthatmothersoftenrequiregirlstostayathome.Theexplanationgivenwasthatwithgirlshelpingthemothersandboyshelpingthefathers,girlsattendingschoolmeansthemothercannotgenerateasmuchincomeasneeded.Itappearsthatthemotherisakeyproviderinsustainingtheday-to-daylifeofthehousehold.Moreover,sincemanymothersthemselveshavenotattendedschool,itappearsthatthereislessunderstandingamongstmothersinsomeIQSsofwhygirlswouldbenefitfromformaleducation.

CBMCsinallsixschoolsconsidersensitisingwomenaskeyinordertoincreasetheenrolmentandattendanceofgirls.However,asdiscussed,thisconstraintisunlikelytoberemovedthroughsensitisationalone,aswomenrequiregirlstoengageinincome-generatingactivities(suchas

173Thesewereperceptionsdrawnfromacombinationofteachers’TPDs,andMallamandLGEAofficerKIIs.


©EDOREN 229

hawking)ortotakecareofhouseholdchoressoamothercan,forexample,gotothefarm.InNigerlowerperformingIQS,forexample,womenrarelyleftthehouse.Girlswereresponsibleforactuallygoingoutandsellingwhatthewomenproduced,and,assuch,mothersaredirectlydependentongirlstobringinanymoney.

ThereasonsforthisareheavilyrootedintheroleofwomenaseconomicagentswithinnorthernNigeriancommunities,andaskeyincomegeneratorsforthefamily.Therunningofhome-basedbusinessesbywomeninnorthernNigeriaisatraditionalpartofthegenderedeconomiccontext,andtheuseofchildrenandyouthtopurchaseinputsandtomarketproductshasalwaysbeenacriticalpartoftheenterpriseinfrastructure.174Unmarrieddaughtersareattheheartofthissystem–moresothanboys,whowillhaveotherdutiesonthefarminruralareas.Thepracticeofaurenkuli,orseclusion,bymanyMuslimwomenwhenmarried,exacerbatesthisrelianceonunmarrieddaughters,aswomenmustmanagetheirentirebusinessinfrastructureremotely.175

ThegenderedpoliticaleconomyofthisrealityisonethattheToCofGEP3’sIQSSdoesnotaddresswhenitcomestoensuringthatgirlsareabletonotonlyaccesstheIQS,butalsotoremainfullyengagedconsistentlyandlongenoughforlearningtooccur.Successfullyaddressingthischallengewillrequireinterventionsrelatedtowomen’spovertyalleviationandalternativeincome-generatingopportunities.Assuch,itwillneedtogobeyondactivitieswithintheeducationsector.

TheextenttowhichCBMCadvocacyandimprovedschoolmanagementalone,andwithoutamoreholisticapproach,canraisegirls’participationinschoolingisquestionable.WithCBMCmembersalsobeingapartofthecommunity’seconomicreality,awillingnesstotakeonthechallengetogirlseducationthatagenderedpoliticaleconomyposes,maybelimitedwithoutfurthersupportinthisarea,despitethepresenceofgoodintentions.Atmidlineitwillbeimportanttointerrogatethisareafurther,tounderstandhowtoreconcilethetensionbetweengirls’successfulretentionandcompletionofschoolingandwomen’sincomegenerationandpovertyalleviationactivitiesinnorthernNigeria.Thisisalsoacyclicalissue,asmanyofthegirlsinquestionwillalsobetransitioningtowomanhood(andneweconomicroles)duetotheanotherkeyfactorthatinfluencesgirls’accesstoschooling–earlymarriage.

Earlymarriage

Oneofthemainreasonsstatedforlowenrolmentandretentionofgirlsisearlymarriage.ThislinkstothefactthatIQSsarelargelymulti-gradelearningenvironments,withclassesoftencateringtoawiderangeofages.IntheBauchicontext,whereintegrationhasbeenamorerecentprocess,thepresenceofolderchildrenreflectsthenumbersofchildrenwhowereout-of-school(formalschooling)untiltheIQSmadesecularlearningavailable.

Whilsttheacquisitionofformaleducation,includingforgirls,maybegainingacceptanceinthesixIQScases,earlymarriageisstillprevalent.Thisisparticularlyproblematicforgirlsovertheageof10whoareacquiringabasiceducationforthefirsttime.CommunitiesandLGEAofficersconsistentlyrefertothisasthekeychallengeforgirls’educationandlearningoutcomes.Yet,withcommunitiesconsideringitaspartoftheirIslamicfaith,theviewisthatitisalmost‘inevitable’.

174Tayloretal.(2014)‘EconomicOpportunitiesandObstaclesforWomenandGirlsinNorthernNigeria’,DFID.175 Zakaria,Y(2001)‘EntrepreneursatHome:SecludedMuslimWomenandHiddenEconomicActivitiesinNorthernNigeria’,NordicJournalofAfricanStudies,10(1):107-123(2001).


©EDOREN 230

Thereisnosetageforagirltogetmarried,ratherthisisdecidedbyreferenceto‘maturity’.176Girlsmaythusreceiveanynumberofyearsofschoolingbeforetheymarry,atagesrangingfrom12to20.Themostcommonlyreferredtoageformarriageis15.Dependingonthe‘ageofmaturity’,somegirlsintheGEP3communitiesmaynothavereceivedanyeducationbythetimecommunitiesdeemthemreadyformarriage:‘Shewasalreadygrownevenbeforetheschoolstarted’(Girls,BauchilowerperformingIQS).Ageofmaturityreferstothegirl’sphysicaldevelopment.IntheBauchilowerperformingIQS,theMallam/headteacherreferredtothefactthatwhenagirlhadphysicallydeveloped,itbecameimportantforhertobemarried.However,agirlwhowasstillundevelopedbeyond15couldcontinueherschooling.

Asmanygirls,especiallyinBauchi,appeartobearound12andarestillinP3,itisplausiblethatearlymarriagewillaffectretention,evenatthestageofprimaryeducation.Withsomecommunities,andinparticulargirls,nothavinghadaccesstoformaleducationpriortointegrationtheremaybegirlsstartingeducationatanolderage.ThequestionofhowthisrelatestoGEP3willneedfurtherexplorationatmidline.

Thereareoftenprovisionsmadeforgirlstocontinueeducationaftermarriage.However,thisisdependentonthewillingnessofthehusbandtoallowhiswifetoattendschool.Participantsstatethatpartiesincreasinglyagreeuponthisbeforemarriage,butsometimesthehusbandchangeshismindaftermarriage.Thoughparentsmaysaythattheirattitudestowardsgirls’educationarepositive,andthattheywantthemto‘gofar’andstudytoahighlevel,thereisanunderstandingthatonceagirlismarriedsheisnolongertheparents’responsibility.Asaresult,changesinparentalattitudeswillnotnecessarilytranslateintoimprovedretentiononcegirlsarebeyondtheageofmarriage.

‘AccordingtothereligionofIslam,agirlchildshouldmarryearly,butyouaskherhusbandtosponsorhercontinuededucation.Ihaveadaughterintheuniversity.Shegotmarriedlastyearandsheisstillcontinuingwithhereducation.Wewillnotdiscouragethegirlsiftheydesiretogotoschool,butwhentheygettotheirhusband’shousetheymaycontinueandtheresponsibilityisnolongeryours.’(Parents,BauchihigherperformingIQS)

Thus,thoughanattitudechangeonthepartofparentsisimportantintheyearsbeforemarriage(andinnegotiatingamarriage),changingtheattitudesofthemenwhoarelikelytomarrygirlsinthecommunityislikelytobeequallyessential.ThoughGEP3indirectlydoesthisthroughcurrentpupils,andparentsofboys,theremaybeaneedtoexplorefurthertheextenttowhichboys/mennotattachedorconnectedtotheIQSareinfluencedbytheintervention,andwhetherthiswillimpactongirls’continuededucationaftermarriage.

ThreeofthesixcasestudyIQSsreferredtoprovidingsomeformofeducationforwomenthatincludedformalsubjects.Integrationinitiativesthusstretchbeyondthedirecttargetgroupsofgirlstoalsoincludeincreasedawarenessoftheimportanceofformaleducationforwomen.However,inmostcasesitappearsthatwomenonlycontinuewithreligiousstudies.Whetherawomancontinuesformalschoolaftermarriageappearsdependentontheattitudesofherhusband-to-be.

‘Someofthemwhoearnsalariesandwhoalsoknowthattheirwivesareresponsibleallowthemtocontinuewiththeireducation.Butsomewillevenpromisethegirlbeforemarriagethatoncetheygetmarried,shewillcontinuewithhereducationonlytogobackonthepromiseoncethegirlbecomeshiswife.Somanymarriageshavebrokenupbecauseofthis.’(Girls,BauchitypicalIQS)

176Thisisareferencetoagirl’sbiologicalandphysicalmaturity,indicatingthepointatwhichherbodystartstodevelop.


©EDOREN 231

Girlscontinuingtoaccessschoolaftermarriagewillalsobedependentontheflexibilityoftheirnewhome-basedresponsibilitiesaswivesandmothers,andtherestrictionsimposedwherethepracticeofseclusionapplies.TheimportanceattachedtoreligiouslearningsuggeststhatgirlsandyoungwomenareencouragedtocontinuetheirQuraniceducationaftermarriage,andtheirmobilityforthisreasonisnotconstrained.Thismayofferawayformarriedwomentoalsocontinuetheirsecularlearning.

ThoughchangingnormsandpracticesaroundearlymarriageisnotadirectobjectiveofGEP3,thequalitativeresearchindicatesthatsuchchangeswillberequiredinordertofacilitatecontinuededucationofgirls.Whilesomestakeholdersseemtobeawareofthechallengeofearlymarriageforgirls’education,thisdoesnotnecessarilychangeattitudesorpractices.Forexample,theMallaminNigerlowerperformingIQSsaidheconsidersearlymarriageamainchallenge,andthathewouldliketoseegirlsstudytothelevelofuniversity.Atthesametime,itappearshehadmarriedhisowndaughtersatyoungages.AsimilarcontradictioncamefromtheBauchilowerperformingcase.FGDswithboysfurthermirrorthis.Someboysarehappytolettheirfuturewivescontinueeducationsince‘Iwantaneducatedwife’(Niger,typicalIQS),whilstothersconsideritsomethingthatisnotdone:‘Iwillnotallowmywife,weinthevillagewedon’tallowourwivestogotoschoolbecauseofsuspicionofinfidelity.’(Bauchi,lowerperformingIQS)

Additionally,changingnormsandpracticesintheIQScontextishighlylimitedbypressuresbothfromwithinthedirectcommunityandpotentiallyfromawiderreligiouscommunity.Parentsfacepressurestomarrytheirdaughterssince‘shewillsoonbecomeworthlessifsheisnotmarriedoff’(Girls,BauchitypicalIQS)

‘Somepeoplewillevenkeepmalicewithyourfatherifherefusestolistentothem.Insomecases,itiseventhegrandparentswhowagewaragainsttheirsonstomarrytheirdaughtersoffearlyandthenthefatherswillsuccumbtopressureandaskthedaughtertobringhomeamanshewantstomarryorahusbandwillbechosenforher.Thatisonemajorchallengewegirlsfacehere.Wearenotallowedtogotoschoolforaslongaswewant.’(Girls,BauchitypicalIQS)

Aslongasgirlsaredependentonmarryingforsocialsecurityandtomakealivingthisislikelytoimpactontheabilityagirlhastostudyfurther.Itisalsolikelytoaffectgirls’aspirationsinschoolsincetherewillbelimitedoptionsavailableforgirlstopursue.InNigerhigherperformingIQStheproprietorisfemaleandactivelystressedtheimportanceofformallyeducatinggirlsevenbeforetheGEP3integration:

‘Nowwehaverealisedthateducationisveryimportant.Beforewemarriedoutourdaughtersatatenderage,butnowwehaveseenthelight.Godwilling,weshallcontinuetoensurethatourchildrengeteducated.Itwasourfaulttohaveallowedourchildrentoplayinthepast,butnowthatwedonotallowourgirlstopayattentiontoboys,theyarenolongerinterestedinmarriage.Allchildrenareveryinterestedineducation.’(Parents,NigerhigherperformingIQS)


©EDOREN 232

Bibliography

Adediwura,A.andTayo,B.(2007).‘Perceptionofteachers’knowledgeattitudeandteachingskillsaspredictorofacademicperformanceinNigeriansecondaryschools’.EducationalResearchandReview,2,pp.165–171.

Anderson,M.andScamporlino,R.(2013)‘TheMasterofTeachingattheUniversityofMelbourne:aclinicalmodelforpre-serviceteacherpreparation’.InternationalSchoolsJournal,32(2),pp.33–42.

Arnold,C.(2011)CashTransfers,LiteratureReview.UKDepartmentforInternationalDevelopment.

Baird,S.,McIntosh,C.andŐzler,B.(2011)‘CashorCondition?EvidencefromaCashTransferExperiment’.TheQuarterlyJournalofEconomics,126(4),pp.1709–1753.

Benhassine,N.,Devoto,F.,Duflo,E.,Dupas,P.andPouliquen,V.(2014)‘TurningaShoveintoaNudge?A“LabeledCashTransfer”forEducation’.NBERWorkingPaper,No.19227.

Burns,D.,andWorsley,S.(2015)NavigatingComplexityinInternationalDevelopment.Rugby,UK:PracticalActionPublishing.

Cert,Y.(2010)‘TeacherEfficacyScale:TheStudyofValidityandReliabilityandPreserviceClassroomTeachers’Self’.JournalofTheoryandPracticeinEducation,6(1),pp.68–85.

CESE.(2015)Whatworksbest:evidence-basedpracticedtohelpimproveNSWstudentperformance.CentreforEducationalStatisticsandEvaluation(CESE).Availableathttp://www.cese.nsw.gov.au/images/stories/PDF/what_works_best.pdf,Accessedon5June2015.

Cogill,J.(2008)‘Primaryteachers’interactivewhiteboardpracticeacrossoneyear:changesinpedagogyandinfluencingfactors’.EdDthesisKing’sCollegeUniversityofLondon.Availableatwww.juliecogill.com.

Coladarci,T.(1992)‘Teachers’SenseofEfficacyandCommitmenttoTeaching’.TheJournalofExperimentalEducation,60(4),pp.323–337.

De,S.,Pettersson,G.,Morris,R.andCameron,S.(2015)TeacherDevelopmentProgramme(TDP),ImpactEvaluationofOutput1:In-ServiceTraining,FinalBaselineTechnicalReport,VolumeIResultsandDiscussion.Oxford,UK:EducationData,Research&EvaluationinNigeria(EDOREN)

Delahais,T.andToulemonde,J.(2012)‘Applyingcontributionanalysis:Lessonsfromfiveyearsofpractice’.Evaluation,18(3),pp.281–293.

DFID(2011)DFIDEthicsPrinciplesforresearchandevaluation.UKDepartmentforInternationalDevelopment.

Dundar,H.,Béteille,T.,Riboud,M.andDeolalikar,A.(2014)StudentLearninginSouthAsia:Challenges,Opportunities,andPolicyPriorities.DirectionsinDevelopment.Washington,DC:WorldBank.

EDOREN(2015)EvaluationFrameworkandPlan,UNICEF’sGirls’EducationProjectPhase3(GEP3).Oxford,UK:EducationData,Research&EvaluationinNigeria(EDOREN).

Getzels,J.W.andGuba,E.G.(1957)‘Socialbehaviourandtheadministrativeprocess’.SchoolReview,65,pp.423–441.

Glover,J.A.andRobert,H.B.(1990)Educationalpsychology:principlesandapplications.HarperColonsPublisher.


©EDOREN 233

Graham,A.,Powell,M.,Taylor,N.,Anderson,D.andFitzgerald,R.(2013)EthicalResearchInvolvingChildren.Florence:UNICEFOfficeofResearch–Innocenti.

Griffin,P.(2014)AssessmentforTeaching.CambridgeUniversityPress.

Guajardo,J.(2011)TeacherMotivation:TheoreticalFramework,SituationAnalysisofSavetheChildrenCountryOffices,andRecommendedStrategies.SavetheChildren.Availableathttp://www.oxfamnovib.nl/Redactie/Downloads/English/SPEF/281-24%20Teacher%20Motivation%20Report.pdf.

Henson,R.(2001)TeacherEfficacyResearchKeynoteAddress–TeacherSelf-Efficacy:SubstantiveImplicationsandMeasurementDilemmas.UniversityofNorthTexas.Availableathttp://www.uky.edu/~eushe2/Pajares/EREkeynote.PDF.

Humphreys,S.andCrawfurd,L.(2014)ReviewoftheLiteratureonBasicEducationinNigeria:IssuesofAccess,Quality,OutcomesandEquity.Oxford,UK:EducationData,Research&EvaluationinNigeria(EDOREN).

Jones,H.,Jones,N.,Shaxson,L.andWalker,D.(2013)Knowledge,policyandpowerininternationaldevelopment:apracticalframeworkforimprovingpolicy.ODIBackgroundNote.

Johnson,D.andHsieh,P.J.(2014)Thecontextofschooling,professionalcompetenciesandlearningoutcomesinnorthernNigeria.DraftSummaryReports.UNICEFGEP3.

Kauffman,D.(2011)‘ValidationoftheMotivationtoTeachScale’.H.U.JournalofEducation,40,pp.279–290.

Kelleher,Fetal.(2011)WomenandtheTeachingProfession:ExploringtheFeminisationDebate.UNESCOandtheCommonwealthSecretariat.

Leach,J.(2002)‘TheCurriculumKnowledgeofTeachers:anAnalyticReviewofaLargeScale,NationalElectronicConferenceEnvironment’.CurriculumJournal,3(1),pp.87–120.

Lynch,KandGood,H.(2011)‘Girlfriendly/childfriendly’schoolsprovideabrighterfutureinMali.UNICEF:Mali.Availableathttp://www.unicef.org/mdg/mali_59595.html.

Masters,G.N.(2013)Reformingeducationalassessment:Imperatives,principlesandchallenges.AustralianCouncilforEducationalResearch(ACER).Availableathttp://research.acer.edu.au/cgi/viewcontent.cgi?article=1021&context=aer,Accessedon21July2015.

Mayne,J.(2012)‘Contributionanalysis:Comingofage?’Evaluation,18(3),pp.270–280.

Mays,N.andPope,C.(1995)‘QualitativeResearch:RigourandQualitativeResearch’.BMJ.311(6997),pp.109–112.

McGrane(2014)UnpublishedreportonESSPINitemfunctioning.OPM,Oxford.

Merriam,S.B.(1988).Casestudyresearchineducation:Aqualitativeapproach.SanFrancisco:JosseyBass.

OECD.(2014)EducationataGlance2014:OECDIndicators.OECDPublishing.

OECDDevelopmentAssistanceCommittee(2010)DACGuidelinesandReferenceSeries:QualityStandardsforDevelopmentEvaluation.OECD.

Patton,M.Q.(2015)QualitativeResearchandEvaluationMethods.SagePublications.

Pedhazur,E.J.andSchmelkin,L.P.(1991).Measurement,DesignandAnalysis:AnIntegratedApproach.Hillsdale,NewJersey,LawrenceErlbaumAssociates.

RANA(2016)RANAQuarterlyReportSeptember–December2015.


©EDOREN 234

SparcandSavi(2015)StateoftheStates–PoliticalEconomy,ProspectsforReformandFiscalConditions(UnpublishedPowerPointPresentation,13July2015).

Suhr,DandShay,M.(2009)GuidelinesforReliability,ConfirmatoryandExploratoryFactorAnalysis.UniversityofNorthernColorado.Availableathttp://www.wuss.org/proceedings09/09WUSSProceedings/papers/anl/ANL-SuhrShay.pdf.

Tayloretal.(2014)EconomicOpportunitiesandObstaclesforWomenandGirlsinNorthernNigeria.DFID.

Tschannen-Moran,M.andWoolfolk-Hoy,A.(2001)‘Teachers’SenseofEfficacyScale’inTeacherefficacy:Capturinganelusiveconstruct.TeachingandTeacherEducation,17,783-805.Availableathttp://u.osu.edu/hoy.17/files/2014/09/TSES-+-scoring-zted8m.pdf.

UNICEF(2015a)UnpublishedStrategyPaper.GEP3TheoryofChange.GEP3.

UNICEF(2015b)UnpublishedStrategyPaper.Girls’EducationProject3,StrategyforIntegratedQur’anicSchools.GEP3.

UNICEF(2015c)UnpublishedStrategyPaper.Girls’EducationProject3,StrategyforEducationManagementInformationSystem(EMIS)CapacityBuilding.GEP3.

UNICEF(2015d)UnpublishedStrategyPaper.Girls’EducationProject3,GEP3ValueforMoneyStrategy2014.GEP3.

UNICEF(2015e)Girls’EducationprojectPhase3,OperationalPlan.RevisedJanuary2015.

UNICEF(2015f)UnpublishedStrategyPaper.Girls’EducationProject3,StrategyPaperonEarlyLearning(literacy&numeracy)RANA.GEP3.

UNICEF(2015g)UnpublishedStrategyPaper.Girls’EducationProject3,StrategyforBuildingtheCapacityofSchool/CentreBasedManagementCommittees.GEP3.

USAID(2014)NigeriaReadingandAccessResearchActivity:Resultsofthe2014HausaandEnglishEarlyGradeReadingAssessments(EGRAs)inGovernmentPrimarySchoolsandIQTECentersinJigawa,Kaduna,KanoandKatsina.RTIInternational.

VoicesforChange(2015)BeingaManinNigeria:PerceptionsandRealities.UKAid.

Yin,R.(1984)Casestudyresearch:Designandmethods.BeverlyHills,CA:SagePublishing.

YuandWatkins(2011)Attitudesandmotivationinsecondlanguageacquisition:AstudyofinternationalstudentsinChinafromaCulturalPerspective.UniversityofHongKong

Zakaria,Y.(2001)EntrepreneursatHome:SecludedMuslimWomenandHiddenEconomicActivitiesinNorthernNigeria.NordicJournalofAfricanStudies.10(1):107–123.


©EDOREN 236

AnnexB BaselineTheoryofChangeassessment

B.1 Focusofkeyinformantinterviewsperstate

Katsina Zamfara Sokoto Niger Bauchi

1. Cashtransfers � � �

2. Enrolmentdrives � � �

3. GirlsforGirls �

4. Earlylearning(EL)intervention � �

5. IQSsupport(TT,HTT,CBMC) � � �

6. SbTD(TTor/andHTT) � � �

7. SBMCsupport � �

8. EMIS � �

9. HilWA � �

10. GESC/advocacy � � � � �

Rationalforfocus‒ ELpilotstate‒ CAPPNGOactiveingirls

spaces

‒ ELpilotstate‒ StrongEMIStrackrecord‒ StakeholdersforEnrolmDr,

SBMCandHiLWAaresimilar

‒ CTpilotstate‒ IQSsupportstate‒ StrongEMIStrackrecord‒ StakeholdersforEnrolmDr,

andSBMCaresimilar

‒ CTpilotstate‒ IQSsupportstate‒ Poor13/14EMIStrackrecord‒ StakeholdersIQSfacilitator

trainingandHTtrainingaresimilar

‒ IQSsupportstate‒ Teachertrainingplanned‒ ActiveHiLWA


©EDOREN 237

B.2 OverviewofkeyinformantsinterviewedforthebaselineTheoryofChange

assessment

National

PersonalinterviewsatnationallevelinAbujatookplacebetween3-8August2015.

Organisation Person Position

FederalMinistryofEducation

1. AgwaomaJoyceEke HeadofGenderUnit

2. H.T.Abdu MemberofGenderUnit

3. N.T.Kpalobi MemberofGenderUnit

4. E.E.Nnorom MemberofGenderUnit

5. O.P.Oshiba MemberofGenderUnit

WorldBank 6. OlatundeAdetoyeseAdekola Educationspecialist

UniversityofGuelph 7. OlabanjiAkinola PdDStudent,researcher

TDP 8. LilianBreakell InterimDeputyProgrammeManager

ESSPIN 9. JohnKay Leadspecialist–educationquality

RTI/RARA 10. R.DrakeWarrick ChiefofParty

SPARC 11. MarkWalker Nationalprogrammemanager

Katsina

PersonalinterviewsinKatsinatookplaceon10-11August2015.TheinterviewwiththeSPARCstateprogrammemanagerwasbyphoned.d.16September2015.


StateMinistryofEducation

1. SagirIbrahimMuhammad PermanentSecretary

2. KabirLawalRuma DPRSSMoE

3. BintaAbdulmuminu Deskofficer

4. HalimaLawal DirectorofSchools

SUBEB5. AbuKasimJibia SecretarySUBEB

6. AliyuUmarImam DPRS

7. LawalMuhammedK/Seruri GEP3Deskofficer

SAME8. AhmedAhmedDoki ActingExecutiveDirector

9. AdamaSuleBakor DirectorWomenEducationFocalperson

CCTPIU,OfficeofSpecialAdviserChildDevelopment

10. MaharazuAbubakarSadiq PIUCoordinator

11. ShehuSadiq MISOfficer

CollegeofEducationDutsinama12. KabirMagaji GEP3FocalPerson

13. AdoIsmaelFuntua HeadofdepartmentEarlyChildEducation

CommunityActionforPopulationParticipation(CAPP) 14. FatimaYusufSaddiqh ProgrammeOfficer

SPARC 15. MohammedOlaniyi Stateprogrammemanager

TDPKatsinastateprogramme 16. HussainaHassan Stateteamleader


©EDOREN 238

17. MarkToscanini Deputystateteamleader

GEP3stateteam18. MaduguIbrahimGobir StateProjectCoordinator

19. SimonOsilamaIzuagie M&EConsultant

20. ShehuSuleimanAbdullahi Output2lead

UNICEFFieldOfficeKatsina 21. ShehuMohammed EducationOfficer

Zamfara

PersonalinterviewsinGusau,Zamfara,tookplaceon13-14August2015.TheinterviewwiththeSPARCstateprogrammemanagerwasbyphoned.d.16September2015.



1. SaniMande DirectorofHigherEducation

2. IbrahimM.Mailalle DPRS

3. MaheBello EMISOfficerSMoE

4. AbdurahmanAbdurazuk EMISOfficerSMoE

SUBEB

5. UsmanAbdullahi DirectorofSpecialProgrammes

6. AminuMusaKanoma DirectorofSocialMobilization

7. MaryamYahayaShantali DPRS

8. KasimuMu’azu EMISOfficerSUBEB

9. IbrahimMuhammadGusau EMISOfficerSUBEB

CollegeofEducationMaru 10. Dr.LawalSaadu FocalpersonGEP3,Lecturer

SBMC 11. AbubakarMohammadDogo SBMCStatechairman

HiLWA 12. MaryamIbrahim MemberofStateHilWA

FOMWAN 13. HalimaMohammedAmira(Leader)BunguduLGA,PermSec.MinistryofTourismZamfara

SPARC 14. HadjaraShibkau Stateprogrammemanager

CSACEFA 15. ShehuAdamuDinawa StatecoordinatorCSACEFA

GEP3stateteam16. Dr.IbrahimYabo StateProjectCoordinator

17. Dr.SalawSahid M&Eofficer

18. ZaraS.Abubakar Output2lead


©EDOREN 239

Sokoto

PersonalinterviewsinSokototookplaceon17-18August2015.



1. Alh.(DR.)M.B.Abubakar PermanentSecretary

2. MuhammadAttahiruAhmad DPRS

3. MamudaGaladima OperationsofficerCTP

4. MuhammadShamsuddeenSambo

EMISofficerCTP

5. AbdulkadirMalami Planning,M&EofficerCTP

SUBEB

6. IbrahimHaliruDingyadi Secretary

7. UmaruA.Boyi IQTEdeskofficer

8. JunaiduUmarJabo DirectorSocialMobilization

SAME9. AhmedBelloGidadawa Permanentsecretary

10. AbubakarMuhammadAlkammu

ProgrammeDirector

SBMC 11. AminuMusaYabo Statesecretary

CSACEFA 12. KabeerAliyu DeputyNationalModerator

CouncilofUlama 13. HamzaAlhajiSuleiman Stateassistantsecretary

UNICEFFieldOfficerSokoto14. TukurLabbo-Yabo Educationofficer

15. SafiyaTahirAbdullahi Educationspecialist

GEP3stateteam16. YahayaAliyuMaiyama Stateprojectcoordinator

17. HadizaI.Ibrahim Output1lead

Niger

PersonalinterviewsinMinna,Niger,tookplaceon20and24August2015.TheinterviewwiththeSPARCstateprogrammemanagerwasbyphoned.d.21September2015.


StateMinistryofEducation1. HajiyaFatimaAbdullahi ActingPermanentsecretary

2. GarbaA.Yahaya DPRS

3. GimbaShuaibu Chiefstatistician

SUBEB

4. AyubaUsmanKatajo Permanentsecretary

5. AlhajiMahmoudSaniBabanna

DirectorofSchoolServices

6. ChadoIsahDokogi IQTEdeskofficer

PIUCashtransferprogramme/EducationResourceCentre

7. SuleAgboola Coordinator

8. RamatuK.Umar PlanningofficerSMoE

9. AminatAbdullahi MISofficerSMoE

10. AdamuIdrisArzika TrainingofficerSUBEB

11. SaniMuhammedKusharki OperationsofficerSUBEB


©EDOREN 240

12. MohammedShuaibuKagara AccountantCTPSUBEB

SAME13. YunanaGagare Executivedirector

14. UmarBerryWalo Chiefaccountant

15. AbubakarAdamu GEP3focalperson

CollegeofEducationMinna

16. Prof.FarukR.Haruma Provost

17. Dr.RuthGaladimaDeputyprovostAcademics&coordinatorGirlChildEducation

18. Dr.MohammedIsahTsado Deputyprovost

19. IsahAbdullahiAloma GEP3deskofficer

AmeenLiteracyFoundation 20. HajiyaAminaKereAhmed Founder

LifeRehabFoundation

21. FatimaBintaFarouk Coordinator

22. Dr.Mrs.LadiShambo

23. YhuzaA.Abdullahi

24. EmmanuelAmuta

25. KabiruAhmed

SPARC 26. GraceOkuchuwi Stateprogrammemanager

UNICEFFieldOfficeKaduna 27. ElizabethObaba Educationofficer

GEP3stateteam

28. IbrahimU.Nagwamatse Stateprojectcoordinator

29. SakaAdebayoIbrahim M&Eofficer

30. HauwaAbdulkadir Output1lead

31. YusufKoboAngo Output2lead

Bauchi

PersonalinterviewsinBauchitookplaceon25-27August2015.


Stateministryofeducation

1. NasiruMuhammadYalwa PermanentSecretary

2. DanAzumiZakariTilde DPRS

3. UmarSani GEP3focalperson

4. BintaSambo DirectorSchoolServices

5. LouisB.Musa DirectorPolicy

SUBEB

6. SaboBappayoAhmed PermanentSecretary

7. AbdullazeezSaibo StateAlmajiriofficer

8. MuhammadBello IQEDeskOfficer

9. IsmailH.Umar DirectorSocialmobilization

CollegeofEducationAzare10. BoseOdeyemi

HeadofDepartmentEarlyChildhoodCareEducation(HeadTeacherDeskOfficer)

11. ShehuMohammedSeniorLecturerSchoolofScience(DeskOfficerSbTD)

BASAME12. HauwaAbubakarBalewa ExecutiveSecretary

13. HaliluUsmanRishi DirectorLiteracy/IQS

14. AsabeLawa GEP3focalperson


©EDOREN 241

15. AbubakarMaishanu DeskOfficerIQS

RARA16. SadiqIlelah Researchcoordinator

17. EmenguAugustus M&Eadvisor

HiLWA

18. JummeiS.Abudcar ChairHiLWA

19. TabittaaSabo DeputychairHiLWA

20. ZainabA.Gidado MemberHiLWA

21. VictoriaYusuf SecretaryHiLWA

22. SalamatuA.Lasan MemberHiLWA

23. TabithaC.Shawulu MemberHiLWA

RahamaWomenDevelopmentProgramme 24. MiriamIilya Executivedirector

FahimtaWomenandYouthDevelopmentInitiative 25. MaryamGarba Executivedirector

UNICEFFieldOfficeBauchi 26. MuntakeM.Mukhtar Educationofficer

GEP3stateteam27. SalehMuhammed Stateprojectcoordinator

28. BimbiTahir Output1lead


©EDOREN 246

AnnexD CalculationofWeightsandFinitePopulationCorrection

FactorsforGEP3School,TeacherandPupilData

D.1 CalculationofGEP3Weights

TheweightingproceduresfortheGEP3datareflecttheoveralldesignandimplementationofthesample,describedinaprevioussectionofthisreport.ThecombinedsamplingframeconsistsofalltheeligiblepublicandintegratedQur’anicschools(IQS)includedintheGEP3projectinfourstatesinnorthernNigeria:Zamfara,Katsina,NigerandBauchi.TheGEP3projectoperatesin6selectedLGAsineachstate.InZamfaraandKatsina3LGAsoutofthe6availablehavebeenselectedtobeincludedinthiscurrentstudy.TheselectionoftheLGAswaspurposiveandleadbyUNICEF.Thestratumcorrespondstostate/LGAandtype(publicschoolorIQS)intheframeforeachdomain,andtheprimarysamplingunits(PSUs)aretheschools.

TheGEP3baselinesurveyisdesignedtoevaluatetwoseparateinterventionsunderaoverallGEP3project:earlylearningintervention(RANA)andIQSsupport.Inthecaseoftheearlylearninginterventiontheschoolswererandomisedintotreatmentandcontrolgroupsaspartoftheevaluation.Sincethesampletreatmentandcontrolschoolsareeachrepresentativeoftheframeforthecorrespondingdomain,theweightswerecalculatedseparatelyforeach.

Inordertomaintaintheeffectivesamplesize,anysampleschoolforwhichdatacouldnotbecollected(becauseofeligibilityorresponseproblems)wasreplaced.Itwasnecessarytoselectmorethanonerandomsetofreserveschoolsforreplacementinsomestrata.Eachsetofreplacementschoolswasselectedrandomlyfromtheremainingschoolsthathadyetnotbeenselectedinthecorrespondingstratum.Sinceallschoolswithinthestratumhadthesameprobabilityofbeingselectedeitherintheoriginalsampleorinoneofthesetsofreplacements,ultimatelyeachschoolinthefinalcompletedsampleforeachstratumhadthesameprobabilityofselectionandcorrespondingweight,regardlesswhetheritwasanoriginalsampleschoolorfromoneofthesetsofreplacements.

Whensomeofthesampleschoolswerecontactedduringthesamplingimplementationitwasfoundthattheydidnotmeetthecriteriaforeligibility.ThiswasespeciallyaproblemforIQS,sincemanyschoolswerenotimplementingtheteachingprogramsasmandated.Asaresult,thecorrespondingframeforeachstratumwasreduced.Alloftheschoolsintheoriginalandreplacementsamplesthatwerecontactedwereclassifiedaseligibleorineligible.However,theeligibilityoftheremaining(non-contacted)schoolsintheframeisunknown.Inordertoestimatehowmanyschoolsintheframeforeachstratumwereeligible,itwasnecessarytocalculatetheproportionofeligibleschoolsamongthosewithinformation,andmultiplythetotalnumberofschoolsintheframeforthatstratumbythecorrespondingproportion.

Withineachstratum,thesampleschoolswereselectedwithequalprobability.Thereforetheweightfortheschoolsineachstratumwouldbecalculatedasfollows:

,

where:

= weightforthesampleschoolsinstratum(state/LGA,type)h

= estimatednumberofeligibleschoolsintheframeforstratumh,calculatedasdescribedpreviously

h

hSh s

SW^

=

ShW^

hS


©EDOREN 247

= numberofsampleschoolsinstratumhwithcompletedata

Thisweightistheinverseofthecorrespondingprobabilityofselectionoftheschoolswithineachstratum.Separateweightsarecalculatedforthetreatmentandcontrolschools.Thenumeratoroftheweight(estimatednumberofeligibleschoolsinthestratum)isthesameforthetreatmentandcontrolschools,sincetheyareselectedfromthesameframe,butthenumberofsampleschoolsinthedenominatorisbasedonthenumberofsampleschoolsinthecorrespondingtreatmentorcontrolgroup.Actually,sincethenumberofsampletreatmentandcontrolschoolsinastratumisgenerallythesame,bothtypesofschoolshavethesameweight,buteachgrouprepresentstheframeseparately.

TheheadteacherandtheCBMCineachsampleschoolwouldhavethesameweightasthecorrespondingschool.

Withineachsampleschoolasampleofteachersandpupilswasselectedforthecorrespondinginterviewsandtests,sotherearetwostagesofselection.Thefirststageprobabilityisthesameasthatforthecorrespondingschool,andthesecondstageprobabilityisequaltothenumberofselectedteachersorpupilswithsurveydatadividedbythetotalnumberofteachersorpupilsintheschool.Thereforethecorrespondingweightsarecalculatedastheinverseoftheprobabilitiesasfollows:

,

where:

= weightforthesampleteachersinthei-thsampleschoolofstratumh

= totalnumberofteachersinthei-thsampleschoolofstratumh

= numberofsampleteachersinterviewed/testedinthei-thsampleschoolof

stratumh

,

where:

= weightforthesamplepupilsinthei-thsampleschoolofstratumh

= totalnumberofpupilsinthei-thsampleschoolofstratumh

= numberofsamplepupilstestedinthei-thsampleschoolofstratumh

Inordertoascertainthedenominatorforestimationofwithinschoolselectionprobabilitiesanumberofsourcesneededtobecombined.Theprimarysourceofinformationonthetotalnumberofpupilsandteachersateachschoolwerethefieldworkmonitoringformsfilledoutbysupervisorsateachschool.Duetologisticchallenges(acartransportingsomeoftheformsgotstolen)othersourceshadtobeusedtocomplementthemissinginformation.Thisalsoallowedtobenchmarkthenumbersreportedonthe

hs

hi

hiShThi tTWW ×=

ThiW

hiT

hit

hi

hiShPhi p

PWW ×=

PhiW

hiP

hip


©EDOREN 248

monitoringforms.OthersourcesincludetheteacherselectionCAPIquestionnaireforthenumberofteachers,informationextractedfromunstructuredsupervisornotes,andsurveyresponsestoanumberofquestionswithintheheadteacherinterview177.Thefinaltotalnumberofeligiblepupilsandteachersineachschoolwasreachedbycomparingdifferentdatasources.Fieldworkmonitoringdatawasassumedtobethemostcrediblesourceofinformationandothersourceswereonlyusedifthefieldworkmonitoringdatawasmissingordeemnotcredible.Thecredibilityofthedatawasascertainedthroughbenchmarkingofdifferentsources.Decisiononthefinalnumberofeligiblerespondentswasreachedonschoolbyschoolbasis.

Theweightsforschools,teachersandpupilsspecifiedherearefullweightsthatexpandthedatatotheframelevel,andalsocorrectlyweighteachstratumincalculatingdomain-levelestimates.Theywillresultinthesamevaluesforestimatesofallrelativeindicators(suchasmeans,proportionsandotherratios)asnormalisedweights.Ifanyparticularanalysisrequiresnormalisedweights,theanalystcanstandardisetheweightsbydividingthembytheaverageweightforfullsampleincludedinthatparticularanalysis.

D.2 FinitePopulationCorrectionFactorsfortheGEP3Analysis

IncalculatingthesamplingerrorandconfidenceintervalsforestimatesofdifferentindicatorsfromtheGEP3data,itisveryimportantthatthevarianceestimatortakeintoaccountthefinitepopulationcorrection(fpc)factorsforeachstageofselection,giventhesmallframeandrelativelyhighsamplingrates.Basicallythefpcfactorisequalto1minusthesamplingrateforthecorrespondingstageofselection.Itreducesthecomponentofvariancefromeachsamplingstagebythecorrespondingproportion.Inthecaseofthefirststageofselectionthefpciscalculatedatthestratumlevelsincetheprobabilitiesforallschoolsarethesamewithinastratum.Forthesecondstageofselectionthefpcfactoriscalculatedattheschoollevel,sincethesecondstageprobabilitiesforteachersandpupilsvarybyschool.

FortheSVYcommandinStata,toapplythefpcfactorsinthevarianceestimationitisnecessarytoincludeinthedatafilethefirststageprobabilityforsampleschoolsinthecorrespondingstratumandthesecondstageprobabilityforthesampleteachersandpupilsinthecorrespondingschool.TheStatasoftwarethenusestheseprobabilitiestocalculatethefirstandsecondstagefpcfactors.

Basedonthediscussionoftheprobabilitiesandweightsabove,theprobabilityoftheschoolsforthefirststagefpcwouldbecalculatedasfollows:

,

where:

= firststageprobabilityforthesampleschoolsinstratumh

Inthecaseofthesecondstagefpcfactorsforthesampleteachersandpupils,thecorrespondingprobabilitieswouldbecalculatedasfollows:

177Thevariablesconsideredforauxiliaryinformationwere:totalnumberofteachersteachingnon-religioussubjectsingradesP1-P3,totalnumberofpupilsenrolledinP2,totalnumberofpupilsinP2presentonthedayonthesurvey(headcountperformedinclass),totalnumberofpupilsattendingP2asreportedbytheteacherduringtheheadcount.

Shhi

hh WS

sp 1^1 ==

hp1

hi

hiThi T

tp =2


©EDOREN 249

,

where:

=secondstageprobabilityforthesampleteachersinthei-thsampleschoolofstratumh

=secondstageprobabilityforthesamplepupilsinthei-thsampleschoolofstratumh

hi

hiPhi P

pp =2

Thip2

Phip2


©EDOREN 251


©EDOREN 252

AnnexF TeachermotivationsubscalesforGEP3

F.1 Distributionofthefiveteachermotivationsubscales

Thegraphsbelowshowsthedistributionofscoresoneachofthefivesubscales178179.Theaxesarethesameacrosseachofthefivegraphstomakethemeasiertocompare.Themean,standarddeviationandnumberofcasesarealsoshownwitheachgraph.

Threeofthefivesubscalesarestronglyskewed:

1. Interestandenjoyment,theskewheremaybeduetoacomplianceeffect

2. Effortandimportance,againwiththeskewpossiblyduetocomplianceeffects

3. Teacher-to-teacherinteraction.

178Thesubscales‘Importanceofteachingeffortvpupilbackground’and‘Importanceofteachingeffortvschoolinfrastructureandprofessionalsupport’makeupageneric‘perceivedefficacy’subscale.179The‘Perceivedteacherefficacy’andthe‘Pressureandworkrelatedtension’subscalesusedatathatwerereversecodedsoallgraphscanberead,fromtheperspectiveoftheGEP3,withvaluestotherightofthehorizontalscalebeingseenasabetteroutcomethanthosetotheleft.Forexample,theitem‘Iteachtoomanyclasses’inthePressureortensionsubscalewasreversecodedsothataresponseof‘Stronglyagree’ineffectbecomesequivalentto‘stronglydisagree’ontheotherscales.


©EDOREN 253


©EDOREN 254

AnnexG Randomisationchecksoftreatmentandcontrolgroup

G.1 SchoolLevelBalance

Table31: AllHeadTeachers

VariableNameTreatme

ntN

Treatme

ntMean

Control

N

Control

MeanTotalN

Total

MeanP-value

HeadTeacherisfemale 120 0 120 0 240 0.02 0.653

Hastheheadteacherbeenabsentinthelastterm? 113 0.6 112 0.7 225 0.61 0.114

NumberofdaystheHeadTeacherwasabsentinthelastterm?

63 7.7 74 8 137 7.88 0.911

Shareofschoolsthathaveseparatefunctioningtoiletforgirls

120 0.3 119 0.2 239 0.26 0.583

Numberofyearssinceintegration 60 1.8 59 2 119 1.92 0.652

TotalnumberofboysenrolledinintegratedsubjectsfromP1-P6

57 173.8 58 201.1 115 187.56 0.522

TotalnumberofgirlsenrolledinintegratedsubjectsfromP1-P6

57 121.4 58 145.2 115 133.41 0.487

Ratiooftotalnumberofchildrenstudyingintegratedsubjectstoteachers

54 42.4 60 42.2 114 42.33 0.978

RatiooftotalnumberofchildrenstudyingintegratedsubjectstoteachersinP1-P3

57 39.6 58 42.2 115 40.94 0.694

Ratioofgirlstoboysenrolledintotal 54 0.7 57 0.7 111 0.71 0.347

Headteachersatinonanylessonfortheentireduration

113 0.5 112 0.5 225 0.46 0.846

SchoolRepairs 120 0.9** 119 1 239 0.95 0.047

RatioofgirlstoboysinZamfarainP1-P3 25 0.6 30 0.6 55 0.64 0.841

RatioofgirlstoboysinKatsinainP1toP3 29 0.8 27 0.9 56 0.81 0.354

Ratiooftotalpupilsenrolledtototalroomsusedforteachingontheday

51 49.9 56 77.7 107 64.43 0.182

Fractionoffemaleteachersofintegratedsubjects 120 0.1 120 0.1 240 0.09 0.172

OverallF-test,F-stat 1.875 P-Value: 0.1722

Note:Statisticalsignificanceofmeandifferencesidentifedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.


©EDOREN 255

Table32: IQSHeadTeachers

VariableNameTreatme

ntIQSN

Treatme

ntIQS

Mean

Control

IQSN

Control

IQS

Mean

Total

IQSN

Total

IQS

Mean

P-value

HeadTeacherisfemale 60 0 60 0 120 0



33 5.9 39 7.4 72 6.71 0.412


60 0.1 60 0.1 120 0.06 0.7

Numberofyearssinceintegration 60 1.8 59 2 119 1.92 0.652


14 36.9 12 27.9 26 32.73 0.23


14 24.9 12 28 26 26.35 0.756


13 42.2 15 23.5 28 32.16 0.142


14 46 12 27.8 26 37.6 0.156



60 0.4 58 0.4 118 0.44 0.872

SchoolRepairs 60 0.9** 60 1 120 0.97 0.042




10 43.6 13 45.6 23 44.71 0.916

Fractionoffemaleteachersofintegratedsubjects 60 0.1** 60 0 120 0.04 0.017




©EDOREN 256

Table33: PublicSchoolHeadTeachers

VariableName

Treatme

ntPublic

Schools

N

Treatme

ntPublic

Schools

Mean

Control

Public

Schools

N

Control

Public

Schools

Mean

Total

Public

Schools

N

Total

Public

Schools

Mean

P-value

HeadTeacherisfemale 60 0.1 60 0 120 0.04 0.651



30 9.8 35 8.7 65 9.17 0.811


60 0.5 59 0.4 119 0.46 0.644


43 218.4 46 246.2 89 232.79 0.586


43 152.8 46 175.8 89 164.69 0.583


41 42.5 45 48.5 86 45.64 0.526


43 37.5 46 46 89 41.92 0.262



53 0.5 54 0.5 107 0.48 0.92

SchoolRepairs 60 0.9 59 0.9 119 0.92 0.313




41 51.4 43 87.4 84 69.84 0.175

Fractionoffemaleteachersofintegratedsubjects 60 0.1 60 0.1 120 0.14 0.855




©EDOREN 257

Table34: AllTeachers

VariableNameTreatme

ntN

Treatme

ntMean

Control

N

Control

MeanTotalN

Total

MeanP-value

Teacherisfemale 236 0.1 241 0.1 477 0.12 0.613

DoestheteacherhaveanSSCE? 236 0.7 241 0.7 477 0.72 0.515

DoestheteacherhaveanNCE? 236 0.6 241 0.5 477 0.53 0.33

DoestheteacherhaveaGrade2Certificate? 236 0.1 241 0.1 477 0.08 0.344

Hastheteacherbeenabsentinthelastterm? 221 0.7 229 0.7 450 0.66 0.862

Numberofdaystheteacherwasabsentinthelastterm 221 6.1 228 5 449 5.52 0.341

DoestheteacherspeakHausa? 177 1 179 1 356 1

Hastheteacherattendedtraininginthelast2years? 233 0.4 240 0.5 473 0.46 0.238

Istheteacherabletoidentifylowperformers? 231 0.4 232 0.5 463 0.49 0.43

Istheteacherabletogiveevidenceforjudgementsanddiagnose?

231 0.1 232 0.1 463 0.07 0.556

Doestheteacherhavewritingskills? 231 0.2 232 0.2 463 0.17 0.578

DoestheteacherhaveHausaknowledge? 231 2.9 232 2.8 463 2.87 0.763

Doestheteacherhavecomprehensionskills? 231 2.1* 232 2.3 463 2.21 0.091

Istheteacherabletointerpretwordsandphrases?

231 0.2 232 0.1 463 0.16 0.652



Table35: IQSTeachers

VariableNameTreatme

ntIQSN

Treatme

ntIQS

Mean

Control

IQSN

Control

IQS

Mean

Total

IQSN

Total

IQS

Mean

P-value

Teacherisfemale 88 0 90 0 178 0.02 0.306





©EDOREN 258


Numberofdaystheteacherwasabsentinthelastterm 85 8.7 87 7.4 172 8.04 0.524



Istheteacherabletoidentifylowperformers? 85 0.3 88 0.6 173 0.45 0.103


85 0 88 0 173 0.02 0.534



Doestheteacherhavecomprehensionskills? 85 1.6 88 1.9 173 1.76 0.261


85 0.1 88 0.1 173 0.1 0.211



Table36: PublicSchoolsTeachers

VariableName

Treatme

nt

Public

Schools

N

Treatme

nt

Public

Schools

Mean

Control

Public

Schools

N

Control

Public

Schools

Mean

Total

Public

Schools

N

Total

Public

Schools

Mean

P-value

Teacherisfemale 148 0.2 151 0.2 299 0.18 0.817





Numberofdaystheteacherwasabsentinthelastterm 136 4.4 141 3.5 277 3.95 0.479



Istheteacherableto 146 0.5 144 0.5 290 0.52 0.901


©EDOREN 259

identifylowperformers?


146 0.1 144 0.1 290 0.1 0.418



Doestheteacherhavecomprehensionskills? 146 2.3 144 2.6 290 2.48 0.173


146 0.2 144 0.2 290 0.19 0.278



G.2 PupilLevelBalance

Table37: AllPupils

VariableNameTreatme

ntN

Treatme

ntMean

Control

N

Control

MeanTotalN

Total

MeanP-value

TotalscoreinEnglishAssessment 1323 354.374 1326 356.946 2649 355.662 0.429

TotalscoreinHausaAssessment 1322 507.341 1327 509.568 2649 508.457 0.6

SpeaksHausaathome 1318 0.991 1305 0.994 2623 0.992 0.381

Femalepupil 1324 0.457 1327 0.444 2651 0.45 0.498

Ageofpupil 843 9.61* 834 9.345 1677 9.478 0.077

Hasachair 1290 0.9*** 1288 0.897 2578 0.875 0.001

Hasamotorcycle 1291 0.673 1290 0.656 2581 0.664 0.352

Hasacar 1290 0.176 1288 0.193 2578 0.185 0.256

HasaTV 1290 0.375 1289 0.386 2579 0.38 0.587

Hasacomputer 1286 0.05 1288 0.049 2574 0.049 0.92

Hasacamera 1289 0.052 1287 0.041 2576 0.047 0.194

Hasamobilephone 1291 0.9** 1289 0.951 2580 0.941 0.025

Hascattle 1291 0.556 1288 0.53 2579 0.543 0.187

Hasagoat 1291 0.865 1288 0.866 2579 0.865 0.973

Hasahorse,donkeyormule 1290 0.178 1289 0.164 2579 0.171 0.352

Hasasheep 1291 0.7** 1289 0.684 2580 0.705 0.018

Hasachicken 1291 0.92 1290 0.91 2581 0.915 0.356




©EDOREN 260

Table38: IQSPupils

VariableNameTreatme

ntIQSN

Treatme

ntIQS

Mean

Control

IQSN

Control

IQS

Mean

Total

IQSN

Total

IQS

Mean

P-value

TotalScoreinEnglishAssessment 618 390.115 644 393.839 1262 392.015 0.467


SpeaksHausaathome 617 0.987*** 643 1 1260 0.994 0.05

Sexofpupil 619 0.417 644 0.402 1263 0.409 0.598

Ageofpupil 409 11.675** 428 11.273 837 11.47 0.034

Hasachair 586 0.882*** 607 0.931 1193 0.907 0.004

Hasamotorcycle 586 0.683 607 0.664 1193 0.673 0.492

Hasacar 586 0.184 607 0.155 1193 0.169 0.176

HasaTV 586 0.416* 606 0.37 1192 0.393 0.099

Hasacomputer 586 0.07** 606 0.038 1192 0.054 0.015

Hasacamera 585 0.067*** 607 0.03 1192 0.048 0.003

Hasamobilephone 586 0.945 607 0.952 1193 0.949 0.593

Hascattle 586 0.584 606 0.561 1192 0.572 0.432

Hasagoat 586 0.882 607 0.883 1193 0.883 0.967


Hasasheep 586 0.783 607 0.751 1193 0.767 0.191

Hasachicken 586 0.935 607 0.931 1193 0.933 0.764



Table39: PublicSchoolPupils

VariableName

Treatme

nt

Public

Schools

N

Treatme

nt

Public

Schools

Mean

Control

Public

Schools

N

Control

Public

Schools

Mean

Total

Public

Schools

N

Total

Public

Schools

Mean

P-value




Sexofpupil 705 0.492 683 0.483 1388 0.488 0.737

Ageofpupil 434 7.7*** 406 7.313 840 7.494 0.005

Hasachair 704 0.8* 681 0.866 1385 0.848 0.066

Hasamotorcycle 705 0.665 683 0.649 1388 0.657 0.514

Hasacar 704 0.2*** 681 0.228 1385 0.198 0.006

HasaTV 704 0.3** 683 0.4 1387 0.37 0.023


©EDOREN 261

Hasacomputer 700 0** 682 0.059 1382 0.046 0.022

Hasacamera 704 0.04 680 0.051 1384 0.046 0.298


Hascattle 705 0.533 682 0.503 1387 0.518 0.258

Hasagoat 705 0.851 681 0.85 1386 0.851 0.965


Hasasheep 705 0.7** 682 0.625 1387 0.652 0.032

Hasachicken 705 0.908 683 0.892 1388 0.9 0.317



Table40: AllFemalePupils

VariableName

Treatme

ntGirls

N

Treatme

ntGirls

Mean

Control

GirlsN

Control

Girls

Mean

Total

GirlsN

Total

Girls

Mean

P-value

TotalscoreinEnglishAssessment 605 339.4** 589 351.064 1194 345.169 0.011



Sexofpupil 605 1 589 1 1194 1

Ageofpupil 375 9.176 367 9.155 742 9.166 0.923

Hasachair 604 0.9** 587 0.906 1191 0.887 0.035

Hasamotorcycle 605 0.674 588 0.651 1193 0.663 0.401

Hasacar 605 0.177 586 0.212 1191 0.194 0.13

HasaTV 604 0.4* 588 0.425 1192 0.399 0.072

Hasacomputer 603 0.043 586 0.055 1189 0.049 0.359

Hasacamera 603 0.033 586 0.038 1189 0.035 0.683

Hasamobilephone 605 0.932 588 0.947 1193 0.94 0.275

Hascattle 605 0.529 586 0.483 1191 0.506 0.113

Hasagoat 605 0.853 588 0.84 1193 0.847 0.542


Hasasheep 605 0.701 588 0.667 1193 0.684 0.205

Hasachicken 605 0.909 588 0.896 1193 0.903 0.455




©EDOREN 262

Table41: AllMalePupils

VariableName

Treatme

ntBoys

N

Treatme

ntBoys

Mean

Control

BoysN

Control

Boys

Mean

Total

BoysN

Total

Boys

Mean

P-value




Sexofpupil 719 0 738 0 1457 0

Ageofpupil 468 10** 467 9.495 935 9.726 0.025

Hasachair 686 0.8** 701 0.889 1387 0.866 0.012

Hasamotorcycle 686 0.672 702 0.66 1388 0.666 0.623

Hasacar 685 0.175 702 0.178 1387 0.177 0.888

HasaTV 686 0.376 701 0.352 1387 0.364 0.359

Hasacomputer 683 0.056 702 0.044 1385 0.05 0.327

Hasacamera 686 0.1* 701 0.044 1387 0.056 0.05


Hascattle 686 0.58 702 0.57 1388 0.575 0.696

Hasagoat 686 0.876 700 0.887 1386 0.882 0.525


Hasasheep 686 0.7** 701 0.699 1387 0.724 0.036

Hasachicken 686 0.93 702 0.922 1388 0.926 0.552




©EDOREN 263

AnnexH QualitativeEvaluationMatrixforSupportforIQS

Contribution Hypotheses Assumptions(outputtooutcomelevel) Coreareastoprobe(includingchangeovertimeforeach)Sourceof

information

Improvedlearningoutcomes(literacyandnumeracy),especiallyforgirls

Teachertrainingandmentoring,accesstofit-for-purposeteachingandlearningmaterialsandimprovedheadteacherpedagogicalleadershipandsupportwillleadtomoreeffective,gender-sensitiveteachingoftheintegratedcurriculum.

• Teachersandotherstakeholderswill,throughtheIQSactivities,bemorewillingandabletotakeonnewskills,rolesorwaysofworking.

• High-qualitymentoringcanimproveteachingquality.• Teacherpedagogiesandattitudesregardingtheroles,

responsibilitiesandcapabilitiesofgirlsandboyscanchangeasaresultoftrainingandmentoringingendersensitivity.

• Fit-for-purposeteaching/learningmaterialscanincreasequalityofteaching.

• Teachermotivationcanbeimprovedthroughteachermentoring,improvedheadteacherpedagogicalleadershipandsupport.

• Teachersandpupilswillbepresent,andsufficienttimewillbeallottedtotheteachingofformalsubjects.

• Theharmonisedcurriculumallotmentofeighthoursfortheteachingoftheintegratedcurriculumwillbeadheredto.

• Trainingandmentoringwillbewelltargeted.• ThereissufficienttimeforGEP3todemonstrateresults

withinthepilotperiod.

• Howthementoringandtrainingisimplemented.• Perceivedbenefitsandchallengesofworkingasateacher

inIQS.• HowtheorganisationoftheIQS,thetypeofpupilsit

attractsanditsrelationshipwiththecommunityinfluencestheeffectiveteachingoftheintegratedcurriculum.

• Perceivedbenefitsandchallengesoftheintegratedcurriculumandteachingandlearningmaterialssuppliedthroughtheintervention.

• Satisfactionwithacquiredskillsandskillsthatcanbeimproved.

• Willingnessandabilitytousenewskills.• Teacherperceptionsandattitudestowardgirlsandboys.• Waysinwhichtraininghasimpactedonteacherpraxis.• Extenttowhichgirlsareabletospeakupaboutissuesand

challengestheyface.• Schoolleaders’perceptionsandattitudestowardgirlsand

boys.• Perceivedchangesbeyondlearning,teachingandschool

leadershipthattheIQSsupporthascontributedto.• TheplausibilityofGEP3interventionsdemonstrating

resultswithinthepilotperiod.

• KIIswithmallams

• KIIswithheadteachers

• FGDswithparents

• FGDswithCBMCmembers

• FGDswithpupils

• QCOwithteacher

• TPDwithteacher

Improvedretentionofgirls

Improvedschoolplanning,mobilisedcommunityandGEP3resources,andimprovingheadteachersplanningwillimprovetheschoolexperienceforgirlsandtheywillbemotivatedtostayinschoolandwillbeexposedtoopportunitiestolearn.

• CBMCmembersarewillingandabletotakeonnewskills,knowledgeandwaysofworking.

• Schoolleadersarewillingtoreflectonandconsideredgenderequityissuesinschoolplanning.

• CBMCsarewillingtoinvestcommunityandGEP3resourcesinschoolinginfrastructurethatbenefitsgirls.

• Governmentandcommunitieshavesufficientcapacitytoretaingirlsandsustainquality.

• Communitymembersarewillingtoreflectonandconsidertheroles,responsibilitiesandcapabilitiesofgirls.

• CBMCpresenceandestablishmenthistory.• Securityandsafetysituationinthelocation.• Proprietorviewsoftheintegratedcurriculum.• Proprietor,communityandCBMCperceptionsand

attitudestowardgirlsandboys.• CBMCrepresentationofcommunityandinfluencing

power.

• KIIswithmallams


• KIIwithcommunityleaders

• FGDswithparents


• FGDswithpupils

Increasedenrolmentofgirls

CBMCsensitisation,animprovementinteachingqualityandagirl-friendlylearningenvironmentwillleadtoanincreaseddemandforgirls’education.

• Parentsandcommunitiesarewillingtoreflectonandconsidertheroles,responsibilitiesandcapabilitiesofgirls.

• Parentsandcommunitieshavesufficienteconomicresourcestoenrolmoregirls.

• CBMCsensitisationiswelltargetedandofhighquality.

• Parental,communityandCBMCperceptionsandattitudestowardgirlsandboys.

• Effectivenessofthecashtransferinalleviatingfinancialbarriers.

• Communityandparentalperceptionsoftheintegratedcurriculum.

• KIIswithmallams


• KIIswithcommunityleaders

• FGDswithparents


• FGDswithpupils


©EDOREN 264

AnnexI Evaluationgovernance,managementandindependence

TheevaluationoftheGEP3isdesignedandimplementedbyEDOREN,aDFID-fundedproject,andthereforehasclearaccountabilitytoDFIDthroughprojectreporting.TheevaluationistobegovernedbyasteeringgroupcomposedofDFIDstaff,aGEP3representative,andanEDORENrepresentative.Theindependenceoftheevaluationisassuredthroughtransparencyandrigorouspeerreview.

I.1 Governanceandmanagement

ThisevaluationframeworkandtheevaluationthatitsetsoutaremanagedandimplementedbyEDOREN,aDFID-fundedprojectmanagedbyOPM,andspecificallyEDORENworkstream1.EDORENworkstream1isresponsibleforthedesignoftheevaluationassetoutinthisdocumentandtheimplementationoftheEDOREN-ledevaluationactivities.TheEDOREN-ledevaluationactivitiesdrawextensivelyoninformationanddatagatheredbyOPM’sAbujaOfficeaspartofamilestonecontractundertheEDORENcontract.EDOREN-ledqualitativedatacollectionandanalysishasbeenundertakenbyanOPMQualitativeLeadwithateamofnationalqualitativeresearchersundertheEDORENcontract.TheOPMAbujabaselinesurveyteamreportregularlytotheEDORENworkstream1GEP3evaluationteaminordertoensurethatthebaselinemeetstherequirementsoftheevaluationframework.EDORENteammembersandtheOPMAbujaFieldworkManagerreporttotheGEP3EvaluationProjectManageronallaspectsoftheworkthatrelatetoevaluationresources,clientliaisonandtimelines.EDORENteammembersreporttotheGEP3TeamLeaderonalltechnicalcomponentsofthework.TheOPMAbujaFieldworkManagerreportstotheEDORENSurveyLeadonalltechnicalcomponentsofthework.

TheEDORENFieldworkManagerliaiseddirectlywithstateandLGAgovernmentrepresentativesregardingGEP3evaluationsurveyactivities.LiaisonwithstateandLGAgovernmentofficeswasundertakenpriortoanyschoolvisitsforpre-piloting,pilotingandsurveydatacollection.Afterundertakingschoolvisits,theGEP3FieldworkManagerreportedbacktogovernmentofficestodetailthedata-collectionactivities.

Figure83: GEP3evaluationgovernancearrangements

TheevaluationisgovernedbyDFID,whichentailsthattheevaluationteamreportstotheDFIDNigeriaeducationteamonprogresstowardevaluationobjectives.ThistakesplacethroughregularEDOREN


©EDOREN 265

quarterlywrittenreportingtoDFIDandsix-weeklyverbalprojectmanagementupdates.ThisprocessmeansthattheEDORENGEP3evaluationteamprovidesfortnightlyupdatesdirectlytotheDFIDEducatioAdvisor.

Inaddition,itwasrecommendedintheEvaluationFrameworkthatanEvaluationSteeringCommitteebeformalised,composedofDFIDNigeriaeducationandresultsadvisers,GEP3andEDORENstaff–agroupofpeoplethatalreadymeetsregularly.Thiscommitteecouldformallymeeteverysixmonthstodiscussevaluationprogress,andwouldberesponsibleforpeerreviewandQA(inadditiontoEDOREN’sinternalQAprocesses).

TheissueofgovernmentrepresentationontheGEP3EvaluationSteeringCommitteewasdiscussedbyEDOREN,UNICEFandDFID,180withUNICEFnotingthatgovernmentparticipationisimportantforownershipandtake-upofresults.EDORENfullyagreedandindicatedthatitwouldwelcometheparticipationofagovernmentrepresentativebutwouldseektohaveDFIDandUNICEFtaketheleadinapproachingandliaisingwithgovernmenttosetupthisparticipation.DFIDandUNICEFagreedtoliaisewithgovernmentonthisissue.Atpresent,DFIDandUNICEFplantousethebaselineevaluationreporttoengagegovernmentandbegintheprocessofestablishingaGEP3EvaluationSteeringCommitteeasoriginallyenvisaged.

I.2 Stakeholderinvolvementinframeworkdevelopment

TheGEPTeamLeadandtheGEP3ProjectManager/EducationSpecialistengagedwithDFIDNigeria,DFID’sResearchandEvidenceDivision,GEP3management,UNICEF,internationalresearcherswithinOPMandEDOREN,othereducationprojectsandimplementingagenciesinbothface-to-facemeetingsandoverSkypeonmultipleoccasionsthroughouttheevaluationdesignprocess.EngaginggovernmentthroughouttheevaluationdesignwascomplexinthecaseofGEP3astheEvaluationFrameworkwasputonholdduringtheredesignoftheGEP3programme.Therefore,theevaluationteamplacedaspecificfocusonengagingwithgovernmentduringthestate-levelstakeholderengagementactivityintheearlystagesoftheevaluation.

AftertheGEP3programmedesignwasfinalised,theimplementingpartner,DFIDandtheEDORENGEP3Evaluationteamspent10daysinMarch2015completingtheevaluationmatrix,byfinalisingtheevaluationquestions,assigningresponsibilitiesandresourcesagainstevaluationactivities,andfirmingupthetimingofvariousactivities.

TheGEP3evaluationteamhasdocumentedthediscussionsundertakenthroughoutthisengagementandhaskepthighlydetailedminutesofmeetingswithkeystakeholders.Thishasenabledtheteamtoreviewtheviews,prioritiesandconcernsofstakeholders,indetail,whiledevelopingtheevaluationframework.Atpointswheretrade-offsorcompromiseswerenecessaryduetoscopeorresources,theGEP3EvaluationTeamLeadandProjectManagercommunicatedtheoptionsandtrade-offstokeystakeholdersinordertoensureallpartieswereinvolvedinthedecision-makingprocesses.

DFIDeducationspecialistsandtheprogrammeimplementerswereprovidedwithafinaldraftoftheevaluationframeworkpriortofinalisationsoastoensurealldecisionsandagreementsarereflectedinthefinalproduct.

1802July2015EDORENGEP3Evaluationmeetingminutes,circulated8July2015.


©EDOREN 266

AnnexJ RobustnessChecksonAgeforMainRegressionModel

Table42: Hausa-MainModelwithAgeImputedforPupilswithMissingAgeData

Variables Coeff. StdErr T-Stat Pvalue


Agedfrom7to10 20.709 5.418 3.823 0.000***

Agedfrom11to15 60.352 6.835 8.830 0.000***

Agedover16 110.609 21.360 5.178 0.000***

PupilinsecondHWItertile 9.662 5.174 1.867 0.063*


Publicschool(notIQS) -73.045 9.629 -7.586 0.000***

Schoolhasgirltoilets 4.140 7.389 0.560 0.576

Schoolhaswater 9.821 7.219 1.360 0.175

Pupilattendsotherschool -1.819 5.956 -0.305 0.760

Schoolinruralareas -35.313 11.824 -2.987 0.003**



Teacherknowledge1 22.352 16.166 1.383 0.168




constant 475.843 58.768 8.097 0.000***Numberofobservations:2,538R-squared=0.280

Table43: English-MainModelwithAgeImputedforPupilswithMissingAgeData

Variables Coeff. StdErr T-Stat P-Value


Agedfrom7to10 7.771 5.020 1.548 0.123

Agedfrom11to15 39.531 6.544 6.041 0.000***

Agedover16 65.032 13.021 4.994 0.000***

PupilinsecondHWItertile 13.196 4.378 3.014 0.003**




Schoolhaswater 9.997 5.356 1.866 0.063*


Schoolinruralareas -35.278 9.084 -3.884 0.000***




©EDOREN 267


Teacherknowledge2 -0.172 3.563 -0.048 0.961



constant 302.603 50.886 5.947 0.000***Numberofobservations:2,538R-squared=0.226

Table44: Hausa-Fixed-EffectsModelwithAgeImputedforPupilswithMissingAgeData



Agedfrom7to10 12.423 4.899 2.536 0.012**

Agedfrom11to15 40.786 6.077 6.712 0.000***

Agedover16 77.086 21.974 3.508 0.001***



Pupilattendsotherschool 4.316 4.868 0.887 0.376Numberofobservations:2,580R-squared=0.502

Table45: English-Fixed-EffectsModelwithAgeImputedforPupilswithMissingAgeData



Agedfrom7to10 3.465 4.960 0.699 0.485

Agedfrom11to15 26.774 6.466 4.141 0.000***

Agedover16 42.177 13.284 3.175 0.002**

PupilinHWItertile 7.630 4.275 1.785 0.076*


Pupilattendsotherschool 8.633 5.009 1.723 0.086*Numberofobservations:2,580R-squared=0.421

ExclusionofAgeVariable

Table46: Hausa-MainModelwithBoardingVariableandNoAgeVariable



Pupilisaboarder 28.227 12.449 2.267 0.024**

PupilinsecondHWItertile 10.091 5.016 2.012 0.045**





©EDOREN 268

Schoolhaswater 11.559 7.874 1.468 0.143

Pupilattendsotherschool -2.310 6.238 -0.370 0.711

Schoolinruralareas -34.169 12.822 -2.665 0.008**

SchoolinKatsina 12.939 7.442 1.739 0.083*






constant 488.359 64.009 7.630 0.000***Numberofobservations:2,538R-squared:0.232

Table47: English-MainModelwithBoardingVariableandNoAgeVariable



Pupilisaboarder 15.718 9.213 1.706 0.089*

PupilinsecondHWItertile 13.515 4.180 3.233 0.001***




Schoolhaswater 10.959 5.863 1.869 0.063*


Schoolinruralareas -34.187 9.692 -3.527 0.001***







constant 310.617 54.694 5.679 0.000***Numberofobservations:2,538R-squared:0.190

Table48: Hausa-Fixed-EffectsModelwithBoardingVariableandNoAgeVariable



Pupilisaboarder 21.837 11.471 1.904 0.058*



Pupilattendsotherschool 4.041 4.799 0.842 0.401Numberofobservations:2,580


©EDOREN 269

R-squared:0.482

Table49: English-Fixed-EffectsModelwithBoardingVariableandNoAgeVariable



Pupilisaboarder 17.875 7.862 2.274 0.024**

PupilinHWItertile 7.400 4.198 1.763 0.079*


Pupilattendsotherschool 8.328 4.853 1.716 0.088*Numberofobservations:2,580R-squared:0.407


©EDOREN 270

AnnexK GEP3AssetIndex

TheassetsincludedintheGEP3PupilQuestionnairetoapproximatethehouseholdwealthwereacombinationoftheassetsavailableintheHNLSSandtheMICS.TheHNLSSincludesarangeofassets(bed,chair,car,radio,TV,etc.)butdoesnotincludeanylivestockvariables.AlthoughtheMICShassomelivestockvariables(cattle,sheep,goats,etc.)ithasfewerassetsoverall.Sinceapriori,wewereunsureofwhetherHNLSSassetsorMICSassetswoulddoabetterjobofdifferentiatingoursampleaccordingtohouseholdwealth,weincludedbothsetsofvariablesinthepupilquestionnaire.

Atthetimeofdesigningthequestionnaire,themodelinmindtodeveloptheassetindexincludedthefollowingassets(allfromHNLSS).ThisdecisionwasmadebasedontheabilityofthissetofassetstodifferentiatetheHNLSSsamplefortheGEP3statesintoquantiles.Insteadofassigningeachassetanequalweightingintheindex,PrincipleComponentAnalysiswasusedtoderivecoefficients(orrelativeweights)foreachassetinthecompositeindex181.Table50givesthecoefficientsderivedfromtheHNLSSdatawhileTable51givesanindicationoftheinitialmodel’sabilitytodifferentiatethesampleintomeaningfulquantiles.

Table50: CoefficientsoftheVariablesincludedintheassetindex,usingPCAonHNLSSdata

Variable Level HNLSScoefficient

Car 0 -0.03928 1 0.567445Fridge 0 -0.0619

1 0.511007

Radio 0 -0.11876 1 0.052846Bed 0 -0.11163 1 0.03721TV 0 -0.12148 1 0.370352Motorcycle 0 -0.06149 1 0.121296Chairs 0 -0.069 1 0.28338Mat 0 -0.02556 1 0.004833Sewingmachine 0 -0.02329

1 0.345258

Airconditioning 0 -0.01403 1 0.75289Fan 0 -0.11178

1 0.39432 Generator 0 -0.04403

1 0.54349

Washingmachine 0 -0.00099

1 0.474309

181Recommendedbymany,including:Filmer,D.,&Pritchett,L.H.(2001),McKenzie,DavidJ.(2005),Vyass,Seema,andLilaniKumaranayake(2006).


©EDOREN 271

Stove 0 -0.07671 1 0.385413Gascooker 0 -0.00486 1 0.730374Camera 0 -0.0033 1 0.6756Computer 0 -0.01056 1 0.793258Mobilephone 0 -0.16452 1 0.26767Bicycle 0 0.002975 1 -0.00737

Theabilityofthecompositeindextodividethesamplewasverifiedbytheproportionofthepopulationthatfellwithineachx-tile:

Table51: X-tiledivisionswiththeassetindex

Population SubdivisionwithHNLSS

Decile Quintile Quartile Tertile

1stDivision 11.78 20.31 25.27 40.89

2ndDivision 8.53 20.58 24.97 26.72

3rdDivision 10.31 19.16 24.95 32.39

4thDivision 10.27 20.11 24.81

5thDivision 9.35 19.85

6thDivision 9.81

7thDivision 10.07

8thDivision 10.04

9thDivision 9.97

10thDivision 9.88

AftertheGEP3datacollectionwascomplete,theintendedplanwastoapplytheHNLSSderivedcoefficientsonthedatacollectedtodevelopacompositeassetindexafterverifyingtherobustnessoftheproposedindexbyderivingcoefficientsusingPCAontheGEP3data.PCA,usingthecompletesetofassetsdiscussedabovecouldnotbecarriedoutontheGEPdatasincegascookersandwashingmachineswerenotcommoninthesample.

Table52: Ownershipofgascookerandwashingmachine

Variable Mean Std.Deviation TotalNAssetownedby

(n)

Gascooker .0236011 .1518316 2,627 62

Washingmachine .0125619 .1113947 2,627 33

Asshownabove,only1.25%ofthesampleownedawashingmachinewhileonly2.4%ofthesampleownedagascooker.Asaresult,thesevariableswerenotsuitableforinclusionintheGEP3assetindex.


©EDOREN 272

Were-ranthemodelbyexcludingtheabovevariablesi.e.washingmachineandgascooker,fromtheinitiallistofvariables(Model1),andbyexcludingtheabovevariablesbutaddingthelivestockvariables(Model2).ThePCAcoefficientsforeachmodelarepresentedbelow(Table53)alongwiththeabilityofthemodeltodifferentiatethesamplebyhouseholdwealth(Table54).

Table53: CoefficientsoftheVariablesincludedintheassetindex,usingPCAonGEP3data

Variable Variablevalue Model1coefficients Model2coefficients

Car 0 -0.085882 -0.079132Car 1 0.3937 0.362757Fridge 0 -0.066786 -0.060538Fridge 1 0.479059 0.43424Radio 0 -0.303788 -0.298376Radio 1 0.06644 0.065257Bed 0 -0.460478 -0.453385Bed 1 0.014353 0.014132Tv 0 -0.184467 -0.167311Tv 1 0.30777 0.279146Motorcycle 0 -0.258315 -0.258875Motorcycle 1 0.133362 0.133651Chairs 0 -0.288097 -0.283432Chairs 1 0.040449 0.039794Mat 0 -0.250035 -0.292829Mat 1 0.002247 0.002632Sewing_Machine 0 -0.142277 -0.135761Sewing_Machine 1 0.246866 0.23556Ac 0 -0.006957 -0.006557Ac 1 0.468972 0.44199Fan 0 -0.141947 -0.127169Fan 1 0.375635 0.336527Generator 0 -0.12895 -0.123251Generator 1 0.371828 0.355396Stove 0 -0.153832 -0.145716Stove 1 0.274951 0.260447Camera 0 -0.024753 -0.023854Camera 1 0.52441 0.505365Computer 0 -0.026476 -0.024326Computer 1 0.510992 0.469496Mobile 0 -0.47705 -0.476349Mobile 1 0.030202 0.030157Bicycle 0 -0.224093 -0.231451Bicycle 1 0.115498 0.119291Goat 0 -0.165671Goat 1 0.025922Cattle_Cows_Bulls 0 -0.125839Cattle_Cows_Bulls 1 0.107044Horse_Donkey_Mule 0 -0.049963Horse_Donkey_Mule 1 0.241427


©EDOREN 273

Sheep 0 -0.179079Sheep 1 0.075002Chicken 0 -0.221941Chicken 1 0.020529

Thougheachofthemodelssuccessfullydividedthesampleintotertiles(with33%ofthepopulationineachband),thedivisionintoquartiles,quintilesanddecilesvaried.

Table54: DivisionIntoTertiles,ByModel

Population Model1 Model2

1stDivision 33.61 33.35

2ndDivision 33.08 33.35

3rdDivision 33.31 33.31

Table55: DivisionIntoQuartiles,ByModel






Table56: DivisionIntoQuintiles,ByModel







Table57: DivisionIntoDeciles,ByModel











©EDOREN 274



Model1andModel2assignsimilarweightstocommonassetsincludedinthemodelwhichsuggeststhatModel1isarobustspecificationsinceityieldssimilarcoefficientswithasmallersetofvariables.Model2faresbetterindividingupthesampleintogroups–i.e.itsdivisionsholdforquintilesaswellasdeciles.Model1,ontheotherhand,successfullydividesthesampleintotertilesandquartiles.

AnadditionalrobustnesscheckforModel1istoderivethisnewmodelusingtheHNLSSdataforGEPstates,aswasdoneinitially.ThiscannotbedoneforModel2sinceithaslivestockvariableswhicharenotincludedintheHNLSSdataset.BycalculatingtheHNLSScoefficientsaccordingtoModel1,weseethatthiscompositeindexstillsplitsupthesampleintoapproximatelyequivalentquartilesandquintiles.

Table58: X-tiledivisionswiththeassetindex(Model1)onHNLSSdata

PopulationSubdivisionwithHNLSS

Decile Quintile Quartile Tertile

1stDivision 12.01 20.09 25.49 42.73

2ndDivision 8.08 22.64 26.22 23.95

3rdDivision 10.80 18.84 23.38 33.31

4thDivision 11.85 18.59 24.91


6thDivision 9.87

7thDivision 8.46

8thDivision 10.12

9thDivision 9.84

10thDivision 10.00

ByapplyingtheHNLSScoefficientsfromthenewmodel(excludinggascookerandwashingmachine),wefindthattheHNLSScoefficientswhenappliedtotheGEPdatastilldifferentiatethesampleintoequivalentquantilesandquartiles.Thoughtheabsolutevaluesoftheasset‘score’varybasedonthecoefficientsused,theabilitytosplitupthepopulationholds.


©EDOREN 275

Table59: ComparisonofModel1usingGEPandHNLSScoefficients

PopulationModel1

UsingGEPcoefficients UsingHNLSScoefficients

Min Max Mean Min Max Mean

1stDivision -3.486999 .702507 -1.001735 -1.083544 -.65499 -.8612033

2ndDivision -2.450946 1.155099 -.0800134 -.649942 -.371386 -.5463656

3rdDivision -2.225513 1.661305 .2626109 -.364134 .399303 -.0181731

4thDivision -1.6906 2.165238 .8327104 .401963 5.160044 1.389426*differentdatasourcesusedtoderivecoefficients,appliedtoGEP3dataset ThefinalchoiceofthecoefficientsusedtodeveloptheGEP3assetindexwasafunctionofthevariationinassetownershipintheactualdata,robustnesscheckonthemodelusingtheHNLSSdata,andtheabilityofthemodeltodifferentiatethesampleintogroupsbasedonassetownership.

Lastly,theassetindexwasonlyusedinthebaselineanalysisforthepupilsintheEarlyLearningevaluationsample.TheindexwasnotconstructedforpupilsintheIQSsupportsampleduetoasmallsampleofthesampledpupilsowning‘luxury’assetscombinedwithlimitedvariationinassetownership(Table60).Principalcomponentanalysisimputesascoreforeachobservationusingthevariationacrossthesample.However,inthecaseofIQSsupportpupils,ownershipofassetssuchascamera,computerandairconditionerwereperfectpredictorsofownershipofotherassetssuchasbed,matandmattress.Duetothis,principalcomponentanalysiswasnotpossibleforthissampleusingtherobustmodeldevelopedfortheEarlyLearningsample.HenceithasnotbeenreportedorusedintheanalysisofIQSsupportschoolsandassociatedpupiloutcomes.

Table60: OwnershipofassetsinIQSsupportsample

Variable Mean Std.Deviation TotalNAssetownedby

(n)

Computer .0664336 .2492564 572 38

Airconditioner .0313589 .1744377 574 18

Camera .0596491 .2370436 570 34


©EDOREN 276

AnnexL EarlyLearningDescriptiveStatistics

Table61: SchoolLevelDescriptiveStatistics

VariableLabel TotalMean Meanbystate

Katsina Zamfara

HeadTeacherPersonalCharacteristics

Respondentisfemaleefef

Mean 0.02 0.03 0.01

StandardError 0.009 0.016 0.009

N 240 120 120

Age

Mean 42.16 38.05 46.34


N 236 119 117

HeadTeacherProfessionalCharacteristics

NumberofyearsinHeadTeacherroleincurrentschool

Mean 6.43 4.19 8.72


N 233 118 115

Everworkedasateacher?

Mean 0.96 0.89 1

StandardError 0.036 0.113 0

N 28 9 19

Numberofyearsofteachingexperience

Mean 15.46 10.19 20.81


N 234 118 116

DoestheHeadTeacherhaveanNCE?

Mean 0.49 0.53 0.44


N 240 120 120

HeadTeacherhasSSCEorhigherdegree

Mean 0.58 0.7 0.47


N 232 115 117

HeadTeacherTrainings

HeadTeacherhasattendedtrainingorworkshop

Mean 0.63 0.61 0.66


N 237 118 119

MaximumtrainingsandworkshopsattendedbytheHeadTeacher

Mean 1.54 1.4 1.67


N 150 72 78

Numberofdaystrained

Mean 9.54 7.11 11.78


N 150 72 78

IstheHeadTeacher Mean 0.31 0.21 0.4


©EDOREN 277

trainedonmanagementandplanning?


N 150 72 78

IstheHeadTeachertrainedonteachinginHausa?

Mean 0.2 0.18 0.22


N 150 72 78

IstheHeadTeachertrainedonteachingmethods?

Mean 0.63 0.72 0.55


N 150 72 78

IstheHeadTeachertrainedoncommunityinvolvement?

Mean 0.25 0.08 0.41


N 150 72 78

IstheHeadTeachertrainedonintegrationactivities?

Mean 0.22 0.19 0.25


N 150 72 78

IstheHeadTeachertrainedondevelopinginstructionalmaterials?

Mean 0.09 0.04 0.13


N 150 72 78

IstheHeadTeachertrainedonschoolleadership?

Mean 0.32 0.19 0.45


N 150 72 78

IstheHeadTeachertrainedonextra-curricular?

Mean 0.08 0.07 0.09


N 150 72 78

IstheHeadTeachertrainedoncurriculumsubjects?

Mean 0.3 0.29 0.31


N 150 72 78

IstheHeadTeachertrainedonliteracyandnumeracy?

Mean 0.27 0.29 0.26


N 150 72 78

HeadTeacherabsenteeism

HowmanydaysistheHeadTeacherabsent?

Mean 7.9 5.96 10.05


N 137 72 65

HeadTeacherSchoolLeadership

Observeanylessonfortheentireduration

Mean 0.46 0.34 0.57


N 225 112 113

Totalnumberofwritten Mean 0.2 0 0.31


©EDOREN 278

observationsavailable StandardError 0.1 0 0.158

N 103 38 65

TakeactiontoimproveHeadTeacherattendance

Mean 0.71 0.67 0.75


N 225 112 113

Takeactiontoimprovepupilattendance

Mean 0.97 0.99 0.96


N 225 112 113

Receiveanymonitoringvisits

Mean 0.78 0.77 0.79


N 240 120 120

Numberofmonitoringvisitsperschool

Mean 8.27 8.47 8.07


N 186 92 94

Note:ForeachvariablethetableaboveincludesMeanvalue,StandardErrorandNumberofobservations.


Katsina Zamfara

SchoolStructure

Yearssinceschoolestablishment

Mean 36.34 32.86 39.58


N 191 92 99

PublicIsthereaPublicPrimarySchoolnearby?

Mean 0.89 0.85 0.93


N 120 60 60

Subjectstaughtinschool-English

Mean 0.87 0.81 0.92


N 240 120 120

Subjectstaughtinschool-Mathematics

Mean 0.97 0.95 0.98


N 240 120 120

Subjecttaughtinschool-SocialStudies/CivicEducation

Mean 0.57 0.53 0.61


N 240 120 120

Subjectstaughtinschool-GeneralScience/BasicScience

Mean 0.54 0.54 0.54


N 240 120 120

Subjectstaughtinschool- Mean 0.95 0.93 0.97


©EDOREN 279

Hausa StandardError 0.014 0.024 0.015

N 240 120 120

Subjectstaughtinschool-Arabic

Mean 0.65 0.47 0.83


N 240 120 120

Schoolteachesall5coreintegratedsubjects

Mean 0.07 0.03 0.1


N 120 60 60

Shareofintegratedteachersthatarepaid

Mean 0.82 0.83 0.81


N 116 60 60

IntegratedteachingperweekatP2level(hours)

Mean 9.41 10.14 8.72


N 211 102 109

DoesthisschoolhaveboardersinP2?

Mean 0.82 0.78 0.85


N 120 60 60

TeacherRoster

Numberofteachersofintegratedsubjects-AllLevels

Mean 6.43 4.61 8.24


N 240 120 120

Ratiooffemalestototalintegratedsubjectteachers

Mean 0.09 0.09 0.09


N 240 120 120

Numberofteachersofintegratedsubjects-P1-P3

Mean 4.31 3.41 5.22


N 240 120 120

Numberoffemaleteachersofintegratedsubjects-P1-P3

Mean 1.18 0.92 1.44


N 159 78 81

Ratiooffemalestototalintegratedsubjectteachers-P1-P6

Mean 0.12 0.13 0.11


N 159 78 81

Teacherturnoverasashareoftotalteachers

Mean 0.21 0.25 0.18


N 240 120 120

SchoolInfrastructure

SchoolRepairsMean 0.95 0.95 0.94



©EDOREN 280

N 239 120 119

Schoolneedsclassrooms

Mean 0.77 0.63 0.82

StandardError 0 0 0

N 40 11 29

Electricity

Mean 0.23 0.18 0.28


N 239 120 119

Electricitytoday

Mean 0.05 0.04 0.06


N 239 120 119

Schoolhasasourceofdrinkingwater

Mean 0.44 0.36 0.51


N 240 120 120

Availabilityofwaterfromsourcetoday

Mean 0.7 0.67 0.72


N 104 43 61

Numberofrooms

Mean 5.93 5.78 6.07


N 239 120 119

Numberofroomsusedforclassestoday

Mean 3.73 3.46 4


N 239 120 119

Numberoffunctioningtoilets

Mean 1.7 2.03 1.36


N 239 120 119

Schoolhasfunctionaltoiletsforgirls

Mean 0.26 0.23 0.3


N 239 120 119

Fenceorboundarywall

Mean 0.12 0.09 0.15


N 239 120 119

Collectionofbooks

Mean 0.11 0.14 0.09


N 239 120 119

Playgroundorsportsarea

Mean 0.55 0.53 0.58


N 239 120 119

GirlFriendliness

RatioofgirlstoboysinP1-P3

Mean 0.73 0.82 0.64



©EDOREN 281

N 111 56 55

Ratioofgirlstoboysintotal

Mean 0.71 0.81 0.6


N 111 56 55

RatioofgirlstoboyspresentonthedayofthevisitinP2

Mean 1.8 1.63 1.96


N 220 107 113


Mean 0.09 0.09 0.09


N 240 120 120


Mean 0.12 0.13 0.11


N 159 78 81

Schoolhasseparatefunctionaltoiletsforgirls

Mean 0.26 0.23 0.3


N 239 120 119

Committeesorgroupswithinschool

Mean 0.35 0.2 0.5


N 240 120 120

SchoolRecords

AttendancerecordexistsforP2forcurrentyear

Mean 0.78 0.84 0.71


N 240 120 120

UpdatedattendancerecordexistsforP2forcurrentyear

Mean 0.25 0.25 0.24


N 240 120 120

Updatedattendancerecordforlastyearavailable

Mean 0.24 0.23 0.25


N 240 120 120

EnrolmentrecordexistsforP2currentyear

Mean 0.48 0.47 0.5


N 240 120 120

Updatedenrolmentrecordforcurrentyearavailable

Mean 0.5 0.51 0.49


N 240 120 120

Updatedenrolmentrecordforlastyearavailable

Mean 0.49 0.48 0.5


N 240 120 120


©EDOREN 282

Table62: TeacherLevelDescriptiveStatistics

Writtenrecordsofmeetingskept

Mean 0.25 2.3 0.28


N 200 93 107

Numberofmeetingrecordsavailable

Mean 1.84 2.34 1.48

StandardError 0 0 0

N 50 21 29


Mean 0.6 0.51 0.69


N 240 120 120

Updatedteacherattendancerecordexists

Mean 0.48 0.44 0.52


N 240 120 120


Mean 0.81 0.85 0.77


N 240 120 120

TimetableprovidedforP2orequivalent

Mean 0.49 0.45 0.53


N 240 120 120


VariableLabel TotalMean

Meanbystate

Katsina Zamfara

TeacherPersonalCharacteristics

Respondentisfemale

Mean 0.15 0.17 0.13


N 477 221 256

Age

Mean 36.08 35.08 36.74


N 471 219 252

SpeaksYoruba

Mean 0.02 0.02 0.03


N 356 139 217

SpeaksHausa?

Mean 1 1 1

StandardError 0 0 0

N 356 139 217

SpeaksEnglish?Mean 0.85 0.86 0.85



©EDOREN 283

N 356 139 217

SpeaksArabic?

Mean 0.22 0.21 0.23


N 356 139 217

ProfessionalCharacteristics

TeacheriscurrentlyteachingP1-P3equivalent

Mean 1 1 1

StandardError 0 0 0

N 357 140 217

TeacheriscurrentlyteachingP3-P6orequivalentlevel

Mean 0.3 0.55 0.17


N 357 140 217

Teacheriscurrentlyteachingatupperandlowerlevels

Mean 0.3 0.55 0.17


N 357 140 217

DoestheteacherhaveanNCE?

Mean 0.49 0.64 0.48


N 240 221 256

Doestheteacherhaveprofessionaleducationalqualifications?

Mean 0.7 0.82 0.62


N 477 221 256

DoestheteacherhaveGrade2orequivalentqualification?

Mean 0.07 0.05 0.09


N 477 221 256


Mean 0.54 0.64 0.48


N 477 221 256

Highestacademicqualificationoftheteacherisprimaryeducation

Mean 0.1 0.09 0.11


N 477 221 256

HighestacademicqualificationoftheteacherisJSS

Mean 0.04 0.05 0.03


N 477 221 256

HighestacademicqualificationoftheteacherisSSCE

Mean 0.72 0.75 0.7


N 477 221 256

HighestacademicqualificationoftheteacherisOND

Mean 0.08 0.05 0.09


N 477 221 256

HighestacademicqualificationisBA/BSc

Mean 0.02 0.02 0.03



©EDOREN 284

N 477 221 256

Doestheteacherhavereligiouseducationqualification?

Mean 0.19 0.16 0.21


N 477 221 256

Totalexperienceasateacherinthisschooloranyotherschool

Mean 10.03 8.63 10.96


N 476 221 255

Totalyearsasateacherincurrentschool

Mean 6.2 6.63 5.96


N 406 167 239

Teacherhasatleast2yearsofteachingexperienceinanyschool

Mean 0.89 0.81 0.95


N 477 221 256

Teacherhasatleast2yearsofteachingexperienceinthecurrentschool

Mean 0.71 0.75 0.69


N 406 167 239

TeacherTrainings

Hastheteacherattendedtrainingduringlast2years?

Mean 0.42 0.44 0.41


N 473 218 255

TotalnumberoftrainingsattendedbytheTeacher

Mean 1.31 1.25 1.35


N 216 101 115

Numberofdaystrained

Mean 8.9 8.61 9.11


N 216 101 115

HastheteacherattendedtrainingorganisedbySUBEB?

Mean 0.17 0.16 0.17


N 216 101 115

HastheteacherattendedtrainingorganisedbyGEP/UNICEF?

Mean 0.47 0.41 0.51


N 216 101 115

HastheteacherattendedtrainingorganisedbyLGEA

Mean 0.09 0.11 0.08


N 216 101 115

IstheteachertrainedonteachinginHausa?

Mean 0.19 0.12 0.24


N 216 101 115

Istheteachertrainedon Mean 0.77 0.77 0.77


©EDOREN 285

teachingmethods? StandardError 0.035 0.047 0.05

N 216 101 115

Istheteachertrainedoncommunityinvolvement?

Mean 0.11 0.08 0.13


N 216 101 115

Istheteachertrainedonintegrationactivities?

Mean 0.11 0.11 0.12


N 216 101 115

Istheteachertrainedondevelopinginstructionalmaterials?

Mean 0.11 0.09 0.12


N 216 101 115

Istheteachertrainedonmanagementandplanning?

Mean 0.15 0.15 0.16


N 216 101 115

Istheteachertrainedonschoolleadership?

Mean 0.15 0.14 0.17


N 216 101 115

Istheteachertrainedonextra-curricular?

Mean 0.07 0.08 0.07


N 216 101 115

Istheteachertrainedoncurriculumsubjects?

Mean 0.21 0.31 0.14


N 216 101 115

Istheteachertrainedonliteracyandnumeracy?

Mean 0.34 0.24 0.4


N 216 101 115

Teacherabsenteeism

Teacherisabsentatleastonceduringlast3months

Mean 0.63 0.57 0.67


N 450 209 241

Numberofdaysteacherisabsentduringthelast3months

Mean 7.73 6.21 8.58


N 294 127 167

TeacherMotivation

Teachermotivationcompositeindex

Mean N/A 2.98 2.99

StandardError N/A 0.022 0.02

N N/A 220 254

TeacherKnowledge


©EDOREN 286

Table63: PupilLevelDescriptiveStatistics


Katsina Zamfara

PupilCharacteristics

Femalepupils

Mean 0.48 0.48 0.47


N 2651 1294 1357

Age

Mean 9.14 9.3 8.96


N 1677 871 806

PupilspeaksEnglishathome

Mean 0 0 0

StandardError 0.001 0 0.002

N 2623 1279 1344

PupilspeaksHausaathome

Mean 0.99 0.99 0.99


N 2623 1279 1344

CurrentlyattendingP2orequivalentlevel

Mean 0.98 0.97 0.99


N 2651 1294 1357

Attendingotherschoolbesidesthisone

Mean 0.54 0.46 0.61


Teachersubjectknowledgecompositeindex

Mean 0.49 0.51 0.48


N 463 211 252

Teacherpedagogicalknowledgecompositeindex

Mean 0.05 0.06 0.05


N 463 211 252

Mother-Tongueteaching(Hausa)inearlygradesindex

Mean 38.33 39.77 37.4


N 459 209 250

Hausateaching/learningmaterialscompositeindex

Mean 0.02 0.03 0.02


N 469 217 252

Syllabus/Curriculumknowledge

Mean 0.89 0.77 0.97


N 463 211 252



©EDOREN 287

N 2651 1294 1357

Pupilisaboarder

Mean 0.09 0.09 0.1


N 2651 1294 1357

Facesdifficultyingettingtoschool

Mean 0.03 0.03 0.02


N 2317 1119 1198

PupilAssessmentScaledScores

Hausaliteracyscaledscore

Mean 504.03 506.67 501.47


N 2649 1293 1356

Englishliteracyscaledscore

Mean 352.95 353.27 352.64


N 2649 1293 1356



©EDOREN 288

AnnexM IQSchoolSupport(IQSS)DescriptiveStatistics

Table64: SchoolLevelDescriptiveStatistics


Bauchi Niger

HeadTeacherPersonalCharacteristics

Respondentisfemale

Mean 0.03 0 0.07


N 60 30 30

Age

Mean 40 44.07 35.63


N 56 29 27

HeadTeacherProfessionalCharacteristics

NumberofyearsinHeadTeacherroleincurrentschool

Mean 9.61 14.48 4.8


N 60 29 30

Everworkedasateacher?

Mean 0.89 1 0.76

StandardError 0 0 0

N 9 5 4

Numberofyearsofteachingexperience

Mean 13.58 17.41 9.62


N 59 30 29

DoestheHeadTeacherhaveanNCE?

Mean 0.23 0.27 0.2


N 60 30 30

HeadTeacherhasSSCEorhigherdegree

Mean 0.4 0.23 0.57


N 60 30 30

HeadTeacherhasprofessionaleducation

Mean 0.33 0.4 0.27


N 60 30 30

HeadTeacherhasreligiouseducation

Mean 0.38 0.56 0.2


N 60 30 30

HeadTeacherTrainings

HeadTeacherhasattendedtrainingorworkshop

Mean 0.43 0.37 0.5


N 60 30 30


©EDOREN 289

MaximumtrainingsandworkshopsattendedbytheHeadTeacher

Mean 1.34 1.45 1.26

StandardError 0 0 0

N 26 11 15

Numberofdaystrained

Mean 4.72 5.28 4.31

StandardError 0 0 0

N 26 11 15

IstheHeadTeachertrainedonmanagementandplanning?

Mean 0.27 0.36 0.2

StandardError 0 0 0

N 26 11 15

IstheHeadTeachertrainedonteachinginHausa?

Mean 0.27 0.46 0.13

StandardError 0 0 0

N 26 11 15

IstheHeadTeachertrainedonteachingmethods?

Mean 0.85 0.82 0.87

StandardError 0 0 0

N 26 11 15

IstheHeadTeachertrainedoncommunityinvolvement?

Mean 0.16 0.18 0.14

StandardError 0 0 0

N 26 11 15

IstheHeadTeachertrainedonintegrationactivities?

Mean 0.31 0.36 0.27

StandardError 0 0 0

N 26 11 15

IstheHeadTeachertrainedondevelopinginstructionalmaterials?

Mean 0.04 0 0.07

StandardError 0 0 0

N 26 11 15

IstheHeadTeachertrainedonschoolleadership?

Mean 0.31 0.63 0.07

StandardError 0 0 0

N 26 11 15

IstheHeadTeachertrainedonextra-curricular?

Mean 0.04 0 0.07

StandardError 0 0 0

N 26 11 15

IstheHeadTeachertrainedoncurriculumsubjects?

Mean 0.11 0.09 0.13

StandardError 0 0 0

N 26 11 15

IstheHeadTeachertrainedonliteracyandnumeracy?

Mean 0.31 0.36 0.26

StandardError 0 0 0

N 26 11 15

HeadTeacherabsenteeism


©EDOREN 290

HowmanydaysistheHeadTeacherabsent?

Mean 5.87 5.88 5.86


N 47 28 19

HeadTeacherSchoolLeadership

Observeanylessonfortheentireduration

Mean 0.42 0.3 0.56


N 57 30 27

Totalnumberofwrittenobservationsavailable

Mean 0.04 0.11 0

StandardError 0 0 0

N 24 9 15

TakeactiontoimproveHeadTeacherattendance

Mean 0.61 0.63 0.59


N 57 30 27

Takeactiontoimprovepupilattendance

Mean 0.91 0.93 0.89


N 57 30 27

Receiveanymonitoringvisits

Mean 0.38 0.44 0.33


N 60 30 30

Numberofmonitoringvisitsperschool

Mean 3.75 4.24 3.1

StandardError 0 0 0

N 23 13 10

Whocameformonitoringvisits-LGEAOfficer?

Mean 0.7 0.69 0.7

StandardError 0 0 0

N 23 13 10

Whocameformonitoringvisits-SAMEOfficer?

Mean 0.09 0 0.2

StandardError 0 0 0

N 23 13 10

Whocameformonitoringvisits-SUBEBOfficer?

Mean 0.13 0.08 0.2

StandardError 0 0 0

N 23 13 10

Whocameformonitoringvisits-MinistryofEducationOfficer?

Mean 0.04 0.08 0

StandardError 0 0 0

N 23 13 10

Whocameformonitoringvisits-UBECOfficer?

Mean 0.04 0 0.1

StandardError 0 0 0

N 23 13 10


©EDOREN 291

Whocameformonitoringvisits-TrainersfromSBMC/CBMCtraining?

Mean 0.04 0 0.1

StandardError 0 0 0

N 23 13 10

Whocameformonitoringvisits-RepresentativesfromNGOs,donorsorotherexternals?

Mean 0.17 0.23 0.1

StandardError 0 0 0

N 23 13 10

Whocameformonitoringvisits-Others(Specify)

Mean 0.26 0.38 0.1

StandardError 0 0 0

N 23 13 10

Whocameformonitoringvisits-Don’tknow?

Mean 0.13 0.15 0.1

StandardError 0 0 0

N 23 13 10

Whywasactionnottaken?–Teacherattendanceisnotaproblem

Mean 0.23 0.37 0.1


N 60 30 30

Reasonsforteacher’sabsence-Transport

Mean 0.05 0.1 0


N 60 30 30

ReasonsforTeacher’sAbsence-OwnIllness

Mean 0.47 0.6 0.33


N 60 30 30

Reasonsforteacher’sabsence-Illnessoffamilymembers

Mean 0.25 0.27 0.23


N 60 30 30

Reasonsforteacher’sabsence-Lateornon-paymentofsalary

Mean 0.15 0.07 0.23


N 60 30 30

Reasonsforteacher’sabsence-salarycollection

Mean 0.02 0 0.03


N 60 30 30

Socialorreligiousobligations

Mean 0.17 0.17 0.17


N 60 30 30



©EDOREN 292


Bauchi Niger

SchoolStructure

Yearssinceschoolestablishment

Mean 26.65 34.81 18.15


N 53 27 26

PublicPrimarySchool?

Mean 0.98 0.97 1


N 60 30 30

Subjectstaughtinschool-English

Mean 0.92 0.97 0.87


N 60 30 30

Subjectstaughtinschool-Mathematics

Mean 0.92 0.97 0.87


N 60 30 30

Subjecttaughtinschool-SocialStudies/CivicEducation

Mean 0.23 0.17 0.3


N 60 30 30

Subjectstaughtinschool-GeneralScience/BasicScience

Mean 0.2 0.2 0.2


N60

30 30

Subjectstaughtinschool-Hausa

Mean 0.73 0.7 0.77


N 60 30 30

Subjectstaughtinschool-Arabic

Mean 0.6 0.6 0.6


N 60 30 30

Schoolteachesall5coreintegratedsubjects

Mean 0.12 0.03 0.2


N 60 30 30

Shareofintegratedteachersthatarepaid

Mean 0.18 0.03 0.33


N 60 30 30

IntegratedteachingperweekatP2level(hours)

Mean 2.98 2.2 3.81


N 58 30 30

DoesthisschoolhaveboardersinP2?

Mean 0.35 0.43 0.27



©EDOREN 293

N 60 30 30

TeacherRoster

Numberofteachersofintegratedsubjects-AllLevels

Mean 2.78 2.39 3.18


N 60 30 30

IQSonlyhasoneteacherofintegratedsubjects

Mean 0.25 0.33 0.16


N 60 30 30


Mean 0.9 0.03 0.16


N 60 30 30

Numberofteachersofintegratedsubjects-P1-P3

Mean 2.22 1.93 2.5


N 60 30 30

Numberoffemaleteachersofintegratedsubjects-P1-P3

Mean 0.13 0 0.27

StandardError 0 0 0

N 23 12 11


Mean 0.03 0 0.07

StandardError 0 0 0

N 23 12 11

Teacherturnoverasashareoftotalteachers

Mean 0.23 0.24 0.22


N 60 30 30

SchoolInfrastructure

SchoolRepairs

Mean 0.9 1 0.8


N 60 30 30

Schoolneedsclassrooms

Mean 0.79 0.7 1

StandardError 0 0 0

N 14 10 4

Electricity

Mean 0.4 0.3 0.5


N 60 30 30

Electricitytoday

Mean 0.28 0.3 0.27


N 60 30 30

Schoolhasasourceofdrinkingwater

Mean 0.3 0.3 0.3


N 60 30 30


©EDOREN 294

Availabilityofwaterfromsourcetoday

Mean 0.89 0.89 0.89

StandardError 0 0 0

N 18 9 9

Numberofrooms

Mean 2.67 2.73 2.61


N 59 30 29

Numberofroomsusedforclassestoday

Mean 2.08 2.1 2.05


N 60 30 30

Numberoffunctioningtoilets

Mean 0.35 0.3 0.4


N 60 30 30

Schoolhasfunctionaltoiletsforgirls

Mean 0.08 0.1 0.07


N 60 30 30

Fenceorboundarywall

Mean 0.22 0.13 0.3


N 60 30 30

Collectionofbooks

Mean 0.03 0.03 0.03


N 60 30 30

Playgroundorsportsarea

Mean 0.36 0.53 0.2


N 60 30 30

GirlFriendliness

RatioofgirlstoboysinP1-P3

Mean 1.22 1.55 0.85

StandardError 0 0 0

N 19 10 9

Ratioofgirlstoboysintotal

Mean 1.31 1.64 0.95

StandardError 0 0 0

N 19 10 9

RatioofgirlstoboyspresentonthedayofthevisitinP2

Mean 1.15 1.21 1.09


N 58 29 29


Mean 0.09 0.03 0.16


N 60 30 30

Ratiooffemalestototalintegratedsubject

Mean 0.03 0 0.07

StandardError 0 0 0


©EDOREN 295

teachers-P1-P3 N 23 12 11

Schoolhasseparatefunctionaltoiletsforgirls

Mean 0.08 0.1 0.07


N 60 30 30

Committeesorgroupswithinschool

Mean 0.03 0 0.06


N 60 30 30

SchoolRecords

AttendancerecordexistsforP2forcurrentyear

Mean 0.33 0.43 0.23


N 60 30 30

UpdatedattendancerecordexistsforP2forcurrentyear

Mean 0.07 0.1 0.03


N 60 30 30

Updatedattendancerecordforlastyearavailable

Mean 0.05 0.07 0.03


N 60 30 30

EnrolmentrecordexistsforP2currentyear

Mean 0.32 0.33 0.3


N 60 30 30

Updatedenrolmentrecordforcurrentyearavailable

Mean 0.32 0.33 0.3


N 60 30 30

Updatedenrolmentrecordforlastyearavailable

Mean 0.17 0.23 0.1


N 60 30 30

Writtenrecordsofmeetingskept

Mean 0.06 0.08 0.04


N 49 25 24

Numberofmeetingrecordsavailable

Mean 1.33 2 0

StandardError 0 0 0

N 3 2 1


Mean 0.12 0.1 0.13


N 60 30 30

Updatedteacherattendancerecordexists

Mean 0.08 0.07 0.1


N 60 30 30


©EDOREN 296


Mean 0.38 0.4 0.37


N 60 30 30

TimetableprovidedforP2orequivalent

Mean 0.22 0.23 0.2


N 60 30 30


Table65: TeacherLevelDescriptiveStatistics


Bauchi Niger

TeacherPersonalCharacteristics

Respondentisfemale

Mean 0.09 0.02 0.16


N 96 46 50

Age

Mean 31.62 33.64 29.87


N 95 45 50

SpeaksYoruba?

Mean 0.02 0 0.03


N 76 38 38

SpeaksNupe?

Mean 0.2 0 0.39


N 76 38 38

SpeaksIgbo?

Mean 0.01 0.03 0


N 76 38 38

SpeaksHausa?

Mean 0.96 1 0.93


N 76 38 38

SpeaksFulfude?

Mean 0.07 0.13 0.02


N 76 38 38

SpeaksEnglish?

Mean 0.9 0.9 0.91


N 76 38 38

SpeaksEbira?

Mean 0.02 0 0.03


N 76 38 38


©EDOREN 297

SpeaksArabic?

Mean 0.17 0.21 0.13


N 76 38 38

ProfessionalCharacteristics

TeacheriscurrentlyteachingP1-P3equivalent

Mean 1 1 1

StandardError 0 0 0

N 76 38 38

TeacheriscurrentlyteachingP3-P6orequivalentlevel

Mean 0.12 0.15 0.09


N 76 38 38

Teacheriscurrentlyteachingatupperandlowerlevels

Mean 0.12 0.15 0.09


N 76 38 38


Mean 0.35 0.51 0.2


N 96 46 50

SubjectsTaught-Mathematics

Mean 0.65 0.65 0.65


N 76 38 38

SubjectsTaught-English

Mean 0.71 0.69 0.74


N 76 38 38

SubjectsTaught-Hausa

Mean 0.32 0.36 0.28


N 76 38 38

Doestheteacherhaveprofessionaleducationalqualifications?

Mean 0.45 0.59 0.33


N 96 46 50

DoestheteacherhaveGrade2orequivalentqualification?

Mean 0.08 0.06 0.09


N 96 46 50

Highestacademicqualificationoftheteacherisprimaryeducation

Mean 0.12 0.15 0.09


N 96 46 50

HighestacademicqualificationoftheteacherisJSS

Mean 0.04 0.05 0.04


N 96 46 50


©EDOREN 298

HighestacademicqualificationoftheteacherisSSCE

Mean 0.66 0.61 0.7


N 96 46 50

HighestacademicqualificationoftheteacherisOND

Mean 0.16 0.15 0.16


N 96 46 50

HighestacademicqualificationisBA/BSc

Mean 0.02 0.04 0


N 96 46 50

Doestheteacherhavereligiouseducationqualification?

Mean 0.21 0.33 0.11


N 96 46 50

Totalexperienceasateacherinthisschooloranyotherschool

Mean 6.81 8.79 5.09


N 95 45 50

Totalyearsasateacherincurrentschool

Mean 3.39 4.42 2.33


N 79 41 38

Teacherhasatleast2yearsofteachingexperienceinanyschool

Mean 0.82 0.87 0.77


N 96 46 50

Teacherhasatleast2yearsofteachingexperienceinthecurrentschool

Mean 0.63 0.7 0.56


N 79 41 38

TeacherTrainings

Hastheteacherattendedtrainingduringlast2years?

Mean 0.43 0.36 0.49


N 96 46 50

TotalnumberoftrainingsattendedbytheTeacher

Mean 1.29 1.3 1.29

StandardError 0 0 0

N 40 16 24

Numberofdaystrained

Mean 6.46 7.63 5.73

StandardError 0 0 0

N 40 16 24

HastheteacherattendedtrainingorganisedbySUBEB?

Mean 0.11 0.24 0.04

StandardError 0 0 0

N 40 16 24

Hastheteacherattendedtrainingorganisedby

Mean 0.71 0.77 0.67

StandardError 0 0 0


©EDOREN 299

GEP/UNICEF? N 40 16 24

HastheteacherattendedtrainingorganisedbyLGEA

Mean 0.19 0.18 0.2

StandardError 0 0 0

N 40 16 24

IstheteachertrainedonteachinginHausa?

Mean 0.25 0.53 0.07

StandardError 0 0 0

N 40 16 24

Istheteachertrainedonteachingmethods?

Mean 0.78 0.77 0.78

StandardError 0 0 0

N 40 16 24

Istheteachertrainedoncommunityinvolvement?

Mean 0.16 0.12 0.18

StandardError 0 0 0

N 40 16 24

Istheteachertrainedonintegrationactivities?

Mean 0.38 0.59 0.25

StandardError 0 0 0

N 40 16 24

Istheteachertrainedondevelopinginstructionalmaterials?

Mean 0.12 0 0.2

StandardError 0 0 0

N 40 16 24

Istheteachertrainedonmanagementandplanning?

Mean 0.05 0.12 0

StandardError 0 0 0

N 40 16 24

Istheteachertrainedonschoolleadership?

Mean 0.18 0.47 0

StandardError 0 0 0

N 40 16 24

Istheteachertrainedonextra-curricular?

Mean 0.04 0 0.7

StandardError 0 0 0

N 40 16 24

Istheteachertrainedoncurriculumsubjects?

Mean 0.24 0.18 0.27

StandardError 0 0 0

N 40 16 24

Istheteachertrainedonliteracyandnumeracy?

Mean 0.18 0.18 0.18

StandardError 0 0 0

N 40 16 24

Teacherabsenteeism

Teacherisabsentatleastonceduringlast3months

Mean 0.76 0.9 0.64



©EDOREN 300

N 93 45 48

Numberofdaysteacherisabsentduringthelast3months

Mean 6.77 7.91 5.33


N 72 40 32

TeacherMotivation

Teachermotivationcompositeindex

Mean N/A 3 2.93

StandardError N/A 0.043 0.05

N N/A 46 50

TeacherKnowledge

Teachersubjectknowledgecompositeindex

Mean 0.44 0.54 0.35


N 96 46 50

Teacherpedagogicalknowledgecompositeindex

Mean 0.06 0.09 0.03


N 96 46 50

Mother-Tongueteaching(Hausa)inearlygradesindex

Mean 31.2 41.2 22.32


N 96 46 50

Hausateaching/learningmaterialscompositeindex

Mean 0.03 0.01 0.04


N 96 46 50

Syllabus/Curriculumknowledge

Mean 0.81 0.87 0.76


N 96 46 50


Table66: PupilLevelDescriptiveStatistics


Bauchi Niger

PupilCharacteristics

Femalepupils

Mean 0.48 0.44 0.52


N 575 296 279

Age

Mean 8.33 8.52 8


N 387 260 127

PupilspeaksEnglishathome

Mean 0 0 0



©EDOREN 301

N 573 295 278

PupilspeaksFulfuldeathome

Mean 0.02 0.03 0.01


N 573 295 278

PupilspeaksHausaathome

Mean 0.73 0.93 0.54


N 573 295 278

PupilspeaksKanuriathome

Mean 0.01 0.03 0


N 573 295 278

PupilspeaksNupeathome

Mean 0.22 0 0.43


N 573 295 278

PeoplespeaksOtherathome

Mean 0.01 0.01 0.02


N 573 295 278

CurrentlyattendingP2orequivalentlevel

Mean 0.89 0.96 0.81


N 574 296 278

Attendingotherschoolbesidesthisone

Mean 0.41 0.72 0.13


N 574 296 278

Typeofotherschoolbeingattended

Mean 0.88 0.94 0.55

StandardError 0 0 0

N 256 215 41

Pupilisaboarder

Mean 0.06 0.1 0.03


N 574 296 278

Facesdifficultyingettingtoschool

Mean 0.03 0.03 0.02


N 539 273 266

PupilAssessmentScaledScores

Hausaliteracyscaledscore

Mean 487.61 518.3 458.4

StandardError 8.256 10.625 10.877

N 575 296 279

Englishliteracyscaledscore

Mean 362.46 371.97 353.38


N 574 296 278

PupilAssetOwnership


©EDOREN 302

Hasacar

Mean 0.24 0.29 0.19


N 574 296 278

Hasafridge

Mean 0.25 0.25 0.25


N 574 296 278

Hasaradio

Mean 0.78 0.89 0.68


N 574 296 278

Hasamattress/bed

Mean 0.94 0.99 0.88


N 574 296 278

HasaTV

Mean 0.57 0.53 0.61


N 574 296 278

Hasamotorcycle

Mean 0.74 0.72 0.75


N 574 296 278

Hasachair

Mean 0.87 0.89 0.85


N 574 296 278

Hasamat

Mean 0.98 1 0.96


N 574 296 278

Hasasewingmachine

Mean 0.37 0.42 0.32


N 574 296 278

Hasanairconditioner

Mean 0.03 0.02 0.03


N 574 296 278

Hasafan

Mean 0.5 0.4 0.6


N 574 296 278

Hasagenerator

Mean 0.38 0.32 0.44


N 572 296 276

Hasastove

Mean 0.39 0.41 0.38


N 573 296 278


©EDOREN 303

Hasacamera

Mean 0.06 0.08 0.05


N 570 296 274

Hasacomputer

Mean 0.06 0.08 0.04


N 572 296 276

Hasamobilephone

Mean 0.93 0.96 0.91


N 574 296 278

Hasabicycle

Mean 0.62 0.63 0.6


N 574 296 278

Hasagoat

Mean 0.77 0.81 0.73


N 574 296 278

Hasacattle

Mean 0.47 0.51 0.43


N 570 296 274

Hasahorse,donkeyormule

Mean 0.05 0.06 0.05


N 574 296 278

Hasasheep

Mean 0.5 0.63 0.37


N 571 296 275

Hasachicken

Mean 0.87 0.88 0.87


N 574 296 278

Hasapig

Mean 0.01 0 0.02


N 573 296 277



©EDOREN 304

AnnexN CBMCIQSSDescriptiveStatistics


Bauchi Niger

CBMCMembership

TotalMembership

Mean 14.24 15.83 12.88


N 52 24 28

Totalfemales

Mean 3.65 4.79 2.66


N 52 24 28

Femalesasashareoftotalmembers

Mean 0.24 0.29 0.19


N 52 24 28

Totalchildren

Mean 0.32 0.62 0.07


N 52 24 28

Childrenasashareoftotalmembers

Mean 0.02 0.04 0


N 52 24 28

YearssinceCBMCestablishment

Mean 2.99 2.25 3.63


N 52 24 28

Yearssinceintegration

Mean 2.44 2 2.82


N 52 24 28

CBMCformedbeforeschoolintegration

Mean 0.37 0.25 0.47


N 52 24 28

HeadTeacherinCBMC

Mean 0.65 0.67 0.64


N 52 24 28

Chairperson-Headteacher

Mean 0.23 0.29 0.17


N 52 24 28

Chairperson-ProprietorMean 0.32 0.37 0.28



©EDOREN 305

N 52 24 28

ProprietorinCBMC

Mean 0.6 0.58 0.61


N 52 24 28

NumberofteachersinCBMC

Mean 2.9 3.32 2.55


N 52 24 28

NumberofparentsinCBMC

Mean 9.6 9.76 9.47


N 52 24 28

NumberofoldorcurrentpupilsinCBMC

Mean 3.03 3.08 2.99


N 52 24 28

NumberofreligiousleadersinCBMC

Mean 0.83 1.04 0.64


N 52 24 28

CBMCMeetings

CBMCMeetingbetween2014-2015

Mean 0.9 0.88 0.93


N 52 24 28

CBMCMeetinginthecurrentyear

Mean 0.65 0.79 0.53


N 52 24 28

Numberofmeetingsheldbetween2014-2015

Mean 4.4 6.19 2.95


N 47 21 26

Attendancerateatlastmeeting

Mean 0.41 0.36 0.46

StandardError 0 0 0

N 30 16 14

Attendancerateoffemalesatlastmeeting

Mean 0.35 0.26 0.48

StandardError 0 0 0

N 26 15 11

CBMCtrainings

CBMCmembersattendedtraining

Mean 0.63 0.54 0.71


N 52 24 28

TrainingbyLGEA

Mean 0.16 0.23 0.11

StandardError 0 0 0

N 32 13 19


©EDOREN 306

TrainingbyGEP/UNICEF

Mean 0.66 0.62 0.68

StandardError 0 0 0

N 32 13 19

TrainingbySIP/TSP

Mean 0.03 0.08 0

StandardError 0 0 0

N 32 13 19

TrainingbySUBEB

Mean 0.06 0.08 0.05

StandardError 0 0 0

N 32 13 19

TrainingbySAME

Mean 0.03 0 0.05

StandardError 0 0 0

N 32 13 19

Totaltrainings

Mean 1.34 1.23 1.42

StandardError 0 0 0

N 32 13 19

TotalnumberofdaysofCBMCtrainings

Mean 4.03 4 4.06

StandardError 0 0 0

N 31 12 19

CBMCMonitoringandPlanningActivities

CompleteWCDPexists

Mean 0.19 0.21 0.18


N 52 24 28

WrittenformofWCDPisavailable

Mean 0.67 0.78 0.59

StandardError 0 0 0

N 21 9 12

Numberofvisittomonitorprogress

Mean 7.25 8.2 6.43


N 52 24 28

CBMCmonitoringofpupils’attendance

Mean 0.85 0.88 0.82


N 52 24 28

CBMCstepstoincreasepupils’attendance

Mean 0.85 0.83 0.86


N 52 24 28

CBMCmonitoringofteacher’sattendance

Mean 0.82 0.84 0.81


N 44 18 26

CBMCtookactiontoimproveTattendance

Mean 0.94 1 0.89

StandardError 0 0 0


©EDOREN 307

N 33 15 18

CBMCqualityofteachingmonitoring

Mean 0.85 0.75 0.93


N 52 24 28

CBMCResources

CBMCefforttomobilizecash

Mean 0.65 0.63 0.68


N 52 24 28

CBMCeffortatcashmobilization

Mean 0.82 1 0.68

StandardError 0 0 0

N 34 15 19

Communitymembers(includingCBMCmembers,localbusinessesandindividuals)

Mean 1 1 1

StandardError 0 0 0

N 28 15 13

GovernmentBodies

Mean 0.07 0 0.15

StandardError 0 0 0

N 28 15 13

NGO,donorsorotherexternalagencies

Mean 0.07 0.07 0.08

StandardError 0 0 0

N 28 15 13

AmounttheCBMCmobilizefromcommunity

Mean 39171.9 31210.9 53255.68

StandardError 0 0 0

N 22 14 8

Totalfundsraisedlastyear

Mean 39474.2 31210.2 54092.8

StandardError 0 0 0

N 22 14 8

Othermonetarysupportwithoutfundraising

Mean 0.21 0.25 0.18


N 52 24 28

CBMCreceivedfundingfromUNICEF

Mean 0.73 0.84 0.6

StandardError 0 0 0

N 11 6 5

Receivedschoolgrant

Mean 0.35 0.42 0.28


N 52 24 28

Amountofschoolgrant

Mean 124892.3 105911.4 148190

StandardError 0 0 0

N 20 11 9


©EDOREN 308

TotalfundingavailabletoCBMC

Mean 112258.5 94258.66 135938.1

StandardError 0 0 0

N 30 17 13

Shareoftotalfundsinvested

Mean 85.68 90.93 78.19

StandardError 0 0 0

N 29 17 12

Utilizationofanynon-cashresource

Mean 0.42 0.46 0.39


N 52 24 28

CBMCFinancialManagement

Useofcashbook

Mean 0.27 0.38 0.18


N 52 24 28

Cashflowisrecordedincashbook

Mean 0.63 0.57 0.75

StandardError 0 0 0

N 11 7 4

Useabankaccounttostorethefundsreceived

Mean 0.69 0.75 0.65


N 52 24 28

Accountinschool’sname

Mean 0.69 0.89 0.5


N 36 18 18

AccountisinCBMC’sname

Mean 0.28 0.06 0.5


N 36 18 18

Evidenceofbanktransactionavailable

Mean 0.53 0.63 0.43


N 15 8 7

CBMCCommunitySupport

Communitymembers(includingCBMCmembers,localbusinesses,andindividuals)

Mean 1 1 1

StandardError 0 0 0

N28

15 13

AmounttheCBMCmobilizefromcommunity

Mean 39171.9 31210.19 53255.68

StandardError 13846.3 10168.79 32507.92

N 22 14 8

Proportionoftotalfundsraisedfromcommunity

Mean 98.41 100 95.81



©EDOREN 309

N 21 13 8

CBMCmobilisedfundsfromcommunity

Mean 0.27 0.16 0.4

StandardError 0 0 0

N 11 6 5

CBMCRecordKeeping

Cashflowisrecordedincashbook

Mean 0.63 0.57 0.75

StandardError 0 0 0

N 11 7 4

Evidenceofbanktransactionsavailable

Mean 0.53 0.63 0.43


N 15 8 7

EvidenceofCBMCmeetingsinthelastschoolyear

Mean 0.66 0.53 0.76


N 32 15 17

WrittenformofWCDPisavailable

Mean 0.67 0.78 0.59

StandardError 0 0 0

N 21 9 12


Cover Photo Credit: © Global Plus News

EDOREN is a consortium of leading organisations in international development and education: Oxford Policy Management (OPM), and the Institute of Development Studies (IDS) at the University of Sussex, and is supported by UK Aid. EDOREN cannot be held responsible for errors or any consequences arising from the use of information contained in this report. Any views and opinions expressed do not necessarily reflect those of OPM, IDS and EDOREN or any other contributing organisation.

evaluation of unicef girls’ education project phase 3 (gep3)€¦ · for advice on the...

Documents