effects of the elm software on the math learning of kenyan ... · effects of the elm software on...

EffectsoftheELMSoftwareontheMathLearningofKenyanGrade/Standard1ElementaryStudents:AReportonthe2016PilotStudy1

Summary:ThispilotstudyexploredthefeasibilityofimplementingELMinKenyanmathclassrooms.Theresultswereobtainedusingastandardizedtestofmathematicsachievementanddemonstratedthatthismeasurepossessesasufficientdegreeofsensitivitytocapturethechangeinmathematicskillsforkindergartenandgrade-onestudentsbetweenpre-andpost-test.ItisimportanttonotethatasaresultofinstructionwhereELMwasintegrated,a)theinitialpre-testgapbetweenlowandhighperformingstudentsstartedclosingforthesubtestmeasuringlanguage,vocabularyandrepresentationsofmathematics;b)studentsofbothgendersgainednearlyequally.Yet,intheabsenceofacontrolgroupwedonotknowhowlarge,ifany,theeffect(s)ofELMwereonstudentmathematicability.WealsolearnedabouttheimplementationofELMinKenyanclassrooms.Instructionwouldbeadvantagedasteachersdevelopgreatercomfortwithusingthetechnologyinlargeclasses.Notsurprisingly,technologyinthecomputerlabsneedstobereliabletomaximizelearningopportunitiesforthestudentsinlargeclassesandimplementationofcooperativeteachingapproachescouldaidmathinstruction.

1ThisresearchwasmadepossiblebyagranttotheauthorsfromtheSESEAproject,AgaKhanFoundationCanadaandGlobalAffairsCanada

2

SampleThreeteachersandtheir168studentsfromtwoprimaryschoolsinMombasaarea,Kenyaparticipatedinthispilotdesignedasaone-grouppretest-postteststudy.Thereweretwograde-oneclasses(N=96)andonekindergartenclass(72).InformedconsentwasobtainedfromtheparticipantsfollowingCanada’sTri-CouncilPolicyontheethicaltreatmentofresearchparticipants.Ofthe168studentswhoparticipatedintheELMfieldtest,162participatedinthepre-andpost-testing.Areductioninsamplesizeoccurredafteranumberofstudentsmissedonetimeoftesting:145and141studentscompletedeitherpre-testorpost-testonly.Thefinalsampleusedforanalysiscontainedthescoresof127students:56kindergartenand71gradeonestudents.Gender-wisethegroupwassplitasfollows:69maleand58femalestudents.InterventionInthispilottheteachersfollowedtheKenyancurriculumrequirementsforteachingmathematicsandwereatlibertytodecideonthemethodofclassroominstructionaswellastheinstructionaltoolsandtechniquestheywoulduse.Inthewinterof2016thethreeteachersparticipatinginthepilotattendedaone-dayinitialtrainingworkshoponhowtouseELMtoteachmathematics.ThroughouttheyeartheyaspartoflargerteaminterestedinteachingwithELMalsoattendedanumberofplanningsessionstohelpthemincorporateELMinMathcurriculum.OurELMcoordinator,amastertrainerinNumeracy,facilitatedthesesessions.Inadditiontothesupport,thesesessionswereintendedtoprovidetheteacherswithaforumtoplantheirteachingwithELM.Teacherswereprovidedwithteachingmaterialsincludinglessonplans,classroomactivities,andjobaidsforteachers.Theuseofthesematerialswassuggestedratherthanprescribedandtheirusewasleftateachteacher’sdiscretion.MultimediascaffoldingandsupportforteachersandstudentsembeddedinELMwerealsoavailable.Additionally,thenumeracytraineroccasionallyvisitedtheteachersduringtheELMsessionsinthelabandbyprovidingfeedbackontheirMathlessons.IntotaltheELMimplementationunfoldedbetweenMarchandSeptemberof2016.InstrumentationStudents’skillsinmathematicswereassessedusingGroupMathematicsAssessmentandDiagnosticEvaluationGMADE(Williams,2004),astandardizedachievementmeasure.Sincekindergartenstudentsmadeabout56%ofthepilotsample,wechosetomeasurethechangeinthestudents’mathematicskillsusinglevelR,thelowestlevelofGMADE.Thislevelcoverstheagebandfrom5to8yearsoldbyofferingitemsatawiderangeofdifficultythatallowstoreliablymeasurelow-,average-andhigh-performingstudents.Parallelforms(AorB)wereusedalternativelytocollectpre-andpost-data.Eachformcontainedfifty-sixmultiplechoiceitemsmeasuringconceptsandoperationsaddressedbyELMsuchascounting,comparing,adding,subtracting,decomposingandplacevalue.LevelRofGMADEcontainsthetwosubtests,ConceptandCommunicationandProcessesandApplications.

3

TheConceptandCommunicationsubtestaddressesthelanguage,vocabularyandrepresentationsofmathematics.The26questionscontainsymbols,wordsandphrasestoassessthefivecontentstandardsincludingnumberandoperations,algebra,geometry,measurement,dataanalysisandprobabilityasreflectedbythefollowingcategories:comparison,geometry,measurement,numeration,quantity,sequence,andtime.TheProcessandApplicationssubtestmeasuresthestudents'abilitytotakelanguageandconceptsofmathematicsandapplytheappropriateoperation(s)andcomputationtosolveawordproblem.Atthislevelthestudentssolveonlyone-steporsingle-operationproblems.The26itemsassignedtocomparison,numeration,quantityandsequenceassessthenumberandoperationsstandard.Theitemsinthegeometrycategoryassessthegeometrystandardwhereastimeitemaddressthemeasurementstandard.AnalysesAllstudentdatawereenteredintoSPSS24forMacOSXandverifiedforaccuracy.Studentsforwhomthetestdataweremissingwereexcludedfromanalyses.Standardscreeningproceduressuggesteddatanormality.Pairedsamplet-testwasruntoexamineifmathscoreschangedafterELMwasusedforXXXweeks.Inaddition,toexploreifthegainsdifferedforgrade1andkindergartenstudentsaswellasformaleandfemalestudents,weperformedindependenttwo-samplet-testsofgroupdifference.ResultsThefollowingsectionpresentstheresultsthatweobtainedafteranalyzingthestudentandteacherdata.StudentdataFirst,weattempttoanswerthequestionifafterbeingtaughtmathematicswiththeuseofELM,studentsdemonstratedgainsinmathematicalskills.Astable1showsELMstudentsgainedintheirmathematicalskillsasmeasuredbytheGMADEstandardizedtest.WeobservedsignificantimprovementsontheonthesubtestsofConceptsandCommunications,ProcessandApplicationsaswellasthetotaltest.Table1.Descriptivestatistics(meansandstandarddeviations),t-testvalues(significancelevelswithGMADEscoresforELMstudents(N=127)

GMADEscalesPost-testMeanscore(SD)

Pre-testMeanscore(SD)

t-test,p<.000

ConceptsandCommunication(28) 25.46(3.01) 22.57(4.03) 6.70***

ProcessandApplications(28) 20.98(6.28) 13.80(5.36) 12.85***TotalGMADE(56) 46.65(8.1) 36.46(7.86) 12.65***Inadditiontothemainanalysis,weexaminedifgainsinmathematicsdifferedforboysandgirls.Table2showstheresultsoftheanalysesoftheGMADEgainscoredifferencesforboysandgirlsinELMclasses.

4

Table2.ReadinggainsbygenderinELMclasses

Tovisuallyrepresenthowstudentsofbothgenders’differedontheGMADEscores,webuiltGraphs1,2and3.Graph1.ConceptsandCommunication Graph2.ProcessandApplications

Graph3.TotalGMADE

ThedatasummarizedinthetableandgraphsshowthatgirlsintheELMclassesdemonstratehighergainsthanboysonallsubtests.Howeverthedifferencesbetweengainscoresfortwogendersarenotstatisticallysignificant.

20

21

22

23

24

25

26

Pretest Posttest

Boys

Girls

10

12

14

16

18

20

22

Pretest Posttest

Boys

Girls

3537394143454749

Pretest Posttest

Boys

Girls

GMADEscales ELMboys(N=69)

ELMgirls(N=58)

Fvalueandsignificance

ConceptsandCommunication(28) 2.55 3.31 0.76StandardDeviation 4.78 4.99ProcessandApplications(28) 6.65 7.79 1.04StandardDeviation 5.65 6.97TotalGMADE(56) 9.24 11.31 1.64StandardDeviation 8.43 9.74

5

Finally,wecomparedgainsinGMADEscoresofthosewhoscoredhigh(N=48)andlow(N=47)atthepre-test.Theanalysisrevealedthatstudentsinthebothgroupsbenefitedfromtheinstruction.However,theimprovementsoflow-scoredstudentsweresignificantlyhigherthanthoseoftheirhigh-scoredpeersonallGMADEsubscalesincludingConceptsandCommunications(t=6.2,p<.000),ProcessesandApplications(t=4.3,p<.000)andTestTotal(t=6.2,p<.000).Graphs4,5and6showhowtheaveragescoreschangedbetweenpre-andpost-testimplyingthatthegapbetweenthegroupsreducedconsiderably.ThisisespeciallynoticeablefortheConceptsandCommunicationsubscale(Graph.4)wherethepre-testdisparitybecamenon-existent.Graph4.ConceptsandCommunication Graph5.ProcessandApplications

Graph6.TotalGMADE

TeacherSelf-reportsThefollowinginformationabouttheclassroomimplementationofELMsurfacedfromtheteachers’self-reports.EachclassusedtheirschoollabwithaccesstoELM.Onaveragetheratiowas5studentspercomputer.Thereforetoincreasetheexposuretimetothetechnology,teachersplacedstudentsingroupsduetothelargeclasssizes.Inadditiontolimitedaccesstotechnology,theteachersreportedhavingexperiencedafewotherimportantchallenges.Amongthemostrecurrentwerepowerblackouts,networkfailures,

10121416182022242628

Pretest Posttest

Low

High

5

10

15

20

25

30

Pretest Posttest

Low

High

20

25

30

35

40

45

50

55

Pretest Posttest

Low

High

6

notworkingcomputersandearphones.Atthemomentofself-reports(term-two,summerof2016)theclassroomusesofELMinbothgradeoneandkindergartenclassestargetedcountingactivities.Theteachersreportedhavingimplementedtheextensionactivitiesprovidedtothem.ThesewereusefulastheyallowedengagingtheirlargeclassesintheELM-coveredcontextwhileonlyabout20%oftheirstudentscouldhavedirectexposuretoELMactivitiesatonetime.TeachersreportedhavingusedplanningandreportingfeaturesofELM;theyaccessedELMreportstolearnabouttheprogresstheirstudentsmadeinELMactivitiesaswellaspreparedindividualsequenceofactivitiesforthestudents.Accordingtotheavailableself-reports,studentsalsodevelopedskillofusingtechnologyandLTK+.Accordingtotheteachers,theirstudents“wereabletologinandoutwithease”aswellasdevelopedtechnologyskillssuchasusingthecomputermouse.Accordingtotheteachers,ahavingaclassroomprojectorwouldstrengthenteachingasitwouldallowdemonstratingELMactivitiestothewholeclasswhichisextremelyimportantteachingstrategyespeciallyinlargeclasses.Inadditiontoachievingtheobjectivessetoncounting,teachersreportedthattheirstudentsweremotivatedtousethetool:theyenjoyeddoingpuzzlesandmakingnewfriends;theyalsolikedtoworkinteams;andcompletingtheactivitiesattheirownpace.ConclusionsThepilotstudyallowedustoachieveafewgoalsinregardstothefeasibilityofusingELMinKenyanmathclassrooms.Evengivenaweakresearchdesignwewereabletocapturelearninggains.Theresultsobtainedusingastandardizedtestofmathematicachievement(GMADElevelR)demonstratedthatthismeasurepossessesasufficientdegreeofsensitivitytocapturethechangeinmathematicskillsforkindergartenandgrade-onestudentsbetweenpre-andpost-test.Substantiallyitisimportanttonotethata)thegirlsintheELMclassesdemonstratedhighergainsthanboysalbeitnon-significantimplyingthatbothgendersbenefitedfromELMaboutequally;b)theinitialpre-testgapbetweenlowandhighperformingstudentsstartedclosingandbecamenon-existentforthesubtestmeasuringlanguage,vocabularyandrepresentationsofmathematics.Yet,intheabsenceofacontrolgroupwedonotknowhowlarge,ifany,theeffect(s)ofELMwereonstudentmathematicability.Secondly,welearnedaboutELMimplementation.ForELMtoworkinthehandsofteacher,theperformanceofcomputerlabsneedtobemorereliabletomaximizealearningopportunityforthestudentsinlargeclasses.Instructionwouldbenefitasteachersbuildmorecomfortusinggroupsinteachingwithtechnology.Futurestudiesshouldensurethesystematicdatacollectiononclassroomimplementation.ClassroomobservationsshouldbecompletedinELMclassesatleasttwiceperschoolyear.RegularteacherreportsabouttheiruseofELMwouldbeanimportantcomplementtotheELMtracedatathatiscollected.