i two field experiments on social support and attendance...

1

©BehaviouralInsightsLtd

Igetbywithalittlehelpfrommyfriends:Twofieldexperimentsonsocialsupportandattendanceinfurthereducationcolleges

intheUK1

BibiGrootInstituteofEducation,UniversityCollegeLondon

MichaelSanders

BlavatnikSchoolofGovernment,UniversityofOxfordHarvardKennedySchool,HarvardUniversity

ToddRogers

HarvardKennedySchool,HarvardUniversity

AbstractInrecentyears,therehasbeenanincreasingrecognitionthatbroadersocioeconomicand environmental conditions such as school climate and the availability of peer,parental,andtutorsupportshapemotivationandeducationalsuccessasmuchas,ormorethan,individual-levelfactorssuchaspersonality.Whetherastudentgrowsupfeelingnurturedandsupportedbytheirfamilyandpeers isstronglycorrelatedwiththeiremotionalandphysicalwell-being(Scales&Taccogna,2001;Wentzel&Caldwell,1997).However,fewstudiesprovidecausalevidencethatsocialnetworkinterventionscan influence students’ success. We investigate, through two randomised fieldexperiments, whether providing students’ social networks with personalisedinformationaboutupcomingexamsandcoursecontentleadstoimprovementsinclassattendance. Students nominated two ‘Study Supporters’ and were subsequentlyindividuallyrandomisedintotwoarms:inonearmtheStudySupportersreceiveweeklytextmessages,intheotherarmtheydonotreceiveany.Weconsistentlyfindpositiveeffectsofthisintervention,particularlyforstudentswhoarestudyingtowardsGCSEexams.We also find that the intervention appears to be particularly effective forstudentsatthelowerendofthedistributionofattendance.Wediscussthisresultinthecontextofthebroadersocialsupportliterature.

1WearegratefultothedepartmentforBusiness,InnovationandSkillsforfundingthisresearchthroughtheAdultSkillsandKnowledgeResearchCentre,andtoEskilForsellandDavidNolanforexcellentresearchassistance.Thanksarealsoduetocountlesstutorsandstudentsatcolleaguesatcollegeswhohavemadethisresearchpossible,andparticularlytoLisaArmitage,PatCarvalho,SusannahClose,DarrellDeSouza,ElizabethFrost,BeverleyHancockSmith,NaomiJulings,RekhaKaul,SarahKnowles,SusanLindley,BarbaraMasters,TonyParry,MichelleProudman,RebeccaPurple,MarkStacey,MumtazVanDerVord,KellyVennerandBarbaraWorms.TofriendsattheBehaviouralInsightsTeam(criticalandotherwise),wewillbeforevergrateful.GrootwouldliketothankDavidMallowsandJakeAndersfromUCLInstituteofEducationfortheirsupervisionandcomments.GrootconductedpartsofthisprojectinpursuitofherPhDattheInstituteofEducation,UCL.WearegratefultoseminarparticipantsattheESAmeetinginJerusalemin2016forcomments.

2


I. Introduction

Duringperiodsofeducation,manyofusmighttakeforgrantedthatsomeoneinourlivescaredaboutourlearningandwouldregularlytalktousaboutit.Ifwedidnothavethatsomeone,be itaparent,grandparent, family friend,or someoneelse,oureducationmighthavepannedoutdifferently.Atthesametime,wehaveallexperiencedwhatitfeelsliketostrugglewithsomethingandnotknowwhototurnto.Unfortunately,however,theproblemmaynotbethatthepeoplearoundyoudonotwanttobeinvolved.Rather,theymaynotknowhowtohelp,andwemightnotknowhowtoask.Drawing on the literature on parental engagement and behavioural interventions,wedevelopasocialsupportinterventionthatseekstoimprovethefrequencyandqualityofcommunicationbetweena studentand theirclose friendsand family.By focusingonsocio-emotional factors in thestudent’senvironment,wemaybeable toharness thepowerofsocial interactions to improvestudentsuccess.JohnsonandJohnson (1985)articulatethisconceptwell:

Motivation to learn is inherently interpersonal, created by internalized pastrelationshipsandbycurrentinterpersonalinfluences.Otherpeopleprovidethepurpose,meaning,andsupportforlearningbyprovidingapprovalandrespectfor one’s academic accomplishments. Humans, not machines, materials,personalitycharacteristics,orexternalappeals,motivatehumans.(p.250)

Increasingtheextenttowhichpeoplefeelsupportedbytheirfriendsandfamiliesmightincrease theirmotivation, and,with it, their likelihood of persistence. Amodest butgrowingnumberofstudiesattempttoboostsupportivecommunicationusingnudge-typeapproachestoimproveoutcomes(Kraft&Rogers,2015;Rogers&Feller,2016).Thispaperaims to contribute to this literature. We conduct two field experiments in furthereducation colleges in the United Kingdom, testing an intervention inwhich studentsnominate‘studysupporters’tosupporttheirlearning,whoarethensentaseriesoftextmessagesprovidingthemwithinformationaboutthestudent’scourse,andencouragingthemtosupportthestudent.Previouscommunicationinterventionswithadolescentshavetypicallyfocusseddistinctlyonprevention.Forexample,theymightfocusonimprovingparent-childcommunicationsurroundingriskyhealthbehaviourssuchassmokingorteenagepregnancy(Blakeetal.,2001).Thispaper,on theotherhand,presentscausalevidence that a social supportinterventioncanhelppromotepositivebehaviourssuchasschoolattendance.

3


The structure of this paper is as follows. In section 2 we review the theoreticalbackgroundofthesocialsupportinterventionandbrieflydiscussthepolicycontextofourinterventions.Section3focusesonourprimaryoutcomemeasure:classattendance.Thisisfollowedbyadescriptionofexperiment1anditskeyfindingsinsection4,thenbyadescriptionofthemethodsandresultsofexperiment2insection5,andanalysisofthepooleddatafrombothexperimentsinsection6.Finally,weofferconclusions.

II. Areviewoftheliterature

Socialsupport:aconceptualframeworkSocial support can be defined as an “individual’s perception of general support orspecificsupportivebehaviours(availableoractedon)frompeopleintheirsocialnetwork”(Malecki&Demaray,2003,p.232).Thisglobaldefinitionofsocialsupportoverlapswithconceptsofsocialnetworks,supportsystemsandsocialintegration,inthesensethatallfour are concernedwith the social resources available in our personal communities(Gottlieb&Bergen,2010).Throughoutthispaper,weconceiveofsocialsupportasanindividual’s perception, rather than an observable social structure. It is difficult toobservesocialsupport inaction,as it isoftennotthematerialisationorexpressionofsupportthatcounts,butthebeliefthatsupport isavailable (Gottlieb&Bergen,2010).Additionally,thequalityofsupportprovided,fromtherecipient’sperspective,isdifficulttomeasurebehaviourally.Theconceptofsocialsupportisthereforebestmeasuredusingself-reportinstruments,inordertocaptureperceivedsupport.Thereare several sourcesand typesof social support.The sourceof supportcanbeanyone close to the individual, including parents, teachers, classmates or friends(Demaray & Malecki, 2002). Specific types of social support include emotional,instrumental, informational and appraisal support (House, 1981).Emotional support ischaracterised by emotionally close, encouraging, accepting and caring relationships.Instrumental support,on theotherhand, involvesactivehelpingand theprovisionofmaterialresources,suchashelpwithtransportation.Informationalsupportinvolvesthecommunication of expectations, knowledge, guidance and feedback. Lastly, appraisalsupportistypifiedbyreceivingpositiveappreciationforone’sefforts(Gottlieb&Bergen,2010;Malecki&Demaray,2003).

TheprotectiveroleofsocialsupportandtheharmfuleffectsofitsabsenceFeeling supported andencouragedby thepeople inour immediate socialnetwork isimportant.BaumeisterandLeary(1995)firsttheorisedthatthis‘senseofbelonging’isabasichumanneed.Theauthorsfoundevidencethattheabsenceofsocialsupportand

4


belonging linkstopoorphysicalandemotionalwell-beingoutcomes.Adolescentswhofeelalienatedfromtheirschoolcommunities,andsocialcommunitiesmorewidely,arenotonlymore likelytodrop-out,butalsofeelmoreschool-relatedanxiety, lonelinessanddepression(Chipuer,2001;Demaray&Malecki,2002;Khatib,Bui&Stansfeld,2013).Bycontrast,thepresenceoravailabilityofsocialsupporthasalsobeenlinkedtobetterphysical and emotional health outcomes (Viner et al., 2012).University studentswhoreportedthattheyhadsocialsupportresourcesavailabletothemreportedlowerlevelsof poor health symptoms than those who could not identify such resources (Hale,Hannum,&Espelage,2005).Otherresearchershavefoundthatthepresenceofsocialsupportreduceschronic,labour,andcardiacpain(Macdonald&Leary,2005).Inameta-analysisof81studiesontheeffectsofsocialsupportonhealthoutcomes,Uchinoetal.(1996)concludethatthesupportisreliablyrelatedtobetterimmuneresponsestoacutestress,lowerratesofmorbidityandmortality,lowercoronaryheartdisease,andlowerbloodpressure,aftercontrollingforpersonalityfactors.Therearealsoclearindicationsthatboththesubjectivesenseofsocialbelongingandtheavailabilityofsocialsupportpromoteengagementandlong-termlearning.Childrenwhoexperience secure attachmentswith people in their social networks showed greateremotionalandbehaviouralengagementinschool(Furrer&Skinner,2003;Ruzeketal.,2016),reportedhighergenerallifesatisfaction(Danielsenetal.,2009)andshowgreaterprosocialgoalpursuit(Wentzel,1998).Instudieswithadolescentsatriskofschoolfailure,thosewhodidnotfeelsociallysupportedreportedlowerattendance,fewerpro-socialbehaviours, having fewer close friends, and lower self-esteem than adolescents notidentifiedasat-risk(Rosenfeld,Richman,&Bowen,1998;Rosenfeld&Richman,1999).Socialsupportmayhavepositiveeffectsonstudentoutcomesinvariousways.Supportivecommunicationmayhelpthestudentfeelmore incontrol. Interactingwith individualswho provide supportmay also help them develop strong interpersonal skills. Lastly,recognisingthathelpisavailable,andknowingwheretofindit,mayhelpstudentstodealwithstressfulsituationsmoreeffectively(Rosenfeld&Richman,1999).

ParentalinvolvementandeducationaloutcomesAlthough adolescence is often seen as a transition period where parental supportgradually becomes less important while peer support becomes crucial to wellbeing,longitudinal studies rather suggest that parents’ supportive involvement is stronglypredictiveof their child’s adjustment (Stice,Ragan&Randall,2004;Rogers,Wiener,Marton&Tannock,2009).Otherstudiessuggestthatqualityoftheparent-adolescentrelationship fosters the development of identity (Meeus,Oosterwegel, & Vollebergh,

5


2002). These studies suggest that more research is needed on how schools, andspecificallyforthisstudyfurthereducation(FE)colleges,canfosterbothabetterparent-school relationship and supportive parent-adolescent communication. In the studiesreportedinthispaper,studentscannominateanyonefromtheirsocialnetworkstobeStudySupporters.Theynominatedawidevarietyofpeople,includinggrandparents,auntsanduncles,parents,brothersandsisters,sportcoaches,familyfriends,andclassmates.However,weaddress the literatureonparental involvementasonaverage 1outof3studentsinoursamplechoseatleastoneparentorguardianastheirstudysupporter.The literature shows a clear association between parental involvement and theeducationaloutcomesoftheirchildren.Inalongitudinalstudyofimpoverishedyouthinanurbanarea, theprobabilityofhighschoolcompletion increasedsignificantlywhenparentsparticipatedintheirchild’sschoolactivities(Smokowskietal.,2004).Similarly,a synthesis of ninemeta-analyses shows that parental involvement is positively andconsistently correlatedwith the academic achievement of their children, across agegroupsandethnicity(Wilder,2014).Additionally,perceivedparentalsupportisassociatedwith lower incidence of internalising problems in adolescence, such as anxious anddepressive symptoms (Stice, Ragan, & Randall, 2004). Although these studies focusmostlyonstudentsuntiltheageof16,thestudentsinoursampleoftenstilllivedathomewiththeirparentsandwereagedbetween16and18yearsold.Itisthereforereasonableto believe that parents’ involvement would still have a positive influence on theirchildren’sacademicachievement.Althoughmostparentswanttobe‘good’andinvolvedparents,manyreportbarrierstosupportingtheirchildren’sschoolingastheygetolder(Harris&Goodall,2008).Parents’level of engagement varies strongly by their socio-economic status, educationalattainmentandemotionalcapital.Whilemanyparentsreportedthattheywouldliketoengagemoreintheirchildren’sschooling,theysometimesstruggletoovercomematerialand psychological barriers. For example, parents often feel restricted by workcommitmentsand theirown senseofpersonalefficacy (Harris&Goodall,2008).Theconditionsthatmustbeinplaceforparentstogetinvolvedaremanifold.Theymustknowwhatitistheyaresupposedtodo,whentheyshoulddoit,andtheymustbelievetheirinvolvementwillmakeapositivedifference.Parentsalsoneedtogetthefeelingthattheeducationalinstitutioninvitesthemtogetinvolved(Hoover-Dempseyetal.,2005).Extrapolatingfromthisfocusonparentalbeliefsandbehaviours,asimilardynamicforclosefriendsandextendedfamilymembersmaybeplausiblehere.Whenstudents’closecontactsarenotconfidentintheirabilitytohelp,orareunclearhowmuchtheyshould

6


be involved, their persistence and motivation to help the student succeed may beimpaired. Their involvementmay be strengthened if the college invites them to getinvolved, and if they receive clear instructions for involvement. The current textmessaging trial was designed to provide study supporters with timely, relevant andactionableprompts from thecollege,with theaim to increasestudysupporters’self-efficacy and involvement with the student’s education. These studies support thehypothesis that physical and emotional wellbeing in adolescence is at least in partdeterminedbysocialprocesses.Growingupinasupportivefamily,andbeingsurroundedbypositiveandsupportivepeersiscrucialtoimproveyoungpeople’schancesinlife.Nevertheless, thesestudiesoftenmerelydescribe thepresenceofsupportasa fixedfactor.Itiseitherpresent,oritisnot.However,recentinterventionstudiessuggestthatsupportivecommunicationcanbepromoted, in turn improvingeducationaloutcomes(Castleman&Page,2016;Rogers&Feller,2016;Robinson,Lee,Dearing&Rogers,2017).Leveragingparents,peersandteachershasthepotentialtoincreasenotonlyeducationalsuccess,butalsostudents’emotionalandphysicalhealthoutcomes.Below,weturntothegrowingbodyofresearchthatshowshowsocialsupportcanbestimulatedorelicited.

Empoweringkeyindividualstohelpimproveyoungpeople’seducationaloutcomesAn independently evaluated randomised controlled trial (RCT) used textmessages toinform students’ parents ofmissing homework and absences inmaths, science andEnglishclass(Milleretal.,2016).93%ofpeopleintheUKpersonallyownamobilephone(Ofcom,2015),makingtext-messagingtechnologyaninexpensiveandscalablemethodtomotivate,inform,andremindindividualsatkeymoments.TheParentEngagementProject(PEP)wasalarge-scalemulti-siteclusterRCT,with15,697studentsacross36secondaryschools.Studentswhoseparents received textmessagesoutperformed theirpeers insciencecomparabletoonemonthofadditionalprogress,butthisimprovementintestscores was not observed for English or maths. It is important to understand whatmechanismsledtotheseobservedoutcomes.Throughphoneinterviewswithover1900parents of intervention pupils, the research team surveyed parents about theirengagementintheirchild’slearning.Theoddsofparentswhoreceivedthetextmessages(treatment)talkingtotheirchildaboutstudyingforanupcomingtestwerealmostthreetimesgreaterthantheoddsofparentswhodidnotreceivetextmessagestalkingtotheirchildaboutrevising.Interestingly,nodifferenceswerefoundfortheothermeasuresofparental involvement, such as whether the parent has talked with their child aboutattendance,grades,orwhattheylearntinschool.Nevertheless,receivingshort,weeklyprompts fromschoolmayhavehelpedparentstakeamoreactiverole intheirchild’seducation.

7


AsimilarstudybyKraftandRogers(2015)foundthatweeklytextmessagesfromtutorstoparentshelpedpreventdrop-out,withadecreaseinstudentswhofailedtoearncoursecreditfrom15.8%to9.3%.Furthermore,theprobabilityofastudentbeingabsentwasreduced from 12% in the control condition to 9.5% in the teacher-to-parentcommunication condition. The mechanisms through which the messages affectedstudent success were assessed through teacher and student surveys, and phoneinterviews with parents. Although the researchers do not find evidence of the textmessagesincreasingtheoccurrenceofsupportiveconversationsoverall,studentsinthetreatmentreportedthattheirparentsspoketothemmoreoftenaboutthingstheyshouldworkontodobetterinschool(Kraft&Rogers,2015).Theauthorsproposethatthetextmessageshelped to improve theeffectivenessofconversations, rather than increasetheirfrequency.Theabove studies focuson leveragingparental support.Todate, few studies seek toleveragepeersupporttopromoteeducationalsuccess.Peer-ledinterventionstypicallyfocus on prevention, such as smoking, substance abuse or dating violence (see forexampleCampbelletal.,2008;Faggianoetal.,2010;Wolfeetal.,2009).Additionally,these interventions are often delivered inside the classroom.Nevertheless, peer-ledinterventionswhere studentsare trained toactaspeer supporterscan lead toclearimprovementsinthetargetbehaviour.Forexample,Campbelletal.(2008)evaluatedtheeffectivenessofawhole-school interventiondeliveredacross59schools intheUnitedKingdom.Externaltrainerstrained influentialstudentstoencouragetheirpeersnottosmoke, by providing themwith information about the short- and long-term risks ofsmoking.Theyalso taught these ‘peersupporters’ listeningandcommunicationskills.Campbelletal.(2008)foundthattheoddsratioofbeingasmokerininterventionschoolscomparedtocontrolschoolswas0.75.Thereductioninuptakeofregularsmokingwassustained for2 years afterprogrammedelivery.However, this typeof intervention isrelatively costly in comparison to textmessaging interventions, which can often bedeliveredfor lessthan£5perstudent.Thepeer-ledcommunication interventionsalsotypically do not focus on promoting educational success. Before we turn to ourexperimentaldesign,webrieflydiscusstheeducationalcontextofourstudiesconductedinFurtherEducationcollegesinEngland.

Thepolicychallengeofhelpingpost-16studentssucceedinMathsandEnglishOnaverage, thereturns toachievingmathsandEnglishqualificationsarepositiveandsignificant.Forexample,thosewhoachievetheirmathsandEnglishatFEcollegesearnonaverage4.2%morethanthosewhodidnotachieve(Cerqua&Urwin,2016).Achieving

8


basic levelsof literacyandnumeracyarealsoassociatedwith improved lifeoutcomes,including thechancesofgettinga job interview (Sandersetal.,2016), retainingwork(Hanusheketal.,2015;McIntosh&Vignoles,2000),aswellassocialbenefitssuchaslowercrime rates (Feinstein, 2002), and increased civic participation and social cohesion(Feinstein,Budge&Vorhaus,2008).Higherskillslevelsalsoaffectoutcomesatanationallevel.Thedistributionofskillsiscentraltointergenerationalmobilityandthecountry’sgrowthrate(Burgess,2015).Studentssit theGeneralCertificateofSecondaryEducation (GCSE)mathsandEnglishexams at the end of compulsory school education at age 16. Last year, 41.3% of allstudents inEnglandfailedtoachieveeithermaths,EnglishorbothGCSEsatA*-C,thescoresthatqualifyasa‘pass’(DfE,2016b,p.8).Since2015,allUKstudentsarerequiredtostayineducationortraininguntiltheir18thbirthday(EducationandSkillsAct2008;c.25).Additionally,ifstudentsdidnotpasstheirmathsandEnglishGCSEsatage16,theyarenotrequiredtocontinuepursuingtheseorotherequivalentqualificationsat16-18(conditionoffundingrequirement,EFA,2014).Asaresult,60%and54%ofstudentswhofailedtheirGCSEsinmathsand/orEnglishatage16goontoretakethesubjectatanFEcollege,respectively(Porter,2015).In2015,110,811and100,239studentswenttoanFEcollegetoretaketheirGCSEsinmathsandEnglish,respectively(Porter,2015).FEcollegesalsoprovidemathsandEnglishqualificationstoadult (19+) learners,althoughmanyofthese courses are part-time or short-term intensive (BIS, 2016). The experimentsdescribedbelowonly includeadult learnerson full-time,year-longmathsandEnglishcourses,ofwhichtherearefewerthan16-18yearoldstudents.FEcollegestudentsgenerallypursueeitherFunctionalSkills(FS)orGCSEqualificationsinmathsandEnglish,wherethelatterweredesignedtopreparestudentsforhigherstudyandtheformertopreparestudentsfortheworldofwork.GCSEsareregardedasthe‘goldstandard’bymanysincetheWolfreportwaspublished(Wolf,2011),butotherspositthatFSqualificationsaremorerelevanttothejobsyoungpeoplearebeingtrainedtodo(The Education and Training Foundation,2015). FSqualifications aredelivered at fivelevels (Entry1,2,3,Level1,2),butonlystudentspursuingLevel1or2tookpart intheexperiments reportedbelow. FS Level 1 is equivalent toGCSED-G and FS Level 2 isequivalenttoGCSEA*-C.ThisistoensurethatstudentsonGCSEandFSqualificationsareofsimilarskilllevels.DespitetheimportantroleFEplaysinpromotingbetterskilllevels,thereisrelativelylittlequantitative research into the causes of success or failure in this environment. Thegreaterandgrowingliteratureinvestigatingimprovingeducationaloutcomesistypically

9


focused on traditional learning environments, such as school and university, andoperatingacrossthepopulationofstudents.TherearegoodreasonstosuspectthattheenvironmentofanFEcollege,andthepeopleattendingthem,aredifferentinmanywaystomainstreameducation.Firstly, students re-takingGCSEsaredistinguishedbyhavingalready failed theexamstowardswhich theyareworking.MaierandSeligman (1976)coined the term “learnedhelplessness”,which isamentalstatewhere individuals’motivation isunderminedbyexperiencingnegativecontextualconditions,causingthemtofaileven inrelatively lowstakessituations.Havingalreadyfailedthehigh-stakesmathsandEnglishGCSEexamsonce,andhavingtoretakethese,canleadtolowself-efficacy,orthefeelingofnotbeingincontrolofone’sownlearning.ArecentqualitativestudycarriedoutintheUKshowsthat many FE college students who previously failed their GCSEs at age 16 feeldemotivated, have weak coping skills, and struggle to feel intrinsically motivated(Anderson&Peart,2016).Additionally, the latestGovernment statisticsonmathsandEnglishprogressshowthatfewFEcollegestudentsgoontoachievebettergradesbyage18thantheydidatage16(DfE,2017).Onaverage,astudent’spointscoregoesbackwardsduringtheir16-18studies,by-0.28forEnglishand-0.31formaths.In2016,only22.8and22.4percentof studentspursuingGCSEsatage 16-18pass theirEnglishandmaths,respectively(atgradesA*-C;DfE,2017).The16-18cohortismorelikelytobemadeupoflow attainers in comparison to the age 16 cohort, and therefore require additionalsupport.FEcollege studentshave alsobeen found tohave lowereducational aspirations thanstudents from non-FE settings; instead they often focus on career andwealth goals(Baird, Rose &McWirter, 2012). Experimental evidence supports the hypothesis thatstudent motivation is low in FE colleges. Chande et al. (2015), found that averageattendancestartsat70%atthebeginningoftheacademicyear,decliningsharplyoverthefirsttermofenrolment.Policymakers,researchersandtheFEcollegesectorhavecalledforrenewedfocusonhelpingpost-16studentssucceedintheirmathsandEnglishstudies.However, FE colleges are facing significant challenges, both in terms of therecent 25% funding cuts (BIS, 2012) and sharp increases in student numbers. Ourintervention istherefore low-cost,scalable,andeasytoadminister.Beforedescribingthefirstexperiment,weturntotherelevanceofouroutcomevariable:attendance.

10


III. Outcomesandmeasurement:thepredictivepowerofattendance

Student non-attendance has been a cause for concern both in further and highereducationinstitutions(Newman-Fordetal.,2008).Classattendanceisabetterpredictorofgradesinaclassandoverallgradepointaverage(GPA)thananyotherknownpredictorof academic achievement, including study habits and scores on standardised tests(Credé, Roch & Kieszczynka, 2010). Looking at administrative data from a highereducationinstitutionintheUK,Newman-Fordandcolleagues(2008)findastatisticallysignificant correlation between class attendance and academic attainment. The linkbetween absence and attainment has also been studied by the UK Department forEducation using full-year absence and attainment data to calculate odds ratios (DfE,2016a).StudentsatKeyStage4(age15-16)are2.2timesmorelikelytoachievefivegoodpassesfortheirGCSEexamsthantheirpeerswhomissbetween10–15%ofclasses.Aftercontrollingforotherfactors,suchashavingspecialeducationalneedsorbeingeligibleforfreeschoolmeals(aproxyforbeingdisadvantaged),absencewasstillsignificantlyandnegatively related to attainment (DfE,2016a).In summary, correlational yet real-worldstudies have shown that attendance is predictive of academic achievement. It ischallengingtotestinafieldexperimentwhetherbetterattendanceresultsinimprovedachievement,asthiswouldrequiretheresearchertomanipulateattendancebyrandomlyassigningstudentstoattendorskipclass.Thosewhoattendallclassesare likelyquitedifferent intermsofmotivationormayfacefewerpracticalobstaclesthanthosewhomissasignificantnumberofclasses.Nevertheless,arecentRCTevaluationofastudentmentoring program shows that these barriers to attendance can be targeted bypersonalisedinterventions(Guryanetal.,2016).Helpingthosewhomissclassesregularlytoattendclassmoreoftenisaprimaryaimoftheexperimentsdiscussedinthispaper.Atapracticallevel,attendanceisanimportantandrelevantoutcomemeasurebecauseitcanbeeasilyandpreciselymeasured.Awealthofinformationonstudentbehaviourisavailable fromcollege administrativedatasets.Theuseof administrativedatasetshasrecentlybecomemorepopularinimpactevaluation,asitallowstheresearchertogatherinformationabout real-lifebehaviourof largeamountsofpeople (Figlio,Karbownik&Salvanes,2015).We collect attendance in a week-by-week format, where potential and missedattendances are recorded for all participating students. As described inmore detailunderthedataheaders inExperiment1and2,weaverageattendanceforthesubject(mathsorEnglish)inwhichthestudentwastreated.

11


IV. Experiment1

OurfirstexperimentwascarriedoutinfivecollegesinEngland.Collegeswererecruitedaspartofanationwiderecruitmentcampaignandweredeemedeligibletoparticipatebasedontheirhavinggivenconsentandbeingofasuitablesize.

DesignandInterventionStudentswhosignedupatthebeginningoftheacademicyear2015/16totakepartintheinterventionwereaskedtonominatetwoindividualstheybelievedwouldbesupportiveoftheirlearningas“studysupporters”.Studentsnominatedtwo‘StudySupporters’andwere subsequently individually randomised into two arms: in one arm the StudySupportersreceiveweeklytextmessages,intheotherarmtheydonotreceiveany.Figure1,below,showstheflowofparticipantsthroughtheexperiment.

Figure1:ParticipantFlowDiagram,Experiment1

12


The messages were co-authored by college academic tutors and were designed toencouragethesupporterstoaskthestudenthowrevisionisprogressing,topraisetheeffort the student is making and to wish the student luck ahead of exams andassessments.Theinterventionwasintendedtobealow-costandlight-touchprocessforcollegestaffandadministrators.Textmessagesweresenttostudents’‘studysupporters’mobilephonesatweekly intervalsduringtheacademicyear. ItwasdeliveredbetweenSeptember2015andJune2016tostudentsstudyingthesubjectsofmathsand/orEnglish,takingeitherGCSEsorFS (Level1or2)qualifications.Thetypesoftextmessagesthatnominated study supporterswere sent included the following information: (a)coursecontent,(b)advancenoticeofupcomingexams,(c)academicresourcesavailabletothestudent,and(d)generalpositivereflexiveconversationprompts.Allsupportersweresentabalancedmixof theabovecategories.Forexample,amessage for tosupportersofstudentstakingaGCSEmathsclassreadasfollows:

Hi [supporter forename]2, [learner forename] has recently learnt aboutpercentages.Ask [him/her] tocalculate the finalpriceofa£250TVafteradding20%VAT(taxonthingsyoubuy)andshowyouhow[he/she]workeditout.Thanks,[College]

Similarly,amessagetosupportersofstudentstakinganFSEnglishclassread:

Hi[supporterforename],pleaseask[learnerforename]if[he/she]hasbeenworkingon[his/her]individualSPaG(personaldevelopment)targetsontheBKSB website. Also, please ask if [he/she] is practising spelling andpunctuation:hardworkpaysoff!Thanks,[College]

Allcollegesreceivedauniquescheduleoftextmessages,astheircontentwastailoredtocollege exam and termdates, and the course curriculum.However,members of theresearchteamwrotethemajorityoftextmessagesandcheckedwhetherthelengthandcontentofmessageswerecomparableacrosscolleges. SeeAppendixFforadditionaltextmessageexamples.Atotalof35weeklymessagesweresentouttostudysupporters,anaverageofonemessageperweek.Therewasnovariationindosagebetweencollegesortypesofcourses.

2Thefieldsinbrackets‘[...]’wereautomaticallymergedwithstudentdataonthetextmessagingplatform,FireText,ensuringthatallrecipientsreceivedpersonalisedtextmessages.

13


DataThestructureofourdataareasfollows.EachstudentwastreatedonlyintheirmathsorEnglishcourse,eveniftheytakebothcourses.Thecourseassignmentwasdeterminedatacollegelevel:somecollegesimplementedtheinterventiononlyintheirmathscourses,and some colleges only did so in English courses. See Table 1 for the college-levelassignmenttomathsandEnglish.IfastudentatCollegeA(assignedtomaths)tookbothmaths and English, they were only treated in their maths class. Therefore, eachobservation inthedatasetcorrespondstoan individual inthecourse (eithermathsorEnglish) their Study Supporters were texted about. However, a small number ofparticipants(N=25)areidentifiedinourdataastakingbothGCSEandFSclasses,andour model distinguishes between the two. Appendix A shows the distribution ofattendancerateswithinourstudy,andTable1displaysthemeanattendancerateforeachcollege.College administrative datasets of week-by-week attendance data are merged withconsentformdata,whichprovideanindicationofwhoconsentedtobepartofthestudysupporterintervention,aswellastheirrandomassignmenttotreatmentortocontrol.Table1,below,showsthesamplesizeforeachcollege,dividedbetweenparticipantswhoconsentedtobepartofthetrialbutwhowereassignedtothecontrol,andthosewhoconsentedandwereassignedtotreatment.

Table1:ConsentandTreatmentAssignment,byCollege,Experiment1College Control Treatment Total Subject

TreatedAverage

Attendance(SD)

CollegeA 90 93 183 Maths 60.3%(31.5%)

CollegeB 125 128 253 English 61.0%(28.8)

CollegeC 97 99 196 English 77.1%(24.2%)

CollegeD 90 112 202 Maths 55.5%(21.7%)

CollegeE 93 94 187 Maths 58.0%(21.7%)

Total 495 526 1021 - 62.4%(28.4%)

Note:Randomisationisconductedstratifiedonparticipantsclass,preferencingtreatmentassignment.Hence,ifaclasscontains10individuals,5willbetreatedand5controlled,butifaclassconsistsof11individuals,6willbetreatedand5controlled.

14


ParticipantsDataongenderandageiscollectedsporadicallythroughdataprovidedbythecollege.Evenwhereitiscollected,thisdataisnotcompleteinitsrecording,andsothedataarepatchy.Inordertoavoidlosingdata,wecodeageandgenderas0wheretheyaremissingandcreateabinaryindicatorformissingnessforeachofageandgender.29%ofstudentsdonotreporttheirgender.Amongthosethatdo,weseearoughlyevensplit,with51.4%ofstudentsaremaleand48.6%arefemale.Themajorityofstudentsarebetween16and18yearsold(56.4%),andweseerelativelyfewadultstudents(aged19orabove;14.5%).For29.1%ofstudentswedonothavedataonageorgender(thisdoesnotappeartobeassociatedwithaparticularproblem).Classidentifiersdonotuniformlyreportthesubjectofstudy(MathsorEnglish),orthelevelbeingstudied(FS,GCSEs).Havingidentifiedstudents’coursesasbestwecan,weexcludecourses,wherepossible,thatwerenottreatedinthatcollege.Forexample,incollegeD,onlymathsclassesweretreated,andsoEnglishclassesareexcludedfromourdataset.Forsomeparticipants,weareunabletocorrectlyidentifywhethertheirclassesare forEnglishorMathsdue toambiguousclass identifiers,andsoanaverageof theavailableattendancedataforallclassesinwhichthatstudentisregisterediscreated.Inmostcases,thisproducesadatasetcontainingoneobservationperparticipant.

RelationshipwithnominatedStudySupportersDuringthesign-upprocedureatthestartoftheAutumntermsemester,studentswereaskedtodescribetheirrelationshipwiththeperson(s)theynominated.Theirfree-textresponsesweresubsequentlycodedandcategorisedintobroadtypesofrelationships,seeAppendixB.Themajorityofstudentsnominateeitheramemberoftheirnuclearfamily(41.3%)orapeerinsideoroutsideofcollege(40.5%).

Analysisstrategy&ResultsWenowproceedtoanalysis.Ourfirstregressiontable,Table2,reportsestimatesfromthefollowinglinearregressionmodel:

𝐴"#$ = 𝛼 = 𝛽)𝑆" + 𝜷𝟐𝑳𝒈 + 𝛽0𝐶$ + 𝑢"#$where A is the percentage of classes that individual i in class (group) g in college cattendedintheperiodcoveredbyourdata.Sisabinaryvariablesetto1ifaparticipantisassignedtothestudysupportertreatment,and0otherwise.Lisavectorofclass-levelbinaryvariablesindicatingwhethertheclassstudiesforFSqualificationsorGCSEs(theomitted category iswhere theparticipant’s levelof study ismissing), andwhether a

15


participantismale,Cisavectorofcollegefixedeffects,anduisanerrortermaccountforclusteringattheclasslevel.Column1ofTable2reportsthisanalysisforourfullsample.Column2reportsitonlyforGCSEstudents,column3onlyforFSstudents.Column4reportsthesameanalysisforstudentsonEnglishcourses,andcolumn5forstudentsonmathscourses.

Table2:Maineffects,Experiment1(OLSRegressions) (1) (2) (3) (4) (5)

All GCSE FS English MathsStudySupporter

2.719 4.757* 2.138 -0.266 5.018*

(1.717) (2.364) (3.096) (2.719) (2.187)

Gender:male

-7.425*** -6.554** -4.058 -10.812*** 0.678

(2.003) (2.519) (3.084) (2.800) (3.063)

FS 6.612* 3.151 0.000 10.593 2.803

(3.337) (5.372) (.) (5.998) (4.417)

GCSE 0.285 0.000 -4.254 5.199 -3.104

(2.344) (.) (3.822) (4.464) (2.821)

Constant 61.700*** 60.612*** 67.890** 56.053*** 69.709***

(3.352) (2.634) (23.489) (5.729) (4.621)

CollegeFixedEffects

Yes Yes Yes Yes Yes

N 1021 558 231 442 579

Notes:Ordinaryleastsquaresregressions.Standarderrors,adjustedforclusteringattheleveloftheclass,inparentheses.Notethatcolumns2and3donotsumto1021asforsomeclassesweareunabletodeterminewhetherparticipantsstudyFSorGCSEs.*=p<0.05,**=p<0.01***=p<0.001.

16


In thismain analysis,we find a small overall effect of our intervention,which is notstatisticallysignificant(p=.114).AmongparticipantstakingtheirGCSEs(whoaretypicallyyoungerinoursample(p=.045)),wefindastatisticallysignificant4.76%pointincreasein attendance as a result of our intervention (62.7% attendance for treated GCSEstudents,comparedtoacontrolgroupmeanof57.9%,p=.045).Wealsofindthattheinterventionwaseffectivewhendelivered inmathscourses (p= .022),butnotwhendeliveredinEnglishcourses.

V. Experiment2

DesignandInterventionInoursecondtrial,conductedcontemporaneouslywiththefirst,isasubsetofalargertrial(reportedinBehaviouralInsightsTeam(2016)),inwhichasubsampleofclassesarerandomlyassignedtobepartofthestudysupportertrial.ThecollegesinExperiment1wereunabletoimplementthesuiteofthreeinterventionsasthetwootherinterventions,GritandValuesAffirmationrequiredmore intensivecollegesupportand in-classtime.Collegeswhowereunabletocommittoimplementingallthreewerethereforeassignedtoimplementoursocialsupportinterventiononlywhichrequiresverylittlein-classtime.Nevertheless, implementationofthestudysupporter interventionwasnotaffectedbycollege organisational constraints. Members of the research team, external to thecolleges,collected thedataandadministeredall textmessages.TheprocedureswereidenticalbetweenExperiment1and2,andcarriedoutsimultaneously.ForcollegesinExperiment2,participantswithintheclassesassignedtobepartofthestudysupportertrialwereaskedtoconsenttobeingpartofthestudy,andtoprovidedetails of two study supporters. Participants who consented and provided thisinformationwereindividuallyrandomisedtotreatmentandcontrol.Withintheseclasses,48%ofparticipantsconsenttobeingpartofthestudy.

17


Figure2:ParticipantFlowDiagram,Experiment2

DataThestructureofourdataareasfollows.Thecollegeadministrativedatasetsofweek-by-weekattendancedataaremergedwithconsentformdata,whichprovideanindicationofwhoconsentedtobepartofthestudysupporterintervention,aswellastheirrandomassignment to treatment or to control. See Table 3, below, for the sample size ofparticipatingstudentsineachcollege.AppendixCshowsthedistributionofattendanceratesinExperiment2,andTable3displaystheaverageattendanceratesforeachcollegeinthisstudy.

18


Table3:ConsentandTreatmentAssignment,byCollege,Experiment2

College Control Treatment Total SubjectTreated

AverageAttendance

(SD)

CollegeF 27 25 52 English 44.3(35.3)

CollegeG 40 42 82 English 69.2(28.9)

CollegeH 109 126 235 Maths 65.7(25.0)

CollegeI 49 36 85 Maths 25.1(36.1)

Total 226 229 455 - 56.3(33.6)

ParticipantsInExperiment2,wehavecompletedataforageandgender.Weseearoughlyevensplitofgenderwhere56.3%aremaleand43.7%are female.As inExperiment 1, studentsbetween theagesof 16and 18aremorecommonly found in thedataset (93.2%) thanthose aged 19 or above (6.8%).Class identifiers do not uniformly report the subjectstudies(MathsorEnglish),orthe levelbeingstudied(FS,GCSEs).VariablesarederivedheresimilarlytoExperiment1,andaredisplayedinthetableinAppendixD.

RelationshipwithnominatedStudySupportersDuringthesign-upprocedureatthestartoftheAutumnterm,studentswereaskedtodescribetheirrelationshipwiththeperson(s)theynominated.Theirfree-textresponsesweresubsequentlycodedandcategorisedintothefollowingbroadtypesofrelationships,seeAppendixE.SimilartoExperiment1,studentsmostoftennominatetwocategoriesofstudysupporters,namelynuclearfamily(44.6%;parents,brothersandsisters)andpeers(42.2%;bothfrominsideandoutsideofcollege).

AnalysisWenowproceed toanalysis.Oursecond regression table,Table4, reportsestimatesfromthefollowingmodel;

𝐴"#$ = 𝛼 = 𝛽)𝑆" + 𝛽3𝐿# + 𝛽0𝐶$ + 𝑢"#$Where A is the percentage of classes that individual I in class (group) g in college cattendedintheperiodcoveredbyourdata.Sisabinaryvariablesetto1ifaparticipantisassignedtothestudysupportertreatment,and0else.Lisavectorofclasslevelbinary

19


variablesindicatingwhetheraclassstudiesEnglishorMaths(theomittedcategoryisthatneither English norMaths are recorded), and other treatments applied to this classoutside of the study supporter trial,whether the class studies towards FS or GCSEqualifications,Cisavectorofcollegefixedeffects,anduisanerrortermaccountingforclusteringattheclasslevel.Column1ofTable4reportsthisanalysisforourfullsample.Column2reportsitonlyforGCSEstudents,column3onlyforFSstudents.Column4reportsthesameanalysisforstudentsonEnglishcourses,andcolumn5forstudentsonmathscourses.

Table4:MainResults-Experiment2

(1) (2) (3) (4) (5)

Fullsample GCSE FS English Maths

StudySupporter 7.596** 7.990* 6.865 7.433 7.479*

(2.752) (3.281) (4.497) (5.483) (3.172)

Gender:male -2.374 -1.507 -4.605 3.823 -4.668

(2.788) (3.307) (4.622) (5.763) (3.169)

16-18yearsold 6.192 8.710 -0.002 7.797 4.880

(5.493) (6.561) (8.969) (8.421) (7.492)

Constant 86.182** 82.797** 38.457*** 78.380* 70.120*

(29.763) (28.915) (9.949) (33.079) (29.169)

CollegeFixedEffects

Yes Yes Yes Yes Yes

N 455 296 159 134 321Notes:Standarderrors,clusteredattheclasslevel,inparentheses.*=p<0.05,**=p<0.01***=p<0.001.

Overall,wefindapositiveandsignificanteffectofthestudysupporterinterventiononattendance(p=.006).Averageattendanceisonaverageat60.1%forthestudentswhosestudysupporterreceivesweeklytexts,comparedto52.5%forthoseinthecontrolgroup,adifferenceof7.6%points.Inthepartitionedanalysis,weseethatespeciallystudentsenrolled on aGCSE qualification benefit from the intervention (p= .016), and thosestudyingmaths(p=.019).

20


VI. PooledAnalysis

Followingtheconclusionofbothexperiments,weconductapooledanalysisacrossthetwostudies.Todoso,wepooldatafrombothstudiesandestimateamodelidenticaltothemainanalysisofthetwostudiesabove.TheresultsofthiscombinedanalysiscanbeseenbelowinTable5.

Table5:PooledanalysisofExperiments1and2

(1) (2) (3) (4) (5)

All GCSE FS English Maths

StudySupporter 4.233** 5.956** 4.148 1.871 5.698**

(1.464) (1.918) (2.596) (2.457) (1.806)

Gender:Male -5.496*** -4.863* -3.429 -8.152** -2.015

(1.648) (2.032) (2.662) (2.547) (2.201)

16-18yearsold 3.739 8.198 -7.971 10.459 0.279

(4.677) (5.919) (7.762) (7.601) (5.938)

Constant 93.737** 84.033** 72.190** 88.335** 65.701***

(28.514) (28.520) (26.324) (30.519) (6.196)

CollegeFixedEffects

Yes Yes Yes Yes Yes

N 1476 854 390 576 900

Notes:Ordinaryleastsquaresregressions.Standarderrors,adjustedforclusteringattheleveloftheclass,inparentheses.Notethatcolumns2and3donotsumto1476(thetotalnumberofparticipantsacrossbothstudies)as forsomeclassesweareunable todeterminewhetherparticipantsstudyFSorGCSEs.*=p<0.05,**=p<0.01***=p<0.001.

Ourpooledanalysissuggests,inlinewithourpreviousfindings,thatourinterventionhasapositiveandstatisticallysignificanteffectonattendance(p=.004).Averageattendanceisat58.4%forstudentsinthecontrolgroup,whereasthosewhoweretreatedhaveanaverageattendanceof62.6%,ora4.2%point increase. Itsuggeststhatthiseffect isprimarilydrivenbyan increase inattendancebyparticipantsstudying forGCSEs (p=.002)andforstudentsstudyingmaths(p=.002).

21


SecondarypooledanalysisAswellasaverageeffectsacrossparticipants,wearealso interested inwhethersomegroupsaremoreaffectedbyourinterventionsthanothers.Unfortunately,thequalityofrecordingofageandgenderislow,anditisthereforedifficulttoconsiderthesegroupsseparately.However,giventhatourinterestisprimarilyaroundparticipants’motivationforlearning,wenowconsiderhowparticipantswithdifferentlevelsofpriormotivationare influenced by the intervention.We do not have baselinemeasurement of eitherattendance,whichearlyinacoursewewouldotherwiseuseasameasureofmotivation.As such, we conduct quantile regression at the 10th, 25th, 50th, 75th and 90thpercentilesofthedistributionofattendance,theresultsofwhichareshowninTable6,below.

Table6:Quantileregressionsusingpooleddata

(1) (2) (3) (4) (5)

Quantile 0.1 0.25 0.5 0.75 0.9

StudySupporter

3.704 3.704 1.389 0.000 0.000

(2.818) (2.231) (1.839) (1.483) (0.344)

Gender:Male -7.407* -3.704 -3.704 -3.030 0.000

(3.398) (2.212) (2.330) (2.518) (0.476)

16-18yearsold 14.550 0.265 4.000 3.030 -0.000

(8.307) (11.668) (5.114) (4.301) (2.616)

>18yearsold 14.550 0.265 5.389 7.143 -0.000

(8.445) (11.469) (5.256) (3.788) (2.347)

Constant 84.048*** 94.034*** 91.664*** 86.537*** 92.976***

(11.026) (13.121) (8.157) (5.714) (3.779)

CollegeFixedEffects

Yes Yes Yes Yes Yes

ClusteredErrors

Yes Yes Yes Yes Yes

Controlgroupvalue

8% 40.7% 70% 85.7% 92.8%

N 1476 1476 1476 1476 1476Notes:AnalysisconductedusingbootstrappedquantileregressionsinStata.1000repetitions.Standarderrors,clusteredattheclasslevel,inparentheses.Controlgroupvaluesarethevalueforthecontrolgroupatthepointinthedistributiontowhichthatregressionrelates.*=p<0.05,**=p<0.01***=p<0.001.

22


Ascanbeseeninthetableabove,theoveralleffectsofourinterventionappeartobeconcentratedatthelowerendofthedistribution,amongthe10thand25thpercentiles,andatthemedian,althoughtheseeffectsarenotindividuallysignificantatconventionallevels,whilenoeffectsareobservedforparticipantsinthe75thand95thpercentiles.

VII. Conclusionanddiscussion

WehaveconductedtwofieldexperimentsthataimtotesttheeffectoftriggeringsocialsupportonattendanceoffurthereducationcollegesintheUK.Inboththeexperimentsand in the pooled analysis of the two,we find a positive effect of our intervention,although this isnotstatisticallysignificant inExperiment 1.Overall,weargue that theeffectsizeofourinterventionissubstantial,amountingtoanincreaseof0.10standarddeviations inattendance inourpooledanalysis.Chandeetal. (2017) foundeffectsonattendanceoveranentireschoolyearandattainment (pass/fail) inasimilaracademiccontext,andfindanincreaseinattendanceof0.19standarddeviationsleadingtoan8.7%pointincreaseinpassratesatthefinalexam(althoughwenotethatinChandeetal.’s,2015,interimanalysis,theeffectsonattendanceweremoremodest).In our pooled analysis, we also find that the intervention effect is largest amongparticipantsstudyingforGCSEexamsandnotstatisticallysignificantforthosestudyingforFSqualifications.AsstudyingGCSEsratherthanFSisnotexperimentallymanipulated,therearemultiplepotentialexplanationsforthisdifference.First,inmostcollegesthecurriculumwaswell-defined forGCSE classes andmore variable for FS classes.As aresult,FStutorswereoftenlessabletoletusknowwhenexamswerehappening(i.e.astheseweretypicallyspreadoutoverafewweeks,andstudentswithinoneclassmightbetakingtheexamatdifferenttimes).Therefore,theexamremindertextswewereabletosendoutforFSclasseswereoftenlessspecificandperhapsthereforealsolessactionablefor the study supporter. A second set of potential explanations lies in differencesobserved inourdataforGCSEversusFSstudents.ParticipantsstudyingforGCSEsaretypicallyyoungerthanFSstudentsinoursample(meanage=17.5vs18.4),andwemightanticipateeitherthatyoungerstudents(who,asdiscussedintheintroduction,arelikelytohavepreviouslyfailedthesesamequalifications),mightbelessmotivatedinthefirstplaceandhencehavemoretogainfromsocialsupport.Sincethesestudentsalsoaremorelikelytoliveathome,itisplausiblethatitmaybemorestraightforwardfortheirsocialsupporttobeactivatedtoour interventions.Additionally,GCSEstudents inoursampleweremore likelytobetakingmathscourses,whicharealsoassociatedwithagreatertreatmenteffect,perhapsbecauseoftheintervention’scapacityforcombating

23


mathsanxiety.OursamplesizeforthisstudydoesnotpermitrigorousanalysissplitbybothGCSEstatusandsubject.Socialsupporthasbeenfoundtobufferstudentsfromdysfunctionalthoughtsandanxiety(Song,Bong,Lee&Kim,2015),andasthereisastrongnegativeculturearoundmathslearningamongyoungadultsonvocationalcourses(TheEducation & Training Foundation, 2014), the interventionmay have helped studentsovercometheirmaladaptivebeliefsaboutlearningmaths.Although our second study and our pooled analysis yield positive and statisticallysignificantresults,itisworthconsideringwhyExperiment1onedidnotproduceresultsthatmet this threshold. The variance within that sample and a higher intra-clustercorrelationrate,suggestthatthismayinpartbearesultofalackofstatisticalpower.Itshouldbenoted,however,thatthepointestimateoftheeffectinthissamplewasalsolowerthanintheotherstudy,andsopowerdoesnotappeartobetheonlyexplanation.There are a few possible explanations here. First, the overall level of attendance inExperiment1washigherthanExperiment2,perhapsindicatingaceilingeffect.Second,wefindconsistentlysignificantandpositiveimpactsofourinterventiononGCSEstudents(including in Experiment 1), who alsomake up a smaller share of the population inExperiment1thaninExperiment2,whichcouldexplainourresults.Thereare severalpossibleexplanations for theeffectivenessofour intervention.Theresultsdemonstratethattextmessagessenttoindividualswhowerenominatedbythestudent can generate improvements in college attendance.However,we cannot testwhether itwas an increase in perceived social support on the student’s part, or anincrease inmonitoring behaviour on the supporters’ part, or both. A previous textmessagingtrialwithparentshasfoundthatindividualsinthetreatmentgroupwere85%morelikelytocontacttheschoolregardingschoolworkorgrades(Bergman,2015)thanparentswhodidnotreceivetextmessages.Controlling(i.e.exertionofpressurethroughtheuseofcommands,punishment,orcoerciveinteractions)versusautonomysupportive(i.e.demonstratinginterest,attention,praise,andreinforcement)stylesofinvolvementareassociatedwithdifferentoutcomes(Deci&Ryan,1987).Rogersetal.(2009)foundthatsupportiveparentalinvolvementwasassociatedwithhigheracademicachievementinchildren,whereascontrollinginvolvementproducedtheoppositeoutcome.Sincethestudysupporters’styleofinvolvementwasnotmeasuredinthepresenttrial,wecanonlyspeculate about the behaviours they engaged in after receiving the text messages.Additionally, we may see a positive effect of the intervention because the studysupporters simplypassedon the informationcontained in themessages, rather thanactivelyengaginginsupportivebehaviourssuchashelpingwithassignmentsorprovidingemotional support. The present trial design does not allow examination of these

24


mechanisms.GiventhepolicygoalstoimproveattendanceandachievementinmathsandEnglish courses at FE Colleges, it is essential to better understand themechanismsthroughwhichpersonalisedmessageshelpleveragestudents’socialsupportnetworkstoimproveacademicmotivation.Overall, we have presented robust evidence from two field experiments that socialsupport is effective at increasing the attendance of students at FE colleges. Thereremainsmuch to learn about the underlyingmechanisms of the intervention and, inparticular, whether some groups find this leveraging of social support particularlybeneficialand/orwhethertheinterventionisparticularlysuitedtosometypesofcourseorsubjectmatter.Futurestudieswillbegintotackletheseissues.

25


VIII. Bibliography

Anderson,N.,&Peart,S.(2016).Backontrack:exploringhowafurthereducationcollegere-motivatesstudentstore-sitpreviouslyfailedqualificationsatGCSE.ResearchinPost-CompulsoryEducation,21(3),196-213.

Baird,J.A.,Rose,J.,&McWhirter,A.(2012).Sotellmewhatyouwant:acomparisonofFEcollegeandotherpost-16students’aspirations.ResearchinPost-CompulsoryEducation,17(3),293-310.

Barnard,W.M.(2004).Parentinvolvementinelementaryschoolandeducationalattainment.Childrenandyouthservicesreview,26(1),39-62.

Bergman,P.(2015).Parent-childinformationfrictionsandhumancapitalinvestment:Evidencefromafieldexperiment.Retrievedfromhttp://www.columbia.edu/~psb2101/BergmanSubmission.pdf

Blake,S.M.,Simkin,L.,Ledsky,R.,Perkins,C.,&Calabrese,J.M.(2001).Effectsofaparent-childcommunicationsinterventiononyoungadolescents’riskforearlyonsetofsexualintercourse.Familyplanningperspectives,52-61.

Burgess,S.M.(2016)HumanCapitalandEducation:TheStateoftheArtintheEconomicsofEducation.IZADiscussionPaperNo.9885

Campbell,R.,Starkey,F.,Holliday,J.,Audrey,S.,Bloor,M.,Parry-Langdon,N.,...&Moore,L.(2008).Aninformalschool-basedpeer-ledinterventionforsmokingpreventioninadolescence(ASSIST):aclusterrandomisedtrial.TheLancet,371(9624),1595-1602.

Castleman,B.L.,&Page,L.C.(2013).Summernudging:Cantextmessagesandpeermentoroutreachincreasecollegegoingamonglow-incomehighschoolgraduates.InSoc.forResearchonEducationalEffectivenessSpringConf.,DC.

Castleman,B.,&Page,L.C.(2016).ParentalInfluencesonPostsecondaryDecisionMaking:EvidencefromaTextMessagingExperiment.EdPolicyWorksworkingpaper,retrievedfromhttp://curry.virginia.edu/uploads/resourceLibrary/50_Parental_Influences_Postsecondary_Decision_Making.pdf

Cerqua,A.,&Urwin,P.(2016).ReturnstomathsandEnglishlearning(atlevel2andbelow)infurthereducation:May2016.

Chandeetal.,(2015)Curbingadultstudentattrition:Evidencefromafieldexperiment.(WorkingPaperNo.14/335).CentreforMarketandPublicOrganisation.Retrievedfromwww.bristol.ac.uk/media-library/sites/cmpo/documents/WP14335Web_Version.pdf

26


Chandeetal.,(2017)Alargescalefieldtrialevaluatingtheeffectivenessofsocio-psychologicalinterventionstoimproveacademicachievementatFurtherEducationColleges.

Chen,J.,&Lin,T.F.(2008).Classattendanceandexamperformance:Arandomizedexperiment.TheJournalofEconomicEducation,39(3),213-227.

Credé,M.,Roch,S.G.,&Kieszczynka,U.M.(2010).Classattendanceincollege:Ametaanalyticreviewoftherelationshipofclassattendancewithgradesandstudentcharacteristics.ReviewofEducationalResearch,80(2),272-295.

Danielsen,A.G.,Oddrun,S.,Hetland,J.,&Wold,B.(2009).School-relatedsocialsupportandstudents’perceivedlifesatisfaction.TheJournalofEducationalResearch,102(4).

Deci,E.L.,&Ryan,R.M.(1987).Thesupportofautonomyandthecontrolofbehavior.Journalofpersonalityandsocialpsychology,53(6),1024.

DepartmentforBusinessInnovation&Skills(BIS),(2012).SkillsFundingStatement2012-2015.TheSkillsFundingAgency,December2012.https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/82774/bis-12-p172x-skills-funding-statement-2012-2015.pdf

DepartmentforBusinessInnovation&Skills(BIS),(2016).PriorQualificationsofadultsundertakingMathsand/orEnglishlearningin2013/2014.Retrievedfromhttps://www.gov.uk/government/uploads/system/uploads/attachment_data/file/522944/bis-16-192b-prior-qualifications-adult-survey-classroom-2013-14.pdf

DepartmentforEducation(DfE),(2016a).ThelinkbetweenabsenceandattainmentatKS2andKS4.Retrievedfromhttps://www.gov.uk/government/uploads/system/uploads/attachment_data/file/509679/The-link-between-absence-and-attainment-at-KS2-and-KS4-2013-to-2014-academic-year.pdf

DepartmentforEducation(DfE),(2016b).ProvisionalGCSEandequivalentresultsinEngland,2015to2016.Retrievedfromhttps://www.gov.uk/government/uploads/system/uploads/attachment_data/file/467603/SFR37_2015.pdf

DepartmentforEducation(DfE),(2017).RevisedAlevelandother16-18resultsinEngland,2015/2016.Retrievedfromhttps://www.gov.uk/government/uploads/system/uploads/attachment_data/file/584124/SFR05_2017__A_level_and_other16-18_results_in_England_SFR_revised.pdf

27


Demaray,M.K.,&Malecki,C.K.(2002).Therelationshipbetweenperceivedsocialsupportandmaladjustmentforstudentsatrisk.PsychologyintheSchools,39(3),305-316.EducationandSkillsAct2008,ElisabethII.Chapter25(2008).London.Retrievedfromhttp://www.legislation.gov.uk/ukpga/2008/25/pdfs/ukpga_20080025_en.pdf

EducationFundingAgency(EFA),(2014).16to19funding:mathsandEnglishconditionoffunding.Retrievedfromwww.gov.uk/guidance/16-to-19-funding-maths-and-english-condition-of-funding

TheEducationandTrainingFoundation(ETF),(2015).MakingmathsandEnglishworkforall.Retrievedfromwww.et-foundation.co.uk/wp-content/uploads/2015/03/Making-maths-and-English-work-for-all-25_03_2015001.pdf

Faggiano,F.,Vigna-Taglianti,F.,Burkhart,G.,Bohrn,K.,Cuomo,L.,Gregori,D.,...&vanderKreeft,P.(2010).Theeffectivenessofaschool-basedsubstanceabusepreventionprogram:18-monthfollow-upoftheEU-Dapclusterrandomizedcontrolledtrial.Drugandalcoholdependence,108(1),56-64.

Figlio,D.N.,Karbownik,K.,&Salvanes,K.G.(2015).Educationresearchandadministrativedata(No.w21592).NationalBureauofEconomicResearch.

Furrer,C.,&Skinner,E.(2003).Senseofrelatednessasafactorinchildren’sacademicengagementandperformance.JournalofEducationalPsychology,95(1),148–162.

Garnefski,N.,&Diekstra,R.F.(1996).Perceivedsocialsupportfromfamily,school,andpeers:Relationshipwithemotionalandbehavioralproblemsamongadolescents.JournaloftheAmericanacademyofChild&adolescentPsychiatry,35(12),1657-1664.

Gottlieb,B.H.,&Bergen,A.E.(2010).Socialsupportconceptsandmeasures.Journalofpsychosomaticresearch,69(5),511-520.

Guryan,J.,Christenson.,S.,Claessens,A.,Engel,M.,Lai,I.,Ludwig,J.Turner,A.C.,&Turner,M.C.(2016).TheEffectofMentoringonSchoolAttendanceandAcademicOutcomes:ARandomizedEvaluationoftheCheck&ConnectProgram(NorthwesternWorkingPaperSeries:WP-16-18).

Hale,C.J.,Hannum,J.W.,&Espelage,D.L.(2005).Socialsupportandphysicalhealth:Theimportanceofbelonging.JournalofAmericanCollegeHealth,53(6),276-284.

Hanushek,E.A.,Schwerdt,G.,Wiederhold,S.,&Woessmann,L.(2015).Returnstoskillsaroundtheworld:EvidencefromPIAAC.EuropeanEconomicReview,73,103-130.

Hoover-Dempsey,K.V.,Walker,J.M.,Sandler,H.M.,Whetsel,D.,Green,C.L.,Wilkins,A.S.,&Closson,K.(2005).Whydoparentsbecomeinvolved?Researchfindingsandimplications.TheElementarySchoolJournal,106(2),105-130.

28


House,J.(1981).Workstressandsocialsupport.Reading,MA:Addison-WesleyJeynes,W.H.(2007).Therelationshipbetweenparentalinvolvementandurban

secondaryschoolstudentacademicachievementameta-analysis.Urbaneducation,42(1),82-110.

Johnson,D.,&Johnson,R.(1985).Motivationalprocessesincooperative,competitive,andindividualisticlearningsituations.Researchonmotivationineducation,2,249-286.

Kraft,M.A.,&Rogers,T.(2015).Theunderutilizedpotentialofteacher-to-parentcommunication:Evidencefromafieldexperiment.EconomicsofEducationReview,47,49-63.

OECD(2016).BuildingSkillsforAll:AReviewofEngland.Retrievedfromhttps://www.oecd.org/unitedkingdom/building-skills-for-all-review-of-england.pdf

OECD(2016).Countrynote,ResultsfromPISA2012.UnitedKingdom.Retrievedfromhttps://www.oecd.org/unitedkingdom/PISA-2012-results-UK.pdf

Ofcom(2015).Factsandfigures.Retrievedfromhttp://media.ofcom.org.uk/facts/Macdonald,G.,&Leary,M.R.(2005).Whydoessocialexclusionhurt?Therelationship

betweensocialandphysicalpain.PsychologicalBulletin,131(2),202–223.Maier,S.F.,&Seligman,M.E.(1976).Learnedhelplessness:Theoryandevidence.Journalofexperimentalpsychology:general,105(1),3.

Malecki,C.K.,&Demaray,M.K.(2003).WhatTypeofSupportDoTheyNeed?InvestigatingStudentAdjustmentasRelatedtoEmotional,Informational,Appraisal,andInstrumentalSupport.Schoolpsychologyquarterly,18(3),231.

Malecki,C.K.,&Demaray,M.K.(2006).Socialsupportasabufferintherelationshipbetweensocioeconomicstatusandacademicperformance.SchoolPsychologyQuarterly,21(4),375.

McGivney,V.(2004).Understandingpersistenceinadultlearning.OpenLearning,19(1),33–46.http://doi.org/10.1080/0268051042000177836

McIntosh,Steven,andAnnaVignoles.(2001)Measuringandassessingtheimpactofbasicskillsonlabourmarketoutcomes.OxfordEconomicPapers53(3,453-481.

Meeus,W.I.M.,Oosterwegel,A.,&Vollebergh,W.(2002).Parentalandpeerattachmentandidentitydevelopmentinadolescence.Journalofadolescence,25(1),93-106.

Miller,S.,Davison,J.,Yohanis,J.,Sloan,S.,Gildea,A.,&Thurston,A.,(2016).TextingParents.Evaluationreportandexecutivesummary.EducationEndowmentFoundation,UK,Retrievedfromhttps://educationendowmentfoundation.org.uk/public/files/Projects/Evaluation_Reports/EEF_Project_Report_Texting_Parents.pdf

29


Newman-Ford,L.,Fitzgibbon,K.,Lloyd,S.,&Thomas,S.(2008).Alarge-scaleinvestigationintotherelationshipbetweenattendanceandattainment:astudyusinganinnovative,electronicattendancemonitoringsystem.StudiesinHigherEducation,33(6),699-717.

Porter,N.(2015).CrossingtheLine:ImprovingSuccessRatesamongStudentsRetakingEnglishandMathsGCSEs.APolicyExchangePolicyBite.Retrievedfromhttps://policyexchange.org.uk/wp-content/uploads/2016/09/crossing-the-line.pdf

Robinson,C.,Lee,M.G.L.,Dearing,E.,&Rogers,T.(2017).ReducingStudentAbsenteeismintheEarlyGradesbyTargetingParentalBeliefs.HKSFacultyResearchWorkingpaper,retrievedfromhttps://research.hks.harvard.edu/publications/workingpapers/Index.aspx

Rogers,M.A.,Wiener,J.,Marton,I.,&Tannock,R.(2009).Supportiveandcontrollingparentalinvolvementaspredictorsofchildren’sacademicachievement:Relationstochildren’sADHDsymptomsandparentingstress.SchoolMentalHealth,1(2),89-102.

Rogers,T.,&Feller,A.(2016).Reducingstudentabsencesatscale.Unpublishedworkingpaper,retrievedfromhttp://scholar.harvard.edu/files/todd_rogers/files/02292016.pdf

Rosenfeld,L.B.,Richman,J.M.,&Bowen,G.L.(1998).Lowsocialsupportamongatriskadolescents.SocialWorkinEducation,20(4),245–260.

Rosenfeld,L.B.,&Richman,J.M.(1999).Supportivecommunicationandschooloutcomes,PartII:Academically“at-risk”lowincomehighschoolstudents.CommunicationEducation,48(4),294-307.

Sandersetal.(2016)Isitbettertobesmartortobekind?Isittoomuchtoaskforboth?Evidencefromtwoexperiments.Manuscriptinpreparation,BehaviouralInsightsTeam,London.UnitedKingdom.

Scales,P.C.,&Taccogna,J.(2001).Developmentalassetsforsuccessinschoolandlife.TheEducationDigest,66(6),34.

Smokowski,P.R.,Mann,E.A.,Reynolds,A.J.,&Fraser,M.W.(2004).Childhoodriskandprotectivefactorsandlateadolescentadjustmentininnercityminorityyouth.ChildrenandYouthServicesReview,26(1),63-91.

Song,J.,Bong,M.,Lee,K.,&Kim,S.I.(2015).Longitudinalinvestigationintotheroleofperceivedsocialsupportinadolescents’academicmotivationandachievement.JournalofEducationalPsychology,107(3),821.

Stice,E.,Ragan,J.,&Randall,P.(2004).Prospectiverelationsbetweensocialsupportanddepression:differentialdirectionofeffectsforparentandpeersupport?JournalofAbnormalPsychology,113(1),155–159.

30


TheEducation&TrainingFoundation(2014).EffectivePracticesinPost-16VocationalMaths,FinalReport.Retrievedfromhttp://www.et-foundation.co.uk/wp-content/uploads/2014/12/Effective-Practices-in-Post-16-Vocational-Maths-v4-0.pdf

Uchino,B.,Cacioppo,J.,&Kiecolt-Glaser,J.(1996).Therelationshipbetweensocialsupportandphysiologicalprocesses:Areviewwithemphasisonunderlyingmechanismsandimplicationsforhealth.PsychologicalBulletin,119(3),488–531.

Viner,R.M.,Ozer,E.M.,Denny,S.,Marmot,M.,Resnick,M.,Fatusi,A.,&Currie,C.(2012).Adolescenceandthesocialdeterminantsofhealth.Thelancet,379(9826),1641-1652.

Vorhaus,John,KathrynDuckworth,DavidBudge,andLeonFeinstein.(2008)Thesocialandpersonalbenefitsoflearning:Asummaryofkeyresearchfindings.CentreforResearchontheWiderBenefitsofLearning,InstituteofEducation,UniversityofLondon,London.

Wentzel,K.R.(1998).Socialrelationshipsandmotivationinmiddleschool:Theroleofparents,teachers,andpeers.JournalofEducationalPsychology,90(2),202–209.

Wentzel,K.R.,&Caldwell,K.(1997).Friendships,peeracceptance,andgroupmembership:Realtionstoacademicachievementinmiddleschool.Childdevelopment,68(6),1198-1209.

Wentzel,K.R.,Russell,S.,&Baker,S.(2015).Emotionalsupportandexpectationsfromparents,teachers,andpeerspredictadolescentcompetenceatschool.JournalofEducationalPsychology2016,108(2),242–255

Wilder,S.(2014).Effectsofparentalinvolvementonacademicachievement:ameta-synthesis.EducationalReview,66(3),377-397.

Winemiller,D.R.,Mitchell,M.E.,Sutliff,J.,&Cline,D.I.(1993).Measurementstrategiesinsocialsupport:Adescriptivereviewoftheliterature.JournalofClinicalPsychology,49(5),638-648.

Wolfe,D.A.,Crooks,C.,Jaffe,P.,Chiodo,D.,Hughes,R.,Ellis,W.,...&Donner,A.(2009).Aschool-basedprogramtopreventadolescentdatingviolence:Aclusterrandomizedtrial.Archivesofpediatrics&adolescentmedicine,163(8),692-699.

Wolf,A.(2011).Reviewofvocationaleducation:theWolfreport.Retrievedfromhttps://www.gov.uk/government/uploads/system/uploads/attachment_data/file/180504/DFE-00031-2011.pdf

York,B.N.,&Loeb,S.(2014).Onestepatatime:Theeffectsofanearlyliteracytextmessagingprogramforparentsofpreschoolers(No.w20659).NationalBureauofEconomicResearch.

31


IX. Appendices

AppendixA:DistributionofAttendance,Experiment1

AppendixB:distributionofnominatedStudySupporters,Experiment1Relationship Studysupporter1 Studysupporter2

N % N %

Nuclearfamily 422 41.33 286 28.01

Partner 56 5.48 0 0

Relative 50 4.90 66 6.46

Peer 413 40.45 401 39.28

Colleague 2 0.20 6 0.6

Professionalsupport 25 2.45 20 1.96

Missing 43 4.2 204 19.98

Total 1,021 100 1,021 100

Notes:Nuclearfamilyisdefinedastheparentsorbrothersandsistersofparticipants.Apartnerisdefinedasgirlfriends,boyfriends,husbandsandwives,etc.Peersareotherstudentsatthecollegeorfriendsoutsideofcollege,whileacolleagueissomeonewhomtheyworkwithinajobnotassociatedwiththecollege.Relativesincludegrandparents,aunts,uncles,stepfamilyandcousins,whileprofessionalsupportincludescollegetutors,socialworkers,andteachersfrompreviousschools.

32


AppendixC:DistributionofAttendance,Experiment2

AppendixD:distributionofsubjectsandqualifications,Experiment2

English Maths Total

FunctionalSkills 55 104 159

GCSE 79 217 296

Total 134 321 455

33


AppendixE:DistributionofrelationshipswithnominatedstudysupportersExperiment2Relationship Studysupporter1 Studysupporter2

N % N %

Nuclearfamily 192 44.6 153 41.6

Partner 8 1.9 5 1.4

Relative 28 6.5 33 9.0

Peer 182 42.2 160 43.5

Colleague 0 0 0 0

Professionalsupport 9 2.1 12 3.3

Other 12 2.8 5 38.3

Total 431 100 368 100

Missing 24 - 87 -

Notes:Nuclearfamilyisdefinedastheparentsorbrothersandsistersofparticipants.Apartnerisdefinedasgirlfriends,boyfriends,husbandsandwives,etc.Peersareotherstudentsatthecollegeorfriendsoutsideofcollege,whileacolleagueissomeonewhomtheyworkwithinajobnotassociatedwiththecollege.Relativesincludegrandparents,aunts,uncles,stepfamilyandcousins,whileprofessionalsupportincludescollegetutors,socialworkers,andteachersfrompreviousschools.Inthecasesof‘missing’,studentsleftthe‘relationshiptoSupporter’blank,but(1)inthecaseofSupporter1,didprovideuswithamobilephonenumberforthenominatedsupporter,and(2)inthecaseofSupporter2,onlyprovideddetailsforoneSupporter(andlefttheformforSupporter2blank).

34


AppendixF:exampletextmessages

Subject Course Textmessagecontent

General GCSEandFS

Hi[SSforename],[studentforename]returnedtotheir[English/Maths]classthisweek.Pleaseaskif[he/she]hasmadeaplanforwhen,where,andhow[he/she]planstostudygoingforward.Thanks,[College]

General GCSEandFS

Hi[SSforename],asareminder:youarereceivingthistextbecause[studentforename]pickedyoutobe[his/her]StudySupporter.Wearesendingyouweeklytextsabouthowyoucanhelp[him/her]succeedin[his/her][English/maths]class.Pleasehaveachatwith[him/her]aboutthesetopics.Thanksforsteppingupwhen[he/she]askedforyourhelp![College]

Maths GCSEandFS

Hi[SSforename],pleaseask[studentforename]tothinkofsomethingthatwaschallengingthisweekandwhatshecandiscussaboutitin[his/her]nextMathsclass.Thanks,[College]

Maths GCSE Hello[SSforename],pleaseask[studentforename]whether[he/she]hascompletedthetasksonwww.MyMaths.co.uk.Keepingupwiththehomeworkisoneofthekeyingredientsforsuccess!Thanks,[College]

Maths FS Hi[SSforename],pleaseaskif[studentforename]hasalreadypracticedusingacalculator.[He/she]isallowedtouseitduringtheupcomingmathsfunctionalskillsexams.Remind[him/her]tobringittoclass!Thanks,[College]

English GCSE Hi[SSforename],[studentforename]islearningaboutwarpoetrythisweek.Pleaseask[him/her]whypoemsabouttheFirstWorldWarcanteachusaboutwhat1915waslike.Thanks,[College]

English FS Hi[SSforename],pleaseaskwhat[studentforename]readovertheChristmasbreak.Whattypesofbooksdoes[he/she]enjoymost?Thanks,[College]

i two field experiments on social support and attendance...

Documents