learning analytics community exchange...

LearningAnalyticsCommunityExchangewww.laceproject.eu

“VisionsoftheFuture“,HorizonReportPublicDeliverable–D3.2

Deliverable Coordinator: DaiGriffiths

Coordinating Institutions: UniversityofBolton,TheOpenUniversity,UK

Datedue:31December2015(+fiveweeksextension)

Keywords:LACE,visions,future,2025,desirability,feasibility,action

Abstract:ThisdocumentreportsonaPolicyDelphistudy focusedonthestateoflearning analytics in 2025. The study centred on eight visions of thefuture,whichwereratedandcommentedonbyexpertsintermsoftheirdesirability,feasibilityandrequirementsforaction.

“VisionsOfTheFuture“,HorizonReport

1

HighlightsThisreportdescribestheVisionsof theFutureofLearningAnalyticsstudycarriedoutbytheLACEproject. Eight visions of the future of learning analyticswere developed by the LACE project, andthese were the basis of consultations with the learning analytics community. The principalinstrumentusedwas a SurveyMonkey survey, includingboth Likert scale and free text responses.The survey attracted 103 responses from invited experts and LACE contacts. Consultations withstakeholdershavealsobeencarriedoutinfacetofacemeetings.

Theresultswereanalysedinthreeways:

1. ChartsoftheLikertscalescores2. AnalysisoftheindividualvisionsinthelightoftheLikertscoresandthefreetextcomments

providedbyrespondents3. Thematicanalysisofthecorpusoffreetextcommentsfromallvisions.

Highlightsofthefindingsinclude:

1. ThereisalotofenthusiasmforLearningAnalytics,butconcernthatitspotentialwillnotbefulfilled

2. Policiesandinfrastructurearenecessarytostrengthentherightsofthedatasubject.Interoperabilityspecificationsandopeninfrastructuresareanenablingtechnology.

3. Learninganalyticsshouldnotimplyautomationofteachingandlearning.4. Issuesofsocialandpoliticalpower,ethics,andownershiparecentralfactorstothefutureof

learninganalytics5. Thereissomedisagreementbetweeneducationalsectors,raisingthepossibilityshouldbe

consideredthatasocio-technicaleliteisproposingsystemsandmethodsthatarenotentirelywelcomedbypractitionersinthefield.

6. Thenecessaryunderlyingtechnologyisalreadyavailable,andwillcontinuetodevelop.

ReadingguideThepurposeanddesignofthestudyisdescribedinSection1.

Thedetailedresultsarediscussedinsections2&3,andadditionalchartsareprovidedinappendix1.Theanalysishasbeenkeptasconciseaspossible,butinevitablythesesectionsincludeaconsiderableamountofdetail.Somereadersmaywishtoskimorskipthesesectionsonafirstreading,andmovestraighttoSection4,Conclusions.

TheConclusionssectionsummarisesthefindingsofsections2&3,contraststhem,andconsiderstheirimplications.Thesectionsmakesfrequentreferencetothedetailedanalysis,andinterestedreaderscanreferbacktospecificsectionsreferenced.


2

Contents1. Introduction...................................................................................................................................4

1.1. Theneedtounderstandwherecurrenttrendsareleading....................................................4

1.2. Methodology...........................................................................................................................4

1.3. Theworkcarriedout...............................................................................................................6

2. DiscussionoftheLikertscaleresults,relatedtofreetextresponses..........................................10

2.1. Vision1:In2025,classroomsmonitorthephysicalenvironmenttosupportlearningandteaching...........................................................................................................................................11

2.2. Vision2:In2025,personaldatatrackingsupportslearning.................................................13

2.3. Vision3:In2025,analyticsarerarelyusedineducation......................................................15

2.4. Vision4:In2025,individualscontroltheirowndata............................................................17

2.5. Vision5:In2025,opensystemsforlearninganalyticsarewidelyadopted..........................19

2.6. Vision6:In2025,learninganalyticssystemsareessentialtoolsofeducationalmanagement 21

2.7. Vision7:In2025,mostteachingisdelegatedtocomputers................................................23

2.8. Vision8:In2015,analyticssupportself-directedlearning...................................................25

3. Themesthatemergedfromthedata...........................................................................................27

3.1. Theme:Affect........................................................................................................................28

3.2. Theme:Alienation.................................................................................................................28

3.3. Theme:Complexity................................................................................................................30

3.4. Theme:Cost...........................................................................................................................31

3.5. Theme:Ethics........................................................................................................................32

3.6. Theme:Experience................................................................................................................32

3.7. Theme:Pedagogy..................................................................................................................34

3.8. Theme:Power.......................................................................................................................35

3.9. Theme:Privacy......................................................................................................................36

3.10. Theme:Regulation..............................................................................................................37

3.11. Theme:Standards...............................................................................................................38

3.12. Theme:Temporality............................................................................................................39

3.13. Theme:Validity...................................................................................................................40

4. Conclusions..................................................................................................................................42

4.1. Differingjudgementsondesirabilityandfeasibility..............................................................42

4.2. Judgementsontheeightvisions,andtheirimplications......................................................43

4.3. Therangeofthemesinformingthejudgementsonthevisions............................................46

4.4. Overallfindings......................................................................................................................47


3

5. FutureSteps.................................................................................................................................52

Referencelist.......................................................................................................................................53

Appendices..........................................................................................................................................54

Appendix1.GraphicalrepresentationoftheLikertscaleresults....................................................54

Appendix2:Codingsummarychart.................................................................................................60

Appendix3:VisionsoftheFuture....................................................................................................61

Appendix4:Informationforparticipants........................................................................................63

About...................................................................................................................................................65

ListoffiguresFigure1:Vision1desirabilityandfeasibility.......................................................................................11Figure2:Vision2desirabilityandfeasibility.......................................................................................13Figure3:Vision3desirabilityandfeasibility.......................................................................................15Figure4:Vision4desirabilityandfeasibility.......................................................................................17Figure5:Vision5desirabilityandfeasibility.......................................................................................19Figure6:Vision6desirabilityandfeasibility.......................................................................................21Figure7:Vision7desirabilityandfeasibility.......................................................................................23Figure8:Vision2desirabilityandfeasibility.......................................................................................25Figure9:Disparityinattitudestodesirabilityandfeasibility..............................................................42Figure10:Respondentsknowledgeoflearninganalytics...................................................................54Figure11:Respondentsbysector.......................................................................................................54Figure12:Alldesirabilitydata.............................................................................................................55Figure13:Allfeasibilitydata...............................................................................................................55Figure14:Desirabilitydatachartedbysector.....................................................................................56Figure15:Feasibilitydatachartedbysector.......................................................................................57Figure16:Desirabilitydatachartedbyrespondents(respondedtodirect/general)invitation..........58Figure17:Feasibilitydatachartedbyrespondents(respondedtodirect/general)invitation............59

ListoftablesTable1:Overviewofthevisions(seeAppendix1:VisionsoftheFutureforthetextinfull)................6Table2:Themesthatemergedfromthedata.....................................................................................27Table3:Summarytableoftheapplicationofcodes,withkeywords.................................................46Table4:Numberofapplicationsofcodesinthe8visions..................................................................60


4

1. Introduction

1.1. TheneedtounderstandwherecurrenttrendsareleadingTheLAK15conferenceinPoughkeepsieendedwithapanelthatconsideredthecurrentstateofthefield.Fourinternationalexperts–fromEurope,NorthAmericaandAustralasia–discussedthecurrentpositionoflearninganalyticsandfuturepossibilities.

SimonBuckinghamShumnotedthattheLAKcommunitymustmoveonfrombuildinganalyticsfortheschoolsanduniversityof2015andstarttodesignthefabricofanalyticsin2025.

Thefutureof learninganalyticsdependstoa largeextentonthepolicyadoptedby institutionsandgovernments.Itspracticewillbegreatlyshapedbytheregulatoryframeworkwhichisestablished,theinvestment decisions made, the infrastructure and specifications which are promoted, and theeducationaldiscourse.(BuckinghamShum2015)

Thisisnosmallchallenge,inpartbecausethetechnologywithwhichweworkischangingsofast.“Typically,wefindthatthedoublingtimefordifferentmeasures–price-performance,bandwidth,capacity–ofthecapabilityofinformationtechnologyisaboutoneyear”(Kurzweil2005,p.4)Communitiesarebecomingmoreconnected,pedagogiesarechanging,andeducatorsarelookingfornewwaystoengagestudents.Somealreadybelievethat‘existingsolutionsdon’taddressthemosturgentneedsineducation’(USDepartmentofEducationOfficeofEducationalTechnology2015,p.6).Thefastpaceofchangemeansthatif,inApril2006,wehadbegundevelopinglearninganalyticsfor2016,wemightnothaveplannedspecificallyforlearningwithandthroughsocialnetworks(TwitterwaslaunchedinJuly2006),withsmartphones(thefirstiPhonewasreleasedin2007),orlearningatscale(thetermMOOCwascoinedin2008).However,byconsultingwithexperts,wemighthavecomeprettyclosebytakingintoaccountexistingworkonnetworkedlearning(Goodyearetal.2004),mobilelearning(Sharples2000)andconnectivism(Siemens2005).

Itisimportantthatthelearninganalyticscommunitylookstothefuture,becausethefutureofthefieldwilldependtoalargeextent,asBuckinghamShumnoted,onpoliciesadoptedbyexternalbodies.Itspracticewillbemouldedbyregulatoryframeworksthatareestablishedexternally,theinvestmentdecisionsmadebyothers,theinfrastructureandspecificationsthatarepromotedacrosstheworld,andtheeducationaldiscoursethatisemployed.Bydevelopingaclearviewofwhatisdesirableandfeasibleinthefuture–andwhatweneedtoavoid–wecanequipourselvestomakepolicyrecommendations,toadvisefunders,andtotakealeadingroleinshapingtheframeworks,theinfrastructure,thespecificationsandthediscoursewithwhichweshallbeworking.

Tomeetthisneedunderstandwhatisdesirableandfeasible,theLACEprojecthasundertakenaPolicyDelphi.AsstatedintheDOW(p.12),thisworkseeksto“drawoutdifferencesofperceptionandvisionfromagroupofresearcherandpractitionerexpertsdrawnfromourliaisonorganisationsandfrompeoplewhohaveparticipatedinouractivities.Thiswillconsiderviewsonwhatisdesirable,whatisfeasibleandtheobstaclestomakingwhatisdesirablehappen.”

1.2. Methodology

1.2.1. Whatisa‘PolicyDelphi’TheoriginsoftheDelphimethodaredescribedbytheRandCorporation:


5

RANDdevelopedtheDelphimethodinthe1950s,originallytoforecasttheimpactoftechnologyonwarfare. The method entails a group of experts who anonymously reply to questionnaires andsubsequently receive feedback in the form of a statistical representation of the "group response,"after which the process repeats itself. The goal is to reduce the range of responses and arrive atsomethingclosertoexpertconsensus.(RANDCorporationn.d.).

Furtherdetailsareavailablein(Helmer-Hirschberg1967).TheunderlyingassumptionsoftheDelphimethodarethatgroupjudgementsaremorevalidthanindividualjudgements,andthataseriesofcoordinatedactivitiescanuncoveragroupjudgement.Thismethod,withincrementalchangesandadditions,isstillappliedtoachieveconsensus.However,amajoroffshootofthemethodwasinstigatedbyTuroffinthe1970s.AsTurofexplains:

Delphias itoriginallywas introducedandpracticedtendedtodealwith technical topicsandseekaconsensus among homogeneous groups of experts. The PolicyDelphi, on the other hand, seeks togeneratethestrongestpossibleopposingviewsonthepotentialresolutionsofamajorpolicy issue.(Turoff&Linstone2002,p.80)

Turoffgoesontoarguethatwhenconfrontedbyaquestionofpolicy,ratherthanofprediction,analysisandresearch“candonomorethansupplyafactualbasisforadvocacy.”Thefuturewilldependonpolicydecisions,andTuroffargues,“thedecisionmakerisnotinterestedinhavingagroupgeneratehisdecision;butrather,haveaninformedgrouppresentalltheoptionsandsupportingevidenceforhisconsideration”(Turoff&Linstone2002,p.80)

Thefutureoflearninganalyticsdependstoalargeextentonthepolicyadoptedbyinstitutionsandgovernments.Itspracticewillbegreatlyshapedbytheregulatoryframeworkwhichisestablished,theinvestmentdecisionsmade,theinfrastructureandspecificationswhicharepromoted,andtheeducationaldiscourse.Consequently,followingtheDOW,theLACEprojectisconductingapolicyDelphi,ratherthanaconsensusDelphi.

Consequently,theLACEPolicyDelphidoesnotseekconsensus,butrathertounderstanddiverseviewsofthepreferredfuture,withthemembersandassociatesoftheLACEprojectfulfillingtheroleofthe‘informedgroup’.

1.2.2. AimsandobjectivesAdaptingthedefinitionofaPolicyDelphiin(Turoff1970)wedefineourinterventionasfollows:

Aim:thesystematicsolicitationandcollationofinformedjudgmentsonlearninganalytics

Objectives:

• Toexploreorexposeunderlyingassumptionsorinformationleadingtodifferingjudgmentsonlearninganalytics

• Tocorrelateinformedjudgmentsonthetopicoflearninganalytics,whichspansawiderangeofdisciplines.

Futureaddressed:Thedateinthefuturewhichourvisionsrelatetois2025,whichcoincideswiththeEUtenderrecentlypassed:2025-2030.

1.2.3. ThedesignofthePolicyDelphiThedesignoftheLACEPolicyDelphiisasfollows.


6

1. ThefirstphaseofthePolicyDelphidrawsontheexpertiseofLACEconsortiummemberstodevelopvisionsoflearninganalyticsin2025intheformofshortscenarios.Followingamatrixanalysis,thesevisionsareselectedtoprovidegoodcoveragefor(a)relevancetostakeholders,and(b)theunderlyingthemesoftechnology,privacyandethics,andpedagogy.

2. Thesecondphaseinvolvesanonlinesurveyofdesignatedexperts,andvolunteerswhorespondedtothefocusedpublicitygeneratedbyLACE.TheexpertsaredrawnfromthethreefocusdomainsofapplicationofLACE(schools,highereducation,andtheworkplace)andthethreeprincipalcontributingdiscoursesoflearninganalytics(technology,privacyandethics,pedagogy).

3. Thethirdphasefocusesoninputfromstakeholders.Followinganalysisofearlierresults,thescenarioswiththeirdesirabilityandfeasibilityratingsaresharedwithstakeholders,whoaddedtheirresponses.Theresultsofthisphasefedintoananalysisofwhatisfeasibleanddesirable,andofwhatwouldneedtochangetomakeanyofthesevisionsareality.

4. Thefinalphaseisstrategicanalysisoffindings.Thisisdesignedtoclarifyanydisagreementsbetweenexpertsandthestakeholders,andtoidentifygapsbetweencurrentinfrastructureandpracticeandthosethatwillberequiredforthefuture.Itwillalsoidentifythedriverswhichareimpliedbytheresponsestothevisions.Thepresentreportpresentsthesefindings.

1.3. Theworkcarriedout

1.3.1. DevelopmentofthevisionsTheexpertisewithintheprojectwasmobilisedtoidentifyarangevisionswhichreflectedthewidevarietyofopinionsabouthowlearninganalyticswilldevelop.Followingamatrixanalysis,thesevisionswerecutdowninnumber,followingtwocriteria:(a)toidentifythemeswhichwouldberelevanttotherangeofexpertstakeholderswhowereourtargetpopulation,and(b)toincludeagoodcoverageoftheunderlyingareasoftechnology,privacyandethics,andpedagogywhichhaveemergedaskeythemesintheLACEproject.Thetitlesaregiveninthefollowingtable,andthefulltextsofthefinalvisionsareprovidedinAppendix1.

Table1:Overviewofthevisions(seeAppendix1:VisionsoftheFutureforthetextinfull)

No. Visiontitle1 In2025,classroomsmonitorthephysicalenvironmentto

supportlearningandteaching2 In2025,personaldatatrackingsupportslearning3 In2025,analyticsarerarelyusedineducation4 In2025,individualscontroltheirowndata5 In2025,opensystemsforlearninganalyticsarewidely

adopted6 In2025,learninganalyticssystemsareessentialtoolsof

educationalmanagement7 In2025,mostteachingisdelegatedtocomputers8 In2025,analyticssupportself-directedautonomous

learning


7

Thevisionswerealsosubstantiallyreducedinlength,tomakeiteasierforrespondentstoengagewiththeprocess.Thevisionswhichwereselectedwerecuttingedgebutalsonotfar-fetched.Theeightscenariosthatweredevelopedinthisprocessformedthebasisforallsubsequentphases.

1.3.2. DevelopmentofanonlinesurveyAnonlinesurveywasdesignedusingSurveyMonkey.Itwasconsideredthatitwouldbetoogreataburdentoaskrespondentstoreadandcommentonalleightvisions.Itwasthereforedecidedtopresentthevisionstotheexpertsinrandomorder.Aftereachvisiontherespondentwasaskediftheywerewillingtoansweranother.Itwasrequestedthateachrespondentshouldaddressatleastthreevisions.

Therespondentswereaskedtwoquestionsaboutthemselves:

1) Howwellinformedareyouaboutlearninganalyticsintermsof• Learningtechnology• Analyticsandmachinelearning• Ethicalissues• Pedagogyandprofessionalpractice

2) Whichsectorsofeducationhaveyouworkedin?Pleasetickallthatapply.• Schools• Workplace• HigherEducation

Respondentswerethenaskedtoassesseachvisionintermsofitsfeasibility,itsdesirability,andthestepswhichwouldberequiredinordertomakeitareality.Standardquestionswereaskedforeachvision:

Likertscale:Howdesirableisthisvision?Pleasegiveyouransweronascalefrom(1)to(4)

Likertscale:Howfeasibleisthisvision?Pleasegiveyouransweronascalefrom(1)to(4)

(Participantsweregiventheoptionofselecting‘Idonotfeelqualifiedtorespond’insteadofratingdesirabilityorfeasibility).

Freetext:Inthelightofthescoreyouhavegivenfordesirable----undesirable,whatactionsdoyouthinkshouldbetaken?Forexampletheremaybelegal,policy,technicalorotherdevelopmentswhichyouthinkareneededtomakethisvisionareality,ortopreventithappening.Pleasedescribetheinitiativeswhicharenecessary,andwhoshouldtakethem.

RespondentswerealsoinvitedtomakefreetextcommentsontheirLikertscaleresponses.

1.3.3. InformationprovidedtoparticipantsParticipantsweregivenafullexplanationofthepurposeofthestudyandhowtheirdatawouldbeusedbeforecommencingthestudy.ThistextisavailableinAppendix4:Informationforparticipants.Respondentswerealsoofferedtheopportunitytostatethattheydidnotwanttheirfreetextinputtobequoted.


8

1.3.4. PopulationThepopulationofrespondentswasdrawnfromtwogroups.FirstlyexpertsweredesignatedbytheLACEwork-packagesdealingwithSchools,WorkplaceandHigherEducation,payingattentiontothethreeprincipalcontributingdiscoursesoflearninganalyticsinLACE(technology,privacyandethics,pedagogy).TheLACEAssociatePartnerswereallincludedinthisexpertgroup.SecondlyvolunteersweresolicitedinpublicitygeneratedontheLACEwebsiteandtheLACEnewsletter.Atotalof193designatedexpertsweresentemailswithdirectlinkstothesurvey,whichenabledtheteamtokeeptrackoftheproportionofresponseswhichweremadebyinvitedexperts,andtocontrasttheirinput.

1.3.5. ResponseIntotal133peoplestartedtheprocessofansweringthequestionnaire.Ofthese103respondedtoatleastonevision.Thenumberofcompleteresponsestovisionswas487,anaverageof3.6perrespondent(comparedwiththerequested3.0perrespondent).

1.3.6. ConsultationwithstakeholdersDirectconsultationwithstakeholdershasalreadytakenplaceatanumberofevents.

• SolarFlare,OpenUniversityoftheUK,• InstitutionalReadinessDayforLearningAnalyticsTechnologies• OnderwijsdagemEducationDays,Rotterdam,Netherlands.• BETT,LondonUK

1.3.7. AnalysisThedatawasdownloadedfromSurveyMonkeyasCommaSeparatedValues,andanalysedusingaspreadsheet.Thoserespondentswhodidnotwanttobequotedwerehighlighted.

Theintentionoftheanalysiswasnottousethedatatodetermineanorderoflikelihoodforthe8visions,butrathertoidentifythedrivers,issuesandconcernswhichexpertsbelievewillconditionthefutureefficacyoflearninganalytics.Becauseofthisastatisticalapproachwasnotappropriate.

TheresultsoftheLikertdatawereusedtogeneratechartsindicatingthedegreeofdesirabilityandfeasibilityascribedtothevisions.Theseweresubdividedbythreesectors(School,Workplace,andHigherEducation),andbysourceofrespondent(invitedexpert,throughtheweblink),andtheresultingchartsareavailableinAppendix1.Theteamthenexaminedthechartstoidentifythebroadattitudesoftherespondentstowardsthevisions,andtohighlightanycontrastsbetweendifferentsubdivisionsofrespondents,orbetweentheresponsestodifferentvisions.

TheLikertchartsforeachvisionwerethencontrastedwithfreetextresponsestothatvision,inordertoachieveamorenuancedviewofrespondents’attitudes.

Inordertoachieveamoreglobalunderstandingoftherespondents’viewsoftheissuesdrivingthedevelopmentoflearninganalytics,thedataforalleightvisionswascumulated,andthefreetextentrieswerecodedaccordingtothirteenthemes.Thesethemesweredevelopedbyfourresearchersinaday-longworkingsession.Theyfirstcollaboratedinthecodingofthefreetextresponsesfor‘desirability’,‘feasibility’and‘necessaryactions’forasinglevision,thenreviewedtheresultingcodesandmergingthemwhereappropriate,andfinallyindividuallycodedtheremainingdatawhile


9

conferringonanydoubtswhichtheyencountered.Thecodingwasnon-exclusive,i.e.eachtextcouldbeassignedmorethanonecode.Thetextsassociatedwitheachcodewerethencompiled,anddistributedtotheresearchers,whowereindividuallyassignedthetaskofsummarisinganumberofthemes.TheresultsofthisprocessareavailableinSection3.


10

2. DiscussionoftheLikertscaleresults,relatedtofreetextresponsesInthissectionwebrieflydiscussthepersonalinformationprovidedbytherespondents.WethencontrasttheLikertscaleresultsforeachvision,andinterpretthemwithreferencetothefreetextresponsesprovidedforthatvision.

TheVisionsoftheFuturestudywasdirectedatexperts,andsoweshouldnotbesurprisedtofindthattherespondentsingeneralassessedthedegreetowhichtheyare‘wellinformedaboutlearninganalytics’asbeinghigh.Thereportedknowledgeof‘analyticsandmachinelearning’waslowerthanforanyothercategory,althougheveninthiscaseamajoritystillconsideredthemselves‘wellinformed’or‘verywellinformed’.Thissuggeststhatmanyofthoseworkingwithlearninganalyticsdonothaveastrongworkingknowledgeoftheunderlyingmethodswhicharebeingused.Learninganalyticsisafieldwhichbringstogethereducationalistsandtechnologists(Suthers&Verbert2013),anditistobeexpectedthateducationalistsfindsomeaspectsoftheworkofthetechnologiststobeopaque.

Thegreatmajorityofrespondentshadexperienceofhighereducation.Notethatthetotalofresponsesisgreaterthan100%becauserespondentswereabletoclaimexperienceofmorethanonesector.Thissuggeststhatmanyrespondentshavemovedtohighereducationfromschoolsortheworkplace(orvice-versa).Themuchhigherscoreforhighereducationreflectsthedominanceofthesectorinthedevelopmentoflearninganalyticssystems,andintheproductionofresearchonthedesignanduseoflearninganalyticssystems.


11

2.1. Vision1:In2025,classroomsmonitorthephysicalenvironmenttosupportlearningandteaching

2.1.1. Textofvision1In2025,classroomsmonitorthephysicalenvironmenttosupportlearningandteaching

In2015,learninganalyticsweremainlyusedtosupportonlinelearning.By2025,theycanbeusedtosupportmostteachingandlearningactivities,whereverthesetakeplace.Furniture,pens,writingpads–almostanytoolusedduringlearning–canbefittedwithsensors.Thesecanrecordmanysortsofinformation,includingtilt,forceandposition.Videocamerasusingfacialrecognitionareabletotrackindividualsastheylearn.Thesecamerasmonitormovements,andrecordexactlyhowlearnersworkwithandmanipulateobjects.Allthisinformationisusedtomonitorlearners’progress.Individualsaresupportedinlearningawiderangeofphysicalskills.Teachersarealertedtosignsofindividuallearner’sboredom,confusion,anddeviationfromtask.Teachersandmanagersareabletomonitorsocialinteractions,andtoidentifywheretheyshouldnurturesocialisationandcooperativebehaviour.

2.1.2. Likertscaleresults

Figure1:Vision1desirabilityandfeasibility

TheLikertscaleresultsforthisvisionshowthatamajorityofrespondentsfoundittobeundesirable,andthatovertwiceasmanyfoundittobe‘veryundesirable’thanfoundit‘verydesirable’.Asregardsfeasibility,however,amajorityofrespondentsdescribedthevisionasbeingfeasibleorveryfeasible.Thissuggestsaconcernthattheavailabilityoftechnologymayleadtheeducationsystemintomakingchangeswhicharenotpedagogicallyappropriate.

Thisvisionisclearlyaimedattheschoolclassroom,andtheresultsshowthatrespondentsfromschoolsweresubstantiallymorepositiveintheirassessmentofthevisionthanwerethosefromhighereducationortheworkplace.


12

Respondentstotheweblinkwereonbalanceagainstthevision,andalmosttwiceasmanyrespondentstotheweblinkfoundthevisiontobe‘veryundesirable’thandidinvitedexperts,raisingthepossibilitythattheremaybeagapinunderstandingbetweenexpertsandpractitionersontheground.

2.1.3. FreetextresponsesFreetextcommentsonthedesirabilityofthisvisionfocusedontwoareas:theefficacyofthepedagogicintervention,andthesocialandethicalconsequences.23commentswerenegative,11ambivalent,andonly9favourable,suggestingthatitwasmainlythosewhofoundthevision‘veryundesirable’thatweremotivatedtocomment.Thestrengthoftheircommentsalsosuggeststhattheymightbewillingtoactivelyresisttheimplementationofsuchsystems.Negativecommentsincluded“veryintrusive”,“justridiculous”,“BigBrotherscenario”,“lossofhumanity”,“TherealfuelofLearningismotivationandvolition,whichyoucannotcapturewithexternalsensors”,etc.

Somerespondentsbelievedthatthestateoftheartinpervasivecomputingshowthatthisvisionisfeasible,andthatdevelopmentsrelatedtotheInternetofThingsandQuantifiedSelfmovementswilldrivepricesdownsoby2025makingthisvisionverylikelytooccur.Otherrespondentswereoftheopinionthatalthoughthetechnologyforgatheringthenecessarydatacouldbeavailableby2025,thereisdoubtastowhetherdataprocessingalgorithmswhichcanproducevalidandreliableconclusionsrelevanttoeducationwillbedeliveredby2015.Concernswereraisedaboutthecosttoinstitutionsandorganisationsofequippingclassroomswiththenecessarytechnology,andthecapacityandcapabilityofteacherstodigestandreacttothedata,meaningthisapproachwouldbedifficulttoimplementatscale.Questionsrelatingtopeoples’attitudestobeingcloselymonitoredwereraised,e.g.howwouldthisapproachworkifsomelearners(orparents)orteacherswereabletorefusetohavetheirdatacollected?Moregenerally,itmaybehumanfactors,rulesandregulationsthatcouldaffectthetimescaleandeaseofimplementationofthisvision.

Asregardstheactionsthatneedtobetaken,respondentsgenerallyindicatedthattherequiredunderlyingtechnologywaseitheralreadyavailableorwouldsoonbecomeso.Requiredactionsrelatedtothetwoprincipalconcernsexpressedindesirability.Firstlymanycommentsindicatedthatresearchisneededtodemonstrateifandhowthisapproachtolearningandteachingiseffective.Secondly,manycommentsrequestedprotocols,regulationsorlegislationtogovernethicsandprivacy.

2.1.4. KeywordsCommercialpressure,efficacy,ethicsprivacy,funding,pedagogic,policy,reliability,scalability,validity.


13

2.2. Vision2:In2025,personaldatatrackingsupportslearning

2.2.1. Textofvision2In2025,personaldatatrackingsupportslearning

In2015,peoplewerebeginningtoweardevicessuchasheart-ratemonitorsandrun-trackersastheywentabouttheirdailylives.By2025,sophisticatedsensorscangatherpersonalinformationaboutfactorssuchasposture,attention,rest,stress,bloodsugar,andmetabolicrate.Peoplecollectthisinformationabouttheiractivities,andfeeditintoprogrammesoftheirchoicethatproviderecommendationsonhowtoactinwaysthatimprovetheirlearning.Learnerscandownloadthestatisticsanddatathatareassociatedwithsuccessfullearninginacertainarea.Aligningpersonaldatawiththese‘ideal’setsisclaimedtohelppeopletomasterskillsasdiverseasswimming,driving,carryingoutsurgeryandpassingexaminations.Academicstarssellprogrammesusingthisdatatooptimiselearningfordifferentagesandcourses.Businessgurusmarketsimilarprogrammesfortopicssuchaspresentationskillsandworkloadmanagement.Somelearnerscreateandsharetheirowndataanalysisprogrammes,whichproviderecommendationsthatoftenincludetheconsumptionofhigh-energyfoodsandstimulants.Themajorityofhigh-schoolanduniversitystudentsfollowself-monitoringprogrammes,anddiscussthemeritsoftheseonsocialmedia.



ThepatternofresultsforVision2issimilartothatforVision1:thevisionisconsideredtobefeasible,butrespondentsareambivalentaboutitsdesirability,withaslightmajorityfindingittobeundesirable.Theresponsesfromthethreesectorsarebroadlysimilar,asaretheresponsesfrominvitedexpertsandweblinkrespondents.

2.2.3. FreetextresponsesThefreetextresponsesalsoechoedthecommentsmadeonVision1,withafocusontheefficacyofthepedagogicintervention,andthesocialandethicalconsequences.Thenumberoffreetextresponseswasslightlylower,with5positivecomments,11ambivalent,and18negative.Giventhat


14

privacyisamajorconcerninbothvisions,itiscoherenttosuggestthatdatagatheringbytheindividualwasseenasmoreacceptablethandatagatheringbytheinstitution,andthatthismadetheresponsemilder.Indeedonecommentexplicitlystated“Iamneutralonthis,butwarmtoitmuchmorethan#1becauseusersappeartohavecontroloverthemonitoringanduseofthedata”.However,oneusercommentedthat“thevisionfailstomentionwhoownsthedata”,whileothersmentionedthepotentialmisuseofdata.Otherswereconcernedatthe“lossoffreewill”,andthat“soundslikewearerobotscontrolledbyouralgorithmicoverlords.

Thepositivecommentsondesirabilitymostlymentionedthepedagogicbenefitswhichcouldbegained,withmention,forexample,ofachievingpersonalisedlearning,and“improvingthesuccessandimpactoflearning”.Thesewerebalancedbyalargernumberofnegativecomments,forexamplereferringto“snakeoil”,andtheundesirabilityofsubstitutingphysicalforconativeindicators.Therewerealargernumberofcommentswhichtookanuancedviewofdesirability,whencomparedwithVision1.Theserecognisedthepotentialofmonitoringpersonalperformance,butraisedconcernsabouttheinterpretationofthedata,oritsethicaluse.

Somerespondentssaidthatpersonaldatatrackinginitiativesarecurrentlyrunning,butonasinglemeasure,andnotcombinedwithothermeasuresintosuchabigvisionaryconcept,andthattheavailabilityofproductsusingoneormoremeasuresislikelyby2025.Othersfeltthatalthoughtechnologycapableofacquiringthedatadescribedislikelytobeavailablein2025,doubtsaboutbeingabletousethisdataoptimiselearningwereraised.Therewereconcernsthatpersonaldatatrackingasdescribedinthevisionwouldhavelimitedeffect,becauseitignoresconstraintsonlearningwhichmayhavemoreimpact:pressuresofwork,family,sociallife;lackofintrinsicmotivation;makingcurriculasufficientlyrelevanttopeople'slives,andotherfactorswhichdrawindividuals’attentionawayfromtheirlearning.Otherconcernsincludedthedevelopmentandsuccessfulmarketingofproductsakintothoseinthevision,butbasedonpoorresearchandspuriousstatistics,resultinginnegativefeedbackwhenmarketingclaimsarenotmatchedbyrealchangesinperformance.

TheactionsrequiredtomakethevisionarealitywerealsosimilartothoseforVision1.Nomentionwasmadeofaneedtodeveloptheunderlyingtechnology.Researchintothepedagogicpotentialofthesemethodswasrequestedinordertocreateastrongerevidencebase.Theneedtoregulatedatamanagement,andprivacypolicies,werealsomentioned.

2.2.4. KeywordsEthics,learningoutcomes,motivation,pedagogy,personalisation,privacy,profit,reliability.


15

2.3. Vision3:In2025,analyticsarerarelyusedineducation

2.3.1. Textofvision3In2025,analyticsarerarelyusedineducation

In2015,manypeoplehopedthatanalyticswouldbeabletoimproveteachingandlearningandtheenvironmentswherethesetakeplace.However,in2025,itisclearthattherearemanyproblems.Coursesthatareautomatedbyanalyticsareseenasinferior,andlearnershaverealisedthattheycangamethesystem.Therehavebeenmajorleaksofsensitivepersonaldata,anditisclearthat,evenwherethishasnothappened,manycompanieshavemisusedthedatageneratedbytheiranalytics.Manygovernmentshaveruledthatindividualsarethesoleownersofthedatatheygenerate.Alluseofdataforeducationalpurposesnowhastobeapprovednotonlybythelearnerbutalsobynewinspectorates.Inpracticethishasmeantthatuseofanalyticsisrestrictedtosummativeassessmentcarriedoutbygovernmentagencies.Aconsensushasemergedineducationalpolicy:themoveawayfromlearninganalyticsisnotonlyethicallydesirableitisalsoeducationallyeffective.



TheLikertscaleresultsshowthatalargemajorityofrespondentsviewedtheideathatlearninganalyticswouldberarelyusedin2025asextremelyundesirable,withonly1personviewingitasdesirable(andinthiscasetheratingwasqualifiedbyafreetextcommentstatingthattheratingassumedthat‘learninganalyticshadbecomecorruptoruntrustworthy’).Ontheotherhandthishighlyundesirableoutcomewasseenbymanyasbeingfeasible,withaslightmajorityviewingitasbeingfeasibleorveryfeasible.

2.3.3. FreetextresponsesIntheirfreetextresponsesondesirability,respondentsmadecleartheirsupportforanalyticsmakingcommentssuchas“Educatorshavealwayscollectedandused‘data’”,“Don'tthrowawaytheentirefieldbecauseofsomemistakesanderrors!”and“Ithinkitismadnesstohavelotsofdata


16

andnotuseit”.Somerespondentstooktheopportunitytoexplainthattheirsupportforlearninganalyticswasconditional,withstatementssuchas“Myidealscenarioiswherehumanteachersuselearninganalyticsethicallyandprofessionallytoenhancelearningdesignandunderstandlearning”,and“Thecasesofmisuseofdataarepossible,buttherisksareminimalcomparedtothepotentialofbetterstudentpersonalization.”

Somethoughtthisvisionisnotfeasiblebecauselearninganalyticsarealreadyinfluencingandimprovingeducationpracticeandpolicyinapositiveway.Also,inseveralotherfieldsanalyticsisusedsuccessfully(e.g.professionalsports),andlessonscanbelearnedfromhowithasbecomeimplementedandembeddedintopractice.Othersthoughtitunlikelyasinstitutionshavecollectedandmanagedsensitivedataabouttheirlearnersformanyyearswithoutdisastersoccurring,andtherearenosignsthattheriskofleakshasrisen,orthattherehasbeenagreatincreaseinthesensitivityofthedata.Somerespondentsopinedthatalthoughthevisionmayoccurinsomecontextsinwhichlearninganalyticsisimplemented‘badly’orwhereethicalorlegalconcernsimpingeonlearninganalyticsbeingused,itisunlikelytohappenglobally.Insomecasesthereactionofthepopularmediamaybeveryinfluential.

Theactionsrecommendedbyrespondentsintheirfreetextcommentsfocusedontwoprincipalareas.Firstly,policyfordataprivacyandappropriateuse,withmanyrequestsformoreeffectiveregulation(althoughonerespondentarguedthatinafreemarketnoactionwasrequired).Fourcommentsmadethecogentpointthattobeeffective,policieshavetobeimplementedinthesystemsandtoolsusedinlearninganalytics,andthatthisimpliestheneedforstandardswithteeth.Giventhelownumberofrespondentswithdeeptechnicalknowledgeoflearninganalytics,itmaybearguedthatmoreattentionshouldbeplacedonthispointthantherelativelysmallnumberofcommentssuggests.Secondly,respondentsassertedtheneedtogenerateanddemonstratepedagogicbenefits.Somementionedtheneedtofocusonstudentcentrededucation,andthepresenceofhumansintheanalyticsprocess.

2.3.4. KeywordsCommercialinterests,ethics,governmentcontrolofdata,infrastructure,pedagogicbenefits,policy,privacy,regulation,usercontrol.


17

2.4. Vision4:In2025,individualscontroltheirowndata

2.4.1. Textofvision4In2025,individualscontroltheirowndata

In2015,itwasnotclearwhoownededucationaldata,anditwasoftenusedwithoutlearners’knowledge.By2025,mostpeopleareawareoftheimportanceandvalueoftheirdata.Learnerscontrolthetypeandquantityofpersonaldatathattheyshare,andwithwhomtheyshareit.Thisincludesinformationaboutprogress,attendanceandexamresults,aswellasdatacollectedbycamerasandsensors.Learnerscanchoosetolimitthetimeforwhichaccessisallowed,ortheycanrestrictaccesstospecificorganisationsandindividuals.Thetoolsformakingthesechoicesareclearlylaidoutandeasytouse.Inthecaseofchildren,datadecisionsaremadeinconsultationwithparentsorcarers.Iftheydonotengagewiththesetools,thennodataissharedandnobenefitsgained.Mosteducationalinstitutionsrecognisethisasapotentialproblem,andruncampaignstoraiseawarenessoftheboththerisksofthoughtlessexposureofdata,andthebenefitstolearnersofinformedsharingofselectededucationaldata.



TheLikertscaleresultsforVision4showverystrongsupportfortheideathatlearnersshouldbeabletocontroltheirowndata,withamajorityofrespondentsratingthisas‘verydesirable’.Therespondentsalsoindicatedthatthisvisionwasfeasible,althoughtheresultwaslessclearthanfor‘desirability’(nearly50%ratedthisasa‘3’,ratherthana‘4:veryfeasible’).Theresultsforallthreesectorsweresimilar,althoughschoolsrespondentswerealittlemoreoptimisticaboutthefeasibility,whileworkplacerespondentswerealittleless.Theresultsfortheinvitedexpertsandthosewhoaccessedtheweblinkweresimilar.

2.4.3. FreetextresponsesThefreetextresponsesondesirabilityreflectthestrongLikertscalescoresinfavourofthisvision.Theyalmostallstresstheimportanceofindividualshavinggreatercontroloverthedatathatthey


18

generate.Thereareafewdissentingvoices.Threerespondentswereconcernedthatthevisionwouldpreventlearninganalyticsfromfulfillingitspotential,asonesaidit“GreatlylimitsourabilitytoeffectivelyuselearninganalyticstoimprovelearningforALLstudents”.InthistheyareechoingthepositiontakenbythecodesofpracticeforlearninganalyticspublishedbyJisc(Sclater2014)andOUUK(TheOpenUniversity2014),whichusethisargumentinfavouroftheinstitutionmakinguseofalleducationaldata,withoutexplicitpermissionfromstudents.

Althoughmanytypesofdatawillberoutinelycaptured,feasibilityofthisisbasedonovercomingsignificantsocialandpoliticalhurdles.Althoughitisfeasibletechnicallytherewillbemanypartieswithconflictingintereststhatwilllobbyagainstfullusercontrolofdata,e.g.commercialandgovernmentinterests.Changesinpolicyandlawwillbenecessary,anditwillbeimpossibletoachieveharmonisedlegislationduetotheeffectofdifferentlegislativeregions.Furthermore,somethoughtitunlikelythatgovernmentsororganisationswouldrelinquishtheircontroloverlearnerdataandmaintainsomerightstodataaspartoftheconditionsofusingtheirservices.Somethoughtthateducationandawarenessiskeytothisvision,inthattheuserpopulationneedabetterunderstandingofthevalueofpersonaldatafortheirownbenefitaswellasthatoforganisations,andtobeactivelyengagedwithprivacyanddatacontrolissues.

Intermsoftheactionswhicharenecessary,mostcommentsaffirmedtheneedforrulesandpoliciesaboutthemanagementofpersonaldata.Anumberofcommentsexpressedtheconcernthatcommercialpressurewouldmakethevisionimpossible,forexample“Idoubtthough,thatitissofeasible.Therewillalwaysbepeople/companies/etc.tryingtomakeuseofotherpeople'sdataforcommercialreasons”.Afewcommentsassertedtheneedfordatasecuritystandardsandprivacystandards,andthetoolswhichcouldmakethisareality.Oneoftherespondentswhowasconcernedaboutthisvisionpreventinglearninganalyticsfromachievingitspotentialsuggestedthatinstitutionsshould“MakepoliciesaboutUSEnotcollection.Withclear,consistent&transparentpoliciesaboutuse,peoplewillnotbeasconcernedaboutcollection.”

2.4.4. KeywordsCommercialpressures,governmentcontrol,legislation,pedagogicbenefits,privacy,regulation,usercontrol.


19

2.5. Vision5:In2025,opensystemsforlearninganalyticsarewidelyadopted

2.5.1. Textofvision5In2025,opensystemsforlearninganalyticsarewidelyadopted

In2015,companiesproducedarangeoflearninganalyticstools,usingdifferentapproachesandstandards.Thealgorithmsandmodelsthatcompaniesuseareoftenprotectedasintellectualproperty.By2025,the‘openlearninganalytics’establishedbytheOpenLearningAnalyticsFoundationhasmadeamorejoined-upapproachpossible.EducationalorganisationsseelearninganalyticsasacentralelementoftheirITprovision.Theydemandcontroloverthesetools,howtheyrunandwhattheyareusedfor.Thetoolstheyselect,althoughtheycomefromdifferentproviders,useopenalgorithmsandsharedataaccordingtoanagreedsetofstandardsthatfacilitatetransparencyandindependentvalidation.Asetofwell-tested,accessibleandstandardisedvisualisationmethodsiscommonlyused,sothatlearnersandteacherscanconfidentlyusearangeoftools.Institutionscaneasilyworkwitharangeofproviderstodesignlearninganalyticssystemsthatsupporttheirstrategicvision.



TheresultsforVision5areverysimilartothoseforVision4.Indeed,whiledataprivacy(Vision4)andopensystems(Vision5)arequiteseparateissues,thereisastrongconnectionthroughtheperceivedabilityofopensystemstoensuretransparencyandaccountabilityindatamanagement.Ofallvisions,thiswastheonewiththehighestproportionofrespondentsratingthevisionas‘highlydesirable’,withnorespondentsratingitas‘highlyundesirable’.

RespondentsweremuchlessconfidentthatVision5wasfeasible,withequalnumbersofrespondentsratingthevisionas‘feasible’and‘notfeasible’,althoughsubstantiallymorerespondentsratedit‘veryfeasible(4)’thantheydid‘notatallfeasible(1)’.Therewasthereforeaconsensusthatthisvisioncouldbeachieved.


20

Theresultsfordesirabilitywerealmostidenticalforthethreesectors.Therewas,however,abigdifferencebetweensectorsintheresultsforfeasibility.ForboththeSchoolsectorandtheWorkplacesectorthelargestnumberofrespondentsindicatedthatthevisionwasratherinfeasible(2).InHigherEducation,ontheotherhandthelargestnumberofrespondentsthoughtitfeasible(3),withalargenumberassertingthatitwasveryfeasible(4).ThefactthatthereweremanymorerespondentsfromHigherEducationthanothersectorsmeansthatthisviewprevailsintheoverallresults.Butitwouldbewisetotakenoteofthegreaterbarriersthatareperceivedtoopenlearninganalyticsinschools.

2.5.3. FreetextresponsesThegreatmajorityoffreetextcommentsondesirabilityreaffirmedtheimportanceofopensystems,andtheiressentialroleinlearninganalytics.Therewasfrequentmentionoftheneedforinstitutionstobeabletocontroltheirownsystems,andofthesupportthiscouldofferfortransparencyondatause.Anumbermentionedthatitisimportanttorealisethatopensystemsareentirelycompatiblewithcommercialservicesandproducts.Therewereafewdissentingvoices.Onerespondentassertedthatthis‘restrictsinnovationinthelongrun’,andintheirresponseson‘feasibility’and‘action’mentionedIMSspecificationsasthebestrouteforward.

Asregardsfeasibility,anumberofcommentsindicatedthatvendorshaveaninteresttolock-incustomersandwillworkagainstthevision.Sharingdataforlearninganalyticsrequiresextremelycarefulattentiontoprivacy,problemswhichavendor-specificsolutionscanavoidcompletelybykeepingdatacarefullyprotectedandonlysharingprivacy-neutralanalytics.Also,marketpressureswillpullprovidersindifferentdirections,andalthoughmethodsandprotocolsforsharingareuseful,thestateoftheartwillcontinuetochangemakingthistaskextremelydifficult.Fundingstreams,organisationalandgovernmentpolicychangeswillberequiredtosupportdevelopmentoftheseopentools;althoughitisentirelyfeasibleinatechnicalsensewidespreadimplementationatscalewillstilldependoncost,infrastructure,culturalchangeanddevelopmentofappropriatedatapolicies.

Theactionsrecommendedbyrespondentsintheircommentsincludedpracticalsuggestionsfordevelopmentofcommondatamodelsandstandards,andappropriatepolicies.Somespecificsuggestionswereoffered,forexamplefollowingtheapproachesestablishedbytheApacheFoundation1andbytheInternationalStandardsOrganisationinitstechnicalcommitteeSC36ISO/IECJTC1/SC36Informationtechnologyforlearning,educationandtraining2.Aneedwasalsoidentifiedtodemonstrateopensystemsinaction.

2.5.4. KeywordsCommercialpressures,controlofsystems,funding,marketplace,usercontrolofdata.

1http://www.apache.org/foundation/2http://www.iso.org/iso/iso_technical_committee%3Fcommid%3D45392


21

2.6. Vision6:In2025,learninganalyticssystemsareessentialtoolsofeducationalmanagement

2.6.1. Textofvision6In2025,learninganalyticssystemsareessentialtoolsofeducationalmanagement

In2015,companieswerebeginningtodevelopsystemstorecommendresourcesandtopredictoutcomes.By2025,thesesystemsarehighlydeveloped.Awiderangeofdataaboutlearnerbehaviourisusedtogenerategoodquality,real-timepredictionsaboutlikelysuccess.Learners,teachers,managersandpolicymakersallhaveaccesstoliveandaccurateinformationabouthowwellalearnerislikelytodo.Learnersandteachersplantheirworkonthebasisofreliabletoolsthatcanproducedetailedandpersonalisedrecommendationsaboutwhatshouldbedonetoachievethebestlearningoutcomes.Agrowingindustryoffersservicestoinstitutionsandindividuals,advisingonhowtorespondtopredictionsgeneratedbyanalytics,andhowtotakeappropriateactioninthelightofrecommendations.Accuratepredictiveinformationenablesmanagersandpolicymakerstoexpandorcontractlearningprovisionbeforesuccessorfailureisevident:youdon’thavetowaittoseeifacourseisboomingorfailing,withfundingchangeshappeningquickly.



Respondentsfoundthistobeadesirablevision,withthemostpopularchoicebeing‘desirable’(3),followedby‘verydesirable’(4).Verysimilarscoresweregivenforfeasibility.

Thevisionwasseenasmarkedlylessdesirablebyrespondentsfromtheschoolssector,withmorerespondentsfindingit‘notdesirable’(1),than‘verydesirable’(4).Similarlyrespondentsfromtheschoolssectorfoundthevisiontobelessfeasiblethandidothersectors.

Respondentswhoaccessedthesurveythroughtheweblinkfoundthevisiontobesubstantiallymorefeasiblethandidtheinvitedexperts.


22

2.6.3. FreetextresponsesThecommentsondesirabilityweremostlysupportiveofthisvision(21comments).Thismightwellbeexpected,as,manywouldpeoplewouldfindthisvisiontobe,inthewordsofonerespondent“thegoalofusinglearninganalytics”.Ontheotherhandasubstantialnumberofrespondentsexpressedambivalence(11responses)becauseofspecificconcernswiththevision,while8expresseddeepconcernwiththepedagogicimplications.

Asregardsfeasibility,somerespondentsthoughtthatthisvisionwasachievablebutnotwithinthe10yeartimescale,duetolackofdatanecessarytoproducereliablepredictors(e.g.scarcityorunavailabilityofhealth,family,motivation,volitiondata).Othersthatitwillnotbedifficulttomeasureandactonmetrics,andthismayhappeninsomecontexts,butsomelearnersmaynotbenefitbecausepredictionswillnotbeuniversallyreliable,orpredictionsmaymeanthatinvestmentinsomegroupsiswithdrawnornotoffered.Somethoughtthatbecauseofthecomplexityofestablishingcausality,thatbusinessandtechnicalcustomisationandflexibilitywillbeimportanttoenablesuccessofthisvision.Itwillalsodependonculturalchangeanddevelopmentofeffectivepoliciesonethics,securityandprivacy,andappropriatebusinessmodelstomeetcostsofimplementation.

Therequiredactionsrecommendedbyrespondentscentredonfourthemes.Firstly,ensuringthatlearninganalyticswasinformedbypedagogy.Secondly,thenecessityforfurtherresearchbecause,asonerespondentputit,“wearestillinthe‘proofofconcept’phase.Thirdly,respondentsexpressedtheneedforinternationalandnationalcollaborationbetweeninstitutions.Fourthly,respondentscommentedontheneedforappropriatepolicies,and,inonecase,forrelieffromregulationwhenexperimenting.

2.6.4. KeywordsPedagogiceffectiveness,policy,prediction,reliability,research,socialimplications,technicaleffectiveness.


23

2.7. Vision7:In2025,mostteachingisdelegatedtocomputers

2.7.1. Textofvision7In2025,mostteachingisdelegatedtocomputers

In2015,peoplewerebeginningtoassembledatasetsthatcouldrepresentlearner’sactivities.By2025,theseareusedonalargescaleinteaching,andthishasledtothedevelopmentofenormousdatasetscontaininginformationabouthundredsofthousandsoflearners.Analysingindetailtheprogressofsuchawidevarietyoflearnershasmadeitpossibletoprovidereliableevidence-basedrecommendationsaboutthemostsuccessfulroutestolearning,aswellasidentifyingthelearningmaterialsandapproachesthataremostsuitableforeachindividualateachpointintheirprogress.Theserecommendationsarebetterinformedandmorereliablethanthosethatcanbeproducedbyeventhebest-trainedhumans.Learnersnowspendmostoftheirtimeworkingwithanalytics-drivensystems,andtheroleofteachershasbeenreduced.Theevidencegeneratedbytheuseofthesesystemsdriveseducationpolicy.



Thisvisionwasratedthesecondleastdesirableofallthevisions,aftervision3.Indeed,asvision3wasthenegativevisionthat“analyticsarerarelyusedineducation”,thisvisionconstitutestheleastpopularofallthewaysofusinganalyticsthatwerepresentedtorespondents.Vision7wasconsideredslightlymorefeasiblethanitwasdesirable,butwithamajorityratingiteither‘notfeasible’(4),or‘ratherunfeasible’(3).

Theresultswerelargelysimilarforallthreesectors,butrespondentsfromtheworkplaceweremarkedlystrongerintheirassessmentthatthevisionwasnotfeasible.

Therewaslittledifferencebetweentheinvitedexpertsandrespondentsreachedthroughtheweblink,althoughthelatterfoundthescenariotobealittlelessdesirable.


24

2.7.3. FreetextresponsesThemajorityofcommentsondesirabilitystressedtheviewthat,asonerespondentputit,“humansarelearningbestwhentaughtbyhumans”.Someaddedargumentsastowhythiswasthecase,suchasthepaucityofinformationprovidedbybrowsingpatterns,inabilitytoworkwithmeaninginanautomatedway,theneedforhumancollaborationandinspiration,andreductioninthecreativityagencyofthelearner.Afewdissentingvoicessuggestedthattherewerepositiveaspectstothevision.

Asregardsfeasibility,oneviewwasthatthisisnotfeasibleinsomecontexts(e.g.workplacelearning)becausetheproportionofeLearningthatgoesoninthosecontextsisnothighenoughnow,andisnotincreasing,forcomputerdrivenlearningtohavemuchimpact.Anotherviewwasthatwhilstitistechnicallyfeasibleinsomeeducationalcontextsanddisciplines,itwillnotbeinothersasitignoreslearningthroughinteractionswithbothpeersandthewidersocialenvironment,andwhereattitude,skillsandcompetenceslikecreativityarerequiredtoproduceindividualandnovelsolutions.Somewerescepticalthatremovalofhumansfromeducationalcontextswasachievable,e.g.thatahumanteachercouldbebettersupportedbyanalytics(butnotreplaced).Marketing(bycompaniessellingsolutions)andconsiderationofcost-cutting(reductionsinwagebill)couldinfluencehowquicklyanddeeplythisvisionispursued.

Inlinewithcommentsondesirabilityandfeasibility,thegreatmajorityofcommentsonnecessaryactionsemphasisedtheneedtoensuretheparticipationofhumansinteaching,andtounderstand,asonerespondentputit,that“learningtakesplaceinasocio-technicalsystem”.Somerespondentsmentionedtheneedforpolicyandlegislativechecksonintrusiveanalyticssystems.Adissentingcommentsuggestedthatbettercommunicationofthebenefitswouldbevaluable,whileanothersaidthatitwasimportantto“ensurethattheunionsarenotkillingtheseinitiativesbecauseofdataprivacyworries”.

2.7.4. KeywordsCommercialinterests,effectiveness,humanvalues,pedagogy.


25

2.8. Vision8:In2015,analyticssupportself-directedlearning

2.8.1. Textofvision8In2025,analyticssupportself-directedautonomouslearning

In2015,learnersineducationalinstitutionsandinbusinesseshadtofollowacurriculumdevelopedbyothers.In2025,theycreategroupsthatworktogethertodecidetheirlearninggoalsandhowtoachievethese.A‘LearningTrajectorySystem’usesanalyticstosupportinformationexchangeandgroupcollaborations,andlearnersreceivesupportfrommentors,ratherthanteachers.Activitytowardsalearninggoalismonitored,andanalyticsprovideindividualswithfeedbackontheirlearningprocess.Thisincludessuggestions,includingpeerlearnerstocontact,expertstoapproach,relevantcontent,andwaysofdevelopinganddemonstratingnewskills.Formativeassessmentisusedtoguidefutureprogress,takingintoaccountindividuals’characteristics,experienceandcontext,replacingexamsthatshowonlywhatstudentshaveachieved.Textsandotherlearningmaterialsareadaptedtosuittheculturalcharacteristicsoflearners,revealedbyanalysisoftheirinteractions.Asaresult,learnersarepersonallyengagedwiththeirtopics,andaremotivatedbytheirhighlyautonomouslearning.Thecompetencesthattheydeveloparevaluableinasocietyinwhichcollectionandanalysisofdataarethenorm.Thereisalsoconvergencebetweenthelearningactivitiesoftheeducationsystemandthemethodsusedbyemployeestodeveloptheirknowledgeandskills.



Thisvisionwasratedthethirdmostpopularintermsofdesirability.Indeed,thetwovisionsratedhigher(4&5)areconcernedwithopeninfrastructureandcontrolofdata,andsovision8isthemosthighlyratedvisionwhichfocusesonthepedagogicapplicationoflearninganalytics.Halfoftherespondentsratedthisvisionas‘verydesirable’(4),withoveraquarterofrespondentsratingit‘desirable’(3).

Respondentsweremorecautiousregardingfeasibility,with‘feasible’(3)themostpopularresponse.


26

Thedifferencesbetweenresponsesfromthethreesectorswassmall,butitisworthnotingthat‘schools’and‘workplace’respondentswereasomewhatlessenthusiasticthan‘highereducation’respondents.Similarlytheinvitedexpertsweremorepositiveaboutthevisionthanrespondentsfromtheweblink,inbothdesirabilityandfeasibility.

2.8.3. FreetextresponsesThecommentsondesirabilitylargelyconcernedthepedagogicmeritsofthelearningscenariodepictedinthevision(29comments).Therewere,however,asignificantnumberofcommentswhichwereeitherambivalentaboutthevision(11comments)ornegativeaboutit(7comments).Theseambivalentandneutralcommentsraiseddoubtsabout,forexample,theabilityoflearnerstounderstandlearninggoals,ortocorrectlyinterprettheresultsofanalytics,andthesocialconsequencesoftailoringmaterialstoculturalpreferences.

Keyviewsexpressedincludethattherearetoomanydependencies,includingthesystems,linkstorelevantcontentandlackofexistingstructuresforthisvisiontobeachievedatwidescalewith10years.Itunderestimatesthestrengthofinstitutionalinertia,i.e.itmaybepossibleintheworkplacecontextsbutisunlikelyinschooloruniversitysystems.Somewerescepticalthattechnologicalsolutionstomeetthisvisionwouldbewouldbereadyby2025,andeveniftheywere,that‘teachingbyrobots’wouldnotbepoliticallyandsociallyacceptable.However,othertooktheviewthatformsofcollectiveandsocialearninghavealreadybeenimplementedwithinformaleducationandMOOCs,sothisvisioncanbeseenassupportingexistingpractices.Theissuesofhowcertificationchangesinresponsewillbecritical.

Manyoftherequestedactionsconcernedpromotionofthepedagogyassociatedwiththevision.Somementionedtheneedtoresearchintoanddemonstratetheeffectivenessofthesetechniques.Othersmentionedtheneedtosupporttheapproachwithinteroperabilityandprivacyspecifications,andwithchangestocertification.

2.8.4. KeywordsPedagogy,effectiveness,inertia,technicalchallenges,certification,policy,privacy.


27

3. ThemesthatemergedfromthedataWehavesofardiscussedtheresultsonavision-by-visionbasis.Ourpurpose,however,isnottoestablishasharedvisionforlearninganalytics.Norisitsimplytorateourvisionsinorderofdesirabilityandfeasibility.Ratherweseektousetheresponsestothevisionsasawayofunderstandingtheissueswhichconcernthoseparticipatinginthefield,thedriverswhichareimpactingonthefutureoflearninganalytics,andtheactionswhichitwouldbeappropriatetotaketoenhancethatfuture.Wehavethereforecumulatedthefreetextdatawhichwehavecollectedforalltheeightvisions,andcodedthemtoidentifythethemeswhichrecurinthem.Themethodwhichwehaveusedisdescribedinsection1.3.7above,andthethemeswhichemergedfromthisprocessareshowninthefollowingtable:

Table2:Themesthatemergedfromthedata

No. Theme Whenthethemecodewasapplied1 Affect Anyreferencetofeelingsoremotion2 Alienation Commentswhichstressedtheneedtoincludehumansintheanalytics

process,orwhichwereconcernedthenegativeimpactofanalyticsonsocietyandrelationships.

3 Complexity Discussionoftheorganisationalortechnicaldifficultiesandchallenges,ortheneedtocreatenewtools.Alsostatementsarguingtheopposite,thatdeploymentisstraightforward/easy.

4 Cost Alldiscussionoffinancialmatters,not‘resources’inabroadsense.Alsomentionoftheimportanceofthemarket.

5 Ethics Concernsaboutwhetherlearninganalyticsinterventionsweregoodforthepeopleinvolved.Ethicswasdefinedasbeingdistinctfromprivacy.

6 Experience Argumentonthebasisofrespondentsowncurrentpractice,andtheirexperienceofachieving,orfailingtoachieve,somethingspecificwithanalyticsintheworldtoday.

7 Pedagogy Discussionofeducationalmethods,includingtrainingforteachers8 Power Personal,socialandpoliticalcoercion;organisationalstructures;and

contextswherelearninganalyticswillbeunabletofunction.9 Privacy Personal,professionalandpoliticalconcernsaboutthecontrolanduseof

data.Thiswasdefinedasbeingdistinctfromethics.10 Regulation Referencestolaws,rules,policies,etc.11 Standards Referencetotechnicalstandardsfortheinteroperabilityofsystems.12 Temporality Alldiscussionoftime,including‘justintime’,delay,lateruseofdata,

timeliness,etc.13 Validity Discussionofreliability,generalizability,comprehensibility,

worthwhileness,correctness,meaning.

Intheremainderofthissectionwediscusseachofthethemeswhichemergedfromthecumulatedtexts,andsummarisetheassociateddata.AnoverviewofthenumberofcodesappliedtoeachvisionisprovidedinAppendix2:CodingSummaryChart.

Fordetailsofhowmanytimeseachelementwasmentionedinthefree-textresponsestothesurvey,seeAppendix2:CodingSummaryChart.


28

3.1. Theme:Affect

3.1.1. Affect:KeywordsMotivation,perception,transparency,volition

3.1.2. Affect:MainaspectsTheword‘affect’asweuseitherereferstotheexpressionofemotioninrelationtothevisions.Severalrespondentsdiscussedlearners’andteachers’attitudestobeingtrackedand/ormonitored.Otherdescriptionsofteachersandstudentsfeelingsincludedthatbothgroupsmaybefrightenedbytheintroductionoflearninganalyticssolutionsduetolossofcontrol(visions1,2and5).Somenotedthattheattitudesofcompaniesandgovernmentswillinfluencethedegreeandspeedoftakeupofanalytics,whetherornottheseattitudesareinformedbyevidence.Transparentbehaviour(supportedbyopensystems)wasseenasdesirable,andsometeacherscanbeinspiringand/orconfrontational(andcomputerdriventeachingmaynotbe).Theideaofanalyticssystemsenabling‘covert(?)shapingoflearning’wasmentioned(vision7),andtheideaofanalyticssolutionproviders‘over-promising’onthecapabilityoftheirsystems.Aviewthatanalyticswouldnotbeabletogainunderstandingoflearners’personalneedsandmotivations(visions1and7)wasputforward.

3.1.3. Affect:MostrelevantvisionsIssuesofaffectappearedmostfrequentlyinresponsestoVision1,closelyfollowedbyvision5,and7and3.WithrespecttoVision1,somementionedthatthevisiondescribedstudentsbeingmonitored,butnotteachersandthat“Learningisanessentiallysocialactivitywhichreliesonmutualtrustandconfidence.

3.1.4. Affect:LeastrelevantvisionsAffectwasnotmentionedatallinconnectionwithvision8,andonlyafewtimeswithrespecttovisions4(e.g.“ThecynicinmesuggestIambeingidealisticifIthinkthiswillreallyhappen”)and6(e.g.“Wowbigbrotherahead……iwouldapplythosetoolstopolicymakersfirst”).Thesearevisionsthatmostrespondentssawasbeingdesirable(seeFigure12:Alldesirabilitydata).

3.2. Theme:Alienation

3.2.1. Alienation:KeywordsAutomation,communication,dystopia,humanity,intrusion,policy,privacy,resistance,society.

3.2.2. Alienation:MainaspectsAlienationisusedheretoindicatecommentswhicheither

a) rejectedlearninganalyticsinaffectivelanguage,orintermsofsocialexclusionb) ascribedanemotionalresponseorexperienceofsocialexclusiontoothers.

Manyrespondentssaidthatanalyticstechniquescombinedwithinvasivetrackingmethodswouldmeetwithapersonalresponsethatwouldbeabarriertosuccessfuladoptionoflearninganalytics.Otherspredictedabacklashandsocialresistance“teacherswillresisthavingthesensorspointedatthem”,orlamentedthelackofresistance.Itwassuggestedthatsomeareasoflearninganalyticsshouldsimplybeavoided,inthewordsofonerespondent“therearesomebarriersthatweshouldn’tcross”.


29

Akeycommonconcernwasacontrastbetween“Realhumanbeings”andthedesignersofsystems,whoaremembersofatechnocraticelite.MentionwasmadeofOrwelland‘bigbrother’andofthelossofhumanityandinter-personalcontact.Onerespondentcapturedthissocialuneasebyaskingifthedesignersofmachine-drivenlearningsystemswouldwanttheirownchildrenandgrandchildrentobeeducatedwiththem,orforthesystemstoreplacetheirowneliteuniversities.The“scary”prospectof“technologyfanaticism”isseeninathreatbecause“wearenotrobots”or“automatons”tobecontrolledby“algorithmicoverlords”.

Morespecificallywithregardtoeducation,manycommentsexpressedthefearthatteacher-studentrelationswouldbeunderminedbytheuseofanalytics,tothedetrimentofauthenticeducationalpractice.Therewasaperceptionthatthe“solerelianceontechnologywillsimplypigeonholelearners”,creatingstereotypesthatunderminetheflexibilityneededtorespondtodiverseneeds.Theresultcouldbegreatersocialexclusion.Alargenumberofcommentsstressedthathumansshouldremaininchargeofdecisionmaking,andthatinformationoverloadandnon-negotiablelearningoutcomesarealreadyathreattothis.Thiswasrelatedinsomecommentswitharejectionoftheprofitmotiveinanalyticsinterventions,“…otherwisepeoplewillbeatthemercyofjudgementbyproprietarysystems”.

Respondentssuggestedtwopossiblecoursesofaction.Somefeltthatcertainscenariosshouldbeabandonedbecausetheyweretoointrusiveorotherwiseobjectionable.Othersmentionedimprovedpopularengagementinpolicyandbettercommunicationofpolicy,enhancedprivacycontrol,anddemonstrablevaluetousers,asmeanswherebyalienationcouldbecounteracted.

3.2.3. Alienation:MostrelevantvisionsThevisionwhichstimulatedmostdiscussionofalienationwasVision7:Analyticshelplearnersmaketherightchoices(25comments).Respondentsgenerallyreferredtotheundesirabilityofmachinestakingdecisionmakingingeneraloutofhumanhands,andinparticularinthefieldofeducation.Thiswasfelttobedehumanising,toignoreimportantaspectsofhumannature.Itwassuggestedthatmechanisedanalysiswouldtendtoleadtoreinforcementofexistingsocialdivisions,andcreatepressureforconformity.Thosewhocommentedvariedbetweenfearingtheconsequencesoftechnologicalomnipotence,andexpressingscepticismthatdataanalyticswouldbecapableofsubstitutingforhumanjudgement.

TheothervisionswhichreceivedlargenumbersofcommentscodedforalienationareVision6:Learninganalyticsareessentialtools(17comments)andVision1:Learnersaremonitoredbytheirlearningenvironments(16comments).AswithVision7,commentsonthesevisionsoftenreferredtotheundesirabilityofcedinghumanjudgementtomachines,andthelossofhumanvaluesandindependence.However,visions1and7additionallydependonmassivecollectionofdata.Invision1thisconcernsthegatheringofveryhighdensitydataintheclassroom,whereasinvision7thedataisgatheredandcompiledfromeducationalrecordsacrosstheeducationsystem.Thisfocusonmassivedatacollectionseemsrelatedtoastrandofcommentsreferringtoinvasionofprivacy,ubiquitousmonitoringandincreasedcontrolofpersonalaction.

3.2.4. Alienation:LeastrelevantvisionsThevisionswiththeleastcommentscodedforalienationareVision5:OpensystemsarewidelyadoptedandVision4:Learnerscontroltheirdata.Bothofthesevisionsimplicitlyaddressconcerns


30

aboutprivacyandsocialcontroloflearninganalyticswhicharethekeythreadsrelatedtoalienation.Itthereforeseemsreasonabletosuggestthatthelackofcommentscodedforalienationonvisions4&5suggeststhatthesevisionsareacceptabletorespondentswhoexperiencealienationwithregardtoothervisions,andthatopendevelopmentoflearninganalyticsandmechanismsforpersonalcontrolofdataareimportantinitiativesiflearninganalyticsistobewidelyacceptable.

3.3. Theme:Complexity

3.3.1. Complexity:KeywordsBarrier,challenge,social,technology,understanding.

3.3.2. Complexity:MainaspectsComplexitywasamajorthemeintheresponses–itwasthethirdmost-frequentlyoccurringtheme.(Forfullcounts,seeAppendix2:CodingSummaryChart.)Itcoverscomplexityandchallenges,whethertechnicalormorebroadly.Aswellasresponsessayingtherearecomplexityandchallenges,itincludesthosemakingtheoppositecase,thatitisrelativelystraightforward,simple,orhasalreadyhappenedinotherfields(asdistinctfromthePowertheme,whichcodescontextswherelearninganalyticsmaybemoreorlessdesirable,feasible,etc.).

Therearetwomainaspectsofthistheme.Thefirstisfromrespondentsarguingthatthereisalotofcomplexity,intermsofthetechnicalchallengestobesolved,andintermsofthehumansmakingsenseanduseofthedata.Theserespondentstendedtoseethiscomplexityaspresentingabarriertothevisions,althoughviewsvariedonhowmuchofabarrier,fromaminorblockage,to‘verydifficulttoachieve’,tomakingthevisioncompletelyinfeasible.Thesecondaspectisfromrespondentsarguingthatitwouldberelativelystraightforward,thattherearefewornotechnicalorsocialchallenges,orthatifthereare,theyareeasilyaddressed.Theserespondentsoftenmadereferencestosituationsorcontextswherethisaspectsofthevisionwasalreadyhappening,andcitedtherapidpaceofdevelopmentinthetechnicalfield.

Itishardtosimplysummarisethebalanceofmanyresponsestomanydifferentvisions,butmostoftheresponseswereverymuchinlinewiththerespondentwhosaid:“Thefeasibilityofthisisbasedonovercomingsignificantsocialandpoliticalhurdles.Itisfeasibletechnically.”Itisimportanttonotethatalmostnorespondentsarguedthatitwasnotcomplextechnically,andsomeofthefewthatdidimpliedthattheywerenotfamiliarwiththetechnicaldetails.Thustherespondentsingeneralagreedthattherewasconsiderabletechnicalcomplexity,aswellassocialandpoliticalcomplexity,butonbalancethoughtthetechnicalchallengesweresignificantlymorelikelytobeovercomethanthesocialandpoliticalones.Therewasmuchsupportfortheideathatthesebroaderconsiderationswerethemostimportantones.

3.3.3. Complexity:MostrelevantvisionsResponsestoalmostallthevisionsdealtwithcomplexityasanissue.Respondentsgenerallyengagedwiththeparticularcomplexityofthevisionassetout.So,forinstance,respondentstoVision2(Learners’personaldataaretracked)gaveexamplesofexistingsensorsandInternetofThingstechnologiesasargumentsforthevisionbeingfeasible.


31

3.3.4. Complexity:LeastrelevantvisionsTherewasrelativelylittlementionofcomplexityinresponsestoVisions3and7.ForVision3,thisisprobablybecausethatvision(Analyticsarerarelyused)doesnotrequirechallengesofcomplexitytohavebeenovercome.ForVision7(Analyticshelplearnersmaketherightchoices),itmayhavebeenbecauserespondentswerefocusingonthepedagogicaspectsofthevision.

3.4. Theme:Cost

3.4.1. Cost:KeywordsBudgetsavings,businessmodels,commercialmodels,costs,economicmodels,fundingmodels,governmentfunding,marketplace,value

3.4.2. Cost:MainaspectsTheuseoflearninganalyticswasassociatedwithaperceivedneedtobringdownthecostofeducationandreducethecostofworkplacetraining.‘Economicscontinuetopressurethetraditionalclassroomwhichmakesthisvision[7]anecessity.’‘Bringingdowncostsisamajordriverineducationaldevelopment.’

Manyrespondentswhomentionedelementsofcostfelttheywereatthemercyofmarketforcesandthatanalyticswouldbeintroducedbecausecompanieswantedtomakemoneybysellingthem.OnecommentedpessimisticallyonVision3,‘Commonsensewon'tprevailinlightofthepowerofmoney’.AnotherrespondedtoVision7,‘Economicscontinuetopressurethetraditionalclassroomwhichmakesthisvisionanecessity.’

Conversely,someexpectedVision5(opensystemsforlearningarewidelyadopted)tofailbecauseitwouldstopcompaniesmakingmoneyfromanalytics,soeconomicforceswouldlimitopenness.

InthecontextofVision4(individualscontroltheirowndata),thevalueofdatawasanissue:‘weneedtoincreaseawarenessofthevalueandtradingpowerofdata’and‘Humansneedtounderstandthattheirdataisanequaltoacurrency.Respectivemeasuresmustbetakentoworkouttheexchangerate.’

Elsewhere,costwastypicallyassociatedwithmoney,althoughonerespondentidentifiedotherformofvalue:‘‘Alotofthevalueofeducationisnotinsupportingindividualgrowthbutindevelopingcommunitygrowth,norms,andsoon.”

Therewasaconcernthatmoneywouldbewastedonunfruitfullinesofresearchandvaluelesstools.

3.4.3. Cost:MostrelevantvisionsIssuesofcostappearedmostfrequentlyinresponsestoVision6(opensystems).Severalpeoplefearedthatthevisionwouldbeblockedbyvendorlock-in.Somesuggesteddevelopinganeweconomicmodeloranewfundingmodel,usinggovernmentfundingtosupportthisvision.

ReferencestocostrelatedtoVision1(classroomsmonitorthephysicalenvironmenttosupportlearningandteaching)focusedontheneedtoprioritisethoughtfullyandavoidwastingmoney.‘‘Focusvaluableandrestrictedresearchtimeandfundingonwhatismorefeasible,reliable,validandofimmediatevalue.”


32

3.4.4. Cost:LeastrelevantvisionTherewerefewresponsesrelatedtocostinrelationtoVision3(analyticsnotused)andnosuggestionsthatexpensewouldlimittheuseanddevelopmentofanalytics

3.5. Theme:Ethics

3.5.1. Ethics:KeywordsAbuse,context,culture,exploitation,personalisation,policy.

3.5.2. Ethics:MainaspectsTherewereconcernsexpressedacrossthevisionsabouttheneedtoensuretheethicaluseofdataforeducationalpurposes,anditsexploitationforotherpurposes,e.g.throughthedevelopmentofpoliciestoguidetheethicaluseofanalyticssystems.Respondentsmadereferencetounethicalpracticessuchasattemptstogamelearninganalyticssystems,andremarkedthatunethicalpracticescouldcausevision3(“In2025,analyticsarerarelyusedineducation”)tooccur.Therewerealsomentionsofthemannerinwhichlearninganalyticscouldbeappliede.g.thatteachers’rolesshouldnotbereduced,butchangedaslearningisa‘human,sociallyembedded,communalactivity’(inresponsetovision7).

3.5.3. Ethics:MostrelevantvisionsThevisionswhichdrewthemostcommentsthatmentionedethicalissueswithrespecttodesirability,feasibilityandactionswerevisions2,3and1respectively,withvision2drawingthemostoverall.Responsestovision2mentionedpotentialofabuseofdataandexploitationofdataforpurposesotherthaneducation.Therewerealsoconcernsthatthevisionreliesonanideal,thatitaimsatmatchingpeoplewiththatideal,ofmeetingexternallydefinedgoalsinsteadofmaximisingindividualpotentialinalocalcontext.(Aresponsetovision6mentionedarelatedconcern,i.e.thepotentialforanalyticstoblockopportunityforindividualstotranscendtheirbackground).Responsestovision2includedconcernsthat‘thosemostinneedwillnotoptin’tohavingnecessarydatacollected,promptingneedtoactontheirbehalf.Requirementoftechnologycouldlimitaccess.

3.5.4. Ethics:LeastrelevantvisionsVision8drewthefewestcommentsmentioningethicalissues.Thiscouldbebecausethevisionstressestheuseofreliableandvalidlearningroutesforindividuallearners.However,oneofthecommentsraisedaconcern:‘selectingstudymaterialbasedon"theculturalcharacteristicsoflearners"soundslikeagood(bad!)waytoensureadeepsplitbetweendifferentgroupsbygivingthemdifferenteducationsontopofotherdifferences’.

3.6. Theme:Experience

3.6.1. Experience:KeywordsCommercialpressure,communicationbarriers,failure,policies,success,technology.

3.6.2. Experience:MainaspectsCommentswhicharecodedunderasrelatingtoexperiencearethoseinwhichtherespondentappealstotheirpersonalexperienceoflearninganalytics,ortheaccumulatedexperienceofthecommunity,asabasisforarguinginfavouroragainstthevisionconcerned.AdetailedbreakdownofhowmanytimesthiscodewasusedisavailableinAppendix2:CodingSummaryChart.


33

Averywiderangeoftypesofexperiencewerementionedasbeingrelevanttothevisions.Moreover,theexperiencesweredividedbetweenthosethatsupportedapositiveviewofthefutureoflearninganalytics,andthosewhichwereprovidedasexamplesofconstraintsonthefutureoflearninganalytics,withabout60%findingpositiveevidenceintheirexperience,andabout40%findingnegativeevidence.Thisisinlinewithourcommentselsewherethat

a) learninganalyticsisdeeplyentwinedwithwidersocial,economic,organisationalandtechnologicalprocesses

b) thereissubstantialdisagreementamongexpertsandenthusiastsforlearninganalyticsaboutthewayinwhichthefieldislikelytodevelop,andthedegreetowhichitwillfulfilitspromise.

Muchofthepositiveevidenceidentifiedinexperiencesconcernedthepresenttrajectoriesoftechnologiesandpracticesthatarerequiredforthesuccessfulimplementationoflearninganalytics.Theseincludepervasivecomputing,smartenvironments,theinternetofthings,biometricsandwearables.TherewerealsodissentingvoiceswhopointedoutthatITdepartmentsdonottalktoeachotherorhavegoodcommunicationwiththeirinstitution,andcomplainedof“Over-promising,thearroganceofthepeoplewhopontificateaboutthis,andtheirignoranceofwhatitactuallytakestolearnandtoteach”.

Inotherareastheconclusionsfromexperienceweremorecontested.Forexampleopenlearninganalyticswassupportedbythefactthatmanyalgorithmsarealreadypublic,thatopencollaborationisunderwaytosupportlearninganalyticsinfrastructure,andthatopeninnovationisanestablishedbusinessmodel.SimilarlyIPregulationoverhumangenomepatentswasgivenasapositiveexample.Ontheotherhand,itwasargued“moneytalks”andcompaniesliketokeeptheirmethodsanddatabehindIPbarriers,whilepublicinterestsarenotwellcoordinatedtofightagainstprivateinterests.Gatekeepers,itwassuggested,wouldmaintaincontrol,astheyhaveinmedicalanalytics.

Anumberofrespondentspointedtotheworkbeingdoneinthecorporateworldandinmilitarytraining,asevidencethatdatadrivensystemscanbeeffective.Howeverotherspointedtoexperiencesofeducationwhichdonotfittheseexamples,forexamplethelackofwidespreadadoptionofprogrammedlearningandintelligenttutoringsystems,andareaswherepredictionisnotperfect(theweather).Anotherrespondentpointedtotheneedforteacherstostimulatelearnerstoengageinself-pacedcourses,andthegenerallackofprogressinincreasingtheproportionofonlineeducation.

Anumberofrespondentspointedatexperiencewhichindicatesthatconcernsaboutprivacyareexaggerated.Theymentionedthatmedicaldataandfinancialdataisalreadysharedandmanagedwithoutmajorproblems.Education,theypointout,alreadykeepsextensiverecordsonlearners,andcarriesoutpredictiveanalysisbymeansofformativeassessment.Policiesforethics,datagovernanceandsecurityarebeginningtobeputinplace.Othersseeatrendtoabuseofdata,andaneo-liberalagendawasmentioned,thereductionoflearningtooutcomes,andKPIs.TheInBloomexperience(Horn2014)indicatesthatprivacyshouldbetakenseriouslyandbuiltinfromthebeginning.

Experiencealsoshowedthatmoreneedstobedoneinsharingknowledgeandexperience,bothinacademiaandintheEU.


34

3.6.3. Experience:MostrelevantvisionsCommentsmadebyrespondentswerecodedfor‘experience’mostfrequentlyinVision3(25comments).Thisdescribesafutureinwhichlearninganalyticshasbeenside-lined,whichwouldclearlybeamatterofconcernforourrespondents,whoarealmostallengagedinthefieldinonewayoranother.Thisvisionisalsoonewhichreliesontheextrapolationofcurrentsocialandpoliticalprocesses,whichistopicthatmanypeoplewillhaveanopinionon.Theopinionswereverydivided,with12commentscitedexperiencetoinsupportofthepredictionthatanalyticswouldberarelyused,while13commentsarguedthecontrary.Thecommentswerealsoverypolarised,withcommentsrangingfrom“Thevisionissoimplausibleitbarelywarrantscomment”toassessingitas“veryprobable”.

Vision7,alsogeneratedmanycomments.Thisvisionsuggeststhatmassivedatabasesoranonymisedstudentinformationwillenablesystemstomakebetterrecommendationsthatteachers,asituationthatwouldthreatenthecurrentnatureoftheeducationsystem,andalsotheprofessionalactivitiesofmanyrespondents.8commentscitedexperiencetosupportthevision,while10arguedthathumaninputwasanessentialpartofeducation.

Asregardsvision5onopensystems(18comments),noexperiencewascitedtoindicatethatthevisionwasundesirable,butrespondentswereequallydividedbetweenarguingthatcurrentworkwouldleadtoopensystems,andarguingthatcommercialinterestsandlackofcommunicationwouldpreventthevisiondeveloping.

Visions1(16comments)and6(14comments)bothproposeaworldinwhichlearninganalyticshasdevelopedandworkseffectively.Inthesescenariosexperiencewaslargelycitedtoindicatethatthevisionwouldbepracticable,whileobjectionstothevisionaretobefoundunderothercodes,suchasalienationandprivacy.

3.6.4. Experience:LeastrelevantvisionsThethemeofexperiencewasleastfrequentlycodedforvision2,perhapsindicatingthatthetechnologyofsensorsisunfamiliartorespondents.Itisinterestingthattherewaslittleevidenceofexperienceforvisions4(5coded)and5(6coded),bothofwhichareratedashighlydesirablebyrespondents.Thisenthusiasmpluslackofexperiencesuggestsagapintheresearchbeingcarriedout,andpresumablyalsointheinstitutionalandfinancialsupportforsuchresearch.

3.7. Theme:Pedagogy

3.7.1. Pedagogy:KeywordsEducators,learn,learner,learning,pedagogy,teach,teacher,teaching.

3.7.2. Pedagogy:MainaspectsPedagogywasthemostcommonthemeintheresponses.(Forcounts,seeAppendix2:CodingSummaryChart.)Manyrespondentsraisedissuesaboutlearninganalyticsandtheprocessesofteachingandlearning,andaboutawarenessandtrainingforlearners,teachersandmanagerstomakebestuseoflearninganalytics.

Themostcommonresponseinthepedagogythemewereconcernsthatlearninganalyticsrepresentspoorpedagogy,orrisksbeingusedforpoorpedagogy.Theseresponsesarguedthatthe


35

dataisnotusefultounderstandlearning,orparticularly“higher-orderlearning”,asopposedtomechanicalbehaviours,physicalskills,and“purelytheacquisitionoffacts”,whichwerethoughttobemorelikelytobethefocus.Theyarguedthatanalyticsdoesn’t(ormaynot)captureessentialelementsoflearning,suchasmotivation,volition,andthehumanconnection(seealsotheAlienationtheme).Manyrespondentswerefirmlyoftheviewthateffectivelearningrequiredahumanteacher,butnotallofthem.Oneevencommented,“Totheextentthattheteachernegativelyaffectsthequalityoflearning,theteachershouldberemoved.”

Somerespondentsimplicitlybelievethatourcurrentunderstandingoflearningisgood;mostofthoserespondentswereatleastsomewhatscepticalthatanalyticscouldcapturewhatweknowaboutgoodpedagogy.Othersarguedthatweneedmoreresearchintolearningitself,andthatlearninganalyticsmighthelp.

Personalisation,responsivenessandadaptationwereacommonissueraisedunderthistheme.Mostresponseswereoftheviewthatlearninganalyticswouldhelpachievepersonalisationandindividualadaptation,butnotall,andsomewerenotconvincedthatthesewouldbegoodthingstoachieveanyway.Asmallnumberofresponseswereconcernedabouttheopposite:thatanalyticswouldmakelearningmoreone-size-fits-all.

Therewasconcernaboutwhatvaluessuchsystemswill“embed”:whetherexistingpoorpedagogicalpracticeorineffectivepowerstructures(cross-referencePower),theill-informedviewsofprogrammersanddatascientists.Somerespondentswereconcernedthatamarketviewoflearningwouldbeembedded;asmallgroupofotherswereconcernedabouttheopposite:thatitwouldallbeabout“educationalapproaches”withnoroleforthemarket.

Mostresponseswerecautiouslypositive:it“doeshavethepotentialtohelpteachersandeducators”,buttheywerenuanced:it“hasaroletoplay”,ratherthanbeingtheentireanswer.

Akeyissueraisedunderthisheadingwasaperceivedneedfortraining,developmentandawareness-raisingaroundanalytics.Mostoften,itwasteacherswhowereseenasneedingthisprofessionallearning,butrespondentsalsomentionedaneedformanagersandlearnersthemselvestogainskillsininterpreting,using,andactingonthedatainappropriateways:“increasinglylearners,families,educatorsandothers[will]acquireexperienceandskillsinjudgingwhentoacceptpredictionsandwhentoviewthemwithsomescepticismandapplyadditionalknowledge”.

3.7.3. Pedagogy:MostrelevantvisionsThisthemewasmentionedbymanyrespondentstoallvisions,probablybecauserespondentsfeltthatpedagogywasanimportantfactorforallofthem.

3.7.4. Pedagogy:Leastrelevantvisions

Notapplicable,becausethisthemewasmentionedinrelationtoallvisions.

3.8. Theme:Power

3.8.1. Power:KeywordsBigBrother,datacontrol,empowerment,humanrights,law,misuse,policy,surveillance


36

3.8.2. Power:MainaspectsRespondentsidentifiedseverallevelsofpowerassociatedwiththesevisions.Atanindividuallevel,somewelcomedopportunitiestoempowerthelearner;whilesomeworriedaboutalossofagencybythelearner.Othersbelievedthateducationshouldbeledbyateacher,whocanmakedecisionsthatlearnersarenotexperiencedenoughandanalyticsnotnuancedenoughtomake.Ataninstitutionallevel,therewerefearsthatcontroloverlearningwouldbeputinthehandsofsystemdesigners,programmersandmanagersratherthanteachers.Atanationallevel,therewereconcernsaboutthedevelopmentofasurveillancestate,andlarge-scalemonitoringprogrammes.Ataninternationallevel,therewereissuesabouttheaimsandvaluesofeducation,andaboutapossibleneedtoexpandourdefinitionofhumanrights

3.8.3. Power:MostrelevantvisionsIssuesofpowerwerefrequentlyraisedinrelationtoVision5(openmodels),particularlytheneedtobalancepowerbetweenusersandvendors:“agoodbalancebetweenempoweringinstitutionswithoutcripplingcompaniesfrombeinginnovative”.Theimportantroleofgovernmentandnationalorganisationsinproposingandimplementingshiftsinpolicies,standardsandpracticeswasassociatedwiththis.

Vision4(datacontrol)provokedstrongreactionsaroundhumanrights:“therearenewwaystoexploit,manipulateandassertpoweroverothersthankstothedataweholdaboutthem.Thispowerneedstobecurtailed.”Therewerecallsforactiononthisissuefromparents,studentbodies,grassrootsmovements,politicalrepresentatives,policymakersandgovernments.

Severalofthevisions,butparticularlyVision1(monitoringtheenvironment)provokedreferencestoBigBrotherandadystopianOrwellianfuture,including:‘atoolforassertingpower’,‘ArewereadyforaEducationalBigBrother?’“theimplicationsforlearningoutsidetheclassroomrequireasurveillancesocietybeyondanythingsocietywouldcurrentlyfindacceptable’and“teacherswouldresist”.Threepeopledescribedthisvisionasalittle“scary”.

3.8.4. Power:LeastrelevantvisionsVisions2(datatracking)and6(essentialtool)provokedthefewestreferencestopower.However,manyofthesereferenceswereintense.OnerespondenttoVision2believedthat“Peoplewouldbecomeslavesoftheirsensorsandtheirdiagnosis&recommendationsapps.’Anothercommented,‘soundslikewearerobotscontrolledbyouralgorithmicoverlords”.Morepositively,onerespondenttoVision6sawlearninganalyticsasawayofgivingmorepowertoeducation:“Rationaleducationalsystemdesignedtomaximizestudentoutcomes.Increaseshumancapital.Educationovercomeshistoricalinjustices.”

3.9. Theme:Privacy

3.9.1. Privacy:KeywordsBigbrother,control,power,regulation,surveillance.

3.9.2. Privacy:MainaspectsTheresponsesontheprivacythemecanbesplitintotwobroadcategories,i.e.privacyrelated‘threats/barriers’tosuccessfulimplementationoflearninganalyticssolutionsandprivacyrelated‘enablers/opportunities’forthesuccessfulimplementationoflearninganalyticssolutions.


37

Inthebarrier/threatcategory,somerespondentsconsideredtheamountofmonitoringnecessaryforsomevisionswouldbetoomuchforsomesocialgroups,e.g.“aBigBrotherscenario,withdeepintrusionintotheprivacyandintegrityofstudents,whichisnotneededforeffectivelearninganalytics”andinsomesituations“theimplicationsforlearningoutsidetheclassroomrequireasurveillancesocietybeyondanythingsocietywouldcurrentlyfindacceptable”(vision1).AnotherbarrieridentifiedisthatIfstudentsareabletooptoutofsomeoralloftheirdatabeinganalysed,thenpotentiallythosewhooptoutmaybethosemostinneedofsupport(asacounter,somerespondentsobservedthatthereisaneedfor“institutionsandmechanismsthatareindependentandactonbehalfofstudentsandteachers”).Someidentifiedprivacyregulationsasapotentialbarrierinsomejurisdictions“itisdifficulttosayhowfeasibleitis,especiallyinEU”(vision1).Therewasaconcernaboutthecostofenablingeasy-to-usetoolstosetandadjustprivacysettings,intermsofbothtechnologydevelopment,andalsointermsoftimetoraiseawareness.

Inthe‘enablers/opportunities’category,onerespondentremarkedthatrisksofintrusionarenotasufficientreasontoreduceinvestmentinlearninganalytics(vision3),and“manypeoplewill(do)continuetobecomfortablewiththeirdatabeingused”.Thereisaneedforneedforan“up-to-date,transparentlegalframeworkstoprotecttheindividual”(vision2).“Focusonhowtomanageprivatedatahastobethemostimportantissue”.Individualsmusthavepowertodecide,andorganisationsmustbetransparentaboutwhatdatatheyuseandhowtheyuseit.Thereare“conceivablescenarioswhereitismorebeneficialfortheuserstonothavefullcontroloftheirowndata,particularlywhereill-informeddecisionsmayhavelifeconsequences”andthereisaneed“toconsiderthegreatergoodofsharingnon-identifiabledata”.Itis“likelythatinstitutionsandtoolsdeveloperswillaskuserstosignawaysomerightstodataaspartoftheconditionsofthemorusingtheirservices.Somedegreeofaccesstodataseemsnecessarytoruntheinstitution”.Thereisaneedfor“clearguidancearoundtheretentionofanonymiseddata”(vision4).”Openingupaccesstodatacreatesavibrantmarketforthirdpartytools”(vision5).

3.9.3. Privacy:MostrelevantvisionsVision4withatotalof99commentsofwhich38wereactions).Onlyoneothervisionhadmorethan20comments(Vision1,ofwhich15ofthe28relevantcommentswereaboutdesirability).

3.9.4. Privacy:LeastrelevantvisionsVision8(only1comment,aboutactions).Visions2,5,6and7had11orfewercommentsaboutprivacy.

3.10. Theme:Regulation

3.10.1. Regulation:KeywordsGovernment,guides,law,laws,legislation,legislature,policies,policy,protocol,protocols,regulation,regulations,rules.

3.10.2. Regulation:MainaspectsThisthemeisconcernedwithlaws,regulations,policyandrules.Comparedtootherthemes,itwasmentionedbyrespondentsamediumamount.

Almostallresponsesmentioningregulationsaidorimpliedthattherewasaneedformoreorbetterregulation:veryfewresponsesarguedforderegulation,repealoflegislationoralooseningof


38

existingpolicy.ThemainexceptionwasinresponsestoVision3(Analyticsarerarelyused),whereseveralresponsesarguedthattherestrictionssuggestedinthatnegativevisionwouldbeundesirableorunnecessary.Therewerealsosomeresponsesarguingthat“rulesandregulations[…]couldcausedelaysandhindercarryingitout”.

Regulationwassuggestedatawiderangeoflevels:fromrulesorpoliciesforindividualorganisationsuptoformallegislationatnationalorsupra-nationallevel.

Overwhelmingly,respondentswhomentionedregulationwereconcernedtoenforceethics,privacy,ownership,andtransparency,buildingonorstrengtheningexistingDataProtectionpractice.(SeealsoEthics(Section3.5)Privacy(Section3.9),Power(Section3.8)).Otherrespondentscitedaneedforregulationtoensuresecurity,andtoencourageormandateopenness.

Althoughmostrespondentswereimplicitlyconfidentinthepossibilityofeffectiveregulationbeingdeveloped,manywereconcernedthatitwouldbeadifficultandcomplexprocess.Therewereevensomecommentsvoicingdoubtthatpolicymakerswouldbeabletorisetothischallenge,e.g.“Idon'tseepolicymakersabletoreachsuchsophisticateddecisions”.

Manyresponsesmentioninggovernmentsandpolicymakersalsocalledforfundingtosupportresearchandawareness-raisinginthisfield.

3.10.3. Regulation:MostrelevantvisionsThisthemewasmentionedunderallthevisions.Thevastmajorityofresponsesinthisthemewereinthefinalopentextsectionconcernedwithactionsneeded.Indeed,regulationswerethemostcommonsuggestionforactionrequired,oroneofthemostcommon,forallvisions.Thissuggeststhattherespondentssawregulationsasthemostimportantactionrequiredinthefieldoflearninganalytics.

3.10.4. Regulation:LeastrelevantvisionsTheleastrelevantvisionsareVision2:In2025,personaldatatrackingsupportslearning,andVision7:In2025,mostteachingisdelegatedtocomputers.ForVision1,theproposalthatpersonaldatawouldbemanagedbythedatasubjectappearstohaveallayedconcerns.InVision7,theconcernwasnotwiththeregulationofdata,butratherwiththeimplicationsoftheresultsofanalysis.

3.11. Theme:Standards

3.11.1. Standards:KeywordsStandards,standardization,interoperability,API,IMS,LTI,Caliper,ExperienceAPI,xAPI,TinCan,SC36

3.11.2. Standards:MainaspectsStandardswasthesecondleast-commonthemeraisedbyrespondents.Itwasnotafrequentresponse,butthoserespondentswhodidraiseittendedtoleavelengthyandinformedcommentsaboutit.

Almostalltheresponsesaboutstandardssaidorimpliedthattheywerenecessaryordesirableinthisfield:“thereisaneedforstandards”,“clearstandardsshouldbedeveloped”.“Limitedinteroperability”wascitedasaproblemthatstandardscouldhelpovercome.


39

Awiderangeofstandardsandstandardsbodieswereraised,includingSC363(theappropriatetechnicalcommitteeoftheInternationalStandardsOrganisation),IMSLearningToolsInteroperability4,andIMSCaliper5,andExperienceAPI/TinCan6.

Aswellasstandardsaboutthedatainlearninganalytics,somerespondentsalsosawaneedforstandardsaboutbroaderissuessuchasprivacy,ethics,datasecurity,andgoodpractice(seealsotheRegulations,PrivacyandEthicsthemes).

3.11.3. Standards:MostrelevantvisionsAlmostalltheresponsescodedundertheStandardsthemeaddressedVision5(Opensystemsarewidelyadopted),whichislargelyconcernedaboutstandards.TherewereseveralStandardsresponsestoVision4(Learnerscontroltheirowndata),whichmainlysaweffectivestandardsasakeyenablerforthisvision.

3.11.4. Standards:LeastrelevantvisionsVeryfewcommentsaboutstandardswereraisedinresponsetotheothervisions,andwheretheywere,theytendedtobefromrespondentswhoclearlyhadexpertiseinstandards.Thissuggeststhatexpertsareunlikelytomentionstandardsinlearninganalyticsunlesstheyarespecificallypromptedortheyareofparticularinteresttotheexpert.

3.12. Theme:Temporality

3.12.1. Temporality:KeywordsDuration,initialstate,just-in-time,rate,speed,sufficiency,timescale.

3.12.2. Temporality:MainaspectsTherewerecommentsacrossthevisionsaboutthespeedatwhichchangeswouldoccur.Somerespondentsthoughtthataspectsofthevisionswerealreadyoccurring,othersthatwithin10yearsisachievable,andsomethatitwilltakelonger.Somenotedthatvariousrequirementsnecessaryforsomeofthevisionsmaytakedifferenttimescalestocometopass,e.g.thetechnologymaybeinplaceby2025,butpoliciesanduseracceptancemaytakelonger.

Othercommentsreflectedontimewithrespecttothenatureofsupportthatlearninganalyticscouldenablee.g.just-in-time,ortoenableteacherstodetectproblemsearlier.Somewereconcernedthatteachersmaynotenoughhavetimetoactonthedata.

Therewerecommentsaboutthetime-spanthatthedatanecessaryforsomeaspectsofthevisionscouldlegallybestoredfor.

3.12.3. Temporality:MostrelevantvisionsVision5(“In2025,opensystemsforlearninganalyticsarewidelyadopted”)hadthemostcommentscodedasbeingrelevanttotemporality(i.e.15).Theseincludedaremarkthatstandardisedtoolsandmethodswillrestrictinnovationinthelong-run,causingthefieldtodevelopmoreslowly.Othe

3http://www.iso.org/iso/iso_technical_committee%3Fcommid%3D453924https://www.imsglobal.org/activity/learning-tools-interoperability5https://www.imsglobal.org/activity/caliperram6https://tincanapi.com/overview/


40

otherhand,interoperabilitystandardswillensurethatLeaningAnalyticstechnologymaybeembeddedintonextgenerationofeducationalsystems.Therewereremarksaboutspeed(“Similartothe(slow)growthoftheOERmovement,2025seemsreasonable”),comparisontospeedanddepthofadoptioninrelatedfields(despitethepopularityofLearningManagementSystems,interoperabilityspecificationsarenotcompletelysupportedbymost;howeveritispossiblethatdefactostandardsarise)andchange(theLearningAnalyticsstateoftheartwillcontinuetochangemakingstandardisationextremelydifficult).

Vision1andvision7had10commentseach.WithrespecttoVision1,respondentsnotedthatitwouldbeadvantageoustohavedatasoastobeabletoprovidejustintimesupport,or(Vision3)todetectproblemsearlier.However,somewereconcernedthatteachersmaynotenoughhavetimetoactonthedata.Respondentsalsonotedthingsthatmightaffectspeedofimplementation“howeveritmightbehumanfactors,rulesandregulationsthatcouldcausedelaysandhindercarryingitout”.ForVision7respondentscommentedthatlearninganalyticsshouldmeanthatlearnerscanspendsameamountoftimewithteachers,butthetimeisspentmoreeffectively.Othersthoughtthatitwilltakemorethan10yearstoachieve,e.g.theslowpaceofinstitutionalchangewillactagainstthishappeningwithin10years;“Aculturalandeducationalshiftisneeded–andthoseareslow”.

Vision3had9comments,ofwhichtwiceasmanywereaboutdesirabilityascomparedtofeasibility(i.e.6:3).Referencewasmadetowavesofhigherandlowerusageduetoavailabilityoftechnologiesandprivacyissues,andtotheincreasingriskofprivatedataleakingunlessprivacyisdesignedintoalllearninganalyticssystemsfromthebeginning.

3.12.4. Temporality:LeastrelevantvisionsVisions2and8hadthefewestcommentsthatwerecodedasrelatingtotemporality,i.e.3foreachvision.Thismaybebecausealthoughtemporalissuessuchthespeedatwhichnecessarychangeswouldoccur,ordifferentratesofevolutionofnecessarycomponentsarerelevanttothesevisions,otherissuesweremoreimportantformostoftherespondents.

3.13. Theme:Validity

3.13.1. Validity:KeywordsAssumptions,education,generalizability,learning,reliability,research.

3.13.2. Validity:MainaspectsHere,‘validity’isusedtoreferalsotothereliabilityandgeneralizabilityoflearninganalyticsandtoallaspectsoftheresearchonwhichtheyarebased.Thereisaperceivedneedto‘Focusvaluableandrestrictedresearchtimeandfundingonwhatismorefeasible,reliable,validandofimmediatevalue.’However,carryingoutthistypeofresearchposesseveralproblems.Oneoftheseisconcernedwithphilosophicalquestionsaboutthenatureoflearning,whetheritissomethingthatteachersdotopupils,aprocessof‘change,growth,transformation’,orsomethingelse.Withoutanagreeddefinitionoflearning,itisdifficulttodefinelearningsuccess,howitcanbeoperationalized,whatdatawillprovideevidenceofit,andhowquicklyitshouldbeapparent.

Somerespondentsfeltlearninganalyticsresearchshouldfocusonareasthatappearstraightforwardandclosetosolution;othersfeltifitshouldbe‘tiedtopedagogicaloutcomes’orshouldbeginbydiggingintothe‘deepcomplexityoflearning’.Manysuggestedareasforresearchthatfocusedon


41

areasnoteasytodefineorcapturesuchasorchestrationoflearning,higher-orderthinkingskills,attitudes,skills,creativityandcriticalunderstanding.Theyalsoidentifiedconfoundingvariablesincluding‘pressuresofwork,family,sociallife;lackofintrinsicmotivation;makingcurriculasufficientlyrelevanttopeople'slives’.

Respondentsstressedaneedforusersto‘havetheaccesstoquestiontheprocessesandassumptionsunderwhichthedataisinput,massaged,andoutput’,toopenupthealgorithmsandsuccesscasesandtoprovideareliableevidencebaseforthesuccessofdifferentmeasures.Therewasalsoacallformorework‘tobedoneonhowstudentsreacttodatatheyaregivenandhowthatdataispresentedtothem’.

3.13.3. Validity:MostrelevantvisionsValiditywasidentifiedasanissueinrelationtoVision6(essentialtool).Again,thiswasinthecontextofproblems.Onerespondentpointedoutthat‘verylittlecredibleresearchhasdemonstratedanyreallarge-scalebenefitstolearnersorinstitutions.’Therewasaworrythatthatuseofanalyticscanleadto‘self-fulfillingprophecies’andanassumptionthat‘thatthefutureoutcomesareafunctionofpastandpresentvalues’.Therewasalsoacallfor‘qualitativeapproachestostudytheeffectsoflearninganalyticsineducation’inordertotakeintoaccountthemanycomplexissuesthatcannotbeconsideredadequatelyusinganentirelyquantitativeapproachtoeducationalresearch.

3.13.4. Validity:LeastrelevantvisionsOnlyonekeyissueaboutvaliditywasraisedinrelationtoVision4(datacontrol).‘Ifyoudon'thaveafulldatasetcanyoureallydrawmeaningfulconclusions?


42

4. ConclusionsTheobjectivesofthisstudy,asdefinedinsection1.2.2,are:

• Toexploreorexposeunderlyingassumptionsorinformationleadingtodifferingjudgmentsonlearninganalytics

• Tocorrelateinformedjudgmentsonthetopicoflearninganalytics,whichspansawiderangeofdisciplines.

Wenowdiscussourfindingsinrelationtothesetwoobjectives.

4.1. Differingjudgementsondesirabilityandfeasibility

Figure9:Disparityinattitudestodesirabilityandfeasibility

Figure9makessalientthosevisionsforwhichtherewasadisparitybetweendesirabilityandfeasibility.TheclaritythatthisrepresentationbringscomesatthecostofgeneratinganaverageforeachLikertscale,withlossofinformationaboutspread.Intheresultingfigureaunanimousminimumnegativeresponseof‘notdesirable’or‘notfeasible’wouldberepresentedaszero%,whileaunanimousmaximumpositiveresponsewouldberepresentedas100%.Intermediatevaluesareweightedaccordingly.


43

Firstly,thischartshowsthatthereisawidevariationinrespondents’viewsofthedesirabilityofthevisions,whereasthereismuchgreateragreementontheirfeasibility.Thisimpliesthatthereis,tosomeextent,asharedunderstandingofthecapabilitiesofthetechnologieswhichareavailabletoimplementlearninganalytics,butalsoawidedisparityofviewsregardingthepurposesforwhichthistechnologyshouldbeused.

Secondly,somevisionsstrongdiscrepancybetweendesirabilityandfeasibility:

• Threevisionsareseenassimilarintheirdesirabilityandfeasibility(visions1,6&7).• Twovisionshaveasubstantialdegreeofdiscrepancybetweendesirabilityandfeasibility

(visions2&8).• Forthreevisionsthereisaverystrongcontrastbetweendesirabilityandfeasibility:

o Vision3:In2025,analyticsarerarelyusedineducationo Vision4:In2025,individualscontroltheirowndatao Vision5:In2025,opensystemsforlearninganalyticsarewidelyadopted.

Inallthreecasestherespondentsindicatethatthediscrepancyisdrivenbythemismatchbetweentechnicalcapability,ontheonehand,andsocialandpoliticalimplications,ontheother.InthecaseofVision3,theconcernisthatsocial,politicalandpedagogicfactorswillbringabouttheundesirableabandonmentoflearninganalytics.Inthecaseofvisions5and6,theconcernisthatinitiativeswhichcouldenablelearninganalyticstomakeapositivecontributiontoeducationandsocietywillbepreventedbysocialandpoliticalfactors.

Thisresultwarnsusagainstthinkingofthefutureoflearninganalyticssolelyintermsofthetechnicalconcernsofanalyticsmethodsandpedagogicalapplications.

4.2. Judgementsontheeightvisions,andtheirimplicationsInthissectionwediscussthemainconclusionsthatcanbedrawnfromresponsestotheindividualvisions,andthedriverswhichtheseimply.Wethenidentifythethreeareasofactionwhichcanaddressthesedrivers.

4.2.1. Vision1.In2025,classroomsmonitorthephysicalenvironmenttosupportlearningandteaching

Amajorityfoundthisvisionundesirable,butfeasible.Rejectionofthevisioncentredontheintrusivenessofdatagatheringandconcernsaboutprivacy.Thetechnologyisavailable,butthepedagogicapplicationisnotready.Thereissomedisagreementbetweenthesectorsandbetweeninvitedexpertsandweblinkrespondents.

Theprincipaldriversidentifiedinthisvisionare:• Policyonandregulationofprivacyanddataownership• Lackofclearconsensusonefficacyoflearninganalytics• Pedagogicapproachinstantiatedinlearninganalytics,andfittopedagogiccontext

4.2.2. Vision2:In2025,personaldatatrackingsupportslearningThemajorityofrespondentsfoundVision2tobeundesirable,butmostrespondentsalsofoundittobefeasible.Manyoftherequiredtechnologiesareperceivedasbeingcurrentlyavailable.Privacy


44

wasaconcern,butusers’controlofdatawasseenasapositiveaspect.Doubtswereexpressedaboutthepedagogicefficacyofthevision.

Theprincipaldriversidentifiedinthisvisionare:• Users’controlofdata• Pedagogicefficacyoflearninganalytics• Readyavailabilityoftechnology• Distrustoftheprofitmotiveinlearninganalytics

4.2.3. Vision3:In2025,analyticsarerarelyusedineducationVision3wasratedtheleastdesirableofallvisions,withovertwiceasmany‘veryundesirable’(1)scoresasanyothervision.Thisresultcontrastswiththemixedandoftenambivalentresponsestotheparticularvisionsoftheuseoflearninganalyticspresentedtorespondents.Thisshowsgreatfaithintherelevanceandutilityoflearninganalytics,butalackofclarityofexactlyhowitthetechniquesshouldbeused.Respondentswereevenlysplitonwhetherthisvisionwasfeasible,inthebiggestcontrastbetweendesirabilityandfeasibilityofanyvision.Thereis,therefore,realconcernamongparticipantsinlearninganalyticsthatitwillnotachieveitspotential.

Theprincipaldriversidentifiedinthisvisionare:

• Confidenceinthefutureoflearninganalytics• Demonstrationoftheopportunitypresentedbytheavailabilityofdata,andthepotential

pedagogicbenefitsofanalytics• Policiesandregulationstoensuredataprivacy

4.2.4. Vision4:In2025,individualscontroltheirowndataVision4wasratedasbeingverydesirable,andamajorityofrespondentsalsothoughtthatitwasfeasible.Thisresultcorrelateswiththeconcernforusers’privacyandcontrolofdataexpressedinmanyoftheothervisions.Thereisthereforeastrongconsensusinthepopulationconsultedthatitisethicallyessentialtoregulatecontrolofusersdatasoastostrengthentherightsofthedatasubject,andthatthisisalsoanecessarystepiflearninganalyticsistobewidelyacceptable.Thereweresomeminorityvoiceswhichpointedoutthatthiswouldleadtolostopportunities,andthateducationinstitutionsalreadycollectedalotofdataaboutlearners.

Theprincipaldriversidentifiedinthisvisionare:• Concernsaboutlackofregulationtoensureusercontrolofdata• Socialacceptanceofmonitoring• Demonstrationofpedagogicefficacy

4.2.5. Vision5:In2025,opensystemsforlearninganalyticsarewidelyadoptedVision5hadmore‘verydesirable’(4)ratingsthananyothervision.Thereisthereforeaverystrongconsensusamongtherespondentsthatanopenandstandardscompliantinfrastructureforlearninganalyticsisessential.Theyalsobelievethatthisisanachievablegoal,butthatitwillrequireappropriatefundingfromnationalbodiesandtheEuropeanCommission.Aplaceisseenforcommercialorganisationswithinopenanalytics.Therewereveryfewnegativecommentsrelatingtothisvision.


45

Theprincipaldriversidentifiedinthisvisionare:• Controlbyinstitutionsoftheirowninfrastructure• CommercialpressurefromeLearningproviders• Transparencyonthecollectionanduseofdata

4.2.6. Vision6:In2025,learninganalyticssystemsareessentialtoolsofeducationalmanagement

Vision6was,onbalance,positivelyperceivedinbothLikertscaleandfreetextresponses.However,respondentsfromtheschoolssectorwerelessenthusiasticthanothersectors,withmorerespondentsfindingit‘notdesirable’(1),than‘verydesirable’(4).Theyalsofounditlessfeasible.Thisindicatespotentialconflictintheimplementationofpredictiveanalyticsinschools.

Invitedexpertsassessedthescenarioasbeinglessfeasiblethandidrespondentstotheweblink,whichmayindicatethatthiscoreusecaseforlearninganalyticsmaybelesswellestablishedthanisgenerallyassumed.Thisalignswiththeconcernsexpressedbysomerespondentsthattheeffectiveimplementationofsuchsystemswillbemorecomplexthanisforeseen.

Theprincipaldriversidentifiedinthisvisionare:• Thedemonstrableeffectivenessoflearninganalyticsmethods• Socialacceptabilityofmonitoring• Thefitbetweenthepedagogyinstantiatedinlearninganalyticsmethods,andthepedagogic

context.• Policyonandregulationofethicsandprivacy

4.2.7. Vision7:In2025,mostteachingisdelegatedtocomputersVision7wastheleastpopularofallthepedagogicapproachestolearninganalyticsthatwerepresentedtorespondents.Objectionscentredontheproposalthat“Learnersnowspendmostoftheirtimeworkingwithanalytics-drivensystems,andtheroleofteachershasbeenreduced”,whichwasrejectedbymostrespondents,oftenstronglyso.Thisisconsistentwiththeresultsforothervisions,inwhichmanyrespondentsalsoplacegreatimportanceonface-to-facecommunicationwithteachers.Itisinterestingthatrespondentsfromtheworkplacewerestrongerintheiropinionthatthisapproachwasnotfeasible,eventhoughonemightexpectcomputerbasedtrainingtobebetterestablishedinthatsector.Thereweresurprisinglyfewreferencestoeconomicfactors,eventhoughthisisakeyaspectofthereplacementofteachersbymachines.

Theprincipaldriversidentifiedthisvisionare:• Pedagogicvisioninstantiatedinlearninganalytics,anditsfitwiththepedagogiccontext.• Lackofconsensusontheeffectivenessoflearninganalyticsinterventions• Policychecksonintrusiveanalyticssystems

4.2.8. Vision8:In2015,analyticssupportself-directedlearningVision8wasratedthemostdesirableofthepedagogicscenarioswhichwerepresentedtorespondentsthroughthevisions.Inbroadterms,thisisthekindofpedagogythattherespondentswouldliketoseeadopted,buttheywerealittlemorecautiousintermsoffeasibility.Inalignmentwithothervisionsthereisaperceivedgapbetweenthepedagogicwishesoftherespondents,andthelikelydevelopmentofthefield.


46

Respondentsfrom‘schools’andthe‘workplace’weresomewhatlessenthusiasticthan‘highereducation’.Similarlytheinvitedexpertsweremorepositivethanrespondentstotheweblink,inbothdesirabilityandfeasibility.Thisdistinctioninvitesreflectionastowhetherasocio-technicaleliteisdrivinganagendawhichmaynotbeentirelywelcometothepeoplewhowillbeusingthesystems.Thelackofconsensuswasclearerinthefreetextresponses,where,despitethepopularityofthepedagogicapproachdescribed,manycommentsquestioneditsefficacy.Thisvisionraisesasyetunresolvedissuesofinteroperability,privacyandcertification.

Theprincipaldriversidentifiedinthisvisionwere:• Pedagogicapproachinstantiatedinthelearninganalyticsmethods,andfitwithpedagogic

context• Lackofconsensusonpedagogicmethodsforlearninganalytics• Interoperability• Policyonprivacyandcertification

4.3. TherangeofthemesinformingthejudgementsonthevisionsTable3showsthenumberoftimeseachthemecodewasappliedtothecorpus,togetherwiththekeywordsidentifiedinthecodedtexts.

Table3:Summarytableoftheapplicationofcodes,withkeywords

Theme No.codes

Keywords

Pedagogy 355 Educators,learn,learner,learning,pedagogy,teach,teacher,teaching.Power 313 BigBrother,datacontrol,empowerment,humanrights,law,misuse,

policy,surveillance.Complexity 238 Barrier,challenge,social,technology,understanding.Validity 187 Assumptions,education,generalizability,learning,reliability,researchPrivacy 180 Bigbrother,control,power,regulation,surveillance.Regulation 169 Government,guides,law,laws,legislation,legislature,policies,policy,

protocol,protocols,regulation,regulations,rules.Ethics 132 Abuse,context,culture,exploitation,personalisation,policy.Experience 109 Technology,communicationbarriers,failure,success,commercial

pressure,policies.Affect 102 Motivation,perception,transparency,volition.Cost 92 Budgetsavings,businessmodels,commercialmodels,costs,economic

models,fundingmodels,governmentfunding,marketplace,value.Alienation 91 Automation,communication,dystopia,humanity,intrusion,policy,

privacy,resistance,society.Standards 79 Standards,standardization,interoperability,API,IMS,LTI,Caliper,

ExperienceAPI,xAPI,TinCan,SC36.Temporality 64 Duration,initialstate,just-in-time,rate,speed,sufficiency,timescale.

Total: 2111

Thecodingofthecorpusoffreetextresponsesindicatesthatthetwokeyissueswhichconcerntherespondentsare:

• thepedagogicaleffectivenessand/orappropriatenessoflearninganalyticsinterventions• thesocio-politicalimplicationsofdatagatheringandoflearninganalyticsmethods.


47

Thesefindingscomplementtheanalysisoftheindividualvisions,andlargelyconfirmtheconclusionswhichcanbedrawnfromtheindividualvisions,aswenowdiscuss.

4.4. OverallfindingsInthissectionwedrawtogethertheresultsoftheLikertscaleanalysis,theanalysisbyvisions,andthethematicanalysisofthecorpusoffreetextresponses.Foreachofthevisionsthedriversidentifiedintheresponsestothevisionswererelatedbytheteamtopolicyinterventionswhichwouldbeconsistentwiththeaspirationsforlearninganalyticsexpressedbytherespondents.

Weidentifysixprincipalfindings:

Finding1:AquestionmarkovertheprospectsforlearninganalyticsachievingitspotentialTheresponsetoVision3:Learninganalyticsarerarelyusedineducationwas,asonemightexpectfromthispopulation,aresoundingrejection.Therespondentsfeltstronglythatlearninganalyticshadalottocontributetoeducation,andthattheopportunitytoreapthebenefitsshouldnotbelost.However,thishighlyundesirableoutcomewasseenbymanyasbeingfeasible,withaslightmajorityviewingitasbeingfeasibleorveryfeasible.Learninganalyticsisthereforenotseenasanunstoppabletrend,butasanapproachwithgreatpotentialwhichisstillatanearlyandsensitivestageofitsdevelopment.

Thistrendwasalsoseenintheresponsestoothervisions.Indeedinonlyonevisionisthereagoodmatchbetweendesirabilityandfeasibility.ThisisVision6:Learninganalyticssystemsareessentialtoolswhichisseenquitepositivelyinboth.IncontrastVisions4,5,and8areverypositivelyratedfordesirability,butrespondentsaremuchlessoptimisticthattheyarefeasible.Theconverseistrueforvisions1,2,3and7,whichratedasclearlyundesirable,butrespondentsareconcernedthattherearerealprospectsthattheywillcomeabout.Thisisindicativeofalearninganalyticscommunitywhichhasseriousdoubtsaboutthedesirabilityofmanyofthepotentialscenariosforlearninganalytics,andhaslowlevelsofconfidencethatpositiveoutcomeswillbeforthcoming.

Thedriversforlearninganalyticsassociatedwiththisresultare:

• Confidenceinasuccessfulfutureforlearninganalytics• Theopportunitypresentedbytheavailabilityofdataandthepotentialpedagogicbenefitsof

analytics• Policiesandregulationstoensuredataprivacy

Therecommendedactionsalignedwiththesedriversare

• Providesupportforpilotingandthedemonstrationofpedagogicbenefitsfromlearninganalytics

• Providesupportforresearchintotheefficacyoflearninganalytics• Developalternativecertificationprocesses• Ensuretheparticipationofhumansinteaching• Ensurelearninganalyticsisinformedbypedagogy• Researchandpromoteappropriatepedagogiesforlearninganalytics• Formulateandimplementationofpoliciesandregulationstoensuredataprivacy

• Institutionsshouldcreateandapplyprotocolsandpoliciestogovernethicsandprivacy(x4)


48

• Regulatorybodiesshouldacttogoverndatacollectionanduse,andenforcedataprivacypolicyintooling

Finding2:PoliciesandinfrastructureTheresponsestoVision4:In2025,individualscontroltheirowndatarateditasbeingverydesirable,andamajorityofrespondentsalsothoughtthatitwasfeasible.Thereisastrongconsensusthatitisethicallyessentialtoregulatecontrolofusersdatasoastostrengthentherightsofthedatasubject,andthatthisisalsoanecessarystepiflearninganalyticsistobewidelyacceptable.Thereweresomeminorityvoiceswhichpointedoutthatthiswouldleadtolostopportunities,andthateducationinstitutionsalreadycollectedalotofdataaboutlearners.Thisresultisalsoatoddswithcurrentpracticeinlearninganalytics,andinfluentialcurrentcodesofpracticepublishedbyJiscandOpenUniversityUK.

TherewasasimilarlystrongconsensusforVision5,indicatingthatanopenandstandardscompliantinfrastructureforlearninganalyticsisessentialinordertomakeprogress.Therewerealmostnodissentingvoicestothisvision.

Theseresultsrelatetotwostrongdriversforthefutureoflearninganalytics:

• Availabilityofasharedopeninfrastructureforlearninganalytics• Thecreationofastrongregulatoryframeworktogoverndataownership,collectionanduse.

Thesedriversarealignedwithrecommendedactionsforgovernments,agenciesandtheCommissiontosupport

• Institutionsshouldcreateandapplyprotocolsandpoliciestogovernethicsandprivacy• Regulatorybodiesshouldacttogoverndatacollectionanduse,andenforcedataprivacy

policyintooling• Developofcommondatamodels,specifications,standardsandpolicies.• Developanopeninfrastructure.Respondentsmentionedpotentialpartnersinthiswork,

includingApereoandSoLAR.• Supportinternationalandnationalcollaborationbetweeninstitutions

Finding3:AconsensusonpedagogyIncodingthefreetextresponses,‘pedagogy’wasthemostfrequentlyusedtag,byasubstantialmargin(seeTable4:Numberofapplicationsofcodesinthe8visions,below).Inbroadtermsthepedagogicapproacheswhichwerefavouredbymostrespondentsintheircommentscouldbecharacterisedasconstructivistandteacherled,ratherthanbeingfocusedoncompetencesandmasteryofcontent.Doubtwasalsoexpressedaboutthemetricsusedtoassessteachingandlearning.Therewasaconsensusthatitisessentialforhumanbeingstoremainatthecentreoftheteachingprocess,andthatlearninganalyticsshouldnotimplyautomationofteachingandlearning.The‘pedagogy’tagcanbeassociatedwiththe‘validity’tag(thefourthmostused),whichwasusedtoindicatethosecommentswhichquestionedthepedagogiceffectivenessoflearninganalyticsinterventions,orstressedthatthisshouldbedemonstratedanddisseminated.Togetherthe‘pedagogy’and‘validity’tagsaccountedformorethanaquarterofthecodesapplied.

ThistrendwasconfirmedbytheresponsetoVision7,whichwastheleastpopularofallthepedagogicapproachestolearninganalyticsthatwerepresentedtorespondents.Objectionscentred


49

ontheproposalthat“Learnersnowspendmostoftheirtimeworkingwithanalytics-drivensystems,andtheroleofteachershasbeenreduced”,whichwasrejectedbymostrespondents,oftenstronglyso.Thisisconsistentwiththeresultsforothervisions,inwhichmanyrespondentsalsoplacegreatimportanceonface-to-facecommunicationwithteachers.Itisinterestingthatrespondentsfromtheworkplacewerestrongerintheiropinionthatthisapproachwasnotfeasible,eventhoughonemightexpectcomputerbasedtrainingtobebetterestablishedinthatsector.Thereweresurprisinglyfewreferencestoeconomicfactors,eventhoughthisisakeyaspectofthereplacementofteachersbymachines.ThemostpositivelyviewedpedagogicscenariowasthatinVision8,inwhichcollaborativeinquirybasedlearningwassupportedbyanalytics.

Theconsensuswasnotunanimous,andsomerespondentsstressedthepotentialbenefitsthatautomatedteaching.

Theseresultsrelatetothefollowingdrivers

• Alignmentbetweenthepedagogiesinstantiatedinlearninganalyticsmethodsandthecontextinwhichtheyaredeployed

• Enthusiasm,orevenjustacceptance,fromtheteacherswhowillbeusinglearninganalyticssystems.

Thesedriversalignwiththefollowingactions:

• Fundandcarryoutresearchintothepedagogicuseoflearninganalytics.• Buildtrustandcollaborationbetweeneducationalistsandtechnologists• Providesupportforresearchintotheefficacyoflearninganalytics,andintothe

appropriatepedagogiesforlearninganalytics• Providesupportforpilotingandthedemonstrationofpedagogicbenefitsfromlearning

analytics

Finding4:Power,ethics,anddataownershipIssuesofsocialandpoliticalpower,ethics,andownershiparecentralfactorstothefutureoflearninganalytics.Theclusterofcodes‘power’(thesecondmostappliedcode),‘privacy’,‘regulation’and‘ethics’togetheraccountedformorethanathirdofthecodesapplied.Whilethecodescanbeclearlydistinguished,buthaveacommonthreadintheresults.‘Power’wasrelatedtoboththeexerciseofpowertoobtaindata,andtheuseofdataanalysistoreinforcepower.‘Privacy’and‘regulation’bothprincipallyrelatedtothedesireamongrespondentsfordatasubjectstohavemorecontroloverthedatawhichtheygenerate,andtheusestowhichitisput.Ethics,similarly,veryoftenrelatedtothemanagementofdata.Togetherthesecodesindicateawidespreadconcernthatthebenefitsofanalyticsmaybethreatenedbyareactionagainstintrusivecollectionofdata,andinappropriateuseoftheresultsofanalysis.

Theseresultsofthecodingarereflectedintheresponsestothevisions,inwhicheveryvisionreceivedcommentsfromusersthatmentionedtheseissues.

Theseresultsrelatetothefollowingdrivers:

• Socialandpoliticalconsensusontheappropriatecollectionanduseofdata.• Trustintheresponsiblecollectionanduseofdata.


50

Theactionsalignedwiththesedriversare

• Developmentofpoliciesandsystemsthatsupportthecontrolofdatabydatasubjects.• Promotionoftransparencyandaccountabilityinthecollectionanduseofdata• Institutionsshouldcreateandapplyprotocolsandpoliciestogovernethicsandprivacy• Regulatorybodiesshouldacttogoverndatacollectionanduse,andenforcedataprivacy

policyintooling

Finding5:Disagreementbetweensectors,andbetweendifferentgroupsofrespondentsThepopulationforthestudywasdividedintothreesectors(school,workplaceandhighereducation),andtwosourcesofrespondents(invitedexpertsandthosewhorepliedtotheweblink).Thedifferencesbetweenthesefivegroupsarenotverylarge,butareworthattendingto.Togivesomeexamples:

• Vision1isclearlyaimedattheschoolclassroom,andtheresultsshowthatrespondentsfromschoolsweresubstantiallymorepositiveintheirassessmentofthevisionthanwerethosefromhighereducationortheworkplace.Almosttwiceasmanyrespondentstotheweblinkfoundthevisiontobe‘veryundesirable’thandidinvitedexperts,

• ForVision5forboththeSchoolsectorandtheWorkplacesectorthelargestnumberofrespondentsindicatedthatthevisionwasratherinfeasible(2).InHigherEducation,ontheotherhandthelargestnumberofrespondentsthoughtitfeasible(3),withalargenumberassertingthatitwasveryfeasible(4).ThefactthatthereweremanymorerespondentsfromHigherEducationthanothersectorsmeansthatthisviewprevailsintheoverallresults.Butitwouldbewisetotakenoteofthegreaterbarriersthatareperceivedtoopenlearninganalyticsinschools.

• Vision6wasseenasmarkedlylessdesirableandfeasiblebyrespondentsfromtheschoolssector.Similarly,respondentswhoaccessedthesurveythroughtheweblinkfoundthevisiontobesubstantiallymorefeasiblethandidtheinvitedexperts.

Thesedifferencesuggestthattheremaybeagapinunderstandingbetweenexpertsandpractitionersontheground.Thepossibilityshouldbeconsideredthatasocio-technicaleliteisproposingsystemsandmethodsthatarenotentirelywelcomedbypractitionersinthefield.Itshouldberememberedthattherespondentswerealleitherselectedorself-selectedenthusiastsforlearninganalytics,andthiseffectmightbeconsiderablylargerinthewidercommunity.

Thedriverindicatedbythisfindingis:

• Alignmentoftheresearchanddevelopmentinterestsofthelearninganalyticscommunitywiththeinterestsandprioritiesofthosewhosefocusisthedaytodayactivitiesofteachingandlearning

Theactionalignedwiththisdriveris

• Supportresearchintotherealityoflearninganalyticsincontext• Fundcoordinationactivitieswhichreachacrossthecommunitiesofpractice


51

Finding6:TechnologyRespondentsseemedsatisfiedwiththetechnologywhichisalreadyavailable,andconfidentthatitwillcontinuetodevelop.

Thisresultrelatestothefollowingdriver:

• Continuingtechnologicalinnovation

Theresultssuggestthatnopolicyinterventionisrequiredtomaintainthepaceofinnovationintheunderlyingtechnologyusedbylearninganalytics.


52

5. FutureStepsWehavefounditfascinatingtoengagewiththelearninganalyticscommunityincarryingoutthisstudy.Webelievethatpolicymakerswillfindtheresultstobeveryrelevant,andthatthecommunitywhichhascontributedtothestudywillbeveryinterestedinseeingthereflectionofitsjudgementsonthefutureoflearninganalytics.

Wealsobelievethatthedatawhichwehavegeneratedisarichresourcewhichcanbefurthermined.Inparticulartherearestatisticalmethodswhichcanbeappliedtothedatawhichmayresultinadditionalinsight,inparticularinthecomparisonoftheviewsofdifferentgroupsofrespondents.Furtheranalysisofthecorpusoffreetextmayalsobeworthwhile,perhapsusingstatisticaltools.

WehavealreadyheldanumberofconsultationswithstakeholderconcerningtheVisionsoftheFuturestudy.ThisworkwillbecontinuedatLAK,whereapanelabouttheVisionsoftheFuturestudyhasbeenaccepted.Theseactivitiesareimportantintwoways.Firstly,theyreflectbackjudgementsofthefutureoflearninganalyticstothecommunitythatgeneratedthem,providingtheopportunitytotestthatthepopulationincludedinthestudyisindeedrepresentativeofthefullrangeofactorsinvolvedinlearninganalytics.Secondly,theconclusionsareatahigherlevelofabstractionthanthevisionswhichstimulatedtheresponses,andtheyproposeaninterpretationoftheimplicationsoftheresponses.Itisimportanttocontrastthisinterpretationwiththejudgementsofthelearninganalytics,bothtoexposeinconsistenciesbetweenparticipants’judgementsaboutlearninganalyticsindifferentcontexts,andalsotocritiquethestudies’interpretationofthedata.

FollowingtheLAKconferencetheteamintendedtobuildontheseconsultations,andonfurtheranalysisofthecorpus,inthewritingofajournalpaperwhichwillsetoutthefinalconclusionsofthisworkwithintheLACEproject.Thepaperwillbereadyforsubmissionbythecloseoftheproject.


53

ReferencelistBuckinghamShum,S.,2015.WritinganalyticsandtheLAK15“StateoftheField”panel(20March

2015).Simon.BuckinghamShum.net.Availableat:http://simon.buckinghamshum.net/2015/03/writing-analytics-lak15-panel/[AccessedFebruary5,2016].

Goodyear,P.etal.,2004.AdvancesinResearchonNetworkedLearning,KluwerAcademicPublishers,Norwell,Massachusetts.

Helmer-Hirschberg,O.,1967.AnalysisoftheFuture:TheDelphiMethod,Availableat:http://www.rand.org/content/dam/rand/pubs/papers/2008/P3558.pdf.

Horn,M.,2014.inBloom’sCollapseOffersLessonsForInnovationInEducation.Forbes.

Kurzweil,R.,2005.TheSingularityisNear,London:Duckworth.

RANDCorporation,TheDelphiMethod.Availableat:http://www.rand.org/topics/delphi-method.html[AccessedFebruary5,2016].

Sclater,N.,2014.Codeofpracticeforlearninganalytics,Availableat:http://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf.

Sharples,M.,2000.Thedesignofpersonalmobiletechnologiesforlifelonglearning.Computers&Education,34(3-4),pp.177–193.

Siemens,G.,2005.Connectivism:Alearningtheoryforthedigitalage.InternationalJournalofInstructionalTechnologyandDistanceLearning,2(1).

Suthers,D.&Verbert,K.,2013.Learninganalyticsasa“middlespace.”InDanSuthersetal.,eds.ProceedingsoftheThirdInternationalConferenceonLearningAnalyticsandKnowledge(LAK’13).ACM,NewYork,NY,USA,pp.1–4.

TheOpenUniversity,2014.PolicyonEthicaluseofStudentDataforLearningAnalytics.,p.11.Availableat:http://www.open.ac.uk/students/charter/sites/www.open.ac.uk.students.charter/files/files/ecms/web-content/ethical-use-of-student-data-policy.pdf.

Turoff,M.,1970.TheDesignofaPolicyDelphi.TechnologicalForecastingandSocialChange,2(2),pp.149–171.

Turoff,M.&Linstone,H.A.,2002.ThePolicyDelphi.,2(2),pp.80–96.

USDepartmentofEducationOfficeofEducationalTechnology,2015.EdTechDeveloper’sGuide:APrimerforSoftwareDevelopers,StartupsandEntrepreneurs,Availableat:http://tech.ed.gov/files/2015/04/Developer-Toolkit.pdf.


54

Appendices

Appendix1.GraphicalrepresentationoftheLikertscaleresultsChartswerepreparedshowingrespondentsresponsestotheLikertscalequestionsonthefeasibilityanddesirabilityofeachvision.Thechartsareasfollows:

1.1Personalinformation

Figure10:Respondentsknowledgeoflearninganalytics

Figure11:Respondentsbysector


55

1.2 DesirabilityandFeasibility:alldata

Figure12:Alldesirabilitydata

Figure13:Allfeasibilitydata

0

10

20

30

40

50

60

70

Vision1(N=53)

Vision2(N=48)

Vision3(N=62)

Vision4(N=54)

Vision5(N=58)

Vision6(N=55)

Vision7(N=53)

Vision8(N=59)

Percen

t

Feasibility

1.Notfeasible

2

3

4.Veryfeasible


56

1.3Desirabilitybysector

Figure14:Desirabilitydatachartedbysector


57

1.4 Feasibilitybysector

Figure15:Feasibilitydatachartedbysector

0

10

20

30

40

50

60

70

Vision1(N=14)

Vision2(N=19)

Vision3(N=22)

Vision4(N=18)

Vision5(N=20)

Vision6(N=16)

Vision7(N=15)

Vision8(N=21)

Percen

t

Feasibility/School

1.Notfeasible 2 3 4.Veryfeasible

05

101520253035404550

Vision1(N=12)

Vision2(N=13)

Vision3(N=16)

Vision4(N=15)

Vision5(N=15)

Vision6(N=13)

Vision7(N=16)

Vision8(N=19)

Percen

t

Feasibility/Workplace


0

10

20

30

40

50

60

Vision1(N=47)

Vision2(N=40)

Vision3(N=54)

Vision4(N=47)

Vision5(N=50)

Vision6(N=49)

Vision7(N=47)

Vision8(N=50)

Percen

t

Feasibility/HE



58

1.5 Desirabilitybysourceofrespondents

Figure16:Desirabilitydatachartedbyrespondents(respondedtodirect/general)invitation


59

1.6 Feasibilitybysourceofrespondents

Figure17:Feasibilitydatachartedbyrespondents(respondedtodirect/general)invitation

0

10

20

30

40

50

60

70

Vision1(N=25)

Vision2(N=22)

Vision3(N=33)

Vision4(N=28)

Vision5(N=30)

Vision6(N=30)

Vision7(N=30)

Vision8(N=34)

Percen

t

Feasibility

1.Notfeasible

2

3

4.Veryfeasible

Respondentsinvitedbyemail

0

10

20

30

40

50

60

Vision1(N=28)

Vision2(N=26)

Vision3(N=29)

Vision4(N=26)

Vision5(N=28)

Vision6(N=25)

Vision7(N=23)

Vision8(N=25)

Percen

t

Feasibility

1.Notfeasible

2

3

4.Veryfeasible

Respondentsviaweblink


60

Appendix2:CodingsummarychartThischartshowshowmanytimeseachofthe13themesappearedinthecodingoftheSurveyMonkeydata.Thecodingispresentedtoshowthecodingfordesirability,feasibilityandactionsforeachoftheeightVisionsthatwerepresentedforcomment.

Table4:Numberofapplicationsofcodesinthe8visions

AffectAlienationComplexityCosts Ethics Experience Pedagogy Power Privacy Regulations Standards TemporalValidity

Vision1 Learnersaremonitoredbytheirlearningenvironments

Desirability 8 12 6 2 4 4 24 15 15 1 1 4 12

Feasibility 4 2 17 6 2 13 9 5 4 3 0 3 10

Actions 11 2 10 6 8 1 12 11 9 11 2 3 13

Vision2Learners’personaldataaretracked

Desirability 6 3 10 2 15 0 26 13 5 0 0 1 11

Feasibility 3 3 19 0 5 2 20 3 1 2 0 2 8

Actions 0 2 8 1 7 1 11 8 5 6 0 0 7

Vision3 Analyticsarerarelyused

Desirability 9 2 2 0 9 9 13 9 8 8 0 6 8

Feasibility 4 2 5 2 7 12 8 13 3 8 0 3 2

Actions 2 4 4 2 5 4 9 13 6 18 3 0 2

Vision4 Learnerscontroltheirowndata

Desirability 1 0 5 2 9 2 19 30 26 9 0 1 5

Feasibility 3 3 18 4 2 0 8 17 35 13 8 2 2

Actions 1 2 14 7 4 3 6 12 38 22 4 3 0

Vision5 Opensystemsarewidelyadopted

Desirability 13 2 7 6 7 7 11 23 3 3 18 7 9

Feasibility 3 1 17 15 3 7 6 20 4 3 9 8 3

Actions 6 0 4 7 6 4 7 23 4 12 13 2 4

Vision6 Learninganalyticsareessentialtools

Desirability 5 6 5 4 5 2 20 8 2 0 0 3 16

Feasibility 1 5 14 4 2 11 4 8 3 6 1 3 11

Actions 0 6 11 4 7 1 9 11 4 10 2 0 8

Vision7 Analyticshelplearnersmaketherightchoices

Desirability 13 15 5 2 9 7 39 14 2 2 3 3 10

Feasibility 7 5 7 3 3 11 19 11 1 2 4 5 8

Actions 2 5 4 3 6 1 13 14 1 5 3 2 10

Vision8 Analyticshavelargelyreplacedteachers

Desirability 0 6 12 3 4 0 33 11 0 0 0 1 12

Feasibility 0 1 22 5 1 6 5 13 0 5 2 2 11

Actions 0 2 12 2 2 1 24 8 1 20 6 0 5

TOTALS 102 91 238 92 132 109 355 313 180 169 79 64 187

AffectAlienationComplexityCosts Ethics Experience Pedagogy Power Privacy Regulations Standards TemporalValidity


61

Appendix3:VisionsoftheFutureThisAppendixsetsouttheeightscenariosthatwereusedinthePolicyDelphistudy.Thescenariosweredevelopedbythefiveauthorsofthisreport,eachofwhomhaswideexperienceofthefieldoflearninganalytics.TheywerethenrevisedandextendedinconsultationwiththewholeLACEteam,inordertobringinotherexpertiseandperspectives.Thevisionswereintendedasprovocationsthatwouldproducereactions;theywerenotintendedtoreflecttheviewsofprojectmembersoroftheLACEprojectasawhole.

Eachoftheeightscenariosbeginswithashortsummaryandthenbrieflycontraststhesituationin2015withtheenvisagedscenarioin2025.Thebodyofthescenariosetsoutthisvision,andsomeofitspossibleimplications,inmoredetail.

3.1 Vision1:In2025,classroomsmonitorthephysicalenvironmenttosupportlearningandteaching

In2015,learninganalyticsweremainlyusedtosupportonlinelearning.By2025,theycanbeusedtosupportmostteachingandlearningactivities,whereverthesetakeplace.Furniture,pens,writingpads–almostanytoolusedduringlearning–canbefittedwithsensors.Thesecanrecordmanysortsofinformation,includingtilt,forceandposition.Videocamerasusingfacialrecognitionareabletotrackindividualsastheylearn.Thesecamerasmonitormovements,andrecordexactlyhowlearnersworkwithandmanipulateobjects.Allthisinformationisusedtomonitorlearners’progress.Individualsaresupportedinlearningawiderangeofphysicalskills.Teachersarealertedtosignsofindividuallearner’sboredom,confusion,anddeviationfromtask.Teachersandmanagersareabletomonitorsocialinteractions,andtoidentifywheretheyshouldnurturesocialisationandcooperativebehaviour.

3.2 Vision2:In2025,personaldatatrackingsupportslearningIn2015,peoplewerebeginningtoweardevicessuchasheart-ratemonitorsandrun-trackersastheywentabouttheirdailylives.By2025,sophisticatedsensorscangatherpersonalinformationaboutfactorssuchasposture,attention,rest,stress,bloodsugar,andmetabolicrate.Peoplecollectthisinformationabouttheiractivities,andfeeditintoprogrammesoftheirchoicethatproviderecommendationsonhowtoactinwaysthatimprovetheirlearning.Learnerscandownloadthestatisticsanddatathatareassociatedwithsuccessfullearninginacertainarea.Aligningpersonaldatawiththese‘ideal’setsisclaimedtohelppeopletomasterskillsasdiverseasswimming,driving,carryingoutsurgeryandpassingexaminations.Academicstarssellprogrammesusingthisdatatooptimiselearningfordifferentagesandcourses.Businessgurusmarketsimilarprogrammesfortopicssuchaspresentationskillsandworkloadmanagement.Somelearnerscreateandsharetheirowndataanalysisprogrammes,whichproviderecommendationsthatoftenincludetheconsumptionofhigh-energyfoodsandstimulants.Themajorityofhigh-schoolanduniversitystudentsfollowself-monitoringprogrammes,anddiscussthemeritsoftheseonsocialmedia.

3.3 Vision3:In2025,analyticsarerarelyusedineducationIn2015,manypeoplehopedthatanalyticswouldbeabletoimproveteachingandlearningandtheenvironmentswherethesetakeplace.However,in2025,itisclearthattherearemanyproblems.Coursesthatareautomatedbyanalyticsareseenasinferior,andlearnershaverealisedthattheycangamethesystem.Therehavebeenmajorleaksofsensitivepersonaldata,anditisclearthat,evenwherethishasnothappened,manycompanieshavemisusedthedatageneratedbytheir


62

analytics.Manygovernmentshaveruledthatindividualsarethesoleownersofthedatatheygenerate.Alluseofdataforeducationalpurposesnowhastobeapprovednotonlybythelearnerbutalsobynewinspectorates.Inpracticethishasmeantthatuseofanalyticsisrestrictedtosummativeassessmentcarriedoutbygovernmentagencies.Aconsensushasemergedineducationalpolicy:themoveawayfromlearninganalyticsisnotonlyethicallydesirableitisalsoeducationallyeffective.

3.4 Vision4:In2025,individualscontroltheirowndataIn2015,itwasnotclearwhoownededucationaldata,anditwasoftenusedwithoutlearners’knowledge.By2025,mostpeopleareawareoftheimportanceandvalueoftheirdata.Learnerscontrolthetypeandquantityofpersonaldatathattheyshare,andwithwhomtheyshareit.Thisincludesinformationaboutprogress,attendanceandexamresults,aswellasdatacollectedbycamerasandsensors.Learnerscanchoosetolimitthetimeforwhichaccessisallowed,ortheycanrestrictaccesstospecificorganisationsandindividuals.Thetoolsformakingthesechoicesareclearlylaidoutandeasytouse.Inthecaseofchildren,datadecisionsaremadeinconsultationwithparentsorcarers.Iftheydonotengagewiththesetools,thennodataissharedandnobenefitsgained.Mosteducationalinstitutionsrecognisethisasapotentialproblem,andruncampaignstoraiseawarenessoftheboththerisksofthoughtlessexposureofdata,andthebenefitstolearnersofinformedsharingofselectededucationaldata.

3.5 Vision5:In2025,opensystemsforlearninganalyticsarewidelyadoptedIn2015,companiesproducedarangeoflearninganalyticstools,usingdifferentapproachesandstandards.Thealgorithmsandmodelsthatcompaniesuseareoftenprotectedasintellectualproperty.By2025,the‘openlearninganalytics’establishedbytheOpenLearningAnalyticsFoundationhasmadeamorejoined-upapproachpossible.EducationalorganisationsseelearninganalyticsasacentralelementoftheirITprovision.Theydemandcontroloverthesetools,howtheyrunandwhattheyareusedfor.Thetoolstheyselect,althoughtheycomefromdifferentproviders,useopenalgorithmsandsharedataaccordingtoanagreedsetofstandardsthatfacilitatetransparencyandindependentvalidation.Asetofwell-tested,accessibleandstandardisedvisualisationmethodsiscommonlyused,sothatlearnersandteacherscanconfidentlyusearangeoftools.Institutionscaneasilyworkwitharangeofproviderstodesignlearninganalyticssystemsthatsupporttheirstrategicvision.

3.6 Vision6:In2025,learninganalyticssystemsareessentialtoolsofeducationalmanagement

In2015,companieswerebeginningtodevelopsystemstorecommendresourcesandtopredictoutcomes.By2025,thesesystemsarehighlydeveloped.Awiderangeofdataaboutlearnerbehaviourisusedtogenerategoodquality,real-timepredictionsaboutlikelysuccess.Learners,teachers,managersandpolicymakersallhaveaccesstoliveandaccurateinformationabouthowwellalearnerislikelytodo.Learnersandteachersplantheirworkonthebasisofreliabletoolsthatcanproducedetailedandpersonalisedrecommendationsaboutwhatshouldbedonetoachievethebestlearningoutcomes.Agrowingindustryoffersservicestoinstitutionsandindividuals,advisingonhowtorespondtopredictionsgeneratedbyanalytics,andhowtotakeappropriateactioninthelightofrecommendations.Accuratepredictiveinformationenablesmanagersandpolicymakerstoexpandorcontractlearningprovisionbeforesuccessorfailureisevident:youdon’thavetowaittoseeifacourseisboomingorfailing,withfundingchangeshappeningquickly.


63

3.7 Vision7:In2025,mostteachingisdelegatedtocomputersIn2015,peoplewerebeginningtoassembledatasetsthatcouldrepresentlearner’sactivities.By2025,theseareusedonalargescaleinteaching,andthishasledtothedevelopmentofenormousdatasetscontaininginformationabouthundredsofthousandsoflearners.Analysingindetailtheprogressofsuchawidevarietyoflearnershasmadeitpossibletoprovidereliableevidence-basedrecommendationsaboutthemostsuccessfulroutestolearning,aswellasidentifyingthelearningmaterialsandapproachesthataremostsuitableforeachindividualateachpointintheirprogress.Theserecommendationsarebetterinformedandmorereliablethanthosethatcanbeproducedbyeventhebest-trainedhumans.Learnersnowspendmostoftheirtimeworkingwithanalytics-drivensystems,andtheroleofteachershasbeenreduced.Theevidencegeneratedbytheuseofthesesystemsdriveseducationpolicy.

3.8 Vision8:In2025,analyticssupportself-directedautonomouslearningIn2015,learnersineducationalinstitutionsandinbusinesseshadtofollowacurriculumdevelopedbyothers.In2025,theycreategroupsthatworktogethertodecidetheirlearninggoalsandhowtoachievethese.A‘LearningTrajectorySystem’usesanalyticstosupportinformationexchangeandgroupcollaborations,andlearnersreceivesupportfrommentors,ratherthanteachers.Activitytowardsalearninggoalismonitored,andanalyticsprovideindividualswithfeedbackontheirlearningprocess.Thisincludessuggestions,includingpeerlearnerstocontact,expertstoapproach,relevantcontent,andwaysofdevelopinganddemonstratingnewskills.Formativeassessmentisusedtoguidefutureprogress,takingintoaccountindividuals’characteristics,experienceandcontext,replacingexamsthatshowonlywhatstudentshaveachieved.Textsandotherlearningmaterialsareadaptedtosuittheculturalcharacteristicsoflearners,revealedbyanalysisoftheirinteractions.Asaresult,learnersarepersonallyengagedwiththeirtopics,andaremotivatedbytheirhighlyautonomouslearning.Thecompetencesthattheydeveloparevaluableinasocietyinwhichcollectionandanalysisofdataarethenorm.Thereisalsoconvergencebetweenthelearningactivitiesoftheeducationsystemandthemethodsusedbyemployeestodeveloptheirknowledgeandskills.

Appendix4:Informationforparticipants

4.1 Pre-surveyOnenteringthesurvey,theparticipantswerepresentedwiththefollowinginformation:

WelcometoLACE'svisionsofthefuturestudy

Theaimofthisstudyistoconsiderviewsonthefutureoflearninganalyticsintermsofwhatisdesirable,whatisfeasibleandtheobstaclestomakingwhatisdesirablehappen.TheLACEProjecthascreatedeightvisionsofthefutureoflearninganalytics.Eachvisionillustratesadifferentaspectofthewaythatlearninganalyticscouldtransformourlivesbytheyear2025.Thestudyintendstodrawoutdifferencesofperceptionandvisionfromyourselfandawidegroupofstakeholders,researchersandpractitionerexpertsonlearninganalytics.Inthequestionnaireyouwillbepresentedwithashortvisionstatement,andaskedifyoufindthisvisiontobedesirableandfeasible,andwhatwouldbeneededtobringitabout.


64

Therearesomequestionsforwhichananswerisrequired,markedwithanasterisk(*).Ifyoufeelyoudonothavetheknowledgetoprovideaninformedresponsepleaseselecttheanswer"Idonotfeelqualifiedtorespond".Togiveyoutimeforfullconsiderationofyourfreetextresponses,youwillinitiallybeaskedtogiveyourviewsonthreevisions.Whenyouhavefinishedthese,youhavetheoptiontostop,oryoucancontinue,and,ofcourse,wewillbeverygratefulifyoucandothis.Weestimatethatitwilltakeyou60minutestorespondtoalleightvisions.Youcanstopatanytimeandreturntorestartatthepointyoufinished.Ifyoufinishalleightvisions,youwillbeinvitedtoaddyourownvisions,ifyouthinkthatanythingismissing.TheresultsofthestudywillbepublishedasaLACELearningAnalyticsReviewpaper.Theinformationwhichyouprovideinthissurveywillbeanonymisedbeforepublication.Wearehowever,requestingthatyouprovidealittleinformationaboutyourknowledgeofthearea.Wewouldalsobegratefulfortheopportunitytocontactyouifwehaveanyqueriesorfurtherquestionsinrelationtoyourcomments,andsowerequestthatyouprovideyouremailaddresswhenyoufinishanswering.However,ifyouwouldprefertoansweranonymously,thatisalsopossible.Inrecognitionofyourparticipationwewillsendyouanadvancecopyofour“VisionsoftheFuture”papersprovidedyougiveusyouremailaddress(youwillhaveanopportunitytothisonalaterpage).IfyouhaveanyquestionsaboutthestudypleasecontactAndrewBrasher:[email protected].

Pleasenote:ThissurveyisusingSurveymonkeyandanyinformationyouenterwillbestoredtemporarilyintheUS.Bytakingpartinthesurveyyouareconsentingtoanyinformationthatcanidentifyyouasanindividualbeingstoredinthisway.

4.2 PostsurveyOnlastpageofsurveytheparticipantswereasked:

Wemaywanttoquoteyourresponsestothissurveyanonymouslyinreportsandpublications.Pleaseletusknowifyouagreethatwecan.

Yes,youcanquotemyresponsesanonymouslyinreportsandpublications

No,youcannotquotemyresponsesanonymouslyinreportsandpublications


65

About

VersionHistoryDate Notes Person11Jan2016 Initialdraft DaiGriffiths,UOB

AndrewBrasher,OUUKRebeccaFerguson,OUUKDougClow,OUUKLiYuan,UOB

29Jan2016 Versionforinternalreview DaiGriffiths04Feb2016 Annotatedreviews RebeccaFerguson,Maren

Scheffel,HendrikDraschler09Feb2016 Finalversion DaiGriffiths09Feb2016 ClearedforsubmissiontotheEC HendrikDrachsler,Maren

Scheffel

Aboutthisdocument(c)2016,DaiGriffiths,AndrewBrasher,DougClow,RebeccaFerguson,LiYuan

LicensedforuseunderthetermsoftheCreativeCommonsAttributionv4.0licence.Attributionshouldbe“byDaiGriffiths,AndrewBrasher,DougClow,RebeccaFerguson,andLiYuan,fortheLACEProject(http://www.laceproject.eu)”.

Formoreinformation,seetheLACEPublicationPolicy:http://www.laceproject.eu/publication-policy/.Note,inparticular,thatsomeimagesusedinLACEpublicationsmaynotbefreelyre-used.

AboutLACETheLACEprojectbringstogetherexistingkeyEuropeanplayersinthefieldsoflearninganalytics&educationaldataminingwhoarecommittedtobuildingcommunitiesofpracticeandsharingemergingbestpracticeinordertomakeprogresstowardsfourobjectives.

Objective1–PromoteknowledgecreationandexchangeObjective2–IncreasetheevidencebaseObjective3–ContributetothedefinitionoffuturedirectionsObjective4–Buildconsensusoninteroperabilityanddatasharing

http://www.laceproject.eu@laceprojecthttp://lanyrd.com/profile/laceproject/https://www.linkedin.com/groups/Learning-Analytics-Community-Exchange-LACE-8133802

ThisdocumentwasproducedwithfundingfromtheEuropeanCommissionSeventhFrameworkProgrammeaspartoftheLACEproject:grantnumber619424

http://www.laceproject.eu)

http://www.laceproject.eu/publication-policy/

http://www.laceproject.eu/publication-policy/

http://www.laceproject.eu

learning analytics community exchange...

Documents