privacy-driven design of learning analytics applications ... · with privacy as a barrier...

(2016).Privacy-drivendesignoflearninganalyticsapplications:Exploringthedesignspaceofsolutionsfordatasharingandinteroperability.JournalofLearningAnalytics,3(1),139–158.http://dx.doi.org/10.18608/jla.2016.31.9

ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0) 139

Privacy-Driven Design of Learning Analytics Applications: Exploring the Design Space of Solutions for Data Sharing and Interoperability

ToreHoel

OsloandAkershusUniversityCollegeofAppliedSciences,[email protected]

WeiqinChen

UniversityofBergen,NorwayABSTRACT:Studieshaveshownthatissuesofprivacy,controlofdata,andtrustareessentialtoimplementation of learning analytics systems. If these issues are not addressed appropriately,systemswilltendtocollapseduetoalegitimacycrisis,ortheywillnotbeimplementedinthefirstplacedue to resistance from learners, their parents, or their teachers. This paper askswhat itmeanstogiveprioritytoprivacy intermsofdataexchangeandapplicationdesignandoffersaconceptualtool,aLearningAnalyticsDesignSpacemodel,toeasetherequirementsolicitationanddesignfornewlearninganalyticssolutions.Thepaperarguesthecaseforprivacy-drivendesignasanessentialpartoflearninganalyticssystemsdevelopment.Asimplemodeldefiningasolutionastheintersectionofanapproach,abarrier,andaconcernisextendedwithaprocessfocusingondesign justifications to allow for an incremental development of solutions. This research isexploratory innature,and furthervalidation isneededtoprovetheusefulnessof theLearningAnalyticsDesignSpacemodel.Keywords: Learning analytics, privacy, data sharing, trust, control of data, privacy by design,interoperability

1 INTRODUCTION Learninganalytics (LA) isdevelopingrapidly inhighereducation,and it isbeginningtogaintraction inschools,accordingtomanyforesightanalysts(Johnsonetal.,2016;Johnson,AdamsBecker,Estrada,&Freeman,2014a;Johnson,AdamsBecker,Estrada,&Freeman,2014b;Griffiths,Brasher,Clow,Ferguson,&Yuan,2016).Nevertheless,marketplayersexperience severe setbacks related to lackof trust in LAsystems(Singer,2014;Drachsleretal.,2016).Amainbarrierformainstreamadoptionofthistechnologyrevolvesaroundconcernsaboutprivacy,controlofdata,andtrust(Hoel,Mason,&Chen,2015;Mason,Hoel,&Chen,inpress;Griffiths,Hoel,&Cooper,2016;Hoel&Chen,2014,2015;CooperandHoel,2015;Scheffel, Drachsler, Stoyanov, & Specht, 2014). This paper promotes the idea that LA systemsdevelopment should be based upon a “privacy by design” approach, rather than addressing privacyconcernsasanunpleasantafterthought.Ifsystemsthathaveintegratedprivacyconcernsintheirdesignswereprioritized,itwouldhelpresearchanddevelopmenttofocusonviableprojectsinsteadofwastingtimeandmoneyonblue-skytechnologies.Privacymay,however,bedefinedasbeyondthescopeofLAsystemsandLAinteroperabilityspecificationdevelopment(ADL,2013;IMSGlobal,2015),asonemightthinkthatprivacyissuesaredealtwithbyfront-



endsystemsthatprovidethedataexhaustforanalytics.Thispositionisflawed,bothconceptuallyandpractically. First, privacy cannot be handled only by a sign-on process or a consent form; privacypermeatesallprocessesoftheLAprocesscycle(Hoel,Chen,&Cho,2016).Second,ifprivacyrequirementsarenotreflectedatthetimeofdesign,thedevelopedsolutionsmaynotdeliveraccordingtolawormarketneeds(Hoel&Chen,2015).Thatsaid,privacyisalsoanequivocalconceptthatneedstobeunderstoodincontextofemergingLApractices.“Theprinciplesofdataprotectionbydesignanddataprotectionbydefault”(EC,2012,p.27)haverecentlybeenbuiltintoEuropeanandUSpolicies,respectively,throughtheGeneralDataProtectionRegulation(CouncilDirective95/46/EC)andRecommendationsforBusinessandPolicy-makersfromtheUSFederalTrade Commission (FTC, 2012). The privacy-by-design (PbD) framework was developed within theInformation and Privacy Commission of Ontario, Canada,with goals of “ensuring privacy and gainingpersonal control over one’s information and, for organizations, gaining a sustainable competitiveadvantage” (Cavoukian, 2012, pp. 36–37). The PbD framework laid down by Cavoukian (2012)encompassesITsystems,accountablebusinesspractices,physicaldesign,andnetworkedinfrastructuresandfollowsthesesevenfoundationalprinciples:

1. Proactivenotreactive;preventativenotremedial2. Privacyasthedefaultsetting3. Privacyembeddedintodesign4. Fullfunctionality–positive-sum,notzero-sum5. End-to-endsecurity–fulllifecycleprotection6. Visibilityandtransparency–keepitopen7. Respectforuserprivacy–keepituser-centric(p.37)

As long as these principles are maintained as high-level concepts left open to be defined by theorganizationseekinga“competitiveadvantage,” thePbDapproachwillhavedifficulties in leavinganyfootprintonaparticulardomain.Theprinciplesneedtobeappliedincontext,bothintermsofdomain(inourcaselearning),anddesign(i.e.,systemsengineering)activities.Thispaperaimstodevelopadesignprocessmodelthatwillmakeiteasiertocreateprivacy-awaredesignsforlearninganalytics.Thepaperisorganizedasfollows:Section2offersaliteraturereviewthatlooksathowprivacyhasbeenthefocusofresearchanddiscoursewithintheLAcommunityinthelastfewyears.Contextsandcontextintegrityareidentifiedasanimportantbackdropforunderstandingprivacy.Basedtheauthors’previouswork,anLADesignSpaceconcept isdevelopedandamodelofferedasausefuldiscourseartefact forachievingprivacy-drivendesignofLA(Section3).InSection4,thecurrentstateoftheartrelatedtodatasharingisdescribedinthecaseusedinSection5toconstructaProblemSpace,aSolutionSpace,and,basedontheseconstructs,aDesignSpaceanalysisofviablesolutionsfordealingwithprivacyinLA.TheresultisdiscussedinSection6,andSection7concludeswithideasforfurtherwork.



2 RELATED WORK IsprivacyrecognizedasanissueincurrentLAresearch?TheyearlyinternationalconferencesonLearningAnalyticsandKnowledge(LAK)arearepresentativeoutletforLAresearch.Lookingatthemainconferenceproceedingsof LAK ’14and LAK ’15,onemay say thatprivacy is recognized,butonly superficially so.However,from2014to2015,weseesignsofanewapproachthatnotonlyidentifiesprivacyasaconcernbutpointstoprivacysolutionsatdifferentlevels.AtLAK’14,12of57papersmentionedprivacy,threeofthemdescribinghowdatawasanonymizedtoprotectprivacy.Therestofthepaperswereconcernedwith privacy as a barrier (Ferguson, De Liddo,Whitelock, de Laat, & Buckingham Shum, 2014a); as arestriction for data tracking (Drachsler, Dietze, Herder, d’Aquin, & Taibi, 2014b); and as a cluster ofstakeholderconcernsrevolvingaroundrisks(Drachsler,Stoyanov,&Specht,2014a).However,privacyisclearlyanobstaclethatshouldbeovercomeinordertoreapthebenefitsofLAsince“Learnersneedtobeconvincedthat[LAsystems]arereliableandwill improvetheir learningwithoutintrudingintotheirprivacy”(Fergusonetal.,2014b,p.251).“Manymythssurroundingtheuseofdata,privacyinfringementandownershipofdataneedtobedispelledandcanbeproperlymodulatedoncethevaluesoflearninganalyticsarerealized”(Arnoldetal.,2014,p.259).Someauthorsremindedtheaudiencethatoneshouldbemindful(ofprivacy)whendesigninguserinterfaces(Aguilar,2014).Indoingso,anotherpaperpointedoutthatwhileethicsandprivacyarefeaturesofeducationaldatasciences,publicentitiesarerequiredtoadheretoFERPAandothersuchregulations,whereas“intheprivatesectortherearefewerrestrictionsandlessregulationsregardingdatacollectionanduse”(Pietyetal.,2014,p.198).OnepapercalledforethicalliteracybyLAknowledgepractitioners,“maintaininganethicalviewpointandfullyincorporatingethicsintotheory,research,andpracticeoftheLAKdiscipline”(Swenson,2014,p.250).Oneyearlater,atLAK’15,privacywasstillnotamajortheme(mentionedin10outof82papers),buttheissuewasputontheagendabyresearchersactiveinEuropeanprojectsinapaneldiscussion(Fergusonetal.,2015)andaworkshopdedicatedtoethicsandprivacy1(Drachsleretal.,2015a).ThemainconferencepapersofLAK’15stilllookedatprivacyasasearchterm(Sekiya,Matsuda,&Yamaguchi,2015),acoursesubject(Vogelsang&Ruppertz,2015),oranabstractconcern(Scheffel,Drachsler,&Specht,2015),whichcouldlimitaccesstodata(Wang,Heffernan,&Heffernan,2015;Drachsleretal.,2015b),oronethatmust“beaddressedgiventhelargerscaleofthetoolsusagecomparedwithpilotstudies”when“testingthetoolin-the-wild”(Martinez-Maldonadoetal.,2015,p.6).However,twopapersadvocatedthatinstitutions“mustengagemoreproactivelywithstudents,toinformandmoredirectlyinvolvetheminthewaysinwhichbothindividualandaggregateddataisbeingused”(Prinsloo&Slade,2015,p.8).PrinslooandSlade (2015)explored theconceptof studentprivacy self-managementandissuesaroundconsentandtheseeminglysimplechoicetoallowstudentstoopt-inoropt-outofhavingtheirdatatracked.Theyconcludedthatthewayforwardcannotsimplybetointroducea choicebetweenopt-inoropt-outas “Onlyby increasing the transparencyaround learninganalytics

1AmajorityofthecontributionstothisspecialissueofJLA(Vol.3,No.1)arebasedoninputtothisLAK’15workshop.



activitieswillHEIsgainthetrustandfullerco-operationofstudents”(2015,p.8).Kitto, Cross, Waters, & Lupton (2015), the authors of the second paper, discussed privacy vs. dataownership andproposed a technical solution, theConnected LearningAnalytics Toolkit, as a radicallydifferentsolutiontocurrentsystemsinthemarketsince“Manyoftheethicalproblemsthatarisefromwithintheprivacyperspectiveevaporatewhenstudentsaregivenfullaccesstotheirdata”(p.5).Kittoetal. (2015) referencedaworkbyPardoandSiemens (2014) thatadvocatesacontextualapproachwithrespecttoinformationprivacy;sometimeswewantourinformationtobepublic,sometimesnot.Nodoubt,theupcomingLAK’16conferencewillmovetheresearchfrontieronethicsandprivacyforLA;sowilloutputsfromtheEuropeanLACEproject,whichhaspublishedaReviewReportoncurrentissuesand their solutions (Drachsler et al., 2016a), as well as this special issue of the Journal of LearningAnalytics.ApreprintofaLAK’16paperbyDrachslerandGreller(2016)promotesachecklistapproachtotrusted learning analytics building on a number of catch phrases (determination, explain, legitimate,involve,consent,anonymize,technical,external)makinguptheDELICATEchecklist.“[W]ewouldliketoencourage the LearningAnalytics community to turn the privacy burden into a privacy quality label,”DrachslerandGrellerstate,seeingthechallengesas“a‘soft’issue,rootedinhumanfactors,suchasangst,scepticism,misunderstandings,andcriticalconcerns”(p.5).Referencingtheauthorsofthispaper(HoelandChen),DrachslerandGreller spellout that they“would refrain fromsolvingaweakness inanewlearning technology by proposing technical fixes or technological solutions, such as standardizationapproaches”(2016,p.5).Inchoosingbetweensoftchecklistsandhardtechnicalfixes,thereisaneedforaconceptualtoolthatcould help usmove from barriers and concerns towell-argued solutions. The aim of this paper is todevelopsuchaconceptualframework.However,beforedoingsothereisstillaneedtounpackprivacyasasocio-culturalconcepttobringitmoretothecentreofLAapplicationdesign.2.1 A Contextual Approach to Privacy PrivacyinLAisrelatedtohowdataareused,stored,andexchanged.Whendatacontaininformationthatcanbelinkedtoaspecificperson,wetalkabout“personaldata.”Wealsotalkabout“privatedata”thatarepartofaperson’sprivacy.Theboundariesputaroundpersonalandprivatedataaresocialagreementsthatdependonwho theperson isand inwhat social settings thedataarecreatedandshared.Akeyquestionrevolvesaroundwhoistheownerofthedata.Theanswercertainlyinvolvesthepersonathand,buttoleavethecontroltothispersonaloneisoftentoosimpleasolution.Heath (2014), discussing contemporary privacy theory contributions to LA found that the “debateregardingprivacyhasswungbetweenargumentsforandagainstaparticularapproachwiththelimitationtheoryandcontroltheorydominating”(p.142).Controltheoryfocusesonallowingindividualstocontroltheirpersonal information,while limitationtheory isconcernedwith the limitationssetonthosewho



could gain access to personal information. Heath puts more confidence, however, in theories thathighlightcontextsastheorganizingconcept,oneofthecontextsbeingLA.Ataninternationalworkshoponthefutureofprivacy,Dartiguepeyrouconcludedthattherewillbeanincreasedacceptanceofsharingdataforcommongood,increasedsocialandpublicvalue,withafollowinglikelyevolutionofthenotionof privacy from the “‘ability to control one’s personal information’ (collection, disclosure, use) to ‘adynamicprocessofnegotiatingpersonalboundariesinintersubjectiverelations’”(2014,p.13).Thus,agoodunderstandingofthemeaningof“context”isneeded.HelenNissenbaum(2014)hasmovedtheprivacydebatebeyond“control”and“limitation,”promotingrespectforcontextasabenchmarkforprivacyonline.Hertheoryofcontextual integrity isatheoryofprivacy regarding personal information “because it posits that informational norms model privacyexpectations; it asserts thatwhenwe findpeople reactingwith surprise, annoyance, indignation, andprotest that their privacy has been compromised, we will find that informational norms have beencontravened,thatcontextualintegrityhasbeenviolated”(Nissenbaum,2014,p.25).Contextis,however,anelusiveconcept thatneeds tobedefined.Nissenbaumhasstudiedthecontexts thatshapeprivacypolicy, i.e., contextas technologysystemorplatform;contextasbusinessmodelorbusinesspractice;contextassectororindustry;andcontextassocialdomain.InthediscourseonLAandinteroperability,itisnaturaltofocusontechnicalcharacteristicsasthecontext,e.g.,propertiesdefinedbyrespectivemedia,systems,orplatformsthatshapethecharacterofouractivities,transactions,andinteractions.“Ifcontextsareunderstoodasdefinedbypropertiesoftechnicalsystemsandplatforms,thenrespectingcontextswillmeanadaptingpoliciestothesedefiningproperties”(Nissenbaum,2014,p.14).However,NissenbaumdoesnotthinkthebestsolutionistodevelopprivacycontextrulesforTwitter,Facebook,specificlearningapplications,etc.Sheaspirestopromoterespectforcontexts,understoodasrespectforsocialdomains,asit“offersabetterchancethantheotherthree[technologysystem,businessmodel,orindustrysector]forthePrincipleofRespectforContexttogeneratepositivemomentumformeaningfulprogressinprivacypolicyandlaw”(Nissenbaum,2014,p.25).Willis,Campbell,andPistilli(2013)seemtobewellalignedwithNissenbaum’scontextualintegritytheoryintheirpaperexploringtheinstitutionalnormsrelatedtousingbigdatainhighereducation,particularlyforpredictiveanalytics.Theyconcluded,“theinstitutionisresponsiblefordeveloping,refining,andusingthemassive amountof data it collects to improve student success and retention.” Furthermore, “theinstitution is responsible forprovidingacampusclimate that isbothattractiveandengagingand thatenhancesthelikelihoodthatstudentswillconnectwithfacultyandotherstudents”(Willisetal.,2013,p.6).Recentdevelopmentofcodesofethicsbyhighereducationalinstitutionsshowsthattheeducationalsystemsarerespondingtothechallengestoimprovethecontextualintegrityoftheirstudents(Sclater,2016).Fromacontextualintegrityperspective,theinstitutionmaynothaveviolatedtheinformationalnormifthe roles of the actors involved— e.g., students, teachers, administrators— are acknowledged, theagreed information typeswere used, and the agreed data flow terms and conditionswere followed.



Actors, information types, and transmission principles are the three key parameters offered byNissenbaum for describing a context in terms of integrity and informational norms. By looking ateducation as a social domain instantiated in a number of specific contexts, the tools provided byNissenbaum’sprivacytheoryarewellsuitedtoanalyzethedesignspaceforLAapplications,providingprivacyischosenasakeyfoundationforapplicationdevelopment.3 FROM PROBLEMS TO SOLUTIONS: CONSTRUCTING A LEARNING ANALYTICS DESIGN SPACE (LADS) MODEL ThispaperwillcarryoutafirstdevelopmentandtentativevalidationoftheLADSmodel.Thisresearchispositioned in the first Relevance Cycle of the three research cycles of Design Science Research (DSR)(Hevner, March, Park, & Ram, 2004; Hevner, 2007), addressing requirements and field-testing. Thepurpose is to come upwith amodel thatwillmake the ideas of PbDmore relevant for LA solutionspromotingdatasharingandinteroperability.However,thescopeoftheLADSmodelisnotlimitedtoissuesof privacy, control of data and trust. This initial cycle of DSR process focuses on “generating designalternativesandevaluatingthealternativesagainstrequirementsuntilasatisfactorydesignisachieved”(Hevner, 2007, p. 90). In this paper, we do the first design and testing of the LADS model againstrequirementssolicitedthroughcommunityexchangeandanalysisofcasesderivedfromLApractices.Inordertoprovetheusefulnessofthemodel,rigorousevaluationneedstobedone.SomeideasonhowthisfutureresearchcouldbedonearepresentedinSection7.Inlookingforthelow-hangingfruitsofLAInteroperability,HoelandChen(2014)builtonInteroperabilityandEnterpriseArchitecturetheoriesandcameupwithaconceptofasolutionspace.Thesetheoriesareconcerned with how organizations are able to solve problems by communicating and exchanginginformation,usingtheinformationexchanged,andgettingaccesstothefunctionalityofathirdsystem(Chen&Daclin,2006).Thesolutionspaceisconceivedasathree-dimensionalmodel,describingconcerns,barriers,andsolutions(Figure1).

Figure1:Solutionsastheintersectionofapproaches,barriers,andconcerns(Chen&Daclin,2006).



In thispaper, thisconceptofasolutionspace is furtherdeveloped intoaLAdesignspace (LADS). It isunderstoodasarangeofpotentialdesignsthatcouldsolveidentifiedLAproblems,e.g.,thoserelatedtoprivacy, control of data, and trust. These designs are justified according to a design space analysis.MacLean,Yong,Bellotti,andMoran(1991)presenteddesignspaceanalysisasanapproachtorepresentdesignrationale,focusingonthreeaspects:questions,options,andcriteria.Questionsarekeyissuesforstructuring the space of alternatives, options are possible alternative answers to the questions, andcriteriaarethebasisforevaluatingandchoosingamongtheoptions.TheLADesignSpacemodel(Figure2)isbasedonathree-stepprocess,identifyingconcerns,barriers,anddesignsolutions.ThefollowingwalkthroughthethreestepswillexplaintheLADSmodel.

Figure2:TheLearningAnalyticsDesignSpaceModel.

1. Constructing the problem space: For this paper, the concerns are related to data sharing andinteroperability, which revolve around issues of privacy, control of one’s own data, and trust inapplications and service providers (Hoel & Chen, 2014). The barriers related to data sharing andinteroperabilityarepartofthechallengeofscalingupLA.AsFergusonetal.(2014b)observe,fewreportscurrently exist in the LA literature regarding deployment of scale. Moving from research and pilotenvironmentsto large-scaleapplicationscouldprovedifficultdueto lackofdatafor learninganalytics(Cooper&Hoel,2015;Griffiths,Hoel,&Cooper,2016).ForthepurposeofthispaperwehaveexploredhowLAdatacouldbecollected(Section4)toidentifybarriersandproposesolutions. 2. Constructing the solution space: Solutions should be developed along many dimensions, (e.g.,technical,organizational,legal,orpolitical),tryingoutboth“soft”and“hard”approaches(seeFigure2wherethesolutionsarerepresentedbycoloureddots.)



3.Constructingthedesignspaceandselectingafirstsolution:Inthelaststep,thequestionsderivedfromtheProblemSpaceanalysisareusedtoanalyzethecandidatesolutions(inFigure2,seeOptioncolumn“O”inDesignSpace),andcriteria(C1andC2)derivedthroughmovingfromproblemtosolutiontodesign.These will be used to select one ormore solutions (green dots) for further analysis in a continuousdevelopmentcycle.Forthesakeofargument,onesolutionmightbethata“technicalfix,”e.g.,adata-sharingconsentdashboardneedstobedeveloped,andthatcodesofpracticeandorganizationalpolicieswerenotenoughtoprovidesolutionstotheidentifiedproblems.Inthefollowingsection,wewillselectsomedataasinputforafirstdemonstrationoftheviabilityofthemodel.4 CASES OF DATA SHARING: ISSUES TO BE ANALYZED USING THE LADS MODEL Inordertoconductafirstrunthroughofthemodel,wewillidentifyconcernsandbarriersselectedfromafewcaseswehavebuiltforthispaperexploringwhichdatacouldbeavailablefor learninganalytics.Afterexaminingdifferentaspectsofdatasharinginthissection, inSection5wewillusetheresultsasinputtoseeiftheLADSmodelisaviableinstrumentforanalysis.LAbeginsandendswithdata.Dataaregeneratedfromlearneractionsandthecontextsoflearning;thentheanalyticsproducesnewdata,which isusedby follow-upactionsand interactionwith the learner,whichinturnproducenewdatatofeedintothenextLAcycle.Thedataarestoredinstandardizedformatsofsorts,andaresubjecttodataclearanceproceduresfollowingnational,institutional,orcompanyrulesandregulations.AstudyofthedataelementsoftheUSCommonEducationData(CEDS,2014)concludesthatmuchofthedataresiding inStudentManagementSystemsorLearningActivityRecordStoresarenot imbuedwithprivacyissuesraisedbytheintroductionofnewLApractices.Ofcourse,therearesensitiveissuesrelatedtotheidentificationofaperson;andtheaggregationofdisparatedataaboutapersoncanalwaysbefeltasathreat,especially ifonelosestrust inthesystemitself.However,thesedatahavebeenaroundineducationfordecadeswithoutcausingtoomuchconcern. It isthelearningprocessdata,sittingintheintersectionbetweenorganizations,people,andlearningresourcesthatnowhavebecomesomuchmoreimportant.Process data are, as observed in new LA applications, captured in formats defined in activity streamspecifications,e.g.,ADLExperienceAPI,2Tin-Can,3IMSCaliper4(Griffiths,Brasher,Clow,Ferguson,&Yuan,

2www.adlnet.gov/tla/experience-api3tincanapi.com4www.imsglobal.org/caliper



2016).Thesespecificationsestablishacorelanguagetodescribeactivitiesbyprovidinginformationonsubject, verb, object, context, etc. On top of these core specifications, community profiles providespecializedvocabulariesforeducationalsettingslikeschools,highereducation,workplacetraining,etc.Withapowerfulandextensiblecore languageone is, inprinciple,able todescribeanyactivity,whichopensupthequestionofwhatLApractitionerswanttodescribe.FergusonandBuckinghamShum(2012)introducedfivecategoriesofanalyticsthatmakeuseoffivepartlyoverlappingclassesofdata:

• Socialnetwork(analyzesrelationshipsusingdataaboutidentifiablepersonsandtheiractivities,e.g.,publishingpapers,participatinginsocialplatforms,etc.)

• Discourse(analyzeslanguageasatoolforknowledgenegotiationandconstructionusingfull-textdatafromdiscussionfora,talk,andotherwrittentextsources)

• Content(analyzesuser-generatedcontentusingdatafromWeb2.0applications)• Disposition (analyzes intrinsicmotivations to learn using a range of activity data, in principle

generatedbyallthetoolsusedbythelearner)• Context (considers formaland informal learningbasedondatadescribing the contextswithin

whichlearninghappens,e.g.,useoftools,educationalsetting,groups,etc.)MostofthedifferenttypesofanalyticsdescribedbyFergusonandBuckinghamShum(2012)wouldnotbepossiblewithoutdatafromsocialsoftware,alsocalledWeb2.0applications.Withmobiledevicesnowinnearlyeverystudent’spocket,useofsocialmediaispartofeverydaylife,includingoncampusorintheclassroom.Evenwhen institutionalpolicies try to restrict theiruse in formaleducationsettings, socialmediastillpervadestheeducationalspace.GaraizarandGuenaga(2014)exploredhowHTML5browserAPIscouldshedsomelightonhowtheuseofWebappsinmobileenvironmentshasthepotentialtoenhancelearning.TheAPIsallowwebpagestomakeuseofdatacollectedbydifferentsensors,e.g.,sensorsembeddedinwearablecomputers(mobilephones,wristbands,watches,etc.).Thisopensuparangeofnewdatasources.Table1liststhedatatypesusedbyHTML5APIsandderivesquestionsastowhatpedagogicalorlearninganalyticsusesthesedatatypescouldpotentiallyhave.

Table1:DatatypesinHTML5APIsandtheirpotentialuseforLA.

Datatype Informationprovided Potentialquestions

Geolocation Latitude / longitudechanges

Is the learner at school or at home? Is shecommuting?Wheredoesthelearningtakeplace?

3-DOrientation Accelerationchanges Isthecontextsuitableforlearning?




Battery Statusofbattery,charging Doesthebatterystatusaffectthelearningcontext?How?

Networkinformation Costofnetworkaccess Doesthecostofnetworkaccessdisruptthelearningscenario?How?

Offlineandonlineevents Connectivitystatus Whichproblemsarecausedbythelackofcontinuityinconnectivity?

DOM storage: file, indexeddatabase

Localstorage Whatdidthelearnerdowhenshewasoffline?Diditaffectthelearningprocess?

Ambientlight Light surrounding thelearner

Isthelearningenvironmentsuitableforlearningormoresuitableforrelaxation?

Temperature Temperature around thelearner

Isthelearningenvironmentsuitableforlearning?

Atmosphericpressure Heightaboveground Isthecontextsuitableforlearning?

Proximityofobjects Are learning aids accessible to the learner duringworkwithaparticularapp?

Gestures Swipe,pinch,twist,etc. Whatisthelearnerfocusedon?

Bloodpressure What is the physical state of the learner duringlearningevents?

Heartbeat What is the physical state of the learner duringlearningevents?

Perspiration Isthelearnernervous?

getUserMedia Nativeaccesstoaudioandvideodevices

Whatisthelearnerlookingat?Whatisshelisteningto?How is the learning context in termsof space,luminosity,noise,etc.?

WebRTC Send and receivemultimedia betweenbrowsers

How can the multimedia streams be collected,stored,analyzed,andenrichedinrealtime?

WebVVT Subtitles and audiodescriptions

Whatistheimpactofaddingsupplementarytextualinformationtomultimediastreams?

Animations (CSS, SMIL, rAF,SVG,Canvas2D,WebGL)

Declarativeandproceduralanimations

What is the impactofaddingsupplementaryvisualinformationtomultimediastreams?




Timers (high resolution, user,resource,navigation)

Timestamps permillisecond

How long does it take to perform an action(download a learning activity, render a web app,etc.)?Isthelearnermultitasking?Isshebored?Isshecheatingviaautomaticresponses?

DOM 4 mutation observes,draganddropevents,focus

Fine-grained userinteractions

Whichwebcontrolsareeasyorhardtouse?Whichgesturesand/orcomplexinteractionsarepreferredbylearners?

Page visibility, full screen,pointerlock

Single task / multitaskscenarios

Isthelearnermultitasking?How?When?Dosingletask/multitaskactivitiesenhancelearning?

History Historyofwebsession Istheworkflowofthelearningappappropriate?

Followingthedatatrail,literallyspeaking,fromtheheadmaster’sfilingcabinettothepocketofthelearnerhasmovedourfocusofanalysisawayfromthedataelementsandtheirpotentialprivacyissuestodataincontext.Privacyisnotaunidimensionalconceptdescribingtherelationshipbetweenthedataelementandthepersonaboutwhomthiselementholdsinformation.Bybringinginthecontextdimension,weseethatdatabelongtomorethanthepersondescribed;itisthecharacteristicsofthesetting(context)thatimpacttheprivacyconcerns.Exploringthesecasesofdataavailableforlearninganalytics,wehaveshownthatthecontextofformalstudyorteachingisessential,asitestablishestheboundaryforwhatiswithinoroutsidethescopeofdataavailableforlearninganalytics.Fromaninstitutionalperspective,ifthisboundaryiscrossed—e.g.,byintroducingsocialsoftwareservicesrunbyathirdparty—thiscanonlyhappenbyindividualconsentonacase-by-casebasis.Fromanindividualorathird-partyperspective,thisboundarymaybelessdefinitive,whichleadstotensionsamongdifferentstakeholdersintheuseofLAtosupportlearning.However,theboundariesbetweenformalandinformallearningarefarfromclear,asMalcolm,Hodkinson,andColley(2003) have demonstrated. They found (before socialmedia took off in learning) “a complete lack ofagreement in the literature about what informal, non-formal and formal learning are, or what theboundariesbetweenthemmightbe”(Malcolmetal.,2003,p.313).TheinputforconstructingtheProblemSpaceisconcernsandbarriers.ThefirstworkshoponLAatICCE2014expandedontheprivacy,control,andtrustclusterofissuesreferredtoabove(Hoel&Chen,2014),andmappedconcerns(Mason,Hoel,&Chen,inpress).Someconcernspointinthedirectionofrestrictivesharingofdataandputtingacaponservicesthatinteroperate.However,therearealsoconcernsaboutnotbeingabletoreapallthebenefitsofLA,understandingandoptimizinglearning(Duval,2011).Thesebenefitsaredirectlyintheinterestofthelearnerwhowishestobeincontrolofherdata.Sincewehavemultiplestakeholderswithlegitimateinterests,theeventualsolutionsmustbalancetheinterestsofallparties.



Concerning barriers, the Educause Center for Applied Research identified four major challenges toachievingsuccesswithanalyticsinhighereducation:affordability,data,culture,andexpertise(Bichsel,2012).Fromaninstitutionalperspective,costisthemainobstacle;however,factorslikemisuseofdata,regulationsrequiringtheuseofdata,inaccuratedata,andindividualprivacyrightsarebarriersthathighereducationleadersworryaboutsincetheyarecollectingmoredatathaneverbefore(Bichsel,2012).Hoel,Mason,andChen(2015)analyzedacorpusofmorethan200questionsgatheredbytheLearningAnalyticsCommunityExchange5andfoundthatthediscussionondatasharingandbigdataforeducationisstillinanearlystage.Conceptualissuesdominateandthereisstillalongwaytogoinmovingtowardssolutionsfortechnicaldevelopmentandimplementation.5 A FIRST DEMONSTRATION OF THE LEARNING ANALYTICS DESIGN SPACE MODEL BasedontheconcernsandbarriersderivedfromtheselectedcasesinSection4,weconstructaProblemSpaceforLAdatasharing.ThisProblemSpaceleadstoanexplorationofsolutions,whichinturnwillbeselectedascandidatesfordesign. 5.1 Building the Problem Space Fromalearner’sperspective,twoconcernsarepullingthe“datasharingslider” inoppositedirections:prioritizing privacy and individual control of data tends to limit data sharing, while wanting to takeadvantageofthelatestpersonal learningapponthemarket isaninvitationtotickanumberof“give-access-to”boxes.Thebarriersarerelatedtotheconceptofa“userincontext.”Informalandindividuallearningleavesthedecisionsofgivingaccesstopersonaldatatotheuser,andisamatteroftheappreciationofbenefits,feeling of control, trust in applications, companies, institutions, and so on. In the current situation,individualsseemtobemorewillingtotakerisksandgofornewandinnovativesolutions(Xu,Luo,Carroll,&Rosson,2011).Whileformallearningisledbyinstitutionswantingtohaveethicaluseofstudentdatapoliciesinplace,theytendtostaywithinstitutionallearningplatformsthatuseonlyalimitedsetofdatasourcesforLA.Fortheinstitutions,lackofprivacyframeworksisamajorbarriertodatasharingandusingsensitivedatasourcesthatotherwiseareonlyavailabletocommercialLAproviders.Thebarriersseemtobemoresocio-culturalororganizationalthantechnicalorlegal,tousetheEuropeaninteroperability frameworkdimensions (IDABC, 2004); however, the solutionswill need to address alltheseinteroperabilitychallenges.

5www.laceproject.eu



5.2 Building the Solution Space Solutionsarefoundbyaddressingtheconcernsandbreakingdownthebarriers,which inourcasewedefineasbeingofatechnical,socio-cultural,andlegalnature.Goingfora“radical”alternative,usingavarietyofdatasourcesandahighdegreeofdatasharing,wecanseethesetentativesolutionsbasedonrequirementsfromthecasesdiscussedinSection4:

• Technical:designaspecificationallowingusers toexpressdetailedconditions fordatasharingwhensigningupforLAapplications,withopt-outpossibilities

• Socio-cultural: boost trust in LA systems,developmentofprivacydeclarations, industry labelsguaranteeingadherencetoprivacystandards,andothermeansofsupportingcustomerdialogueaboutprivacy

• Legal: strengthen ownership and control of data from learning activities in national andinternationallaw

Thenextstepistochooseoneormoreofthesealternativesolutionsfordesign.5.3 Design Space Analysis Whichsolutionshouldbefocusedon?Thedesignspaceanalysisstartswithquestioningtherationaleofaprojectasarefinementoftheproblemspaceanalysis.Forourpurpose,wemaintaintheambitiousgoalofusingapplicationssupportingpersonalizedandadaptivelearning.Furthermore,weask,isthesolutionsafe from “losing face” through leakage of personal information? And does the solution supportubiquitouslearningbyallowingbothformalandinformallearninginthesameapplication?Thecriteriaforwhichoptionstochoosedrivethedesignprocessbasedontheidentifiedsolutions.Theprivacy-by-designapproachadvocatedbyNissenbaum(2014)gaveprioritytothesocialdomainasthecontexttoexplore—toseeifcontextualintegrityismaintainedwhendataareshared.Therefore,doesthe proposed option pass the test of having been subject to an informed public deliberation on thebenefitsofLAandtheconsequencesofdatasharingfortheuseraswellasfortheinstitution,theserviceprovider,andothers?Inthecaseofthetechnicalsolutionproposedabove,thedesignmustgobeyondaquicktechnicalfixtosolvetheproblemandgivetheuserabsolutecontrol.Theinstitution(schooloruniversity)shouldhaveasay,sinceitisalsoresponsibleforthegreatergood,theclassorgroup,theparents,andsociety.Technicalsolutions should, therefore, include an element of permanent negotiation, thus requiring simple,transparentsolutions(Hoel&Chen,2015).Thelegalsolutionisalsoanoptionbutnotthefirstpriority.Ofcourse, solutionsmust have legal backing, but theprivacy concerns surroundingdata sharing are notsolvedbylegalmeasuresalone.Ouranalysispointsinsteadtothesocio-culturaldomainforsolutionsanddesignrequirements.A socio-cultural design solution must focus on the communication between user and system/service



provider.Trustisnota“thing”that,negotiatedonce,lastsforever;itmustberenegotiatedrepeatedly.Especiallyinadynamicenvironmentcrowdedwithactorswithdifferentinterests,large-scale,complex,non-transparentsolutionswillthereforebechallenged.Itwillbeeasiertomaintaincontextintegritywithsmallersolutions.SmallerLAsolutionsmayseemacontradictioninterms,astheideasofbigdataanddata sharing across systems often lead to plans for large-scale solutions, perhaps with a centralizedLearningRecordStoreordatawarehouseaggregatingdatafromanumberofsystems.Nevertheless,ifmaintainingtrustispivotaltoLAsystemsinthecurrentstageofdevelopment,ourdesignspaceanalysisconcludesthatthesocio-culturalaspectsofnegotiatingaccesstodatashoulddirectthedesignoftechnicalsolutions, legalframeworks,andimplementation.WiththatresultofthefirstdesigncycleoftheLADSmodel,newconcernsandbarriers shouldbemapped inorder toarrive,after several iterations,atanimplementabledesign.

Table2:SummaryofthefirstiterationoftheLADSmodel.

Questions Solutions Criteria Design SolutionCandidate

Will student privacy self-management bemaintained?

Userdatasharingconsenttool

Promotecontextintegrity

Will privacy in differentcontextsberespected?

Data sharing dashboardwithconsentandopt-outmechanisms

Continuous negotiationbetween learner,institution, and thirdparties

Will different user groupstrustthesolutions?

Learner/institutiondialoguepractices

Avoid obfuscation,promotetransparency

Solution that prioritizesthe socio-cultural aspectsfor negotiation of accessto data for learninganalytics

Will the solutions supportubiquitouslearninginbothformal and informalsettings?

Regulation of dataownership and controlthroughlaw

Harvestlow-hangingfruits

Table2summarizesthefirstiterationofusingtheLADSmodeltoformquestionsanddesignsolutions.ThistablemapstheprocessillustratedinFigure2withexamplesofproblems,solutions,criteria,andacandidatedesignsolutionidentifiedfortheselectedcasesinSection4.



6 DISCUSSION Educational institutions have always used learner behaviour and performance data to determine,visualize,andsortstrengthsandweaknessesofindividuallearnersandgroups.WhatisnewwithLAistheability to process this information in real time and on demand. Furthermore, LA can go far beyondclassroomassessmentprocedures.Bydoingso,LAisworkingwithdatathelearneroftendoesnotknowarebeingused(Williamson,2015).LAcanbeusedtocomputetherelationshipsbetweenlearnersbasedontheirinteractions,tocomparethecommitmentofalearnerinacoursebasedontimespentonthelearningmaterial,ortocomparetextwrittenbystudentsagainstpre-existingcorpora.Thus,LAaffectstheprivacyrightsoflearnersinanewmanner,makingitnecessaryforthelearnerandtheinstitutiontonegotiate the boundaries between personal and institutional spaces, between informal and formallearning, and between institutionally provided tools and technology for personal use. As Thomas hasargued, “learning spaceshave tobeplannedon the strength thatdifferent kindsof learningwill onlyemerge once these spaces are used by students” (Thomas, 2010, p. 508).When “much, if notmost,learningdoesnotoccurinformallydesignatedlearningspaces,”itistimeto“wrestthelocusofcontrolfrom the traditional conceptionof learning spaceplanningas theexclusiveprovinceofarchitectsandphysicalfacilityplanners”(Thomas,2010,pp.503,510).Thisneedtore-assesswherelearninghappensisreinforced by the introduction of LA as a support technology. LA is, however, an emerging discipline(Siemens,2013),andmostofthetechnologicalideasarestillonthedrawingboard.Therefore,thereisastrong need to do the right thing from the outset, to avoid setbacks and the need to correctmisconceptionsandrebuildtrustafterprivacycollapses.This paper contributes a conceptual tool to ease the requirement solicitation and design for new LAsolutions.Asimplemodeldefiningasolutionastheintersectionofanapproach,abarrier,andaconcernwasextendedwithaprocessfocusingondesignjustificationstoallowfortheincrementaldevelopmentofsolutions.Weusedprivacy-by-designprinciplestosteerthedevelopmentofideastowardsolutions;however,otherprinciplescouldbeusedtotestalternativedesignsolutions,likepedagogicalprinciplesfocusingonlearningefficacy,learner-centredapproaches,ubiquitouslearning,andsoon.7 CONCLUSIONS AND FUTURE RESEARCH PrivacyawarenessisreportedtobeoneofthemajorfeaturesofsmartLAwhenresearcherssummarizetheirexperiences“fromthefield”(Ebner,Taraghi,&Saranti,2015).LAisayoungfieldbothinresearchandinapplicationdesign.Newideasarebeinglaunchednearlyeveryday,andthereisaneedfortestingto see if they meet the requirements of different stakeholders. For example, Kennisnet, a Dutchgovernmentalschoolagency,haschosenPbDprinciplesasastartingpointfortheirnewdesign:“Next,weusetheopenUserManagedAccess(UMA)standard.Thestudent,orparentforunderagestudents,hasacentralplaceandistheownerofhisowneducationaldata”(Bomas,2014).WillgivingstudentsandparentsfullownershipoftheirdatausingtheUMAstandardbenefiteducationalgoals?Inordertoanswerthisquestion,onemustanalyzehowthestandardis implementedandhowthedifferentconcernsare



addressed.Inthispaper,wehaveproposedtheLADSmodelasatooltoanswersuchquestions.Thetoolallowsuserstomaptheproblemspaceandanalyzedifferentsolutionsaccordingtodifferentcriteria.ThefirsttentativevalidationofthemodelpresentedinthispapershowsthatithasthepotentialtomakearequirementdiscourseonLAapplicationsmorefruitful.However,inordertoverifythisconclusion,furthertestingisnecessary.TheEuropeanLearningAnalyticsCommunityExchange(LACE)projecthas identifiedprivacyandethicsasmajor themes forcommunitydiscourse todevelop the fieldofLA.ThisprojectwillbeasuitabletestinggroundfortheLADSmodel.8 ACKNOWLEDGMENTS This paper was partly produced with funding from the European Commission Seventh FrameworkProgrammeaspartoftheLACEProject,grantnumber619424. REFERENCES ADL(AdvancedDistributedLearning).(2015).xAPIspecification.ProducedbytheExperienceAPIWorking

GroupinsupportoftheOfficeoftheDeputeAssistantSecretaryofDefense(Readiness)AdvancedDistributed Learning Initiative. Retrieved from https://github.com/adlnet/xAPI-Spec/blob/master/xAPI.md

Aguilar,S.(2014).Perceptionsanduseofanearlywarningsystemduringahighereducationtransitionprogram.Proceedingsofthe4th InternationalConferenceonLearningAnalyticsandKnowledge(LAK’14),390–391.http://dx.doi.org/10.1145/2567574.2567625

Arnold,K.E.,Lynch,G.,Huston,D.,Wong,L.,Jorn,L.,&Olsen,C.W.(2014).Buildinginstitutionalcapacitiesandcompetenciesforsystemiclearninganalyticsinitiatives.Proceedingsofthe4thInternationalConference on Learning Analytics and Knowledge (LAK ’14), 257–260.http://dx.doi.org/10.1145/2567574.2567593

Bichsel, J. (2012). Analytics in higher education: Benefits, barriers, progress, and recommendations.Louisville, CO: EDUCAUSE Center for Applied Research.https://net.educause.edu/ir/library/pdf/ERS1207/ers1207.pdf

Bomas,E.(2014,29August).Howtogivestudentscontroloftheirdata.[Weblogpost]Retrievedfromhttp://www.laceproject.eu/blog/give-students-control-data/

Cavoukian,A.(2012).Privacybydesign:Fromrhetorictoreality.Toronto,ON:InformationandPrivacyCommissioner of Ontario. Retrieved from https://www.ipc.on.ca/images/Resources/PbDBook-From-Rhetoric-to-Reality.pdf

CEDS. (2014). Common education data standards (CEDS): Version 4 data model guide. Retrieved frohttps://ceds.ed.gov/

Chen,D.,&Daclin,N.(2006).Frameworkforenterpriseinteroperability.InProceedingsoftheWorkshopsandtheDoctorialSymposiumoftheSecondIFAC/IFIPI-ESAInternationalConference:EI2N,WSI,IS-TSPQ.Retrievedfromhttp://chen33.free.fr/M2/Elearning/CIGI2009.Chen.final.pdf

Cooper,A.,&Hoel,T.(2015).Datasharingrequirementsandroadmap.PublicDeliverableD7.2fromtheLACE project (European Commission Seventh Framework Programme, grant number 619424).



Retrievedfromhttp://www.laceproject.eu/deliverables/d7-2-data-sharing-roadmap/Dartiguepeyrou, C. (Ed.). (2014). The futures of privacy. France: Fondation Télécom, Institut Mines-

Télécom. Retrieved from https://www.fondation-telecom.org/wp-content/uploads/2016/01/2014-The-Futures-of-Privacy-.pdf

Drachsler,H.,&Greller,W.(2016)Privacyandlearninganalytics—it’saDELICATEissue:Achecklisttoestablishtrustedlearninganalytics.Proceedingsofthe6thInternationalConferenceonLearningAnalyticsandKnowledge(LAKʼ16),89-98.http://dx.doi.org/10.1145/2883851.2883893

Drachsler,H.,Hoel,T.,Greller,W.,Kickmeier-Rust,M.,Steiner,C.,&Griffiths,D.(2016)Ethicsandprivacyforlearninganalytics:Areviewofcurrentissuesandtheirsolutions.RetrievedfromLACEprojectwebsitehttp://www.laceproject.eu/

Drachsler, H., Hoel, T., Scheffel, M., Kismihók, G., Berg, A., Ferguson, R., Chen, W., Cooper, A., &Manderveld, J. (2015a). Ethical and privacy issues in the application of learning analytics.Proceedingsofthe5thInternationalConferenceonLearningAnalyticsandKnowledge(LAK’15),390–391.http://dx.doi.org/10.1145/2723576.2723642

Drachsler, H., Dietze, S., Herder, E., d’Aquin, M., Taibi, D., & Scheffel, M. (2015b) The 3rd LAK datacompetition.Proceedingsofthe5thInternationalConferenceonLearningAnalyticsandKnowledge(LAK’15),396–397.http://dx.doi.org/10.1145/2723576.2723641

Drachsler,H.,Stoyanov,S.,&Specht,M.(2014a).TheimpactoflearninganalyticsontheDutcheducationsystem.Proceedingsofthe4thInternationalConferenceonLearningAnalyticsandKnowledge(LAK’14),158–162.http://dx.doi.org/10.1145/2567574.2567617

Drachsler,H.,Dietze,S.,Herder,E.,d’Aquin,M.,&Taibi,D.(2014b).Thelearninganalytics&knowledge(LAK)datachallenge2014.Proceedingsofthe4thInternationalConferenceonLearningAnalyticsandKnowledge(LAK’14),289–290.http://dx.doi.org/10.1145/2567574.2567630

Duval,E.(2011).Attentionplease!:Learninganalyticsforvisualizationandrecommendation.Proceedingsof the 1st International Conference on Learning Analytics and Knowledge (LAK ’11), 9–17.http://dx.doi.org/10.1145/2090116.2090118

Ebner,M.,Taraghi,B.,&Saranti,A.(2015).Sevenfeaturesofsmartlearninganalytics:Lessonslearnedfrom four years of research with learning analytics. eLearning Papers, 40. Retrieved fromhttp://www.openeducationeuropa.eu/en/article/Seven-features-of-smart-learning-analytics----lessons-learned-from-four-years-of-research-with-learning-analytics-

EC (European Commission). (2012).Proposal for a Regulation of the European Parliament and of theCouncilontheprotectionofindividualswithregardtotheprocessingofpersonaldataandonthefreemovementofsuchdata(GeneralDataProtectionRegulation)COM(2012)11Final.

CouncilDirective95/46/EContheprotectionofindividualswithregardtotheprocessingofpersonaldataandonthefreemovementofsuchdata[1995]OJL281/31

Ferguson,R.,&BuckinghamShum,S.(2012).Social learninganalytics:Fiveapproaches.Proceedingsofthe 2nd International Conference on Learning Analytics and Knowledge (LAK ʼ12), 23–33.http://dx.doi.org/10.1145/2330601.2330616

Ferguson,R.,Cooper,A.,Drachsler,H.,Kismihók,G.,Boyer,A.,Tammets,K.,&MartínezMonés,A.(2015).Learning analytics: European perspectives.Proceedings of the 5th International Conference onLearningAnalyticsandKnowledge(LAK’15),69–72.http://dx.doi.org/10.1145/2723576.2723637

Ferguson,R.,DeLiddo,D.,Whitelock,D.,deLaat,M.,&BuckinghamShum,S.(2014a).DCLA14:Secondinternational workshop on discourse-centric learning analytics. Proceedings of the 4thInternational Conference on Learning Analytics and Knowledge (LAK ’14), 283–284.http://dx.doi.org/10.1145/2567574.2567631



Ferguson, R., Clow, D.,Macfadyen, L., Essa, A., Dawson, S., & Alexander, S. (2014b). Setting learninganalytics in context: Overcoming the barriers to large-scale adoption. Proceedings of the 4thInternational Conference on Learning Analytics and Knowledge (LAK ʼ14), 251–253.http://dx.doi.org/10.1145/2567574.2567592

Federal Trade Commission (FTC). (2012). Protecting consumer privacy in an era of rapid change:Recommendations for businesses and policymakers. Retrieved from the Federal TradeCommission website http://www.ftc.gov/reports/protecting-consumer-privacy-era-rapid-change-recommendations-businesses-policymakers

Garaizar, P., & Guenaga, M. (2014). A multimodal learning analytics view of HTML5 APIs: Technicalbenefits and privacy risks. Proceedings of the 2nd International Conference on TechnologicalEcosystems for Enhancing Multiculturality (TEEM ’14) (pp. 275–281). New York: ACM.http://dx.doi.org/10.1145/2669711.2669911

Griffiths,D.,Brasher,A.,Clow,D.,Ferguson,R.,&Yuan,L.(2016).“VisionsoftheFuture,”HorizonReport.Public Deliverable D3.2. LACE Project. Retrieved from http://www.laceproject.eu/wp-content/uploads/2016/02/LACE_D3_2.pdf

Griffiths,D.,Hoel,T.,&Cooper,A.(2016).Learninganalyticsinteroperability:Requirements,specificationsandadoption.PublicDeliverableD7.4.LACEProject(EuropeanCommissionSeventhFrameworkProgramme, grant number 619424). Retrieved from http://www.laceproject.eu/d7-4-learning-analytics-interoperability-requirements-specifications-and-adoption/

Heath,J.(2014).Contemporaryprivacytheorycontributionstolearninganalytics.JournalofLearningAnalytics,1(1).Retrievedfromhttps://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/3339

Hevner,A. (2007).A three cycle viewofdesign science research.Scandinavian Journalof InformationSystems,19(2),87–92(article4).

Hevner,A.,March,S.T.,Park,J.,&Ram,S.(2004).Designscienceininformationsystemsresearch.MisQuarterly,28(1),75–105.

Hoel,T.,&Chen,W.(2014).Learninganalyticsinteroperability:Lookingforlow-hangingfruits.InC.-C.Liuetal. (Eds.),Proceedingsofthe22nd InternationalConferenceonComputers inEducation (ICCE2014)(pp.253–264).Japan:Asia-PacificSocietyforComputersinEducation.

Hoel, T., & Chen,W. (2015). Privacy in learning analytics: Implications for system architecture. In T.Watanabe&K.Seta(Eds.),TheoryandPracticeforKnowledgeManagement:Proceedingsofthe11th International Conference on Knowledge Management (ICKM 2015). Retrieved fromhttp://hoel.nu/publications/Hoel_Chen_ICKM15_final_preprint.pdf

Hoel, T., Chen,W., & Cho, Y.-S. (2016). Privacy requirements for learning analytics: From policies totechnicalsolutions.PaperpresentedattheWorkshoponEthicsandPrivacyforLearningAnalyticsat the 6th International Conference on Learning Analytics and Knowledge (LAK ’16), 25 April,Edinburgh,UK.

Hoel, T.,Mason, J., & Chen,W. (2015) Data sharing for learning analytics: Questioning the risks andbenefits.InH.Ogataetal.(Eds.),Proceedingsofthe23rdInternationalConferenceonComputersin Education (ICCE 2015). Asia-Pacific Society for Computers in Education. Retrieved fromhttp://hoel.nu/publications/ICCE2015_Hoel_Mason_Chen_final.pdf

IDABC.(2004).Europeaninteroperabilityframeworkforpan-EuropeaneGovernmentservices.Version1.0.Luxembourg: Office for Official Publications of the European Communities. Retrieved fromhttp://ec.europa.eu/idabc/servlets/Docd552.pdf?id=19529

IMSGlobal.(2015).CaliperAnalytics™Background.RetrievedfromthewebsiteofIMSGlobalLearningConsortiumhttp://www.imsglobal.org/activity/caliperram



Johnson, L.,AdamsBecker,S.,Cummins,M.,Estrada,V., Freeman,A.,&Hall,C. (2016).NMCHorizonReport:2016HigherEducationEdition.Austin,Texas:TheNewMediaConsortium.Retrievedfromhttp://cdn.nmc.org/media/2016-nmc-horizon-report-he-EN.pdf

Johnson, L., AdamsBecker, S., Estrada, V.,& Freeman, A. (2014a).NMCHorizon Report: 2014HigherEducation Edition. Austin, Texas: The New Media Consortium. Retrieved fromhttp://www.nmc.org/pdf/2014-nmc-horizon-report-he-EN.pdf

Johnson,L.,AdamsBecker,S.,Estrada,V.,&Freeman,A.(2014b).NMCHorizonReport:2014K–12Edition.Austin,Texas:TheNewMediaConsortium.Retrievedfromhttp://cdn.nmc.org/media/2014-nmc-horizon-report-k12-EN.pdf

Kitto,K.,Cross,S.,Waters,Z.,&Lupton,M.(2015).LearninganalyticsbeyondtheLMS:Theconnectedlearninganalyticstoolkit.Proceedingsofthe5th InternationalConferenceonLearningAnalyticsandKnowledge(LAK’15),11–15.http://dx.doi.org/10.1145/2723576.2723627

MacLean, A., Yong, R. M., Bellotti, V. M. E., &Moran, T. P. (1991). Questions, options, and criteria:Elements of design space analysis. Human–Computer Interaction, 6, 201–250.http://dx.doi.org/10.1207/s15327051hci0603&4_2

Malcolm, J., Hodkinson, P., & Colley, H. (2003). The interrelationships between informal and formallearning. Journal of Workplace Learning, 15(7/8), 313–318.http://dx.doi.org/10.1108/13665620310504783

Martinez-Maldonado, R., Pardo, A.,Mirriahi, N., Yacef, K., Kay, J., & Clayphan, A. (2015). The LATUXworkflow: Designing and deploying awareness tools in technology-enabled learning settings.Proceedingsofthe5thInternationalConferenceonLearningAnalyticsandKnowledge(LAK’15),1–10.http://dx.doi.org/10.1145/2723576.2723583

Mason,J.,Hoel,T.,&Chen,W.(inpress).Questionsasdata:Illuminatingthepotentialoflearninganalyticsthrough questioning an emergent field. Accepted for publication in Research and Practice inTechnologyEnhancedLearning.

Nissenbaum,H.(2014).Respectforcontextasabenchmarkforprivacyonline:Whatitisandisn’t.InC.Dartiguepeyrou (Ed.), The futures of privacy (pp. 19–30). France: Fondation Télécom, InstitutMines-Télécom. Retrieved from https://www.fondation-telecom.org/wp-content/uploads/2016/01/2014-The-Futures-of-Privacy-.pdf

Piety,P.J.,Hickey,D.T.,&Bishop,M.J.(2014).Educationaldatasciences:Framingemergentpracticesfor analytics of learning, organizations, and systems. Proceedings of the 4th InternationalConference on Learning Analytics and Knowledge (LAK ’14), 193–202.http://dx.doi.org/10.1145/2567574.2567582

Pardo,A.,&Siemens,G. (2014).Ethical andprivacyprinciples for learninganalytics.British JournalofEducationalTechnology,45,438–450.http://dx.doi.org/10.1111/bjet.12152

Prinsloo, P., & Slade, S. (2015). Student privacy self-management: Implications for learning analytics.Proceedingsofthe5thInternationalConferenceonLearningAnalyticsandKnowledge(LAK’15),83–92.http://dx.doi.org/10.1145/2723576.2723585

Scheffel,M.,Drachsler,H.,Stoyanov,S.,&Specht,M. (2014).Quality indicators for learninganalytics.EducationalTechnology&Society,17(4),117–132.

Scheffel,M.,Drachsler,H.,&Specht,M.(2015)Developinganevaluationframeworkofqualityindicatorsforlearninganalytics.Proceedingsofthe5thInternationalConferenceonLearningAnalyticsandKnowledge(LAK’15),16–20.http://dx.doi.org/10.1145/2723576.2723629

Sclater,N.(2016)Developingacodeofpracticeforlearninganalytics.JournalofLearningAnalytics,3(1),16—42.http://dx.doi.org/10.18608/jla.2016.31.3

Singer,N. (2014,21April). InBloomstudentdata repository toclose.NewYorkTimes.Retrieved from



http://bits.blogs.nytimes.com/2014/04/21/inbloom-student-data-repository-to-closeSekiya,T.,Matsuda,Y.,&Yamaguchi,K.(2015).CurriculumanalysisofCSdepartmentsbasedonCS2013

by simplified, supervised LDA. Proceedings of the 5th International Conference on LearningAnalyticsandKnowledge(LAK’15),330–339.http://dx.doi.org/10.1145/2723576.2723594

Siemens,G.(2013).Learninganalytics:Theemergenceofadiscipline.AmericanBehavioralScientist,57(10),1380–1400.http://dx.doi.org/10.1177/0002764213498851

Swenson, J. (2014). Establishing an ethical literacy for learning analytics. Proceedings of the 4thInternational Conference on Learning Analytics and Knowledge (LAK ʼ14), 246–250.http://dx.doi.org/10.1145/2567574.2567613

Thomas,H. (2010). Learning spaces, learningenvironments and thedis‘placement’of learning.BritishJournal of Educational Technology, 41(3), 502–511. http://dx.doi.org/10.1111/j.1467-8535.2009.00974.x

Vogelsang,T.,&Ruppertz,L.(2015).Onthevalidityofpeergradingandacloudteachingassistantsystem.Proceedingsofthe5thInternationalConferenceonLearningAnalyticsandKnowledge(LAK’15),41–50.http://dx.doi.org/10.1145/2723576.2723633

Wang,Y.,Heffernan,N.T.,&Heffernan,C.(2015).Towardsbetteraffectdetectors:Effectofmissingskills,classfeaturesandcommonwronganswers.Proceedingsofthe5th InternationalConferenceonLearningAnalyticsandKnowledge(LAK’15).31–35.http://dx.doi.org/10.1145/2723576.2723618

Williamson,B. (2015).Digitaleducationgovernance:Datavisualization,predictiveanalytics,and“real-time” policy instruments. Journal of Education Policy, 31(2), 123–141.http://dx.doi.org/10.1080/02680939.2015.1035758

Willis,J.E.,Campbell,J.,&Pistilli,M.(2013,6May).Ethics,bigdata,andanalytics:Amodelforapplication.Educause Review. Retrieved from http://www.educause.edu/ero/article/ethics-big-data-and-analytics-model-application

Xu, H., Luo, X. R., Carroll, J. M., & Rosson, M. B. (2011). The personalization privacy paradox: Anexploratory study of decisionmaking process for location-awaremarketing.Decision SupportSystems,51(1),42–52.http://dx.doi.org/10.1016/j.dss.2010.11.017

privacy-driven design of learning analytics applications ... · with privacy as a barrier...

Documents