learning analytics - competen-seacompeten-sea.eu/wp-content/uploads/2018/02/... · recommendations...
TRANSCRIPT
![Page 1: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/1.jpg)
LearningAnalytics–
what,how,why
MarenScheffelIoanaJivet
OpenUniversiteitNederland
![Page 2: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/2.jpg)
Outline
• Whatislearninganalytics?• Whereareweat?• Howaredashboardsused?• Howcanweevaluatelearninganalyticstools?• Howaboutaninstitutionallearninganalyticspolicy?• Wheredowegofromhere?
![Page 3: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/3.jpg)
Insight is the buzzword in the data revolution. Peoplewho have insight into their behaviour will be able toworkmoreefficientlyfromnowon.Thisequallyappliesto organisations, higher education institutions andnations. The digital revolution has brought about atremendous increase in the volume of data and it iscomparativelyeasytoaccess.Itwouldseemtobealostopportunitytoignoresuchdata.
MarjoleinvanTrigt(2016)
![Page 4: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/4.jpg)
https://cdn.static-econo
mist.com
/site
s/de
fault/files/im
ages/prin
t-ed
ition
/201
5022
8_LD
P001
_0.jp
g
![Page 5: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/5.jpg)
TheUnitedNationsarecallingforactions:
“dataarethelifebloodofdecision-making”
“the new world of data,a world in which dataare bigger, faster andmore detailed than everbefore”
![Page 6: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/6.jpg)
But what about theeducationaldomain?
Dataanalysisiscommon inbusinessandcommerce.
http://arcbridge.com
/wp-conten
t/up
loads/20
15/07/bu
siness-analytics.pn
g
![Page 7: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/7.jpg)
Collecting,storingandanalysingstudentrecords
hasalwaysbeendone
inonewayoranother.
http://balfour.rb
e.sk.ca/sites/balfour/files/Da
vid-Oscar-Brown-Stud
ent-Re
cord-side-2.jpg
http://abo
vethelaw
.com
/wp-conten
t/up
loads/20
13/03/file-cabine
t1.jp
g
![Page 8: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/8.jpg)
learningacquisitionofknowledgeorskills
throughstudy,experience,orbeingtaught
analyticssystematiccomputationalanalysisofdataor
statisticsor
informationresultingfromthesystematicanalysisofdataorstatistics
![Page 9: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/9.jpg)
Learning analytics is the measurement, collection,analysis and reporting of data about learners andtheir contexts, for purposes of understanding andoptimising learningand theenvironments inwhichitoccurs.
Siemens(2011)
![Page 10: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/10.jpg)
Outline
• Whatislearninganalytics?• Whereareweat?• Howaredashboardsused?• Howcanweevaluatelearninganalyticstools?• Howaboutaninstitutionallearninganalyticspolicy?• Wheredowegofromhere?
![Page 11: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/11.jpg)
LevelsofLearningAnalytics
![Page 12: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/12.jpg)
LearningAnalyticsFramework
Greller,W.andDrachsler,H.(2012).TranslatingLearningintoNumbers:AGenericFrameworkforLearningAnalytics.EducationalTechnology&
Society,15(3):42–57.
![Page 13: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/13.jpg)
LAshouldbeforteachersandlearners
![Page 14: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/14.jpg)
LASophisticationModel
Siemens,G.,Dawson,S.,&Lynch,G.(2014).ImprovingtheQualityandProductivityoftheHigherEducationSector–PolicyandStrategyforSystems-LevelDeploymentofLearningAnalytics.Canberra,Australia:OfficeofLearningandTeaching,AustralianGovernment.
![Page 15: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/15.jpg)
Drachsler,H.andGreller,W.(2016).PrivacyandAnalytics–it’saDELICATEissue.AChecklisttoestablishtrustedLearningAnalytics.InProceedingsoftheSixthInternationalConferenceonLearningAnalyticsandKnowledge,LAK’16,pages89–98,NewYork,NY,
USA.ACM.DOI:10.1145/2883851.2883893.
![Page 16: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/16.jpg)
http://www.laceproject.eu/ethics-privacy-learning-analytics/
![Page 17: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/17.jpg)
http://www.eugdpr.org/
![Page 18: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/18.jpg)
Outline
• Whatislearninganalytics?• Whereareweat?• Howaredashboardsused?• Howcanweevaluatelearninganalyticstools?• Howaboutaninstitutionallearninganalyticspolicy?• Wheredowegofromhere?
![Page 19: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/19.jpg)
LearningAnalyticsDashboards
http://w
ww.learno
nsolutions.com
/wp-conten
t/up
loads/20
16/10/learning-analytics-learno
n-85
0x40
0.jpg
![Page 20: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/20.jpg)
Structure
Source:https://www.flickr.com/photos/bruneluniversity/12926707974/
![Page 21: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/21.jpg)
Learninganalyticsdashboardsare…
…“singledisplaysthataggregatedifferentindicatorsaboutlearner(s),learningprocess(es)and/orlearningcontext(s)intooneormultiplevisualisations”Targetusers:teachers(75%),learners(51%),administratorsandresearchersLearningscenarios:formal(91%),non-formal,informalPurpose:Supportinformeddecisionmaking
Schwendimann,B.A.,Rodriguez-Triana,M.J.,Vozniuk,A.,Prieto,L.P.,Boroujeni,M.S.,Holzer,A.,...&Dillenbourg,P.(2017).Perceivinglearningataglance:Asystematicliteraturereviewof
learningdashboardresearch.IEEETransactionsonLearningTechnologies,10(1),30-41.
![Page 22: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/22.jpg)
Thelearninganalyticsprocess
Verbert,K.,Duval,E.,Klerkx,J.,Govaerts,S.,&Santos,J.L.(2013).Learninganalyticsdashboardapplications.AmericanBehavioralScientist,57(10),1500-1509.
![Page 23: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/23.jpg)
Visualisation
Indicators
Educationalfoundation
• Barchart• Linegraph• Piechart• Scatterplot• Socialgraphs• Text
• Learnercharacteristics• Contextoflearning• Content• Learneractions• Learningoutcomes• Socialinteractions
• Feedbacktheory• Self-regulatedlearning• Collaborativelearning• 21stcenturyskills• Bloom’staxonomy• Socialcomparison
Learninganalyticsdashboards
![Page 24: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/24.jpg)
Learninganalyticsdashboards
Jivet,I.,Scheffel,M.,Drachsler,H.,andSpecht,M.(2017).Awarenessisnotenough.Pitfallsoflearninganalyticsdashboardsintheeducationalpractice.InProceedingsofthe12thEuropean
ConferenceonTechnologyEnhancedLearning(EC-TEL2017),LNCS,pages82-96.Berlin.Springer.
![Page 25: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/25.jpg)
Learninganalyticsdashboardsforteachers
Monroy,C.,Rangel,V.S.,&Whitaker,R.(2013).STEMScopes:contextualizinglearninganalyticsinaK-12sciencecurriculum.InLAK’13(pp.210-219).ACM.
![Page 26: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/26.jpg)
Learninganalyticsdashboardsforlearners
Corrin,L.,&deBarba,P.(2014).Exploringstudents’interpretationoffeedbackdeliveredthroughlearninganalyticsdashboards.ASCILITE2014.pp.629-633.
![Page 27: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/27.jpg)
Learninganalyticsdashboardsforlearners
Beheshitha,S.S.,Hatala,M.,Gašević,D.,&Joksimović,S.(2016).Theroleofachievementgoalorientationswhenstudyingeffectoflearninganalyticsvisualizations.InProceedingsLAK’16(pp.54-63).
ACM.
![Page 28: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/28.jpg)
Learninganalyticsdashboardsforlearners
Aljohani,N.R.,&Davis,H.C.(2013).Learninganalyticsandformativeassessmenttoprovideimmediatedetailedfeedbackusingastudentcenteredmobiledashboard.NGMAST2013,pp.262-267.IEEE.
![Page 29: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/29.jpg)
Learninganalyticsdashboardsforteachers
![Page 30: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/30.jpg)
Learninganalyticsdashboardsforlearners
![Page 31: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/31.jpg)
TheLearningTracker
Designgoals1. Traceable2. Scrutable3. ActionableIterativedesign1. Interactive2. Planningahead3. Framing
![Page 32: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/32.jpg)
TheActivityWidget
![Page 33: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/33.jpg)
TheLearningPathWidget
![Page 34: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/34.jpg)
Recommendationsfordesigningdashboards
1. Define the goal of the dashboard and desired impact on the learners.
2. Research educational concepts as theoretical grounding. 3. Describe data that supports the goal. 4. Experiment with different visualisations. 5. Customise the dashboard to provide the same support to all
users. 6. Integrate the dashboard into the learning activities. 7. Focus dashboard evaluation on goal fulfillment, impact on
learners' affect and motivation, and on the usability of the tool. 8. Tool's usability should also assess how learners interpret
information. 9. Use multiple data sources to validate dashboard effects.
Jivet,I.,Scheffel,M.,Drachsler,H.&Specht,M.(2017).Awarenessisnotenough.Pitfallsoflearninganalyticsdashboardsintheeducationalpractice.EC-TEL2017.
![Page 35: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/35.jpg)
Outline
• Whatislearninganalytics?• Whereareweat?• Howaredashboardsused?• Howcanweevaluatelearninganalyticstools?• Howaboutaninstitutionallearninganalyticspolicy?• Wheredowegofromhere?
![Page 36: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/36.jpg)
Whereistheevidence?
http://www.laceproject.eu/evidence-hub/
![Page 37: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/37.jpg)
Learninganalyticstools
• Demandsofthestakeholders?• Empiricalevidence?• Desiredeffect?• Indicatorsfortoolevaluation?• Comparability?• Standardisedevaluation?• Impact?• Benefit?
![Page 38: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/38.jpg)
TheEvaluationFrameworkforLA
![Page 39: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/39.jpg)
Finally, if there isn’t a relevant questionnaire foryoursituation,youcandeviseone.
JohnBrooke(2013)
![Page 40: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/40.jpg)
ObjectivesforEFLAcreation
• Identifyqualityindicatorsforlearninganalytics
• Createanapplicableevaluationinstrumentbasedontheseindicators
• Validatetheevaluationinstrument
![Page 41: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/41.jpg)
EFLA1
ObjectivesAwareness Reflection Motivation Behavioural
Change
Learning SupportPerceived Usefulness Recommendation Activity
ClassificationDetection of
Students at Risk
Learning Measures and OutputComparability Effectiveness Efficiency Helpfulness
Data AspectsTransparency Data Standards Data Ownership Privacy
Organisational AspectsAvailability Implementation Training of Educa-
tional StakeholdersOrganisational
Change
![Page 42: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/42.jpg)
EFLA2
DATA-I know what data is being collected-I have access to my data-I understand the presented results
L E A R N E R S
-I know what data is being collected-I have access to my students’ data-I understand the presented results
AWARENESS-I am aware of my current learning status-I comprehend my current learning status-I can project my future learning status
-I am aware of my students’ current learning status-I comprehend my students’ current learning status-I can project my students’ future learning status
REFLECTION-I reflect on my learning activities-I reflect on alternative learning activities-I know when to change my behaviour
-I reflect on my teaching activities-I reflect on alternative teaching activities-I know when to change my behaviour
T E A C H E R S
IMPACT-I can detect whether I am falling behind-I study more efficiently-I study more effectively
-I can detect whether my students are falling behind-My students learn more efficiently-My students learn more effectively
![Page 43: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/43.jpg)
EFLA3
DATA-For this LA tool it is clear what data is being collected-For this LA tool it is clear why the data is being collected-For this LA tool it is clear who has access to the data
L E A R N E R S
-For this LA tool it is clear what data is being collected-For this LA tool it is clear why the data is being collected-For this LA tool it is clear who has access to the data
AWARENESS-This LA tool makes me aware of my current learning situation-This LA tool makes me forecast a possible future learning situation given my (un)changed behaviour
-This LA tool makes me aware of my students’ current learning situation-This LA tool makes me forecast my students’ possible future learning situation given their (un)changed behaviour
REFLECTION-This LA tool stimulates me to reflect on my past learning behaviour-This LA tool stimulates me to adapt my learning behaviour if necessary
-This LA tool stimulates me to reflect on my past teaching behaviour-This LA tool stimulates me to adapt my teaching behaviour if necessary
T E A C H E R S
IMPACT-This LA tool increases my motivation to study-This LA tool stimulates me to study more efficiently-This LA tool stimulates me to study more effectively
-This LA tool increases my motivation to teach-This LA tool stimulates me to teach more efficiently-This LA tool stimulates me to teach more effectively
![Page 44: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/44.jpg)
EFLA4
DATA-For this LA tool it is clear what data is being collected-For this LA tool it is clear why the data is being collected
L E A R N E R S
-For this LA tool it is clear what data is being collected-For this LA tool it is clear why the data is being collected
AWARENESS & REFLECTION-This LA tool makes me aware of my current learning situation-This LA tool makes me forecast my possible future learning situation given my (un)changed behaviour-This LA tool stimulates me to reflect on my past learning behaviour-This LA tool stimulates me to adapt my learning behaviour if necessary
-This LA tool makes me aware of my students’ current learning situation-This LA tool makes me forecast my students’ possible future learning situation given their (un)changed behaviour-This LA tool stimulates me to reflect on my past teaching behaviour-This LA tool stimulates me to adapt my teaching behaviour if necessary
T E A C H E R S
IMPACT-This LA tool stimulates me to study more efficiently-This LA tool stimulates me to study more effectively
-This LA tool stimulates me to teach more efficiently-This LA tool stimulates me to teach more effectively
![Page 45: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/45.jpg)
EFLA
• isvalid&reliable,
• addressesthelackofevaluationinstruments,
• complementsevaluationonaninstitutionallevel,
• addressestheneedforevidencethathasbeencalledfor,
• iseasytousewithtemplates&interactivespreadsheet
![Page 46: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/46.jpg)
DATAFor this LA tool it is clear what data is being collected
The Evaluation Framework for Learning AnalyticsE F L A
AWARENESS & REFLECTION
IMPACT
forL E A R N E R S
For this LA tool it is clear why the data is being collectedstrongly
disagreestrongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
This LA tool makes me awareof my current learning situation
This LA tool stimulates me to reflecton my past learning behaviour
This LA tool makes me forecast my possible future learning situation given my (un)changed behaviour
strongly disagree
strongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
stronglydisagree
strongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
strongly disagree
strongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
stronglydisagree
strongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
This LA tool stimulates me to adaptmy learning behaviour if necessary
stronglydisagree
strongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
This LA tool stimulates me to study more efficientlystrongly
disagreestrongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
This LA tool stimulates me to study more effectivelystrongly
disagreestrongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
DATAFor this LA tool it is clear what data is being collected
AWARENESS & REFLECTION
IMPACT
forT E A C H E R S
For this LA tool it is clear why the data is being collectedstrongly
disagreestrongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
This LA tool makes me awareof my students’ current learning situation
This LA tool stimulates me to reflecton my past teaching behaviour
This LA tool makes me forecast my students’ possible future learning situation given their (un)changed behaviour
stronglydisagree
strongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
stronglydisagree
strongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
stronglydisagree
strongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
stronglydisagree
strongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
This LA tool stimulates me to adaptmy teaching behaviour if necessary
stronglydisagree
strongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
This LA tool stimulates me to teach more efficientlystrongly
disagreestrongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
This LA tool stimulates me to teach more effectivelystrongly
disagreestrongly agree
1O
2O
3O
4O
5O
6O
7O
8O
9O
10O
EFLA scoring per stakeholder groupStep 1: Calculate the average value for each item based on the answers given for that item.
Step 2: Calculate the average value for each dimension based on the average of its items.
Step 3: Calculate the dimensional scores by rounding the results of ((x-1)/9)*100 where x is the average value of a dimension.
Step 4: Calculate the overall EFLA score by taking the average of the three dimensional scores.
![Page 47: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/47.jpg)
http://www.laceproject.eu/evaluation-framework-for-la/
![Page 48: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/48.jpg)
Outline
• Whatislearninganalytics?• Whereareweat?• Howaredashboardsused?• Howcanweevaluatelearninganalyticstools?• HowaboutaninstitutionalLApolicy?• Wheredowegofromhere?
![Page 49: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/49.jpg)
PolicyFramework
http://sheilaproject.eu/
![Page 50: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/50.jpg)
PolicyFramework
ACTION CHALLENGES POLICY
culture capabilities datamanagement
ethics&privacy culture evaluation
financial&humanresources
ethics&privacy methodology
infrastructure infrastructure policymanagement
internal&externalsupport management purpose
methodology methodology stakeholderengagement
purpose
stakeholderengagement
![Page 51: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/51.jpg)
Outline
• Whatislearninganalytics?• Whereareweat?• Howaredashboardsused?• Howcanweevaluatelearninganalyticstools?• Howaboutaninstitutionallearninganalyticspolicy?• Wheredowegofromhere?
![Page 52: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/52.jpg)
LASophisticationModel
Siemens,G.,Dawson,S.,&Lynch,G.(2014).ImprovingtheQualityandProductivityoftheHigherEducationSector–PolicyandStrategyforSystems-LevelDeploymentofLearningAnalytics.Canberra,Australia:OfficeofLearningandTeaching,AustralianGovernment.
![Page 53: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/53.jpg)
ProblemofLAUptake
EducationalScience
ComputerScience
LearningAnalytics
MIDDLESPACE
![Page 54: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/54.jpg)
Solution:UseLAasarealmiddlespace
EducationalScience
ComputerScience
LearningAnalytics
MIDDLESPACE
![Page 55: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/55.jpg)
Learninganalytics…
… is not about the creation of intelligent or stupidsystems…… but about the creation of intelligent and successfulstudents…
…andthepromotionofeducation.
![Page 56: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/56.jpg)
Ifeducationisviewedasavehicleforadvancingtheglobaleconomy,thenitmustbetheNorthStarthatguides societies to the next big thing, illuminatingnew ideas that solve pressing challenges andcreatingopportunitiestoshapeabetterfuture.
SamanthaAdamsBeckeretal.(2017)
![Page 57: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/57.jpg)
Nowit’syourturn…
…tocreateyourowndashboard!
![Page 58: Learning Analytics - Competen-SEAcompeten-sea.eu/wp-content/uploads/2018/02/... · Recommendations for designing dashboards 1. Define the goal of the dashboard and desired impact](https://reader033.vdocument.in/reader033/viewer/2022042222/5ec8824337185c26cf048d8e/html5/thumbnails/58.jpg)
Thankyouforyourattention!
[email protected] [email protected]@m_a_s_c @IoanaJJEFLA:www.laceproject.eu/evaluation-framework-for-la/LAattheOU:www.ou.nl/en/welten-open-learning-analytics