learning analytics: exposing student data
TRANSCRIPT
Learning Analytics: Exposing Student DataMyles Blaney – University of Edinburgh
2
University of Edinburgh FAQs
• Founded 1582 (its old!)
• Ranked 17th in the world (QS Ranking)
• Over 25k students
• Over 5k staff
• Dolly the sheep & Higgs Boson
• Large and Diverse
3
VLE FAQs
• Blackboard Learn is the main VLE– Moodle trial for Online Distance Learning– Custom VLEs
• 5,500 courses created a year
• Over 70% course coverage
• Multiple 3rd Party Tools Installed and developed
4
Learning Analytics
• What do we mean by analytics?– is the discovery and communication of meaningful patterns in data
Analytics. (2015, March 12). In Wikipedia, The Free Encyclopedia. Retrieved 11:40, March 30, 2015, from http://en.wikipedia.org/w/index.php?title=Analytics&oldid=651076924
• Models– predictive\retention\informative
• Case Studies– Signals, Purdue University (US)– EBEAM, University of Huddersfield (UK) – fullCRM, University of Roehampton (UK)
• BIG DATA (bigger than Godzilla)!
5
Its just data!
• Course Context
• You cant see the data trees from the data forest
• Actionable data
• Interpretation & Repercussions (for all)
• Ethics
• Play the system
• Staff and students have different needs
• Are you interested?
• Image Source: https://media.licdn.com/mpr/mpr/p/3/005/067/090/0500a7a.jpg
6
Its just more data!
• Staff impact\support– What do I do with this data?
• Student impact\support
– I am below\above or just average
• An evolving environment
Image source: http://www.bashyamspiro.com/immigration-meditation/2012/03/08/h-1b-amendments-the-times-they-are-a-changin%E2%80%99/
7
Why now?
• Demand from students (via NSS and EUSA)– How am I doing?– How is the class doing?– Enhance student experience– VLE is part of the student experience
• Demand from schools– info on student activity– Personal Tutors– Courses
• Horizon Report (the curve of doom)
• VLE Focus although wider interest throughout University
8
Were we doing this already?
• Recent Appointment of the Chair in Learning Analytics and Informatics : Professor Dragan Gasevic
• School of Physics - CourseMarks
• Msc Surgical Science (P\T)- Dr Paula Smith– 2014/15 project – ODL Students– Identify students falling behind in their studies– Email student & discuss– Dashboard
9
1ST Step is the hardest
• Project created to review Learning Analytics (Tel009)– What do staff want?– What do students want?– What is available? (is it being used)– Have other institutions implemented?
• Project team– Wilma Alexander (Lead)
– Stephannie Hay
– Myles Blaney
• Image Source:http://www.anneatulaegwu.com/2014/09/baby-steps.html
10
What did we do?
• Literature review– JISC Analytic Series Papers– Case studies– LACE (Learning Analytics Community Engagement)– SOLAR (Society of Learning Analytics Research)
• VLE Tool Review
• Student information gathering events
• Outreach to VLE communities
• User Stories– As a…..– I want to….– So that …..
Image source: https://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcRDSWD4Nf5lb0SDbf9jlDe-fGoN6ARs8PlsGDSZP-eyqSIizNQb
11
User Stories Feedback
• 92 stories created
• 18 Staff and 2 students attended the sessions
• Stories were grouped with the project scope and aims:
Mandatory 4
In Scope 38
Already Available 35
Not in scope but of interest 21
Not in scope and already available 6
12
Tool Analysis (Learn)
– BIRT Reports– Course Reports – Stats column option in the grade center – Average grade display – Retention Centre– Performance Dashboard– Google Analytics– 3rd Party tools
• BB Learning Analytics• Eesysoft
13
1ST Step Conclusions
• Manual (e.g. spreadsheets and time)
• Existing tools don’t meet the requirements– however staff do have access to them
• Simplicity (generic)
• Students want to see the cohort (and don’t trust their friends)
• Access (what\when\where\who)
• Impact
• Its not a magical fix
• 2nd Project – if you build it they might come!
14
Student Feedback Sessions
• They love grade data! (especially grades)
• They love charts
• Personal Tutor access beneficial
• Simplicity
• No concerns around data collection
• They like a free lunch
15
2nd Step is meant to be easier
• Build Project Created
• Prototype Pilot
• Student feedback sessions
• Steering group
• Design
• Test, feedback, test, feedback, test feedback, test, feedback
• Live target date July 2015
Image Source: https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTyKuJU5_Opr9DZEQH457P85StD0TfIW_qOj6_UCysPeUNoLvFL
16
University of Edinburgh Student Analytics
• Building Block
• Access controlled at a course level
• Staff will need to enable the tool
• Staff will have access to an admin panel to select grade columns and discussion threads for use.
• Tool can display the following to students– Users click count within the course (heat map) – 4 week period – Their grade vs cohort via bar chart for all releases grades in the GC– Their discussion forum posts vs cohorts per forum
• Staff can access the same information
17
Pilot
18
Student Feedback from Pilot and Survey
• My grader is too strict compared to others
• It makes me paranoid
• I would rather have face-to-face contact
• It will motivate me
• I am happy with the info I already receive
• I can see how I am performing in the class
19
Student Feedback from Survey
• Please scale what information you would find most to least useful?
Yes57%
No40%
N/A3%
• Will it change your Learning Behaviour?
Activity
Grade
Discussion
0 20 40 60 80 100 120 140
• Not really except for a slight increase in paranoia regarding grades
• whether my grades are good or not I can find out myself by talking to my PT, data like this is not instructive to me individually, I have my own needs and goals
20
Survey Q: What do students want to see?
• 38% wanted to have more grade visibility
• 6% didn’t answer the question
• Replies varied– Attendance– Student satisfaction– Grading turn over
“worried this extra info would result in yet another useless email from the university” – student 91
21
3rd Step. Footloose!
• Currently more thoughts & questions than concrete statements
• Mixed response from students
• Follow on project for Moodle
• Management Information Project
• Empower students to take action via data
• Image Source: https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcQScVwS6ErIA-sbJMIPrlobTexjNRCp30x2SqpqHbqMC8ODdRje
22
Any Questions
Image Source: http://www.independent.co.uk/migration_catalog/article6240886.ece/alternates/w620/Untitled-2.jpeg