a-lasi getting started in learning analytics (lockyer, rogers and dawson)
DESCRIPTION
Workshop held at the Australian Learning Analytics Summer Institute (A-LASI) run by Lori Lockyer, Tim Rogers and Shane DawsonTRANSCRIPT
Getting Started with Learning Analytics
Lori LockyerTim RogersShane Dawson
What about today?
• Introductions and background• From base camp to summit• Data – it seems important• Analytics for teachers• Wrap up• Beer and cookies• Questions, concerns or issues
…is the collection, collation, analysis and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning
Learning Analytics
Ed theory, Ed practice, SNA, Data mining, Machine learning, semantic, data visualisations, sense-making, psychology (social, cognitive, organisational), learning sciences
Learning Analytics
Examine large data sets – trends/ patterns or anomalies.
What do patterns indicate and what do changes in habit indicate?
Higher education: • Lots of isolated work targeting attrition. Very few
large enterprise egs.
• Commercial – IBM, SAS, Hobsons, D2L Insight, BB analytics
Current State
Education Examples
Education Examples
Education Examples
Education - Purdue
Education - UMBC
Education – UniSA
Pass/Fail, RetentionConcept understanding
Current Focus
Kentucky: 1.3% - 80 stds approx 400k
Where next?
Beyond dashboards
Predictive and recommender states
Future
Learner control
NLP – video annotations
Emotions/ face tracking
Future
Confusion Engaged
FrustratedActivity modified
Continuous state of challenge
First steps – the why, what and how
of dataImproving feedback in mass higher education
Data, data, everywhere…
•Where data is accessible it is usually lagged, scattered, indecipherable, requires manipulating, lacks context…
•Yes, there are BI reports, but they are mostly for the converted and don’t flag exceptions
…but not a digit of use•Currently, despite all the data,
•Students often don’t know how they are going
•Academics don’t know if their teaching is effective
•Program/degree owners don’t know how their students navigate their way through
•Management don’t know if the Uni is on track
Outline of data thinking process
•What is the purpose for the data?
•What data is needed (and who ‘owns’ it)
•How to work with the data?
•How to make the data actionable?
Data for what purpose?
•Student level support (success and retention)
•Educator needs – improving teaching and learning
•Program designer/owner needs – curriculum flows
•Management/QA requirements – are courses/subjects meeting standards and improving?
Who owns the data…
•…aka where do you get it? IT, Business Intelligence, Admin?
•And others, e.g. class rolls, library data, orientation attendance, in-class formative and summative assessments etc
Working with data
•All data will need various degrees of extraction and transformation
•All data needs contextualisation, and a decision about how fine-grained that needs to be
•For example, is this a problem?…
42
42 Student
42 Student Test score/100
42 Student Test score/100
10% course mark
42 Student Test score/100
10% course mark
Degree: Chemical Engineering
42 Student Test score/100
10% course mark
Degree: Chemical Engineering
Course: Shakespeare and Society
42 Student Test score/100
10% course mark
Degree: Chemical Engineering
Course: Shakespeare and Society
Class Position: 1/36
Making data actionable
•Visualising the data for summary and exception highlighting
•Trends, key junctures, cumulative risk
•Tools for action, e.g. CRMs, and business processes
Visualising critical metrics
The work of Stephen FewContext is everything
Data and analytics to support learning design and implementation
Learning Analytics:
… measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.
(LAK11 - https://tekri.athabascau.ca/analytics/)
Where and how does learning occur in HE in Australia?
• Within courses/units• which are designed predominately by
teachers (not instructional designers)• who interact with students as they
are learning• who can, may, may not, intervene in
the learning process.
How might a university teacher use data and analytics?
• Analytics to inform design decisions
• Just-in-time analytics to understand learner activity and experience during implementation
• Recommendations for learner action
• Analytics for post-implementation reflection and revision
• Support scholarship of teaching
What can data help us with?
• Moe than…– retention/attrition– “… and they liked it”
• To are they…– doing what you intended?– understanding the task?– on-task/off-task?– motivated, engaged?– actually learning anything?
Learning analytics can only help us answer these questions if they are:
- specific to the learning outcomes of the unit
- related to how we think learning occurs for such outcomes, in the discipline…
- relevant to the learning design we have put in place
In other words…
… the teaching and learning context matters.
Mapping a design
Case Based Learning Design adapted from Bennett (2002) available at http://needle.uow.edu.au/ldt/ld/4wpX5Bun.
Case Based Learning Design adapted from Bennett (2002) available at http://needle.uow.edu.au/ldt/ld/4wpX5Bun.
LMS log: Student log in; access case
Network diagram: even pattern of participation
Network diagram: teacher-centred pattern
Network diagram: even pattern of participationDocument sharing logs of contribution
LMS log: access to teacher feedback
LMS log: submission of reflection templateContent analysis: depth of reflection
Case Based Learning Design adapted from Bennett (2002) available at http://needle.uow.edu.au/ldt/ld/4wpX5Bun.
LMS log: Student log in; access case
Network diagram: even pattern of participation
Network diagram: teacher-centred pattern
Network diagram: even pattern of participationDocument sharing logs of contribution
LMS log: access to teacher feedback
LMS log: submission of reflection templateContent analysis: depth of reflection
Case Based Learning Design adapted from Bennett (2002) available at http://needle.uow.edu.au/ldt/ld/4wpX5Bun.
LMS log: Student log in; access case
Network diagram: even pattern of participation
Network diagram: teacher-centred pattern
Network diagram: even pattern of participationDocument sharing logs of contribution
LMS log: access to teacher feedback
LMS log: submission of reflection templateContent analysis: depth of reflection
Now it is your turn:Sample design or
Your learning design?
• What are the learning outcomes?• What does the design look like? Map it?• What do you want to know?• What data will inform these?• What patterns do you anticipate? • What can you do about it?
Summary
• We are already capturing a lot of data
• There’s a lot of information we are not systematically capturing
• Current or possible answer might answer our questions
• First we have to have relevant questions and know what we are prepared to do with the answers