Learning analytics:

answering the FAQs

Rebecca Ferguson

The Open University

SCONUL: June 2014

Take a minute to write down

or tweet the three questions

that would head your list of

FAQs about learning analytics

#sconulfaqs

Who are you?

• The Open University: largest in Europe

• Making use of data for more than 40 years

• Informal learning: iTunes, YouTube…

• MOOCs on FutureLearn and OpenLearn

• Learning analytics research / events

• LACE project

What are analytics?

High-level figures Brief overview for internal and external reports

Academic analytics Figures on retention and success, for the institution to

assess performance

Educational data mining Searching for patterns in the data

Learning analytics Use of [big] data to provide actionable intelligence for

learners and teachers

Is there a precise definition?

The measurement, collection, analysis and

reporting of data about learners and their

contexts, for purposes of understanding and

optimizing learning and the environments in

which it occurs.

Why do learners use analytics?

• Monitor their own activities and interactions

• Monitor the learning process

• Compare their activity with that of others

• Increase awareness, reflect and self reflect

• Improve discussion participation

• Improve learning behaviour

• Improve performance

• Become better learners

• Learn!

Why might I use analytics?

• Monitor the learning process

• Explore student data

• Identify problems

• Discover patterns

• Find early indicators for success

• Find early indicators for poor marks or drop-out

• Assess usefulness of learning materials

• Increase awareness, reflect and self reflect

• Increase understanding of learning environments

• Intervene, advise and assist

• Improve teaching, resources and the environment

Can you give me an example?

• Aligned with

clear aims

• Huge and

sustained effort

• Agreed proxies

for learning

• Clear and

standardised

visualisation

• Driving

behaviour at

every level Individual assessment within cohort

Is that an ideal example?

Can we achieve this?

• Aligned with clear aims

• Huge and sustained effort

• Agreed proxies for learning

• Clear and standardised visualisation

• Driving behaviour at every level

Can we avoid this?

• Instructivist approach

• Stressed, unhappy learners

• Analytics with little value for learners or teachers

• Omission of key areas, such as collaboration

Don’t start with the data –

start with the pedagogy

How do people learn?

How can I use data to facilitate that process?

Social learning analytics:

How do people learn socially & in social settings?

How can I use data to facilitate that process?

How could we achieve that?

What questions should I ask?

• Which elements are learners struggling with?

• Which sections engage them the most?

• What prompts them to ask questions?

• How are they navigating resources?

• What misconceptions have they shown?

• Are there any accessibility issues?

How can analytics be used to

achieve desired learning outcomes?

• Demographic data

• Calendar information about assignments

• VLE activity data (including forums).

• Lists of required reading

• Library resources usage data

• Library helpdesk enquiries

• Library website usage and analytics data

• Assessment results

• User survey results

• Student retention and attainment data

What data could I use?

Does HE use learning analytics?

Purdue

University

SNAPP:

British Columbia

Are all analytics like that?

Disparate data

VLE (Moodle) data

Survey data

Help desk data

Library data

Assessment data

Registration data 250,000 students

around the world

Faculty expertise

Learning design

Learning outcomes

Assessment strategy

Module connections

Resource understanding

Data

Wranglers

Do you have an OU example?

Resource page? Sub page? URL?

Significance of dates?

What does a Wrangler do?

Reach agreement on conventions

80% agreed or strongly agreed that….

Is that

80% of registered students?

80% of those who completed the survey?

80% of those who completed both survey and course?

Or is it

80% of survey respondents who completed the

course and did not select ‘not applicable’

What else do Wranglers do?

Relate analytics to learning design

How many forums?

What were their roles?

How were they timetabled?

Anything else?

Support, not surveillance

Reporting at module level, not individual level

Survey responses kept apart from activity data

Making data accessible, not overwhelming

Negotiating meaning, not dictating interpretation

Are there any ethical issues?

Shared knowledge building

Is it all about wrangling?

Shared knowledge building

Does your library use analytics?

What about informal learning?

Social

Network

Status

Domain

Expert

Reputation

Massive

thousands may sign up

Open

no payment is required

Online

resources on the Internet

Courses

time-bounded cohorts

Any links with MOOCs?

(What are MOOCs?)

Introduction to

Forensic Science

University of Strathclyde

You have been actively engaged in

the discussions, which is excellent,

thank you, but with more than

23,000 participants it means that our

responses and comments risk

getting lost.

Analytics could help to filter discussions

How could MOOCs use analytics?

Introduction to

Forensic Science

University of Strathclyde

This will be primary school material

for some of you and exactly the

opposite for others. It is just not

possible to tailor the material to

each of you […]

How could MOOCs use analytics?

Analytics could help to group learners

Analytics could help to focus attention

How could MOOCs use analytics?

Helping to identify useful analytics

What do learners need to know in order to

network, collaborate, browse or reflect?

What do educators need to know to support them?

Helping to identify gaps in the data

Which data do we need to collect?

Helping to identify gaps in our toolkit

Which design elements can we look at easily?

Which ones still pose problems?

Any links with learning design?

Helping to frame and focus analytics questions

What did they learn?… in relation to learning outcomes

Were they social?... when they were collaborating

Did they share links?... when encouraged to browse

Did they return to steps?... when encouraged to reflect

Helping to identify appropriate forms of analysis

The same content, but with a focus on

Number of visits if content

Length, quality, number of comments if conversational

Dwell time and repeat visits if reflection

Any links with learning design?

How are analytics used in libraries?

Mission

To impact student measures of satisfaction

To demonstrate value-added to users

To inform and justify policy

Teaching and learning

To target and assess interventions

To position library teaching effort

To track user journeys

Recommendations

To promote use of content beyond reading list

To link discovery, lists and recommendations

To surface unused content jisclamp.mimas.ac.uk

How are analytics used in libraries?

Collection management

To deliver better / best value for money

To promote evidence-led decision-making

To identify how faculties use the library

Data

To merge data from multiple systems

To incorporate web analytics

To provide compelling visualisations

Service

To correlate feedback, enquiries and collection strength

To understand access patterns

To target skills development jisclamp.mimas.ac.uk

How are you using analytics?

Collection management

Mission

Data

Teaching and learning

Recommendations

Service

jisclamp.mimas.ac.uk

bit.ly/1130R0Y

bit.ly/VUGBfJ

Where can I find out more?

jisclamp.mimas.ac.uk

bit.ly/1qulB06