learning analytics - what do stakeholders really think?

@PlymUniASTI /[email protected]/asti

Learning Analytics: What Do Stakeholders Really Think?

Prof Neil Witt, Dr Anne McDermott, Prof Pauline Kneale & Prof David Coslett

Academic Support, Technology & Innovation

Teaching and Learning Support

ASTI and Teaching and Learning Support investigating the potential of using Learning Analytics as a means of enhancing the student experience

Grant from the HEA’s Strategic Excellence Initiative for Vice-Chancellors

Background



The electronic footprint our students leave behind when they interact with our digital systems

e.g. digital learning environment, electronic library, ePortfolio

This and other data sources used to track student engagement and to identify those who may be in danger of failing

Web pages and apps used to present various data visualisations for personal tutors, students and others

Definition of Learning Analytics



Definition of Learning Analytics

Institution Faculty School Programme Module Individual

Academic Analytics

LearningAnalyticsD

ata

Granularity of Data



A student perspective?



Understanding our Stakeholders’ Perspectives

Students

Staff

Governors

Senior Leaders



Mostly information already collected in a range of ways, from a range of systems for a range of purposes

Array of challenges Systematic identification of ‘at risk’ students may place an

unsustainable obligation to act on the University

Challenges: Ethical Legal Data Technical Policy Process

Key issues, challenges and concerns



Transparency vital to maintain trust

High degree of confidence that staff would deal with their data in a ‘professional way’

Varied in degree of comfort with easy access to data about their own performance - closely monitored students seemed least worried e.g. health areas

Most, but not all, keen to compare their profile with anonymised cohort average or an ‘ideal’ student, so consider ‘opt out/in’

Could motivate some but discourage others

University’s response to a ‘red flag’ should not be an automatic process but the start of a conversation

Stakeholder Perspectives: Students



More concerns than students over legal / ethical data use

Support and training to understand responsibilities

Need for openness and transparency

Focus should be on benefitting student not institution

Questioned effect on retention students leave for many reasons

Policy changes needed, e.g. attendance monitoring

Much of this data is already collected in disparate ways

Need to ensure compliance with legislation for current, retrospective and future use of data

Stakeholder Perspectives: Academic, Technical & Support Staff



Need shared vision of what is meant by Learning Analytics

Culture of respect for information and anonymity required

Concerns about the scope and quality of current data

Analytics data should be triangulated with other information

Opposing views about students having own and cohort data

‘why would they want to know?’‘what are they afraid of?’

Some concerns it could be demotivational, a distraction or encourage a strategic approach to study

Could give insight into characteristics of a successful programme, trajectory of a successful student, value added over the course of a programme (Learning Gain)

Stakeholder Perspectives: Senior Leaders



Potential to increase retention and enhance performance but would need to show a return on investment

Varied responses to having personal access

With Analytics, data must become everyone’s responsibility

Academic Analytics could aid institutional decision-making

Enable Plymouth to offer something distinctive to its students

Stakeholder Perspectives: Governors



Recommendations



LA owned at a very high level Plan for success e.g. fewer

withdrawals Define goal(s) and specify initial

measures Audit policies to identify

amendments and gaps

Implement single version of truth for data & policies

Set and resource Institution-wide standards for responding

Build-in to future procurements Consider offering ‘opt-out’

Recommendations: Policy



Consent agreements and statements in line with planned use

Choice of Learning Analytics solution Implement institution-wide

standards for responding Governance requires a

multidisciplinary team including students

Digital literacy & training for staff Be open and transparent,

particularly with students Be aware there will be false

negatives/ false positives Staff development to make

responsibilities clear and support policy changes

Recommendations: Process



Bring silos together (e.g. data warehouse)

Single version of truth needed for chosen data

Specify data currently easily accessible

Establish ownership, stewardship and users of data

Agreements with 3rd party provider to reflect new use

Unique identifier work required Work out synergies with other

existing projects (e.g. S3, Mobile With Plymouth app)

Recommendations: Technology



A response to a ‘red flag’ should not be an automatic process but the start of a conversation

Implement single version of truth for data and policies

Policy changes needed, e.g. attendance monitoring

Support and training to understand responsibilities and support policy changes

Culture of respect for information and anonymity required

Establish ownership, stewardship and users of data

Policies and data need to be owned centrally (i.e. Academic Registry)

With Analytics, data must become everyone’s responsibility

Institutional Checklist



Senior Sponsorship is essential

Single Version of the truth for all data and policies

Have an “owner” for data and revised/updated/new policies

Use available solutions (i.e. Jisc toolkit, Mobile With Plymouth, S3)

Use analytics to support personal tutoring and institutional decision making

Learning Analytics is about Culture Change, not technology

Making it so


Thank you

Questions?

learning analytics - what do stakeholders really think?

Education