learning analytics in higher education

1

www.iadlearning.com

Learning AnalyticsYour next movement towards the future

of education

Jose A Omedes

Jose.omedes@itoptraining.com

2

www.iadlearning.comWhat raises interest in LA area?

Analysis of around 300 posts on the topic

20.11%

11.41%

9.24%

8.70%8.15%4.89%

3

www.iadlearning.comIndex

• Defining Learning Analytics

• The three elements of analytics: data, analysis and action

• Learning Analytics maturity and the predictive bridge.

• Learning Analytics benefits and experiences

• A new learning era

• Implementing learning analytics

• Learning analytics ethics

• Key take aways

4

www.iadlearning.comLearning Analytics Defined

“Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments

in which it occurs”

International conference on learning analytics

5

www.iadlearning.comLearning Analytics Defined

Learning analytics is the measurement, collection, analysis and reporting of data aboutlearners and their contexts, for purposes of understanding and optimizing learning and

the environments in which it occurs

DATA

Basic asset.Raw material

to be transformed into analytical insights.

ANALYSIS

Process to addintelligence to data usingalgorithms.

ACTION

Critical step towards achieving the purpose:

Understanding& optimizing learning

International conference on learning analytics

6

www.iadlearning.comLearning Analytics and EDM

Educational Data Mining (EDM)

EDM focuses on the development of methods for exploring the unique types of data that come from an educational context. […] the objective of data mining in education is largely to improve

learning […]

Handbook of educational data mining

Educational Data Mining (EDM) ≈ Learning Analytics

7

www.iadlearning.comLA and EDM on Google

8

www.iadlearning.comData Types

Internal External

PROVIDED / OBSERVED

Learning Analytics

INFERRED

DERIVED

LearningAnalytics

Algorithms

PROVIDED: Consciously given

OBSERVED: Recorded automatically

DERIVED: Produced from other data

INFERRED: Produced using analytics

Usually based on correlation between data sets

9

www.iadlearning.comData Sources

Demographic DataNot Sensitive

Sensitive

Name, birthdate, Sex

Ethnicity, Disability, Scholarship

Academic Data

Prior Performance

Current Performance

Learner Content

Maths: A, Physics: B, Electromagnetism: B

Course 1: Assignment 1: 89% Assignment 2: 56%Course 2: Assignment 1: 35%, Assignment 2: 64%

Essay 1, Group Report B, Chat 1 …

Learning Activity Data Activity Records

2017/10/02- 10:50 Logged into LMS2017/10/02- 11:50 Accessed Library Catalog2017/10/02- 12:00 Check out library book “Human body anatomy”

Educational Context Data Context InfoCourse 1: start: 2017/09/02, duration: 10 weeks, instructor: Allan GreenCourse 2: start: 2017/08/05, duration: 15 weeks, instructor: Mike Brown

External DataSocial Account Profiles

Other apps info

Facebook, Twitter, Google +

eBook apps, xAPI enabled apps …

10

www.iadlearning.comData Sources

SIS(Student Information

Systems)

LMS / LRS(Learning Management

System)

Other Internal Systems

External Systems

Learning Analytics go beyond the LMS !!!

11

www.iadlearning.comAnalysis

Process of obtaining insights from data based ona set of statistical and machine learning based algorithms

ANALYSIS

Data Learning Analytics

12

www.iadlearning.comAnalytics maturity model

analysiscomplexity

(imposes demands on

data: volume, type, timeframe,

etc.)

Diagram Source: https://www.ibm.com/developerworks/community/blogs/jfp/entry/the_analytics_maturity_model?lang=en

13

www.iadlearning.comPredictive Analytics Bridge

DescriptiveDiagnostic

Predictive

AlgorithmsData

ReactiveUnderstand the past

ProactiveInfluence the present

14

www.iadlearning.comAction

Is the overall target of the learning analytics process

No action = Failure

Having in place the internal processes that lead to action is critical

CultureLeadership

15

www.iadlearning.comIs it worth? Benefits

Reduce drop out rates

Increase learners’ performance

Targeted proactive tutoringIncrease retention

Targeted proactive tutoring

Offer personalized learning experiencesAdaptive Learning / Adaptive content

Understand content consumption patterns & quality issues

Improve content & course quality

Instructional design

16

www.iadlearning.comIs it worth? Benefits

Cost efficient allocation

Understand which resources work and which don`t

Data-driven investment decisions

Identify and promote student success factors

Create student structured pathways towards graduation

Proactively drivesuccess

Curriculum design

17

www.iadlearning.com

Source: https://www.jisc.ac.uk/reports/learning-analytics-in-higher-education

Does it work?

18

www.iadlearning.comDoes it work?

Source: https://www.jisc.ac.uk/reports/learning-analytics-in-higher-education

Case studies show:

• Validity of the predictive models applied to learning analytics systems

• Interventions with at-risk students are effective• There are other benefits to taking a data-driven approach

19

www.iadlearning.comA new learning era … or not?

Learning Analytics: The coming third waveMalcolm Brown, Director, EDUCAUSE Learning Initiative

The Predictive Learning Analytics RevolutionEDUCASE ECAR Working Group Paper

Adaptive Learning Holds Promise for the Future of Higher Education BARNES6NOBLE at EDUCATION DIVE

20

www.iadlearning.comReality …

The world is more and more data-driven … Education is no exception.

“Learning analytics” is an important tool to improve education and to make high education institutions more competitive.

“Learning analytics” are successful only if there is action as a result of its implementation.

Institutions must cross the predictive analytics bridge to benefit from a new way of driving students success.

21

www.iadlearning.comThe decision automation leap

Prescriptive PrescriptiveHuman supported Fully automated

Learning Analytics

22

www.iadlearning.comThe decision automation leap

Impressive and completelyautomated

Really impressive !!!80% Unknown …

Never forget the human factor in education !!!

23

www.iadlearning.comImplementing Learning Analytics

How do I implement my learning analytics project?

24

www.iadlearning.comImplementing Learning Analytics

1 Start smallDurationScope

2 Don’t focus on technology Focus on specific problems you want to address

3 Go after quick winsShow there are positive outcomes and possible impact

4Involve from day one all the critical stakeholders

Don’t forget the people that would use the technology in the end

25

www.iadlearning.comLearning Analytics ReadinessUnderstand whether institutions are ready for learning analytics from multiple dimensions

LEARNING ANALYTICS READINESS JISC

Culture & Vision

Ethics & Legal Issues

Strategy & Investment

Structure & Governance

Technology & Data

LEARNING ANALYTICS READINESS INSTRUMENT

Culture & Processes

Data Management Expertise

Data Analysis Expertise

Governance / Infrastructure

Readiness perception

Kimberly E. Arnold

Steven Lonn

Matthew D. Pistilli

ANALYTICS MATURITY INDEX

26

www.iadlearning.comWhat is Readiness about?

Strategy & Vision

Culture

Governance and Processes

Ethics and Legal Issues

Investment

Technology

Data

ACTION

ANALYTICSMATURITYINDEX

27

www.iadlearning.comDon’t get trapped by the readiness loop

Strategy&Vision

Culture

GovernanceandProcesses

EthicsandLegalIssues

Investment

Technology

Data

ACTION

Readiness Assessment

1 Startsmall

2 Don’tfocusontechnology

3 Goafterquickwins

4Involvefromdayoneallthecriticalstakeholders

Start your seed project!

28

www.iadlearning.comTechnology Challenges

Data Capture Not all educational interactions happen in a digital environment

Predictions Accuracy All statistical processes draw conclusions subject to an estimated error

Partial View Learning processes go beyond what data tell us

Data Literacy Analytics consumers need the skills to interpret analytics properly

Data Variety Combine data coming from multiple sources and systems

Comparable AnalyticsNot all systems use same algorithms or measure the same wayOpen standards?

29

www.iadlearning.comLearning Analytics Ethics

Source: Accenture Tecnology: https://www.accenture.com/us-en/insight-data-ethics

Data Supply Chain

DATA ANALISYS ACTION

30

www.iadlearning.comLearning Analytics Ethics

DATA ANALISYS ACTION

• Consent

• Privacy

• Access

• Validity

• Right to opt out

• Safety

• Transparency

• Accuracy

• Validation

• Systematic &

random errors

• Obligation to act

• Failure to act

• Adverse impact

• Abuse / Gaming

• Discrimination /social

status

• Pedagogical impact

Complete ethical issues taxonomy by Sclater: https://analytics.jiscinvolve.org/wp/2015/03/03/a-taxonomy-of-ethical-legal-and-logistical-issues-of-learning-analytics-v1-0/

31

www.iadlearning.comKey Take Aways

• Learning analytics are an important asset for institutions providing important benefits but also competitiveness

• Institutions must cross the predictive analytics bridge and start influencing the future by changing the present.

• Learning analytics are about action. No action = Failure. Get ready to act and change !

• Launch your analytics seed project: start small, don’t focus on technology, go after quick wins, involve stakeholders.

• Develop your code of conduct and run analytics protecting all the stakeholders and specially the students.

32

www.iadlearning.comWhat raises interest in LA area?

Analysis of around 300 posts on the topic

20.11%

11.41%

9.24%

8.70%8.15%4.89%

33

www.iadlearning.comFor more info …

Thanks a lot!Jose A Omedes

Research and Development Director

Jose.omedes@itoptraining.com