Social Learning Analytics

1

CALRG 2011

Simon Buckingham Shum & Rebecca Ferguson Knowledge Media Institute & Institute of Educational Technology The Open University, Milton Keynes, UK @sbskmi / @R3beccaF

How we’re going to do this...

10mins Imagine.../background/critical questions – Simon

15mins A taxonomy of Social Learning Analytics – Rebecca

10mins SLA: more than just a bunch of techniques – Simon

15mins Open discussion...

2

Coming soon to a future near you?...

Analytics Report Application from Ali Bloggs to study Z0001 This applicant has a high risk profile: 1.   No academic study for last 15 years 2.   Low socio-economic background 3.   English as a second language 4.   Weak ICT skills 5.   His responses to the learning styles survey indicate a loner,

rather than a collaborative learner, known to be a disadvantage on this course

[click to view the 3 other risk factors] Without a Grade 3 tutor (advanced skills in 1-1 support), based on the last 5 years data there is a 37% chance of dropping out by Week 6.

[ACCEPT] [REJECT]

3

Coming soon to a future near you?...

“Hi Ann, In the last 2 weeks, it looks like you’ve been really stretching yourself. You seem to have been working on your critical thinking, with that challenge to Mike’s assumption, and the evidence-based claim about nuclear waste in your blog. Check out Donna Winter, who seems to have very different views to yours on global warming. How would you assess her position? In your next video conference tutorial, try to improve on the last three, in which you seem to have contributed only once each time.”

4

Coming soon to a future near you?...

“Did you know that two other people you know have used the Smith & Jones 2009 framework graphic? Finally, you seem to have really become a pivotal member of the Local-Global Climate Network. Good work: only a month ago you were on the edge! Why not reflect in your blog on how these groups are helping you in your long term goal to Work for the UN in Africa?”

5

6

L. Johnson, R. Smith, H. Willis, A. Levine, and K. Haywood, The 2011 Horizon Report (Austin, TX: The New Media Consortium, 2011), http://www.nmc.org/pdf/2011-Horizon-Report.pdf

Learning analytics “Learning Analytics is concerned with the collection, analysis and reporting of data about learning in a range of contexts, including informal learning, academic institutions, and the workplace. It informs and provides input for action to support and enhance learning experiences, and the success of learners.”

2nd Int. Conf. Learning Analytics & Knowledge 2012

dougclow.wordpress.com

“Academic Analytics”

8

Goldstein, P. J. (2005). Academic Analytics: The Uses of Management Information and Technology in Higher Education: Key Findings. Boulder, Colorado: Educause Center for Applied Research http://net.educause.edu/ir/library/pdf/EKF/EKF0508.pdf

•  Stage 1—Extraction and reporting of transaction-level data

•  Stage 2—Analysis and monitoring of operational performance

•  Stage 3—What-if decision support (such as scenario building)

•  Stage 4—Predictive modeling and simulation

•  Stage 5—Automatic triggers of business processes (such as alerts)

“Academic analytics can be thought of as an engine to make decisions or guide actions. That engine consists of five steps: capture, report, predict, act, and refine.” “Administrative units, such as admissions and fund raising, remain the most common users of analytics in higher education today.” Campbell, J. P. & Oblinger, D.G. (2007) Academic Analytics. EDUCAUSE http://connect.educause.edu/Library/Abstract/AcademicAnalytics/45275

Academic analytics in English schools

9

OU Analytics service: Predictive modelling

§  Probability models help us to identify patterns of success that vary between: §  student groups §  areas of curriculum §  study methods

§  Previous OU study data – quantity and results – are the best predictors of future success

§  The results provide a more robust comparison of module pass rates and support the OU in identifying aspects of good performance that can be shared and aspects where improvement could be realised

10 OU Student Statistics & Surveys Team, Institute of Educational Technology

Purdue University Signals

11

Purdue's premise: academic success is defined as a function of aptitude (as measured by standardized test scores and similar information) and effort (as measured by participation within the CMS). Using factor analysis and logistic regression, a model was programmed to predict student success based on:

•  ACT or SAT score •  Overall grade-point average •  CMS usage composite •  CMS assessment composite •  CMS assignment composite •  CMS calendar composite

Campbell et al (2007). Academic Analytics: A New Tool for a New Era, EDUCAUSE Review, vol. 42, no. 4 (July/August 2007): 40–57. http://bit.ly/lmxG2x

http://www.itap.purdue.edu/studio/signals

critical questions

12

Pause for thought...

§  in the discourse of academic analytics, there is little mention of pedagogy, theory, learning or teaching

§  what models of “learning” currently underpin analytics? If we can’t log and measure it, it’s invisible...

§  what learning phenomena should analytics track to equip learners for the complexities of C21?

§  classification schemes are the mechanisms by which we choose not only how to remember, but also systematically forget (Bowker and Star, 1999)

§  power: who is defining the measures, to what ends, and who gets to see which results? 13

social learning analytics

14

a taxonomy

Social Learning Analytics

•  Social learning network analysis

•  Social learning discourse analysis

•  Social learning content analysis

•  Social learning dispositions analysis

•  Social learning context analysis

Social network analytics

•  Networked learning uses ICT to promote connections

•  Networks consist of actors (people and resources) and the ties between them.

•  Ties can be classified by their frequency, quality or importance

SNAPP

•  Trace the growth of course communities •  Identify disconnected students •  Highlight the role of information brokers

GEPHI

•  Networks with interconnected interests •  Interests that are shared by actors in a network •  Role of information brokers in sharing resources, •  Roles played by resources in connecting networks

Tony Hirst blog.ouseful.info

Social network analysis and social learning

•  Identify and support types of interaction that promote the learning process

•  Identify interventions that are likely to increase the potential of a network to support the learning of its actors

Social learning discourse analytics

•  Educational success and failure may be explained by the quality of educational dialogue, rather than simply in terms of the capability of individual students or the skill of their teachers

•  The ways in which learners engage in dialogue are indicators of how they engage with other learners’ ideas, how they compare those ideas with their personal understanding, and how they account for their own point of view

Cohere

•  Annotations or discussion as a network of rhetorical moves

•  Users must reflect on, and make explicit, the nature of their contribution

Simon Buckingham Shum, Anna De Liddo

Exploratory dialogue

Rebecca Ferguson

Open Mentor

Analyse, visualise and compare quality of feedback

Denise Whitelock

Content analytics

Automated methods to examine, index and filter online media assets, with the intention of guiding learners through the ocean of available resources

LOCOanalyst

Provides feedback for content authors and teachers that can help them to improve their online courses (Jovanovic et al., 2008)

Visual search

Visual similarity search uses features of images such as colour, texture and shape in order to find material that is visually related

Suzanne Little

iSpot

Social content analytics draw upon the tags, ratings and additional data supplied by learners

Social learning dispositions analytics •  Learning dispositions provide a way of identifying and

naming the qualities of a good learner.

•  They comprise the seven dimensions of ‘learning power’: changing & learning, critical curiosity, meaning making, dependence & fragility, creativity, relationships/interdependence and strategic awareness

•  Dynamic assessment of learning power can be used to reflect back to learners what they say about themselves in relation to these dimensions

Ruth Deakin Crick, University of Bristol

•  Effective Lifelong Learning Inventory (ELLI) responses produce a learning profile

•  This profile forms the basis for a mentored discussion with the potential to spark and encourage changes in the learner’s activities, attitude and approach to learning

ELLI

ELLIment

Thomas Ullmann: http://people.kmi.open.ac.uk/ullmann/projects/ELLIMent

EnquiryBlogger

Rebecca Ferguson, Simon Buckingham Shum, Ruth Deakin Crick http://learningemergence.net/tools/enquiryblogger

Social learning context analytics

Taking context into account:

•  Formal settings

•  Informal settings

•  Mobile learning

•  Synchronous environments

•  Asynchronous environments

Identifying and using context

My OU Story: Liam Green Hughes Stuart Brown Tony Hirst

Affinity groups

reflections...

36

if these reshape our conception of the

future of learning – do they not also

reshape our conception of the future of learning analytics?

Tectonic shifts in the learning landscape...

TECH: online, personalised, real time, multimedia, mobile...

FREE/OPEN: expected initially: I’ll pay if it’s good enough

SOCIAL LEARNING: innovation now depends on it

VALUES: autonomy, diversity, self-expression, participation

POST-INDUSTRIAL: new institutional roles in post-industrial education system

37

Taken together, these are profound shifts in power,

relationships, economics...

Tectonic shifts in the learning landscape...

38

The emerging “2.0” landscapes for learning, scholarship and knowledge work demand new, more meaningful indicators than conventional BI/MIS

e.g. social capital, critical thinking, citizenship, habits of mind, resilience, collaboration skills, creativity, emotional intelligence…

SLA: it’s not just what they do (taxonomy) but how we use them (credibility/integrity)

39

Analytics should step beyond the C20 business

intelligence mindset (cf. C21 “pervasive BI”)

Beyond a tool for institutions to track

learners, these are tools to place in the hands of

those being tracked

Concerns about the abuse of analytics may rest on the old

power configuration of an institutionally wielded instrument, to gather

summative data

SLA are about helping people to grow as learners through

personal + collective formative feedback

Some links...

Learning Analytics blog, resources & open course http://www.learninganalytics.net 2nd Int. Conf. Learning Analytics, Vancouver, Apr 2012 http://lak12.sites.olt.ubc.ca KMi Learning Analytics R&D http://people.kmi.open.ac.uk/sbs/tag/learning-analytics

40