Three paths for learning analytics and beyond: Moving from rhetoric to reality
Colin Beer
CQUniversity
David Jones
University of Southern Queensland
Rolley Tickner
CQUniversity
http://www.flickr.com/photos/dragnfly78/235652252/
0
5
10
15
2010 2011 2012 2013 2014
Learning analytics @ ASCILITE
http://flickr.com/photos/boskizzi/3241710/ http://flickr.com/photos/boskizzi/3241710/
Oz/NZ Horizon Report
Year Time Frame Label
2010 4 to 5 years Visual data analysis
2012 1 year or less (#2) Learning analytics
2013 1 year or less (#1) Learning analytics
2014 2 to 3 years Learning analytics
http://www.flickr.com/photos/rchughtai/2121560287/
Learning analytics is essential for penetrating
the fog that has settled over much of higher
education(Siemens and Long, 2011, p. 40)
Learning analytics can contribute to course
design, student success, faculty
development, predictive modelling and
strategic information(Diaz & Brown, 2012)
Learning analytics has been identified as a
key future trend in learning and teaching
(Johnson et al, 2013; Lodge & Lewis, 2012;)
http://www.flickr.com/photos/mikecogh/5959192031/
I’m not familiar with (m)any universities that
have taken a systems-level view of LA.
http://bit.ly/16uz8vU
Most of what I’ve encountered to date is specific
research projects or small deployments of LA. I have
yet to see a systemic approach to analytics use/adoption
http://www.flickr.com/photos/johnhaydon/5042881685/
Basing decisions on data and evidence
seems stunningly obvious
(Siemens and Long, 2011, p. 31)
http://flickr.com/photos/boskizzi/3241710/
http://flickr.com/photos/boskizzi/3241710/
Management fashion is "relatively transitory collective
beliefs, disseminated by the discourse of knowledge
entrepreneurs, that a management technique is at the
forefront of rational management progress”
(Abrahamson and Fairchild, 2003)
Amplified by hyperbole…, the fashionable vision
may exert a strong, if transitory, normative pull
among managers.
(Swanson and Ramiller, 2004)
Fad cycle
http://www.flickr.com/photos/moriza/308483890/
1. Technological spark
2. Growing revolution
3. Minimial impact
4. Resolution of dissonance
(Birnbaum, 2000)
http://www.flickr.com/photos/tambako/8592709246/
Learning Analytics
The typical
approach to
implementation
=
Three paths
Do it to Do it for Do it with
Early Alert Student Indicators (EASI)
BI Dashboard
Moodle
EASI
LA Database
Early Alert Student Indicators (EASI)
Early Alert Student Indicators (EASI)
Early Alert Student Indicators (EASI)
Early Alert Student Indicators (EASI)
Early Alert Student Indicators (EASI)
Early Alert Student Indicators (EASI)
Early Alert Student Indicators (EASI)
Early Alert Student Indicators (EASI)
Early Alert Student Indicators (EASI)
Early Alert Student Indicators (EASI)
Early Alert Student Indicators (EASI)
• Formal project proposed during 2012
• 12+ iterations of the project initiation documentation
• Details on project scope, deliverables, budget, milestones and quality
• 3 x major budget revisions
• Project officially started in 2014 term 1 with an institution wide pilot
Early Alert Student Indicators (EASI)
• Over 85,000 nudges delivered• Used by a majority of teaching staff• Deemed by senior management to be a
successful project• Contributed to Tier 1 & 2 L&T awards• Project was delivered on time and on budget
https://flic.kr/p/6zTqgL
https://flic.kr/p/5hb8RQ
https://flic.kr/p/5Py8CH
http://flickr.com/photos/tonymangan/754511201/
Pitfalls
Complex and likely to fail
Resistance Compliance
Failures of rationality
Disappearing data
Loss of information
Tail wagging the dog
Do it to
http://flickr.com/photos/tonymangan/754511201/
Pitfalls
Complex and likely to fail
Resistance Compliance
Failures of rationality
Disappearing data
Loss of information
Tail wagging the dog
Do it toAuthor Planning Learning
Weick & Quinn (1999) Episodic change Continuous change
Brews & Hunt (1999) Planning school Learning school
Seely Brown & Hagel (2005)
Push system Pull systems
Hutchins (1991) Supervisor reflection and intervention
Local adjustment
Truex et al (2000) Traditional design Emergent design
March (1991) Exploitation Exploration
Boehm & Turner (2003) Plan-driven Agile
Mintzberg (1989) Deliberate strategy Emergent Strategy
Kurtz & Snowden (2007)
Idealistic Naturalistic
http://flickr.com/photos/tonymangan/754511201/
Pitfalls
The chasm
Blackbox
We don’t know how?
Do it for
Arguably, teachers are the primary change agents
in any educational system.
(Mor & Mogilevsky, 2013, p.1 )
http://www.flickr.com/photos/davidking/2202649444/
Academics are pivotal to implementing changes in
Learning and teaching…
(Radloff, 2008)
Teachers operate in a complex and dynamic domain –
the background knowledge and practices of their
students constantly change, the technologies and
resources at their disposal are perpetually evolving,
and the guidance and directives they receive are
frequently updated
http://www.flickr.com/photos/davidking/2202649444/
..underlined the importance of understanding context,
and
of involving teachers in the process of developing
and
deploying analytics (Sharples et al, 2013, p. 15)
Any attempt to introduce wide-scale educational
analytics and accountability processes thus requires
a thorough
understanding of the pedagogical and technical
context in which the data are generated.(Lockyer et al., 2013, p. 2)
http://www.flickr.com/photos/9422878@N08/7788404750/
..drop the language of planning, controlling, and
measuring through which organisations, teams
and projects have been managed so far.
That language stems from heavy and slow-changing
industries and infrastructures.
(Cioborra, 2002)
http://www.flickr.com/photos/9422878@N08/7788404750/
The power of bricolage, improvisation and hacking is
that these activities are highly situated; they exploit,
in full, the local context and resources at hand, while
often pre-planned ways of operating appear to be
derooted, and less effective because they do not fit
the contigencies of the moment.
(Cioborra, 2002)
http://www.flickr.com/photos/9422878@N08/7788404750/
Competitive advantage related to ICTs can only stemfrom the cognitive and organisational capability toconvert such systems, applications and data into practical, situated, and unique knowledge for action.
(Cioborra, 2002 )
It’s a question of balance
http://www.flickr.com/photos/dullhunk/202872717/
Colin Beer – [email protected]
David Jones – [email protected]
Rolley Tickner – [email protected]