learning analytics: realizing the big data promise in the csu
DESCRIPTION
The word “analytics” has become a buzzword in current educational technology conversations, applied to everything from analysis of student work to LMS usage reporting to institutional analysis of ERP data. Broadly speaking, Learner Analytics refers to the analysis of student data using statistical techniques to improve decision-making. In the context of educational technology, Learner Analytics promises to improve our understanding of effective (and ineffective) student learning and technology usage. What progress have we seen in realizing this promise? This session offers a discussion of the promise of Learner Analytics, current research findings and tools, and explores examples from CSU Chico and the CSU Office of the Chancellor.TRANSCRIPT
John Whitmer, CSU Office of the Chancellor & CSU Chico
Learner AnalyticsRealizing the “Big Data” Promise in the CSU
Download slides at: http://bit.ly/HqaHBF
Outline
1. Big Data & Analytics Promise(s)
2. National Examples of Tools & Systems
3. Learner Analytics @ Chico State
4. Q & A
1. BIG DATA & ANALYTICS PROMISE(S)
Steve Lohr, NY Times, August 5, 2009
Economist. (2010, 11/4/2010). Augmented business: Smart systems will disrupt lots of industries, and perhaps the entire economy. The Economist.
7
Source: jisc_infonet @ Flickr.com
Source: jisc_infonet @ Flickr.com
Current GPA: 3.3First in family to attend collegeSAT Score: 877
Hasn’t taken college-level math
No declared major
8
What’s different with Big Data? 4 V’s:1. Volume2. Variety3. Velocity4. Variability
(IBM & Brian Hopkins, Forrester)
Academic Analytics
“Academic Analytics marries large data sets with statistical techniques and predictive modeling to
improve decision making”
(Campbell and Oblinger 2007, p. 3)
10
Academic Analytics1. Term adopted in 2005 ELI research report
(Goldstein & Katz, 2005)
– Response to widespread adoption ERP systems, desire to use data collected for improved decision making
– 380 respondents; 65% planned to increase capacity in near future
2. Call to move from transactional/operational reporting to what-if analysis, predictive modeling, and alerts
3. LMS identified as potential domain for future growth
DD Screenshot
Learner Analytics:
“ ... measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.” (Siemens, 2011)
or said plainly:
What are students doing?
Does it matter?
Learner Analytics1. Analyze combinations of data including:
– Frequency of ed tech usage (e.g. clickstream analysis)– Student learning “outputs” (e.g. quiz scores, text answers)– Student background characteristics (e.g. race/ethnicity)– Academic achievement (e.g. grades, retention, graduation)
2. Current rsch: mostly data mining, not hypothesis-driven
3. More complex than Academic Analytics, considering: – Immaturity of ed tech reporting functionality– Translation of usage into meaningful activity– No significant difference: not what technology used, it’s how it’s
used, who uses it, and for what purpose
A few promises of analytics for faculty and students …
1. Provide behavioral data to investigate student performance
2. Inform faculty about students succeeding or at risk of failing a course
3. Warn students that they are likely to fail a course – before it’s too late
4. Help faculty evaluate the effectiveness of practices and course designs
5. Customize content and learning activities (e.g. adaptive learning materials)
What’s the promise of analytics for academic technologists?
1. Decision-making based on actual practices (not just perceptions) and student outcomes
2. Support movement of A.T. into strategic role re: teaching and learning by:– demonstrating the link between technology
and learning– distinguishing our role from a technology
infrastructure provider
Our 2 biggest barriers
Image Source: http://bit.ly/Hq9Cdg
Image Source: Utopian Inc http://bit.ly/Hq9sCq
Image Source: Privacy in the Cloud: http://bit.ly/HrF6zk
2. NATIONAL EXAMPLES OF TOOLS & SYSTEMS
SIGNALS
http://www.itap.purdue.edu/studio/signals/Purdue Signals Project
SNAPP
http://www.snappvis.org/SNAPP (Social Networks Adapting Pedagogical Practice)
KHAN
http://www.khanacademy.org/Khan Academy
OLI
http://oli.web.cmu.edu/openlearning/initiative/processCM Open Learning Initiative
PARCHMENT
http://www.parchment.com/c/my-chances/Parchment
26
3. LEARNER ANALYTICS @ CHICO STATE
27
LMS Learner Analytics @ Chico StateCampus-wide
– How are faculty & students using the LMS? – What meaningful activities are being conducted? – How does that usage vary by student background, by college, by
department?
Course level– What is the relationship between LMS actions, student background
characteristics and student academic achievement? (6 million dollar question)
– Intro to Religious Studies: redesigned in Academy eLearning, increased enrollment from 80 to 327 students first semester
Ultimate goal: provide faculty and administrators with what-if modeling tools to identify promising practices and early alerts
28
Chart from Scott Kodai, Chico State
CSU Practice
INTRODUCTION TO RELIGIOUS STUDIES (RELS 180)
34
CLOSING THOUGHTS
Call to Action
1. Metrics reporting is the foundation for Analytics
2. Don’t need to wait for student performance data; good metrics can inspire access to performance data
3. You’re *not* behind the curve, this is a rapidly emerging area that we can (should) lead ...
4. If there’s any ed tech software folks in the audience, please help us with better reporting!
Want more? Resources on Analytics
Googledoc: http://bit.ly/HrG6Dm
37
Q&A and Contact Info
Resources Googledoc: http://bit.ly/HrG6Dm
Contact Info: • John Whitmer ([email protected])• Hillary C Kaplowitz ([email protected])• Berggren, Kate E ([email protected])
Download presentation at: http://bit.ly/HqaHBF