Visualising activity in learning networks using open data and educational

analytics

Andrew Deacon & Michael Paskevicius

Centre for Educational Technology, University of Cape Town

Southern African Association for Institutional Research

(SAAIR) Forum 2011

Centre for Educational Technology within the Centre for Higher Education Development

– Michael Paskevicius (Learning Technologist)

• Interested in social media and open education

• Previously MIO at Polytechnic of Namibia

– Andrew Deacon (Learning Designer)

• Experienced learning designer

• Significant experience analysing assessment

Agenda

• Definition of educational analytics

• Explore the data landscape of institutional learning environments, personal learning environments and social media

• Learning analytics – approaches & challenges at the University of Cape Town (Michael)

• Visualizing complex data – beyond univariant dashboards (Andrew)

• Available toolsets and concluding thoughts

An age of data

• Massive increase in data storage capability

• What about data collected within learning environments?

Source: Deloitte Consulting Source: Telegraph

Source: The Economist

Educational analytics

• The measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs. (Learning Analytics 2011 Conference site: https://tekri.athabascau.ca/analytics)

• Exploring the unique types of data that come from educational settings, and using those methods to better understand students, and the settings which they learn in. (Baker & Yacef, 2009)

• Academic analytics can be used to profile and even predict students who may be at risk, by analysing demographic and performance data of former students. (Fritz, 2011)

http://lak12.sites.olt.ubc.ca/

Educational analytics data landscape

Social media

Personal learning environments (PLE)

Institutional learning environments

• ERP Systems • Historical performance data • Learning management system data • Libraries • School application data • Turnitin Reports • Demographics

Attributes • Owned data • Accessible • Found in various databases

Attributes • External data • Mostly difficult to obtain if at all • Difficult to connect to

institutional data

Attributes • External data • Mostly difficult to obtain if at all • Difficult to connect to

institutional data • Perhaps not academic at all

The social web

If our aim is to understand people’s behaviour rather than simply to record it, we want to know about primary groups, neighbourhoods, organizations, social circles, and communities; about interaction, communication, role expectations, and social control. Allen Barton, 1968 cited in Freeman, C. (2004)

Tool name Example

Twitteralytics

Tools used

Data sources

Source: CC BY-SA 3.0

Web and activity log scraping • How do people connect with each other in collaborative

academic environments? • What types of interaction occur in a forum or chat room

discussion?

Social network analysis • What are people saying about our university in social

networks? • How are students related within social networks?

Extract method:

• Query select data via API or script (Python, PHP, screen-scraping programs)

• Group by hashtags, groups, users, topics, keywords • Often requires addition of semantic understanding (and

associated documentation)

Institutional Learning Environments

Starting point: UCT Learning Management System

How and when do students use the learning management system?

Site visits

Chat room activity

Sectioning of students

Polling of students

Content accessed

Submission of assignments

Submission of assignments

Does student LMS activity correlate to course grade?

How do students and academics engage in a course chat room? Academics

and support staff

Days in which chat occurred

Students

Chat messages linked to day of occurrence

What do students who drop the course chat mostly about?

How do students engage with academics in a chat room?

Said more by students

Said more by educators

PLEs / Social Media

Exploratory data analysis

• Getting actual social media data (vs surveys / aggregated data)

• Usage and trends Confirm what happened

• Relationships Explain how things are connected

• Comparisons Serendipity as new questions arise

UCT and social media

• Prominent links to:

– Flickr

– YouTube

– Facebook

– LinkedIn

Twitter: student survey

Would use on my cell phone Yes

SMS 99%

Webmail 94%

Facebook 92%

Wikipedia 90%

Library journals 85%

Flickr, YouTube 74%

Google Docs 63%

Skype 61%

Twitter 26%

Vula student survey, 2010 data set

Twitter: UCT chatter

• Six months of data (April – Sept 2011)

• Tweets including a UCT hashtag #UCT, #Ikeys, …

• Attributes; how tweets are amplified

• Just over 5,000 tweets

• Cannot capture everything referring to something

• Clean dataset to exclude other uses of hashtags

Twitter: apps & locations

27%

36%

20%

17%

Blackberry Twitter Ubersocial Others

Smartphone geo-location

Cell phones

Cell phones: Blackberry

Twitter: viral #UCT

Varsity Cup final

Helicopter crash

6 months of tweets

Twitter: tweeter relationships

Small number of frequent tweeters

1. Drama student (162)

2. UCT Radio (132)

3. Science student (84)

Flickr: helicopter crash at UCT

Ian Barbour - http://www.flickr.com/people/barbourians/

Twitter: helicopter crash at UCT

2 hours after the

event

• Crash or hard-landing?

• Media outlets getting re-tweeted

• Peak: 140 in 5 min

Facebook: all friend relationships

Paul Butler http://www.facebook.com/notes/facebook-engineering/visualizing-friendships/469716398919

UCT: first-year courses

Psychology and Economics courses have students registered for the largest number of other course (node size is the number of edges)

Data acquisition & preparation

• Social media data challenges

– Tools and data APIs changing

– Being commercialised (and throttled)

– Data cleaning required

Correlation and causation

• Correlation does not imply causation

– Covariation is a necessary but not sufficient condition for causality

– Correlation is not causation (could be a hint)

Conclusions

• Exploring emerging data sources – Combined institutional data sets

– Acknowledge Personal Learning Environments

– Highly fragmented social media data

– Collectively enrich existing information

• Visualisations and multivariant analysis – New exploratory tools

– Making information more accessible

Literature references

• Baker, S.J.D., Yacef, K. (2009) The State of Educational Data Mining in 2009: A Review and Future Visions: http://www.educationaldatamining.org/JEDM/images/articles/vol1/issue1/JEDMVol1Issue1_BakerYacef.pdf

• Freeman, C. (2004) The Development of Social Network Analysis: A Study in the Sociology of Science. Empirical Press: Vancouver, BC Canada.

• Fritz, J. (2011) Learning Analytics. Presentation prepared for Learning and Knowledge Analytics course 2011 (LAK11). http://www.slideshare.net/BCcampus/learning-analytics-fritz

• Kirschner, P.A., Karpinski, A.C. (2010) Facebook and academic performance. Computers in Human Behavior, 26: 1237-1245.

Software references

• Gephi – network analysis, data collection

• NodeXL – network analysis, data collection

• Twitteralytics – data collection (Google Doc)

• Word cloud – R package (wordcloud)

• Geo-location map – R package (RgoogleMaps)

• Excel – spreadsheet, charts

• SPSS – statistical analysis, graphs