learning analaytics and information visualization

Learning Analytics and Information Visualization Monitoring students’ activity and participation during online course MathGeAr and MetaMath, 26 – 27 June 2014 in Tampere

Tampere University of Technology Department of Mathematics Intelligent Information Systems Laboratory (IISLab) Kirsi Kuosa, Development manager Anne Tervakari, Juho Koro, Meri Kailanto, Jukka Paukkeri

Content of the presentation

•  Learning Analytics

●  Learning Analytics for TUT Circle and Moodle

●  Research articles and references

2 Intelligent Information Systems Laboratory (IISLab)

Learning analytics

●  Learning analytics can be defined as collection of methods for measuring, collecting, analyzing and reporting of data about learners, their actions and contexts, which can be utilized for understanding and optimizing learning (e.g. LAK 2011.)

●  Focus is on the learning process including an analysis of the relationship between learner, content, teacher, and institution (Long & Siemens 2011.)


Learning Analytics – Why?

●  Information about students' participation and activity is required while the course is still in progress, for example, to

●  motivate and activate students, and promote their ongoing participation

●  help identify students who may be at risk of failing, and ●  help make decisions on pedagogical strategies, actions and

interventions. ●  The rich information about students' actions is recorded and stored

automatically in the log data of learning environment. However,

●  large amounts of data is difficult to utilize for developing learning and teaching

–  data mining techniques, analysing methods, and visualizations tools are usually not developed for pedagogical purposes

–  methods and tools may be cumbersome for use by teachers and students.


LEARNING ANALYTICS FOR TUT CIRCLE AND MOODLE

Analysing and visualizing educational learning data

TUT Circle ●  A social media enhanced learning environment developed by IISLab

of TUT. http://www.tut.fi/piiri

●  Built on Drupal, an open source content management framework.

●  Contains all the basic functionalities of a modern social media web service including possibilities

●  to publish different types of content (wiki pages, blog posts, forum messages, news, events)

●  to form groups and friendships with each others ●  to send private messages, chat, exchange opinions, create,

contribute and comment on contents, share resources within and between the groups

●  to control the visibility of their information and control the access to the information.



TUT Circle Desktop layout

Mobile layout

An interactive learning analytics tool ●  The tool implemented in TUT Circle (later on in Moodle) supports analysis

and visualization of log data from online learning environment in real time, and offers several different visualizations representing information about students’ behaviour during an online course.

●  For example, interactive visualizations representing

●  students’ activity and participation during the course in general, which offers students the possibility to compare their own activity with those of other students

●  (social) networks based on information exchange among students ●  connections between students' and learning materials

●  relationships between students' active/passive participation and academic performance

●  how students use keywords describing the core content of the learning material and terms that are relevant to the subject matter.


An interactive visualization dashboard, part 1


Time Unique viewers

Date

Views

The content can be filtered according to time, date, unique viewers and views by clicking and dragging on charts.

In the histogram presenting the distribution of students’ participation, all peaks are at the date of a deadline. This indicates a strong element of procrastination.

An interactive visualization dashboard, part 2

10

Type of content Activity of students

X-axis: number of content produced. Y-axis: views of content Size of bubble: size of content produced Green: Active Red: Passive

For teacher: possibility to make a comparison between students' activity and filter content according to an author.

For student: possibility to compare his/her own activity to other students’ activity and filter content according to an author.

Intelligent Information Systems Laboratory (IISLab)

The content can be filtered according to an author or type of content by clicking on the visualization.

Network based on information exchange (comments) among students

11

Visualization presenting network based on the students' comments on the other students' messages and content. Study programs:

Information technology

Inform. & knowl. man.

Industrial engineering

Communication engin.

Teacher


Network based on information exchange (readings) among students

12

Visualization presenting network based on students' readings of the other students' content. The nodes may be filtered by study program. Study programs:

Information technology

Electrical engineering

Mechanical engineering

Inform. & knowl. man.

Electrical engineering

Industrial engineering

Communication engin.

Teacher


Participation in weekly assignments


Comparison between discussion forums (here: weekly assignments) according to amount of content produced (number of words: green area=median, pink area=average)

Comparison between content produced by one student (orange area) and content produced on average/median

Connections between students and learning materials


Students

Learning material available in TUT Circle

Discussion forums

Cited web sources

The visualization shows connections between the student (here: Petri) and 1. discussion forums he participated in 2. chapters of learning material he cited in his messages 3. web sources he cited in his messages.

For student: Possibility to examine his/her use of references, and compare his/her own actions to other students’ actions. For teacher: Also possibility to filter information presented in visualization according to grades.

Visualization based on content analysis

Keyword added by the teacher

Most used words by the students

Blog messages

Discussion forums

Chapters of ebook

The visualization shows, for example, have the students used the keyword “verkkopalvelu” added by the teacher, and where they have used it. Thus the visualization helps a social navigation among the students.

For student: Possibility to find interesting information related to the keyword from learning material or from content produced by other students, which supports peer learning For teacher: Support for quality evaluation of the students performance (i.e. helps teacher to find out if the students use keywords in meaningful way in their works).

Content of the online learning material


The visualization represented the hierarchically structured content of the online learning material. Below the circle pack is a tag cloud visualization that represents the issues discussed in the learning material. When the user selects one or more keywords from the tag cloud, the chapters related to those words will be emphasized in the circle pack. The user can navigate to the content by clicking the chapter represented in the circle pack.

Relationship between academic performance and participation


Grade Words produced Content produced Content viewed Content read

Passive participation Active participation



Grade Content read Content viewed Content produced Words produced

Passive participation Active participation



For teacher: Possibility to make a comparison between students' participation and academic performance, and filter content according to grade and student.

For student: Possibility to compare his/her own participation to other students’ actions and academic performance.

Summa summarum ●  Learning analytics and visualization of learning data can produce valuable

information for both teachers and students. For example possibilities to

●  verify student interaction and collaboration activities, ●  see latent ties among students and learning materials or external web

resources

●  observe the evolution of student active/passive participation ●  investigate relationship between active/passive participation or

undesired behaviour like procrastination and academic performance.

●  Visualizations based on content analysis can support to obtain overviews, categorize, navigate, search for relevant content, and evaluate the quality of the content.


Please, see the video that gives an overview of the different visualizations done at IISLab in 2009-2013 in a Finnish national Campus Conexus project financed by the European Social Fund: http://vimeo.com/84297046

Benefits for the teachers ●  Help to evaluate the quality of a course’s instructional design from

viewpoint of pedagogical usability (Silius & Tervakari 2003,2007).

●  Provide useful information about students’ learning types and activity, and evolution of the participation.

●  Help to identify undesired behaviour such as procrastination, and students who might have risk of failing or dropping out. à Possibilities to make strategic interventions or provide guidance just in time.

●  Support comparison of activity and active/passive participation among the students, and the quality evaluation of the students’ performance. à Support for fair and just assessment of the students’ learning performance.

●  Help the to maintain the quality of the learning material, for example, by helping to identify the least used parts of the material.


Benefits for the students

•  Helps to monitor and evaluate own performance processes and learning outcomes, and compare own activity levels with those of other students. à Support strategic adjustments for improving own performance.

•  Helps to find new and interesting references to utilize in own works.

•  Visualization based on content analysis can help to see the overview of the actual topics discussed, reflect the quality of own content, and support educational knowledge discovery.

•  May increase motivation (especially among the students who have a competitive spirit).


Research articles 1/3 ●  Kuosa, K., Koro, J., Tervakari, A-M., Paukkeri, J. & Kailanto, M. 2014. Content analysis and

visualizations, Tools for social enhanced learning environment. 17th International Conference on Interactive Collaborative Learning (ICL 2014), 3 – 6 December 2014 in Dubai. Accepted for publication.

●  Silius, K., Kailanto, M. & Tervakari, A-M. 2011. Evaluating the quality of the social media in an educational context. International Journal of Emerging Technologies in Learning, vol. 6, issue 3, 21 -27. doi: http://dx.doi.org/ijet.v6i3.1732

●  Silius, K. & Miilumäki, T. 2009. Students’ Motivations for Social Media Enhanced Studying and Learning. In: Proceedings of International Technology Enhanced Learning Conference 2009 (TELearn 2009) on the 6-8 October 2009 Taipei, Taiwan. CD-ROM.

●  Silius, K., Miilumäki, T., Huhtamäki, J., Tebest, T., Meriläinen, J. & Pohjolainen, S. 2010. Social Media Enhanced Studying and Learning in Higher Education. In Education Engineering (EDUCON), 2010 IEEE, Conference Proceedings, 14-16 April 2010, Madrid, Spain, pp. 137-143.doi: http://dx.doi.org/10.1109/EDUCON.2010.5492586

●  Silius, K., Miilumäki, T., Huhtamäki, J., Tebest, T., Meriläinen, J. & Pohjolainen, S. 2010. Students’ Motivations for Social Media Enhanced Studying and Learning. In Knowledge Management & E-Learning: An International Journal (KM&EL), Vol. 2, No. 1, pp. 51-67. Available at http://www.kmel-journal.org/ojs/index.php/online-publication/article/view/55/39 .


Research articles 2/3 ●  Silius, K., Tervakari, A-M. & Kailanto, K. 2013. Visualizations of user data in social media

enhanced web-based environment in higher education. In: Proceedings of Global Engineering Education Conference (EDUCON), Synergy from Classic and Future Engineering Education, 2013 IEEE, the 13-15 March 2013 in Berlin, Germany. Extended version published in International Journal of Emerging Technologies in Learning, vol. 8, Special issue. DOI: http://dx.doi.org/10.3991%2Fijet.v8iS2.2740

●  Silius, K., Tervakari, A-M., Huhtamäki, J., Tebest, T., Marttila, J., Kailanto, M. & Miilumäki, T. 2011. Programming of Hypermedia – Course Implementation in Social Media. In: Proceedings of the 2011 2nd International Congress on Computer Applications and Computational Science. Advances in Intelligent and Soft Computing, 2012, Volume 144/2012, 369-376. Springer Berlin/Heidelberg. DOI: http://dx.doi.org/10.1007/978-3-642-28314-7_50

●  Silius, K., Tervakari, A-M., Kailanto, M., Huhtamäki, J., Marttila, J., Tebest, T. & Miilumäki, T. 2011. Developing an Online Publication – Collaborating among Students in Different Disciplines. In: Proceedings of the 2011 2nd International Congress on Computer Applications and Computational Science. Advances in Intelligent and Soft Computing, 2012, Volume 144/2012, 361-367. Springer Berlin/Heidelberg. DOI: http://dx.doi.org/10.1007/978-3-642-28314-7_49

●  Tervakari, A-M., Kuosa, K., Koro, J., Paukkeri, J. & Kailanto, M. 2014. Teacher’s learning analytics tools in social media enhanced learning environment. 17th International Conference on Interactive Collaborative Learning (ICL 2014), 3 – 6 December 2014 in Dubai. Accepted for publication.


Research articles 3/3 ●  Tervakari, A-M., Marttila, J., Kailanto, M., Huhtamäki, J., Koro, J. & Silius, K. 2013.

Developing Learning Analytics for TUT Circle. In: Ley, T., Ruohonen, M., Laanpere, M. & Tatnall, A. (eds.). Open and Social Technologies for Networked Learning. IFIP Advances in Information and Communication Technology. Springer Berlin/Heidelberg, 101 – 110. http://dx.doi.org/10.1007/978-3-642-37285-8_11

●  Tervakari, A-M., Silius, K. & Kailanto, K. 2013. Students’ Participation in a Social Media Enhanced Learning Environment. In: Proceedings of Global Engineering Education Conference (EDUCON), Synergy from Classic and Future Engineering Education, 2013 IEEE, the 13-15 March 2013 in Berlin, Germany. Extended version is published in International Journal of Emerging Technologies in Learning, vol. 8, Special issue. DOI: http://dx.doi.org/10.3991%2Fijet.v8iS2.2740

●  Tervakari, A-M., Silius, K., Koro, J., Paukkeri, J., and Pirttilä, O. 2014. Usefulness of information visualizations based on educational data. In Proceedings ofn the 4th IEEE Global Engineering Education Conference (EDUCON 2014) on 2nd – 6th of April 2014 in Istanbul, Turkey, 142 - 151. doi: http://dx.doi.org/10.1109/EDUCON.2014.6826081

●  Tervakari, A-M., Silius, K., Tebest, T. Marttila, J., Kailanto, M, & Huhtamäki, J. 2012. Peer learning in Social Media Enhanced Learning. International Journal of Emerging Technologies in Learning, vol. 7, issue 3, 35-42. http://dx.doi.org/10.3991/ijet.v7i3.2173


References

•  LAK 2011. 1st International Conference on Learning Analytics and Knowledge, Banff, Alberta, 27 Feb – 1 March, 2011, https://tekri.athabascau.ca/analytics/

•  Long, P, and Siemens, G. 2011. Penetrating the Fog: Analytics in Learning and Education. EDUCAUSE Review, vol. 46, no. 5, 30 – 40. https://net.educause.edu/ir/library/pdf/ERM1151.pdf

•  Silius, K. & Tervakari, A-M. 2003. An Evaluation of the Usefulness of Web based Learning Environments, The Evaluation Tool into the Portal of Finnish Virtual University,” in: Pearrocha, V. & alt. (ed.) mENU 2003 – Int. Conf. on University Networks and E-learning 2003, 8-9 May 2003 in Valencia, Spain. Proc. of mENU.

•  Silius, K. & Tervakari, A-M. 2007. Variety of Quality Experiences on Web-Based Courses. In: Spector, J. M. et al. (eds.). Proc. of the 7th IEEE Int.Conference on Advanced Learning Technologies ICALT 2007, July 18-20 2007, Niigata, Japan, 858-86. DOI: http://dx.doi.org/10.1109/ICALT.2007.278

Thank you for your attention! Questions?


Photo: Mika Hirsimäki

Campus Conexus (2009-04/2014)

•  This study is part of a Finnish national project called Campus Conexus.

•  Purpose is to strengthen the cultural practices of five Finnish universities and to promote learning and teaching.

•  Aim is to study how to engage students into university studies by enriching learning experiences for example with online communities.

•  Financed by European Social Fund.