tools and methods to enhance information seeking, sensemaking and learning

Tools and Methods to Enhance Information Seeking,

Sensemaking, and Learning

Dragan Gašević@dgasevic

https://semtech.athabascau.ca

Exciting times

MOOC-mania

http://www.scientificamerican.com/article.cfm?id=massive-open-online-courses-transform-higher-education-and-science

Scientific American, March 13, 2013

Aren’t they great?!

They offer good resources, but…

CHALLENGES

What skills to promote?

How about – critical thinking, creativity, research-intensive learning,

information seeking, sensemaking,self-directed and self-regulated learning,

…

“We teach what we can measure. If we don't measure what we care about,

it will never be taught.”

Paulo Blikinsein, LAK 2013

Three Generation of Distance Education Pedagogies

Anderson, T. & Dron, J. (2011). Three Generations of Distance Education Pedagogy, International Review of Research in Open and Distance Learning 12(3), 80-97, http://goo.gl/j3mRF

Why does it matter?!

ChallengeInformation seeking skills

Judd, T., & Kennedy, G. (2011). Expediency-based practice? Medical students’ reliance on Google and Wikipedia for biomedical inquiries. British Journal of Educational Technology, 42 (2), 351-360. doi:10.1111/j.1467-8535.2009.01019.x

Why does it matter?!

ChallengeSensemaking paradox

Butcher, K. R., & Sumner, R. (2011). Self-Directed Learning and the Sensemaking Paradox. Human–Computer Interaction, 26(1-2), 123-159. doi:10.1080/07370024.2011.556552

Why does it matter?!

ChallengeMetacognitive skills

Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-Regulated Learning: Beliefs, Techniques, and Illusions. Annual Review of Psychology, 64, 417-444. doi:10.1146/annurev-psych-113011-143823

How can we help?

Information seeking, sensemaking, & learning

Complex processes w/ dynamic feedback loops

APPROACH

Evidence-based Discipline

Integration of best research evidence with practitioner expertise and stakeholder values

The goal made up based on

Evidence-based discipline

Information seeking,sensemaking,

learning

Practice

Real world

Research

Evidence-based discipline

Information seeking,sensemaking,

learning

Practice

Real world

Research

Self-reports, laboratory, intrusive,

causality,…

Evidence-based discipline

Information seeking,sensemaking,

learning

Practice

Real world

Research

Self-reports, laboratory, intrusive,

causality,…

Users “on average”, GIGO, ..

GIGO – Garbage In Garbage Out

Intervene/instrument

Evidence-based discipline

Information seeking,sensemaking,

learning

Practice

Real world!

Research

Self-reports, laboratory, intrusive,

causality,…

Users “on average”,GIGO, …

GIGO – Garbage In Garbage Out

Intervene/instrument

Collect/Analyze

Learning Analytics – What?

Measurement, collection, analysis, and reporting of data about

learners and their contexts

Learning Analytics – Why?

Understanding and optimising learning and the environments

in which learning occurs

TOOLS

Data Collection

Importance of context

Tool and format independent Aggregates and integrates

Learning Context Ontology: LOCO

Jovanovic, J., Knight, C., Gasevic, D., Richards, G. (2007). Ontologies for Effective Use of Context in e-Learning Settings. Educational Technology & Society, 10(3), 47-59.

http://intelleo.eu

LOCO-Analyst

OAST and LOCO-Analyst

Ali, L., Hatala, M. Gašević, D., Jovanović, J., (2012). A Qualitative Evaluation of Evolution of a Learning Analytics Tool," Computers & Education, 58(1), 470-489

Learning Environment

LOCO-Analyst

Learning Analytics

Visual analytics requested (77.8%)

Ali, L., Hatala, M. Gašević, D., Jovanović, J. (2012). A Qualitative Evaluation of Evolution of a Learning Analytics Tool. Computers & Education, 58(1) 470-489.

LOCO-Analyst

LOCO-Analyst

LOCO-Analyst

LOCO-Analyst

Student comprehension

Student comprehension

Gašević, D., et al. (2011). An Approach to Folksonomy-based Ontology Maintenance for Learning Environments. IEEE Transactions on Learning Technologies, 4(4), pp. 301-314.

METHODS

Learning Analytics Acceptance Model

Ali, L., Asadi, M., Gašević, D., Jovanović, J., Hatala, M. (2013). Factors Influencing Beliefs for Adoption of a Learning Analytics Tool: An Empirical Study," Computers & Education, 62, 130–148.

Learning Analytics

What to measure?

We don’t need page access counts!

Wilson, T.D. (1999). Models in information behaviour research. Journal of Documentation, 55(3), 249 - 270, doi:10.1108/EUM0000000007145

Cognitive Presence in Online Discussions – Moderator Role

Cognitive presence Non-moderator Moderator t(df)

Control

group

Triggering event 4.29 (4.51) 1.05 (0.23) 4.18(37)*

Exploration 4.92 (4.12) 5.26 (3.43) -0.46(37)

Integration 1.42 (2.07) 2.68 (1.76) -3.05(37)**

Resolution 0.24 (0.49) 0.74 (1.01) -3.15(37)**

Other 0.89 (1.45) 0.89 (1.03) 0.00(37)

Interventi

on grou

p

Triggering event 1.43 (1.98) 1.02 (0.15) 1.35(43)

Exploration 3.64 (2.47) 2.82 (2.09) 1.83(43)

Integration 3.86 (3.00) 3.93 (2.32) -0.15(43)

Resolution 0.43 (0.85) 1.07 (1.50) -2.74(43)***

Other 0.86 (1.36) 0.61 (0.90) 1.21(43)

* p < 0.001; ** p < 0.005; *** p < 0.01

Cognitive Presence in Online Discussions – Association w/ Grades

Cognitive presence TMA1 TMA2 TMA3 TMA4 Final

Control group

Triggering event -.226 .005 -.046 -.050 -.010Exploration -.001 .141 .009 -.037 .048Integration .128 .060 .034 .043 .113Resolution .201 .027 -.023 -.054 .074Other -.028 .078 .113 .106 .154

Intervention group

Triggering event .149 -.077 -.070 .000 .016Exploration .216 .197 .163 .223 .243

Integration .156 .396** .417** .338* .454**

Resolution -.041 .060 .154 .083 .129Other .219 .046 .050 .075 .088

** p < 0.01; * p < 0.05

Analytics for Social Learning Environments

Automated content analysis and role miningOpen learner modeling

Recommendations

Self-regulated (Workplace) Learning

Trace-based Measurement Protocol

Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University.

Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University.

Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University.

Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University.

Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University.

Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University.

Self-regulated (Workplace) Learning

Self-regulated (Workplace) Learning

http://learningworksforkids.com/EF/metacognition.html

Information Interaction

Zhou, M., & Winne, P. H. (2012). Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction, 22(6), 413–419. doi:10.1016/j.learninstruc.2012.03.004

Information Interaction

Zhou, M., & Winne, P. H. (2012). Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction, 22(6), 413–419. doi:10.1016/j.learninstruc.2012.03.004

Achievement goal orientation (2x2)

Concept and relation filtering

Zouaq, A., Gašević, D., Hatala, M., "Towards Open Ontology Learning and Filtering," Information Systems, Vol. 36, No. 7, 2011, pp. 1064-1081.

Concept and relation filtering

Zouaq, A., Gašević, D., Hatala, M., "Towards Open Ontology Learning and Filtering," Information Systems, Vol. 36, No. 7, 2011, pp. 1064-1081.

FORWARD-LOOKING

Analyzing dynamic processes

http://www.processmining.org

Enough text obsession!

Forward-looking

George Siemens, LAK 2013

CLAS – Collaborative Lecture Annotation System

Multimodal analytics

Forward-looking

http://www.solaresearch.org

SUMMARY

Thank you!