Augmented Reality in learning

How should we design AR learning applications and evaluate their effectiveness?

ARea 14 / Oulu / 13.6.2014joanna.kalalahti@uta.fi

Game Research LabTampere Research Centre for Information and Media

School of Information SciencesUniversity of Tampere

"ProGE" — Rapid prototyping of games for learning

PARTNERS:

Trends in the use of AR in learning

New Media Consortium Horizon Reports: http://www.nmc.org/publications

Scopus abstract and citation database, search terms "augmented reality" AND (learning OR teaching OR education) (21.1.2014)

What have we learned from the learning technology effectiveness studies?• Learning game effectiveness studies as an example:

– Empirical research is fragmented – contradictory results caused by methodological and reporting flaws

– Standardized guidelines for studying the effectiveness?– Preference for mixed methods revealing something about the

learning process– Effectiveness studies should be only carried out when we have

developed a good learning game!

• Many problems are common for learning technology effectiveness studies in general and should be studied within the same framework

AR in learning - the state of the art of the research• The research results are mainly from simple trials, not yet ready to

be generalized• Some promising possibilities seem to be emerging for AR...

... as a cognitive scaffold for the transitions between abstract and concrete (for example, Augmented Anesthesia Machine)http://www.cise.ufl.edu/~jpq/Site/Mixed_Reality_for_Training.html... as a support for situational learning (for example, Environmental Detectives)http://education.mit.edu/ar/ed.html... as a support for spatial cognition (for example, Construct3D)https://www.ims.tuwien.ac.at/projects/construct3d... as a technology enabling learners to create their own content —together and alone (for example, Spicewood elementary school)http://www.aurasma.com/education/new-technology-opens-up-world-of-learning-for-marble-falls-isd-students/

AR seems to be useful for learning when...(Carmichael, Biddle & Mould 2012)

• ...we want the attention to stay in the task at hand, not to be split away from the task

• ...we want to support natural user interfaces and the possibility to manipulate the object

• ...we want to offer contextual meaning for the virtual information

• ...there is a meaningful connection between the virtual object and the real world:"When reality doesn't play a prominent role in the application, it is difficult to make a meaningful connection between virtual and real objects."

Experiences from our own pilot• Cleaning the toilet is one of the least

motivating topics in the studies of domestic and consumer services students

• The use of an AR application might make it more interesting and help students to learn

• The application was built to support the revision of the work

• Promising results• The application was not AR in a strict sense,

since AR was not suitable for this application when used with tablet computers

• Still – if we would have used AR, very similar instructional methods would have been embedded in the application...

• Was the new instructional method more important than the technology?

"Media will never influence learning" (Clark 1994)

• Is it the media that affects learning or the instructional method embedded in the media?

• We should separate instructional methods and media from each other"I claim that our failure to separate medium from method has caused enormous confounding and waste in a very important and expensive research area. We continue to invest heavily in expensive media in the hope that they will produce gains in learning. When learning gains are found, we atteribute them to the delivery medium, not to the active ingredient in instruction." (Clark 1994)

• Identification of the suitable instructional method – the active ingredient in instruction – and the most cost-effective media for the instruction

• Different media – different affordances• What about using media for its own sake?

How should we design AR applications for learning?• The existing research gives us clues about the affordances of AR for

learning• We should first define the aspired learning goals / outcomes and

match the most suitable instructional methods with them in order to achieve them

• Then we should evaluate if there exists some technological advantages in AR which could enable the instructional methods to be carried out cost-effectively

• When evaluating the effectiveness of AR in learning, the evaluation method should be matched with the type of the learning goalWhat kind of learning goals and instructional methods seems AR as a technology to be most connected with?A taxonomy might help practitioners choose AR when it seems to be a useful technology

A good start has been already made:

• A classification scheme for learning outcomes for training evaluation (Kraiger, Ford & Salas 1993, 323)

• Research Linking Game Attributes and Learning Outcomes (Bedwell et al 2012, 745-746)

• Cross-media comparison of factors that can influence student learning (Radu 2014)

Thank you!

References

• All, A., Nuñez Castellar, E.P. & Van Looy, J. 2013. A Systematic Literature Review of Methodology Used to Measure Effectiveness in Digital Game-Based Learning. Proceedings of the European Conference on Games Based Learning 2013, Reading, UK: Academic Conferences, Publishing International Limited, pp. 607-616.

• Anderson, L.W., Krathwohl, D.R., Airasian, P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R., Raths, J. & Wittrock, M.C. 2001. A Taxonomy for Learning, Teaching, and Assessing. A Revision of Bloom's Taxonomy of Educational Objectives. Addison Wesley Longman Inc.: New York.

• Backlund, P. & Hendrix, M. 2013. Educational Games – Are They Worth The Effort? A literature survey of the effectiveness of serious games. Proceedings of 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES), 11 13 September 2013, Poole, UK.

• Billinghurst, M. & Dünser, A. 2012. Augmented Reality in the Classroom. Computer. IEEE Computer Society, vol. 45, no. 7, pp. 56 63.

• Carmichael, G., Biddle, R. & Mould, D. 2012. Understanding the Power of Augmented Reality for Learning. Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, Oct 09, 2012 in Montréal, Quebec, Canada.

• Clark, R.E. 1994. Media Will Never Influence Learning. Educational Technology Re-search and Development, vol. 42, no. 2, pp. 21 29.

• Connolly, T.M., Boyle, E.A., MacArthur, E., Hainey, T. & Boyle, J.M. 2012. A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, vol. 59, no. 2, pp. 661-686.

• Dunleavy, M. & Dede, C. 2013. Augmented Reality Teaching and Learning. In J.M. Spector, M.D. Merrill, J. Elen & M.J. Bishop (Eds.) The Handbook of Research for Educational Communications and Technology (4th ed.). Springer: New York.

• Dunleavy, M., Dede, C. & Mitchell, R. 2009. Affordances and Limitations of • Immersive Participatory Augmented Reality Simulations for Teaching and Learning. Journal of Science Education and

Technology, vol. 18, no. 1, pp. 7 22.• Gagné, R.M., Wager, W.W., Golas, K.C. & Keller, J.M. 2005 (Fifth Edition). Principles of Instructional Design. Wadsworth

CENGAGE Learning: Belmont, CA, USA.• Gee, J.P. 2011. Reflections on empirical evidence on games and learning. In S. Tobias & J.D. Fletcher (Eds.) Computer

Games and Instruction. Charlotte, NC: Information Age Publishing, Inc.

References

• Girard, C., Ecalle, J. & Magnan, A. 2013. Serious games as new educational tools: how effective are they? A meta-analysis of recent studies. Journal of Computer Assisted Learning, vol. 29, no. 3, pp. 207-219.

• Hays, R.T. 2005. The effectiveness of instructional games: a literature review and discussion. Technical report 2005-004. Naval Air Warfare Center Training Systems Division: Orlando, Florida.

• Henderson, S.J. 2011. Augmented Reality Interfaces for Procedural Tasks. Columbia University: New York.• Henderson, S. & Feiner, S. 2011a. Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair.

In IEEE Transactions on Visualization and Computer Graphics 17(10), 1355-1368.• Henderson, S.J. & Feiner, S. 2011b. Augmented Reality in the Psychomotor Phase of a Procedural Task. In IEEE

International Symposium on Mixed and Augmented Reality 2011. Science and Technology Proceedings, 191-200.• Jenkinson, J. 2009. Measuring the Effectiveness of Educational Technology: What Are We Attempting to Measure?

Electronic Journal of e-Learning, vol. 7, no. 3, pp. 273 280.• Kaufmann, H. 2004. Geometry Education with Augmented Reality. Dissertation. Vienna University of Technology.• Kaufmann, H. & Schmalstieg, D. 2003. Mathematics and geometry education with collaborative augmented reality.

Computers & Graphics, vol. 27, no. 3, pp. 339 345.• Ke, F. 2009. A Qualitative Meta-Analysis of Computer Games as Learning Tools. In R. E. Ferdig (Ed.) Handbook of

Research on Effective Electronic Gaming in Education. IGI Global: New York.• Klopfer, E. 2008. Augmented Learning. Research and Design of Mobile Educational Games. MIT Press: Campbirdge,

Massachusetts. • Klopfer, E. & Squire, K. 2008. Environmental Detectives – the development of an augmented reality platform for

environmental simulations. Education Tech Research and Development 56(2), 203 228.• Kraiger, K., Ford, J.K. & Salas, E. 1993. Application of Cognitive, Skill-Based, and Affective Theories of Learning Outcomes

to New Methods of Training Evaluation. Journal of Applied Psychology, vol. 78, no. 2, pp. 311 328.• Prensky, M. 2011. Comments on Research comparing Games to Other Instructional Methods. In S. Tobias & J.D. Fletcher

(Eds.) Computer Games and Instruction. Charlotte, NC: Information Age Publishing, Inc.

References

• Quarles, J., Lampotang, S., Fischler, I, Fishwick, P. & Lok, B. 2009. Scaffolded learning with mixed reality. Computers & Graphics, vol. 33, no. 1, pp. 34 36.

• Quarles, J., Lampotang, S. Fischler, I., Fishwick, P. & Lok, B. 2008. Tangible User Interfaces Compensate for Low Spatial Cognition. IEEE Symposium on 3D User Interfaces, 8-9 March, Reno, Nevada, USA, pp. 11 18.

• Radu, I. 2014. Augmented reality in education: a meta-review and cross-media analysis. Personal and Ubiquitous Computing, January 2014 (published online).

• Radu, I. 2012. Why Should My Students User AR? A Comparative Review of the Educational Impacts of Augmented Reality. Science and Technology Proceedings, IEEE International Symposium on Mixed and Augmented Reality 2012, 5-8 November 2012, Atlanta, Georgia, USA, pp. 313 314.

• Rosenbaum, E., Klopfer, E. & Perry, J. 2007. On Location Learning: Authentic Applied Science with Networked Augmented Realities. Journal of Science Education and Technology, vol. 16, no. 1, pp. 31 45.

• Schmitz, B., Klemke, R. & Specht, M. 2012. Effects of mobile gaming patterns on learning outcomes: a literature review. International Journal of Technology Enhanced Learning (IJTEL), vol. 4, no. 5/6, pp. 345 358.

• Shelton, B.E. & Hedley, N.R. 2002. Using Augmented Reality for Teaching Earth-Sun Relationships to Undergraduate Geography Students. Proceedings of the First IEEE International Augmented Reality Toolkit Workshop, Darmstadt, Germany, September 2002. ACM Press.

• Sitzmann, T. 2011. A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel psychology, vol. 64, no. 2, pp. 489-528.

• Squire, K. & Klopfer, E. 2007. Augmented Reality Simulations on Handheld Computers. Journal of Learning Sciences, vol. 16, no. 3, pp. 371 413.

• Squire, K.D. & Jan, M. 2007. Mad City Mystery: Developing scientific argumentation skills with a place-based augmented reality game on handheld computers. Journal of Science Education and Technology, vol. 16, no. 1, pp. 5 29.

• Vandercruysse, S., Vanderwaetere, M. & Clarebout, G. 2012. Game-Based Learning: A Review on the Effectiveness of Educational Games. In M.M. Cruz-Cunha (Ed.) Handbook of Research on Serious Games as Educational, Business, and Research Tools. Hershey, PA: IGI Global.

References

• Vogel, J.J., Vogel, D.S., Cannon-Bowers, J., Bowers, C.A., Muse, K. & Wright, M. 2006. Computer gaming and interactive simulations for learning: a meta-analysis. Journal of Educational Computing Research, vol. 34, no. 3, pp. 229-243.

• Wouters, P., van Nimwegen, C. & van der Spek, E.D. 2013. A Meta-Analysis of the Cognitive and Motivational Effects of Serious Games. Journal of Educational Psychology, vol. 105, no. 2, pp. 249-265.

• Wouters, P. & van Oostendorp, H. 2013. A meta-analytic review of the role of instructional support in game-based learning. Computers & Education, vol. 60, no. 1, pp. 412 425.