Evaluating virtual and VR sims

Suzie Kardong-Edgren PhD, RN, ANEF, CHSE, FSSH, FAANNurse Scientist

“Virtual” screen based simulations…computer screen based• Computer screen based simulations• Ryerson University free and beautifully done • Many commercial companies • Many companies excellent back-end metrics• Work all sims yourself so you can debrief

Virtual reality

• Telepresence• Emerging research at this time• IT support…are they there?• Beware cyber sickness…around 5-10%• Does VR translate into Kirkpatrick level III behaviors in clinical?

Transfer validity from VR to clinical…yes!

Briefing and Debriefing for VSim or VR

• Assess student experience with programs and equipment• Clarify misperceptions• Stay in touch with student learning • Relate virtual experience to actual clinical• DASH tool- https://harvardmedsim.org/debriefing-assessment-for-

simulation-in-healthcare-dash/• Facilitator Competency Rubric-

https://sites.google.com/view/evaluatinghealthcaresimulation/fcr?authuser=0

For any new product…Usability studies needed • Ask any company what research they have done with their product• System Usability Scale from sus.gov • Research page at sus.gov https://www.usability.gov/how-to-and-

tools/index.html• User research methods at SUS https://www.usability.gov/how-to-and-

tools/methods/user-research/index.html

References

• https://business.oculus.com/• Kardong-Edgren, S., Breitkreutz, K., Werb, M., Foreman, S., & Ellertson, A.

(2018). Evaluating the usability of a second-generation VR game for refreshing sterile urinary catheterization skills. Nurse Educator, Published ahead of print July. doi: 10.1097/NNE.0000000000000570 .• Butt, A., Kardong-Edgren, S., Ellertson, A. (2018). Using game-based virtual

reality with haptics for skill acquisition. Clinical Simulation in Nursing, 16(3), 25-32. doi:10.1016/j.ecns.2017.010. (Awarded 2018 Research Article of the Year)• Kardong-Edgren, S., Fara, S., Young, M., Alinier, G. (2019). A call to unify

definition of virtual reality. Clinical Simulation in Nursing, 31, 28-34. Doi: https://doi.org/10.1016/• j.ecns.2019.02.006.

References

• Verkuyl, M., Atack, L., McCulloch, T., Liu, L., Betts, L., Lapum, J. L., Hughes, M., Mastrilli, P., & Romaniuk, D. (2018, June). Comparison of debriefing methods after a virtual simulation: An experiment. Clinical Simulation in Nursing, 19, 1-7. https://doi.org/10.1016/j.ecns.2018.03.002.• Verkuyl, M., Lapum, J. L., Hughes, M., McCulloch, T., Liu, L., Mastrilli, P.,

Romaniuk, D., & Betts, L. (2018, July). Virtual gaming simulation: Exploring self-debriefing, virtual debriefing, and in-person debriefing. Clinical Simulation in Nursing, 20, 7-14. https://doi.org/10.1016/j.ecns.2018.04.006.• The PEARLS Healthcare Debriefing Tool Academic Medicine 93(2):336,

February 2018.

Marge Verkuyl…Ryerson University Canada

• Trailer https://www.youtube.com/watch?v=oMk7Fyqqm3o• https://de.ryerson.ca/games/nursing/mental-health/• https://de.ryerson.ca/games/nursing/post-op/• https://de.ryerson.ca/games/nursing/maternity/prenatal/game/#/• https://de.ryerson.ca/games/nursing/maternity/labour-and-

delivery/game/#/• https://de.ryerson.ca/games/nursing/maternity/postpartum/game/#