Virtual Experiences Research GroupPhD StudentsKyle Johnsen, Aaron Kotranza, John Quarels, Andrew Raij, Xiyong Wang, Brent RossenUndergraduatesJoshua Horton, Harold RodriguezFundingNational Science Foundation (CAREER, REU), University of Florida Colleges of Engineering and Medicine, Medical College of Georgia, Keele University, School of Pharmacy

Graduating Students

Kyle Johnsenhttp://www.cise.ufl.edu/~kjohnsen

Andrew Raijhttp://www.cise.ufl.edu/~raij

Harold RodriguezUndergraduate

Immersive Virtual Humans for Educating Medical and Pharmacy Communication SkillsK. Johnsen, A. Raij, B. Rossen, A. Kotranza, X. Wang, B. LokComputer and Information Science and EngineeringM. Cohen, A. StevensSurgeryJ. CendanCommunity Health and Family Medicine R. FerdigEducation

A. Deladisma, D. S. LindSurgical Oncology

S. Chapman, L. BracegirdlePharmacy

Can Virtual Humans Enable

VOSCE Project OverviewStarted Spring 04n > 250 studentsMedicalNursingPhysician AssistantPharmacyOne of the most popular VH/VR experiencesThree institutesUniversity of FloridaMedical College of GeorgiaKeele University (U.K.)TeamVR/HCI/CS 1 PhD, 6 grad students, 2 undergradsMedicine 6 MDs, 2 medical studentsEducation 2 PhDsResearch focus on interfaces

Play Video

Why Virtual Humans?StudentsRepetitionFeedbackLongitudinal learningEducatorsStandardizationDynamicAbnormal findingsCultural competencyAggregate performanceResearchersStudy the extent of impact of VHsEasy to run studies (Twiddle one thing)

What can a Virtual Human do?Respond to a sneezeShow empathyIm scared, can you help me?Could this be cancer?SneezingEvoke emotion (e.g. anxiety)Physiological effectsSocial conventionsSimilar to a humanSo what happens with VHs of different backgrounds?

VHs and BiasVHs elicit racial/ethnic biasWorking to mitigate effects of bias

Mixed Reality HumansVirtual humans have limitationsOpen research problems (e.g. AI, speech)No tactile feedbackMerge the real and virtual spacesReal toolsReal simulatorsComplete patient interactionPhysical Exams (e.g. eye exam)Point at eye chartOphthalmoscopeFollow my finger1 or 2 fingers?

Breast Simulator IntegrationBreast simulator integratedDr. Carla Pugh, NorthwesternStudent does a patient historyAsks to remove gown (physiological measures)Performs a breast examVP winces at too much pressureFuture workPelvic simulatorCentral line simulator[Kotranza (submitted) VR2008]

Potential Applications for VHsTraining Interpersonal ScenariosEmployee TrainingSocial situationsSexual HarassmentCultural Sensitivity/CompetencyBusiness DealingsPatron diversity (physical, emotional, mental)Park Visitor Experience EnhancementKiosks for patrons to interact with their own culturally knowledgeable guidePersonalized ride that allows patrons to have immersive experiences

IPS SystemInputsNatural speechTracking data (head, hands, chair, tools)VideoPhysiological measuresOutputsSpeech and animationLife-size projection (or HMD)Perspective correct rendering Reactive virtual humanCOTS components2 PCs2-4 video camerasData projector or HMDWireless microphoneBodymedia Sensewear< $10,000(USD)Potential:Every Hospital

BiasStudy (n=9), people reported that the African-American VH had less education and moneyIdentical: animations, words spokenDifferences: skin-tone, voiceOther biases to studyRace/EthnicityGenderAgeWeightSocial hot topics

Abnormal FindingsConditions difficult to represent with existing education methodsConditionsPsychomotorNeurologicalAge/race/ethnicity dependent conditionsBlurry vision scenarioCranial Nerve III (due to brain tumor)Corneal ulcerRetinal detachmentBenefitsCurricular planning of medical student exposureSupplement SP experiencesLeverage the dynamic nature of VPs[Wang ISMAR 2007, submitted VR 2008]Play Video

Current WorkClassroom incorporationCommunication course2nd year MSSurgical rotation 3rd year MSPharmacology 1st yearn>120 per yearPotentially 30,000 interactions a year @ existing institutionsVirtual InstructorReal-time response to tracked cuesPhysiological measuresPosture cuesVerbal cues

Scenarios and VPsAcute abdominal painBreast massDyspepsiaSexual historyEye examPatientsDIgital ANimated Avatar (DIANA)Elderly Diana (Edna)Manniquin Diana (Mandi)Building the following VPsMale Personality (e.g. irate)Intelligence (e.g. mentally retarded)Appearance (e.g. disfigurement, limbs, burns)

Join Us!Application domain outside medicineAdditional locations to install and test systemSupportResearch directionsStudents

Thank you!http://www.cise.ufl.edu/research/vegroupQuestions?

Virtual Experiences Research GroupPhD StudentsKyle Johnsen, Aaron Kotranza, John Quarels, Andrew Raij, Xiyong Wang, Brent RossenUndergraduatesJoshua Horton, Harold RodriguezFundingNational Science Foundation (CAREER, REU), University of Florida Colleges of Engineering and Medicine, Medical College of Georgia, Keele University, School of Pharmacy

After Action Review of VH InteractionsTaking a cue from flight simulatorsEnd-UsersSelf-reflectionUnderstandingFeedbackEducatorsQuantitative analysisIdentify trends and outliersResearchersCapturing H-VH interactionAnalysisProcess, Filter, Visualize[Raij (submission) VR 2008, SIGCHI 2008]

Fix slideNegative training transferHow can we avoid dragnet style questionsBody lean , eye gaze, jargon, audio frequency analysisCant game the systeWhat is the definition of equal?

Caption on top of diagram