Medical Education Research Methods and Innovative Designs (MERMAID)

How To Turn Your Educational Presentation into an On-Line Module

Gary TabasMarch 13, 2015

Goals

• Why do it?

• How to do it

• What works ?

• A look at some of what we all have done

A Case

You are a faculty member who runs a workshop for a rotating group of residence each month. The workshop runs about 2 hours and the faculty members that use to help you are no longer available. Your own workload has recently increased. Additionally not all of the residents have been able to get to the workshop because of other demands. You are thinking that you would like to ease the burden on yourself but make sure that all the residents continue to receive the necessary education.

Why Do It?

• Delivery repetition• Multiple sites• Increasing faculty demands• Increasing demands on learners’ time• Differing learning styles - adaptive learning• Flipped classroom

– Shorter group meeting time– Targeted education

Why Do It?

Learners can:

– Work at their own pace (individualized learning)

– Practice using repetition

– Complete the learning at work or at home

– Work independently or in small groups

– Work with or without a facilitator

Types of On-Line Learning

• Recorded slides with audio• Purely online - no face-to-face meetings• Blended Learning - combination of online and

face-to-face• Synchronous (chat, video conferencing)• Asynchronous (email, listserv, blog)• Self-study or group - Instructor-led

Adapted from WorldWideLearn

Simulation in Medical Education

Medical Simulation in Medical Education: Results of an AAMC Survey

On-line Module Structure

• Linear– All learners take the same path

• Cases, tutorials, quizzes, graphics

• Branching– Two or more paths through the module

• Cases, management options, tutorials, quizzes

Linear

Case Vignette

Quiz (diagnosis, treatment, management)

Tutorial

Vignette

Branching

VignetteManagement

Option 1 Option 2 Option 3

Result of option 1 Result of option 2 Result of option 3

new mgt option 1 new mgt option 2 new mgt option 3

How to do it?

How to get started:

Needs analysis, learning goals and objectives

What is the problem you want to address?

What is the gap between current knowledge and ideal?

Will your on-line instruction be primary or supplemental?

Instructor involvement?

Design

Shovelware

Use the power and flexibility of the web to enhance learning, not just put on-line what can be done without the web

Principles

1. Develop clinical scenarios (vignettes) that actively engage learners

This and subsequent slide principles 1-13 From: Posel, McGee, Fleiszer

2. Practice data acquisition

3. Emphasize critical analysis

4. Require decision-making (with consequences), hypothesis generation, treatment and management

A week later, Joshua calls the office and requests to be seen. He complains about feeling completely "wiped out" and is concerned about the new drooling out of his mouth. You agree to see him at noon.In the office he complains he has been too exhausted to work for 3 days, which is costing him income. He admits to feeling intermittently warm and achy all the time. He denies chills, night sweats, sore throat, cough, nausea, vomiting, diarrhea, dysuria, joint swelling and pain.

5. Consider uncommon situations that learners might not otherwise see.

6. Make cases longitudinal

7. Link to EBM explanations, guidelines, references

8. Emphasize the importance of communicationKathy suddenly becomes very angry, saying: "WHAT?? What do you mean you're not going to let me get a mammogram until I turn 50? Do you want me to get breast cancer? Everyone I know gets mammograms as soon as they're 40. Forget it, I'll find a doctor who wants to take care of me." She storms out of your office before you can reply.

You call Kathy the following day, and start the conversation by saying: "I am sorry that I upset you yesterday. I know you're understandably very scared since your friend found out that she has breast cancer.

My intention was to explain the risks and benefits of mammograms, because we know that unfortunately, when to start mammograms is a complicated issue. But I know that I did not make this clear yesterday, and I'm hoping we can start the conversation over."

9. Provide continuous and immediate individualized feedback

10. Provide opportunity to learn from errors:Consequences of suboptimal choices can be:

– Suboptimal outcome– Less than high value care– Patient upset

• Provide Summary

• A gluten-free diet should be initiated only after the diagnosis of celiac disease is confirmed by endoscopy/biopsy as it involves lifelong adherence and is an expensive and socially inconvenient diet.

• Asymptomatic family members can obtain genetic testing to assess their risk for celiac disease (and determine whether periodic screening should be initiated).

• While lifelong therapy is indicated, it is unclear what the real long-term risks to non-adherence are.

• Dermatitis herpetiformis is associated exclusively with celiac disease and should be treated in the same manner.

12.Use your case in other settingsPCC

13. Encourage learners to author a case– Learn by doing, think how learners think

(learning and teaching at the same time)

– Search for own evidence– Scholarship opportunity

Web design

• Consistent page organization• Good use of space• Concise text

– Bulleted, short phrases– Limit sentences to 20 words– Limit scrolling (3 screens to a page max)

• Easy navigation

Software

• Easier to purchase commercial software

• Program from scratch

• Combination of above

Case Authoring Software

• Campus System for Virtual Patients• Casus• DecisionSim (VpSim)• OpenLabyrinth• Tusk• Web-SP

http://www.medbiq.org/virtual_patient/implementers

VpSim

What Works

What does the Literature Say?

• Web-based Learning in Clinic: An RCT Cook, DA Academic Med 2005

Residents randomized to paper vs web for ambulatory topics: asthma, depression, DM, nicotine dependence

97 residents were randomly assigned 2 units on paper and 2 on web.

Results

• No difference in knowledge scores– Pre to post test

• 68 to 75% for web• 66 to 73% for paper• (another study showed higher scores for multimedia

module on physical dx compared to paper)

• 78% preferred web format (95% CI, 67-86%)– 10 items (efficient, convenient, easier, effective)

• Less time spent on web, 47 min web, 59 min paper (learning efficiency)

• < 50% used links to additional learning

materials

Self-Study Web Vs PrintBell DS. Ann Int Med 2000

• In proctored sessions, residents assigned to review post acute MI care

• Both had MCQ’s and identical guideline text

• Web had tutorials, links to guideline passages and graphic evidence animations

Results

• Immediate post-test scores same• Less time spent w web (27 min vs 38 min)• So better learning efficiency w web

– Score gain 8.6/hr vs 6.7/hr• Greater satisfaction w learning w web

– 17 vs 15 on 20 pt scale (P< 0.001)• After 4-6 mo, knowledge decrease to same

extent (similar scores, P = 0.12)

Internet-based Learning in the Health Professions: A Meta-analysis (Cook 2008)

• Internet vs non-internet learning was superior for knowledge gains, but similar for satisfaction, skills and behaviors.

• Practice exercises, tutorials, on-line peer discussions and longer duration courses had positive effects.

JAMA 9/10/08; Vol 300 No. 10

Cook Meta-analysis

In this landmark article, Cook concluded:

Some methods of implementing CAI would likely be more effective than others and recommended that future research clarify which features of CAI are most effective.

Self-Assessment Questions: An RCT

• 149 residents randomized to web-based learning with or with out self-assessment questions imbedded in the module

• Topics = cervical cancer screening, dementia, osteoporosis, dyspepsia

Cook; Academic Medicine March 2006

Results

• Higher post-test scores when using question format: – 78.9% vs 76.2% (p=0.006)

• Residents preferring question format scored higher than those preferring standard format:– 79.7% vs 69.5% (p<0.001)

• 83% preferred question format and 71% said it was more efficient (took longer: 60 vs 44 min)

Adaptive Web-based Instruction

• An intervention to adapt web-based instruction to learners’ prior knowledge

• 122 IM residents• 4 ambulatory medicine modules• Learners who correctly answer case-based

questions skip corresponding content

Cook; JGIM 23(7):985-90

Methods

• 17-21 case-based questions per module• Didactic info appearing as feedback to each

question• Standard: 1-sentence explanation + didactics

(1-4 paragraphs of text, tables, pictures, links)• Adaptive: if correct response to MCQ, learner is

directed to go to next MCQ w option to view didactics. If incorrect, no option to skip didactics

ResultsAdaptive Standard P

Knowledge 76% 77% .34

Time Spent (min) 29.3 35.6 .0003

75% preferred adaptive format

Conclusion: adaptive format is associated with greater learning efficiency

Educating Residents Using Virtual Case-Based Simulation Improves Diabetes

Management: An RCT• N=341; 9 Primary care residencies• Intervention Gp: 18 VPs that respond to

provider actions• CG: Sham non-DM case with instructions• Assessment: 4 VPs, pre-post test, survey• Results: Achieve clinical goals on VP2

15-48% vs 1.8-18% (p = .02 - <.001)

Academic Medicine vol 89, No 12/Dec 2014

Findings in the Literature

• Learning efficiency improved• Knowledge about the same• Case-based MCQ’s improved knowledge• Links may not be used – limit them

Our Use of an On-line Teaching Case -DKA

• 51 Third year medical students– AIMC

• 84 pharmacy students– Clinical pharmacology course

• 11 Endocrine fellows

Pretest

VP (path score, time-on-task)

Posttest

Survey

Pre-Posttest Scores *P = .026

† P< .001 vs all posttest

†

Selected Posttest Questions

NS

*

†

* P = .01 † P<.001

Scores (%)

VP Time-On-Task

* P = .004 vs other groups

Survey Data

The ability to see and react to the consequences of my decisions in this module was more effective in teaching me clinical decision making than with other learning methods

All differences NS

The module was effective in helping me learn how to adjust therapy in patients with DKA who are notresponding appropriately

* P = .02 vs Med students

*

The module improved my confidence in managing DKA.

* P = .01 vs other groups

*

The module was of high educational value

Summary

• It is feasible to create a interactive branched-narrative VP to teach the management of a complex medical illness

• One way to fill an educational gap• Effectiveness was significant as evidenced by

pre-posttest score improvement• Students felt that the VP was effective in

teaching management of DKA and preferred the VP over other teaching methods

A look at some local uses of on-line learning

• A group of MS looking at which aspects of curriculum can be taught with on-line cases

• Individual MS and residents developing cases• Integrated Case Studies (MS 2)• General Medicine Modules• Johns Hopkins Modules• Yale Ambulatory Curriculum• Annals Virtual Patients

April Release Pulmonary Hypertension Avoiding Problems in Transitions of Care Hyperthyroidism A Woman with Dyspnea and Edema

(HFpEF) Parkinson Disease Migraine Fall ’15 Release Approach to the Patient with HIV Obstructive Sleep Apnea Care of the Adult Cancer Survivor Concussion Stable Ischemic Heart Disease Neurocognitive Decline Spring ’16 Release Insomnia Generalized Anxiety Disorder Back Pain

Michael Barnett Maggie BensonMamta Bhatnagar Rene ClaxtonJennifer Corbelli Dave DemoiseAnna Donovan Andrea ElliottKristian Feterik Michelle FreemanAlda Maria Gonzaga Rosanne GranieriBrian Heist Scott HerrleHarish Jasti Amar KohliMary Korytkowski Melissa McNeilAlexandra Mieczkowski Anuradha MunshiRuth Preisner John RagsdaleHarsha Rao Adam SawatskyGaetan Sgro Jamie SternJo-Anne Suffoletto Gary TabasSarah Tilstra Dianne Zalenski

Annals VP Authors

Virtual Patient Simulation Approach to Teaching Appropriate Use of Cardiac Stress Testing

Daniel Nguyen MD, Amar Kohli MD, Kathryn Berlacher MD