Preparing for Licensure Examinations

Lisa Travis, MS, EdS

Medical Librarian, LMU-DCOM

Chelsea A. Nickolson, OMSIII COSGP Medical Education Representative, LMU-DCOM

Donald J. Sefcik, DO, MBA Senior Associate Dean, MSUCOM

AA

CO

M A

nnua

l Mee

ting

201

2

Co-sponsored by Council of Osteopathic Student Government Presidents

Council of Osteopathic Librarians Society of Osteopathic Medical Educators

March 30, 2012

Goal of this session

From librarian, student, and faculty perspectives, this session will describe the current practices and best practices in preparing for the National Licensure Exams.

AA

CO

M A

nnua

l Mee

ting

201

2

Part 1 – Librarian Perspective Survey about COMLEX-USA Level 1 Resources

in 2010 and 2011 Survey about COMLEX-USA Level 2 Resources

in 2012 So many review books and databases Few available electronically

AA

CO

M A

nnua

l Mee

ting

201

2

COMLEX-USA Level 1 Survey

2010 – 205 students 2011 – 395 students

7 osteopathic medical schools

Q1: List the three books that you consider most helpful for reviewing for the COMLEX Level 1 exam.

2010 2011

First Aid for USMLE 47 95

OMT Review (Savarese) 18 61

Rapid Review Pathology (Goljan) 34 56

First Aid 24 53

Kaplan Review Books 26 30

Microbiology Made Ridiculously Simple 8 22

BRS Physiology 5 12

First Aid for COMLEX 4 7

USMLE Step 1 Secrets 1 6

BRS Pathology 1 4

First Aid Cases 1 4

Kaplan for COMLEX 11 4

Step-Up to USMLE Step 1 (Mehta) 3 4

Boards Boot Camp Materials 3

BRS 1 3

Kaplan COMLEX Level 2 1 3

BRS Biochemistry 2

Crush 2

Flashcards (generic) 2

Q2: How helpful do you perceive the following non-book resources to be in preparing for the COMLEX Level 1 exam?

Very Helpful Helpful Somewhat

Helpful Not Very Helpful

Weighted Score

USMLEWorld 75 11 6 0 94

COMBANK 30 12 11 0 84

COMSAE 23 17 14 1 78

Kaplan Qbank for USMLE 32 21 18 3 78

USMLE Rx 10 6 5 1 78

Kaplan Qbank for COMLEX 28 26 38 4 70

USMLE Consult 1 0 4 0 60

ExamMaster 2 2 19 0 57

USMLEasy 0 1 5 1 50

USMLE QuickPrep 0 0 4 0 50

Q3: Please identify the board review resource (book or non-book) that you consider to be the most helpful resource. 2010 2011

COMBANK 3 24

First Aid for USMLE 17 19

First Aid 16 18

Doctors in Training 2 12

Kaplan Course 7 11

USMLEWorld 14 11

Rapid Review Pathology (Goljan) 3 7

Boards Boot Camp 4

Question banks (generic) 1 4

OMT Review (Savarese) 2

PASS Program 1 2

BRS Pharmacology Flashcards (Kim) 1

BRS Physiology 1

ExamMaster 1

USMLE Step 1 Secrets 1

Wikipedia 1

Microcards 1

Unpublished notes 1

USMLE Consult 1

COMLEX-USA Level 2 Survey Class of 2011 – 13/212 students

(6.13%) Class of 2012 – 143/212 students

(67.45%) Class of 2013 – 56/212 students

(26.42%) 7 osteopathic medical schools

COMLEX-USA Level 2 Survey

212 responses – 1 incomplete 7 GA-PCOM 34 LMU-DCOM 46 NYCOM 18 OU-HCOM 43 PCOM 28 UNECOM 33 VCOM-VC

AA

CO

M A

nnua

l Mee

ting

201

2

COMLEX-USA Level 2 Survey Three most helpful books – 15 responses 1st book (most helpful book)

3 – First Aid 3 – Step Up To Step 2 1 – Step Up To Medicine 1 - Boards and Wards by Carlos Ayala and Brad Spellberg 1 – Case Files 1 – Master the Boards 1 – Crush Step 2 1 – OMT review by Savarese 1 – NMS Series 1 – USMLE Step 2 (not a book) 1 – USMLEWorld (not a book)

AA

CO

M A

nnua

l Mee

ting

201

2

COMLEX-USA Level 2 Survey Three most helpful books – 13 responses 2nd most helpful book

4 – First Aid 3 – OMT review by Savarese 2 – Step Up To Step 2 1 – Step Up To Medicine 1 – Crush Step 2 1 – Doctors in Training (not a book) 1 – Combank (not a book)

AA

CO

M A

nnua

l Mee

ting

201

2

COMLEX-USA Level 2 Survey

Three most helpful books – 8 responses

3nd most helpful book 5 – OMT review by Savarese 3 – First Aid

AA

CO

M A

nnua

l Mee

ting

201

2

COMLEX-USA Level 2 Survey – How helpful for CE exam (19 options)? Very helpful – top five 42.93% (82/191) – COMBANK for COMLEX-USA

Level 2 CE 40.64% (76/187) - USMLEWorld for USMLE Step 2

CK 34.20% (66/193) - COMQUEST for COMLEX-USA

Level 2 CE 26.70% (51/191) – Doctors in Training for

COMLEX-USA Level 2 CE 23.91% (44/184) – COMSAE Practice Questions

for COMLEX-USA Level 2 CE

AA

CO

M A

nnua

l Mee

ting

201

2

Not very helpful – top five 10.22% (19/181) – Exam Master for USMLE Step 2 CK 8.15% (15/184) - COMSAE Practice Questions for

COMLEX-USA Level 2 CE 8.06% (15/186) – Kaplan Qbank for USMLE Step 2 CK 6.56% (12/183) – Northwestern Medical Review for

COMLEX-USA Level CE 5.49% (9/164) – Falcon Review materials for COMLEX

Level 2 5.39% (9/167) – Falcon Review materials for USMLE

Step 2 CK

AA

CO

M A

nnua

l Mee

ting

201

2

COMLEX-USA Level 2 Survey – How helpful for CE exam (19 options)?

COMLEX-USA Level 2 Survey – How helpful for CE exam (19 options)? Not familiar with the resource – top five 95.81% (160/167) - USMLEWeapon 95.18% (158/166) – Score95 93.33% (167/180) – MasteringBoards.com for COMLEX-

USA Level 2 CE 92.86% (169/182) – Premier Review for COMLEX-USA

Level 2 CE 91.26% (167/183) – PASS Program for USMLE Step 2

CK; will design a program to meet a student’s needs as well

AA

CO

M A

nnua

l Mee

ting

201

2

COMSAE Practice Questions for COMLEX-USA Level 2 CE (19 options) 23.91% (44/184) - Very helpful 20.11% (37/184) - Helpful 21.20% (39/184) - Somewhat helpful 8.15% (15/184) - Not very helpful 26.63% (49/184) - Not familiar with resource

AA

CO

M A

nnua

l Mee

ting

201

2

COMLEX-USA Level 2 Survey – How helpful for PE exam (9 options)? Very helpful – top three 14.59% (27/185) – COMSAE information for

COMLEX-USA Level 2 PE 13.71% (24/175) - USMLEWorld for USMLE Step 2

CS 5.11% (9/176) – Kaplan materials for USMLE Step

2 CS 2% or less - All other resources

AA

CO

M A

nnua

l Mee

ting

201

2

COMLEX-USA Level 2 Survey – How helpful for CE exam (9 options)? Not very helpful – top five 10.81% (20/185) – COMSAE information for COMLEX-

USA Level 2 PE 6.36% (11/173) – Falcon Review materials for USMLE

Step 2 CS 6.36% (11/173) – PASS Program for USMLE Step 2 CS;

will design a program to meet student’s needs as well

6.25% (11/176) – Northwestern Medical Review for COMLEX-USA Level PE

5.11% (9/174) – Premier Review materials for USMLE Step 2 CS

AA

CO

M A

nnua

l Mee

ting

201

2

COMLEX-USA Level 2 Survey – How helpful for CE exam? Not familiar with the resource – top five 93.71% (164/175) – Ximedus for USMLE Step 2 CS 91.48% (161/176) – MasteringBoards.com for

COMLEX-USA Level 2 PE 91.38% (159/174) – Premier Review materials for

USMLE Step 2 CS 91.33% (158/173) – Falcon Review materials for USMLE

Step 2 CS 90.91% (160/176) – Northwestern Medical Review for

COMLEX-USA Level 2 PE

AA

CO

M A

nnua

l Mee

ting

201

2

COMSAE Informationfor COMLEX-USA Level 2 PE (9 options) 14.59% (27/185) - Very helpful 20.00% (37/185) - Helpful 12.43% (23/185) - Somewhat helpful 10.81% (20/185) - Not very helpful 42.16% (78/185) - Not familiar with resource

AA

CO

M A

nnua

l Mee

ting

201

2

Best Review Resource – Book or Non-Book (15 responses) 4 – Doctors in Training 3 – First Aid 3 - COMBANK 2 – Kaplan Qbank 1 – COMQUEST 1 – Boards Boot Camp 1 – USMLEWorld A

AC

OM

Ann

ual M

eetin

g 2

012

Survey Results

Lisa.travis@lmunet.edu

Part 2- Student Perspective •  COMLEX-USA Level 1 is the unknown

–  How to study? –  What to study?

•  Time commitment –  Keeping up with coursework –  Maintaining balance

•  Stress increase –  Consequences of not passing –  Length of exam –  Attaining competitive score –  Cost of exam and prep materials

AA

CO

M A

nnua

l Mee

ting

201

2

Stress Effects on Learning and Memory

•  Varies with type of learning –  ↑Simple Recall –  ↓Problem Solving

•  Other Effects –  Health and fitness –  Relationships –  Psych and mood –  BALANCE!

AA

CO

M A

nnua

l Mee

ting

201

2

Mentorship •  Students

–  OMS-III/IVs –  National Organizations (COSGP!)

•  Faculty •  Physicians •  Administration

–  Correlations •  Mentorship ≠ Sharing items from past/

future exams •  Stay Positive!

AA

CO

M A

nnua

l Mee

ting

201

2

COMLEX-USA Level 1 Survey 2012

•  Designed by COSGP Research Representative, Kevin Ku, OMS-IV

•  458 Student Participants, OMS-IIIs only •  14 Schools Represented •  Participation Window: 2/27/12- 3/27/12 •  Data from April 2011 available for COMLEX-

USA Level 2 A

AC

OM

Ann

ual M

eetin

g 2

012

Time To Study A

AC

OM

Ann

ual M

eetin

g 2

012

How much time off from classes did your school provide between second and third year for board studying?

What were you doing academically at the time you took COMLEX-USA Level 1?

69%

18%

7%

5%

Methods Used A

AC

OM

Ann

ual M

eetin

g 2

012

None 3 1% Independent Study 401 90%

Informal Group Study 141 31%

School-sponsored, non-commercial review course

73 16%

Commercial review course/program

243 54%

Commercial question banks

377 84%

Other 16 4%

Most Useful Source A

AC

OM

Ann

ual M

eetin

g 2

012

Of the following, please indicate the #1 resource that you found the MOST helpful in your preparation

2nd Most Useful Source A

AC

OM

Ann

ual M

eetin

g 2

012

Least Helpful Source

Which resource did you find the LEAST helpful?

AA

CO

M A

nnua

l Mee

ting

201

2

In Retrospect…PLANNING

“More questions and setting a higher personal goal. I met mine, but then afterwards wondered what would have happened if I aimed for a 700 or 750 instead.” –Score of 602 “Study over a longer period of time, not try to

memorize it all too late. Do more practice questions. Needed more time to study.” – Score of 380 “Started using my board review book during

systems courses and more questions.” –Score of 500

AA

CO

M A

nnua

l Mee

ting

201

2

In Retrospect…ANXIETY

“I honestly believe I failed the first time not because I was under prepared but because I had too much anxiety and was way too 'in my head'. Perhaps if there was a course on handling stress during these exams or even just an assembly of solid facts on how much our board scores affect our residency applications.” –Score of 390

AA

CO

M A

nnua

l Mee

ting

201

2

In Retrospect…TIMING

“I started USMLE World too early, I think it would have been more beneficial to work through it closer to my exam date” –Score of 674 “Focus more last week ... Was burned out when

I took my test and couldn't focus” –Score of 570

"Taken the test later so I had more time to study after classes were done.” –Score of 487 A

AC

OM

Ann

ual M

eetin

g 2

012

What Else?

COMLEX-USA Level 1 Student Survey also included:

•  Percent of lectures attended •  Percent working (hours included in inc. of 10) •  Specialty Choice •  USMLE scheduling vs. COMLEX •  USMLE scores •  Requirements of COM/SOMs to take

COMLEX-USA Level 1

AA

CO

M A

nnua

l Mee

ting

201

2

Interested?

Contact COSGP for more survey data. •  Chelsea Nickolson, OMS-III

Medical Education Representative COSGPMedEdRep@aacom.org -or-

•  Kevin Ku, OMS-IV Research Representative COSGPResRep@aacom.org

COSGPResRep@aacom.org

AA

CO

M A

nnua

l Mee

ting

201

2

Part 3 – Faculty Perspective

Standardized Tests

AA

CO

M A

nnua

l Mee

ting

201

2

Part 3 – Faculty Perspective Standardized Tests

Roadmap - Where can you find it? Test Blueprint Table of Specifications Bulletin of Information www.nbome.org

AA

CO

M A

nnua

l Mee

ting

201

2

Part 3 – Faculty Perspective Standardized Tests

Roadmap – How does it help? What you will see When you need to be there How you should get there Why it’s important

AA

CO

M A

nnua

l Mee

ting

201

2

Part 3 – Faculty Perspective Standardized Tests

Roadmap – Greatest Utility? What you will see When you need to be there How you should get there Why it’s important

AA

CO

M A

nnua

l Mee

ting

201

2

Content

Context

Part 3 – Faculty Perspective Standardized Tests

Content and Context What you study How you study

Determines your score!!

AA

CO

M A

nnua

l Mee

ting

201

2

MBTI and COMLEX Scores

Sefcik, Prerost & Arbet, JAOA 2009

AA

CO

M A

nnua

l Mee

ting

201

2

Importance of Vignette

Examinees with Higher Scores

Examinees with Lower Scores

No Vignette 99%

Short Vignette 98%

Long Vignette 98%

http://www.nbme.org/pdf/itemwriting_2003/2003iwgwhole.pdf

AA

CO

M A

nnua

l Mee

ting

201

2

Importance of Vignette

Examinees with Higher Scores

Examinees with Lower Scores

No Vignette 99% 90%

Short Vignette 98% 82%

Long Vignette 98% 66%

http://www.nbme.org/pdf/itemwriting_2003/2003iwgwhole.pdf

AA

CO

M A

nnua

l Mee

ting

201

2

Standardized Tests

Content = What Rehearsal Techniques

AA

CO

M A

nnua

l Mee

ting

201

2

Stimulus

Facilitation

Outcome

Inadequate because SAME Context ~ 30%

Standardized Tests

New Context ~ 70% Metacognition

AA

CO

M A

nnua

l Mee

ting

201

2

Stimulus

Outcome AB

C

Stimulus* Stimulus ̂

Standardized Tests Metacognition = Thinking about Thinking

How to study? Associations

Retrieval

AA

CO

M A

nnua

l Mee

ting

201

2

Standardized Tests More than content!

Study with context in mind!!

AA

CO

M A

nnua

l Mee

ting

201

2

Don’t skim / Strive to Clarify Avoid - Illusion of Competence

Annotate / Summarize Create your own questions Practice Recall Study with Peers (with a PAL) Be aware of your MBTI type / impact

ST

Scoring

Content

Skills

Timed

Guessing

Items

Take-Home: Standardized Tests

Test-Wise Test-Takers: First Step in Preparing

Blueprint

AA

CO

M A

nnua

l Mee

ting

201

2

R E S O U R C E S

Selection Attention

Storage Organize

EncodingAssociate

Jairam & Kiewra (2009)

SOAR

Thank you!

Are there some questions?

AA

CO

M A

nnua

l Mee

ting

201

2