national board of medical examiners® (nbme®)

NATIONAL BOARD OF MEDICAL EXAMINERS® (NBME®)

Serving the Public Through Evaluation of Healthcare Professionals Worldwide

OVERVIEW OF NBME SERVICES AND ASSESSMENTS FOR MEDICAL SCHOOLS AND

STUDENTS

Agata P. Butler, PhD Vice President, NBME

Serve the assessment needs of medical schools by providing high-quality assessment tools as well as information and assistance to medical educators

NBME Subject Exam Program

NBME Subject Tests: Basic Science Exams

Gross Anatomy (+/-Embryology) Histology & Cell Biology Biochemistry Physiology (+/- Neurophysiology) Neuroscience (+/- Neuropath, Neuropharm) Behavioral Sciences Microbiology (+/- Immunology) Pharmacology Pathology Introduction to Clinical Diagnosis

NBME Subject Tests: Clinical Science Exams

Ambulatory Medicine Family Medicine (modular) Internal Medicine Clinical Neurology Obstetrics & Gynecology Pediatrics Psychiatry Surgery

Core Clerkship Exams

NBME Subject Tests: Clinical Science Exams

Internal Medicine (2009) Surgery (2010) Pediatrics (2011) Emergency Medicine (2013)

Advanced Clinical Exams

NBME Subject Tests: Comprehensive Exams

Comprehensive Basic Science Examination

Comprehensive Clinical Science Examination

Used by many medical schools as end-of-course and end-of-clerkship exams National norms and descriptive information

of students at the same stage of training

Multiple parallel forms of exam available Test scores are equated and can be directly

compared over time

NBME Subject Exam Program

High-quality, USMLE-like items that have been

carefully reviewed and pretested Items written by a "national faculty" (USMLE Test Committees) Gives students practice for USMLE

NBME Subject Exam Program

Report performance profiles to schools for individual examinees taking the comprehensive subject exams, clinical science exams, and web-based basic science exams.

Performance Profiles

Student Performance Profile

Provide a look-up table for schools and students to use as a benchmark on passing USMLE (comparable score)

Look-up Table

A table provides approximate performance equivalents for USMLE Step 1, making it possible to translate the CBSE score to the scale used for USMLE Step 1.

Web-Based Subject Exams

Transitioned subject tests from a paper format to web-based platform Enhance security Streamline score reporting Provide new opportunities for growth

NBME provides test administration software that

provides a mechanism for the school to use laptop and/or desktop computers

“locks” the browser, preventing examinees from copying screens or communicating exam content

enables the Proctor to open exam access, monitor examinee progress and control other aspects of the test administration

New Directions in 2012

NBME subject examinations are now available at Prometric Test Centers Initially limited to the comprehensive and

clinical science subject tests

Future Directions

Include pretest items in subject tests Include multimedia in items Pilot modular tests

Collaborate with schools and faculty in developing new and innovative item types

Faculty members can access the NBME item bank to design an assessment to fit their needs

Customized Assessment Services

Customized Assessment Services: Uses

In-course tests Mid-term exams

End-of-course tests Individual discipline-based courses Individual organ-system blocks

Integrated, tests End-of-semester/block exams

Basic science progress tests

Self-assessment services provided directly to students Comprehensive Basic Science (Step 1) Comprehensive Clinical Science (Step 2 CK) Comprehensive Clinical Medicine (Step 3 MCQ)

NBME Self-Assessment Services

NBME Self-Assessment Services

Designed to reflect the format and content coverage of USMLE

Provided as a service to students and can be taken via the web at any time and from any location that the examinee chooses

CBSSA: Sample Performance Profile

A table provides approximate performance equivalents for USMLE Step 1, making it possible to translate the CBSSA score to the scale used for USMLE Step 1.

Recent Research Publication on NSAS

12,224 U.S. and Canadian medical students Completed an English version of the CBSSA exam between

12/06 and 12/08 Took the exam under standard-paced conditions First Step 1 attempt was on or before 11/17/09.

The most recent CBSSA prior to their first Step 1 attempt and first Step 1 score was used

CBSSA performance is strongly related to Step 1 performance (67 – 69% of variance in Step 1 scores explained by CBSSA)

Research Publications on NSAS

Using the NBME Self-Assessments to Predict Performance on USMLE Step 1 and Step 2: Impact of Test Administration Conditions. Academic Medicine. Vol. 70. No. 10. October/Supplement 2004.

Relationship Between Performance on the NBME Comprehensive Basic Sciences Self-Assessment and USMLE Step 1 for US and Canadian Medical School Students. Academic Medicine. 85(10):S98-S101, October 2010.

Content Feedback

Longitudinal Performance Profile Performance profile graphically displays

the relative strengths and weaknesses in general content areas as horizontal bars

Enhance feedback to display the performance bar for each content area vertically with each column (bar) of the report identified to show progress over multiple testing sessions

Longitudinal Performance Profile

New: Clinical Science Mastery Series

New self-assessments for the clinical science subject examinations

Two, 50-item forms for each of the following:

Clinical Neurology Medicine Obstetrics/Gynecology Pediatrics Psychiatry Surgery

Content Feedback for Psychiatry Self-Assessment

A table provides approximate performance equivalents for the Medicine Subject Test, making it possible to translate the self-assessment score to the scale used for the Medicine Subject Test.

Future Directions: Enhance Feedback for Students

Enhance feedback regarding timing and pacing for the self-assessments

Enhance score reports with more specific feedback about the items

Provide opportunity for students to review the questions and answers after the test for customized assessments

To learn more about the

National Board of Medical Examiners, visit us at

www.nbme.org

Performance on Most Recent CBSSA, First Step 1, and Pass/Fail Outcomes

Group N

CBSSA Step 1

Mean

SD

Mean

SD

% Passing

Study Sample 12,224 456 109 225 21 96.5%

All US/Canadian Examinees 57,329 221 23 93.1%

CBSSA Attempts N

Percent of Study

Sample

CBSSA Score Step 1 Score

Mean SD Mean SD

1 6489 53% 438 110 224 21

2 3546 29% 466 103 225 21

3 1402 12% 484 105 227 21

4 602 5% 503 108 230 21

5 113 1% 525 115 232 23

6 65 <1% 550 94 236 19

CBSSA and Step 1 Performance by Number of CBSSA Takes

Model Summary

Model R R Square

Adjusted R Square

Std. Error of the Estimate

1 .819(a) .671 .671 12.140

2 .828(b) .685 .685 11.875

3 .831 (c) .691 .691 11.762

a. Predictors: (Constant), LAST_CBSSA_BF_STEP1

b. Predictors: (Constant), LAST_CBSSA_BF_STEP1, NUM_DAYS _BET_STEP1

c. Predictors: (Constant), LAST_CBSSA_BF_STEP1, NUM_DAYS__STEP1, NUM_CBSSA_TAKES_BF_STEP1

Stepwise Multiple Regression

Dependent variable: First Step 1 score Independent variables:

Most recent CBSSA before first Step 1 attempt Number of days between most recent CBSSA and first

Step 1 attempt Mean = 14 days Median = 8 days Mode = 7 days

Number of CBSSA takes before first Step 1