Seth D. Goldstein, MD1, Brenessa Lindeman, MD1, Jorie Colbert-Getz, PhD3, Trisha Arbella1, Robert Dudas, MD2, Anne Lidor, MD, MPH1,
Bethany Sacks, MD, MEd1
Johns Hopkins Hospital Departments of Surgery1 and Pediatrics2
Johns Hopkins School of Medicine Office of Medical Education Services3
Objective and Subjective Knowledge Correlation
Investigate convergent validity between subjective ratings of surgery student clinical knowledge and scores on the NBME subject examination Inter-rater reliability of faculty and resident
evaluations of global clinical knowledge
1 2 3 4 5
Unacceptable
Needs Improveme
nt
At Expected Level
Above Expectation
s
Outstanding
Unable to apply preclinical knowledge to understand basic medical problems.
Inconsistent understanding of patient problems. Limited differential diagnosis.
Knows basic differential diagnoses of major/active problems in patients. Understands team’s choice of therapy.
Knows expanded differential diagnoses, including recognition of emergencies. Can discuss therapeutic options.
Knows nuances of differential diagnosis, including disease prevalence and anticipated history & exam findings. Able to independently formulate a management plan. Able to assign prognoses.
Subjective Student Evaluation Scale
Retrospective, cross-sectional study from July 2009 – June 2011
Rating Attendings% (n)
Residents% (n)
1 N/A N/A
2 <1% (4) 1% (6)
3 20% (178) 28% (244)
4 57% (498) 47% (402)
5 22% (193) 24% (211)
0.76 0.81
ρ2:Attendings <0.01Residents 0.01
Medical Student Ratings by NBME Percentile Group
Quartile
Attending
n (SD)
Resident
n (SD)
Bottom 25th
3.96 (0.45)
3.81 (0.48)
Middle 50th
4.00 (0.35)
3.95 (0.43)
Top 25th
4.08 (0.40)
4.10 (0.49)
Bottom 25th
Middle 50th
Top 25th
1Pulito AR, Donnelly MB, Plymale M, et al. What do faculty observe of medical students' clinical performance? Teach Learn Med 2006;18:99-104.