evaluating resident candidates: does closed file review yield better information? shahnaz chowdhry...
TRANSCRIPT
![Page 1: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/1.jpg)
Evaluating Resident Candidates: Does Closed File Review Yield Better Information?
Shahnaz Chowdhry MS MD1, Linnea S. Hauge PhD1, Steve Stroessner PhD2, and Norman L. Wool MD1
Department of General Surgery, Rush University Medical Center, Chicago, IL1
and Department of Psychology, Barnard College, New York, NY2
![Page 2: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/2.jpg)
Background
Bias created by open file interviewsRobin, Bombeck, Pollak, Nyhus (1991)Miles, Shaw, Risucci (2001)
Closed file interviews designed to reduce bias
![Page 3: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/3.jpg)
Purpose of the Study
To determine the efficacy of enhanced interview and rating process
![Page 4: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/4.jpg)
Research Questions
Is there a difference between faculty ratings of applicant characteristics in open and closed file conditions?
Do ratings of applicants, in these conditions, vary by gender and known status?
![Page 5: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/5.jpg)
Study Participants
Large, urban academic general surgery program
70 residency candidates (interviewees)44 Male26 Female
17 surgery faculty members (interviewers)12 Male5 Female
![Page 6: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/6.jpg)
Closed File Rating Process
Each applicant was interviewed by 2 faculty members (140 total interviews)
Closed file interviewNamePersonal statementUndergraduate institutionMedical school
Faculty completed evaluation instrument
![Page 7: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/7.jpg)
Open File Rating Process
Faculty reviewed applicant filesERAS applicationTranscriptUSMLE ScoresMSPELetters of Recommendation
Faculty completed the same evaluation instrument
![Page 8: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/8.jpg)
Evaluation Instrument Motivation for learning Motivation to do research Ability to work effectively with
others Commitment to general surgery Likelihood to succeed in surgery Overall fit in our program
Self-confidence Communication Intelligence Maturity Compassion Interpersonal
Skills Motivation for
Surgery
Miles WS, Shaw V, and Risucci D. The Role of blinded interview in the assessment of surgical residency candidates. The American Journal of Surgery. 182 (2001) 143-162.
![Page 9: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/9.jpg)
Data Analysis
Factor analysis to yield rating components MANOVA on:
Interview type (closed vs. open)Known status (known vs. unknown)Gender (male vs. female)Ratings
![Page 10: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/10.jpg)
Results
No significant differences betweenRatings between open vs. closed file
(F=.34, p=.56)
Open vs. closed file ratings by gender (F=.04, p=.84)
![Page 11: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/11.jpg)
Results: Interaction Effects
Significant interaction effects between:Open vs. closed file ratings by known status
(F=4.7, p=.04)Open vs. closed file ratings by trait (F=1.8,
p=.04)Open vs. closed file ratings by trait, gender
and known status (F=1.98, p=.02)Trait by gender (F=3.1, p=.001)
![Page 12: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/12.jpg)
Results: Closed vs. Open
By trait (F=1.8, p=.04)
Closed OpenIntelligence 4.19 4.33Maturity 4.16 4.29Commitment to general surgery 4.32 4.41Qualifications to succeed 4.20 4.32
![Page 13: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/13.jpg)
Results: Closed vs. Open
By known status (F=4.7, p=.04)
Known UnknownMaturity (open) 4.02 4.43Motivation for general surgery (closed) 4.26 4.59Working with others (closed) 4.91 4.36Qualifications to succeed (open) 4.04 4.41Likelihood to succeed (open) 4.00 4.45Fit to program (open) 4.00 4.33
![Page 14: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/14.jpg)
Results: Closed vs. Open Closed
Open
Known females Communication 4.4 4.0 Interpersonal skills 4.4 3.7 Motivation for research 3.6 3.1
Known males Qualifications to succeed 4.4 4.0
![Page 15: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/15.jpg)
Results: Closed vs. Open
Maturity Ratings
Closed Open
Known Male 3.8 4.0
Unknown Male 4.2 4.3
Known Female 4.6 4.4
Unknown Female 4.2 4.6
![Page 16: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/16.jpg)
Results: Trait by GenderSelf-confidence 4.3 4.4
Communication 4.3 4.3
Intelligence 4.4 4.3
Maturity 4.1 4.4
Compassion 4.3 4.4
Interpersonal skills 4.3 4.2
Motivation for surgery 4.5 4.6
Motivation for learning 4.6 4.6
Motivation for research 4.2 3.6
Ability to work effectively with others 4.5 4.4
Commitment to general surgery 4.4 4.6
Qualifications to succeed in surgery 4.2 4.3
Likelihood to succeed in surgery 4.2 4.3
Overall fit for our program 4.2 4.2
M F
![Page 17: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/17.jpg)
Factor Analysis Results
Social Traits Communication = 0.87 Maturity = 0.85 Self-confidence = 0.77 Interpersonal skills =0.76
General Surgery Commitment to general surgery = 0.871 Motivation to do general surgery = 0.85
Research Motivation to do research = 0.89
![Page 18: Evaluating Resident Candidates: Does Closed File Review Yield Better Information? Shahnaz Chowdhry MS MD 1, Linnea S. Hauge PhD 1, Steve Stroessner PhD](https://reader036.vdocument.in/reader036/viewer/2022080914/56649d0d5503460f949e2811/html5/thumbnails/18.jpg)
Conclusions
No significant change in closed and open file evaluations except:Differences in ratings of known vs. unknown
applicantsDifferences in ratings of some traits by the
gender of the applicant Constructs evaluated are social traits,
motivation for general surgery, and research