Residency Director’s Forum: Applicant Pools - Resident Selection ProcessR. Tim Yoho, DPM, MS, FACFASProfessor and Dean, College of Podiatric Medicine & Surgery Des Moines University

The Education & Training for Health Care Transformation Conference

The Ten Pillars Shaping the Future of Health Science Education:

External Forces, Ongoing Evolution and Future Trends

Jeffrey P. Gold, MDChancellor & Executive VP Health Affairs

Executive Dean, College of Medicine & Life Sciences

The University of Toledo

The Future of Health Science Education

Curriculums will become more learner and learning focused.

Students will be assessed by individual competencies and the outcomes of their learning.

Inclusion of inter-professional components.

Integration of simulation and virtual immersive reality.

Academic health centers as learning and teaching hubs.

Standardization of knowledge, skill and professionalism competencies.

Heightened emphasis on team-based care assessments.

Increased accountability for healthcare outcomes, need for lifelong learning skills.

A more diverse and international healthcare workforce.

Focus on healthcare delivery to ensure the sustainability of the field..

Competency and outcome based assessment strategies will not link the educational programs to time of study, but will link them to their individual competencies and to the outcome of their learning.

The ability to perform in the domains of knowledge and skill, as well as professionalism and communication, will be increasingly measured on an individual basis and the chronological passage through the curriculum well be adjusted accordingly.

Just a individuals learn at different paces, the best way to swing a golf club or acquire a foreign language , others learn the health professions at different paces.

680 680 680 680 680 680 680 680 690

1.31.4

1.5 1.5 1.5

1.8

1.3 1.3

0

0.5

1

1.5

2

2.5

3

3.5

4

0

200

400

600

800

1000

1200

1400

2014 2015 2016 2017 2018 2019 2020 2021 2022

Total Apps Total Enr Seats Ratio A/S

Applicant/Enrollment Data for Podiatric Medical Schools

2.8

2.9

3.0

3.1

3.2

3.3

3.4

0.0

5.0

10.0

15.0

20.0

25.0

2014 2015 2016 2017 2018 2019 2020 2021

APP MCAT Mat MCAT App SGPA Mat SGPA

APMLE 1 First Time Pass Rate 2014-2019: 85%

Four-Year Graduation Rate 2014-2018: 80-85%

MCAT Score 2019-2021 Scaled to Old System

Applicant/Enrollment Academic Data for Podiatric Medical Schools

Residency Application Data 2014-2018

• Number of applicants per program• Mean: 79.5 Range: 72.5 - 85.4

• Number of programs per applicant• Mean: 28 Range: 27 – 30

• What does this tell us?

Importance Ratings by Program DirectorsHarfmann KL, Zirwas MJ. Can performance in medical school predict performance in residency? A compilation of correlative studies. J Am Acad Dermatol, 2011;65: 1010-22.

• Top Five• Interpersonal skills during the

interview• Interactions with faculty

during the interview• Professional attributes

exhibited during the interview• Perceived commitment to

specialty• Clerkship performance

• Bottom Five• Personal statement• Graduate of highly regarded

medical school• Honors membership• Demonstrated involvement in

research• Interest in academic career

Resident Selection CriteriaShofler D. Selecting residents in podiatric medicine and surgery. Jr Foot and Ankle Surgery, 2015;54:565-76.

• Top Five RD’s• Previous disciplinary action• Number of classes failed• Performance during clerkship

or visit• Opinions of other attending

physicians and residents• Number of Part I attempts

• Top 5 Students• Performance during clerkship

or visit• Opinions of other attending

physicians and residents• Performance during social

portion of the interview• Previous disciplinary actions• Performance during skills

portion of the interview

Resident Selection Criteria(Shofler)

• Bottom Five• Reputation of podiatric medical

school• Extracurricular non-academic

activities• Student undergraduate

experiences• Letters of recommendation• Communications with dean of

the podiatric medical school

• Bottom Five-Students• Student undergraduate

experience• Communication with the dean

of the podiatric medical school• Reputation of podiatric medical

school• Involvement in research• Extracurricular non-academic

activities

What Directors Value

• Social appropriateness/emotional maturity• Self and situational awareness

• Teachable• Motivated/Growth Mindset/Work Ethic• Team Player• Integrity• Humility• Likeability• Balance of Knowledge and Social Skills

Performance on USMLE Steps 1 and 2(Harfmann)

• High performance on examinations at one level likely predicts high performance on examinations at future levels.

• Correlation of USMLE to noncognitive performance measures are unclear and unpredictable.

Performance in Preclinical Coursework(Harfmann study)

• Lower performance on any academic marker was associated with an increased risk of poor knowledge and professionalism ratings.

• Correlations between resident performance and preclinical GPAs from different institutions are unlikely to be useful.

• Preclinical performance from a single institution is consistently measured and is more likely to be a valid tool for comparison of students from that institution.

Clinical Performance(Harfmann study)

• Clerkship specific performance failed to find a strong correlation between clinical performance and future noncognitive performance measures.

• Identified a positive correlation between overall medical student performance and future noncognitive performance.

• Can be used to predict future cognitive performance.

Clinical PerformanceYoho RM, Tallerico V, Vardaxis V. Relationship between student academic and clinical performance in podiatric medical education at Des Moines University. 2012; 102:4, 314-18

• Single institution study• N=202• Strong correlation between academic performance in the first-year to

the second-year• Third-year clinical training consistent for all students• Professionalism objective scored (noncognitive dominant)• A significant positive correlation between academic achievement and

clinical performance for pooled data.

Letters of Recommendation

• Can be used to predict future noncognitive performance if evaluation is standardized and objectified. Strong correlation. (Harfmann)

• The development of a standardized letter of recommendation in podiatric medicine and surgery might be appropriate to improve this aspect of the residency application. (Shofler)

Interview (Harfmann)

• Interviewers blinded to cognitive application data to more accurately assess noncognitive traits. Very strong.

• Can be used to predict future noncognitive performance. Very strong

• Orthopedic programs characterized the residency interview as “getting to know the applicant.”

• Appears to be a key factor in selecting the best fit. • Interview should be “primarily social.” (Shofler)

Performance Rating of Current Residents

• 8% different levels of dissatisfied.• 12% Somewhat satisfied• 33% Satisfied• 47% Very satisfied

• 52% identified a joint decision of the residency director, attending physicians and resident physicians as their selection process.

APMLE 1

Pass Fail

6Academic deficiencies

Academic deficiencies

No deficiencies

ProfessionalismAssessment 

3GPA/Rank

1GPA/Rank

Grades

4Lower quartile

5Lower quartile

Grades

No Deficiencies

2Lower Quartile

Residency selection funnel

Clinical Competence (Knowledge of Medicine)

Communication Skills

Ethical and Legal Understanding

Excellence

Humanism

Accountability

Arnold and Stern, 2006

Altruism 

Professionalism“Professionalism is demonstrated through a foundation of clinical competence, communication skills, and ethical and legal understanding upon which is built the aspiration to and wise application of the principles of: excellence, humanism, accountability, and altruism.”

Constructing a clerkship and Interview evaluation process/tool that’s assesses the Principles of Professionalism • Look at what others are doing for interviews:

• Interviewers should be blinded• Applicants should be aware of blinding and rationale• Interviewers should have specific questions addressed to each applicant intended to

investigate a specific quality or characteristic deemed critical by the program.• Applicants should be scored numerically on the basis of answer content and delivery

• Clerkship performance• Consider task-based interview techniques (standardized patient interviews, short

teaching presentations…) • Develop own evaluation tool based on what is important to the program.

Summary• You can compare academic performance of candidates from the same institution but unpredictable to

compare to other programs. • Lower performance on any academic marker (APMLE or course failure) is associated with an increased

risk of poor knowledge and possibly lower professionalism attributes.**• A common set of professionalism/noncognitive competencies incorporated into the fourth-year clerkship

evaluation for all colleges may level the playing field. • Clerkship performance matters but don’t immediately eliminate applicants that did not rotate at your

site. • Task based clerkships with grading rubrics.• Standardized letters of recommendation may have merit. • Value the interview process. Focus on the values your program in seeking a candidate. Multiple sets of

eyes. • REMEMBER: Students will be assessed by individual competencies and the outcomes of their learning.

Competency and outcome based assessment will not link the educational program to time of study, but will link them to their individual competencies and to the outcome of their learning.

Building an Applicant Pool

• The podiatric profession has to have the will to address this threat.• Pride in the profession.• Active outreach to high school and college students.• Mentoring/shadowing opportunities, including major scientific

events (http://www.dpmnetwork.org/membership/find-a-mentor) • Podiatric medical school based programs (http://www.aacpm.org) • Post-bac programs (https://apps.aamc.org/postbac/#/index) • MCAT/DAT/GRE cohorts• Tour for Diversity• Marketing a third type of MEDICAL SCHOOL

Acknowledgements:ACFASMori North AACPM Executive Director and StaffPodiatric Medical School Leadership

Thank You!