COMLEX-USA FOR RESIDENCY PROGRAM DIRECTORS

Evidence–based assessment designed specifically for osteopathic medical students and residents that measures competencies required for the provision of safe and effective osteopathic medical care to patients.

COMLEX-USA

PATHWAY TO LICENSURE COMLEX-USA, the Comprehensive Osteopathic Medical Licensing

Examination of the United States, is the exam series used by all

medical licensing authorities to make licensing decisions for

osteopathic physicians. COMLEX-USA is accepted in all 50 United

States and recognized by a number of international jurisdictions.2

COMLEX-USA is designed to assess osteopathic medical knowledge,

fundamental clinical skills, and other foundational competencies

considered essential for the practice of osteopathic medicine. The

primary and intended use of COMLEX-USA is for licensure.

RECOGNIZED EVIDENCE FOR VALIDITY A rigorous scientific process is used to produce exams of the highest

quality with evidence-based validity and reliability. Following a

comprehensive review of COMLEX-USA and USMLE, the FSMB

(Federation of State Medical Boards) endorsed both exams as valid

and reliable for their intended respective purposes, and concluded

that support for the validity of COMLEX-USA is exemplary. A recently

published collaborative study with the Federation of State Medical

Boards (Roberts et al) on the predictive validity of COMLEX-USA

demonstrates a strong correlation between successful completion of

COMLEX-USA and a lower likelihood of state licensing board

disciplinary action.

ELIGIBILITY AND ATTESTATION Eligibility criteria for COMLEX-USA exams require attestation by a

dean or residency program director to confirm that the candidate is in

good academic and professional standing and is approved to take the

exam. Good standing denotes that the osteopathic medical student or

resident meets the academic and professional requirements of the

college of osteopathic medicine or residency program and is eligible

to continue in the program.

It is recommended but not required that COMLEX-USA Level 3 be

taken after a minimum of six months in residency. The attestation

process for COMLEX-USA Level 3 helps to fulfill the NBOME mission to

protect the public, and adds value and entrustability to state licensing

boards and patients. Additionally, attestation provides COMLEX-USA

score reports to residency program directors and faculty.

COMPETENCY AND EVIDENCE-BASED DESIGN In 2019, COMLEX-USA completed a transition to a contemporary, two

decision-point, competency-based exam blueprint and evidence-

based design informed by extensive research on osteopathic

physician practice, expert consensus and stakeholder surveys.3 The

enhanced COMLEX-USA blueprint4 assesses measurable outcomes

of seven Fundamental Osteopathic Medical Competency Domains5

and focuses on high-frequency, high-impact health issues and clinical

presentations that affect patients.

DO candidates are not required to pass the United States Medical Licensing Examination (USMLE®) to be eligible to apply to ACGME-accredited residency programs. The ACGME does not specify which licensing board exam(s) (i.e., COMLEX-USA, USMLE) applicants must take to be eligible for appointment in ACGME-accredited residency programs.

Frequently Asked Questions: Single Accreditation System Accreditation Council for Graduate Medical Education, 20191

The COMLEX-USA Level 3 exam blueprint, designed for osteopathic physicians, provides a good example of how medical examiners can update their assessment systems to stay ahead of the curve of change… focuses on ‘knowledge, skills, experiences, attitudes, values, and/or behaviors that are observable and measurable and can be directly assessed in a reliable manner’— all factors that can help medical regulators make sound, balanced decisions as they continue their work in protecting the public and ensuring quality in medical practice, keeping our patients safe.”

—Heidi M. Koenig, MD Editor-In-Chief, Journal of Medical Regulation, October 2018

EQUALITY FOR COMLEX-USA AND USMLE IN ACGME PROGRAMS The American Medical Association6, the Accreditation Council for Graduate Medical Education, and the Federation of State Medical Boards recognize equivalent uses for COMLEX-USA and USMLE.

RESIDENT SELECTION AND SECONDARY USES COMLEX-USA and USMLE scores are used increasingly to screen

and evaluate DO residency applicants. In addition to use by residency

program directors, other secondary uses of COMLEX-USA scores

include student and resident promotion and assessment.

NBOME recommends residency and fellowship program directors

develop a strong understanding of what COMLEX-USA exams

measure, how standards are set, what the scores mean, and how

COMLEX-USA scores correlate to performance in residency.

EQUIVALENT USES OF LICENSURE EXAMS COMLEX-USA is recognized by the ACGME for acceptance into

graduate medical education (GME). The ACGME Single Accreditation

System for GME aligns accreditation standards for all residency

programs in the U.S., providing graduates of DO- and MD-granting

medical schools the ability to complete residency and fellowship

training in all programs. A single accreditation system for GME is

based on mutual recognition and understanding of the important and

unique qualifications and philosophies of both DO and MD graduates.

The ACGME publically communicates its policy that COMLEX-USA and

USMLE are both acceptable for ACGME residency programs and has

clarified that DO candidates are not required to pass USMLE exams in

order to be eligible to apply to ACGME-accredited residency programs.

An individual who has completed an ACGME-accredited residency

program or an AOA-approved residency program in any given specialty

is eligible for appointment to any ACGME-accredited fellowship

program that allows prerequisite education and training in that specialty,

regardless of which licensing exam(s) the individual has pursued.1

Acceptance of COMLEX-USA by all ACGME programs may reduce

stress and support wellness in residency applicants, and diminish

barriers for DO students applying to programs and improve holistic

resident screening processes.

INTERPRETING COMLEX-USA SCORES COMLEX-USA exams use a criterion-referenced standard setting

methodology. The minimum passing score for COMLEX-USA Level 1 or

Level 2-CE is 400, and for Level 3 a score of 350 is passing.

Passing Levels 1 and 2 means a candidate has demonstrated

competence to enter into supervised clinical practice settings, enter

graduate medical education, and prepare for lifelong learning. Passing

Level 3 means the candidate has demonstrated competence in

foundational competency domains required for generalist physicians to

deliver safe and effective osteopathic medical care of patients as required

for entry into the unsupervised practice of osteopathic medicine and to

continue lifelong learning and practice-based learning and improvement.

PERCENTILE RANK SCORE CONVERSION A percentile score conversion tool is accessible via the NBOME

website as well as the ERAS® Program Director Workstation. This tool

converts an applicant’s 3-digit COMLEX-USA score to a percentile rank

to help programs compare an applicant’s relative performance to other

applicants within a testing cycle, and correctly interpret assessment

scores when filtering applications. OVER 82% OF ACGME- ACCREDITED RESIDENCY PROGRAM DIRECTORS USE COMLEX-USA AS PART OF THE APPLICATION PROCESS FOR DO APPLICANTS

All osteopathic medical students must pass COMLEX-USA Level 1, Level 2-CE and Level 2-PE to graduate from a college of osteopathic medicine with the DO degree.7

CONVERSION VARIES BY TESTING CYCLE

COMLEX-USA Level 1, May 2018-April 2019 Cohort 3-DIGIT SCORE/PERCENTILE RANK

99

750

97

700

91

650

80

600

62

550

39

500

20

450

7

400

Students and their abilities as a physician are not defined solely by a number. Program directors need to get comfortable with DO applicants and using COMLEX-USA as just one tool to review which applicants have the skills necessary to thrive in a GME program — just like they learn about other things in their program, they need to learn to adapt to this change.”

—Kenneth B. Simons, MD, Senior Associate Dean for GME and Accreditation, Medical College of Wisconsin

NBOME cautions residency program directors to avoid the sole use of any examination score, or the overuse or sole use of any examination program, in screening or hiring residents.

RESEARCH AND HOLISTIC ADMISSION REVIEW Holistic admission review processes in medical education consider the

“whole” applicant to avoid disproportionate focus on a single factor in

order to achieve diversity in a physician population that is prepared to

address varied needs across health care populations. As part of its

research program, NBOME conducts studies designed to help

residency program directors understand COMLEX-USA exams and the

validity support for using these scores in holistic residency application

decisions. A recent study demonstrates a strong association between

COMLEX-USA Level 1 and USMLE Step 1 performance of osteopathic

medical students who took both exams.8 With growing support for

equivalent uses of COMLEX-USA and USMLE for residency

applications, NBOME provides program directors with insight into the

uses of COMLEX-USA and how COMLEX-USA scores can be an asset

in holistic resident screening and selection processes. SCORE CONCORDANCE STUDIES • Sandella JM, Gimpel JR, Smith LL, Boulet JR. The use of COMLEX-

USA and USMLE for residency applicant selection. Journal of

Graduate Medical Education. 2016; 8(3):358-363.

• Lee AS, Chang L, Feng E, Helf S. Reliability and validity of

conversion formulas between Comprehensive Osteopathic Medical

Licensing Examination of the United States Level 1 and United

States Medical Licensing Examination Step 1. Journal of Graduate

Medical Education. 2014;6(2):280-283.

• Schenarts PJ, Termuhlen PM, Pasley J, Rose JS, Friedell ML. A

primer on how to select osteopathic applicants to an allopathic

general surgery residency. Journal of Surgical Education.

2011;68(3):239-245.

• Chick DA, Friedman HP, Young VB, Solomon D. Relationship

between COMLEX and USMLE scores among osteopathic medical

students who take both examinations. Teach Learn Medicine.

2010;22(1):3-7.

PREDICTIVE VALIDITY PERFORMANCE IN IN-TRAINING AND CERTIFICATION EXAMS • Roberts WL, Gross GA, Gimpel JR, Smith LL, Arnhart K, Pei X,

Young A. An investigation of the relationship between COMLEX-USA

licensure examination performance and state licensing board

disciplinary actions. Academic Medicine. 2019; Volume Publish

Ahead of Print.

• Hudson K, Tsai T-H, Finch C, Dickerman J, Liu S, and Shen L. A

validity study of COMLEX-USA Level 2-CE and COMAT clinical

subjects: Concurrent and predictive evidence. Journal of Graduate

Medical Education: 2019;11(5):521-526.

• Hudson KM, Feinberg G, Hempstead L, Zipp C, Gimpel JR, Wang Y.

Association between performance on COMLEX-USA and the

American College of Osteopathic Family Physicians in-service

examination. Journal of Graduate Medical Education. 2018;10(5):

543-547.

• O’Neill TR, Peabody MR, Song H. The predictive validity of

NBOME’s COMLEX-USA with regard to outcomes on ABFM

examinations. Academic Medicine. 2016;91(11):1568-1575.

• Li F, Gimpel JR, Arenson E, Song H, Bates BP, Ludwin F. The

relationship between COMLEX-USA scores and performance on the

American Osteopathic Board of Emergency Medicine Part I

Certifying Examination. Journal of the American Osteopathic

Association. 2014;114(4):260-266.

• Pierce DL, Mirre-Gonzalez MA, Carter MA, Nug D, Salamanca Y.

Performance on COMLEX-USA exams predicts performance on EM

residency in-training exams. Academic Emergency Medicine.

2013;20(5) Suppl. 1.

• Langenau EE, Pugliano G, Roberts W. Relationship between high-

stakes clinical skills exam scores and program director global

competency ratings of first-year pediatric residents. Medical

Education. 2011;16:7362.

• Langenau EE, Pugliano G, Roberts W, Hostoffer R. Summary of

ACOP program directors’ annual reports for first-year residents and

relationships between resident competency performance ratings and

COMLEX-USA test scores. Electronic Journal of the American

College of Osteopathic Pediatricians. 2010;2(7).

REFERENCES

1. Frequently Asked Questions: Single Accreditation System. Accreditation Council for Graduate Medical Education (ACGME).

2. Federation of State Medical Boards of the United States. U.S. Medical Regulatory Trends and Actions.

3. Horber D, Gimpel JR. Enhancing COMLEX-USA: Evidence-based redesign of the osteopathic medical licensure examination program. Journal of Medical Regulation. October 2018, 104(3):11-18.

4. National Board of Osteopathic Medical Examiners. COMLEX-USA Master Blueprint effective beginning September 2018.

5. National Board of Osteopathic Medical Examiners. Fundamental Osteopathic Medical Competency Domains 2016: Guidelines for Assessment for Osteopathic Medical Licensure and the Practice of Osteopathic Medicine, Chicago, IL, 2016.

6. American Medical Association press release, COMLEX, DO matching and GME accreditation: What you need to know. January 23, 2019.

7. American Osteopathic Association. Commission on Osteopathic College Accreditation, Accreditation of Colleges of Osteopathic Medicine: COM Continuing Accreditation Standards.

8. Sandella JM, Gimpel JR, Smith LL, Boulet JR. The use of COMLEX-USA and USMLE for residency applicant selection. Journal of Graduate Medical Education. 2016;8(3):358-363.

March 2020

FOR MORE INFORMATION ABOUT IMPORTANT RESEARCH ON COMLEX-USA VALIDITY AND

RELIABILITY, VISIT THE NBOME WEBSITE.