MCAT Update: What You Need to Know

2018 NAAHP National Meeting, July 1, 2018

Marc Kroopnick, Ph.D., M.Eng., Director MCAT Development and PsychometricsJudy Byrne, M.A., Sr. Director, MCAT Business OperationsAAMC

Robert Witzburg, M.D.Dean of AdmissionsBoston University School of Medicine

Julie Chanatry, Ph.D. (moderator)Chairperson, Health Sciences Advisory CommitteeColgate University

Agenda

▪ What Do We Know About Students Who Have Taken the Current Version of the MCAT Exam?

▪ How Did Admissions Officers Use MCAT Scores in 2017 Student Selection?

▪ What Are We Learning about the Impact, Use, and Predictive Validity of the Exam?

▪ MCAT Testing Program Updates for 2018

▪ Resources for Advisors and Students

▪ Questions

2

What Do We Know About Students Who Have Taken the Current Version of the MCAT Exam?

3

2018 is the fourth year of the MCAT exam

4

ExamineesApplicants/

Matriculants

Year 1

Coursework

Year 2

Coursework

USMLE

Step 1

Clerkships

Graduate in

3-4 Years

Graduate in 5

Years

USMLE Step 2

Exams

2015

2016

2018

2017

20172016 2018 2019 2020 2021

20182017 2019 2020 2021 2022

20192018 2020 2021 2022 2023

20202019 2021 2022 2023 2024

To many this exam is no longer “new”

▪ Most applicants do not know the old test

▪ Pre-health advisors are more familiar with the concepts and skills tested on the exam

▪ Some admissions committee members may only have used scores from THIS exam to select students

▪ Fewer applicants are applying with scores from the previous exam (1% submitted only old scores in 2018)

▪ Aspirants and pre-health advisors are beginning to learn more about how these MCAT scores are used in admissions

5

3% more examinees tested in 2017 than in 2016

78,410 80,997

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

90,000

100,000

2016 2017

Unique Examinees with Scores

6

Examinees tested in the same proportions as in the past

Percentage of Examinees Taking This Version of the MCAT Exam from 2015 to 2017,by Gender, Race/Ethnicity (N = 186,450)

46%

54%

48%

10%

11%

27%

1%

<1%

4%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Male

Female

White

Black or African American

Hispanic

Asian

American Indian or Alaska Native

Native Hawaiian or Other Pacific Islander

Other

Gender

Race/Ethnicity

7

Percentage of Examinees Taking This Version of the MCAT Exam from 2015 to 2017, by Fee Assistance, Parental Education, Testing Condition, and Repeater Status (N = 186,450)

Examinees tested in the same proportions as the past

8

Most examinees completed coursework in the natural, behavioral, and social sciences

Percentages of MCAT Examinees from 2015 to 2017 who Completed College Coursework in the Natural, Behavioral, and Social Sciences

9

Many examinees took preparation courses, used Khan tutorials, and used practice materials

Percentages of MCAT Examinees in 2017 Who Prepared for the MCAT Exam by Using an MCAT Preparation Course, Khan Academy, or AAMC Practice Materials

10

MCAT Total Score Distribution (2015-2017 Testing Years)

11

Section Score Summary for Exams Administered in 2015, 2016, and 2017 (N = 239,681) Mean (SD)

Chemical and Physical Foundations of Biological Systems 125.1 (3.0)

Critical Analysis and Reasoning Skills 124.8 (2.9)

Biological and Biochemical Foundations of Living Systems 125.3 (3.0)

Psychological, Social, and Biological Foundations of Behavior 125.4 (3.1)

How Did Admissions Officers Use MCAT Scores in 2017 Student Selection?

12

13

Admissions

committees build

classes that help

them meet their

missions and goals

Individualized consideration

is given to how each

applicant may contribute to

teaching, learning, and the

practice of medicine while

weighing and balancing the

range of criteria needed in a

class to achieve the outcomes

desired by the school.

Medical schools use holistic review to admit students that enable them to meet their missions, goals, and diversity interests

14

The new MCAT exam was designed, in part, to support holistic review

▪ The new test includes more questions per section, providing better information about examinees’ strengths and weaknesses on the exam

▪ The score scale uses a nice round number at 500 to draw attention to applicants who might otherwise be overlooked

▪ The score reports use confidence bands to describe measurement precision and score profiles to describe strengths and weaknesses

15

About 90% of 2017 applicants submitted scores from THIS version of the MCAT exam

16

Both the old and new exams

Only the old exam

Only the new exam

Admissions committees used holistic review practices to put MCAT scores in context in 2017 selection

17

Percentage and Number of 2017 Applicants Accepted into at Least One Medical School, by New MCAT Total Score and Undergraduate GPA Range

New MCAT Total

GPA Total 472-485 486-489 490-493 494-497 498-501 502-505 506-509 510-513 514-517 518-528 All

3.80-4.00 4% 5% 10% 21% 34% 54% 67% 76% 85% 89% 67%

2/51 5/98 23/241 111/528 353/1,040 920/1,692 1,538/2,310 1,870/2,461 1,798/2,112 1,796/2,014 8,416/12,547

3.60-3.79 0% 1% 5% 15% 26% 38% 54% 68% 76% 85% 48%

0/126 3/223 23/442 129/867 366/1,430 764/1,996 1,197/2,225 1,388/2,040 987/1,301 639/755 5,496/11,405

3.40-3.59 2% 1% 4% 10% 21% 29% 40% 55% 61% 71% 33%

4/183 3/293 22/552 100/955 265/1,286 479/1,635 661/1,659 694/1,260 405/660 235/329 2,868/8,812

3.20-3.39 0% <1% 3% 9% 18% 24% 32% 40% 56% 58% 22%

0/226 1/306 14/527 64/749 158/884 246/,1027 279/873 255/631 176/314 63/109 1,256/5,646

3.00-3.19 0% 2% 1% 7% 14% 21% 25% 35% 42% 37% 15%

0/248 4/253 4/363 36/490 74/511 109/507 100/395 98/282 43/103 21/57 489/3,209

2.80-2.99 1% 1% 2% 5% 6% 17% 25% 30% 29% 42% 10%

2/247 2/181 5/201 12/238 14/233 34/203 40/160 25/84 14/49 5/12 153/1,608

2.60-2.79 0% 1% 0% 2% 6% 18% 18% 17% 40% - 6%

0/157 1/117 0/142 3/132 8/126 14/79 10/57 6/36 6/15 53/868

2.40-2.59 0% 2% 2% 5% 3% 13% 25% 24% - - 5%

0/118 1/57 1/48 3/59 1/40 4/30 6/24 4/17 21/398

2.20-2.39 0% 0% 0% 0% 12% 33% 8% - - 5%

0/69 0/29 0/24 0/21 2/17 4/12 1/12 9/192

2.00-2.19 0% 0% 0% - - - - - 0%

0/41 0/13 0/10 0/86

<2.00 0% - - - - 3%

0/15 1/31

All 1% 1% 4% 11% 22% 36% 50% 64% 75% 84% 42%

8/1,481 20/1,576 93/2,556 458/4,051 1,241/5,571 2,574/7,188 3,832/7,717 4,341/6,819 3,430/4,558 2,765/3,285 18,762/44,802

Some 2017 applicants w/ high UGPAs and MCAT scores weren’t accepted

18

Percentage and Number of 2017 Applicants Accepted into at Least One Medical School, by New MCAT Total Score and Undergraduate GPA Range

New MCAT Total

GPA Total 472-485 486-489 490-493 494-497 498-501 502-505 506-509 510-513 514-517 518-528 All

3.80-4.00 4% 5% 10% 21% 34% 54% 67% 76% 85% 89% 67%

2/51 5/98 23/241 111/528 353/1,040 920/1,692 1,538/2,310 1,870/2,461 1,798/2,112 1,796/2,014 8,416/12,547

3.60-3.79 0% 1% 5% 15% 26% 38% 54% 68% 76% 85% 48%

0/126 3/223 23/442 129/867 366/1,430 764/1,996 1,197/2,225 1,388/2,040 987/1,301 639/755 5,496/11,405

3.40-3.59 2% 1% 4% 10% 21% 29% 40% 55% 61% 71% 33%

4/183 3/293 22/552 100/955 265/1,286 479/1,635 661/1,659 694/1,260 405/660 235/329 2,868/8,812

3.20-3.39 0% <1% 3% 9% 18% 24% 32% 40% 56% 58% 22%

0/226 1/306 14/527 64/749 158/884 246/,1027 279/873 255/631 176/314 63/109 1,256/5,646

3.00-3.19 0% 2% 1% 7% 14% 21% 25% 35% 42% 37% 15%

0/248 4/253 4/363 36/490 74/511 109/507 100/395 98/282 43/103 21/57 489/3,209

2.80-2.99 1% 1% 2% 5% 6% 17% 25% 30% 29% 42% 10%

2/247 2/181 5/201 12/238 14/233 34/203 40/160 25/84 14/49 5/12 153/1,608

2.60-2.79 0% 1% 0% 2% 6% 18% 18% 17% 40% - 6%

0/157 1/117 0/142 3/132 8/126 14/79 10/57 6/36 6/15 53/868

2.40-2.59 0% 2% 2% 5% 3% 13% 25% 24% - - 5%

0/118 1/57 1/48 3/59 1/40 4/30 6/24 4/17 21/398

2.20-2.39 0% 0% 0% 0% 12% 33% 8% - - 5%

0/69 0/29 0/24 0/21 2/17 4/12 1/12 9/192

2.00-2.19 0% 0% 0% - - - - - 0%

0/41 0/13 0/10 0/86

<2.00 0% - - - - 3%

0/15 1/31

All 1% 1% 4% 11% 22% 36% 50% 64% 75% 84% 42%

8/1,481 20/1,576 93/2,556 458/4,051 1,241/5,571 2,574/7,188 3,832/7,717 4,341/6,819 3,430/4,558 2,765/3,285 18,762/44,802

11% of applicants

with GPAs at or above 3.8 and

MCAT scores at or above 518 were

not admitted into any medical

schools

Other 2017 applicants with modest credentials were accepted

19

Percentage and Number of 2017 Applicants Accepted into at Least One Medical School, by New MCAT Total Score and Undergraduate GPA Range

New MCAT Total

GPA Total 472-485 486-489 490-493 494-497 498-501 502-505 506-509 510-513 514-517 518-528 All

3.80-4.00 4% 5% 10% 21% 34% 54% 67% 76% 85% 89% 67%

2/51 5/98 23/241 111/528 353/1,040 920/1,692 1,538/2,310 1,870/2,461 1,798/2,112 1,796/2,014 8,416/12,547

3.60-3.79 0% 1% 5% 15% 26% 38% 54% 68% 76% 85% 48%

0/126 3/223 23/442 129/867 366/1,430 764/1,996 1,197/2,225 1,388/2,040 987/1,301 639/755 5,496/11,405

3.40-3.59 2% 1% 4% 10% 21% 29% 40% 55% 61% 71% 33%

4/183 3/293 22/552 100/955 265/1,286 479/1,635 661/1,659 694/1,260 405/660 235/329 2,868/8,812

3.20-3.39 0% <1% 3% 9% 18% 24% 32% 40% 56% 58% 22%

0/226 1/306 14/527 64/749 158/884 246/,1027 279/873 255/631 176/314 63/109 1,256/5,646

3.00-3.19 0% 2% 1% 7% 14% 21% 25% 35% 42% 37% 15%

0/248 4/253 4/363 36/490 74/511 109/507 100/395 98/282 43/103 21/57 489/3,209

2.80-2.99 1% 1% 2% 5% 6% 17% 25% 30% 29% 42% 10%

2/247 2/181 5/201 12/238 14/233 34/203 40/160 25/84 14/49 5/12 153/1,608

2.60-2.79 0% 1% 0% 2% 6% 18% 18% 17% 40% - 6%

0/157 1/117 0/142 3/132 8/126 14/79 10/57 6/36 6/15 53/868

2.40-2.59 0% 2% 2% 5% 3% 13% 25% 24% - - 5%

0/118 1/57 1/48 3/59 1/40 4/30 6/24 4/17 21/398

2.20-2.39 0% 0% 0% 0% 12% 33% 8% - - 5%

0/69 0/29 0/24 0/21 2/17 4/12 1/12 9/192

2.00-2.19 0% 0% 0% - - - - - 0%

0/41 0/13 0/10 0/86

<2.00 0% - - - - 3%

0/15 1/31

All 1% 1% 4% 11% 22% 36% 50% 64% 75% 84% 42%

8/1,481 20/1,576 93/2,556 458/4,051 1,241/5,571 2,574/7,188 3,832/7,717 4,341/6,819 3,430/4,558 2,765/3,285 18,762/44,802

14% of applicants

with GPAs 3.00 -3.19 and

MCAT scores 498-501 were

admitted into at least one medical

school

What Are We Learning About the Impact, Use, and Predictive Validity of the Exam?

20

21 medical schools and 2 pre-health advisors from undergraduate institutions are working together to evaluate the MCAT exam

21

The MCAT Validity Committee is conducting research in three broad areas

22

Academic Preparation,

Diversity, and Fairness

Admissions Decision Making

Predicting Academic

Performance

Type of Fairness

Definition

Exam Fairness

MCAT scores have the same meaning and predict student performance equally well for examinees from different backgrounds.

Procedural Fairness

Admissions officers and their committees have ample information and

resources to make appropriate and balanced use of MCAT scores in

admissions.

Societal Fairness

Aspiring physicians from different groups have equity in access to

preparation materials and opportunities to prepare for the exam.

Fairness is front and center in evaluating the exam

23

Today’s findings show summative performance across year 1 courses from the 2016 validity school entrants with scores from the current exam

24

Matriculants

w/ current scoresN ≈ 8,000

Participants

N ≈ 1,000

w/ current

scores

2016 MatriculantsN ≈ 21,000

National

population

Validity

school

sample

At the validity schools, MCAT scores and UGPAs predict summative performance across year 1 courses. Combined, they predict better than either one alone.

25

MCAT scores provide comparable prediction for students from different sociodemographic backgrounds

▪ Research studied these early relationships for students grouped by:

▪ Race/ethnicity

▪ Socioeconomic status

▪ Gender

▪ The associations we just reported hold for examinees from different backgrounds.

▪ So far, MCAT scores neither over- nor under-predict the performance of students from any of these groups.

26

We have a lot more to learn about how students do in their second year, in their clerkships

and on their USMLE exams, and their graduation from undergraduate medical school.

When viewed in context with UGPAs, experiences, and attributes, MCAT scores provide useful information about applicants’ likely success in medical school

▪ In total, these results show that MCAT scores predict how well, on average, students with a given MCAT score are likely to do in medical school.

▪ It is also true that students’ performance in medical school is influenced by many factors.

▪ In the end, some medical students perform better, as well as, and less well than their MCAT scores predict.

▪ These findings call for giving balanced consideration in admissions decisions to each applicant’s past experiences and personal attributes as well as to grades and MCAT scores.

27

MCAT Testing Program Updates for 2018

28

What’s new and in the queue?

▪ We continue to make improvements to test administration and test preparation

▪ Enhancements to registration system, as well as test delivery in 2018 testing year

▪ Improvements to test preparation delivery in 2019 testing year

29

Pearson VUE began administering the MCAT exam in January 2018

▪ Selected after a comprehensive review as part of our due diligence to students, advisors, and admissions officers

▪ Consistent, high quality testing experience

▪ More Friday & Saturday test dates

▪ Vast majority in convenient locations

▪ Enhanced font type and size, screen resolution, and usage of keyboard short cuts

30

Selecting dates and locations is easier for examinees

31

There are 30 U.S. test dates in 2018 from January - September

32Students-residents.aamc.org/mcatdates

We’ve seen significant improvements in examinees receiving their preferred exam date and testing center

33

So far in 2018 This time last year in

2017

Change

Preferred Date

• Were able to register for their

preferred exam date

82% 64% +18.1%

Preferred Test Center

• Were able to register within 10

miles of preferred test center

• Registered greater than 50

miles from preferred test center

92%

4%

71%

17%

+20.3%

-13.1%

2018 testing by the numbers

• 34,359 unique examinees with scores who tested from January through May 2018

• Scores are released 30-35 days after administrations

• 1% of examinees tested with accommodations

34

Improvements to test preparation delivery

▪ Test preparation transition to new delivery system

▪ No impact to students testing in 2018 testing year

▪ The new platform will provide

▪ One place for students and advisors to easily access our free and low-cost products and resources

▪ A solid foundation from which we can build

▪ More information will be available later this summer

35

Resources for Advisors and Students

36

AAMC works to provide equity in access to information and resources about preparing for the MCAT exam

▪ Outreach campaigns directly to students from sociodemographic groups underrepresented in medicine

▪ Distribute monthly newsletter, the Pre-med Navigator, with important information, resources, and tips

▪ Outreach to students through their advisors

▪ The Khan Academy has over 1,100 free tutorials on exam content

▪ AAMC has a host of free and low cost resources that help students prepare and practice for the new exam

37aamc.org/premednavigator

AAMC offers a Fee Assistance Program

▪ Approval valid for up to 2 calendar years

▪ Financial information required for all living parents

▪ DACA students eligible

MCAT Benefits

▪ Reduced registration fees

▪ Free MCAT prep materials

▪ Stipend for updated evaluation for accommodations

Benefits are not retroactive

38 aamc.org/fap

Additional Benefits

• AMCAS application

submission (up to 16 schools)

• MSAR access

MCAT student preparation resources introduced for 2018 testing year

▪ Practice Exam 3 ($35)

▪ The Official Guide to the MCAT Exam, Fifth Edition ($30)

▪ Enhanced Guide to Creating a Study Plan (FREE)

▪ How I Prepared for the MCAT Exam Testimonials (FREE)

39Students-residents.aamc.org/mcatprep

See handout for full list

of resources.

Resources are available on AAMC’s Advisor Hub

▪ Information on MCAT scores and reporting dates

▪ MCAT test preparation resources and pricing (30% discount)

▪ MCAT data and research

▪ Additional information and resources

40aamc.org/advisors

See handout for full list

of resources.

Questions?

41

Pre-health Advisor Support

T: 202-828-0950

E: advisors@aamc.org

42