1

Transitioning MathTranscript Research in Support of

TMP

Dave PavelchekWashington State University

Social & Economic Sciences Research Center Puget Sound Division

360-586-4583 http://www.sesrc.wsu.edu/k12

dpavel@wsu.eduDecember 2007

2

Background SESRC has been providing technical assistance to the TMP project(s) on evaluation issues

SESRC has also been conducting transcript studies for districts, analyzing the relationship between courses and grades and post high school outcomes

3

The Shoreline Transition Math Partnership Shoreline School District and Shoreline Community College

Interested in the patterns of students currently making the post-secondary transition

For insight into current dynamics

As a potential baseline to measure change

4

Study Sample

Graduated from Shoreline District in 2003, 2004 or 2005

Enrolled in district at least 3 years

SPED & ELL students excluded

5

Data High School transcripts for 1,785 students

Community college math placement scores, courses, and grades for the 713 enrolled at Shoreline Community College

Baccalaureate Math Placement Test (MPT) scores for 219 students at five public universities

Post-secondary followup ranged from 2 to 4 years for the three classes

6

Definitions of “College Ready” A student who enrolled in a college-level course at a community college without first taking a remedial course.

OR

A student whose placement test score placed them at a college math level (either CTC or BA)

7

Coding Courses from Transcript to Ordinal Scale

Math Level Math Course0 Pre-high school math course or variable content1 Integrated Math 1 - "Typical” freshman math class 2 Integrated Math 2 - "Typical” sophomore math class 3 Integrated Math 3 -"Typical” junior math class

3.5Math Analysis at Shorecrest (some review of Math 3 and some Pre-Calculus content)

4 Pre-calculus, AP Statistics, Math Analysis at Shorewood5 Calculus, AP Calculus

8

What the Graduates Do

College Type of SSD Graduates

25%15%

7%

33%

4%16%

0%

20%

40%

60%

80%

100%

SCC Only SCC and 4 year

2 year(other than

SCC)

4 year 2 and 4year

No College

% st

uden

ts

9

College Type by Gender

College Type by Gender

8%

26%20%

4%

30%

12% 11%4%

37%

7%17%

24%

0%

20%

40%

60%

80%

100%

SCC Only SCC and 4year

2 year (otherthan SCC)

4 year 2 and 4 year No College

% o

f stu

dent

s

Male Female

10

College Type by Ethnicity

College Type by Ethnicity

25%

15%7%

35%

4%

14%

29%

11% 11%16%

2%

31%

0%

10%

20%

30%

40%

50%

% o

f stu

dent

s

White or Asian (n=1,666) Non-white/non-Asian (n=119)

11

Math Grades and College Attendance(Note - percentages are of each grade level)

College Type for Each High School Math GPA Level

4%

51%

14% 19%

4% 8%11%

40%

11% 13%4%

21%15%

46%

4% 1%2%

32%

0%

20%

40%60%

80%

100%

SCC Only SCC and 4year

2 year(other than

SCC)

4 year 2 and 4year

No College

% o

f st

uden

ts

A or B C D or F

12

Math Preparation (Revised to 29%, 20%, 50% for highest passed)

Highest High School Math Level(for all SSD Grads)

Int. Math 322%

Math 0.5 - Math 2

26%Math Anal.-

Calc.52%

13

Math Preparation & College Type(Note – the percents are of each Prep Level) Highest Level Taken

College Type for Each Highest High School Math Level

17%

36%

1%5%

39%

2%14%

5%

19%17%

38%

8% 5%

56%

5%

19%13%2%

0%

20%

40%

60%

80%

100%

SCC Only SCC and 4year

2 year(other than

SCC)

4 year 2 and 4year

No College

% o

f st

ud

ents

Math 0.5 - 2 Int. Math 3 Math Anal.- Calc.

14

Composition of Math Preparation for Shoreline Grads at Different College Types Highest Math Taken

Highest Level High School Math for Each College Type

60% 62%

7%8%

40%

2%

21%

26%12%

25%

33%

23%

18%

86%67%66%

27% 17%

0%

20%

40%

60%

80%

100%

SCC Only SCC and 4year

2 year(other than

SCC)

4 year 2 and 4year

No College

% o

f st

ud

ents

Math 0.5 - 2 Int. Math 3 Math Anal.- Calc.

15

The Traditional College ‘Gateway’

Grade in Integrated 3 Math(for all SSD Grads)

C23%

A or B65%

D or F12%

16

Int 3 Grade & College Type(Note – the percents are of each Grade Level)

College Type for Each Integrated Math 3 Grade Category

6%5%

56%

2%

20%

11% 10%5%

24%

8%18%

36%

17% 14%3%

11%7%

48%

0%

20%

40%

60%

80%

100%

SCC Only SCC and 4year

2 year(other than

SCC)

4 year 2 and 4year

NoCollege

% o

f st

ud

ents

A or B C D or F

17

Composition of Int 3 Grades for Shoreline Grads at Different College Types

Grade in Integrated Math 3 for Each College Type

50%70%

84%

33%

68%

35%

30%

22%13%

46%

22%

39%

10%20%

8%3%21%26%

0%

20%

40%

60%

80%

100%

SCCOnly

SCCand 4year

2 year(otherthan

SCC)

4 year 2 and 4year

NoCollege

% o

f st

ud

en

ts

A or B C D or F

18

The Senior Math Question

Math in Senior Year(for all SSD Grads)

No41%

Yes59%

19

Senior Math Composition for Shoreline Grads at Different College Types

Percent of Students in Each College Type Who Took Math Senior Year

35%70%82%

34%67%

45%

65%

30%18%

66%33%

55%

0%20%

40%60%

80%100%

SCC Only SCC and4 year

2 year(other than

SCC)

4 year 2 and 4year

NoCollege

% o

f st

ud

ents

Math Senior Year No Math Senior Year

20

What Level of Math at Community College?

Level of Math Taken at Shoreline Community College

College Math Only22%

Remedial Math Only

21%

Remedial and College Level

Math 14%

No Math Taken43%

21

Delayed Math at Community College?

Math in 1st Year at SCC

No Math in 1st Year

18%

Math in 1st Year38%

No Math in College

44%

22

Grades in Community College Math

College Level Math GPA

C29%

D or F23%

A or B48%

23

Grades in Remedial Mathat Community College

Remedial Level Math GPA

D or F31%

A or B32%

C37%

24

Remedial Grades & Further Progressat Community College

Remedial Math Grade by College Level Math Grade

60%

20%

15%40%

85%

80%

0%

20%

40%

60%

80%

100%

A or B C D or F

Remedial Math Grade

% o

f S

tude

nts

C or higher in College Level Math Lower than C in College Level Math

25

Adaptive Placement Test Components at Shoreline CC

Highest Level Placement Test Taken

Inter. Algebra

48%

College Algebra

19%Elem.

Algebra13%

Numeric20%

26

Placement Results Shoreline CC

Course Placement from Placement Test Score

Math070/ Bus175

5%

Math060/ ABE13%

College Math9%

Math08015%

Math090/ 09758%

27

Is There Math After Placement?(Shoreline CC)

Placement Test Score Math Level by Subsequent Math Courses Taken

20%29%

13%

38%

7%16%

0%

77%

0%20%40%60%80%

100%

Remedial&

CollegeMath

RemedialMathOnly

CollegeMathOnly

No Math Remedial&

CollegeMath

RemedialMathOnly

CollegeMathOnly

No Math

Remedial Level Placement Math Score College Level Placement Math Score

% o

f stu

dent

s

28

Empirical Evidence for Placement Process (Shoreline CC)

Next Math Enrollment by Placement Test Results

0

100

200

300

400

Didn't take test <90 90-99 college level

Placement Test Results

College Level

Remedial and College Level

Remedial

No Math

29

Complexity and the Colinearity Problem What we would like to know is causation – what

‘gives’ students better math skills and retention

If not that, predictive value – what tells us how what a student should tackle next, and how hard it will be

Unfortunately, the usual suspects travel in packs. The same students who get higher math grades in Int 3, tend to get higher math grades overall, and take more and more advanced math, and are more likely take math in senior year

This is called “colinearity” & it is why we resort to multivariate regression

30

Research Questions on Remedial Enrollments How well are Placement Test Scores

predicted by high school math preparation?

How well is Enrollment in Remedial Math predicted by high school math preparation?

How well is Enrollment in College Level Math predicted by high school math preparation?

How well are Grades in College Level Math predicted by high school math preparation?

31

College-Readiness Rates

Percentage of Students who Passed Placement Test or Enrolled Directly in College-level at CC

by Highest Math Course Passed

16%23%

42%52%

72% 76%

0%

20%

40%

60%

80%

100%

Int. Math 1 orlower (N=119)

Int. Math 2(N=124)

Int. Math 3(N=210)

Math. Anal.(N=46)

Pre-Calc.(N=279)

Calc. (N=98)

32

College-Readiness Rates and Preparation – Shoreline CC

Placement Test Results of Graduates by Highest Math Completed

10% 18%

58%

21%

45%68%

80%75%

42%

79%52%

30%10% 7% 0%

2%3%0%

20%

40%

60%

80%

100%

Int. Math 1 orlower (N=470)

Int. Math 2(N=77)

Int. Math 3(N=107)

Math. Anal.(N=10)

Pre-Calc.(N=79)

Calc. (N=19)

% st

uden

ts

College Level Math Level 097-099 Math Level <090

33

College-Readiness Rates and Preparation – Public Baccalaureates

Placement Test Results of Graduates by Highest High School Math Passed

(Public BA Students Only)

40%

11% 4%

58%

89% 96%

0%

20%

40%

60%

80%

100%

Int. Math 3 (N=31) Math. Anal. (N=15) Pre-Calc. (N=122) Calc. (N=51)

Remedial College

34

Untangling Complexity, What Factors Predict Best

Cox & Snell

Nagelkerke

INCLUDING BA STUDENTS

Placed at Remedial Level - inc BA 0.485 0.653 4 4 -1

SHORELINE CTC ONLY

Placed at Remedial Level 0.207 0.458 3 2 -2

Placed below Math 97 0.224 0.307 4 -1*

Taking Remedial ~ 0.365 ~ 0.500 4 -3 4

Going on to College Level 0.109 0.147 4

Going on to College Level** w/ remedial grade 0.25 0.337

-3 4

College Level Course w/wo remedial** .228-.235 .312-.321 4 1 -1*

College-Level Math grades 0.605 1.352 0.769

BOLDED series use "Highest Math Passed" - unbold use "Highest Math Taken"**Forward and Backward Regression significantly disagree about some factors - show best fit (R-square)

Remedial Course Grade

Gender (female)

Grade in Highest Math Course

Math in 12th grade

EthnicityR-square estimates

0.225

HighestMath Course

Math GPA in HS

35

Factors in the Remedial Process

Cox & Snell Nagelkerke

INCLUDING BA STUDENTS

Placed at Remedial Level - inc BA 0.485 0.653 4 4 -1

SHORELINE CTC ONLY

Placed at Remedial Level 0.207 0.458 3 2 -2

Placed below Math 97

Taking Remedial ~ 0.365 ~ 0.500 4 -3 4

BOLDED series use "Highest Math Passed" - unbold use "Highest Math Taken"

R-square estimatesHighest Math

Math GPA in HS

Gender (female)

Grade in Highest Math Course

36

College Performance Factors –Shoreline CC

Cox & Snell

Nagelkerke

Going on to College Level 0.109 0.147 4

Going on to College Level** w/ remedial grade 0.25 0.337

-3 4

College Level Course w/wo remedial ** .228-.235 .312-.321 4 1 -1*

College-Level Math grades 0.605 1.352 0.769

**Forward and Backward Regression significantly disagree about some factors - show best fit (R-square)

Remedial Course Grade

Gender (female)

Grade in Highest Math Course

Math in 12th grade

EthnicityR-square estimates

0.225

Highest Course Passed

Math GPA in HS

37

Findings from Similar Projects

Different districts are interested in different questions.

Data availability changes

Methods improve (hopefully).

38

Examples of Other Uses of Transcript Research

Investigating Remedial English

Patterns of enrollment & persistence by type of college

Role of CTE in college selection, enrollment & persistence

39

Transcript Research on Remedial English (Not Shoreline SD Data)

Percentage of Graduates who are "College Ready" by Highest Level of English Taken

25%

47%

69% 67%

93%

0%

25%

50%

75%

100%

Freshman Level (N=4)

SophomoreLevel (N=19)

Junior Level (N=164)

Senior Level (N=314)

AP/Honors (N=184)

40

Findings from Regression on English College Readiness(Pooled Data from 2 Districts)

Logistic regression with an R2 value of approximately 20% (N=717)

Highest level English course taken strongest

English GPA was second-most important

ELL, & Ethnicity were also significant

Non-significant variables: gender and taking any English in the senior year

41

However, remedial English is not just a problem for recent immigrants(Not Shoreline SD Data)

E nro llm e nts in R e m e dia l E ng lis h by R a c e a nd E L L S ta tus

M in o rit y & N o t ELL3 3 %W h it e & ELL

E lig ib le2 %

M in o rit y & ELL E lig ib le

2 1 %

W h it e & N o t ELL4 4 %

42

Persistence from First to Second College Year by Level of HS Coursework (Not Shoreline Data)

70%75%

84%

92% 93%

78%

86% 86%

94%

86% 85% 87%94%

83%88%86%

0%

25%

50%

75%

100%

1 (2

0)

2 (7

3)

3 (2

79

)

4 (3

11)

5 (1

38

)

RS

(21

)

2 (2

7)

3 (2

22

)

4 (4

11)

5 (1

49

)

RS

(32

)

2 (7

)

3 (9

3)

4 (5

46

)

5 (1

24

)

RS

(66

)

Highest Level Math Highest Level Science Highest Level English

43

Percent Transferring from a Local 2-Year to 4-Year College by HS Grades(Not Shoreline SD Data)

17%

31%

46%

14%

29%

43%

9%

24%

39%

0%

25%

50%

75%

100%

0 to

1.9

(48)

2 to

2.9

(81)

3 to

4.0

(28)

0 to

1.9

(37)

2 to

2.9

(70)

3 to

4.0

(49)

0 to

1.9

(11)

2 to

2.9

(80)

3 to

4.0

(66)

Cum GPA Math Cum GPA Science Cum GPA English

44

To view other WSU-SESRC research reports on education topics please see our website at:

http://www.sesrc.wsu.edu/k12/Email:

dpavel@wsu.edusternpo@wsu.edu

For more information