three studies to drive campus conversations on student success

An initiative of the Research & Planning Group

for California Community Colleges

Three Studies to Drive Campus Conversations on Student Success

Rob Johnstone, RP Group

An initiative of the Research & Planning Group

for California Community Colleges

Today we’ll cover:

Working toward a “Culture of Inquiry” – selected findings from statewide projects

Grades & Success in Sequenced CoursesLevels of Preparedness in GE Courses –

success of simultaneously enrolled students in Math / English courses

Cohort Tracking in Basic Skills

An initiative of the Research & Planning Group

for California Community Colleges

Ground Rules

Ask lots of questions – especially nagging and tough ones

If I’m talking too fast, throw something at me (preferably not something sharp)

Data doesn’t make meaning; we make meaning from data

Spontaneous outbursts of applause are encouraged

An initiative of the Research & Planning Group

for California Community Colleges

Data, Evidence, College Decision-making, Student Success, and You…

An initiative of the Research & Planning Group

for California Community Colleges

How do colleges make good decisions ?

“Learning emerges from an ongoing conversation about things that matter with both passion and discipline”

- Parker PalmerThe Courage to Teach

If you want to learn you have to take part in the conversation

Learning is founded on evidence and inquiry

An initiative of the Research & Planning Group

for California Community Colleges

Framing Thought #1

“Data do not speak for themselves. What is also needed are occasions that bring educators together to examine evidence about student learning, reflect on its meaning and identify approaches that yield better results.” Carnegie Foundation (2009)

An initiative of the Research & Planning Group

for California Community Colleges

Framing Thought #2

Decision making in complex systems is a highly social process, not solely an intellectual exercise.

The process is composed of many small acts, carried out by different people at different points in time.

An initiative of the Research & Planning Group

for California Community Colleges

Institutional Research (IR) & You

IR is about helping the college and its practitioners make better decisions

Institutional research is fundamentally an interventionist discipline.

We are using non-experimental data to examine behavior in complex human systems

We are not seeking absolute truths; rather we are looking for patterns of evidence that inform action-oriented decisions

An initiative of the Research & Planning Group

for California Community Colleges

Linear Model of Institutional Research

Faculty/Staff

Researcher

Primary Responsibility

Key Features:

• Request-driven• Highly delineated tasks• Emphasis is on generating

metrics

An initiative of the Research & Planning Group

for California Community Colleges

Collaborative Model of Inst. Research

Faculty/Staff

Researcher

Joint Activity

Primary Responsibility

Key Features:• Dialogue-rich• Jointly-driven processes

An initiative of the Research & Planning Group

for California Community Colleges

Colleges may be focusing on the wrong data.

Finding from RP Research

An initiative of the Research & Planning Group

for California Community Colleges

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

Retention Rate

Success Rate

Consider the multitude of changes over this period:

- faculty/staff turnover- program successes/failures- changing student demographics - budget contractions/expansion- evolving state & accreditation mandates- leadership turnover

and yet these performance metrics remained amazingly stable.

What does that tell us about the usefulness of these metrics in setting institutional strategies?

Twenty Year Trend for Common Institutional Outcome Measures

An initiative of the Research & Planning Group

for California Community Colleges

Much of what is needed to support good decision making lies beyond the current grasp of our institutional databases.

An Observation

An initiative of the Research & Planning Group

for California Community Colleges

What is Our College’s Transfer Rate?

An initiative of the Research & Planning Group

for California Community Colleges

Insight from the Field: A Look at Three Studies

An initiative of the Research & Planning Group

for California Community Colleges

Section 2: Grades & Success in Sequenced

Courses

An initiative of the Research & Planning Group

for California Community Colleges

82%

67%

49%

37%42%

59%

18%

33%

51%

63%58%

41%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

A in English 826 B in English 826 C in English 826 Other Grade in English 826

English 836/846 Non-Success

English 836/846 -First English Course

Success Rates of English 836/846 Students by Entry Group

Success Rate Non-Success Rate

An initiative of the Research & Planning Group

for California Community Colleges

88%

80%

58%

47% 45%

65%

12%

20%

42%

53% 55%

35%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

A in English 836/846 B in English 836/846 C in English 836/846 Other Grade in English 836/846

English 100/105 Non-Success

English 100/105 -First English Course

Success Rates of English 100 Students by Entry Group

Success Rate Non-Success Rate

An initiative of the Research & Planning Group

for California Community Colleges

60%

51%

38%

19%

43%

54%

40%

49%

62%

81%

57%

46%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

A in Math 811 B in Math 811 C in Math 811 Other Grade in Math 811

Math 110/111 Non-Success

Math 110/111 - First Math Course

Success Rates of Math 110 Students by Entry Group

Success Rate Non-Success Rate

An initiative of the Research & Planning Group

for California Community Colleges

77%

55%

35%

21%

45%

57%

23%

45%

65%

79%

55%

43%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

A in Math 110/112 B in Math 110/112 C in Math 110/112 Other Grade in Math 110/112

Math 120/122 Non-Success

Math 120/122 - First Math Course

Success Rates of Math 120 Students by Entry Group

Success Rate Non-Success Rate

An initiative of the Research & Planning Group

for California Community Colleges

96%

79%

40%

71%

61%

77%

4%

21%

60%

29%

39%

23%

0%

20%

40%

60%

80%

100%

120%

A in Chemistry 210 B in Chemistry 210 C in Chemistry 210 Other Grade in Chemistry 210

Chemistry 220 Non-Success

Chemistry 220 - First Chemistry Course

Success Rates of Chemistry 220 Students by Entry Group

Success-Rate Non-Success Rate

An initiative of the Research & Planning Group

for California Community Colleges

94%91%

76%

64%

81%85%

89%

6%9%

24%

36%

19%15%

11%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

A in Biology 250 B in Biology 250 C in Biology 250 Other Grade in Biology 250

Biology 260 Non-Success

Biology 260 - First Biology Course

Simultaneous / Other Group for

Biology 260

Success Rates of Biology 260 Students by Entry Group

Success Rate Non-Success Rate

An initiative of the Research & Planning Group

for California Community Colleges

91%

81%

68%

29%

66%71%

9%

19%

32%

71%

34%29%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

A in Economics 100 B in Economics 100 C in Economics 100 Other Grade in Economics 100

Economics 102 Non-Success

Economics 102 - First Economics Course

Success Rates of Economics 102 Students by Entry Group

Success Rate Non-Success Rate

An initiative of the Research & Planning Group

for California Community Colleges

Section 3: Success in GE Courses by Simultaneously

Enrolled English / Math Students

An initiative of the Research & Planning Group

for California Community Colleges

Psychology 100 Success of Students Simultaneously Enrolled in English

Simultaneously Enrolled English Course

Count in Psyc 100

% of Psyc 100 Enr.

Success in Psyc 100

English 826 139 2% 31%

English 836 / 846 730 9% 48%

English 100 / 105 1,209 15% 68%

English 110 530 7% 75%

Not in English Course 5,207 67% 64%

Total 7,815 100% 63%

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade

An initiative of the Research & Planning Group

for California Community Colleges

Psychology 100 Success of Students Simultaneously Enrolled in Math

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade

Simultaneously Enrolled Math Course

Count in Psyc 100

% of Psyc 100 Enr.

Success in Psyc 100

Math 811 157 2% 43%

Math 110 / 111 / 112 970 12% 51%

Math 120 / 122 /123 773 10% 63%

Transfer-Level Math 997 13% 76%

Not in Math Course 4,918 63% 64%

Total 7,815 100% 63%

An initiative of the Research & Planning Group

for California Community Colleges

Success in 5 Highly Enrolled GE Courses by English Enrollment Level

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade

Simultaneously Enrolled English Course

Success in Psyc 100

Success in SPCH 100

Success in ECON 100

Success in HIST 201

Success in HSCI130

English 826 31% 54% 27% 48% 60%

English 836 / 846 48% 57% 56% 43% 74%

English 100 / 105 68% 73% 73% 51% 82%

English 110 75% 82% 79% 64% 88%

Not in English Course 64% 74% 69% 59% 73%

Total 63% 73% 69% 58% 75%

Total Enrollments 7,815 5,793 5,643 4,603 4,475

An initiative of the Research & Planning Group

for California Community Colleges

Success in 5 More Highly Enrolled GE Courses by English Enrollment Level

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade

Simultaneously Enrolled English Course

Success in FITN 334

Success in ACTG 100

Success in MUS 202

Success in BIOL 250

Success in SOCI 100

English 826 48% 37% 31% N/A 40%

English 836 / 846 58% 50% 64% 41% 50%

English 100 / 105 68% 66% 74% 60% 59%

English 110 76% 73% 80% 65% 74%

Not in English Course 66% 69% 70% 70% 69%

Total 66% 68% 71% 68% 68%

Total Enrollments 4,447 3,718 3,624 3,577 2,813

An initiative of the Research & Planning Group

for California Community Colleges

Success in 5 Highly Enrolled GE Courses by Math Enrollment Level

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade

Simultaneously Enrolled Math Course

Success in PSYC 100

Success in SPCH 100

Success in ECON 100

Success in HIST 201

Success in HSCI 130

Math 811 43% 54% 26% 38% 49%

Math 110 / 111 / 112 51% 63% 46% 48% 71%

Math 120 / 122 /123 63% 67% 62% 51% 79%

Transfer-Level Math 76% 78% 80% 63% 86%

Not in Math Course 64% 75% 69% 59% 74%

Total 63% 73% 69% 58% 75%

Total Enrollments 7,815 5,793 5,643 4,603 4,475

An initiative of the Research & Planning Group

for California Community Colleges

Success in 5 More Highly Enrolled GE Courses by Math Enrollment Level

Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade

Simultaneously Enrolled Math Course

Success in FITN 334

Success in ACTG 100

Success in MUS 202

Success in BIOL 250

Success in SOCI 100

Math 811 59% 43% 50% N/A N/A

Math 110 / 111 / 112 61% 46% 58% 51% 46%

Math 120 / 122 /123 68% 55% 73% 54% 55%

Transfer-Level Math 77% 77% 82% 69% 75%

Not in Math Course 66% 70% 70% 70% 69%

Total 66% 68% 71% 68% 68%

Total Enrollments 4,447 3,718 3,624 3,577 2,813

An initiative of the Research & Planning Group

for California Community Colleges

Section 4: Cohort Tracking in Basic Skills Sequences

An initiative of the Research & Planning Group

for California Community Colleges

Cohort Tracking

Cohorts tracked include students whose first math course at Skyline was between 2002-03 and 2006-07. Students were tracked for three academic years.

Students were tracked based on the first math/English/ESL course that they took at Skyline (not on their placement).

An initiative of the Research & Planning Group

for California Community Colleges

Math Cohort Tracking – Starting in Fundamentals

Starting Level Att. Fund

Pass Fund

Att. Beg.

Pass Beg.

Att. Inter.

Pass Inter.

Att.Tran.

Pass Tran.

Fundamentals

1,863

1,215 939 624 331 212 134 99

% of starting cohort

100% 50% 18% 7%

% from previous

step--- 65% 77% 66% 53% 64% 63% 74%

*Note: Fundamentals = Math 811, Beginning Algebra = 110 or 111+112, Intermediate Algebra = 120 or 122+123, Transfer = 130,

150, 200-level

11%33% 5%65%

An initiative of the Research & Planning Group

for California Community Colleges

Math Cohort Tracking – Starting in Beginning Algebra

Starting Level Att. Beg.

Pass Beg.

Att. Inter.

Pass Inter.

Att.Tran.

Pass Tran.

Beginning Algebra

3,449

2,306

1,363 934 577 440

% of starting cohort

100% 40% 27% 17%

% from previous

step--- 67% 59% 69% 62% 76%

*Note: Beginning Algebra = 110 or 111+112, Intermediate Algebra = 120 or 122+123, Transfer =

130, 150, 200-level

27% 13%67%

An initiative of the Research & Planning Group

for California Community Colleges

Math Cohort Tracking – Starting in Intermediate Algebra

Starting Level Att. Inter.

Pass Inter.

Att.Tran.

Pass Tran.

Intermediate Algebra

2,086

1,494 891 647

% of starting cohort

100% 43%

% from previous

step--- 72% 60% 73%

*Note: Intermediate Algebra = 120 or 122+123, Transfer = 130, 150, 200-level

31%72%

An initiative of the Research & Planning Group

for California Community Colleges

English Tracking – Starting in Two Levels Below Transfer

Starting LevelAtt. Two

Below

Pass Two

Below

Att. One

Below

Pass One

Below

Att. Trans.

A

Pass Trans.

A

Att. Trans.

B

Pass Trans.

B

Two Levels Below

Transfer

2,328

1,566

1,179 924 742 591 440 362

% of starting cohort

100% 51% 32% 19%

% from previous

step--- 67% 75% 78% 80% 80% 74% 72%

*Note: Two Levels Below = 826, One Level Below = 836 or 846, Transfer A = English 100 or 105, Transfer B = 110 or 165

25%40% 16%67%

An initiative of the Research & Planning Group

for California Community Colleges

English Tracking – Starting in One Level Below Transfer

Starting LevelAtt. One

Below

Pass One

Below

Att. Trans.

A

Pass Trans.

A

Att. Trans.

B

Pass Trans.

B

One Level Below

Transfer3,017 2,246 1,666 1,391 1,038 879

% of starting cohort 100% 55% 27% 34%

% from previous step --- 74% 74% 83% 75% 85%

*Note: One Level Below = 836 or 846, Transfer A = English 100 or 105, Transfer B = 110 or 165

46% 29%74%

An initiative of the Research & Planning Group

for California Community Colleges

English Tracking – Starting in Transfer A

Starting LevelAtt.

Trans. A

Pass Trans.

A

Att. Trans.

B

Pass Trans.

B

Transfer A 3,320 2,615 1,641 1,406

% of starting cohort 100% 49%

% from previous step --- 79% 63% 86%

*Note: Transfer A = English 100 or 105, Transfer B = 110 or 165

42%79%

An initiative of the Research & Planning Group

for California Community Colleges

ESL Tracking – Starting in Pre Intermediate

Starting Level

Att. Pre

Inter.

Pass Pre

Inter.

Att. Inter.

Pass Inter.

Att. High Inter.

Pass High Inter/

Att. Trans

.

Pass Trans

.

Pre Intermediate 259 177 123 91 78 56 44 32

% of starting cohort

100% 68% 47% 35% 30% 22% 17% 12%

% from previous

step--- 68% 69% 74% 86% 72% 79% 73%

*Note: Pre Intermediate = 820 / 821-822 / 852-862-872; Intermediate = 830 / 831-832 / 853-863-873; High Intermediate = 840 / 841-842;

Transfer = 400 or English 100/105

22%35% 11%68%

An initiative of the Research & Planning Group

for California Community Colleges

ESL Tracking –Starting in Intermediate

Starting Level

Att. Inter.

Pass Inter.

Att. High Inter.

Pass High Inter/

Att. Trans.

Pass Trans.

Intermediate 361 250 177 133 99 79

% of starting cohort 100% 69% 49% 37% 27% 22%

% from previous step --- 69% 71% 75% 74% 80%

*Note: Intermediate = 830 / 831-832 / 853-863-873; High Intermediate = 840 / 841-842; Transfer = 400 or

English 100/105

37% 22%69%

An initiative of the Research & Planning Group

for California Community Colleges

ESL Tracking – Starting in High Intermediate

Starting LevelAtt.

High Inter.

Pass High Inter/

Att. Trans.

Pass Trans.

High Intermediate 366 243 158 123

% of starting cohort 100% 66% 43% 34%

% from previous step --- 66% 65% 78%

*Note: High Intermediate = 840 / 841-842; Transfer = 400 or English 100/105

34%66%

An initiative of the Research & Planning Group

for California Community Colleges

Something to consider…

25%40% 16%67%

46% 29%74%

42%79%

English Progression

34%

37% 22%69%

22%35% 11%68%

ESL Progression

66%11%33% 5%65%

27% 13%67%

31%72%

Math Progression

An initiative of the Research & Planning Group

for California Community Colleges

25%40% 16%67%

46% 29%74%

42%79%

English Progression

34%

37% 22%69%

22%35% 11%68%

ESL Progression

66%11%33% 5%65%

27% 13%67%

31%72%

Math Progression

Something to consider…

25%40% 16%67%

46% 29%74%

42%79%

English Progression

34%

37% 22%69%

22%35% 11%68%

ESL Progression

66%11%33% 5%65%

27% 13%67%

31%72%

Math Progression

An initiative of the Research & Planning Group

for California Community Colleges

25%40% 16%67%

46% 29%74%

42%79%

English Progression

34%

37% 22%69%

22%35% 11%68%

ESL Progression

66%11%33% 5%65%

27% 13%67%

31%72%

Math Progression

Something to consider…

An initiative of the Research & Planning Group

for California Community Colleges

Rob Johnstone, Project Directorjohnstoner@smccd.edu

Priyadarshini Chaplot, Project Coordinatorpchaplot@rpgroup.org

Lead Technical CoachesDarla CooperJanet FulksBob GabrinerCathy HassonKen MeehanBob PachecoGreg StoupIan Walton

BRIC TAP Core Team

An initiative of the Research & Planning Group

for California Community Colleges

Find Out More!RP Group Websitewww.rpgroup.org

BRIC TAPhttp://www.rpgroup.org/projects/BRIC.html

Rob Johnstone, BRIC Project Directorjohnstoner@smccd.edu

Kathy Booth, RP Group Executive Directorkbooth@rpgroup.org

Priya Chaplot, BRIC Project Coordinatorpchaplot@rpgroup.org