THE DATA MADE ME DO IT:

LESSONS LEARNED FROM DEI AND CBD

Workshop BFebruary 6, 2013

Presenters:

George Fouts – Senior Partner for Completion by Design, former Director of Developmental Education InitiativeDebbie Smith – Interim Division Chair, Student Development and LearningAlison Wiers – Associate Vice President, Student Support ServicesClark Wright – Institutional Research Associate for Completion by Design

Welcome to AtD Community College

Math Placement Exam Exercise

•Scrap paper and calculator allowed•Five minutes to complete

Math Placement Question #1•What is the average (arithmetic mean) of 8, 7, 7, 5, 3, 2, and 2?•A. 3•B. 4•C. 4•D. 5•E. 6

Math Placement Question #2•Phillip charged $400 worth of goods on his credit card. On his first bill, he was not charged any interest, and he made a payment of $20. He then charged another $18 worth of goods. On his second bill a month later, he was charged 2% interest on his entire unpaid balance. How much interest was Phillip charged on his second bill?•A. $8.76•B. $7.96•C. $7.60•D. $7.24•E. $6.63

Math Placement Question #3•What is the equation of the line that contains the points with (x,y) coordinates (-3,7) and (5,-1)?•A. y = 3x – 2•B. y = x + 10•C. y = - X + 8•D. y = - X + •E. y = -x + 4

Math Placement Question #4•For i = , if 3i (2 + 5i) = x + 6i then x = ?•A. -15•B. 5•C. 5i•D. 15i•E. 27i

Math Placement Question #5•For positive real numbers x, y, and z, which of the following expressions is equivalent to X1/2 Y2/3 Z5/6?•A. •B. •C. •D. •E.

Overview•Data is used to inform decisions •Major initiatives at Guilford Technical Community College: DEI & CbD•Link DEI lessons learned to CbD efforts•Design intervention at AtD Community College

Using Data to Inform Decisions

Uncover Major Areas with Solvable Problems

•What is the question?•Data analysis•Focus Groups•Engaged faculty, staff, administrators, and students•Synthesized information gained from discussions

Using Data

Engaging the College•Focus groups and forums vital to identify weak areas•For DEI, two areas within developmental education were identified early: 1. Lack of preparation by students for placement test2. New student confusion about developmental

courses and navigating them alongside curriculum college courses

•For CbD, all groups reported similar issues

Using Data

CBD Engagement FeedbackUsing Data

•More info up front and opportunity to opt out of developmental.•Lack of preparation with placement tests leading to lower scores.•Placed in developmental education, making it difficult to access core curriculum.•Lack of follow up advising.

CBD Engagement Feedback

•Online registration process is not explained.•Academic advising to include relationship building with faculty and soft skills.•No advisor assistance.•Ability to track progress.•Students don’t know where and how they are doing (progress alert, early alert).

Using Data

Engagement Process

•BIG things we learned:•Advising needs to continue past orientation.•Students need to continually know how they are progressing.•Remedial courses impact student motivation.

Using Data

Data Analysis

•IR assistance supported areas of concern and analyzed results•Weekly “data team” meetings for DEI and CbD •Discussed data results, issues, and ideas•Highlights were taken to stakeholders

Using Data

Major Initiatives at GTCC

DEI Initiatives

•Specialized Student Orientation, Advising, and Registration (SSOAR)•Modified version of SOAR for students needing 2+

developmental subjects

Major Initiatives

•COMPASS review• Scaled up and improved existing face-to-face review•Created online version•Promoted to students

DEI Results – COMPASS Review

ENG RED MAT-Pre Alg MAT-Alg02468

1012141618

16.1

8.1 7.3

4

15.7

8.8

6.4 6.6

F2F Workshop Online Workshop

Average point increase (scale of 0-99) in test scores during the implementation year when a workshop was taken between first and second attempt.

Major Initiatives

% of Students placing out of one or more classes after taking the COMPASS workshop and retesting.

Fall 2010 Spring 2011 Fall 2011 Spring 20120%

10%

20%

30%

40%

50%

60%

70%

EnglishReadingMath

DEI Results – COMPASS Review

Major Initiatives

Data Analysis

•Face-to-face and online sessions had similar success• F2F scaled down, online scaled up• Students retaking COMPASS test must first complete

review

Major Initiatives

Data Analysis

•Build on success of COMPASS review• Summer bridge program with extended COMPASS

review •Will potentially include study skills and student

orientation information

Major Initiatives

SSOAR – Specialized SOAR

Specialized SOAR (SSOAR) is a specialized student orientation, advising and registration session designed to provide in-depth one-on-one advising sessions to new students who test into two or more developmental areas.

SSOAR Initiative Evaluated using Student Persistence Rate

Major Initiatives

• What is the impact of SSOAR on persistence among developmental education students?

• Is there a significant difference in persistence rates between developmental education students who attended SSOAR and those who did not attend SSOAR?• Is the improvement in persistence rates of

developmental education students who received SSOAR consistent across cohorts?

Research QuestionMajor Initiatives

Spring 2010 Fall 2010 Spring 2011 Fall 20110.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

83.3%

58.3%

50.8%

37.7%

76.9%

48.2%

39.9%

28.2%

Persistence Rates for new Fall 2009 Students Placing into 2+ Developmental Subjects

Attended SSOAR (N=671)Did not attend SSOAR (N=968)

Term

Pers

iste

nce

from

Firs

t Fal

l

• Developmental education students who attended SSOAR have a 1 year persistence rate about 10% higher than those who did not attend SSOAR.

• The 1 year persistence rate for the 2010FA cohort also show a similar trend.

• Beginning 2011FA GTCC instituted a policy that all new students must attend regular SOAR and all developmental education students will have to attend a SSOAR session prior to registration.

Result Implications Major Initiatives

Result Implications

• For the 2011FA cohort the proportion of developmental education students who attended SSOAR increased by 30%.

• The real projected benefits are that at least 270 more developmental education students will persist after the second Fall from the 2011FA cohort.

2009FA 2010FA 2011FA0%

20%

40%

60%

80%

100%

40% 41%

70%

% of 2+ Developmental Subject Students Attending SSOAR

Major Initiatives

DEI Results - SSOAR•Problem collecting SSOAR data•“Checkbox” method not effective•Preventing SSOAR students taking SOAR

•CbD to integrate SSOAR information into redesigned orientation process•Many more required online modules•All students receive information emphasizing

developmental education and placement testing

Major Initiatives

Developmental Placement – Multiple Measures

•Lack of efficacy of student developmental placement•High school GPA may be better placement indicator•Local analyses and literature review resulted in formation of new state policy

Major Initiatives

Local Analysis:Can high school GPA predict success despite developmental reading level?

Level ofReading

Level ofHSGPA Group N

Fall09toSpr11 College GPA

Mean Std DevDid not take Reading 090

HSGPA <2.75 Did not take Reading 090, HSGPA<2.75

61 1.18 1.06

Did not take Reading 090

HSGPA≥2.75 Did not take Reading 090, HSGPA≥2.75

14 2.30 1.33

Took Reading 090

HSGPA<2.75 Took reading 090, HSGPA <2.75

563 1.29 0.91

Took Reading 090

HSGPA≥2.75 Took reading 090, HSGPA ≥2.75

130 2.24 0.90

Major Initiatives

Local Analysis:Does that effect continue with other developmental courses?

Level ofDEVEDS

Level ofHSGPA Group N

Fall09toSpr11 College GPA

Mean Std DevDid not take any req’d Dev-Eds

HSGPA <2.75 Did not take deveds, hsgpa<2.75

253 1.75 0.96

Did not take any req’d Dev-Eds

HSGPA≥2.75 Did not take deveds, hsgpa≥2.75

110 2.54 0.94

Took 1+ req’d Dev-Eds

HSGPA<2.75 Took at least one deved, hsgpa<2.75

459 1.55 0.92

Took 1+ req’d Dev-Eds

HSGPA≥2.75 Took at least one deved, hsgpa≥2.75

159 2.44 0.89

Major Initiatives

CCRC - Severe Error Rate Major Initiatives

CCRC - Next Steps1. Reduce stakes of placement testing2. Focus on those who fail placement tests3. Use better information from other sources:• Student-level high school information• Institutional information: Schools and local education

agencies• ACT• Non-cognitive tests

Major Initiatives

Resulting Policy Change

•NCCCS has developed proposed policy change •Allow placement of students scoring above certain HSGPA

into college courses

•Policy targeted for Fall 2013 semester•Exact numbers still in debate• Last proposal allowed students with HSGPA ≥ 2.6

graduating within the last 10 years

Major Initiatives

Linking the Lessons Learned

Use Previous Knowledge to Drive New Ideas

•SSOAR CbD’s proposed new orientation process•COMPASS Review CbD’s work with policymakers•First-year-experience findings driving new QEP and PRESS for Completion

Lessons Learned

Overlap and Connection

•Overlaping key members•Connections between members•DEI members frequently contacted and shared

knowledge with CbD members• Sharing across administrative staff, team members, and

IR staff

Lessons Learned

Promote Culture of Completion

•Gain buy-in through frequent and open sharing of information•Hold all-personnel meetings •Create larger intervention sub-teams involving individuals across the entire campus community

Lessons Learned

Keep Channels of Communication Open

•Weekly meetings held for various teams•Planning team, implementation team and sub-teams,

and a data team

•GTCC’s CbD directors attend all meetings, members attend multiple meanings

Lessons Learned

Intervention Design

AtD Community College

•You will be assigned to a department at AtDCC:• Administration• Institutional Research• Faculty• Student Services Staff

•Notecards• Knowledge known to your position

•Speak to those around you from different departments to come up with a solution to the problem•Solve big issues using lessons learned•15 minutes then report out

Intervention Design

Summary

•Discussed how data is used to inform decisions •Major initiatives at Guilford Technical Community College: DEI & CbD•Linked DEI lessons learned to CbD efforts•Designed intervention at AtD Community College