STUDENT SUCCESS:SIMPLE QUESTIONS, COMPLEX ANSWERS

Karen RothsteinResearch Analyst I

Andrew FuenmayorResearch Analyst I

Will students be more successful if they take this

course(e.g., Counseling, Study

Skills, College Readiness)?

THE SIMPLE QUESTION…

Students who took Study Skills

Students who did not take Study Skills

70%60%

Following Term Success Rate

THE SIMPLE ANSWER

Students who success-fully

completed Study Skills

Students who did not successfully complete

Study Skills

80%

50%

Following Term Success Rate

THE LESS SIMPLE ANSWER

“Well that depends on several factors, one of which is the course in question. In addition, we need to examine other variables…”

In other words…

THE COMPLEX ANSWER

Student Success: More than meets the eye!

To encourage more complex and complete interpretations of student success Issues of self-selectionComparison group

To include student-level variablesStudent characteristics that could influence success

Two applicationsSpecific target course vs. comparison courseStudent support services

PURPOSE

TARGET COURSE VS. COMPARISON

COURSE

Target course Course of interest “TC” in syntax

Comparison course How do you select comparison course? “CC” in syntax

Time frame Proximal effect of target course Distal effect of target course

Uses fi rst-time studentsUses fi rst attempt at target course and comparison

courseSuccess = A, B, C, P, CR, IPNo success = D, F, W, NP, NC, IF

BEFORE YOU START

Create a long MIS data file

Create a first-time student (FTS) fileSB15 = 1See “Creates first-time student file” syntax

Merge long MIS and FTS fileSee “Creates data file for analysis” syntax

CREATE DATA FILE

STEP I: Cleans data fileData earlier than 2000 will need a different term variable

Change target course (line 61)Change comparison course (line 63)

STEP 2: Creates 3 cohortsTarget course firstComparison course firstBoth target and comparison course in same semester

Change target course (lines 19, 21, 24, 26)Change comparison course (lines 20, 22, 25, 27)

NOW FOR THE FUN PART!

STEP 3: Identifies termsConcurrentFollowingSubsequent

STEP 4: Creates variables to calculate success rates ConcurrentFollowingSubsequent

STEP 5: Generates success rate outputChange destination for output as Excel file (line 208)

NOW FOR THE FUN PART!

DOES TAKING THE TARGET COURSE LEAD TO SUCCESS?

Enrolled in target course

Did not enroll in target course

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

60.2% 61.5%

Success Rate in Term Following Target Course

MAYBE IT’S SUCCEEDING IN THE TARGET COURSE?

Successful in target course

Unsuccessful in target course

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

66.1%

41.2%

Success Rate in Term Following Target Course

MAYBE IT’S MORE COMPLICATED…

0.0%30.0%60.0%90.0% 65.7%

40.3%

71.8%46.4%

Success Rate in Term Following Target Course

Move from success rates to regressionUse high school data to include prior

success ratesWhat else can we examine?

GenderEthnicityAgeEducation PlanConcurrent GPA (excluding target and comparison courses)

Other student-level variables?

GET CRAZY WITH THE CHEEZ WHIZ!

STUDENT SUPPORT SERVICES

Will students be more successful

if they use this support service

(e.g., tutoring, workshops, counseling appointments,

review sessions, etc.)?

AGAIN, THE SIMPLE QUESTION

DOES THE SUPPORT SERVICE LEAD TO SUCCESS?

Participated in support service

Did not participate in

support service

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

71.4%

28.0%

Success Rate in Target Course

Possible independent variablesParticipation vs. no participationFulfill required hours vs. failure to fulfill required hours

Total number of hours of support serviceType of support servicePrior exposure to support servicePrior success ratePrior number of units

Dependent variableSuccess in a specific course (tied to the student support service)

Regression

THE COMPLEX ANSWER

EFFECT OF STUDENT SUPPORT SERVICE ON COURSE SUCCESS

 Variable BWald Stat. Sig.

Exp(B)

Standardized   Student Success Rate

1.698 18221.96 .000 5.46

Course with student support services -.922 542.66 .000 .398Number of Hours spent in support services .636 1489.53 .000 1.889(Number of Hours spent in support services )2 (to capture non-linear effects) -.035 333.05 .000 .965

NOW WHAT?

Carefully and thoughtfully explain why simple answers are inadequate

Highlight need to consider variables other than the course in and of itself

Present data in a user-friendly way

IMPROVING DIGESTION OF DATA

INSTEAD OF THIS…

 Variable BWald Stat. Sig.

Exp(B)

Standardized   Student Success Rate

1.698 18221.96 .000 5.46

Course with student support services -.922 542.66 .000 .398Number of Hours spent in support services .636 1489.53 .000 1.889(Number of Hours spent in support services )2 (to capture non-linear effects) -.035 333.05 .000 .965

TRY THIS…

0%10%20%30%40%50%60%70%80%90%

100%

4 hours3 hours2 hours1 hours0 hours

Students’ Prior Success Rate

Co

urs

e S

ucc

ess

Ra

te

“Why aren’t you doing it the same way as last time?”

“We need to look at the right subset of students.”

“There’s more than just numbers.”

“So what you are saying is that what I do doesn’t matter.”

BE PREPARED FOR QUESTIONS LIKE…

The reason these interventions and courses exist is to help students succeed

If the research suggests this is not the case, how can we make the intervention more effective?Where has it worked? Why?

WHEN IT COMES DOWN TO IT…

Please feel free to contact me to discuss.

Suggestions for improving syntax always welcome!

Karen Rothstein: krothstein@lbcc.eduAndrew Fuenmayor: afuenmayor@lbcc.edu

THANKS FOR LISTENING!