first-year seminars and living-learning communities: are ... · first-year seminars and...

23
First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton Missouri State University Fall 2013

Upload: others

Post on 24-Aug-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment?

Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

Fall 2013

Page 2: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

2    

TABLE OF CONTENTS

Purpose of Study…………………………………………………………………….. 1

Description of Departmental Unit(s) and Services………………………………… 2

Description of Programs Being Assessed…………………………………………… 3

Operationalization of Variables…………………………………………………….. 4

Study Design…………………………………………………………………………. 4

Population and Sampling Strategy…………………………………………………. 5

Data Collection Instrument…………………………………………………………. 5

Detailed Presentation of Data Collection Activities……………………………….. 6

Analysis of Data……………………………………………………………………… 7

Results of Data Analysis…………………………………………………………….. 8-18

Discussion of Results and Implications…………………………………………….. 19

Action Plan for Program Changes and/or Recommendations……………………. 19

Limitations and Challenges…………………………………………………………. 20

Logistics and Tips for Future Researchers…………………………………………. 20

Appendix A: General Timeline

Appendix B: Executive Summary and Supplemental Material

Disclaimer: Further analysis of the Living-Learning Community Programs, First-Year Programs, and data findings may be required to refine the enclosed documentation and

recommendations. Please consult with the listed researchers with any questions, comments, or suggestions. Thank You!

Page 3: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

3    

Purpose of Study

The purpose of this study was to conduct an assessment to determine if differences existed in the academic success and fall-to-fall persistence rates in first-time-in-college (FTIC) students, who participated in a Living-Learning Community (LLC) in the academic year 2012-2013, at Missouri State University (MSU). The two LLCs that were assessed included: (1) an LLC with a connected GEP 101(First-Year Seminar) course designated for students who are in the process of deciding their major, as compared to (2) a stand-alone First-Year Experience (FYE) LLC where students do not have a designated GEP class. ***Note: The key that was used for the two types of LLCs throughout this assessment documentation included the following:

1. Deciding Living-Learning Community à LLC+GEP 2. First-Year Experience Living-Learning Community à FY-LLC

Assessment Approach: Two dimensions of assessment were used to conduct this study, and determined potential developments or changes that should be considered based on the findings.

1. The tracking dimension looked at how many and what types of students participated in both LLCs. To examine the profile of these students, pre-existing inputs including high school GPA, class rank, ACT composite scores, gender, and race/ethnicity were examined. These inputs determined if there were pre-existing differences between the students in the LLC+GEP versus the FY-LLC.

2. The program outcome dimension illustrated what the program sought to accomplish by looking at students’ MSU GPA, credit hours completed, and fall 2012 to fall 2013 retention in the LLC+GEP versus the FY-LLC.

Research Questions:

The following research questions served as a guide for the researcher to frame and develop this assessment project. Questions one and two correlated with the tracking dimension, while question three and four were linked to the program outcome dimension of assessment.

1. Are there pre-existing differences in academic ability (as measured by ACT, high school GPA, and class rank) in students who participate in the living-learning community (LLC) with a GEP 101 component (Deciding Students) as compared to LLC students without a designated GEP 101 class (i.e. First-Year LLC)?

2. Are there pre-existing differences in demographic factors (e.g. gender, race/ethnicity, age, etc.)?

Page 4: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

4    

3. Are there differences in the academic success (as defined by GPA and credit hours)

completed in the first year in students who participate in the LLC with GEP 101 component (Deciding Students LLC) as compared to LLC students without a designated GEP 101 class (First-Year LLC)?

4. Are there differences in the first to second year persistence rates (measured fall-to-fall)

for Deciding Students LLC+GEP compared to FY-LLC? References: (Dr. Belinda McCarthy fall 2013 SAE 767 syllabus, Missouri State University) (Bresciani, M., Zelna, C and Anderson, J. (2004). Assessing student learning and development: a handbook for practitioners. Washington, D. C.: NASPA.)

Description of Departmental Unit(s) and Services

Collaborative efforts between two units are the driving force for the implementation, development, and (hopeful) success of the LLC+GEP and FY-LLC. These units are Student Development and Public Affairs, and Residence Life, Housing, and Dining Services. Each provided a context and framework for the assessment study.

Student Development and Public Affairs:

This unit focuses on students’ academic success in addition to measuring and evaluating student learning through continuous assessment projects each year. It is also responsible for supporting the MSU unique Public Affairs Mission by developing opportunities to educate students on the mission’s three pillars: ethical leadership, community engagement, and cultural competence.

The First-Year Program department is housed in this unit, and is in charge of assisting new students in achieving a successful transition to MSU, both academically and socially. The coordination, operation, and implementation of the First-Year Seminar course (GEP101) is the core focus in this department. This course, GEP 101, is a required foundational course that fulfills five general learning goals for the general education program at MSU. These five general goals include the areas of Public Affairs, information literacy, and collaboration. Additionally, the department provides a common academic and intellectual experience, and creates a greater sense of community among students, faculty, and staff by the implementation of a Common Reader; a book that is selected by a committee each year that all freshmen are required to read.

References: (http://www.missouristate.edu/sdpa/) (http://www.missouristate.edu/FirstYearPrograms/66155.htm)

Page 5: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

5    

Residence Life, Housing, and Dining Services:

This unit is responsible for the management, maintenance, and upkeep of living and dining facilities for students, in addition to implementing extra and co-curricular programs for residents. The safety and disciplinary oversight for student residents is also a priority for this unit. Providing support, services, and activities (educational and social) are key components for this unit.

The Living-Learning Communities department is housed in this unit, and is designed to ease students’ transition to college; familiarize students with the campus and resources; provide faculty and staff contact; promote active learning; and assist students in connecting with other students. A plethora of incentives are provided to students who choose to participate in this optional program. Those incentives include: attending programs related to academic interest; social and educational activities; priority move-in date; interact with faculty fellows; and personal growth and success.

References: (http://reslife.missouristate.edu/livinglearning/LLCAbout.htm) (http://reslife.missouristate.edu/) (Missouri State University Living-Learning Communities 2013-2015 brochure)

Description of Programs Being Assessed

Below is a description, taking from the 2013-2015 Living-Learning Community brochure, of each LLC that will be assessed in this study.

Deciding Student LLC (LLC+GEP):

The ideal community for first-year students who are exploring career options or are considering majors at the University. This LLC includes components of the First-Year Experience LLC along with guided activities in selecting a major and career, and take GEP101 class (First-Year Foundations Seminar) taught by academic advisors. Students also receive guidance on exploring and selecting leadership opportunities and other options available at Missouri State University that enhance their student experience and career plan. There are 2 floors (one female and one male) located in Woods House.

First-Year Experience LLC (FY-LLC):

Designed to enhance each student’s first year of college and help ease the transition into college life. Students on these floors receive numerous opportunities to interact with other students and to get involved on campus. In addition, students interact with faculty members in a relaxed setting outside the classroom. There are three first-year floors (two female and one male) located in Freddy and Wells House.

Reference: (Missouri State University Living-Learning Communities 2013-2015 brochure)

Page 6: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

6    

Operationalization of Variables

Below are the variables that were used in the assessment study. No instrument was used to collect the data, but IBM SPSS Statistics version 20 software was used to analyze and interpret the findings.

Variables in the LLC+GEP and FY-LLC were measured using the assessment dimensions, tracking and program outcome.

• Pre-existing Variable (before college entrance): 1. High School GPA 2. High School Rank 3. ACT Composite Score 4. Gender 5. Race/Ethnicity

• Outcome Variables (after college entrance and post fall 2012/pre fall 2013):

1. MSU GPA 2. Credit Hours Completed 3. Fall 2012 to Fall 2013 Retention

Study Design

The study was quantitative in nature, and was quasi-experimental (i.e. before-and-after) control group design. In this design, the experimental group A and control group B were selected without random assignment. Only the experimental group received the treatment. This design was chosen, because it could measure change and group differences between the LLC+GEP and FY-LLC. In addition, this design was appropriate, because the researcher sought to determine and measure the effectiveness of both LLCs. The historic data collected was only retrieved once; however, it included pre and post variables of data for each participant; that is, pre-existing inputs of participants prior to entering MSU, and participant outcomes after their first completed academic year.

The reference period or time frame in which the study explored the aforementioned phenomenon was retrospective, in that it examined historical data that was already gathered and available. This data was provided by two sources, the assistant director in the Assessment Office and the Office of Institutional Research.

Lastly, the overall assessment sought to determine if the program goals for both LLCs positively or negatively impacted the participants’ academic progress (post-outcomes).

Reference: (Kumar, Ranjit. (2011). Research methodology: A step-by-step guide for beginners. Washington, DC. Sage Publications.) (Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage.)

Page 7: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

7    

Population and Sampling Strategy

The study population was identified by the coordinator of the LLCs and by the director of First-Year Programs using student M-numbers. This population was comprised of FTIC students who voluntarily applied and were selected to participate in LLCs in the fall of 2012. The targeted sample size from the study population was filtered to only include those students who participated in the LLC+GEP or FY-LLC. All participants (i.e.154 students) were included in the study with 66 students being participants of the LLC+GPE, and 88 students participating in FY-LLC. See chart below for the sample population descriptive breakdown.

Living-Learning Communities Type Breakdown

Frequency Percent Valid Percent Cumulative Percent

Valid FY-LLC 88 57.1 57.1 57.1 LLC+GEP 66 42.9 42.9 100.0 Total 154 100.0 100.0

The sampling strategy involved the director of First-Year Programs requesting pre-existing (historical) data on student participants in both LLCs from the Office of Institutional Research (OIR). All students who were in the LLC+GEP and FY-LLC in fall of 2012 were included in the data set.

References: (http://www.missouristate.edu/oir/) (http://www.missouristate.edu/rstats/)

Data Collection Instrument

There was no data-gathering tool (e.g. survey) constructed for this study; however, institutional records were used. The director of First-Year Programs obtained student M-numbers from the coordinator of the LLCs, and submitted the student protected identification to the Office of Institutional Research (OIR). This method was used in order to receive historic data (i.e. aforementioned variables) on each student participant in order to compile in a single file for a detailed review and analysis of data. This was necessary in order to enable researchers to correlate institutional data for the LLC+GEP and FY-LLC. Only M-numbers were used for this assessment project. Student names were not used or included in the study at any time.

Page 8: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

8    

Additionally, information interviews were conducted with the director and assistant director of First-Year Program, and coordinator of the Living-Learning Communities. These interviews were not to collect evidence or data for the assessment project specifically; rather, they were conducted to give the student researcher information about the mission, vision, purpose, and goals for the LLC+GEP and FY-LLC communities. This background information created a framework for the student research, because the researcher had not worked with LLCs prior to the beginning of the study.

References: (http://www.missouristate.edu/oir/)

Detailed Presentation of Data Collection Activities

In order to collect the historical data needed for this study, the researchers completed the following steps. These are suggested steps that should be taken in the future in order to replicate this study or similar study. See Appendix A for a general timeline of the process that was used.

Step 1: The graduate student researcher and First-Year Program director had a consultation with the assistant director in the MSU Assessment office to determine the following:

• Solidify the language of study’s research questions for proper assessment to be completed • Determine what variables needed to be obtained in order to answer the research questions • Determine steps needed to be taken to get IRB approval and begin data collection • Determine how data would be assessed, analyzed, and reported

Step 2: The First-Year Program director obtained M-numbers for each student who took GEP with the LLC component (i.e. those LLC+GEP integrated courses taught by advisors in the Academic Advisement Center).

Step 3: The First-Year Programs director emailed the LLC coordinator to request a list be sent of all residents who lived on the LLC+GEP floors, and who were enrolled in the course.

Step 4: The two lists were compared to ensure that the names and M-numbers matched.

Step 5: The graduate student researcher completed an IRB application and submitted a draft to First-Year Program director. Note: This step begun after step 1, along with director completing steps 2-4.

Step 6: The First-Year Program director submitted the list of M-numbers to the Office of Institutional Research and requested a data set for the LLC+GEP and FY-LLC variables (i.e. high school GPA and rank, ACT composite score, gender, race/ethnicity, MSU GPA, credit hours complete, and fall 2012-fall 2013 retention)

Step 7: Once the Office of Institutional Research emailed the historical data set in an excel document, the graduate student researcher scheduled an appointment with the RStats (Research,

Page 9: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

9    

Statistical Training, Analysis, and Technical Support) Institute. This meeting provided the researchers assistance with analyzing and interpreting the data. The RStats representative also provided assistance in building t-test and charts to compare the two LLCs to answer the research questions.

The timeframe for all data collection activities was approximately two months. It should be noted, however, that in the two-month window expedited requests were made. In future studies, researchers should allow more time for the assessment project’s data collection process.

References: (http://www.missouristate.edu/oir/) (http://orc.missouristate.edu/84658.htm) (http://www.missouristate.edu/rstats/)

Analysis of Data

The software utilized to analyze the data was IBM SPSS Statistics, version 20. A copy of this program was obtained from MSU’s Cheek Hall computer lab, and downloaded/saved on the researchers’ personal computer.

With the excel file containing the data from the Office of Institutional Research, the researchers visited the RStats Institute to get assistance with analyzing and interpreting the data; two appointments were needed to complete this task (i.e. one for analyzing and one for interpreting). The RStats graduate student walked the researchers through the following:

• How to arrange the data into two categories: descriptive (e.g. gender and race/ethnicity) and continuous/numerical (e.g. GPA, credit hours, etc.). This process was necessary to ensure information was aligned properly when test were ran.

• How to rename categories in the excel data sets, so that the software would recognize columns when running the analysis.

• How to code variables with a number. For example, each race/ethnicity was assigned a number (i.e. Caucasian/White = 1, African American/Black = 2, etc. OR… female = 2 and male = 1).

• How to set-up a variety of t-tests in order to compare the LLC+GEP to the FY-LLC to answer the research questions. Here, crosstabs and chi-square test were performed on nominal variables to create the t-tests or independent sample test, Levene’s test of Equality of Error Variances, tests between subjects effects, estimated marginal means, Pairwise Comparisons, Univariate Analysis of Variance, group and descriptive statistics, frequency tables, graphs, etc.

o All of these tests were run in order to analyze the data and answer the research questions. Additionally, they were run in order to check the validity and reliability of the data.

• How to create bar graphs that would display the data via illustration.

Page 10: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

10    

***Note: Future researchers should be aware that when you go to receive a copy of the SPSS software, they should bring a blank DVD with them in exchange for the copy. Additionally, it is highly suggested and encouraged to seek assistance from the RStats Institute if researchers have no prior knowledge on how to analyze or interpret data or the use of SPSS.

References: (http://www.missouristate.edu/oir/) (http://www.missouristate.edu/rstats/)

Results of Data Analysis

Below is each research question proposed in this assessment study, along with the results of the data analysis and the researchers interpretation of the data.

Research Question 1:

Are there pre-existing differences in academic ability (as measured by ACT, high school GPA, and class rank) in students who participate in the living-learning community (LLC) with a GEP 101 component (i.e. LLC+GEP) as compared to LLC students without a designated GEP 101 class (i.e. FY-LLC)?

Pre-Existing Academic Inputs for Both LLC+GEP and FY-LLC LLC_Type N Mean Std. Deviation Std. Error Mean

HS_RANK FY-LLC 76 75.1842 78.95580 9.05685 LLC+GEP 52 114.1923 121.54932 16.85586

HS_PERCENTILE FY-LLC 76 73.8421 19.31704 2.21582 LLC+GEP 53 60.5849 21.85128 3.00150

ACT_SCORE FY-LLC 82 24.16 3.564 .394 LLC+GEP 62 22.48 3.243 .412

HS_GPA FY-LLC 86 3.59 .474 .051 LLC+GEP 61 3.31 .492 .063

Research Question 1 Interpretation: Based on the results, the FY-LLC participants’ pre-existing academic inputs were higher than the LLC+GEP participants.

Page 11: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

11    

Research Question 2:

Are there pre-existing differences in demographic factors (e.g. gender, race/ethnicity, age, etc.)?

Race/Ethnicity for Both LLC+GEP and FY-LLC Frequency Percent Valid Percent Cumulative

Percent

Valid

White 120 77.9 77.9 77.9 Black 16 10.4 10.4 88.3 Hispanic 7 4.5 4.5 92.9 Asian 1 .6 .6 93.5 More than one 6 3.9 3.9 97.4 Non-resident 3 1.9 1.9 99.4 Unknown 1 .6 .6 100.0 Total 154 100.0 100.0

Race/Ethnicity for FY-LLC Frequency Percent Valid Percent Cumulative

Percent

Valid

White 69 78.4 78.4 78.4 Black 9 10.2 10.2 88.6 Hispanic 5 5.7 5.7 94.3 Asian 1 1.1 1.1 95.5 More than one 2 2.3 2.3 97.7 Non-resident 1 1.1 1.1 98.9 Unknown 1 1.1 1.1 100.0 Total 88 100.0 100.0

Page 12: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

12    

Race/Ethnicity for LLC+GEP Frequency Percent Valid Percent Cumulative

Percent

Valid

White 51 77.3 77.3 77.3 Black 7 10.6 10.6 87.9 Hispanic 2 3.0 3.0 90.9 More than one 4 6.1 6.1 97.0 Non-resident 2 3.0 3.0 100.0 Total 66 100.0 100.0

Research Question 2 Interpretation: Race/Ethnicity Based on the results, in the LLC+GEP and FY-LLC there are more White/Caucasian participants than other race and ethnicities.

Gender for Both LLC+GEP and FY-LLC Frequency Percent Valid Percent Cumulative

Percent

Valid Male 55 35.7 35.7 35.7 Female 99 64.3 64.3 100.0 Total 154 100.0 100.0

Gender for FY-LLC Frequency Percent Valid Percent Cumulative

Percent

Valid Male 28 31.8 31.8 31.8 Female 60 68.2 68.2 100.0 Total 88 100.0 100.0

Page 13: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

13    

Gender for LLC+GEP Frequency Percent Valid Percent Cumulative

Percent

Valid Male 27 40.9 40.9 40.9 Female 39 59.1 59.1 100.0 Total 66 100.0 100.0

Research Question 2 Interpretation: Gender Based on the results, in the LLC+GEP and FY-LLC there are more female participants than male.

Current Age for FY-LLC Frequency Percent Valid Percent Cumulative

Percent

Valid

18 70 79.5 79.5 79.5 19 12 13.6 13.6 93.2 20 3 3.4 3.4 96.6 21 2 2.3 2.3 98.9 35 1 1.1 1.1 100.0 Total 88 100.0 100.0

Current Age for LLC+GEP Frequency Percent Valid Percent Cumulative

Percent

Valid

18 46 69.7 69.7 69.7 19 16 24.2 24.2 93.9 20 1 1.5 1.5 95.5 21 3 4.5 4.5 100.0 Total 66 100.0 100.0

Research Question 2 Interpretation: Age

Based on the results, in the LLC+GEP and FY-LLC most participants are traditional college-age students with one outlier. Note: Because numerical data is continuous, previous ages when students entered MSU was not included.

Page 14: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

14    

Research Question 3: Are there differences in the academic success (as defined by GPA and credit hours) completed in the first year in students who participate in the LLC with GEP 101 component (Deciding Students LLC) as compared to LLC students without a designated GEP 101 class (First-Year LLC)?

MSU GPA and Credit Hours Complete for Both LLC+GEP and FY-LLC

LLC_Type N Mean Std. Deviation Std. Error Mean

MSU_GPA_SP13 FY-LLC 88 3.07471431830 .735618790555 .078417226555

LLC+GEP 66 2.47782643017 .996007167501 .122600007274 MSU_CREDITS_PASSED_SP13

FY-LLC 88 30.76 12.507 1.333 LLC+GEP 66 25.53 18.169 2.236

Research Question 3 Interpretation:

Based on the results, participants in the FY-LLC have higher MSU GPAs and completed more credit hours than those participants in the LLC+GEP. An illustration of this data is below.

Page 15: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

15    

Page 16: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

16    

Page 17: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

17    

Page 18: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

18    

Page 19: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

19    

Research Question 4: Are there differences in the first to second year persistence rates (measured fall-to-fall) for Deciding Students LLC+GEP compared to FY-LLC?

Retention for Both LLC+GEP and FY-LLC Retention Total

No Yes

LLC_Type

FY-LLC

Count 8 80 88 Expected Count 18.9 69.1 88.0 % within LLC_Type 9.1% 90.9% 100.0% % within Retention 24.2% 66.1% 57.1%

LLC+GEP

Count 25 41 66 Expected Count 14.1 51.9 66.0 % within LLC_Type 37.9% 62.1% 100.0% % within Retention 75.8% 33.9% 42.9%

Total

Count 33 121 154 Expected Count 33.0 121.0 154.0 % within LLC_Type 21.4% 78.6% 100.0% % within Retention 100.0% 100.0% 100.0%

Research Question 4 Interpretation:

Based on the results, participants in the FY-LLC had higher persistence rates from fall 2012 to fall 2013 than those participants in the LLC+GEP. A breakdown of each LLC and an illustration of this data are below.

Retention for FY-LLC Frequency Percent Valid Percent Cumulative

Percent

Valid No 8 9.1 9.1 9.1 Yes 80 90.9 90.9 100.0 Total 88 100.0 100.0

Page 20: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

20    

Retention for LLC+GEP Frequency Percent Valid Percent Cumulative

Percent

Valid No 25 37.9 37.9 37.9 Yes 41 62.1 62.1 100.0 Total 66 100.0 100.0

Page 21: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

21    

Discussion of Results and Implications

Based on the data analysis and interpretation, some general conclusion can be determined that student participants in the FY-LLC entered MSU with higher academic inputs than those who were in the LLC+GEP. Additionally, after completing a full academic year at MSU, those participants in the FY-LLC performed better academically than those in the LLC+GEP. Thus, it is not surprising to see these students completing more credit hours with higher persistence rates. However, even after controlling for group differences with the univariate analysis significant difference between the groups was shown. Because the LLC+GEP has an additional academic component to their program, the data suggests that including all deciding students in a cohort is unlikely to positively the impact or students’ performance, unlike the FY-LLC with no GEP component.

As a result, the student researcher encourages program coordinators for both the FY-LLC and LLC+GEP to consider conducting a program improvement assessment to create outcomes that will provide substantial (positive) changes in participants in the LLC+GEP. Or, perhaps the coordinators could consider completing an academic year without the GEP component tied to a LLC to see if results are the same or worse in student participants.

Action Plan for Program Changes and/or Recommendations

Several steps may be taken at this point to improve the LLC+GEP in order to have similar results as the FY-LLC; however, the student researcher offers the following recommendations:

• Administrators/decision makers should consider feedback from program coordinators who are in charge of each LLC, First-Year Programs, as well as other stakeholders before automatically continuing programs if group differences are minimal.

• Create a committee to engage in conversation to determine how much time, resources, and collaboration is needed to organize both types of LLCs. Determine if this time, resources, and collaboration can be utilize in other areas of the department, or for improvements of the LLC+GEP. If the latter is chosen, consider working on improvements without running the program for an academic year to provide a baseline group for comparison purposes.

• Consider conducting a qualitative assessment with previous participants to determine their level of satisfaction and whether or not having the GEP component helped or hindered their academic performance. While the researcher understands satisfaction is only one measure, additional information may be gained.

• Consider a more thorough assessment that is conducted yearly that will include findings for program improvement purposes.

• Consider that there may be no advantage to adding a GEP component based upon the analysis and the resources that involved for coordinating this LLC.

Page 22: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

22    

Limitations and Challenges

Having only a semester to conduct an entire assessment project was the primary barrier and limitation of this study; however, it can be done. The student researcher relied heavily on the administrative support (i.e. director and assistant director of First-Year Programs) for resources and access to data and program information. The administrative support equally had an interest in the potential findings; due to the fact the results would determine possible future program suggestions and resource allocations.

Another challenge that the student researcher faced was personnel changes in the Office of Institutional Research. Because both the director and assistant director were in transition, it was difficult to receive the data needed for the assessment project. Again, the researcher turned to administrative support in order to lighten this barrier. With the resources and access of the director of First-Year Programs, the data was received in enough time to be analyzed and interpreted with the help of the RStats Institute.

Finally, the student researcher’s personal barrier was having no prior knowledge of SPSS and how to utilize this software system. The RStats Institute provided assistance in walking the researcher through each step to analyze the data and determine findings; however, replicating what was done after leaving the help session was difficult. The researchers recorded the session, but duplicating what was previously learned and done was challenging.

Logistics and Tips for Future Researchers

1. It was important to have a strong administrative support staff to assist the student researcher in this project. These individuals were equally invested in the assessment project, and their buy-in made it easy to receive access to data, resources, and a clear understanding of expectations. For future researchers, it is encouraged to invest in a study where administration fully support the project and will provide necessary resources. It helps ensure student researchers are asking appropriate questions with the greatest potential for positive impact and change.

2. Because of the time constraints a semester provides, it is important to create a timeline of necessary steps early on in the project. This timeline should be flexible and have the ability to adapt to unforeseen changes. An example of the timeline used for the study can be found in Appendix A.

3. It is important to have regular meetings with administrative support so that all researchers are on the same page at all times. If possible, whenever there is a meeting with parties involved in the research process, all team members should attend. Additionally, seek permission to record meetings so that the conversation can be a resource later. This suggestion is especially important when meeting with RStats. There is a lot of information shared in a short amount of time, and if the researcher has no prior knowledge of how to use statistical software, it will be difficult to replicate with no recordings.

Page 23: First-Year Seminars and Living-Learning Communities: Are ... · First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen,

First-Year Seminars and Living-Learning Communities: Are They Worth the Investment? Dr. Mike Wood, Tracey Glaessgen, and Robin Hamilton

Missouri State University

23    

4. Unforeseen personnel changes can occur in the mist of completing an expedited (semester long) assessment project that limits the likelihood of gathering necessary data. It is highly encouraged to have an additional plan to obtain data. For example, this plan may include looking up each variable on each student one by one. If this situation occurs, consider only assessing a portion of the sample of the population instead of the entire sample. It will only give the researcher a rough idea of the findings that will suffice the projects requirements; however, the assessment process experience will still be there.

5. Conducting assessment will be long, tiresome, difficult, and at times frustrating, so choose an interesting topic that makes a difference in an office/department.