engagement and retention: assessment of a general education initiative for first-year students

Engagement and Retention: Assessment of a General Education Initiative for First-Year Students


Frederick S. Foster-Clark, Daniel F. O’Neill, Laurie Hanich, and Carol Y. Phillips

Millersville University of Pennsylvania

Presented at AAC&U’s Network for Academic Renewal conference, General Education and Assessment: Engaging Critical Questions, Fostering Critical

Learning, Miami, FL, March 1, 2007

Dr. Fred Foster-Clark General Education Coordinator Dr. Linda McDowell First Year Experiences Coordinator Dr. Daniel O’Neill Counseling and Human

DevelopmentDr. Laurie Hanich Educational FoundationsDr. Thomas Burns Associate Provost for Academic

AdministrationDr. Carol Phillips Associate Provost EmeritaMs. Kris Vitucci Graduate Assistant

FYE Assessment Team



• Undergrad enrollment - 7200– 91% fulltime – 14% minority– 96% instate (Pennsylvania)

• Entering fall 2006 class of 1363 students (new freshman) –

– Mean SATs = 1050; – Mean Percentile Rank = 69%

• 12th Ranked Public in US News & World Report’s Master’s Universities in the North

• Top Majors: – Business Administration (892)– Elementary Education (808)– Undecided (785)– Biology (503)– Psychology (449)– Industry & Technology (444)– Communications (438)

Profile of Millersville University


• Started FYE in Fall 2001: Outgrowth of the Student Alcohol Abuse Task Force recommendations

• Develop potential models for a holistic first-year program to encourage: Social engagement Civic engagement Intellectual engagement

• Components to be included: Enhanced advisement Seminar course, the integrating element Living/learning community with related programming

The Charge


Components

• All freshman housing

• Special programming (WOW*)

• Peer mentors

• Team building experiences

• Community building

• Tutoring on-site

*”What’s on Wednesday”

Outcomes

• Students make friends, congregate in groups

• Students form study groups with classmates

• Students like the residence hall experience

• Students involved in campus life

Collaboration with Student Affairs/Resident Life


UNIV 101• 1-credit extended

orientation Seminar with linked fundamentals course (i.e., ENGL 110 or COMM 100)

• Socratic format• Problem-based learning• Co-curricular/

extracurricular assignments• Service-learning• Faculty as advisor

UNIV 179• 3-credit, content-rich

passion Seminar linked with fundamentals course

• General education credit• Co-curricular/

extracurricular assignments• Service-learning• Faculty may or may not

serve as advisor

Seminar/Learning Community Models


– FY Perspectives course paired with first-year fundamental course (Composition or Speech)

– Living-learning connection (Students live together in Freshman residence halls)

– Service Learning (15 hours recommended)

– Attentive advising by seminar instructors with support from Resident Life and Exploratory programs

– Peer mentors (live in residence halls; one assigned to each seminar)

Components of the First-Year Perspectives


First Year Perspectives (FYP) is a component of General Education specifically designed for first semester students and offered in a seminar format, typically linked to a foundations course (either ENGL 110 or COMM 100) as a part of a living/learning community. A major function of these seminars is to introduce a process of critical inquiry applied to important social, cultural, scientific, technological, and/or aesthetic problems. Each FYP course will introduce multiple perspectives related to the understanding and resolution of these problems.

First Year Perspectives (UNIV 179)


Common learning objectives:1. Communicating orally and in writing2. Obtaining and evaluating information

(information literacy)3. Engaging in critical inquiry4. Appreciating the importance of civic engagement5. Understanding the importance of a liberal arts

education6. Making a successful transition into university

life, both academically and socially

First Year Perspectives (UNIV 179)


• Dream of America• Why We Hate• Facing Fear• Homes and Homelessness• The Amish and the Media • Witchcraft in 17th Century England and New England • Our Bodies/Ourselves: Sexuality and Gender in the Global Village • A Different View: How Can We Change the World? • Scientific Revolutions: An Exploration of Method • Culture, Science and Mathematics in the Pre-Columbian Americas • Food or Free Speech? Human Rights and You • The Monsters Under Our Beds • Liberty and Justice for All: The Promise of American Education • The Deindustrialization of America: Jobs Today, Gone Tomorrow

First-Year Perspectives Topics (Fall 2006)


• 110 of the 284 Exploratory (Undecided) students were randomly assigned to the pilot-test program

• Pilot-test students assigned to one of five learning community topics based on their stated preferences

• Students completed mid-semester and end-of-semester Web-based surveys

• Focus group of pilot-test students held at end of semester by assessment staff

• Instructors completed opened-ended surveys

• Students followed to check academic progress and persistence

Fall 2005 Pilot-test Evaluation Methods


Students in the freshman seminar had more serious conversations with different students, worked more with classmates outside of class, researched for a paper more, contributed more to class, and came to class more prepared than students who were not in a freshman seminar.

Freshman Year Mid-Term Survey – Fall 2005 Results

Students who were involved in a freshman seminar . . .

Means (all differences p < .05)[Based on scale of 1 (Never) to 4 (Very Often)]

Had more serious conversations with students who were very different in terms of their religious beliefs, political opinions, or personal values than students who were not involved in a freshman seminar.

Seminar = 2.28No Seminar= 2.02

Worked more with classmates outside of class to prepare class assignments than students who were not involved in a freshman seminar.


Worked more on a paper or project that required researching ideas or information from various sources than students who were not involved in a freshman seminar.


Asked more questions in class and contributed to class discussions than students who were not involved in a freshman seminar.


Came to class without completing readings or assignments less than students who were not involved in a freshman seminar.



• Pretest-Posttest Survey – Higher Education Values Inventory (HEVI) – Openness to Diversity & Challenge

• Pretest-Posttest Open-Ended Responses– Civic Responsibility– Liberal Arts

• Information Literacy Assessment (forthcoming)• NSSE (End of spring term)• End-of-Semester Focus Groups (students)• Faculty Survey• Persistence/Retention

Fall 2006 Assessment Components


Comparison of Pre-Test/Post-Test Means (Fall 2006)

Note: N’s ranged from 212 to 215; standard deviations in parentheses.Ranges:

HEVI Subscales: (0 = strongly disagree to 4 = strongly agree)Openness to Diversity/Challenge Scale: (1 = strongly disagree to 5 = strongly agree)

Scale Pre-Test Post-Test p value

HEVI: Family Expectations1.98 (.79)

1.98(.86)

ns

HEVI: Scholastic Focus2.03(.74)

1.84(.79)

<.001

HEVI: Achievement Value2.39(.66)

2.37 (.65)

ns

HEVI: General Education Value2.45 (.59)

2.41 (.71)

ns

HEVI: Achievement Obstacles1.35 (.64)

1.49 (.78)

.002

Openness to Diversity/Challenge 3.80(.56)

3.94(.68)

.002


Writing Prompts (Fall 2006)

Civic Responsibility– What is civic responsibility? [pre- & post-]

– How has your understanding of civic responsibility changed as a result of your experiences this semester? What in particular had an impact? [posttest only]

Liberal Arts– What is a liberal arts education? [pre- & post-]

– How has your understanding of a liberal arts education changed as a result of your experiences this semester? What in particular had an impact? [posttest only]


Responses to Liberal Arts Writing Prompt (Pretest -- Fall 2006)

Off Target (1) Middle category (2) On Target (3)

Description The response is not clear with regard to conceptualizing or understanding liberal arts.

The response indicates a more limited understanding of a liberal arts education. Responses focus on individual skills or dispositions that may be gained through a liberal arts education but fail to capture a larger conceptualization of the liberal arts.

The response indicates a well developed conceptualization and understanding of liberal arts. Responses focus on the notion of a liberal arts education providing broad exposure to different disciplines and ways of knowing across math and science, social science, and humanities.

Examplesof responses Developing personal character

Required for graduation Related to arts, music,

performance Related to politics or political

ideology I don’t know Blank responses Other “off target responses”

that fit the description above.

Study of different cultures Exposure to different

perspectives General education (or a basic

education) Reading, writing, thinking Other “middle responses” that

fit the description above.

Exposure to different disciplines, courses, or subject areas

Well rounded Other “on target responses”


Percentages(N)

32%(39)

35%(43)

33%(41)


Responses to Civic Responsibility Writing Prompt (Pretest-Fall 2006)

Off Target (1) Middle category (2) On Target (3)

Description The response is not clear with regard to conceptualizing or understanding civic responsibility.

The response indicates a developing conceptualization or understanding of civic responsibility yet is limited in scope. Responses focus on individual characteristics or interpersonal traits/skills, but don’t extend to how collective members of society are affected or how the community is impacted.

The response indicates a well developed conceptualization and understanding of civic responsibility. Responses focus on the notion of civic responsibility in a broad sense, by identifying how collective members of society are affected or how the community in which one resides is impacted.

Examplesof responses Doing well in school

I don’t know Other “off target responses”


Descriptive traits---respectful, kind, nice, etc.

Personal/individual responsibilities

Other “middle responses” that fit the description above.

Being a good citizen Giving back to the community Helping others in need Adhering to laws or standards

of the community Other “on target responses”


Percentages (N)

17% (20) 33% (39) 50% (58)


Levels of Elaboration in Writing Prompts (Pretest -- Fall 2006)

Level of Elaboration – Liberal Arts (Pretest)

Shallow (1) Partial/Emerging (2) Deep (3)

Description Response includes shallow ideas with limited and unelaborated details. Response is superficial.

Response includes minimal ideas that are supported with vague or simplistic details. Response lacks insight and/or reflection.

Response includes deep and complex ideas that are supported by rich details. Response includes evidence of reflection and/or insight.

Percentages (N)

48% (59) 41% (50) 11% (14)

Level of Elaboration – Civic Responsibility (Pretest)

Shallow (1) Partial/Emerging (2) Deep (3)

Description Response includes shallow ideas with limited and unelaborated details. Response is superficial.

Response includes minimal ideas that are supported with vague or simplistic details. Response lacks insight and/or reflection.

Response includes deep and complex ideas that are supported by rich details. Response includes evidence of reflection and/or insight.

Percentages (N)

40% (47) 42% (49) 18% (21)


Comparison of Pre-Test/Post-Test Means (Fall 2006)

Scale Pre-Test Post-Test p value

Liberal Arts Content2.03(.80)

2.32(.71)

<.001

Liberal Arts Level of Elaboration1.64(.68)

1.83(.63)

.016

Civic Responsibility Content2.37(.72)

2.45 (.74)

ns

Civic Responsibility Level of Elaboration1.79(.74)

1.86(.74)

ns

Note: N = 108 for Liberal Arts coding; N = 106 for Liberal Arts coding; standard deviations in parentheses.


0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Off Target Middle On Target

Response Coding

Pretest

Posttest

Pretest and Posttest Responses – Conceptions of the Liberal Arts – Fall 2006


Criteria for Information Literacy CompetencyThe student…

– Identifies a variety of types and formats of potential sources.

– Summarizes the main ideas.

– Synthesizes main ideas to construct new concepts.

– Compares new knowledge with prior knowledge to determine the value added, contradictions, or other unique characteristics of information.

– Applies new and prior information to the planning and creation of the essay.

Being assessed by Library faculty using a sample of 41 student papers drawn from five different sections of UNIV 179.


The Seminar/Learning Community• Really focused in depth on one

area• Service-learning really opened

my eyes• I got to pick the class• Made lots of friends –living and

studying together• Professor knew everybody; small

class great• Peer mentors really helpful

The Living Community• Really liked living together• Appreciated the fact that we were

all freshmen going thru the same things

• Easier being with people in the same situation

• Great for working on group presentations and studying

• Peer mentors lived with us and helped us with classes, with adjustment and registration

• Liked the relational aspects the best

Student Focus Group Findings 2006


Faculty Reactions: Rewards• “Energizing--got me out of a teaching rut!”

• “I enjoyed a chance to spread my wings after a steady diet of required courses.”

• “Camaraderie with other faculty”

• “Got to know students well—both academically and personally.”

• “Being both teacher and advisor made me better at both!”

• “I could convey to students that college involves a high level of intellectual rigor…students rose to my high expectations.”

• “I enjoyed seeing the students blossom. They came in quiet and reserved and by the end of the semester they were engaging in discussion and debate.”

• “Students were ‘incredible’…attendance was phenomenal, the students were engaged, they participated, and they asked questions routinely. It was like teaching seniors.”

• “The students knew each other very well, and they often worked on course material or discussed that material in the dorms.”


Faculty Reactions: Challenges• Developing a brand new course involves a tremendous amount of

work and preparation.

• Choosing the course topic well. The ideal topic “has clear impact on students’ personal lives…and can be treated with academic integrity.”

• Balancing course content and goals with attention to students’ general academic and personal progress

• Service Learning component is often the most difficult part of the course to administer. Identifying service opportunities, arranging transportation and other logistics etc. requires considerable advance planning and coordination with other campus offices.

• “Students didn’t get the seminar concept. At the beginning of the semester they were frustrated I wasn’t lecturing. I need to better prepare them for what a seminar is.”


Persistence into Sophomore Year for Exploratory Students at Millersville – Fall 2001 to Fall 2005

0%10%20%30%40%50%60%70%80%90%

100%

Fall2001

Fall2002

Fall2003

Fall2004

Fall2005

FYE Program

Other Undecided


• Starting small and building up

• Close collaboration between Academic and Student Affairs

• Faculty development

• Early and heavy use of assessment data– Learning how to do assessment better

• Getting innovations to fit existing curricular structures

Conclusion: Lessons Learned


• Starting small and building up– Began with more traditional one-credit FYE as

experimental course, evolved and developed itself, morphed into three-credit passion seminar.

– New FYE programming started as small pilot-test with assessment opportunities, expanded in second and third years, with full adoption planned for Year 4.



• Close collaboration between Academic and Student Affairs

– Brings the living and learning together– Tutoring in dorms, study groups, and support

programming benefits student learning and student socialization

– Faculty more attuned to student learning and advisement needs

– Maximizes the “bang for the buck”



• Faculty development– Outside experts (e.g., Jodi Levine Laufgraben, Peggy

Maki, John Gardner, Randy Swing, Steven Briggs, Ed Zlotkowski, Nancy King, Ed Napieralski, John O’Connor, Doug Howard) brought to campus for consultation & training

– Spring/summer workshop– Continued opportunities for discussion and faculty

development during semester– End-of-term follow-up luncheon/workshop– Peer mentor training



• Early and heavy use of assessment data– Use of assessment for program development– Use of assessment to justify expansion– Learning how to do assessment better



• Getting innovations to fit existing curricular structures

– Counting for credit in General Education – Faculty and department buy-in



Thanks for attending our presentation. We hope our work is helpful to you in your own program development and assessment efforts.

Further information is available from our website:http://muweb.millersville.edu/~gened/

Contact information follows on the next slide.


Frederick S. Foster-Clark, Department of Psychology and Coordinator of General EducationEmail: [email protected]: 717-872-3933

Daniel F. O’Neill, Department of Counseling and Human DevelopmentEmail: Daniell.O’[email protected]: 717-872-3122

Laurie B. Hanich, Department of Educational FoundationsEmail: [email protected]: 717-871-2231

Linda L. McDowell, Department of Educational Foundations and Coordinator of First Year ExperiencesEmail: [email protected]: 717-871-2388

Thomas D. Burns, Associate Provost for Academic AdministrationEmail: [email protected]: 717-872-3703

Carol Y. Phillips, Associate Provost EmeritaEmail: [email protected]: 717-872-3703

Go to http://muweb.millersville.edu/~gened/ for information about our Gen Ed program and FYE efforts.

Contact Information

engagement and retention: assessment of a general education initiative for first-year students

Documents

general education initiative

elementary education

year studentsuniv

year studentsfrederick

year program

year studentsengagement

year studentsdr

year studentsstarted