Evaluation Summary

Maine College Transitions Program

2008 Fiscal Year

Evaluation Design

• Based on Logic Model• Used Evaluation Questions:

1. Based on data collected from local programs, what is the demographic picture of Maine College Transitions Program? What are the common characteristics across the state? What are the unique characteristics?

2. What elements of established best practices are in place? What new ones have emerged?

Evaluation Methodology

• MAEMIS Data

• Program Qualitative Reports

• Student Surveys

• Professional Development/Conference Evaluations

• Advisory Board minutes

• Coordinator updates

Overall MAEMIS Findings

• Reaching target audience:– 60% of students are over the age of 25– 31% are between 19-24– 9% are between 16-18

• Most are employed– 48% employed– 32% unemployed– 20% not in the labor force

Overall MAEMIS Findings (cont)

Of those who are counted:

• 69% are female (n=783)

• 65% are first generation college candidates (n=310)

• 67% have enrollment in a 2 year college as their goal(n=375)

Overall MAEMIS Findings (cont)

• Most who obtained a GED prior to enrollment did so within 3 years or more than 10 years of enrollment in College Transitions

• 55% were referred by Adult Education programs, and only 15% by Post Secondary Education (however, where there is good articulation, it can be high. Presque Isle had 85% of referrals from the community college.)

MAEMIS Findings (cont)

• Overall, at least 75% of students improved their Accuplacer scores in Reading, Writing, Arithmetic, or Algebra

• 76% (n=125)of completers placed into a 2 year college

Overall MAEMIS Findings (cont)

• CT Co-location and/or enrollment, or a close relationship with a college or community college, appears to show a correlation with increased student college enrollment (87% Vs 53%) upon completion of the CT program (data not complete for all programs reporting)

Overall MAEMIS Findings (cont)

• Most students want to enter healthcare (29%), with 8% wanting business administration, 6% art, 6% early childhood, and 5% trades and technical training as career choices.

• Engineering, information technology, technical trades were chosen by 1% - 3% of students.

Overall MAEMIS findings, cont.

• Biotechnology, construction, financial services, fire science, hospitality, and manufacturing had less than 1% or no student interest for career choices

Themes from Qualitative Data

• Recruitment• Retention• Curriculum• Program Elements• Relationships with Partners• Scheduling• Student Confidence• Professional Development

Recruitment

• Overall request for statewide effort/plan

• Help/Strategies for attracting appropriate participants

• Strategies for attracting participants with commitment

Need Strategies for Retention

• Overcoming student challenges such as childcare, transportation

• Adult issues that impact participation

• Commitment to the program

• Understanding of work involved

Curriculum/Teaching Changes from Last Year

• Integrated technology• Classes mesh with student needs, aligned with

assessments• Classes scaffolded to increase learning and pace• Classes based on developmental courses and run

like college class• Research and research papers assigned

Curriculum, cont.

• Writing prompts with real life applications

• PLATO allows open exit/open entry and self-paced learning

Program Elements/Best Practices

• Goal setting• Notebook/portfolio documenting goal achievement

and work examples• Specific topic workshops or classes OR• Integrating technology, study skills, career planning,

college success into an orientation or into the core curriculum

• Advising/counseling accessible, 1:1, and on going and includes review of achievement and/or goal attainment

Program Elements, cont.

• Partner agencies provide core services such as financial aid, college readiness, time management, self esteem,etc.

• College visits preceded by information and introduction to campus enabling students to ask informed questions

• New technology utilized to familiarize students with campus life (UNET, Blackboard, Moodle, PLATO on line, wireless laptops, Virtual college visits)

Relationship with Partners

• Co-location allows students to access campus services such as advising, library, food services

• Co-location allows participants to mingle with and meet enrolled students

• Co-location beneficial for staff and results in increased collaboration and coordination: computer labs, library resources, shared staff

Relationship with Partners, cont.

• Agreements with higher education for placement, scholarships, credits

• Partner agencies enhance program offerings and allow for additional workshops and courses

• Sharing assessments not always smooth• Some partnerships work better than others

Scheduling

• Self paced VS set entry/exit classes

• One day VS Class by Class

• Extra workshops VS Integrated skills in core curriculum

• Set counseling times VS only as needed

Student Surveys

• Pre and post matches were not consistent, and therefore results not reliable

• Trends were evidenced in low number of matched results received

• Trends indicate value to using this instrument

Student Survey Trends

• Student surveys show increased confidence in college and career success

• Counseling key to building self esteem• Goal setting and incremental achievement helps build

confidence and is a retention factor• Student relationships with family and friends

important factors for success• Student relationships with other students key factor in

student motivation, retention and success• Relationships with teachers and counselors important

factors in building self-efficacy

Professional Development Valued by All Programs

• College Transitions meetings and workshops/conferences

• MELMAC peer learning sessions• NCTN Tool kit• Other College Transitions programs• Request for more sharing time for programs to

learn from each other• Request for more professional development

that includes partners from secondary education and other partners

Conclusions

• Project is recruiting the target population

• Students are increasing their academic skills and self confidence to enter college

• Programs are contextualizing design to their communities

Conclusions (cont)

• There are 2 emerging schedules for Maine’s College Transition program: – Students take set classes as a group scheduled in

one day, evening, or over a few hours over several days. Academic and career counseling, college life skills, and other classes are included in the curriculum

– Students select the classes, based on assessment scores, that meet their academic and career goals, with college life skills classes and academic and career counseling mandatory supports

Conclusions (cont)

• Some programs have begun to offer both types of schedules to accommodate all students

Conclusions (cont)

• Preliminary data indicates that students need fewer developmental courses upon enrollment into secondary education, however this needs further study for validation of this trend

• Emphasis on counseling and skills for college life and success builds student self-efficacy

Conclusions (cont)

• There are some regional trends that can be highlighted for the entire project. For example, Aroostook recruited 50% male and 50% female participants, compared with 32% male and 68% female project-wide

• Fort Kent and Old Orchard Beach programs had 100% first generation students enrolled, followed by Piscataquis/Penobscot region with 68%.

Conclusions (cont)

• Programs value the professional development offered, as well as the guidance and leadership of the project coordinator

• Programs would appreciate increased opportunities for partners to be included in and attracted to CT professional development events

Recommendations

• Continue to refine the MAEMIS and Student Survey data collection protocols and instruments to obtain consistent data

• Put in place more consistent mechanisms and methods to track students after CT completion

Recommendations (cont)

• Examine regional and local trends that enhance the project’s outcomes (such as recruitment of males, increased completers, increased number of students who do not need developmental courses)

• Develop recruitment strategies for those who obtained a GED or High School credential five to 10 years ago. This group is under-represented in program data.

Recommendations (cont)

• Build in professional development opportunities for partners to attend as a team. Utilize the Bridges conference to include topics and workshops that are conducive to this.

• Examine the benefits of the two models of scheduling as well as the student outcomes for each model

Recommendations (cont)

• Utilize labor market information on job trends in the career counseling component. Data this year indicates some fields are not being explored as career options

• Monitor how the economy impacts the project. For example, the choice of healthcare as a career may be impacted by news of layoffs of healthcare workers in local communities or enrollments in CT programs increase because of a lack of employment