CHEO EvaluationRutgers University, School of

Management and Labor Relations

Heather McKayDirector, Innovative Training and Workforce Development Research

Rutgers CWW and Evaluation

• Conducted numerous evaluations and research projects in education, training and workforce throughout the US and abroad

• Rutgers CWW has worked in Colorado since 2008

• Conducted three research projects in Colorado to date: online learning project, sectors project evaluation, TAACCCT Round 1

Rutgers Team

• Heather McKay• Suzanne Michael• Debbie Borie-Holtz• Sara Haviland• Laura Barrett• Renee Edwards• Joseph Rua

Why Evaluate?• Federal grant and an evaluation is required• Understand the type and degree of impact of TAA on program

development, academic success and employment• Identify best strategies/practices• Collect observations, insights and lessons learned to inform,

refine and/or develop more effective programs/services to meet existent and emergent needs of students, colleges and industries

• Identify issues/needs that can inform academic practice and public policy

• Provide feedback to colleges throughout the lifetime of the grant so that changes can be made mid-course

• Share information between the colleges• Tell Rutgers what you need from this evaluation

Process Evaluation

• Qualitative

• Quantitative

• Feedback ongoing

Focus of the Qualitative Evaluation

• Assess the implementation of CHEO across all colleges• Examine how each college is implementing their

individual project goals.• Document and understand the partnerships in the

project. Assess how these groups are working together and the influence these partnerships are having on the project.

• Look at the process of recruiting TAA and TAA eligible/like students to programming

• Examine the experience of students. • Study the development, delivery, and use of the online

and hybrid curriculum

Focus of Qualitative Evaluation Cont.

• Examine the relationship and partnership with local workforce systems and employers

• Track career coach contact• Track the development of stackable

credentials in the region.• Examine faculty training• Examine the use of NANSLO

Focus of the Quantitative Evaluation

• Group level comparison cohorts for redesigned programming and coursework

• Individual matching of students in redesigned programming to others at the colleges

• Analysis of individual level data with workforce data including UI, TAA, and where possible other workforce data sources.

• Analyze the path and assess the population of students pursuing the stackable credentials identified in the grant.

• Follow the path of students using the services of the career coach

• Document increased capacity of institutions for serving students.

• Attending group Sessions/calls

• Career Coach Sessions

• College Site Visits• NANSLO Meetings

•College Administrators•Grant Personnel •Partner Employers •State-level Stakeholders•Faculty•Students•NANSLO partners

•Historical and Current Academic & Student Profile Data from College Data Sources

•UI Data

•Career Coach data •Pre & Post Course Surveys

•Interviews•Focus Groups

Individual Student Data Comparative

Cohorts

ObservationsInterviews & Focus Groups

\

COETC EVALUATION QUALITATIVE AND QUANTITATIVE ANALYSES

DATA PROCESS: INDIVIDUAL QUANTITATIVE DATA

Student Variables

Banner/or state equivalent

Career Coach reporting Wage Data

Possible workforce data

Data Sources Cross-walked

Data Reported

Individual & Group Level

(Individual Level Data: Identifiers Removed)

Important Issues To Date

• Institutional Review Boards• Cross walking data• Access to data and confidentiality• Comparative cohorts• Reporting data• Communication flow between Rutgers,

Pueblo CC and the colleges

Timeline

• Data conversations have begun• IRB conversations have begun• College interviews will be scheduled over

the coming months• Following these conversations the

evaluation plan will be refined and revised

Contact Information and Questions

Heather McKay hmckay@work.rutgers.edu

848-932-1275