California Integrated Service

Delivery Evaluation Report

Phase I

November 18, 2010

Prepared for: The California Workforce Investment Board

By Richard W. Moore, Ph.D. Gerard Rossy, Ph.D. William Roberts, Ph.D Kenneth Chapman Ph.D. Urte Sanchez MBA Chris Hanley

The College of Business and Economics

i

Table of Contents I: Introduction 1

WIA Experience and ISD 2

ISD Design 4

II: Approach 6

Overview 7

Case Method 7

Data Systems10 10

III: Big City One Stop 11

The Context 12

Local WIA Area 12

Decision to Adopt ISD 13

Rolling Out ISD 14

Case Study Site 19

Lessons Learned 41

IV: CV Rural 43

An Integration Microcosm 44

Site Overview 44

Context 45

Local WIA 45

ISD Delivery Development 46

ISD and Service Delivery 54

ISD and Performance 56

Integration 57

Organizational Philosophy 58

Organizational Culture 59

Data Systems 60

Cost Analysis 61

Lessons Learned 67

V: Transportation City9 69

WIA Local Area 70

ISD and Performance 78

Case Study Site 80

Business Services 88

Cost Analysis 88

Lessons Learned 97

ii

VI: Alpha 99

Introduction 100

Introduction of ISD 101

ISD Process and Service 102

Implementation Challenges, Barriers, and Issues 107

ISD and Performance 112

Cost Analysis 114

Lessons Learned 118

VII: Integrated Service Delivery Data and Systems 119

Introduction 120

Big Picture 120

VIII: Lessons Learned 128

Overview 129

Critical Success Factors 129

Contextual Factors 130

Local and Regional Context 131

Implementation and Process Factors 132

ISD as a System 139

Final Thoughts 140

IX: References 141

X: Appendix 143

iii

Tables and Figures

Executive Summary

Figure E- vi

Figure E-2: ISD A Systems View x

VII Big City

Figure III-1: Participants Enrolled and Exited in WIA 15

Figure III-2: Training 2007-08 to 2009-2010 16

Figure III-3: Entered Employment Rate 17

Figure III-4: Retained in Employment 18

Figure III-5: Earnings Measure 18

Figure III-6: Current Process Flow 23

Text Box III-A: Getting Oriented 26

Text Box III-B: A Tale of Two Intakes 28

Figure III-7: Total Clients Served and Registered 30

Table III-1: ISD Phase II Tier 1 and 2 Services 31

Table III-2: Partner Contribution Report 32

Table III-3: Partner Contribution Report: Job Seeker & Business Service Process 33

Table III-4: Cost per Unit of Job Seeker Service 35

Table III-5: Units of Job Seeker Service 36

Table III-6: Cost of Business Services 38

Table III-7: Cost per Unit of Job Seeker Service 39

Table III-8: Number of Business Services 40

IV CVRural

Text Box IV-1: Potluck 48

Flow Chart IV-1: Customer Process Chart 50

Text Box IV-2: OneStop Orientation 51

Figure IV-1: Participants Enrolled and Exited 54

Figure IV-2: Number of Distinct Individuals Served 55

Figure IV-3: Number of Trainees in Training 56

Figure IV-4: Entered Employment Rate 57

Figure IV-5: Retained in Employment 57

Figure IV-6: Earnings Measure 57

Figure VI-7: Comparison of WIA and EDD Culture 59

Table IV-1: Costs by Activity 61

Table IV-2: Costs per Unit of Job Seeker Service 64

Table IV-3: Partner Contribution Report 65

iv

V Transport City

Figure V-1: Clients Served 74

Figure V-2: Clients Enrolled 75

Figure V-3: Total WIA Enrolled Clients and All Clients Area 76

Figure V-: Percentage of Clients Enrolled 77

Figure V-5: Clients Receiving Training 78

Figure V-6: Entered Employment Rate 79

Figure V-7: Retention Rate 79

Figure V-8 Average Earning Performance Measure 80

Figure V-9: Total Clients Served 82

Figure V-10 Transport City OneStop Process Map 83

Text Box V-1: A Half Hour at Reception 84

Text Box V-2: Job Club 86

Table V-I: Partner Contribution Report 89

Table V-2 Partner Contribution Report 90

Table V-3: Job Seeker Costs by Activity 92

Table V-4: Cost per Unit of Job Seeker Service 93

Table V-5: Units of Job Seeker Service 94

Table V-6: Cost of Business Services 95

Table V-7: Cost per Unit of Job Seeker Service 96

VI Alpha

Flow Chart VI-1 103

Figure VI- -Step Registration Process 104

Table VI-1 EDD v. WIA 109

Figure VI-2: Participants Enrolled and Exited 111

Figure VI-3: Enrollees in Training 112

Figure VI-4: Entered Employment Rate 113

Figure VI-5: Retained in Employment 113

Figure VI-6: Earnings Measure 114

Table VI-2: Costs Per Activity 115

Table VI-3: Percent Costs Per Activity Compared 116

Table VI-4: Costs Per Unit of Job Seeker Services 117

VII Integrated Service Delivery Data and Systems

Figure VII-1: Rough Pre-ISD Structure 124

Figure VII-2: Rough Post-ISD Structure 125

Figure VII-3: Database 126

Figure VII-4: JTA vs VOS at RC Rural 127

VIII Lessons Learned

Figure VIII- 130

Figure VIII-2: ISD A Systems View 139

v

Executive Summary

Overview In January 2010 the California Workforce Investment Board (CWIB) contracted with

egrate the work of local Workforce

Investment Act Programs with Wagner-Peyser programs inside California OneStops. At the

time this study began 12 local WIA areas had joined the program and integrated their programs

in the 2008-09 program year. The sites

through their second year of operation as ISD sites when our work began.1 Our evaluation

project has two phases. In Phase I a formative evaluation of the program examines the

implementation of ISD, and in Phase II a summative evaluation of the program will measure

-

depth case studies of four ISD sites or Learning Labs.

To guide our work in the formative evaluation we developed, in consultation with our

advisory committee, six evaluation questions.

1. How did the process of service integration proceed? How did the experience vary from site to site and how did the process vary from the planed process?

2. What specific methods and activities contributed most to successful integration? 3. What barriers and challenges emerged in the process and how were the overcome? 4. How were the broad goals of the program operationalized at the local level? 5. What data are available at the local and state level to measure the achievement of service

6. What lessons have been learned that would help other local areas implement service integration?

In this report we use our analysis of these four diverse Learning Labs to answer these

questions and to provide insights about the ISD program.

Findings

We studied four OneStops in some depth. This allows us to examine their experience in

detail, but it does not provide a large enough sample to generalize to the entire population of ISD

sites. It does allow us to identify factors that emerged across the four sites that seem to either

support or hinder the implementation of ISD. Based on this analysis we have identified a series

give us some evidence to suggest strategies that help other OneStops implement ISD.

context and the implementation process. Within the two categories are five critical success

factors, which are listed below:

1 Two other local areas became ISD sites in the 2009-10 program year as well.

vi

o Context Organizational Culture and Political Context Local and Regional Context

o Implementation and Process Management Structure Decision-making Processes Formal and Informal Implementation Strategies

Each critical factor can be viewed as having both a driving force one that promotes the

desired change and a barrier that inhibits the desired change. This model of change or

strategy implementation was first proposed by Kurt Lewin2 to provide a way of conceptualizing

the critical factors affecting change management in organizational settings.

Figure E-

The key to effective change is to design strategies and processes that enhance and

support the driving forces and diminish or remove the barriers to change. Next we summarize the

driving forces and barriers for each critical success factor.

Context

Organizational Culture and Political Context

The drivers of, and barriers to, integration resulting from organizational culture and

political factors are summarized as follows

2 Lewin K. (1943). Defining the "Field at a Given Time." Psychological Review. 50: 292-310. Republished in Resolving Social Conflicts & Field Theory in Social Science, Washington, D.C.: American Psychological Association, 1997.

Desired Results

Integrated

Service Delivery

Driving Forces Positives Forces for Change

Barriers Obstacles to Change

vii

Drivers

Barriers

Shared mission and vision

Common goals and measures

Common values and priorities

Shared commitment to customers

Compatible data systems