process evaluation of employment support in wellington ... · 27 evaluation of employment support...
TRANSCRIPT
TE POU LIMITED TRADING AS TE POU
Process evaluation
of employment
support in
Wellington general
practices An initiative between Workwise, the
Ministry of Social Development,
Work and Income, Compass Health
and Newtown and Waitangirua
general practices
2 Evaluation of employment support in Wellington general practices
Evaluation of employment support in Wellington general practices 3
Acknowledgements
4 Evaluation of employment support in Wellington general practices
Contents
Acknowledgements ............................................................................................... 3
Contents ............................................................................................................ 4
List of figures ...................................................................................................... 5
List of tables ....................................................................................................... 5
Frequently used terms ........................................................................................... 6
Key findings ........................................................................................................ 8 Client profile ....................................................................................................................8 Partnership roles and relationships .........................................................................................9
Executive summary ............................................................................................. 10 Background .................................................................................................................... 10 Evaluation goals and objectives ........................................................................................... 10 Methods ........................................................................................................................ 10 Key findings .................................................................................................................... 10 Issues to consider ............................................................................................................. 12
Programme background ....................................................................................... 13
Evaluation approach and method ............................................................................ 15 Data collection methods .................................................................................................... 15 Data analysis and ethics ..................................................................................................... 16 Limitations ..................................................................................................................... 17
Establishing the partnership .................................................................................. 18 Integration and enrolment .................................................................................................. 20
Client demographics, activities, outcomes and experiences ........................................... 23 Client demographics ......................................................................................................... 23 Client activities and outcomes ............................................................................................. 25 Client experiences............................................................................................................ 28
Partnership ....................................................................................................... 31 Organisational roles .......................................................................................................... 31 Developing relationships, communication and information sharing ................................................. 40 Challenges ..................................................................................................................... 46 Outcomes and learnings ..................................................................................................... 48 Suggestions for improvement and next steps ............................................................................ 52
Discussion ........................................................................................................ 54 Organisational roles in employment support pathways ................................................................ 54 Operational and strategic roles ............................................................................................ 58
Conclusions ....................................................................................................... 60
Appendices ....................................................................................................... 61 Appendix A: Evaluation objectives and questions ...................................................................... 62 Appendix B: Evaluation approach and method .......................................................................... 64 Appendix C: Relationship survey results .................................................................................. 67 Appendix D: Interview schedules .......................................................................................... 76
References ....................................................................................................... 85
Evaluation of employment support in Wellington general practices 5
List of figures
List of tables
6 Evaluation of employment support in Wellington general practices
Frequently used terms
Evaluation of employment support in Wellington general practices 7
8 Evaluation of employment support in Wellington general practices
Key findings
Clients’ profile
Clients’ early outcomes
Evaluation of employment support in Wellington general practices 9
Partnership roles and relationships
10 Evaluation of employment support in Wellington general practices
Executive summary
Background
Evaluation goals and objectives
Methods
Key findings
Evaluation of employment support in Wellington general practices 11
Partnership
Client outcomes
12 Evaluation of employment support in Wellington general practices
Issues to consider
Evaluation of employment support in Wellington general practices 13
Programme background
14 Evaluation of employment support in Wellington general practices
Evaluation of employment support in Wellington general practices 15
Evaluation approach and method
Data collection methods
Literature review
16 Evaluation of employment support in Wellington general practices
In-depth interviews
Review of routine programme data
Data analysis
Evaluation of employment support in Wellington general practices 17
Limitations
18 Evaluation of employment support in Wellington general practices
Establishing the partnership
Table 1: Key implementation milestones
Date Milestone
Evaluation of employment support in Wellington general practices 19
Table 2: Practice and catchment area characteristics
20 Evaluation of employment support in Wellington general practices
Integration and enrolment
Table 3: Employment consultant integration with the two sites and referral data October 2012 to July
2013
Evaluation of employment support in Wellington general practices 21
Figure 1: Referrals accepted to July 2013, Hamilton v Wellington
Referral process
0
2
4
6
8
10
12
14
16
18
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Nu
mb
er
of
acce
pte
d r
efe
rral
s
Month PH Waikato - Referrals accepted by practice PH Wellington - Referrals accepted by practice
22 Evaluation of employment support in Wellington general practices
Cohort trial
Evaluation of employment support in Wellington general practices 23
Client demographics, activities, outcomes and
experiences
Client demographics
Figure 2: Age (in years) and gender of accepted Clients, October 2012 to July 2013
Age (in years)
24 Evaluation of employment support in Wellington general practices
Figure 3: Ethnicity of accepted clients, October 2012 to July 2013
Figure 4: Mental health issues of accepted clients, October 2012 to July 2013
Evaluation of employment support in Wellington general practices 25
Client activities and outcomes
26 Evaluation of employment support in Wellington general practices
Figure 5: Length of stay in the service of accepted clients by phase
Phase/Employment plan
Length of stay
Evaluation of employment support in Wellington general practices 27
Figure 6: Employment outcomes to July 2013, Hamilton vs. Wellington
Figure 7: Job titles and number of hours worked per week of the five Wellington clients
0
2
4
6
8
10
12
14
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Nu
mb
er
of
em
plo
yme
nt
ou
tco
me
s/ N
um
be
r o
f p
eo
ple
Month
PH Waikato - Employment outcome by practice PH Waikato - Number of people who secured jobs by practice
PH Weliington - Employment outcome by practice PH Wellington - Number of people who secured jobs by practice
Shop assistant (4 hours/week)
Security guard (40 hours/week)
Graphic designer (40 hours/week)
Meat processor (45 hours/week)
Sales consultant (40 hours/week)
28 Evaluation of employment support in Wellington general practices
Client experiences
Evaluation of employment support in Wellington general practices 29
30 Evaluation of employment support in Wellington general practices
Evaluation of employment support in Wellington general practices 31
Partnership
Figure 8: Organisational partnership levels
Organisational roles
Strategic management
Operations management
Frontline staff
32 Evaluation of employment support in Wellington general practices
Evaluation of employment support in Wellington general practices 33
Working with clients: Work and Income
34 Evaluation of employment support in Wellington general practices
Evaluation of employment support in Wellington general practices 35
Working with clients: Workwise
36 Evaluation of employment support in Wellington general practices
Figure 9: Time spent (in minutes) on activities, October 2012 to July 2013
030
6090
120
150
180
Act
ivity
tim
e (
min
utes
)
1 2 3 4 5 6 7 8 9Introduction & Employment CV/Cover letter Interview skills, Job Job Work On-going Clinicalassessment planning development training & search applications experience in-work liaison/MDT
support & job try outs support
Evaluation of employment support in Wellington general practices 37
38 Evaluation of employment support in Wellington general practices
Evaluation of employment support in Wellington general practices 39
Duplication of services
40 Evaluation of employment support in Wellington general practices
Developing relationships, communication and information
sharing
Work and Income and Workwise
Evaluation of employment support in Wellington general practices 41
42 Evaluation of employment support in Wellington general practices
Work and Income and GPs
Evaluation of employment support in Wellington general practices 43
44 Evaluation of employment support in Wellington general practices
Workwise and GPs
Evaluation of employment support in Wellington general practices 45
46 Evaluation of employment support in Wellington general practices
Challenges
Privacy and consent
Evaluation of employment support in Wellington general practices 47
48 Evaluation of employment support in Wellington general practices
Outcomes and learnings
Evaluation of employment support in Wellington general practices 49
Establishing roles and relationships
50 Evaluation of employment support in Wellington general practices
Frontline organisational changes
Evaluation of employment support in Wellington general practices 51
Value of the partnership
52 Evaluation of employment support in Wellington general practices
Suggestions for improvement and next steps
Communication and resources
Sustainability
Evaluation of employment support in Wellington general practices 53
54 Evaluation of employment support in Wellington general practices
Discussion
Organisational roles in employment support pathways
Standard pathway
Evaluation of employment support in Wellington general practices 55
Figure 10: Standard pathway to access employment support services
56 Evaluation of employment support in Wellington general practices
Integrated pathway
Figure 11: Integrated pathway to access employment support services
Evaluation of employment support in Wellington general practices 57
Client experiences with employment support pathways
58 Evaluation of employment support in Wellington general practices
Operational and strategic roles
Evaluation of employment support in Wellington general practices 59
60 Evaluation of employment support in Wellington general practices
Conclusions
Evaluation of employment support in Wellington general practices 61
Appendices
Appendix A: Evaluation objectives and questions
Appendix B: Evaluation methodology
Appendix C: Relationship survey results
Appendix D: Interview schedules
62 Evaluation of employment support in Wellington general practices
Appendix A: Evaluation objectives and questions
Table 4: Evaluation objectives and questions
Evaluation Objectives Evaluation Questions
Evaluation of employment support in Wellington general practices 63
Evaluation Objectives Evaluation Questions
64 Evaluation of employment support in Wellington general practices
Appendix B: Evaluation approach and method
Process evaluation
Theory-based evaluation
Utilisation evaluation methodology
Evaluation of employment support in Wellington general practices 65
Data analysis methods
Analysis of qualitative data
Inductive and deductive analysis
66 Evaluation of employment support in Wellington general practices
Analysis of quantitative data
Summary
Evaluation of employment support in Wellington general practices 67
Appendix C: Relationship survey results
Evidence-based supported employment demonstration in Compass Health
primary care practices, baseline and follow up survey results
Working relationship
68 Evaluation of employment support in Wellington general practices
Figure 12: How important is it to have a working relationship with the other party when supporting
clients/patients?
Frequency and type of contact
0
2
4
6
8
10
12
14
16
18
20
Very important Somewhatimportant
Neutral Unimportant Not at allimportant
GPs
W&I
GPs 2
W&I 2
Evaluation of employment support in Wellington general practices 69
Figure 13: In the last six months, how often have you contacted the other party in relation to a client?
Figure 14: In the last six months, how often are you contacted by the other party in relation to a
client/patient?
0
2
4
6
8
10
12
14
Daily 2-3 times perweek
2-3 times perfortnight
1 time permonth
less than 1 timeper month
Not at all
GPs
W&I
GPs 2
W&I 2
0
5
10
15
20
25
Daily 2-3 times perweek
2-3 times perfortnight
1 time permonth
less than 1time permonth
Not at all
GPs
W&I
GPs 2
W&I 2
70 Evaluation of employment support in Wellington general practices
Figure 15: Preferred mode of contact
Information Sharing
0
5
10
15
20
25
Telephone calldirectly to me
Telephone call tosomeone else at my
practice, e.g.reception, which I
will return
E-mail Letter Not applicable
GPs
W&I
Evaluation of employment support in Wellington general practices 71
72 Evaluation of employment support in Wellington general practices
Figure 16: How confident are you that the other party can provide appropriate support for your
patient/client?
0
2
4
6
8
10
12
14
16
18
Very confident Somewhatconfident
Neutral Unconfident Very unconfident
GPs
W&I
GPs 2
W&I 2
Evaluation of employment support in Wellington general practices 73
Limitations
74 Evaluation of employment support in Wellington general practices
Conclusions
Evaluation of employment support in Wellington general practices 75
Appendix: Participant characteristics
Table 5: Participant characteristics
Length of time in role Time spent working in Wellington
GP Work and
Income
GP Work and
Income
Table 6: Participant characteristics at follow up survey
Length of time in role Time spent working in Wellington
GP Work and
Income
GP Work and
Income
76 Evaluation of employment support in Wellington general practices
Appendix D: Interview schedules Interview schedule for clients
Part one: Factors that lead to participation or declining
o
o
o
o
o
Part two: During the programme
o
o
o
o
Part three: Outcomes
Evaluation of employment support in Wellington general practices 77
o
o
o
o
78 Evaluation of employment support in Wellington general practices
Interview schedule for high level staff
Part one: Vision and Purpose
o
o
o
o
Part two: Establishing partnership
o
o
o
o
o
Part three: Partnership during the programme
o
o
Evaluation of employment support in Wellington general practices 79
o
o
o
o
Part four: Outcomes
o
o
o
80 Evaluation of employment support in Wellington general practices
Interview schedule for Work and Income
Part one: Vision and Purpose
o
o
Part two: Establishing partnership
o
o
Part three: During the partnership
Communicating with GPs
o
If doesn’t contact GPs (or says role of RHA to talk to GPs)
o
If does contact GPs
All
Evaluation of employment support in Wellington general practices 81
Communicating with other employment agencies (including Workwise)
o
o
o
o
o
o
Part four: Outcomes
o
o
o
o
o
o
o
82 Evaluation of employment support in Wellington general practices
Interview schedule for GPs
Part one: Vision and Purpose
o
o
Part two: Establishing partnership
o
o
Part three: During the partnership
o
o
o
o
o
o
o
o
Part four: Outcomes
o
o
o
o
o
o
o
Evaluation of employment support in Wellington general practices 83
Interview schedule for employment consultant
Part one: Vision and Purpose
o
o
Part two: Establishing partnership
o
o
Part three: During the partnership
Communicating with GPs
o
o
o
o
o
o
o
Communicating with Work and Income
o
o
o
84 Evaluation of employment support in Wellington general practices
o
o
o
Part four: Outcomes
o
o
o
o
o
o
o
Evaluation of employment support in Wellington general practices 85
References
86 Evaluation of employment support in Wellington general practices
a u c k l a n d
new zea land
t f
h a m i l t o n
n ew zea land
t f
w e l l i n g t o n
n ew zea land
t f
c h r i s t c h u r c h
n ew zea land
t f