common indicators:
DESCRIPTION
Common Indicators:. Transforming the Culture of Evaluation in Career and Employment Services Saskatchewan Abilities Council Saskatchewan Education, Employment and Labour New Brunswick Post-Secondary Training, Education and Labour Canadian Career Development Foundation (CCDF) - PowerPoint PPT PresentationTRANSCRIPT
Transforming the Culture of Evaluation in Career and Employment Services
Saskatchewan Abilities CouncilSaskatchewan Education, Employment and Labour
New Brunswick Post-Secondary Training, Education and Labour
Canadian Career Development Foundation (CCDF)Goss-Gilroy Inc.
Canadian Research Working Group on Evidence Based Practice in Career Development (CRWG)
Common Indicators:
CCDF: Lynne Bezanson & Céline Renald
Dave Redekopp, Life-Role Development Group
Guylaine Michaud, Université de Sherbrooke
Ken Organ, Goss-Gilroy
Provincial Partners
◦ Managers and Practitioners◦ Clients: all who agree to participate; no criteria except not
one-off’s
Research Team
U Name It StructureInput Process Outcome
Indicators of client change 1. Learning outcomes
• Knowledge and skills linked to intervention
2. Personal attribute outcomes• Changes in self-
esteem, self efficacy, well-being, self confidence, self-awareness)
3. Impact outcomes • Impact of #1 & #2 on
client’s life, e.g., employment status, training status, quality, fit, standard of livinga
Activities to promote positive change
Generic interventions• Working alliance,
client engagementSpecific interventions and progress in:
Job ReadinessCareer Decision MakingSkill EnhancementWork SearchWork MaintenanceLife Circumstances
The support you provide and progress in personal attributes:
self-esteem, self efficacy, well-being, self confidence, self-awareness)
Resources available
• Staff: Number of staff, level of training, type of training
• Funding: Budget• Agency mandate• Community
resources• Access to
Employment Opportunities and Training
• Client Employment History
• Client Life Circumstances
What common indicators are applicable across different client contexts, different client groups, different agencies, and different interventions?
What statements can be made about service effectiveness by tracking common indicators of inputs, processes and outcomes? If numbers permit, it will be possible to aggregate data to begin to address the ultimate question, “What kinds of interventions in what contexts produce what kinds of outcomes?”
Research Questions
5
Input ◦ LABOUR MARKET REALITIES: Catchment Area
Profile Local labour market opportunities – what is the quality and
range of work opportunities? Educational opportunities – easily accessible or not? Community supports/resources – adequate? accessible?
◦ CLIENT REALITIES: Client Employment Potential (demographic info) Work experience Level of education Language Health factors Other
6
Input ◦ PRACTITIONER PROFILE
Age Gender Experience Education/Training
◦ SERVICES OFFERED BY YOUR OFFICE Provided by managers
Process Build & Maintain Working Alliance with Client
Conduct Employability Needs Assessment◦ 5 Employability Dimensions and Additional Life
Circumstances
Develop and Update Action Plan
Interventions (throughout 6 weeks)
Track Client Progress (as appropriate throughout 6 weeks)
PRE-EMPLOYABILITY/JOB READINESS: Has the intention to seek employment and/or enter training but faces challenges, vulnerabilities and/or skill deficits that need to be addressed before the client can be successful in achieving employment/training goals
CAREER DECISION MAKING: Not yet clear about work/learning goals; needs to understand personal skills, interest, values and personality and more about different potential occupations in order to develop a vision for a preferred future
SKILLS ENHANCEMENT: Has career goal, but needs to explore skills and qualifications
needed for the work they want and how to acquire them WORK SEARCH: Has career goal and necessary skills/qualifications for desired work; needs
skills, strategies and tools necessary to conduct a successful job search EMPLOYMENT MAINTENANCE: Has difficulty keeping work or may need to work on the
skills/attitudes required for success at work OTHER: Many circumstances beyond those covered by the employability dimensions may
impact on the client’s employability outcomes. Some of these life circumstances may necessitate referral outside of career and employment services. Life circumstance issues may need to be addressed and resolved before progress toward employment can realistically be made.
Employability Dimensions
Client Progress Indicators Employability Needs
◦ 5 Employability Dimensions◦ Additional Life Circumstances
Personal Attributes Needs/Working Alliance◦ Self-management, self-esteem, well-being, self-efficacy,
self-awareness◦ Working alliance◦ Client engagement
Labour Market Indicators◦ Tailored to client status
Not at all
Not much
A little
Quite a lot
A lot
Identify and clarify future direction (e.g.,training, education, employment or change in life circumstances goal)
Identify personal strengths/resources that support future direction (e.g.,training, education, employment or change in life circumstances goal)
Resolve specific challenges/vulnerabilities that may impact on future direction (e.g., mortgage, public transit, day care etc.)
Access community resources/supports that address specific personal challenges/vulnerabilities and assist in helping move goal forward (e.g., mental health services; addictions counselling; public housing etc.)
Develop necessary personal supports needed to move toward future direction (e.g., family, childcare, transportation)
Acquire life/employment skills and attitudes that support future direction
Acquire attitudes which support future direction Develop and follow a plan of action to move future
direction forward
Keep appointments and sustain efforts Other: (please specify)________________________________
Indicators of challenges in client life circumstances are in evidence:
Not at all
Not much
A little
Quite a lot
A lot
Improved housing is needed Improved transportation is needed Improved capacity to work and/or study is
needed
Increase in sense of responsibility for own choices and behaviours is needed
Increase in ability to set short and long term goals is needed
Increase in understanding expectations and demands of employers is needed
Reduction in destructive behavior is needed
Following medical and medication protocols is needed
Increased openness to change is needed Increased access to constructive and
positive support systems is needed
Improved relationships with family and friends are needed
Increased trust in other people is needed Other: _________________________________
Life Circumstances
Client needs assistance in the following areas: Not at all
Not much
A little
Quite a lot
A lot
Ability to Self-Manage: More info
Developing stronger Self-Esteem: More info
Improving sense of Well-Being: i.e.: More info
Developing stronger Self-Efficacy: i.e.: More info
Increasing Self-Awareness: More info
Other (Please specify): ________________________________________
Personal Attributes Example
more info: Self-Esteem:
“a person’s overall emotional evaluation of his or her own worth, an attitude to self and a judgement of oneself. Self-esteem encompasses beliefs (e.g., I am competent; I am worthy) and emotions (pride, shame). Self-esteem is the positive or negative evaluation of the self”
Demonstrates an optimistic outlook Sees self as competent and able Has confidence in ability to interact with others Uses positive self-talk Acts with little hesitation Raises few objections Asks questions Articulates skills assertively Makes eye contact Is hopeful Takes reasonable risks
Self-Esteem Attribute
Outcome Indicators Employability Needs
◦ 5 Employability Dimensions◦ Additional Life Circumstances
Personal Attributes Needs/Working Alliance◦ Self-management, self-esteem, well-being, self-efficacy,
self-awareness◦ Working alliance◦ Client engagement
Labour Market Indicators◦ Tailored to client status
quality and “fit” of employment or education/training, salary, location
Input & Outcome Connections◦ More educated clients do better than less
educated clients◦ Success is positively related to labour market
attachment ◦ “No Barrier” clients do better than “Single
Barrier” clients who do better than “Multiple Barrier” clients
◦ Success is positively related to entry self-efficacy◦ Success is positively related to entry self-esteem
What We Hope/Expect to Say
Outcomes and Processes◦ Learning Outcomes
Client skill/knowledge increases with any intervention◦ Personal Attribute Outcomes
Client attributes improve with any intervention Client self-esteem improves with any intervention Client self-efficacy improves with any intervention
◦ Labour Market Outcomes Employment/training increases with any intervention Quality/fit increases with any intervention Working alliance is positively related to employment/
training outcomes
What We Hope/Expect to Say (cont’d)
Relationships between Outcomes◦ Learning outcomes are positively related to:
personal attribute outcomes labour market outcomes
◦ Personal attribute outcomes are positively related to labour market outcomes
Personal Attribute Outcomes & Processes◦ Working alliance is positively related to self-
efficacy changes◦ Working alliance is positively related to self-
esteem changes
What We Hope/Expect to Say (cont’d)
Learning Outcomes & Processes◦ Client skill/knowledge increases are positively
correlated with working alliance Measurement Relationships
◦ Practitioner and client assessments of working alliance are positively related
◦ Practitioner assessment of personal attributes is positively related to client self-esteem scores
◦ Practitioner assessment of personal attributes is positively related to client self-efficacy scores
What We Hope/Expect to Say (cont’d)
U Name It StructureInput Process Outcome
Indicators of client change 1. Learning outcomes
• Knowledge and skills linked to intervention
2. Personal attribute outcomes• Changes in self-
esteem, self efficacy, well-being, self confidence, self-awareness)
3. Impact outcomes • Impact of #1 & #2 on
client’s life, e.g., employment status, training status, quality, fit, standard of livinga
Activities to promote positive change
Generic interventions• Working alliance,
client engagementSpecific interventions and progress in:
Job ReadinessCareer Decision MakingSkill EnhancementWork SearchWork MaintenanceLife Circumstances
The support you provide and progress in personal attributes:
self-esteem, self efficacy, well-being, self confidence, self-awareness)
Resources available
• Staff: Number of staff, level of training, type of training
• Funding: Budget• Agency mandate• Community
resources• Access to
Employment Opportunities and Training
• Client Employment History
• Client Life Circumstances
Where “U-Name-It” is headed… More evidence for the field to justify the power
and efficacy of career interventions
More integrity around outcomes
A culture of evaluation built into career and employment service practice
Over time, the ability to answer:◦ “What kinds of interventions in what contexts
produce what kinds of outcomes”
20
Research Reports will be completed by May 31, 2013
Follow the CRWG website at:
www.crwg-gdrc.ca
Thank You
To follow the study: