new - powerpoint template - nhs employers/media/employers/documents/recruit/vbr... · development...

mailto:[email protected]



•

•

•

•

•

•

•

•

Evidence base for VBR methodologies

Dr Máire Kerrin

Work Psychology Group

www.hee.nhs.ukwww.hee.nhs.uk

What are values?

• There is a complex relationship between values and other attributes

such as personality, ability and motivation.

– Values are motivational goals that influence behaviour.

– Values primarily impact the goals that individuals choose to pursue

(goal content) while personality traits primarily impact the amount of

effort that individuals exhibit in pursuit of those goals (goal striving).

– Personality represents behaviours that come most naturally, whereas

values reflect effort (a choice) to behave a certain way. This is an

important distinction when considering selection tools.

• Assessing values in recruitment is more challenging than assessing

abilities and skills.


Key Messages from the VBR Literature Review

• VBR is only one part of embedding the NHS values, research emphasises the

need for a multi-faceted approach.

• Individuals recruited with optimal values for the delivery of high quality

compassionate care, could lose these values if placed within teams with

suboptimal values.

• Evidence relating to VBR directly is limited, yet there exists a plethora of

measurement tools claimed to be of use.

• A single VBR tool used for attraction and marketing purposes may be

appropriate.

• When making selection decisions, a tailored approach is more likely to

accurately assess the diverse requirements of different job roles across the

NHS.

• Recruiters (particularly when interviewing) should represent the values the NHS

is seeking to attract and be trained to assess for values during recruitment.


Evaluation criteria for selection methods

• Evaluation criteria with which to judge the effectiveness and efficiency of

methods for selection methods have been established.

(See Arnold et al, 2010; Smith & Robertson, 1993; Schmitt, 2012;

Patterson, 2008; 2012; Cleland et al, 2012)

• Sixteen key evaluation criteria relevant to the VBR agenda.

• Each criterion is not mutually exclusive - it is possible for some criteria to be

at odds with one another.

• Recruiters may decide to prioritise (weight) certain criteria.


Evaluation CriteriaAccuracy and

effectiveness

1. Evidence of reliability

2. Evidence of validity

3. Arrangements for on-going validation, evaluation and

development are in place

4. Susceptibility to coaching

5. Fairness, promotes diversity/widening access

6. Legality

Costs and efficiency 7. Scalability for high volume recruitment

8. Efficiency

9. Utility

10. Generality of use

Practicalities and

implementation

11. Practicality (ease of administration)

12. Expertise required for analysis of information generated by

the tool

13. Ease of interpretation

Stakeholder

acceptance and

feedback

14. Positive employee/trainee/student perceptions

15. Generates appropriate feedback

16. Educational impact/value

Evaluation criteria for selection methods


•

Review of selection methods

Selection Method for

VBR

Reliability Validity Candidate

acceptability

Cost (to the

organisation)

Promotes

diversity

Susceptibility

to coaching

Traditional Interviews Low Low High Moderate to high Low High

Structured Interviews e.g. competency-based,

situational, multi-mini

interviews

Moderate to

high

Moderate High Moderate to high Moderate Moderate

Group Interviews Low Low Moderate Moderate Low High

Personal statements Low Low High Low to moderate Low High

References Low Low High Low to moderate Low N/A

Situational judgement

tests

High High (only if

based on a robust

psychometric

methodology)

Moderate to

high

Low to moderate High Low to

moderate

Personality testing High Moderate Low to

moderate

Low to moderate Moderate Moderate to

high

Selection centres

using work samples e.g. group exercise,

written/in-tray task,

presentations, interactive

exercises

Moderate to

high

High (only if

exercises are

used in

combination

based on a multi-

trait, method

approach)

High High Moderate Moderate


Implications for pre selection in VBRSelection

Methods

Implications of the evidence for implementing VBR

Personal

statements

Whilst candidate acceptability is high, susceptibility to coaching is also

high. (Ineffective method for VBR)

References Use of references remains widespread despite little research supporting

validity or reliability. (Ineffective method for VBR)

Situational

judgement tests

Improved validity over other selection tools (IQ and personality tests),

and can be mapped to organisational values. Whilst SJTs can be

relatively costly to design, SJTs are machine-markable and can be

delivered on-line, producing cost savings in high volume selection.

(Effective method for VBR)

Personality

testing

Concerns regarding faking in operational selection. Where there is a

high risk of susceptibility to coaching, personality assessment is best

used to drive more focused questioning at interviews (rather than a

stand-alone instrument without verification). (Personality tests may be

more useful at the attraction phase of VBR as part of self

assessment/selection.)


Selection Methods Implications of the research evidence for VBR

Traditional Interviews Across most evaluation criteria, traditional interviews perform

poorly. (Ineffective for VBR)

Structured Interviews

e.g. competency-based,

situational, multiple-mini

interviews

Interviews based on a thorough role analysis, use structured and

standardised questions with trained interviewers and appropriate

scoring can be reliable and valid. Candidates prefer interviews to

other methods although they are relatively resource intensive.

(Effective method for VBR)

Group Interviews Whilst group interviews appear more cost efficient in terms of

assessor time, evidence for reliability, validity and fairness is

lacking. (Ineffective method for VBR)

Selection centres

using work samples e.g.

group exercises,

written/in-tray task,

presentations, interactive

exercises

When designed appropriately (using a multi-trait, multi-method

approach with work samples), SCs are valid predictors of job

performance. Candidates are positive towards SCs as they have

multiple opportunities to perform. SCs are relatively expensive to

design and implement. (Effective method for VBR)

Implications for selection in VBR

Values-Based Recruitment Interviews (VBI)

Train the Trainer

Overview of Training Package Content

2014

VBI Train the Trainer: Objective

“To ensure individuals attending the

workshop have the requisite skills,

knowledge and confidence to be able to

train colleagues within their own

organisation on VBR interview technique”

Train the Trainer: Workshop Objectives• Create opportunities for learning, sharing knowledge and discussing

experience in relation to VBR.

• Refresh and clarify knowledge of assessment best practice, with

particular reference to VBR.

• To summarise the VBR evidence base, including recent literature review.

• Provide information about effective VBI practices.

• To increase understanding of VBI techniques, including interviewer

skills.

• Provide support in critically appraising existing VBI practices and

effectively implementing new VBI practices.

• Provide opportunities for interaction, participation and feedback.

• Facilitate peer support networks.

Workshop Content: Overview

• Two days (10am-4pm), approx. 10 delegates & 1

facilitator; designed with flexibility.

• Blended approach to learning: classroom based learning

and practical activity; emphasis on experiential learning.

.Day One (All Delegates)

• Overview of VBR: Background, Context & Evidence Base

• Understanding of VBI Techniques

• Refresh of Assessment / Interview Skills

• VBI in Practice: Practical Skills Development Sessions

Workshop Content: Overview

Day Two – Option One

(VBI Implemented)

• Key Considerations for

VBI Implementation

• Enhancing

Understanding of VBI in

Others

• Training Others in VBI

techniques

Day Two – Option Two

(VBI not implemented/in refinement)

• Review of VBI progress

• Key Considerations for

VBI Implementation

• VBI Action Planning

• Training Others in VBI

techniques

Evaluation Strategy

• Pre- and post-training evaluation utilising evidence-based

model1, focused on three key learning outcomes.

• As a result of this training intervention…

– Cognitive - Has knowledge increased (moving from novice to

expert) relating to values-based recruitment interviews?

– Behaviour / Skill - Has task proficiency and competence

relating to values-based recruitment interviews improved?

– Motivational / Affective - Has motivation and commitment to

the concept of values-based recruitment increased?

Documentation: Overview & Key Features

• High-quality, relevant content

• Ensure consistent approach to train the trainer delivery

– Clear, detailed materials and guidance documentation

• Consideration of ‘paperless’ delivery

• Two Suites of Materials:

1. Train the Trainer Workshop

2. Delegate Materials: ‘take home’, VBI training in Trusts

• NHS Employers Corporate Branding

• Flexibility of Delegate Materials

– Editing enabled for adaptability

Documentation: Train the Trainer

• Train the Trainer Presentation Slides (x 3)

• Facilitator Guidance

• Delegate Pre-Reading

• Delegate Learning Log (x 2)

• Supporting Exercise Material (x 1)

• Certificate of Completion

Documentation: Delegate Materials

• VBI Example Training Content (Slides)

• Example Exercise Templates

• Example VBI Training Pre-Reading

• VBI Training Facilitator Guidance

• VBI Training Evaluation Templates

• VBI: A Resource for Managers

Documentation: Evaluation

• Train the Trainer: Guidance for Evaluation

– Including Six Month Follow-Up Interview Guide

• Pre-Training Evaluation Questionnaire

• Immediate Post-Training Evaluation Questionnaire

•

•

•

•

•

•

•

Situation

Task

Action

Result•What did you learn from it?

•What impact did it have on you?

•What would you do differently?

•How did you feel about it?

•Why did you decide to…?

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

Thank You

Any Questions?

Recruiting for Values in HEIs

Values Based Recruitment (VBR) Partner Workshop

Sandra Hatton, Director Project One

18 July 2014

Etc Venues, Birmingham


NHS Employment Journey – A Continuum of Values Based Employment

Values Based Recruitment (Values tested at multiple assessment points)

Attracting Candidates

Values of NHS Constitution marketed to prospective candidates

(students, trainees and employees),including use of

NHS Careers Service.

Pre-selection Selection

Values Based Environment

Values Based Employment Systems

Post SelectionEntry into

Employment & Beyond

Values based short-listing

criteria.Pre-selection

tools to assess values.

Use of selection tools, methods and approaches to assess values.

Evidence of values in

education, training,

development and organisation

culture.

Embedding values in organisation processes and

continuous learning and professional

development.

Culture & Leadership

Education, Training & Continuous Development

Values Driven Behaviour

Recruitment Post selection

NHS Constitution


Programme deliverables

By October 2014, the VBR programme will deliver a national values

based recruitment framework and suite of associated tools and

resources.

Other key deliverables include:

• Common working definition of values based activity

• Evidence to support employers and HEIs in recruiting for values

• Tool to support NHS organisations in testing and mapping their locally

agreed values to those of the NHS Constitution

• Access to values based interview and assessment training

• Methodology for measurement of long term impact and benefits


National VBR framework

HEE will oversee delivery of a national values based recruitment

framework and suite of associated tools and resources by October 2014.

The framework will:

• provide a common set of evidence-based national VBR principles and

standards against each stage of the recruitment process (attraction,

screening, selection and induction);

• provide guidance and access to resources to successfully recruit in HEIs,

NHS employers and LETBs and to prepare organisations for VBR;

• provide easy access to a toolkit of resources and evaluated techniques;

and

• access to good practice and case studies.


Stage Good practice standards for VBR

Attraction

• Marketing materials, websites and job descriptions and person

specifications include specific reference to the NHS

Constitution / Local Values.

• Local values (where they exist) are mapped to those of the

NHS Constitution. Evidence of

involvement

of patients

and public in

some stage

of the

recruitment

process.

Screening

and/or

Selection

Where designed appropriately, the following are considered to be

evidence-based approaches for VBR:

• Situational Judgement Tests (SJTs);

• Structured interviews;

• Multiple Mini Interviews (MMIs); and

• Selection Centres.

As good practice, the overall selection process should encompass

a structured face-to-face interview on an individual basis.

Post

selection

• Providing feedback to unsuccessful candidates if requested.

• Embedding the NHS Constitution in induction processes.

Professional and regulatory standards are integral to the overall selection process.

National VBR Framework

In October 2014, HEE will launch a national Values Based Recruitment

framework, including associated resources and good practice standards, to

support HEIs and providers to effectively recruit for values.

Case Study HEIs: Using Multiple Mini-Interviews (MMIs)

to recruit midwifery students with the right values

Alison Callwood, Teaching Fellow, University of Surrey

Presented by Jan Zietara,

Project Manager for Project 1 ‘Recruiting into HEIs’


Case study: Using MMIs to recruit midwifery students with the right values

• MMI development informed by current evidence including: K. Eva, McMaster

University, Ontario; C. Roberts, Sydney Medical School; A. O’Brien St George’s,

University of London

• Objective structured clinical examination (OSCE) style format

• Candidates respond to questions relating to a specific scenario at a ‘station’ in a

timed circuit e.g. x7, 4 minute stations with 1 minute between

• Each station scenario is designed to assess a specific attribute/NHS value

• No prompt questions and no dialogue between interviewer and candidate

• Practitioners integral to process

• Applicants assessed and rated against generic and station specific criteria

• Preliminary research shows reliability (internal consistency) is excellent

• Predictive validity: awaiting data


Scenario Example: Initiative and problem solving

You have recently taken the initiative to volunteer at a homeless shelter. On

one particular day you meet a 42 year old woman who is dying from an alcohol

related liver disease. Her skin and eyes are very yellow from a lifetime of

alcohol abuse and she complains of significant generalised pain which

worsens with movement.

She confides in you that she hasn’t had a drink in two days and “wants to get

hammered” before she dies but cannot get the bottle of vodka out of her

locker. You know that there is a strict policy for residents that alcohol is not

allowed in the shelter. There is another woman lying on her side three beds

away who is watching your interactions closely.

What would you say or do? Please explain…


Selected FeedbackStudents

“The questions are challenging but this is what interviews are supposed to be about,

pushing yourself to show your true personality and nature not rehearsed answers”

“You can move from a bad station in the MMI and not let it impact on other stations”

“In the traditional interview I felt I could relax more, in the MMI it felt like I was starting

all over again at each station”

“It doesn’t allow you to get a feel for staff like the personal interview…we are picking

the University as well as the University picking us”

Interviewers

“I think it gives you a much better idea than our current interview of whether they are

able to rationalise, analyse, synthesise …you can see those students who have very

concrete thinking and those who are able to look at both sides of the argument”

“I think it does demonstrate caring, certainly in my scenario with the alcoholic, it came

over very clearly… I found the compassion came out much clearer…”

“What’s quite nice about this is that… they come in and sit down and they are the

person they are and we are not biased by anything on their application forms”


On-going Research: Longitudinal Studies

(predictive validity)

• Sept 2014 BSc midwifery studies students exploring correlations between

their MMI score at interview and subsequent programme performance at

end of years 1, 2, 3 (n = 60).

• The University of Surrey is replacing the personal interview with MMIs for

all pre-registration health care student selection in 2014-15 recruitment

cycle. A large follow-up study is planned (n > 350), commencing Autumn

2014.

• For further information [email protected]



Thank you &

Questions


Contact details and further information

• Via email: [email protected]

• http://hee.nhs.uk/work-programmes/values-based-recruitment

• Via twitter: @NHS_HealthEdEng

@NHSE_Caroline

http://hee.nhs.uk/work-programmes/values-based-recruitment/


Key Questions

Q1. Looking at the table in the draft framework, please discuss and feedback

any concerns with the good practice standards included?

Q2. Are there any gaps we need to consider?

Q3. What are your challenges to using values based recruitment methods?

NHS Jobs

Values based recruitment

Agenda

• Introduction

• NHS Jobs update

• Pre-application and additional questions

• Scorecards

• Interviews

• Questions

52

Introduction

• OJEU

• Design and development sessions

• User acceptance testing sessions

• Pilot – June 2013

• Train the trainer – October 2013

• Launch date – 4 March 2014

• Planned activity

• Helpdesk activity

• Future development

53

NHS Jobs Usage

54

0

5,000

10,000

15,000

20,000

25,000

March (13/14) April (13/14) May (13/14) June (13/14)

Adverts Placed

NHS Jobs 1

NHS Jobs 2

NHS Jobs Usage

55

0

100,000

200,000

300,000

400,000

500,000

March (13/14) April (13/14) May (13/14) June (13/14)

Applications submitted

NHS Jobs 1

NHS Jobs 2

News

56

System Administrators were reduced to max 3 per organisation on 31st May – However, please ensure you have more than 1!

There was a software update on 4th June, which contained:– Fix for some issues when printing interview schedules– Fix for jobseeker error when requesting re-send of activation code– Fix for error when changing the vacancy team– Fix for error received when finalising shortlisting in some scenarios

On 12th June, the software update contained:– Supporting Information on applications from 1000 to 1500 words– Fix for Twitter adverts– Fix to prevent referees from being chased after email address is

changed– IIP standard logo added

News

57

On 26th June, the software update contained:– Improvements to candidate search– Improvement to allow Vacancy Bulletin report to be downloaded– Ability to report on primary and/or secondary organisation

Today, 10 July, the software update contained:- Ability to print the application form declaration section- Include information on when interview invitations were sent to

applicants- Include interviewees that have declined or withdrawn in the

interview schedule - Recruitment teams having the ability to un-reject applicants,

returning them to the logical previous status- Ability for Recruiting Managers to record interview outcome appoint

recommendations

Documents and Links

Pre application and additional

application questions

Creating a scorecard template

NHS Jobs

Contact details

[email protected]

Mobile No. 07827872313

65


Questions

NHS Jobs

Creating an interview

Questions

Contact details

[email protected]

Mobile No: 07827872313

Questions

Contact details

[email protected]

Mobile No: 07827872313