interim evaluation report 01 - nhs employers/media/employers...this report summarises the results...
TRANSCRIPT
SupportiveLeadership &Management BehaviourTraining
01
PSYCHOLOGY WITH BUSINESS IN MIND
Pilot Training Programme MAY 2016Report Number: NHSE008Based on date received up to November 2015
Interim Evaluation Report
CONTENTSCONTENTS
SUPPORTIVE LEADERSHIP AND MANAGEMENT BEHAVIOUR - EVALUATION REPORT 2016
02
BACKGROUND 3
An integrated model of leadership 3Programme evaluation 4
MANAGER/LEADER BEHAVIOUR 6
MANAGER/LEADER CAPABILITY 8
TRAINING REACTIONS 10
QUALITATIVE COMMENTS 12
CLOSE 13
1
6
2
3
4
5
BACKGROUND
www.zeal-sbt.co.uk Email: [email protected] Telephone: 01159 932 324
03
1 - BACKGROUND
There were three broad objectives of the Supportive Leadership &Management Behaviour SBT training programme. These objectiveswere met by trainers delivering a series of activities and exercisesusing scenario based videos and other types of exercises as part ofa one day training programme.
1. Increase awareness and importance of everyday manager/leader behaviour2. Reflect on your own potential leadership/ manager behaviour3. Develop a plan to action supportive behaviour in your own work situation
AN INTEGRATED MODEL OF LEADERSHIP
Although the programme itself was designed to increase supportive leadership behaviour, helpingleaders to become more aware of their own performance and the factors that can influence theirperformance was integral to the success of this training intervention. Leaders were thereforeintroduced to the model of human performance that is shown below in Figure 1.
The approach used in this training starts out from the basis of understanding human health andperformance and then extends this understanding towards leadership and management andthen situates this behaviour within an organisational context. The supportive leader behaviourmodule is built upon the principle that human performance in any given situation is the culminationof three components that interact with and influence each other. These components are whatwe call 1) Personal Factors; 2) Behavioural Factors; and 3) Environmental Factors.
PERSONAL
ENVIRONMENTAL BEHAVIOUR/SKILL
MANAGEMENT/LEADERSHIPEFFECTIVENESS
Figure 1The interactive nature ofpersonal, environmentaland behavioural factors
BACKGROUND
SUPPORTIVE LEADERSHIP AND MANAGEMENT BEHAVIOUR - EVALUATION REPORT 2016
04
The use of this model helps leaders to understand that no one component can be isolated fromthe other, and knowledge of all three is crucial to understanding human behaviour, learning andperformance. As shown above, these three components/factors act in a constant cyclical pattern,interacting with each other and dictating human behaviour. It also helps leaders to realise thatthe performance of their staff is similarly influenced by these factors. By developing a greaterunderstanding and appreciation of these factors, leaders can begin to target any specific areasfor personal development and also apply the model to develop those they lead using a commonframework and language. In addition, and to enable leaders to develop a greater understandingof their own personal, behavioural and environmental factors, the training programme was alsooffered with a 360 assessment called the Supportive Manager Behaviour Index (SMBI). TheSMBI enables leaders to seek feedback on their own approach from their staff, their peers andtheir managers. This process results in a personal feedback report that can be used to plan theirpersonal development.
PROGRAMME EVALUATION
Evaluating the impact and effectiveness of the programme was an important part of this project.As shown below, the SBT is being evaluated in multiple ways:
1. Reviewing the responses that have been recorded on the evaluation questionnairescompleted before and after training and in a transfer survey that was issued to asample of managers in October.
2. Reviewing any change in the responses recorded through the SMBI (360 behaviourassessment).
3. Reviewing any impact of the SBT management development training on measuredorganisational outcomes.
This report summarises the results from reviewing and – where possible - comparing pre andpost evaluation questionnaires (Figure 2)
Figure 2Illustrating the evaluation process for the SBT management development programme
Pre-courseevaluation
TRAININGPost-courseevaluation
BACKGROUND
www.zeal-sbt.co.uk Email: [email protected] Telephone: 01159 932 324
05
Various measures where used to assess the impact of the training, and these are summarisedin Table 1 below. Some items (e.g. personality and mindset) were included as control factors toallow for the fact that individual differences may play a role in determining how effective thetraining is.
Measure Pre Post Follow-up
Management behaviour (awareness) a a aManager capability perceptions a a aPsychological symptoms of stress (PSS) a aJob satisfaction a aIntentions to quit a aOrganisational commitment a aReactions to training aTransfer of training aPersonality aMindset aPerceived organisational support a
Table 1Evaluation Measures
SUPPORTIVE LEADERSHIP AND MANAGEMENT BEHAVIOUR - EVALUATION REPORT 2016
06
BEHAVIOUR A key aspect of this training is to also help managers/leaders
become more aware of supportive and unsupportive behaviours aswell as to develop a more informed understanding about the impactthat certain behaviours can have on employee health, well-being,motivation and performance.
To assess any change in awareness and understanding, delegates were asked to rate 17 behavioursin terms of how supportive or unsupportive they considered each behaviour to be. Assessments ofbehaviour were provided both before and immediately after the training. The 17 behaviours can begrouped into three main categories as shown below in Figure 3. An example behaviour is alsoprovided for information.
2 - MANAGER/LEADERBEHAVIOUR
Figure 3Behavioural categories assessed before and after training.
The before and after training scores (immediately post training) on the three behavioural categoriesare shown below in Table 2.
Positive orsupportivebehaviour
Encourages staff to sharetheir views
Performanceand targetmanagement
Sets demandingexpectations for their staff
Negative orunsupportivebehaviour
Is quick to blame whenthings go wrong
Behavioural Category
Positive or Supportive Behaviour
Performance and Target Management
Negative or Unsupportive Behaviour
AfterTraining
6.20
4.30
2.63
BeforeTraining
5.90
4.27
2.95
SignificantDifference
YES P<.001NOYES P<.001
Information:
N=485
Scale range: Minimum =1 (Very Unsupportive) and Maximum = 7 (Very Supportive)
P<.001 - there is less than a 1 in 1000 probability of such a difference in awareness ofthe supportiveness of manager behaviour happening ‘by chance’
Table 2Scores on the behaviour categories before and after training
BACKGROUND
www.zeal-sbt.co.uk Email: [email protected] Telephone: 01159 932 324
07
These results suggest that immediately post course, delegates had a shift in their awarenessof supportive and unsupportive behaviours so that;
1. Delegates perceive the positive behaviours as being more supportive than theyperceived them to be at the start of the training. This change in awareness andunderstanding of the impact of the supportive behaviours was found to bestatistically significant.
2. At the end of the training, delegates perceive the negative behaviours to be moreunsupportive than they perceived them to be at the start of the training. This changein awareness and understanding of the impact of the negative behaviours was alsofound to be statistically significant.
3. Scores on the behaviours that were associated with performance managementand target setting did not change as a result of the training and remained neutral(i.e. at the scale mid-point).
CAPABILITY
SUPPORTIVE LEADERSHIP AND MANAGEMENT BEHAVIOUR - EVALUATION REPORT 2016
08
Leadership capability is assessed using a five item scale. Each itemassesses the confidence with which leaders believe they can dealwith various scenarios and the scale range is from 0 (no confidence)to 10 (complete confidence). Leaders are asked to assess theirconfidence to deal with specific workplace situations/scenarios.Each scenario is described in detail in the evaluation survey, for thepurpose of this report a summary of the scenario domain is providedbelow in Table 3.
Analysis of the data showed that manager’s perceived capability to deal with different scenariossignificantly increased following attendance on the Supportive Leadership & Management Trainingprogramme. This difference is evident when comparing pre-course training scores with theimmediate post-course training scores on all of the measured scenarios (Table 3). This findingsuggests that managers left the training feeling more confident in their ability to deal with a rangeof management scenarios.
3 - MANAGER/LEADERCAPABILITY
Scenario Domain
Dealing with team conflict
Dealing with staff sickness absence
Dealing with organisational change
Dealing with violence and aggression
Dealing with poor communications
AfterTraining
7.57
7.71
7.35
7.87
7.69
BeforeTraining
6.72
6.61
6.20
7.08
6.64
SignificantDifference
YES P<.005
YES P<.005
YES P<.005
YES P<.005
YES P<.005
Information:
N = 498
Scale range: Minimum = 0 (not at all confident) and Maximum = 10 (completely confident)
P<.005 - there is less than a 5 in 1000 probability of such an increase in perceived capability happening ‘by chance’
Table 3Manager Capability scores
In addition to looking at the scores on the individual capability items, the five leader capability surveyitems can be aggregated to enable an overall leader capability score to be calculated. Using thedata from across the participating organisations (N= 498), Figure 4 below depicts the trend line inleadership capability from before to immediately after training.
CAPABILITY
www.zeal-sbt.co.uk Email: [email protected] Telephone: 01159 932 324
09
Impact of training on: Management capability perceptions
40
38.15
33.20
39
38
37
36
35
34
33
32Before Training
Management Capability P
erceptions
Significant p<0.005
Time of measurementAfter Training
Figure 4Management capability perceptions pre-course and immediately after training
REACTIO
NS
SUPPORTIVE LEADERSHIP AND MANAGEMENT BEHAVIOUR - EVALUATION REPORT 2016
10
As part of the post course evaluation questionnaire, delegate’sexperience of the training was measured. Although a number ofitems were asked within the questionnaire, further analysis of theserevealed that an overall measure of training reactions could bereduced to 7 items – example items are shown in Table 4 below.
Items are measured on a scale of 1- 5 (1=strongly disagree, 5 = strongly agree), with a higherscore indicating a more positive reaction/ experience of the training programme.
Table 5 below provides an overview of the delegate’s scores on the training reaction indicators.Although, and on their own, trainee reactions do not provide a rigorous assessment of any trainingprogramme, they are useful indicators of training quality. The response option with the highestfrequency for each training reaction indicator is highlighted in Table 5.
4 - TRAINING REACTIONS
StronglyAgree
163
116
158
138
133
171
255
Agree
218
268
261
272
267
215
203
Neutral
90
89
69
79
81
71
39
Disagree
23
29
14
22
20
38
5
StronglyDisagree
7
4
4
4
6
12
2
ReactionIndicator
The training met myexpectationsThe training context waswell suited to the trainingprocessIn my opinion, the plannedobjectives were metThe training was realisticThe issues were dealt within as much depth as thecourse allowedThe training enabled meto share professionalexperiences with colleaguesThe trainer/s wereconfident in their delivery
Training Reactions sample items
In my opinion, the planned objectives were met
The trainer(s) were confident in their delivery
The training met my expectations
Table 4Trainee Reactions sample items
Table 5Training reaction scores
REACTIO
NS
www.zeal-sbt.co.uk Email: [email protected] Telephone: 01159 932 324
11
Based on a sample of approximately 507 (note: not all rows will add up to 507 due tomissing cases)
75% of delegates agreed or strongly agreed that the training had met theirexpectations
76% of delegates agreed or strongly agreed that the training context was wellsuited to the training process
83% of delegates agreed or strongly agreed that the planned objectives were met
81% of delegates agreed or strongly agreed that the training was realistic
79% of delegates agreed or strongly agreed that the issues were dealt with inas much depth as the course allowed
76% of delegates agreed or strongly agreed that the training enabled them to shareprofessional experiences with colleagues
91% of delegates agreed or strongly agreed that the trainer(s) were confident intheir delivery
COMMENTS
SUPPORTIVE LEADERSHIP AND MANAGEMENT BEHAVIOUR - EVALUATION REPORT 2016
12
5 - QUALITATIVE COMMENTS
To capture any additional reactions that delegates have to thetraining, the post training questionnaire provides delegates with theopportunity to add further comments. A sample of these can be seenbelow and further support the positive reactions to the training.
Very inspiring at present time. Having the opportunity to reflect upon personalstrengths and weaknesses has been very refreshing.
Useful and practical course. Techniques can be applied easily and seamlesslyinto operational practice. Good opportunities to share experience, reflect and learnfrom others.
Very reflective – open discussions help reflect on own practice. Good looking atdepth – the positives/ negatives of managers behaviours. Liked coming up with goals.
I found the training very relevant to my development and also what I need to knowand change in being a better manager and leader. This has helped me to recognisequalities that I have and areas which I need to develop. There was a good mix of staffon the training course and the experience shared has really helped me and I feel betterequipped to manage and hopefully become a good leader and merge both together inachieving organisational goals.
Group discussion initiated by real life scenario was very helpful in looking atdifferent ways of tackling issues… the experience of others gave me a differentperspective of looking at situations.
I have greatly enjoyed this training and I would suggest that it becomes mandatoryin my trust both for new but also existing managers and especially senior managers.
“ ”“
”
“
”“
”“ ”
“ ”
CLOSE
www.zeal-sbt.co.uk Email: [email protected] Telephone: 01159 932 324
13
6 - CLOSE
On the basis of the evaluation results, it can be seen that the trainingitself is both valuable and valued by trainees. Of particular note is thatmanagers are leaving the training with a significantly increased senseof their own ability to deal with various potential managementsituations that they may encounter in their job roles. In addition tothis, managers left the session with a greater awareness ofsupportive and unsupportive leadership behaviours and reportedbeing keen to apply this learning. In addition, 91% felt that the trainerswere confident in their delivery, which suggests that the delivery oftraining across the different Trusts has been to a high standard.
Following the distribution of the follow-up (time 3) survey in October/ November 2015, it isanticipated that the long term impacts of the training will be explored within the next full report.
NOTES
SUPPORTIVE LEADERSHIP AND MANAGEMENT BEHAVIOUR - EVALUATION REPORT 2016
14
ZEAL SO
LUTIONS
www.zeal-sbt.co.uk Email: [email protected] Telephone: 01159 932 324
15
At Zeal, we are all passionate about the role of psychology in theworkplace. We bring together leading edge thinkers, doers andcommunicators who know what it means to solve customerproblems in practical and useful ways. Our core aim is to enhanceindividual and organisational health and effectiveness. What doesthis mean, you might ask?
Do you ever ask questions like… How can we improve motivation at work?
Do our leaders and managers have all the skills and competencies they need?
How can we better manage workplace violence and aggression?
How can staff retention be improved?
What can we do to reduce absence?
Are we doing all we can to manage workplace bullying/harassment?
Do we have a 'stress problem'? What can we do about it?
What do people really think about their job and the organisation?
Do we have the 'right' organisational culture?
Does the design and layout of our workplace supportwhat we ask people to do?
Is our training effective? Can it be improved?
Our aim is to answer questions suchas these by:
Bringing useable science to bear on allaspects of human psychology at work
Making sure that there is evidence forany solutions advocated
Translating fundamental knowledge intoeveryday ‘jargon free’ tactics, strategiesand actions
Evaluating actions and interventions
Encouraging individual and organisationalproblem solving
Bringing energy and enthusiasm toeverything we do
Management& Leadership
Health &Well-Being
Recruitment& Selection
Training &Development
Consultancy Advice
Research Evaluation
We offer consultancy, advice, researchand evaluation services. And we workacross key organisational functions.
Call us now for a discussion.www.zeal-sbt.co.uk
Email: [email protected]: 01159 932 324
ABOUT ZEAL SOLUTIONS
PSYCHOLOGY WITH BUSINESS IN MIND
www.zeal-sbt.co.ukEmail: [email protected]: 01159 932 324
Zeal Solutions Ltd15 Beck StreetNottinghamNG1 1EQ.
Zeal
Sol
utio
ns L
td ©
201
6. R
ef.S
LM
BM
IER
.
“”
Evaluation isa tool thatcan supportlearning.