IKT. 1 . LANGUAGE & COMMUNICATION DISOKI>EKS, 1998, VOL. 33, SUPP1,EMENT

WHEN STAFF TRAINING ISN’T ENOUGH: AN EVALUATION OF A COLLABORATIVE APPROACH

Christine Jenkins’ , Linda Baxter ’, Joanna Dowton2 , Catherine Gibbs’ and Jan Partridge’

Department of Psychology, University of Portsmouth, Portsmouth, PO1 2DY’

East Wiltshire Healthcare Trust, Postern House, Marlborough, Wiltshire, SN8 4AS2

This paper describes an evaluation of a training workshop for staff working with people who have severe physical and learning disabilities and who need extensive support with eating and drinking. Training involved joint input from speech and language therapy, dietetics, occupational therapy and physiotherapy and was evaluated at three levels: participants’ satisfaction, gains in knowledge and video-tapes of mealtimes pre- and post-training to assess changes in practice. Results indicated very positive staff perceptions of the workshops and highly significant increases in knowledge, both immediately (p < 0.001) and after six months (p < 0.001). However there was no evidence of significant change in practice. Results are discussed in the context of bringing about changes in staff behaviour.

Introduction

A significant group of people with learning disabilities who use residential and day services also have physical disabilities and are dependent on others for daily living activities and personal care. Many need support with eating and drinking and some will have complex and severe problems in this area, possibly including dysphagia. The nature of this support has implications for a number of health professions (Hotaling 1990). The advice of dietitians is vital in achieving a balanced and healthy diet especially for those who have difficulty in managing normal quantities, variety or consistency of food whilst speech and language therapists have a role not only with managing the process of eating, drinking and swallowing but also in making staff aware of the opportunities for communication which occur at mealtimes. The expertise of physiotherapists and occupational therapists is essential to ensure that optimal positioning is achieved for both client and carer and that appropriate equipment is used. For most of us eating is a pleasurable, frequently social activity which we expect to enjoy in a pleasant environment and every effort should be made, challenging though it may be, to achieve the same for this client-group.

Bennett (1994) reports that those who carry out these daily care activities in everyday life need to have an appropriate range of knowledge, values, skills, experience and competencies. Lack of these attributes can result in reduced quality of life for service users and has potential risks for physical health and may in extreme cases lead to death (Rogers et al. 1994). In order to meet the need to develop staff skills a series of one-day,

41 0 People with learning disabilities

multidisciplinary workshops was held in 1995/6, which resulted in very favourable comments from participants and requests for more workshops from other staff. However, mindfhl of the necessity to provide clear, objective evidence that the considerable professional time and effort invested was worthwhile, it was decided to carry out an in- depth evaluation of the training.

When considering evaluation of staff training, it is important to reflect on its primary purpose, i.e. that the increase in knowledge and skills of staff should result in better outcomes for service users. Cullen (1988) emphasises the importance of this: ‘There will almost certainly be short-term effects, such as immediate improvements in staff morale, and positive evaluations of the training programme by participants, however there is virtually no reliable evidence that such changes are related to benefits to mentally handicapped people’ (p. 3 10).

It was therefore planned to evaluate on three levels by obtaining the perceptions of participants about the training, as it is important that people view the workshop as a positive experience, by establishing what has been learned by comparing levels of knowledge before and after the course and to assess long-term retention of the information by re-testing after an interval of six months and by evaluating whether the training has resulted in beneficial changes in practice by video-taping mealtime episodes before and after the training.

Method

Participants 20 staff who support people with severe and multiple disabilities attended one of

two oneday workshops. Staff were from two units (seven from a small group home and thirteen from the special needs unit of a day centre) with a wide range of age and experience in each group. In each case a ‘whole establishment’ approach was used so that all of the staff from that particular unit was included in the training regardless of previous experience or qualifications. Both staff groups had requested training on this topic and when the purpose of the evaluation was explained to them, agreed to take part. Assurances of confidentiality were given.

The people with learning disabilities who use the two services were also participants in the evaluation and for them the issue of consent is more complex. They all have cognitive impairments such that it is unlikely that they have capacity to give informed consent so permission, with assurances of confidentiality and anonymity, was sought on their behalf from family or carers.

Training Training followed the format of a one-day interactive workshop divided into nine

sessions and presented by relevant professionals. The first four sessions described aspects of normal eating and drinking, positioning, nutrition and environmental considerations whilst three sessions concentrated on meeting special needs in each of these areas. The day finished with a practical session, comprising role-play exercises and experimenting with food and equipment.

Knowles (1980) describes four assumptions of andragogy or adult learning, as a tendency towards self-directedness in learning, more effective learning through experiential techniques, awareness of specific learning needs and the fact that adults are competency based learners, desiring to acquire skills and knowledge relevant to their

People with learning disabilities 41 1

current circumstances. These factors were central to the design of the workshop which had a balance between theoretical and practical sessions and an informal atmosphere which allowed for discussion and shared experiences. The workshop presenters were familiar with the environment within which the staff worked and with the needs of the service users and were therefore able to make the material immediately relevant.

At the end of the day participants were provided with a written summary of each session together with details of how to contact and make referrals to the multidisciplinary team.

Evaluation Participants were given a questionnaire before the first session of the workshop to

assess the extent of their knowledge and this was repeated after the final session. The assessments were directly related to the learning outcomes for the wurse and used a multiple choice format. These questionnaires had been piloted by staff from a similar unit and minor amendments made in the light of their comments. At the end of the workshop participants were also asked to complete their evaluation of the wurse. The knowledge assessment was repeated six months after the course, when questionnaires were completed in the workplace.

For the evaluation of practice, each participant was video recorded feeding a client immediately prior to the workshop and again between four and six months later. This time interval between the workshop and the second episode served a dual purpose, i.e. to allow for individual referrals to be made to the relevant professionals and the resulting recommendations to be implemented and to give some idea of the medium-term sustainability of the results of the training. Staff within the units were provided with written guidelines and recorded their own mealtime videos. They were also asked to provide information about the content of the meal and their perceptions of how typical the client’s behaviour had been with any other relevant information.

Results

Participant evaluations These were overwhelmingly positive for both workshops, with an overall

satisfaction rating of 89.3%. In response to the question ‘What did you find most usehl?’ several people responded ‘Everything’. Most participants felt that they had sufficient opportunity for discussion and questions and suggestions for additions to the content largely centred around more advice on specific areas of difficulty. The practical sessions were particularly appreciated.

Knowledge assessments Twenty questionnaires were completed before and immediately after the course but

by the time of the follow-up assessment six months later, six staff were not available to complete the questionnaire (either through sickness or having left the unit). One of these staff was from the residential unit and five from the day centre. The mean score for the pre-course assessment was 65.5% which rose to 82% immediately post-training. After six months the mean score remained high at 83.5%, a slight gain from the immediate post- course assessment. Statistical analysis was carried out using paired sample t-tests, comparing the pre-course scores with each of the post-test scores. Results showed a

41 2 People with learning disabilities

highly significant difference both between the pre-course and immediate post-course responses (t = 5.73, df = 19, p < 0.001), and between the pre-course and six months post- course scores (t = 4.69, df = 13, p < 0,001). A one-way analysis of variance was carried out to determine whether there were any significant difference in scores between the two groups of staff, The results were not significant.

Change of practice Fourteen recordings were made prior to the workshop, four in the residential unit

and ten in the day centre. Each client had a mealtime filmed. Some recordings involved two members of staff, one giving a first course and the other a pudding. There were eight follow-up videos, the difference in number being due to staff changes and sickness as with the questionnaires. As far as possible staff were recorded with the same client but this was not always practicable. The view of the clinical professionals involved in the training was that it should be regarded as ‘whole unit training’ and so all staff should be able to meet the needs of each client in their unit. On the whole the quality of the videos was satisfactory and the guidelines had been followed but in a small number the light was poor making it difficult to see details of the food, facial expression, etc. Meals at the day centre take place in a large dining hall and extraneous noise sometimes made it difficult to hear interaction between staff and client.

A checklist was compiled which reflected both the content and learning outcomes of the workshop and essentials for good practice (Burge 1994, Rogers et al. 1994, Pope 1997). It consisted of seven main sections (sensory impairment, positioning, equipment, food, eating process, communication and environment) each with a number of sub- headings. A simple three-point scale was used to rate each of the items on the list and detailed criteria for scoring were established. The checklist and criteria were piloted using other videos and appropriate changes were made before being used for this study.

It was decided that the different sections of the videos would be rated by the professionals with the most appropriate knowledge and expertise in that particular area so sensory impairment, eating process and communication were assessed by speech and language therapists, food by dietitians and positioning, equipment and the environment jointly by physiotherapy and occupational therapy. Rating of each section was carried out by small groups, some of whom had been part of the training team and some with other relevant experience. For example, three speech and language therapists took part, one with extensive experience of learning disability and two with dysphagia training and some learning disability experience. Inter-rater reliability was calculated using Cohen’s Kappa and levels of agreement of between 0.70 to 0.90 were achieved. Robson (1993) states that a Kappa of 0.60 to 0.75 is ‘good’ and above 0.75 is ‘excellent’. The individual videos were re-recorded in random order by someone not otherwise involved in the study, to avoid any bias by the raters.

Scores for each video were then totalled and pre- and post-course scores compared. There was in fact a slight decrease in scores from 44.3% pre-training to 44.1% post- training, indicating no effect of training on practice. These results were analysed using a Wilcoxon matched-pairs signed-ranks test and showed no significance (z = 0.1529, p > 0.05). A one-way analysis of variance to test for differences between staff groups showed no significance.

People with learning dirabilities 41 3

Discussion

On two of the three evaluation measures, the training workshops achieved their aim of providing staff working with people who have extensive mealtime support needs with increased knowledge and skills. Participants not only enjoyed the training but felt that it provided them appropriate learning for the needs of their job. This was confirmed by the gains in knowledge which were significant and persisted over time. Although there was slight uncertainty about whether the follow-up questionnaires were completed independently (as this took place in the workplace without outside supervision) it seems unlikely that results could have been affected substantially by collusion.

However in marked contrast to the other results, the third level of evaluation (and arguably the most important) i.e. of changes in practice which benefit clients, showed no evidence of effectiveness.

This raises the question of why this training seems to be highly effective in increasing knowledge and yet ineffective in translating that knowledge into practice. Of the many possible explanations, two seem to be particularly relevant. There is the possibility that the checklist as an evaluation tool was insufficiently sensitive to detect any changes that did occur, which a wider rating scale, of five or perhaps ten points might have. However the ratings did show individual changes of up to 20% but these were not always post-course improvements. In some cases these were decreases in scores after training. It could also be argued that only one meal for each cliendstaff pair before and after the workshop did not give a true picture. In some cases the behaviour of either clients or staff may not have been typical. Although a greater amount of video information would undoubtedly have been useful, the study was reliant on the goodwill of the staff to take part in the videos, an extra time-consuming activity, and it would have been unreasonable to expect them to do more recording, quite apart from the additional commitment required from the raters.

An alternative explanation is that the checklist and rating scale accurately reflected the reality of the situation and that there was no measurable overall improvement in practice. There is extensive evidence to suggest that training on its own has little effect on staff behaviour (Cullen 1988, Axtell et al. 1997). Landesman-Dwyer and Knowles (1987) stress the importance of an ecological approach where the working environment enables and supports the changes in practice. The training programme in this study attempted to take account of this by ensuring that team leaders and managers also took part in the training but maybe more attention to the environment, over a longer period, is required. Porterfield (1987) suggests that improvements in staff practice are best maintained by positive monitoring where appropriate feedback is provided by supervisors and managers.

Conclusion

This study was an attempt by a group of professionals to measure the outcomes of a collaborative approach to staff training. The subject of the training, i.e. managing eating, drinking and swallowing difficulties, is crucial to the health and quality of life of people with severe learning and physical disabilities. Although the training achieved significant gains in staff knowledge, the problem remains of how to make positive changes in staff behaviour. This is an issue which affects clinical practice both within and across professional groups wherever direct, day-to-day intervention is provided by others.

41 4 People with learning disabilities

The checklist used to rate the videos could be developed further as a useful tool for multidisciplinary assessment and guide for intervention and detailed study of the videos of practice may provide useful data for the development of future training. In the longer term other indicators of the success of the training would lie in improvements in health and nutrition for clients and appropriateness of referrals to the multidisciplinary team.

Acknowledgements

The authors wish to thank the clients and staff who took part in the study, together with Chns Chandler, Lucie Crompton, Robin Jenkins, Tanya Klopper and Elizabeth Sanday for their contributions to the project.

References

Axtell, C. M., Maitlis, S. and Yearta, S. K., 1997, Predicting immediate and longer-term transfer of training. Personnel Review, 26, 20 1-2 13.

Bennett, L., 1994, Staff - A Valuable Resource. Guest Editorial: British Institute of Learning Disabilities Bulletin, 95, December, 1-3.

Burge, P., 1994, Textured soft diets and feedmg techniques among the elderly mentally ill. Journal ofHuman Nutrition and Dietetics, 7, 191-198.

Cullen, C., 1988, A review of staff training: the emperor’s old clothes. The Irish Journal ofPsychology, 9, 309-323.

Hotaling, D. L., 1990, Adapting the mealtime environment: setting the stage for eating. Dysphagia, 5 , 77-83.

Knowles, M. S . , 1980, The Modern Practice of Adult Education: From Pedagogy to Andragogy (New York: Cambridge Books), 2nd edition

Landesman-Dwyer, S. and Knowles, M., 1987, Ecological analysis of staff training in residential settings. In Staff Training In Mental Handicap, edited by J. Hogg and P. Mittler (London: Croom Helm)

Pope, P., 1997, Management of the physical condition in people with chronic and severe neurological disabilities living in the community. Physiotherapy, 83, 1 16-122.

Porterfield, J., 1987, Positive Monitoring: A Method of Supporting Staff and Improving Services for People with Learning Disabilities, (Kidderminster: BIMH Publications)

Robson, C., 1993, Real World Research. (Oxford: Blackwell Publishers Ltd) Rogers, B., Stratton, P., Msall, B., Andres, M., Champlain, M. K., Koerner, P. and

Piazza, J., 1994, Long term management strategies of tracheal aspiration in adults with developmental disablities. American Journal of Mental Retardation. 4, 490- 498.