echo report 2015-6 max - european commission · tyc transforming your care usa united states of...
TRANSCRIPT
1
Evaluation of Project ECHO (Extension for Community Healthcare Outcomes) Northern
Ireland programme 2015-6
Report for Health and Social Care Board
May 2016
2
Contents
TITLE PAGE
NUMBER
Contents 2
Abbreviations 3
Executive summary 4
Acknowledgements and Contributors 7
Chapter 1 – Introduction 9
Chapter 2 - Methods for Evaluation of ECHO with Healthcare Professionals 12
Chapter 3 - Diabetes ECHO 15
Chapter 4 - Optometry ECHO 33
Chapter 5 - Nursing Homes ECHO 55
Chapter 6 - Dermatology for GP trainees ECHO 72
Chapter 7 - Carers support ECHO 92
Chapter 8 - Hub feedback 112
Chapter 9 – Overall Discussion 118
Chapter 10 – Recommendation Summary 124
References 125
3
Abbreviations
AMD Age-related Macular Degeneration
DHSSPS Department of Health, Social Services and Public Safety
ECHO Extension for Community Healthcare Outcomes
GP General Practitioner
GPST GP Speciality Trainee
GPwSI GP with a special interest
HCP Healthcare Professional
HSC Health and Social Care
ICP Integrated Care Partnership
NH Nursing Home
NI Northern Ireland
NIH Northern Ireland Hospice
NIMDTA
OCT
Northern Ireland Medical and Dental Training Agency
Optical Coherence Tomography
PIL Participant Information Leaflet
TYC Transforming Your Care
USA United States of America
4
Executive Summary
Introduction
Project ECHO (Extension for Community Healthcare Outcomes) uses tele-conferencing
technology to improve access to specialised care through supporting and training primary
health care professionals (HCPs) remotely (at ‘spokes’) from a centralised ‘hub’ of experts.
ECHO has been proven to improve care across the United States, and a pilot study in
community hospice nurses in Northern Ireland (NI) showed an improvement in knowledge
and self-efficacy of HCPs.(1) In order to determine if ECHO would be effective in other
contexts in NI, the HSC funded five ECHOs knowledge networks: Diabetes, Optometry,
Nursing Homes, Dermatology for GP trainees, and one for supporting carers of palliative
care patients. This report includes the results of the evaluation of the project and
recommendations for the future use of ECHO in NI.
Methods
The four ECHOs with Healthcare Professionals (HCPs) were evaluated using similar
methods. A pre ECHO knowledge and self-efficacy assessment was undertaken, along with
collection of demographic data for spoke participants. At the end of each ECHO program, a
post ECHO knowledge and self-efficacy assessment was undertaken by spoke participants,
to determine if there was a change over the period of the project. They were also asked to
complete a retrospective-pretest evaluation of self-efficacy (i.e. how competent and
confident do they feel they were before the ECHO project with the benefit of hindsight). A
survey of all participants’ (hub and spokes) views on ECHO in general was performed, and
focus groups were held with spoke members of each ECHO network.
The carers support ECHO was evaluated using a questionnaire of all participants, and a
focus group of hub members.
Results
Two networks (optometry and dermatology for GP trainees) demonstrated a statistically
significant improvement in knowledge and self-efficacy. The other two clinical networks
(nursing homes and diabetes), while they demonstrated an improvement in knowledge and
self-efficacy, the response rates were too low for statistical analysis. All four networks with
HCPs demonstrated very positive views towards ECHO and the education and support that
it provided, both through a questionnaire and through focus groups of spoke members. Both
hub and spoke members valued being involved in ECHO networks, and all would participate
again in ECHO if given the opportunity. All said it had improved the care they provided for
patients.
5
Due to difficulties with recruitment and retention, only four carers participated in the carers
program. While feedback from the carers was positive, and all said they would participate
again and recommend the program to others, half stated that they would prefer a face-to-
face carers service. Hub members of the carers ECHO identified that it was a good forum
for information sharing, but that it lacked the face-to-face support that carers can particularly
benefit from. They identified issues with recruitment and some strategies that may be helpful
for future carers ECHOs.
Conclusion
The quantitative and qualitative findings from this study support the use of ECHO in Northern
Ireland for healthcare professionals by demonstrating statistically significant improvements in
knowledge and self-efficacy in two networks, and positive feedback across all the networks
replicating findings in other ECHO knowledge networks in the US setting,(2-4) and one pilot
study in Northern Ireland.(1) Due to low numbers secondary to issues with recruitment and
retention of carers, it is not possible to draw firm conclusions on the usefulness of ECHO for
carers and further research is needed.
As a low-cost high-impact model, ECHO can be adapted to meet the needs and resources of
different communities and populations. At a time when health care providers are under
mounting pressure to do more and spend less, this model provides an affordable solution to
addressing growing need in the UK in training and supporting healthcare professionals.
Further research is needed to look at the impact on patient care and service delivery.
Suggestions from evaluation
• ECHO should continue to be developed and implemented to help educate healthcare
professionals across Northern Ireland in a cost effective manner
• Adequate funding is necessary to allow protected time for all to participate, both at
the hub and the spokes
• The importance of the skills of the facilitator was a recurring theme, and emphasises
the need for good quality facilitation training and supervision to ensure networks
exhibit fidelity to the ECHO model
• Future ECHO networks should continue to be evaluated to ensure they meet the
needs of the population they are trying to educate and support
• To improve the response rates in future evaluations, funding for individual practices
or participants could be dependent on participating in the evaluation process. This
should therefore reduce the bias of a partial response, and improve the
generalisability and hence usefulness of the results
6
• Further studies are required to determine if the ECHO model improves patient care
and reduces costs e.g. through reduced referral rates, improved quality of life, and
better patient and staff outcomes
• Minor technological issues should be addressed to make connectivity easy and not
detract from the learning environment. Issues of not being able to connect from
some sites due to HSC security policies also need addressed
• Ensuring that preparatory work is available in good time for the spoke members, and
that additional online resources are easily accessible were highlighted as contributing
to the overall benefits associated with being part of an ECHO network
• More research is needed to determine the usefulness of a carers ECHO network
The suggestions made in this report could contribute to designing a future Carers
ECHO and through further evaluation determine more conclusively if the ECHO
format is a useful methodology in providing information and support to carers in a
cost effective manner.
7
Acknowledgements
This project could not have been completed without the enthusiastic support and
commitment of all the teams who were involved in the setting up and running of the five
ECHO clinics in a short period of time. They, along with the Education Department at the
Northern Ireland Hospice, established and maintained the ECHO clinics throughout the
project. Their commitment to providing excellent education to others and hence improving
patient care was outstanding, and the evaluation of the clinics would not have been possible
without their expert involvement in the evaluation assessments.
We are particularly indebted to all the participants who so willingly gave of their time and
effort to participate both in Project ECHO and in its evaluation, and who throughout the
project demonstrated a commitment to patient care which was truly inspiring.
Administration support for the ECHO evaluations through Patricia Marshall, Áine McMullan,
Rebecca Donnelly, Ciara McClements, Tracey McTernaghan and Claire Armstrong was
invaluable.
Our sincere thanks also go to the Health & Social Care Board for funding this study, and in
particular to the HSC ECHO project Board, to Dr Sanjeev Arora and the ECHO team at the
University of New Mexico for ongoing support and encouragement and to the Senior
Management Team at the Hospice for having the vision to invest in the ECHO way of
working.
Contributors to the report
Dr Clare White (Principle Investigator), Consultant Palliative Medicine, N. Ireland Hospice – all aspects of the study Dr Clare McVeigh, Nurse Research Lead, N. Ireland Hospice – focus groups, carers ECHO evaluation and write up Prof Max Watson, Medical Director, N. Ireland Hospice – set up and running of Project ECHO NIH Dr Lynn Dunwoody, School of Psychology, Ulster University, N. Ireland – statistical analysis Administration Support Patricia Marshall, Information Systems Administrator, N. Ireland Hospice - data entry Áine McMullan, Project manager, Project ECHO NI - Coordination of ECHO information
8
Tracey McTernaghan, N. Ireland Hospice – coordination of ECHO data collection and marking Claire Armstrong, N. Ireland Hospice – coordination of ECHO data collection and marking Rebecca Donnelly, Regional ICP Administrative Support, Health and Social Care Board – coordination of ECHO data collection Ciara McClements - Project Manager Transforming Your Palliative and End of Life Care (TYPEOLC) Delivering Choice Programme, Northern Ireland - coordination of ECHO data collection and marking
Dermatology ECHO Dr Nigel Hart, Senior lecturer in General Practice, Queens University Belfast and Associate Director in General Practice, NIMDTA Dr Siobhan McEntee, GP in Glengormley Practice, GP Trainer, GP Program Director with NIMDTA Claire Loughrey, Director of General Practice, NIMDTA Optometry ECHO Mr Raymond Curran, Head of Ophthalmic Services, Health and Social Care Board, Belfast Mrs Margaret McMullan, Clinical Ophthalmic Adviser, Health and Social Care Board, Belfast Mr Michael Williams, Senior Lecturer, Centre for Medical Education, Queen's University Belfast | Honorary Consultant, Medical Ophthalmology, Belfast Health and Social Care Trust Prof Augusto Azuara-Blanco, Clinical Professor Centre for Vision and Vascular Science, Queen’s University Belfast | Consultant Ophthalmologist, Belfast Health and Social Care Trust Diabetes ECHO Dr Roy Harper, Consultant Endocrinologist, South Eastern Trust/ HSCB Dr Neil Black, Consultant Endocrinologist, Western Trust Nursing Home ECHO Sue Foster, Head of Education, Northern Ireland Hospice Corrina Grimes, Allied Health Professinal Consultant, Public Health Agency
9
Chapter 1 - Introduction
Accessible education for healthcare professionals (HCPs) is essential if they are to be
enabled to provide high quality care in a rapidly evolving health care environment. This is
particularly challenging within a community context as practitioners may work in isolated
settings, with varied access to educational opportunities and peer review of practice. With
the constraints of limited resources and the isolated setting in which many HCPs work, new
innovative or creative approaches to education and mentorship that facilitate individual and
collective learning and changes in practice are required.
One such approach is to use the structure of ECHO (Extension for Community Healthcare
Outcomes), which was developed by University of New Mexico and uses point to point
video-conferencing technology involving a central specialist ‘hub’ linking with primary care
providers remotely at ‘spokes’. ECHO uses a collaborative model of medical education and
clinical support, and aims to empower and equip HCPs to provide better care to more
people, right where they live.(5) Participants in the primary care setting (at the spokes)
receive evidence-based or best practice guidance from specialists at the hub, case-based
learning from peer-presentations and have opportunity for live questions and answers. The
ECHO model does not provide direct patient care, but through training using real life cases
provides front-line HCPs with the knowledge and support to manage similar patients with
complex conditions. It does this by engaging HCPs in a sustained learning system and
partnering them with specialist mentors to form a community of practice.(5) It is therefore
different from telemedicine (picture 1), which aims to treat patients directly. Instead it aims
to ‘multiply’ knowledge through educating others to treat more patients than could ever be
directly cared for by one individual HCP. Through ECHO there is opportunity to quickly
translate new knowledge into practice, and thus improve outcomes for patients in more
remote settings.
ECHO is now used across the United States of America (USA) for 45 different disease and
health conditions including hepatitis C, diabetes, asthma, pain management and
rheumatology, and has been shown to improve patient outcomes.(2-4) The impact on such
outcomes, and the reduction in waiting lists at central specialist clinics have been key in
promoting its widespread uptake in health systems which are struggling to meet the needs of
patients living far from central services.(2-4) Indeed the Senate in the USA has just passed a
bill supporting the widespread use of ECHO across the country.
10
Picture 1 ECHO vs. Telemedicine
ECHO uses cloud based ‘Zoom’ software which is compliant with encryption standards and
which allows staff to connect with ECHO sessions using standard computers, laptops,
tablets and hand held devices. Zoom allows for up to 100 spokes to join a single ECHO
session and does not require expensive dedicated telemedicine equipment though is
dependent on a reliable broadband connection.
In 2014 the Northern Ireland Hospice (NIH) undertook a six month pilot project of ECHO with
its community hospice nurses, and the evaluation showed statistically significant
improvements in knowledge and self-efficacy(1); 96% recorded gains in learning, and 90%
felt that ECHO had improved the care they provided for patients; 83% would recommend
ECHO to other healthcare professionals (HCPs); 70% stated the technology used in ECHO
had given them access to education that would have been hard to access due to
geography.(1) This was the first evaluation of ECHO in the UK and Europe.
In 2015 the Health and Social Care (HSC) Board received funding of £403k from the
Executive Change Fund in 2015-16 and worked in partnership with NIH to pilot the use of
the ECHO model across the HSC to determine if the successes of ECHO in the USA could
be replicated in Northern Ireland. The project pilot period was from October 2015 until 31st
March 2016.
11
There were approximately 40 ECHO sessions set up (November 2015 – March 2016)
across the following Knowledge Networks using the hub and spoke model:
• Diabetes for GPs
• Optometry for Optometrists
• Palliative Care and Quality Improvement for Nursing Homes (26 Nursing Homes
involving over 90 healthcare professionals)
• Dermatology for GP Trainees
• Support for Carers of Patients with Palliative Care needs
Each of these ECHO knowledge networks was evaluated to help inform decisions around
the future of ECHO in Northern Ireland. The ECHO knowledge networks are described in
turn along with the results of their evaluation.
12
Chapter 2 - Methods for Evaluation of ECHO with Healthcare Professionals
These methods relate to the evaluation for each of the four clinical ECHO networks with
HCPs- Diabetes, Optometry, Nursing Homes and Dermatology for GP trainees. The carers
support evaluation is described separately as it required a different evaluation strategy.
Aims and Objectives
The aim of the study was to evaluate the use of ECHO Knowledge Networks for HCPs
working in Northern Ireland.
In order to do this, the study objectives were:
• To determine if the use of ECHO improves HCPs knowledge and self-rated skills and
confidence (self-efficacy) in the management of patients
• To explore the HCPs experience and perceived usefulness of ECHO in meeting
knowledge and support needs
• To determine if hub participants benefited from participating in ECHO
Study Design
A prospective longitudinal cohort study was undertaken for each ECHO, using a combination
of qualitative and quantitative methods. The study comprised of two stages:
Stage 1: Baseline assessment – prior to ECHO commencement for spoke participants
Stage 2: End of ECHO assessment for hub and spoke participants.
All ECHO evaluations, with the exception of the Dermatology for GP trainees, were run by
an independent evaluation team from the NIH research department. Assessment tools were
specifically written for each particular ECHO, but adapted from a master format. The creation
of the assessment tools appropriate to each network was undertaken by the clinical lead
running that network.
The Dermatology ECHO for GP trainees evaluation was run by the hub members (Dr Hart
and Dr McEntee) and the evaluation team from NIH was only involved in obtaining general
feedback on ECHO from participants and through running the focus group.
STAGE 1 –Baseline data collection prior to ECHO commencement:
Spoke participants: Each HCP participating in the ‘spokes’ (or community) was asked to
complete the following tools, based on the research aims for the study:
1. A confidential self-efficacy tool focusing on participant confidence in managing different
situations that they face in delivering the relevant service in their particular clinical area.
13
2. A written knowledge assessment. This was developed by each individual ECHO clinical
leads team with input from the research team as required.
3. Demographic data for all participating HCPs included gender, age, profession, area
worked in (rural / urban), and length of time working in their particular area.
The tools took a maximum of 60 minutes to complete and were undertaken prior to
commencement of the ECHO network, either at a training day or they were sent out to
participants and returned once completed. Participants’ names were not recorded on the
tool; instead each participant was allocated a unique code that was only identifiable to the
administration team. Each participant was provided with information about the evaluation
using a participant information leaflet (PIL). Participation in the evaluation was considered
an essential requirement of being involved in the ECHO, and hence if participants did not
wish to participate in the evaluation the intention was that they would be unable to
participate in the ECHO network. Consent to take part in the evaluation was presumed on
completion and return of the questionnaires.
STAGE 2- After ECHO Knowledge Network completed:
Spoke participants - at the end of the ECHO project all HCPs at the spokes were asked to
complete the following survey data:
• Questionnaire of self-efficacy to determine if there was a change over the period of
the project. They were also asked to complete a retrospective-pretest evaluation of
self-efficacy (i.e. how competent and confident do they feel they were before the
ECHO project with the benefit of hindsight). This was used to try to reduce the bias
of self-evaluation and facilitate HCPs to be more objective in assessing the impact of
ECHO on self-efficacy.(6, 7) The pre-test, post project and retro-pretest
questionnaires were all compared using a unique identifier code
• Knowledge assessment (the same as pre-ECHO) with results being compared with
their pre-ECHO assessment
• Questions relating to participants overall views on ECHO and the network they were
involved in.
Hub participants - All hub participants were asked to complete a questionnaire via Survey
Monkey®(8) of their experiences of participating in an ECHO network.
Focus Groups
To address the second objective of exploring the HCPs experience and perceived
usefulness of ECHO in meeting knowledge and support needs, focus groups were
14
undertaken with the HCPs who had attended the ECHO programme at the spokes and who
indicated a willingness to participate in a focus group discussion. All spoke participants who
participated in ECHO received an invitation letter via email requesting if they would be willing
to participate as well as a PIL. The intention of this stage of the study was to provide greater
insight into the perceptions and experiences of participating in the ECHO knowledge
networks, problems encountered and benefits, and whether or not it would be useful to
continue with the programme. A focus group schedule, based on the aims and objectives of
the project was used to elicit information. There was a cooling off period of a minimum seven
days between receiving the letter inviting participation, the PIL and participating.
With the participant’s permission, the discussion was audio-taped and supplemented by field
notes. Participants’ names were not recorded on the tape; instead each participant was
allocated a unique code that was only be identifiable to the researcher. The groups were
facilitated by a researcher who was not involved in the running of ECHO using a structured
framework. Focus groups were conducted using the ECHO technology following the final
ECHO session or at another pre-arranged time.
Inclusion Criteria
All HCPs participating in the ECHO programme were invited to participate in the relevant
aspects of the study.
Governance and Consent
Approval was granted from N. Ireland Hospice / Ulster University research governance
committee. Consent to take part in the study was presumed on completion and return of the
questionnaires. Formal consent was taken for the focus groups. No patients were involved
in this aspect of the evaluation, only HCPs who were assured that this evaluation was about
evaluating ECHO, not about evaluating HCPs and their individual knowledge and practice.
Data analysis - Due to the small sample, descriptive statistics and where possible, non-
parametric tests (Friedman and Wilcoxon) were used to summarise the participant
characteristics and survey data. The data from the focus group was analysed by adopting a
thematic analysis framework described by King and Horrocks (2010). (9) This thematic
analysis approach allowed the investigator to be flexible in their interpretation and
development of the themes, and not to be confined by a rigid analytical framework.
15
Chapter 3- Diabetes ECHO
Background
There are over 83,000 people living with diabetes in Northern Ireland with rates of
approximately 5% of the general population. Approximately 90% of people living with
diabetes have type two diabetes, the prevalence of which is rising rapidly. Diabetes care is
delivered in both primary and secondary settings, and in NI approximately two-thirds of
people are looked after solely by primary care through Practice Nurses and GPs.
Historically a larger proportion of people in Northern Ireland are looked after in secondary
care when compared to other areas in the UK. GPs and Practice Nurses are experienced
professionals, but their confidence and knowledge base in diabetes care varies from practice
to practice. It is estimated that 10% of the healthcare budget is spent on delivering diabetes
care including the management of complications from diabetes. Diabetes is therefore a key
target for the HSC and is one of the priority conditions on which Transforming your Care
(TYC) and Integrated Care Partnership (ICP) initiatives focus.
Diabetic patients need multidisciplinary input to deliver all the care that is needed. This is
appreciated to varying degrees in different care settings. It is also a complex therapeutic
area which continues to change rapidly, so it is hard to keep up to date particularly if it is not
the clinician’s main clinical area. These factors can decrease confidence in using new
therapies to manage patients. This can contribute towards therapeutic inertia or encourage
premature referral to speciality care, which results in overloading of such services when the
problem could be dealt with without attending a secondary care clinic. The converse is that
delay in an appropriate referral can lead to an increased complication risk. Primary care
professionals therefore need supported and educated to manage patients appropriately in
the primary care setting, and to know when to refer to secondary care.
The Intervention
The Diabetes ECHO trained and supported primary care providers, GPs and practice
nurses, to improve their knowledge and skills in the management of patients with diabetes.
The ECHOs were held weekly on a Wednesday afternoon from 1.30-3.00pm and covered a
curriculum devised by spoke participants including diagnosis, appropriate use of Type 2
agents, use of insulin, and management of diabetes at the end of life. For detail see table 1.
Hub - The hub included diabetologist’s from the regional diabetes strategy group who
expressed an interest in being involved in the ECHO network. There was also an ICP
Clinical Lead with an interest in diabetes, a Trust dietician, a Trust Special Diabetic
Pharmacist and a National Care Advisor from Diabetes UK.
16
Spokes - Involvement in ECHO was opened to all GP’s in the region. Thirteen GP’s, who
were joined by some of their practice nurses and practice based pharmacists took part.
Table 1 Diabetes ECHO Network
Diabetes ECHO Network
Clinical
Champion/Lead
Facilitator
Admin Support
Dr Roy Harper – Diabetologist South Eastern Trust
Dr Neil Black - Diabetologist Western Trust
Dr. Glynis Magee - Diabetologist Southern Trust
Florence Findlay White (Diabetes UK)
Tracey McTernaghan
Frequency of clinics
Weekly and then bi-weekly. 9 sessions in total.
Dates: Dec – 9th,16th/Jan – 13th,20th,27th/Feb – 3rd,17th/Mar –
2nd,16th
Start date Wednesday 9th December
Training Date 25th November 2015 at Jennymount
Hub members
• Barney McCoy (ICP Clinical Lead – South)
• Lesley Hamilton (Western Trust)
• Neil Black (Western Trust)
• Roy Harper (SE Trust/HSCB)
• Brid Farrell (PHA)
• Magee, Glynis (diabetologist – Southern Trust)
• Lynne Thomas (dietician) – SE Trust
• Rosemary Donnelly - Specialist diabetic Pharmacists (SE
Trust)
Hub costs
As per costing schedule
Based on 12 weeks of clinic and 4 hours clinic time (inc prep) and
one session of training
£11,481 paid to the 5 Trusts on 1st December 2015
Spoke members 13 GP’s confirmed (Representing all ICP areas across the region
except West)
Spoke costs £170 for GP practices per session, “backfill”.
Equipment needed
for spokes
Only 4 webcams required as most using their own equipment,
laptop/ipad
17
Curriculum
Curriculum developed at training day.
• Currently, what is the best way to diagnose diabetes in
both adults and children?
• Communicating the diagnosis news to patients and
inspiring change.
• When & how do I start insulin in Type 2 diabetics?
• What Type 2 agents should I be using and when? (Part 1)
• What Type 2 agents should I be using and when? (Part 2)
• The practicalities of Insulin including the ‘sick day rule’.
• What has my diabetic patient been told in their structured
education programme?
• What do I do when my diabetic patient is dying? How do I
recognise this and manage the last year of life?
• Rules – Diabetes & driving
Evaluation Methods
Evaluation was undertaken as described in Chapter two. Participants completed their pre-
test evaluations at the training day, and were emailed their post-test evaluations which were
returned by post or email.
Results – Diabetes ECHO
There were 13 GP sites who participated in ECHO, and while practice nurses joined
intermittently, only the GPs were invited to take part in the evaluation as they were the
primary target of the ECHO and attended consistently. Eight spoke participants completed
the pre ECHO assessments (response rate 61.5%), with six completing the post ECHO
assessments (response rate 46.2%). Demographic data for all participants in the pretest
evaluation are shown in table 2. For the rest of the results, only the participants who
completed the pre and post evaluations are included.
18
Table 2 Diabetes Demographic data Pre ECHO Evaluation
Range Totals
Age 20-30 0
31-40 1
41-50 4
51-60 3
60+ 0
Gender Male 6
Female 2
Work area Urban 3
Rural 3
Mixed 2
Years in practice <5 1
6-10 0
11-20 3
21+ 4
Knowledge and Self-efficacy Assessments
The mean scores of knowledge improved slightly between the pre ECHO and post ECHO
assessments for the six participants. Average knowledge scores improved from 21.63 to
22.86 (out of a possible 40 marks); from 54% to 57%. Two participants’ score dis-improved,
one was the same and three improved. Means and standard deviations are shown in table
3. Due to the low response rate no further statistical analysis was possible. Error bars are
shown in figure 1.
Table 3 Diabetes ECHO Knowledge and total self-efficacy results
Pre- ECHO Post-ECHO Retro-Pre ECHO
Outcome Mean SD Mean SD Mean SD
Knowledge 21.63 3.25 22.86 2.79 - -
Total Self
efficacy
3.55 0.70 4.50 0.33 3.60 0.48
19
Figure 1 Error bars representing the mean and two standard deviations either side of the mean for the diabetes knowledge assessment
Self efficacy results improved (table 3), and a higher average score for the post-test
evaluation in all areas compared with the pretest and retro-pretest evaluation. The different
domains are shown in table 4. Due to the low response rate no further statistical analysis
was possible.
Table 4 – Diabetes ECHO Self efficacy assessment - Participants self-rated confidence in
each area (1 - not confident at all, 5 - very confident)
Question
Pre ECHO average (Range)
Retropre ECHO Average (Range)
Post ECHO average (Range)
Diagnose and classify patients with diabetes 4.38 (3-5)
4.00 (3-5)
4.83 (3-5)
Treat patients with diabetes using optimised treatment regimens
4.00 (2-5)
3.67 (2-5)
4.50 (2-5)
Understand the possible side-effects and limitations of most pharmacological treatments used in diabetes
3.88 (2-5)
3.33 (2-5)
4.33 (2-5)
Initiate and support patients as they use injection therapies
2.63 (1-5)
3.17 (1-5)
4.17 (1-5)
Assess and expertly manage diabetes complications and co-morbidities
3.50 (2-5)
3.33 (2-5)
4.50 (2-5)
Educate and motivate patients with diabetes 3.75 (3-5)
4.00 (3-5)
4.83 (3-5)
Serve as a local expert within my practice and area for diabetes questions and issues
3.38 (1-5)
3.67 (1-5)
4.50 (1-5)
20
General Feedback of Diabetes ECHO
Box 1 demonstrates participant’s views on ECHO in general. The six respondents were very
positive, with 100% having learnt ‘a lot’ through ECHO participation and enjoyed it, 100% felt
it helped translate knowledge into practice more than other teaching sessions they had been
involved in and had improved the care they provided for patients, and 100% would
participate again.
Box 1 General ECHO Diabetes Results
1. Rating on a scale of 1-5 the quality of learning / usefulness from each area (1- poor, 5- excellent)
1 2 3 4 5
Review of previous session 0 0 1 2 3 Presentations 0 0 0 1 5 Case based discussions 0 0 0 1 5
2. Overall do you feel you have learnt through participating in ECHO?
A lot 6
A little 0 NO 0
3. Did you find participating in ECHO enjoyable?
A lot 6
A little 0 No 0
4. Do you think that participating in ECHO has improved the care you provide for patients?
A lot 6
A little 0 No 0
21
5. Do you think the format of ECHO helps translate knowledge from teaching into practice more than other teaching sessions you have been involved in?
Yes 6
No 0
6. Would you recommend ECHO to other healthcare professionals in your area?
Yes 5
No 0
7. Have you used any of the online resources via Moodle, and if yes have you found these useful?
Used and found useful
Used and found NOT useful
Power point presentations 4 0 Video of the teaching
sessions 4 0 Video of case presentations 4 0 Other supporting materials 2 0
8. Regarding ECHO technology…
Agree Disagree Unsure It has given me access to education that would have been hard to access due to geography 4 2 0 It was a good medium to access teaching / education at a different location from where I work 6 0 0 Any technical difficulties were acceptable and did not put me off participating in ECHO 5 0
No issues
Any technical difficulties did not significantly reduce my learning 5 0
No issues
9. How do you rate your overall ECHO experience? (1- poor, 5- excellent)
1 2 3 4 5 0 0 0 0 6
22
10. Would you participate in ECHO sessions in the future if the opportunity arose?
Yes 6
NO 0
Focus Group Results
The focus group was conducted with five of the spoke GPs. Analysis of the focus group data
uncovered two overarching themes, each with their own descriptive and interpretative
themes that are outlined in thematic diagrams (figure 2 and 3). Overarching theme 1, ECHO
Enhanced Clinical Knowledge and Skills, explored the reasons why participants perceived
that ECHO optimised their clinical practice. Interpretative themes 1A and 1B highlighted the
contributing factors that resulted in participant’s perceived enhancement of clinical practice.
Overarching theme 2, Consideration for the Future of ECHO, displayed the key conceptions
that participants perceived should be considered for future ECHOs.
Overarching Theme 1: ECHO Enhanced Clinical Knowledge and Skills
This theme encompassed how participants perceived ECHO to be an education platform
that enhanced their clinical knowledge and skills. Interpretative theme 1A (figure 2) depicted
how participants perceived that ECHO optimised the care they delivered their patients
through the knowledge they had gained, and also through their increased confidence in
caring for patients with diabetes. Interpretative theme 1B (figure 2) illuminated how ECHO
also created a safe learning environment for participants. Findings highlighted that this was
perceived to have been achieved by the encouragement of the hub and the sharing of
knowledge. The learning environment was also enhanced as ECHO was perceived as
providing a platform which met the differing learning styles of participants.
23
Figure 2 Theme 1 ‘ECHO Enhanced Clinical Knowledge and Skills’ for Diabetes ECHO
Descriptive Themes Interpretative Themes Overarching Themes
1 ECHO Enhanced
Clinical Knowledge and Skills
1A Optimised Patient
Care
Increased GPs confidence in their skills when caring for
patients with diabetes
Increased knowledge and skills regarding diagnosing
diabetes
Increased knowledge regarding new treatments
1B Created an effective learning environment
Importance of the Hub
Encouraged sharing of knowledge
Highlighted areas for further learning
Adapted to participants learning styles
24
Figure 3 Theme 2, ‘Consideration for the Future of ECHO’ for Diabetes ECHO
Descriptive Themes Interpretative Themes Overarching Themes
Interpretative Theme 1A: Optimised Patient Care
The majority of participants described how they perceived that the ECHO program had a
positive impact on patient care. Many felt this was due to the impact of ECHO on increasing
their confidence when providing care to patients with diabetes in the primary care setting:
“I think my confidence in dealing with some patients with diabetes has risen significantly.”
(DIA/12)
For some, this was achieved through collaborative working at the ECHO clinics:
2 Consideration for
the Future of ECHO
2A There needs to be
funding
Funding allows protected time
Funding ensures quality teaching
2B Furture ECHO
participants
Team approach at the spokes
Future hub members
2C Technological considerations
"Technical hitches"
What worked well
25
“It really raised my confidence in dealing with diabetic patients. I really enjoyed sharing
cases with other GPs and realising that I wasn't on my own when I got stuck as to where to
go next, and it just reassured me and made me feel more confident in taking the next step in
treating my diabetic patients.” (DIA/14)
“It's a confidence thing. It's really improving everybody's confidence in how they manage
these patients and bringing our expertise up a little bit more and helping to understand how
other people deal with their problem cases that we have, which is extremely important for all
of us.” (DIA/15)
One participant felt that ECHO would also be beneficial to his colleagues in helping build
their confidence in managing patients with diabetes:
“Some of my other partners tend to be very reluctant (regards diabetes)…. anybody who has
a blood sugar above 7 tends to get landed at my door, but I would encourage my partners to
take part in something like this, which would help.” (DIA/15)
One participant perceived that the ECHO program had increased their knowledge and skills
in relation to diagnosing the type of diabetes a patient has:
“There were quite a few things I learned from this. I thought it was useful to go through the
diagnosis of diabetes, which at first glance seems relatively straightforward but in a small
number of cases can be difficult and different, and looking at things outside of the normal
Type 1 Type 2 diabetes was useful and made me think again about categorising people with
diabetes.” (DIA/05)
Some participants also illuminated that the ECHO clinics had increased their awareness and
confidence in relation to considering new treatment methods:
“I thought all the presentations were useful but I thought it was particularly useful, the
discussions around some of the newer drugs and how they can be used effectively.”
(DIA/05)
“I think the thing for me was where to go after you’d started a couple of different types of
drugs and you’re thinking about referring the patient for consideration of insulin and
becoming more comfortable with some of the newer drugs, the sodium-glucose co-
transporter 2 (SGLT2) inhibitors, and things like that, and certainly it has helped with that
significantly.” (DIA/12)
26
Overall, these findings have highlighted how the learning gained through the ECHO platform
enhanced the confidence and clinical skills of the participants. This was perceived to
enhance patient care due to raised awareness amongst participants on how to optimally
diagnose and treat patients with type 1 and type 2 diabetes.
Interpretative Theme 1B: Created an effective learning environment
Many participants expressed that the enthusiasm and encouragement at the hub added to
the positive learning environment provided through the ECHO platform. Many felt that this
positively attributed to the success of the program:
“The enthusiasm at the hub is really, really important and I think on this occasion was
extremely successful.” (DIA/15)
“There was great enthusiasm from the hub, which sort of encouraged and motivated
everybody, and I think that was extremely useful and helpful.” (DIA/15)
Findings additionally illuminated that participants found the multi-professional presence at
the hub beneficial:
“The other point I would add, which I thought was particularly good, was the multi
professional aspect to it, the fact that we had not just GPs and consultants but there were
diabetic specialist nurses, dieticians and others there. So there was a good breadth of
experience and complementary knowledge sets and experience sets there.” (DIA/05)
“I think just to emphasise what was said earlier about the multidisciplinary approach, I think
having groups of people who are coming at it from a different angle really means that you're
sharing a lot more information. Having the diabetic specialist nurses and the dieticians and
the consultants and GPs and the practice nurses, all in the one group, really enriched the
experience and really helped us all to appreciate where others are coming from.” (DIA/05)
Findings also highlighted how participants perceived ECHO as providing an effective
learning environment through encouraging the sharing of knowledge and experiences:
“One of the big benefits for me has been hearing what my colleagues [the spokes] are doing
and realising that I can be a little bit more adventurous with some things and that I can be a
little less adventurous with other things, and actually realising that we’re pretty much doing
much the same stuff, maybe approaching it in slightly different ways but very definitely doing
very similar things.” (DIA/12)
27
One participant highlighted that the ECHO experience had highlighted areas for further
learning:
“For me it’s highlighted a lot I don't know, really it’s highlighted areas within diabetes that I
realise actually I need to do a lot more reading in that area, and then it’s reinforced other
areas where I feel more comfortable. But it certainly highlighted a lot of things that I didn’t
know, that I didn’t know.” (DIA/05)
The majority of participants conveyed that ECHO provided and effective learning
environment due to the formats ability to adapt to their learning styles. This perception
resulted from the mixture of didactic and experiential learning provided at the clinics:
“In terms of the content, I thought the content was very good. It was well presented and
there were lots of new information and lots of experiential learning, as well as factual
learning.” (DIA/05)
“The presentations, I thought, were very good, very thought provoking and encouraged me
to read around them. But I got the most out of our discussions. Our case discussions, I think,
were extremely well thought through.” (DIA/15)
One participant also illuminated that they benefited from the interaction with other clinicians:
“The way I like to learn is interaction with people, so this is very easily accessible interaction
with other colleagues and with consultants and other specialists. So I was very grateful for
the experience.” (DIA/14)
Accounts demonstrated how the ECHO platform created an effective learning environment
for GPs with a special interest in diabetes. This was achieved by the presence of an
encouraging hub and an environment that nurtured the sharing of knowledge amongst
clinicians. The learning environment was also optimised as the platform lended itself to
meeting the varying learning styles of participants and providing them with the opportunity to
further consider their clinical practice.
28
Overarching Theme 2: Consideration for the Future of ECHO
Findings were indicative of the need for further considerations when providing future
ECHOs. This was conveyed though participant’s views on the need for appropriate funding
to participate in ECHO and who should participate in ECHO, both at the hub and the spokes.
Findings also illuminated the technological considerations that may be needed to enhance
the ECHO experience. (Figure 3)
Interpretative Theme 2A: There needs to be funding
Participants conveyed that for ECHO to successful in the future, there needs to be adequate
funding. The majority of participants highlighted the need for protected time to be funded for
future ECHOs:
“For this particular set of 10 [current ECHO clinics] we all got payment for it so we were able
to protect the time, we were able to set aside the time to do it. But if this was an ongoing
thing and we were looking at doing it once a fortnight, I suspect that the enthusiasm might
wane as people realise that life is very busy and if I'm not being funded to provide backfill for
this how am I possibly going to keep it going?”(DIA/12)
“One of my concerns for the future of it, is how do we manage to get protected time in what
is a very busy day, a very busy week, in order to get a group of people together? Time is a
big factor here for all of us in general practice and as it is obviously in hospital as well, and
the funding does give us protected time.” (DIA/12)
One participant also highlighted the need for appropriate funding to ensure the teaching
remains of a good quality:
“What it necessitates though [funding] is that the teaching is of high quality and it is really
worthwhile.” (DIA/15)
In summary, the presence of appropriate funding is needed to facilitate future ECHOs. This
is perceived to be needed not just to protect the time of clinicians to participate, but also to
ensure the high standard of education provided is being maintained.
Interpretative Theme 2B: Future ECHO Participants
Findings also conveyed perceptions of who should participate in future ECHOs to optimise
the learning gained. One participant highlighted that having more than one team member at
each spoke would be of benefit to clinical practice:
29
“There was one session where my practice nurse joined me, and that was useful because
we were able to follow and complete the discussion outside of the ECHO session and it
meant that the learning was much more widespread within the practice. And I think that is
something I would encourage for the future, is we try and have teams of people meeting
together, not just one individual from the team, and I think that if you can create enthusiasm
in our practice then the standard of care will rise much higher.” (DIA/12)
Participants also illuminated their thoughts on who should be present at the hub for future
ECHOs. One participant conveyed how the presence of other specialist clinicians who
advise on conditions related to diabetes, would benefit the hub:
“Maybe also to consider other specialties with regards, for example... there's so much, foot
care, renal disease etc. obviously as part of the programme for a future date would be
helpful as well.” (DIA/15)
Accounts displayed how membership at the hub and spokes must be carefully considered to
ensure optimal multi-professional collaboration. It is also perceived that increased
membership at each spoke will increase the impact of learning and thus optimise patient
care.
Interpretative Theme 2C: Technological considerations
Findings were indicative of the need to consider the potential technological needs of
participants for future ECHOs. Some participants conveyed that technical issues from the
hub were often encountered:
“I thought the volume sometimes at the hub, or hearing what they were saying clearly at the
hub was difficult at my end, that may have been our problem at our computer end, and every
time the speaker [facilitator or teacher] moved or turned their head I seemed to lose the
sound quite a bit. (DIA/15)
“The sound in the hub, occasionally it would drop, again, as people were turning their head,
or it wasn’t the person directly in front of the camera, if it was someone else who was
contributing.” (DIA/12)
One participant expressed that they found it difficult to connect to ECHO through Wi-Fi in
their workplace:
30
“One day I had to work on Wednesday and I tried to link in from work, but we don't have Wi-
Fi and the work computer wasn't set up, I didn't have the speaker and the headphones. It
was my own fault. I tried to bring my laptop in but couldn't connect up because the laptop
can't connect to work. So issues with NHS or GP connection that was all.” (DIA/14)
However, the technology was also considered as promoting optimal participation in ECHO
due to the flexibility it provided:
“The other thing just to say is I set myself a target of trying to be in a different location for
every session, and the car was parked in different car parks around Northern Ireland, so 3G,
4G signal was perfectly adequate, which I didn't expect it to be. I actually expected it to drop
out a lot more than it did. So I think the technology worked really well.” (DIA/14)
One participant perceived however that the success of the technology can often be
attributed to initial installation of the equipment and the ongoing administrative support:
“I think just the initial setting up of the various places where you're going to do ECHO is very,
very important and have shadow runs prior to the actual meetings to make sure everybody is
up and running would be helpful and less stressful. The ECHO administrator did a great job
[in coordinating these], I have to say.” (DIA/15)
Discussion
While the knowledge assessment for the diabetes ECHO did not demonstrate a substantial
improvement in knowledge scores and no statistical analysis was possible due to low
numbers, self-efficacy scores improved following the completion of the ECHO knowledge
network, and the feedback received from participants was extremely positive. When asked
about their overall views of participating in the network, 100% stated they had learnt ‘a lot’
through participation and enjoyed it, 100% felt it helped translate knowledge into practice
more than other teaching sessions they had been involved in and had improved the care
they provided for patients, and 100% would participate again. A potential reason for the lack
of improvement in knowledge score when compared with the perception of learning was the
assessment itself- it was difficult (average scores were 54% and 57%) and may not have
fully assessed the topics that were being taught in the hub. This was likely due to the tight
timeframe of setting up the evaluation before the ECHO started when the curriculum had not
been finalised, and could easily be addressed in future evaluations, with knowledge
assessments being written based on the curriculum that will be covered in the upcoming
31
ECHOs sessions. It also highlighted the disconnect that existed before the ECHO between
what consultants in Diabetes perceived as the diabetic knowledge level among GPs.
The focus group findings highlighted the safe and effective learning environment that the
ECHO platform can provide for GPs working with patients with diabetes in the primary care
setting. ECHO optimised the delivery of care to patients through effectively enhancing the
knowledge and skills of GPs in relation to a variety of areas such as: diagnosing diabetes
and utilising new and innovative treatment options. Participants also illuminated the
importance of the hub and how effective it had been in motivating and encouraging the
spokes. For future ECHOs it has been suggested that specialists in other areas should be
invited to the hub, and spokes should consist of multiple members of the care delivery team.
ECHO was conveyed as an effective way to deliver clinically relevant education that
promotes patient care and it provided a key network of support for all involved.
The response rate to the evaluation was low and future evaluations should have
mechanisms in place to encourage participation, for example funding being dependant on
participation in the evaluation process or protected time being made available through
additional funding. This could help prevent bias and improve the generalisability of results.
While spoke members commented that they felt that participating in ECHO had improved the
care they provided for patients, it was not within the scope of this evaluation to look at the
impact on service delivery or patient care, and future studies should look to address this to
consider the cost effectiveness of ECHO through potential reductions in referral rates to
secondary care, improvements in diabetic control and reduction in long term complications.
Suggestions from evaluation
- The diabetic ECHO network should continue and be made available to more GPs
and other primary care professionals
- Technological issues need addressed including sound quality from the hub and
connectivity from NHS / GP sites
- Funding needs to be continued to allow protected time for participation and ensure
high quality teaching from specialists at the hub is maintained
- More than one participant at each site should be encouraged to enhance the
‘community of practice’ locally and to allow spoke participants to continue the
discussions and learning after the ECHO session is finished
- Future knowledge assessments should be based on the curriculum being taught
32
- Participation in the ECHO network and receipt of funding could be dependent on
participation in the evaluation process, or funding should be made available to allow
participants to take part in the evaluation. This could improve the generalisability of
results and prevent bias
- Future studies should look at impact on service delivery and patient care, for
example referral rates to secondary care and diabetic control of patients managed in
the primary care setting, to determine if participation in the ECHO knowledge network
has a direct impact on these areas.
33
Chapter 4 - Optometry ECHO
Background
Ophthalmology is a high demand specialty, typically accounting for 7-8% of all outpatient
appointments, both regionally and nationally, each year. In the Northern Ireland context, this
results in an excess of 100,000 out-patient appointments annually. Many ophthalmic
conditions are age-related, and many are long-term conditions (LTC’s) where ‘cure’ may not
be possible, and management to maintain useful vision is the goal. Glaucoma and macular
degeneration are two such conditions.
The Optometry ECHO had the objective of, providing primary care optometrists a safe space
to improve their knowledge base, and in turn helping them to better manage patients with
glaucoma or macular eye disease (age-related macular degeneration - AMD). By tele-
mentoring and case-sharing, ECHO aimed to enhance the available knowledge and clinical
diagnostic skills in primary care, thus helping to improve case-handling and referral patterns.
The Intervention
The Optometry ECHO knowledge networks were on Friday mornings from 9.30 – 11.00 am
and covered topics relating to AMD and Glaucoma. Twenty-one Optometrists from across
the region took part in 12 ECHO sessions to gain additional knowledge, skill and confidence
in the diagnosis and management of patients with these long term ophthalmic conditions.
The ECHO curriculum supported these primary care providers in elements of care, such as
diagnosing macular disease ‘at the margins’ and how and when to refer. For more details
see table 5.
Hub - The hub members included academic clinicians who were specialists in the identified
sub-specialties of AMD and Glaucoma. These included the HSCB Ophthalmic Service lead,
a Clinical Senior Lecturer and Consultant Ophthalmologist, a Professor of Ophthalmology
and Consultant Ophthalmologist, a Biomedical Scientist, a Primary Care Optometrist, a
university research Optometrist and a Clinical Optometric Adviser from the HSCB.
Spokes - The spokes were selected because they are all holders of the College of
Optometrists’ Professional Certificate in either (i) glaucoma or (ii) medical retina. The ECHO
programme was alternately around these sub-specialities.
34
Table 5 Optometry / Ophthalmology ECHO
Optometry/Ophthalmology ECHO
Clinical
Champion
Facilitator
Educators
Admin support
Raymond Curran
Margaret McMullan
Prof Agusto Azuaura-Blanco (Glaucoma)
Mr Michael Williams (AMD)
Dr Ruth Hogg: Lecturer QUB
Dr Linda Knox
Dr Gerry Mahon: Biomedical Scientist QUB/BHSCT
Claire Armstrong
Frequency of
clinics
Weekly (12)
Fridays at 9.30-11am
Training day Friday 13th November 2015 9.30-2.00 at Jennymount
Sessions
20, 27 November
11, 18 December
8, 15, 22, 29 January 2016
5, 12, 19, 26 February 2016
Hub members
Mr Michael Williams:Clinical Senior Lecturer, Honorary Consultant
Ophthalmologist QUB/BHSCT
Prof Agusto Azuara-Blanco: Clinical Professor/Consultant
Ophthalmologist QUB/BHSCT
Dr Ruth Hogg: Lecturer QUB
Dr Gerry Mahon: Biomedical Scientist QUB/BHSCT
Mr Alan Rundle: Primary Care Optometrist
Raymond Curran: AD of Integrated Care, Head of Ophthalmic
Services, HSCB
Mrs Margaret McMullan: Clinical Optometric Adviser, HSCB
ECLO’s (Eye care liaison officer) – invited as required pertinent to
curriculum
Hub costs
Based on 12 weeks of clinic and 4 hours clinic time (inc prep) and one
session of training
£11,481 paid to the 5 HSC Trusts on 1st December (for participation in
all ECHO networks as required)
Spoke members 21 Primary care optometrists to allow 1-1.5 hours
Spoke costs £165 per ECHO session
35
Spoke
Equipment
8 webcam & microphones/headsets
Majority using their own equipment
Equipment costs 8 webcams & microphones/headsets x £50 = £400
Curriculum Developed. Alternating between glaucoma and AMD with a session on
Optical Coherence Tomography
• AMD - Presentations, photos and images to make decisions on
• Glaucoma - The optic disc and glaucoma: Importance of disc
size. Recognising a normal disc. The ISN'T rule. Difficult
discs. Tilted discs. Glaucoma suspects
• AMD - Diagnosing macular disease at the margins. How and
when to refer
• Glaucoma - Fields. Some dots are missing: Is it glaucoma?
What do I do? And: is the patient fit for driving?
• Glaucoma - IOP: Ocular hypertension - OHTS study - central
corneal thickness - refractive surgery
• AMD - What should be done for patient with early AMD? Are
supplements worth it?
• Glaucoma - Systemic medications and glaucoma. Compliance-
adherence: what to do?
• AMD - OCT Machines: “Believers, Non-Believers and Doubters”
why buy one/why not buy one/what to do with one?
• AMD - Neovascular AMD: diagnosis, monitoring and
management. What to tell patients about it?
• Glaucoma - Rare Discs AMD - Dry AMD and the Eye Clinic
Liaison Officer: how patient’s needs are addressed
• Glaucoma – Gonioscopy
Evaluation Methods
Evaluation was undertaken as described in Chapter two. All participants completed their
pre-test evaluations at the training day, and were emailed their post-test evaluations which
were returned by post or email.
Results
Twenty one spoke participants completed the pre-ECHO assessments (response rate
100%), with 11 (response rate 52.4%) completing the post ECHO assessments (only 10
completed the retro-pretest evaluation of self-efficacy). Demographic data for all participants
are shown in table 6. For the rest of the results only the participants who completed the pre
and post evaluations are included.
36
Table 6 Optometry Demographic data Pre ECHO Evaluation
Range Totals Age 20-30 2 31-40 7 41-50 10 51-60 2 60+ 0 Gender Male 4 Female 17
Qualifications BSC Honours Optometry 18
Other 3 Do you already hold Prof Cert in Yes 20 Glaucoma? No 1 Do you already hold Prof Cert in Yes 3 Medical Retina? No 18 Years in profession <5 0 6-10 4 11-20 8 21+ 9
LCG area of practice Belfast 8 Southern 2
South Eastern 2 Northern 7 Western 1 North & West 2 Current Area of Work Rural 5 Urban 10 Mixed 6 Current area of work
Independent Practice 14 Corporate / Multiple (>3 practices) 6
Do you have access to OCT in your Yes 6 practice? No 15 Do you regularly use OCT in your Yes 5 clinical practice? No 16 Is the Optometry practice you work in Yes 17 connected to FPS BSO web portal? No 4
Knowledge and Self-efficacy Assessments
The mean scores of knowledge improved between the pre-ECHO and post-ECHO
assessments. Average knowledge scores improved from 23.7 to 27.6 (out of a possible 38
marks); from 62% to 73%. Only one participants score dis-improved, one was the same and
the rest improved. Due to the small sample, non-parametric tests were used to analyse the
37
data. Wilcoxon results indicated a significant improvement in knowledge (p=0.008) pre and
post-ECHO. More details are shown in table 7 and figure 4.
Eleven participants completed the pre and post-test self-efficacy evaluations, with ten the
retro-pretest evaluation. Self-efficacy results improved (table 7), with a lower average retro-
pretest evaluation score than pretest score (in all but one area), and a higher average score
for the post-test evaluation in all areas. The different domains are shown in table 8. Due to
the small sample, non-parametric tests were used to analyse the data. A Friedman test
showed that overall self-efficacy improved significantly across the three (pre, retro-pre and
post) time points (p=0.006); post hoc tests using a Bonferroni adjusted alpha (p=0.017) to
control for Type I errors revealed statistically significant improvement in post ECHO self-
efficacy (p=0.007) and between Retro-Pre and Post ECHO self-efficacy (p=0.008). More
details are shown in table 7 and figure 5.
Table 7 Optometry ECHO Knowledge and total self-efficacy results
Pre-ECHO Retro-pre ECHO Post ECHO
Centiles 25th Median 75th 25th Median 75th 25th Median 75th
Knowledge
19.0 24.0 25.5 - - - 25.0 27.0 32.0
Self-efficacy
3.29 3.57 4.29 3.07 3.43 3.93 4.21 4.23 4.71
Figure 4 Box Plot for Optometry Knowledge Scores
38
Table 8 – Optometry ECHO Self efficacy assessment - Participants self-rated confidence in
each area (1 - not confident at all, 5 - very confident)
Question Pre test
Average
(Range)
Retro Pretest
Average
(Range)
Post test
Average
(Range)
To examine and assess optic nerve head
appearance
4.00
(3-5)
3.80
(3-4)
4.60
(4-5)
Identify signs of glaucomatous damage to
the optic nerve head
4.10
(3-5)
3.90
(3-5)
4.70
(4-5)
Interpret visual field plots 3.80
(2-5)
3.70
(2-5)
4.40
(4-5)
Identify signs of macular disease (both
types dry and wet)
4.00
(3-5)
3.30
(2-4)
4.50
(4-5)
Differentiate the clinical appearance of wet
AMD and Dry AMD
3.70
(2-5)
3.80
(2-5)
4.50
(4-5)
Interpret OCT scans with respect to
macular changes
2.70
(1-4)
2.30
(1-4)
3.80
(3-5)
Overall I am confident in my ability to do
my job well and provide a safe and quality
service to patients
4.30
(3-5)
3.80
(3-4)
4.60
(4-5)
Figure 5 Box Plot for Optometry Self Efficacy Scores
39
General Feedback of Optometry ECHO
Box 2 demonstrates participant’s views on ECHO in general. Participants were very positive
about their experience of ECHO, with 100% having learnt through participation, 100% felt it
helped translate knowledge into practice more than other teaching sessions they had been
involved in, and 100% would recommend it to others and participate again.
Box 2 General ECHO Optometry Results
1. Please rate each on a scale of 1-5 the quality of learning / usefulness from each area (1- poor, 5- excellent)
1 2 3 4 5 Review of previous session 0 0 0 3 7 Presentations 0 0 0 0 11 Case based discussions 0 0 0 2 8
2. Overall do you feel you have learnt through participating in ECHO?
A lot 10
A little 1 No 0
3. Did you find participating in ECHO enjoyable?
A lot 10
A little 1 No 0
4. Do you think that participating in ECHO has improved the care you provide for patients?
A lot 7
A little 3 No 0
5. Do you think the format of ECHO helps translate knowledge from teaching into practice more than other teaching sessions you have been involved in?
Yes 11
No 0
40
6. Would you recommend ECHO to other healthcare professionals in your area?
Yes 11
No 0
7. Have you used any of the online resources via Moodle, and if yes have you found these useful?
Used and found useful
Used and found NOT useful
Power point presentations 4 1 Video of the teaching sessions 6 1 Video of case presentations 4 1 Other supporting materials 8 2
8. Regarding ECHO technology…
Agree Disagree Unsure It has given me access to education that would have been hard to access due to geography 8 1 2 It was a good medium to access teaching / education at a different location from where I work 11 0 0 Any technical difficulties were acceptable and did not put me off participating in ECHO 11 0 0 Any technical difficulties did not significantly reduce my learning 11 0 0
9. How do you rate your overall ECHO experience? (1- poor, 5- excellent)
1 2 3 4 5
0 0 0 1 10
10. Would you participate in ECHO sessions in the future if the opportunity arose?
Yes 11
No 0
41
Focus Group
Ten optometrists participated in the focus group from the spokes for the Optometry ECHO.
Analysis of the focus group data uncovered three overarching themes, each with their own
descriptive and interpretative themes that are outlined in thematic diagrams. Overarching
theme 1, ECHO Enhanced Clinical Knowledge and Skills, explored the reasons why
participants perceived that ECHO optimised their clinical practice (figure 6). Interpretative
themes 1A, 1B and 1C highlighted the contributing factors that resulted in participant’s
perceived enhancement of clinical practice. Overarching theme 2, ECHO Exceeded
Expectations and Changed Misconceptions, highlighted the preconceptions participants had
before ECHO commenced and how these altered throughout the project (figure 7).
Interpretative themes 2A and 2B illuminated the intricacies of why and how perceptions
changed in a positive way. Overarching theme 3, Consideration for the Future of ECHO,
displayed the key conceptions that participants perceived should be considered for future
ECHOs (figure 8).
Overarching Theme 1: ECHO Enhanced Clinical Knowledge and Skills
This theme encompassed how participants perceived ECHO to be an education platform
that enhanced their clinical knowledge and skills (figure 7). Interpretative theme 1A depicted
how ECHO provided an environment that facilitated the development of interdisciplinary
relationships and communication, within the field of optometry and ophthalmology.
Participants also perceived that ECHO optimised the care they delivered to their patients
through the knowledge they had gained, and also through their increased confidence in
making appropriate referrals. Interpretative theme 1C illuminated how ECHO also created a
safe learning environment for participants. Findings highlighted that this was perceived to
have been achieved by the hub members encouraging learning and treating all queries with
respect. The learning environment was also enhanced by the relationships that were built
between members of the spokes.
42
Figure 6 Theme 1 ‘ECHO Enhanced Clinical Knowledge and Skills’ for Optometry ECHO
Descriptive Themes Interpretative Themes Overarching Themes
1 ECHO Enhanced
Clinical Knowledge and Skills
1A Nurtured
interdisciplinary relationships and communication
Strengthened the link between primary and
secondary care providers
"Relationship between ophthalmologists and
optometrists has increased because of ECHO."
1B Optimised Patient
Care
Increased optometrist confidence in their skills
Reduced unnecessary referrals
1C Created a safe learning
environment
Expertise at the Hub
Encouraged sharing of knowledge
Built relationships between optometrists
"No questions are silly."
43
Figure 7 Theme 2 ‘ECHO Exceeded Expectations and Changed Misconceptions’ for
Optometry ECHO
Descriptive Themes Interpretative Themes Overarching Themes
Figure 8 Theme 3 ‘Consideration for the Future of ECHO’ for Optometry ECHO
Descriptive Themes Interpretative Themes Overarching Themes
2 ECHO Exceeded Expectations and
Changed Misconceptions
2A "Didn't know what to
expect."
Would it be too formal?
How would the technology work?
2B ECHO challenged
misconceptions
Addressed concerns about technology
"Afraid that knowledge may not
be up to par."
3 Consideration for
the Future of ECHO
3A Changes to be made
"Cutting it fine to have your homework
done."
Accessibility of resources
3B Furture ECHO
participants
Potential for Enhanced
relationships between optometrists and GPs
Relevant to Other professionals in the
community
44
Interpretative Theme 1A: Nurtured interdisciplinary relationships and communication
The majority of participants described how they perceived the relationships that were
fostered within the ECHO environment. Many felt that the ECHO platform provided a forum
which strengthened the links between primary and secondary care providers. Participants
perceived that this was optimised through inter-professional learning about the role of
optometrists in the community, and ophthalmologists in the secondary care setting:
“They [ECHO Hub] also learnt a lot from us [Spokes], what it's like in the front line
[Community setting] and the decisions you have to make. So I think ECHO has been
great for understanding, we understand more about how they work [Secondary Care
Setting], what happens with patients but they also understand more about what it's like in
the community.” (OP/15)
Participants perceived that through raising awareness of professional’s roles, ECHO actively
participated in facilitating interdisciplinary relationships:
“Realistically what it's [ECHO’s] doing is building relationships, where beforehand there
was this barrier of a hospital that stops us [Community Optometrists] from building that.”
(OP/1)
Findings also illuminated that ECHO enhanced relationship building between optometrists
and ophthalmologists:
“I think everybody would agree that certainly the relationship between ophthalmologists
and optometrists has increased because of ECHO. I'm sure that the ophthalmologists
would probably think a lot more about our knowledge and certainly understand what
we're looking at in practice, and we also understand more about what they're looking at
and what they would want to see and that relationship has been fantastic.” (OP/1)
Overall, these findings suggested that the unique learning environment that was nurtured by
the ECHO platform facilitated the building of relationships between professionals across the
primary and secondary care setting. This led to direct enhancement of interdisciplinary
awareness of the key role various clinicians have in patient care.
45
Interpretative Theme 1B: Optimised Patient Care
Participants also envisaged that the ECHO program had a positive impact on patient care.
Many felt this was due to ECHO increasing the perceived confidence they had in relation to
their clinical skills. This resulted from the education provided at ECHO:
“I felt so much more confident on the glaucoma case about what to look out for and
what way to manage it.” (OP/14)
“I think we've done macular and glaucoma in depth and it was fantastic, what we've
learnt out of it and certain learning points, I definitely feel more confident managing
this in practice.” (OP/15)
For many this was a very positive outcome from ECHO as they had hoped taking part would
improve patient care:
“I definitely went into ECHO hoping that it would be able to improve the care that I
provide to patients, and I definitely feel that I have.” (OP/12)
Findings additionally illuminated the perceived impact of ECHO on referrals to secondary
care services. The majority perceived that the learning they had achieved through ECHO
enabled them to make more appropriate referrals in relation to various conditions:
“Our disc assessment, I think, is better, our ability to reason and to look and decide
whether we want to keep the patient in practice or refer, I think it's better. I think it'll
reduce, certainly for me it will reduce some unnecessary referrals but it will also help
me pick up on things that I really need to refer.” (OP/1)
“I think I feel more confident now, when I'm doing things with patients, to look close,
especially looking at discs or looking at the macula more closely and thinking 'what
else could be going on that we can't see?' that it's perfectly okay to send somebody
for further investigation without necessarily knowing exactly what you can see or
what you can't see, but also being a bit more confident about what you can see and
thinking 'yeah, this is definitely something that needs sorted out' rather than being a
bit woolly about what you're sending in.” (OP/13)
The majority of participants also perceived that the learning gained had helped them to feel
more confident in dealing with particular conditions. This resulted in participants reducing
unnecessary referrals:
46
“I found that knowing the pathway and knowing the treatment that our patients were
going to be receiving in the hospital helped me to recognise when I'm actually more
happy and confident to keep them in practice and when it's safe to do that, and that's
been a really useful tool. So of course, it's lovely to be able to refer them when I'm
sure and I have the confidence to do that, but also the reverse, to keep the patients
here and to have confidence in our own ability and skill set.” (OP/11)
Findings also highlighted perceptions that reducing the number of unnecessary referrals
would result in decreased pressure on secondary care services. One participant expressed
the perception that more education through ECHO on anterior segment conditions would
help decrease the demand for acute eye service referrals:
“If we were more confident about dealing with these [anterior segment conditions] the
pressure on acute eye services would be lessened.” (OP/9)
In summary, these findings have highlighted how the learning gained through the ECHO
platform enhanced the confidence and clinical skills of the participants. This resulted in
improved patient care due to raised awareness amongst participants of how to act on and
manage specific conditions, and appropriately refer to other services. Awareness of
appropriate referral criteria also resulted in the perception that ECHO could reduce referrals
to secondary care services. Thus, enhancing the provision of care in the primary care setting
and ultimately reducing the demand on secondary care services.
Interpretative Theme 1C: Created a Safe Learning Environment
Many participants expressed that the expertise at the hub added to the positive learning
environment provided through the ECHO platform. Findings illuminated how the
ophthalmology presence at the hub was beneficial to the spokes:
“I personally found it really beneficial, especially the two ophthalmologists that sat in
on a lot of the sessions, you were getting feedback from them when they were
receiving our referrals, and local protocols were particularly beneficial. So I certainly
definitely felt I had benefitted from it.” (OP/12)
The majority also expressed that alongside the hub, the spokes provided encouraging peer
support and advice at the sessions:
47
“Just the fact that you had an ophthalmologist there and you have your peers, that
you can get an immediate answer, it's just brilliant, because quite often you're just left
wondering about things, 'am I doing the right thing?' and so it’s just to have that
immediate response was just great.” (OP/14)
The majority of participants also referred to the important role of the facilitator at each
session. Findings indicated that participants perceived the facilitators to be key in optimising
the learning gained at the ECHO session:
“Both the facilitators had listened to what we were looking for them to talk about at
ECHO and they really stuck to that, they really looked at it, and they listened to that
and then they addressed them, and it was incredibly useful.” (OP/1)
The ECHO hub was also regarded by participants as providing a safe learning environment
through the support provided by the hub when asking questions:
“I think every question that was brought up, it was made to feel significant and you
felt you could bring up any issue, which I think is very important.” (OP/14)
Alongside the perceived benefits of ECHO to those directly involved, participants also
expressed that the learning they gained through ECHO was shared amongst other
professional colleagues. Many expressed that colleagues would come to them for guidance
due to their involvement in ECHO:
“Colleagues that weren't involved in the ECHO session, they would sometimes come
to me with cases that they're not so sure about and I definitely feel more confident in
providing answers to their queries as well.” (OP/12)
This resulted in a perceived benefit to the other colleagues’ patients:
“I found other colleagues asking me things and it was something that I could maybe
bring as a question in ECHO, so the whole team did learn, and because I work in
different practices, that was spread among different practices too. So yeah, it's
helped in the care of my patients and in the patients of my colleagues too.” (OP/15)
The majority of participants highlighted that the ECHO program allowed for relationship
building between the spokes. Many expressed that they felt less isolation within their field
48
due to building relationships with the various optometrists who were partaking in the
program:
“So I think, from a relational point of view, it's been fantastic as well, as well as
getting to know the spokes. I had never met X [member of the spoke] and yet last
week I walking into a room and chatted to him as if I had known him for months. So,
at the end of the day, I've got to know another 20 optometrists that I would feel very
confident going and talking to if I was to meet in a meeting. So I think that's been
fantastic.” (OP/1)
“I think with meeting all the spoke optometrists too, you know, we know names but
we never see each other really, so it was great to put a face to a name.” (OP/09)
Accounts demonstrated that participants viewed ECHO as a safe learning environment that
was enhanced through the expertise, and skilled facilitation, present at the hub. Key learning
was also nurtured from the peer support provided by the spokes and the ability to build
relationships with other colleagues. Evidence also displayed how ECHO can often reach
wider than the participants themselves. Participants shared the learning gained with other
colleagues and thus perceived the impact of ECHO to go beyond them and their patients to
the patients of others.
Overarching Theme 2: ECHO Exceeded Expectations and Changed Misconceptions
Overarching theme 2 encompassed how participants perceived ECHO as exceeding their
expectation and altering the preconceptions they held before participating (figure 7).
Interpretative theme 2A conveyed how participants were wary of ECHO before it started due
to a lack of awareness of what it specifically entailed. However, interpretative theme 2B
encapsulated how the misconceptions participants held were changed throughout the
program.
Interpretative Theme 2A: “Didn’t know what to expect.”
Many participants conveyed how they were concerned that ECHO would be too formal.
However this perception changed:
“I think you probably did think it might be a little bit more formal than it was, and I
think it was really, like the others have said, because it was quite informal, even with
the ophthalmologists in there and their lectures and things, it never felt so formal that
you couldn't interact and ask questions,” (OP/13)
49
This participant expressed that the more comfortable environment was directly related to the
expert communication skills of the hub members:
“The ophthalmologists that we had, seemed to be very good communicators. The
level they were pitching things at, the things they were saying, the informality of the
whole thing, I think they were extremely good at communicating.” (OP/13)
Findings also highlighted that many had concerns about the influence of technology on the
ECHO program. Many were unsure if this would negatively impact on the interaction
between the spokes:
“At the beginning I was unsure about the technology and I thought we would be very
detached from one another.” (OP/9)
However, for this particular participant this concern was unfounded as the program went on:
“But as the weeks went on, I think the spokes started to talk more to one another and
ask each other questions.” (OP/9)
Some were concerned about the use of technology due to their own computer skills:
“I was a bit apprehensive, first of all, before using it because I'm not a very computer
person.” (OP/20)
Overall, the concerns regarding the use of technology and the format of the sessions were
reduced as participants partook in the program. This was facilitated by the ECHO
equipment, and also the members of the hub and spokes.
Interpretative Theme 2B: ECHO Challenged Misconceptions
The misconceptions held by participants before ECHO were reduced throughout the
program. The reasons for this are illuminated within interpretative theme 2B. Accounts
highlighted how initial fears regarding technology through the IT support given prior to
commencing the program:
“But they came out and installed the camera for me in my office, and you literally just
go in, type in the code and up it pops, and it's been fantastic, no bother. So very
good.” (OP/20)
50
“I couldn't find any fault with it at all. It was as easy as anything. I literally pushed the
button and the thing, the camera comes on, the sound works. I never had any bother
with it at all. Once you put your code in it just turns everything on by itself, it was
great.” (OP/13)
Participants also found it useful to use alternative tablet and Smartphone devices to
participate in the ECHO clinics:
“I think it was very easy to use. I just used it on an iPad mini and I'm just amazed
how easy it is to use, and I think people have been even accessing it on a phone, so
it's been fantastic technology.” (OP/15)
Using mobile devices also allowed participants to have flexibility in where they joined the
clinic:
“Because it's on the iPad, and I do locum and I've done it in three other practices and
at home, so it's been amazing, you take your iPad and connect to the Wi-Fi, and you
can really do it anywhere and that's the amazing thing about it.” (OP/15)
It was expressed by some participants that they held the misconception that ECHO would
not be aimed at a high enough level to meet their learning needs. However one participant
highlighted that this was not the case:
“I think my concern going into ECHO was that it was going to be pitched at a very low
level and we would end up listening to an awful lot of stuff that we already knew, and
it turned out to be not like that at all.” (OP/1)
On the other hand, some participants expressed that prior to partaking in ECHO many may
feel they do not have the knowledge required. However it was perceived that ECHO does
provide a safe environment for learners at various levels:
“Others may be afraid that their knowledge may not be up to par, and I think it's
important that people do realise that ECHO is a safe environment and that you're not
going to be wrong, as such, and you can get your questions answered without
anybody judging you, which I think is very important.” (OP/14)
In summary, findings conveyed that misconceptions regarding ECHO were soon altered
after the program commenced. This was facilitated by the ease of use of IT equipment and
the safe learning environment provided by the ECHO platform.
51
Overarching Theme 3: Consideration for the Future of ECHO
Findings were indicative of the need for further considerations when providing future ECHOs
(figure 8). This was conveyed though participant’s views on changes to be made that were
discussed in interpretative theme 3A. Findings also illuminated the role of ECHO for other
participants and this was demonstrated in interpretative theme 3B.
Interpretative Theme 3A: Changes to be made
Participants conveyed that in order to effectively repeat the ECHO program, more
consideration may be needed regarding the amount of time participants are given to review
the case presentations. Many expressed that they did not feel they were given sufficient time
to prepare for the session:
“For me personally, the information was usually emailed, but it tended to come out on
the Thursday, just the day before ECHO, and I work late on a Thursday and it was
just you were getting home and by the time you were looking at it, it was maybe 9:30,
10 o'clock at night, which was when you were a bit more tired. So if we had it
maybe... it did sometimes comes out on the Wednesday, but if it did come out a
couple of days beforehand it would be useful to have a bit more time to look over the
information.” (OP/20)
“There were some weeks that you were cutting it quite fine to have done your
homework, so it would be useful. It's not always possible but it would have been
useful to have had it a bit earlier.” (OP/11)
“Even 24 hours earlier, if we had the information 24 hours earlier. I, as well, struggled
and I was looking at the case presentations at 9 o'clock before ECHO started and it
would have been better to have it a day earlier.” (OP/9)
Many participants also conveyed that they had difficulty accessing online resources that
were made available to those participating in ECHO:
“I found a little bit of difficulty getting onto the site with the resources.” (OP/09)
“I did have a look on the online resource site, it did take me a while, I did find some
resources but it was a wee bit of ‘hoking’ around to find it.” (OP/20)
“I did find it difficult to get on to the online resource site and some of the links were
difficult to open.” (OP/19)
52
In summary, findings highlighted the need for future ECHOs to consider the time participants
are giving to prepare for the session. Accessibility of resources that are made available
throughout the program also needs to be addressed to ensure participants receive optimal
information and learning.
Interpretative Theme 3B: Future ECHO Participants
Findings indicated that ECHO is perceived to be of benefit across multiple professions, and
healthcare settings. Participants conveyed that ECHO may be a beneficial platform to use to
enhance interdisciplinary working between optometrists and GPs:
“So communication can only be good, and you could even imagine this [ECHO]
between us [optometrists] and GPs, perhaps, or within different professions, which
would perhaps be beneficial as well.” (OP/15)
Ultimately this could promote partnership working in the primary care setting:
“I think GPs don't necessarily understand what we know, some of the time. The GPs
in the rural area where I work, send me a lot of patients but I'm not sure that GPs in
general understand how much more useful we can be to them than they think we
are.” (OP/13)
Participants also envisaged that the ECHO model would be beneficial across other
disciplines:
“I would imagine this model could be shared out among any profession. In any
healthcare profession there would be cases that could be brought and discussed [at
ECHO] with perhaps somebody, maybe a consultant with more junior doctors, in the
same situation, we would have peers and also those who can provide answers. So I
imagine it would be quite easily shared out.” (OP/14)
“I was saying to a dentist a few months ago, he's not long qualified but he's taken a
different route than most dentists, he's gone in to hospital dentistry and he's seeing a
lot of very interesting cases, and one comment he made was I'm glad I'm not out in
the community setting. So it sounded to me like oh dear, that's quite like our
profession. So I think dentistry, yeah, a general dentist would probably gain the same
way that we have from ECHO.” (OP/9)
53
Interpretative theme 3B illuminated the benefits of ECHO across disciplines and how it could
promote partnership working, especially within the primary care setting.
Discussion
Optometrists who participated in the ECHO knowledge network demonstrated statistically
significant improvements in knowledge and self-efficacy following the 12 ECHO sessions.
When asked about their views on ECHO in general, 100% stated they had learnt through
participation, 100% felt it helped translate knowledge into practice more than other teaching
sessions they had been involved in, and 100% would recommend it to others and participate
again.
The focus group findings highlighted the safe and effective learning environment that the
ECHO platform provided for optometrists working within the primary care setting. ECHO
optimised the delivery of care to patients through effectively enhancing the interdisciplinary
relationships between optometrists and ophthalmologists. Patients also benefited from the
enhanced knowledge, skills and clinical confidence that participant’s experienced as a result
of ECHO. Although participants often had their doubts about the effectiveness of ECHO due
to technological concerns, or a desire to maintain ‘credibility’ amongst their peers, the ECHO
pilot soon changed these perceptions by the reassuring and encouraging environment
provided by the hub, and by the hub facilitators. ECHO was perceived as an effective way to
deliver clinically relevant education that will promote safe and quality patient care and reduce
unnecessary referrals to secondary care services. It provided a key network of support for all
involved and their colleagues. Irritants such as the tight timeframe of receiving ‘homework’
24 hours before the ECHO session leaving little time to prepare and the accessibility of
resources on-line should be easily addressed in future networks.
The response rate to the evaluation was lower than expected (52.4%) and future evaluations
should have mechanisms in place to encourage participation, for example; funding being
dependant on participation in the evaluation process, protected time being made available
through additional funding and processes in place to encourage non-responders.
While spoke participants felt that participating in ECHO had improved the care they provided
for patients and had improved the appropriateness of referrals to secondary care, it was not
within the scope of this evaluation to look at the impact on service delivery or patient care,
and future studies should look to address this to consider the cost effectiveness of ECHO
through potential reductions in referral rates to secondary care and improvements in
optometry care for patients in the community.
54
Suggestions from evaluation
- The optometry network should continue and be made available to more optometrists
with consideration for involvement of other primary care professionals where
relevant.
- Case presentations and ‘homework’ should be made available earlier to allow more
time for preparation.
- Training on how to access online resources needs to be improved.
- Funding needs continued to allow protected time for participation.
- Participation in the ECHO network and receipt of funding should be dependent on
participation in the evaluation process, or funding should be made available to allow
participants to take part in the evaluation. This should improve the generalisability of
results and prevent bias.
- Future studies should look at impact on service delivery and patient care, for
example referral rates to secondary care and management of patients in the primary
care setting, to determine if participation in the ECHO knowledge network has a
direct impact on these areas.
55
Chapter 5 - Nursing Homes ECHO
The Intervention
The Nursing Home ECHO knowledge network supported nursing home staff throughout
Northern Ireland to improve their knowledge and skills in the care and management of
patients with a wide range of healthcare needs. Issues such as renal failure, palliative and
end of life care, drug management, and heart failure were discussed with a team of multi-
disciplinary professionals.
To enable the Nursing Homes to embed changes in their systems, the HSC Safety Forum
also supported the nursing home staff to build knowledge in quality and safety improvement
tools and techniques during the ECHO clinics. This included the Model for Improvement, the
Plan Do Study Act (PDSA) Cycle, and measuring for improvement, along with a range of
other ideas for change.
The sessions were on a Tuesday afternoon from 2-4pm. This network completed 10 ECHO
sessions. More information on the set up of this ECHO is shown in table 9.
Hub - In the ECHO hub there were a number of HCPs including a Palliative Medicine
Consultant, a Lecturer in Palliative Care, the HSC Safety Forum Clinical Director and Patient
Safety Advisor, a Pharmacist, a Dietician and other HCP’s as required including a
Physiotherapist, Occupational Therapist, Speech and Language Therapist, Chaplain and
Social Worker.
Spokes - Recruitment was via three methods
1. Nursing Homes were recruited through the existing HSC Safety Forum Nursing
Home collaborative.
2. Invitations were also send to homes within the Four Seasons and Care Circle
organisations.
3. Expression of interest were sought through members of the Regional Transforming
Your Palliative Care Programme Board.
There were a total of 26 nursing homes involved with up to 70 staff participating at the
spokes in any one session.
56
Table 9 Palliative Care in Nursing Homes ECHO
Palliative Care – Nursing Homes Clinical Champion/ Facilitator Admin support
Max Watson & Sue Foster Ciara McClements/ Rebecca Donnelly
Frequency of clinics
10 sessions – 2 hours each (Tuesdays 2pm-4pm) Weekly initially (November) and then every other week
Session dates
November – 3rd, 10th, 17th, 24th December – 8th January – 12th, 26th February – 9th, 23rd March – 15th
Training 15th September and 29th October Hub members
Facilitator/Educator – Max Watson / Sue Foster and other NIH staff as required (Consultant / GP, Nurse, Pharmacist, Dietitian, Physiotherapist, Occupational Therapist, Speech and Language Therapist, Chaplain, Social Worker) Corrina Grimes - PHA Janet Haines – Woods - HSC QI Safety Forum Gavin Lavery – HSC QI Safety Forum
Hub costs Hospice staff costs Spoke members Staff from 26 Nursing Homes across Northern Ireland. Spoke costs
Nursing Homes will be paid an ‘Education Grant’ in February of approximately £1500
Equipment needed for spokes
21 webcams required (9 for Four Seasons) 12 installed Extra equipment required for Four Seasons: 2nd hand pc - £120 plus vat KVM switch - £37 plus vat USB wireless adapter (may be required) - approx £12 So approximately £200 x 9 homes = £1800
Total Spoke Equipment costs
Approx £ 5,000
Curriculum development update
1. Communication 2. Symptom Management in Palliative & EOL Care 3. Pain Management 4. Ethical Issues 5. Drug Management 6. Recognising Death & Dying 7. Delirium 8. Behavioural & Psychological Symptoms in
Dementia (BPSD) 9. Heart Failure
57
Evaluation Methods
Evaluation was undertaken as described in Chapter 2. There were different assessments for
registered and unregistered staff in both knowledge and self-efficacy. All participants
completed their pretest evaluations at the training day or in their individual nursing homes
under the supervision of their manager, and managers were emailed the post-test
evaluations to administer to staff and then return them by post or email.
Results – Nursing Home ECHO
Sixty two registered and 30 unregistered spoke participants completed the pre ECHO
assessments (92 total), with eight registered nurses (response rate 12.9%) and two
unregistered staff (response rate 6.7%) completing the post ECHO knowledge assessments.
Seven registered and three unregistered staff completed the post ECHO self-efficacy
assessments. Demographic data for all participants are shown in table 10 (registered staff)
and 11 (unregistered staff). For the rest of the results only the participants who completed
the pre and post evaluations are included.
Table 10 Nursing Home Demographic data Pre ECHO Evaluation for Registered Staff
Range Totals
Age 20-30 15
31-40 6
41-50 22
51-60 13
60+ 6
Gender Male 2
Female 60
Profession Registered Nurse 36
Team Leader/ Deputy Manager 12
Manager 13
Other 1
Years in profession <5 22
6-10 4
11-20 7
21+ 29
Current Area of Work Rural 8
Urban 17
Mixed 29
Not stated 8
58
Table 11 NH Demographic data Pre ECHO Evaluation for Unregistered Staff
Range Totals
Age 20-30 13
31-40 6
41-50 7
51-60 2
60+ 0
Not stated 2
Gender Male 3
Female 27
Profession Healthcare assistant 23
Other 7
Years in profession <5 14
6-10 9
11-20 5
21+ 2
Current Area of Work Rural 8
Urban 6
Mixed 9
Not stated 7
Knowledge and Self-efficacy Assessments
The mean scores of knowledge improved between the pre-ECHO and post-ECHO
assessments. For registered staff average knowledge scores improved from 41.8 to 45.6
(out of a possible 50 marks); from 84% to 91%. No participants score dis-improved, two
were the same and the rest improved. For unregistered staff one participants score
improved from 76% to 84%, the other participant dis-improved by 1 mark from 90% to 88%.
Due to the low response rate no statistical analysis was possible.
Seven registered and three unregistered participants completed the three self-efficacy
evaluations. Self-efficacy results improved (table 12 for registered and table 13 for
unregistered staff), with a lower average retro-pretest evaluation score than pretest score in
all areas for registered staff and in all but one area (heart failure patients) in unregistered
staff, and a higher average score for the post-test evaluation in all areas. For registered
staff, overall confidence, happiness and feeling of support improved, however, so did stress
levels. Due to the low response rate no statistical analysis was possible.
59
Table 12 – NH ECHO Self efficacy assessment for Registered Staff- Participants self-rated confidence in each area (1 - not confident at all, 5 - very confident)
Question Pre ECHO Average (Range)
Retropre Average (Range)
Post Average (Range)
Understand the principles of palliative care and apply them in my daily work
3.93 (1-5)
3.50 (2-5)
4.43 (3-5)
Break bad news to clients/residents and their families 4.04 (2-5)
3.50 (2-5)
4.57 (4-5)
Communicate effectively with clients/residents and their families
4.61 (3-5)
4.0 (3-5)
4.71 (4-5)
Recognise when a client/resident is approaching the dying phase and manage the dying phase appropriately
4.29 (2-5)
3.83 (2-5)
4.71 (3-5)
Know what medications will help a client/resident to control their symptoms and to give these appropriately
4.00 (1-5)
3.67 (3-5)
4.29 (4-5)
Play a role in providing good symptom management to clients/residents
4.05 (1-5)
3.50 (2-5)
4.43 (3-5)
Assessing and managing a client/resident’s pain 4.13 (2-5)
3.50 (2-5)
4.29 (3-5)
Provide appropriate bereavement support to clients /residents /their families
4.16 (2-5)
3.33 (2-5)
4.29 (2-5)
Make decisions about clients/resident’s care within an ethical framework
4.02 (1-5)
3.17 (2-5)
4.14 (2-5)
Support and facilitate advance care planning 3.89 (1-5)
3.67 (1-5)
4.43 (3-5)
Manage clients/residents with dementia 4.27 (2-5)
3.83 (2-5)
4.57 (3-5)
Manage clients/residents with heart failure 3.84 (2-5)
3.50 (2-5)
4.00 (2-5)
Manage clients/residents with renal failure 3.05 (0-5)
2.83 (2-4)
4.14 (3-5)
Manage clients/residents with COPD 3.32 (0-5)
3.17 (2-5)
4.14 (3-5)
Overall my confidence in my ability to do my job well and provide an excellent service to clients/residents is…
3.46 (0-5)
3.83 (2-5)
4.57 (3-5)
Overall my happiness in my job is… 3.57 (0-5)
3.67 (2-5)
4.43 (3-5)
Overall my stress level in relation to my job is… 2.61 (0-5)
3.17 (2-4)
3.43 (3-5)
Overall I feel supported in doing my job. 3.39 (0-5)
4.00 (3-5)
4.43 (3-5)
60
Table 13 – Nursing Home ECHO Self efficacy assessment for Unregistered Staff- Participants self-rated confidence in each area (1 - not confident at all, 5 - very confident)
Question Pre ECHO Average (Range)
Retropre Average (Range)
Post ECHO Average (Range)
Understand the principles of palliative care and apply them in my daily work
3.60 (2-5)
2.67 (2-3)
4.00 (4)
Comfort clients/residents/families who have received bad news
3.74 (2-5)
2.00 (2-3)
4.67 (4-5)
Communicate effectively with clients/residents and their families
4.17 (1-5)
2.33 (2-3)
4.33 (4-5)
Recognise when a client/resident is approaching the dying phase and play my role in helping with the dying phase appropriately
3.54 (1-5)
2.00 (1-4)
4.67 (4-5)
Know when a client/resident needs medications to control their symptoms and to advocate for them
3.43 (1-5)
2.33 (1-4)
4.67 (4-5)
Play a role in providing good symptom management
3.34 (1-5)
2.00 (1-3)
3.67 (3-4)
Helping to monitor and help with a client’s/resident’s pain
3.60 (1-5)
2.00 (1-3)
4.00 (3-5)
Provide appropriate bereavement support to clients/residents/ their families
3.46 (1-5)
1.67 (1-3)
4.00 (4)
Make decisions about clients/residents care within an ethical framework
3.00 (1-5)
1.67 (1-3)
3.67 (3-4)
Support and facilitate advance care planning 2.60 (1-5)
2.00 (2)
3.67 (3-4)
Care for clients/residents with dementia 4.26 (2-5)
2.67 (2-3)
4.67 (4-5)
Care for clients/residents with heart failure 3.40 (1-5)
1.33 (2)
3.00 (0-5)
Care for clients/residents with renal failure 3.09 (1-5)
2.00 (1-3)
4.00 (3-5)
Care for clients/residents with COPD 3.49 (1-5)
2.00 (1-3)
4.33 (4-5)
Overall my confidence in my ability to do my job well and provide an excellent service to clients/residents is
4.11 (2-5)
3.67 (3-4)
4.67 (4-5)
Overall my happiness in my job is… 4.43 (0-5)
2.67 (2-3)
3.00 (3)
Overall my stress level in relation to my job is… 3.34 (0-5)
2.00 (2)
2.00 (2)
Overall I feel supported in doing my job. 4.11 (0-5)
2.33 (2-3)
2.67 (2-3)
61
General Feedback of NH ECHO
Box 3 demonstrates participant’s views on ECHO in general. Due to low numbers of
responses the registered and unregistered staff have been combined. Participants were
very positive about their experience of ECHO, with 100% having learnt through participation
and found it enjoyable, 100% felt it helped translate knowledge into practice more than other
teaching sessions they had been involved in and improved the care they provided for
patients, and 100% would recommend it to others and participate again.
Box 3 General feedback from NH ECHO- registered and unregistered staff
1. Please rate each on a scale of 1-5 the quality of learning / usefulness from
each area (1- poor, 5- excellent)
1 2 3 4 5
Review of previous
session 0 0 2 2 2
Presentations 0 0 0 6 5
Case based
discussions 0 0 1 2 7
2. Overall do you feel you have learnt
through participating in ECHO?
A lot 9
A little 1
No 0
3. Did you find participating in
ECHO enjoyable?
A lot 10
A little 0
No 0
62
4. Do you think that participating in ECHO has improved
the care you provide for patients?
A lot 9
A little 1
No 0
5. Do you think the format of ECHO helps translate
knowledge from teaching into practice more than other
teaching sessions you have been involved in?
Yes 10
No 0
6. Would you recommend ECHO to other
healthcare professionals in your area?
Yes 10
No 0
7. Have you used any of the online resources via Moodle, and if yes
have you found these useful?
Used
and
found
useful
Used
and
found
NOT
useful
Power point
presentations 7 0
Video of the
teaching sessions 6 1
Video of case
presentations 6 1
Other supporting
materials 7 0
63
8. Regarding ECHO technology
Agree Disagree Unsure
It has given me access to education that would have been hard to
access due to geography 9 0 1
It was a good medium to access teaching / education at a different
location from where I work 9 0 1
Any technical difficulties were acceptable and did not put me off
participating in ECHO 8 0 1
Any technical difficulties did not significantly reduce my learning 8 0 0
9. How do you rate your overall ECHO
experience? (1- poor, 5- excellent)
1 2 3 4 5
0 0 0 3 7
10. Would you participate in ECHO sessions in
the future if the opportunity arose?
Yes 10
NO 0
Focus Group
Five registered nurses and one nursing home manager who participated at the spokes for
the nursing home ECHO participated in the focus group. Analysis of the focus group data
uncovered two overarching themes, each with their own descriptive and interpretative
themes that are outlined in thematic diagrams. Overarching theme 1 (figure 9), ECHO
Enhanced Clinical Knowledge and Skills, explored the reasons why participants perceived
that ECHO optimised their clinical practice. Interpretative themes 1A and 1B highlighted the
contributing factors that resulted in participant’s perceived enhancement of clinical practice.
Overarching theme 2, Consideration for the Future of ECHO, displayed the key conceptions
that participants perceived should be considered for future ECHOs (figure 10).
64
Figure 9 Theme 1 ‘ECHO Enhanced Clinical Knowledge and Skills’ for Nursing Home ECHO
Descriptive Themes Interpretative Themes Overarching Themes
1 ECHO Enhanced
Clinical Knowledge and
Skills
1A Optimised Patient
Care
Enhanced relationships between the Nursing
Homes and GPs
Increased knowledge, confidence and skills
regarding palliative and end of life care
1B Created an effective learning environment
Importance of the Hub
Healthcare assistants benefited from ECHO
Benefited from interaction with other homes
Resources were excellent
65
Figure 10 Theme 2 ‘Consideration for the Future of ECHO’ for Nursing Home ECHO
Descriptive Themes Interpretative Themes Overarching Themes
Overarching Theme 1: ECHO Enhanced Clinical Knowledge and Skills
This theme (figure 9) encompassed how participants perceived ECHO to be an education
platform that enhanced their clinical knowledge and skills. Interpretative theme 1A depicted
how participants perceived that ECHO optimised the care they delivered to their patients
through the knowledge and confidence they had gained, and also through enhancing
relationships with GPs. Interpretative theme 1B illuminated how ECHO also created an
effective learning environment for participants. Findings highlighted that this was perceived
to have been achieved by the effectiveness of the hub and the interaction between the
spokes. The learning environment was also enhanced as ECHO was perceived as providing
a platform which benefited healthcare assistants, alongside registered nurses.
2 Consideration for
the Future of ECHO
2A Challenges
Time to participate
Not on a weekly basis
Too much in one session
Staff enthusiasm lacking at times
2B Future ECHOs
Seperate sessions for staff without a medical or nursing background
ECHO for GPs and Nursing Homes
66
Interpretative Theme 1A: Optimised Patient Care
The majority of participants described how they perceived that the ECHO program had a
positive impact on patient care. Many felt this was achieved by enhancing the relationship
they had with GPs. ECHO aided in increasing participants confidence when discussing
patients with the GP, and helping the GP to value their opinions on patient care:
“I think it’s definitely made me feel more confident dealing with the likes of the doctors. After
our last presentation, regarding our client with the pain in the hands, I was able to go back to
his GP and have quite a lengthy discussion, and they were asking where I had got these
suggestions from and once I mentioned this ECHO project and the consultant facilitator, she
was straightaway on board with giving out the treatment. You’re not just seen as a nurse
trying to tell the doctor what to do; you’ve a bit more evidence to back up what you’re saying,
I think, which has helped.” (NH/3)
“We did our case study on a resident in her pain relief, we discussed what we had discussed
on the case presentation with the GP and her medication has been reviewed as a result of
that. So yes, it’s been very productive.” (NH/86)
Many also felt that ECHO increased their knowledge, confidence and skills regarding
palliative and end of life care:
“I think it’s [ECHO] given a level of confidence and also I think the discussions have given
the nurses something to check against, particularly say, around some drugs and their
interactions and things like that, and I think it’s just how people think a little more, which is
just great.” (NH/50)
“After the heart failure session it promoted us to have a discussion about monitoring and
blood results for our resident that was given high doses of diuretics. So it just helped us to
focus as well, rather than just blindly follow the treatment plan without any questioning or
checking. So it was helpful for that resident as well.” (NH/1)
Overall, ECHO was perceived as optimising patient care through enhancing the relationship
between nursing homes and GPs and increasing the knowledge, skills and confidence of
participants.
67
Interpretative Theme 1B: Created an effective learning environment
Participants conveyed that ECHO created an effective learning environment for the spokes.
Accounts portrayed that this was contributed to by the effectiveness of the hub and the
facilitator:
“The consultant facilitator was amazing, very respectful of what’s in nursing homes and what
we’re doing and always constantly very positive about the work that we’re already doing. It
was very much supported.” (NH/86)
“I believe they [the hub] were very supportive and it was clear how respectful everybody was
to each other in the hub. The respect when it was you’re turn and not making you feel silly,
any questions or queries that you raised, because sometimes it can be very isolating in a
nursing home, trying to lead and drive forward quality improvement. But nobody was made
to feel stupid about any queries they raised. It was treated that it was supportive for
learning.” (NH/1)
Participants also conveyed that healthcare assistants also benefited from the ECHO
program:
“The carers that attended did feel it was very, very useful, and it was all carers at that
session.” (NH/1)
For some participants ECHO was perceived as being beneficial for healthcare assistants in
the future as well:
“I do feel that it has opened our minds to they [healthcare assistants] need as much
information as we require as nurses, because they’re the ones that are doing the hands on
care every day, they’re the ones that are assessing the residents every day and reporting
back to the nurses. So definitely I think the care assistants will benefit from this programme
greatly.” (NH/86)
The majority of participants perceived that they benefited from the interaction with the other
spokes that they got at ECHO:
“Overall perceptions have been very positive, that’s my personal opinion. I feel I have
benefited from it, plus I've really enjoyed networking with everybody.” (NH/1)
“But I think the interaction between all the care homes was brilliant.” (NH/50)
68
This interaction also helped to dispel participants concerns regarding ECHO:
“I suppose, going into it, I was apprehensive but that was totally cleared very quickly, and it
was lovely, the whole camaraderie between everybody, because we’re all in the one boat,
we’re all having the same problems and it’s great to share knowledge and information with
each other. So I really thought it was very good.” (NH/86)
“I suppose with us being in a nursing home we’re quite isolated so the beauty of this is that it
is reachable to everybody and we can all share information and we are part of the group,
we’ve been talking about this outside of the session as well, so it’s been a great resource.”
(NH/86)
The resources were also viewed as being extremely beneficial to all participants:
“I think the resources that have been supplied have been fantastic. The information, there's
a hell of a lot of printing to be done, even yet! So it is definitely valuable material and I
thought we were going to get some really good material out of it.” (NH/1)
“Plus, a lot of the stuff on the website, I could print that off and if staff weren’t able to attend
these sessions we were able to disseminate all the information. So they got a lot of
information out of it.” (NH/86)
Overarching Theme 2: Consideration for the Future of ECHO
Findings were indicative of the need for further considerations when providing future ECHOs
(figure 10). This was conveyed though the challenges participant’s perceived regarding
getting the appropriate time to be able to dedicate to ECHO and also the timing of the
sessions themselves. Accounts also illuminated the challenges of getting staff enthusiastic
regarding ECHO. Interpretative theme 2B depicted perceptions of how the future of ECHO
may be considered.
Interpretative Theme 2A: Challenges
Accounts conveyed the challenges that participants perceived in relation to ECHO. Findings
suggested that getting time to participate in ECHO was an issue for participants:
“I think at the start I thought a lot more staff would be involved and I was very keen for that. It
was very difficult with staffing issues and that to get staff off the floor, so that would be the
only thing that I personally was disappointed in.” (NH/86)
69
Many perceived that having the ECHO session on a weekly basis was challenging:
“I would like things spaced out a little bit more and not on a weekly basis. So maybe
fortnightly.” (NH/50)
“I think weekly for a period was a bit of a challenge because time is always precious to
everybody.” (NH/1)
Some also felt that there was a lot covered in one session which made it difficult to engage
fully in the ECHO and cover all the information:
“I felt sometimes maybe the second presentation wasn't starting until maybe 3:50, 3:45 and
so we’d maybe spent a lot of time on one that didn’t have an awful lot of issues and then
maybe the second case presentation had a lot more that we could have discussed but
because the presentation was done first.” (NH/86)
“I think we probably tried to squeeze too much into a session and I do think that having the
quality framework going on as well was probably overcrowding the session and sometimes
that meant we didn’t get all of the case studies done etc.” (NH/50)
Some felt challenge by staff’s lack of enthusiasm to take part in ECHO:
“We didn’t have the enthusiasm as much from the staff, the three of us seemed to be the
Three Musketeers wanting to do everything but we didn’t get as much enthusiasm from the
staff.” (NH/1)
“Well, as I said, I thought that the staff would have been more participating in it. I was the
lone ranger over here [laughter] I really, really hoped that the staff would have got involved
because I thought they would have got a lot out of it, they would have really enjoyed it, as I
did.” (NH/86)
Interpretative Theme 2B: Future ECHOs
Findings highlighted that for future ECHOs participants perceived that separate sessions for
staff without a medical or nursing background may be required:
“It’s better to have a separate one [ECHO], a simpler session, because they [non-registered
nursing staff] did enjoy it and the amount they learned from it but there are areas which they
couldn’t quite understand. So probably better split the sessions.” (NH/10)
70
“I think some of the sessions were a little bit kind of medical based, I do think sometimes you
get more benefit by segregating groups of staff but I also think there is a place for mixing
staff as well. So I would go for both.” (NH/50)
“I feel that a lot of the information was very nurse led and I had my activity therapist in on
some of the sessions and it just looked like it was completely over her head.” (NH/86)
Many also felt that future ECHOs would be of benefit between GPs and Nursing Homes:
“ECHO could really be used between nursing homes and GPs as well I think that would be
of great benefit to a nursing home.” (NH/1)
Discussion
The response rates for the post ECHO evaluation were very low despite repeated and
persistent efforts to encourage participation. There was an improvement in knowledge and
self-efficacy, but no statistical analysis was possible due to the low numbers.
General feedback on the ECHO experience was positive, and from the low number of
responses, 100% stated they had learnt through participation and found it enjoyable, 100%
felt it helped translate knowledge into practice more than other teaching sessions they had
been involved in and improved the care they provided for patients, and 100% would
recommend it to others and participate again.
The focus groups confirmed this- the ECHO platform was viewed as a positive learning
experience that promoted the awareness of palliative and end of life care amongst nursing,
healthcare assistants and other staff within the nursing home setting. Of particular benefit
was the role ECHO had in nurturing the relationships between GPs and nursing home staff
and the confidence and skills it gave staff in the delivery of palliative care. Nursing homes
also conveyed that the interaction between spokes, that the ECHO platform lends itself to,
also enhanced the ECHO experience. For future ECHOs with this group of participants it
may be useful to consider the timing of the sessions to allow more staff to join and get the
most out of each session. Funding for protected time may have helped this issue, as staff
reported that with the busyness of the job it was hard to get time to attend due to patient
care needs.
A weakness is the low response rate that is likely to have led to a biased sample of more
enthusiastic HCPs participating. This was reinforced in the focus group where participants
commented on the lack of enthusiasm of some other staff. The ability to generalise these
71
results is therefore limited. More research is necessary to determine the benefits of ECHO
in this setting, and whether it improves staff knowledge, self-efficacy and ultimately patient
care.
Suggestions from evaluation
- Due to the very low response rate further research is needed in this area to
determine how useful ECHO is in nursing homes for healthcare professionals. While
the responses from participants were very positive, the response rate of
approximately 10% makes the results ungeneralisable as it was likely a biased
sample.
- Consideration should be given to incentives for participating in the evaluation for staff
e.g. funding dependent on this or funding for protected time to participate.
- Funding needs to be considered for protected time for staff to participate.
- The timing and frequency of any future sessions would need to be considered closely
with the involved spokes.
- Research looking at the direct impact on patient care and analysis of evidence of
using the quality improvement learning would be likely to provide very valuable
information as to the cost effectiveness and direct outcomes of ECHO investment.
72
Chapter 6 - Enhanced Dermatology for GP Trainees ECHO
Background
Skin disease is common and distressing. Around 24% of the population consult their GP
with a skin problem in any 12-month period.(10) It is estimated that approximately 14% of all
GP consultations are in relation to disorders or concerns about the skin.(9) Establishing and
maintaining competence in this area of medicine is therefore essential for any GP. There is
variable (and often limited) training in dermatology at undergraduate level leading to
confidence and competence gaps at the post-graduate stage.(11-13) Most skin disease can
be appropriately and efficiently managed in primary care.
Of the nearly 13 million people presenting to GPs with a skin problem each year in England
and Wales, around 6.1% (0.8 million) are referred for specialist advice.(14) While there are
well over 1000 dermatological diseases, 10 of them (eczema, psoriasis, acne, urticaria,
rosacea, infections/infestations, leg ulcers and stasis eczema, lichen planus and drug
rashes) account for 80% of consultations for skin disease in General Practice and specially-
collected data from four specialist Dermatology departments in England show that
specialists most commonly see people with skin lesions (35-45%), eczema, psoriasis and
acne.
Within one health trust area in Northern Ireland demand for secondary care dermatology
services has been seen to grow year on year with a growth of 4.5% for 13/14 on the
previous year.(15) This growth in demand contributes to greater waiting times, delays in
effective clinical management and an increase in patient distress. Additionally, new initiatives
around models of care are being developed. Prominent among these is ‘Transforming Your
Care’ (TYC) which aims to increase the proportion of care delivered within community
settings.(16) The Kings Fund suggests a number of approaches to demand management
among these being ‘Schemes to manage GP referrals’.(17) It is likely that progress in
respect of confidence and competence among GPs and the adoption of best practice
guidelines/ frameworks is likely to yield the greatest dividends.
GP Trainees follow a well-defined curriculum(18) which is broad in its reach and completed
within a short 3 year time period. “Care of People with Skin Problems” is one of the core
competencies of this curriculum for the ‘Membership’ exam of the Royal College of General
Practitioners (MRCGP).(18) Some GP Trainees will get the opportunity to gain deeper
experience in the management of dermatology problems through secondary-care clinical
attachments but the majority will not. An opportunity to increase knowledge and competence
73
during training in respect of the management of dermatological conditions has the potential
to have a career-long impact for future cohorts of GPs.
Trainees at the GP ST2 level have 100+ consultations per week. These consultations are
unselected and undifferentiated and will cover the full range of clinical areas for which
anyone might consult their GP. On the basis of average consultation rates it is therefore
expected that each GPST2 will see and manage 14+ patients with dermatological issues
each week.
The Northern Ireland Medical and Dental Training Agency (NIMDTA) is an organisation
which operates on behalf of the Department of Health, Social Services and Public Safety
(DHSSPS) to train medical and dental professionals. NIMDTA commissions, promotes and
oversees postgraduate medical and dental education and training throughout Northern
Ireland. NIMDTA organises and delivers the recruitment, selection and allocation of doctors
and dentists to foundation, core and specialty training programmes and rigorously assesses
the performance of trainees through annual review and appraisal. It works in close
partnership with local education providers to ensure that the training and supervision of
trainees support the delivery of high quality, safe, patient care. NIMDTA now recruit
approximately 85 trainees each year to the GP Training programme. These GP specialist
trainees (GPST) have a 3 year run-through training programme (GPST1, GPST2 and
GPST3) of which 18 months takes place within hospital attachments and 18 months in
General Practice (6 months in GPST2 and 12 months during GPST3).
The Intervention
Twenty-eight GP Trainees (GPST2 level) from across the region took part in this ECHO over
5 sessions to learn about the common dermatological conditions of eczema, psoriasis and
acne. For more details see table 14. The Programme Lead (a GP Training Programme
Director with NIMDTA who is also a GP with enhanced dermatology training developed a
Programme Resource Pack comprising a range of written material, pictures and videos that
were used in the sessions. (These resources were based on her work previously done with
the Northern Area Primary Care Dermatology Pathway and were reviewed by local
consultants).
The ECHO sessions were scheduled to take place on a 2-weekly basis and ran for 1.5 hours
around lunchtime to facilitate maximum participation and minimum disruption to the working
day. The meetings were conducted via video-conference technology that allowed GP
Trainees to join from any location that provided a webcam and internet connection thus
avoiding the need to journey to a central location.
74
Participants were asked to prospectively identify ‘Cases’ from their GP surgeries and to use
the agreed ‘Case Proforma’ setting out the anonymised details of a non identifiable patient,
the challenges faced, the treatment modalities used and the outcomes achieved. The hub
team provided feedback and guidance on management options and advice on refinements
to the management options available. It was expected that in each meeting up to 3 cases
would be discussed.
Most of the GP Trainees joined the ECHO Clinics via videoconference principally from one
of two locations
1. One of the seven community offices of the NIH for those GPST2s in close proximity
or
2. From the nearest GP Practice which had a webcam facility
Some of the GP Trainees joined using their own tablet or laptop devices from home as
access to the GP Practice Wi-Fi network was forbidden by the local Health and Social Care
security policies. A map of the locations of trainees, Northern Ireland Hospice community
offices and locations of the GP Webcams can be seen in Picture 2.
Hub - The hub team was chaired by the Programme Lead (GP with enhanced Dermatology
Skills) and a variety of invited guest team members (e.g. Consultant or Specialist Nurse
Dermatologist and a Pharmacist with enhanced knowledge of prescribing for dermatological
conditions).
Spokes - The spokes were identified as the half of the GPST2 cohort who are placed in
General Practice during the first 6 months of their ST2 year. There were 28 GP trainees
involved in total. Some based at NIH community sites (16) and some at training practices
(12).
Picture 2 - Locations of trainees, NIH community offices and locations of the GP Webcams
75
Table 14 GP Trainee/ Dermatology Knowledge Network
GP Trainee/ Dermatology Knowledge Network Clinical Champion/Lead Facilitator Teacher
Dr Nigel Hart Dr Siobhan McEntee
Frequency of clinics 5 clinics Start date
21st October 4th & 25th November 2nd December 13th January
Date for training September 2015 GP Core Day Hub members
Programme Lead GP with enhanced skills Invited others; consultant, Specialist Nurse, Pharmacist,
Hub costs None Spoke members
GP trainees – some based at NIH community sites (16) and some at training practices (12)
Spoke costs No backfill requirement
Equipment needed for spokes
8 webcams/microphones required – installed by NI Hospice Project ECHO Jonathan Pope (BSO) attended installation of a few to ensure no impact on HSC network
Equipment costs 8 webcam/microphones x £145 Curriculum development update
Dermatology curriculum developed • Facial Rashes, eg acne vulgaris • Psoriasis • Eczema in Children • Eczema in Adults • When should I refer?
Evaluation framework Nigel and Siobhan developed a pre & post ECHO
evaluation
Objectives:
The objectives of this pilot study were to determine if the ECHO model can be used to
support the improvement of confidence and clinical knowledge among GP ST2 Trainees in
the management of common dermatological conditions presenting in primary care.
Methods
The knowledge and self-efficacy assessments were run by the network facilitators (Dr Hart
and Dr McEntee). The general ECHO feedback and focus groups were undertaken by the
evaluation team from NIH as described in chapter 2.
76
The questionnaire for pre- and post-programme self-efficacy and knowledge assessment
were conducted using Survey Monkey®(8) and was carried out with all participants before
and after the Enhanced Dermatology ECHO Programme.
Results
Twenty-six trainees took part in the Enhanced Dermatology ECHO Knowledge Network. All
trainees successfully completed the Pre-ECHO questionnaire. One trainee left the
programme early due to being rotated to a new attachment. One trainee did not fully
complete the post- questionnaire and two trainees did not start the post- questionnaire at all.
Out of 26 original programme participants pre- and post- programme paired data was
collected for 22 trainees (response rate 84.6%).
The Likert scale questions in the self-efficacy part of the questionnaire (self-efficacy and
confidence in respect of managing clinical cases in dermatology) were coded 1 to 5 (1 =
Strongly Disagree Strongly, 2 = Disagree, 3 = No strong views, 4 = Agree, 5 = Strongly
Agree), Question 4 (confidence in relation to strength of steroid usage) was coded 1 to 4 (1
= Mild, 2 = Moderate, 3 = Potent, 4 = Very Potent). The paired results were analysed using
the Wilcoxon signed rank test. All questions showed a significant improvement (P < 0.001) in
assessment of self-efficacy and confidence for managing clinical cases in dermatology from
before the programme to after.
The responses to knowledge assessment were marked, scored and totalled for each of the
trainees. Completion of the knowledge assessment achieved a maximum total score of 50.
The scores were analysed using the t-test. The pre- and post- mean percentage scores
showed a significant improvement (Pre- 35.5% (SD 7.6) and Post- 60.6% (SD 4.3),
P<0.001).
General ECHO Feedback
There were 8 responses (30.8% response rate) to a Survey Monkey for GP trainees about
their general views on ECHO (box 4). All felt they had learnt through participating in ECHO,
found it at least ‘a little’ enjoyable, and 100% felt it had improved the care they provided for
patients. All would participate again in future ECHOs.
77
Box 4 General ECHO Feedback for GP Trainees Dermatology Network
1. Please rate each on a scale from 1-5 the quality of the learning / usefulness from each
area (1 – Poor, 5 - Excellent)
Answer Options 1 2 3 4 5
Review of previous session 0 0 3 5 0
Presentations 0 0 1 4 3
Case based discussions 0 0 1 6 1
2. Overall do you feel you have learnt through participating in
ECHO?
Answer Options
Response
Percent
Response
Count
A lot 87.5% 7
A little 12.5% 1
No 0.0% 0
Unsure 0.0% 0
If Unsure please state why 0.0% 0
3. Did you find participating in ECHO enjoyable?
Answer Options
Response
Percent
Response
Count
A lot 62.5% 5
A little 37.5% 3
No 0.0% 0
Unsure 0.0% 0
If Unsure please state why 0.0% 0
4. Do you think that participating in ECHO has improved the
care you provide for patients?
Answer Options
Response
Percent
Response
Count
A lot 75.0% 6
A little 25.0% 2
No 0.0% 0
Unsure 0.0% 0
78
5. Do you think the format of ECHO helps translate
knowledge from teaching into practice more than other
teaching sessions you have been involved in?
Answer Options
Response
Percent
Response
Count
Yes 50.0% 4
No 12.5% 1
Unsure 25.0% 2
6. Would you recommend ECHO to other Healthcare
professionals in your area?
Answer Options
Response
Percent
Response
Count
Yes 87.5% 7
No 0.0% 0
Unsure 12.5% 1
7. Regarding ECHO technology...
Answer Options Agree Disagree Unsure
Response
Count
It has given me access to education
that would have been hard to access
due to geography
5 2 0 7
It was a good medium to access
teaching / education at a different
location from where I work
8 0 0 8
Any technical difficulties were
acceptable and did not put me off
participating in ECHO
7 0 1 8
Any technical difficulties did not
significantly reduce my learning 7 0 1 8
8. How do you rate your overall ECHO experience? (1 - Poor, 5 -
Excellent)
Answer Options 1 2 3 4 5
Experience Rating 0 0 1 5 2
79
Focus Group
The focus group was conducted with eight GP trainees who participated at the spokes.
Analysis of the focus group data uncovered two overarching themes, each with their own
descriptive and interpretative themes that are outlined in thematic diagrams. Overarching
theme 1, ECHO Enhanced Clinical Knowledge and Skills, explored the reasons why
participants perceived that ECHO optimised their clinical practice (figure 11). Interpretative
themes 1A and 1B highlighted the contributing factors that resulted in participant’s perceived
enhancement of clinical practice. Overarching theme 2, Consideration for the Future of
ECHO, displayed the key conceptions that participants perceived should be considered for
future ECHOs (figure 12).
Overarching Theme 1: ECHO Enhanced Clinical Knowledge and Skills
This theme encompassed how participants perceived ECHO to be an education platform
that enhanced their clinical knowledge and skills (figure 11). Interpretative theme 1A
depicted how participants perceived that ECHO optimised the care they delivered their
patients through the knowledge they had gained, and also through their increased
confidence in caring for patients with dermatological conditions. Interpretative theme 1B
illuminated how ECHO also created an effective learning environment for participants.
Findings highlighted that this was perceived to have been achieved by the expertise of the
hub and the sharing of knowledge from remote places. The learning environment was also
enhanced as ECHO was perceived as providing learning that previously had been limited for
this group of clinicians.
9. Would you participate in ECHO sessions in the future if the
opportunity arose?
Answer Options
Response
Percent
Response
Count
Yes 100.0% 8
No 0.0% 0
Unsure 0.0% 0
80
Figure 11 Theme 1 ‘ECHO Enhanced Clinical Knowledge and Skills’ for Dermatology for GP
trainees ECHO
Descriptive Themes Interpretative Themes Overarching Themes
1 ECHO Enhanced
Clinical Knowledge and
Skills
1A Optimised Patient
Care
Optimised the approach to patient care
Increased confidence dealing with dermatological
conditions
Enhanced the quality and relevance of referrals
1B Created an effective learning environment
Expertise at the Hub
"It covered all the basics"
Limited training in this area pre- ECHO
Preconceived it would be harder
"We could do it from remote places."
81
Figure 12 Theme 2 ‘Consideration for the Future of ECHO’ for Dermatology for GP trainees
ECHO
Descriptive Themes Interpretative Themes Overarching Themes
Interpretative Theme 1A: Optimised Patient Care
The majority of participants described how they perceived that the ECHO program had a
positive impact on patient care. Some felt it helped them to consider how they were
managing their patients and feel more prepared to take alternative approaches that were
discussed at ECHO:
“I think even one of the most helpful things was some of the resources in the lecture notes
that we got emailed, things like emollient ladders and the steroid ladders, which are just very
helpful to have on your desk in front of you and probably made you feel more confident in
prescribing those and using them. From my perspective, that was one of the most useful
things in terms of its impact on practice.” (GPT/02)
2 Consideration for
the Future of ECHO
2A Changes to be
made
Cameras in more practices
Have on a dedicated study
day
2B Challenges
Getting spokes to mute and unmute
Too Many spokes
2C Furture ECHO
participants
Consideration of the hub members
Ability to view missed sessions
82
“I thought it was quite good overall and quite a good idea, and I think I benefitted from it and
it probably changed the way I managed a few patients, and involved some of the stuff that
we went through.” (GPT/06)
Findings also suggested that the ECHO experience had encouraged participants to adopt a
more holistic approach to the care of their patients:
“It made me think, not just about the physical aspect of the condition but more about the
psychological, the emotional impact that the conditions have on the patients as well, and that
was something I learned through ECHO and something that I was able to explore with the
patients as well. I think it's going to help my management of patients in the future, so I think
it's had a good impact already and it will continue to do so.” (GPT/07)
“You take a more holistic view of things, what their work is, has their condition improved
when they’re off work for a while or off on holidays, things that you probably wouldn't have
asked before.” (GPT/08)
Many also felt that patient care was optimised due to the impact of the ECHO program on
increasing their confidence and skills when providing care to patients with dermatological
needs in the primary care setting:
“I think where I notice it most, I wasn't able to get to the psoriasis ECHO and I still, compared
to all the other conditions [covered at ECHO] that we're managing, I would probably feel less
confident about managing psoriasis. So I think the ECHO does probably add a different
component. I do see the extra dimension that ECHO has added to understanding and
treating conditions, because I can compare the acnes and the others, eczemas, and I'm
more comfortable with those, given I've had the ECHO experience of it.” (GPT/03)
Findings also illuminated how participant’s confidence was enhanced due to the case
presentations that were used during the ECHO sessions:
“I think it was really useful having the case discussions, because even though obviously all
the ECHO communication was given and everyone said it was really useful to have that on
your desk to flick through, but it was really good then hearing real life cases and what would
be different... obviously every patient is different and what we could do, what we need to
think about. I thought that was good and increased our confidence in dealing with those
slightly more difficult situations.” (GPT/04)
83
ECHO also provided a platform to increase GP trainee’s confidence in relation to the use of
topical medications. One GP trainee expressed that the confidence they had gained through
ECHO had been translated into practice as they were more effective in information giving to
patients. This was also perceived as helpful in building relationships and promoting
compliance:
“I think that patients benefit overall if we are able to be more confident in our approach, and
obviously confidence and competence, and I think certainly it is about the practical things,
how much of an emollient to give someone, how much steroid cream, the practical points of
how to explain how to put it on, and I think whenever you are able to explain those things to
patients it gives them more confidence that you know what you're talking about rather than
just saying 'here, have a steroid' but not really... explain things like fingertip, how much to
use and give them the right amount for that week and then tell them to come back. I think
that really helped them to be compliant and come back and see you about it. So it improved
the relationship but also they're kind of ... it helped. So I do think it really did benefit
patients.” (GPT/05)
Accounts also highlighted how the ECHO platform enabled participants to improve the
quality of referrals they made to other services
“There were people who I had seen who needed referred and I felt more confident making
that decision because I had the knowledge that we had gained.” (GPT/04)
Some participants also expressed that the learning they had gained from ECHO gave them
the knowledge and skills to provide further treatment to their patients and avoid unnecessary
referrals to secondary care services:
“One case in particular that I had was a baby with eczema that wasn't responding to
treatment that we'd given, and I didn't really have much experience with that sort of thing, the
woman was quite anxious so I was thinking about maybe referring to paediatric dermatology
just to have a look because I wasn't really sure what was going on. But then the session on
eczema, so it was going into that that I felt a bit more confident and actually rang the mum
back and just had a chat with her and said we can try these different things instead of
referring you on to paediatrics. So I think she was reasonably happy with how that worked
out. So I think that's a good example of changing practice and probably preventing a referral
to secondary care.” (GPT/06)
84
The presence of an experienced interdisciplinary team at the hub was also perceived to be a
factor in reducing the number of unnecessary referrals due to the clinical knowledge and
skills gained:
“I value the fact that it was an interdisciplinary team at the table [hub], you did get the
perspective of a pharmacist, you got the perspective of a nurse, who's maybe seeing some
of these patients, and all the little things that they teach the patients, we have been able to
glean a bit off, and in a way that can help avoid referral because maybe we have chosen the
right treatment but it maybe wasn't being applied right or wasn't being applied well enough.
So we learn from that.” (GPT/03)
Participants also illuminated that in reducing inappropriate referrals, ECHO may also have
the potential to help alleviate issues regarding waiting lists to specialist services:
“I think now with the fact that waiting times are so long to see specialists, I think something
like ECHO could be used for GP’s to discuss with a specialist to say is there anything I could
do with this in the meantime, does this need referred? That sort of advice would be useful
and would be needed at primary care.” (GPT/06)
In summary, participant’s views highlighted the impact of ECHO on optimising patient care
through increasing the knowledge and skills base of GP trainee’s in relation to
dermatological conditions. Findings also highlighted how the learning gained from the hub
provided information on specific conditions that aided participants to optimise the
appropriateness of their referrals to secondary care services.
Interpretative Theme 1B: Created an effective learning environment
Many participants expressed that the expertise at the hub added to the optimal learning
environment provided through the ECHO platform. Some participants felt that the facilitator
was vital in enhancing the quality of the ECHO sessions:
“Generally, the facilitators do quite a good job. It was maybe a bit tricky initially with the
muting and unmuting function that people had mentioned previously. But I think overall it
was well done, I think everybody got a chance to participate. So I think overall it was pretty
well done.” (GPT/06)
Many also felt that the expertise at the hub was able to provide good practical guidance for
the management of patients with dermatological conditions:
85
“I think the relative experience at the table [hub] was quite good as well. You had a nurse
and a pharmacist, and it was kept very practical, which is helpful, because that's what we
have to deal with rather than maybe all the exact signs behind everything. The day-to-day
steps of it, I found quite helpful.” (GPT/03)
“The practical advice is excellent, things like apply the emollient half an hour before the
steroid and then also the guidelines, as the guys have said before, are excellent.” (GPT/08)
Accounts also expressed that ECHO “covered the basics” and provided useful information
on the day-to-day care of this client group:
“I thought it was very helpful. I think it covered all the basics. It covered the day-to-day things
quite well and the ladders and pathways for treatment have been quite useful. So yeah, I
think it's benefited me, certainly.”
(GPT/08)
The resources were also highlighted as being a contributing factor to the effective learning
environment within ECHO:
“To me, the resources were very helpful. I suppose the main thing, it was great to have the
lecture before it so you could have it printed out and have a look at it, but the main thing I
found the most helpful were all the guidelines, just having them in a folder and being able to
look at them throughout the day whenever you were in the GP practice dealing with patients.
So I thought they were very good.” (GPT/01)
“I think it was good having all the handouts, that after the first session we had the handouts
sent out before, and I think it was good to have the presentation there and rather than
making lots of notes you could just allocate it as you went along, and I suppose knowing
about what cases were coming up. So being able to use that resource was really useful.”
(GPT/04)
The education provided by the ECHO sessions were particularly welcome as participants
expressed the training they received regarding dermatology had been limited:
“I think our undergraduate dermatology teaching is only a week, so you're coming into a GP
with very little knowledge about how to manage all the common skin conditions, and inside a
couple of weeks of being in a GP practice you realise how common dermatology issues are.
So definitely, it's been really helpful and I don't really know what they would have done,
86
coped with all the dermatology without it, to be honest, because you're not really taught that
at all.” (GPT/01)
“We have very little, certainly personally, day-to-day experience of dermatology, before
general practice. In hospital it tends to not be really dealt with on a medical ward, it would
just be "see your GP about that." So we were on the receiving end of that and I think it has
been helpful.” (GPT/03)
However, prior to the start of ECHO many held preconceptions that it would be harder to
engage in the sessions but felt the facilitator helped them get involved:
“I suppose it's a very new format to all of us. I've never had any teaching delivered that way
before so I wasn't sure what to make of it. I thought it would be harder. Initially, before we
had done any, I thought it would be quite hard to feel a part of it or feel engaged with it, but I
think, you know, the facilitator did quite a good job of keeping everyone involved and asking
different people things. So I found it easier to engage with what I had thought whenever the
idea was explained at the beginning.” (GPT/03)
Some felt unsure of how they would find using the technology:
“I thought it was going to be a lot harder to be involved with and thought it was going to be
more like watching a lecture up on a screen and it was much more high tech. I thought, apart
from a few initial hiccups, it went very smoothly, it went a lot better than I thought it would.”
(GPT/04)
However perceptions changed as the ECHO progressed and using the technology became
more ‘natural’:
“I initially thought this was going to be a bit strange and a bit odd but eventually it seemed to
be quite natural, was probably my perception, it actually came a lot more naturally than I
thought it was going to.” (GPT/02)
Many also felt it was positive learning environment as it could accessed even in “remote
places” and without having to travel:
“I think having the Zoom in your own practice is a lot more practical than travelling
somewhere.” (GPT/05)
87
“When I heard about it I was quite interested to see how it would work, because I know
dermatology is really big area and we do need to focus more time on it, so I think it was
interesting to see how it was set up all over Northern Ireland and that we were able to make
it work where we didn't have to keep attending one area to get to each input but we could do
it from kind of remote places. So it was quite an interesting experience to be a part of.”
(GPT/07)
“Certainly I would just echo the fact that the camera was in my practice. I'm in quite a remote
location and it just wouldn't have been feasible to be involved in it. So yeah, that was very
helpful.” (GPT/02)
Overall, the ECHO platform contributed to an effective learning environment that was
facilitated by the effectiveness of the hub at providing clinical expertise and good facilitation.
Participants expressed negative preconceptions that they had before ECHO, and the lack of
training they perceived they had in this area previously. However, these perceptions
changed as the ECHO program commenced and the training provided practical guidance
from an interdisciplinary perspective.
Overarching Theme 2: Consideration for the Future of ECHO
Findings were indicative of the need for further considerations when providing future ECHOs
(figure 12). This was conveyed though participant’s views on the need for appropriate
equipment in various locations, alongside protected study time, to enhance the accessibility
of ECHO. Participants additionally highlighted the specific challenges of participation that
may be addressed for future ECHOs. Finally, interpretative theme 2C explored the accounts
expressed that considered the hub at future ECHOs and the availability of sessions to those
who did not attend.
Interpretative theme 2A: Changes to be made
Accounts provided guidance on potential changes that could be considered for future
ECHOs. Participants expressed that GPs would benefit from ECHO equipment being
installed in more practices:
“The doctor that I work with had to travel very far for teaching, and certainly when I was here
and the camera wasn't I would have had to go a great distance also and that just wouldn't
have been feasible. I think they've worked very well with that and they got the cameras into
most practices. Certainly a way for improvement would be maybe, if possible, to get them in
more. If that's possible, I don't know, but certainly I think they did very well from that point of
view.” (GPT/08)
88
Many participants also expressed that future ECHOs may benefit from clinicians having
dedicated study time to take part in ECHO:
“The only thing I would maybe suggest would be, so on Thursdays, we have teaching in our
locality groups, and I just wonder, sometimes on a Wednesday afternoon it was really quite
rushed to set that hour aside for ECHO and then come back to your work afterwards and
maybe leave a home visit until later on that evening, whereas on the Thursday, because it is
a teaching day we would be a lot more relaxed, and I think in a sense we don't feel as
pressured. I don't know if anyone else had that experience or felt like that, because certainly
I was rushing to go or to come and sit down and then having to finish when I got back and
that kind of took away from the teaching because you were kind of thinking what do I have to
do later, kind of thing. (GPT/05)
“Even if it's not about kind of work pressure and having to do stuff on the day, I think it would
be more relaxed to have more time. Like, when you have a day put aside for a study day,
because that was the aim of it, to learn more about it.” (GPT/07)
Interpretative Theme 2B: Challenges
Some participants expressed the challenges they faced when participating in ECHO. Some
felt that getting the spokes to mute and unmute was difficult and often took up time:
“I think it was difficult at times trying to get everyone involved in muting and unmuting and
there was a lot of the group time that way, but I suppose that's something that can't be
helped with the technology.” (GPT/01)
“I agree with the muting and unmuting point. Sometimes that's my fault as well so I can't
really blame anyone for that, but it was an issue throughout.” (GPT/08)
Some participants also expressed that there may have been too many spokes present at the
sessions to enhance optimal participation:
“I suppose at times it was a bit difficult trying to get participation because there maybe were
so many web cams.” (GPT/01)
“Probably I would agree with that and I suspect people would be even happier to take part or
to speak up if they're in a smaller, in their locality group. So I think you might find if you were
able to do that you would get even more involvement with people because they don't feel
like they're just out on their own talking to a web cam.” (GPT/02)
89
Interpretative Theme 2C: Future ECHO Participants
For future ECHOs, participants also expressed that it may be beneficial to consider the
membership at the hub. Some felt that if there were more experienced clinicians taking part
at the spokes, then the hub would need to reflect this:
“I think if you're setting it up for more experienced GPs it would be quite useful to have the
dermatology ECHO where you could speak to dermatologists about problem cases maybe
or cases that are difficult to manage or cases where it wasn't quite sure that you should refer
or you shouldn't.” (GPT/06)
Some felt that if there were too many experienced members at the hub then that may hinder
the participation of the spokes:
“I think if it was a mixed crowd of attendees, it would be different if we had an expert at the
table, if it became a mixed crowd with some very experienced GPs with a special interest,
some of the questions that we would ask among ourselves, you might begin to wonder is it a
silly question and you're not going to ask it. So that would be the only thing, if you were
going to broaden it out to a more than GP trainee core attendee then it might limit what we'd
ask and what we'd get out of it at times.” (GPT/03)
Participants also highlighted the benefits of being able to review missed sessions online:
“There may be the odd session, obviously, that you can't make, and I think that having them
online would be quite nice, to go back and have a look. Often there's important notes and
information in the discussion that you might not get in the printed material from the lecture.
So that would be useful.” (GPT/06)
Discussion
The results of this study demonstrate that the ECHO programme among GPST2 Trainees
has been successful in improving confidence among the participants and also resulted in an
improvement in their knowledge base.
The programme created a collaborative learning environment within which participants could
improve their knowledge and confidence to manage patients with dermatology issues
through the activity of case discussion and refinement of management options. With a
minimal intervention of five sessions, GP Trainees grew in confidence and showed an
improvement in knowledge for the management of eczema, psoriasis and acne. It is
anticipated that this programme will result in better decision-making and management plans
for patients presenting to this cohort of future GPs. Whilst it was not one of the research
90
questions or a measured outcome of this study, it is a reasonable expectation that the
improved confidence and knowledge among this cohort of GP trainees would lead to a
reduction in the need for further advice from or a referral to secondary care dermatology
services. To answer this question a different study design would be required.
While there were only eight responses (30.8% response rate) to a survey about participants
general views on ECHO, all respondents felt they had learnt through participating in ECHO,
found it at least ‘a little’ enjoyable, and 100% felt it had improved the care they provided for
patients. All would participate again in future ECHOs. While there may be bias in this low
response rate, the results indicate positive views on ECHO that were confirmed by the focus
group.
The focus group findings highlighted the safe and effective learning environment that the
ECHO platform can provide for GP Trainees in the primary care setting. The ECHO network
helped to improve care delivery by enhancing the knowledge and skills of GP trainees in
relation to a variety of areas such as: taking a holistic approach to patient care, awareness of
alternative treatments and how to effectively prescribe and administer topical medications.
Through this increased confidence, participants also voiced that the ECHO program had
increased their ability to make more appropriate referrals. Findings were also indicative that
ECHO may be an effective platform with which to reduce referrals to secondary care
services and promote care delivery by primary care providers. Due to its ability to reach
participants in remote areas, ECHO was seen as an effective learning platform that saved
clinicians time into having to travel for teaching.
In relation to potential future ECHO networks the timing of sessions may need to be
considered to incorporate protected study time. Better knowledge on how to access the
recorded session would be useful for those who have missed a particular network meeting.
This facility was actually available but required participants to share their email address with
the network administrator.
For any future network, consent should be obtained in advance of ECHO starting.
Membership of the hub is also an important factor and it was perceived that these members
need to be carefully adapted to the needs of the spokes. In summary ECHO has been
perceived by GP trainees as an effective way to deliver clinically relevant education that will
promote patient care amongst primary care services.
91
Suggestions from evaluation
- The dermatology network for GP trainees was found to be useful and should
continue for future trainees with consideration for involvement of other professionals
where relevant.
- Addressing minor technological issues with connectivity and muting microphones
could enhance the experience further
- The number of spokes in a session should be considered so there are not too many
that may detract from learning.
- Access to a recorded session would be useful for those who miss a session.
- Timing of the sessions needs considered as to whether they should be moved to a
non-clinical day.
- Participation in the ECHO network and receipt of an incentive e.g. certificate of
participation for trainees should be dependent on participation in the evaluation
process, or time should be made available to allow participants to take part in the
evaluation. This should improve the generalisability of results and prevent bias.
- Future studies should look at impact on service delivery and patient care, for
example referral rates to secondary care and management of patients in the primary
care setting, to determine if participation in the ECHO knowledge network has a
direct impact on these areas.
92
Chapter 7 - Carer support ECHO
Background
The carer of a patient can be defined as, “ The informal caregiver who provides a patient
with unpaid support for their physical, financial and psychological needs”.(19) Caring for a
patient with palliative care needs can be stressful for a wide variety of reasons as the
physical, psychological and social needs associated with a life-limiting disease are not only
experienced by the patient, but also by the family carer..(20) Reasons for stress for the carer
may include a perceived lack of knowledge, lack of confidence, fear of the unknown or fear
of the future, and a feeling of isolation to name but some.
HCPs have the potential to increase or decrease the burden felt by the carer by
acknowledging that the illness is experienced by both the patient and the carer.(21)
However, although HCPs have a responsibility to recognise the needs of the carer alongside
the patient to provide holistic palliative care,(22) obtaining support can be challenging for
carers. One challenge carers’ may face is trying to access formal carers’ support. This can
be potentially challenging due to the need to travel, the additional time travel takes on top of
the actual time at the meeting, and often the need to arrange for someone else to be present
with the person they care for whilst they are absent.
In 2015 the NIH carer’s service ceased due to fiscal issues. This carer’s service involved a
group of carers coming together for a weekly meeting over a six week period at a central
location with a facilitator. At this meeting carers were provided with relevant education to
help them in their caring role and it also allowed carers to provide and receive support for
and from one-another. This service had a positive impact on the holistic support provided to
carers, but unfortunately had to stop when charity funding ended as it was not commissioned
The Intervention
In order to continue to provide support to carers, across various geographical locations, NIH
hoped to re-establish the delivery of support to carers through the utilisation of ECHO. This
innovative carer’s service ran in collaboration with three other hospices, in rural and urban
areas, throughout NI (Southern Area Hospice, Macmillan Unit Antrim and Foyle Hospice).
The carers ECHO service involved a group of carers linking in from their own homes via
computer / laptop / hand-held device, using zoom technology and a webcam / inbuilt
computer camera. Carers were able to see each other and the ECHO facilitator at the
central ‘hub’.
93
Carers were provided education on the topics:
Ø Nutrition and Diet
Ø Medication
Ø What to expect
Ø Relaxation and mindfulness: coping with stress
Carers also had the opportunity to discuss personal experiences and learn from one-
another. In order to determine if this innovative method of providing carer support and
education was useful and acceptable to carers, the service was evaluated. The initial
evaluation was to assess carers’ views of ECHO and to explore the views of the hub
members.
Methods
Carers Perspective
At the end of the ECHO programme all carers who started the ECHO (including those who
did not complete the programme) were sent an invitation letter and PIL explaining the
evaluation. They were asked to complete a survey which included questions about their
demographic data, experience of technology prior to ECHO, and their overall views on the
acceptability and usefulness of the ECHO carers project. The survey could either be
completed on paper and returned or completed via Survey Monkey. Each carer was
assigned a unique confidential identifier code. Participants’ names were not recorded on
any tool. Consent to take part in the evaluation was presumed on completion of the survey.
This was made explicit to the carer in the PIL. The master list was kept in a locked area,
and no-one had access to it except the lead evaluator if required. Participants were
informed that the code would not be de-identified unless in the situation described under
ethical considerations.
Focus Group with Hub Members
Each hub member participating in the ECHO was invited via email to partake in a focus
group following the final ECHO session. Each member was provided with information about
the evaluation using a PIL. Before the focus group was commenced written informed
consent was obtained from each member. A focus group guide was used for the focus
group.
Ethical Considerations
It was recognised that there could have been ethical implications when approaching active
carers to take part in the survey, and the time period when the carer was approached
needed to be carefully considered.(23) Therefore the evaluation team liaised directly with the
94
carers ECHO program facilitator to ensure that it was a suitable time to ask the carer to
participate in a survey.
Carers and HCPs were provided with PILs to ensure they fully understood the evaluation
and to enable them to give informed consent. Consent was implied by completion of the
written tool (carers) or returned consent forms (hub members). Carers and HCPs were made
aware prior to data collection that every endeavour would be taken to ensure their privacy
and identity was fully protected throughout the duration and after the evaluation was
complete. Each carer and HCP was allocated a unique confidential identifier code in order
for documentation related to that carer to be numbered instead of their details being used;
only the evaluation team had access to the carer’s or HCP’s identification number details.
The access to original data was restricted to the evaluation team.
Analysis
All data was analysed using descriptive statistics in aggregate. Focus groups were
recorded, transcribed and analysed using thematic analysis.
Results - Carer Survey
Fifteen carers were recruited to ECHO but only four carers went on to participate in the
carers ECHO and all responded to the survey. Demographic data of participants are shown
in box 5. Carers were of a wide age range, and the majority were female. One was still
working, one was a carer in the community, one was retired and one had given up work to
care for her husband. Three attended all four sessions. Participants had varied previous
experience with technology prior to participating in ECHO.
Box 5 Carers Demographic Data
What is your age?
Answer Options Response
Percent Response
Count
35 to 44 25.0% 1
45 to 54 50.0% 2
65 to 74 25.0% 1
What is your gender?
Answer Options Response Percent
Response Count
Female 75.0% 3
Male 25.0% 1
95
With regard to l iv ing arrangements, please t ick al l the answers that apply
Answer Options Response
Percent Response
Count
Live with the person you are caring for long term 50.0% 2
You stay with the person you care for sometimes 25.0% 1
You and the person you care for don't live at the same address 25.0% 1 When taking into account your caring commitments, do you f ind i t easy to attend face-to-face meetings aimed at support ing you as a carer i f you want to?
Answer Options Response Percent
Response Count
Yes 0.0% 0
No 33.3% 1
Unsure 66.7% 2
With regard to transport do you...
Answer Options Response Percent
Response Count
Have your own car 75.0% 3
Get public transport 25.0% 1
What is your relat ionship to the patient?
Answer Options Response Percent
Response Count
Spouse 50.0% 2
Child 25.0% 1
Other 25.0% 1
What age is the person you are caring for?
Answer Options Response Percent
Response Count
41-50 25.0% 1
61-70 25.0% 1
71-80 50.0% 2
What gender is the person you care for?
Answer Options Response
Percent Response
Count
Male 50.0% 2
Female 50.0% 2
96
Does the person you care for need someone with them at al l t imes?
Answer Options Response Percent
Response Count
Yes 25.0% 1
No 50.0% 2
Unsure 25.0% 1
Have you previously attended a carer support group?
Answer Options Response Percent
Response Count
Yes 75.0% 3
No 25.0% 1 With regard to your experience with technology prior to start ing the ECHO please put one t ick in each row that best applies to you
Answer Options No Experience
A l i t t le Experience
Quite a lot of
Experience
Very Experienced
Rating Average
Using a computer 0 2 1 1 2.75
Using Skype 1 2 1 0 2.00
Emailing 0 1 1 2 3.25 Using social media e.g. Facebook, twitter etc.
2 1 1 0 1.75
Use of the internet to find out information 1 1 2 0 2.25
Online Shopping 1 0 2 1 2.75
When asked their views about the ECHO network, all respondents found the sessions either ‘very’ or ‘quite’ interesting, all found them ‘very’ relevant, and one person found it ‘quite’ stressful. The others did not. See figure 13 for more information.
Figure 13 Carers Views on the ECHO Sessions
0 0.5
1 1.5
2 2.5
3 3.5
4 4.5
Generally interesting
Generally relevant
Helped me in my
caring role
Enjoyable Tiring Stressful
How did you find these sessions?
Not at all
A little
Quite a lot
Very much
Unsure
97
There were four sessions in the program and all respondents stated the sessions were
‘excellent’. When asked about the technology, 100% stated it worked well but only one
person stated they found it easy to use. Half felt they could communicate with others during
the sessions via the technology well, three people found it easy to relate to others, one
found it difficult to speak out and two would have preferred being in the same room. Figure
14 shows participants views when asked about using the technology compared with
traveling to a meeting. Half would prefer a face to face carers support service, and half
preferred the ECHO technology. All would participate again and would recommend the
service to others. Reasons given for stopping attending are in table 15.
Figure14 Carers views on ECHO compared with traveling to a meeting and feelings of
support
0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0%
I found it convenient time
wise
I found it convenient not having to travel
I found it convenient not having to leave
the person I was caring for to attend the meeting
I would have preferred to do
this from somewhere that was not home
I would have preferred to travel to a meeting
How did you find being able to attend a session using the technology rather than having to travel to a meeting? (Please tick all that apply to you)
0
1
2
3
4
5
By other carers
Did you feel supported during the sessions? (Please put a tick in each line)
Not at all A little Quite a lot Very much Unsure
98
Table 15 Reasons for stopping attending sessions
If you stopped attending sessions, was it because you...(please tick any applicable
answers)
Answer Options Response
Percent Response Count
Would have preferred to meet people face to
face 25.0% 1
Too busy due to other life circumstances 50.0% 2
The person I care for became too unwell 50.0% 2
Other (please specify) 50.0% 2
Results of Focus Group with Hub Members
Five HCPs who participated at the hub for the ECHO programme for Carers participated in
the focus group. Participant’s profiles are shown in table 16.
Table 16 – Hub focus group Participants Profile
Profession Role
Palliative Care Nurse Consultant Hub Member
Social Worker Hub Member
Physiotherapist Hub Member
Project Manager Facilitator and co-coordinator
Occupational Therapist Hub Member
The data was analysed by adopting a thematic analysis framework as described by King and
Horrocks.(24) Analysis of the focus group data uncovered two overarching themes, each
with their own descriptive and interpretative themes that are outlined in thematic diagrams.
Overarching theme one, challenges and suitability of Echo for carers explored the
challenges faced when conducting this particular ECHO program, and how suitable it was for
this group (figure 14). Overarching theme two, consideration for the future of ECHO for
carers, depicted the particular elements of ECHO that participants perceived needed
considered to enable the success of future carer’s programmes via the ECHO platform
(figure 15).
99
Figure 14 Theme 1 ‘Challenges and Suitability of ECHO for Carers’ for Carers ECHO
Descriptive Themes Interpretative Themes Overarching Themes
1 Challenges and
Suitability of ECHO for Carers
1A Suitability of ECHO
for carers
Built a rapport with carers
Carers gained knowledge and support
Encouraged consideration of end of
life
Support in their own home
1B Challenges of ECHO
for carers
Lacked support of face-to-face
Did it address carers needs?
Too many healthcare professionals
Technology
Recruitment and retention was challenging
100
Figure 15 Theme 2 ‘Consideration for the future of ECHO for Carers’
Descriptive Themes Interpretative Themes Overarching Themes
Overarching Theme 1: Challenges and Suitability of ECHO for Carers
This theme encompassed how participants perceived ECHO to have some elements of
suitability in relation to providing support and information to carers of people with a life-
limiting condition. Interpretative theme 1A depicted how participants perceived that ECHO
did facilitate building a rapport with carers and also helped carers to gain knowledge and
support for their caring role. Perceptions also highlighted how ECHO provided a platform
which was accessible in carers own homes and also encouraged carers to consider the end
of life for the person they cared for. Interpretative theme 1B however illuminated the
perceived challenges of using this platform for carers to provide support and information.
2 Consideration for
the Future of ECHO for Carers
2A Future Hub Members
Facilitator is key
Rolling programme
2B Future for carers
Do not need a disease specific
ECHO programme
Blended approach needed
Recruitment considerations
Good to know about the person being
cared for
101
Accounts demonstrated issues regarding the lack of face-to-face support for carers and
concerns from hub members regarding optimally addressing the needs of the spokes.
Findings also conveyed concerns regarding the number of professionals at the hub in
comparison to carers, and issues regarding technology. Recruitment was also perceived as
a challenge and this was discussed further in interpretative theme 1B.
Interpretative Theme 1A: Suitability of ECHO for Carers
Accounts portrayed how the ECHO for carers allowed professionals at the hub to build a
rapport with the carers. Some felt this rapport was similar to that at face-to-face carers
groups:
“I think there was a feeling of a bit of rapport with the carers. I know there was a small
number but you did feel that it was kept quite informal and you did feel that you got a bit of
personality from them. So I don't think it was very cold compared to face to face, I think there
still was that bit of interaction with people on a certain level.” (CE3)
Many also felt that the programme provided carers with knowledge and support in relation to
being a carer:
“I just wanted to say one of the real positive experiences has been the feedback carers have
given me. A carer has been in touch to say he felt supported in it.” (CE5)
Many participants perceived that carers benefited from the information provided at the
sessions:
“I think the information sessions and the teaching element of it was good and I think the
carers who were involved gained a lot of knowledge through that. The presentations that I
saw were very good and very informative and I'm sure those bits of knowledge gained.”
(CE2)
Findings also highlighted that similarly to other carer groups, ECHO encouraged carers to
consider the end of life for the person they cared for:
“I think the other positive thing is one of the carers started to think about her end of life
wishes for her mum. I know that would happen in any carer session really, that if people are
feeling supported they feel confident to make those decisions. But that was one of the
things.” (CE5)
102
“I think the good thing was that it maybe started to make them [carers] think. Like the lady
who said she was going to think about end of life care for her mum and what her mum's
wishes would be and things. It maybe made them think about things that they wouldn't have
if they hadn't been part of the group.” (CE4)
Findings illuminated perceptions that ECHO was a positive form of delivering a carers
program as the support was provided in their own home:
“It [ECHO] allows the carers to be in their own home.” (CE1)
“I was just thinking about there's times [a carer] would have been sitting with a cup of coffee,
and in some ways that's nice that people can be relaxed in their own home.” (CE3)
One carer expressed that this would have enabled them to still carer for their husband whilst
participating:
“I think one of the positive things, and I know there was a carer, she was caring for her
husband with Motor Neuron Disease. She said she would have liked to join just for that
security of being able to access that training but still being there to be able to care for him in
the home.” (CE5)
Interpretative Theme 1B: Challenges of ECHO for Carers
Findings also conveyed that participants perceived some challenges to delivering an ECHO
program via the ECHO. Some felt that it lacked the support that would be available at a face-
to-face carers group:
“There wasn't much room to actually provide support to people, compared to, say, a group
setting where if someone was in distress you could spend more time with them or if
someone was very quiet and just wanted to listen to the others. I think it was more
information sharing, and that was very good but it lacked, I feel, the support of one to one or
group face to face contact. So I think that was missing in it, just because of the set up of
Echo.” (CE2)
It was also conveyed that carers may have missed the support of each other that is
perceived to be provided at a face-to-face session:
“I suppose I'm just comparing it with what I've known in the past with carers' groups, and I
think the main thing about a carers' group, I see myself as facilitating it but the carers
103
support each other, although there's a programme but the carers sort of learn from each
other and support each other. And I suppose that's the other side of it. The information
sessions were all very good but I still think that element was missing.” (CE2)
Findings portrayed that hub members also had concerns before ECHO started as to whether
it would optimally address the needs of carers:
“I suppose I did have my concerns how it would work with carers in that particular setting
and I thought would it address all their needs?” (CE5)
“I've been involved in a couple of other ECHO projects and my experience has been with
other professionals. I suppose my concern with the carers was, as has already been said, is
that we're working with service users and protection around them.” (CE2)
Many also expressed concerns regarding if ECHO would provide optimal support and how
this would be managed:
“I was also concerned around, when I had heard it talked about before, I had heard the word
"support" used, as well as information, and I suppose I was concerned about how you could
support a group remotely and how, if there was a huge group of carers, which wasn't the
case in the end, but if it had been, how to facilitate remotely, and if you had someone with an
agenda or who had very strong views, I was concerned how that could be managed via the
ECHO technology.” (CE1)
One participant expressed that prior to ECHO commencing; they had perceived it would
entail more practical support:
“I thought it would have been more kind of practical support that they would be wanting,
more of hands on type learning.” (CE4)
Accounts additionally illuminated that participants perceived some the sessions which carers
requested may not have been fully addressed. Some felt that if sessions didn’t take place
then the carer may have missed out:
“I suppose the weeks that were cancelled for a certain reason then didn't happen, and they
were issues that people had asked at the sessions to be addressed. So it's just if you missed
your week then it was gone. So the full programme wasn't really delivered as such.” (CE4)
104
Some also portrayed that they were unsure if the sessions fully addressed what the carer
had requested:
“The first week was about getting the agenda and it was supposed to be getting what carers
wanted, but I suppose we were all new to it as well and you were never sure if you really got
the topics or got the things that were actually important to them, and is there a way of doing
that or do you need an introductory session and then for them to come back to do that, or
afterwards. But I don't know if in some ways we sort of set some of the sessions; it wasn't
always what they wanted.” (CE3)
Findings illuminated that participants perceived there may have been too many professionals
at the hub at times, in comparison to carers:
“Sometimes I just felt that there were six or seven of us [professionals at the hub] and one or
two carers and it was just unbalanced a bit.” (CE2)
Accounts also portrayed that having a lot of professionals at the hub also meant using a lot
of professionals at the onetime to deliver a programme to a small group:
“You need a lot of professionals [for the hub] and you're using a lot of professional time for a
small group for [ECHO], if you want it you can have it teaching but then you could use a few
professionals but then you don't have the support element. So it's hard to get a balance
between the two.” (CE3)
Accounts additionally highlighted the challenges faced during the ECHO programme in
relation to the technology used. Prior to the commencement of ECHO some were concerned
how the technology would work:
“I suppose I was a bit nervous about technology and how that would work and whether it
would be complicated or not. I thought the idea sounded good but for me, it was just sort of
wondering how the technology would go.” (CE3)
Some felt this was the reasoning behind some carers not partaking in the programme:
“So I think technology was one of the reasons why some carers didn't participate.” (CE5)
105
However many of these concerns were eased during the ECHO programme:
“One of my fears had been about the technology, but actually that side of it was very easy.
So it is a good system to link into. It works well on a teaching perspective, being able to get
information across.” (CE3)
Some technical difficulties were experienced during the ECHO sessions though:
“Our only problem was we couldn't hear very well the first couple of weeks, so we ended up
buying speakers and then that helped, but we found it very hard to hear. Anyone hadn't
muted themselves there was a lot of background noise so it was hard to hear who was
talking. So that was the only thing with the technology for us.” (CE4)
“One week [at ECHO] I couldn't see myself, and from then on every time I tried to access
Zoom, IT had worked on it an hour beforehand, yesterday for an hour and a half, I had to
come here today [the Hub], they still couldn't get it to work. But I've had them [IT] working on
it on three occasions recently for different things and we still can't get the network
connection, for some reason.” (CE1)
Findings depicted the challenged the programme faced due to issues regarding the
recruitment and retention of carers. Within the recruitment stage there were 15 carers to
partake but only four participated. Many felt it was hard to recruit due to difficulties in
explaining a new mode of delivery for carer support and information:
“I think recruitment was one of the really challenging things with this pilot. Because it was
new I think it was hard to explain the process to people, because it was just a concept at that
time.” (CE5)
“Although I have some experience of ECHO in other settings, it was trying to explain how it
would work, and I think, even if you're running a carers' group face to face, it's difficult to
recruit people, to get them to come, because you couldn't explain what it is, and sometimes I
felt that I was trying to explain something that I didn't know... I couldn't put it across well, I
think, to some of the carers just how it would work. So I think that put people off as well.”
(CE2)
106
Some perceived prognostication of the person being cared for challenged recruitment:
“From our point of view it was very hard for us, because the original brief was that they had
to have a prognosis of at least six months and for us that ruled out anybody we had as a
patient at the time. So we had to look towards oncology, the Macmillan nurses, to see if they
could get anybody for us, and the ones they had been dealing with had maybe just got bad
news or things like that. So they found it hard to recruit so it was people who were already in
a support group were our best target audience for the pilot because it was just hard to get
anybody else that fitted what we were looking for.” (CE4)
Retention was also a challenge within the programme:
“We had 15 participants that wanted to participate in the ECHO. Other lines just took over
and they couldn't always participate in the sessions, and I suppose that's one of the things
that's always going to happen in this type of setting,” (CE5)
“it was smaller numbers, very small numbers and not consistent people attending, so that
made it more difficult.” (CE1)
Overarching Theme 2: Consideration for the Future of ECHO for Carers
Overarching theme 2 encompassed the perceptions of the hub members regarding issues
that should be considered for future ECHO programmes for carers. Many felt that for an
optimal hub the facilitator was very important and it was also perceived that other
professionals may benefit from a rolling programme to enable attendance. In relation to the
carers, participants portrayed that the ECHO programme for carers does not need to be
disease specific; however a blended approach may be of benefit. Accounts also illuminated
ways in which to optimise recruitment and the benefits of knowing about the person the carer
is caring for.
Interpretative Theme 2A: Future Hub Members
Findings highlighted the importance of an effective facilitator in coordinating the programme
and providing consistency to the carers:
“I think we were blessed in having a very good facilitator who coordinated the whole thing.
That was a lot of work, a lot of pressure, and she facilitated the meetings extremely well.
That's a big commitment and I think for it to go forward and for it to be successful you need
somebody like that.” (CE1)
107
“You still need that one person [facilitator] who's a good coordinator, to be consistent for the
carers as well, to make it feel like they're connecting to something rather than just looking at
a programme. “ (CE3)
Some felt it was also important to have a rolling programme to enable professionals to
attend relevant sessions and not every session:
“I do think it would be very hard for us [professionals] to justify our time here [at ECHO] when
there's so many other things that we are supposed to be involved in with our patients here
[in work]. If it's a small group, and a support group, you know justifying our time to our
managers would be very hard whereas if it's a rolling programme that you're there for set
weeks or certain weeks I think that would be more sustainable.” (CE3)
“I wonder if it's to be a rolling programme or an ongoing teaching programme or something
could you rotate it around the trusts, because it just means it would keep it fresh but also it
would open it up to different staff and maybe allow them to maintain it so that you're only
doing it every so often, which would also justify it to managers.” (CE3)
Interpretative Theme 2B: Future for Carers
Participants highlighted their perceptions of carers taking part in future ECHOs. Findings
highlighted perceptions that future ECHO sessions for carers did not need to be disease
specific:
CE2: “Carers are carers and even though they're caring for someone with a different illness,
even if it was all cancer related illness there would be differences in care for someone with a
brain tumour as opposed to something else. So I think there's more common ground, no
matter what the illness is. There's more commonality between just being a carer than trying
to break it up. It's just an opinion, you know?
CE1: I would agree.”
“Carers had the same issues, the same feelings of isolation and the same family issues. The
specific stuff around symptom control, some of that might be similar as well but we didn't go
into that detail so we felt it was only fair to open it up to people caring for any [life-limiting]
condition.” (CE5)
However some felt that a blended approach to the support provided may be needed in the
future:
108
“It’s one of them things where one size doesn't fit all and maybe if we used the ECHO for the
information and then followed up face to face or had a blended approach then you would
use the ECHO to save professionals time, especially if you were doing the one to one
sessions with the carers, because I can imagine that would be quite time restrictive. But
yeah, maybe look at different ways of incorporating this way to have reached them in the
home and then bring them out for further support, if that's possible.” (CE5)
Participants also talked of how recruitment issues may be addressed in future Carers
ECHOs. Some felt that recruiting through primary care providers may enhance participation:
“I'm just wondering, for us, we're seeing people in an acute setting or in a hospice setting,
but it might be people that you could target, people like the community nurses and district
nurses and the hospice nurses might be the people, because they're going into the homes
and they're seeing how people are managing, they would be better suited to recruit people in
their own home and talk about it there.” (CE3)
Findings additionally highlighted that knowing details of the person being cared for may help
in providing optimal care:
“It may also be useful to know a little bit about the person that the carer is caring for and their
condition. There could be many and varied conditions and if you're giving a talk you need to
be sure that you're actually addressing what their needs are and not going off on a tangent
about a topic that has no relevance to the condition of the person that they're caring for.”
(CE1)
Discussion
This ECHO program for carers found it hard to recruit and retain sufficient numbers of
carers. While all of the carers who participated in the ECHO network participated in the
evaluation, it was not possible within the scope and timeframe of this evaluation due to the
need for formal ethical approval to survey the people who did not participate in the ECHO
network to determine their reasons.
The survey results highlighted that the majority of carers viewed participating in the carers
group via ECHO as a positive experience. Some were unsure if they would find it easy to
attend face–to-face meetings due to caring commitments; although the majority had
previously attended a carer’s support group. Although many felt it was convenient to use the
ECHO model for the carers group, some still would have preferred communicating with
109
others in the same room. Carers gave excellent feedback regarding the sessions delivered
and felt very much supported by all professionals involved. However, one carer found some
of the sessions stressful but it was not possible to explore the reasoning for this. Overall, it
was perceived that the technology worked well and all participating carers would recommend
the ECHO programme and participate again. While this feedback is positive, it would be
dangerous to draw firm conclusions based on the responses of four participants.
The focus group findings illuminated the intricacies of the experiences of hub members
involved in this particular ECHO programme. Similarly to the survey findings, professionals
highlighted the need for face-to-face support alongside the ECHO sessions. However it was
perceived that the ECHO platform enabled carers to gain knowledge and support whilst also
fostering relationships between the carers and professionals. The ECHO platform provided
support in the carers own home which was seen as beneficial, however there were concerns
as to whether the sessions appropriately addressed the areas which carers wanted covered.
Many acknowledged the benefits of ECHO in encouraging consideration of end of life needs,
however it was also highlighted that this was a feature of many carer support groups.
Recruitment and retention was challenging, and professionals felt that this was due in part to
the difficulties in explaining the new concept to carers, that the technology may have been
off-putting to some, and also due to the prognosis of the people being cared for. It was
highlighted that for future recruitment it may be of benefit to utilise primary care providers to
identify potential carers. For future ECHOs the facilitator needs to play a key role as the pilot
programme was well supported and coordinated by the facilitator. Professionals depicted the
difficulties in justifying time to attend the ECHO clinics and suggested a rolling programme
may be more feasible for future sessions. Findings also conveyed that although disease
specific carers groups are not perceived to be required, it would be beneficial to hub
members to be aware of the needs of the person being cared for.
Due to the very low numbers of participants it is difficult to draw firm conclusions on the
usefulness or not of the ECHO model in providing support to carers. While there were
positive feedback from both carers and hub members, there were also significant issues with
recruitment and retention, and likely cost effectiveness due to the large numbers of HCPS at
the hub and few carers at the spokes. Perhaps most significantly the perceptions of some of
the hub members towards seeing ECHO as a potentially useful means of carer support grew
stronger through the process. An economic analysis was not possible within the scope of
this study. The lack of face to face contact is also a recurring theme for both carers and hub
members. A larger study with sufficient time and funding to get ethical approval to explore
the issues raised here more in depth is necessary to determine the future use of ECHO in
110
this setting. The feedback from participants and hub members has helped identify some
issues that could potentially be addressed in future programs e.g. using a mixed or blended
approach with some ECHO sessions for information sharing and some face to face sessions
for more supportive care.
It was not within the scope of this evaluation to look at the impact on the carer or the person
they cared for, and future studies should look to address this to consider the outcomes for
carers.
Suggestions from evaluation
ECHO may be a viable method with which to deliver carers support in the future but
further research is needed as this study was of insufficient size to draw firm conclusions.
Future studies with appropriate time frames and funding need to explore the feasibility of
ECHO as a platform to deliver carers support and also examine the outcomes for carers
and those they care for.
- A blended approach may be most helpful to also allow face-to-face sessions within
the programme.
- Carers need to be given time to reflect on the chosen topics to ensure they address
their needs.
- Recruitment needs to be carefully considered and other professionals who are
working with this client group need to be involved i.e. primary healthcare providers.
- A rolling programme needs to be considered to ensure that professionals can attend
sessions that are relevant to their expertise.
- The facilitator role needs to be appropriately assigned and remain consistent for the
carers and professionals.
111
Chapter 8 - Hub Feedback
Methods
To determine if there was a benefit to hub members in participating in ECHO knowledge
networks and to determine their views on having participated in ECHO, all hub members
were sent out a survey via Survey Monkey as described in Chapter 2.(8)
Results
Eighteen hub members responded to the survey- seven from the optometry ECHO, nine
from the Diabetes ECHO, one from GP dermatology and one from the NH ECHO. Results
are shown in table 16. All participants (100%) who participated in the survey indicated they
had learnt through ECHO, both from hub and spoke members, they found it enjoyable and
would recommend it to others. They felt it helped translate knowledge into practice, and all
responders indicated they would be willing to participate in future ECHOs. They were
positive about the use of ECHO technology.
Table 16 Hub feedback on ECHO
Number %
Profession Physician
GP
Optometrist
Allied Healthcare Professional
Other (please specify)
6
1
4
4
3
33.3
5.6
22.2
22.2
16.7
Whilst participating at the ECHO
Hub did you (all that apply)?
Facilitate a session
Deliver a teaching presentation
Participate in discussions regarding a case
Provide specialist advice / guidance on
topics being discussed
5
11
15
14
27.8
61.1
83.3
77.8
Do you feel ECHO offered you
an effective platform with which
to pass on relevant knowledge
and skills to others?
A lot
A little
No
17
0
1
94.4
0
5.6
Overall do you feel you have
learnt personally through
participating in ECHO?
A lot
A little
No
17
1
0
94.4
5.6
0
112
Did you learn from other Hub
members?
A lot
A little
No
13
5
0
72.2
27.8
0
Did you learn from participants
at the spokes?
A lot
A little
No
14
4
0
77.8
22.2
0
Did you find participating in
ECHO enjoyable?
A lot
A little
No
18
0
0
100
0
0
Do you think your participation
in ECHO has changed your
practice at all?
A lot
A little
No
Unsure
4
12
0
2
22.2
66.7
0
11.1
Do you think the format of
ECHO helps translate
knowledge from teaching into
practice?
Yes
No
Unsure
18
0
0
100
0
0
Would you recommend ECHO
to other Healthcare
professionals in your area?
Yes
No
Unsure
18
0
0
100
0
0
Would you like to participate in
ECHO sessions if the
opportunity arose in the future?
Yes
No
Unsure
Unanswered
17
0
0
1
100
0
0
Do you have ideas as to how
ECHO could be embedded
within your own working week to
provide a sustainable model of
maintaining a community of
practice which would benefit
patients?
Yes
No
Unsure
Unanswered
14
0
2
2
87.5
0
12.5
113
Participants had the opportunity to write comments about how ECHO could be embedded
within the working week to provide a sustainable model of maintaining a community of
practice which would benefit patients. Comments were generally positive and included the
need for having participation job-planned with appropriate funding / backfill, that either hub
members should be able to link virtually or there should be other hub sites across the region,
and the need to gather evidence that the ECHO program in NI affects practice. Comments
were:
“ECHO could be adopted to facilitate sharing experience & learning for my local trust
diabetes team as the team spans primary & secondary care & across three hospital
sites, could be intra/inter- professional potentially be use to facilitate training for d-
Nav service reviews across the trust & wider if the service gets funding to rollout
across NI or even UK.”
“A shorter programme to address diabetes education for staff, as a rolling
programme, aiming to provide base line information and support practices. To create
specific " levels" / programmes of support and education. To engage practice nurses
in sessions and identify topics of use to them. Engage other members of specialist
diabetes teams across various trust to input and spread learning regionally. ? spread
of practices in spokes- to enhance learning from all areas. Establish as part of
Diabetes Network ( regional strategy)”
“As detailed in draft submission for 2016-2017: Key themes in the responses to the
first draft I did with ideas about how it could have wider participation were to keep
disciplines together (not separate GP and practice nurse ECHOs) and to recognise
and facilitate spoke participation by finding backfill. The hope is that GP Federations
could help engagement. However, advice from Barney McCoy was that spoke
participants needed funded. I have no idea about this. The devil is in the detail of how
this will run to get wider representation from spokes and even within the hub. I
envisage a core hub, inviting other clinical staff to participate here and there perhaps
via video link rather than physical presence. I also suspect that we should run 2-3
diets of a fixed didactic programme in 1 year, and have other sessions in between
that focus on cases in the classic ECHO model. We would need to discuss the detail
of this based on your SuperHub experience. We can then adapt a specific running
order in agreement with core hub members. I talked to Damien McMullan, palliative
care consultant in Derry who is a regional clinical lead. He is very keen to bring this
to palliative care more and will talk to Max. However, he’d like us to club together as
a few specialities like diabetes, palliative care and perhaps another like respiratory to
114
create another hub in the northwest, perhaps focused in the Hospice or Hospital. A
second one could run from NI Hospice Belfast. That would allow much wider
inclusion, but would mean funding that possibly exceeds our current capacity. It
would mean that a session would take up 2-3h for people from this area, rather than
5h as at current (travel time included).”
“I personally have flexibility in my working week.”
“Needs properly job planned. Also the funding for ECHO was in my opinion not done
well. There was only 2 weeks notice given to hub members at the beginning to
cancel clinical sessions rather than the mandatory 6. Also, trust were paid in advance
for backfill of these sessions but to date have not passed on this money for the
purpose that it was intended resulting in a significant increase in my hospital waiting
lists because I participated in ECHO under the ?false pretence that it would be
backfilled.”
“ECHO is an ideal methodology to reach and work with a significant number of HSC
and other staff which could be otherwise difficult due to current constraints on staff
time. The Safety Forum's model for working is through themed "breakthrough QI
Collaboratives". This involved bringing HSC and other staff (e.g. nursing homes)
together for a 12-18 month period through learning sets (face to face\) and action
period in local organisations. An area of focus is identified, for example, sepsis, falls,
etc... and front line teams come together to learn how they can improve practice in
the identified area; making changes, measuring and striving for reliability and
sustainability over time. ECHO provides a way to bring more staff together - they can
meet in their own local organisations through the multi-web technology and have
access to colleagues in other Trusts. This helps to reduce time away from their own
organisation promoting more rapid QI and also means that more frequent sessions
can be run rather than waiting for 4 months between each face to face learning set. It
is felt that this would be an excellent way to progress taking account of current
constraints and building that community of practice in a number of areas.”
“An hour or so allocated to this during the working week would seem to me to be
perfectly acceptable. However as with other associated professions, enhanced
remuneration should be taken into account as part of the CET (Continued Education
and Training) grant in recognition of the improvement both to the quality of patient
care and improved efficiencies within the primary and secondary care interface.”
115
“Could become part of working week if properly job planned. As it stands funding for
backfill went direct to trust finance departments, there was insufficient notice to
reschedule clinics and my clinics and patients have suffered as a result of my
absence to attend ECHO, backfill money has not to date been passed on by trust.”
“other specialist diabetes dieticians GP's within own trust regional dietetic projects
e.g. outcomes”
“No harm to the excellent staff at Hospice Hub, but it takes me 3h round trip to get to
and from the hub from work in Derry - half a day. If we did this at NW Hospice or
WHSCT location, more of us from the West and South West could take part as Hub
more regularly and more often. I'd definitely like to have second hub at West/NW
region from which different specialities could run ECHO at different stages of the
week. There should be two hubs: Belfast and Derry to greatly increase our reach and
capacity. Include and actively invite practice nurses. The pilot had mainly GPs with
expressed interest and some with less and a few PNs. PNs deliver most diabetes
care throughout NI, particularly where the GPs in their practice have less interest in
diabetes. We need GPs from the WEST involved -conspicuous absence that
unnerved me! Maybe they wouldn’t engage, maybe they weren’t asked. Openly and
widely invite and encourage, targeting a geographical spread of GPs across an area.
That may facilitate other participants. Is it feasible that more people could observe
online while live, though not be able to speak/be seen - reducing bandwidth
requirement at hub for extra people to add in? I think a regular session with staff in
the hub rotating in is a good idea from same Trust, missed Trusts or, different Trusts
in rotation. Rotating from one Trust manning hub to another would allow attraction of
GPs and PNs from their area I hope, spreading the love around. An ongoing
programme with advance notice of topics will allow participants to perhaps attend a
series in a row for a cycle of topics, and dip in and out of others later as the series of
topics come round again as they inevitably will. I want more!”
“Since being involved in echo I have now become and enthusiast and can think of a
lot of different scenarios where this would be excellent for educational delivery and
also it lends itself beautifully to multidisciplinary interaction”
“Although I am very enthusiastic about ECHO and I have greatly enjoyed I think it
would be important to gather evidence that ECHO is achieving what is meant to be
achieving Evidence from the USA may not be generalisable to our setting. We may
need to do a trial”
116
“ECHO not only provides a safe space for tele-mentoring, but it also offers an
excellent vehicle to share experiences, two-way and every way. By facilitating peer
discussion and peer-to-peer learning, it provides a governance framework, building
trust, that will enable care to be shared, and delivered, safely, closer to the patient's
home. ECHO is a model of excellence for demonopilisation of medical knowledge
and building a strong community of practice.”
“ECHO was a fantastic experience. As a hub member, I arrived thinking it was all
about cascading my knowledge and experiences to the spokes, but I learnt a great
deal from the spokes, from their insightful questions on my presentations, directly
from the comments but particularly from conversations between spoke members.
The nature of the work done by the spoke members was brought home to me by
ECHO, and ideas and motivation to carry on work between hub and spokes greatly
increased. I hope can continue, ideally monthly, and be developed, for example to
ensure dissemination of spokes' learning to 'sub spokes'. The phrase 'community of
learning' may have sounded trite at the beginning, but that's truly what ECHO was all
about, and has created: this community now needs nurtured.”
“It may prove difficult to continue on a weekly basis as I would be time limited, but
perhaps some form of monthly platform?”
“Ophthalmic staff within the Health and Social Care Board are progressing work to
reform eyecare service by delivering an integrated approach to the commissioning
and provision of eyecare services. This work involves the development of a pathway
approach for high demand and long term ophthalmic conditions. Better ways of
working must be considered in order to manage demand and afford patients good
outcomes and experience by adopting a multidisciplinary team approach spanning
the primary and secondary care settings. Ophthalmic services are one such service
in which patients have access to qualified, skilled and well equipped professionals in
both settings and hence significant opportunities exist for joined up patient-centred
care. Eyecare pathways incorporate models of care which have governance, intra
and inter professional communications and evidence based care as core
requirements. ECHO facilitates an innovative approach to embedding and supporting
all of these essential components. Implementation and roll out of ECHO within the
work I do in HSCB Ophthalmic services (Optometry/Ophthalmology) would have
several benefits: 1. Regular and consistent opportunities for the two way sharing of
knowledge and experience between ophthalmic professionals 2. Opportunities for
117
service development in relation to a shared approach to the management and review
of risk stratified patients who have 'stable' long term eye conditions. This would
reduce demand on secondary care review appointments and allow patients to
receive safe and effective care closer to home. 3. Optometrists in primary care within
a community of practice supported by ECHO would, in time, transform into pockets of
excellence which would become a valuable resource adding resilience into the
system. 4. ECHO facilitates the development of cross-professional links to support a
holistic approach to addressing the needs of patients. For eyecare this will
incorporate links with the voluntary sector, low vision support services, medical
services, pharmaceutical services and other groupings.”
Discussion
While many hub members thought that participating in ECHO was going to be of benefit to
the spokes, all members also learnt themselves from both other hub members and those at
the spokes. All found participation enjoyable, and that the format of ECHO helps translate
knowledge from teaching into practice. All would participate again in the future and would
recommend to other HCPs in their area. The need for adequate funding and job-planning
was highlighted in order to allow hub members to continue to participate.
118
Chapter 9 - Overall Discussion
Over the last six months, four ECHO knowledge networks have been operating, through a
total of 37 sessions, involving over 150 HCPs in Northern Ireland. In addition a small ECHO
with four carers of palliative care patients has been facilitated, with much learning about the
options for future use and research.
Two networks (Dermatology for GP trainees and Optometry) showed statistically significant
improvements in knowledge and self-efficacy, in keeping with the results from the initial NI
ECHO pilot in community hospice nurses.(1) The response rates for the Diabetes and
Nursing Home networks were low and statistical analysis couldn’t be performed.
Feedback from the four networks in HCPs showed an overall very positive view on the
ECHO networks and their impact on patient care by both the participants in the spokes and
the hubs, and this was confirmed by the focus groups. The combined feedback from the
four networks is shown in box 5, and demonstrates that the most useful areas in the
sessions are the presentations followed by the case based discussions. All participants felt
they had learnt through participating in ECHO, all had enjoyed it on some level, and all felt it
had improved the care they provided for patients, perhaps the best marker of the networks.
Only one person across the networks did not think the format of ECHO helped translate
knowledge from teaching into practice more than other teaching sessions they had been
involved in, and no one said they wouldn’t recommend it to other HCPs. The majority found
ECHO gave them access to education that would have been hard to access due to
geography (26 out of 34 respondents – 76.5%), that it was a good medium to access
teaching / education at a different location from where they worked (34 out of 35
respondents – 97%), that any technical difficulties were acceptable and did not put them off
participating in ECHO (31 out of 33 respondents – 93.9%) or reduce their learning (31 out of
32 respondents – 96.9%). On a scale of 1 to 5 (1- poor, 5 – excellent) 34 out of 35
respondents (97.1%) rated it 4 or 5, and 100% would participate in ECHO sessions in the
future.
119
Box 5 Summary of Spoke Results on General ECHO feedback
1. Rating on a scale of 1-5 the quality of learning /
usefulness from each area (1- poor, 5- excellent)
1 2 3 4 5
Review of previous session 0 0 6 12 12
Presentations 0 0 1 12 24
Case based discussions 0 0 2 11 19
2. Overall do you feel you have learnt through
participating in ECHO?
A lot 32
A little 3
NO 0
3. Did you find participating in ECHO
enjoyable?
A lot 31
A little 4
NO 0
4. Do you think that participating in ECHO has improved the care
you provide for patients?
A lot 28
A little 6
NO 0
5. Do you think the format of ECHO helps translate knowledge from
teaching into practice more than other teaching sessions you have
been involved in?
Yes 31
NO 1
Unsure 1
120
6. Would you recommend ECHO to other healthcare
professionals in your area?
Yes 33
NO 0
Unsure 1
7. Have you used any of the online
resources via Moodle, and if yes have you
found these useful (dermatology network
did not have access to this)?
Tick if used
and found
useful
Tick if
NOT
useful
Power point presentations 15 1
Video of the teaching
sessions 16 2
Video of case presentations 14 2
Other supporting materials 17 2
8. Regarding ECHO technology…
Agree Disagree Unsure
It has given me access to education that would
have been hard to access due to geography 26 5 3
It was a good medium to access teaching /
education at a different location from where I
work 34 0 1
Any technical difficulties were acceptable and
did not put me off participating in ECHO 31 0 2
Any technical difficulties did not significantly
reduce my learning 31 0 1
121
9. How do you rate your overall ECHO
experience? (1- poor, 5- excellent)
1 2 3 4 5
0 0 1 9 25
10. Would you participate in ECHO sessions in the
future if the opportunity arose?
Yes 35
NO 0
Unsure 0
The pilot ECHO program for carers found it hard to recruit and retain sufficient numbers of
carers, with only four participating in the network. The survey results highlighted that the
majority of carers viewed participating in the ECHO carers network as a positive experience,
although while many felt it was convenient to use the ECHO model, some still would have
preferred a face-to-face meeting. Carers gave excellent feedback regarding the sessions
delivered and felt very much supported by all professionals involved. Overall, it was
perceived that the technology worked well and all participating carers would recommend the
ECHO programme and participate again. The focus group findings with hub members
highlighted the need for face-to-face support alongside the ECHO sessions. However it was
perceived that the ECHO platform enabled carers to gain knowledge and support whilst also
fostering relationships between the carers and professionals.
Hub members felt that the issues with recruitment and retention were due in part to the
difficulties in explaining the new concept to carers, that the technology may have been off-
putting to some, and also due to the prognosis of the people being cared for. It was
highlighted that for future recruitment it may be of benefit to utilise primary care providers to
identify potential carers. The feedback from participants and hub members has helped
identify some issues that could potentially be addressed in future programs e.g. using a
mixed or blended approach with some ECHO sessions for information sharing and some
face to face sessions for more supportive care.
Weaknesses in this evaluation of the four ECHO networks in HCPs include the low
evaluation response rate. Despite efforts to improve the response rates, with administration
122
staff contacting participants to encourage on several different occasions, the rates remained
low. Future ECHOs evaluations could improve this response rate by considering several
options, for example recognition (certificate) for participation in ECHO being dependent on
completing the evaluation, funding being dependant on completing the evaluation, or
additional funding being made available to allow protected time to participate in the
evaluation.
Another weakness is that some participants post ECHO were able to complete their
knowledge evaluations unsupervised and return them by email or post. This opens up the
opportunity for this not to be a true reflection of knowledge as they could have referred to
notes or other sources of information. As there was no opportunity to get everyone together
at the end of ECHO, it was unlikely that many would have travelled to come to an evaluation
session without one of the incentives described above. In terms of knowledge evaluations it
may the case that less is more and having a shorter Survey Monkey assessment which was
not too long might ensure higher rates of completion.
For the Nursing Home focus group, the moderator had sat in on some hub sessions,
although had not provided the teaching. It is recognised that this may cause bias for the
nursing home focus group results, but due to the tight timeframe for the report it was not
possible to get another moderator. The moderator did not participate in any other ECHO
network.
Due to the short timeframe between the confirmation of funding and setting up of the
networks and needing to get a pre-ECHO evaluation performed on the training days, some
of the curricula had not been fully set before the pre ECHO evaluation was carried out This
led to difficulties in writing the knowledge papers, which may have led to questions being
asked that were not then covered during the sessions. This could be better address in future
studies with a longer timeframe to set curricula and then write focussed knowledge
assessments.
The low numbers of participants in the carers ECHO makes it impossible to draw firm
conclusions on its future potential and more research in this area is needed.
Due to the short timeframe for this evaluation and the level of funding it was not possible to
consider the direct impact on patient care and service delivery, and future studies should
consider these areas to determine the cost effectiveness of ECHO.
123
Conclusions
ECHO is being adopted across the United States and the world: by academic medical
centres, large clinical systems, national medical care providers, and even federal
government agencies such as the US Department of Defence. As a low-cost high-impact
model, ECHO has potential to meet the needs and resources of different communities and
populations where there is good access to broadband, and a suitable trained facilitator and
a group of experts willing to share knowledge and support from a central hub. At a time
when health care providers are under mounting pressure to do more and spend less, this
model provides an affordable solution to addressing growing need.
Through the four ECHO knowledge networks that have been evaluated in this report, and
building on the evaluation in community hospice nurses,(1) the results support the on-going
use of ECHO for HCPs education and support locally in Northern Ireland. There was a
statistically significant improvement in knowledge and self-efficacy in two of the networks,
with positive feedback about learning and impact on patient care across all HCP networks.
Due to the very low numbers of participants it is difficult to draw firm conclusions on the
usefulness or not of the ECHO model in providing support to carers. While there were
positive feedback from both carers and hub members, there were also significant issues with
recruitment and retention, and likely cost effectiveness due to the large numbers of HCPs at
the hub and few carers at the spokes. An economic analysis was not possible within the
scope of this study. The lack of face to face contact is also a recurring theme for both carers
and hub members. A larger study with sufficient time and funding to get ethical approval to
explore the issues raised here more in depth is necessary to determine the future use of
ECHO in this setting.
In addition, by demonstrating that the previously documented successes of ECHO from the
United States and in Northern Ireland have been replicated in this prospective study we are
encouraged to believe that the UK could benefit from widespread use of the ECHO model to
help support the delivery of high quality care irrespective of geographical location as in the
United States.
Research is now necessary to determine if the positive outcomes from this study lead to an
impact on service delivery and ultimately improved patient care.
124
Chapter 10 – Recommendation Summary
• ECHO should continue to be developed, implemented and evaluated to help educate
healthcare professionals across Northern Ireland in a cost effective manner.
• Adequate funding is necessary to allow protected time for all to participate, both in
the hub and at the spokes.
• The importance of the facilitator was a recurring theme, and so this high standard
needs maintained through training and supervision to ensure the networks remain
useful.
• Future ECHO networks should continue to be evaluated to ensure they meet the
needs of the population they are trying to educate and support.
• To improve the response rates in future evaluations, funding for individual practices
or participants should be dependent on participating in the evaluation process, or
funded time should be given. This should therefore reduce the bias of partial
response, and improve the generalisability and hence usefulness of the results.
• Funded studies should be performed looking at the direct impact on patient care to
determine if the ECHO model improves patient care and reduces costs e.g. through
reduced referral rates, improved quality of life, better prescribing etc.
• Minor technological issues should be addressed to make connectivity easy and not
detract from the learning environment. Issues of not being able to connect from
some sites due to HSC security policies also need addressed.
• Further addressing other minor issues such as having preparatory work available in
good time for the spoke members and easily accessible online resources will only
improve on the experience of ECHO
• More research is needed to determine the usefulness of a carers ECHO network.
The suggestions made in this report should be considered if designing a future
Carers ECHO and adequate time and funding is needed to evaluate if the ECHO
format is useful in providing information and support to carers in a cost effective
manner.
125
References
1. White C, McIlfatrick S, Dunwoody L, Watson M. Supporting and improving community health services—a prospective evaluation of ECHO technology in community palliative care nursing teams. BMJ Supportive & Palliative Care. 2015. 2. Arora S, Thornton K, Murata G, Deming P, Kalishman S, Dion D, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. New England Journal of Medicine.364(23):2199-‐207. 3. Colleran K, Harding E, Kipp BJ, Zurawski A, MacMillan B, Jelinkova L, et al. Building capacity to reduce disparities in diabetes: training community health workers using an integrated distance learning model. Diabetes Educator.38(3):386-‐96. 4. Harkins MS, Raissy H, Moseley K, Luttecke K, S. A. Extension for Community Healthcare Outcomes Project -‐ Asthma Specialty Consultation Via Telehealth to Improve Asthma Care in rural New Mexico. US Respiratory Diseases. 2011;7(1):7-‐9. 5. http://echo.unm.edu/about-‐echo/ Accessed 26.02.15. 6. Ammentrop J, Sabroe S, Kofeod PE, J M. The effects of training in communication skills on medical doctors' and nurses' self-‐efficacy: A randomised controlled trial. Patient Education and Counseling. 2007;66:270-‐7. 7. Nimon K, Zigarmi D, Allen J. Measures of Program Effectiveness Based on Retrospective Pretest Data: Are All Created Equal? American Journal of Evaluation 2011;32(1):8-‐28. 8. https://www.surveymonkey.com/ (accessed 25 Mar 2016). 9. Kerr OC, Benton EC, Walker JJ et al Dermatological workload: primary versus secondary care. British Journal of Dermatology 2007: 157 (suppl. 1). . 10. RCGP Birmingham Research Unit. Weekly Returns Service Annual Report 2006. 11. APPGS reports (1998, 2004, 2013). 12. Davies E, Burg,S. Audit of dermatological content of U.K. undergraduate curricula British Journal of Dermatology 2009: 160, 999-‐1005. 13. Chiange Y et al (including C Griffiths & S Burge) Undergraduate dermatology education: a survey of UK medical students British Journal of Dermatology 2008: 159 (Suppl.1). 14. Skin Conditions in the UK: A Health Care Needs Assessment: Schofield, Grindlay, Williams 2009. 15. Personal communication from Northern Health and Social Care Trust. 16. Transforming Your Care: A Review of Health and Social Care in Northern Ireland 2011 (http://www.dhsspsni.gov.uk/transforming-‐your-‐care-‐review-‐of-‐hsc-‐ni-‐final-‐report.pdf). 17. The Kings Fund. Approaches to demand management: Commissioning in a cold climate 2010. 18. http://www.rcgp.org.uk/training-‐exams/gp-‐curriculum-‐overview/online-‐curriculum.aspx. 19. Miles, A. Asbridge, J. E. (2014) ‘Carers–vital partners in increasing the person-‐centeredness of health and social care systems.’ European Journal for Person Centered Healthcare, 2 (4), p.p. 421-‐424. 20. Matzo, M; Witt Sherman, D. (2010) Palliative Care Nursing Quality of Care to the End of Life. New York: Springer Publishing Company. . 21. Hynes, G; Stokes, A; McCarron, M. (2012) ‘Informal care‐giving in advanced chronic obstructive pulmonary disease: lay knowledge and experience.’ Journal of clinical nursing. 21 (7‐8), p.p. 1068-‐1077. 22. Mc Veigh, C. (2015) Palliative Care for patients with non-‐malignant respiratory disease. Nursing Standard. 29 (36), p.p. 44-‐51. 23. Addington-‐ Hall, J. and Mc Pherson, C. (2001) ‘After-‐ Death Interviews with Surrogates/ Bereaved Family Members: Some Issues of Validity.’ Journal of Pain and Symptom Management. 22 (3), p.p. 784-‐ 790. . 24. King, N. and Horrocks, C. (2010) Interviews in Qualitative Research. London: Sage Publications Ltd. .