evaluation of the knowledge mobilisation team …...e-mail [email protected] michelle farr...
TRANSCRIPT
i
Evaluation of the Knowledge Mobilisation Team
From September 2013 – April 2016
An independent formative evaluation of the Avon Primary Care Research Collaborative (APCRC), associated commissioning
organisations and the University of Bristol School of Social and Community Medicine Knowledge Mobilisation team
Authors Kate Beckett Michelle Farr
Andrée le May
April 2016
ii
Executive Summary (including section and page number references to main report)
This report details the formative evaluation of a small, innovative Knowledge Mobilisation Team
(KMT) that spans academic and commissioning contexts. The KMT was established in September
2013, by Avon Primary Care Research Collaborative (APCRC) in partnership with local Clinical
Commissioning Groups (CCG) and the School of Social and Community Medicine (SSCM) at the
University of Bristol (UoB). Currently the team includes seconded healthcare managers (known as
‘Management Fellows’ [MFs]) embedded within academia and seconded health researchers (known
as ‘Researchers-in-Residence’[RRs]) embedded within commissioning organisations under the
leadership of a National Institute for Health Research KM fellow; they are supported by a SSCM
Communications Officer. Since its inception eight people have worked within the team (Section 1:
p3). Our overall findings suggest that this team is outperforming itself whilst being stretched close
to its limit.
Data for this formative evaluation were gathered through interviews with the team, its key contacts
and stakeholders, documentary analysis and observations (Methods: p80). Our synthesis of these
data summarises how the KMT impacts on those it sets out to influence and the types of knowledge
and skills it uses to achieve change (Section 4: p15). We use illustrative case-studies in the report as
examples of KMT activities, their impact and factors which sustain or hinder them (Section 5: p29).
The KMT spans two very different worlds - academia and commissioning (Section 2: p6). It
undertakes this bridging role by; 1) cultivating relationships, 2) targeting activities to suit its various
audiences and tasks, and 3) using different types of knowledge to enable these organisations to
work together in the co-production of varied health-related projects and outcomes. KMT activities
can be broadly divided into two main spheres of work - specific commissioner-researcher
collaborative projects and raising wider KM awareness and capacity. The KMT is enabled, and
challenged, in these endeavours by the context(s) within which it works, the specifics of its projects,
the skills it has (collectively and as individuals) and can use, and the resources available to it (Section
3: p10). The data demonstrates the broad scope and range of KMT activities and impact, however
there is also considerable evidence of the potential for overload.
Whilst impact is hard to measure the following summarises the team’s impact:
Impacts on individuals who work with the KM team (Table 4.2: p19)
Academics have learnt how to enable non-academic people to input into research and to
communicate research results in a way that reaches broader audiences.
Some researchers are more aware of mobilising knowledge through the lifecycle of research
projects rather than just at the end.
Researchers and commissioners are both considering how to put research into practice.
Commissioners are thinking in the longer term and have wider perspectives.
Commissioners understand the value of systematic reviews within decisions.
Greater understanding of what research funders are looking for around knowledge
mobilisation has been achieved.
Greater commissioner appreciation and understanding of evaluating service changes has
been achieved.
Greater researcher knowledge and understanding about commissioning (including budget
and Government pressures) has been achieved.
iii
There is greater commissioner knowledge and understanding about university worlds (how
research projects are put together, evaluation, and systematic reviews).
Commissioners are better supported to reflect on own practice.
Researchers are more confident and enthusiastic to engage with commissioners.
Understanding how to give other people what they need in a timely and appropriate way.
Impacts on relational or network development (Table 4.3: p21)
New personal links developed between commissioners and researchers.
Connecting researchers with commissioners has facilitated new and ongoing research
projects, co-creating and distributing knowledge.
Greater numbers of bridging relationships between commissioners and researchers.
MFs have a network of research contacts to connect with on their return to work within
commissioning.
RRs have a network of commissioners and practitioners to support their research work.
Smoothing relationships between researchers and commissioners where there is
misunderstanding or lack of clarity has led to the better running of research projects.
Greater knowledge and understanding of the local Health Integration Teams, and increasing
commissioners’ roles within them.
Impacts on organisations (Table 4.4: p22)
A series of co-produced research projects has been developed.
Commissioners are co-applicants on research projects, and part of advisory and steering
groups for a number of local research projects.
Co-produced service evaluations are underway.
Greater service evaluation capacity.
Greater knowledge and understanding and embedding of evaluation within commissioning.
Uptake of research findings within commissioning decisions.
More research informed commissioning and commissioning informed research.
Increased knowledge mobilisation capacity within both commissioning and research worlds.
Greater awareness of knowledge mobilisation within academic senior management.
Greater awareness of the world of research within commissioning senior management.
Putting researchers into commissioning groups makes people think and reflect differently on
what they are doing.
People are making decisions differently.
Cultural change is beginning to be seeded.
More bids to the CLAHRC concerning service evaluations are being made.
People questioned the extent to which awareness and outcomes of KM extended beyond
those people who worked closely with KM team members.
Wider health and social care impacts (Table 4.5: p25)
Research is being communicated to a wider number of stakeholders within the health and
social care environment and beyond.
Events that have attracted a wider range of different practitioners have been designed.
More creative ways of getting research messages across such as animations and blogs are
being used.
Wider promotion of knowledge mobilisation and contributing to academic knowledge in the
area.
iv
Practice is changing as a result of knowledge mobilisation.
Impact on the KM team (Table 4.7: p26)
Development of greater knowledge mobilisation skills and enhanced understanding of
knowledge mobilisation as an academic (and practical) subject.
Development of skills in how to engage different people and bring them together.
Greater confidence from working outside their home organisation.
Better understanding of seconded organisation and its work.
New networks and contacts developed.
New research skills acquired.
New research projects developed.
The report closes by making recommendations related to the various contexts within which the KMT
works, the tasks/projects it is involved with, the team and how it, and the individuals working within
it, might work in the future – both in their day-to-day workings within the KMT and when they
return to their seconding organisations. (Section 8: p77)
Author details:
Kate Beckett Research Fellow The University of the West of England, Bristol. Centre for Child & Adolescent Health Oakfield House, Oakfield Grove Bristol BS8 2BN E-mail [email protected]
Michelle Farr
Senior Research Associate
Collaborations for Leadership in Applied Health Research and Care West (NIHR CLAHRC West), University Hospitals Bristol NHS Foundation Trust, 9th Floor, Whitefriars, Lewins Mead Bristol BS1 2NT Email: [email protected]
Andrée le May
Professor Emerita
The University of Southampton, Highfield, Southampton. SO17 1BJ E-mail [email protected]
v
Contents
Executive Summary ................................................................................ ii
Introduction ........................................................................................... 1
1: The KM team ...................................................................................... 3
Diagram 1.1: The 8 members of the Knowledge Mobilisation team ................................ 4
Diagram 1.2: The KM team from 2013-2016...................................................................... 5
2: Worlds Apart ...................................................................................... 6
2.1 Key differences .................................................................................................................... 6
2.2 Implications for the KM team ........................................................................................... 9
3: Key activities and factors which sustain or impede the KM team ....10
Diagram 3.1: What the KM team does ............................................................................ 11
Diagram 3.2: The 9 types of knowledge mobilised by the KM team .............................. 12
Diagram 3.3: Factors which HELP in achieving these activities ....................................... 13
Diagram 3.4: Factors which HINDER these activities ....................................................... 14
4: Impact ..............................................................................................15
4.1 Difficulties in measurement and the long term nature of the work ...................... 15
4.2 How impact measurement can be developed ...................................................... 16
Table 4.1: How KM team impacts may be measured .......................................................... 17
4.3 What has changed? .............................................................................................. 18
Table 4.2: Impacts on individuals who work with the KM team ......................................... 19
Table 4.3: Impacts on relational or network development ................................................. 21
Table 4.4: Impacts on organisations ..................................................................................... 22
Table 4.5: Wider health and social care impacts ................................................................. 25
Table 4.6: Unintended consequences .................................................................................. 26
Table 4.7: Skills and knowledge acquired by the KM team ................................................. 26
5: KMT involved projects/studies & in depth case studies ...................29
vi
5.1 KM team involved studies .................................................................................... 29
Table 5.1: Showing KM team involved projects/studies ...................................................... 30
5.2 Case Studies ......................................................................................................... 32
Case Study 1 (Project): Red Cross Project………………………………………………………………….34
Case Study 2 (Project): Elizabeth Blackwell project ....................................................... 36
Case Study 3 (Project): 3D Research Project ................................................................... 38
Case Study 4 (Project): PROVIDE ...................................................................................... 41
Case Study 5 (Project): Knowledge exchange in commissioning.................................... 46
Case Study 6 (Project): Diabetes evaluation .................................................................... 49
Case Study 7 (Project): Benchmarking data in commissioning PhD research ............... 51
Case Study 8 (Role): KM Team Leader ............................................................................. 53
Case Study 9 (Activity): KM awareness raising ................................................................ 57
5.3 Case studies and activities summary .................................................................... 60
6: What could be done differently? …………………………………………………59
6.1 Optimising the KM team members’ skills and impact………………………………………59
6.2 Suggestions for taking KM initiatives forward ...................................................... 65
7: Linking with wider KM initiatives ......................................................69
Table 7.1: Other KM initiatives and joint activities .............................................................. 69
Diagram 7.1: Map of wider Bristol area KM initiatives. ................................................... 73
8: Recommendations for taking the KM initiative forward ...................77
Diagram 8.1: Recommendations: 1) setting and context, 2) task or project .................. 78
Diagram 8.2: Recommendations: 3) KM team, 4) individual ........................................... 79
Appendix ..............................................................................................80
Methods ..................................................................................................................... 80
References ................................................................................................................. 85
Acknowledgements .................................................................................................... 85
1
Introduction
The need for health research funders, producers, key NHS intermediaries and end users to
effectively share knowledge and skills has long been recognised but poorly achieved. Indeed despite
over £8.5 billion annual expenditure on health services research (UKCRC, 2015) much of this work
fails to have a significant or lasting impact on practice. Furthermore acknowledgement of contextual
factors inhibiting the uptake of academic research has been slow and contributed to the persistent
‘gulf’ between research and practice (e.g. Eccles et al, 2009). This has led to burgeoning interest and
emphasis on knowledge mobilisation or “making knowledge readily accessible and useful to
individuals and groups by developing ways to work collaboratively” (Health Information Research
Unit). Many strategies to improve researcher-practitioner collaboration have been tried however
systematic evaluation of their impact is acknowledged to be both difficult and generally poor (Davies
et al, 2015). This formative evaluation, jointly commissioned by Lesley Wye as part of her NIHR
Knowledge Mobilisation Research Fellowship and the Avon Primary Care Research Collaborative,
provides details of the workings of a small, innovative knowledge mobilisation team that spans
academic and commissioning contexts.
Since September 2013, Avon Primary Care Research Collaborative (APCRC), in partnership with local
Clinical Commissioning Groups (CCG) and the School of Social and Community Medicine (SSCM) at
the University of Bristol (UoB), has pioneered an innovative Knowledge Mobilisation (KM) approach
by establishing a dedicated KM team (KMT). This team includes seconded healthcare managers
(known as ‘Management Fellows’: MF) embedded within academia and seconded health researchers
(known as ‘Researchers-in-Residence’: RR) embedded within commissioning organisations under the
leadership of Lesley Wye (a National Institute for Health Research KM fellow: KMRF); the team is
supported by a communications officer (CO). Their key focus is on improving knowledge
mobilisation between health service researchers and local healthcare commissioners by:
1. Sharing knowledge and skills;
2. Fostering collaborations between commissioners and researchers;
3. Encouraging commissioning-informed research and research-informed commissioning.
A useful explanatory video can be found at https://www.youtube.com/watch?v=yZnmeVWHWPk.
2
This KM initiative has been in operation since 2013 and current KM team members are well-
embedded having been in post at least 12 months. This provides a propitious time to systematically
investigate the extent to which the key aims above are being met, identify any barriers and
facilitators to achieving them and develop recommendations for what could be done differently or
encouraged to be sustained. The aims of this formative evaluation are therefore to:
Learn about the experiences and activities of current and past KM team members and the
views of their commissioning and academic colleagues.
Ascertain ways to identify and measure the impact on the KM team members themselves
and, in turn, their impact on key stake holding organisations.
Explore how KM team members envisage using and developing knowledge and skills gained
during and post- fellowship, and how well host and original organisations are supporting
this.
Investigate how the KM team fits within other local and wider KM initiatives
Inform decision-making about the future of the KM team and explore their longer term
sustainability.
The evaluation drew on data from interviews with the KM team and its key stakeholders,
documentary analyses and observations of relevant meetings. Details of our methods are provided
in the Appendix (p80). Throughout the text we have selected key quotations to characterise the
themes that emerged from our triangulated analysis and drawn together a number of case-studies
that, when seen together, reflect the breadth of the work undertaken by the KM team. We have
anonymised data by using numbers/letters as source identifiers but recognise that individuals will
know their own words.
Our evaluation is presented in the following eight sections: 1) the KM team, 2) the context(s) of their
work, 3) their key activities and factors which sustain or impede them. This is followed by, 4) a
discussion of the challenges and possible means of measuring their impact followed by actual KMT
impacts and activities that have led to these and 5) a comprehensive list of KMT engaged projects
and 9 in-depth case studies providing specific examples of their contributions. Finally, 6) we explore
the best means to exploit KM team members’ skills and optimise impact before/during/after
secondment, 7) their place and unique contribution within wider KM and 8) make recommendations
for taking the KM initiative forward.
3
1: The KM team
In order to understand the impact of the KM Team it is important to know who this involves and the
terms and duration of their KM role. Diagram 1.1 overleaf details the 8 past and current members of
the KM team, showing their movement from home to host organisation, substantive position, KM
team role and source of funding (right hand column). Diagram 1.2 provides a timeline to illustrate
the growth and development of the KM team. It demonstrates that while KMT numbers have
steadily grown (in terms of FTE allocated to KM activities) the maximum input at any one time is 2.9
FTE. Of this 0.7 FTE is funded by a NIHR KMRF award, 0.1 FTE by the University of Bristol (UoB) and
APCRC for the Communications Officer1, and the remaining elements (MF and RR posts) (2.1 FTE) are
funded by APCRC.
Note: Quotes used within this report are labelled according to whether they were made by members
of the KM team (A) or by KM contacts or senior stakeholders (B). The latter two groups have been
combined for the purpose of anonymity.
However Section 5 includes in-depth case studies of particular projects, to protect participants
identity quotes in this section have been kept to a minimum and those used are not referenced
(except in Case study 9 which relates to general activities rather than a specific project) .
1 Note Nadya Anscombe’s role as Communication Officer extends beyond her work with the KMT. She is jointly funded by UoB and APCRC for a range of KM related activities but estimates that specific KMT activities amount to the equivalent of 0.1 FTE
4
Diagram 1.1: The 8 members of the Knowledge Mobilisation team (past and present) Showing their substantive
employing organisation (University of Bristol - red, NHS - blue) and position, secondment role and funding
source
Team Leader: Lesley Wye NIHR Knowledge Mobilisation Research
Fellow & Senior Research Fellow School of Social & Community medicine (SSCM)
Nadya Anscombe Communications Officer
Centre for Academic Primary Care SSCM
Management Fellow seconded to UoB
Management Fellow seconded to UoB
Co
ho
rt 1
: 20
13
-201
4
Co
ho
rt 2
: 20
14
-20
16
Helen Baxter Senior Research Associate
Centre for Academic Primary Care SSCM
Helen Cramer Research Fellow
Centre for Academic Primary Care SSCM
James Rooney Senior Project Manager NHS South, Central and West Commissioning Support Units
Becca Robinson Service Improvement Lead
Bristol Clinical Commissioning Group
Management Fellow seconded to UoB
Management Fellow seconded to UoB
Researcher in Residence seconded to BCCG
Research
Cap
ability Fu
nd
ing, A
von
Prim
ary Care R
esearch C
ollab
orative
NIH
R K
MR
F A
PC
RC
& U
oB
Rachel Anthwal
Senior Project Manager, Transformation Team, NHS South & West
Commissioning Support Unit
Jude Carey Business and Project Manager Bristol Clinical Commissioning Group
Researcher in Residence seconded to BCCG
5
Diagram 1.2: The KM team from 2013-2016, including pre-secondment role, FTE allocated for KM activities and additional KM related activities
KM team member Pre-secondment role
J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D
Lesley Wye Research Fellow, Centre for Academic
Primary Care, UOB
Nadya Anscombe Communications officer Centre for
Academic Primary Care, UOB
Cohort 1
Jude Carey Business and Project Manager, Bristol
CCG
Rachel Anthwal Senior Project Manager,
Transformation Team, NHS South &
West Commissioning Support Unit
Cohort 2
Helen Baxter Senior Research Associate at UoB
Helen Cramer Post-doctoral research fellow, UoB
Centre for Academic Primary Care
James Rooney Project & Service Redesign Manager,
South West Commissioning Support
Unit
Becca Robinson Service Improvement Lead, Bristol
Clinical Commissioning group
2013 2014 2015 2016
Researcher in res idence seconded to Bris tol CCG
(0.4FTE year 1/0.5 FTE year 2)
Management Fel low seconded to
UOB (0.5 FTE)
Under l ine management of Les ley Wye/KM
team (0.1 FTE)
Co-applicant/KM lead NIHR
funded RE-PROVIDE study 2
hours/week for 6 years
Management Fel low seconded to UOB (0.6
FTE)
Management Fel low seconded to UOB (0.5 FTE)
NIHR Knowledge Mobi l i sation Research Fel lowship 0.7 FTE
(reducing to 0.6/0.5 FTE)
Manager: KM
fel lows
Management Fel low seconded
to UOB (Ful l time)
Researcher in res idence seconded to Bris tol CCG
(0.25 for fi rs t 2 months then 0.5 FTE)
6
2: Worlds Apart In this section we review the contexts in which the KM team are situated namely the worlds
of academia and the NHS (in particular commissioning organisations) exploring key
differences and their implications for the KM team. The data reveal how mutual lack of
understanding or contact can sustain stereotypes, barriers and impede collaboration.
Section 3 shows how the KMT works to overcome some of these challenges.
2.1 Key differences
Organisational culture
Commissioning organisations are characterised as fast-paced, noisy, reactive work
environments in which individuals have limited autonomy or time to develop or reflect.
Objectives are externally imposed, have immediate impact and are subject to constant
change. Pragmatism, opportunism, success and speed are essential.
‘The goalposts change all the time… a culture where they are consistently told what they have
to do all the time and are just utterly reactive … everything is relational and open plan’ (A16)
‘Everything we do is complex and difficult …the system is bigger than any of us... at this
moment and it’s very difficult to be influential’ (B22)
Commissioning organisations also differ from each other:
‘How we function in (x) isn’t how (y) CCG function and how (z) CCG function. And there’s 211
CCGs, or something like that, nationally, and if they’re all working a bit differently …’ (A4)
By comparison the academic environment is perceived of as quiet, open and respectful.
Researchers have autonomy and time to pursue their own interests, set their own
objectives, experiment and develop ideas. But it is also characterised as inward looking and
self-serving with the focus being on personal interests and careers.
‘A bit indulgent, that I had all this time to be reading and thinking...it felt like a real luxury.’
(A14)
7
‘There’s always an element of the tail wagging the dog, in terms of what academics will do.’
(B10)
‘The dark depths of the University, and it’s like these people are sucked into the dark and,
and they’re there and they’re busying away and working away and …… calling up patients
and doing surveys of things’ (B20)
Language
Within these very different cultures language is an important tool to maintain separate
identity and status:
‘I was looking round the room, and I was thinking, God, does anyone understand this,
because it is totally. I don’t even, you know, you’re talking a different language to me’ (A14)
‘All these very clever people talking in a language I don’t understand. But then again, I
imagine if you go and sit in a commissioning meeting, that’s exactly how you feel.’ (A3)
Time frames
The university and CCG operate to completely different time frames:
‘We work at I think a different speed … the commissioning world … the real world, the NHS
world has to go, keep going continuously, commissioning work’s very fast … we’re (university)
not at the same speed or working in the same process’ (B18)
‘The problem is, because it’s an annual cycle, commissioners are only really interested in next
year, not what they might be commissioning in five years’ time.” (A3)
Remembering a collaborative workshop (A17) recalled:
‘the big question, the question that got everybody talking was how long would it take a
commissioner to put together a business case for … an intervention that costs half a million,
and then the same question is how long would it take a researcher to put together a grant
proposal … the commissioners said, oh, it would take me from anything from a few hours to
8
a couple of days … to put together a business case, and the researchers were saying, well,
anything from three months to six months to put together the grant proposal.’
Priorities and purpose
Commissioner priorities are characterised as urgent, immediate, pragmatic and of wide
importance yet frequently externally imposed. They are concerned with day-to-day service
delivery, strategic objectives and costs:
‘Quite often in commissioning … we’re not always talking about a clinical intervention, we
might be talking about aim, model of care, or a way of doing things.’ (B19)
‘An important consideration for us is the cost effectiveness and quite often that’s the element
that’s missing from the research study.’ (B19)
‘If one of (the) targets isn’t being delivered, actually the government is going to jump up and
down on us in three weeks’ time if we haven’t improved something in that time’ (B8)
‘We have to do something today which will make a difference tomorrow and the next day’
(B22)
‘The challenge we’ve obviously got is who is the customer and what are they prepared to pay
for?’ (B22)
While academic priorities are regarded as more individual or directed towards more
esoteric purposes or specialised topics.
‘The impact of research is not currently perceived as linked to career progression for
researchers and therefore may not be viewed as a priority regardless of their interest in
getting their research into practice or policy.’ (Doc 11, Feb 2015)
‘We can see the amount of research that’s going on into specialist, you know, high-end
technical brilliance … we hear about the 12 patients in the UK who will benefit next year as a
result of some ...multi-million pound investment in research. For me, the burning platform is
…the general population… how do we keep people fit, healthy and well’ (B22)
9
‘It’s the age old thing, there is a mismatch between the research that the researchers are
producing and the questions that the commissioners really want asked.’ (A2)
‘I suppose, to get really philosophical, in a free society there must always be the opportunity
to go and ask questions and do such just because you want to, actually if we’re spending vast
sums of public money on research then it should be with a view to it actually being used and
being useful.’ (A3)
Relationship to evidence and research
These cultural and organisational differences impact on these institutions’ relationships to
evidence and research:
‘Commissioners need an evaluation or … a piece of work that’s at a good enough level for
you to make decisions. You don’t want something that takes three, five years…when we’ve
moved on into another… frenzy of stuff’ (B20)
‘What really came out was that in the way that commissioners access and use knowledge…
is through relationships… they call on the people that they know… usually literally within
their immediate vicinity of their open plan office’ (A16)
‘The only way to do an evidence appraisal is to do the so-called gold standard Cochrane
approach…which probably is not appropriate for the needs of the healthcare person because
they can’t wait for a year and a half’ (B11)
‘It’s quite frantic, the timelines are so short. The… corner cutting which is essential and
pragmatic…they cut so much off the corner that in the end, the academic who is placed
there, feels they’re getting quite far from their own integrity’ (B21)
2.2 Implications for the KM team
The aim of the KM team is to bridge these differences but this is not any easy task, it
amounts to:
‘Learning what an Anthropologist would learn when they go into another culture’. (B21)
10
‘Being thrown in to a totally different environment where everything seemed quite foreign -
the culture, the language, the style of working' (Doc 11 Feb 2015)
3: Key activities and factors which sustain or impede the KM team
In this section we explore the types of activities engaged in by the KM team, factors which
help or hinder them and types of knowledge shared. The data suggests that KM team
activities can be broadly divided into two main spheres of work - specific commissioner-
researcher collaborative projects and raising wider KM awareness and capacity.
Management Fellows and Researchers-in-Residence are instrumental in furthering these
activities within their host organisations however the leadership role of the KMR Fellow and
the support of the Communications Officer are essential in facilitating and extending this
work. Diagrams 3.1, 3.3 and 3.4 illustrate what types of activity the KM team engage in and
what helps and hinders them.
The KMT uses a number of forms of knowledge in their work (Diagram 3.2). These forms of
knowledge are shared with host organisations and within the KM team, however relational
and organisational knowledge appear to have the greatest currency. Furthermore, some
activities achieve or are directed towards different types of knowledge sharing (see Case
Studies Section 5: p29).
11
1)
REL
ATI
ON
AL
AC
TIV
ITIE
S
Create bridges, cross boundaries and hierarchies
Increase mutual understanding and awareness of two very different worlds
Enhance reciprocity and reveal common ground
Facilitate networks, links and collaborations
Act as a sounding board
Map key stake holders and list key personnel
Provide an important independent perspective outside the host organisations usual norms
Participate in and introduce new members to steering groups, HITS and other stakeholder meetings
Engage with wider health, social care and voluntary sector organisations
2)
RES
EAR
CH
& E
VA
LUA
TIO
N
Develop, contribute to and initiate collaborative projects
Add capacity and/or new skills to exisiting projects
Assist in securing additional funding and resource
Enhance the application, utility and accessibility of academic research
Emphasise and develop NHS service evaluation capacity and skills
Develop research capacity and skills
Optimise impact eg researchers taught how to ‘hook’ commissioners interest in their research, commissioners how to evaluate services and attract funding for research
Collaborate on development and delivery of short courses for researchers and NHS staff
3)
DIS
SEM
INA
TIO
N A
CTI
VIT
IES
Target and engage with wider audiences
Tailor messages to varied audiences & translate research into commissioner language
Develop evidence summaries and key message briefs
Embrace social media and alternative techniques
Organise conferences and presentations
Contribute to scientific papers
Review research to feed into commissioning decisions
Present regularly at informal & formal local & national symposiums and conferences
4)
TRA
NSF
ERR
AB
LE S
KIL
LS D
EVEL
OP
MEN
T
Work in a different orgnisational culture
Consolidate or learn new research, evaluation or relational skills
Manage complex organisations, change and uncertainty
Comminicate with different audiences
Organise conferences and presentations
Meet and overcome obstacles and barriers
Lead and initiate change
Manage conflicting and mutiple demands
Lead on self-initiated research or evalutaion
Develop personal links and networks
5)
INN
OV
ATI
ON
AN
D K
M A
WA
REN
ESS
RA
ISIN
G
Raise the profile of KM as an essential component of research/funding bids
Advise & lead on KM elements within existing/new research
Introduce creativity and innovation, experiment at the interface, seed new ideas
Add value, inspire, act as a catalyst
Question and/or change the way research is initiated, designed, done and disseminated
Initiate debate about the knowledge required for effective healthcare
Promote the KM team to wide audiences
Exemplify a new KM model and make Bristol visible as a KM HUB
Increase KM capacity and knowledge
Diagram 3.1: What the KM team does: showing 5 key areas of activity namely 1) relational activities, 2) research and evaluation, 3) dissemination activities,
4) transferrable skills development, 5) innovation and KM awareness raising
12
KM team
Relational knowledge
Organisatio-nal Knowledge
Research design & methods
Evaluationdesign & methods
Diversedisseminati-on
methods
Experiential or clinical
knowledge
KM theory and practice
Research evidence
Funding sources & opportuniti-es
Diagram 3.2: The 9 types of knowledge mobilised by the KM team
13
CTIVITIES
1)
SETT
ING
OR
CO
NTE
XT
Perceived reciprocity, clear mutual benefit and balance
Buy-in, belief understanding
Ongoing support and clear objective setting
Readiness & stability
Continuity & long term commitment
Research friendly environment
Organisational culture, structures and senior managers which support and promote KM activity
Linkage to other KM initiatives eg BHP HITS
Security of funding and resource
Evidence of impact
2)
TASK
OR
PR
OJE
CT
Time, effort, flexibility
Co-location & embededness
Opportunism and chance
Willingness to seize an opportunity and try new ideas
Developing personal and organisational links
Small steps, incremental benefits and small pilot feasibility projects
Large scale research projects with established teams and pre-existing emphasis on applied research
Building on quick wins to develop reputation
Appropriate project scale and stage for setting
Delivery of key actionable messages
Being clear on what the KM individual can offer
3)
KM
TEA
M Visbility and opportunities to sell the idea
Mutual support and team membership
Leadership (KMRF)
Freedom to experiment
Shared values & ideals
Learning from each other, having a counterpart in the other 'camp'
Strong relational contacts within host organisation
Informality and friendship between members
A safe area where people can sound out ideas and share problems
The KM reading group (for learning & literature review)
Maintaining some independence from stakeholder organisations
4)
IND
IVID
UA
L
Communication & networking skills
Topic interest and/or knowledge
Trust, openness, adaptability, energy, enthusiasm & persistence
Organisational knowledge
Willingness to learn
Creativity
Foresight to identify common concerns
Someone who 'dances in-between'
Task and person fit
Diagram 3.3: Factors which HELP in achieving these activities in terms of 1) the context or setting, 2) the task or project, 3) the KM team and 4)
the individual
14
1)
CO
NTE
XT
OR
SET
TIN
G Different timescales & demands
Organisational demands, priorities, flux and change
Differing political & organisational agendas and cultures
Lack of strategic buy-in, understanding or senior endorsement (NHS/HEI)
Tensions between ‘micro’ and ‘macro’ level expectations, measurable or relational impacts
Not seeing the value of engagement with another world
Lack of planning for sustainability, future resource and post secondment roles
Perceived inequality in impact and costs
Political and organisational pressures on the commissioing world
Rapid expansion of KM initiatives, risk of overload
Unrealistic expectations of research
2)
TASK
OR
PR
OJE
CT Lack of time or energy invested
to optimise KMT assimilation and impact
Brevity of placement it takes time to assimilate
Unrealistic or differing expectations
Lack of clarity about secondee's role, scope and availability
Suitability of scale, scope and focus of task for setting or individual
Expectation that research can provide straightforward solutions
Time consuming nature of collaboration
Mismatch between researcher and commissioners requests
Misunderstandings and different expectations
Failed bids and time wasted3
) K
M T
EAM
Dependence on individual charisma and personality traits
Demands, limits to what 8 people can do
Lack of funded resource to co-ordinate and support the team
Having too many taskmasters
Exclusivity of the KM brand and poor linkage with other strands
Being thinly stretched over different agendas and changing priorities
4)
IND
IVID
UA
L Brevity of placement (it takes time to adapt)
Retreat to comfort zones
Different individuals – different skills
Placement not matched to skills and knowledge
Excessive and conflicting demand
Post secondment underutilisation of skills learned
Diagram 3.4: Factors which HINDER these activities in terms of 1) the context or setting, 2) the task or project, 3) the KM team and 4) the individual
15
4: Impact This section explores the difficulties of measuring the impact of the KM team work (4.1) and
offers some suggestions made by interviewees about a range of potential ways to evidence
impact (4.2), some of which are already included within this report2. Section 4.3 covers the
wide range of impacts that the KM team’s work has contributed to, providing evidence to
support these and details of KM team’s activities that led to these impacts. This section also
covers the skills that KM team members have developed themselves, alongside impacts on
KM team members’ own roles and careers.
4.1 Difficulties in measurement and the long term nature of the work
Evidencing clear impact of KM team work was seen as important, yet a challenge. Some
interviewees saw that there was a clear need to be able to ‘justify’ (B11) investments in the
KM team, and engage others who may be ‘on the fence’ (B20).
Activities
The activities that the KM team are involved in often do not have easily measurable results.
‘Separating out what made the difference is hard to tell… it’s only one ingredient in
lots of things that are going on’ (B7)
‘I think we’re supporting a wider definition of what impact can be, and it might not
always be beautifully measurable but it’s important work to be trying to engage in’
(A1)
‘Impact shouldn’t be in a box on the side; it’s about things gelling into the work that
we do’ (B13)
2 For example, case studies (Section 5), the projects and co-produced research that KM team members are involved with (Section 5) and the skills people have developed as a result of working with the KM team (Section 4.3).
16
Influencing decision making: One of the KM team’s roles is to influence decision-making so that
it is more informed by research/ commissioning worlds. However there are difficulties in
evidencing how people make decisions and act upon information that is communicated to
them. It was also considered as less easy to measure direct impact on policy and practice,
when feeding research into practice.
Culture change: Culture change was seen as an important element in KM work, however it
was considered that this was both very difficult to measure and achieve.
Impacts of relationship building and networking: Whilst KM team members may (or may not) see
that conversations that they have initiated have gone onto further collaborative work,
evidencing this may be difficult. There are different subjective viewpoints and others might
have different perspectives on the same conversation. This was indeed the case in two
interviews. There are difficulties in tracing causal connections in how relationships and
networks developed, may then lead onto more substantive influence.
Attribution to KM activities: Identifying what changes can be pinpointed directly to KM team
activity and what is a result of other wider KM drives and initiatives is not easy to evidence.
‘It’s one brick in a series of bricks that build up a picture’ (B7)
Long term nature of the work: Some interviewees saw that impact was a long term issue that
had parallels with other healthcare movements such as quality improvement and patient
and public involvement:
‘By the nature of the work, it’s going to be a slower track…it probably needs to be a
five year project’ (B8)
4.2 How impact measurement can be developed
Whilst acknowledging the difficulties of impact measurement, there were various
suggestions about how impact could be evidenced (Table 4.1).
17
Table 4.1: How KM team impacts may be measured
ACTIVITY MEASURES
Assessing research awareness and use
within commissioning
Having benchmark and baseline data would be
helpful here. One early MF carried out a research
aware questionnaire with commissioners that may
be able to act as a baseline measure.
Another way to measure this is to count how many
commissioning business cases have evidence to
support them.
Research dissemination Measures such as website access statistics; total
number and composition of mailing lists; Twitter
followers and retweets; and media interest are
already being used.
The number and composition of audiences at
external conferences and formal feedback from
conference delegates has been used.
Knowledge mobilisation within research Counting how many research projects have
delivered knowledge mobilisation within their
work. How many research bids have KM phases
within them?
Embedding evaluation How many commissioning business cases have
evaluation, assessment or outcome measurement
within them?
Co-produced research Number of co-produced bids that are developed.
Number of research grants with KM team members
named on the grant as co-applicants.
Other suggestions covered later include:
The skills people have developed (see 4.3).
Documented case studies from the CCG and the university about what has happened
differently as a result of the KM team activities (see section 5 Case Studies).
Identifying the projects and co-produced research that KM team members have
been involved with (see section 5 Table of Projects).
18
4.3 What has changed? Impacts at an individual, relational and organisational level and
beyond
This section covers the wide range of impacts of the KM team’s work, and the activities that
led to these impacts. Impacts covered include those on:
other people, both commissioners and researchers who work with the KM team
(Table 4.2)
relationships, networks and understanding between commissioners and researchers
(Table 4.3)
organisational impacts (Table 4.4)
wider health and social care outcomes (Table 4.5)
and unintended consequences (Table 4.6).
Impacts of the work on KM team members, in terms of skills and knowledge they have
developed is presented in Table 4.7, and a summary of how these have affected their
careers and roles ends this section.
19
Table 4.2: Impacts on individuals who work with the KM team
OUTCOMES EVIDENCE AND EXAMPLES OF ACTIONS THAT LEAD TO THIS
Academics have learnt how to engage non-academic people to input
into research and to communicate research results in a way that
reaches broader audiences
Some researchers are aware of mobilising knowledge through the
lifecycle of research projects rather than just at the end.
Meetings with KM team members to discuss how this may be done with examples
Engaging in translating research to the ‘real world’ (B18) in collaboration with MFs
Commissioners thinking more long term and having a wider
perspective on decisions
Researchers and commissioners both considering how to put
research into practice
From a commissioning world: ‘I think the challenge of having [KMT member] and
other members from her team about has been powerful in terms of giving a trust to
the research world and an input potentially, which allows us to think about doing
things in quite a radically different way’ (B8).
From the research world: ‘focusing on what it really meant for the people who
want to know the results locally was very useful, and ironically we’ve had some
good academic outputs from it, but it sort of flipped the whole thing on its head’
(B25)
Commissioners understanding the value of systematic reviews
within decisions
Working alongside KM team members to specifically design systematic reviews
that fit with commissioners needs as opposed to accessing generic Cochrane ones.
‘Although I’ve known they happen, you look at Cochrane’s …when it’s actually
something which is more around the work that you’re doing it’s very helpful’ (B8).
Understanding what funders are looking for around knowledge
mobilisation
KM team giving support on KM and being seen as someone to go to for KM advice
Greater commissioner appreciation and understanding of evaluating
service changes
‘I have to say my tolerance of and respect for evaluation has increased a great
deal’ (B6).
KM team working with commissioners to support evaluations
20
Greater knowledge and understanding about commissioning
(including budget and Government pressures)
Greater knowledge and understanding about university worlds (how
research projects are put together, evaluation, systematic reviews)
Understanding how to give other people what they need in a timely
and appropriate way.
Awareness raising, facilitating relational connections, facilitating organisational
understanding
Researchers and commissioners talking with KM team about the different worlds,
working alongside them, and developing collaborative relationships
Learning different languages and drivers from KM team to advance exchange
between commissioners and researchers
Providing insight into how ‘other’ worlds work, and how to engage with them
Supporting commissioners to reflect on own practice ‘She became a very trusted reference point for me. She still is.’ (B6)
‘I think it’s been very powerful for me in terms of making me think in a different
way’ (B8)
Researchers more confident and enthusiastic to engage with
commissioners
‘I’ve definitely got more and more enthusiastic about this need to build
relationships and connections across the academic service divide because of
working with [KMT member]’ B7)
21
Table 4.3: Impacts on relational or network development
OUTCOMES EVIDENCE AND EXAMPLES OF ACTIONS THAT LEAD TO THIS
New personal links developed between commissioners and
researchers
Foresight and knowledge of different people’s interests and how they may link
together.
Facilitation of initial meetings between commissioners and researchers to highlight
how and where collaborations could be developed.
Connecting researchers with commissioners to facilitate new and
ongoing research projects, co-create and distribute knowledge
‘I’m sure without a doubt having commissioners on board, all that contact with the
CCG, it makes you write better, more applied research projects’ (B18)
Giving commissioners insight into ‘the complicated picture of all these different
funders’ (B19).
Greater numbers of bridging relationships between commissioners
and researchers
Drawing on own contacts and relationships to bridge divides between the two
worlds:
‘It helps knowing these people’s names when I’ve seen them in other meetings
since’ (B5)
MFs have a network of research contacts to connect with on their
return to work within commissioning
Being seconded into the University. Since MFs have returned to commissioning
they still get contacted by researchers to help with making connections.
RRs have a network of commissioners and practitioners to support
their research work
Being seconded into the CCG. It was noted that maintaining a physical presence at
the CCG beyond a secondment period is important in maintaining those links. ‘The
relationships are there so I know I can access them’
Smoothing relationships between researchers and commissioners
where there is misunderstanding or lack of clarity. This has led to the
better running of research projects
Previous trusting, relational connections. Communications between the two
groups, facilitating organisational understanding
Greater knowledge and understanding of the Health Integration
Teams, and increasing commissioners roles within them
Linking researchers and commissioner with these and updating them on
developments
22
Table 4.4: Impacts on organisations
OUTCOMES EVIDENCE AND EXAMPLES OF ACTIONS THAT LEAD TO THIS
A series of co-produced research projects
Commissioners being co-applicants on research projects,
and being part of advisory and steering groups for a
number of local research projects
Identifying problems, potential solutions and bringing together a range of stakeholders
interested in the problem. E.g.: Elizabeth Blackwell project – ‘A frailty toolkit for Primary care’
(see case studies); abnormal test results for out-of-hours service.
Researchers building stronger collaborative relationships with commissioners leading onto the
development of co-produced research bids rather than a tick box exercise to put a name on a
grant.
MFs working within University research projects and then being named as a co-applicant on
new and successful research grants, alongside being part of advisory and steering groups for a
number of research projects (e.g. RE-PROVIDE, ALT-CON, 3D).
KM team members being involved in drafting KM sections of research bids.
Co-produced evaluations
Greater service evaluation capacity
Greater knowledge and understanding and embedding
of evaluation within commissioning
Negotiation around quantity, quality and speed of evaluation is needed to meet both
researchers and commissioners need.
Researchers exploring different ways of doing evaluation to speed up process.
MFs having considerable experience of these co-produced evaluations gives them the skills to
continue this work on their return to home organisations.
Being able to discuss evaluations informally with previous MFs before talking to ‘experts’ about
them.
‘The evaluation stuff alone, is worth its weight in gold, because that’s where we’ve been quite
weak in the past, properly having that expertise to help us think through that, but also helping
them understand what’s good enough… for us to make a decision’ (B19).
23
Uptake of research findings within commissioning
decisions
More research informed commissioning
Systematic reviews based upon commissioners’ concerns: ‘The cardiovascular review has
definitely informed our commissioning plan around what’s the best thing we should spend our
money on’.
‘It’s informed some of our commissioning questions and decisions or given us the opportunity
to get someone to go and research it’ (B9).
PROVIDE team commented on draft commissioning proposals for the proposed re-
commissioning of a range of domestic violence services.
‘Within the commissioning brief, people now have to come up with examples of evidence that
supports their business cases which wasn't there before. So actually got that as part of the
template’ (A16). With this commissioning brief there were differences in perceptions as to the
extent to which this was attributable to the KM team.
Increased knowledge mobilisation capacity within both
commissioning and research worlds
Greater awareness of knowledge mobilisation within
academic senior management
Greater awareness of the world of research within
commissioning senior management
KM team leader now a member of CAPC Senior Management Team and has role of Deputy
Director of Impact UoB School of Social and Community Medicine, contributing to cross-
Faculty research meetings.
The communications officer post has been extended within CAPC.
The presence and work of the KM team has contributed to a greater commitment to using
evidence and research within commissioning teams that are aware of its work.
Putting researchers into commissioning groups makes
people think and reflect differently on what they are
doing
‘[It’s] really important to start changing a very entrenched language in the NHS. We need to
start introducing new concepts, new words, new phrase, new ways of talking, new ways of
understanding. That is the importance of this project.’ (B6).
Making decisions differently People spoke of commissioners being more aware of the need for evaluation and evidence in
decision making: ‘It’s about considering all the different pieces of the jigsaw before you get to
make a decision as a commissioner, that’s the bit that’s quite important, and using all the
24
evidence, knowledge and expertise you’ve got in order to make the best decision you possibly
can’ (B13).
Beginning to seed cultural change ‘I think [the KM Team] have helped bring about or catalyse a cultural shift, which is still early’
(B21).
Greater commissioning informed research Developing and editing research tools, to make them more commissioner friendly.
Better tailored research messages going to wider audiences:
‘We’ve tried to take on board what commissioners are saying… practical things … their
priorities …. So I would say our research has been informed by commissioning (B21).
More bids to the CLAHRC concerning service evaluations ‘I think the Knowledge Mobilisation Team, have been key perhaps to catalysing that sea
change of understanding’ (B25).
People questioned the extent to which awareness and
outcomes of KM extended beyond those people who
worked closely with KM team members
KM team activities seem to impact most on those they are collaborating with, and less so at
wider organisational levels.
25
Table 4.5: Wider health and social care impacts
OUTCOMES EVIDENCE AND EXAMPLES OF ACTIONS THAT LEAD TO THIS
Communicating research to a wider number of
stakeholders within the health and social care
environment and beyond.
Designing events that have attracted a wider range
of different practitioners
Interactive workshops
More creative ways of getting research messages
across such as animations and blogs
Databases and mailing lists created and developed incorporating contacts from a range of health
and social care organisations for dissemination of key research messages for different research
projects and conferences.
Asking conference attendees to pledge that they will act differently in some way as a result of
knowledge learnt
Publications developed for practitioners journals e.g. HSJ
Twitter account developed with growing numbers of followers and usage.
Conferences being more thoughtfully promoted to achieve good representation from a broad
range of sectors
Training researchers to work with journalists
MFs developing dissemination plans for the research projects they are associated with
Providing dissemination advice and tools
Changed forms of practice as a result of knowledge
mobilisation
‘[The] domestic violence work that [MF] was involved with, I am sure the impact of that on the
service has been so much greater, because of [MF]’s involvement in that’ B25).
Wider promotion of knowledge mobilisation and
contributing to academic knowledge in the area
The KM team have met and presented to a range of different audiences regarding the
management fellowships and researchers in residence to promote knowledge mobilisation
techniques.
A range of conference presentations have promoted the work of the KM team nationally e.g. Royal
College of GPs conference, Glasgow, October 2015. Workshop entitled ‘Researchers in residence -
a new model to improve patient care’.
26
Table 4.6: Unintended consequences
OUTCOMES EVIDENCE AND EXAMPLES OF ACTIONS THAT LEAD TO THIS
Making choices about how to fund different activities ‘I would say that it is quite expensive, and that for every time you’re investing in
this, it’s a choice, and you’re investing in this and not in something else’ (B7)
‘Resentment’ where commissioners have given time with no visible
results (Interviewee 3).
Commissioners have worked with researchers to develop co-produced joint bids
but some of these have not been successful
Commissioners being without key staff for a period of time Consequences of seconding process
Table 4.7: Skills and knowledge acquired by the KM team
ALL KM TEAM NHS MANAGEMENT FELLOWS RESEARCHERS-IN-RESIDENCE
Greater knowledge mobilisation
skills and understanding
knowledge mobilisation as an
academic subject
Understanding and practising
different ways of sharing
knowledge
Intention to base commissioning work on research and evidence,
alongside measuring effectiveness of interventions
More critical, analytic and questioning approach
Developing knowledge mobilisation and dissemination plans for
research, use of social media in disseminating research, developing
commissioner briefings on research reports, workshop planning and
facilitation, conference organisation
Making research projects relevant to
commissioners, how to facilitate genuine
interaction between researchers and
commissioners
Knowing how to present things to
commissioners, skills in speaking to different
audiences
How to engage different people
and bring them together
Linking researchers with commissioners where there are shared
interests and concerns
Facilitation, running meetings, diplomatic skills
Greater confidence having worked
outside of own home organisation
Confidence in own skills and abilities – this was noted by both MFs
themselves and the commissioners whom they work with
More competent and confident in presentations
Skills and confidence in working with senior
managers
27
New understanding of seconded
organisation and its work
Understanding of the research world – skills, structures, funding,
career progression, pressures and motivations of research staff
Broadening understanding of own organisation by explaining it to
others
Understanding how the commissioning cycle
works, timescales and pressures within the
commissioning world, understanding different
aspects of the health service and being able to
explain these to other people.
New networks and contacts New research contacts and networks, including becoming a co-
applicant on research applications
New networks of commissioning and
professional contacts to develop co-produced
research and evaluation
Developing new research skills Research project management including protocol writing, research
proposal writing, understanding ethical review processes,
developing and commenting on research bids.
Academic writing, co-authoring an article, commenting on research
reports and journal articles.
Understanding different methods: e.g. economic evaluation,
randomised control trials, process evaluation, different evaluation
methods and skills, systematic reviews and meta-analysis.
Research skills: literature searching and critical analysis of
literature, focus groups, interview skills, interviewing patients,
observations, evaluation skills, questionnaire design and delivery,
combining qualitative and quantitative data and triangulation,
qualitative data analysis, individual and cross case analysis.
Developing own research skills and knowledge
e.g. basic epidemiology; designing formative
evaluations; evaluation logic modelling and
theories of change; regression analysis and
various quantitative skills (with the aim of
passing these onto NHS analysts)
Developing new research projects Developing own research projects e.g. research aware
questionnaire within Bristol CCG, professionals experiences of joint
working with South Gloucestershire CCG and Sirona Care & Health.
Developing new co-produced evaluations,
systematic reviews and research projects,
bringing in new funding for projects
28
Impact on KM team members’ role and careers
The experience of KM work within the team clearly develops people’s skills and knowledge.
Working across two organisations provides new insights, skills and understandings; yet it
can also be a conflicting experience, where people are stretched too far, trying to manage
different demands. The question of how skills and experiences are utilised and developed
beyond the seconded placements is tackled within section 6 p59. However in terms of
people’s own ambitions and careers there was a sense that people generally wanted to
return to different roles that incorporated, developed and appreciated these new skills.
Where people potentially lost contact with the KM team on their returns to their home
organisations there was some sense of isolation. There was also a fear that in returning to
home organisations, networks and relations that had developed, may be lost, due to no
longer having a presence in the seconded organisation. Most secondees wished for a new
role that had a greater focus on knowledge mobilisation activities within their work.
29
5: KMT involved projects/studies & in depth case studies
In this section we present a comprehensive list of studies in which the KM team are/have
been involved indicating those which the KM team initiated or lead, local lead organisation,
scale and project funder(s). We have also selected 9 projects, activities or roles to present as
in-depth case studies.
5.1 KM team involved studies Since September 2013 the KM Team has contributed/are continuing to contribute to 32
studies (see Table 5.1 overleaf). These studies vary enormously in their size, setting, focus
and methods. In order to represent these differences we categorised projects into small or
large scale projects based on their type, complexity, duration, resources and number of
contributing sites, we subsequently verified this description with key informants.
Of the 32 KMT involved projects/studies 17 were/are large scale complex studies, 11 small,
2 small scale studies within larger ones and 2 relate to KMT involvement in Health
Integration Teams (HITs). KMT contributions to these studies/projects vary. KMT members
led 12 studies and initiated 12 (they are more likely to lead and initiate small scale studies
rather than larger ones). This list is not exhaustive and the data suggest that the KM team
may have contributed to additional studies/projects, however those listed have been
verified by different data sources and by members of the KM team.
Note Table 5.1: It is important to note that the number of studies per KMT member is not
significant, all have very different skills, have been placed in very different contexts and
studies differ in their complexity and demands.
30
Table 5.1: Showing KM team involved projects/studies, those which the KM team initiated or lead, local lead organisation, scale and project funder(s) KM team member(s) involved KMT
led? KMT
initiated? Lead local
organisation
Small or large scale
(SS/LS)
Study funder – source of funding for overall project not necessarily for KMT input
Project/study LW JC RA HC HB JR BR NA
3D study X x x x N N UoB LS NIHR Hs & DR
Abnormal test results for out of hours service
x Y N UoB SS APCRC
A frailty toolkit for Primary care – Elizabeth Blackwell project
x x Y Y UoB/SGCCG SS EBI Research for Health Challenge
ALSPAC x N N UoB LS Welcome trust
Alt-Con Alternatives to GP face to face consult
x x x N N UoB LS NIHR HS&DR
An admission avoided systematic review x x Y Y UoB/SGCCG SS APCRC
Benchmarking data in commissioning PhD research
x x N N UoB SS Not Known
Cardiovascular review x x Y Y CCG SS Public Health
Delays to discharge & clinical decision making
x Y Y UoB SS APCRC/CLAHRC West
Detecting bacterial resistance to antibiotics x N Y UoB SS Bristolbridge UoB
Diabetes evaluation x x x Y Y BCCG/UoB LS BCCG & NIHR KMRF
Discovery programme x x N N UoB LS NIHR
End of life care co-ordination centre evaluation (NB didn’t go ahead)
x x Y Y BCCG/UoB LS BCCG & NIHR KMRF
Healthlines trial x N N UoB LS NIHR Programme Grant for Applied Research
IMPRove HIT x N N BHP NA NA
ITHACA HIT x x x N N BHP NA NA
House of care (discrete part of HG wells diabetes project)
x Y Y BCCG SS in LS BCCG
31
KM team member(s) involved KMT led?
KMT initiated?
Lead local organisation
Small or large scale
(SS/LS)
Study funder – source of funding for overall project not necessarily for KMT input
Project/study LW JC RA HC HB JR BR NA
Joint Front door project x x N N CCG LS Application submitted to CLAHRC West
Professionals experience of Joint working x x Y Y SGCCG SS APCRC
Knowledge exchange in commissioning x x x x Y Y UoB SS NIHR HDSR
LTC group (observation of) x x N N UoB/CCG LS APCRC
Primary care factors (associated with use of unscheduled secondary care)
x N N UoB LS NIHR HDSR
PROVIDE – domestic violence x x x N N UoB LS NIHR Programme grant
QuICN (quality indicators for community services)
x UWE LS NIHR HSDR
Red Cross Project x x N N UWE SS Red Cross
RE-PROVIDE (JC & HC are co-applicants) x x x N N UoB LS NIHR Programme grant
Research aware questionnaire – research use by commissioners survey
x Y Y UoB SS APCRC
Telehealth evaluation x x x x x Y Y UoB LS BCCG & NIHR KMRF
TimeLi Dementia Diagnosis study - PhD research
x N N UoB SS in LS Wellcome trust
'Twilight study' A qualitative study to explore the journey taken by people aged over 85 in their last years of life
x N N UoB LS Not known
Enhanced Hospital Alternatives Programme x x N N UWE/UoB/ BCCG
LS NIHR programme grant under development
GPs in ED x N N UWE/UoB/ BCCG
LS NIHR HS&DR – second stage development
32
5.2 Case Studies The following 9 case studies were selected to illustrate different types of engagement, different
settings and scope/scale of projects, or significant activities or roles. These case studies present
concrete examples of the KM team’s contribution and impact. Each case study is based on all the
data sources not individual interviews, we have kept direct quotes to a minimum in these case
studies to protect the anonymity of sources.
Each case study follows a set format which includes: 1) a brief description of the study/context, 2)
what happened (process, what did KM team member contribute and how), 3) what was the impact
or outcomes, 4) what helped (in terms of supporting the KMT member and/or optimising KM), 5)
what didn’t help, 6) key take-away messages, 7) types of knowledge shared (see Diagram 3.2), 8)
Context: Mechanism: Outcomes based on this case study – i.e. what can be said about ‘why the KMT
programme works, for whom it works and the sets of circumstances within which it works.’ (See
Methods in Appendix). The following 9 case studies are divided into: project, role and activity case
studies.
For quick reference Section 5.3 p.60 provides a summary of the key messages that arise from the
case studies and answers the question: what works, for whom and in what circumstances in these
knowledge mobilisation activities.
A. Project case studies 1. Red Cross Project ‘Exploring the role of first aid education in supporting people attending A&E’
2. Elizabeth Blackwell Project
3. 3D study
4. Programme of Research on Violence in Diverse Domestic Environments (PROVIDE)
5. Knowledge exchange in commissioning
6. Diabetes Evaluation
7. Benchmarking data in commissioning PhD research
B. Role case studies 8. Lesley Wye team leader KMT
C. Activity case studies 9. KM awareness raising
33
Note: All project and role case studies were reviewed and verified by the relevant KMT member and
other key informants before inclusion. Case Study 9 relates to broader activities and draws on wider
data sources so this was not deemed necessary for this one.
34
Case Study 1: Red Cross Project ‘Exploring the role of first aid education in supporting people attending A&E’
1. Brief description of study/context
KM Team Members involved: Primary: Helen Baxter (RR) & Nadya Anscombe (CO)
Aim:
To work with the Red Cross to develop a first aid intervention to help reduce unnecessary hospital admissions via A+E.
Method: Mixed methods
Initiated by:
Email sent around SSCM and a small research team formed followed by KM team member emailing interested parties, having opportunistic conversations with key people or while attending Urgent Care Steering Group meetings.
Setting: UWE, UoB and Bristol CCG
Collaborators: Bristol CCG, UWE, Red Cross, UoB, BHP ITHAcA & CIPIC HIT
Project Lead: Julie Mytton, UWE
Local lead organisation: University of the West of England
Funding: £59,913 British Red Cross
Scale: Small
2. What happened (process, what did KM team member contribute and how) HB and the CCG had already identified the Red Cross as key players in urgent care and were keen
to engage with them.
HB had multiple conversations via email and within Urgent care Steering group meetings which helped to develop ideas and link interested parties.
The Red Cross call for research tenders came out.
A small research team formed at UoB and HB suggested forming a small steering group of lead academics and stakeholders.
The morning after HB met with a key person in the lift by chance, this led to further discussions, and agreement to sit on steering group with other lead clinicians.
Other stakeholders were approached and a project team developed.
Project proposal submitted for a collaborative research led project (across Universities and HITs) with clinician and commissioner advice and support.
Funding secured
NA has drafted a press release about successful funding for this project
3. What was the impact or outcomes? Funded collaborative project with involvement of multiple organisations
Voluntary sector funding and engagement
Collaboration across 2 HITs and 2 Universities with commissioner involvement.
Relationships built with potential for further collaborative work
35
4. What helped? Prior interest and network development
Opportunity and chance – the ‘magic lift’
KM team member linking and networking, knowledge and skills
KM team member prior reputation, networks and embeddedness in CCG & ITHAcA HIT
Established networks across the CIPIC and ITHAcA HITs enabled the bidding team to come together quickly this was particularly important as there was such a short period (2 weeks) between tender and submission
British Red Cross tender
Willingness of all parties to engage and share knowledge and skills
5. What didn’t help? This was a highly successful bid – the project started on March 1st.
Note: this case study demonstrates KM team success in initiating, co-designing and securing funding for collaborative projects, an important element in KM. The project is underway and it is not yet possible to evaluate whether it has any impact on practice.
6. Key take away messages Relational activities can have tangible outcomes
KM team members are ideally placed to link disparate parties around common interests and goals
The case highlights the importance of researchers being in commissioning premises on a regular basis to be able to network and develop relationships and projects within commissioners own environment.
7. Types of knowledge shared: relational, organisational, research methods, funding sources, research evidence (to inform proposal)
8. Context Mechanism Outcomes The RR used her prior knowledge and networks within the field of Urgent care to help a group of
interested academics and stakeholders to respond to a funding call.
An opportunistic meeting assisted in moving the project forward.
RR placement at the CCG optimised commissioner involvement and her knowledge of academics with relevant expertise helped link interested parties.
36
Case Study 2: Elizabeth Blackwell project – ‘A frailty toolkit for Primary care: an explorative study of the utility and acceptability of a Comprehensive Geriatric Assessment (CGA) toolkit within primary care’
1. Brief description of study/context
KM Team Members involved: Primary: Helen Baxter (RR) and James Rooney (MF)
Aim: To investigate the existing evidence and the acceptability of the use of a frailty toolkit in primary care
Method: Systematic Review of evidence, GP interviews
Initiated by: Helen Baxter at the request of SG CCG
Setting: SG CCG
Collaborators: UoB, Bristol CCG & SG CCG, ITHAcA HIT
Project Lead: Helen Baxter
Local lead organisation: UoB & SG CCG
Funding: £25,000 Research for Health Challenge, Elizabeth Blackwell Institute
Scale: Small
2. What happened (process, what did KM team member contribute and how) HB was already involved in an NBT project on the topic of ‘frailty’ in secondary care.
In support of this project and the wider care agenda she met with other NHS/HEI stakeholders to discuss services and research ideas relating to frailty
HB met with a GP Clinical Evidence Fellow and the Urgent Care Clinical Lead at South Gloucs CCG to discuss how frailty services are currently being designed and evaluated to see if there are ways and mechanisms to do this more effectively. They agreed to work together to avoid duplication of effort, encourage different initiatives and make use of lessons learned elsewhere. HB involved Bristol CCG as well who were also interested in the topic.
They identified a gap in the literature with regards to frailty screening tools for use in primary care
Elizabeth Blackwell Institute introduced a scheme to encourage healthcare practitioners and University of Bristol researchers to work together to develop innovative thinking around clinical problems. The EBI Research for Health Challenge
The GP Clinical Evidence fellow alerted South Gloucs CCG to the funding opportunity (which HB was also aware of).
HB assisted commissioner(s) in formulating a research question and completing the funding application by interpreting research language. She helped ‘me understand it in University terms … how researchers think as opposed to how we think’
HB introduced a qualitative component to permit better understanding of GP’s concerns
HB helped commissioners canvas GP’s to ensure their views were represented in the funding application
HB consulted and worked with GP Clinical Evidence Fellows on project design
HB brought in two senior UoB researchers with expertise in systematic reviews 1) as a co-applicant and 2) to join the Steering Group
Latterly JR has facilitated contact with commissioning colleagues and attended project meetings, it is anticipated he will have more involvement in this study in the future
37
3. What was the impact or outcomes? A funded project with a primary care focus: previously the majority of topics were perceived as
stemming from secondary care
Researcher-Commissioner collaboration - using research knowhow to answer a commissioner identified problem
Relationships built with potential for further collaborative work
4. What helped?
HB could translate some of her work in secondary care into a primary care focus
Regarded by commissioners as an important topic needing answers
Interest and engagement of commissioners
Support of ITHaCa HIT and North Bristol Trust
Knowledge of funding opportunities
Linkage with GP Clinical Evidence Fellows
HB supported commissioners in understanding what was required in submitting a proposal
Linked to an existing HB project and area of interest in secondary care
Existing links with HEI and NHS experts
‘Very applied’, small scale project
Willingness of all parties to collaborate
5. What didn’t help? ‘Political must do’s’ and short timeframe for delivery
6. Key take away messages Collaborative research works well when the idea is developed within the commissioning environment
and meets an identified need.
Different KM initiatives (the KMT and GP Clinical Evidence Fellow) can complement each other’s skills and expertise and assist commissioners in designing and securing funding for applied research
RR can assist commissioners in understanding research language, in formulating their questions and by introducing specialist research expertise
7. Types of Knowledge shared: Organisational, relational, research methods, research evidence, funding sources
8. Context Mechanism Outcomes Opportunities to develop collaborations between commissioners and researchers can be proactively
developed by the RR, especially where the topic fits within their own expertise and knowledge and with commissioner priorities. Assisting commissioners to develop their ideas and proposals can lead to successful funding for collaborative research.
Note: this case study demonstrates KM team success in initiating, co-designing and securing funding for collaborative projects, an important element in KM. The project is underway and it is not yet possible to evaluate whether it has any impact on practice.
38
Case Study 3 (Project): 3D Research Project
1. Brief description of study/context
KM Team Members
involved
Becca Robinson (MF), Lesley Wye (KMRF); Rachel Anthwal (MF);
Nadya Anscombe (CO)
Aim
Multi-centre study trying to help general practices to improve how they
treat patients with multiple long term health problems. KM team
involvement has concerned linking with NHS organisations, undertaking
research and developing communications and dissemination plans.
Method Multi-centred cluster randomised trial of a complex intervention.
Setting University of Bristol
Collaborators Multiple partner study collaborating across different universities
including Bristol, Manchester, Glasgow and Dundee
Project Lead Professor Chris Salisbury
Local lead organisation University of Bristol
Funding NIHR HDSR
Scale Large multi-site
2. What happened (process, what did KM team member contribute and how)
The MF supported ongoing communications links between the NHS and the study, increasing
commissioners’ awareness of this study, and facilitating communications between researchers
and health organisations including BNSSG CCGs.
The MF helped with the process evaluation, supporting recruitment, supporting PPI training,
undertaking interviews with commissioners and practices.
The MF helped with the work to understand how individuals living with long term conditions are
currently managed in general practice
The MF helped to develop a dissemination plan for the 3D study team
The MF will be helping facilitate patient focus groups to understand their perceptions of care as
part of the intervention.
The MF participates in local and other site project team & programme management group
meetings contributing to project processes & progress monitoring and supplying additional
expertise based on her knowledge of commissioning organisations and KM.
39
3. What was the impact or outcomes?
The research team were given information about other interventions that GP practices were
involved in. These included the schemes and initiatives run by the CCG, and secondly, other
research studies that the CCG were aware of but the researchers were not. This enabled
negotiation around which practices became participants, avoiding cross contamination of
different research studies.
The MF was more easily able to communicate with commissioners than researchers had been
able to, knowing their motives and ‘what buttons to push’. This supported the implementation of
the research study and ongoing communications with CCGs, strengthening relationships. Links
with the local CCG also facilitated relationships with other national CCGs that were involved in
the project.
An evolving dissemination plan and outputs other than journal articles that cover a broader
range of audiences, including charities and PPI groups, conducting KM throughout the study,
rather than just at the end.
Members of the study put more ‘thought and effort and attention’ into developing newsletters
and the website for the project throughout the study.
4. What helped?
MF’s ability to find out who were the people that researchers needed to speak to, to facilitate the
research, knowing how to get hold of people and understanding the organisational context.
Having a commissioning manager speak to other NHS managers was perceived to be more
effective than a researcher-to-manager conversation. Knowing the system and being perceived as
part of it makes a difference.
Commissioners knowing about developments in the commissioning world that may impact on a
study.
Having a practitioner investigate what normal and usual care is taps into their practitioner
knowledge
Having a CCG MF as part of the project has helped to engage other commissioners within the
project, ‘they feel it’s their project too’.
Generating links with other CCGs beyond Bristol including SG and NS.
5. What were the difficulties?
The MF was involved in contacting other CCGs outside the regional area, which may have been
more difficult as there was less knowledge of how different locality systems work
It took some time for the MF to feel like they got up to speed with the research project, to a point
in feeling confident about their contributions.
Through the secondment, as the MF has been developing their skills their work has been
focussed on a greater variety of different projects, so they have less time to contribute, when
their skill set is more developed.
40
6. Key take away messages
Organisational and relational knowledge of the MF helped the research study navigate and
implement the research
This also provided the MF with opportunities to develop their research skills.
Both researchers and commissioners learnt from each other through the collaboration
7. Types of knowledge shared
Relational knowledge, organisational knowledge, experiential/clinical knowledge, research design
and methods knowledge, diverse dissemination methods
8. Context Mechanism Outcomes
Complex studies involving multiple health partners benefit from NHS managers being on board
to navigate and communicate with many health partners. This leads to more smooth running
research and greater commissioner engagement with the research, and wider dissemination
opportunities.
41
Case Study 4 (Project): Programme of Research on Violence in Diverse Domestic Environments (PROVIDE)
1. Brief description of study/context
KM Team Members involved Primary Jude Carey (MF) & Becca Robinson (MF), Nadya Anscombe (CO)
Aim
PROVIDE was a UK National Institute of Health Research funded programme which ran between October 2009 and September 2014. It brought together a group of key DVA researchers from the health and social science fields. PROVIDE comprised 4 work streams sited in general practice, sexual health clinics and 3rd sector domestic violence agencies. The topics range from the prevalence of domestic violence in health care settings and the health care response for men as victims or perpetrators to improving outcomes for women seeking help from domestic violence agencies with a psychological intervention.
Collaborators UoB, RESPECT, Bristol Next Link, London School of Hygiene and Tropical Medicine, Cardiff Women’s aid, Domestic Violence training Ltd., Guys & St Thomas’ NHS Foundation Trust, Kings college London, Galop, Bristol City Council, APCRC
Local lead organisation UoB
Academic Lead Gene Feder
Funding NIHR Programme Grant
Scale Large scale complex multi-site study with 4 work streams
2. What happened (process, what did KM team member contribute and how) JC was embedded within the PROVIDE team at the beginning of her fellowship when the
project was approaching the dissemination stage
JC worked with Jayne Bailey (PROVIDE trial manager) to develop a KM strategy
JC attended the quarterly PROVIDE Advisory Group meetings as well as other meetings related to the programme where appropriate (trial management and steering group meetings).
JC & NA created a database comprising 355 contacts from a range of organisations and sectors which included amongst others: third sector, health commissioners, local authority commissioners, NHS safeguarding leads, public health leads, and clinicians from a range of backgrounds including primary care and mental health and criminal justice commissioners and professionals.
JC & NA created a mailing list using MailChimp (an online e-mail management service) and a series of e-mail campaigns related to the PROVIDE conference planned for November 2014 (see below).
JC & NA contributed to development and ongoing review of PROVIDE website including development of two briefing notes for research papers.
JC & NA established a Twitter account (DV_Bristol) which has been used to promote output from the research programme including papers, chapters, briefing notes, videos and events.
JC completed a stakeholder mapping exercise informed by individual interviews with all members of the PROVIDE research team and Advisory Group, including representatives from commissioning and third sector organisations
JC ran a KM workshop attended by same individuals to agree on a KM approach for the programme
42
JC completed preparatory work with researchers encouraging them to present their research findings and key messages in max 3 slides; raise awareness of KM and translation of research findings
JC consulted with commissioners of domestic violence services from the Local Authority and Public Health. They provided suggestions regarding ways to disseminate PROVIDE findings to commissioning and public health audiences, through specific strategy groups and a number of public health conferences. They also commented on the priority and relevance of the research findings to commissioners
JC produced (filmed and edited) three short videos including researchers talking about their studies and a third sector collaborator discussing engagement in the research programme. These have been uploaded to a Domestic Violence YouTube channel that was established for this purpose. These videos produced in response to PPI recommendations.
JC contributed to development of guidelines for IRIS (domestic violence training, identification and signposting in general practice) national and local implementation teams that outline the types of information that would be valuable and persuasive to commissioners of services.
JC & NA significant involvement in organising end of programme conference which included innovative KM elements:
Used a targeted proportional ticketing approach to ensure representation of Public Health, commissioning, primary care and mental health professions alongside third sector and academic staff.
Conference programme designed to acknowledge the wide audience expected. Guidance notes developed for presenters and chairs which included advice about not just
presenting key findings but also focussing on the key messages and anticipated impact of the research.
Conference programme allowed time for discussion after each group of presentations to stimulate cross-sector responses to our research findings.
Offered presenters an opportunity to practice their presentations and four took up this option. As a result researchers were able to tailor their presentation for a wider audience by changing the focus, terminology and approach to the presentation.
JC completed in-depth evaluation of the conference based in part on work undertaken by the Australian Primary Health Care Research and Information Service. JC asked attendees to assess the extent to which attending the conference had broadened their knowledge, stimulated their thinking and would impact on their work in future using a 5 point Likert scale
JC contributed to research activities for work stream 2 by interviewing domestic violence support workers about their experiences of being involved in the PATH trial. The data provided will support the production of a number of published papers around implementation, ethics and fidelity with the model. (May 2014)
JC attended meetings to comment on draft commissioning proposals proposed re-commissioning of a range of domestic violence services by Bristol City Council
JC facilitated presentation of research findings at Men and Boy’s stakeholder group (mix of commissioners, providers, local authority reps) in 6 May 2014.
JC used experience within PROVIDE to inform informal and formal presentations on KM
Arranged for Bristol City Council to consult with Domestic Violence (DV) researchers on their commissioning plans for DV services - agreed on mutual benefits and means for researchers and commissioners to work more closely together.
BR ongoing support for commissioner engagement by arranging meetings to share research findings and discuss who should be engaged in further communications.
43
3. What was the impact or outcomes?
Specific impacts & outcomes
Conference outcomes
a) delegates included representatives from
Four local NHS commissioning organisations including North Somerset CCG, Bristol CCG, South
Glos CCG and South West Commissioning Support Unit.
Four local authorities – North Somerset Council, Leicester City Council, Bristol City Council and
City of Cardiff Council.
Third sector organisations in the following areas: children’s, women’s, men’s, domestic violence
and abuse, drugs and alcohol, housing and homelessness, mental health and wellbeing, equality
and diversity.
Academic presence nationally from Bristol, Leicester, London, Sunderland, York, Glasgow,
Nottingham, Hull, Oxford and South Wales and internationally from Deakin University,
Melbourne.
b) Feedback on the event
53 (89%) people agreed or strongly agreed that the conference had broadened their knowledge
of the subject. (6 were neutral)
57 (97%) people agreed or strongly agreed that the conference had stimulated their thinking on
the subject. (2 were neutral)
55 (93%) people agreed or strongly agreed that attending the conference would impact on their
future work (3 were neutral, 1 disagreed)
c) The KMT helped to make the conference more accessible and creative to diverse audiences ‘so I
was juggling at one point and… somebody had … helium balloons to represent different things
and, so everybody could do what they wanted … in their three minutes and did different things, so
it was really great for us cos we were able to be a bit more creative’
d) JC ‘was crucial in helping us design that (the conference), so that we had then this lovely mixture
of um researchers, service providers, DV service providers and, and Commissioners. Which helped
design that, the kind of, the interaction that happened between those groups at the event, which I
think was very, actually very helpful.’
Mail Chimp has permitted tracking of a range of metrics including; the number of people
subscribing and unsubscribing from the list, the proportion of recipients opening and forwarding
the correspondence and the geographical reach of each campaign. The list has now increased to
417 through additional subscriptions; during the same period a total of 5 people have
unsubscribed.
The Twitter account was established in April 2014 and now has a following of 186 people including
NHS professionals and organisations, domestic violence researchers and professionals and third
sector representatives.
JC has continued involvement and contributed to successful (£2.5 million) follow-on RE-PROVIDE
programme grant application as a co-applicant/KM lead, she has been funded for 2 hours dedicated
KM input per week on this 6 year study
Arranged for Bristol City Council to consult with Domestic Violence (DV) researchers on their
commissioning plans for DV services - agreed on mutual benefits and means for researchers and
commissioners to work more closely together.
PROVIDE team presented formal feedback to Bristol City Council on meeting to discuss draft
commissioning proposals, identified a range of collaborative proposals and subsequently two
researchers from Bristol University, Prof Feder and Dr Natasha Mulvihill from the Centre for Gender
and Violence Studies were invited to sit on the Bristol Domestic and Sexual Abuse Strategy Group
44
More general impact and outcomes on PROVIDE
KMT ‘made a substantial contribution because we were at the sort of half way through a programme grant, with, with some very real patient benefit type outcomes in prospect’ getting help with the KM elements ‘made a tremendous contribution’
JC helped researchers use the appropriate language to get their messages across to different audiences: ‘we have to use very specific wording … like, er, potentially abusive behaviours which doesn’t drip off the tongue … and isn’t great if you’re trying to mobilise knowledge … to people who’ve got five minutes. So we had a lot of conversations about that, about the sensitivities’
“So we do kind of have the links locally (with commissioners), but it was also about getting the wider national links I think … and that’s what she was able to help with particularly”
‘JC is a Commissioner, so she could talk to her mates and say “Come to a meeting … we felt that we were able to talk to Commissioners. We knew who they were, we knew where they lived, and that was with the help of the KM Team.’
Wider impacts & outcomes
JC ‘helped across the board, you know, senior, junior researchers, got them in thinking about what, what concrete needs to be done for KM.’
Wider impact: Jayne Bailey was selected to attend the Health Innovation Programme to present IRIS (precursor to PROVIDE) as a social enterprise, where she won the award for best business idea with the most social impact. Alison Gregory, a CAPC researcher, along with Gene Feder then received 3 months of funding from HEFCE and Innovate UK to work with commissioners and others to develop a not-for-profit business to disseminate and implement findings from IRIS. Their application was unusual, as most successful applications are for gadgets or tools. Although the KM team had no input into the grant application, the fact that the domestic violence team applied for and won this recognition is a marker of the changing culture within CAPC around KM.
PROVIDE outputs incorporated into a London Councils good practice guide on domestic and sexual violence
PROVIDE - recommendations on opportunities and mechanisms for the dissemination of research findings within the NHS will be incorporated within the UoB knowledge mobilisation strategy due to be drafted in Jan/Feb 2014
4. What helped?
KM literature review performed as part of JC’s development assisted her in developing KM strategy for PROVIDE
Embeddedness within the team, going ‘native’ becoming part of the team
Personal enthusiasm, drive, interest, and willingness to learn
Working within an established successful team with pre-existing emphasis on applied research and working with multiple stakeholders.
Seeing the programme valued and expand
JC ‘got the politics’ and understood the complexities and importance of getting the message across
Willingness of both parties to invest time, energy and support
Provided extra capacity – an extra pair of hands
NA’s communication and social media expertise
LW managerial and pastoral role
Ongoing involvement of the KM team
45
5. What didn’t help? The extent to which researchers actively participated in developing briefing notes, key
messages documents and creating videos varied • Location of the researchers due to multi-centre nature of the programme lead to difficulty in
developing good working relationships over a short time period and over distance • Availability – some researchers were no longer actively involved in the project • Level of interest – there was definitely a broad spectrum in terms of interest and enthusiasm
for knowledge mobilisation
The impact of research is not currently perceived as linked to career progression for researchers and therefore may not be viewed as a priority regardless of their interest in getting their research into practice or policy
6. Key take away messages The MF and KMT can be pivotal in developing more accessible and creative dissemination
methods for applied research
Their organisational, relational knowledge and communicative know-how can assist in translating research findings for diverse non-academic audiences
The KMT can provide a broad range of additional skills and expertise to enhance the KM potential and impact of research
The fit between individual skills, expertise and task/project context and phase is important
7. Types of knowledge shared: relational, organisational, dissemination methods, KM theory and practice, research evidence, evaluation design and methods
8. Context Mechanism Outcomes The KMT in particular MFs can have wide ranging impact on large scale university led projects
within established research teams with applied emphasis.
Becoming fully embedded optimises the impact on host and secondee, it can lead to fruitful cross-fertilisation of skills and ideas, with potential for wider KM awareness raising and further collaborative work.
46
Case Study 5. Knowledge exchange in commissioning
1. Brief description of study/context
KM Team members involved
Primary: Lesley Wye (KMRF) & Rachel Anthwal (MF), James Rooney (MF), Nadya Anscombe (CO), Becca Robinson (MF)
Aim
To understand more about commissioning, commissioners’ information seeking behaviour and the role of research in their decisions and to learn how research can influence policy.
Method Empirical Qualitative study
Collaborators UoB: Emer Brangan, Ailsa Cameron Southampton University: John Gabbay, Jonathan H. Klein and Catherine Pope
Project lead Lesley Wye/UoB
Funding NIHR HSRD
Scale Large
2. What happened (process, what did KM team member contribute and how) LW was PI on this study
RA started her MF in the final six months project and contributed to: 1. Analysis of case studies to pull out answers to the research questions for the study including key points from a commissioning perspective 2. Research team meetings & teleconferences 3. A workshop with wider stakeholder colleagues, including a number of local commissioners. 4. Compilation of the final report, including contributing to the review of case studies and draft chapters, testing the findings against her experience in the ‘real world’. 5. Working with an external advisor (part of the project team) on the key findings and actionable messages from the external providers case studies. 6. Planning of a session with the research team, additional commissioners and analysts to discuss the key findings and actionable messages to inform the dissemination strategy for the project. Identifying and engaging commissioners to take part in the design and implementation of the knowledge mobilization strategy. 7. Organised and conducted dissemination workshop with researchers and commissioners to identify key actionable messages on 8 April 2014. These were included in the final report as an appendix.
The findings broke down into 2 areas: 1) the use of external providers to inform commissioner decision making (disseminated via point 4) and 2) ways that researchers could inform commissioning (disseminated via points 5, 6 & 7).
LW, JR, BR & NA contributed to a mass media campaign for the first outputs from this project including: e-mail to 600 contacts from the contact list with links to these documents, a press release to major specialist commissioning press and twitter feeds, a blog with PolicyBristol and an 800 word comment piece for the Health Service Journal.
As the second area of findings was targeted to researchers, conferences and scientific journals were a perfect forum. LW gave three oral presentations to South West Society for Academic Primary Care (SAPC) conference, national SAPC conference and the Health Service Research Network conference all in 2015. LW and BR also gave elevator pitches on commissioners’ information seeking behaviour and the subsequent development of the KM team at 3 conferences.
LW launched the PenCLAHRC implementation seminar series by presenting these findings to an audience of 30 researchers.
LW met and discussed the findings with local MP Charlotte Leslie.
47
RA developed a briefing for senior managers at the Commissioning Support Unit to disseminate study findings and inform the units approach to working with external partners e.g. PWC as well as the CCG.
Preparation and publication of 2 peer reviewed papers (BMJ Open and BMC Health Services Research)
3. What was the impact or outcomes?
Provided an opportunity for KMT MF to develop research and dissemination skills and to use their knowledge of commissioning to inform both. RA fulfilled her ambition to co-author a scientific paper
Outputs
2 Peer reviewed articles:
1. Wye, L, Brangan, E, Cameron, A, Gabbay, J, Klein, J, Anthwal R, Pope, C, (2015) ‘What do external consultants from private and not-for-profit companies offer healthcare commissioners? A qualitative study of knowledge exchange’. BMJ Open, vol 5.
2. Wye L, Brangan E, Cameron A, Gabbay J, Klein J, Pope C (2014) Evidence based policy making and the ‘art’ of commissioning – how English healthcare commissioners access and use information and academic research in ‘real life’ decision-making: an empirical qualitative study BMC Health Services Research. 15:430 DOI: 10.1186/s12913-0
Final report: 3. Wye L, Brangan E, Cameron A, Gabbay J, Klein J, Pope C (2015) Knowledge exchange in health-
care commissioning: case studies of the use of commercial, not-for-profit and public sector agencies, 2011-14 Health Services and Delivery Research, 3 (19 )
Generating new knowledge & influencing wider KM practice
This study underpinned the development of LW’s application for a Knowledge Mobilisation Fellowship and the design of the KM team, so a major impact was the development of the KM team itself.
Findings of this study could be explicitly linked to the development and role of the KMT and used to inform KMT activities more generally
The study topic is central to the whole KMT exercise and furnished the team/others with better knowledge and understanding
Key findings from this study are that a way to bridge the researcher-commissioner gap is through collaborations with commissioners through co-produced, commissioner-initiated evaluations.
LW was contacted to help others set up similar schemes in Oxford, Liverpool and Southampton.
Knowledge Exchange in Commissioning - recommendations on opportunities and mechanisms for the dissemination of research findings within the NHS incorporated into the UoB Knowledge Mobilisation strategy
The NIHR Dissemination Centre visited Bristol to see what they could adopt nationally.
4. What helped? LW as PI was able to support the KMT in acquiring new research and dissemination skills she was
‘really linked in’ and ‘very practical’. Regular meetings and supervision were really helpful
LW was vital in making MFs feel part of the team and giving them opportunities to get involved
The phase in the project ‘was a really good point for a fellow to come in because it was kind of writing up stage, where they were … doing the analysis so great learning’ a great development opportunity
48
The phase in the project was ideal to utilise MF’s organisational, relational and experiential knowledge to good effect so they could ‘really influence some of the key messages coming out of reports’, they also felt that their contribution was useful and their knowledge valued
Creative and imaginative use of social media and NA communications know how
Targeting researchers as a key audience to help them learn how to interact with commissioners better.
5. What didn’t help? Personal issues and other work demands affected work flow planning and KMT contributions
Tensions arose between need to deliver peer reviewed outputs vs dissemination to non-academic audiences ‘it’s difficult to push forward on both fronts simultaneously’
Mass media outputs received an unexpectedly poor response leading to re-evaluation of the approach.
Researchers were really interested in this study, while NHS managers less so.
6. Key take away messages Comprehensive understanding of commissioner decision making and the relationship between
commissioners and research is essential for improving KM and for the development of an effective KMT
Having a clear evidence base to inform the design and development of the KM team is strategically beneficial and highly useful
The MF’s organisational and relational knowledge helped contextualise this research to the real world of commissioning
This also provided the MF with opportunities to develop her research skills.
Commissioner’s engagement with research dissemination is limited by many factors, much wider dissemination to researchers is possible.
Targeting effort and resources and tailored dissemination can assist in ensuring wide dissemination, but presents challenges and requires further work
7. Types of knowledge shared: Research methods, KM theory and practice, research evidence, dissemination methods, organisational and relational knowledge, experiential knowledge
8. Context Mechanism Outcomes This study provides the rationale for creation of the KMT.
Embedding commissioner know how was important and helped the secondee develop new skills.
LW’s senior support, expertise and drive were significant in sustaining KMT involvement, momentum and wide dissemination of study findings
Achieving balance between university and commissioning organisations engagement and priorities is difficult
49
Case Study 6 (Project): Diabetes evaluation
1. Brief description of study/context
KM Team Members involved:
Helen Cramer (RR), Becca Robinson (MF), Rachel Anthwal (through NHS role as LTC programme lead for Bristol CCG).
Aim:
To work with the long term conditions (LTC) steering group to develop an evaluation for diabetes services
Initiated by: RR in collaboration with LTC steering group members and diabetes lead.
Setting: BCCG/UoB
Collaborators: BCCG, UoB, CSU
Project Lead: Helen Cramer, University of Bristol
Funding: BCCG & NIHR KMRF
Scale: Large
2. What happened (process, what did KM team member contribute and how) With members of the LTC steering group HC agreed to develop an evaluation of an integrated
model of care for diabetes incorporating shared decision making for 16 GP practices.
With help from the Programme Lead, LTC steering group members, APCRC, and KM team members and leader, an evaluation protocol was agreed for the diabetes integrated model of care or ‘HG Wells project’.
This was a formative evaluation case study approach, focussing on what 3-4 practices do with additional data about their management of diabetes patients. The focus of this evaluation is on the changes GP practices make following the installation of new software and 3 monthly reports giving them enhanced information about diabetes management in their practice.
The RR brought in a MF to help with the evaluation and in turn develop the MF’s evaluation skills.
A data sharing contract between the university and the CCG was agreed and signed.
3. What was the impact or outcomes? An evaluation is currently being delivered. It evaluates a new model of working within diabetes
which gives GPs enhanced information about diabetes management in their practice.
The MF involved has developed their evaluation and interviewing skills.
The RR has contributed toward project planning and developing service evaluations with the CCG.
Practitioners have been able to discuss and apply evaluation methods to their work.
4. What helped? Working with LTC steering group members, and having the input of other KM team members
and APCRC in the design of the evaluation. Developing a carefully tailored and designed evaluation to the context, drawing on the RR’s research skills and methods.
Working closely with the diabetes project team to design a logic model/ theory of change, and agree on a manageable scale for the evaluation
Working with commissioners who have a background and understanding of different evaluation methods.
Discussing different forms of evaluation with the group, to agree on the way forward.
The RR bringing an external perspective to discussions, being ‘a very trusted reference point’
The RR helping to design the project’s initial participatory team building and goal setting meeting in pilot practices.
The MF was able to develop skills in interviewing patients, analysing data and observing diabetic reviews. She was also able to use her nursing practice to help inform the topic guide for
50
interviewing patients. This contributed to a two way exchange where the RR benefitted from the MF’s knowledge and the MF has learnt research and evaluation skills.
5. What were the difficulties encountered? Agreeing a focus for the evaluation was difficult, and discussion was needed to clarify what
aspects of the programme were most important and what an evaluation could realistically cover.
Substantial negotiation was needed to agree on quantity, quality and speed e.g. the need for transcription (research quality versus speed and resource).
Whether control groups are needed. Installing software and then asking practices to do nothing is sensitive politically.
Whether those practices willing to be evaluated represent those with most potential to change. Here the RR was able to influence the sampling of practices for the pilot.
Possible overlaps with other research studies and ‘contamination’.
Target journals for the planned papers have not yet been specified but some papers such as the qualitative evaluation may be constrained by the evaluation work only having CCG ethical approval.
A series of losses of key CCG staff has made the evaluation more difficult
Associated delays mean that the evaluation time frame may extend beyond the time limit of the RR role.
6. Key take away messages ‘I think that’s been a really, really fruitful and productive relationship’
Large scale evaluations take considerable time and resource to achieve
The benefits of being able to specifically design appropriate evaluations, as opposed to taking evaluation designs off the peg.
The collaborative work between RRs and MFs and commissioners supports knowledge exchange
7. Types of knowledge shared Evaluation and research design and methods, practice knowledge, relational knowledge,
organisational knowledge.
8. Context Mechanism Outcomes Loss of key personnel makes collaboration more difficult to achieve
Service evaluations need to be carefully tailored to meet the needs of both commissioners and researchers
51
Case Study 7 (Project): Benchmarking data in commissioning PhD research and beyond
1. Brief description of study/context
KM Team Members involved James Rooney (MF), Becca Robinson (MF)
Aim
Supporting researchers in their networking and research with commissioners
Initiated through
Networks and relationships within the University of Bristol that linked a PhD researcher with NHS Management Fellows.
Setting University of Bristol
Collaborators UoB, MFs, CCG
Project Lead PhD Researcher
Local lead organisation University of Bristol
Scale Small
2. What happened (process, what did KM team member contribute and how)
A UoB PhD researcher wanted to understand how commissioners use benchmarking data and what tools they use.
A MF tested out their survey questions and suggested changes to make them more commissioner friendly. This included making changes to the language, assessing what was most relevant, editing less useful questions and suggesting improvements.
The questionnaire was piloted by another MF.
MFs were particularly helpful in talking the researcher through commissioning structures and processes, identifying potential commissioning stakeholders and ‘opening the doors’ to be able to talk with some senior people within commissioning.
3. What was the impact or outcomes?
This collaboration helped to increase the PhD researcher’s number of interviews from around 3-4 to 10.
Following this experience, after the PhD was submitted, this researcher went onto a new job. A link with a commissioner that was initiated through one of the interviews conducted above, developed further.
Because the researcher already knew some of the commissioners, they ‘got talking’ and found some shared ground of research interests.
This has resulted in putting in a bid to the CLARHC call, with other researchers. This is focussed on improving the business cycle and the evidence underpinning commissioning decisions, with the aim of improving the quality of business cases with cost effectiveness evidence.
52
4. What helped?
Giving insight into how commissioning works and what commissioners think and do.
Being pointed in the right direction to specific commissioners. Giving specific contact details and information about commissioners that is not available on websites.
It was considered that the original project may never have got off the ground without the input of the MFs. Because of the links developed here, a further research bid has been developed and submitted to the CLAHRC.
MFs openness and helpfulness.
5. What didn’t help?
Both groups (commissioners and researchers) are very busy so ‘getting round the table’ can be very difficult.
‘My experience was solely positive… I’ve not much negative to say’
6. Key take away messages
Relational activities can have tangible outcomes
Outcomes that can develop from relational activities are not easy to predict and may happen some time after initial collaborations
7. Types of knowledge shared: Practice knowledge, relational knowledge, organisational knowledge.
8. Context Mechanism Outcome
Relational activities are important in mobilising knowledge.
Researchers benefit from having commissioners (NHS MFs) on an informal basis to talk to within the University buildings
MFs own networks were key to networking the PhD researcher, which has helped to lead to their own further network development.
53
Case Study 8 (Role): Lesley Wye – this case study explores Lesley Wye’s role, activities and impact as KM team lead and NIHR KMRF.
1. Activities – these have been grouped according to 5 key themes: A. KM team management &
support, B. Increasing KM capacity and awareness, C. Skills & Career Development, D. Researcher Commissioner collaboration, E. Dissemination of research
A. KM team management & support
Managed Cohort 1 Management Fellows from September 2013 (pre KMRF) and has ongoing responsibility for managing the whole team
Provides glue, leadership, strategy and support
Embeds MFs into the academic environment – an ‘essential’ role
Facilitates fortnightly KM team meetings
Set up and facilitates the Qualitative skills group and literature & wine evenings
Develops, champions and nurtures KM team knowledge and skills
Assigns secondees to research projects, supports and co-ordinates their involvement
Develops RR and MF job specifications, objective setting and work programmes and plays a pivotal role in their recruitment and induction.
Facilitates MF’s and RR’s to progress their ideas and broker connections
Shares her vision and passion within & beyond the team
Active in expanding, evaluating and diversifying the KM team
Values individual team members and their skills, makes them feel ‘special’ and part of a team
B. Increasing KM capacity and awareness
Applied for and awarded NIHR KMRF, supports other aspirant KMRF’s, active founder member of national KMRFs ‘Action Learning Set’
Wide and varied presentations at local and national conferences and symposiums (about KM theory and practice and the commissioning world)
First speaker at launch of CLAHRC ‘Implementation’ seminar series
Attends wider national KM forums & events e.g. Fellows Connect event
Publishes peer reviewed academic papers on KM (3 in 2015)
Collaborates with other KM initiatives and experts e.g. Martin Marshall, London CLAHRC, Imperial College, John Gabbay Southampton University
Leads on provision of ad hoc KM advice service for UoB researchers
Provides advocacy and PR for KM
Co-applicant GW4 KM event across 4 Universities (Bath, Bristol, Exeter, Cardiff)
Invited participant Health Foundation discussion ‘National Strategy for KM’
Supported MF’s in completing a KM literature review
Contributes to development of KM strategies for large scale funded research e.g. PROVIDE, Primary Care Factors
54
C. Skills & career development
Self: Successful KMRF application (APCRC support equivalent to 20% of Lesley’s time for six months to write Fellowship application, actively engaging with the KM agenda through this support). Promoted to Senior Research Fellow, appointed Deputy Director of Impact UoB School of Social and Community Medicine & member of CAPC senior management team.
KM team: supports learning and skills development, e.g. qualitative skills group, KM reading group, advises on research and evaluation design and methods.
Other: provides ad hoc dissemination support to CAPC researchers (plans to set up a ‘Dissemination Advisory Service’ on hold). Collaborates on development and delivery of short courses for researchers and NHS staff including: 1) Towards Impact, 2) Service evaluation, 3) Doctors as leaders and managers (including placements within commissioning). Actively engages with, initiates and supervises postgraduate student projects (PhD and MSc). Identifies and supports future ‘impact leaders’ within CAPC
D. Researcher commissioner collaboration
Relational activities
Brokers relationships between academics and NHS e.g. commissioners input into the Psychological therapies HIT
Explored possibility of developing a questionnaire to measure researcher-commissioner collaboration
Set up and co-ordinates a central mailing list of key stakeholders (with Nadya Anscombe)
Acts as a ‘gateway’ between research and commissioning
Research activities
Leads or contributes to multiple large and small scale research projects e.g. Knowledge Exchange in commissioning (Lead), QuICN (quality indicators for community services), 3D study
Leads or collaborates in developing new areas of research (inc. ethics and funding applications)
Secures additional resource for small scale research projects (finance or research support)
Uses extensive NHS & HEI networks to enlist support and facilitate researcher-commissioner research
Increases research capacity, funding and skills and raises the profile of KM within research
Evaluation activities
Instigated and co-leads large collaborative evaluations as part of her KMRF e.g. Telehealth
Develops CCG/KM/CAPC team service evaluation capacity and expertise
Academic lead development of ‘End of life care co-ordination centre evaluation’ (now discontinued)
Active role in development of service evaluation models e.g. realistic evaluation & regression analysis
E. Dissemination of research
55
In addition to the above activities a key work stream for the KM team and LW involves translation of research into commissioner accessible messages and use of social media and more creative dissemination techniques.
2. What helps sustain KM team leader role/activities
acting ‘as a lynchpin…she holds the team’
engendering a sense that it’s OK to experiment and helps secondees realise their ideas
having passion, energy and enthusiasm
ability to communicate her vision and ideas to multiple audiences
having large, well developed diverse networks and respect within the NHS and academia
believing in the potential and need for research with impact
seizing opportunities, generating lots of ideas and acting on them
creating an informal supportive learning environment
understanding the commissioning world
having extensive and prolonged experience in change management and working at the interface between NHS/HEI
support of senior people in both camps who are positive and innovative, respect each other and are ready to take the long view
close friendship and support of the KM team (past and present)
organisational support, funding openness, understanding and buy in
3. What doesn’t help sustain KM team leader activities/role
Risk of overload/ over dependence and reliance on her goodwill ‘Her expertise helps to co-ordinate things and they go off the boil in her absence’
It requires a great deal of time and patience to embed and make KM team members comfortable and useful in their new roles
Multiple taskmasters, conflicting pressures, expectations and demands
Requirement for measurable outcomes and return on investment ‘KM team absorbing considerable resource’
4. Key take away messages
LW has a pivotal role in the success of the KM team and in increasing local KM awareness and researcher-commissioner collaborations
She has been ‘a good catalyst in terms of talking about how this (research impact) can happen more clearly’ she is ‘quite key’
Lesley’s managerial supportive role has been integral to the setting up and supporting of the KM team. However this time is funded by her KM fellowship which ends in 2017. This is an important issue to address for the long term sustainability of NHS Management Fellow and Researcher in Residence posts.
5. Types of knowledge shared: relational, organisational, research methods, evaluation methods, dissemination methods, experiential knowledge, KM theory and practice, research evidence, funding sources & opportunities
56
6. Context Mechanism Outcome
Having clear leadership, expertise, passion and drive are essential to the development, support and impact of local and wider KM initiatives.
Individuals who assume such roles have exceptional communicative, creative and relational skills but there is a risk of overload without sufficient organisational support and resources.
57
Case Study 9 (Activity): KM awareness raising
1. Brief description. All KM team members were/are involved in this activity the aim of which is central to their role. - in this case study we review activities undertaken by the KM team which were explicitly or implicitly directed towards raising KM awareness
Aim. To raise awareness of the theory and practice of Knowledge Mobilisation within host organisations, and wider health & social care environment and academia. To actively increase KM capacity and effectiveness through diverse means.
2. What happened (process, what did KM team contribute and how) KM team members secondment – to span organisations and use diverse methods to improve
mutual collaboration and understanding
Presentations, seminars, conferences, elevator pitches, workshops e.g. NHS change day, SWSAPC Knowledge mobilisation workshop, CAPC seminars, HSRN symposium, PROVIDE conference (see Case study 4), Fellows connect event, Royal College of GP’s conference
Informal awareness raising via personal means: spending ‘a lot of time explaining what KM is’ (Interview 14)
Linked key people from opposing camps, including wider social & healthcare, to share ideas about effective knowledge mobilisation
Raising the visibility of KM at key meetings (within team and beyond) e.g. LTC steering group, KM team Meetings, ITHAcA HIT
KMT knowledge and skills development: completed a literature review entitled ‘What are effective ways to put research into practice?’, “I have referred back to this throughout my secondment” (Doc 11, Feb 2015) & KM reading group (where new articles are discussed and critiqued)
UoB/CCG short course development: e.g. Towards Impact
Peer review and professional journal publications e.g. Evidence based policy making and the ‘art’ of commissioning – how English healthcare commissioners access and use information and academic research in ‘real life’ decision-making: an empirical qualitative study online at http://www.bristol.ac.uk/primaryhealthcare/km
KMT website development (including animation) online at: http://www.bristol.ac.uk/primaryhealthcare/km
Proposed that ‘research and evidence based commissioning’ should be a required competency for CCG managers
Extended and diversified social media use e.g. Twitter account and KMT website
Completed a stakeholder mapping exercise to better understand the landscape and aid communication with key personnel
Contributed to national think thank events: NIHR Dissemination Centre visit and LW invitation to Health Foundation event ‘National Strategy for KM’
Initiated and designed collaborative projects (see project list Diagram N) and improve understanding of how to coproduce research
Collaborated and/or network with experts in the field
Provided informal and formal advice on how to Improve dissemination by using creative methods and targeting diverse audiences e.g. EPSOS workshop on prostate cancer, Dissemination strategy for TARGET trial and PROVIDE
Provided formal and informal support for developing/writing KM elements of small/large scale research projects e.g. KM workshop with PROVIDE advisory group and lead on KM elements of major follow on programme grant RE-PROVIDE
Contributed to discussions about the future of cross-institutional KM
58
Developed transferrable skills and inspired KM careers
Supported other areas in setting up similar KM initiatives e.g. Wessex, Nottingham, Cambridge, Oxford
3. What was the impact or outcomes? (See section 4 for more detailed impacts) Increasing interest and attendance at CAPC seminars & positive feedback from conference
presentations
18 SWSAPC conference attendees signed a pledge to make a change to support the KM agenda in their work e.g. pledges include a research administrator routinely asking researchers how their bid would make a difference to the NHS and a clinical researcher taking on a commissioning role.
More proportional attendance at conferences and seminars (NHS staff, policy makers and academics) e.g. PROVIDE and AHSN conference
Wide and varied dissemination using different means
Overview: ‘My overall impression is that they are making slow but steady progress in raising awareness of what they do and, perhaps more importantly, of the purpose of what they do through building these relationships’. (Interviewee 11) ‘my prediction would be that Lesley’s influence, particularly if there’s been a research Fellow …KM Fellow working with the Team, is that (KM) would be more, it would be visible. Visible you know in the way that … PPI is visible. (Interviewee 21)’
4. What helped?
Energy, enthusiasm and hard work: ‘in the 14 month period that she was there … she gave 10 presentations, she alone …delivered 10 different events and gave 10 different presentations.’
Flexibility and creativity
Collaboration and support within the team ‘having the combination of the researchers and the commissioners with their two different views on things and the dialogue that that creates and the discussion that that generates I think has been really, really positive.’
Lesley Wye’s leadership and vision
Visibility and approachability
Using multiple approaches and opportunities
Institutional readiness and fit: University - REF IMPACT agenda and requirements, CAPC/SSCM already doing applied research, “our neck of the woods, the sort of downstream applied research neck of the woods, it’s certainly the one that’s most amenable to this” and CCG’s needing to optimise use of diminishing resource
Knowledge mobilisation is ‘active’ not passive, practical not theoretical so easier to engage with
Use of innovative means – cross institutional secondments
Increase in capacity: “that’s a powerful way of you know … influencing people, actually giving them a, another member of their Team to do work. Um, I mean, the image of the “gift horse in mouth”
59
5. What didn’t help?
KMT and others overambitious expectations about what can be achieved and how quickly cultural shift can occur
Developing KM teams own KM skills frequently gets squeezed out due to other demands
Lack of clarity about who the KMT are and what their purpose is ‘I don’t think people are really aware of my role, so they don’t know why I do it’
Not so visible in all areas
Lack of connectivity to other local KM initiatives
KM awareness raising is manifest in these activities but hard to measure and fund, currently outcomes on patient care are unknown: ’I guess most of the conference presence has been around the idea of the KM Team rather than demonstrating these specific hard impacts that would have never happened without the KM Team.’
Insufficient capacity within the team for so many projects and ideas
6. Key take away messages
Over the course of 2.5 years (approx. since September 2013) the KMT have developed an enormous portfolio of diverse activities aimed at increasing awareness of KM within HEI and the NHS both locally and nationally.
The sheer energy and scope of their efforts is astounding
Through this process the KMT have developed new skills and expertise and knowledge of what works where and of future challenges. Some audiences are more receptive, future emphasis should continue to focus on both commissioning and HEI audiences.
7. Types of knowledge shared: KM theory and practice, dissemination methods, relational and organisational knowledge, research evidence, research and evaluation design methods
8. Context Mechanism Outcomes
Raising KM awareness is challenging and requires different skills and methods in different settings and for different audiences.
The KMT can maximise opportunities for raising KM awareness through their mix of diverse knowledge and skills
60
5.3 Case studies and activities summary: what works, for whom and in what
circumstances?
Secondees’ prior knowledge, skills and relationships make an important contribution to
their new seconding organisation. Using networks, relationships and knowledge they can
facilitate people to come together in new ways across organisations to:
create new and innovative projects, research and outcomes;
embed knowledge mobilisation into existing work, creating more commissioning
informed research and research informed commissioning;
support commissioners to include research and evaluation within their work;
support researchers to embed knowledge mobilisation activities within their work;
support commissioners and researchers to further develop their own networks,
creating further bridges across organisations.
This enables a process where knowledge can be both co-created and embedded within
organisationally relevant work.
Through secondees, knowledge about research and commissioning can be directly applied
to the host organisations’ projects and everyday work, leading to new perspectives,
activities and decisions.
If secondees can become fully embedded within the host organisation, this optimises the
impact on both the host and secondee. It can lead to fruitful cross-fertilisation of skills and
ideas, with potential for wider KM awareness raising and further collaborative work.
Clear leadership through the KMT leader role supports and facilitates the embedding of
secondees into their host organisations, which is vital to the success of such roles. Lesley’s
managerial supportive role has been integral to the setting up and supporting of the KM
team. However, this time is funded by her KM fellowship which ends in 2017. This is an
important issue to address for the long term sustainability of NHS Management Fellow and
Researcher-in-Residence posts.
Achieving balance between university and commissioning organisations’ engagement and
priorities is difficult to achieve.
Benefits of KM activities
Over the course of 2.5 years (approximately since September 2013) the KMT has
developed an enormous portfolio of diverse activities aimed at increasing awareness
of KM within HEIs and the NHS both locally and nationally.
Complex research studies involving multiple health partners benefit from MFs being
on board to navigate and communicate with many health partners. This leads to
more smooth running research and greater commissioner engagement with the
research, and wider dissemination opportunities.
61
MFs can have wide ranging impact on large-scale university led projects within
established research teams with an applied emphasis. Organisational and relational
knowledge of MFs help research studies navigate and implement research and
embed research knowledge into practice. This also provides the MF with
opportunities to develop their research skills.
MFs and the wider KMT can be pivotal in developing more accessible and creative
dissemination methods for applied research. Their organisational, relational
knowledge and communicative know-how can assist in translating research findings
for diverse non-academic audiences.
Service evaluations can take considerable time and resource and need to be carefully
tailored to meet the needs of both commissioners and researchers. They have to be
sufficiently flexible to take account of changing commissioning contexts. There are
benefits of being able to specifically design appropriate evaluations with skilled
researchers, as opposed to taking evaluation designs ‘off the peg’.
Collaborative research works well when the idea is developed within the
commissioning environment and meets an identified need.
Relational activities can have tangible outcomes. KM team members are ideally
placed to link disparate parties around common interests and goals. Opportunism
and identifying potentially fruitful avenues for collaboration are important foresights
that the KM team use to build their collaborations. Outcomes that can develop from
relational activities are not easy to predict and may happen some time after initial
connections.
The KMT risk substantial overload as they respond to different organisations,
agendas, ideas and collaborative potentials. Sufficient organisational support and
resources are needed for such work.
Raising KM awareness is challenging and requires different skills and methods in
different settings and for different audiences. The KMT can maximise opportunities
for raising KM awareness through their mix of diverse knowledge and skills.
Different KM initiatives (the KMT and GP Clinical Evidence Fellow) can complement
each other’s skills and expertise and assist commissioners in designing and securing
funding for applied research.
62
6: What could be done differently? Developing the future – evaluation
participants ideas
This section puts forward ideas for optimising the KM team members’ skills and impact before,
during and after secondment and makes suggestions for taking KM initiatives forward. Suggestions
within this section were drawn directly from participant interviewee responses to questions
regarding possible improvements to the KM team initiative. The recommendations in section 8 are
derived from our full analysis of all evaluation material and are based on the sum of all our work.
6.1 Optimising the KM team members’ skills and impact before/during/after
secondment.
Managing the placements and their activities
In terms of suggestions for developing MF and RR placements, there seemed to be slightly more
focus on suggestions from a commissioner perspective than a University perspective. This may be a
result of having had two rounds of MFs within the University, with a formative evaluation conducted
of the first Round of MFs (Wye and Baxter 2014), and more time for reflection and development of
their roles. In this KM team evaluation interviewees highlighted the possible areas for development
as follows:
Having a clear rationale for where secondees are placed, possibly linking with CCG and University
strategies
Importance of allocating people so that they are aligned to CCG strategic priorities. There is a
five year Sustainability and Transformation Plan within commissioning. Can the KM team be
used to inform that plan?
‘Actually I think the Sustainability and Transformation Plan is a more strategic opportunity...
To say, “How are you trying to create a sustainable health care system?” That’s, that’s what
it’s about’ (B22)
‘There’s a lot around the longer term strategic CCG vision which potentially could be helped
through this’ (B8)
Further focus on understanding where clinical interventions are not effective, and do not have a
clinical benefit. Evidence needed for improving quality and reducing costs.
63
Too much emphasis of delivering service evaluations?
‘I don't think having researchers come in there and trying to do service evaluations for
commissioners fundamentally is going to work longer term because … we really want to build
up the skills within the healthcare services themselves’ (A16)
Linking MFs within University research projects to develop links with commissioners and further
knowledge mobilisation and dissemination strategies has worked well (see PROVIDE and 3D case
studies). This type of placement fits well with University strategies surrounding impact and
public/community engagement.
Preparing the organisation for the secondee’s role and managing expectations
Supporting the people who work with MFs and RRs in understanding people’s roles:
‘How much I can ask or what I should ask and what I shouldn’t ask’ (B18)
‘It’s better understanding and appreciation of what they do, what they can offer, and
opportunities to make better use of them’ (B20)
‘I’m interested to know what else we could be offered or how else we could work together’
(B9)
Enhancing opportunities for RR’s to seed collaboration within CCGs
CCG steering groups are very operational and there is less time and space for research contributions
within these operational meetings. They are useful for people to get to know the RR but actual
collaboration between RRs and commissioners often occurs elsewhere. It seems to be that HITs have
operated well as an additional arena for collaboration. The lift, the kitchen and meetings over soup
all suggest opportunistic and creative ways to develop relationships, but it also shows that there are
less clear forums for this collaboration in a CCG. Lesley as part of her KMRF role has introduced some
key groups to support MFs in their embedding within a University context, such as the qualitative
skills group and the KM reading group. Further thought needs to be given as to how to enhance the
potential for collaboration in the commissioning context using brokering roles and dedicated spaces.
64
Other activities that may help commissioners embed evidence
Within commissioning, new business case templates have been designed that specifically ask for
evidence to support business proposals. These have been specifically designed to encourage
more evidence-based commissioning. However ‘people haven’t really filled it in very well, or they
weren’t sure where to go’ (B19). It was suggested that KM supporters could help people to fill in
these sections of the business case templates to help embed further evidence-based
commissioning.
Updating the transformation team with published systematic reviews so that they can feed this
into the work of commissioners.
Supporting commissioners in assessing the quality of evidence that they use.
Managing the return to home organisations
Setting a course to come back in from the beginning of the secondment.
Having progress meetings through the placement between APCRC, MF/RR, MF/RR’s line
manager and LW have been hard to organise. These seem to have occurred more often
through the second phase of MFs.
How do the projects that people work with connect to the strategic priorities of the CCG?
Can these be incorporated into MFs work planning, so there is more continuity when they
come back to their original roles?
Further work needs to be done with employing organisations and secondees to develop
roles that can account for and develop MFs and RRs new skills and networks on their return
back full time to their home organisations. There may need to be some work done with
organisations to educate them as to how they can employ returning secondees’ new skills
and value these, developing flexibility and receptiveness to new skills.
Raising the profile of knowledge mobilisation
Communications and ‘branding’ still needs to be attended to – particularly focussing on extending
people’s understanding of what is knowledge mobilisation and how it can be made understandable
to a wider range of people.
‘Make a bigger noise about what you are doing’ (B6)
65
‘I think sometimes the term ‘knowledge mobilisation’ scares people a little bit, if they don’t
understand what it means’ (B13)
‘Raise the profile of this initiative…. not many people know about it. It needs to be made a
more important thing than it is’ (B6)
‘I think quite a few of my colleagues are still saying, “Who’s [RR]? What does she do?”’ (B8)
Importance of quick wins and small gains to promote engagement and win over people
Small scale projects with quick wins may work better to initially engage people within a
commissioning environment that constantly changes.
Developing wider spread
People sometimes commented that the KM team’s work is still channelled through a few individuals
and there was a sense of pressure to expand and diversify the KM team model. However, it was also
stated that the current KM team can be overstretched within their capacity and resources. Within
current resources spreading the KM team’s activity out too far too soon may reduce effectiveness.
Any wider spread of KM team activities will need to take account of the wider range of posts that
APCRC have recently developed such as the Graduate Evaluation and Evidence Assistants. Please see
section 7 for further details.
‘We can’t send them all on attachments to another organisation, so we need to give thought
to how we make sure that the learning that some individuals get does affect everybody else
within the NHS environment’ (B7)
‘There is a question for me about how we do make knowledge mobilisation something that
means something to everybody’ (B13)
6.2 Suggestions for taking KM initiatives forward
Agreeing clear expectations of the KM team
The KM team seems to have developed organically and was originally conceived of as an
‘experiment’. However as it has grown, there has been a perceived need to capture its work and
evidence its impact. In any future development of the KM team, clear agreed expectations and
66
deliverables, that are seen as achievable to different parties need to be discussed and agreed.
However there is a tensions within this between ‘micro’ and ‘macro’ level expectations, measurable
or relational impacts. Not all of the KM team’s activities are easily measurable, and an emphasis on
just measurable activities such as developing collaborative bids, may not acknowledge the more
intangible goals of the KM team, such as creating cultural changes and embedding evidence into
commissioning decisions.
Developing the future of the scheme
Having both RRs and MFs has been particularly important both from the KM team’s and
commissioners’ perspectives
‘For me, it feels like it’s the combination of both that has been most effective’ (B19)
‘I think one of the strengths of it has been that core team, those regular meetings. You
know, I do think that you need to look after each other to make it work’ (A4)
Should there be greater reciprocity between RRs and MFs placement positions?
Should RRs go into the organisation that the MF have come out of? There may be difficulties
with this dependent on the networks and relationships that have to be developed within the
organisation, and the support that the secondee will be able to receive.
Sustainability of the KM team
Planning for the future sustainability of KM activities needs to be considered at this stage. Lesley
Wye’s managerial and supportive role is funded by her KM fellowship rather than through RCF
money, which will be completed in 2017. Her role has been integral to the setting up and supporting
of the KM team. This is an important issue to address for the long term sustainability of NHS
Management Fellow and Researcher in Residence posts.
‘More supervisory capacity in the team’ (B25)
‘PPI or Knowledge Mobilisation … those roles have been handed out perhaps without
resource attached … and I don’t think you can do it as an add-on to other jobs’ (B23)
67
What are the alternative models that can be developed?
One commissioner argued that it was important to have a long term, strategic approach to
this work:
‘If it’s not delivered by five years, I think we can think quite differently. Because we’re
looking initially at one year, and we always do this because it’s funding and it’s about
evidence of doing, and I know the research world is very much more around short periods of
intervention. To really align with the commissioning world and the CCG world that continuity
I think is more important’ (B8)
Learning from patient and public involvement (PPI) movements may be important, whereby
such initiatives have become slowly ingrained within communities. Can anything be learnt
from the process of how PPI became more embedded in research? What models of PPI can be
translated? What can be learnt from its success as a movement? Can you have a KM expert,
like those of PPI, where a person acts as the go-to person for advice and knowledge? How
would this position then be funded?
Connecting with CSUs and NHS England. How can RR positions connect with NHS England? Can
Commissioning Support Units be connected with further, with possibly a RR being placed
within an organisation? Would a service evaluation capacity be well suited to being situated
within CSUs? ‘CSUs need to develop qualitative and quantitative skills’ (Interviewee 16)
More permanent roles that bridge the two organisations, more formal structures were
suggested. This would be one way of continuing the leadership post that LW is currently doing,
which is funded by her NIHR KMRF. The post-holder would need to have commissioning
networks, know KM literature, and be able to develop strategy to develop KM between
researchers and commissioners.
Structure of the KM team
Enable people who enjoy this role, to be able to professionally develop further in it. There is an
important question about how, if future MF and RR roles are continued, these are structured
and staffed.
‘I think it’s a real trade-off between allowing lots of different people to have a go, versus
having fewer people for longer time. So do we turn them over once every twelve months or
do we allow people a little bit more time to get properly embedded? Because I’ve got a
feeling they get much more effective after a time’ (B25).
68
Current post holders have developed strong skills and networks, yet these have taken time
to embed. There was a sense from some people that having an option of longer continuity in
posts may be of benefit, as opposed to replacing everyone for a short period of time. One
option could be to allow current secondees to reapply for an extension of their role, if they
would so wish, and the seconding organisation allows this to happen.
Developing an advisory group was suggested by one interviewee as a way to develop the
team to have more people to be involved in the strategy and in the running of knowledge
mobilisation activities.
‘First of all to guide us with strategy, to give us the back-up and governance to press ahead
with certain things, and secondly to help, with our own contacts, disseminate the work of the
KM Team and promote it. So I’m keen to set up that. I would be really keen to have a senior
person – I guess I have in my mind a career structure’ (B11).
Continuing funding
There was a strong call from both commissioners and researchers to continue this work. Some
example quotes include:
‘Please don’t stop. Whatever you do, do not stop. This has been one of the most important
things that happened in my life in the last year’ (Commissioner)
‘I’ve seen first-hand how valuable it can be, so it would be nice if it was continued to be
funded’ (Researcher)
‘With every one of our projects it would be useful to have input if we had the time and the
manpower to have, and not just from their team but from our own commissioning team,
then I’m sure it would better inform all of our commissioning decisions. I don’t really see a
downside to it so, yes, we’ll slurp up whatever is given us’ (Commissioner)
‘I think what has been done so far is superb and it’s been limited by people’s time and
capacity’ (Commissioner)
69
7: Linking with wider KM initiatives There are a series of different local knowledge mobilisation initiatives and other organisations that work in the area of knowledge mobilisation. This section
overviews some of these wider KM activities (Table 7.1) and how the KM team work and fit with them. Diagram 7.1 maps the following wider Bristol Area
KM initiatives showing key collaborative partnerships, major stakeholders and dedicated KM roles (by home organisation) which contribute to a growing
KM Hub. Suggestions for how to further develop links between KM initiatives within this section were drawn directly from participant interviewees. The
recommendations in section 8 are derived from our full analysis of all evaluation material and are based on the sum of all our work.
Table 7.1 Other KM initiatives and joint activities with the KM team in the locality
ORGANISATION THEIR ROLE IN KM JOINT KM ACTIVITIES WITH KM TEAM
APCRC
APCRC fund the NHS MFs and RRs.
Head of Evidence & Effectiveness leads the Evidence Informed Commissioning work at APCRC.
Head of Evaluation leads on the strategic development and delivery of the service evaluation
3 Graduate Evaluation Assistants and 3 Graduate Evidence Assistants have recently been recruited, one of each being placed within each of Bristol, North Somerset and South Gloucestershire (BNSSG) CCGs (18 month fixed term posts).
One Health and Social Care Economist provides economic support to South Gloucestershire clinical commissioning group (CCG), hosted by the public health intelligence team in South Gloucestershire Council.
Specific examples of where the KM team have worked with APCRC include:
the service evaluation short course
an APCRC evaluation toolkit, contributing to ensuring its accessibility to commissioners
support in designing the diabetes evaluation
sharing service evaluation skills
support in how frailty services are currently being designed and evaluated, encouraging the different initiatives to make use of the lessons learned elsewhere
working to promote service evaluation within commissioning The graduate assistant posts have only recently been appointed.
West of England Academic Health Science Network (WE
11 GP Clinical Evidence Fellows (CEF) have been seconded into 7 CCGs in the West of England, funded by the WE AHSN.
A RR has connected with the N Somerset and South Glos GP CEFs, one of these meetings led to some work together.
70
AHSN) GP Clinical Evidence Fellows
Provide support for one day a week within each of the 7 CCGs.
Part-time GPs whose role is to promote the use of evidence and evaluation and conduct some evidence appraisals, to support the growth of an evidence culture.
Here an urgent care lead, a SG GP CEF and a RR put in a challenge to the Elizabeth Blackwell Institute to understand more about how complex geriatric assessments could be used in primary care.
The RR has taken this project up as a researcher, both initiating and leading this project (see project list table).
Health Integration Teams, Bristol Health Partners
Health Integration Teams (HITs) bring together researchers, clinicians, commissioners, providers and patients to tackle health priorities and to help improve services.
HITs work under Bristol Health Partners, a strategic collaboration between nine partner organisations, including NHS trusts, 3 clinical commissioning groups (NS, SG and Bristol), Bristol's universities (UoB and UWE) and Bristol City Council.
One of the first NHS MFs was involved with the ITHAcA HIT. This work supported ITHACA to maximise its collaborations with commissioners.
Current NHS Management Fellows do not seem to be involved in HITs in their seconded roles within the University.
A RR is co-ordinator of the ITHAcA HIT focussing on Avoiding Hospital Admissions. She links ITHAcA and the urgent care steering group.
Commissioner involvement within the ITHAcA HIT has been a strong focus and it was commented by a commissioner in an interview that now there is ‘actually more clarity in terms of a joined up approach and an involvement from commissioners’. This is something that one of the first NHS MFs was working to achieve within ITHAcA
Within the ITHAcA HIT a RR leads: - one current project on delays to discharge from secondary into
primary care for elderly patients: clinical decision-making and risk management.
- a project developing a 'rule' for managing abnormal test results in out-of-hours primary care (see project list table for further details).
A RR has also worked with the co-directors, of the IMPROVE HIT, focussing on perinatal mental health, writing a business case for a perinatal mental health team as part of the IMPROVE HIT. They also organised a workshop for this HIT on setting up the work streams.
There is currently no LTC HIT, this potentially constrains the ability of the RR working with the LTC steering group or others to develop collaborations in this field. HITs are a useful extra body that explore where researchers, clinicians and commissioners can collaborate within research. Without them it is less clear and easy for RRs to track where
71
there is ripe potential for commissioners and researchers to come together and network with different people. HITs provide important extra spaces for RRs to develop collaborative working.
Public Health Christina Maslen works within the Public Health Evidence Service and works with Bristol CCG to provide evidence reviews to inform commissioning decisions
Christina worked with the KMT on the CLAHRC service evaluation course, she carried out the literature review for the telehealth evaluation and RRs have linked with Christina around her evidence reviews for commissioners.
CLAHRC West The Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West works with the NHS, local authorities and universities, to conduct applied health research and implement research evidence. They also have a focus on conducting service evaluations and are interested in evidence utilisation.
KM team members have worked with the CLAHRC in a variety of different ways, including:
Jointly running a service evaluation course, in partnership with APCRC and CLAHRC West.
Submitting bids to them, with some success, such as the ‘Delays to discharge & clinical decision making’ project. Another application has been recently submitted to work on the Joint Front door project (see table of projects).
UWE KM Strategy team: Meetings are taking place to develop a KM strategy within UWE. The KM strategy team and senior management are actively contributing to local discussions with key stakeholders including APCRC, BCCG, and UoB to develop KM capacity across Bristol.
NIHR KMRF applications: Two researchers within UWE Health & Life Sciences faculty submitted NIHR KMRF Round 4 (2015) applications entitled: 1) ‘The mobilisation of musculoskeletal knowledge in the NHS’ (Jen Pearson) and 2) ‘Using rich evidence and forum theatre to enhance practitioner mindlines: a pilot study of a new knowledge mobilisation method applied to post-injury psychological care’ (Kate Beckett). The latter was successful and KB will start her KMRF in June 2016.
NIHR KMRF applications LW provided informal support and advice on NIHR KMRF application to both applicants. She is identified as an expert advisor on the successful bid. A number of GP Clinical Evidence Fellows contributed to the design of KB’s successful KMRF bid by providing insight into the topic from a primary care perspective. Introductions were made by Peter Brindle APCRC LW has introduced KB to the national KMRF Action Learning Set which meets regularly to support each other and KM
Commissioning project Members of the KMT have supported Jen Pearson’s work looking at commissioner decision making by helping her to contact relevant people.
72
This project is supported by Lesley Wye (KMT), Peter Brindle (APCRC), KM experts (Andree Le May and John Gabbay, Jonathan Benger (UWE, UHBristol) and UHBristol Foundation Trust trauma practitioners. It will involve collaborative working across primary and secondary care.
Commissioning project Jen Pearson at UWE has been employed on a study entitled ‘Exploring how musculoskeletal evidence is embedded in the NHS: A commissioners’ perspective’. The aim was explore how musculoskeletal evidence is embedded in context of the NHS within the boundary of the AHSN and CLAHRC West.
Health Improvement Research Fellow appointments Jen Pearson and Charlene Ronquillo have recently been appointed to work at the interface between Bristol CCG and UWE to develop collaborative research bids (posts funded by APCRC).
Other Jane Powell provides academic support to the APCRC funded Health and Social Care Economist at SGCCG
73
KM HUB
WEAHSN
NIHR CLAHRC
WEST
BHPHITS
NBT
PRIMARY
CARE
UWE
NHS SERVICE
PROVIDERS
HEALTH
IMPROVE
MENT RF.
KM STRATEGY
TEAM
NIHR
KMRF
UHBT
MF (2)
MF
(1)
EVIDENCE
ASSISTANTS (3)
UoB
PUBLIC HEALTH
BRISTOL
CC
UNIVERSITIES
Note: APCRC funds RR, MF, Evidence and Evaluation
assistants & Health Improvement Research Fellows.
WE AHSN funds the GP Clinical Evidence Fellows.
Diagram 7.1 Map of wider Bristol area KM
initiatives. Showing key collaborative
partnerships, major stakeholders and dedicated
KM roles (by home organisation) which
contribute to a growing KM Hub (in green).
ECONOMIST
EVALUATION ASSISTANTS
(3)
NIHR
KMRF
South Glos CC
PUBLIC HEALTH
APCRC
VOLUNTARY
SECTOR
OTHER
RR (2)
GP CLINICAL EVIDENCE
FELLOWS (11)
SW. Clinical
Support
Unit
CLINICAL COMMISSIONING
GROUPS
BCCG, SGCCG,
NCCG
74
Networking with national knowledge mobilisation initiatives The KM team has developed a variety of different national links with different KM programmes, such
as:
Health Foundation Fellows Connect – the KM team’s approach to building collaborations
across organisations is different from many of the Fellows who attend this event, who tend
to work with clinicians on improvement projects within provider organisations.
NHS MFs have met with two fellows from Oxford, one from the CQC and one from NHS
England.
Presenting at a variety of different local, regional and national conferences, including the
HSRN, AHSN, South West and national Society for Academic Primary Care conferences and
organising a variety of local events.
The School of Social and Community Medicine hosted a seminar by Martin Marshall,
Professor of Healthcare Improvement at UCL and lead for Improvement Science London.
This resulted in an invitation from Martin for the KM team to co-present at a workshop on
researchers in residence for the Health Services Research Symposium in July 2015.
One RR sat on a panel and gave a presentation about researchers in residence organised by
Martin Marshall at the Health Service Research Network conference in June 2015.
One RR gave a workshop at the annual Royal College of GPs conference in Glasgow October
2015, delivering this workshop as part of a team of people including Martin Marshall
working across the UK on different researcher-in-residence models.
The KM team has also developed links with people such as Kieron Walshe from University of
Manchester, Bill Lucas from The University of Winchester and the Health Foundation and Liz
Maddox Brown from NHS Improving Quality.
The KM team leader sat on a panel for a one-day evaluation symposium organised by the
Nuffield Trust, where she talked about the KM model. Nuffield are interested in developing
the evaluation capacity of healthcare analysts in CSUs and elsewhere and want further
collaboration.
The KM team leader launched Pen CLAHRC ‘Implementation’ seminar series.
In addition to this formal work, the KM team have been contacted nationally and met with a variety
of different people interested in their models of working.
Developing KM initiatives: spread and working together
As can be seen from Table 7.1 above there are a series of KM initiatives within the local area. Issues
arising from this included:
75
‘If you spread too much you just dilute … you need to really build something really strong and
then work out from that’ (A16).
In introducing KM staff into commissioning organisations there needed to be a steady
approach - laying foundations, making sure communication is clear, managing expectations,
anticipating workload and the support for people, both people going into organisations and
the organisations themselves. Organisations need to have clear guidance and support on
how to use people and what they can offer and contribute to.
Planning how to engage secondees into the organisation, considering different strategies to
engage people within the main parts of the organisation.
‘For us as commissioners, it’s sometimes a bit of a confusing picture. Like who’s doing what?
And what’s that? … It’s just a matter of coordinating it all. So it’s another bit of that jigsaw’
(B19).
The potential of creating ‘a bit of overload’ (Interviewee 20).
‘I think if we could find a way of the KM Team being more inclusive and bringing,
encouraging more people to join their meetings, and perhaps changing the venue of the
meetings. So basically, if they can start to really incorporate interested people from the
University of the West of England, that would be good, so that they are a wider team, rather
than the University of Bristol huddle’ (B11).
From a university perspective, it was also considered that knowledge mobilisation was part of a
much bigger academic agenda where ‘there’s lots of activity that’s all overlapping’ around impact
and knowledge translation and public engagement.
‘They’re all the same agenda, it seems to me, which is not having academic research done in
a vacuum but trying to make a difference to the real world’ (B7).
It was seen that the added value and contribution of the KM team within the wider knowledge
mobilisation initiatives was that:
For researchers there was a clear benefit of having NHS MFs within the University: ‘there
was a geographical closeness… I guess it’s more personal’ (B5).
‘KM team individuals feel as though they’re actually quite involved in fingers in multiple pies
with multiple organisations with multiple people. And that just feels from my side to be a
difference’ (B8).
The KM team were ‘the only ones that have actually bridge across to academia’ (A16).
76
The independence of RRs within commissioning steering groups was highly valued, having an
external person to be able to bounce ideas off and provide a critical and external
perspective, to help people think in a different way.
The value of having NHS MFs being able to support and navigate researchers through the
NHS and commissioning world, and having a wider perspective on how to ensure that
research projects can be more embedded into NHS and commissioning worlds.
How to link KM initiatives together
There was a sense from KM team members of the importance of linking with the different KM
initiatives, both locally and nationally. However within limited resources, there was also a sense of
making sure that those links supported and enabled their work. Different ways to link these wider
knowledge mobilisation initiatives were discussed:
An event: Several people spoke about the development of an event to link everyone together and
share their work and perspectives. However there is also a broader question of how these different
people in their diverse organisations can link and network further over time, rather than just at a
one-off event.
A network: In addition to the APCRC funded posts, the collaborative work of the Academic Health
Science Network and the CLAHRC was also seen as important to link in with, to develop a network of
people, interested in and practising knowledge mobilisation.
‘It may be that some strategic appointments of Researchers in Residence or Management
Fellows could actually really achieve a significant jump in the links and the flow of
information between groups or teams’ (B25).
Creating a culture of this approach rather than a new institutional structure: Embedding this way of
working into the culture of organisations.
New structures: Having longer term substantial posts, such as lecturers, senior lecturers and
professors:
77
‘Using the work to grow good, coproduced grant applications while also exploring,
developing the whole theory of knowledge mobilisation in general… develop the discipline of
knowledge mobilisation in itself’ (B11).
For example, the People in Health West of England is a joint strategy group that co-ordinates
integrated regional public involvement in health, across various regional organisations and provides
a potential model for developing KM understanding and resource.
APCRC is asking the question of how can the KM activities of the University of Bristol, UWE, Bristol
Health Partners and the HITs, APCRCs KM initiatives, the CCGs and providers be best co-ordinated.
Would new institutional structures support or create greater confusion, adding another level to an
already busy picture. One interviewee asked:
‘Should Bristol Health Partners or some other sort of umbrella group actually be thinking
about a city wide initiative here, should we be expanding it out beyond what’s mainly
Primary Care and CCG’ (B25).
Whatever the way forward, a clear goal must be to learn from and build on the range of existing
work, creating further mutually beneficial collaborations.
8: Recommendations for taking the KM initiative forward
In this formative evaluation we have gathered evidence from different sources (both internal and
external, documentary, verbal and observational) regarding the KM team activities, their facilitators
and barriers, outcomes and impacts. We have also developed in-depth case studies to illustrate
these findings in relation to particular projects, roles and activities. In Section 6 we reviewed what
evaluation participants said about the KMT’s future direction, barriers and facilitators and in Section
7 we situated the KMT within the wider KM context. Diagrams 8.1 and 8.2 on the following pages
summarise our recommendations based on all of this work.
78
1)
SETT
ING
OR
CO
NTE
XT The KM team should be recommissioned and supported
to continue their work
Resources to ensure future sustainability, in particular the team leader role are required
There are benefits for both UoB and CCGs in supporting the KMT, future equity in resourcing and secondments will assist in sustaining mutual benefit
KM is a rapidly developing and important field, this investment is both timely and apt, it places Bristol in a strong position. However impact is hard to evaluate and takes time, freedom to experiment is essential
Planning for pre, during and post secondment roles needs more emphasis to ensure long-term returns on this investment
Rapid diversification and expansion of KM initiatives should be approached with caution, organisational readiness is essential
Alternative or additional host organisations should be considered eg CSU, if there is organisational readiness
2)
TASK
OR
PR
OJE
CT Project, setting and KM secondee fit should be carefully considered before placement
MFs can have enormous impact on large scale university-led projects within established research teams with applied emphasis
The potential KMT impact on large service evaluations is mixed and subject to many obstacles. Assessments should be made to determine if such work is an effective use of KMT resource
RRs can be extremely effective in seeding new small scale collaborative evaluations or research projects within CCGs
The KMT are extremely effective in relational and KM awareness raising activities, these should not be eclisped by pressure to secure research grants and work on specific projects
Co-location & embededness are essential with support to navigate organisations
KMT's pivotal role in improving dissemination and accesibility of research should be sustained
Secondees need time and support to cross the bridge to their host and back again
Diagram 8.1 Recommendations for taking the KM initiative forward 1) Setting and context, 2) task or project
79
3)
KM
TEA
M The KM team requires effective strong leadership to provide support and help embed secondees into organisations
Mutual support and team membership are essential and should be supported
Reciprocity and collaboration between MFs and RRs is key and should be maintained
The need for a separate supportive KMT identity should be balanced by efforts to reach out to other local KM initiatives
The emphasis on experimentation needs to be balanced by recognition of the need to demonstrate impact
KMT secondees should be encouraged to develop and present case-studies to illustrate their work
KMT should prioritise realistic goals and potential impact to avoid excessive demand
Some audiences are more receptive, future emphasis should continue to focus on both commissioning and HEI audiences
Discussions about optimising linkage between local KM initiatives and impact e.g. an event, network or new career/ organisational structures should continue
Greater linking to CCG and/or University strategic priorities may be of benefit
4)
IND
IVID
UA
L KMT members should be central in planning their pre, during and post secondment personal development, activities and roles
The timimg and setting of project placements should be carefully matched to both use and develop individual skills
Culture change takes time, resource and tenacity, and requires structural and senior management support
The exceptional qualities required for these KM roles should be recognised and supported
RRs and MFs should have the opportunity to re-apply for their roles to allow continuity, further skills development and networking
Post secondment MFs and RRs should have dedicated time to mentor new secondees to maintain organisational links and transfer knowledge
Longer term more permanent bridge spanning roles should be considered for those who wish to develop a KM career
Rigid career or organisational structures may not be the best means to sustain individual's creativity and flexibility and ultimately support KM
Diagram 8.2 Recommendations for taking the KM initiative forward 3) KM team, 4) individual
80
Appendix
Methods (extracted from proposal accepted by Ethics Committee)
The evaluation team
Data collection, analysis and write up were carried out by Michelle Farr, who conducted the previous
evaluation of Lesley Wye’s fellowship (Farr 2015) and Kate Beckett, a UWE knowledge mobilisation
researcher. The team was overseen by Professor Andrée le May, an international expert in
knowledge mobilisation, the team approach ensured greater rigour and independence from the KM
team, while also providing sufficient resources to produce timely output.
Theoretical underpinning
This evaluation was informed by realist evaluation methods (Pawson and Tilley, 1997). Realist
evaluation is a form of theory-based evaluation, which is often used within the analysis of complex
social interventions. Realist evaluation aims to understand why a programme works, for whom it
works and the sets of circumstances within which it works. This helps in understanding how
particular actions affect researcher-commissioner collaborations, and any contextual drivers and
constraining factors. We also explored how and why particular connections and relationships were
developed and the impact of these in relation to encouraging commissioning-informed research and
research-informed commissioning.
Data collection and analysis This evaluation employed multiple data sources and methods including documentary analysis, semi-
structured interviews and non-participant observations. This provided rich and diverse data and
permitted triangulation of data from different sources.
Documentary evidence
We identified and analysed the following 23 key documents relating to KM team activity:
Quarterly KM team progress reports (10)
Reflective essays of KM team members at the end of their fellowships (2)
KM team strategy documents (11)
These documents provided rich written evidence about KM team objectives and activities but were
mainly authored by members of the KM team. There was also considerable repetition and similarity
81
between themes, it was therefore not deemed necessary to analyse further documents from our
secondary list:
1. KM team member activity logs
2. KM team meeting minutes
3. Meeting minutes for other groups e.g. Long Term conditions steering committee
Interviews
We conducted 25 total semi-structured interviews among the following 3 groups.
Group A: Past and current KM team members including LW as KM team manager
Group B: Commissioning, clinical and academic colleagues who:
o Are members of the groups/ teams with whom KM team members are fostering
collaborations
o Have had an introduction to a commissioner/ researcher/ clinician via a KM team
member as part of their linkage and exchange activities
o Work with the KM team member at their home organisation
Group C: Senior individuals from stake-holding organisations including CAPC, the CCGs, the CSU
and the funders APCRC.
Sample selection
Group A: All KM team members were interviewed.
Group B: KM team members were asked to nominate 3-4 people who fulfilled the above criteria
to invite to interview. Participants were selected purposively for maximum variation, ensuring a
balance of commissioners and researchers, senior and less senior staff were included. We aimed
to interview 2 out of the 3-4 people nominated by each KMT member, which was achieved in all
but one interview. Where group B people had been interviewed as part of the first KMRF
evaluation (Farr, 2015) we prioritised new interviewees for maximum variation.
Group C: Senior individuals from the stake-holding organisations who were involved in KM team
activities were identified by the KM team. Participants were again selected for maximum
variation and balance between organisations.
82
There was considerable overlap between Group B & C nominees and between individuals nominated
by different KMT members. Potential participants were invited into the study verbally or by e-mail and
sent information sheets. Engagement with this evaluation was extremely good and despite the
relatively short timescale we were able to recruit sufficient participants. The following table shows
numbers of potential/nominated participants per group, numbers approached and numbers
interviewed.
Potential participants Number approached Number interviewed
Group A 8 8 8
Group B 15 14 13
Group C 6 6 4
Total 29 28 25
MF and KB conducted all 25 semi-structured interviews using topic guides. These were designed and
ethically approved to cover areas such as KM team member activities and roles, helpful and
unhelpful factors in developing collaborations, impact and career opportunities. Interviews were
conducted face-to-face or by telephone, depending on the preference of the participant. Written or
recorded consent was taken by the evaluator before the interview began, although in the case of
telephone interviews recorded verbal consent was taken and followed up by written consent.
Interviews lasted between 20-45 minutes, were digitally recorded and transcribed by an external
transcriber.
Observations
The evaluation team planned to attend the following 6 meetings attended by KMT members to
observe collaborations in real time, as they happen.
Long Term Conditions Steering Group
Urgent care Steering Group
3D research team
Telehealth evaluation group
Diabetes evaluation group.
KM team meetings
KM reading group
However a number of factors prevented 6/7 observations taking place, these include
83
1. Meetings planned but not occurring within the evaluation period (2)
2. No further meetings planned (2)
3. The evaluation team decided not to attend KM team internal meetings as they had already
been observed in the previous evaluation of Lesley Wye’s fellowship (Farr 2015) and there
were considerable time constraints (2)
The 3D meeting was observed using the following method:
At least a week before observation, KB requested permission to observe the group via the chair or
lead facilitator. He was asked to e-mail group participants with a request to observe and an
information sheet describing the study. The chair or facilitator reaffirmed consent on the day of the
meeting with participants before the evaluator entered the room. Before the meeting begins, the
evaluator explained the study and mentioned that participants’ can ask him/ her to leave at any
time. KB took verbal consent from the group as a whole and written consent from the chair or lead
facilitator. This group consent process is preferable to individual consent, as many of these groups
include 15-20 people and obtaining individual consent was considered too time-consuming for
participants.
The observations of the above meeting sought to understand the following:
The purpose and aim of the meeting
Major issues arising in the development of collaborations between researchers and
commissioners
Contextual issues mentioned. Enablers/ constrainers.
Content of the meeting
The role and contribution of the KM team member(s)
Who is doing the talking/ contributing?
Who is leading/ facilitating?
Outcomes of the meeting
Impact of the meeting on furthering/ hindering collaborations
The evaluator took contemporaneous notes during the meeting, these were typed up as soon as
possible after the meeting and fed into the analysis.
84
While we were unable to complete all the planned observations we were able to draw on secondary
data regarding KMT member’s participation from documentary evidence, interviews and prior
observations completed in Farr (2015).
Analysis
Interviews and observations were analysed thematically, using a coding framework derived
inductively from the data. Each evaluator applied the coding framework to the data that she
collected. To increase rigour, a subset of data (10%) was coded by both. Documentary data was
drawn on to supplement and challenge emerging findings. Observation, interview and documentary
data was also analysed to map out existing and developing relationships and networks across the
organisations. Data collection and analysis was iterative, with teleconferences and meetings
between the evaluators and Professor le May at key junctures to ensure that emerging findings were
critically examined. Andrée le May led a facilitated data management workshop to discuss any
disagreements within the data and the synthesis of the analysis.
Strategy documents
Significant differences in the purpose and content of strategy documents as opposed to quarterly
team progress reports and reflective essays were noted; the former alluding to aspirations and
objectives whereas the latter contained information about actions. As a result the 11 strategy
documents were subject to separate content analysis by KB, her preliminary results were discussed
and refined by the team.
Case Studies
Preliminary analysis and discussions identified a need for case studies to exemplify some of the
activities the KMT engaged in and their impact. All potential case studies were listed and discussed
and 9 selected to represent broad themes and issues. Case study data was drawn from the analysis
and written up by KB or MF. Preliminary drafts were then circulated among the team and subsequently
sent to key informants within the KMT for review and verification prior to inclusion in the report.
KMT involved projects list
KMT involved projects were frequently mentioned in all data sources, a comprehensive list was
formulated by KB and MF separately then combined and reviewed. Missing data and verification was
sought from members of the KMT.
85
References
Eccles M, Armstrong D, Baker R et al (2009) An implementation research agenda. Implementation Science. 4:18.
Davies HTO, Powell AE, Nutley SM. (2015) Mobilising knowledge to improve UK health care: learning
from other countries and other sectors – a multimethod mapping study. Health Serv Deliv Res. 3:27.
Health Information Research Unit. What is KT? Available at: http://whatiskt.wikispaces.com/.
Pawson R and Tilley N. (1997) Realistic Evaluation, London: Sage.
UKCRC (2015) UK Health Research Analysis. Medical Research Council. Additional data from: Farr M. (July 2015) Independent review of Lesley Wye’s work on the NIHR Knowledge Mobilisation Fellowship.
Acknowledgements
We would like to thank all the participants in this evaluation for sharing their time and thoughts and
the KM team for all their support and for allowing us the privilege of focussing on their work.
We would also like to thank the University of Bristol, University of the West of England, Avon
Primary Care Research Collaborative and associated commissioning organisations, and Lesley Wye
for supporting this work.