1

HS&DR Protocol

Project ref 10/1009/09

Version 4

Date March 3, 2013

Project Title

Innovations in major system reconfiguration in England: a study of the effectiveness,

acceptability and processes of implementation of different models of stroke care.

Chief investigator Professor Naomi Fulop

Sponsor David Wilson (on behalf of University College London)

Funder National Institute for Health Research Health Services and Delivery (NIHR HS&DR) Programme

NIHR Portfolio number UKCRN 11917

REC approval September 19, 2011 (ref 11/LO/1396)

Start date September 1, 2011

End date March 31, 2016

2

1. AIMS AND OBJECTIVES

This research aims to use formative evaluation methods to support and analyse reconfiguration of stroke services in different regions of England and, in doing so,

identify lessons that will guide future reconfiguration work in other services.

1. To identify the barriers and facilitators for major system reconfiguration,

implementation and sustainability. 2. To study whether the reconfigurations have delivered clinical and cost effective

improvements that patients and public think are worthwhile. 3. To identify lessons about major service reconfiguration that might be applied in

other settings (i.e. other locations and other service domains).

Research questions

1. What are the key processes of and factors influencing the development and implementation of the stroke service reconfigurations?

2. To what extent have system changes delivered process and outcome improvements?

3. Have changes delivered improvements that stakeholders (e.g. commissioners,

staff, patients and the public, and reconfiguration leads) think are worthwhile? 4. Have changes delivered value for money?

5. How is service reconfiguration influenced by the wider context of major structural change in the NHS?

2. BACKGROUND

A major review of the literature on the diffusion of innovations in service organisations drew attention to the lack of research on the processes by which

innovations in healthcare organisations are implemented and sus tained (or not) and in what particular contexts 1. This project aims to contribute to the development of

this evidence base by studying in depth the implementation of major service innovation, using the example of stroke services. Research, including that funded by the National Institute for Health Research, has highlighted challenges in implementing acute service reconfiguration 2, 3, as has previous research on mergers of healthcare providers 4, particularly where there is resistance from professionals and the public 5. Other research has highlighted challenges of major system delivery change at local level 6. This project will build on this work by studying the implementation of system reconfigurations across a whole patient pathway in several regions in England focused on the same clinical condition but in different contexts and using different models.

Significant changes in provision of clinical care within the English National Health Service (NHS) have been discussed in recent years, with the proposal to concentrate specialist services, such as major trauma, cardiac surgery, and specialist paediatrics, in fewer centres 7, 8. The case for such change in stroke services was strong, with

clear evidence of unacceptable variations in the quality of care, and many patients

denied access to evidence based care 9. Major system change for stroke was

3

prompted by the publication of the Department of Health National Stroke Strategy 10 and London and Manchester led the way in making changes.

Reconfiguration in London

The London reconfiguration was conducted at the request of the London Strategic Health Authority (SHA). An additional £20m per annum were provided by PCTs to be

paid through an enhanced tariff providing that the quality standards set by a multidisciplinary steering group were met. The model was developed with the

support of a Joint Committee of PCTs representing all Commissioners in London. The Greater Manchester and Cheshire Cardiac and Stroke Network (GMCCSN) were

charged by the Greater Manchester Association of PCTs to reconfigure services to allow universal access to hyperacute stroke treatment in the area.

In London, Trusts participated in a bidding process to host 24/7 Hyperacute Stroke Units (HASUs), Stroke Units and Transient Ischaemic Attack (TIA) services. Following this process, of the stroke services provided by 32 London hospitals pre-reconfiguration, 8 services were designated as Hyperacute Stroke Units, 24 as Stroke Units, while 4 services were withdrawn. Any person presenting with a suspected

stroke is transferred to a HASU for assessment and treatment, then repatriated to a Stroke Unit, to a nursing home, or to their own home. The reconfigured London

model was implemented in July 2010.

Reconfiguration in Greater Manchester

In Greater Manchester’s reconfigured model, the local population is served by 10 hospital Trusts, each providing District Stroke Centre (DSC) services; one Trust also hosts a Comprehensive Stroke Centre (CSC), which offers hyperacute stroke services in a neurosciences centre with access to interventional neuro-radiology and neurosurgery (24 hours per day, seven days per week); and two Trusts host Primary Stroke Centres (PSCs), providing thrombolysis (from 7 am to 7 pm, Monday to Friday). Any individual presenting within four hours of developing stroke symptoms is transferred to either the CSC or PSC for hyperacute care; once stable, he/she is repatriated either to a DSC, to a nursing home, or their own home. If presenting outside this four-hour ‘window,’ stroke patients in Greater Manchester are taken to the DSC to which they are nearest, much as they did prior to reconfiguration. In Manchester, the reconfiguration was achieved in a stepwise fashion, commencing in December 2008 and completed in April 2010.

Further changes in Greater Manchester

When the Greater Manchester reconfiguration was originally designed, it was agreed that a formal review of performance should be conducted 12 months post implementation. This review was delivered in December 2011 and based on this, the reconfiguration’s External Advisory Group concluded that the Greater Manchester model had not fully delivered on its aim to provide local population with equal access to high quality stroke services. It was agreed that further changes in the model should be considered.

In June 2012, consultation on further reconfiguration of the Greater Manchester model began. It is proposed that from September 2013 a revised model will be

implemented (‘Manchester B’), where anyone suspected to have had a stroke will be

4

taken to either the Comprehensive Stroke Centre (CSC) or a Primary Stroke Centre (PSC; PSC hours will be extended to cover 7am-11pm, 7 days per week) and District Stroke Centres will no longer receive suspected acute stroke patients. These changes would bring stroke services in Greater Manchester more in line with the London model.

Reconfigurations planned in the Midlands and the East of England

The Midlands and East of England covers an area in excess of 48,000km2 with a

population of 15.5 million people. This region has 20,000 hospital admissions following stroke every year 11. In comparison, London covers an area of 1570km2

with a population of 8.17 million people, and has approximately 8,000 annual hospital admissions following stroke; and Greater Manchester covers 1276 km2 with

a population of 2.68 million people, and has approximately 4,500 hospital admissions following stroke per year.

NHS Midlands and East SHA identified variation in stroke service performance and outcomes, both across the region and in comparison with other parts of the country. There was interest in the improvements in performance and outcomes achieved in

London as a result of its major stroke reconfiguration in 2010.

The SHA recognised substantial differences between the Midlands and the East of

England and London, not least the demography, and predominately rural nature of large parts of the Midlands and the East of England. Consideration was given to how, with a different geography, demography, and economic circumstances, the Midlands and the East of England could achieve a step change improvement in stroke outcomes. A review was commissioned to identify the arrangements necessary to achieve this.

An External Expert Advisory Group (EEAG) has developed a detailed best practice specification to guide local service provision. These cover seven ‘domains’ along the care pathway: primary prevention; pre-hospital care; acute care (including hyperacute, acute, Transient Ischaemic Attack, and tertiary care); community rehabilitation (including Early Supported Discharge and stroke specialist community rehabilitation); long term care and support; secondary prevention; and end of life support.

Local health systems, coordinated by the nine local Stroke Networks, are developing

proposals of how they will meet this specification.

The geography of the region is such that there will be implemented a number of ‘urban’ models, ‘rural’ models and ‘combined’ models to reflect small urban areas with rural catchments. The extended evaluation will analyse examples of the different models, and draw learning from each.

As was the case in Greater Manchester’s initial reconfiguration, the reconfigurations in the Midlands and the East of England will take a phased approach to

implementation. For example, changes in Primary Care settings are expected to commence in April 2013, whereas the first changes in Acute Care settings are

anticipated in July 2013 (following consultation processes), with all changes expected to be implemented by March 2014.

5

3. NEED

HEALTH NEED

In England, 110,000 people experience a stroke each year; of these people,

approximately 30% die within a year; and half the remainder are left with lifelong impairments and disability10. A National Audit Office (NAO) report on stroke care in

England was highly critical of the quality and cost effectiveness of services, with the majority of patients not receiving the most effective treatment 12. It is estimated that

stroke costs the English economy £7 billion per annum, much of which is spent providing long term support to people disabled by strokes that might either be

prevented altogether or their effect minimised by high quality acute care and rehabilitation.

The National Stroke Strategy 10 identified 24 markers of high quality services, to be achieved by health and social care services within 10 years. It is argued that achieving these markers may improve quality and cost-effectiveness of services. They include many aspects of stroke care, such as public awareness, prevention of stroke, timely access to stroke services, including thrombolysis, development of

specialised stroke services, and provision of specialist rehabilitation services and long term support. The service models evaluated here represent efforts to reorganise

services so that many of these markers are achieved and are therefore a valuable source of learning on how effective stroke care might be organised and delivered.

RESEARCH NEED

While there have been significant reconfigurations of acute services over recent years, there have been few similar examples within the NHS of services being completely restructured across a whole pathway for such large populations over such a short period of time; and nowhere else in the world has attempted anything similar for stroke. Consequently, processes of system wide change and the impact of ‘hub and spoke’ models in the context of stroke care are not yet well understood. This research represents an opportunity not only to understand the processes by which such large scale change is brought about, but also the effects the models implemented have on the quality, cost, and outcome of stroke services.

The lessons learned, such as how to develop innovative structures and processes from these reconfigurations should be directly applicable to other situations where

major reconfiguration is being planned, such as cardiac and vascular surgery and major trauma. This work also provides an opportunity to investigate health economic arguments about investing to save for stroke 13. The lessons learned from this project will therefore continue to have relevance not only for stroke but future service reconfigurations.

This evaluation takes place at a time of significant change in the English NHS. The UK government’s Health and Social Care Act 14 brought with it a number of changes to how services are organised, with several bodies abolished and their functions transferred to new organisations. Exploring the impact of this turbulence will form a

key element of this evaluation, providing an opportunity to learn whether and how reconfigurations can be conducted or sustained through a period of major structural change.

6

This study was originally funded by the Health Services and Delivery Research programme (HS&DR) of the National Institute for Health Research (NIHR) to study the reconfiguration of stroke services in London and Greater Manchester 15 for three years from 1st September 2011 – 31st August 2014. An extension to the original study was funded by NIHR HS&DR from 1st November 2012 to 31st March 2016. Both quantitative and qualitative components of the research will be extended, to incorporate additional analysis of further stroke service reconfiguration in Greater

Manchester and planned changes in the Midlands and the East of England.

4. METHODOLOGY

DESIGN

We propose to study these major system innovations in two contrasting, but complementary ways. First, we will take a more traditional health technology assessment approach to address 'what works and at what cost?'. On its own, however, this approach pays little attention to structural pressures, e.g. professional pressures or processes whereby organisations innovate as a result of economic,

regulatory or legal reasons 16, 17. This approach also assumes innovation is always progressive and poor adopters are conservative, whereas resistance may be a

rational response 18.

A major review of the evidence on diffusion of innovations identifies the characteristics of innovations that are more likely to be sustainable 1:

the nature of the innovation (relative advantage, low complexity, s cope for reinvention) and its fit with the organisation’s existing skill mix, work practices and strategic goals

motivation, capacity and competence of individual practitioners

elements of organisational structure (e.g. devolved decision making, internal networks) and capacity (e.g. change skills, evaluation skills)

resources and leadership early involvement and co-operation of staff at all levels

personalised, targeted and high-quality training

evaluation and feedback linkage with the resource system from development of the innovation through to

implementation embeddedness in inter-organisational networks

conducive external pressures e.g. synchrony with local priorities and policymaking streams.

While the study period precludes establishing the long term sustainability of the changes brought about by reconfiguration, the evaluation will assess the extent to which the approaches taken by the reconfigurations reflect these characteristics,

particularly in the light of unprecedented changes to the English NHS.

Drawing on the innovations theory, particularly that which emphasises the role of structural factors, we propose to study the implementation of these innovations in

7

terms of Webster's 18 notion of a 'social matrix' i.e., not as a technical, rational set of issues but in terms of requiring 'co-creation' by a range of stakeholders - where such processes create a 'social matrix' that is only as strong as the network of relations that hold it together. Studying the implementation of different models of system innovation in a range of contexts (London, Greater Manchester and the Midlands and the East of England) will allow us to understand the relationship between implementation and organisational context. Given the changing nature of the

healthcare landscape, we are proposing a formative evaluation, providing lessons to the regions involved and the wider NHS over the course of the study.

A multi-method approach will be employed, including documentary analysis, analysis of relevant performance data (including costs), and interviews and observations.

Understanding what works and at what cost

Identifying what process and outcome changes have occurred, and at what cost, provides evidence about the extent to which the reconfigurations have succeeded in their objectives of changing the system of service provision and quality of care. This component of the evaluation will analyse documentary evidence to establish the

models applied in London, Greater Manchester, and selected sites in the Midlands and East of England; it will also identify whether any changes in process and

outcomes can be associated with these changes.

In assessing the nature and results of the reconfigurations, the evaluation will apply a controlled before-and-after design 19. This will compare the participating regions in terms of the impact they had on processes, outcomes, and costs of care. In addition to comparing these sites pre- and post-reconfiguration, the analysis will make a wider comparison with the rest of England. This approach will permit observed changes to be analysed in the context of changes that take place in the rest of England over this period. A two-step approach will be taken to the analysis of outcomes: in the first instance, the impact of the reconfigurations on mortality and length of stay will be investigated; if evidence of impact on these is found, a more detailed cost-effectiveness analysis will be undertaken.

Understanding implementation and sustainability

To develop worthwhile lessons for future reconfigurations, it is important to establish not just whether process and outcome changes took place, but also how

and why they occurred. This will be achieved through qualitative methods (documentary analysis, stakeholder interviews, and non-participant observation). These data will be used to explore themes drawn from the evaluation’s conceptual framework, and thus establish the relationships between activities in support of change, the context, the complex interactions between stakeholders, and perceived process and outcome changes.

Synthesis of approaches

A multi-method case study approach will be employed to draw together the learning from the approaches described above. The case study method permits development

and testing of theories on how efforts to bring about change interact with the context in which they take place; a multiple case study approach allows the

development and testing of theories in several contexts 20-22, although more local

8

contextual factors will also be considered: a series of local ‘service-level’ case studies will be conducted. The qualitative study is designed to allow change to be evaluated at, first, the planning and governance level of the reconfiguration of stroke services at the region level and, second, in relation to ‘service level’ case studies within each region of the changes to the provision of stroke services. The case studies will reflect the main ‘trajectories’ experienced by organisations participating in changes of this kind, including developing new services, and closing existing services. Each case

study will draw together evidence from the evaluation’s quantitative and qualitative components to develop and test theories on how change activities interacted with

local contexts to produce the outcomes observed.

A total of 12 case studies will be conducted, covering London (5 case studies),

Greater Manchester (3 case studies) and the Midlands and East of England (4 case studies). Table 1 presents the timescales of the reconfigurations, disaggregated over

three phases – ‘before’, ‘during’ and ‘after’ – as applied in the quantitative analysis.

Table 1. Timescales of reconfigurations

Before During After

London January 2004 – June 2008

July 2008 – June 2010

July 2010 – December 2013

Greater Manchester A January 2004 – December 2008

January 2009 – May 2010

June 2010 – Aug 2012

Greater Manchester B June 2010 – Aug 2012

Sept 2013 – April 2014

May 2014 – March 2015

Midlands and the East of England

January 2004 – March 2013

April 2013 – March 2014

April 2014 – March 2015

Note. Timings of reconfigurations yet to be implemented are based on current plans, and may alter

The Greater Manchester region will be studied contemporaneously as it changes its

reconfiguration model. Four further case studies to cover the changes taking place in the Midlands and the East of England will be conducted contemporaneously: the

selection of case studies in this region will permit the evaluation to examine models of stroke care applied in urban and rural settings, and an area combining urban and

rural locations (see Table 2 for summary of reconfiguration models by setting).

Table 2. Summary of reconfiguration models by setting

Model Reconfigurations

Urban London - 5 service level case studies: HASU (x2), SU (x2), decommissioned service (x1)

Greater Manchester A & B - 3 service level case studies: CSC (x1), PSC (x1), DSC (x1)

Midlands and the East of England 1

Rural Midlands and the East of England 2 & 3

Combined Midlands and the East of England 4

9

5. DATA COLLECTION

Documentary analysis

We will collect documentation on the reconfigurations at all sites. These documents

will cover how the reconfigurations developed and the processes of implementation. They will include records of public consultations conducted by SHAs and Stroke and

Cardiac networks; Gateway reviews (where available); reconfiguration proposals; and project documentation. Project update documentation will be collected over the

lifespan of the evaluation.

Quantitative analysis – what works and at what cost

Using routinely collected data, the evaluation will assess whether the reconfigurations were associated with changes in process, outcomes and costs of care. Data will be collected retrospectively from 2004, up to the point at which services began to change; whilst ‘after’ data will be collected from the points at which the new models were fully in place, through to March 2015.

Care process measures have been selected for the evidence indicating relevance to

quality of care and their positive association with improved outcomes 23-25. Measures include proportion of patients appropriately receiving thrombolysis, admitted to a stroke unit for 50% of their stay, appropriately receiving aspirin in the acute period, assessed by a multidisciplinary team, and receiving a swallowing test within the first 24 hours of care. These data will be drawn from national data sources (the Royal College of Physicians’ Sentinel Stroke Audit, the Stroke Improvement National Audit Programme (SINAP), and the Sentinel Stroke National Audit Programme (SSNAP)).

Outcome measures will include patient’s length of hospital stay, discharge destination, readmission rates, and mortality (in hospital, at 30 days, and at one

year): these data will be drawn from HES and ONS datasets. Patient’s independent function (measured through Barthel Index score at discharge, and/or Modified

Rankin score) will be drawn from Sentinel audit data, SINAP and SSNAP.

Qualitative analysis – understanding implementation and sustainability

First, qualitative data will be collected at the planning and governance level for each region as a whole. Second, data will be collected in the context of specific case

studies of the implementation of the planned changes to stroke services at the service level.

Interviews

Semi-structured one-to-one interviews will be conducted with a range of people who commission, plan, deliver and experience stroke services within the different

regions, including people who led the reconfigurations; local commissioners; staff working in relevant services; staff who worked in decommissioned services ; and stroke patients and their caregivers.

Observations

Along with the interviews, non-participant observation of meetings in the Greater Manchester and the Midlands and East of England regions will take place to support

the contemporaneous analysis of implementation. At the planning and governance

10

level for each region, researchers will observe significant events in the development of the reconfigurations, including planning meetings, consultation events, and processes by which new services are set in place (e.g. training events). In the context of local provider case studies, service level meetings within provider organisations will be observed, such as multidisciplinary meetings.

Participants

Our sample reflects the areas in which reconfigurations have taken place, i.e.

Greater Manchester, London, and the Midlands and East of England. These service reconfigurations have been selected because, as outlined above, the scale of the

work is unprecedented in this context.

For the interview component of the evaluation, we will sample informants purposively. We aim to obtain perspectives from a wide range of informants, reflecting the organisations and groups involved in and affected by the reconfiguration.

To support longitudinal analysis, interviews will be conducted at several points in time (see timeline). We will aim to carry out follow-up interviews with staff

interviewees, while service user and carer interviews will be conducted with people who have recently experienced stroke services – i.e. the service users and carers

interviewed in Phase 1 and Phase 2 will not be the same people.

Staff

For each reconfiguration, we will carry out one-to-one interviews, and follow up interviews with leaders of the reconfigurations (N=10-20). For each service level case study, we will interview up to 15 informants (with follow-up interviews), including local commissioners, taking into account changes to commissioning of services, and

service providers (including doctors, nurses, therapists, management and ambulance staff), i.e. up to 165 ‘staff’ interviewees across the participating sites, and up to 330

interviews in total.

Service users and carers

We will also carry out one-to-one interviews with patient and carers. To ensure participants have recent experience of reconfigured stroke services, we will

approach and recruit people just before discharge from hospital and will interview them approximately three months after discharge. We will interview up to 8 service users and carers per service level case study, making a total of up to 80 interviews

(NB one London case study is of a decommissioned service, meaning no current patients will be available to be interviewed).

A number of patient interviews will be conducted by service user expert and co-investigator, Mr Nanik Pursani, who has an Honorary contract with King’s College

London and has received formal social research methods training.

Over the two phases, we estimate that we will carry out a total of up to 410

interviews: up to 330 with reconfiguration leads, commissioners and service providers, and up to 80 interviews with patients and carers.

Interviews will take place in a private location agreed with the participant, or over the telephone.

11

Recruitment

Staff

Relevant staff interviewees will be identified in discussions with local stroke network

leads and through documentary analysis, and will be limited to those who had some type of involvement in the reconfiguration process or its results . Potential staff

participants will initially be approached by the study researchers. Contact, including provision of information sheets, will be made through e-mail and telephone.

For observational data, permission to observe meetings will be gained from the meeting’s chair prior to the meeting commencing. An information sheet will be circulated to all participants in advance. At the start of each meeting, participants will be briefed on the study’s aims, what participation entails, and that they can decline to participate at any time. Verbal consent of the participants will be recorded. If participants do not agree to participate, any contributions they make to the meeting will be excluded from the researcher’s field notes, or the researcher will withdraw from the meeting if appropriate. Staff involved are unlikely to experience any risks from this study. Staff and individual organisations will be granted

anonymity and will not be identified by name in any reports.

Service users and carers

Clinical staff will identify service users (or carers) who are medically stable and close to discharge. A member of clinical staff will then ask these potential participants if they are willing to speak with a project researcher about the study. If the individual is willing, a study researcher will then verbally explain the purpose of the study, and provide written information. If the potential participant is still interested and agreeable, the researcher will allow at least 24 hours to elapse, then contact the potential participant again to ask for his/her agreement to participate in an interview. If the potential participant agrees, interviews will take place within three months of discharge, at a time and place mutually agreed with the patient and/or carer. The potential participant will be able to refuse to participate at any point: when first approached, again when asked for agreement 24 hours later, and at any point subsequently, up to and including the actual interview.

This approach to recruitment of people with stroke and their carers has been used in the ongoing South London Stroke Register since 1995 (approval by the ethics

committees of Guy’s and St Thomas’ Hospital Trust, King’s College Hospital, Queen’s Square, and Westminster Hospital (London). It has also been discussed with the King's College London Stroke Research Patients and Family Group (http://www.kcl.ac.uk/schools/medicine/research/hscr/sections/stroke/patients.html) and found to be an appropriate way of inviting people in this patient group to participate in research.

Interview topic guides

Interview topic guides will be developed over the initial phase of the project and will be informed in part by evidence provided by the documentary analysis ; the topic

guides for follow up interviews will be informed in part by findings derived from initial qualitative and quantitative analyses. The main themes of interviews are

summarised below.

12

Interviews with leaders of the reconfigurations will include the following topics :

how reconfigurations were developed, e.g. national and local contextual factors influencing selection of models, consultations, process mapping, etc;

processes of implementation;

impact on staff and services, including health and social services;

impact on patients and the public; and

contextual factors influencing the reconfiguration (e.g. finance and the organisational setting).

Interviews with local stroke services staff (including those in decommissioned services) and local commissioners will address:

processes of implementation; impact on staff and services, including health and social services;

impact on patients and the public; and

contextual factors influencing the reconfiguration (e.g. finance, the organisational setting, etc).

Interviews with patients and carers will address their experiences of the reconfigured stroke services.

6. DATA ANALYSIS

Documentary analysis

The reconfigurations at both sites have been documented effectively (see above)

and the proposed evaluation will have full access to these data.

Documentary evidence will inform the evaluation in terms of the development and

progress of the reconfigurations. We will analyse the documentary evidence – with a focus on documents covering planning, events and progress of the reconfigurations

to build an understanding of the development and implementation of the work. Themes will be drawn from the evidence base on effective organisational change 26,

covering issues such as

preconditions, e.g. organisational context, drivers for change, and local leadership ‘buy in’;

planning, e.g. consultation with relevant stakeholders, consideration of resource needs and selection of effective methods;

implementation, e.g. engagement and training of relevant staff, provision of resources to support engagement, effective measurement and ongoing, constructive feedback on progress of the work; and

follow-up, e.g. whether the work is evaluated in terms of generating evidence and developing theory.

This information will feed into the development of ‘timelines’ detailing the progress of the evaluations, which will contribute to the overall structure of the evaluation. It

will reveal the approaches taken to the reconfigurations and identify events that may have influenced progress of the work.

13

Quantitative analysis – what works and at what cost

Analysis of process and outcome data

Routinely collected data will be analysed for two purposes. First, to analyse

outcomes, econometric and cost effectiveness analyses will be conducted. Second, there will be an analysis of whether the reconfigurations are associated with

significant changes in how services were provided, i.e. an analysis of process measures.

Analysis of outcomes

Econometric analysis

The outcome variables measured at the Trust level in each time period will be regressed against the covariates, with particular interest in interactions showing the

impact of region following the introduction of stroke service reconfigurations. The regression model is:

ititittiit eXRYAy 43210 [1]

where y is the outcome of interest (mortality, LOS), i indicates Trust, t indicates year, A is region, Y is year, R is an indicator variable taking the value one if stroke services in Trust i in each region A in year t have been reconfigured (1=yes, 0 otherwise), X is

a set of patient and Trust characteristics , the s are coefficients to be estimated, and e is an error term. The regression model used will depend on the nature of the dependent variable. Of particular interest are the sign and statistical significance of

the coefficient 3 . If for reconfigurations in participating regions 3 indicates a

favourable result, i.e. a reduction in mortality and/or on LOS, the following cost-effectiveness analysis will be undertaken. A different version of Eq.[1] may be required if the time period during which reconfiguration occurred was substantial. In this case an alternative regression model is:

itit

A

it

D

ittiit eXRRYAy 543210 [2]

where , RD is an indicator variable taking the value one if stroke services in Trust i in each region A in year t are in the process of being reconfigured (“During”; 1=yes, 0 otherwise) and RA is defined analogously for the period after reconfiguration has been implemented (“After”). In this case 4 is the variable of interest.

Analysis of cost-effectiveness

A detailed Discrete Event Simulation model, with costs and outcomes summarised at 30 days, 90 days and 10 years, was developed to assess the cost effectiveness of the new London stroke model implemented in 2010. Following successful implementation of the model(s), and a sufficient period of time to collect adequate patient numbers, we will populate the model using the data sources mentioned above in Data Collection, for the participating regions ‘before’ and ‘after’ implementation. Costs will be assessed from the perspective of the NHS and

personal social services (PSS). The proposed cost-effectiveness measure in the short run model is the incremental cost per death avoided at 90 days; in the long run it is the incremental cost per quality adjusted life year gained. Cost components will include: ambulance; scans; thrombolysis; length of stay on wards; and discharge

14

destination. Cost-effectiveness will be calculated as the mean cost difference between the comparators divided by the mean difference in outcomes (90 day mortality/QALYs) to give the incremental cost-effectiveness ratio (ICER). We will undertake deterministic (one-, two- and multi-way) and probabilistic sensitivity analysis.

Analysis of functional independence

To analyse the effect of the reconfigurations on stroke patients’ functional

independence, the regression model described for the econometric analysis of impact on outcomes will be applied to functional independence measures (e.g.

Barthel Index score on discharge) where available, covering the same ‘before’ and ‘after’ periods.

Analysis of process changes

To analyse process changes, the regression model described for the econometric analysis of impact on outcomes will be applied to process variables, covering the same ‘before’ and ‘after’ periods. This will allow an assessment of whether relative performance across the participating regions (London, Greater Manchester and the

Midlands and East of England) remains constant pre and post reconfiguration, i.e. whether the reconfigurations are associated with changes in processes of care.

Extending the ‘location’ comparison to include the rest of the English NHS will permit these changes to be assessed in the context of any nationwide changes that may have taken place over the period analysed.

Interviews

All interviews will be digitally recorded for transcription in full. Fieldwork notes will also be kept by the researcher. Data will be managed with NVIVO software. Ongoing

iterative and thematic analysis of all data will be undertaken concurrently, according to well established procedures of constant comparative analysis 27. Initial analysis

and category building will be conducted by the researcher and will include category mapping and constant comparison. Validity will be assessed in relation to Patton’s

four criteria of validity in qualitative research: verification, rival explanations, negative cases and triangulation 28.

Non-participant observation

A structured template will be used to record observational data collected by members of the research team that attend meetings for this purpose. The

observational data will be iteratively and thematically analysed using the same methods as for the interview data.

7. DISSEMINATION

One product of the research will be the final report to the funder, NIHR HS&DR. Alongside the main report, which will describe the research and its findings in detail,

we will provide short summaries of the research; and appendices presenting e.g. data tools, raw data where appropriate, and an archive of the stakeholder update reports.

15

To ensure national and international dissemination of the learning from this proposed research, we will aim to publish our findings in high impact peer reviewed journals.

In addition, we will produce summaries of our ongoing findings for professional healthcare and management journals.

In tandem with development of peer reviewed publications, the project will present its findings to national and international scientific meetings and conferences.

As we propose a formative evaluation, we will disseminate our learning to stakeholders by presenting at regular events held by the participating Stroke Networks, covering progress of reconfiguration, process, outcome and cost measures; and any learning relevant to progressing work further. These will be supplemented by electronic updates, to be distributed to all relevant stakeholders every six months. Given the potential to transfer learning from this evaluation to reconfigurations in other domains, we will also provide management briefings for NHS policy makers, managers and clinical leaders using a range of electronic delivery mechanisms, e.g. Webex.

A ‘lay’ version of these updates will be produced to ensure that patients and the public are suitably informed. Co-applicant (CM) produces a biannual research

newsletter for stroke survivors and carers (http://www.kcl.ac.uk/schools/medicine/ research/ hscr/sections/stroke/patients.html) and this expertise will be extremely valuable in ensuring production of useful and accessible bulletins.

8. PLAN OF INVESTIGATION

Task Period (months)

Research tool development 1-12

Process and outcome data – collection and analysis 11-20

27-31

44-55

London interviews – collection and analysis

7-22

25-32

Manchester interviews – collection and analysis

7-22

25-32

38-48

Midlands & East of England interviews – collection and analysis

20-28

38-48

Manchester observation data – collection and analysis 20-48

Midlands and East of England observation data – collection and 20-48

16

analysis

Progress reports to funder 6, 12, 17, then monthly

Interim report to funder 31

Final write-up 46-55

Note. Month 1 = September 2011

9. PROJECT MANAGEMENT

Management arrangements

As Chief Investigator, Naomi Fulop provides overall oversight and leadership of the

project team. She also manages the two London-based researchers.

A London-based researcher is responsible for day-to-day management of the project. They share responsibilities for coordinating the research and preparing research

team meetings, e.g. providing regular electronic updates to the research team and SSC.

The Manchester researcher was appointed in month 14, and will lead data collection in the Manchester area. She is managed by co-investigator RB, and has weekly

update meetings with the London researcher.

The research team will meet monthly over the course of the project. To ensure maximum participation, teleconference facilities will be used. The research team will meet in person once per year.

Study Steering Committee

The Study Steering Committee (SSC) is chaired by Sir Roger Boyle. It is composed of

academics, stakeholder representatives, including staff drawn from provider, commissioner and other relevant organisations in the areas covered by the

reconfigurations, and service user representatives from local stroke service user groups.

The SSC will meet at least annually throughout the duration of the project.

10. SERVICE USERS/PUBLIC INVOLVEMENT

Service users/public involvement

We will involve service users in a number of ways. Nanik Pursani, a stroke service user, is a co-investigator on the evaluation. He has contributed to project meetings, the development of research tools (patients and carers’ interview topic guide), and will carry out interviews with patients and carers.

We will present any research instruments and the resultant thematic analysis to user

groups, for example, the KCL Stroke Research Patients and Family Group (http://www.kcl.ac.uk/schools/medicine/research/hscr/sections/stroke/patients.ht

ml) for their feedback.

17

We will communicate progress of the evaluation to stakeholders through established avenues, including updates through local clinical networks, and bulletins on local stroke newsletters.

End-user involvement

End-users of this research were directly involved in the development of this proposal

and will remain involved as co-investigators (AR, PT, and SS) throughout all stages of the project. In addition, as mentioned above, our Study Steering Committee has

membership from a range of stakeholders from relevant organisations in London, Greater Manchester, the Midlands and the East of England.

11. FUNDING

The project ‘Innovations in major system reconfiguration in England: a study of the effectiveness, acceptability and processes of implementation of different models of

stroke care’ is funded by the National Institute for Health Research’s Health Services and Delivery (NIHR HS&DR) Programme.

12. REFERENCES

1. Greenhalgh T, Robert G, Bate P, Kyriakidou O, Macfarlane F, Peacock R. How to spread good ideas: a systematic review of the literature on diffusion, dissemination and sustainability of innovations in health service delivery and organisation. London: National Co-ordinating Centre for NHS Service Delivery and Organisation; 2004 Contract No.: 12th May 2009.

2. Spurgeon P, Cooke M, Fulop N, et al. Evaluating models of service delivery: reconfiguration principles. Southampton: Final report to NIHR SDO Programme; 2010.

3. Fulop N, Walters R, 6 P, Spurgeon P. Implementing changes to hospital services: Factors influencing the process and ‘results’ of reconfiguration. Health Policy 2011 doi:10.1016/j.healthpol.2011.05.015.

4. Fulop N, Protopsaltis G, King A, Allen P, Hutchings A, Normand C. Changing organisations: a study of the context and processes of mergers of health care providers in England. Social Science & Medicine 2005;60:119-30.

5. Fulop N, 6 P, Spurgeon P. Processes of change in the reconfiguration of hospital services: the role of stakeholder involvement. In Organising and Reorganising - Power and Change in Health Care Organisations.Edited by McKee L, Ferlie E, Hyde P. Basingstoke: Palgrave MacMillan; 2008.

6. Greenhalgh T, Humphrey C, Hughes J, Macfarlane F, Butler C, Pawson R. How do you modernize a health service? A realist evaluation of whole-scale transformation in london. The Milbank Quarterly 2009;87:391-496.

7. Imison C. Reconfiguring hospital services. London: King's Fund; 2011.

8. The Royal College of Surgeons of England and British Orthopaedic Association. Better care for the severely injured. London: RCSE; 2001.

9. Intercollegiate Stroke Working Party. National Sentinel Audit of Stroke. London: Royal College of Physicians; 2008.

18

10. Department of Health. National Stroke Strategy. London: Crown; 2007.

11. Lintern S. Analysed: review aims to improve stroke services in the Midlands and East. 2012; Available from: http://www.hsj.co.uk/hsj-local/briefing/midlands-and-east/analysed-review-aims-to-improve-stroke-services-in-the-midlands-and-east/5047149.article.

12. National Audit Office. Reducing brain damage: faster access to stroke care. London: The Stationery Office; 2005.

13. National Audit Office. Progress in improving stroke care: modelling paper. London: National Audit Office; 2010.

14. Health and Social Care Act. London: The Stationery Office; 2012.

15. Fulop N, Boaden R, Hunter R, et al. Innovations in major system reconfiguration in England: a study of the effectiveness, acceptability and processes of implementation of two models of stroke care. Implement Sci 2013;8 doi:10.1186/1748-5908-8-5.

16. Ferlie E, Dopson S. Studying complex organizations in health care. In Knowledge to action? Evidence based health care in context.Edited by Dopson S, Fitzgerald L. Oxford: Oxford University Press; 2005.

17. Lemieux-Charles L, Barnsley J. An innovation diffusion perspective on knowledge and evidence in health care. In Using Knowledge and Evidence in Health Care: Multidisciplinary Perspectives.Edited by Lemieux-Charles L, Champagne F. Toronto, ON: University of Toronto Press; 2005.

18. Webster A. Health, Technology, and Society: A Sociological Critique. Basingstoke: Palgrave Macmillan; 2007.

19. Ashenfelter O, Card D. Using the longitudinal structure of earnings to estimate the effect of training programs. Rev Econ Stat 1985;67:648-60.

20. Yin RK. Case study research: design and methods. 4th ed. London: Sage Publications; 2009.

21. Yin RK. Enhancing the quality of case studies in health services research. Health Services Research 1999;34:1209.

22. Baker GR. The contribution of case study research to knowledge of how to improve quality of care. BMJ Qual Saf 2011;20:i30-i5 10.1136/bmjqs.2010.046490.

23. Wellwood I, Wu O, Langhorne P, et al. Developing a Tool to Assess Quality of Stroke Care Across European Populations The EROS Quality Assessment Tool. Stroke 2011;42:1207-11.

24. Wiedmann S, Norrving B, Nowe T, et al. Variations in Quality Indicators of Acute Stroke Care in 6 European Countries. Stroke 2012;43:458-63.

25. Saposnik G, Kapral MK, Coutts SB, Fang J, Demchuk AM, Hill MD. Do all age groups benefit from organized inpatient stroke care? Stroke 2009;40:3321-7.

26. Worrall A, Ramsay A, Gordon K, et al. Evaluation of the Mental Health Improvement Partnerships programme. London: National Co-ordinating Centre for NHS Service Delivery and Organisation; 2008.

27. Mays N, Pope C. Qualitative research: rigour and qualitative research. BMJ 1995;311:109.

28. Patton M. Qualitative evaluation and research methods. Thousand Oaks, CA: Sage; 2002.