wapagfinrep

8/10/2019 WAPAGFinRep

1/70


2/70

THE FUTURE OF AMBULANCE SERVICES IN WALESA REVIEW FOR THE WELSH AMBULANCE POLICY ADVISORY GROUP

FOREWORD BY DR. GARETH JONES 1

FOREWORD 1

EXECUTIVE SUMMARY 2

1. INTRODUCTION & STRATEGIC CONTEXT 5

1.1 BACKGROUND TO THE REVIEW 5

1.2 P ROJECT GOAL AND OBJECTIVES 6

2. PROJECT DESIGN 7

2.1 OVERVIEW 7 2.2 P ROJECT CONTROL 7 2.3 ROLES & RESPONSIBILITIES 8

3. SYSTEM OF CARE DESIGN 8

3.1 DESIGN PROCESS 8 3.2 BENEFIT AND RISK ASSESSMENT CRITERIA 9 3.3 BENEFIT & RISK SCORING SCHEME 9 3.4 COST ASSESSMENT 10 3.5 CONSTRAINTS 10 3.6 OPTIONS FOR APPRAISAL 10

4. THE EVIDENCE OF THE OPTION APPRAISAL 10

4.1 THE APPRAISAL P ROCESS 10 4.2 APPRAISAL RESULTS 11

5. CONCLUSIONS 13

5.1 THE FUTURE DIRECTION OF AMBULANCE S ERVICES IN WALES 13 5.2 IMPROVING P ATIENT CARE S ERVICES 13 5.3 METHODS OF ORGANISATION 15 5.4 ISSUES AFFECTING IMPLEMENTATION 19 5.5 S ECURING STAKEHOLDER SUPPORT 20


3/70


4/70


1

Foreword

I am pleased to write as independent adviser to the Welsh Ambulance Policy Advisory Group (WAPAG). The project to review the Future Provision of Ambulance

Services in Wales commenced in December 1996. I became involved in May 1997at the same time that Philip Davies, Director of Service Development, Bro Taf Health

Authority and David Williams, Chief Executive, Swansea NHS Trust, joined theProject Team as independent assessors. We participated in the discussions on theselection of criteria for the evaluation of options, the weighting of the criteria, and inthe discussions of the conclusions reached by the Independent Option AppraisalGroup, leading to this final report. The Group comprised NHS professionals andcommunity representatives, drawn from outside the Welsh Ambulance Service.

We were impressed by the process, developed by the selected project consultants,Fitch & Associates - Europe, Ltd and AEA Technology plc. It enabled the carefulevaluation of the strategic and operational factors to be carried out before reaching adecision of the future optimal configuration of the Ambulance Trusts. Form followedfunction. There was a very high level of participation in the process by interestedparties throughout Wales and the discussions in which we took part were extremelythoughtful, constructive and responsible. Overall, it was, in our opinion, a highlyprofessional exercise, carried out effectively and efficiently.

The evaluation of patient benefits, operational and transitional risks andmanagement costs involved both qualitative and quantitative factors. We agree thatthe best way forward lies in a choice between a three Trust and a one Trustconfiguration. The one Trust option maximises patient benefits and the three Trustoption minimises risks. The differences between these two options, taking potential

benefits and potential risks into account, was extremely close and the advantages ofone over the other too close to call.

However, the balance of views expressed by the WAPAG Project Team, havingreflected upon the data, was that the transitional risks related to a critical stage in theprocess rather than the future outcome. As such, these risks might be treatedseparately through the development of a transitional plan which would have to bemanaged as part of any future reconfiguration process. These issues need to becarefully considered when selecting the optimal Ambulance Service configuration toimplement the strategy presented in the report for the benefit of people in city, urbanand rural communities in Wales.

Gareth JonesChairmanNevill Hall & District NHS Trust


5/70


2

Executive Summary

In December 1996, building on work which they had carried out earlier that year, the Welsh Ambulance Policy Advisory Group (WAPAG) identified the need to review options for the

future development of Ambulance Services within NHS Wales, with the aims of improving performance in terms of patient service and cost effectiveness . This review was driven by thelikely introduction of improved emergency response time standards, the new policy directiongiven in A Fresh Start and criticism of the existing configuration by the Parliamentary Welsh

Affairs Committee. The objectives of the review were to:

improve Ambulance and Paramedic services for the benefit of patients;

produce clear practical recommendations on the method of organising AmbulanceServices in the future;

outline the issues affecting the implementation of these arrangements;

command overall support from Ambulance Services, other NHS providers and purchasersfor its recommendations.

A project was initiated, overseen by a WAPAG Project Team , and a Planning & RiskManagement Group was appointed, comprising an independent project leader and projectconsultants. The role of the Planning & Risk Management Group was to manage the projectand develop products including an option appraisal, risk assessment and written proposals.Four functional groups, covering Operations, Training & Education, Finance and Strategy, were also set up to produce briefing reports for the Planning & Risk Management Group.

The project was designed to include independent and stakeholder involvement at each stageof the process.

A system of care was designed, in accordance with the review objectives, describing thevalues and services required to provide effective, comprehensive and efficient out-of-hospitalcare to the people of Wales, regardless of service configuration. The system of care was theengine which drove the subsequent option appraisal, and was a benchmark for the ability ofeach option to deliver the service described. It consisted of a number of weighted criteria setsfor benefit and risk measurement. This was mandated by the Project Team.

The option appraisal was designed to measure the benefits and risks of each option relative tothose of the Modified Status Quo. Scoring was carried out, over two days, by a group ofindependent stakeholders, none of whom has any direct involvement in the management ofambulance services in Wales. Patient, staff, purchaser, provider and primary care interests

were all represented.

The result of the option appraisal was that:

1. The Three Trust option scored highest when benefits were added to total risks 1;

2. The All Wales option scored highest when transitional risk was removed from thecalculation;

3. These results are relatively insensitive to variations in the weightings of the criteria sets;

4. There was no significant difference in the costs of the two options.

Both the Three Trust and All Wales configurations have the capacity to save 3 million overtheir first five years in operation.


6/70


3

The recommendation of the Independent Option Appraisal Group was that there should bethree ambulance trusts providing all ambulance services in Wales.

This was not a straightforward choice. The Group concluded that the All-Wales AmbulanceTrust option offered more benefits than any other, but also that it carried the second highestlevel of total risk, most of which was transitional.

They judged, however, that the Three Trust option had the greatest capacity to achieve theobjectives of the review because it had the most favourable balance of benefit and total risk.

(See Annexe A for a more detailed exposition of the above factors)

Recognising the closeness of the two leading options, they asked for serious consideration tobe given to:

ensuring a greater degree of inter-service collaboration on suitable issues if the ThreeTrust option is selected;

ensuring maximum local accountability if the All-Wales option is selected.

The review was strengthened greatly by the unanimous support for the process expressed bythe Independent Option Appraisal Group. All participants agreed that the outcome provided afair and balanced representation of their views.

In reviewing the evidence of the option appraisal, the WAPAG Project Team faced the samedilemma as the Independent Option Appraisal Group: that the results for the two leadingoptions (the Three Trust and All Wales options) were so close as to make any firm distinctiondifficult.

However, in the course of their debate, two distinct strains of opinion emerged.

The first of these supported the conclusion of the Option Appraisal Group, feeling that:

the total risks of the All Wales option were greater;

there was a high probability that these risks would not be managed effectively.

The second strain of opinion supported the All-Wales option, holding that:

the benefits of the All Wales option were greater;

the transitional and operational risks could be managed, with professional support, usingrecognised techniques.

No consensus was achieved between these two positions, and the project team thereforemade the following recommendation:

The Welsh Ambulance Policy Advisory Groups Project Team recognises the strengths ofeach of the two leading options in its review. The Team also recognises that it is theresponsibility of the Welsh Office to determine the form of the public consultation which willfollow on this review. The Parliamentary Under-Secretary of State for Health may, however,wish to consider putting both options forward for consultation.


7/70


4

Issues affecting implementation were identified under the following headings:

1. Clinical Development2. Leadership & Management3. Community Ownership4. Staff5. Control & Communications6. Improving Operational Performance7. Financial8. Change Management

If the option which emerges from the public consultation exercise is to be successful inimplementation, it will require the continued good will and support of stakeholders. Theprocess by which the recommendation of this review has been achieved militates strongly, initself, in favour of stakeholder support for the new arrangements. It is suggested that theWelsh Ambulance Policy Advisory Group would benefit from the continued involvement ofindependent stakeholders in the various stages of the implementation process. In the view ofthe independents themselves, the value of an effective, stakeholder-focused communications

strategy cannot be over-emphasised. It is possible that the independent participants in thedevelopment and implementation processes will be prepared to lend practical support to sucha strategy. If so, they will be a very valuable asset indeed.


8/70


5

1. Introduction & strategic context

The context for this review may be summarised as improving the out-of-hospital caredelivered to patients, both in its effectiveness and in terms of the value which the ambulanceservices of Wales, with their expensively trained professional staff and extensive operational

support infrastructure, can add to the range of clinically effective patient services which NHSWales is capable of providing.

This report is the product of significant work and commitment by a large number of groupsand individuals from the Welsh ambulance services, the wider NHS and stakeholder groups.

1.1 Background to the review

In December 1996, building on work which they had carried out earlier that year, the Welsh Ambulance Policy Advisory Group (WAPAG), consisting of the chairmen and chief executivesof the four Ambulance NHS Trusts and high level representation from Pembrokeshire &Derwen NHS Trust, submitted a Project Initiation Document (PID) to the Welsh Office.

Entitled The Future of Ambulance Services in Wales , this document identified the need toreview options for the future development of Ambulance Services within NHS Wales, with theaims of improving performance in terms of patient service and cost effectiveness .

Major drivers for the review included:

the likely introduction of new emergency response time standards from April 1998; new policy direction given within A Fresh Start, creating opportunities for closer

partnerships with primary and secondary care agencies; criticism, by the Parliamentary Welsh Affairs Committee, of existing service configuration,

with particular reference to its mismatch with the new Health Authority boundaries.

Although configuration had been under discussion at WAPAG for 12 months, the approval ofthe PID by the Welsh Office provided new impetus, and a Project Team was set up, whoseterms of reference were to:

1. Consider how Ambulance Services may operate in the future in the light of technologicalchange, new ORCON standards and Fresh Start, and incorporate these views indeveloping services.

2. Develop joint proposals in the form of an Expression of Interest for consideration by WelshOffice officials and Ministers.

3. Ensure the proposals address all possible options, based on robust information and reachconclusions about genuine patient benefits and value for money.

4. Consider the risks to revenue and capital associated with each configuration and thepossible consequences on PFI 2.

5. Develop a transitional plan for the chosen option, commencing April 1998.

6. Co-operate openly in providing whatever information is necessary to develop jointproposals for the future provision of Ambulance Services in Wales, by June 1997 and anyfurther documents regarding reorganisation.

7. Provide information arising from the development of joint proposals to the NHS in WalesService Review Project.


9/70


6

Currently, there are five ambulance services in Wales, four of which are Ambulance NHSTrusts. The fifth, Pembrokeshire Ambulance Service, is an integral part of the Pembrokeshireand Derwen NHS Trust.

North Wales

South &East Wales

Pembrokeshire

WestWales

MidGlamorgan

Covering:ClwydGwynedd

Covering:East DyfedWest Glamorgan

Covering:

Pembrokeshire

Covering:GwentSouth GlamorganPowys

Covering:Mid Glamorgan

Figure 1: Current configuration of Ambulance Services

The income, staffing and population of these services is shown in Figure 2, below.

Service Income (m) Staff (w.t.e.) Population*Mid Glamorgan Ambulance NHS Trust 9.4 337.87 600,400North Wales Ambulance NHS Trust 13.7 537.48 656,000Pembrokeshire Ambulance Service 1.9 59.40 113,600South & East Wales Ambulance NHS Trust 17.5 597.00 938,000West Wales Ambulance NHS Trust 12.1 449.59 610,300Total 54.6 1981.34 2,918,300

Figure 2: Service Profile 1996/97 (Sources: WAPAG Finance Sub Group & *OutlineExpression of Interest)

1.2 Project goal and objectives

The goal of the project was to produce, by mid July 1997, a joint proposal, from WAPAG tothe Welsh Office, for revised organisational arrangements which will:

improve Ambulance and Paramedic services for the benefit of patients;

produce clear practical recommendations on the method of organising AmbulanceServices in the future;

outline the issues affecting the implementation of these arrangements;

command overall support from Ambulance Services, other NHS providers and purchasers

for its recommendations.


10/70


7

2. Project design

2.1 Overview

The project was initiated by the Welsh Ambulance Policy Advisory Group (WAPAG), acting asthe Project Board . They set up a Project Team to oversee the project and a Planning & RiskManagement Group , comprising an independent Project Leader and Project Consultants, tomanage the project and develop products including an option appraisal, risk assessment andwritten proposals. This structure is illustrated in Figure 3, below.

ExternalIndependentreview panel

Agree rangeof options

Sign off project plan

Sign off projectprocesses andmaterials

Agree projectobjectives

Approve PID

Approve termsof referencefor PL & PCs

Project Board (WAPAG)

Develop risk &benefit measures

Develop optionbriefings for theindependent Option

Appraisal Group

Put together theindependent OAG

Indep-endentoptionappraisal

Draftprojectreport

Sign off

Finalreport

Project Team

P & RMG PL* Planning & Risk Management Group

* Project Leader

Sign off

Weight risk,benefit andcost criteria

Key:

= P & RMG or PL = Project Team or WAPAG

Figure 3: Project Structure

WAPAG also set up a Communications Group to control the information aspects of theproject, and four functional groups covering Operations, Training & Education, Finance andStrategy , to produce briefing reports for the Planning & Risk Management Group (included asVolume II of this report).

The project was designed to permit high levels of independent and stakeholder involvementwhilst assuring the retention of overall control by the Project Team. Apart from theindependent membership of the latter body, stakeholders from inside and outside the

Ambulance Service were involved at other levels, including:

the functional groups; the External Independent panel against whom the criteria and rankings were tested; the Independent Option Appraisal Group who scored the options against the benefit and

risk criteria.

2.2 Project control

Overall project control was provided by the WAPAG Project Team, which comprised the chiefexecutives of the four ambulance trusts, an executive director of Pembrokeshire & DerwenNHS Trust and three independent members. The Project Team was chaired by the Chairmanof North Wales Ambulance NHS Trust. This body was responsible, through the formalapproval of plans, products and activities, for the development of detailed operationalmandates for the Planning & Risk Management Group and the Independent Option AppraisalGroup.


11/70


8

2.3 Roles & responsibilities

Project Leader

Alan Murray of Fitch & Associates - Europe, Ltd, was appointed as Project Leader. His role

(see Annexe B for detailed terms of reference) was to provide overall co-ordination and day-to-day management of the project, via membership of the Project Team and leadership of thePlanning & Risk Management Group.

His responsibilities included:

development of project plans;

review of the functional group reports;

briefing & co-ordination of the Project Consultants;

collaboration with the Project Consultants on the production of the option appraisal & riskassessment;

production of a report on the Future Provision of Ambulance Services in Wales, coveringexpected developments within the next three to five years.

Project Consultants

Neil Griffiths and Sam Darby of AEA Technology plc, were appointed as Project Consultants.Their role (see Annexe B for detailed terms of reference) was to work within the projectframework to produce an option appraisal and risk assessment which was consistent with thegoal and objectives set out in 1.2, above.

Their responsibilities included:

facilitating the development of a range of options for the future delivery of ambulanceservices;

facilitating the development of a detailed mandate from the Project Team for the benefit,risk and cost appraisal of those options;

collaborating with the Project Leader on the production of the option appraisal & riskassessment.

3. System of Care design As a prerequisite to the appraisal, it was necessary to describe the system of care required tomeet the project objectives. This comprised the set of values and services required to provideeffective, comprehensive and efficient out-of-hospital care to the patients and communitiesserved, regardless of the configuration chosen. It was the engine which drove the appraisal.

3.1 Design process

The system was designed in strict accordance with the objectives. The Planning & RiskManagement Group (P & RMG) reviewed these, along with the reports of the four functionalgroups 3 and relevant literature 4, and then developed a draft set of criterion statements for

benefit and risk measurement.


12/70


13/70


10

3.4 Cost assessment

Cost assumptions for each option were based on assessments by the WAPAG Directors ofFinance Group. These examined the transitional and implementation costs of each proposedconfiguration. Although further work needs to be done to determine the cost of meeting the

expected new performance standards, Trust Finance Directors consider that this cost isunlikely to differ significantly between the leading options. Whilst the sensitivity analyses ofthe option appraisal result (see Annexe A) make it clear that costing differences were not adecisive factor, it must be emphasised that more detailed costings will be required before anyimplementation process can begin.

3.5 Constraints

The only stated constraint in this review was that no option would be appraised if it was shownto cost more in operation, all other things being equal, than the current arrangements. It wasnot necessary to sift out any option on this basis.

3.6 Options for Appraisal

The Project Team approved the following options for appraisal:

a Modified Status Quo optiona Three Ambulance Trust optiona Two Ambulance Trust option

an All-Wales Ambulance Trust optiona Zero Ambulance Trust 6 option

The Project Team sifted out the Four Ambulance Trust option on the basis that its potentialbenefits were likely to be outweighed by the transitional risks of the reconfiguration (see

Annexe A).

An attempt was also made to identify hybrid options for inclusion in the appraisal. None wereidentified which, in the view of the Project Team, had the potential to produce worthwhileadditional benefits.

4. The Evidence of the Option Appraisal

4.1 The Appraisal Process

Scoring of the options against each criteria set was carried out at a meeting held at PencerrigHotel on 17-18 June 1997 by the Independent Option Appraisal Group . The members of thegroup are listed in Annexe A, Appendix 5, and are independent of WAPAG and the existingambulance trusts.

Scoring schemes were designed for the appraisal of benefits and risks, relative to theModified Status Quo. In other words, the process was designed to identify and estimate therisks and benefits which would arise, above and beyond those which would be expected if noreorganisation of the Welsh Ambulance services were to take place. Thus the ModifiedStatus Quo always scores zero.

The costs were scored using the same scheme as for the criteria scoring, based on thefigures provided by the Finance Group. Cost savings were scored positively (as a benefit) and

additional costs negatively. A detailed description of the scoring scheme may be found in Annexe A.


14/70


11

The group members were provided, several days in advance of the meeting, with detailedbriefings, prepared by the ambulance services, illustrating how each option might best bemade to meet the appraisal criteria. These were used during the appraisal to inform thedebate.

During the appraisal, each criterion was taken in turn and the group members were invited todiscuss how each option might perform against it. After open debate, they were then asked tocomplete a score sheet for that criterion, awarding positive benefit scores on the basis of thehighest expected benefit and negative risk scores on the basis of the highest expected risk.They were also asked to indicate where risks were predominantly transitional in nature. Thescores were added within each set, and averaged across the eleven scorers, beforeweighting.

At the conclusion of the scoring phase, the group members were asked to rank the options inorder of fitness on the basis of what they had discussed over the past two days. They werethen shown the results of their scoring, given the opportunity to discuss these and finallyasked if they wished to take back any of their scoring sheets for amendment. No requestswere made to amend any aspect of the scoring.

4.2 Appraisal Results

The two highest scoring options were the Three Trust option and the All Wales option. Thelatter configuration is self-explanatory, and the suggested configuration of the former is shownin Figure 5, below.

Mid & WestWales

Covering:ClwydGwynedd

Covering:East DyfedWest GlamorganPembrokeshirePowys

Covering:GwentSouth GlamorganMid Glamorgan

North Wales

South Wales

Figure 5: The Three Trust Option - Suggested Configuration


15/70


12

The result of the option appraisal is summarised in Figure 6, below 7.

Mainstreamweighting

Zero costweighting

Equalweighting

3 Trust All Wales 3 Trust All Wales 3 Trust All Wales

Benefits 16.7 18.0 18.3 19.7 16.9 17.3

Risks (total) -9.4 -11.6 -10.5 -12.9 -9.3 -11.9

Sum of benefits & total risks 7.3 6.4 7.8 6.8 7.6 5.4

Sum of benefits &operational risks

13.6 14.5 14.8 16.1 12.8 13.0

Savings ( over 5 yrs, excl.new ORCON & capital exp.)

3,040K 3,078K Base year is 1996/7No inflation increase included

Figure 6: Appraisal Scores for the Two Leading Options, with Sensitivity Analyses

The following features can be seen quite clearly in this table:

1. The Three Trust option scores highest when benefits are added to total risks (operationalrisks plus transitional risks);

2. The All Wales option scores highest when transitional risk is removed from the calculation;

3. These results are relatively insensitive to variations in the weightings of the criteria sets;

4. There is no significant difference in the costs of the two options.

The Independent Option Appraisal Group members were unsurprised at this outcome, whichis supported in validation checks in Annexe A. After discussion of the scores, the grouprecommended the Three Trust option.

This was not a straightforward choice. The Independent Option Appraisal Group concludedthat the All-Wales Ambulance Trust option offered more benefits than any other, but also thatit carried significant risk. Whilst they recognised that most of the risk would be transitional,they expressed concern about the capacity of the system to manage this. They were alsoconcerned about loss of access to the chief executive, in an all-Wales ambulance trust, byCHCs 8, complainants and other local interests if there was a drift towards centralisation.

However, recognising the closeness of the two options, they asked for serious considerationto be given to:

ensuring a greater degree of inter-service collaboration on suitable issues if the ThreeTrust option is selected;

ensuring maximum local accountability if the All-Wales option is selected.

In respect of the former option, the Independent Option Appraisal Group also asked for carefulconsideration to be given to the precise boundaries of the three trusts, to maximiseidentification with the services by their local populations.

It is also extremely significant that the participants in this exercise were unanimous in their

support of the process, and agreed that the outcome provided a fair and balancedrepresentation of their views.


16/70


13

5. Conclusions

5.1 The future direction of Ambulance Services in Wales

In reviewing the option appraisal results, the Project Team faced the same dilemma as theIndependent Option Appraisal Group: that the results for the two leading options were so closeas to make any firm distinction difficult. However, in the course of their debate, two distinctstrains of opinion emerged.

The first of these supported the conclusion of the Option Appraisal Group, feeling that:

the total risks of the All Wales option were greater;

there was a high probability that these risks would not be managed effectively.

The second strain of opinion supported the All-Wales option, holding that:

the benefits of the All Wales option were greater;

the transitional and operational risks could be managed, with professional support, usingrecognised techniques.

A more detailed analysis of these benefits & risks may be found in Annexe A, Appendix 7.

No consensus was achieved between these two positions, and the project team thereforemade the following recommendation:

The Welsh Ambulance Policy Advisory Groups Project Team recognises the strengths ofeach of the two leading options in its review. The Team also recognises that it is the

responsibility of the Welsh Office to determine the form of the public consultation which willfollow on this review. The Parliamentary Under-Secretary of State for Health may, however,wish to consider putting both options forward for consultation.

5.2 Improving Patient Care Services

The main direct improvements to patient care arising from this review, to which all theconstituent services have committed themselves explicitly, via the system of care mandate,are described in the following criteria sets which, whilst they may have limited apparentimportance if viewed separately, combine to produce a powerful impetus for an out-of-hospitalcare system which is comprehensive, appropriate to individual patient need and demonstrablyclinically effective, as well as being efficient, accessible, equitable and responsive.

Clinical Care

This describes the capacity of the system:

through its Paramedic Steering Committee, training and operations structures, to developand implement clinical protocols to meet changing medical practice and comply withJCALC 9 guidelines and patient care procedures which offer clear and relevant guidance toambulance staff;

through its Paramedic Steering Committee, training and operations structures, periodically to review its clinical protocols and procedures, to monitor and assure crew compliance with its approved clinical protocols & procedures;


17/70


14

to deliver the paramedic personal audit profiles required by the IHCD 10 in the AmbulanceService Paramedic Training manual, Section 9, Unit 1, para 21;

to collect, collate, present and analyse clinical data from patient report forms,comprehensively and in a form relevant to its clinical audit, quality assurance &improvement needs;

to evaluate its clinical performance by direct observation, retrospective case review &clinical audit of key topics;

on the basis of that evaluation, to review its practices, protocols and procedures andidentify the need for improvement;

to develop and implement improvement programmes, and evaluate their effectiveness;

to adapt & develop its clinical practice on the basis of relevant evidence from secondaryresearch sources;

to become a provider of clinical data to pre- hospital care research projects;

to attract funding and commission original and relevant research.

Ambulance Operation


to achieve the existing ORCON 11 emergency and urgent response standards, withoutadditional funding;

to improve its performance by developing and operating sophisticated flexible deploymentsystems (e.g., System Status Management);

to achieve a response time of 8 minutes flat to 75% of Category A emergencies 12 byfinancial year 2000/01, without compromise to its response standards in categories B andC 13 ;

to achieve a 90th percentile 8 minute standard for Category A emergencies by 2002/03;

to develop and maintain a first responder system to complete the Chain of Survival;

to collaborate effectively on operational issues with, other emergency services, other pre-hospital care providers (e.g., BASICS 14 ), other NHS providers,

both at the planning stage and on the ground;

to develop and maintain general and specific disaster plans for all reasonable risks, which - comply with current NHS central guidance, are compatible with the plans of other emergency and NHS providers, are known to and understood by all relevant staff, are exercised at agreed intervals;

to exercise command, and participate in control, of a major disaster, effectively, regardlessof location, through its management structure(s);


18/70


15

to categorise emergency calls using an approved system, employing staff trained to recognised standards, and system-approved quality processes;

to provide comprehensive radiotelephone coverage throughout its area(s);

to provide 24 hour Welsh language communications coverage;

to cope flexibly with the demand placed upon it, and with expected demand growth overthe next three years.

How the Two Leading Options Compare on Improving Patient Care Services

The Independent Option Appraisal Group used these statements as the focus for a series ofdiscussions, during which they shared their various perspectives prior to scoring each optionon benefits and risks. After full discussion of the relative merits of the five options, theyproduced a set of scores, the results of which are illustrated in Figure 7, below:

Benefit minus All Risk Benefit minus Operational Risk1. Clinical Care All Wales All Wales2. Ambulance Operation Three Trust Three Trust

Figure 7: Performance of the Leading Options on Patient Care Services Criteria

This shows that the All Wales option was perceived clearly to be better on the clinical carecriteria, whilst the Three Trust option was seen as better, unequivocally, on the ambulanceoperation criteria. The removal of transitional risk had no effect on these rankings, although itdid bring the All Wales option to within 7% of parity on ambulance operation.

Both options scored significantly higher on patient care services criteria than any of their

competitors.

Whilst it did not prove to be a major factor in the appraisal, it should be emphasised that afurther benefit of the two leading options is that, over five years, each has the potential torelease 3 million 15 (see Annexe A) from other applications to fund improvements in patientcare.

5.3 Methods of organisation

The main methods of organisation, to which all the constituent services have committedthemselves explicitly, via the system of care mandate, are described in the following criteriasets.

Accountability Issues


in keeping, inter alia , with the NHS Codes of Conduct and Accountability, to discharge itsduties of accountability to CHCs, local communities, individual patients & staff;

in keeping, inter alia , with the NHS Codes of Conduct and Accountability, to discharge itsduties of accountability to local purchasers, general practitioners and other primary careproviders, the Welsh Office and Parliament.


19/70


16

Human Resources


in a timely fashion, to attract, process & appoint recruits of suitable quality;

to harmonise pay & reward within its organisational boundaries;

to devise, negotiate and implement local pay & reward packages;

effectively to manage employee communications, industrial relations, payroll & pensions services;

to provide or procure work-based training & assessment to confirm, maintain and updatekey competences;

to provide or procure skills & other development for managers at all levels, fromsupervisory to executive;

to provide structured career development opportunities for managers in post;

to attain accreditation against an external human resources development standard;

to provide its staff with a range of staff care services, including, occupational health & welfare services, fitness maintenance programmes, stress management.

Fleet & Equipment Management


to define vehicle and major equipment specifications which optimise, effectiveness of care for patients, efficiency of utilisation, economy of operation;

to acquire vehicles and major equipment, economically, efficiently and within public sector procurement standards and, at least cost;

and to dispose of superfluous vehicles and major equipment efficiently and economically,

on the basis of a needs-orientated replacement plan;

to identify and control, within industry benchmarks, the operating costs of its vehicles andmajor equipment;

to utilise its vehicle and major equipment assets economically, efficiently and inaccordance with manufacturers guidance and relevant service standards;

to operate maintenance arrangements which ensure, within industry benchmarks, theavailability of vehicles and key equipment for their planned use.


20/70


17

Estates Management


critically to assess its estates requirement against the strategic and operational needs ofthe system;

to manage the process of investment and disinvestment in estates as dictated by thestrategic and operational needs of the system;

to maintain its existing estates stock in a condition which provides a safe & adequateworking environment for its staff;

Financial Performance


over the next five financial years, to

attract contract income sufficient to cover its operating costs, respond to fluctuations in the market, remain competitive on quality and price, exercise timely and effective financial management & control;

keep its overheads at a level consistent with the price the market is able to bear;

to plan the delivery of its capital programmes on a horizon of at least three years to enableit, inter alia , to avail itself of the option of PFI.

Executive Leadership


to recruit and retain leadership candidates of suitable calibre;

to manage its key functions effectively, at a level with which local staff, purchasers andother stakeholders can identify;

to develop strategies and plans which identify, programme related activity levels by type, anticipated costs, & hence desired income from that level of activity, planned application of that funding across budgets, the level of capital investment, reinvestment or disinvestment required, and the revenue

consequences of these plans;

to prepare and negotiate contracts with all its purchasers for signature in accordance withWelsh Office timetables;

to provide customer sensitive account management across the full range of its contracts;

to deliver services at local level to the customers specifications.


21/70


18

Strategic Positioning


to provide a sophisticated, dispatch linked, pathway management system for triagingcallers and directing them to the care pathway most appropriate to their need;

to develop an out-of-hospital care model by integrating pre-hospital and primary care, at the initial needs identification stage, at the response determination stage, at the emergency management stage, at the home assessment and care stage.

How the Two Leading Options Compare on Methods of Organisation

Once again, the Independent Option Appraisal Group used these statements as the focus fora series of discussions, during which they shared their various perspectives prior to scoringeach option on benefits and risks. After full discussion of the relative merits of the five

options, they produced a set of scores, the results of which are illustrated in Figure 8, below:

Benefit minus All Risk Benefit minus Operational Risk1. Accountabilities Three Trust Three Trust2. Human Resources Equal All Wales3. Fleet & Equipment Mgt All Wales All Wales4. Estates Management Three Trust Three Trust5. Financial Performance All Wales All Wales6. Corporate Issues Three Trust Three Trust7. Strategic Positioning Three Trust Three Trust

Figure 8: Performance of the Leading Options on Organisational Criteria

It will be noted that the Three Trust option was clearly ahead on accountabilities, estatesmanagement, corporate issues and strategic positioning, whilst the All Wales option wasclearly ahead on fleet & equipment management and financial performance. The removal oftransitional risk militated significantly in favour of the All Wales option in the case of humanresources.

Overall, with or without weightings, the Three Trust option was first when all risks wereincluded, whilst the All Wales option took the lead when transitional risk was removed.

Both options score significantly higher on organisational criteria than any of their competitors.

The Independent Option Appraisal Group asked for emphasis to be placed on the importance

of the following issues to the success of the new services:

delegation of management functions to local level;

ensuring maximum local accountability;

adoption of common corporate approaches to shared issues such as clinical standards,recruitment and procurement;

provision of locally sensitive control & communications services, including staff who cancommunicate in the local Welsh language dialect;

improvement of equal opportunities and racial equality in recruitment, so that the servicesreflect the communities they serve.


22/70


19

5.4 Issues affecting implementation

Arising from the discussions during the option appraisal and the notes made by theparticipants on their scoring sheets, the major issues affecting successful implementation ofthe chosen option are:

1. Clinical Development Issues

In keeping with the prime object of the review, the improvement of Ambulance and Paramedicservices for the benefit of patients, important points to emphasise in this area included:

the need for careful selection of future Local Ambulance Paramedic Steering Committeemembership, if reserves of goodwill are to be protected;

the need for enhanced clinician involvement in the management of services; the need to harmonise patient care equipment within, and even between services; the need to build a research evidence base, to improve future clinical training and practice.

2. Leadership & Management Issues

This was considered to be a most important area, and the areas touched on included:

the calibre of executive leadership, particularly at chief executive level, which wasconsidered to be the most important pre-requisite for success, and essential not only to thegood organisation of the services, but also to the confidence and motivation of their staff;

development of supervisory and middle managers both for their current roles and forsuccession to senior management posts;

the importance of non executive board structures which are representative of thecommunities served.

3. Community Ownership

It was felt that there was a need for an effective, stakeholder-focused communicationsstrategy on the new proposals, their implementation and their subsequent operation, if thecommunities served were to develop and retain confidence in the new arrangements.

4. Staff Issues

There was a view that attention would have to be paid to the following issues if staff were toaccept ownership of the new configuration:

the significant differences in salary rates between the current services, which will have tobe recognised and dealt with as part of the process of reconfiguration, to avoid a situationwhere there are unresolved salary rate differentials within service boundaries;

employment standards & values, which will have to be geared to the recruitment, retention,development and support of high quality staff;

the development of a team culture, where staff feel they belong, and where they know theycan rely on their leaders and their team-mates for support;

a sustained focus on communicating with staff and involving them in the decisions whichaffect their working lives.


23/70


20

5. Control & Communications Issues

This was considered to be an area of particular importance, and a number of keyimplementation issues were identified, including:

the need to ensure that maximum benefit is derived from the capital investments in controlwhich have taken, and are currently taking place in the current services;

the need to ensure that, where control arrangements become less local, local knowledge iseither preserved or replaced by safe and reliable systems with which staff are thoroughlyfamiliar;

the need to harmonise control & communication systems to ensure interoperability;

the need to build reliable communications systems for current and future needs;

the vital need to ensure full redundancy of control & communication systems as asafeguard against catastrophic failure.

6. Improving Operational Performance

The main issue identified here was the need to achieve the new emergency response timestandards against a background of financial restraint and growing emergency demand. It wasfelt that a combination of new approaches and capital investment would be required to effectthis. The need to consider staff morale issues was seen as paramount.

7. Financial Issues

The important issues here included:

the need for longer term capital planning if full advantage is to be taken of the PrivateFinance Initiative;

the need for central consideration to be given to the debt burden carried by one of theconstituent services if the new organisations are to begin their lives in a healthy financialcondition.

8. Change Management

Particularly given the predominance of transitional risk in the two leading options, it was feltthat high priority would have to be given to the development of reliable risk managementtechniques and processes if problems are to be avoided during the implementation of therevised arrangements.

5.5 Securing stakeholder support

The process by which the chief recommendation in this report has been achieved militatesstrongly, in itself, in favour of stakeholder support for the new arrangements. It has beenevident to those involved throughout that process that the ambulance services of Wales enjoythe good will and, subject to local circumstances, the support of the public, their NHScolleagues, their purchasers and their staff. All of these groups, as represented by theindependent membership of the various bodies which have participated in this complexreview, have displayed great interest in the review process and have been prepared to devoteto it significant amounts of their valuable time.


24/70


21

Building on the success of this approach, it is suggested that the Welsh Ambulance Policy Advisory Group would benefit from the continued involvement of independent stakeholders inthe various stages of the implementation process. In the view of the independentsthemselves, the value of an effective, stakeholder-focused communications strategy cannotbe over-emphasised. It is possible that the independent participants in the development andimplementation processes will be prepared to lend practical support to such a strategy. If so,they will be a very valuable asset indeed.

1 Total risks = operational risk + transitional risk: see Annexe A, Appendix 6 for detailedexplanation.2 The Governments Private Finance Initiative.3 Published as Volume II of this report.4 Including, for example:Review of Ambulance Performance Standards: Final Report of Steering Group: NHSExecutive, July 1996 .Developing Emergency Services in the Community: Chief Medical Officer, DoH England,November 1996.

5 This represents an effort to meet the appraisal criteria within the current configuration.6 Integrating ambulance services with hospital and community trusts, as in Pembrokeshire.7 Any disagreements between the figures in this table and those in Annexe A, Appendix 4,tables A4.1 to A4.3 are accounted for by rounding.8 Community Health Councils: patient representative bodies.9 Joint Royal Colleges Ambulance Liaison Committee: the national consensus body on thedevelopment of the NHS ambulance paramedic scheme.10 Institute of Health & Care Development: the awarding body for ambulance qualifications.11 The national emergency response time standards.12 Immediately life threatening emergencies.13 Respectively, serious and minor emergencies.14 British Association of Immediate Care: a GP organisation for improving pre-hospital care.15 Identified by the WAPAG Finance Sub Group.


25/70

RESTRICTED - COMMERCIALRDUP/22554001/r02 Issue 2

RESTRICTED - COMMERCIAL

Annexe A:Review of Ambulance

Services in Wales - OptionAppraisal


26/70

RESTRICTED - COMMERCIALRDUP/22554001/r02 Issue 2

RESTRICTED - COMMERCIAL

i

Review of AmbulanceServices in Wales - OptionA raisal

A report produced for Welsh Ambulance Policy Advisory Group

S DarbyN Griffiths

July 1997


27/70


iii

Title Review of Ambulance Services in Wales - Option Appraisal

Customer Welsh Ambulance Policy Advisory Group

Customer reference WAPAG

Confidentiality,copyright andreproduction

Restricted - Commercial

This document has been prepared by AEA Technology plc in connectionwith a contract to supply goods and/or services and is submitted only onthe basis of strict confidentiality. The contents must not be disclosed tothird parties other than in accordance with the terms of the contract.

File reference oa_rep_3

Report number RDUP/22554001/r02

Report status Issue 2

Neil Griffiths AEA Technology plcE5 Culham

AbingdonOxfordshireOX14 3DBTelephone 01235 464342Facsimile 01235 463389

AEA Technology is the trading name of AEA Technology plc AEA Technology is certificated to BS EN ISO9001:(1994)

Name Signature Date

Author S DarbyN Griffiths

Reviewed by S Darby

Approved by


28/70


iv

Executive Summary

This report describes an option appraisal undertaken on behalf of the Welsh Ambulance Policy Advisory Group (WAPAG) to determine the future direction of ambulance services in Wales. Theappraisal methodology was based on widely used multi-attribute techniques which were extensivelytailored to meet the specific needs of this project.

The key steps in the project included the definition of the set of options to be appraised, the definitionof criteria against which the options would be appraised, the weighting of those criteria, theidentification of specific risks and benefits associated with each option in respect of each criterion, thescoring of each option against each criterion and finally the incorporation of financial information intothe scoring. Important features of the approach were the involvement of a wide range of stakeholdersin the scoring process and the transparency of the process as a whole.

A set of project objectives was defined by WAPAG and care was taken to ensure that the appraisalprocess was aligned with these objectives at every stage. The objectives are:

Improving ambulance & paramedic services for the benefit of patients

Producing clear practical recommendations for the future organisation of services

Outlining the issues affecting implementation

Maximising overall support amongst ambulance services & other NHS providers

The options selected for the appraisal were:

Modified Status Quo (retaining the current 4 trust plus 1 service arrangement with modifications todeliver improved performance)

Three Ambulance Trusts Two Ambulance Trusts A single All Wales Ambulance Trust A Zero Trust option (ambulance services are integrated into acute or community trusts).

The outcome of the appraisal was as follows:

The Three Ambulance Trust Option achieved the highest net score (benefits minus total risks)

The All Wales Ambulance Trust Option achieved the highest benefit score but also attracted a hightotal risk score giving it a lower net score than the Three Trust Option

The three remaining options all scored significantly less than the Three Trust and All Wales options

A sensitivity analysis and a validation check were performed on the scores and were fully consistentwith the results shown above. The effect of removing the financial information scores was alsoexamined but even this did not affect the outcome.

The group which undertook the scoring (the Independent Option Appraisal Group) were shown theresults of their efforts at the end of the scoring exercise. The group, which represented the fullspectrum of stakeholder views, unanimously supported the process and felt that the outcome was afair and balanced representation of the views expressed.


29/70


v

The group did, however, make a number of points which they felt strongly should be given carefulconsideration as the process proceeds:

If the Three Trust Option were selected the potential for realising some of the benefits of scaleafforded by the All Wales option through inter-trust pooling arrangements should be explored fully.

If the All Wales Trust Option were selected the extent to which management arrangements couldbe devolved to maximise local accountability should be explored fully.

Whichever option is chosen, the boundaries of the Trusts (in the Three Trust Option) ormanagement divisions (in the All Wales Option) should be re-examined to ensure maximumcorrespondence with demographic and community boundaries thereby improving both theprospects for management efficiency savings and the support of local communities for the changes.

Beyond the key issues identified above, a number of other points of discussion within the Option Appraisal Group should also be highlighted. The group were concerned that none of the briefingsprovided to it made any reference to the need for greater numbers of women, Welsh speakers andethnic minorities within the ambulance service in Wales, or indeed any wider reference to equalopportunities. The use of part time staff was also omitted.

The issue of pay and conditions attracted considerable discussion, and the harmonisation of thesewithin the chosen option was seen to represent a source both of great opportunity and great risk.

Executive leadership and, in particular, the chief executive post(s), were seen as crucial to the successof the chosen option not only because of the need for good organisational and business managementbut also of the need to carry the support of, and motivate staff throughout the service(s).

The degree to which the views of the Independent Option Appraisal Group converged during theappraisal exercise, and the robustness of the final result are highly encouraging. The commentsexpressed by the group and listed above are intended as a positive contribution to improve the

likelihood of successfully implementing the chosen option. Therefore in addition to providing a clearbasis for a decision this appraisal process has produced a framework for the development of thechosen option and a positive atmosphere within which to take forward that option. On this basis wecan look forward to a most promising future for ambulance services in Wales.


30/70


1

Contents

1 INTRODUCTION 2

2 PROJECT DESIGN 3

2.1 PROCESS DESIGN 32.2 OPTION SELECTION 5

3 APPRAISAL PROCESS DESIGN 5

3.1 BENEFIT AND RISK ASSESSMENT CRITERIA 53.2 WEIGHTING 63.3 OPTION DEVELOPMENT 73.4 SCORING PROCESS 7

4 THE APPRAISAL 8

4.1 APPRAISAL RESULTS 84.1.1 A GGREGATED BENEFIT AND TOTAL RISKS 84.1.2 A GGREGATED BENEFIT AND TRANSITIONAL RISKS 84.2 SENSITIVITY ANALYSIS 94.3 VALIDATION 104.4 VIEWS EXPRESSED BY OPTION APPRAISAL GROUP 11

5 MAJOR RISKS AND BENEFITS 11

5.1 RISKS 115.2 BENEFITS 11

6 CONCLUSIONS 12

Appendices

APPENDIX 1 DESCRIPTION OF OPTIONS APPENDIX 2 APPRAISAL CRITERIAAPPENDIX 3 APPRAISAL TOOLSAPPENDIX 4 DATA AND COMPUTATIONSAPPENDIX 5 PARTICIPANTS IN THE PROCESSAPPENDIX 6 BENEFIT & RISK DEFINITIONS

APPENDIX 7 MAJOR BENEFITS AND RISKS


31/70


2

1 IntroductionThe Welsh Ambulance Policy Advisory Group (WAPAG), with the support of the Welsh Office, isundertaking a project to review the future of ambulance services in Wales. The focus of the project isa review of configurations for ambulance services within the Principality in the form of an optionappraisal.

To assist in this process WAPAG have employed an independent Project Leader and ProjectConsultants. The Project Leader role is being undertaken by Fitch and Associates - Europe, Limitedand AEA Technology plc are acting as Project Consultants.

The project objectives, as defined in the WAPAG Project Initiation Document, are:

Improving ambulance & paramedic services for the benefit of patients

Producing clear practical recommendations for the future organisation of services

Outlining the issues affecting implementation

Maximising overall support amongst ambulance services & other NHS providers

Based on information derived from previous studies, and following a preliminary examination of theavailable options (see Section 2.2), the WAPAG Project Team have agreed that the options to beappraised should be:

Modified Status Quo (retaining the current 4 trust plus 1 service arrangement shown in Figure 1 withmodifications to deliver improved performance)

Three Ambulance Trusts Two Ambulance Trusts A single All Wales Ambulance Trust

A Zero Trust option (ambulance services are integrated into acute or community trusts).

Summaries of the options are given in Appendix 1.

The possibilities of a mixed approach or other variations on the selected options have not beenexplicitly reviewed although views expressed by the Independent Option Appraisal Group are reportedin section 4.4.

The project methodology is described in Sections 2 and 3. The appraisal exercise involved the scoringof the options against a predetermined set of criteria. The scoring exercise was carried out by anindependent Option Appraisal Group, which is familiar with the issues involved in the provision ofambulance services. The composition of the Option Appraisal Group was selected carefully torepresent a broad range of views, but with no direct involvement in the management of the servicesunder review.


32/70


3

North Wales

MidGlamorgan

Pembrokeshire

WestWales South

& EastWales

Current Arrangement of AmbulanceServices in Wales

Figure 5

2 Project Design

2.1 PROCESS DESIGNSelecting the optimum configuration for the future of ambulance services in Wales is a complexproblem. In assessing configuration options, the important differences must be first identified and thenevaluated. Selecting the best option is not an easy task given the diverse and disparate nature of theunderlying imperatives, such as improving patient care, achieving cost-effectiveness and minimisingfinancial risks and risks to patient services. These factors cannot be compared directly, and yet, anyappraisal process must be able to deal with the trade-offs which will inevitably arise betweenconflicting objectives.

The process chosen for this appraisal is based on a Multi Attribute approach (see for example Buede &Watson Decision Synthesis, 1989). This is a formal method for dealing with problems of this typeand has been widely used in many industries. It has many advantages: a consistent basis for optioncomparison; a highly visible means of establishing the important factors and their inter-relationships; anauditable method of assisting decision making. Additionally, the sensitivity of the final option tochanges or uncertainties in the weighting factors can readily be established.

The basis of the approach is to identify the stakeholders and important attributes of the problem.Numerical weights are assigned to the attributes which reflect their relative importance. For eachoption, the attributes are scored and the aggregated or weighted score obtained. The preferred optionis the one with the greatest aggregate score (In fact, the NHS executives recommended approach toCapital Investment is a particular application of a Multi Attribute method). This process was tailored toreflect the Project Objectives stated in the WAPAG Project Initiation Document (see Section 1).


33/70


4

In order to meet the Project Objectives when using this approach, basic principles were formulated toassist in designing the Option Appraisal Process. These, together with the way that they wereimplemented are briefly described below:

Principles Implementation

Inclusion of the full range of views held bythose involved in or affected by thedecision.

Stakeholder Involvement. List ofstakeholder representatives identified andincorporated into the process

A systematic evaluation of the options toidentify the criteria (or attributes) which arerelevant to the decision and which helpdiscriminate between options.

Appraisal Criteria. Defined by the Planning& Risk Management Group with review bystakeholder representatives and agreementfrom WAPAG Project Team.

The assignment of weights to the criteriato reflect their relative importance (and theimportance of the differences betweenoptions)

Weighting. Assigned by WAPAG ProjectTeam, taking into account canvassed viewsof stakeholder representatives.

Scores for each of the criteria, eitherquantitatively as for costings or by

judgement for subjective attributes.

Independent Option Appraisal Group.Scoring by team independent of WAPAG.

Demonstrably robust decision Validation. Confirmation of OA Groupsupport for the process supplemented bysensitivity and validation checks on theoutcome

The first step in the project was option selection. Once the options had been identified the appraisalwas undertaken in a series of stages which are shown in Figure 2 below. The project stages are

described in more detail in subsequent sections.

Only one constraint was applied to the appraisal, which was that the selected option should not costmore in operation than the current arrangements, all other things being equal. Consequently, anyoption found to breach this constraint would have been sifted out.

DefineCriteriaStatements

DefineStandardsandWeights( status quo or newdefined

standard )

Assesscapacity toachievestandard(measured asimpact and likelihood)

Identifyassociatedrisks (using

prompt tree )

SCORE (normalise to status quo )

IMPACT (none - catastrophic )LIKELIHOOD (improbable - v. likely )

IMPACT (none - catastrophic )LIKELIHOOD (improbable - v. likely )

O n c e i m p l e m

e n t e d

D u r i n g t r a n s i t i o n

Project Leader ProjectConsultants QA Briefing Groups Independent OA Group

Option Appraisal Process Flowchart

Figure 6


34/70


5

2.2 OPTION SELECTIONIt was felt worthwhile to appraise as wide a selection of options as possible to ensure that possiblefuture directions for ambulance services in Wales had been reviewed comprehensively. However,options which increased the number of trusts were not considered as the existing trusts all hadbudgets of less than 17.5M and any further decrease would make the organisations financiallyuntenable.

Furthermore, the possibility of a Four Trust Option was sifted out as the benefits of moving from five tofour services were only likely to be marginal and would be far outweighed by the transitional risksarising from the reorganisation.

Discussions on hybrid options did not identify any single hybrid model which had the potential toproduce worthwhile additional benefits.

Finally, as ambulance services would be required to meet significantly higher performance standards inthe future it was felt that even the status quo option would require some modification to satisfy thesestandards.

Consequently, the options selected for appraisal were the Modified Status Quo, a Three Trust Option,a Two Trust Option, a Single All Wales Trust Option and a Zero Trust Option which followed thePembrokeshire model.

3 Appraisal Process Design

3.1 BENEFIT AND RISK ASSESSMENT CRITERIA Appraisal criteria were designed to provide a comprehensive measure of the ability of the organisationof a configuration option to meet the ideal, in terms of both benefits and risks. (The meaning of theseterms is discussed in Appendix 6)

The criteria were designed according to the set of principles shown below and criteria which could notmeet all of these principles were rejected:

Relevance - A criterion must describe a key element in the provision of ambulanceservices (currently or in the future). Specifically, the factor must assist in meeting the Project

Objectives. Other criteria, although perhaps of general relevance to an ambulance service, werenot included.

Discrimination - A criterion must be capable of discriminating between option configurations, eitherby a measurable parameter (e.g. cost) or on the basis of judgement. Criteria which could notdiscriminate were not included.

Measures and Standards - A criterion must include any important standards. Examples includeJCALC and ORCON. Failure to meet or maintain important standards is measured as a risk.

Independence - A criterion must be independent from other criteria, to minimise the degree ofsubsequent double counting in the Multi Attribute model. In this study, such correlation wasavoided by collecting criteria of similar characteristics into groups or criteria sets.

Completeness - all aspects of the ambulance services relevant to the appraisal must be describedby criteria (i.e. no important aspects must be neglected). This is important to ensure a balancedand equitable appraisal.

The Project Leader and Consultants designed a set of appraisal criteria and criteria sets which werereviewed and agreed at a Project Team meeting on 15 May 1997. The Criteria Review Panel comprised representatives of WAPAG with Dr Gareth Jones of Nevill Hall & District NHS Trust and PhilDavies of Bro Taf Health Authority to provide an independent view. The attendees are listed in

Appendix 5 and the finalised list of criteria & criteria sets are listed in Appendix 2.


35/70


6

3.2 WEIGHTING As described above, nine criteria sets (with supporting criteria within each set to define its meaning)were agreed with Criteria Review Panel prior to the appraisal process. Numerical weights wereassigned to the criteria sets.

The purpose of the weights is to reflect the relative importance of the factors described by criteria setsand the relative importance of differences between criteria sets that will arise from the optionconfigurations.

The weighting process was designed to: allow each participant to express and record his views obtain a broad consensus identify extreme views for subsequent sensitivity analysis.

Weights were assigned to the Criteria Sets associated with patient benefits by the Criteria ReviewPanel ; weights for the costings were assigned at a later date.

As the weighting process could critically affect the outcome of the appraisal it was important that arigorous and systematic approach to the exercise was adopted. To ensure that the weightings wererepresentative of all stakeholder views the weighting exercise was preceded by a consultation exerciseamongst external stakeholders to provide an initial ranking to the criteria. A workshop involving theWAPAG project team and external stakeholder representation was then held to assign the detailedweightings using a step-by-step approach and incorporating the views obtained from the consultationexercise.

There was broad agreement as to the three most important issues: Ambulance operation Clinical care Financial performance

which collectively accounted for over 50% of the total weighting, & the three least important: Accountability issues Strategic positioning Estates

which account collectively for about 15% of the total weighting. The remaining criteria (Corporateissues, Fleet & equipment mangt. & Human Resources) account for about 35% of the total weight.

The weighting sets, normalised to sum to 100, are shown in Table 1.

Criteria Set Rank Av. weight Ambulance operation 1 19.3Human resources 6 9.1Clinical care 2 18.8Fleet & equip. mgmt. 5 11.8Financial performance 3 13.1Corporate issues 4 12.7Strategic positioning 8 5.3

Accountability issues 7 6.2Estates 9 3.7

Sum 100Table 1: Criteria Weighting

(The effects of transitional and operating costs are discussed in Section 4.2)


36/70


7

3.3 OPTION DEVELOPMENTThe purpose of the option development phase was to generate a written description for each optionwhich could be subsequently scored by the Independent Appraisal Group .

A successful outcome for this phase required that:

Each of the written developments must describe the option as completely as possible with respectto potential benefits and risks.

Developments must be written consistently to facilitate subsequent scoring.

Assisted by a facilitator drawn from the Project Leader and Consultants , each of the Briefing Groups developed a single option as follows:

Configuration OptionModified Status Quo3 Trust2 Trust

1 Trust0 Trust

Briefing GroupNorth Wales Ambulance Service NHS TrustWest Wales Ambulance NHS TrustSouth East Wales Ambulance NHS Trust

Mid Glamorgan Ambulance NHS TrustPembrokeshire and Derwen NHS Trust

The groups were provided with a pro-forma to complete, which included a preliminary development ofthe option (dummy briefing) to ease the task facing the group and also to ensure consistency ofapproach. The groups were provided with a prompt card, designed to stimulate discussion of issueswhich might affect the Project Objectives and Robert Maxwells six criteria for health service planning.The prompt card was developed specifically for this project and is shown in Appendix 3. The groupswere free to amend the preliminary development text as necessary.

3.4 SCORING PROCESS

Scoring of the options against each criteria set was carried out at a meeting held at Pencerrig Hotel on17-18 June 1997 by the Independent Option Appraisal Group . The members of the group are listed in Appendix 5 and are independent of WAPAG and the management of the existing ambulance trusts.

Scoring schemes were designed for the appraisal of benefits and risks, relative to the Modified StatusQuo. In other words, the process was designed to identify and estimate the risks and benefits whichwould arise, above and beyond those which would be expected if no reorganisation of the Welsh

Ambulance services were to take place. Thus the Modified Status Quo always scores zero.

In both, the scoring was a two-part process requiring heuristic estimates of:

likelihood - how likely or probable the realisation of the risk or benefit was, relative to theModified Status Quo.

Impact - what the effect of the risk or benefit would be if it occurred, relative to theModified Status Quo.

Guidance on likelihood, risk impact and benefit estimation was provided to the Independent Option Appraisal Group. The schemes are shown in Appendix 3 and enable categorisations of:

Risks on a scale of 0 (little or no risk) to -3 (risks of great concern) Benefits on a scale of +3(optimal) to 0 (little or no probable benefit)

to be made in a consistent manner.


37/70


8

4. The Appraisal

4.1 APPRAISAL RESULTSThe five configuration options have been appraised according to the processes described above. They

were scored against the appraisal criteria and the weighted sum of the criteria set scores calculated.The weights may be taken to represent the mainstream opinion of WAPAG and their independentadvisors as to the balance of importance between financial considerations and the issues important toachieving the project objectives.

Results with and without transitional risk are described below.

4.1.1 Aggregated benefit and total risks

A summary of the results is shown in Figure 3; the scores obtained from the Option Appraisal anddetails of the calculations are given in Appendix 4.

The graph shows: benefits transitional risks total (operational + transitional) risks aggregate scores (the sum of benefit and total risk scores)

relative to the Modified Status Quo (MSQ) for each of the option configurations. The units are arbitrarybecause the scores are composed of many disparate factors.The following conclusions can be drawn from the graph:

i. For all configuration options, and unless risks are actively managed, there will besignificant risks during the transition from the Modified Status Quo to the newconfiguration. There will also be significant residual risks, though smaller, remaining afterthe transition phase, i.e. during subsequent service.

ii. The 3-Trust configuration has the greatest aggregate score and maximises the differencebetween benefit and total risk.

iii. The All Wales configuration has the greatest potential for benefit and also a greater totalrisk than the 3-trust. Most of this risk is Transitional and once in operation the All WalesOption achieves the highest net score (see next section).

iv. The 0- Trust configuration has lowest aggregate score, the greatest total risk and is theleast attractive.

v. The 2- Trust configuration has no advantages over the other option configurations.

0.0

16.714.4

18.0

9.2

0 .0 0 .0 0 .0

-9.4

-11.5 -11.6 -12.2

-3.0

7.3

3.0

6.4

-8.2-8.1-8.2-6.3

-15.0

-10.0

-5.0

0.0

5.0

10.0

15.0

20.0

MSQ 3-Trust 2-Trust All Wales 0-Trust

Scores (arbitrary units)

Benefits

Sum of benefits& RisksTrans risk

Risks (total)

Fig 3: Aggregated Option Scores Based on Total Risks


38/70


9

4.1.2 Aggregated benefit and transitional risks

Figure 4 shows the results on omitting all transitional risks; this graph indicates the relative worth ofthe configurations some time in the future, after the transitional phase has passed.

0.0

13.6

11.2

14.5

5.2

0.0

16.7

14.4

18.0

9.2

0.0

-3.1 -3.3 -3.5 -4.0-5.0

0.0

5.0

10.0

15.0

20.0

MSQ 3-Trust 2-Trust All Wales 0-Trust

S c o r e

( a r b

i t r a r y u n

i t s

) Aggregated score

Benefits

Operational risks

Fig 4: Aggregated Option Scores Based on Operational Risks

Note: This graphignores theeffect oftransition riskswhich could beconsiderable

The following conclusions can be drawn from the graph: The All Wales configuration has the potential for greatest benefit. The All Wales configuration has the highest aggregate score (benefits plus operational risk)

It should be noted that after considerable discussion, the Option Appraisal Group came out in favour ofthe 3-Trust configuration. The reason for this was that, although the Group recognised that the All-Wales configuration offered the potential for greatest benefit, they were concerned that it also carrieda greater degree of total risk (relative to the 3-Trust), particularly during the transition period.

4.2 SENSITIVITY ANALYSIS

The range of weightings obtained at the Criteria Weighting and Review meeting indicated that therewere differences in the panels views on the relative importance of the criteria sets. (There was broadagreement as to the most and least important criteria sets at the meeting of 29 May 1997. However,some participants would have preferred greater or lesser weight applied to certain of the sets.)

Another factor requiring evaluation is the effect of transitional and operational costs on the conclusions.

In order to test the sensitivity of the conclusions to these considerations, further analysis wasperformed. Two additional calculations are described below with details in Appendix 4.

a. Sensitivity analysis 1 : Cost estimates for the options were provided by the WAPAG Directors ofFinance Group. Their aim was to provide estimates of:

1. The operating costs for each option2. The costs of effecting the transition3. Additional costs of meeting the new performance standards

The estimates are detailed in Appendix 4. Whilst further work needs to be done to determine thecost of meeting the new performance standards, Trust Finance Directors consider that this cost isunlikely to differ significantly between the leading options.

The results are listed in Appendix 4 (Table A4.5), where it can be seen that the effect of includingoperational and financial costs does not affect the conclusions of Section 4.1

b. Sensitivity analysis 2: All criteria sets were given equal weight, thus removing differences in

importance between criteria based on patient care.


39/70


10

The weighting set effectively reduces the importance of certain patient benefit criteria sets withrespect to others. Costs were included in the analysis. Again, qualitative differences betweenoptions are preserved and the initial conclusions are unaffected.

The explanation for the insensitivity of the model to variations in the weightings is that, in general,

individual criteria set scores favour the 3-Trust option, followed by the All Wales option. The results ofthe multi-attribute model are largely independent of all the weighting schemes of practical interest. Inother words, the results of the study are insensitive to all but extreme opinions regarding relativeweightings.

4.3 VALIDATIONOne of the strengths of the process is the opportunity given to the Independent Option Appraisal Panelto discuss & modify their opinions in the light of others experience & expertise. Ideally, as the processproceeds, errors of perception will be corrected & at the end of the study a consensus will be obtained.

As a check that the process was proceeding as planned, a straw poll was taken at the end of thesession immediately prior to the calculation of the final weighted option scores. Participants were

asked to rank the options in order of fitness on the basis of what they had discussed over the past twodays. The results are shown in Table 2 below:

1st 2nd 3rd 4th 5thScorer 1 All Wales 3-T 0-T 2-T MSQScorer 2 3-T All Wales 0-T 2-T MSQScorer 3 3-T All Wales 2-T 0-T MSQScorer 4 3-T All Wales 2-T 0-T MSQScorer 5 0-T MSQ All Wales 3-T 2-TScorer 6 3-T 0-T MSQ All Wales 2-TScorer 7 3-T 2-T All Wales MSQ 0-TScorer 8 MSQ All Wales 2-T 3-T 0-TScorer 9 All Wales 3-T 2-T MSQ 0-TScorer 10 All Wales 2-T 3-T MSQ 0-TScorer 11 All Wales 2-T 3-T MSQ 0-T

Table 2: Straw Poll Validation Check

Five of the eleven scorers placed the Three Trust Option in first place, and four of them the All WalesOption.

The table of statistics below (Table 3) shows the frequency with which the scorers ranked options asbeing first during the scoring sessions. (The totals exceed the number of criteria because on manyoccasions options were ranked equal first.)

MSQ 3-T 2-T All

Wales

0-T Total

Scorer 1 0 6 6 28 4 44Scorer 2 0 20 10 19 8 57Scorer 3 0 22 18 22 11 73Scorer 4 0 25 18 21 9 73Scorer 5 0 6 3 13 23 45Scorer 6 2 28 7 16 11 64Scorer 7 0 25 5 6 1 37Scorer 8 3 11 14 24 11 63Scorer 9 0 12 15 27 6 60Scorer 10 3 17 17 26 10 73Scorer 11 1 13 18 29 7 68

Table 3: Scoring Pattern Validation Check - No. times configuration scored first


40/70


11

Once again the Three Trust and All Wales options win, showing good agreement between theoption(s) that scorers perceived to be best and the options that they marked for individual criteria asbeing best. This demonstrates that the process reflected the scorers views.

This confirms that the outcome of the appraisal represents a true convergence of views and not simply

the middle way in a widely diverging set of views.

4.4 VIEWS EXPRESSED BY OPTION APPRAISAL GROUPDuring the scoring exercise the Independent Option Appraisal Group made a number of points whichthey felt strongly should be given careful consideration as the process proceeds. The key points theywished to highlight were:

If the Three Trust Option were selected the potential for realising some of the benefits of scaleafforded by the All Wales option through inter-trust pooling arrangements should be explored fully.

If the All Wales Trust Option were selected the extent to which management arrangements couldbe devolved to maximise local accountability should be explored fully.

Whichever option is chosen, the boundaries of the Trusts (in the Three Trust Option) ormanagement divisions (in the All Wales Option) should be re-examined to ensure maximumcorrespondence with demographic and community boundaries thereby improving both theprospects for management efficiency savings and the support of local communities for the changes.

In addition, the group asked that several other points of discussion be recorded in the appraisal report.

The group were concerned that none of the briefings provided to it made any reference to the need forgreater numbers of women, Welsh speakers and ethnic minorities within the service in Wales, orindeed any wider reference to equal opportunities. The use of part time staff was also omitted.

The issue of pay and conditions attracted considerable discussion, and the harmonisation of thesewithin the chosen option was seen to represent a source both of great opportunity and great risk.

Executive leadership and, in particular, the chief executive post(s), were seen as crucial to the successof the chosen option not only because of the need for good organisational and business managementbut also of the need to carry the support of, and motivate staff throughout the service(s).

5 Major Risks and Benefits

5.1 RISKSDuring the Optional Appraisal process, the group were invited to record, on the scoring sheets, their

views. Most of the comments made by participants related to risk issues. Those made against highscoring criteria (those with Category -3 scores for risk) are summarised in Appendix 7 for the AllWales and 3-Trust configurations. These can be viewed as the issues which the Group felt moststrongly about and can be taken to indicate the risks of greatest significance.

5.2 BENEFITSIn Appendix 7, the aggregate score (Net Benefit) for each of the Appraisal Criteria Sets is given for theleading options (3-Trust and All Wales).

Overall Net Benefit (Benefit -Total Risk) and Long Term Net Benefit (Benefit - Operational Risk) areshown.

These data allow comparison of the options against each of the sets, showing where each option was

judged to be superior.


41/70


12

6 ConclusionsThe appraisal demonstrates clearly that the Three Trust Option (see f

wapagfinrep

Documents