electronic decision support tools (edst) …...department of health’s funding agreement for...

ELECTRONIC DECISION SUPPORT TOOLS (EDST) FOR PATHOLOGY REQUESTING: PATHSUPPORT

FINAL REPORT

PREPARED FOR THE

DEPARTMENT OF HEALTH

BY

THE ROYAL COLLEGE OF PATHOLOGISTS OF AUSTRALASIA

28 MARCH 2015

The Royal College of Pathologists of Australasia

ABN: 52 000 173 231

Durham Hall

207 Albion Street

Surry Hills NSW 2010

Ph: 02 8356 5858 Fax: 02 8356 5828

FINAL PathSupport Final Report

i

Executive Summary

The purpose of this Final Report is to advise the Department of Health of the work completed in accordance with the reporting requirement of Item F.6 of the Schedule of Department of Health’s Funding Agreement for Electronic Decision Support Tools (EDST) for Pathology Requesting: PathSupport- (executed 4th April 2014):

• Summary of findings for information gathering for GPs, Desktop Vendors, Pathologists and Laboratories.

• Design principles and possible development options and the feasibility of leveraging existing pathways solutions compared with building such solutions.

• Indicative cost, benefits and feasibility of EDS for pathology requesting • Summary of project activities and potential next steps.

The PathSupport Project builds upon the outcomes from the Electronic Decision Support Scoping Project submitted by the RCPA to the Department of Health in July 2013.

Summary of Findings for Information Gathering

The initial information gathering included four GP Focus Groups with 36 contributors, the CEOs of six Desktop Vendors, several pathologists and three laboratory directors. The information gathering focused on identifying the issues the GPs, the pathologists and the pathology laboratories want solved. These issues focused on the context of GPs using their desktop solutions for the initial requesting of pathology.

There were 14 high impact issues identified within these groups, and two of the most important form the focus of this report. The information gathering also assisted with identifying the key characteristics GPs required of a pathology requesting solution that might address the above issues. Input was also sought from academics specialising in this area to inform the information gathering approach and to rationalise these findings.

Further information gathering was undertaken using the proposed solution to obtain input. The information gathering included a webinar with six GP contributors, a meeting with the CEOs (or their delegates) of six Desktop Vendors, meetings with a cross section of Laboratory CIOs, a meeting with five members of the Australian Association of Practice Managers, a teleconference to discuss Clinical Governance requirements and a Laboratory Impact workshop.

Design Principles and Possible (Solution) Development Options

The project team used the key solution characteristics from the information gathering and the design concepts of the EDS Scoping Project to conclude that it was not effective to leverage the existing pathways solutions. Instead the project team identified a solution that leverages existing pathology modules in GP desktop solutions as the preferred option for further investigation.

The proposed solution encourages (but does not mandate) the GP to firstly enter the clinical context for the pathology being requested (e.g. Diabetes Monitoring). The solution then uses that clinical context to prepopulate the request with an endorsed set of necessary tests and also provides further information to help the GP request appropriately. The proposed solution addresses the three most important issues identified by the pathologists and the GPs. At the same time it satisfies the key solution characteristic requested by the end users

FINAL

ii

PathSupport Final Report

- including “not negatively impacting GP workflow” and providing “the ability for GPs to manually override the suggested tests”.

In the proposed solution the desktop pathology modules need to be modified and provided with sets of clinically relevant pathology tests that are updated through a RCPA led governance mechanism on a regular basis. The recommended approach is to leverage the scientific content of the RCPA Manual which includes the regular maintenance of the Manual to address currency. As a by-product of this process key parts of the “human readable” content of the RCPA Manual (and other sources) could be translated into “machine readable” rules1 that would integrate with desktop pathology modules.

The project would focus on the initial necessary2 tests that should be requested by GPs for a select group of clinical contexts. At the same time the project would capture and provide further information for the GP including links to key resources such as the RCPA Manual (and possibly the RACGP Red Book). If required, the further information could include suggestions regarding what tests are unsuitable, and link to the relevant health pathway algorithms that are being developed by several primary care organisations3.

The rules development (and their ongoing maintenance) required to be undertaken to deliver the solution would be overseen by a governance function that includes representatives from the RCPA, at least one key GP organisation (the RACGP has indicated a desire to participate in this activity), and representatives from other Specialist Colleges/organisations as required. Updates to the rules would be provided to Desktop Vendors for incorporation into their products.

Indicative Costs, Benefits and Feasibility of EDS for Pathology Requesting

One of the contractual requirements of the PathSupport Project is to “provide assistance in capturing the benefits and costs of the decision support solutions as input to the (subsequent) development of a business case to encourage the uptake of electronic decision support tools.”4 In addition to capturing the benefits and costs, the project has also examined the feasibility of the proposed solution.

The proposed solution outlined above should, in summary, could deliver the following benefits:

• Increased requesting of necessary tests with the potential of improved patient outcomes and experience (e.g. avoiding second requesting of pathology, and the early detection of illness).

• Decreased requesting of unsuitable tests with the potential to reduce patient inconvenience, overdiagnosis and inappropriate use of resources.

1 A rule in this context is a mechanism used to look up information using parameters. 2 The Project has considered that “appropriate requesting of pathology tests” is defined as: requesting the maximum number of “necessary” tests and a minimum number of “additional” or “suitable tests”. 3 The organisations involved were Medicare Locals. These are potentially in transition to Primary Health Networks 4 Department of Health PathSupport Funding Agreement 4th April 2014 – page 2 detailed activities.


iii

• Increased provision of clinical context for pathologists that would enable them to

provide improved reporting which in turn would result in improved patient care and refinement of appropriate next steps in patient care.

The quantification of these benefits has not been undertaken as part of this Project. To obtain benefits and other important information the Project recommends undertaking a Proof of Concept (POC), to provide input to the Business Case for a national implementation.

The costs of undertaking the POC project to quantify the benefits and to validate the solution design and acceptability is estimated to be $495,672, and includes:

• Evaluation of GP changes in requesting patterns and other GP impacts;

• Evaluation of the implications for pathologists and pathology laboratories;

• Rules creation (for two clinical contexts), expert input and related governance;

• Assessment of two models of knowledge capture, hosting and access;

• Software development for two participating Desktop Vendors to modify their pathology requesting modules and to capture evaluation information;

• Management of the project; and,

• Possible engagement of external specialist evaluators (additional costs to undertake evaluation of between $50,000 and $317,000 are not included in the estimate provided above).

The POC would provide significant input for the Business Case for a national roll-out. As an initial guide, the cost of implementing the proposed solution nationally is estimated to be $1,136,400, and includes:

• The cost of the RCPA led governance functions associated with the development and maintenance of the rules.

• The cost of creating the basic model for hosting and distributing those rules to the Desktop Vendors. A more comprehensive option of hosting and accessing the rules is also included in the report. Evaluation of the two options would be need to be undertaken during the POC.

• The cost of supporting modifications of Desktop Vendors’ pathology modules.

• The cost associated with RCPA management of the project.

There would be additional costs related to implementing the solution in GP practices and promoting the use of the solution, which would be estimated during the POC. Ongoing costs of the national rollout are estimated to be between $325,120 and $632,920 per annum (depending upon the selection of the preferred model used to host the rules).

The feasibility of the solution is considered moderate to high. Key considerations that have been examined include:

• The concerns of laboratory directors in relation to the implications of the solution. Preliminary indications (from a limited set of clinical contexts investigated) are that the impact on resources may not be significant. The POC will assist in evaluating impact on laboratories.

• The risk that the cost of development and ongoing support costs by the Desktop Vendors may be prohibitive. Indicative costs have been prepared based upon


iv

preliminary solution design. These costs necessitate further validation. These indicative costs are detailed in this report.

• The reluctance of GPs to adopt the new solution based upon concerns regarding the possible impact of their clinical independence and changes to workflow. The GP feedback on the solution indicated this concern can be addressed by a non- prescriptive design.

• The clinical governance required to manage the clinical risk associated with the solution may be considerable. The preferred approach would be for this body of work to be addressed by creating an RCPA-led EDS Clinical Governance committee separate to but complementing the RCPA Manual Editorial Committee . The estimated cost for this is detailed in this report.

Summary of Project Activities and Potential Next Steps The key project activities have been highlighted above and are detailed in Appendix A of this Final Report.

The Project Team identified the following potential next steps:

1. Undertake a Proof of Concept (POC) to gauge the acceptability of the preferred solution. This would involve a small number of Desktop Vendors at a defined number of GP practices in cooperation with a small number of pathology practices. The POC would also trial the generation, distribution and testing of the suggested request sets. The generation of the request set would build on the knowledge previously developed as part of the RCPA’s Pathology Decision Support tools (PDSTs), however the format of the PDST’s presented in the RCPA Manual is likely to change. The POC would include an evaluation of the three benefits highlighted from the information gathering phase. Furthermore, it would look to refine the key costs identified and engage with the Pathologists, Laboratories, Desktop Vendors and GPs to assess the feasibility of the solution. The aim is that this additional work could form the basis of a business case for a national implementation.

2. The next phase of the Project could coordinate the pathology requesting guidance provided to GPs. This could be achieved by working with the organisations that are currently developing primary care pathways that include guidance in pathology requesting. The specific aim is to ensure that pathology information (that defines the initial requesting of necessary tests) in the primary care guidelines is consistent with the information in the RCPA Manual and can be referred to by the GPs when using the proposed solution in this report. The content developed in the current PDSTs could be used during the initial steps in this activity. However, a recommendation from this project is that the PDSTs format should be reviewed (to cater for an audience other than primary care) and their content adapted for use for both the PathSupport Knowledge Services (for definition see later under Proposed Solution) and the primary care pathways.

3. Prepare a funding submission to enable the RCPA in collaboration with other stakeholders to progress the above steps.


v

PathSupport has identified a proposed solution that has the potential to significantly progress the key objective ; that is, “the purpose of the EDST is to optimise the appropriate use of pathology testing to enhance high quality diagnosis and treatment in patient care”5

PathSupport represents only the first part of the second phase mentioned in Clause 35 of the Pathology Funding Agreement. The subsequent part of the second phase incorporates the “significant progress on the development of Electronic Decision Support tools” and “assistance in the development of a business case to encourage the uptake of these tools by requestors”. The RCPA will undertake a funding submission to progress the next phases.

5 Department of Health project agreement page 1 “Program Description and Objectives”


vi

TABLE OF CONTENTS Executive Summary ................................................................................................................. 1

Abbreviations ...................................................................................................................... 1

1 Introduction and Project Overview .............................................................................. 2

1.1 Background and Context ............................................................................................. 2

1.2 Project Aims and Objectives .................................................................................... 3

1.3 Project Scope and Approach ....................................................................................... 3

2 Summary of Findings from Information Gathering .................................................... 5

2.1 Monash University ................................................................................................... 5

2.2 UNSW / AIHI ................................................................................................................ 7

2.3 University of Sydney / FMRC ....................................................................................... 9

2.4 NPS ......................................................................................................................... 9

2.5 Pathology Sector Approach ................................................................................... 11

2.6 Pathology Sector Findings ......................................................................................... 12

2.7 Desktop Vendor Industry Approach ........................................................................... 14

2.8 Desktop Vendor Industry Findings ......................................................................... 14

2.9 General Practitioner Information Gathering ............................................................... 17

2.10 Findings (Issues the GPs Want Solved) ................................................................. 18

2.11 Findings (Proposed Solution) ................................................................................. 20

3 Design Principles and Development Options .............................................................. 22

3.1 Approach ............................................................................................................... 22

3.2 Findings ................................................................................................................. 23

4 The Proposed Solution .................................................................................................. 27

4.1 Confirmation of Scope ............................................................................................... 27

4.2 Further Issues from the Information Gathering Findings ......................................... 27

4.3 The Detail of the Proposed Solution ....................................................................... 28

5 Indicative Benefits, Costs and Feasibility ................................................................. 35

5.1 Benefits ................................................................................................................. 35

5.2 Indicative Costs for Subsequent Business Case ....................................................... 37

5.3 Feasibility .............................................................................................................. 40

5.4 Alignment of EDS with other RCPA Projects .......................................................... 41

5.5 Analysis of Alternatives .......................................................................................... 41

6 Potential Next Steps................................................................................................... 43

6.1 Undertake a Proof of Concept ................................................................................... 43


vii

6.2 Undertake Formal Alignment of Guidelines and Pathway ....................................... 43

6.3 Undertake a Funding Application ........................................................................... 43

7 Management of Funds ............................................................................................... 45

Appendix A: Final Report Deliverables and Activity Performance Indicators ............... 46

Appendix B: Project Governance Reporting ................................................................... 49


1

Abbreviations

AIHI Australian Institute of Health Innovation

BEACH Bettering the Evaluation and Care of Health, University of Sydney

CDS Clinical Decision Support

CHIA Certified Health Informatician Australasia

DoH Department of Health

DSS Decision Support Systems

EDS Electronic Decision Support

EDSS Electronic Decision Support Systems

FMRC Family Medicine Research Centre

MSIA Medical Software Industry of Australia

NEHTA National E-Health Transition Authority

NPS NPS MedicineWise

PCEHR Personally Controlled Electronic Health Record

PDST Pathology Decision Support Tool

PITUS Pathology Information Terminology and Units Standardisation

POC Proof of Concept

PSC Project Steering Committee

RACGP Royal Australian College of General Practitioners

RCPA Royal College of Pathologists of Australasia

UNSW University of New South Wales.


2

1 Introduction and Project Overview

1.1 Background and Context The Electronic Decision Support for Pathology Requesting: PathSupport Project arises from the Pathology Funding Agreement of April 2011, specifically clauses 34 and 35 which reference the use of Electronic Decision Support (EDS) to assist in quality requesting of pathology by GPs through their desktop software. This Project is a continuation of the EDS Scoping Project which was submitted July 2013,

This report addresses the “Final Report” requirements6 described in the PathSupport Funding Agreement between the Commonwealth and the RCPA executed 4th April 2014. Namely

• Summary of findings for information gathering for GPs, Desktop Vendors, Pathologists and Laboratories.

• Design principles and possible development options and the feasibility of leveraging existing Pathway solutions compared with building such solutions (“build or buy”).

• Indicative costs and benefits and feasibility of EDS for pathology requesting.

• Summary of project activities7 and outlining potential next steps.

A request was submitted to the Department of Health to extend the activities of the project to 28 March 2015, and approved in Deed of Variation 2. This extension allowed the Project to undertake some activities that enabled the College to provide greater detail relating to several areas including addressing feasibility benefits and detailed costs to better assist in the development of a subsequent business case. An Interim Final Report was provided to the Department of Health dated November 2014.

Early deliverables in the project included a Project Plan (dated 30th April 2014) and a PathSupport Performance Report (dated 4th June 2014). Since the provision of those deliverables the Project has engaged in extensive information gathering and consultation phases. These activities have been extremely valuable in assessing user requirements and refining suitable solution delivery options. These activities included:

• Research stream activities to complement the research undertaken during the previous Electronic Decision Support Scoping Project (2013). This additional stream focused on the work undertaken by Monash University, UNSW/AIHI, the NPS and the Sydney University/FMRC in order to understand why EDS had not been successful in the past; to help shape the subsequent information gathering and to inform the design of the solution and implementation approach.

• The proposed solution identified during this project effectively removed the need for any future RCPA projects to undertake software development. Consequently the buy or build activities were refined. This refinement focused on linking to pathway

6 Refer to Appendix A for the detail of the project compliance with these reporting requirements 7 Refer to Appendix A for the detail of the projects compliance with the detailed Activity Performance Indicators


3

solutions as information sources rather than using the pathway solutions as an interface to the RCPA pathology request sets.

The engagement with pathologists regarding further clinical content of the rules was refined to use the expert clinical and strategic input of the pathologists on the Project Steering Committee. Their input was further complemented by the two pathologists who were interviewed in their capacity as laboratory directors.

1.2 Project Aims and Objectives The main objective of this project is to “optimise the appropriate use of pathology testing to enhance high quality diagnosis and treatment in patient care”8.

This objective is to be considered in the context of the implementation of EDS tools and the take-up of electronic pathology requesting.

1.3 Project Scope and Approach The project objectives and scope included the following:

1. An assessment of GP attitudes to EDS tools.

2. An assessment of the capacity of (and flow-on costs to) GP Software Desktop Vendors and Pathology Laboratories Information Systems to integrate EDS tools.

3. Evaluate the existing decision support systems. 4. Provide a summary of policy implications arising from the adoption of EDS tools.

5. Identification of next steps including implementation options.

The project approach leading up to the delivery of the Interim Final report is illustrated in the following diagram.

8 Department of Health Project Funding Agreement page 1 “Program Description and Objectives


4

Subsequent to the Interim Final Report a set of five activities were undertaken namely:

1. A Desktop Vendor workshop with several aims including an assessment of the proposed solution feasibility, obtaining cost estimates for a national roll out and preparedness to participate in a POC.

2. A GP Focus group to gauge acceptability of the possible solution.

3. A Pathology Laboratory impact workshop to gauge resource implications of the solution on laboratories.

4. A GP practice manager meeting to establish the practical implications of the solution. 5. A teleconference to discuss clinical governance considerations to assist with defining

the approach for capturing clinical knowledge and to estimate the workload necessary to support the process.


5

2 Summary of Findings from Information Gathering

As per the above diagram, a number of information gathering activities occurred with other stakeholders prior to the GP information gathering phase. The outcomes from those other stakeholder meetings assisted with the GP activities. The organisations and sectors engaged are listed below:

• Monash University • UNSW / AIHI • University of Sydney / FMRC • National Prescribing Service (NPS) • Pathology Sector • Desktop Software Vendors

This work complemented the insights and guidance from the members of the Project Steering Committee and the research previously undertaken during the EDS Scoping project in order to understand why EDS had not been successful in the past and to inform the design of the solution and implementation approach.

The input from the Health Informaticians on the Project Steering Committee has leveraged the CHIA Practitioner’s Guide (Chapter 6) to reduce the ambiguity of terms. In particular PathSupport Project has not focused on the abstract areas of “Clinical Reasoning” and “Clinical Judgment” but rather on the concrete area of “Clinical Decision Making”.

2.1 Monash University A Monash University PhD student, submitted a thesis in August 2008 called “Appropriate Use of Pathology Tests by General Practitioners in Australia: Towards a Blueprint for Intelligent Decision Support”. This study was supported by Dorevitch Pathology and the RACGP. In addition to contributions from academic supervisors, the work leveraged the contribution of an expert pathologist in the area of decision support who was the study’s representative of the Australian pathology industry. The PathSupport Project Team communicated with that expert extensively during this project to help guide the relevance of the study to the PathSupport Project.

The following is a summary of the Monash University study and how it applies to the PathSupport Project:

Terminology Impacting PathSupport Information Gathering

The Monash University study provided an important definition regarding the “appropriate requesting of pathology tests” specifically: requesting the maximum number of necessary tests and the minimum number of unnecessary 9 or unsuitable tests as indicated in published clinical guidelines.

The above definition provides three categories of requested tests which was used during the information gathering and may be useful in defining rules that underpin the human readable information relevant for the solution design. In effect there are tests that:

9 Subsequently the project used the term “additional” tests rather than “unnecessary” tests.


6

1. Should be requested (necessary10 tests).

2. Should not be requested (unsuitable tests).

3. May be requested (unnecessary11 tests) - In a theoretical setting these tests are unnecessary. However, in a practical setting, it may be appropriate for the GP to request these tests based upon their requesting style and patient context (e.g. comorbidities).

This last category reinforces the need for the GP to have flexibility in using the endorsed pathology test sets.

The study identified the variables that could be used to identify a clinical context. Two of these variables were used during the information gathering phases: Clinical Problems (e.g. Diabetes, Lipid Disorder Abdominal Pain) and the reason for which GPs requested tests (e.g. Monitoring, Diagnosis, and Screening).

Key Findings Impacting the Information Gathering

The approach used was to analyse large amounts of pathology data (over 1.5 million requests) to find out “what kinds of patients are consuming what types of pathology tests?” and “what are the clinical purposes for which GPs request these pathology tests?”

Using that data the study then undertook an online survey (with the help of the RACGP) of 85 GPs to answer a number of questions including: “Do the tests requested by GPs in commonly encountered clinical scenarios comply with clinical guidelines?”

Results of the survey showed the following:

The variability of alignment with guidelines based upon a set of 20 clinical scenarios12

was significant. The two most noticeable results were:

• 71% of the monitoring of high user diabetics matched the guidelines albeit with a significant portion of additional and/or unsuitable tests requested.

• 69% of diagnostic testing for high user patients with pregnancy problems missed any of the necessary tests.

All of the other alignment cases identified that “overlap” (i.e. some necessary tests were not requested but additional or unsuitable tests were requested instead) was by far the most common category of alignment with guidelines (typically well over 60% of GPs fell in this requesting category).

Of the tests requested (in the 17 scenarios where pathology should be requested): 41% were necessary and matched the guidelines, 47% were unnecessary (additional) and 12% were unsuitable.

10 During the information gathering with the GPs, several initially used the general term “appropriate” however they also supported the use of the term “necessary” as being more precise in the context of rigorous sets of endorsed tests. 11 Subsequently the project used “additional” rather than “unnecessary” 12 Of the 20 scenarios there were three that had no necessary tests required


7

The study referenced earlier work13 that had identified “human readable” information available through the “RCPA Manual of the Use and Interpretation of Pathology Tests” as being well regarded by at least some GPs but not extensively used.

2.2 UNSW / AIHI The PathSupport Project Team met with the Director of the Centre of Health Informatics at the Australian Institute of Health Innovation (AIHI) of the University of NSW on the 5th August 2014 regarding the PathSupport Project and GP decision support.

The meeting addressed the following topics/questions. Many of the responses endorsed the views held by individual members of the Project Steering Committee.

1. Why has Electronic Decision Support failed even though we have been studying it

for over a decade?

The wider GP decision support has NOT failed and is working in certain areas only (such as prescribing) and that solutions will not succeed if they do not meet the needs of the users. To that end the PathSupport Project must be clear on who it is building the solution for. If the PathSupport Project is building the pathology requesting solution for GPs and they don’t want it, the initiative is not going to be successful. If the PathSupport Project does something as simple as improve GP pathology requesting workflow with a few key strokes then there is a better chance of success. Decision support has been hampered by the diverse desktop programs available to GPs in Australia and a lack of uniformity.

ii. Are there any suggestions regarding the approach used to implementing the solution?

Starting small would demonstrate value and allow to then further refine this approach. A pilot project with a small and simple set of rules, has the advantage of providing a simple platform for subsequent rounds of development. This first round is vital to measure the levels of use and adoption to promote the advantages of the solution for up take, and allows first round developers to practically apply the “rules” and learn from the process.

Suggested the solution should use the term: “request sets by condition which have been defined / endorsed by the RCPA”. Use of simple language may help convey the approach from the start.

iii. Guidance regarding establishment and ongoing maintenance of request sets by clinical context.

Suggest that the Project leverage the existing human readable RCPA Manual and algorithms/flowcharts to develop a machine readable product containing the rules that underpin the appropriate human readable product.

Ultimately, the Project should aim to provide machine readable rules for as many clinical contexts as can be safely written.


8

13 Healthcare Management Advisors 2001


9

The PathSupport Project should aim to incorporate a maintenance structure from the start, as even though the core rules may be foundational, changes will be required in terms of refinement.

Also suggest that there should be obligations put on the release of the rules to the Desktop Vendors. An example of these obligations may be a commitment that the software is upgraded in an appropriate time frame.

iv. Guidance regarding role of the Desktop Vendors in the user interface.

Suggest that the user interface should be developed by the Desktop Vendors. The data quality in the desktop solutions is often limited. It should be up to the Desktop Vendors to guide the characteristics / triggers to entering the rules.

For instance they may connect to the appropriate rules:

• Once the pathology module is manually activated, and then within that module use the local data and user interface to select the appropriate clinical context. And possibly…

• When the software automatically detects certain clinical context that indicates that pathology tests should be requested.

Consideration needs to be given to practicalities around and governance for the process to define the rules in terminology currently used by each Desktop Vendor even if the data quality was high. This area has two components namely the clinical contexts that identify the set of rules and then within that set of rules the individual parameters that define the logic.

For instance the process may provide a set of rules under the heading Type 2 Diabetes whereas the Desktop Vendor software may use other terms.

Rigorous coding of the pathology request sets in the RCPA rules should be used in developing the Desktop Vendor solution.

The process should be as unambiguous as possible in defining the rules to ensure Desktop Vendors are clear in developing the software solution.

v. Guidance regarding the Hosting of Rules

Suggest engaging external third parties to assist with developing and hosting the request sets.

Other Meeting with AIHI

In February 2014 the PathSupport Project Team met with another representative of the UNSW/AIHI.

The AIHI representative provided insights based upon his experience as a GP and 30+ years involvement in decision support systems. This input supported the views of the Project Steering Committee and influenced the information gathering and analysis. Specifically:

• The importance of having similar user interfaces for pathology, diagnostic imaging

and prescriptions. • GPs should start with a simple and focused model, then increase the functionality. • There should be at least two categories of GP – expert and novice (and possibly an

average/middle/typical category). And two levels of guidance – simple and complex.


10

• A GP has the art of “bundle thinking”14 when undertaking a patient consultation. At some time during that process there is a trigger that results in an entry into a defined and robust path.

2.3 University of Sydney / FMRC

Introduction In March 2013 a Senior Researcher from the Family Medicine Research Centre (FMRC) submitted a thesis called “In Evaluation of pathology ordering by general practitioners in Australia” to fulfil the PhD requirements at Sydney School of Public Health Sydney Medical School University of Sydney.

The PathSupport Project Team communicated with the FMRC Senior Researcher extensively during this project to help guide the relevance of the study to the PathSupport Project.

The following is a summary of the University of Sydney /FMRC work and how it applies to the PathSupport Project:

The study used the term “purposes” for requesting pathology and suggested that the percentage break-up of the purposes was diagnosis 31.6%, monitoring of illness or drug therapy 40.6%, and screening 15.1%. The concept of “purpose” has been leveraged in the PathSupport Project information gathering and specifically the three terms Screening, Diagnoses and Monitoring appear in the subsequent sections of this report.

There is a significant concentration of clinical contexts (there were 22 identified) that collectively accounted for more than half of pathology requesting.

A significant conclusion of the study was that “for the ongoing management of chronic problems, pathology testing guidance required improvement. Australia has an ageing population and therefore chronic problem management and the testing associated with it will inevitably increase. Improved guidance regarding pathology testing in chronic problem management could help support GPs’ appropriate ordering in this area”.

The study examines in detail six clinical contexts with high pathology requesting rates and identified significant variability of appropriate requesting based upon the clinical contexts (four were considered good with two being considered poor).

2.4 NPS The PathSupport Project leveraged the work previously undertaken in the EDS Scoping Project and in particular the key aspects of the three NPS reports that were made available to the EDS Scoping Project team.

Some key elements are summarised in this report as they have provided significant guidance for the PathSupport Project Information Gathering and influenced the PathSupport Project recommendations.

14 The CHI Practitioner Guide (Chapter 6) uses the terms “Clinical Reasoning” and “Clinical Judgement” as the abstract aspects that are undertaken prior the concrete actions (or non-actions) of “clinical decision making”.


11

The first report focused on patient safety in general practice management systems. The NPS focus was prescribing rather than pathology requesting. 22 of the 114 features that support patient safety in general practice management systems were identified as decision support features. These were noted as being the most “variable and poorly implemented group of features largely attributed to:

• A lack of suitable knowledge databases.

• Differences in the way knowledge content was incorporated into software systems.

High priority recommendations of the study included a call for high quality knowledge databases with independent, evidence-based, accurate, up-to-date and locally relevant content to underpin clinical decision support and national standards and guidelines for the implementation of safety and quality features in clinical software systems.

In the second report, 14 GPs were interviewed and they indicated that when diagnosing conditions, the search for specific information amongst the various knowledge databases is challenging. It was reported that the electronic Therapeutic Guidelines are very useful for supporting prescribing decisions because all the information is in one place.

Recommendations for four areas of decision support for medical tests were given:

• Creation of a single electronic resource to access information about requesting and interpreting medical tests.

• Improved information sources for patients.

• Decision support which integrates with GP workflow when requesting tests.

• Access to previous results from other providers and standardisation of test names.

It was also suggested that more opinion should be sought to supplement the responses from the 14 general practitioners interviewed.

The third NPS report examined the pathology environment in general practice in Australia. It was noted in the report that “none of the (5) GPs interviewed could think of any scenarios where specific decision support may be useful, apart from some recommended request sets which all agreed could be helpful.”

This first part of the above note appears to be inconsistent with the opinions of the 34 GPs who participated in the GP Focus groups as part of the PathSupport Project. However, the second part, i.e. “some recommended request sets …could be helpful” may provide the explanation. In effect the GPs in the NPS study appear to support the scope of the PathSupport Project: that the value is not in the early abstract Clinical Reasoning and Clinical Judgement15 parts of the consultation but rather the benefit is once the GP has triggered the structured Clinical Decision path down which he/she wishes to proceed.

In addition to leveraging the three NPS reports, the PathSupport Project has interacted with the NPS project teams to exchange information on this Project and NPS activities and programs relevant to pathology and to identify synergies and collaboration opportunities.

The following topics were examined as they impacted the proposed solution and the need to undertake a Proof of Concept (POC):

15 These terms are used in the Certified Health Informaticians Australasia Literature.


12

• The Medicine Insight program which collects data from the desktop solutions of

participating general practices. Currently the focus of this work is on medicines and the review of appropriate prescribing. The prospect of the program being expanded to cater more for pathology was examined, particularly in the context of evaluating a POC. However the timescale for such activities are long term. As a consequence, the NPS Medicine Insight program is not considered an option for evaluating the POC, but may be worth considering for subsequent valuations.

• The Choosing Wisely program which is is directed at clinician-patient information sharing focussing on inappropriate practices across all medical specialities. It is anticipated the program will also cater for pathology testing. The NPS website already has extensive information regarding pathology testing, much of which has been sourced from Lab Tests Online

• The NPS has a network of staff (Clinical Service Specialists) supporting GPs often through personal visits to the General Practice. This support service covers both prescribing and medical tests. The activities of these Clinical Service Specialists are usually guided by a specific topic such as a perceived need for support of a particular clinical condition. An implication for the PathSupport Project is that it may be appropriate to leverage the Clinical Service Specialists during a POC especially if there is an overlap between the clinical contexts used in the POC and current topics that the NPS are supporting.

2.5 Pathology Sector Approach This approach involved an initial number of individual face to face meetings with a cross section of laboratory directors. This cross-section included representatives from both private and public laboratories. Two of the laboratory directors interviewed, were also pathologists.

The directors were advised that the aim of the meeting was to assess the preparedness (including barriers and risks) of their laboratory to leverage possible Desktop Vendor development (resulting from the PathSupport Project) and their business changes required with the implementation of improved pathology requesting modules used by GPs.

The directors were provided with the following scope:

• Pathology requesting only – there is no reporting in scope.

• Initial requesting, not subsequent requesting which is the domain of the laboratory.

• No overt preference for laboratory selection including not participating in the sequence of laboratories appearing on the request screen.

A set of detailed questions were prepared in consultation with the Project team. The detailed responses to the questions are not in this report however a summary of the responses is outlined in Section 2.6 below.

Two further meetings were held using the proposed solution (outlined in Section 4 ) to gauge:

• The Laboratory impact (workflow changes and resource changes) resulting from the changes in requesting by GPs (increased requesting of necessary tests and decreased requesting of unsuitable tests) together with the increase in quality clinical context provided with the requests.

• The impact of the solution on the Laboratory Information Systems

The approach used for the first activity above included convening a small group of experts to examine a set of nine clinical conditions (the five from the original Scoping Project, plus Health Check-ups, Hypertension, Fatigue and Thyroid problems). The group graded the impact the proposed solution was likely to have on the laboratory, the requesting GP and the


13

patient. Baseline data was provided by the FMRC / BEACH to assist with the impact assessments.

The second activity (i.e. the impact on the Laboratory Information Systems) involved a series of individual phone interviews with CIO’s or appropriate nominees at four pathology organisations. Each interviewee was provided with a demonstration of the possible solution and the project context prior to the interview to facilitate informed responses.

2.6 Pathology Sector Findings

Issues the Pathology Laboratory Wants Solved The contributors identified four issues. At this stage there has been no rigorous attempt to quantify or rank the four issues however the second and third issues consumed the majority of the focus of the meetings. Greater analysis of these issues could be examined during the POC as a way to gauge acceptability of the proposed solution by the pathologists and laboratories. Issues one and four were also identified by the GPs as part of their 12 issues (as per 2.10 Findings (Issues the GPs would like solved)). In subsequent sections of this report the unique pathology issues are numbered 13 and 14.

1. GPs are requesting tests unnecessarily as they are often requesting tests without qualifying them as “do if indicated”.

2. Pathologists find it difficult to provide best clinical guidance on many requests because:

i. Over 50% of all pathology requested is received at the laboratory without the clinical condition or the reason16 for requesting the test.

ii. In certain cases (e.g. TFT) the provision of current medications is not provided.

iii. Relevant earlier clinical histories are not provided.

It was suggested that providing much of this information manually (especially the earlier clinical histories) may be too hard although perhaps this information could be provided via online requesting.

3. There was an issue / concern that the directors are unclear of the impact that a new solution may have on the laboratories.

4. There are avoidable data entry errors and unnecessary extra workload due to many GPs not transmitting their requests electronically.

Implications of a Profession-Led Solution The laboratory directors viewed the clinical implications favourably because it provides a shared view and consistency as a starting point for consideration by the GP. It was seen that there is value in an independent RCPA committee guiding the rules.

As mentioned above, the directors considered the non-clinical implications to be unclear regarding the detailed impact of requesting for each clinical context.

16 It was stated that the reasons for ordering pathology are approximately: 40% monitoring, 40% diagnostic and 20% screening


14

2. Laboratory Impact Assessment The following table highlights the key factors that may impact laboratories as well as those considered highest impact on patients and GPs.

Finding Lab Impact Patient Impact

GP Impact

Increased Clinical Comments High (positive) Moderate( Reduced False Positives Low (+) High (+)

Lab Resources (Increased Necessary & reduced

Neutral

Lab Resources (reduced Unsuitable) Moderate (+) Reduced missed necessary tests Low (+) High (+) High (+)

Value of increased clinical comments:

The consistent provision of a clinical context was considered as positively impacting the management of the tests and the provision of clinical comments with the reports, and positive in terms of workflow associated with handling some results.

A major strategic impact (of including clinical context) related to future opportunities to leverage the existing pathology data to undertake research into the efficacy of pathology requesting.

Reduction in unnecessary false positive results

False positives17 if associated with additional and unsuitable tests that are requested unnecessarily can be associated with overdiagnosis and overtreatment of some conditions and this would be reduced.

Laboratory Resources (Necessary and Additional tests)

Of the nine clinical conditions examined in the workshop, the majority of these did not indicate there would be a significant reduction or increase in the laboratory resources associated with necessary and additional tests. In most instances the increase in tests (that should have been requested) or decrease in tests (that should not have been requested) were perceived as being similar in volume. Furthermore the majority of these tests are common high volume tests typically with a low incremental impact on time and resources. .

3. Laboratory Information Systems Impact The feedback focused on the technical changes to the contributors’ LIS environments driven by the provision of an increased number of quality clinical contexts. This functionality was perceived as an “opportunity” to further automate report comments rather than an imposition on resources. LIS impact should not be viewed as a barrier to the introduction of the PathSupport solution. The time and resources required to utilise the extra clinical contexts was seen to be “table configurations” rather than LIS software changes. The workload required to undertake changes to the LIS configuration was considered modest.

17 False Positives relates to abnormal results typically associated with biological variations often associated with diet etc.


15

2.7 Desktop Vendor Industry Approach The approach involved initial individual face to face meetings with a cross section of the CEOs of Desktop systems. Those Desktop Vendors were Best Practice, Genie, StatHealth, Medical Director (previously known as HCN), Zedmed and Medtech.

The CEO’s were advised the aim of the meeting would be to assess the business drivers that impact the preparedness of their company to undertake the development of an improved pathology requesting module.

A set of detailed questions were prepared in consultation with the PathSupport Project Team. In addition the set of guiding concepts from the original EDS Scoping Project were shared with the CEOs.

The detailed responses to the questions are not included in this report however, a summary of the responses is outlined in Section 2.8. In describing the information, the following terms have been used:

• “All” if all solutions or Desktop Vendors fell into this category.

• “Majority” at least 4 Desktop Vendors fell into this category.

• “Several” or “some” when at least two Desktop Vendors fell into this category.

• “At least one” when one fell into this category and others (who had not expressed a view) may also hold the same position.

• “A” Desktop Vendor when only one expressed the view and there was no expectation that others might hold the same opinion.

The subsequent round of information gathering involved a full day workshop with Desktop Vendors and industry representatives. The same six Desktop Vendors who were interviewed individually (along with a representative of the Medical Software Industry Association who is co-chair of the RACGP group reviewing Clinical Decision Support Systems) attended this workshop with the aim of communicating the proposed solution and request their input by providing estimates of the time and resources required to develop, test, maintain and support the proposed solution.

The workshop was also used to assist with developing an overall understanding of the scope and interaction of the solution required with the PathSupport Knowledge Services.

2.8 Desktop Vendor Industry Findings

The initial set of meetings provided the following finding:

Current State of Play

All solutions have a pathology requesting model that generates/prints pathology request forms.

The generation of eRequest messages that underpin the above request forms is not available in all products. When they are available some are produced in proprietary format for nominated large pathology laboratories. At least one Desktop Vendor has developed a standards-based request message including SNOMED test codes as part of the Department of Health funded PITUS Project – administered by the RCPA.

The guidance of test selection is included by at least one Desktop Vendor by using a table (spreadsheet) of tests provided by one of the large laboratories. The table can be searched


16

by clinical condition. Most Desktop Vendors provide favourites (test groups) as a key feature of the local configuration of pathology requesting systems. At least one Desktop Vendor provides a set of model favourites developed by their advisory group.

Plans for Enhancement of Pathology Requesting Module Developing an enhanced pathology requesting module is viewed favourably by many of the Desktop Vendors, however further analysis is required to determine user demand once a prototype is available. This analysis is required because none of the Desktop Vendors had significant demand from their user bases to undertake this development.

Product Development (re Interface to Rules) The majority of Desktop Vendors suggested the PathSupport Project should adopt the simplest and fastest interface options. At least one Desktop Vendor already uses tables with the concept of “table driven logic” being proposed. Web service calls to access remotely hosted rules was considered an acceptable approach provided there were no negative impacts on GP user response times.

Product Review (Testing Approach) The majority of Desktop Vendors requested a collaborative approach be used to define a set of test cases to increase the prospect of the testing being practical. They suggested that the testing should be undertaken by the Desktop Vendor (this testing could be witnessed as needed). This approach is preferred to testing by a third party, which is often costly and time consuming.

Feedback for Improvements (including Rules) The majority of Desktop Vendors suggested starting simple and limited. There was general support that the initial simple set of rules would need continuous improvement and expansion. They also felt that unless previous test results were readily accessible that the credibility of the rules may be undermined.

There was a broad expectation that the GPs would contact the Desktop Vendor with complaints as well as suggested improvements and expansion in rules. They indicated limited support time and resources available for this and requested that the RCPA-led project governance would be responsible (and seen to be responsible) for the accuracy of the rules.

Consequently the majority of Desktop Vendors supported the provision of feedback to be effectively a “pass through” to the RCPA-led project governance, possibly at a detailed level per clinical context.

There was a suggestion from some Desktop Vendors that some requesting information regarding the rules could be captured automatically. The sharing of information would not occur unless there was formal agreement by the GPs. This information might include:

• The occurrences (by clinical context) where the suggested necessary tests were used without modification.

• The occurrences (by clinical condition) where the suggested necessary tests were modified by GPs for individual patients and what the modifications were.


17

In addition to the second item above, it was suggested that the reason for modification may be too much of an imposition for general use but may be acceptable to the GPs as part of a targeted short term review.

Notwithstanding the desire by the majority of Desktop Vendors to ensure that their users know that the RCPA-led project governance is responsible for the content, some Desktop Vendors still saw a role in assisting the RCPA-led project governance in rule reviews through the possible leverage (at a cost) of the Desktop Vendor’s call centres.

Cost Categories and Ongoing Business Models

The majority of Desktop Vendors considered there are three main costs:

• Initial development and testing (one off).

• Maintenance if the number of parameters in the rules change (as required).

• User support dependent of the number of sites.

Some Desktop Vendors suggested that the method of costing the initial development would be based upon an “opportunity cost” rather than “cost recovery”. Furthermore, there may be a need to provide the Desktop Vendors with the rationale that the functionality would be used by the GPs.

Some Desktop Vendors suggested that the PathSupport Project could consider payment per use. Such a business model would help the Desktop Vendors to give extra attention to driving up the use of the pathology requesting module.

Expectation of the PathSupport Project

In addition to:

• the provision of funds (for development and ongoing maintenance and support), and

• the rules and their ongoing maintenance

several Desktop Vendors supported the RCPA-led project governance providing the endorsement (branding) of the rules and hence providing authority for the knowledge used by the Desktop Vendors. The nature of this branding was not discussed in detail.

On a more detailed level, some Desktop Vendors sought the provision of a detailed implementation guide from the PathSupport Project to help develop the integration of the rules into the software solutions.

Subsequent Workshop Responses Following the Desktop Vendor Workshop all Desktop Vendors provided written responses to the following four topics:

Feasibility: All Desktop Vendors considered the solution feasible.

Costs: All Desktop Vendors provided detailed estimates of the initial development, subsequent software release time and resources and support costs. These indicative costings have been incorporated into Section 5.2.

Barriers: All Desktop Vendors considered that a requirement of undertaking the development, and the subsequent success would depend on the availability of a critical mass of clinical contexts.


18

POC Participation: Three of the six Desktop Vendors indicated a willingness to participate in a POC.

2.9 General Practitioner Information Gathering The GP information gathering phase was a major component of the Project.

The detail of the approach used is not included in this report however the following is an extract of that approach.

The initial GP information gathering included four focus groups. The four events together with the number of General Practitioners (GPs), Practice Managers (PMs) and Practice Nurses (PNs) were as follows:

Location / ML Date GPs PMs PNs Total

Pathologist

Brisbane (MNBML) 21st Aug 11 0 0 11 CEO - SNP

Sydney (IWSML) 26th Aug 10 1 0 11 Clinical Pathologist Primary HealthCare

Melbourne (IEMML) 28th Aug 8 0 1 9 Clinical Pathologist Healthscope

Broken Hill (FWML) 3rd Sept 5 0 0 5 Chairman of Board of - NSW Health

Pathology

Totals 34 1 1 36

Subsequent GP Information Gathering

The GP information gathering was conducted as a webinar and included six GPs who had participated in the original focus groups. The GPs were nominated by the local Medicare Local co-ordinators and included a cross section of users of different desktop systems.

The participants were updated on the narrow focus of the possible solution in regards to their initial “issues they wanted solved”. The proposed solution was demonstrated with the support of the pathologist who attended the first Focus Group and one of the GP representatives from the Project Steering Committee. The GPs provided written responses to four questions at the conclusion of the webinar:

• Overall opinion of the proposed solution

• Likely uptake by other GPs (e.g. experienced and inexperienced GP groupings)

• Feasibility re the time and resources necessary to implement compared with the benefits expected

• Likelihood of requesting the functionality from their Desktop Vendors


19

Subsequent Practice Manager Information Gathering The Australian Association of Practice Managers (AAPM) information gathering involved a similar briefing as the GPs with more context provided as none had participated in the original focus groups. The feedback from the practice managers was captured during the meeting and mainly focused on the operational feasibility of the solution.

2.10 Findings (Issues the GPs would like solved) The following is a summary of the output of the original four groups. In total there were 17 issues identified by the 11 teams at the four focus groups.

• 12 issues were identified as being in scope and being high impact.

• Three issues were out of scope. • Two issues were design concepts (rather than issues).

The 12 high impact issues identified are:

1. GPs request inappropriately due to an absence of the easily accessed

guidance The GPs identified the lack of easy access to current best practice guidance as a potential for missing the requesting of important (appropriate) tests. For the same reason they identified that discretionary and even unsuitable testing could be undertaken. At times the influence of the patient to request further tests could be managed more appropriately if current good practice guidelines could be readily accessed.

2. Patients receiving unexpected bills The GPs identified the need for greater information at the time of requesting pathology in regards to Medicare rules that might inadvertently result in a private bill for the patient. The GPs also felt that the current pathology requesting modules incorrectly allowed the requesting of tests that would be rejected by Medicare (e.g. HbA1c for the diagnosis of Diabetes). 3. A lack of overall costing information makes it harder to request appropriately The GPs identified the need for greater information regarding the costs of tests that might influence the requesting of certain additional tests. This is not an issue for the minimum set of tests that are clinically appropriate. In such circumstances, the GPs would typically request the appropriate tests without considering the cost implications. However, GPs will often consider further additional tests – perhaps due to their personal requesting approach and due to patient specific considerations. The GPs consider that their decision making is harder without the benefit of knowing the cost implications of these additional tests.

4. An absence of patient information sheets The GPs identified that verbally providing patients with special instructions can be ineffective with some patients not recalling or misunderstanding the instructions. Consequently, patients may need to return for collections or the quality of the results is compromised through inappropriate preparation.

5. Lack of access to previous pathology

The GPs identified that not knowing previous tests that a patient has had can result in unnecessary retesting.


20

6. An absence of readily available GP collection information

The GPs identified an absence of easy access to the collection information for “doctor collects”.

7. Clinical and administrative issues from (pathology provider) data entry errors

The GPs identified data entry errors associated with multiple elements including patient identification, requesting doctor and their practice identification, copy doctors and their practice identification, missed tests and missed billing entries. This issue can impact on time and can also impact patients’ healthcare.

8. GPs have difficulty in establishing compliance with the requested pathology

The GPs identified that although this issue necessitated new functionality in the handling of pathology results, it would be facilitated by the provision of electronic requesting and electronic acknowledgement from the pathology laboratories of the receipt of the specimens.

9. An absence of information regarding timeframe for results

The GPs identified an absence of easy access to the timeframe for the turnaround of results making it difficult to manage follow-up.

10. Unnecessary testing due to not effectively using “add ons” and “do if indicated tests18”

The GPs identified this as a way to reduce testing (when not required) while at the same time avoiding delays and extra collections when the testing is required. Part of this issue is the difficulty in readily accessing information regarding the appropriate “add on” and “do if indicated” tests.

11. Difficulties in patients getting results due to ineffective instructions

The GPs identified that the provision (and recording) of effective instructions to patients (regarding obtaining their results) would reduce follow-up problems. This issue could also be linked with the above issue relating to turnaround times for results.

12. GPs experience difficulties with paper request forms

The GPs identified that the need for paper request forms has associated issues such as printer issues and reprinting of lost requests.

Reclassification of Issues There were two issues that will be treated as “design concepts” in the following sub-section. Those are:

a. The current pathology modules were already slow. b. The modules lacked flexibility in modifying the test groupings.

18 Do If tests are those tests that are performed only if the results of another test indicates it is appropriate the extra tests.


21

User Demand for the 12 High Impact Issues

The approach used to rank the 12 issues is not detailed in this report however, in overview: The number of teams that identified these issues was used as an indicator of “user demand” for a solution to the issue.

Issue Description User Demand

1. Difficulty requesting appropriate tests due to a lack of suggested tests a. Under requesting of necessary tests b. Over requesting of unsuitable tests

High+

2. Unintended bills due to a lack of knowledge of Medicare rules High

3. Difficulty in requesting extra tests due to lack of knowledge of costs High

4. Specimen collection issues due to a lack patient information sheets High

5. Tests are repeated unnecessarily due to lack of knowing previous test history

Med

6. Specimen collection issues due to a lack of GP collection information Med

7. Time wasted and reduction in the quality of care due to data entry errors Med

8. Difficulty identifying patient compliance due to a lack of a feedback loop Med

9. Difficulties with follow-ups due to a lack of results turnaround information Low

10. Unnecessary testing being performed due to not using “add-ons” Low

11. Difficulties with patients obtaining results due to ineffective record keeping

Low

12. Printing problems and lost request forms due to need for paper requests Low

2.11 Findings (Proposed Solution)

GP Findings on the Proposed Solution

The following information is a summary of the written responses from follow up with GPs who had participated in the initial workshops once the Proposed Solution was developed:

1. The Overall Opinion of the Proposed Solution All six GPs viewed the solution very positively. Individual comments included:

• Very good. Simple Intuitive. • Strongly positive. Love it. • Very Positive. • Brilliant.


22

• Very positive. I would use it. • Would be very useful and something I would look forward to using.

In addition there were supporting detailed comments relating to the screen design.

2. Uptake by GPs The majority of the contributors were reasonably optimistic that most GPs would take up the solution with a qualification that older or non-computer literate GPs may not take up the product.

3. Feasibility to Implement and realise the benefits.

The majority of the contributors considered the time and resources to implement the solution would be manageable in order to achieve the resultant benefits. None of the contributors felt the time and resources was prohibitive

4. Likelihood of Requesting this functionality from your Vendor.

The majority of contributors were positive towards requesting this functionality from their vendors and indicated they would do so.

Cost was raised as an issue in relation to GP’s preparedness to pay software vendors for the enhanced functionality. AAPM Findings on the Proposed Solution. The following information is a summary of the key points raised:

1. The currency of the endorsed test list and information. • The need for a robust process in place to ensure the currency of endorsed

test lists (and the supporting comments). Resistance to use the solution could be encountered if the users are not confident of the accuracy or currency of the information. A method of advising users that changes have occurred to the information should be in place, especially in areas where changes to supporting comments might otherwise go undetected.

2. Medico Legal Implications

• An assessment of medico legal implications for GPs through using the proposed PathSupport Solution should be undertaken. A specific example is the implications for a GP of deleting a test from the endorsed “necessary” test list.

3. Workflow of the solution design.

• It is vital that GPs are able to adopt a consistent workflow process. To achieve this consistency, a large majority (perhaps 80-90%) of pathology requesting needs to have access to the RCPA endorsed tests.

4. Selection of which Clinical Contexts to be included.

• A mechanism whereby Practice Managers can provide feedback on clinical contexts for inclusion by the RCPA


23

3 Design Principles and Development Options

3.1 Approach The EDS Scoping Project had previously identified a set of 10 design concepts (see box below) for the EDS solution. These were a combination of the concepts that are applicable to:

i. The GP’s user interface with their pathology requesting module, and

ii. The interface by the pathology requesting module to the clinical rules.

The original concepts (below) used “should” and “must” as a guide however the information gathering phase to validate these concepts resulted in several changes to the mandatory nature of some of these concepts.

The PathSupport Project was charged with validating the design concepts and establishing if existing solutions (e.g. The Map of Medicine and Health Pathways) could be used to minimise the development of a new solution.

19 The original version in the Scoping Project had “should not disturb” and (following input from the GPs on the PSC) was subsequently changed to “should not negatively impact”

1. The solution should not negatively impact19 GP workflow.

2. The solution must be capable of being disabled by the general practitioner.

3. The solution should isolate the Desktop Vendors as much as possible from maintenance updates.

4. The Desktop Vendors should not be responsible the clinical knowledge but must be responsible for the interface / integration to and from that clinical knowledge.

5. The delivery method, for example, embedded in the clinical desktop, using a central web based service or a hybrid version of embedding and leveraging a central service for updates, should be at the discretion of the Desktop Vendors . There may be a cost in proving multiple delivery models hence they should only be developed as needed.

6. The interface to the solution must support nationally agreed terminology such as SNOMED, LOINC etc.

7. The interface must support the passing of de-identified patient data from the clinical desktop and receive pathology tests back to the clinical desktop.

8. The solution should be capable of being triggered at various points in the GP workflow, that is, not just at the time of pathology requesting. These additional triggers should be at the discretion of the Desktop Vendors.

9. The existence of clinical knowledge for a particular scenario should be visible to the general practitioner without the need to access that clinical knowledge.

10. The solution should support manual override of the suggested pathology tests by the clinician.


24

GP User Interface The approach to validate the GP interface design concepts was undertaken in three parts:

i. Asking the GPs at the focus groups to review the relevant concepts (specifically those associated with their interaction with pathology requesting, i.e. concepts 1,2,8,9 and 10). They were asked to rate the relevant design concepts in terms of the mandatory nature of requiring Desktop Vendors to include these features in their software

ii. Asking the Desktop Vendors to review the relevant concepts.

iii. Seeking additional opinions from visits to the general practices of three expert GPs.

Clinical Knowledge Interface The approach to assess the clinical knowledge interface design elements was based upon:

i. Requesting the Desktop Vendors to review the relevant concepts and provide input.

ii. Information gathered from a key expert in rules engines from the UNSW and Steering Committee Members in the management of rules in a pathology setting.

Buy or Build

The approach to gauge the suitability of leveraging existing solutions was based upon meetings (including product demonstrations) with the Medicare Locals that are championing the solutions:

i. Metropolitan North Brisbane (for Map of Medicine), and

ii. Inner West Sydney and Inner East Melbourne (for Health Pathways).

3.2 Findings

GP User Interface

The findings from the GP Focus Groups are outlined below.

Num Design Concept Description Must Should

1 Not negatively impact GP workflow 21 2

2 Solution to be capable of being disabled 4 19

3 Capable of being triggered in various points 4 19

4 Existence of clinical knowledge without the need to view it 4 19

5 Ability for GP to manually override suggested tests 17 6

Concepts one and five were distinctive in terms of their mandatory category when compared with the other three concepts.


25

Concept 1: “not negatively impact GP workflow” has some element of subjectivity. Further definition of this concept may be required in consultation with individual vendors and their respective users to ensure that such a concept is achieved.

Concept 5: “Ability for GP to manually override suggested tests” is reasonably objective across all solutions and may not require much more definition.

Concepts 2 to 4 were categorised as “should”.

Concept 2: “Solution to be capable of being disabled”, this concept was the subject of considerable discussion centred on the clinical independence of experienced GPs and the need for guidance for the inexperienced GPs. The main proponents for concept 2 settled on the following wording “The new software must have the ability to disable the display of suggested tests and the disabling feature must be provided at an individual GP level.”

Concept 3: “Capable of being triggered in various points” was the least supported concept based upon the individual comments and notations suggesting that it was the most optional of the 5 concepts for a Desktop Vendor to include.

Concept 4: “Existence of Clinical Knowledge without the need to view” was discussed in some groups partly for clarification. Those involved supported the concept primarily to limit the need to view information unnecessarily and possibly to reduce the impact on GP workflow.

The findings of the Desktop Vendors are outlined in section 2.8. There was broad acceptance of the concepts that relate to the GP user interface.

The findings from the three GP experts is outlined below.

i. A concern that there may be difficulties if the main desktop solution exited to other different solutions (that may be web based) with possible response time delays.

ii. A suggestion to stress the importance of the patient-doctor interaction and the decision making in that interaction. A solution that is too “tick-box” driven may take away from enhancing that interaction / decision making. Hence any solution design should seek to avoid negatively impacting the patient-doctor interaction.

iii. Support for the provision of guidance of necessary tests for particular clinical contexts. One of the benefits would be to help manage the pressure from patients if they are pushing for certain tests, e.g. PSAs being done too frequently.

iv. Support for capturing clinical notes / reasons for pathology as part of the current requesting process. Furthermore the availability of other clinical information (other current conditions, etc.) is useful.

v. A suggestion to stress the importance of flexibility and that it is essential that a GP can add more tests to the suggested “necessary” tests. Furthermore if a GP requests tests that are known to be “unsuitable” then the terminology in advising the GP needs to be “respectful” to recognise clinical independence of the GP.

vi. A suggestion regarding the language of what the suggested necessary test might be called (and perhaps how they may be identified for each condition). There may be many different authorities that could differ on the suggested test request sets. However the set of tests might be called something like “current good practice for General Practice”. The use of the word “current” has a flow on to the need to review the suggested tests on an appropriate schedule.


26

vii. A concern was the issue of “who owns the data”. If the overriding of suggested

necessary tests is recorded by the solution then that information must not be shared outside the practice.

viii. Support for the concept of starting small and simple because if the GPs “get turned off they will turn off” the functionality and it may be very hard to get them back.

4. Clinical Knowledge Interface

The detailed findings of the desktop software vendor meetings are in section 2.8. There was broad agreement with the original design concepts amongst the desktop software vendors.

The scope of the table driven approach that was shared during these discussions was refined to the following for each condition / purpose (i.e. clinical context):

• The necessary test request set.

• The unsuitable tests (if any). Not all clinical contexts would anticipate having unsuitable tests listed –these tests would be incorporated into commentary. This would be guidance only and not used to block the entry of tests.

• A commentary describing exceptions and conditions (for example, possibly including Medicare constraints regarding frequency of testing) as well as potential links to other resource sources.

Within the table driven approach, the number of clinical contexts is not restricted hence such a model could be applied to a very wide set of clinical contexts albeit there is almost no complexity in the execution of the rule.

In the context of Desktop Vendor systems there is currently a major barrier (and risk) for the premature automation of rules due to the inconsistent representation of the data that may be used to interpret rules. For instance the Desktop Vendor system is very unlikely to currently record family history of a particular condition in a manner suitable for the execution of a rule.

In summary the Desktop Vendors had a leaning towards starting/continuing with the rules being provided as raw data for embedding in their software programs. The rationale for such an approach was simplicity and the avoidance of potential response time issues with third party access.

However, the meetings with the Desktop Vendors occurred before the meetings with the rules engine expert from UNSW and members of the Project Steering Committee. The sessions examined the pros and cons of using a basic table driven approach compared with using a third party rules engine provider.

One aspect that favours the use of a third party provider is the rigour associated with translating the human readable knowledge into machine readable knowledge. This could be achieved through a formal user interface.

Consequently the clinical knowledge interface approach may warrant an assessment (both technically and commercially) in parallel with the POC.

Buy or Build The investigation of the buy or build options was modified once the GP focus groups had been conducted and the possible solution was identified. The demonstration of the Map of Medicine product and its interface with Medical Director that was viewed at MNBML’s Clinical Advisory Group (CAG) meetings showed that the Map of Medicine product was outside of the current pathology requesting workflow. As a


27

consequence (although the interface from Medical Director to the associated Map was an effective link) it would not conform to one of the mandatory design concepts from the GPs namely that the solution must not negatively impact the GP workflow.

A key finding was that the development of the content was being undertaken with the local clinicians rather than with the Colleges. As such the content of the Map of Medicine pathways has a risk of being inconsistent with the suggested necessary test sets. This finding has influenced the importance of the possible next step relating to alignment of guidelines.


28

4 The Proposed Solution

4.1 Confirmation of Scope The approach to validate the scope for future work has been based upon examining the 14 unique issues identified during the information gathering phases. That initial examination has been guided by the Project Steering Committee towards the appropriateness of a future project (i.e. post PathSupport) being able to solve the issues.

The detailed commentary for each issue is not included in this report however the scope of the solution has been confirmed as:

• Providing GPs with guidance to request pathology appropriately (issue number 1a and 1b from the GP list of issues and issue 1 from the pathologist list) based upon provision of the necessary tests (and further information) for a clinical context.

• Providing a facility which increases the provision of clinical context to pathologists (key unique issue number 13 from the pathologists’ list of issues)

4.2 Further Issues from the Information Gathering Findings This section highlights further issues considered as part of the subsequent solution and recommendations.

Difficulty in Accessing the RCPA Manual

There is broad support for the value and authority of the Project governance (leveraging RCPA Manual) for a clinical context. However, the current pathology requesting workflow used by GPs is not configured so that the RCPA Manual is incorporated in this workflow. The solution should provide a direct link to the relevant Project site (potentially the RCPA Manual) using the GP provided clinical context. Likewise the Pathology Decision Support Tools (PDSTs) currently located within the RCPA Manual are in a form that currently cannot be integrated into GP workflow or into GP desktop programs.

5. Absence of Clinical Context

The clinical benefits of providing clinical context for a pathology request was strongly supported by expert GPs and pathology laboratories / pathologists . However, the Monash University study identified that only 39% of requests are accompanied by clinical context entered by GPs. The implication of this issue is that the proposed solution should encourage but not mandate the entry of the clinical context.

Impact on the Clinical Independence of GPs There is discernible demand for test guidance from the end user GPs, however the level of guidance needs to avoid interfering with clinical independence. As a consequence the solution should not be too prescriptive in design or use of language when providing guidance.

Clarification of Language: Appropriate Requesting and Necessary Tests

During the information gathering it was common that the conversational use of “appropriate” and “necessary” was almost interchangeable resulting in ambiguity. In the proposed solution this ambiguity needs to be removed to give clarity for stakeholders. To that end the approach


29

used has been to distinguish the broader concept of appropriate requesting and the precise concept of necessary tests.

The relationship between these terms is captured in the following diagram:

This highlights appropriate tests (presumably those included in a request is appropriate) is not a precise set of tests because it has been based upon the requester clinical judgement regarding the requesting of some additional tests.

Consequently the proposed solution uses the precision of “necessary” tests as the basis of the rules.

4.3 The Detail of the Proposed Solution The proposed solution encourages (but does not mandate) the GP to firstly enter the clinical context for the pathology being requested (e.g. Diabetes Monitoring). The solution then uses that clinical context to prepopulate the request with an endorsed set of necessary tests and also provides further information to help the GP request appropriately. The identified solution addresses the three most important issues identified by the pathologists and GPs. At the same time it satisfies the key solution characteristic requested by end users.

6. Proposed Solution (Knowledge Services)

A vital component supporting the enhancements to the pathology requesting modules is to provide the Desktop Vendors with a knowledge base that is “machine readable”. The knowledge base is a collection of rules that links a clinical context with an authoritative set of necessary tests (using the APUTS “preferred terms” and SNOMED codes) that can prepopulate the pathology requesting module together with further information that can be viewed by the GP. The knowledge base would also include documentation for the Desktop Vendors to understand the representation of the data elements in the rules. This documentation will enable the Desktop Vendor systems to use the rules and the data effectively and safely.

The creation of the machine readable rules will involve the translation from human readable knowledge (provided by the Project governance, leveraging the structure established for the RCPA Manual). Other sources might include the RACGP Red Book and guidelines from


30

other authorities. A rule is a mechanism used to look up information using parameters (initially just the clinical context). That information is the list of necessary tests and the further commentary that can then be displayed for the GP.

Having created the rules, ongoing regular maintenance and review will be required to maintain their integrity and currency. Hosting and distribution of the rules to the Desktop Vendors will need to form part of the proposed solution.

The governance of the clinical elements of the above “knowledge services” activities will need to be incorporated within the RCPA governance structure in conjunction with other colleges and clinical authorities. The rules may be informed by the implications of the Medicare rules for requesting and the activities of other knowledge providers. Furthermore, the laboratories would seek to be informed of the implications of new rules. As a consequence part of the governance function will be to interact with other stakeholders.

The following graphic illustrates the above commentary.

7. Clinical Governance of the Knowledge Services

A meeting was convened of the key parties to examine the model necessary to support the process of developing and maintaining the content provided to the Desktop Vendors. The participants in that meeting included the a GP representative of the Project Steering Committee as well as the Chair of the PathSupport PSC, RCPA CEO and Deputy CEO and the Editor–in-Chief of the RCPA Manual.

The key outcomes of that meeting were:


31

• The EDS machine-readable specifications could be hosted on the RCPA Manual

website.

• The process needs clinical governance similar to, but separate from, the RCPA Manual Editorial Team processes.

• An overseeing Clinical Governance group should consist of pathology and GP experts provided by the RCPA and RACGP (with access to other expertise, in particular the RCPA Discipline Advisory Committees and other specialist Colleges).

• The POC would include the Antenatal screening tool and the Hyperlipidaemia / Lipid studies tool developed under the Scoping Project.

• The Clinical Governance group would have processes for reviewing and updating the EDS tools as new clinical information/guidelines become available.

One of the PathSupport objectives20 is to “provide a summary of the policy implications arising from the adoption of EDST”.

The role of the Clinical Governance Committee could be to ensure the RCPA’s policies are reflected in the way clinical information is developed and the detailed content that is provided to the Desktop Vendor systems.

Some of the factors that impact this policy setting include:

• Prioritising clinical issues based upon various influences including the input of the RCPA membership; the RACGP membership etc.

• Medico legal implications for the RCPA and the level of involvement in the medico- legal implications for the GP users. Such a position may guide the need for (and wording of) disclaimers with Desktop Vendors and GPs. This activity could examine a variety of scenarios including the use of the recommended tests and the non-use (i.e. removal) of the recommended tests, version control/currency etc.

• The extent of collaboration with other organisations including specialists colleges (e.g. RANZCR, RANZCOG, RACP), other GP colleges (e.g. ACRRM) and further organisations linked to particular conditions (e.g. Diabetes Australia)

• Develop guiding principles for the categorisation of tests i.e. necessary, additional and unsuitable. This categorisation could include explanations on the boundaries between the categories of tests. These principles will have implications for the supporting comments in the information shared with the Desktop Vendors.

• Develop a set of conditions/guidelines for usage of the RCPA information for Desktop Vendors e.g. upgrading systems when there are new releases of information.

• Develop quality review principles for the review of the accuracy of content prior to publication.

• Develop an ongoing review process to manage the currency of the published information

• Develop the approach for handling ongoing feedback from users of the PathSupport Solution.

20 Department of Health Funding Agreement for this project Schedule A2


32

8. The GP Desktop Solution

An example21 (from the Best Practice solution) of the current pathology workflow in the Desktop Vendor system appears below (this is a mock screen shot and therefore not an actual patient). The first screen is the general consulting screen that the GP would have open when consulting with the patient. Having come to a possible path of action, the GP may decide that pathology should be requested and selects the “blue beaker” icon and the subsequent screen would then be presented.

21 These examples have been included in the report with the permission of the CEO of Best Practice


33

The current pathology requesting module screen commences (as below) with an expectation of entering pathology tests. This entry may be facilitated by groups of favourites and by key word search. Having entered the tests the GP has the option (although many GPs do not use this) to enter a clinical note.


34

As mentioned above, the proposed solution involves reversing the sequence of the entry of data. Specifically it encourages (but does not mandate) the GP to enter the (two part) clinical context for the pathology being requested (e.g. Diabetes Monitoring). The proposed solution uses that clinical context to prepopulate the tests requested and provide further information to assist in requesting appropriately and possibly avoid requesting unsuitable tests. The GP is able to modify the suggested tests which would have a logo identifying the authority of the clinical information. This modification of the tests is applicable to the individual request / patient only. The endorsed list is only modifiable globally through the Project governance process.

Hence with reduced key strokes the three most important issues flagged in the information gathering phases are all solved (in part) with in this solution:

• The under requesting of necessary tests,

• The over requesting of unsuitable tests, and

• The absence of clinical context when requesting tests.

Furthermore the proposed solution addresses the key design requirements of:

• Not negatively impacting the GP’s workflow, and

• Allowing the GP flexibility to add change or delete the suggested tests based upon their local context.


35

The above screen is an early prototype. Further enhancements of this prototype may include changes and functionality such as:

• Automatic inclusion of drug lists if the comments suggest this information is relevant.

• Display currency of the clinical knowledge with a date of expiry or review (version control).

• A feedback button to enable GPs to provide feedback (in regards to the necessary tests and the additional information) to the RCPA Governance Committee.

• Reposition the requested tests so the recommended tests lists appear closer to the clinical context that has generated the recommended test lists – currently Favourite Tests appear here.

• Explanation of other rules around pathology requests for a particular clinical context (eg Medicare).

9. Knowledge Translation and Distribution

It is possible that initially the parameters that define the request sets could be focused only on the clinical context (e.g. Diabetes Monitoring). The current quality of the GP data provides a barrier to introducing greater automation of the rules. As a consequence any complexity regarding other parameters might initially appear in the “further information” box. This would enable the GP to be supported in their decision making and manually add tests as needed.

The Project proposes two models for the first iteration / proof of concept (POC):

• A table model, and

• A third party rules engine model.

Both would be evaluated in parallel by a project team possibly lead by expert Informaticians from the RCPA. The generation of the tables might be a by-product of the third party rules generation. Such tables could be administered under the Project governance (possibly via the RCPA Manual) and downloaded on a periodic basis by the Desktop Vendors. The third party rules engine provider could host the rules and provide online real time access for the desktop systems.


36

5 Indicative Benefits, Costs and Feasibility

One of the expectations of the PathSupport Project is to “provide assistance in capturing the benefits and costs of the decision support solutions as input to the (subsequent) development of a business case to encourage the uptake of electronic decision support tools.” In addition to capturing the benefits and costs, the project has also examined the feasibility of the solution.

The following benefits are presented in overview. Further work is required within the POC to quantify the benefits.

5.1 Benefits The proposed solution would expect to deliver the following benefits:

• Increased requesting of necessary tests with the potential of improved patient outcomes and experience (e.g. avoiding second requesting of pathology, and the early detection of illness).

• Decreased requesting of unsuitable tests with the potential to reduce patient inconvenience, overdiagnosis and inappropriate use of resources. .

• Increased provision of clinical content for pathologists to provide improved reporting resulting in improved patient care and better refinement of the appropriate next steps in the patient’s care.

When identifying benefits, it is important to recognise that the PathSupport Project could complement eRequesting initiatives from other RCPA projects such as PITUS. In a separate PITUS project, undertaking a proof of concept for eRequesting using PITUS terminology identified a number of benefits associated with eRequesting which includes:

• Reduced data entry / Specimen Reception Area (SRA) time,

• Elimination of transcription errors with the associated improvement in quality and safety,

• Coding errors are minimised, and

• A reduction in test turnaround times.

The benefits associated with eRequesting are not examined further within this project, however it is important to acknowledge there are a number of potential benefits from this PITUS Project that could positively impact EDS.


37

10. Summary of Benefits and Metrics

The following table outlines the expected benefits and proposes a set of possible metrics to assist in the qualification of the benefits in a subsequent business case.

Benefit Area Description and Metrics

1. Increased requesting of “necessary tests”

a) Expected to reduce the number of occasions that a patient needs to have a subsequent pathology test because not all the necessary tests were originally requested.

Desired Metric (current state): The number of occasions that a second (avoidable) request is placed. Desired Metric (future state): The % reduction in these avoidable extra requests resulting from the introduction of the proposed PathSupport solution.

b) Expected to reduce time taken for the diagnosis of a patient because the most appropriate tests are requested.

Desired Patient Care Metric: to be identified in the business case.

Note that a disadvantage of this increased requesting of necessary tests may be the extra workload placed

th l b t 2. Decreased requesting

of unsuitable tests.

a) Expected to reduce the inappropriate use of resources by the pathology providers.

Desired Metric (current state): the resources that are used on unsuitable tests. Desired Metric: (future state): the % reduction in time and resources.

b) Expected to reduce patient inconvenience where extra specimens are collected unnecessarily

Desired Patient Care Metric: to be identified in the business case.

3. Improved provision of Clinical content

a) Improved ability for Pathologists to provide quality care for patients including the generation of reporting comments.

Desired Patient Care Metric: to be identified in the business case

b) Reduction in unnecessary advice to GPs of abnormal results when an abnormal result may be reasonable for a particular patient.

Desired metric: the number of unnecessary escalations Desired metric: the % reduction.


38

The purpose of the EDST is to “optimise the appropriate use of pathology testing to enhance high quality diagnosis and treatment in patient care”22. A subsequent business case may leverage the outcomes of the POC to devise qualitative and quantitative metrics to measure the benefits of enhanced high quality diagnosis and enhanced treatment in patient care.

However, that subsequent business case may need to consider a limited set of quantifiable metrics to justify the financial costs of the solution described in this report.

11. Role of the Proof of Concept in measuring benefits

Currently it is difficult to quantify benefits as the change in the level of conformance with the recommended tests is not available. It is intended this information will be ascertained through the POC.

This will be undertaken using baseline estimates (with the assistance of external research organisations) of:

• the number of extra follow up requests that are a result of not requesting the necessary tests on the first occasion and

• the number of requests for unsuitable tests

The POC will aim to establish the conformance as a percentage with the guidelines so the evaluation team can extrapolate the possible reduction in unnecessary requests.

5.2 Indicative Costs for Subsequent Business Case The following costs are provided to assist in the development of a subsequent business case “to encourage the uptake of EDST”. During that business case development the comprehensive set of detailed assumptions (see appendix D) would require validation and appropriate options selected.

Prior to undertaking the business case, a POC should be undertaken to validate benefits and feasibility as well as inform the selection of the most appropriate approach to the distribution of clinical rules.

The table below outlines the estimated costs of the POC broken down into 5 key categories.

Costs Item/ Category Relevant Parameters and Metrics Amount (ex 1. Rules creation (includes

knowledge experts, clinical governance functions, development & testing)

Number of clinical contexts: 2 (Antenatal & Hyperlipidaemia/Lipids[monitoring])

Number of hosting models :2 (both RCPA and Third Party hosting)

$77,380

2. GP Desktop Solution development and testing

Number of Vendors: 2

Includes any additional software enhancements identified for the evaluation (Category 3).

$102,000

22 Department of Health project agreement page 1 “Program Description and Objectives


39

Costs Item/ Category Relevant Parameters and Metrics Amount (ex 3. Participation in trial &

Evaluation of: GP workflow changes, rules engine and the impact on pathology practice.

Number of GPs: 100s, Number of Pathology Providers :3

Does not include external evaluation of GP workflow due to beyond scope duration provided in quote. Alternatives need to be investigated and are

$70,280

4. Management Costs (6 months)

$163,400

5. Contingency – 20% $82,612

Total Excluding external evaluation – estimated to be between $50K and $320K

$495,672

The indicative costs for the subsequent business case to implement EDST nationally are outlined in the table below. The key assumptions for the national rollout costs are:

• A further six Desktop Vendors to undertake development (in addition to two Desktop Vendors participating in POC). A total of eight Desktop Vendors will participate in the ongoing support and enhancement activities

• A minimum of an additional 20 clinical contexts would need to be undertaken in the national rollout phase for adequate evaluation

• Determine the type of distribution model of the clinical knowledge (i.e. either (i) a download model from the RCPA Manual website or (ii) a full web services and hosting model via a third party).

• A third party user interface for knowledge creation has been assessed as being acceptable and justifiable during the POC and an interface can produce downloadable tables that can be stored on the RCPA Manual website.

Costs Item/ Category Relevant Parameters and Metrics Initial Costs Annual Cost 1. Rules creation (includes

clinical governance functions, development & testing)

Number of clinical contexts: additional 20 – as a minimum Cost of third party user interface licence included.

Model option to be selected based upon POC evaluation and funding. (i) RCPA hosted and download model (ii) Third Party Hosted and web services model

Initial costs : (incurred in addition to selection of model option)

Ongoing costs pa Option i RCPA Hosted download – Or Option ii: third party hosted web services

$400,000

$151,200 Or

$459,000


40

Costs Item/ Category Relevant Parameters and Metrics Initial Costs Annual Cost 2. GP Desktop Solution Number of Vendors: 8 [additional 6

+ 2 from POC] Initial costs: Ongoing annual costs:

$240,000

$157,120 3. Pathology LIS and

implementation at Pathology

Configuration of LIS and associated Pathologists review of workflow impact :

$60,000

4. Implementation at GP Practices

Configuration of solution and training of GPs – [costs to be absorbed] Promotion of use by RCPA, RACGP and or ACRRM etc. through quality improvement themes. [Not included in costs - pending further input]

$0

5. Management (Based upon 12 months)

RCPA Project Management and Operational costs:

Ongoing annual operational costs

$436,400

$16,800

Total Initial costs

PLUS ongoing annual costs

Option i download hosted at RCPA or Option ii web services hosted at third party

$1,136,400

$325,120 or

$632,920


41

5.3 Feasibility

12. Technical Changes

The technical changes for the Desktop Vendors may not involve significant time and resources. Feedback received from Desktop Vendors has indicated the proposed desktop solution is technically achievable within a reasonably limited time frame. The more challenging issue relates to the development of a critical mass of content [recommended tests and supporting information] via expert knowledge groups. The planned POC will only be based upon two clinical contexts and feedback from the GPs, Practice managers and some Desktop Vendors is that a critical mass of clinical contexts is required prior to implementing the proposed solution.

The technical changes required for rules generation (via third party user interface) and subsequent distribution/ access (RCPA or third party) is more significant. The options proposed in this report should be investigated in the POC.

The following four technical items were identified during the information gathering phases:

• The concerns of laboratory directors in relation to the implications of the solution. Preliminary indications (from a limited set of clinical contexts investigated) are that the impact on resources may not be considerable, however this needs to be validated by inviting a number of pathology practices to take part in the POC.

• The risk that the cost of development and ongoing support costs for Desktop Vendors may be prohibitive. Indicative costs have been prepared based upon preliminary solution design. These costs necessitate further validation. These indicative costs are detailed in this report, and will require validation as part of the POC.

• The reluctance of GPs to adopt the new solution based upon concerns regarding possible impact of their clinical independence and changes to workflow. The GP feedback on the solution indicated this concern may be addressed by a non- prescriptive design.

• The clinical governance required to manage the clinical risk associated with the solution may be considerable. The preferred approach would be for this body of work to be addressed by supporting an RCPA-led Clinical Governance Committee separate to the RCPA Manual Editorial Committee. The costs associated with the time and resources are detailed in this report.


42

5.4 Alignment of EDS with other RCPA Projects The Project aligns with two recently completed RCPA projects: PITUS23 and the “RCPA Manual Update”, and both projects were supported with funding from the Department of Health.

13. PathSupport and PITUS

The aim of the Pathology Information Terminology Units and Standardisation project (PITUS) is “to continue the development of standards for information in pathology requests and reports and to facilitate their uptake by pathology practices and their customers to allow safer and better quality use of pathology”.

The PathSupport Project uses the standards developed by the PITUS Project and specifically the use of standard Preferred Terms and SNOMED codes for requested tests. This information will be rigorously captured in the knowledge provided to the desktop systems. PathSupport may positively facilitate the uptake of these terms and codes by the (GP) customers of pathology practices and through that facilitate uptake by the pathology practices.

14. PathSupport and RCPA Manual

The RCPA Manual Update Project has been guided by the QUPP objectives including “support for referral practitioners that are informed and facilitated by best practice professional relationships and protocols between referrers and providers”. One of the specific objectives of the project is to “increase the volume of monthly hits and page views of the RCPA Manual Website”.

The PathSupport Project’s key alignments with the RCPA Manual are

• Leveraging the knowledge currently contained in the Manual (including the 623 clinical problems and the 34 PDSTs) for the development of the rules that are distributed throughout the GP community via the desktop software. It is likely however that the format of content will require significant modification to able to be incorporated into the proposed PathSupport solution.

• Supporting the RCPA Manual project objectives by providing simple and ready access to the RCPA Manual website through context sensitive links from the desktop system every time a GP requests pathology using the proposed PathSupport solution.

5.5 Analysis of Alternatives Alternative solutions examined during this project primarily focused on the use of existing health pathway solutions as a vehicle to access the RCPA endorsed request sets. By using a separate solution interface (e.g. Health Pathways or Map of Medicine) to access the recommended tests – the GP would have been required to exit their current requesting

23 The future of further PITUS (Pathology Information, Terminology and Units Standardisation) Projects is the subject of a current funding request. Nonetheless the output from previous PITUS projects (including PUTS) has provided the basis for at least the POC.


43

module. This alternative was assessed based upon acceptability to the GPs. When it was established this alternative would negatively impact the GP workflow and would not be acceptable to the GPs, a cost analysis was not undertaken.


44

6 Potential Next Steps

6.1 Undertake a Proof of Concept Undertake a Proof of Concept (POC) to gauge the acceptability of the proposed solution. It would involve two Desktop Vendors with a maximum of 100 GPs trialling the POC in cooperation with a number of pathology practices. The POC would also trial the generation, distribution and testing of a series of suggested request sets (for Antenatal Screening and for Hyperlipidaemia/ Lipids Monitoring) and the governance of those processes in cooperation with the RACGP24 and others as required. It would include an evaluation of the three benefits outlined above (but not included in indicative costs). Furthermore, it would propose to refine the three costs identified and critically also engage with the Pathologists, Laboratories, Desktop Vendors and GPs to assess the feasibility of the solution.

The clinical contexts that may be included in the POC will be Antenatal Screening and Hyperlipidaemia/ Lipids Studies. The POC’s would leverage to content of earlier PDST work to develop the content of the rules for distribution to the desktop systems, however the current format of these PDST’s would be subject to modification.

In addition to the key evaluation aspects mentioned above, the POC will also provide the opportunity to address some of the messages identified during the information gathering. As an example:

The GPs indicated that there is high demand for their desktop pathology modules to be upgraded to provide guidance in the selection of pathology tests.

A key element of the POC and evaluation would be to model costs for a national implementation. It would also provide a platform for suggestions for policy implications. This will assist in establishing a more informed timeline for a subsequent national implementation.

6.2 Undertake Formal Alignment of Guidelines and Pathway

Further discussions are required between the RCPA, the RACGP and other sources of clinical guidelines in pathology requesting, to determine a clinical governance model for producing and maintaining the test lists. This may result in a broader representation on a RCPA led Clinical Governance Committee. The features of the proposed solution and the emergence of health pathways may result in for a change of format of the current RCPA Manual PDSTs.

6.3 Undertake a Funding Application The PathSupport Project has identified a possible solution that has the potential to significantly progress the key objective associated with the original funding agreement. That is “to optimise the appropriate use of pathology testing to enhance high quality diagnosis and treatment in patient care”25

24 The RACGP has indicated a willingness to collaborate in future projects. 25 Department of Health project funding agreement page 1 “Program Description and Objectives”


45

Furthermore the PathSupport Project represents only the first part of the second phase mentioned in Clause 35 of the Pathology Funding Agreement. The subsequent part of the second phase incorporates the “significant progress on the development of Electronic Decision Support tools” and “assistance in the development of a business case to encourage the uptake of these tools by requestors”.

The RCPA proposes to undertake a funding submission along the lines of the model and costings described in this report to progress the next phases.


46

7 Project Budget

There is no invoice associated with the Final Report as per Item F.6 of the Schedule. However, there is currently unspent funding of approximately $28,000 for this Project. In the recently submitted Final Report for the RCPA Manual Update project, the College requested that any unspent funds for this project be approved for future refinements of the PDST’s – pending the outcomes of the proposed PathSupport Solution.

The PathSupport Project Team now recommends that the unspent funds from both of these projects are reallocated toward the next phase of developing the proposed POC outlined in Section 6.1. The RCPA has undertaken some preliminary analysis to identify relevant stakeholders and the steps involved, in particular, placing an emphasis on capturing the clinical content of the relevant algorithms from the Scoping Project into a form suitable for interface with GP desktop solutions. The proposed timeframe to undertake this POC is six months.


47

Appendix A: Final Report Deliverables and Activity Performance Indicators

Final Report Contract Deliverables Item of the Schedule of Deed of Variation 1 required the RCPA to address the following:

Report Requirement Location of information. Summary of findings GPs, Desktop Vendors, pathologists and Laboratories

GP summary: See Section 2.10 and 3.2 Desktop Vendor summary See Section 2.8 Pathologists and Laboratories summary: See Section 2.6

Design Principles

Feasibility of buy/build

GP Input Section 3.2 Desktop Vendor input: Section 2.8 Solution design (GP interface): Section 4.2 Solution design (Knowledge) Section 4.3 Buy or Build: Section 3.2

Indicative Costs and Benefits of EDS for pathology requesting

Costs: See Section 5.2 Benefits See Section 5.1

Summary of project activities and potential next steps

Project Activities: Section 1.3 Activity performance indicators listed below Detailed in the PathSupport Project Plan v1.0 dated 30th April 2014 (available from RCPA PMO) Next Steps: Sections 6.1 6.2 and 6.3


48

15. Activity Performance Indicators (API)

The following ten Activity Performance Indicators are requirement as per Item B3 of the PathSupport Funding Agreement Schedule.:

1. Project Management and Governance

Indicator Description Location of information and/or commentary. Formation of Governance bodies & framework to control & monitor the progress of the project & to guide & authorise clinical & technical components

Please refer to Appendix B. This information reflects that the activities were conducted in accordance with the Project Plan provided to the Department on the 30th April with variations as noted.

Appointment of Committee Members, Project Manager, Project Officers & chairs of the Knowledge Expert Groups engaged

Please refer to Appendix B. This information reflects that the activities were conducted in accordance with the Project Plan provided to the Department on the 30th April with variations as noted

Agreed approaches for information gathering / survey details for GP’s, Desktop Vendors, Pathologists & Laboratories

The approach for the information gathering is described in the various sections including Sections 2.5, 2.7, 2.9 and 3.1. The key elements of the information gathering approaches were agreed with the Project Steering Committee and with the Project Owner. Where necessary additional detailed agreement was sought through relevant PSC members Issues No significant issues regarding the approach were identified.

Agreed approach to assess the build / buy / adopt options & progress the implementation concepts.

The approach to assess the build / buy options was reassessed following the information gathering with the GPs and the subsequent identification of the proposed solution. Please refer to sections 3.1 and 3.2 Identification of the proposed solution assisted with eliminating the need for a separate EDST, hence the build / buy options were no longer a consideration. Please see section 5.5

The preferred approach was for EDST to be delivered through modifications to existing Pathology Requesting Modules in Desktop systems. This reassessment gained consensus with Project Owner and the Project Steering Committee. Issues No key issues identified regarding the approach


49

2. Consultation & Assessment of Information: Indicator Description Location of information and/or commentary. Meetings held with relevant stakeholders, organisations, clinicians and technical advisors. Survey taskforce established and utilised to develop survey tool.…

Please refer to Section 2 and Section 3. A survey tool was not utilised during the Project when the PSC elected to conduct the GP information gathering through the focus groups.

GP attitudes gathered and analysed

Please refer to sections 2.10 and 3.2. Issues: The only issue identified related to the selection of a subset of GPs for the second round information gathering. This issue was addressed by requesting the Medicare Local representatives to ensure this included users of a variety of desktop solutions.

Technical user requirements gathered, Evaluation of existing decision support systems commenced

The technical requirements are captured in sections 2.8 and 3.2.

See API “Evaluation of existing decision support systems” below re the evaluation

3. Development of Reports: Indicator Description Location of information and/or commentary. Set of design principles used as the basis for a clinical knowledge repository and decision support in general practice including the review of the 10 “design concepts” identified during the Scoping Project

The sections 2.2, 2.8 and 3.1

Evaluation of existing decision support systems completed.

In the context of the existing decision support systems not being part of the proposed solution – this activity was superseded with the key technical requirements requested by the Desktop Vendors and the GPs (see 2.8 and 3.2)

Potential technical costs summarised that might inform the subsequent future activity

Please refer to section 5.2 for the relevant cost details and section 5.1 for the benefits


50

Appendix B: Project Governance Reporting.

The formation of the various project governance groups and project teams (and the subsequent appointment of personnel to the relevant positions) was guided by the Project Owner in consultation with other RCPA executives as needed.

Project Governance

For the majority of the project duration the project structure was as follows:

During the final stages of the project, the Project Manager reported to the Head of the RCPA PMO who in turn reported to the Project Owner.


51

16. Project Team Membership

The individuals who fulfilled the above roles were

• The Chair of the Project Steering Committee: A/Prof Peter Stewart • The Project Owner was Dr Bronwen Ross • The Project Manager was Mr Roger Hewitt. • The RCPA PMO Head (Ms Vanessa White) undertook key project reviews during the

last phases of the project and provided the line of reporting between the Project Manager and the Project Owner. .

17. Steering Committee Membership

A/Prof Peter Stewart – Chair Chemical Pathologist and RCPA President

Dr John Bennett – GP Representative General Practitioner, University Health Service, The University of Queensland

Dr Ian Clark – Pathology Australia Representative Anatomical Pathologist, and CEO of Capital Pathology

Dr Glenn Edwards - Catholic Health Representative Chemical Pathologist and National Medical Director, St John of God Pathology

A/ Prof Michael Legg - RCPA Informatics Advisory Committee Representative Consultant Health Informatician,

Dr Dominic Mallon - NCOPP Representative Clinical Immunologist and Chief Pathologist, PathWest

Dr Linda Mann – GP Representative General practitioner – Leichardt NSW

Dr Bronwen Ross – Project Owner Deputy CEO, RCPA

18. Steering Committee Meetings and Project Reporting

There were four Steering Committee Meetings and one Steering Committee Teleconference. All meetings had an agenda, pre reading and minutes. Copies of these documents are available from the RCPA Project Management Office.

The fortnightly reporting was changed to weekly reporting.

The Project Manager provided the Head of the RCPA PMO with fortnightly status reports of activity issues and risks. Copies of these documents are available from the RCPA Project Management Office.


52

Issues that occurred during the process and how they were addressed

There were only minor issues encountered during the formation (and subsequent appointment of the relevant parties) of the Governance bodies and the framework to control and monitor the project. They were:

• There have been two Deeds of Variation issued under the contract. Firstly, the initial GP workshops were conducted through the RCPA by the Project Manager rather than through a secondary funding agreement with a University group. While this sped up the process (necessary due to the delayed execution of the project funding agreement), it contributed to the overspend on the Project Manager’s time. Secondly an extension of time variation was approved late in the project due to the need for further detailed work on the costing and feasibility aspects of the proposed solution. This time extension also contributed to the overspend in the Project Manager’s time however did not impact on the overall cost of the project which is underspent, mainly due to fewer sitting fees paid to GPs than expected.

• The lines of reporting between the Project Manager and the Project Owner were changed. This change resulted from a restructuring of the reporting of all RCPA projects. As part of that restructure the Project Manager reported to the Head of the RCPA’s Project Management Office rather to the Project Owner

• The Project Manager was supported by a number of administration and finance staff on an as needed basis rather than by a dedicated Project administrator.

• The appointment of the Steering Committee was in accordance with the Project Plan and included three of the five KEG Chairs. The engagement with the other two KEG chairs was achieved through one participating in one of the Focus Groups and both being updated verbally with project developments.

electronic decision support tools (edst) …...department of health’s funding agreement for...

Documents