eurorec-sdmi/prorec.si workshop "quality labelling and certification of electronic health...

1EuroRec-SDMI/Prorec.SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe”

Portoroz, Slovenia, May 5th 2008

EuroRec Certification Guidelines and Procedures



Generic Version Model Certification Guidelines & Procedures

Introduction

What is Certification?

Market Assessment

The Management of the Certification Process inEurope

Generic Certification Process

Generic Certification Documentation/Manual

Issues/Options in introducing Certification Legal & Management Organisation & Process

Next steps



Introduction

Generic Version, Model Certification Guidelines & Procedures

Describes “what” to certify and “how” to perform the certificationDocuments preliminary model - generic guidelines andproceduresGeneric documentation - Benchmarking Manual for

Certification and quality labelling of EHRs

In the context of serving as a “template” for countries and/or organisations in Europe planning the certification and quality labelling :

Outlines the process, organisational, management & legal issues to be addressed in order to introduce certification and quality Labelling of EHRs



What is EHR Certification?Critical component in ensuring the quality of EHR systems is the introduction of a Certification and Quality Labelling process

Certification of EHRs can be defined as the procedure and action by which a body duly authorised and recognised as a legitimate provider of this service evaluates and certifies an Electronic Health Record system as meeting predetermined quality standards and certification criteria

Certification addresses functional quality of EHR systems considered as end products.

Certification or quality labelling processes can only evaluate the functionality of a software application.

The technical design (the way the function was implemented) cannot be subject of a certification or labelling exercise. The latter one remains the full responsibility and the exclusive authority of the software vendor.



Market Assessment - Certification of EHRs

Questionnaire administered through national ProReccentres to 4 categories of Respondent

Developers/Suppliers Policy Makers

Purchasers System End-Users

Respondent

Developers/Suppliers 33Policy Makers 31Purchasers 28System End-Users 33

Total 125

Country

Belgium 9 Bulgaria 62 Denmark 2Germany 6Ireland 22Romania 13Slovenia 11

Total 125



91% - 97% of all 4 respondent categories agree that the certification process is important to the development of EHRs

Developers/ Policy Makers Purchasers System

Suppliers End-users

88%-91% 94%-100% 54%-64% 73%-82%

believe that EHR certification will result in one or more of these benefits:

improve prospects of competing favourably in EHR market improve quality, safety & cost effectiveness of healthcare improve business process, including security, interoperability and data reliability allow the provision of a better service to patients

Above suggests need for special focus on purchasers





Developers/Suppliers 88% do not currently have certified EHR systems 66% plan to submit EHR system for certification 66% say their organisation prepared to pay for certification from a

nationally/internationally recognised certification body

Purchasers

61% interested in buying a certified EHR system knowing the financialcommitment this involves

but

only 36% say their organisation is prepared to pay extra for a certified system




All 4 respondent categories give a High rating to certification and other QREC products that will:

Define what an EHR product is expected to provide and how it will meet specifications (Range 81% - 93%)

Significantly reduce investment and associated risks of implementation of EHR systems by specifying interoperability requirements (Range 70%-89%)

High ratings in the main also given to certification and other QREC products that will:

Provide an independent, unbiased, professional and trustworthy quality assurance mechanism for EHR products

Support industry co-operation by convergence towards known and new national standards and criteria, improving interoperability of EHR systems

Improve the quality and safety of healthcare: by assessing the quality and suitability of EHR product by providing an inventory of resource by educating vendors/users on requirements of successful EHRs




A significant majority of all 4 respondent categories say they/their organisation would be interested in:

Using the certification process service - (82%-90%) Utilising a register of certified EHR systems - (90%-93%) Utilising a register of criteria that EHRs need to fulfil in order to gain certification – (82%-93%) Receiving advice on benchmarking processes for formal testing of EHR Products for certification and quality labelling – (86%-91%) Having access to libraries of terminology underpinning certification and quality labelling –

(81%-97%)

100% of Purchasers say their organisation would appreciate if:

suppliers/sellers of their EHR system had access to current up-to-date information regarding terminology underpinning certification

if seller had received advice on the testing of that system on the way to achieving certification

A number of key issues now to be finalised for certification process



How Certification will be Managed

EuroRec will act as a central repository of validated quality criteria and other relevant materials that can be used to harmonise European testing, certification & quality labeling and procurement specification for EHR systems

Rather than impose particular certification models or specific criteria on any member country EuroRec will foster, via ProRec centres and other channels, the progressive adoption of consistent and comparable approaches to EHR system certification and quality labelling

The management of the certification process in Europe will operate through the EuroRec Institute and the National Certification Authorities and/or the National ProRec Centres

The certification process may be carried out by the National Certification Authority itself, or a mandated subcontractor, - National ProRec centre or other agency



How Certification will be Managed

The role of EuroRec - initially through the Q-Rec project

Build and develop the Certification process centrally Distribute the Certification process to National Certification Authorities/ProRec

Centres Assist and support National Certification Authorities/ ProRec Centres in

introduction and adoption Providing guidance and education to all stakeholders in the market Auditors of the process Establish and Maintain Registers Maintaining the standards and certification criteria

The role of the National Certification Authorities/ ProRec Centres

Taking the Certification process and adapting for local needs. Be the centre of certification for each country

Establish and Maintain Registers (based on EuroRec registers) Maintaining the standards and certification criteria



The Generic Certification Process

Generic certification process stages are: Initial contact by supplier/developer/purchaser Formal application Application process

Contract between Certification Authority and ApplicantCertification Manual describing: Certification process

Statement of criteria for CertificationDocumentation to be supplied by vendorTesting Schedule

Test/Retest – 2 levelsPre-test self-assessmentExternal evaluation by C.A.

Pass/Fail Certification Register

Details certification status of each productSchedule of dates for retesting/recertification

Issue of Certificate/Seal - issued on successful completion of test Publication



Generic Certification Documentation/Manual

Generic Certification Documentation/Manual includes specific forms and statement of requirements in regard to: Application form Contract Instruction /information /guidelines for the Certification process Application Test Pack Vendor Documentation



Application Form

Application form, details:

Supplier’s name, address, contact details Product name, version, status (live. pre-live, development -

if product is live, details of at least one site where it is in use – location and contact)

Requested test date/alternative date Signature of director/company representative with

appropriate authority



Contract

Contract is completed between Certification Authority/Agency and Applicant

Contract to cover the certification process, specifying Certification Authority/Agency and Vendors rights, accountability, responsibilities, limitations, confidentiality, freedom of information and waivers.



Certification Manual

Instruction /information /guidelines for the Certification processDetails:

Overview of certification process How to apply Documentation required Criteria to attain certification Overview of range of functional, operational, implementation, standards

and user criteria used for tests Overview of tests and testing methodologies Approval process Retest and appeal process Use of Seals and Certification Fees



Application/Test Pack

Application /Test Pack contains:

Detailed description of certification process Detailed description of test process/methodology Documentation to be completed Supporting documentation required Detailed set of criteria to be utilised for testing Statement of Criteria Requirements for Certification regarding

Functionality Interoperability Security Data Structures, Codes and Standards Test Scripts/Scenarios to test each of the criteria



Vendor Documentation

Documentation to be supplied by vendor:

Software/Hardware Maintenance. Agreement(s) Employment contracts and agreements Procedures document System documentation (including operating systems and database) Hardware/communications specifications User Manual Conformance statement Training manual and arrangements Software/Product Licence Escrow agreement Licences from relevant Third party suppliers.



Implementing the Certification Process

A number of issues and have been identified. Options are proposed for consideration and decision by each national certification authority.

These are categorised into:

Legal & Management Issues/Options Organisation & Process Issues/Options



Legal & Management Issues/Options in the Certification Process

The major issues/options to be decided are:

1. Balance between Pan European vs. National based certification

2. Status of Certification/Accreditation Authority (C.A.) - Government vs. Non-Government

3. Mandatory vs. Voluntary Approach

4. Supplier Participation in Evaluation



1. Pan European vs. National based certification

Process issues/considerations comprise primarily the following:

Considerations here include the following:

Pan European based approach likely to: Be more efficient Assure consistency Prove more acceptable to suppliers Raise National difficulties

National based approach likely to: Be better supported at Country level Diminish consistency/efficiency Increase cost both to C.A. and suppliers Be opposed by suppliers

Joint Pan European/ National European certification - generic criteria/functionality National Certification – specific national requirements



Market Assessment

Should certification be organised/provided on:

A – a Pan-European basis B - a national basis C – a combination of A and B

Suppliers/Developers Policy Makers Purchasers End-Users

A 33% 13% 25% 42%

B 12% 6% 11% 9%

C 54% 74% 64% 46%



2. Certification Authority (C.A.) - Government vs. Non-Government


Government Body likely to: Be less acceptable to suppliers Be more independent

Non-Government Body likely to: Be more acceptable to suppliers Hold more monopolistic status Be dependent on suppliers



Market Assessment

Should certification be organised/provided by:

A – a Government organisation B – a Non-Government organisation


A 9% 52% 39% 33%

B 91% 45% 61% 58%



3. Mandatory vs. Voluntary Approach

Important to determine the approach

Considerations include the following: Mandatory approach likely to:

Give greater assurance to purchasers/end-users Be welcomed by higher performing suppliers Drive product improvements in lesser performing suppliers Create potential difficulty for new entrants

Voluntary approach likely to: Be generally more welcomed by suppliers Effectively produce the same participation outcome as

mandatory approach Create more amenable relational arrangement with suppliers



Market Assessment

Should certification be A Mandatory or B Optional?


A 15% 52% 85% 85%

B 85% 45% 11% 12%



4. Supplier Participation in Evaluation


Supplier participation likely to: Enhance supplier confidence Assure peer status Enhance supplier acceptance Raise questions on objectivity Potentially undermine business confidentiality Advantage larger/more established suppliers

Supplier non-participation likely to: Assure greater objectivity Enhance procurer acceptance Improve confidence of and acceptance by funding authorities To raise arguments on feasibility of the requirements, correctness of the tests

and professionalism of the process



Should developer/supplier participate in certification process? Suppliers/Developers Policy Makers Purchasers End-User

Agree 100% 93% 86% 73%

Market Assessment



Organisation & Process Issues/Options in the Certification Process

The organisation and process issues/options to be decided are: 1. Audit vs. Self-Assessment / Surveyor Review Evaluation Approach

2 Audit/Surveyor Workforce

3. Quality Assurance vs. Quality Improvement Focus

4. Scoring / Rating Scheme

5. Certification Award

6. Evaluation Event For Certification

7. Scheme Review and Evaluation



1. Audit vs. Self-Assessment / Surveyor Review Evaluation Approach


Audit Approach likely to: Be more rigid and extreme Lack nuance Potentially unfairly advantage larger/more established suppliers Be more objective

Self-Assessment /Surveyor Review Approach likely to: Be more rounded in outcome Avoid extremes Be better able to reflect potential of suppliers Be generally more acceptable to suppliers Potentially be more open to surveyor persuasion Potentially be less objective



2. Audit/Surveyor Workforce

Important considerations here include:

Competence / personal profile considerations Job specification Part-time (temporary) vs. full time (permanent) appointees Peer Status Credibility Calibre Selection Training Knowledge / skills maintenance



3. Quality Assurance vs. Quality Improvement Focus

Considerations here include the following: QA approach likely to:

Be more rigid Distance C.A. from procurers and suppliers Minimise value return of scheme in the context of its potential to develop procurer /

supplier synergies

QI approach likely to: Add significant value for procurers/suppliers Encourage and benefit smaller / start up suppliers Optimise value return of scheme in the context of its potential to develop procurer /

supplier synergies Enhance engagement between C.A. and procurers/ suppliers Be more resource demanding Further EuroRec’s mission objectives



4. Scoring / Rating Scheme


Pure numeric scoring / rating vs. combined numerate / judgement scoring / rating

Standards / criteria weightings Mandatory standard / criteria Minimum score / rate requirements

Decision on scoring/rating scheme will impact on final criteria definition



5. Certification Award: Pass / Fail vs. Graded


Pass / fail approach likely to: Benefit established / large scale suppliers Militate against small scale suppliers Potentially inhibit market entry Improve certainty for procurers

Graded approach likely to: Benefit small scale suppliers Encourage / assist new entrant suppliers Create less certainty for procurers



6. Evaluation Event For Certification

Considerations here include:

Necessity to develop a clear and unambiguous template setting out the process to be followed for the evaluation event

Imparting details to / training suppliers with respect to the process

Articulation of the process and mechanism by which the evaluation outcome is converted into a certification award

Need to provide for a formal and objective appeals mechanism with respect to a contested evaluation / certification outcome



7. Scheme Review and Evaluation

There is a need to incorporate provision in the scheme process for:

Scheme review (3/5 year cycle)

Continuous scheme evaluation (i.e. to establish extent to which scheme is meeting its stated objectives / expectations)



Next Steps

Utilise the pilot implementations in Europe of a quality labelling and certification process for EHR systems in compliance with the “good practice requirements” elaborated by EuroRec

Validate Model - generic guidelines and procedures Generic documentation - Benchmarking Manual for

Certification and quality labelling of EHRs

Determine and agree and – on a Country/European level Options are that are proposed for consideration and decision by

each national certification authority

Implement on a Country and European level Quality labelling and certification process for EHR systems