Understanding the Increasing Impact of Clinical Researchon Obtaining Product CE MarksGert Bos and Suzanne Halliday BSI Healthcare

Medical Device Clinical Research ConferenceQ1 productions, London3 November 2009

2

This presentation …

Ten changes in relation to clinical evaluation

Ten “mistakes” that will delay time to complete Notified Body review.

3

This presentation …

Sources of information:

1. 93/42/EEC – amended by 2007/47/EC September/2007

2. Harmonised European Standards• EN ISO 14155 -1 -2 :2003

3. Guidance Documents• MedDev 2.7.1• MedDev 2.12 /2• GHTF Study Group 5 N2R8• NB-MED 2.7.1

10 Changes in clinical evaluation

1. Article 1, 14a and15 - definitions, databank, transparency trials2. Annex I – ER 14 6a - clinical evaluation mandatory3. Annex X – 1.1 - Risk/Benefit based on clinical data4. Annex X – 1.1 - Defined & Methodologically Sound Procedure 5. Annex X – 1.1a - Clinical Investigations on Implantable Devices6. Annex X – 1.1b - Evaluation Shall be Documented7. Annex X – 1.1c - Evaluation Must be Actively Updated8. Annex X – 1.1d - No Clinical Data justified with Risk Management9. Annex X – 2.3.5 - Adverse Events to All Competent Authorities10.Annex VIII - Devices for Clinical Investigation

4

1. “The clinical evaluation is based on a comprehensive analysis of available pre- and post market clinical data …”

Annex X 1.1 – “The ‘clinical evaluation’ is based on:

1.1.1. … critical evaluation of the relevant scientific literature …

1.1.2. … critical evaluation of the results of all clinical investigations …

1.1.3. … critical evaluation of the combined clinical data …”

Search strategy

• Write search protocol

• Document search terms and databases

• Systematic reviews & peer reviewed research work

• Use people qualified in data researching

6

2. “The clinical evaluation is based on a comprehensive analysis of available pre- and post market clinical data …

…relevant to the intended use of the device in question, including clinical performance data and safety data.”

Relevance of data

• Look at Nr of patients, diagnosis, follow up Clinical safety and clinical performance Primary and secondary endpoint …..

Equivalence non-equivalence

8

3. “Document the selection criteria to be applied to published literature and justification for their choice.”

Annex X 1.1 b – “The clinical evaluation and its outcome shall be documented. This documentation shall be included and/or fully referenced in the technical documentation of the device.”

10

Methods Literature Review and Meta-AnalysisTo address the question about the outcomes of Total Knee Arthroplasty, the authors conducted a systematic literature review from 1990 to October 2009. The titles and abstracts of the resulting 3,519 references were then screened, using our inclusion criteria: primary total knee arthroplasty studies

more than 100 knees per studybaseline data and post-op outcomes data providedexperimental or quasiexperimental study designEnglish languagetricompartment

All articles that appeared to meet the screening criteria were abstracted by trained abstractors. Of the original results, 611 references either met the inclusion criteria or needed further screening of the full article to determine if they met inclusion. Of these, 62 studies reported pre- and post-Total Knee Arthroplasty functional data using at least one of the four established measures we relied on: Knee Society score, Hospital for Special Surgery score, WOMAC or SF-36.

Search Strings for Total Knee Arthroplasty Outcomes:The literature search was done using the following combination of MeSH headings, keywords,and publication types: arthroplasty, replacement, knee OR knee prosthesis OR "knee replacement" OR "knee implant" OR TKAR OR knee injuries OR knee joint.

Inclusion Criteria

AHRQ Systematic Review

4. “Appraise each individual data set, in terms of its relevance, applicability…”

Annex X 1.1 – “The evaluation of this data, hereinafter referred to as ‘clinical evaluation’, where appropriate taking account of any relevant harmonised standards, must follow a defined and methodologically sound procedure …”

12

MedDev 2.7.1

Anonymous Client Submission

13

“Equivalence” per MedDev 2.7.1.

Anonymous Client Submission

14EN ISO 14155-1:2003

EN ISO 14155-1:2003 does not use “relevance” or “equivalence” but does use “similar”:

• technology• critical performance• design• principles of operation• patient or study populations• indications for use• severity and type of disease or condition• conditions of use• intended use

15GHTF N2R8

GHTF N2R8 uses “comparable” and “similar”:intended use:• clinical condition being treated• severity and stage of disease• site of application to/in the body • patient population

technical characteristics:• design / specifications• physiochemical properties• deployment methods• critical performance requirements• principles of operation• conditions of use

biological characteristics:• biocompatibility of materials in contact with body fluids/tissues

5. “Appraise each individual data set, in terms of its relevance, applicability…

… quality and clinical significance.”

Annex X 1.1 d – “Where demonstration of conformity with essential requirements based on clinical data is not deemed appropriate, adequate justification for any such exclusion has to be given based on risk management output ...”

Quality and significance

• Weight results of individual papers or studies with a “contribution to demonstrating overall performance and safety of the device.”

• “… evaluate potential sources of bias” i.e. results reported on the same patients, multiple times? i.e. results from the designing surgeon better than all

others?

17

6. “Where a clinical investigation has been carried out by or on behalf of a manufacturer, it is expected that documentation relating to the design, ethical and regulatory approvals, conduct, results and conclusions of the investigation needed for the clinical evaluation will be available for consideration, as appropriate.”

Annex X 1.1 c – “The clinical evaluation and its documentation must be actively updated with data obtained from the post-market surveillance. Where post-market clinical follow-up as part of the post-market surveillance plan for the device is not deemed necessary, this must be duly justified and documented.”

19

Clinical Investigation Data:

• Expected to be designed, conducted and reported in accordance with EN ISO 14155 -1 & -2.

• Documentation to be available:− Clinical Investigation Plan (study design, selection of

participants, treatment, blinding of participants and investigators, dependent variables, follow up, statistical analyses).

− CIP amendments, rationale and evidence of notification of Regulatory Authority.

− Ethics Committed opinion, approved informed consent forms, patient information documents.

− Case report forms, monitoring and audit records.− Regulatory Authority approvals and correspondence.− Final report – signed and dated.

GHTF N2R8

20

“Where the clinical investigation(s) was performed outside the EU, the manufacturer must demonstrate that the use of the device (including clinical practice and techniques) and patient population are equivalent to those for which the device will be used within the EU.”

MedDev 2.7.1

• Declaration of Helsinki

•ISO 14155 / GCP

• Pharma directive clinicaltrials outside EU

7. “What clinical investigation documentation/data should be used in the clinical evaluation?

Clinical Investigation Plan – amendments, rationale and evidence of notification of Regulatory Authority”

Annex X 2.3.5 – “All serious adverse events must be fully recorded and immediately notified to all competent authorities of the Member States in which the clinical investigation is being performed.”

22

8. “Outline clearly the conclusions reached about the safety and performance of the device from theevaluation, with respect to the intended use of the device…”

Is overall residual risk acceptable? (7)

Yes

NoDo medical benefits outweigh the overall

residual risk? (7)

YesYes

No

24

Clinical Evaluation Report:

• Conclusions:− For each proposed indication clinical evidence

demonstrates conformity with relevant ERs:− With or without clinical study on device itself

− Device performs as intended

− Device does not pose any safety concerns to either recipient or end-user

− Risks associated with use of device acceptable when weighed against benefits to patient

GHTF N2R8

9. “Outline clearly the conclusions reached about the safety and performance of the device from theevaluation, with respect to the intended use of the device…

…State whether the risks identified in the risk management documentation have been addressed by the clinical data.”

Annex X 1.1 d – “Adequacy of demonstration of conformity with the essential requirements by performance evaluation, bench testing and pre-clinical evaluation alone has to be duly substantiated.”

Holistic view

• Risks in RMF addressed in clinical evaluation report

• Risks from CIP included in RMF

• Residual risks in RMF and in IFU identical

• Positive acceptance of residual risks

26

10. “The clinical evaluation report should be signed and dated by the evaluator(s) and accompaniedby the manufacturer’s justification of the choice of evaluator.”

Annex X 1.1 – “As a general rule, confirmation of conformity with the requirements concerning the characteristics and performances referred to in ER1 and ER3, under the normal conditions of use of the device, and the evaluation of the side-effects and of the acceptability of the benefit/risk ratio referred to in ER 6, must be based on clinical data.”

28

Clinical Evaluation Report:

• Signed & Dated by someone “suitably qualified.”

• Possess knowledge of:− Device technology and application.− Research methodology (design and biostatistics).− Diagnosis and management of conditions to be treated or

diagnosed.

GHTF N2R8

29

Clinical Evaluation – Ten Biggest Mistakes …

1. “Google” used to identify two or three papers, that are then summarised to demonstrate compliance.

2. Literature review includes ONLY published studies of mechanical testing, computer modelling, animal and in vitro experimentation.

3. Many papers identified, however not all selected for inclusion, without justification.

4. Devices described as “substantially equivalent.”

5. Equal reliance on clinical data from one randomised control trial and one expert opinion.

30

Clinical Evaluation – Ten Biggest Mistakes …

6. Clinical Investigation – without notifying a EU Competent Authority or without any other regulatory authority approval.

7. Clinical Investigation – Agreed a protocol with Competent Authority for 45 patients followed for 52 weeks.Want to CE mark after 20 patients followed for 25 weeks = protocol deviation.

8. Conclusion that the device under review is “substantially equivalent” to other devices in the published literature.

9. No conclusions that benefits outweigh risks.

10. Unsigned and undated Clinical Evaluation.

Literature route easier option ??• Clinical investigations

+ targeted+ precise with good design+ better overall acceptance

- Expensive- Time consuming- Logistically cumbersome- Internal know how

• Literature route

+ less exensive+ fast+ low risk

- Depends on quality of literature- Historical data- No perfect match of parameters- Lower acceptance

32

Contact UsName: Gert Bos

Title: Head of regulatory and Clinical Affairs

Address: BSIKitemark House, maylands AvenueHemel Hempstead, HP2 4SQ, UK

Telephone: +44 (0)1442 278664Fax: +44 (0)8450 765601

Email: Gert.Bos@bsigroup.comLinks: www.bsigroup.com/healthcare