risk based computer validation - cbinet.com of a risk based csv approach ... risk based computer...

Risk Based Computer Validation

Lizzandra Rivera

Sr. Manager Quality Computer Systems Compliance

Alexion

2

Agenda

• What is risk based computer validation?

• Key components of a risk based computer validation

program

• Strategies for aligning current practices into a risk

based computer validation program

• Risk Based Computer Validation Strategies

3

Before we begin…

• Quick Census:

○ Who is in the quality organization?

○ Who is in the engineering organization?

○ Who is in the IT organization?

○ Others?

4

Let’s talk Risk

• What is Risk?

• Definition:

○ /noun/: (1) possibility of loss or injury (2) someone or something that

creates or suggest a hazard

○ /transitive verb/: (1) to expose to hazard or danger

5

For Computerized Systems…

• ISPE GAMP5

○ Section 5 – Quality Risk Management

○ “Systematic process for the assessment, control, communication and

review of risk”

6

Questions for Attendees

• Is the risk associated with a training system the same

as a process control?

• Should we validate a training system the same way as

a process control system?

• What about a document management system?

• Which one has the higher level of risk?

7

Risk Based Computer Validation

• Risk based validation is an approach for validating

computer systems that is based on the system risk

level

• This approach is also based on the risk level of the user

requirements and their impact to the product and

patient safety

What do you need in place?

9

Components of a risk based CSV approach

• Must have:

○ Document hierarchy that enables a risk based approach

• The document hierarchy requires the following:

○ Validation Policy

○ Computer Validation Standard

○ Computer Validation Procedures

10

Document Hierarchy

Validation Policy

Computer Validation Standards

Site Procedures

11

Document Hierarchy

Validation Policy


Site Procedures

12

Validation Policy

• High level document that describes the requirements for a risk

based validation program

• Provides alignment with regulatory requirements and industry

standards

• Provides the high level roles and responsibilities for each

functional area

13

Validation Policy

• Includes the following:

○ Process Validation

○ Equipment Validation

○ Risk Based Computer System Validation

○ Shipping Validation

○ CIP & SIP Validation

14

Document Hierarchy

Validation Policy


Site Procedures

15

Computer Validation Standard

• Describes the specific requirements for the risk based

computer validation program

• Provides alignment with Regulatory requirements and Industry

standards

16

Computer Validation Standard (cont.)

• CSV standard should include the following information:

○ Purpose

○ Scope

○ Process Requirements

○ Roles and Responsibilities

○ Validation Planning

▪ Compliance Assessments

▪ Validation Plan

▪ Risk Assessment (system level & requirements)

▪ Supplier Assessment

17


• Requirements Definition○ User Requirement Specification

○ Functional Requirement Specification

• Design & Coding

• Development Testing

• Risk Based Qualification Activities

○ Commissioning

○ Qualification: IQ, OQ, & PQ

○ Supplier Assessment

18


• Deviation and Discrepancy Handling

• Traceability Matrix

• Validation Reports

19


• Discuss at a high level Operation and Maintenance processes:

○ Incident Management

○ Change Management

○ Configuration Management

○ Backup and Restore

○ Disaster Recovery

○ Security

○ Periodic Review

○ Decommissioning

20

Document Hierarchy

Validation Policy


Site Procedures

21

Computer Validation Procedure

• Document that describes the specific steps to validate computer systems based on their level of risk

• Translates standard requirements into “how to” actions that promote a risk based approach

• Provides roles and responsibilities for all impacted functional areas

• Provides specific instructions and detailed requirements for the creation, review and approval of risk based validation deliverables

22

Computer Validation Procedure

• Describes the Risk Based Process

• Right Balance

○ Too General – Doesn’t provide enough guidance

○ Too Specific – Hinders the process

▪ Specific Details can be moved to Document Templates

23

Legacy Documents – Strategies for Alignment

• Validation policy and standards need to be revised to align with

risk based computer validation strategies

• Validation procedures need to be revised to align with the risk

based standards

• Validation templates need to be revised to eliminate

duplication and non-value activities and content

• Assessment tools need to be created to implement the new

risk based approach

24

Strategies for aligning legacy practices

• The outcome of the system assessments are an input into the

risk based validation strategy

• Assessment tools

○ GxP assessment

○ Part 11/Annex 11 assessment

○ System level risk assessment

○ Requirements risk assessment

System Assessments – Tool Examples

26

System Assessments

• How do you determine the GxP and overall system impact?

○ Meetings

○ Endless debates

○ Relying on SME’s

27

System Assessments - Benefits

• Enables and document the agreed upon risk based approach

• Intended to provide consistency, efficiency and compliance

• Intended to provide consistent interpretations of the applicable regulatory requirements

• Eliminates misinterpretations

• Provides inputs to validation planning phase

• Supports the creation of User Requirements

• Cost efficient approach

28

GxP Assessment

• Initial assessment to determine whether the system has GxPimpact

• Identifies applicable regulatory requirements (high risk?)

• Supports the creation of URS statements related to regulatory requirements

• Provides input to the validation planning phase and risk assessment

GAMP Categorization can also be included here

29

GxP Assessment - ExampleQuestions Yes/ No

Process/Production

Does the computer related system perform functions or calculations affect any of the critical parameters of a product manufacture such as mixing times, heating/cooling temperatures, flow rates etc.?

Does the computer related system interface to any Inputs or Outputs (I/O) that control or monitor manufacturing parameters of a component or drug product?

Does the computer related system generate and/or store alarms, events or transactions that are generated during the manufacturing or packaging of a component or drug product?

Does the computer related system store data about the actual manufacturing parameters for a given component or drug product? (e.g., quantity of components added, mixing time, temperature, rpm, pressure, equipment used etc.)?

Does the computer related system perform manufacturing process control? (DCS, PLC, etc)

Does the computer system, either automatically or based on user inputs, formally initiate any actions that affect the processing of a component or drug product? (E.g. Does the Systems generate a work order that causes someone to make a change to the process?)

Does the computer system, either automatically or based on user inputs, formally approve any actions that affect the processing of a component or drug product?

Packaging and Labeling

Does the computer related system generate labels or formally identifies a component or drug product? (ex. Label printer)?

Does the computer related system store data about the actual packaging parameters for a given component or drug product? (e.g., quantity of components added, mixing time, temperature, rpm, pressure, equipment used etc.)?

Facilities, Equipment and Utilities

Does the computer related system control, monitor and/ or store data of manufacturing equipment configuration data over time or per batch?

Does the computer related system generate schedule, track and/or store equipment maintenance or calibration data?

Does the computer related system record the lots or batches that have been produced using a given set of equipment or procedures?

Does the computer related system control/monitor or store data that support manufacturing equipment operation or performance including Cleaning and Sterilization?

30

Part 11 & Annex 11 Assessments• Initial assessment to determine whether the system has Part 11

and Annex 11 impact

• Identifies applicable Part 11 and Annex requirements○ Open or Close System

○ E-records

○ E-Signatures

○ Operational controls

• Support the creation of URS statements related to Part 11/Annex requirements

• Provide input to the validation planning phase and risk assessment

31

System Risk Assessment

• System level risk assessment to determine whether the system is high, medium or low risk

• Simple risk assessment tool

• Should assess the risk impact to the following:○ Product Quality

○ Patient Safety

○ Compliance

○ Safety

○ Business Process

○ Complexity

Provides input to the validation strategy

32


• The outcome of the system level risk assessment should

support a risk based validation approach

Low Risk

toHigh Risk

Less Testing

toMore

Testing

System

Testing

33

What does this mean to us?

• Procedures should clearly describe the required validation

deliverables for each risk level

• High risk systems should have more testing and validation

deliverables

• Medium or low risk systems should require less testing of low

risk requirements

○ Medium or low risk systems procedures should allow combining

documents and integrating validation activities

34

Requirement Risk Assessment

• Performed to determine whether each requirement is high, medium or low risk

• The requirements risk assessment should assess the risk impact to the following:○ Product Quality

○ Patient Safety

○ Compliance

○ Safety

• The outcome of these risk assessments is the key component for the risk based computer validation

35

Assessment Tools

GxPAssessment


Requirement Risk

Assessment

36

How to apply your results? – Example 1

• System 1

○ GxP , e-signatures, closed

system

○ Medium Risk System

○ Category 3

○ Requirements:

▪ 50 high risk

▪ 30 medium risk

▪ 75 low risk

Approach :

- Test all high risk

- Sample medium low

- Lows will probably be

challenged as you test other

requirements

37


• System 1

○ GxP , e-signatures, open system

○ High Risk System

○ Category 4 with Cat 5 in-house

customization

○ Requirements:

▪ 100 high risk

▪ 30 medium risk

▪ 75 low risk

Approach :

- Test all high/medium risk

requirements

- Sample low

- Lows will probably be

challenged as you test other

requirements

38


• System 1

○ GxP

○ Low Risk System

○ Category 5 – (low complexity)

○ Requirements:

▪ 15 total requirement

Approach :

- Skip Requirement Risk

Assessment

- Test all requirements

39

Example Strategy Summary

• High risk systems should require testing all high and medium risk requirements

• High risk systems should require testing a sampling of low risk requirements

• Medium risk systems should require testing all high risk requirements with a sampling of medium and low risk requirements

• Low risk systems should require testing only high risk requirements and sampling medium requirements

Strategy should be clearly defined in either Plan or Procedure

Risked Based Computer Validation Strategies

40

41

History of the Problem

42

We are producing Two Things: Paper and Drugs

Look familiar ?!

43

Problem Statement

• How much is the cost of Validation?

○ Cost of validation – typically ~25% of total Capital

• How much TIME?

○ Inadequate cycle times

○ Inability to support timelines based on business needs

44

Challenges Impacting a Risk Based Approach

• Conservative interpretation of regulatory requirementsRegulation: “The system must be validated”

Interpretation: We must validate the system in-house and have the vendor also performing full validation!

Regulation: “The need for a vendor audit should be based on a risk assessment”

Interpretation: We must audit all software vendors regardless of their risk level

Regulation: “Systems should have a periodic review process that includes data integrity”

Interpretation: We must perform data integrity periodic reviews for all systems monthly!

45


• Overly conservative Quality unit

• Poor or very vague understanding of what is really high risk

• Lack of understanding of the risk of the status quo vs implementing new system/process

• Conservative and subjective assumptions about regulatory expectations during an audit:○ “I am sure that Health Authorities will be looking for….”

○ “We will be audited for xyz regulatory requirements”

46


• Validation ○ Lack of understanding of the Intended Use of the system

• Engineering○ Lack of understanding of the Regulations

• Quality unit○ Vague regulatory understanding related to computer system

validation

○ Non-technical resources supporting technical implementations

○ Assigning unqualified QA resources to support complex technical projects

47

Effect

• Delays in implementing critical systems that improve efficiency and COMPLIANCE

• Increased implementation cost

• Significant and unnecessary resource commitment

• Significant duplication of effort

• Moving target expectations!

Solution Strategies

49

What can be done?

• Ensure an accurate and clear understanding of high risk factors

• Build a Team:○ Resources with the appropriate experience and background Technical

skills, including the Quality Unit▪ Don’t have them? Train them!

○ Accurate and pragmatic understanding of regulatory requirements by all Team Members

○ Computer validation background and skills

• Clear understanding of the risk of the status quo vs changing the process

Computer Validation Risk Based Approach

Validation Planning , Protocols and Reports

50

51

Validation Planning

• Do you have a Computerized System Master Validation Plan?

• How many Validation Plans do you create for an implementation of a computerized system?

• How often do you create Validation Plans?

• Are you departing from your Master Plan?

More documents are not equal to higher quality!

52

Validation Planning

• A risk based approach for validation planning should consider the size and complexity of the system/implementation○ For large complex implementations or changes one Validation Plan

document should be adequate

○ For small or simple systems implementations the validation plan can be integrated into one of the protocols or change control records

53

Requirements Definitions

• Discussions and interviews○ Include key stakeholders

○ Ask specific questions

• Observation○ Observe business function

54

Requirements Definitions

• Workflow analysis

○ Map your process!

○ Assessment of workflows

○ Development of use cases

• Workshops

○ Classify “must have” and “nice to have”

55

Requirements Definitions Pitfalls

• Multiple requirements within a single requirement statement

• No formal change management process

• Poor scope management

• Not classifying requirements

• Ambiguous requirements

• Delivering a large document instead of clear and concise

requirements

56

Requirements Definition

• Requirements need to be clearly understood by the technical

team

• Unambiguous

• Should articulate the needs of the system owner and business

• Need to translate applicable regulatory expectations into clear

requirements

57

Requirement Definition Risk Based

• Integrate software and equipment requirements into one

document

• Keep document approvers to only those that are key

stakeholders

• Electronic record instead of paper

58

Validation Protocol

• Integrate and combine documents○ IOQ instead of individual IQ,OQ protocols

○ Benefit: One document instead of two reduces cycle times

• Keep document approvers to only those that are key stakeholders

• Electronic record instead of paper, consider paperless validation systems

59

Summary Reports

• For large projects a summary report should be created

• Protocols for routine validation activities can have a summary

page integrated into the protocol instead of a report

○ Summary page should summarize the protocol execution, closed

deviations and be approved with the protocol

○ Summary page should provide an statement on whether the protocol

met all the acceptance criteria

60

Risk Based Summary Reports

• Status Quo (The Problem)▪ A Final Report will be written following execution of the qualification exercise in Test

and Production environment. The report will provide a summary of the test results

generated during execution of the protocol and indicate if the acceptance criteria

listed were met. The report will be approved by the signature approvers for the

protocol.

▪ The report will include the following information:

▫ Execution copy of the protocols with all raw data and associated documents!

▫ A summary of all discrepancies logged during execution of the protocols.

▫ The summary will detail the nature of each discrepancy, the corrective action taken, and the

resolution!

61

Risk Based Summary Report

• Risk Based Approach (The Solution)▪ A summary report will be written following execution of the qualification

exercise in Test and Production environment. The report will provide a

summary of the test results generated during execution of the protocol and

indicate if the acceptance criteria listed were met. The summary report will

be included as the last activity in the executed protocol and will be

reviewed and approved by the signature approvers of the protocol

▪ The summary report will be:▫ included as the last section of the Execution copy of the protocol

• reviewed and approved by the signature chain of the protocol

▫ summarize all test results and whether acceptance criteria were met

▫ summarize all discrepancies and their disposition logged during execution of the protocol

62

Summary Reports (Status Quo)

Risk Based CSV: More Ideas

63

64

Document Approvers

• Typical approval cycle is more than five validation document approvers

• Lean approach for document approvers should be two○ System Owner or SME

○ Quality

65

Benefits – Less Approvers

• Reduced cycle times

• Faster turnaround

• Cost efficient

• If you have an EDM,

○ Reduce the numbers of EDM users (higher privileges)

○ Lower license cost for document approvers

66

Test Script vs Protocol

• Not all validation activities require a large complex protocol

• The decision between a test script or protocol should be based on the complexity of the system or change

• Routine validation activities in support of change control can be done using test scripts

• Protocol should be used for large system implementations and complex changes

• Test Script Examples

○ Parameter changes

○ Simple phase logic changes (open/close valve)

○ Security & authentication

67

Pre- Approved Verification Forms

• Implementation of verification forms instead of protocols

• Driven by SOP

• Forms are pre-approved with SOP

• Installation & Functional Verification forms

• Forms can be created by leveraging existing protocol

68

Pre-Approved Verification Forms

• Forms can be created from requirements and design documents

• Forms can be used for the validation of changes to existing systems

• Examples of verification forms

• Security verification

• Recipe verification

• Audit trail verification

• Parameter verification

• P&ID verification

• Loop check verification

69

Verification Forms(cont.)

System Number: CC # (s) / WO #(s):

System(s) Description: Requirements document:

User Rights Verification

Step Action Expected Response Actual Response

Meets Acceptance

Criteria

(Pass / Fail)

Verified By Initial / Date

1 Login with Administrator account into the application.

Successfully logged into the Administrator account

Does Actual response meet the Expected response?

Yes No

Verification Criteria:

Visual

Screenshot

N/A

Screen shot Attached:

Yes No ,

Attachment #: ______

2

Document the user group to be modified.

___________________

___________________

Appropriate user groups are identified

Privileges are documented

Does Actual response meet the Expected response?

Yes No


Visual

Screenshot

N/A


Yes No ,


3 List the privileges to be

Added /Modified Deleted

___________________________________________________________________________________________________________________________________________________________________________


Meets Acceptance

Criteria

(Pass / Fail)

Verified By Initial / Date

70


Meets Acceptance

Criteria

(Pass / Fail)

Verified By Initial /

Date

Make Copies As Needed

Privilege: _________________

Test Procedure:

___________________________

___________________________

___________________________

___________________________

___________________________

___________________________

___________________________

_

Privileges specified in step# 3

are successfully verified

Does Actual response meet the Expected

response?

Yes No


Visual

Screenshot

N/A


Yes No ,


7 Log off from the User account. Successfully logged off from

the User account

Does Actual response meet the Expected

response?

Yes No


Visual

Screenshot

N/A


Yes No ,


Approved By Quality (Print/Sign):_____________________________ Date:______________

Completed by: Date:

71

Integrated equipment and automation verification

• Installation and functional verification forms○ Pump verification

○ Alarm/interlock verification

○ Valve and miscellaneous equipment verification

○ Agitator verification

• Integrating equipment verification forms into the automation protocols

• Leveraging calibration data

72

Paperless Validation

• Eliminates paper validation documentation and system specifications

• Integrates electronic deviations to protocols

• Integrates the creation of traceability matrix with electronic protocols

• Enables electronic execution of protocol

• Electronic review and approval of protocols and system specifications

73

Paperless Validation

• Automates and manages the validation life cycle

• Provides real time validation status of any system and metrics

• Provides a holistic view of project status and validation

deliverables for internal and external auditors, with real time

status

74

Summary

• We covered a lot of ideas!

○ What is risk based computer validation?

○ Key components of a risk based computer validation program

○ Strategies for aligning current practices into a risk based computer

validation program

○ Risk based Computer Validation strategies

75

Questions