validating computer systems

7/23/2019 Validating Computer Systems

1/2938 APPLIED CLINICAL TRIALS August 200 0

Regulatory authorities require

life cycle management for GCP

systems both during the systems

development (referred to as the

software development life cycle,

or SDLC) and for the rest of their

existence through to retirement

and replacement. The initial

validation plan prepared by users

for user acceptance of a new GCP

system must include provisions

for the care and feeding of a

validated system for the rest of i ts

lifeafter startup through to

retirement. The computerized

system validation (CSV) package

developed at system launch must

also establish the ongoing valida-

tion environment to keep the

system fit and performing as

expected over the years it is

being used. Such long-term

planning requires the involve-

ment of management to ensure a

supply of user resources to

support the system over time.

This fi rst of a series of articles

on the subject of system valida-

tion discusses the users role in

the process. Later articles

describe the roles of the software

supplier and of the system

installers.

An a t o m y o f a G C P s y s t e m

In the 1980s, the computer valida-

tion committee of the Pharma-

ceutical Manufacturers Associa-

tion produced a diagram to help

define terms for validation work.1

Figure 1 shows an updated view

of that classic drawing using an

adverse event (AE) system as an

Validating Computer Systems, Par t 1

A GCP Computer System Isa Lifetime ResponsibilityTeri Stokes

urchasing a computer-

ized system for use in

a GCP-regulated work

process is like buying

a family pet. Like a

well-trained pet, the system is

expected to do what we want it to

and per form as expected. No

mistakes on the carpet, please!

Dont crash in the midst of our

rush to prepare the final study

report, please! Buying a pet is a

lifetime responsibility that begins

with the initial excitement of first

encounters (housebreaking) and

continues on for years with daily

feedings and walks and health

check visits. Buying a GCP

system is also a lifetime responsi-

bility, with initial validation (user

acceptance) and ongoing change

control, repeat testing, mainte-

nance, enhancements, backups,

and audits.

P

example. In this view, a computer

systemis composed of hardware

and software, and a work pr ocess

is composed of people, standard

operating procedures (SOPs),

and often equipment or instru-

mentation.

When a computer system is

used to support a work process,

the two together become a

compu ter i zed systemin the

example, a serious AE manage-

ment system. The computerized

system performs in an operating

environmentsuch as pharma-

covigilance (in the example) or

Techno logy Updat eTechno logy Updat e

Users are

responsible for the

ongoing care and

feeding of their

systems. Is your

GCP system a

pampered pet or a

dirty dog?

Management control

Controlled GCP workprocesses using

computerized systems

System reliability

Consistent, intendedperformance of


Data integri tySecure, accurate, andattributable GCP e-data

Auditable qualityDocumented evidence forcontrol and quality ofe-data and e-systems

e

Lifet ime syst em validation goals

Fig ur e 1 . An example of the configuration of a systemdesigned for serious adverse event management.

Operating environment (pharmacovigilance)

Computerized system (serious AE mana gement system)

Comput er syst em Work process

Hardware Equipment

Software People

Platform system Software application system

SOPs


2/29August 200 0 APPLIED CLINICAL TRIALS 39

product safety. To make user

acceptance validation work

practical, however, it is neces-

sar y to add to the basic diagram

two new concepts. The software

that performs in the work

process and that users see and

touch with a keyboard, mouse,

or bar code wand is the software

appli cation system. Everything

else that is required for the

software application to perform

as expected is par t of t he

pla tform system. The platform

system includes the hardware,

operating systems, databases,

query and other software tools,

and network communications for

ser ver and desktop configur a-

tions that supply the infra-

structure support to the

regulated application.

The software application

cannot function without its plat-

form system support and users

cannot reach the software appli-

cation without desktop or

handheld systems and network

communications. One platform

system can, however, suppor t

more than one software applica-

tion system if it is large enough

and the infrastructure require-

ments are the same for both

applications.

Crea t ion o f va lida t ion

p a c k a g e s

Figure 2 shows the full life cycle

of a regulated software applica-

tion and its platform system. As

shown in the figure, there are

three separate CSV packages to

be prepared at the startup of a

new computerized GCP system:

the performance qualification

(PQ) package, the installation

qualification (IQ) package, and

the operational qualification (OQ)

package.

Software application: The PQ

package. Preparation of the per for-

mance qualification pack-age for

the software application system is

the responsibility of the user

acceptance team. This process

focuses on how the application

per forms in its operating environ-

ment. It consists of a package of

documented evidence that paper

trains your system to behave

itself in the work process. Regula-

tory authorities hold users

responsible for GCP validation of

both the application and its

platform. Because a user team is

seldom equipped to address

platform issues, the users

normally depend on subcontract-

ing development of the platform

CSV package to the information

technology/ information systems

(IT/ IS) depar tment. An internal

or external service level agree-

ment (SLA) i s then used to define

roles and responsibilities for the

startup validation support and the

ongoing service and success of

the system over time.

Platform system: The IQ package.

The installation qualification

package for the platform system

includes startup testing, backup,

recovery, contingency planning,

SOPs, change control, mainte-

nance, configuration manage-

ment, and ongoing services. This

platform CSV package process is

like preparing the yard so your

pet dog can play in safety and

comfort (invisible fencing, dog

house, water dish); it is the

subject of Par t 2 in this series.

This formal division of valida-

tion labor allows the r espective

teams to focus on what they know

best. An IT/ IS team cannot know

the work process as well as the

user team and thus cannot

conduct performance testing that

fully exercises the application in

the same way users can. It is also

impor tant that real users have the

chance to work with the system to

prepare their work ing materials

(SOPs, work instructions, guide-

lines) and fine-tune their work

process to the idiosyncrasies of

the system and vice versa before

the application goes live. Software

problems arising during user

acceptance testing can then be

resolved without interrupting

GCP operations, which can lead

to a troublefree launch of the

system.

Figure 3 illustrates the division

of labor between the user accep-

tance team and the platform

package team.

Software supplier testing: The OQ

package. Yet a thir d team involved

in the star tup of a new software

application system is that of the

software supplier and/ or configu-

ration provider, who perform the

operational qualification of the

system. Regulatory author iti es

also hold the users responsible

for the quality of work and testing

performed by this third team,

whether the team is internal or

external to the users organiza-

tion. Thus, the pedigree of your

GCP system must be as well

documented as the kennel regis-

tration of your purebred pet,

including what kind of shots

(testing) it has had to ensure

good health and immunity to

known diseases.

Fig ur e 2 . The life cycle of a GCP-compliant system, illustrating the PQ, IQ, and 0Q pack-ages in relation to one another.

Application life cycle

Software developmentlife cycle Platform system

5. Test

Verify to SDD& release

1.Configure

3.Test

2.Install

Test fit to install specs.

9. Retire

Decommission& replace

6. Commission

Accept &validate to URS

Test fit to work

Test fit

to design

process

3. Design

How?SDD

2. System plan

What?URS

7. Operate

Use & monitor

1. System idea

Needs analysisreport & RFP

8. Maintain

Fix & modify

IQ packageIT/ IS

OQ packageSuppl ier

PQ packageUser/ owner

4. Build

Program orconfigure

LEGEND

RFP Request forproposal.

URS User requirementsspecification.

SDD Software designdescription.

performance qualification (PQ) Usertesting to answer the question,Does the application systemperform as expected to meet userrequirements specification (URS)in a simulated work processenvironment?

installation qualification (IQ) Platformtesting to answer the questions,Is the software applicationproperly installed, are all itsphysical and logical requirements(per manufacturers specifications)met, and is its platformadequately configured for useraccess in the work process?

operational qualification (OQ)Supplier testing to answer thequestion, Does the applicationsoftware work as intended justabove, just below, and at the

operational limits set in its designspecification?

Te s t in g C o m p o n e n t s



The operational limits pro-

grammed into an application are

tested by the module and system

integration level testing is per-

formed by the software supplier.

This testing is performed before

the release of the application for

shipping to a customer. Usually

the user team exercises its check

of the supplier/ providers work

by conducting an audit or walk-

through review of the software

development life cycle and

writing a formal audit report of

their findings. The suppliers CSV

package is the focus of Par t 3 of

this series.

Ma na g e me nt s r o le in

l i fe t ime system va l ida t ion

Buying a purebred pet requires

taking on daily responsibility for

the animal during its lifetime, and

purchasing a strategic GCP

system requires a similar level of

care and attention. For manage-

ment to be in control of a comput-

erized work process, the organiza-

tion must be set up to support

lifetime validation for a regulated

system. Figure 4 shows how a

sponsor or CRO can be organized

to support lifetime validation for

strategic GCP systems.

The business decision group.

Senior managementsuch as the

director of clinical research and

the directors for clinical data

management, biostatistics, phar-

macovigilance, and clinical quality

assurancehas ultimate regula-

tor y responsibi lity for GCP data in

the clinical trial work process.

This group of managers forms the

logical composition for a businessdecision group (BDG) to plan and

provide resources for compliance

efforts for strategic GCP systems.

The BDG addresses broad

issues, such as: Can we add

another country of users to the

system? Should we add another

application to the same server or

get a new server? How many

users can we send to the

suppliers user training this

quarter? Which work process gets

the next upgrade to its computer-

ized system? When do we retire

this system, and do we upgrade

with the current supplier or move

to a new vendor or new technol-

ogy at retirement? How much do

we redesign our work process to

fit the system? Who is responsible

for keeping the system in a

validated state?

The system sponsor is the senior

manager r esponsible for a par ticu-

lar computerized work process

such as clinical data manage-ment

or pharmacovigilance. The

system sponsor carries the local

responsibility for funding and

approving the validation effort of

computerized systems in that

work process.

The system sponsor addresses

such issues as: Who will I appoint

as system owner to lead a CSV

package team? How can I locate

others to help the three key

package team members with

regular tasks so that they have

time to develop the startup CSV

package on time? How do I

budget and gauge their ongoing

duties for training new users,

troubleshooting, and keeping the

system validated after the system

goes live and they return to their

full-time jobs?

The system owner is usually the

key user in the operating environ-

ment who is held responsible for

having the system available for

users in the computerized work

process. This person assigns a

package manager who drives the

documentation process and a test

coordinator who manages the

testing activities. The system

owner leads this core team indeveloping and maintaining the

CSV package. Other users are

included in an ad hoc capacity as

testers and witnesses, and

specialty resources may be added

as necessar y to provide specific

expertise.

The quality assurance role stays

outside of the package develop-

ment process to be able to audit

the CSV package. QA audits of

the CSV package midway and at

the end of development give the

team and the BDG an indepen-

dent view of the ability of the

package to pass compliance

inspections.

The use r a cce pta nce CS V

p a c k a g e

Once the need for a new GCP

system has been established, the

needs analysis document itself

star ts the application li fe cycle and

becomes the first document in a

users CSV package. The request

for proposal (RFP) becomes the

second document in the

package. When a system is

chosen, it is time to identify the

package team and write a valida-

tion plan. The Institute for Electri-

cal and Electronics Engineers,

Inc. (IEEE) gives a document

outline and explanation of

expected content for a validation

plan in its standard 1012-1986.2

The user acceptance package

for computerized system valida-

tion is designed by the validation

plan to provide documented

evidence for

management control of the

system, its users, and its

regulated data.

reliability of the system to

perform as intended every

time.

protection of data integrity

Fig ur e 3 . Two packages, two teams.

Computer system

Hardware

IT/ IS team

Software

Platform system

Work process

Equipment

People

SOPs

Software application system

PlatformCSV pkg.

User team

UseracceptanceCSV pkg.

SLA

Service/successlevel agreement

Figure 4 . Lifetime validation requires a lifetime package team:system owner, package manager, and testing coordinator.

Business decision group ( BDG)

Funds & approves computerized work process

System sponsor

Funds & approvesCSV package

Qualit y assurance

Audits CSV package &hosts GCP inspections

Testingcoordinator

Package manager

System owner

CSV package team

Develops & maintainsCSV package



The validation plan devel-oped by a user acceptanceteam includes more than

just testing. The boxes onthe left side of Figure 5focus on control of the phys-ical and logical environment

for the application and thetwo boxes on the rightaddress control of userinteraction and the humaninterface to the system.Each of these items is dis-cussed briefly below inorder of appearance in thefigure.

Control of physica l/

logica l e nvironme nt

Application administration

SOPs. Descriptions of usertypes and their privilegeson the system, proceduresfor system administration,backup and restoreactivities.

Application configurationmanagement logs. A logbookbinder used on an ongoing basis to keep a current descriptionof the application, its backup log, maintenance records, supportactions, change control decisions and actions taken, ongoingtesting, problem tracking, release notes, suppliercorrespondence, the service level agreement (see below) withthe platform supplier and/ or with the application provider, record

of user training events, and list of authorized users.Change control log. A change control SOP for the system andmonthly change control reports for system changes made byuser, platform, and/ or supplier actions. Report should includethe extent of repeat testing performed and/or updates made torelevant CSV package items.

QA audit log. A record of any QA audits performed on the CSVpackage itself or on behalf of the CSV package, such as QAaudits of suppliers.

Supplier reports.This log includes any walkthrough reportsdeveloped by the users when reviewing the platform CSVpackage or other supplier activities. It also includes follow-upreports on supplier milestone performance under service level

agreements.BDG minutes.The business decision group, consisting of linemanagers responsible for the operating environment and itscomputerized work processes, documents its financial,operational, and resource decisions about the regulated systemin meeting minutes that are a part of the systems CSVpackage.

Control of user interact ion

Needs analysis.The initial document examining the work processand describing the type of computerization it might benefit from.

Request for proposal (RFP). A formal document sent toprospective suppliers describing the work process needs and

requesting a response from vendors showing how they couldmeet those needs with their products and/ or services.

Contract. A legal document describing the roles, responsibilities,and financial elements in the business relationship (purchase,lease, rental) between system users and external suppliers ofsystems or services.

User requirements specification (URS). A formal, approveddocument describing the users needs for a computerized

system from a work process perspective, such as the types ofdata to be handled, the flow of data across the work process,data inputs and outputs to the work process, size and locationof user groups, types of user roles, and access privilegesneeded.

Service/success level agreements (SLAs). Formal, approveddocuments between system user BDGs and internal or externalsuppliers to describe roles and responsibilities on both sides forkeeping the regulated system in successful operation andcontinually validated.

User manuals. Printed materials from system suppliersinstructing users how to work with the regulated system.

CVs and training records. Curriculum vitae showing the users

educational backgrounds and work experience that indicatecompetence to perform in the work process, and trainingrecords showing formal and on-the-job training given to ensurethat a user is qualified to use a regulated system in the workprocess.

Department SOPs. Department standard operating procedures,guidelines, and/ or work instructions that are specific for usingthe computerized system in the GCP work process.

Problem/help logs. Logbook binder to record user problems asthey arise and their resolution as they are settled. A standardincident report form might be used to facilitate this recording.Department should have work instructions for how to reportproblems and where to go to get help with the system. Recordsdescribing problems and resolution should be reviewed monthlyfor trends and training issues.

C S V P a c k a g e It e m s

Fig ur e 5 . A standard user acceptance CSV package, prepared and maintained by a userdepartments team.

Validation plan

Application administrative SOPs &application configuration

management logs

Change control log,QA audit log, supplier

reports & BDG minutes

Needs analysis, RFP,contract, URS, SLAs

User manuals, CVs &training records, dept.

SOPs, problem/ help logs

Test plan(s)startup & ongoing

Test cases, scripts,data & results logs

Test summaryreport & updates

Users CSV package summary report



during handling by the system.

auditable quality of the system,

its data, and user activities with

the system over time.

Figure 5 shows the contents of

a standard user acceptance CSV

package. Although the testing

functions in the central column

are the usual perception of activi-

ties under a validation plan, the

other two columns are of even

greater importance for ensuring

the smooth, ongoing operation of

the system in a validated state.

See the CSV Package Items box

for brief descriptions of these

other activities.

Formal testing practices in CSV

packages. All testing per formed in

suppor t of releasing a computer-

ized system for GCP use is to be

performed in a formal manner.

That means it is to be fully

thought through, documented,

approved, and reported upon

under the guidance of a signed

validation plan and a signed test

plan.

A startup test plan is used at the

installation of a new system. The

Institute for Electrical and

Electronics Engineers, Inc.

(IEEE) provides a good standard

outline to be used for any system

(see Test Plan Outline box) .2

The test plan points to other

formal testing documentation to

be developed under its directives.

These documents include the

following.

Traceabil ity matri x. A three-

column table that identifies the

key user requirements in the work

process down the first column, the

system features/ functions sup-

porting each key requirement

down the second column, and the

test cases checking and chal-

lenging the respective system

functions down the third column.

Test case descr i pti on. A

document that describes a logical

grouping of test activities associ-

ated with the system and its placein the work process. Test Case 1

should be designed to check for

workplace preparedness by

reviewing system manuals, work

instructions, SOPs, and training

records. Test Case 2 should check

system administrative functions

for setting up users, database

structure, and so on. Test Case 3

should be designed to check and

challenge the system with

examples of the work process

activit ies in a simulation mode.

Test scri pt. A document that

manages the individual testing

experience and records the pass/

fail conclusion. There can be one

or more test scripts per test case.

Step procedur e. A document

specifying the detailed steps to be

performed during testing, the

expected results, and identified

spaces for recording system

response, problems occurring,

and resolution to problems. There

can be one or more step proce-

dure documents per test script.

Resul t log. Identifi ed space in

a step procedure for recording

system response to the testing

activity.

Change control and the ongoing

test plan. For repeat testing after

the system has gone live and fixes

and updates occur, a separate

ongoing test plan is created, as

shown in Figure 6. The ongoing

test plan is written and approved

as one of the final validation plan

tasks to be completed before

writing the CSV package sum-

mary report. Figure 6 also shows

the other CSV package items that

may require updates or adjust-

ments depending on the size and

scope of the fixes, updates, or

enhancements made to the

system.

Keeping a GCP system vali-

Test plan ID number. Coded to relate the testplan to the associated validation plan.

Introduction. Briefly describes system to betested and references the related URS.

Test items. List of software application andsystem versions to be tested.

Features to be tested. System featurestraced to requirements from URS.

Features not to be tested and why. Installed fea-ture not used by work process.

Approach. Describes overall strategy, tech-niques, and limits for testing.

Pass/fail criteria. If just one test script fails,does the whole system fail?

Testing suspension criteria and resumption

requirements.

Test deliverables. Documents required: testplan, test cases, test logs, test sum-

mary report.

Testing tasks.Tasks necessary to preparefor, perform, and report testing.

Environmental needs. Physical, logical, andsecurity requirements for testing.

Responsibilities. Identified for test devel-

oper, tester, and witness. Staff training needs. Specifies skill level for

test writer, tester, and witness.

Schedule.Estimated time to do each taskand definition of milestones.

Risks and contingencies. Identify high-riskassumptions of test plan and specifyhow plan addresses each risk.

Approvals.Specify names and titles of allwho must approve this plan with spacefor signatures and dates.

Tes t P la n Ou t line f o r S t a r tu p o r Ong o i ng Tes t i ng

Fig ur e 6 . The ongoing test plan illustrates management control through to retirement.

Validation plancreated at startup

Application administrative SOPs &application configuration

management logs

BDG minutes,change control log &

QA audit log

URS, SLAs

User manuals, CVs &training records, dept.

SOPs, problem/ help logs

Test planongoing

Change control testcases, scripts, logs

Change control testsummary reports

Users CSV package summary report



dated requires keeping all of its

relevant documentation up-to-date

as the system is tuned to a chang-

ing work process or a problem is

resolved, or when a supplier

provides a new feature. User train-

ing and user instructions in

manuals and SOPs may have to be

adjusted to ensure proper use of

the system. Management needs to

be kept informed of trends in

changes and the scope of changes

over time.

System change requires

repeat testing to some extent

determined by the ongoing test

plan, and all formal testing

documentation must be kept for

audit purposes. The reuse of

startup test scripts provides a

degree of regression testing

that checks for consistent results

with pr ior findings. Some startup

test scripts can also be used for

training new users on the

system. Such training test experi-

ence can also be used for

periodic checks of the system,

and so should be kept and

recorded for double credit

whenever possible.

Test summary report. When any

formal testing is performed, a test

summar y report must be pre-

pared to give management a

review and analysis of the experi-

ence and assessment of the

testings impact on the validation

status of the system tested. A pile

of test result logs remains just a

pile of paper until a responsible

person who understands what the

testing means has analyzed it and

reported the conclusions. A good

test summary report includes the

points outlined in the Change

Control Test box (adapted from

the IEEE Standard for Software

Test Documentation) .3

As Figure 7 shows, many

changes can occur with a GCP

system after startup. The longest

part of a successful systems

lifetime is during its operational

phase in the work process.

The user team continues to

focus on the GCP software appli-

cation system by training new

users, resolving issues and

problems with the system, and

keeping the configuration man-

agement logbook binder up-to-

date with system activities. The

user team also manages the SLA

process with platform and applica-

tion suppliers, performs formal

testing of changes to the system,

and keeps the BDG informed with

quarterly updates on system

status. When the GCP system user

community becomes multisite,

multinational, and multidivisional,

the user team roles become full-

time positions.

A successful re la t ionship

We have all seen uncontrolled,

poorly disciplined pets become a

nuisance to ever yone around

themsometimes their behavior

is a danger to themselves and

others. The same situation can

easily arise with computerized

systems left unmanaged. The

purchase of software for clinical

research is serious business and

the failure of software to function

as intended must be considered

for its impact on the safety,

efficacy, and quality of study data

and the medical decisions made

on the basis of computerized

systems in the GCP work process.

The operational phase of a GCP

software application system is the

longest phase of its life, and man-

agement must exer t ongoing con-

trol of the system to ensure its

continued compliance to valida-

tion standards. The user teams

startup CSV package should also

establish the documented proce-

dures for ongoing control of the

GCP system, and management

should provide the strategic focus

and ongoing resources needed to

achieve the goal of expected per-

formance with lifetime system

validation for its essential GCP

systems.

R e f e r e n c e s

1. Pharmaceutical M anufacturers

Association Computer Systems

Validation Committee, Validation

Concepts for Computer Systems

Used in the Manufacture of Drug

Products, Pharmaceutical

Technology 10 (5), 2434 (1986).

2. Institute for Electrical and

Electronics Engineers, Inc., IEEE

Standard for Software Veri fication

and Validation Plans, Standard

1012-1986 (IEEE, Piscataway, NJ,

1992), pp. 1219.

3. Institute for Electrical and

Electronics Engineers, Inc., IEEE

Standard for Software Test

Documentation, Standard 829-1983

(IEEE, Piscataway, NJ, 1993), pp.

1012, 1516.

Teri Stokes, PhD , is seni or

consult ant an d di rector of

GXP Internat ional , 131

Sudbu ry Road, Concord ,

M A 01742, (978) 287-4393,

fax (978) 369-5620,

gxpint [email protected] tr anet.com.

Fig ur e 7 . User team focus and business decision groupresponsibilities through to system retirement.

Fit ?User / owner

Platformsupport SLA

Business decision group (BDG): Prioritize change & resources

Updates & auditsof user CSV package

GCP

platform

GCPapplication

User team focus

Configuration mgt.log binders

Help desk

Applicationsupport SLA

9. Retire1. Sytem idea

2. System plan

Application life cycle8. Maintain

Fix & modify

Patches &updates

Provider/supplier

6. Commission

Re-accept

7. Operate

Use & monitor

Test summary report identifier. Unique ID traceable to associ-ated ongoing test plan.

Summary. Describes change(s) being tested. Describesitems tested (application version), test environment (plat-form system), and test approach (test cases used).

Variances.States any deviations from test case or test

scripts and reasons for deviations. Comprehensive assessment. Discusses assumptions and limits

to scope of testing. Were scope of testing and resultsobtained sufficient to assess system reliability forchange(s) made? Discuss reasons for limits chosen.

Summary of results. Gives table of testing results per testcase. Table of anomalies and their resolutions. List ofoutstanding issues and risks (unresolved anomalies).

Evaluation.Pass/ fail conclusion based on test results andcriteria in the ongoing test plan.

Summary of activities. Describes tester/ witness staffing, test-ing location(s), and test documentation preparation and

approval process. Approvals. Names, titles, signatures, dates with meaning of

signatures.

Appendix.Table of contents list for test documentationproduced.

C h a n g e C o n t r o l Te s t S u m m a r y R e p o r t


7/29

tem iceberga collection of

server, desktop, and peripheral

hardware, operating systems,

database engines, query/report-

ing tools, and the communications

networks needed so the applica-

tion will work as intended. These

components make up the platform

and infrastructure systems usually

supplied to users by the informa-

tion technology/information sys-

tems (IT/IS) department.

Often the phrase Out of sight,

out of mind can be used to

describe the typical validation

focus for such platform and infra-

structure systems. To meet regu-

latory requirements, however,

what is needed is a focus for com-

puter validation in the IT/IS

Validating Computer Systems, Part 2

GCP Validation of Platformand Infrastructure Systems

Teri Stokes

art 1 in this series dis-

cussed the user accep-

tance validation of

application software

as a lifetime responsi-

bility to ensure its ability to per-

form as intended in a good clinical

practice work process. The

metaphor of buying a dog was

used to compare the ongoing

responsibilities for keeping a fam-

ily pet healthy and fit to the need

to provide ongoing care to keep a

computerized GCP system fit dur-

ing its lifetime.

Part 2 now addresses the other

half of the system operation story

that is often hidden from the users

view. Like the tip of an iceberg,

however, multiuser and global

GCP applications sit atop a huge,

unseen, but necessary, mass of

support. Below the application

software tip is the rest of the sys-

P

department that documents the

health and fitness of the platform

systems and network infrastruc-

ture for the way they are used to

support GCP applications.

Platform system life cycle

The life of a platform system

begins when the user group

decides to purchase a particular

application software to perform a

certain set of GCP work process

tasksclinical data management,

adverse event reporting, elec-

tronic subject diaries, statistical

analysis, Web-based clinical data

entry. For major applications, an

IT/IS person often consults with

the user team to provide technical

expertise in the purchase pro-

cess. This person also advises

about how well the technical

requirements of the application

software candidates fit into the

current IT/IS setting and whether

any new platform/infrastructure

components will need to be pur-

chased to handle a particular

application.

Figure 1 shows the traditional

life cycle for application software

with a division between the steps

performed by the user group pur-

chasing the software (1,2,69) and

the steps performed by the sup-

plier that develops the application

software (35). The life cycle of

the platform system is also shown

in Figure 1, operating in support

of the user group during steps

Technology UpdateTechnology Update

The IT/IS team not

only implements

the platform

system that

delivers the GCP

application to the

work process, but

is responsible for

its continued care.

Figure 1. The platform system life cycle in detail, and how it fits with the rest of the applica-tion development cycle.


Software development

life cycle Platform system

5. Test1. Configure

9. Retire


6. Commission

Validate & retestafter fixes

Test fit to work

Test fit

to design

process

3. Design

7. Operate

GCP workprocess

1. System idea

Needs analysisreport

8. Maintain

Fix & modify

IT/IS dept.Softwaresupplier

User group

4. Build

LEGEND

URS User requirementsspecification

SLA Service levelagreement

URS

2. System plan

2. Install

3. Test fit toSLA needs

6. Retire

5. Maintain

4. Operate

September 2000 APPLIED CLINICAL TRIALS 55


8/2956 APPLIED CLINICAL TRIALS September 2000

69, but also depicted next to the

software supplier that specifies

the physical and logical require-

ments of the application for the

IT/IS department to use when

configuring the platform system.

The life cycle of the platform

system requires that the IT/IS

platform team configure, install,

test, operate, maintain, and

eventuallyretire the system.

Details of each stage follow.

Configuration. The IT/IS plat-

form team must first define the

hardware, software, and network

components to be assembled for

server and desktop systems to be

used for access to and operation of

the application. Platform configu-

ration must meet the specifica-

tions of the application software

supplier, the platform component

suppliers, the user groups work

process, and the IT/IS depart-

ments work process.

Installation. The team assem-

bles and installs all platform com-

ponents according to relevant

manufacturers specifications.

Testing. Team members per-

form an installation qualification

(IQ) test to ensure that the many

components work properly as a

platform system unit. They install

the GCP application and ensure

that platform functions are acces-

sible to the applicationfor exam-

ple, that the desktop can reach

both the database engine and the

printer to produce a report.

Operation. The IT/IS platform

team then performs ongoing oper-

ations using standard operating

procedures (SOPs) to meet the

user group GCP needs stated in a

service level agreement (SLA)

for example, nightly or weekly

backups, security checks, system

administration, disaster recovery,

support for new user groups. The

team performs periodic testing of

the platform for ongoing quality

control.

Maintenance. The IT/IS plat-

form team is responsible for

installing patches and upgrades

from the application supplier.

Team members ser vice, repair,

replace, and install new compo-

nents to the platform system as

needed over time. They perform

problem resolution, maintenance,

and repeat testing, as dictated by a

documented change control SOP.

Retirement. When the time

comes, the IT/IS platform team

will plan and execute the orderly

decommissioning of the platform

and its infrastructure compo-

nentsin part or as a whole

when new technology decisions

are made for the application itself

or for platform support.

Platform system CSV

package

The first regulated application

installed on a new or existing plat-

form system automatically makes

the entire platform subject to reg-

ulatory inspection, even though

the rest of the applications on that

platform are not regulated. Thus,

it is smart for an IT/IS department

The documentation required for a platform system CSV packagealso helps the IT/IS department to be organized in its operations.



to plan its platform environments

with a realistic strategy for estab-

lishing and maintaining a docu-

mented approach to system opera-

tions. Grouping several GCP

applications on the same platform

system configuration can mini-

mize IT/IS work by requiring only

one computerized system valida-

tion (CSV) package for the com-

mon platform.

In many ways, the documenta-

tion required for a platform sys-

tem CSV package also helps the

IT/IS department to be organized

in its operations. The CSV pack-

age makes it possible for new

hires and backup operators to

work with the system and to take a

knowledgeable approach to

resolving issues that arise.

The platform CSV package is

ver y similar in content to the

users CSV package outlined in

Part 1, but reflects the particular

responsibilities of the platform

package team. The IT/IS platform

package needs to reflect the work

process in the data center and

infrastructure support functions.

It also needs to have a practical

approach that eliminates redun-

dant work and allows for a

response to multiple applications

and user support for all regulated

applicationsGCP, GLP (good

laboratory practice), GMP (good

manufacturing practice), Rule

11collectively referred to as

GXP. Goals for the platform CSV

package are the same as for user

acceptance. The platform system

must have documented evidence

to show that it is under IT/IS man-

agements control, that it operates

reliably, that its database engines

and communications networks

protect the integrity of GXP data

being handled, and that documen-

tation is in good order for audit

purposes.

Validation plan

The Inst itute for Electrical and


(IEEE) publishes a standard for

software verification and valida-

tion plans that can easily be

adapted to the purpose of writing a

platform system validation plan

(see Platform Validation Plan Out-

line box).1

Every plan in the CSV pack-

agevalidation plan, startup

test plan, disaster recovery

planmust have a defined task

list stating just what actions are

to be taken to execute the plans

strategy. Figure 2 and the Plat-

form CSV Package box give an

itemized description of each

component of the package to be

addressed by the validation

plan.

Every plan must have its own

summary report written to

explain to management the out-

come of the planned tasks. The

package summary report

includes the outcome of individual

tasks as well as the highlights of

summary reports from subordi-

nate plans. The Platform Package

Summary Report box shows an

outline for such a summary

report.

A collaborative effort

The GCP application software

cannot function without its plat-

Plan identifier. ID number indicating system associated with

the plan.

Summary of all validation life cycle tasks and their current status.

Summary of all CSV package items and their current status.

Summary of unexpected problems/issues and their resolution.

Summary of deviations from the validation plan and rationale

for deviations.

Assessment of overall system quality based on package docu-

mentation, test summary report, and QA audit report.

Recommendations. Release statement from management for

the GXP status of the platform system.

Approval signature(s) and date(s).

Appendix A. Update report form for recording major system

changes with related regression testing.

Appendix B. Summary reports for periodic quality control

testing.aAdapted from IEEE Standard 1012-1986.1

Platform Package Summary Reporta

Purpose and scope

Inclusions, exclusions, and limitations.

Reference documents

SOPs, manuals, and policies referenced by the plan.

Definitions

Terms required to understand the validation plan.

Validation overview

Organization and master schedule for the validation

effort.

Resources summary and responsibilities for validation

tasks (usually a three-column table listing validation

tasks, role(s) responsible, and due date).

Tools, techniques, and methodologies used in the valida-

tion effort.

Life cycle validation tasks at each phase from purchase/install to retirement

Concept phase. SLA based on users needs analysis,

URS, and application suppliers specifications for plat-

form requirements.

Development phase. CM logbook description of platformsystem.

Installation and checkout phase. Test and validation

summary reports.

Operation and maintenance phase. SOPs, logs, audits,

and training.

Retirement phase. Archive plan and transition plan to

next platform system.

System validation reporting

Required and optional records/reports to be written.

Validation administration procedures

Reporting and resolution process for system and user

problems and issues.

Task repetition policy. When and how to repeat testing

and other validation tasks.

Deviation policy. How to handle actions that differ from

the plan.

Control procedures. How software application and plat-

form system(s) are configured, protected, and stored

SOPs for backup/retrieval, disaster recovery, change

control, and system testing.

Standards, practices, and conventions for validation

work. Template formats for logs, reports, and other

items in the validation package.

aAdapted from IEEE Standard 1012-1986.1

Platform Validation Plan Outlinea


10/29September 2000 APPLIED CLINICAL TRIALS 59

Validation plan. Document describing thepurpose, scope, approach, resources,and schedule of intended validationactivities. It identifies the CSV packageitems; the tasks, roles, responsibilities,and schedule for developing packageitems; and the standards, methods,

and procedures used to quality assure acomputerized system throughout allphases of its life cycle (Figure 2).

Configuration management (CM) log. First sec-tion in a CM logbook binder that con-tains a written description of the plat-form system configuration in text and indiagrams. Section includes a list of allmajor components of the platform withrelevant identifiers, such as serveridentifiers, associated disks, softwareversions, overall network topology.

System manuals. Supplier documentation for major compo-nents of the platform such as server hardware, operat-

ing system, database engine, network management. System SOPs. System administration standard procedures

and/or department work instructions for operating theplatform, performing backups, providing security.

Disaster plan. Document describing exactly how the plat-form system can be reconstructed in case of its destruc-tion by fire, flood, theft, or other catastrophe.

Change control log. Section of CM logbook binder whererecords are made for the approval, implementation, andrepeat testing of changes to the platform system. It isexpected that IT/IS has a department level change con-trol SOP and that records in this log conform to thedepartments procedure.

Backup log. CM log record of performing daily, weekly,monthly backups for the regulated applications on theplatform system.

Archive log. CM log record of off-site storage location forany backup media stored that relate to the platformsystem or its regulated applications.

Supplier records. CM logbook binder section(s) for filing anycorrespondence or visit reports from component suppli-ers. Should also include an area for logging telephoneconversations with supplier support desks.

Audit log. CM logbook binder section to record date, time,and participants in any audits or inspections of the plat-form system by internal QA, external clients, or regula-tory authorities.

Test plan. Document that describes the technical and man-agement approach to be followed for testing a system orcomponent. Typical contents identify the items to betested, tasks to be performed, responsibilities, sched-ules, and required resources for the testing activity (seeIEEE standard1).

Startup test plan. Document describing the strategy for test-ing at the time of the initial installation of a platformsystem.

Ongoing test plan. Document describing the approach fortesting performed under change control throughout theoperational life of a platform system.

Test case. Document specifying the details of the testingapproach for a platform feature or combination of fea-

tures and identifying associated test scripts, such assystem power up/down, system backup/recover, server

connectivity to desktops/printers/remote sites across anetwork.

Test script. Document specifying inputs, predicted results,and a set of execution conditions for an individual test.Includes one or more step procedures that describekeystrokes or other tester actions and provide log spacefor recording system response to test activity.

Test summary report. Document summarizing testing activi-ties and results. It also contains an evaluation of theplatform system tested.

Service level agreement(s) (SLAs) with user groups owning aregulated application on the platform system. Documentthat describes the respective roles and responsibilitiesof IT/IS and the user group for successful support of theGCP application on the platform system.

SLA monthly reports. Brief notes describing ongoing mile-stone results and important decisions/actions taken forthe platform system in supporting a GCP application andits user group.

Security log. Section of CM logbook binder that definessecurity levels for the platform system and records anyincidents of security breach and their resolution.

Help desk log. Record of platform system issues arisingand their resolution.

IT/IS deptartment SOPs. Standard operating proceduresshould be defined for physical-logical security of the datacenter and platform systems, change control for applica-tions and platforms, installation/operation of a regu-lated platform system, performing system backup proce-

dures, actions for disaster recovery, and training ofpersonnel.

Curriculum vitae (CVs). Resums of education and profes-sional experience related to current work assignment.

Training records. All IT/IS department personnel working ona platform system supporting GCP applications shouldhave CVs and updated records for ongoing training rele-vant to their work. Training in the regulatory require-ments for computerized systems is also expected.

CSV package summary report. Document summarizing all CSVpackage activities initiated under the validation plan andthe results of those activities. It also contains an evalua-tion of the platform systems readiness to perform andbe operated in compliance with regulatory requirements

(see Platform Package Summary Report outline).

Platform CSV Package

Figure 2. The platform CSV package.

Validation plan

Configuration mgmt. log,

system manuals, & SOPs

Disaster plan, change

control log, backup &

archive logs, supplier

records, audit log

SLA, SLA monthly reports,

security logs, help desk log

IT/IS dept. SOPs, CVs

& training records

Test plan(s)

startup & ongoing

Test cases, scripts,

data & result logs

Test summary

report(s)

Platform CSV package summary report(s)


11/29

form system support, and users

cannot reach the application with-

out desktop or handheld systems

and network communications.

The size and complexity of a plat-

form system varies with the

design of the application software

and the size and diversity of the

user population.

Validation of the computerized

system requires the collaboration

of two teamsthe user accep-

tance team and the IT/IS team

(Figure 3). The user acceptance

team is responsible for the CSV

package that documents whether

the software application performs

as intended during the GCP work

process. The IT/IS team is

responsible for the CSV package

that documents the assembly and

validation of the platform system

configuration to deliver the appli-

cation to the work process. The

platform system configuration will

include the components of the

server system(s) hosting the soft-

ware application, the components

of the desktop delivery system

(PC/laptop/device) bringing the

application to the users, and

all network communications

required for the application to

operate as intended in the work

process.

For GCP systems, the user

team and the IT/IS team must

work closely together for the life

of the system. The need for collab-

oration is usually quite clear dur-

ing the excitement and attention

paid to the initial project for

startup of a new system. But as

time goes on, it is often over-

looked that both teams must con-

tinue to exist and must continue to

work together to operate, main-

tain, and fine-tune the computer-

ized system to its GCP work pro-

cess while keeping it in a validated

state. Ongoing activities and

updates to the CSV package items

of each team must continue.

The SLA. The businesslike way

to address a lifetime partnership

is to develop a service level agree-

ment (SLA) between the parties.

The SLA needs to clearly describe

the expectations on both sides for

application and platform activities

to ensure the success of the com-

puterized system in the GCP work

process and its ability to have

uneventful audits and inspections.

Developing the SLA is the start

of a long-term partnership

between the users and the plat-

form support group. As with any

lifetime partnership, it is impor-

tant to be clear about roles and

responsibilities, to set specific

expectations with measurable

milestones, and to have a problem

resolution process defined and

understood. Platform support can

come from IT/IS departments

that are internal or external to

clinical research or to the com-

pany as a whole, and the SLA pro-

cess is the same for any source of

platform support. Figure 4 shows

the partnership process.

There are many topics to con-

sider when developing a service

level agreement. Topics will vary

depending on the size and com-

plexity of the role the application

plays in the GCP work process.

The following items are given as

examples of the types of questions

to be discussed between the plat-

form team and the user team.

The application users GCP work

process needs. Size and scope of

user population. How many

users? At how many locations?

Across how many time zones and

countries? Likely number of con-

current users? Are users internal

or external to the organization?

Who approves adding new users?

What is the expected expansion

rate for adding new usersCRA

laptops, investigator site entry

systems, subject palm diaries?

What types of physical- logical

security are needed?

Type of user activities to be

per formed. How does data get

into the database? Batch upload/

download of data from internal/

external sources, manual data

entry from what locations, data

acquisition from instruments?

How is data retrieved from the

database? Who manages the data-

base? What kinds of reports are

needed requiring what types of

printers? Location of specialty

printers? Is the application

expected to communicate with

other applications or databases?

How? Diary upload to CRA laptop

followed by upload to platform

server? CRO SAS (statistical anal-

ysis system) tapes to platform

server? Instrument data to lab

server followed by upload to plat-

form server?

IT/IS platform support process

needs. Support services. What

kind of system and data backup

schedule is needed? When can

maintenance be performed with-

out disrupting the users work

process? What hours will help

desk response be available for

user support with desktop and

server issues? For what time

zones? Is special technical

expertise needed to support this

application? How much support

Figure 3. The IT/IS team delivers the system to the desktopsof the users.

Computer system

Hardware

IT/IS team

Software

Platform system

Work process

Equipment

People

SOPs


PlatformCSV pkg.

User team


Local/wide

area networkcommunications

Desktopdelivery system

PC

Figure 4. A service level agreement (SLA) defines the on-

going partnership that must exist between the user team and

the platform team over the life of the application.

Computer system

Hardware

IT/IS team

Software

Work process

Equipment

People

SOPs

PlatformCSV pkg.

User team


SLA

Service/successlevel agreement

Defines:Users GCP work

process needs

IT/IS platform supportprocess needs

User and IT/IS rolesand responsibilities

Monthly progress

60 APPLIED CLINICAL TRIALS September 2000



is the application supplier pro-

vidingfor users and for IT/IS?

What is the application sup -

pliers process for contacting

support services?

Support training. Is special

training needed for platform

operators? How and when is

training available? Can the appli-

cation supplier provide backup

for the IT/IS help desk? Who

delivers the special training?

How will installation and training

for remote site users be handled?

User and IT/IS roles and responsi-

bilities. Business par tnership.

Who speaks for the user group

and who speaks for IT/IS on plat-

form system issues? Who man-

ages contact with the application

supplier and for what purposes?

Who approves change requests?

Who pays for what? What notices

will be given and to whom when

application changes are to be

made that require platform

changes and vice versa? What is

the problem resolution process?

How are unsolved problems

escalated for resolution? What

are the GCP audit/inspection

response roles?

Monthly progress. Lifetime part-

nership success is measured by

application success each month.

SLA success milestones.

Does every SLA requirement

have a measurable outcome? If

the user group or IT/IS expecta-

tion cannot be measured, it is a

wish, not a requirement, and it

should be omitted from the SLA.

IT/IS view. How many help

desk calls are handled and

closed per month? Length of con-

tinuous platform server and net-

work uptime for the application

without any interrupt ions in

work process operations? Num-

ber of printer or desktop prob-

lems reported and resolved for

the application versus number of

printers or desktops in use for

the application? Any changes to

platform system components

resulting in updates to the con-

figuration management logbook

binder? Any change control and

repeat testing records for the

month? Any updates to the plat-

forms CSV package? Any new

announcements from the plat-

form component suppliers for

upgrades, enhancement patches,

or new versions of product com-

ing soon that could affect the

application?

User group view. Number of

new sites or new users added per

month? Number of application

suppliers fixes or updates

installed and retested? Number

and type (system/data/applica-

tion, daily/weekly/monthly) of

backups performed for the appli-

cation? Any audits or inspections

expected for the CSV packages?

Any application training needed

for system fixes and/or new

hires for IT/IS or user groups?

What is the training, who deliv-

As with any lifetime partnership, it is important to be clear aboutroles and responsibilities, to set specific expectations, and to have a

problem resolution process defined.


13/29September 2000 APPLIED CLINICAL TRIALS 63

ers it, and how is it recorded?

Any new announcements from

the application supplier for

upgrades, enhancement patches,

or new versions of product com-

ing soon that could have an

impact on the platform configura-

tion? Any security issues aris-

ingserver, database, network?

It is important for all par ties to

remember that the goal of the

SLA is the successful operation

of the application software in the

GCP work process. The ultimate

measurement of SLA success is

the controlled, reliable handling

of GCP data in clinical research

and protection of the integrity of

such data throughout its process-

ing, storage, and retrieval by the

application software.

Lightening the IT/IS load

To make life simpler, many IT/IS

departments identify one stan-

dard desktop configuration of PC

and core applications to be used

across the clinical research orga-

nization. This standard desktop

configuration is then validated on

its own in a single desktop CSV

package. The CSV package for

the standard desktop is then ref-

erenced across multiple applica-

tion platforms with only a brief

installation qualification (IQ) test

performed per new application.

In similar fashion, many IT/IS

departments identify standard

server configurations for certain

types of server technologies

(HP/UNIX, for example) and

database environmentssuch as

Oraclethat are often used by

regulated applications. The stan-

dard server configuration is then

validated on its own in a single

CSV package to support the first

regulated application being used.

Subsequent GCP applications

going on to the same server con-

figuration would reference the

first platform CSV package and

just add package updates for the

specific needs of the new applica-

tion. Updates might include test-

ing new printer types, network

communications to new user

sites, and special data backup

requirements.

For platform servers dedi-

cated to a single application, the

IT/IS CSV package is performed

separately on the one platform

system. The platform CSV pack-

age should be completed before

the time for formal testing by the

user team. A test environment is

then built on the platform system

in time for the user team to exe-

cute test scripts in a simulated

production environment.

Management control

Controlled GCP work

processes using


System reliability

Consistent, intended

performance of


Data integrity

Secure, accurate, and

attributable GCP e-data

Auditable quality

Documented evidence for

control and quality of

e-data and e-systems

e

Lifetime system validation goals



Management support

It is important that IT/IS manage-

ment provide an organizational

framework for developing CSV

packages for regulated platform

systems. The framework includes

SOPs for how to develop a CSV

package, for how to perform for-

mal testing in a documented fash-

ion acceptable to regulatory

inspection, and for describing a

standard approach to service level

agreements with parties that are

either internal or external to the

company.

In addition to SOPs for per-

forming CSV work, IT/IS man-

agement should agree or approve

a systems quality assurance plan

(SQAP) for data center operations

that provides an overall strategy

for ensuring the quality of system

operations. The IEEE Standard

for Software Quality Assurance

(IEEE Std. 730-1989) can be

adapted to this purpose.2All plat-

form system CSV packages would

then operate under the auspices

of the approved IT/IS systems

QA plan (see Systems QA Plan

box).

CSV package team

Just as we described for the user

CSV package team in Part 1, a

business decision group funds

and approves the IT/IS work pro-

cess for providing platform sys-

tems to clients with regulated

applications (Figure 5). A data

center manager is appointed as

sponsor of a particular platform

system and then assigns a team of

IT/IS operations personnel to

develop and maintain the CSV

package for the platform. The

quality assurance function stays

independent of package work and

System sponsor (CIO/head of IT/IS function)

The IT/IS dept. manager owning the

regulatory responsibility for the GXP

platforms. Provides personnel, budget,

and equipment. Approves CSV package

validation plan and package summary

report. Assigns a team leader.

System owner and team leader (senior manager,

IT dept.) Person responsible for ensuring

the system functions as intended for plat-

form purposes. Functions as team leader

for CSV package effort. Identifies and

leads a package team. Approves test

plan and test summary report. Drives the

package preparation process and

identifies ad hoc members as needed.

QA auditor (IT or GCP QA) Trains the team

on regulatory requirements for the

system and audits the package for

progress toward plans and compliancewith regulations.

Package manager (senior administrator, IT

dept.) Systems analyst trained in CSV

package documentation practices.

Drives item preparation, manages

package archive, and checks the quality

of documents in production for their

ability to pass audits.

Ad hoc members Provide administrative

support, specialty expertise, consulting,

training, testing, or other support as

needed. Platform size and scope

determine the size of this component.

Test coordinator (support specialist) Systems

analyst who understands the specific

use of the platform by the GXP

application. Develops test plan and

other test documentation. Identifies and

trains testers and witnesses in formal

testing practices. Manages formal

testing process.

Test script writer Individual with a

technical understanding of the

configured platform system to be

tested. Creates specific conditions to

test the platform and writes instructions

for how the platform system is to be

used to operate to those conditions.

Tester(s) Trained platform specialist who

executes the directions in the test

script. A tester observes system

response and records it in a testing log

or by capture of a screen or by printing

an expected report. The writer of a test

script can never be the tester for that

same script.

Witness Responsible individual trained in

GXP testing practices (testing SOP).

Ensures that GXP practices are followed

and that test logs contain all the items

requested by the test script. The writerof a script can witness its testing.

Platform CSV Team Roles

Purpose of plan

Business, regulatory, and techni-

cal purposes.

Scope of systems QA plan (SQAP)

Inclusions. Types of platform sys-

tems to be validated.

Exclusions. Internals of platformsystem components.

Limitations. Shared systems con-

trolled elsewhere.

Reference documents

Regulations, company policies,

department SOPs.

Management

Organization. Roles defined for

CSV package team.

Tasks. Validation package activi-

ties described.

Responsibilities. Role responsibili-

ties identified for package activities.aAdapted from IEEE Standard 730-1989.2

Documentation for quality control of systems

Logs, plans, reports.

Minimum documentation require-

ments are defined for standard

system packages and special

types of systems.

Standard formats and templates

are referenced in appendices.

Standards, practices, and metrics

SOPs for responsible installation,

maintenance, and support of vali-

dated platform systems, company

policies, and external regulations

applied to SQAP scope.

Reviews, audits, and inspections

Type and frequency of system

reviews.

Testing

Formal testing practices required

for validating systems.

Problem reporting

Problem reporting and corrective

action process.

Code and media control

Security, archival, and retrieval.

Supplier control

Content for quality and servicelevel agreements (SLAs).

Records collection

Records collection, maintenance,

and retention practices.

Training

Materials and instruction for sys-

tem support personnel.

Risk management system

Risk management, data security,

disaster recovery, legacy systems.

Approvals

Authorized signatures for this

plan.

Systems QA Plana



audits the CSV package for com-

pliance to company and regula-

tory standards.

The various CSV roles and

responsibilities for the package

team effort are described in the

box, Platform CSV Team Roles.

Given the high turnover rate in

many organizations, it is wise to

always keep the CSV package

team populated by at least three

rolesthe system owner, the

package manager, and test coordi-

nator. Thus, several knowledge-

able people are available who

know the system and its package

and can defend both in case of an

audit or inspection.

Developing a CSV package for

a platform system takes time,

costs money, and requires a team

of people. It is not a casual experi-

ence and must be planned for as

an ongoing business responsibil-

ity for organizations supporting

regulated application software

with platform systems and infra-

structure. Good practices for com-

puter validation can, however, be

integrated into normal good prac-

tices for running stable, unsink-

able operations in the IT/IS

department. Lifetime system vali-

dation goals are also good busi-

ness goals for IT/IS services.

References

1. Institute for Electrical and Elec-

tronics Engineers, Inc., IEEE Stan-

dard for Software Verification and

Validat ion Plans , Standard 1012-

1986 (IEEE, Piscataway, NJ, 1993),

pp. 1219.

2. Institute for Electrical and Elec-

tronics Engineers, Inc., IEEE Stan-

dard for Software Quality Assur-

ance Plans, Standard 730-1996

(IEEE, Piscataway, NJ, 1993), pp.

912.

Teri Stokes,PhD, is senior consul-

tant and director of GXP Interna-

tional, 131 Sudbury Road, Con-

cord, MA 01742, (978) 287-4393,

fax (978) 369-5620.

Figure 5. The CSV package team.

Business decision group (BDG)

Funds & approves computerized work process

System sponsor

Funds & approves

CSV package

Quality assurance

Audits CSV package &hosts GCP inspections

Testingcoordinator

Package manager

System owner

CSV package team

Develops & maintainsCSV package


16/29November 200 0 APPLIED CLINICAL TRIALS 1

nine steps, beginning with a sys-

tem idea and needs analysis and

continuing thr ough to retirement.

The software development life

cycle (SDLC) portion is shown in

detail in steps 3 through 5 (design,

build, test). These steps are the

focus of the suppliers computer

system validation (CSV) package.

Figure 1 also illustrates the

areas of close coordination with

the user group for refining the

user requirements specification

(URS, step 2), being audited in the

commissioning step (step 6), and

providing ongoing support and

enhancements under a ser vice

level agreement (SLA).

Validating Computer Systems, Par t 3

GCP Software VerificationTeri Stokes

oftware suppliers have

a tough job. Writing

software for use in a

GCP work process

adds an extra compli-

cation: preparing for many audits

of the software engineering pro-

cess itself as well as the software

application produced. Part 1 of

this series (August 2000) dis-

cussed a user acceptance package

for computerized system valida-

tion (CSV) to prove that the

right software was built for the

user requirements in the GCP

work process. Part 2 (September

2000) addressed the IT/ IS depart-

ments CSV package to document

that the software was installed on

the right platform of compo-

nents and infrastructure. Part 3

explores the software suppliers

CSV package that documents the

way the software was bui lt right

to meet its design specifications.

The life of a software applica-

tion, as shown in Figure 1, has

S

Showing these steps in a flat,

two-dimensional picture with

arrows going in one direction is

misleading, because there is usu-

ally a lot of cycling back and forth

and refinement of specifications

between steps 2 and 3, when users

are shown demonstration or pro-

totype versions of an application.

At some point, however, the URS

must be frozen so that the func-

tional requirements can be final-

ized and the software design

description made for the program-

mers to use as a blueprint for

building the software application.

Requirements. The Questions

on the Path to a URS box lists

questions for defining require-

ments on any size of software

application. The key factor here is

that the users work process

requirements are written down

and agreed to by all parties before

the start of building the software.

Extra time taken to refine the URS

with the supplier can pay huge

dividends, because the rest of the

development process evolves as a

response to the URS. Any error at

the star t builds error into the rest

of the process and the final prod-

uct. The cost to fix errors in speci-

fications grows exponentially

through each succeeding step of

the development cycle. Unclear

Techno logy Updat eTechno logy Updat e

Writing software

for GCP use is

serious business.

Applications must

be tested and

retested to ensure

that they actually

do what they are

designed to do.

Fig ur e 1 . The software development life cycle in detail, and how it fits with the rest of theapplication development cycle.


Software developmentlife cycle

Platform system

9. Retire


6. Commission

Validate & retestafter fixes

Fit ?3. Design

Functional requirementsspecification (FRS)

Software designdescription (SDD)

7. Operate

GCP workprocess

1. System idea

Needs analysisreport

8. Maintain

Fix & modify

IT/IS dept.Softwaresupplier

User group

4. Build

Program/ configureto SDD using standards

LEGEND

URS User requirementsspecification

SLA Service levelagreement

FRS Functionalrequirementsspecification

SDD Software designdescription

URS

2. System plan

5. Test

Peer review, code reviewunit, system, integration,

& regression testing

SLA


17/292 APPLIED CLINICAL TRIALS November 200 0

requirements result in unreliable

software.

Designing. The suppliers

response to the URS may be in one

or two technical documents, but

will include both functional specifi-

cations and design description.

Functional r equirements speci-

fication (FRS)document to

describe the functions a system

or component must perform.1

Software design description

(SDD)document to describe

system or component architec-

ture, control logic, data struc-

tures, input/ output formats,

interface descriptions, and

algorithms.1

The Software Specifications

and Testing box shows that all

requirements must be testable. A

good requirements document

lends itself to direct translation

into test scripts. If a user, function,

or design requirement cannot be

stated in a clear and precise way

so that it can be objectively tested,

then it is a wish and not a

requirement.

Building. Well-defined require-

ments also smooth the path for

the team building the software.

The software application can be

built in two ways.

Programming. The supplier

can use a programming lan-

guage or object tool to write

source code for a unique soft-

ware application such as SAS

(SAS Insti tute, Car y, NC) or

Excel (Microsoft, Redmond,

WA).

Configuration. The supplier

can configure an application by

setting preprogrammed varia-

tions in a configurable system

and writing small routines to

adapt it to the users specific

needs. Examples of such appli-cations are a macro for ran-

domizing treatments and a

spreadsheet for scheduling

patient visits.

Testing. As different levels of

software code are built, tested,

adjusted, and retested, there is

frequently a great deal of recy-

cling between development steps

4 and 5 (Figure 1). Testing is con-

ducted in several ways. Peer

review is informal testing; a pro-

grammer gives the code to

another software professional to

tr y out and asks for comments

and suggestions. Formal types of

testing are defined below as per

IEEE Standard 610.12-1990.1

Code review. A meeting at

which software code is pre-

sented to project personnel,

managers, users, customers, or

other interested parties for

comment or approval.

Unit testing. Testing of individ-

ual hardware or software units

or groups of related units.

System testing. Testing con-

ducted on a complete, inte-

grated system to evaluate the

systems compliance with its

specified requir ements.

Integration testing. Testing in

which software components,

hardware components, or both

are combined and tested to

evaluate the interaction

between them.

Regression testing. Selective

retesting of a system or compo-

nent to verify that modifica-

tions have not caused unin-

tended effects and that the

system or component still com-

plies with its specified require-

ments.

Although this formal develop-

ment and testing process is clearly

needed for large systemssuch

as a sponsors clinical data man-

agement system and high-end sta-

tistical systemmany people

question its usefulness for spread-

sheets, macros, and small applica-

tions used at investigator sites. In

fact, the same nine steps shown in

Figure 1 need to be traversed for

the life of any software used forGCP purposes, but the size and

scope of each step is scaled to the

size and scope of the software

application. The CSV package

documents are also scaled to fit

the size of the software. A small

software systems development

form can be designed to cover

steps 2 through 6 using two sides

of one sheet of paper.2

Sof twa r e suppl i e r CSV

p a c k a g e

Producing a good CSV package in

software development is good

business and exercises the stan-

dard practices for good software

engineering. A well-defined and

well-documented software devel-

opment life cycle provides the

software supplier with manage-

ment control of the engineering

process. Clear requirements and

documented coding standards

lead to efficient and effective

building of software and the abil-

ity to bring new programmers

into a project with minimal ramp-

up time and reduced risk of

errors.

Documented formal testing of

a product under development pro-

vides traceabil ity for changes and

a measure of the reliability of the

software and its capacity to pro-

tect the integrity of data it han-

dles. It lays a strong foundation

for problem resolution when the

application is out on the market or

future enhancements are consid-

ered. Test summary repor ts give

developers and testers the oppor-

tunity to present objective evi-

dence for how good they are at

creating robust, reliable software.

Good online and off-line user doc-

umentation facilitates user train-

ing, increases customer satisfac-

tion with the application, and

Extra time here pays off in asmoother design process.

Who are the sponsors andintended users?

What job is the systemexpected to do in the work

process? Which tasks?

Where is the documentationto describe in detail the usersGCP work process?

When is the software to bereleased, updated, fixed,tested, and maintained?

Why does the work processneed this software? Is itsafety- or efficacy-related?

How are users to be trainedand problems resolved during

ongoing operations?

Qu e s t io n s o n t h e P a t h t o a U R S

User requirements, functional specifications, and designspecifications must represent actual requirements ofthe work process and its platform system.

All requirements must be testable.

Testing must exercise all GCP aspects of specifications,including: Functions Limits Invalid entries

Error messages

Access control Online help messages.

S o f t w a r e S p e c if ic a t i o ns a n d Te s t in g

Work process

Equipment

People

SOPs



18/29November 200 0 APPLIED CLINICAL TRIALS 3

reduces the volume of help desk

support calls.

Experienced software suppli-

ers to the clinical trials market-

place understand that their clients

are responsible to the authorities

for the validation of software they

use. To meet that responsibility,

suppliers will be subjected to

client QA audits to assess the way

quality was built into the software

during development. A well-orga-

nized supplier CSV package that

emerges from the normal good

engineering practices of the sup-

pliers business will reduce audit

time and audit follow-up work.

The Software Supplier CSV box

lists the items in the package.

Whether each item is covered by a

sentence, a paragraph, a page of

text, or a manual of information

will depend on the size and scope

of the software product being

developed and the size of the

effort being described.

Verificat ion plan

The Institute for Electrical and


(IEEE) publishes a standard for

software verification and valida-

tion plans that can easily be

adapted.3 The Software Verifica-

tion Plan box shows a sample of

an outline for this purpose.

Every plan in the CSV pack-

ageverification plan