New England Chapter of the AAPM2013 Winter Meeting

Newton, MA

February 1, 2013 Making High Tech Radiotherapy Safe

Howard Amols, Ph.D.

Memorial Sloan Kettering

Cancer Center

New York, USA

Radiation Therapy Accidents

• WHO World Alliance for Patient Safety– Literature review 1976-2007– 3125 patients affected by errors in radiotherapy– ~1% resulted in death (middle and high income countries*)

• 98% of errors occurred in planning stage or during introduction of new systems or equipment

– Of errors without adverse effects• 55% planning• 25% new systems or equipment• 9% information transfer• 10% treatment delivery

– Mis-information or errors in data transfer- greatest bulk of “near misses” in modern radiotherapy services

Radiation Therapy safety problems

3 Myths

1.Radiation therapy accidents are rare

2.Most accidents happened long ago in the developing world

3.You need new high technology to have an accident

There’s been a focus on high-technology as the source of RT accidents.

This is only partially true.

New technology is often a contributing factor, but most RT disasters involving new technology required significant help from humans in order to escalate.

Really big errors are a team effort!

(more than one person has to make a mistake)

A Major Accident Requires:

Equipment malfunction AND/OR an individual act

of negligence

AND

a deficient QA/QC program, bad

communications, inadequate training,

lack of teamwork

If commercial airlines had the same error rates as hospitals there would be a major airline crash EVERY DAY.

There is something fundamentally wrong with the historical method by which the medical profession has approached QA/QC:

“Those who don't know history are destined to repeat it.” 

Edmund Burke

Radiation Accidents: Common Threadsnew equipment + new software, but no new QA/QC understaffed, overworked, undertrained, rushed

no internal redundancy, no external audits

no common sense, no time outs

bad communication, no central reporting

too much faith in manufacturer design

manufacturer and institutional denial

unusual clinical results ignored by physicians

A recent IAEA analysis of reported treatment errors noted common threads:

1. Possibility of systematic software error never considered;2. Manufacturer failed to inform users of previous incidents;3. Error messages from system ignored or overridden;5. Used software in unanticipated sequence that confused the system;6. Input wrong data, misunderstood input requirements;7. No physical dose measurements made to confirm;8. New or recently upgraded equipment;

Life is more complicated than it used to be!

• Control software for Varian Truebeam Linacr (for example) has > 1 million lines of computer code!

• No single person can completely understand how such a system works

• No government regulatory agency has the expertise to review such a system

• Teamwork and communication is essential

RT is more complicated than it used to be!

• Manufacturers software engineers rarely have clinical experience and do not know how a clinic really operates

• People will always find new ways to make mistakes that are difficult to anticipated

• Quality Assurance must be an evolving process, to be reviewed and modified whenever new technology is implemented and when errors are discovered

• Study near-misses as well as actual errors• Never underestimate the ingenuity of a fool, or someone

who is being rushed!

Special Dangers of Hi-Technology

1.Systematic errors harder to detect2.Humans get complacent. Don’t really

check computers (evolution of R/V systems)

3.Many treatment components too complex for humans to check (e.g., DMLC files, IMRT files)

4. Many treatment aids/devices are invisible (dynamic wedge, MLC, etc.)

5. Errors made on day 1 can propagate 6. Programmers don’t understand what we

do7. We don’t understanding what

programmers do 7. Too easy to `over ride’ warning messages8. Manufacturers training programs often

inadequate

My `favorite’ IMRT accident

• Almost everything that can go wrong in a radiation therapy accident occurred here!

• Best teaching tool I know of on the subject of radiation therapy errors

• Background, IMRT accident

• March 2005, New York City • A patient is to be treated with IMRT for head and

neck cancer (oropharynx)

• March 4 – 7: An IMRT plan is prepared: “1Oropharyn”. Verification plan created by TPS. EPID dosimetry confirms correctness.

• March 8: patient treated correctly with “1Oropharyn”.

• March 9-11: Fractions #2, 3 and 4 also correct. Verification images for the kV imaging system are created and added to the plan, now called “1AOropharyn”.

• March 11: Physician wants modified dose distribution (reducing dose to teeth) “1AOropharyn” is copied and saved to the DB as “1BOropharyn”

• NOTE the dates!

• March 14: Re-optimization for “1B Oropharyn”.

• New optimal fluences saved to DB.

• MLC motion control points for IMRT generated. Normal completion.

What happened?

• March 14• “Save all” is started. All new and modified data

should be saved to the DB.

• In this process, data is sent to a holding area on the server, and not saved permanently until ALL data elements have been received.

• Data to be saved included: (1) fluence data, (2) DRRs and (3) MLC control points

What happened?

• March 14, 11 a.m.• An error message is displayed.

• The user presses “Yes”, which begins a second, separate, save transaction.

• MLC control point data is moved to the holding area.

The purpose of this error message is so that you can click `yes’ to proceed

What happened?

• March 14, 11.a.m.• The DRR is, however, still locked into the faulty first

attempt to save.

• This means the second save won’t be able to complete.

• The software would have appeared to be frozen.

Ctrl-Alt-Del usually gets you out of this so....

What happened?

• March 14, 11.a.m.• Within 12 s, another workstation is used to open the

patients plan to load into VARIS and to treat.

• Unbeknownst to the users, the MLC motion file is NOT properly saved because the error message was over-ridden

What happened?

• March 14, 11 a.m.• No verification plan, no pre-treatment dosimetry, no

review by 2nd physicist (not enough time!!)

• Several computer crashes ignored and over-ridden.

• Plan approved by physician (the plan looked OK on printout, but the computer data files were corrupted)

Big Errors are usually a team effort!So far:• The radiation oncologist did not read port films on time, and then rushed

everybody else• Neither therapists nor physicists confront him• The treatment planner over-rode an error message they did not understand• The physicists did neither a double check, nor a dosimetry test of an IMRT

plan (also called `billing fraud’)• There were real bugs in the software design (it crashed, two people could

open file at the same time)• So far no real harm has been done!• But they’re sowing the seeds for a really great accidents!

And now the therapists add the final piece!

March 14, 2005, 1 p.m.

• What they should have seen:

March 14, 2005, 1 p.m.: What they also didn’t notice:

1324 MU with MLC wide open

Discovery of accident

• March 14-16, 2005• The patient is treated without MLCs for 3 fractions

• On March 16, a verification plan is created and run on the treatment machine. The operator notices the absence of MLCs.

• A second verification plan is created and run with the same result.

• The patient plan is loaded and run, with the same result.

Impact of accident• The patient received 13 Gy per fraction

for three fractions, i.e. 39 Gy in 3 fractions

Was there a bug in the Manufacuters software?Yes, but the software bug needed a lot of help in order to kill somebody:

1. MD didn’t check port films and rushed the plan modification 1. Nobody confronted the MD2. Error messages ignored and not investigated3. Treatment plan QA not performed4. Therapists weren’t watching MLC display5. neither the manufacturer, nor the user, nor the FDA, nor anyone else has the expertise to test this software for safety6. Accident not made public until much later!

Disasters are a team effort!

More radiation therapy accidents

Springfield, Mo., 2004-09. 76 SRS patients overdosed >50 percent. Used too large a dosimeter to calibrate SRS fields. No independent check, no mandated state or federal reporting reguirement, no requirement for physicists or therapists to be certified. Insufficient support from manufacturer.

Tampa, Fla., 2004-5. , 77 SRS patients overdosed >50% because PDD factor not used in TG-51 calibration. Uncovered after 1 year during RPC inspection for participation in RTOG.

10mm

24mm

7mm

For 10mm coneThe Farmer chamberis much larger than the x-ray field, and most of it is in the beam penumbra.Dose reading = (6+4x0.7)/24 = 37%

Penumbra= 3-5 mm

Penumbra= 3-5 mm

Note: dose uniformity should be +3% over 80% of the field width (for 10 x 10 field size).80% of field = 8mm. But for very small fields uniformity is worse.

More radiation therapy accidentsAnother radiosurgery error: Linac collimators not properly set for small SRS cones (hospital staff assumed this was done automatically)

SRS coneInsert (2-3)

SRS coneHolder(5x5)

Linac Collimators(10x10)

More…..

.

UK, 1982-90: incorrect SSD correction (did not know how TPS worked). 1045 patients, 30% underdose, >492 RT failuresBend, Oregon, 1980’s: incorrect TPC. 13% overdoseSpain, 1990: Linac `repair’ led to 36MeV e- beam no matter what was programmed. No dosimetry check. 27 patients, 15 deathsFrance, 2004: incorrect MU for dynamic wedge. 23 patients overdosed 20%, 4 deathsGlasgow,2006: incorrect calculation of MU. Planner thought TPS calculated MU/Gy and not MU/fraction. It didn’t! 67% overdose results in deathFrance, 2006-7: large ion chamber used for SRS. 145 overdoses.

Errors we have Seen at Memorial Sloan Kettering

MLC Error (FSRT) Event Sequence:

FSRS treatment scheduled right after DMLC treatment At end of DMLC treatment leaves are all closed BrainLab mMLC attached to Linac below regular MLC Therapist does not retract primary MLC leaves Primary MLC leaf position not detected by R&V (thinks it’s SRS) Patient treated with correct mMLC apertures and closed primary MLC

Why:

Software not designed for two MLC’s Light field not normally used for SRS

DMLC Error: IMRT treatment with open MLC leaves:

DMLC field selected for treatment after a static MLC treatment DMLC plan loaded, leaves retracted for light field use “Go” selected, leaves fail to return to prescribed position All systems allow treatment to proceed with retracted leaves Therapist fails to detect error Why:

Software did anticipate this sequence of events Could only have happened with exactly the right wrong timing Very similar to Therac-25 disaster

Record/Verify Systematic Error with DMLC:

DMLC (v1) created from TPS and sent to R/V Dosimetry checks done Plan changed MU (only) manually edited in R&V No Independent Check of Data Difference too subtle to see on Portal Image Check

Why:

Human error Improper understanding of software (change in TP system doesn’t automatically

get transferred to R/V)

Upgraded,VARISto ARIA

Radiation Therapy Safety Program at MSKCC

• MSK was the first center in the US to initiate IMRT

• Since 1995 we have treated over 20,000 patients with IMRT

(> half a million treatment fractions)

• We never dismiss the potential for errors, and have put into place a multi-tiered Quality Assurance Program to continually monitor all phases of radiation therapy treatments at MSK

• Almost all radiation therapy accidents result from a combination of equipment (or software) malfunctions PLUS human error

Radiation Therapy Safety Program at MSKCC

- training in all new technologies- staff all board certified and licensed- continuing staff education - all errors, large and small are fully discussed at

monthly meetings of the Rad Onc QA Committee- Minutes of Rad Onc QA Committee are reported

and discussed at the Hospitals QA Committee- QA program is continually modified and upgraded

to reflect new findings and technologies- New equipment and computer systems are tested

prior to release for patient treatments

The human component of most radiation therapy accidents results from a combination of:

• failure to appreciate limitations of new technologies

• inadequate redundency in QA program:

- people checking people

- people checking computers

- computers checking people

- computers checking computers

• static or non-proactive QA program

• System wide QA of all treatment devices, hardware and software

• Patient specific QA for each individual patients customized

treatment plan

Quality Assurance at MSK consists of two equally important components :

System wide QA of all treatment devices, hardware and software

• Annual dosimetry intercalibration test with Radiological Physics

Center

• Periodic dosimetry and treatment planning accreditation from

RTOG for IMRT and IGRT national protocol studies

• Periodic reviews by ACR (ASTRO??)

• Every day, during each treatment fraction for all patients all treatment parameters (dose, mlc settings, x-ray energy, etc. are recorded by the linac computer and stored in a `treatment log file’

• Every evening a `batch’ computer program is run comparing all `IMRT treatment log files’ with the planned MLC files. Discrepancies are investigated immediately by a medical physicist

• On a weekly basis all patient treatment folders are reviewed by both a medical physicist and a radiation therapist to insure consistency between delivered treatments and treatment plans

• On a weekly basis portal films and/or orthogonal x-rays are taken on each patient to insure correctness of patient treatment position

Using Linac Log Files

Human errors usually fall into one of three categories:

1. Department policy is properly followed, but an error occurs anyway. For example, policy requires that all treatment plans and MU calculations be independently checked by a second person before the patient's first treatment. This policy is followed, but the second person also fails to detect the error;

2. Department policy is not followed. For example, no one performs an independent check of treatment plan or dose calculation prior to the first treatment; and

3. Department policy is deficient or incomplete. Most common for new technology

4. Bizarre errors: sequence of events, almost impossible to foresee or prevent

Nothing is foolproof for the sufficiently talented fool!

R/V systems, computer controlled Linacs, image guided patient positioning systems, etc. reduce but do no preventerrors. They enable humans to make different kinds of mistakes faster and more efficiently.

New Paradigm for QA

Most errors are NOT systematic. They are patient specific. Therefore QA should shift from equipment focused to patient focused.

Patient Specific QA:

Treatment plan check (more difficult than before)

R/V, file check-sums (each fraction)

Independent MU check, dosimetry, portal images

Log file analysis, chamber measurement, film dosimetry

Machine Specific QA:

Film test

Dosimetry test

Drift test

MLC and IGRT tests

•

Is QA Reporting Like an Iceberg?

ReportedErrors

Discovered Errors

Actual number of errors

The Aviation Approach

Even good people makemistakes, they are the the rule, not the exception

Mistakes result from flaws in the system

Find the system flaw that made it possible for a particular individualto make a mistake

Start at the top and workdown the system until thecause of the error is found

They rarely make the sameMistake twice

The Hospital Approach

We are Gods. Mistakes are the exceptions not the rule

Mistakes result from individual screw ups

Find and punish the person who made the mistake

Start at the bottom and work up. Find the lowest ranking person you can blame

The Aviation Approach

Even good people makemistakes, they are the the rule, not the exception

Mistakes result from flaws in the system

Find the system flaw that made it possible for a particular individualto make a mistake

Start at the top and workdown the system until thecause of the error is found

They rarely make the sameMistake twice

The Hospital Approach

We are Gods. Mistakes are the exceptions not the rule

Mistakes result from individual screw ups

Find and punish the person who made the mistake

Start at the bottom and work up. Find the lowest ranking person you can blame

The Aviation Approach

Even good people makemistakes, they are the the rule, not the exception

Mistakes result from flaws in the system

Find the system flaw that made it possible for a particular individualto make a mistake

Start at the top and workdown the system until thecause of the error is found

They rarely make the sameMistake twice

The Hospital Approach

We are Gods. Mistakes are the exceptions not the rule

Mistakes result from individual screw ups

Find and punish the person who made the mistake

Start at the bottom and work up. Find the lowest ranking person you can blame

The Aviation Approach

Even good people makemistakes, they are the the rule, not the exception

Mistakes result from flaws in the system

Find the system flaw that made it possible for a particular individualto make a mistake

Start at the top and workdown the system until thecause of the error is found

They rarely make the sameMistake twice

The Hospital Approach

We are Gods. Mistakes are the exceptions not the rule

Mistakes result from individual screw ups

Find and punish the person who made the mistake

Start at the bottom and work up. Find the lowest ranking person you can blame

The Aviation Approach

Even good people makemistakes, they are the the rule, not the exception

Mistakes result from flaws in the system

Find the system flaw that made it possible for a particular individualto make a mistake

Start at the top and workdown the system until thecause of the error is found

They rarely make the sameMistake twice

The Hospital Approach

We are Gods. Mistakes are the exceptions not the rule

Mistakes result from individual screw ups

Find and punish the person who made the mistake

Start at the bottom and work up. Find the lowest ranking person you can blame

The Aviation Approach

Even good people makemistakes, they are the the rule, not the exception

Mistakes result from flaws in the system

Find the system flaw that made it possible for a particular individualto make a mistake

Start at the top and workdown the system until thecause of the error is found

They rarely make the sameMistake twice

The Hospital Approach

We are Gods. Mistakes are the exceptions not the rule

Mistakes result from individual screw ups

Find and punish the person who made the mistake

Start at the bottom and work up. Find the lowest ranking person you can blame

The Aviation Approach

Even good people makemistakes, they are the the rule, not the exception

Mistakes result from flaws in the system

Find the system flaw that made it possible for a particular individualto make a mistake

Start at the top and workdown the system until thecause of the error is found

They rarely make the sameMistake twice

The Hospital Approach

We are Gods. Mistakes are the exceptions not the rule

Mistakes result from individual screw ups

Find and punish the person who made the mistake

Start at the bottom and work up. Find the lowest ranking person you can blame

The Aviation Approach

Even good people makemistakes, they are the the rule, not the exception

Mistakes result from flaws in the system

Find the system flaw that made it possible for a particular individualto make a mistake

Start at the top and workdown the system until thecause of the error is found

They rarely make the samemistake twice

The Hospital Approach

We are Gods. Mistakes are the exceptions not the rule

Mistakes result from individual screw ups

Find and punish the person who made the mistake

Start at the bottom and work up. Find the lowest ranking person you can blame

Use of High Tech In Surgery

Position inthe peckingorder

MD’s

Nurses

Technicians

Number Of chances to misuse hi-tech

e.g.; robotics, lasers, laproscopic

Use of Hi-Tech in RT: inverted training/culpability Pyramid

MD’s

Physicists

Dosimetrists

Therapists

Therapist

Dosimetrists

Physicists

MD’se.g.; Linac, MLC, IGRT, R/V, treatment planning

Position inthe peckingorder Number

of chances to misuse hi-tech

Use of Hi-Tech in RT: inverted training/culpability Pyramid

MD’s

Physicists

Dosimetrists

Therapists

Therapist

Dosimetrists

Physicists

MD’se.g.; Linac, MLC, IGRT, R/V, treatment planning

Position inthe peckingorder Number

of chances to misuse hi-tech