quality assurance for treatment planning systems

Quality Assurance for Treatment Planning Systems PTCOG Educational Session

May 19th 2010

Oliver Jäkel

Heidelberg Ion Beam Therapy Centerand

German Cancer Research Center, Heidelberg

Oliver Jäkel Medical Physics

IntroductionInternational recommendations; common sources of errors; vendor and user responsibility; legal aspects; risk analysis

Framework of QAQA and QC; acceptance tests; commissioning; periodic QA; uncertainty, deviation and tolerance; documentation;

Tools and MethodsSystem verification; software checks; measurements; Monte Carlo algorithms; analysis of results;

Typical test proceduresSoftware and data base; data transfer; generation of control parameters; imaging QA; geometrical and dosimetrical tests on dose distributions; radiobiological issues;

Conclusions

Outline


Fatal Errors in TPIntroduction

621 mistakes in NY state 2001-2009:At average 2 mistakes contributing


Intro: RecommendationsIAEA TRS No.430, 2004: Commissioning and Quality

Assurance of Computerized Planning Systems forRadiation Treatment of Cancer

IAEA-TECDOC-1540, 2007: Specification andAcceptance Testing of Radiotherapy Treatment PlanningSystemsIAEA-TECDOC-1583, 2008: Commissioning of

Radiotherapy Treatment Planning Systems -Testing for Typical External Beam Treatment Techniques

AAPM Report 55, TG 23: Radiation treatment planning dosimetry verification,1995

Van Dyk J et al. Commissioning and quality assurance of treatment planning computers. IJROBP 26, p261, 1993Fraas B, et al. AAPM TG 53: Quality assurance for

clinical radiotherapy treatment planning. Med. Phys. 25: p.1773, 1998

Jacky J, White CP, IJROBP 18:253.Testing a 3-D radiation therapy planning program (1990)


IAEA: Lessons Learned from Accidental Exposures in Radiotherapy, Safety Reports Series No. 17, IAEA, Vienna (2000).

Intro: Common Sources of Error

Inconsistent/incorrect basic dataInsufficient understanding of TPS

Incorrect calculation of open/wedged fields

Calc. error after change of TP Confusion of fractional dose vs.

total doseMisunderstanding of complex TP

given verbally

lack of documentation and verification

Inadequate commissioningInsufficient understanding of

the TPSIncomplete validation

Lack of an independent check of the treatment planLack of effective procedures

Most TP errors can be summarized by a lack of:Education, Verification, Documentation, Communication.


Intro: vendor responsibility Accurate specifications outlining system capabilities,

algorithms (incl. capabilities and limitations)

Detailed system documentation (system design, dose normalization, MU calculations)

User training: (1) basic training

(2) commissioning process

(3) system management

(4) implementation of a QA program

Detailed information regarding software updates, program alterations

Clear communication regarding bugs, error reporting


Intro: legal aspectsIntro: User responsibility

Define responsible physicist for supervision and management (incl. installation, acceptance, commissioning and QA)

Implementation of acceptance, commissioning and QA process

Record keeping associated with acceptance, commissioning, QA

User training / staff education: clinical use of TPS and its output(!)

Implementation, commissioning, QA of software upgrades incl. documentation

Ongoing communication with the vendor regarding software bugs and updates

Ongoing communication with the users of the output of the TPS esp. regarding limitations and bugs.


Intl. Recommendations (IAEA TecDoc 1040 QA in RT, ICRU)European directives (e.g. Medical Device Directive)AAPM (TG 24), ESTRO (Booklet No 4)

National radiation protection regulationGuidelines for medical application of radiationNational and International standards (ISO, IEC), e.g.:

DIN 6870-1: Quality management system in medical radiology Part 1: Radiotherapy

IEC 62083: Medical electrical equipment - Requirements for the safety of radiotherapy treatment planning systems

Intro: Legal Aspects of QA

There are few detailed and hard requirementsThe user always has to define a specific QA program


Identify, characterize, and assess potential risks Assess the vulnerability of critical elements to specific risksQuantify the risk (expected consequences of specific events)

Prioritize risk reduction measures based on a strategy

Intro: Risk Analysis

ProbabilityF: Frequent IIa IIb IIIE: Probable IIa IIb

D: Sometimes IIaC: hardly

conceivable IIa IIb IIb

B: very unlikely IIa IIa

A: : inconceivable I

0 1 2 3 4None Irrelevant Small Critical Fatal

Effects


Framework: QA and QCQuality assurance: All planned and systematic actions necessary to provide confidence that a product will satisfy given requirements for quality.

Quality Control:The regulatory process through which the actual quality performance ismeasured, compared with existing standards, and the actions necessary tokeep or regain conformance with the standards.

The QC process:(a) the definition of a specification; (b) the measurement of performance associated with that specification;(c) the comparison of the measurement with the specification; (d) the possible action steps required if the measurement falls outside the specification.

As part of step (d), one needs to define what is an acceptable deviation(a tolerance) from the known standard.


Assessment of clinical needs

Framework: Implementation of a TPS

Selection and purchase

Installation

Commissioning

Periodic and patient specific QA

Acceptance test

Training

Clinical use


Acceptance testAssure that the specifications of a product and safety standards are fulfilled (radiation and electrical hazards)

CommissioningCharacterization of the equipment's performance over the whole range of possible operation following acceptance incl. the preparation of procedures, protocols, instructions, data for clinical service.It includes development of procedures and QC tests and training.

Periodic QAProcedures which are performed regularly and which allow to assess, if the initial requirements are still fulfilled; may involve different procedures than during commissioning;

Patient specific QAProcedures performed on patient specific treatment plan or equipment.

Framework


Framework: Uncertainty, deviation and tolerance

Deviation: Difference between a measured or calculated value and a reference value.Deviations will depend on many factors:

position in the beam, complexity of phantom and treatment plandose calculation algorithm

Tolerance:Range of acceptability beyond which corrective action is required; Tolerances are always specific for a certain facility and application.Uncertainty: The uncertainty of a measured value has to be included in the tolerance or (preferred) may be added later;

Typically: accuracy of algorithms, beam delivery, beam properties, CT data, dosimetry

Error:Deviation of a quantity obtained through an incorrect procedure; A result maybe within the tolerance although an error was made


Dose measurements (Ion chamber, film, etc)Setup of phantomsBeam delivery (esp. in scanning, field homogeneity, stability)Dose calculation algorithmApproximations of the beam modelGeometric parameters (acc. of readings, instruments) Differences between commissioning and constancy checks

Sources of uncertainty

Example: How to separate beam delivery from TPS:Commissioning: Measure dose and compare with TP-dose

Defines reference for TP and DosimetryTP Periodic checks: Repeat calculation & compare w. safety copy Dosimrtry check: Repeat measurement & compare w. old data


Tools and Methods: Steps in QC

1. Definition of the specifications (performance and test characteristics)

2. Definition of tolerances (incl. uncertainties)3. Definition of tests via

- System verification- Software checks- Dosimetric measurements- Monte Carlo simulations

4. Performing tests and Comparison w. specs5. Possible Action steps if outside tolerance


G4

Mon

te C

arlo Xi

O

1 Gy(RBE)3 Gy(RBE)5 Gy(RBE)7 Gy(RBE)9 Gy(RBE)

11 Gy(RBE)13 Gy(RBE)15 Gy(RBE)17 Gy(RBE)

Range

MC versus TPS dose for passive delivery @ MGHPaganetti et al, Phys. Med. Biol. 53, 2008

Findings (29 fields): differences between TPS (PB-Alg.) and MC especially near the end of range + difference dose-to-water vs. dose-to-tissue


Bone slab

Water slab

Water slab

Bone slab

Underdosage up to ~20%, consistent withdosimetric finding in water phantom

Courtesy of F. Sommerer et al, HIT

FLUKA recalculation of treatment plan

MC vs. TPS for lateral inhomogeneityTPS plan


MC for individual treatment plans

FLUKA

TPS (TRiP)

Test proton plan:TPS (with inputFLUKA-generatedDatabase) vs MC

Courtesy of Parodi K., Sommerer F.,Unholtz D.,Brons S.


Monte Carlo simulations are extremely powerful

They will immediately show differences as compared to TPS

A Monte Carlo is not automatically giving correct results, although its algorithms may be more accurate than the TPS

Monte Carlo requires much more information as input (cross sections, HU-LUT for all materials)

QA for a Monte Carlo may be very complicated:- Documentation of the Code, User Manual - Version control and change management- Benchmarking of code- Control of input parameters (cross sections etc.)

Cautions

Very few (if any) certified codes for p-RT exists!Clinical decisions should drawn very carefully!


Examples of Test procedures @ HITSoftware & data base:Versioning, data base, data transfer, PACS- archive Machine interface & control parametersCalc. energies, ranges, field size, spot positions, fluence optimization Imaging QA Data transfer, HU-checks, geometrical image checks (distortions, scales, contrast), stereotactic tests, DRRGeometrical tests on dose distributionsField size, depth modulation, lateral and distal

gradientsDosimetrical tests on dose distributionsHomogeneous phantom (water, different angles)Depth inhomogeneities (slab phantom)Lateral inhomogeneities (slabs)Irregular inhom. Phantom (Alderson)

Radiobiological issuesCheck p-RBE, recalculate standard plans


Courtesy of S. Klemm

Plans of different complexityDefined measurement positions:

1 High Dose2 Plateau3 Lateral Gradient4 Distal Gradient

Tolerances @ HIT:

3

214

3

21

3

4

3

Dosimetric tests in a water phantom

Regular Target: Max. dev : 5% Mean. dev : 3%*Relative to maximum dose

max,i

max


3

ddd 1H @ HIT

Checks on the data base for treatment planning

To measure or not to measure ?

TPS: - check if input=output- check machine control file- check consistency

Beam delivery: measure!


Empirical range calculation from CT numbers

Range relative to water

CT number

Tissue equivalent phantomsReal tissue measurements


Imaging-QA: Possible distortions of CT numbers

Contrast agent in CT mean shift (25 pat.): 18 HUmax shifts: 36 HU Errors in range < 1.6 %

Reconstruction filters redistribution of HU numbersErrors in range < 3 %

Filter: AH50AH90

Dedicated imaging protocols and imaging QA required


Dose distribution carbon ions

PET-Measurement

Patient with Chondrosarcoma of skull base

PET for Range verification ?

PET is not routinely used for formal QA at GSI/HIT


Courtesy of B. Ackermann

Field geometry: IC in water vs. Films

Lateral

0.00

20.00

40.00

60.00

80.00

100.00

120.00

-60.00 -40.00 -20.00 0.00 20.00 40.00 60.00

lateral position in mm

D/D

(max

)

Measurement lateral rightTPS lateral rightMeasurement lateral leftTPS lateral left

Commissioing: Ion chambers

Periodic QA: films


Dosimetric Checks of dose calculation

Measurement equipment for commissioningWater Phantom PTW Freiburg24 PinPoint chamber array connected to a MP3 controller2 12-channel Multidos dosimeters


Dosimetric Verification of treatment plans for commissioing and patient specific QA

Data-acquisition in the TPSVx Verification Mode in TPS (Syngo PT Planning)Dose calculation in waterfor each PinPoint-ChamberCalculated dose [cGy]Gradient information [Gy/mm]


Tolerance for dose verification @ HIT

12

Regular Target:Max. dev. < 5% Mean. Dev. : < 3%

Irregular/complex Target: Max. dev: < 7% Mean. dev: < 5 %

max,i

max

max,i

max

Tolerances include dosimetric uncertainties!


MC application to active beam delivery @ HITFLUKA dose calculations of scanned fields for comparison withmeasurements and TPS calculation to support TPS-commissioning

Meas.: ICs in water phantom + data acquisition system from C. Karger (DKFZ)

FLUKA: ~ 108 p out of 4.4 1010

irradiated

Parodi, Mairani,Brons, HIT to be published

eye view Along depth Along depth

Protons


Patient Plan Verification: C-12


Radiobiological QA: Verification with cell samplesMitaroff A et al, Rad. Env. Biophys. 1998

Measurements for validation of the model and TPS.

No biological QA routinely used for formal QA at HIT.


MC for biological dose calculations @ GSI / HITFLUKA interfaced during runtime with the Local-Effect-Model (M. Scholz et al, GSI) used by TPS @ GSI / HIT

FLUKATRiP98Exp data

-TPS (TRiP98): biological planning and optimization for CHO cellsFLUKA-LEM: forward calculation of the optimized plan

Exp. Data and TPS calculations: Krämer et al, PMB 48 (2003) 2063MC Calculations: A. Mairani et al, Book of Abstract EW-MCTP, Cardiff,

2009; submitted to PMB


Radiobiological QA @ HIT

No real biological QATest only constancy of algorithmsBenchmark of new algorithm vs. old algorithm Check input in data base

Courtesy of C. Karger


Carefully analyze the clinical needsWrite down performance characteristicsDefine test characteristics, tolerances, actionsDefine how to test

(SOPs for commissioing and periodic QA)

Analyze uncertainties (SOP) Document all results Summarize findings for the users

Conclusion


Always think of the unexpected !

quality assurance for treatment planning systems

Documents