a view from the auditor melbourne 5 th october 2012 ivan williams, joerg lehmann, john kenny,...

A view from the auditor

Melbourne 5th October 2012

Ivan Williams, Joerg Lehmann, John Kenny, Jessica Lye, Leon Dunn

Patient Safety in Radiation Oncology: Australian Edition

Why use / trust an auditor ?

2

1045 patients

Defensive (negative) rationale

International Context• Numerous international examples of radiation

oncology accidents• Analysis has shown that most systemic errors

could have been detected by independent external audit

• International audits have all found issues which needed to be resolved

4

Why audit

5

“Firstly and most directly, in every dosimetry audit programme, measured doses have been observed and reported which have been outside the required tolerances, in some cases significantly so.”

Thwaites DI, SSDL 58, June 2010Izewska J, et al., SSDL 58, June 2011Ibbot, G.S., Followill, D.S., SSDL 58, June 2011.

Positive rationaleA properly constructed auditing program has resources beyond

the capacity of any (many?) radiotherapy facilities.• Time

• Dedicated staff• Dedicated equipment • Technical expertise

• Oversight panels with professional experts

6

The ACDS program has expertise and abilities beyond the capacity of many radiotherapy facilities within its area of audit

Example I: Time

Example II: Technical expertise

Courtesy R.Ganesan, P Harty.

Example III: Equipment

Courtesy R.Ganesan, P Harty.

10

ACDS

ARPANSA (CEO)

Branch HeadMedical Radiation

Daily Administration

Formal Responsibility

Department of Health and Ageing

MOU

CAG

Auditors

Facilities & Professional

Organisations

AdviseApproval of Documents

Positive rationale: Oversight

• Al2O3:C (Sapphire) nanoDotTM OSLDs • Size: 1x1x0.2 cm3

• Retractable Active element: 5 mm diameter, 0.2 mm thick• Material: Aluminium Oxide doped with Carbon• Bar-coded for tracking

1.0 cm

1.0

cm

Technical Expertise: OSLD commissioning

Commissioning - Overview• Reproducibility• Signal depletion during readout • Reader stability• Fading• Linearity• Energy dependence

Why? Accurate calculation of absorbed dose requires a correction for each of these. Kf, Kl, Ke, Kr

y = -0.02840x + 100.03959R² = 0.98430

93

94

95

96

97

98

99

100

101

0 50 100 150 200

Nor

mal

ised

Sig

nal

(%)

Read #

Linear (Reader 1 - Sn: 11340531)

Each readout of the dosimeter depletes the signal by 0.028%.

• Consequence: Each dosimeter can be re-read a number of times and corrected to reduce readout uncertainty.

Signal depletion per read

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

0 200 400 600 800 1000 1200 1400

Sig

nal

x 10

0000

Dose (cGy)

y = -0.058ln(x) + 1.2816R² = 0.9834

0.80

0.85

0.90

0.95

1.00

1.05

1.10

0 200 400 600 800 1000

[cG

y/si

gnal

] / [c

Gy/

sign

al(1

00cG

y)]

Dose (cGy)

(a)

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

0 200 400 600 800 1000 1200 1400

Sig

nal

x 10

0000

Dose (cGy)

y = -0.058ln(x) + 1.2816R² = 0.9834

0.80

0.85

0.90

0.95

1.00

1.05

1.10

0 200 400 600 800 1000

[cG

y/si

gnal

] / [c

Gy/

sign

al(1

00cG

y)]

Dose (cGy)

(b)

• Supralinear response with the degree of supralinearity being dependant on the accumulated dose.

• Decrease in sensitivity as the absorbed dose to the nanoDot OSLD increases.

• The sensitivity = cGy per unit signal.

Linearity

Slight energy dependence for MV photons up to 1% relative to response at 6MV

2.8 % w.r.t response at 6MV

Consistent with (Aznar et al., 2004; Jursinic, 2007; Viamonte et al., 2008), along with the manufacturers stating that there is little to no energy dependence.

Electron beams show similar dependence (1 – 2%) with a larger measurement error associated with higher energies (18 – 20 MeV)

Energy Dependence

• The absorbed dose can be directly calculated using the following equation:

Daudit = [(Counts.kr –Counts(bg).kr(bg).kf(bg)) kf ] ECF.S.kE.kL

– ECF is the individual element correction factor defined as the ratio of the mean batch counts, after 100 cGy irradiation, to the individual OSL counts, after 100 cGy irradiation.

– S is the batch sensitivity, in cGy/counts, to 100 cGy of 6 MV photons.– kE is the energy correction to account for the slight energy dependence in

OSLD response relative to 6 MV photons.– kL is the non-linearity correction to account for the non-linear sensitivity of

the OSLD, normalized to the sensitivity at 100 cGy.– kr is the reader correction to account for the daily variation in the OSLD

reader.– A reader correction, kr(bg), is also applied to the background signal.– kf is the fading correction to account for the reduced signal that occurs

between irradiation and readout date.– A fading correction, kf(bg), is also applied to the background signal from the

initial ECF measurement. "

Block Factors

The ACDS: An auditing programIn July 2010, the Australian Government funded a trial initiative to provide external, independent dosimetric verification for Australian radiotherapy centres: The Australian Clinical Dosimetry Service, ACDS.

Housed within Australian Radiation Protection and Nuclear Safety Agency, ARPANSA, under a Memorandum of Understanding, MoU.

Analysis of the service will be conducted in the third year to determine the outcomes of the ACDS

A decision will be made whether to continue, modify or terminate the program based on the outcomes

ACDS trialFundamentally

To increase the safety of radiotherapy within Australiavia:

1. Three level audit – Level I, II and III.2. National coverage3. Private and public clinics4. Interaction with professional colleges – Level Ib

The service is free and voluntary

20

Within MoU

Extant to MoU

Australian Context: Risk contributors• 50000+ Australians treated per year• On-going roll-out of new radiotherapy clinics and updating of

older machines – technology • Detection of errors within modern machines can be more

difficult in modern cancer therapies• Australian is BIG - logistics• Sparse population and large cities – regional centres, country

centres and some metropolitan centres do not have local support

• Staff shortages

21

ACDS Audit Levels

Level I: Linac output under reference conditionsLevel II: Treatment planning and delivery Level III: End-to-End test

Based on T.Kron et al., IJROBP 52(2), 566–579, 2002

DiagnosticImaging

TargetOutlining

TreatmentPlanning

Beam Calibration

Patient Setup

TreatmentDelivery

Recordand Verify

Level I

Level II

Level III

- based on auditors absolute measurement uncertainty ()- action level: 2, failed audit > 3- reporting to center: “Dose is 0.7 % high with a 2 of 4.2%”

Audit reporting

All audit rest on the fundamental dosimetry, however, as the investigation approaches the dose to patient, the multiple factors affecting the dose to the patient must be considered.

Level I Audit

Level II Audit

Treatment Delivery

Beam Modelling

Level III Audit

Lung / Thorax 4D

Planning /Delivery

IntensityModulated

H&N Pelvis Breast

Phantom geometry

Calc Algorithm

Simulation

TargetingInhomogeneity

Dose to Patient

Dose Delivery / Dose Calculation

Dose to Water

Methodology & Design

With an on-going audit program, such as the ACDS, a variety of Level II audit test capabilities provides a strong foundation for Level III audits and a fall-back approach when questionable Level III outcomes arise and must be investigated.

Methodology & Design

Level I Audit

Level II Audit

Treatment Delivery

Beam Modelling

Level III Audit

Lung / Thorax 4D

Planning /Delivery

IntensityModulated

H&N Pelvis Breast

Phantom geometry

Calc Algorithm

Simulation

TargetingInhomogeneity

Dose to Patient

Dose Delivery / Dose Calculation

Dose to Water

Similarly, issues arising with a Level II audit may be investigated and resolved with a Level I

Methodology & Design

Level I Audit

Level II Audit

Treatment Delivery

Beam Modelling

Level III Audit

Lung / Thorax 4D

Planning /Delivery

IntensityModulated

H&N Pelvis Breast

Phantom geometry

Calc Algorithm

Simulation

TargetingInhomogeneity

Dose to Patient

Dose Delivery / Dose Calculation

Dose to Water

Level IPassive dosimeter, TLD/OSLD, placed in the clinical beam in a regular, reproducible environment with well understood conditions. External audit.

Was TLD (IAEA approach), changing to OSLD for logistical and operational reasons in July 2012

Required: 60% of all linacs in Australia.To-date: ~50%Expected: ~100%

27

-8

-6

-4

-2

0

2

4

6

8

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110

Vari

ation

from

ACD

S (%

)

Beam ID

Output for All Photon Energies: 1st January 2011 to 18 September 2012Level I Audit (off-site, TLD, photon only)

ACDS Action Level (+/- 4.2%) ACDS Tolerance Level (+/- 6.3 %) Individual Photon Beam Measurement Mean

6 MV 10 - 15 MV 18 MV

-8

-6

-4

-2

0

2

4

6

8

0 10 20 30 40 50 60 70 80 90 100 110 120 130

Vari

ation

from

ACD

S (%

)

Beam ID

Output for All Photon Energies: 1st Jan 2011 to 18 September 2012Level I Audit (off-site, OSLD and TLD photon only)

ACDS Action Level ACDS Tolerance Level TLD OSLD

Level Ib – by consumer demandOn-site measurement with chamber for photons and electrons

Required in many European Nations,

Required by Australian Radiation Oncology Practice Standards, criterion 15.1.

Recombination, polarity and output

Organisation supplies water tank, beam data

ACDS supplies chambers, electrometer, meters, cables ...

-4

-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115

Vari

ation

from

ACD

S (%

)

Beam Number

Level 1b Audit (On-site, Ionisation Chamber) as at 18 September 2012Outputs for All Photon & Electron Energies

Individual Beam Measurement ACDS Action Level 1.5% ACDS Tolerance Level 3.0%

-4

-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115

Vari

ation

from

ACD

S (%

)

Beam Number

Level 1b Audit (On-site, Ionisation Chamber) as at 18 September 2012Outputs for All Photon & Electron Energies

Individual Beam Measurement ACDS Action Level 1.5% ACDS Tolerance Level 3.0%

≥ 10 MV6 MV 6 MeV 8-9 MeV 12 MeV 15-16 MeV ≥ 18 MeV

-4

-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115

Var

iati

on fr

om A

CDS

(%)

Beam Number

Level 1B Audit (On-site, Ionisation Chamber) as at 18 September 2012Outputs for All Photon & Electron Energies

Individual Beam Measurement ACDS Action Level 1.5% ACDS Tolerance Level 3.0% Proposed New Action Level Proposed New Tolerance Level

≥ 10 MV6 MV 6 MeV 8-9 MeV 12 MeV 15-16 MeV ≥ 18 MeV

Level II

2D array of detectors placed in the clinical beam in a phantom of solid water. Lung slabs are added and measurement is compared with predictions from the computer planning system. Outcomes are derived from the spatial and dosimetric difference between the predicted and measured doses.

34

DiagnosticImaging

3D TreatmentPlanning

Patient Setup

TreatmentDelivery

Recordand Verify

Level II – Basic Design

Required: 40 % of all linacs in Australia.To-date: Testing and field trialsExpected: ~40 %

Level IIIEntire process check from CT to treatment with a human-like plastic phantom. Outcome is obtained from the spatial and dosimetric difference between measurement and prediction.

Required: 15 linacs within Australia.To-date: 9 linacs auditedExpected: 20+ linacs

36

Humanoid Phantom (Ann D Roger) goes through the complete chain of procedures a patient experiences in Radiation Therapy.

DiagnosticImaging

3D TreatmentPlanning

Patient Setup

TreatmentDelivery

Recordand Verify

CIRS thorax phantom

1

2 3 4 6 5 8

7 9 10

Level III

Radiation Therapists should conduct each of the steps in keeping with routine clinical practice so that the audit assesses the actual patient process.

Level III

Level III Dose Tolerances• measurement uncertainty () cannot be determined with

sufficient accuracy in the given complex geometry• clinical acceptability (5%) is used as a starting point for 3• points in low dose areas / clinically insignificant areas / not well

defined areas are reported but not scored (RNS)• and reporting will be re-evaluated over time as data comes in

with the goal of catching outlying results

Level III – case 2

Field: 6x, 10 cm x 15 cm 45° wedgePrescription: 2 Gy to Point 1

Location Expected dose Plan vs MX (local ref) Plan vs MX (global ref)

Point 1 – WDT ~200 cGy -0.63% (-2.51%, +1.22%)

Point 4 – WDT > 300 cGy 0.06% (-2.62%, +1.65%) 0.11% (-4.02%, +2.56%)

Point 7 – LAA ~4 cGy -17.1% (-42.9%, +18.3%) -0.37% (-1.02%, +0.34%)

average (min, max)

• adjusted for points in low dose areas• Global reference is used instead of local

1 2 3 4 6 5 8 7 9

10

Level IIField ID Level II - Field Description Level III

reference

Field 7case 3 LATopen field, asymmetric

Field 8case 3 LAT

wedged field, asymmetric

Field 9

case 3 LATopen field, asymmetric

inhomogenity

Field 10

case 3 LATwedged fieldasymmetric

Inhomogenity

Level II underpinning Level IIIExample fields

Recommendation IReview the accuracy of all barometers used for clinical dosimetry: Ensure that they are calibrated by a NATA accredited service, which is accredited for barometers.Ensure that the barometer(s?) is re-calibrated according to instructions.

1 % error in pressure ~ 1 % error in dose

42

Early lessons learned

43

• Air pressure issues– Barometer not calibrated (properly) or faulty– Airport pressure

• Equipment (ionization chambers) problems– Outdated styles– Slightly damaged Solvable administrative problems

• Understanding of calibration / QA process– Staff changes– Long living spread sheets – routine QA– Wrong calibration factor used

Successes & Challenges• Level I – ACDS will overshoot 100*% v 60% (accepted

by DoHA)• Level Ib – Outside MoU (accepted (commended?) by

DoHA)• Level II – ensure the ACDS hits target of 40 %• IMRT – Require plan for future• Prepare for review = prepare for post 2014• External Professional Expectations/Desires

45100 % = >95 %

AcknowledgementsJohn KennyJörg LehmannLeon DunnJessica LyeTomas KronAbel MacDonaldAlison McWhirterTracey Rumble

Ramanathann Ganesan Peter HartyDavid WebbDuncan ButlerChris OliverPeter Johnston

'The Australian Clinical Dosimetry Service is a joint initiative between the Department of Health and Ageing and the Australian Radiation Protection andNuclear Safety Agency'