statistical process control for analysis of routine machine...

Statistical Process Control Statistical Process Control for Analysis of Routine for Analysis of Routine

Machine QCMachine QC

Mac Clements, MSMac Clements, MS

Radiological Imaging Technology, Inc.Radiological Imaging Technology, Inc.

©© Radiological Imaging Technology Radiological Imaging Technology -- 20122012 11

BackgroundBackground

�� Recent publications have stressed the Recent publications have stressed the need for managing QA systems.need for managing QA systems.�� Sentinel Event Alert #47 (2011)Sentinel Event Alert #47 (2011)

�� ““Implement a system for centralized quality and Implement a system for centralized quality and safety performance monitoring of inventoried safety performance monitoring of inventoried equipment under the supervision of a qualified equipment under the supervision of a qualified medical physicistmedical physicist””

�� AAPM Task Group 142 (2009)AAPM Task Group 142 (2009)�� ““A sudden and significant deviation from the A sudden and significant deviation from the

expected value should be called to the attention of expected value should be called to the attention of the MP, even if the measurement itself does not the MP, even if the measurement itself does not exceed the table tolerance value.exceed the table tolerance value.””


BackgroundBackground

�� ACR AccreditationACR Accreditation�� Therapy accreditation inspecting to Therapy accreditation inspecting to

TG142 standards (as of 2011)TG142 standards (as of 2011)

�� State legislation (TG142 State legislation (TG142 –– Ohio??)Ohio??)

�� RTOGRTOG�� November 2011 November 2011 –– updated its updated its

requirement and is testing to TG142 requirement and is testing to TG142 standards rather than TG40standards rather than TG40


Definition of Definition of ““ QualityQuality ””

Quality ControlQuality Control

�� Is the process Is the process ““in in controlcontrol”” or or ““out of out of controlcontrol””??

�� Consistency in Consistency in measurements.measurements.

Quality AssuranceQuality Assurance

�� Predictability in Predictability in measurements.measurements.

�� Opposite of the Opposite of the ““rearrear--view mirror approachview mirror approach””..


•• Minimizing process variationMinimizing process variation•• Understanding source of fluctuationsUnderstanding source of fluctuations

•• And how/when to fix itAnd how/when to fix it

Principles for Understanding DataPrinciples for Understanding Data 11

�� ““No data have meaning apart from their No data have meaning apart from their context.context.””

�� ““Every data set contains noise, and some data Every data set contains noise, and some data sets contain signals. Therefore, before you sets contain signals. Therefore, before you can detect a signal, you must first filter out the can detect a signal, you must first filter out the noise.noise.””


1. Understanding Variation; The Key to Managing Chaos. (2nd Edition) Donald Wheeler (2000)

Conventional QAConventional QA


Begin

End

Determine Specifications

Perform the Test

Result

Determine Reason

Document

Unacceptable resultAcceptable result

�� No understanding of variation No understanding of variation inside specificationsinside specifications

�� No understanding of trends in No understanding of trends in datadata

Pawlicki, T., Whitaker, M. “Variation and control of process behavior”, Int J Rad Onc Biol Phys (2008)

The Source of (Some) SpecificationsThe Source of (Some) Specifications


�� Without any context for the data, specification Without any context for the data, specification may give an incomplete picture.may give an incomplete picture.

AAPM

ACR

Manufacturer

1. Understanding Variation; The Key to Managing Chaos. (2nd Edition) Donald Wheeler (2000)

Specifications vs. ContextSpecifications vs. Context

�� Is the measurement on day 14 a signal?Is the measurement on day 14 a signal?

�� Is it something to be concerned about?Is it something to be concerned about?


Possible FailuresPossible Failures

�� Whenever a system attempts to Whenever a system attempts to detect a detect a ““signalsignal””, there are 2 possible , there are 2 possible failures:failures:�� 1: Failure to separate the signal from 1: Failure to separate the signal from

the background noisethe background noise

�� 2: Incorrect identification of noise as 2: Incorrect identification of noise as signalsignal

�� Both result in lost time (looking for Both result in lost time (looking for source of error, or making up for source of error, or making up for missed error)missed error)

�� How do we minimize both?How do we minimize both?©© Radiological Imaging Technology Radiological Imaging Technology -- 20122012 99

Process Behavior ChartsProcess Behavior Charts

�� Developed by Shewhart in 1930Developed by Shewhart in 1930’’ss

�� Used mostly in manufacturingUsed mostly in manufacturing

�� Recently been adopted into Recently been adopted into radiotherapy and diagnostic radiologyradiotherapy and diagnostic radiology

�� Designed to separate the two types of Designed to separate the two types of variation in any processvariation in any process�� Routine variationRoutine variation

�� Exceptional variationExceptional variation


Clinical ExampleClinical Example

�� Daily Flat/Sym OutputDaily Flat/Sym Output

�� Red dash = SPC limitsRed dash = SPC limits

�� Dotted line = 1Dotted line = 1σσ

�� Clinical specification limits Clinical specification limits 3% (Action 1) and 5% 3% (Action 1) and 5% (Action 2)(Action 2)

�� Limits calculated from first Limits calculated from first 10 measurements10 measurements

�� First First ““signalsignal”” at point 9 in at point 9 in process behavior chartprocess behavior chart


Pawlicki, Whitaker, and Boyer “SPC in Radiotherapy” Med Phys 32(9) (2005)


�� Introduction of Introduction of ““systematic errorssystematic errors””clearly indicated in process chart.clearly indicated in process chart.�� Change of output factor for ion Change of output factor for ion

chamberschambers

�� Change of SSD in setupChange of SSD in setup

�� Process Behavior Chart reProcess Behavior Chart re--evaluated evaluated after systematic error was corrected.after systematic error was corrected.



�� Daily Flat/Sym OutputDaily Flat/Sym Output

�� Limits calculated from Limits calculated from subsequent subsequent measurements (9measurements (9--19)19)

�� First First ““signalsignal”” at point at point 2121

�� Incorrect SSD (setup to Incorrect SSD (setup to buildup not surface of buildup not surface of ion chamber)ion chamber)

�� Corrected for next Corrected for next measurementmeasurement

�� Second Second ““signalsignal”” at at point 32point 32

�� Same mistake (on Same mistake (on therapist part)therapist part)

�� Immediately caught and Immediately caught and corrected (measurement corrected (measurement 33)33)



�� At no point were any measurements At no point were any measurements outside 1outside 1σσ

�� At no point were any measurements At no point were any measurements outside clinical specification action outside clinical specification action limits (3%, 5%)limits (3%, 5%)

�� Systematic changes quickly caughtSystematic changes quickly caught

�� Immediate indication of whether or not Immediate indication of whether or not process is process is ““in controlin control””


Use of Process Behavior ChartsUse of Process Behavior Charts


Assume process of routine QA displays a predictable behavior

Calculate process behavior limits (averages and ranges)

Compare observed average and range values with process

behavior limits

If routine QA readings are within process behavior limits, then process may be stable.

If routine QA readings are outside process behavior limits, then process is definitely unstable.

“Proof” of stable process is continued routine QA readings within process behavior limits.

Take action to identify and remove systematic errors in the process

Continue charting routine QA readings to confirm removal of systematic errors

Practical GuidelinesPractical Guidelines

�� PBC should never supersede clinical PBC should never supersede clinical requirements.requirements.�� i.e. If process is outside clinical i.e. If process is outside clinical

requirements, action should be taken requirements, action should be taken regardless of whether or not process is regardless of whether or not process is ““in in controlcontrol””

�� Limits can be calculated with little data, Limits can be calculated with little data, and revised as more data becomes and revised as more data becomes availableavailable�� Increases the sensitivity of the chartIncreases the sensitivity of the chart

�� If a deliberate change is made to the If a deliberate change is made to the process, limits should be recalculatedprocess, limits should be recalculated



MultiMulti --Source Data ManagerSource Data Manager


Automated Routines For:Automated Routines For:

�� MLC Picket FenceMLC Picket Fence


Automated Routines For:Automated Routines For:

�� Beam Profile Measurements Beam Profile Measurements

�� Star / Spoke ShotStar / Spoke Shot

�� Stereotactic (WinstonStereotactic (Winston--Lutz)Lutz)

�� Asymmetric BeamAsymmetric Beam

�� RadiationRadiation--Light FieldLight Field

Use your EPID


Planar kV ImagingPlanar kV Imaging

PTW Normi4 FLU Leeds TOR18 FG

DISC Plus IBA PrimusL


Planar MV ImagingPlanar MV Imaging

PTW EPID QC QC3

Las Vegas RIT EPID


CBCTCBCT


ISOISO--CubeCube

ReferencesReferences

�� Pawlicki T, Whitaker M, Boyer A. Pawlicki T, Whitaker M, Boyer A. ““Statistical Process Statistical Process Control for radiotherapy quality assuranceControl for radiotherapy quality assurance”” Med Phys 32 Med Phys 32 (2005)(2005)

�� Pawlicki T, Whitaker M. Pawlicki T, Whitaker M. ““Variation and control of process Variation and control of process behaviorbehavior”” Int J Rad Onc Biol Phys 71 (2008)Int J Rad Onc Biol Phys 71 (2008)

�� Donald Wheeler Donald Wheeler ““Understanding variation: The key to Understanding variation: The key to managing chaosmanaging chaos”” 22ndnd Edition, SPC Press (2000) Edition, SPC Press (2000)

�� Able C, Bright M. Able C, Bright M. ““Quality control of external beam Quality control of external beam treatment delivery: mechanical parameterstreatment delivery: mechanical parameters”” Med Phys Med Phys 36 (2009)36 (2009)

�� Able C. Able C. ““Statistical Process Control: A new paradigm in Statistical Process Control: A new paradigm in radiotherapy QAradiotherapy QA”” www.ablemedicalphysics.comwww.ablemedicalphysics.com

�� W A Shewhart. W A Shewhart. ““Economic control of quality of Economic control of quality of manufactured product.manufactured product.”” Van Nostrand (1931)Van Nostrand (1931)


The New (And Improved!) RIT FamilyThe New (And Improved!) RIT Family



AppendixAppendix


WeWe’’ve got you coveredve got you covered

�� Ongoing helpOngoing help

�� New recommendationsNew recommendations

�� This is a process / transitionThis is a process / transition

�� RegularRegular�� UpdatesUpdates

�� Online training sessionsOnline training sessions

�� ClassesClasses

�� Tech SupportTech Support

�� Come see us at booth Come see us at booth #10087#10087

statistical process control for analysis of routine machine...

Documents