dir qa options and research developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · dir qa...
TRANSCRIPT
![Page 1: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/1.jpg)
DIR QA Options and Research Development
Neil Kirby, Ph.D. Assistant Professor Radiation Oncology
UTHSCSA
![Page 2: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/2.jpg)
Acknowledgments UCSF Jean Pouliot Josephine Chen Hojin Kim Kamal Singhrao Cynthia Chuang Olivier Morin
UTHSCSA Daniel Saenz Kristen Cline Mohammad Obeidat Sotirios Stathakis
UF Health Cancer Center – Orlando Health Jason Pukala
Yale Sarah Geneser
Rutgers Ke Nie
![Page 3: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/3.jpg)
Overview • Phantom DIR QA.
• Lessons learned from phantoms.
• Patient-specific QA.
![Page 4: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/4.jpg)
Types of DIR QA Patient
• Limited knowledge
of deformation. • DIR errors matter.
Phantom
• Know the deformation.
• Not a real case.
![Page 5: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/5.jpg)
Types of Phantoms Virtual Physical
K. Nie et al. Med Phys 40, 041911 (2013).
![Page 6: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/6.jpg)
Physical Phantoms Pros • Are a full end-to-end QA of
imaging and DIR. Cons • Requires specialized
fabrication. • May not fully represent a
patient
![Page 7: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/7.jpg)
How realistic should physical phantoms be?
• D. Saenz et al. “Quality Assurance of Deformable Image Registration Algorithms: How Realistic Should Phantoms Be?”,
SU-E-J-97, Exhibit Hall .
![Page 8: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/8.jpg)
Virtual Phantoms
Pros • Closely resemble
patients. • Can be created digitally. • Commercial platforms
(ImSimQA from OSL). Cons • Not an end-to-end test.
![Page 9: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/9.jpg)
Deformable Image Registration Evaluation Project (DIREP)
J. Pukala et al. Med Phys 40, 111703 (2013).
• Virtual phantoms are also available online: https://sites.google.com/site/dirphantoms/
![Page 10: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/10.jpg)
How realistic should virtual phantoms be?
• Image noise can leave a “fingerprint” of the deformation.
![Page 11: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/11.jpg)
How realistic should virtual phantoms be?
MIM Mean Error [mm]
Velocity Mean Error
[mm]
Realistic Noise
Scenario 1.24 1.68
No Processing 0.78 1.66
• This fingerprint affects some algorithms more than others.
![Page 12: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/12.jpg)
Virtual phantoms to represent daily imaging
• K. Cline et al. “Comparative Analysis of MIM and Velocity's Image Deformation Algorithm Using Simulated KV-CBCT Images for Quality Assurance
SU-E-J-89, Exhibit Hall
CT simulated CBCT CBCT
![Page 13: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/13.jpg)
How realistic should virtual phantoms be?
Virtual phantoms must include:
• 1. Realistic noise scenarios.
• 2. Realistic deformations – missing tissue, divergences, and discontinuities in the deformation field.
![Page 14: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/14.jpg)
Virtual phantoms can be
20%
20%
20%
20%
20% 1. downloaded from online. 2. created with vendor software. 3. useful for DIR QA. 4. all of the above. 5. none of the above.
10
![Page 15: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/15.jpg)
Some Basic Types of DIR QA
• 1. Contour mapping.
• 2. Landmark Tracking.
• 3. Image similarity.
• 4. Comparison of predictions to known deformation (for phantoms).
Known
Prediction
Error
![Page 16: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/16.jpg)
• Comparison of transferred contours to that drawn on the other image.
Contour Mapping
![Page 17: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/17.jpg)
• Landmarks are iden8fied on the CT images and tracked to test deforma8on predic8ons.
• Any prominent landmark will also stand out to the algorithm.
Measurement
Prediction
Error
Landmark Tracking
![Page 18: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/18.jpg)
• A mathema8cal metric is used to evaluate the similarity between the warped image and the fixed image.
• A few examples -‐ sum of squared difference, mean absolute difference (MAD), cross correla8on, and mutual informa8on.
Image Similarity
M 1st W 2nd W 3rd W F
MAD =1N
Wi −Fii=1
N∑
Image Similarity
![Page 19: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/19.jpg)
• Penalty Func8on = Similarity + λ�Regulariza8on • An op8mal balance between the penalty terms must be found.
Large λ Medium λ Small λ
Balance of Penalty Terms
![Page 20: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/20.jpg)
• Pelvic phantom to represent deforma8ons from bladder filling. • Realis8c soV-‐8ssue varia8ons.
Patient CTs
Phantom CTs
UCSF Deformable Pelvic Phantom
![Page 21: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/21.jpg)
• MIM tested with a variable regulariza8on factor (20 values). • Overall spa8al error (relevant for dose mapping) op8mized with a different factor than contour transfer.
More Image Similarity
16 18 20 22 241.5
2
2.5
3
3.5
4
4.5
Erro
r % >
7 m
m
Mean Absolute Difference (HU)16 18 20 22 245
10
15
Erro
r % >
3 m
m
Mean Absolute Difference (HU)
16 18 20 22 240.6
0.7
0.8
0.9
1
Dic
e C
oeffi
cien
tMean Absolute Difference (HU)
Variable Regularization Factor
![Page 22: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/22.jpg)
Comparison of QA Methods on Virtual Phantoms
• Similar results are found with virtual phantoms.
• M. Obeidat et al. “Comparison of Different QA Methods for Deformable Image Registration to the Known Errors for Prostate and Head-And-Neck Virtual Phantoms”, SU-E-J-117, Exhibit Hall
Ground Truth
![Page 23: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/23.jpg)
Lessons learned from phantoms
1. Contour mapping.
2. Landmark Tracking.
3. Image similarity.
= Actual Accuracy.
![Page 24: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/24.jpg)
• A strong focus on image similarity tends to increase accuracy in regions of contrast, but decrease accuracy in homogenous regions.
• The 3 metrics are dominated by regions of contrast.
Fat Muscle
Reason for discrepancy
![Page 25: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/25.jpg)
Lessons from phantoms
1. Contour mapping.
2. Landmark Tracking.
3. Image similarity.
= Actual Accuracy.
Available for Patients
![Page 26: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/26.jpg)
• For some applica8ons, these metrics are adequate for DIR QA.
• For contour transfer, you simply need to visually inspect these contours.
DIR QA for a patient.
![Page 27: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/27.jpg)
• For other applica8ons, such as dose or standard uptake values, these metrics are not enough.
• These are deposited in both high and low contrast regions.
DIR QA for a patient.
![Page 28: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/28.jpg)
• Evalua8ng DIR spa8al accuracy must be broken into two parts: • 1. Accuracy assessment in regions of contrast. • 2. Evalua8on of how physical the deforma8on field is away from regions of contrast (Jacobian, vector field, deforma8on grid)
DIR QA for a patient.
![Page 29: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/29.jpg)
• You can es8mate the spa8al accuracy from these two terms and mul8ply by local dose gradient. 2.5 Gy/mm x 1.3 mm = 3.25 Gy • You should s8ll assume at least a 1 mm error.
• B. Tewfik et al. “Submillimeter accuracy in radiosurgery is not possible”, Med Phys, 40, 050601 (2013)
Quantitative DIR QA for a patient.
![Page 30: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/30.jpg)
• Manual techniques for quan8ta8ve DIR accuracy assessment are not incredibly prac8cal.
• Automated soVware to create DIR mapping uncertainty is the next step for DIR QA.
Automated DIR QA for a patient.
![Page 31: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/31.jpg)
Automated DIR QA for a patient.
• H. Kim et al. “Validating Dose Uncertainty Estimates Produced by AUTODIRECT, An Automated Program to Evaluate Deformable Image Registration Accuracy”, SU-E-J-92, Exhibit Hall
![Page 32: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/32.jpg)
Overall spatial accuracy of a DIR algorithm can be directly evaluated by its ability to
20%
20%
20%
20%
20% 1. transfer anatomical contours. 2. deform landmarks. 3. produce close image similarity. 4. all of the above. 5. none of the above.
10
![Page 33: DIR QA Options and Research Developmentamos3.aapm.org/abstracts/pdf/99-28330-359478... · DIR QA Options and Research Development Neil Kirby, Ph.D. Assistant Professor Radiation Oncology](https://reader034.vdocument.in/reader034/viewer/2022042212/5eb60fc6e17b794558536660/html5/thumbnails/33.jpg)
Conclusions • How realistic a phantom (virtual or physical)
is will affect DIR accuracy.
• DIR effectiveness for contour mapping, landmark tracking, and image similarity is not the same as overall spatial accuracy.
• Patient QA should assess both image similarity and how physical a deformation is.