new implementation of advanced image guided radiation...

Implementation of Advanced Image GuidedRadiation Therapy

Marc L Kessler, PhD 1

Course OutlineCourse Outline

Image Acquisition

Principles, characteristics & applications

of the available modalities

Image Processing … in the Tx room

Image guided treatment delivery

What can / can’t we do in the room

Image Processing … over the course

Re-planning using structure propagation,

dose mapping and accumulation

Course OutlineCourse Outline

Image Acquisition

Principles, characteristics & applications

of the available modalities

Image Processing … in the Tx room

Image guided treatment delivery

What can / can’t we do in the room

Image Processing … over the course

Re-planning using structure propagation,

dose mapping and accumulation

Jeff SiewerdsenJohns Hopkins University

Jeff SiewerdsenJohns Hopkins University

Jan-Jakob SonkeAVL/Netherlands Cancer Institute

Jan-Jakob SonkeAVL/Netherlands Cancer Institute

Marc L KesslerThe University of Michigan

Marc L KesslerThe University of Michigan

��

��

Disclosure StatementDisclosure Statement

National Center InstituteU.S. National Institutes of Health | www.cancer.gov



I have received slides, images and ideas from many

wonderful people, several of which are in this room!

I have received slides, images and ideas from many

wonderful people, several of which are in this room!

Look here!Look here!

AcknowledgementsAcknowledgements

Adaptive RadiotherapyAdaptive RadiotherapyJaffray / PMHJaffray / PMH

Temporal Scales of InterventionTemporal Scales of Intervention

Real timeReal time Off-lineOff-lineOn-lineOn-line

Adaptive RadiotherapyAdaptive Radiotherapy

DeliveryDeliveryPlanningPlanningImagingImaging

On-lineOn-line

ImagingImagingOff-lineOff-line

downstream , upstream and iterative data flowdownstream , upstream and iterative data flow

… lots of decisions at lots of times… lots of decisions at lots of times

really Off-linereally Off-line




Jeff … taught us how images are acquired and

the pros and cons of the different modalities for

measuring different features of the patient

Jan-Jakob … taught us about image processing

and how we can adapt to certain changes of the

patient … but he ran out of knobs to twiddle

Jeff … taught us how images are acquired and

the pros and cons of the different modalities for

measuring different features of the patient

Jan-Jakob … taught us about image processing

and how we can adapt to certain changes of the

patient … but he ran out of knobs to twiddle

eventuallyeventually^

How Many Knobs?How Many Knobs?

Global

Rigid

Global

Rigid

Fully

Deformable

Fully

Deformable

Local

Rigid

Local

Rigid

Too Few?Too Few? Too Many?Too Many?

3, 6, …3, 6, … 3 x # voxels3 x # voxels

XB = T (XA ,{ ß})XB = T (XA ,{ ß}) XB = T (XA ,{ ß(XA)})XB = T (XA ,{ ß(XA)})


Spatially invariant

Rigid, Similarity, Affine

Spatially variant

Thin-plate splines

B-splines, Cubic splines

Dense deformation fields

Spatially invariant

Rigid, Similarity, Affine

Spatially variant

Thin-plate splines

B-splines, Cubic splines

Dense deformation fieldsmore knobsmore knobs

fewerknobsfewerknobs



6 Knobs / Entire FOV6 Knobs / Entire FOVSonke / NKISonke / NKI

Planning CTPlanning CT CBCTCBCT&&

Pareto

Frontier?

Pareto

Frontier?

6 Knobs / Limited FOV6 Knobs / Limited FOVSonke / NKISonke / NKI

Planning CT & CBCTPlanning CT & CBCT

6 Knobs / Multiple FOV6 Knobs / Multiple FOVSonke / NKISonke / NKI

Planning CT & CBCTPlanning CT & CBCT

Interpolate

between separate

registrations using

Thin-plate splines

Interpolate

between separate

registrations using

Thin-plate splines



Registration using B-splinesRegistration using B-splines

Lot’s of Knobs / Entire FOVLot’s of Knobs / Entire FOV

Planning CTPlanning CT CT from PETCT from PET

Registration FrameworkRegistration Framework

Fixed

image

Floating

image

Geometric

Transformation

Mapped

Image

Mapped

Image

MetricMetric

TransformerTransformerInterpolatorInterpolator

Similarity

OptimizerOptimizer

Adjusted

Parameters

Adjusted

Parameters

10,000 ft View10,000 ft View

Sonke / NKISonke / NKI

Multi-resolution B-SplinesMulti-resolution B-Splines

CoarseCoarse FineFine



Multi-resolution B-SplinesMulti-resolution B-Splines

1.81.8

1.91.9

2.02.0

2.12.1

2.22.2

2.32.3

2.42.4

2.52.5

00 2020 4040 6060 8080 100100 120120 140140 160160 180180

Iteration NumberIteration Number

1.81.8

1.91.9

2.02.0

2.12.1

2.22.2

2.32.3

2.42.4

2.52.5

00 2020 4040 6060 8080 100100 120120 140140 160160 180180

1.81.8

1.91.9

2.02.0

2.12.1

2.22.2

2.32.3

2.42.4

2.52.5

00 2020 4040 6060 8080 100100 120120 140140 160160 180180

Registration Metric vs. IterationRegistration Metric vs. Iteration

Re

gis

tra

tio

n M

etr

icR

eg

istr

ati

on

Me

tric

Change in

knot spacing

Change in

knot spacing

Low ResLow Res

High ResHigh Res

noticebolusnoticebolus




Visual validation looks good!Visual validation looks good!



How many degrees of freedom are needed

to carry out (accurate) deformable image

registration?



registration?

13%

27%

13%

27%

20% 1. 6

2. 12

3. 42

4. 3 x number of voxels

5. More than 12 and less than 3 x # voxels

1. 6

2. 12

3. 42





registration?



registration?

1. 6

2. 12

3. 42



1. 6

2. 12

3. 42




Global

Rigid

Global

Rigid

Fully

Deformable

Fully

Deformable

Local

Rigid

Local

Rigid

Too Few?Too Few? Too Many?Too Many?

3, 6, … 123, 6, … 12 3 x # voxels3 x # voxels

XB = T (XA ,{ ß})XB = T (XA ,{ ß}) XB = T (XA ,{ ß(XA)})XB = T (XA ,{ ß(XA)})




We usually don’t really know

… use more than we need

… depends on the situation

We usually don’t really know

… use more than we need

… depends on the situation

clinicalclinical^


Deformable image registration …

is an over determined and

under constrained problem

Deformable image registration …

is an over determined and

under constrained problem

Under Constrained?Under Constrained?Sonke / NKISonke / NKI




How to Constrain?How to Constrain?Sonke / NKISonke / NKI

Label bones and don’t let them warp!Label bones and don’t let them warp!

Tell the algorithm what is whatTell the algorithm what is what

Registration FrameworkRegistration Framework

Fixed

image

Floating

image

Geometric

Transformation

Mapped

Image

Mapped

Image

MetricMetric

TransformerTransformerInterpolatorInterpolator

Similarity

OptimizerOptimizer

Adjusted

Parameters

Adjusted

Parameters

10,000 ft View10,000 ft View


Add Some Physics!Add Some Physics!

tissue-dependent regularizationtissue-dependent regularization

Etotal = Esimilarity + αααα EstiffnessEtotal = Esimilarity + αααα Estiffness

intensity similarity metricintensity similarity metric

wc(x) |det JT(x) – 1|2 dxwc(x) |det JT(x) – 1|2 dxEvol = Evol =



Add Some Physics!Add Some Physics!

CBCT deformed w/ and w/oCBCT deformed w/ and w/o



ImagingImaging










ImagingImaging







ImagingImaging


… the scanners to the planners to the Tx machine… the scanners to the planners to the Tx machine

Radiotherapy ClassicRadiotherapy Classic


… the scanners to the planners to the Tx machine… the scanners to the planners to the Tx machine


Reuse the

contours

already

defined

Estimate

3D dose

already

delivered

DeliveryPlanningImaging



Image Processing ToolsImage Processing Tools

� Deformable image registration

Compute the transformation that relates the

coordinates of objects in two imaging studies

� Contour and dose mapping

Map information such as anatomic contours

and doses from one image study to another

� Deformable image registration

Compute the transformation that relates the

coordinates of objects in two imaging studies

� Contour and dose mapping

Map information such as anatomic contours

and doses from one image study to another

Data PropagationData Propagation

� Outlines

contours

… data just at boundaries

� Voxel data

dose & image values

… data at every point

� Outlines

contours

… data just at boundaries

� Voxel data

dose & image values

… data at every point

Deformable RegistrationDeformable RegistrationDong / MDACCDong / MDACC

Registration using “Demons”Registration using “Demons”

Delivery

CT

Delivery

CT

Planning

CT

Planning

CT



Deformable RegistrationDeformable RegistrationDong / MDACCDong / MDACC

Registration using “Demons”Registration using “Demons”

Delivery

CT

Delivery

CT

Planning

CT

Planning

CT

Contour PropagationContour PropagationDong / MDACCDong / MDACC

“Cut and Paste” Structures“Cut and Paste” Structures

Delivery

CT

Delivery

CT

Planning

CT

Planning

CT



Delivery

CT

Delivery

CT

Planning

CT

Planning

CT





Delivery

CT

Delivery

CT

Planning

CT

Planning

CT


Apply deformation to StructuresApply deformation to Structures

Delivery

CT

Delivery

CT

Planning

CT

Planning

CT


Deform StucturesDeform Stuctures

Planning

CT

Planning

CT

Delivery

CT

Delivery

CT



Contour PropagationContour PropagationCirca 1985Circa 1985

Drawn ContoursDrawn Contours Derive the ContourDerive the Contour

?


Drawn ContoursDrawn Contours Derived ContoursDerived Contours


aa bb

Stack MR

Outlines

Stack MR

Outlines

Create a

3D Model

Create a

3D Model

cc

Transform

and Slice

Transform

and Slice

dd

Apply Outlines

to CT

Apply Outlines

to CT

Structure Transfer Between Sets of Three Dimensional Medical Imaging Data, G.T.Y. Chen, M. Kessler, and

S. Pitluck, Proceedings of the National Computer Graphics Association, Dallas, TX, vol III, pp 171-77 (1985)

Structure Transfer Between Sets of Three Dimensional Medical Imaging Data, G.T.Y. Chen, M. Kessler, and

S. Pitluck, Proceedings of the National Computer Graphics Association, Dallas, TX, vol III, pp 171-77 (1985)



Orientation MattersOrientation Matters

Axial

“Target”

Axial

“Target”

Coronal

“Target”

Coronal

“Target”

Differences due to sampling plane and voxel shape!Differences due to sampling plane and voxel shape!

Jeff … we want small isotropic voxels!Jeff … we want small isotropic voxels!

Contours defined on one DICOM image

study can be transferred to another

DICOM image study by:




1. using “cut and paste”

2. registering the two imaging studies then mapping the

structures using the resulting geometric transformation

3. transferring the contours and images to a DICOM server

using IHE-RO and then reading them back in

4. scaling the contours by the ratio of the different pixel sizes

5. It is not possible to transfer contours between imaging

studies








studies














studies








studies



Dose MappingDose MappingRosu / UMRosu / UM

Volume AVolume A Volume BVolume B

Dose MappingDose Mapping

C’C

Volume of a voxel changesVolume of a voxel changes

Volume AVolume A Volume BVolume B

Rosu / UMRosu / UM

Dose MappingDose MappingDong / MDACCDong / MDACC

Apply deformation to dose gridApply deformation to dose grid

Delivery

CT

Delivery

CT

Planning

CT

Planning

CT



Dose AlgebraDose Algebra

CBCT day nCBCT day n

Pouliot / UCSFPouliot / UCSF

CBCT day mCBCT day m

Dose AlgebraDose AlgebraPouliot / UCSFPouliot / UCSF

Dose Difference (%)

>5%

>10%

� Change in shape

� Increased cord dose

� Change in shape

� Increased cord dose

Subtraction of two mapped dosesSubtraction of two mapped doses


Dealing with volume elements that may:Dealing with volume elements that may:

change shape / appear / disappear

… need proper spatial re-sampling

don’t necessarily add in a linear fashion

… need some sort of radiobiology?

exist in homogenous intensity regions

… hard to evaluate registration










Dealing with volume elements that may:Dealing with volume elements that may:







The major difference in the process of

transferring doses and contours between

two DICOM studies



two DICOM studies

1. Doses depend on tissue density and contours do not

2. Doses do not change once they are delivered, contours do

3. Transferring doses is more time consuming than

transferring contours

4. Transferring doses requires registration at every voxel,

transferring contours requires this only at boundaries

5. Transferring doses requires knowledge of the alpha-beta

ratio and transferring contour does not











two DICOM studies



two DICOM studies



















Final ThoughtsFinal Thoughts

Final ThoughtsFinal Thoughts

� Need sufficient information in the on-line

loop to indicate the need for off-line loop

� Off-line loop may require additional and

different information

� Efficiency and circumstances will drive the

relative use of the loops

� New technology in image acquisition and

processing may help to merge the loops

� Need sufficient information in the on-line

loop to indicate the need for off-line loop

� Off-line loop may require additional and

different information

� Efficiency and circumstances will drive the

relative use of the loops

� New technology in image acquisition and

processing may help to merge the loops

Thank you for

your attention!

Thank you for

your attention!



Visualization of DeformationsVisualization of Deformations

Overlay of

Arrow glyphs

Overlay of

Arrow glyphs

Color overlay

of magnitude

Color overlay

of magnitude

Deformation of

uniform grid

Deformation of

uniform grid



Vector Field Critical PointsVector Field Critical Points

AttractorAttractor RepellorRepellor

Rotation CenterRotation Center

Saddle Point (I)Saddle Point (I) Saddle Point (II)Saddle Point (II)

Tittlemeyer, et al.Tittlemeyer, et al.

Optical Flow AlgorithmOptical Flow Algorithm

Image AImage A Image BImage B

Foskey/UNCFoskey/UNC

Optical Flow AlgorithmOptical Flow Algorithm

Image AImage A Image BImage B

Foskey/UNCFoskey/UNC



Optical Flow AlgorithmOptical Flow AlgorithmFoskey/UNCFoskey/UNC





ResultsResults

No masking

Note ribs (and tissue ) near diaphragm

No masking

Note ribs (and tissue ) near diaphragm

Mask using segmented lung/abdomen

Ribs not affected by lung registration

Mask using segmented lung/abdomen

Ribs not affected by lung registration

Use of maskingUse of masking



Data MappingData Mapping

Data MappingData Mapping





Build this crazyBuild this crazyBuild this crazyBuild this crazy

contraption contraption contraption contraption

piecepiecepiecepiece----bybybyby----piece as you race piece as you race piece as you race piece as you race

around the track!around the track!around the track!around the track!

Build this crazyBuild this crazyBuild this crazyBuild this crazy

contraption contraption contraption contraption

piecepiecepiecepiece----bybybyby----piece as you race piece as you race piece as you race piece as you race

around the track!around the track!around the track!around the track!

Don’t Get caughtDon’t Get caughtDon’t Get caughtDon’t Get caughtDon’t Get caughtDon’t Get caughtDon’t Get caughtDon’t Get caught


It’s fun It’s fun It’s fun It’s fun

to build this comical wonder but woe to build this comical wonder but woe to build this comical wonder but woe to build this comical wonder but woe

to the mouse thatto the mouse thatto the mouse thatto the mouse that

gets caughtgets caughtgets caughtgets caught

under!under!under!under!

It’s fun It’s fun It’s fun It’s fun

to build this comical wonder but woe to build this comical wonder but woe to build this comical wonder but woe to build this comical wonder but woe

to the mouse thatto the mouse thatto the mouse thatto the mouse that

gets caughtgets caughtgets caughtgets caught

under!under!under!under!

Don’t Get caughtDon’t Get caughtDon’t Get caughtDon’t Get caughtDon’t Get caughtDon’t Get caughtDon’t Get caughtDon’t Get caught

new implementation of advanced image guided radiation...

Documents