IMAGE REGISTRATION IN VETERINARY RADIATION ONCOLOGY: INDICATIONS,

IMPLICATIONS AND FUTURE ADVANCES

Yang Feng1, 2, Jessica Lawrence2, Kun Cheng1, Dean Montgomery1, Lisa Forrest3, Duncan B.

McLaren1, Stephen McLaughlin4, David J. Argyle2, William H. Nailon1

1The University of Edinburgh, Department of Oncology Physics, Edinburgh Cancer Centre,

Western General Hospital, Edinburgh, UK. 2The University of Edinburgh, Royal (Dick) School

of Veterinary Studies and Roslin Institute, UK. 3The University of Wisconsin-Madison,

Department of Surgical Sciences, 2015 Linden Drive, Madison WI, USA. 4Heriot-Watt

University, School of Engineering and Physical Sciences, Edinburgh, EH14 4AS, UK.

Keywords: Oncology, Image Registration, Computed tomography, Radiation therapy, Magnetic

resonance imaging

Running Head: Image Registration in Radiation Oncology

Funding Sources: This work was generously supported by NHS Lothian, Edinburgh and

Lothians Health Foundation (charity number SC007342), the James Clerk Maxwell Foundation,

the Jamie King Uro-Oncology Endowment Fund, a University of Edinburgh Darwin Award and

a University of Edinburgh Individual Stipend.

Previous Abstracts: Material has not previously been presented.

1

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

123456789

Abstract

The field of veterinary radiation therapy (RT) has gained substantial momentum in recent

decades with significant advances in conformal treatment planning, image-guided radiation

therapy (IGRT) and intensity-modulated (IMRT) techniques. At the root of these advancements

lie improvements in tumor imaging, image alignment (registration), target volume delineation,

and identification of critical structures. Image registration has been widely used to combine

information from multi-modality images such as computerized tomography (CT), magnetic

resonance imaging (MRI) and positron emission tomography (PET) to improve the accuracy of

radiation delivery and reliably identify tumor-bearing areas. Many different techniques have

been applied in image registration. This review provides an overview of medical image

registration in RT and its applications in veterinary oncology. A summary of the most commonly

used approaches in human and veterinary medicine is presented along with their current use in

IGRT and adaptive radiation therapy (ART). It is important to realize that registration does not

guarantee that target volumes, such as the gross tumor volume (GTV), are correctly identified on

the image being registered, as limitations unique to registration algorithms exist. Research

involving nNovel registration-frameworks for automatic segmentation of tumor volumes is

ongoing and comparative oncology programs offer a unique opportunity to test the efficacy of

proposed algorithms.

2

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

101112131415161718

Introduction

Imaging data from multiple anatomical and functional imaging studies is becoming a routine

component of veterinary patient management for a variety of medical and surgical conditions.

Spanning from initial diagnosis to determining therapeutic options to assessing response or

recrudescence of disease, these data help direct decisions regarding disease management,

efficacy of treatment and patient outcome. While computed tomography (CT) and magnetic

resonance (MR) imaging are readily available in veterinary medicine, novel imaging techniques

such as molecular imaging offer complementary information to aid in disease recognition, extent

and treatment planning. In order to optimally use information gathered from these various

techniques, the data must be easily compared despite differences in image acquisition and

presentation. Image registration is defined as the process of aligning two or more images from

the same or different imaging modalities to allow for data mapping (data fusion) and

interpretation. While registration carries importance across imaging disciplines, the purpose of

this overview review is to provide a broad overview of image registration and data fusion

techniques used in radiation therapy (RT), given the high frequency with which it is utilized and

radiation oncologists’ reliance on registered images. While RT is the focus of the review, the

principles outlined for registration techniques can be broadly applied to non-oncologic

applications.

It is estimated that greater than 60% of human patients with cancer will receive RT and

although there is no similar strong data in dogs and cats, RT has become a common therapeutic

3

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

192021222324252627

modality in veterinary oncology.1 There is a large body of evidence supporting the use of RT for

optimal local tumor control, therefore identification of the tumor volume is of utmost importance

when determining a treatment plan or evaluating changes in tumor volume. In practice, medical

images acquired from different imaging modalities are used to guide the entire RT process from

the initial treatment plan to fractionated radiation delivery through to dose verification. While

radiation oncologists have always been guided by images in some form, the advent of image-

guided radiation therapy (IGRT) has revolutionized how integral images are to modern radiation

oncology. In a broad sense, IGRT may reflect any aspect of RT that utilizes imaging to improve

treatment, such as weekly or twice weekly portal imaging. In a stricter sense, IGRT refers to

contemporaneous functional and structural imaging to improve target delineation, adjust for

target motion and/or uncertainties in patient positioning, and potentially adapt treatment to the

response of the tumor during adaptive radiation therapy (ART).2 Often without recognition of the

process involved, radiationWhile radiation oncologists are highly dependent on image

registration to ensure that IGRT is successful atin targeting tumor and limiting dose to adjacent

normal tissue, the underlying process is complex and often overlooked. A typical IGRT process,

using veterinary images as an example, where image registration is vital in order to merge

information from multi-modality images and therefore provide an accurate guide for radiation

delivery is illustrated in Fig. 1.

Image registration provides a geometric transform that makes it possible to map

information between the images, often with sub-pixel resolution. As the use of multimodality

4

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

282930313233343536

image data in RT increases, medical image registration is essential to combine the information

from each modality. As a result, it has become a very active area of research.3 In RT, Iimages

utilized in RT can be registered to obtain comprehensive information whether theyregardless of

patient positioning, time point with respect to therapy, or type of imaging acquisition. were

obtained on patients in dissimilar positions or immobilization devices, from different points of

view, at various time points, or by different sensors. Registration may also be used to combine

multiple images from the same imaging modality,4 or to combine information from multiple

modalities such as CT, MR imaging, positron emission tomography (PET) and single-photon

emission computed tomography (SPECT).5 For the purpose of this review, athe reference image

is defined as the source image containing the reference information, while athe target image is

defined as the movable image spatially matched to the reference. After registration, information

from the reference image, for examplesuch as contoured structures, can be used on the target

image.

Image registration methods have been widely used both in human medicine6-11 and in

preclinical studies12-18 to improvefor matching between images acquired on different modalities

such as PET and-MR,19 SPECT and-CT,20 and PET and-CT.21 Registration approaches applied to

humans are also suitable for companion animals.22-31 and may prove useful in comparative

oncology, which refers to the study of cancer etiology, biology and treatment in companion

animals that develop spontaneously-occurring cancer.1, 32-34 Comparative oncology approaches

presents a unique opportunity to improve RT for both human and veterinary patient groups given

5

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

373839404142434445

the number of similarities in tumor biology, imaging techniques and therapeutic

managementmodalities. Image registration approachesmethodologies, regardless of the species

imaged, will help facilitate this by finding the optimal geometric transformation between

corresponding image data.35

In general, an image registration algorithm can be divided into four parts: image

conditioning, geometric transformation, similarity function and optimization (Fig. 2). Any step

that alters the original data to make it more suitable for applying image registration is known as

image conditioning. Due to the diversity of the methods used for this, a comprehensive review of

image conditioning falls outside the scope of this manuscript, but may be found elsewhere.5, 36-37

Image Registration

Table 15 provides a comprehensive list of the most commonly used terminology in image

registration.5 A summary of the kKey technical details of the transformation function, similarity

function and optimization process (Fig. 2) are introduced in this section.

Transformation

As it is common for the patient orientation and immobilization to vary between imaging studies,

the first and keyessential component of registration is transformation. This describes the

geometrical shift that is required for the target image to match it to the reference image. Rigid

transformation methods are most commonly used and are usually applied on images that have no

6

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

464748495051525354

distortion, such as CT to CT. Generally, a rigid geometric transformation can be achieved by

translation and rotation and is commonly used to match between bones on medical images, such

as the skull.38-39 An affine transformation, which is an extension of the rigid transformation,

allows for rotations, translations, scaling and shearing.40 Rigid transformations can also be

applied before a non-rigid registration as an image conditioning step.41-42

The assumption that rigid movement of anatomy occurs globally is incorrect in any

number of situations, therefore limiting the widespread use of rigid registrations for sites other

than the head.40 Most non-rigid registration methods are based on deformable models. These

range in complexity and may be veryvary from simple with relatively few parameters toor more

complicated where each point or voxel moves independently.40 There are two directions in

deformable registration: free form deformable registration (often abbreviated FFD) and guided

deformable registration, which are controlled by models based on prior knowledge of the

registered objects or organs. The fundamental difference between them is that the free form

deformable registration allows any deformations by moving the positions of its control grid. The

most commonly used guided deformable registration methods are elastic-based and flow-based.

Elastic-based methods treat organs as elastic solids and define two forces: internal forces that

oppose the deformation and external forces that try to deform the images. The best transform can

be obtained by finding equilibrium between both internal and external forces. Flow-based

methods such as include fluid flow and optical flow . These methods treat the registration

problem as a motion problem and therefore achieves the best match is achieved by meeting a

7

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

555657585960616263

pre-set constraint in a physical model. Fig. 3 shows tThe application of the popular B-spline

based FFD method registering pre- and post-treatment CT images of a dog with a sinonasal

tumor is demonstrated in Fig 3.43 A similar method called thin-plate spline (often abbreviated

TPS) can also generate FFD registration. However, compared to the B-spine method this

approach is limited by the fact that the deformation is applied globally. 44

Similarity Function

The transformation, or alignment, of the image datasets is assessed by measuring the similarity

between them.40 Generally, tThere are two predominant kinds of similarity measurements:

intensity-based and feature-based measurements. The most popular intensity-based similarity

function is based on intensity difference or intensity correlation. Feature-based similarity

functions depend on the feature structure extracted, such as anatomic structures (bones) or

artificial landmarks (fiducials). Most feature-based algorithms use points, lines, or surfaces for

matching.40, 45-46 A minimum of three or four pairs of points are required in order to compute the

rotations and translations for a rigid or affine transformation.40 Unlike point matching, line and

surface matching do not require a one-to-one match between images but rather attempt to

maximize the overlap between equivalent lines and surfaces such as the skull surface, ribs or

pelvis.40, 45-46 Fig. 4 illustrates tThe transformation process and the corresponding similarity

calculation used in the registration of images from a dog with a nasal tumor is illustrated in Fig

4.

8

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

646566676869707172

Optimization

Optimization is the process used to search for a numerical value produced by the similarity

function, often the minimum value, which is indicative of when the best registration, or match,

between the images has been found. The goal of the optimization algorithm is to find a

maximum or minimum value of the similarity measure accepted; it is common for optimal

registration to be accomplished when the similarity measures are defined by their minimal value.

It is an important step in a registration method because the similarity function can often produce

several local minima, which will trap the search and giveproduces sub-optimal results.

Assessment

The obvious motivation for registering images from different studies is to map clinically useful

information from one study onto another, for example in treatment planning prior to radiation

delivery. If data can be successfully fused, radiation oncologists can then map tumor volumes

such as the GTV, clinical target volume (CTV) and planning target volume (PTV) or organs at

risk (OAR) directly onto the CT needed for dose calculations (structure mapping).40 Therefore,

relying on the ability of a bespoke software system to adequately register images is of utmost

importance in order to adhere to current radiation guidelines – namely to appropriately outline

tumor volumes andto limit normal tissue toxicity. In this vain, it is important to realize the

difficulties in accurately validating the performance of a complex registration method. In some

9

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

737475767778798081

cases, a registration algorithm can be assessed using phantom data,47 but this is not commonly

done as performance on phantoms cannot ensure comparable performance on clinical cases. A

nother commonly used approach is to use synthetic images on which manual definition of

corresponding points such as fiducial markers can provides a straightforward assessment of a

registration method.48-49 The Dice coefficient, which measures the overlap between clinical and

registered contours, placesuts a numerical value on the registration accuracy.50-51 Within the

range 0% to 100%, a Dice value of 100% indicates excellent geometric agreement while a Dice

value of 0% represents poor geometric agreement (Fig. 5). The Jaccard index and Tanimoto

coefficient (both vary from 0% to 100%), which can be calculated directly from the Dice

coefficient, are also used to measure the similarity between regions. However, they need prior

knowledge (contours of the same region) on both reference and target images.

Once the registration process has been completed, there are a number of techniques that

can be used to visualize fused data, including the use of overlays, pseudo-coloring, grey scale

coloring, and side-by-side display of anatomic planes.40 This not only allows the radiation

oncologist to evaluate the fused images but can also be used forand to map ping of the calculated

3D dose distributions fromon the CT treatment plan to the coordinate system of another imaging

study, such as MR. Not only is this helpful to define the tumor and normal tissue dose for tissues

that are not optimally imaged with CT (such as canine or feline intracranial tumors), but the

fusion of 3D dose distributions to post-treatment imaging scans to improve detection and

understanding of radiation changes over time (Fig. 5). As veterinary radiation oncology advances

10

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

828384858687888990

into the realm of IGRT and ART with the use of volumetric imaging systems (on-board

kilovoltage CT or megavoltage CT units), the concept of accurate registration becomes

increasingly criticalmore important to provide input into the ART process.

Current State of Image Registration in Veterinary Radiation Oncology

Image registration and data fusion are useful in veterinary radiation oncology with most

modern treatment planning systems now facilitating the use of a secondary datasets for target and

OAR delineation. In addition iIt is now also possible to map dose information to the secondary

dataset following an appropriate registration.

Dose distributions generated for conformal radiation therapy (CRT) have been accurately

predicted with CT-based treatment planning systems and are already widely used in veterinary

oncology.53 CT data includes the Hounsfield units as a linear transformation of radiation beam

attenuation that varies with electron densities of materials as the beam progresses through tissue.

Most RT planning software obtains the relative electron density from the relationship between

the linear attenuation coefficients and Hounsfield unit values in order to determine heterogeneity

in the dog or cat’s body. Non-contrast CT images are typically utilized for treatment planning,

given that contrast agents are high-Z radio-opaque materials, which would attenuate a radiation

beam more than normal, resulting in higher than normal electron density. Post-contrast CT data

used for a treatment plan would theoretically give rise to higher monitor unit (MU) values, and

therefore radiation dose, compared to MU values taken from calculating a plan using pre-contrast

11

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

919293949596979899

CT data. As veterinary radiation oncologists often use commercially available human treatment

planning software, automatic registration methods may be built-in to the software, although other

“in house” registration algorithms may be adapted for use.22, 24-29 One of the most common

automatic registrations used in routine treatment planning is the merging of pre-contrast and

post-contrast CT data in order to delineate the GTV, CTV, PTV and OAR. Because regions of

interest are often best identified on post contrast CT images, many commercial software

programs (such as Eclipse™, Varian Medical Systems, Palo Alto, used at the authors’

institutions) automatically permit DICOM-coordinated registration and facilitate contouring on

fused multimodality images. However, there are often limited registration capabilities between

multimodal image acquisitions; our current planning system is capable of a fully automatic,

mutual-information-based rigid registration but results are often unsatisfactory between

modalities (i.e. CTs obtained with different DICOM origins or MRI and CT data acquired with

vastly in different animal positionings). This eliminatesAutomatic registration partially

eliminates a potential source of error as radiation oncologists historically may have estimated

tumor volume on the pre-contrast CT data based on visual examination of alternative imaging

studies. Upon eEvaluationng of the veterinary literature to date, it is often difficult to firmly

understand how tumor volumes were identified prior to treatment; for example in one study

evaluating RT for canine intracranial tumors, the GTV was defined on planning CT scans by the

contrast-enhancing area on CT or MR data.54 It is presumed that contours for the GTV were

drawn on pre-contrast CT images after evaluation of diagnostic scans, however this presumption

12

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

100101102103104105106107108

may be incorrect. In most other veterinary studies, it is unclear how the GTV was derived, as

irradiated volumes have been inconsistently defined.52 The delineation of target volumes is

paramount to effective RT, as geographic miss could occur if the GTV, CTV and PTV are

ineffectually contoured and/or unexpected toxicity could result from a poorly defined OAR.

Proper radiation reporting is also critical to permit reproducibility of clinical studies across

institutions.52 It is also important to note that dose distributions generated by treatment planning

are estimated dose distributions, thereby introducing an additional uncertainty to planning. The

variation in registration methods introduces bias that could lead to erroneous conclusions about

the extent and location of various volumes compounded by variations in dose distribution

algorithms. As veterinary reporting of RT planning improves, information regarding registration

should be included.

In addition to RT planning, image registration has made its way into the routine

assessment of daily patient positioning, enabling better tumor targeting.25-29, 55-57 One of the key

concepts of IMRT, stereotactic radiation therapy (SRT) or radio surgery (SRS) and IGRT

planning is that tight dose constraints can beare placed around the PTV provided the clinician is

confident that the patient’s animal’s positioning will be reproducible during treatment. One of

the most common practices immediately prior to treatment delivery is to generate a pair of

orthogonal digitally reconstructed radiographs (DRRs) from the planning CT and to register

simulated radiographs with MV radiographs acquired by a flat-panel imager (electronic portal

imaging devices or EPIDs) attached to the linear accelerator (Fig. 65).55, -56 More advanced on-

13

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

109110111112113114115116117

board imaging equipment involves cone-beam CT systems or helical CT systems built into the

treatment unit.25-29, 57 Automated image registration at the treatment unit can determine the

rotations and translations required to align the datasets, thus decreasing some of the variability

with daily patient setup and intra-patient shifts.25-29, 40 Challenges still exist with registration of

different modalities with different positioning and therefore fiducials and other landmarks may

be needed to improve alignment (Fig. 76). Dose delivered to patients throughout a treatment

protocol also rely on accurate patient positioning and registration (i.e. daily cone beam CT

registered with treatment-planning CT) in order to sum dose delivered over time; monitoring of

dose deviations from the expected distribution allows for intra-treatment adjustments in order to

better target tumor yet limit normal tissue dose. Little veterinary literature has evaluated the

utility or validity of various registration algorithms other than an early study demonstrating an

in-house computer program that was developed for brain imaging.22 This study highlighted the

widespread potential of image registration by evaluating corresponding images from CT and

MR, CT images before and after surgical treatment, and CT and post-mortem cryosection images

in dogs or cats.22 Currently at our institution, for follow-up imaging, standardized response

criteria are applied when assessing tumor response following treatment.58 Responses for tumors

of the head and neck (intracranial, sino-nasal, cervical) are often assessed by the radiology and

radiation oncology team on serial CT scans, without consideration for image registration over

time, which may not accurately reflect the response of the tumor or the assessment of treatment-

induced toxicity.27 Often tToxicities are often inferred by tissue changes, however easier

14

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

118119120121122123124125126

implementation of image registration in the clinic may provide more objective information

regarding tumor and normal tissue changes following radiation, particularly as varying

fractionated protocols are used across veterinary radiation oncologists.52,5 3,59,60

With the incorporation of functional imaging into veterinary oncology, determination of

structural and, indirectly, pathological changes may be better assessed. Changes such as fibrosis

and tumor recurrence can be difficult to distinguish on CT and MR; functional imaging with PET

may help clarify differences and alter future therapy. Currently in veterinary medicine, 2-deoxy-

2-[18F]-fluoro-deoxyglucose (FDG), 3’-deoxy-3’-[18F]-fluorothymidine (FLT), and copper(II)-

diacetyl-bis (N4-methylthiosemicarbazone) (Cu-ATSM) have been used to assess various

neoplastic conditions in dogs and cats.61 59-70 68 In particular, several studies have evaluated

functional imaging over time in an attempt to correlate pre-treatment scans to post-treatment

scans; ideal comparisons would require registration of image datasets to avoid interpretation

errors.67 5-70 68 One study that assessed pre-radiation therapy FDG, FLT and Cu-ATSM PET/CT

scans to post-treatment FDG PET/CT scans demonstrated the complexity and utility of

registration methods in dogs with sinonasal tumors.70 68 Dogs in this study underwent a radiation-

planning CT scan that served as the reference dataset; bony anatomy from pre-radiation PET/CT

scans was registered to the bony anatomy of the radiation-planning CT; rigid registration using

cross-correlation resulted in affine transformations that were applied to the corresponding PET

images. The translated and rotated PET matrices were subsequently resampled using a spline

filter defined by the reference PET image that was obtained at the same time as the treatment-15

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

127128129130131132133134135

planning CT.70 68 While complex, the registration process ensured that each voxel index of the

pre-treatment PET/CT scans corresponded as closely as possible to the same region in the post-

treatment FDG PET image for each patient, enabling assessment of changes in tracer uptake.70 68

Imaging and registration advances will undoubtedly lead to more work being performed in a

similar fashion and may alter current criteria for response assessment and/or response prediction.

Proposed Advances in Veterinary Image Registration

In general, RT is delivered to patients over the course of several weeks in both human and

veterinary oncology. While the GTV, CTV and PTV are currently considered standard targets in

radiation oncology, the PTV is created to make sure that dose plans are robust to

uncertainties.71 69 The GTV and CTV are oncological-anatomical concepts, which are heavily

dependent on the experience and judgment of the radiation oncologist and radiologist. The PTV,

on the other hand, is a geometrical scheme used for treatment planning in order to ensure that the

prescribed dose targets the CTV.71 69 Alternatively, one might view the PTV as decreasing the risk

that patient positioning, setup uncertainties, and OAR or target position uncertainties will lead to

a severe under-dosage of the CTV. As the impact of setup uncertainties decreases with IGRT, the

CTV to PTV margins may be reduced, thus decreasing normal tissue toxicity. Taking this a step

further, if diagnostic imaging, interpretation, and registration with the initial treatment plan are

performed frequently while a patient is on therapy, the treatment plan can be “adapted” to

optimize tumor control and minimize normal tissue damage.

16

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

136137138139140141142143144

ART is defined as changing a radiation plan during RT to account for anatomy or biology

changes of the patients.72 0 Studies have shown clear advantages and gains in clinical application

of adaptive radiation therapy. However, clinical implementation is limited as ART is time-

consuming and places an additional workload on clinicians to quickly contour follow-up images

manually. Adaptive planning could be used at different fractions (off-line) and at different time

points within one fraction (on-line).73 1 Identifying changes in the GTV between images acquired

for the purpose of RT planning and images acquired for the purpose of assessing response to RT

is challenging. This is mainly because of: 1. uncertainties of soft tissues and internal organ

motion;74 2 2. the response of the GTV to radiation or other treatments is difficult to predict,

which may cause anatomical changes;75 3 3. distortions or contrast variability between different

modalities, which makes it difficult to register the information between multi-modality images.76 4

To overcome these difficulties, image guided adaptive radiation therapy (IGART) is the most

reliable method for a comprehensive ART approach.77 5 Research into known, documented

uncertainties in soft tissue and organ motions in prostate,78 6 head and neck79 7 and lung80 78 is well

developed and directly impact ART approaches. Furthermore, registration methods that combine

information and realize deformable mapping between structural and functional images are

essential to make individualized IGART more practicalrapid and feasible and enable

individualized plans.81 79

Preliminary Experience With A Novel Image Registration Framework

17

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

145146147148149150151152153

To develop an efficient and practical IGART approach, an automatic and robust method for the

registration of images acquired at different time points with different modalities is required. In

addition, an accurate segmentation method for non-rigidly tracking the GTV on follow-up

images is also required. Our research group has developed an automatic registration-

segmentation framework, which has the potential to achieve these aims. In this framework the

planning contour of the GTV from the pre-treatment reference image is first registered to the

post-treatment target image both rigidly and non-rigidly. A segmentation method, which uses the

registered data and planning contour as an initial condition, is applied to find the new contour on

the target image. Four Dice coefficients (D1: between the original reference and target contour;

D2: between the rigidly registered reference contour and target contour; D3: between the non-

rigidly registered reference contour and target contour; D4: between the final segmented result

and target contour) are used to assess the step-by-step improvement in this framework (Fig. 7).

Rigid registration is deemed sufficient when there is limited distortion between the target

and reference images, while in the presence of distortion, a non-rigid transform is required. The

proposed automatic registration-segmentation framework has non-rigid capability and to date,

has been applied to images obtained from five dogs (P1-P5) with sino-nasal tumors treated with

helical Tomotherapy.27 Details of data sets used to test this framework are presented in Table 2.

CT imaging was performed using a single detector row unit (GE HiLight Advantage, GE

Medical Systems, Milwaukee, WI, USA) with dogs in sternal recumbency while MR was

performed at 1.0T (GE Signa Advantage) with dogs in dorsal recumbency. All CT images were

18

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

154155156157158159160161162

acquired with 512x512 pixels and a pixel size of 0.39x0.39 mm. All MR images were acquired

with 256x256 pixels and a pixel size of 1.6x1.6 mm. The number of CT and MR image slices

containing the tumor volume varied between dogs and data sets. A board-certified radiation

oncologist involved in the canine trial retrospectively evaluated all images and contoured GTV,

CTV, PTV and OAR for evaluation (JL).

The performance of the framework for automatic registration and segmentation of target

volumes was initially tested on pre-treatment and 6-week post-treatment images from case P5,

where the images were from the same modality (CT) (Fig. 8). The registration results are shown

in blue, target GTV contours are shown in yellow and the automatically segmented GTV contour

produced by the image analysis algorithm are illustrated by a red contour. The Dice coefficient

between the clinical and automatically generated contour for this case was 96.73% indicating

good agreement. The framework was next tested on multi-modality data from case P5

comprising of pre-treatment MR and CT acquired 6 weeks after treatment (Fig. 9). In this case,

the reference contour from the MR image shown in blue was pre-processed by scaling and

rotating prior to matching as different pixel sizes and orientations existed between CT and MR.

With a Dice coefficient of 93.7% between the target contour (yellow) and the automatically

segmented GTV contour (red), once again there is good agreement. The overall performance, as

indicated by the Dice coefficient, in the first 4 dogs was found to be > 80% (Fig. 10), however

additional investigation and analysis is required to assess the performance of the approach on

estimating the position of the GTV for different tumor histologies and over a longer time course.

19

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

163164165166167168169170171

The preliminary results from the proposed registration-segmentation framework

demonstrate that by using prior information from pre-treatment images, it is possible to

automatically segment the GTV on post-radiation therapy images with acceptable clinical

accuracy (mean Dice, 84.44%). The approach also has the potential to be used as part of an ART

strategy for personalized RT that is tailored to time-of-treatment variations of the GTV as it has

the advantage of automatically defining topology changes (Fig. 11). Further work is required to

assess the performance on a larger data set, with different targets (CTV) and using other imaging

modalities such as PET, however, this framework shows promise as a tool to define the GTV

over time and has the potential to be used for ART.

Conclusion

In its simplest context, image registration simply refers to the comparison of one dataset

to another. The complexity behind making quantitative use of this imaging data has been

reviewed here, illustrating that it is necessary to determine the transformations needed to relate

coordinates of one dataset to another. The impact of image registration on RT from treatment-

planning to image-guided delivery to post-treatment follow-up is enormous and has already

impacted veterinary patients. Novel registration frameworks may be ideally tested on companion

animals undergoing RT given similarities between human and veterinary radiation oncology.

Ultimately optimized image registration and data fusion techniques should aid in targeting tumor

20

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

172173174175176177178179180

volumes and organs-at-risk, improve response assessment and increase the availability and utility

of an efficient rapid ART approach.

Acknowledgements:

We are grateful for the generous support from NHS Lothian, Edinburgh and Lothians Health

Foundation (charity number SC007342), the James Clerk Maxwell Foundation, the Jamie King

Uro-Oncology Endowment Fund, a University of Edinburgh Darwin Award and a University of

Edinburgh Individual Stipend.

References

1. Paoloni M, Khanna C. Translation of new cancer treatments from pet dogs to humans.

Nat Rev Cancer. 2008;8:147-56.

2. Mell LK, Song WY, Pawlicki T, Mundt AJ. Image-guided radiation therapy. In: Halperin EC,

Wazer DE, Perez CA, Brady LW (eds): Principles and Practice of Radiation Oncology, 6th

ed. Philadelphia: Lippincott Williams and Wilkins, 2013;246-76.

3. Pluim JPM, Fitzpatrick JM. Image registration. IEEE Trans Med Imaging.

2003;22(11):1341–3.

4. Gooding MJ, Rajpoot K, Mitchell S, Chamberlain P, Kennedy SH, Noble JA. Investigation

into the fusion of multiple 4-D fetal echocardiography images to improve image quality.

Ultrasound Med Biol. 2010;36(6):957-66.

5. Maintz JBA, Viergever MA. A survey of medical image registration. Med Image Anal. 21

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

181182183184185186187188189

1998;2(1):1-36.

6. Freeborough PA, Fox NC. Modeling brain deformations in Alzheimer disease by fluid

registration of serial 3D MR images. J Comput Assist Tomogr. 1998;22(5):838–43.

7. Collins DL, Evans AC. ANIMAL: validation and applications of non-linear registration-

based segmentation. Intern J Pattern Recognit Artif Intell. 1997;11(8):1271–94.

8. Lavely WC, Scarfone C, Cevikalp H, Li R, Byrne DW, Cmelak AJ, et al. Phantom validation

of co-registration of PET and CT for image-guided radiotherapy. Med Phys.

2004;31(4):1083–92.

9. Maurer CR, Fitzpatrick JM, Wang MY, Galloway RL, Maciunas RJ, Allen GS. Registration of

head volume images using implantable fiducial markers. IEEE Trans Med Imaging.

1997;16(4):447–62.

10. Grosland NM, Bafna R, Magnotta VA. Automated hexahedral meshing of anatomic

structures using deformable registration. Comput Methods Biomech Biomed Engin.

2009;12(1):35-43.

11. Maksimov D, Hesser J, Brockmann C, Jochum S, Dietz T, Schnitzer A. Graph-Matching

based CTA. IEEE Trans Med Imaging. 2009;28(12):1940–1954.

12. Martin B, Julien M, Jouke D, Tobias DH, Axel WW, Ivo Q, et al. Atlas-based whole-body

segmentation of mice from low-contrast Micro-CT data. Med Image Anal.

2010;14(5):723-37.

22

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

190191192193194195196197198

13. Anand AJ, Abhijit JC, Changqing L, Joyita D, Simon RC, David WS, et al. DigiWarp: a

method for deformable mouse atlas warping to surface topographic data. Phys Med Biol.

2010;55(20):6197–214.

14. Johnson CA, Seidel J, Carson RE, Gandler WR, Sofer A, Green MV. Evaluation of 3D

reconstruction algorithms for a small animal PET camera. IEEE Trans Nucl Sci.

1997;44(3):1303-8.

15. Green MV, Seidel J, Vaquero JJ, Jagoda E, Lee I, Eckelman WC. High resolution PET, SPECT

and projection imaging in small animals. Comput Med Imaging Graph. 2001;25(2):79-

86.

16. Wildeman M, Baiker M, Reinders MJT, Lowik CWGM, Reiber JHC, Lelieveldt BPF. 2D/3D

registration of micro-CT data to multi-view photographs based on a 3D distance map.

Proc IEEE Int Symp Biomed Imaging. In: Proceedings of IEEE International Symposium

on Biomedical Imaging; 2009; Boston, Massachusetts, USA.

17. Zheng X, Xishi H, Xiaobo Z, Youxian S, Vasilis N, Stephen TCW. Registration of 3-D CT and

2-D flat images of mouse via affine transformation. IEEE Trans Inf Technol Biomed.

2008;12(5):569-78.

18. Yang CC, Wu TH, Lin MH, Huang YH, Guo WY, Chen CL, et al. Multimodality imaging

combination in small animal via point-based registration. Nucl Instrum Methods Phys

Res A. 2006;569(2):240-4.Johnson HJ, McCormick M. ITK software guide. 2013.

23

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

199200201202203204205206207

19. Hayakawa N, Uemura K, Ishiwata K, Shimada Y, Ogi N, Nagaoka T. A PET-MRI

registration technique for PET studies of the rat brain. Nucl Med Biol. 2000;27(2):121-5.

20. Sakdinawat AE, Iwata K, Hwang AB, Tang HR, Wong KH, Hasegawa BH. Development of

external fiducial markers for image registration in small animal SPECT/CT. IEEE Nucl

Sci Symp Conf Rec. 2002;2:842-5.

21. Woo SK, Kim KM, Lee T, Kim JY, Jung JH, Woo KS, et al. Experimental condition and

registration method for the tumor detection of lung metastasis small animal PET and CT

whole body images. IEEE Nucl Sci Symp Conf Rec. 2007;5:3372-5.

22. Yubei S, Koji U, Babak AA, Tsukasa N, Kiichi I, Hinako T, et al. Application of PET-MRI

registration techniques to cat brain imaging. J Neurosci Methods. 2000;101(1):1-7.

23. Bottcher P, Maierl J, Hecht S, Matis U, Leibich HG. Automatic image registration of three-

dimensional images of the head of cats and dogs by use of maximization of mutual

information. Am J Vet Res. 2004;65:1680-7.

24. Shimada Y, Uemura K, Ardenkani BA, Nagaoka T, Ishiwata K, Toyama H, et al. Application

of PET-MRI registration techniques to cat brain imaging. J Neurosci Methods.

2000;101:1-7.

25. Nolan MW, Kogan L, Griffin LR, Custis Jt, Harmon JF, Biller BJ, et al. Intensity-modulated

and image-guided radiation therapy for treatment of genitourinary carcinomas in dogs.

J Vet Intern Med. 2012;26:987-95.

24

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

208209210211212213214215216

26. Harmon J, Yoshikawa H, Custis J, LaRue S. Evlauation of canine prostate intrafractional

motion using serial cone beam computed tomography imaging. Vet Radiol Ultrasound.

2013;54:93-8.

27. Murphy S, Gutierrez A, Lawrence J, Bjorling D, Mackie T, Forrest L. Laparoscopically

implanted tissue expander radiotherapy in canine transitional cell carcinoma. Vet Radiol

Ultrasound. 2008;49:400-5.

28. Lawrence JA, Forrest LJ, Turek MM, Miller PE, Mackie TR, Jaradat HA, et al. Proof of

principle of ocular sparing in dogs with sinonasal tumors treated with intensity-

modulated radiation therapy. Vet Radiol Ultrasound. 2012;51:561-70.

29. Kubicek LN, Seo S, Chappell RJ, Jeraj R, Forrest LJ. Helical tomotherapy setup variations

in canine nasal tumor patients immobilized with a bite block. Vet Radiol Ultrasound.

2012;53:474-81.

30. Forrest LJ, Mackie TR, Ruchala K, Turek M, Kapatoes J, Jaradat H, et al. The utility of

megavoltage computed tomography images from a helical tomotherapy system for

setup verification purposes. Int J Radiat Oncol Biol Phys. 2004;60:1639-44.

31. Kaczka DW, Cao K, Christensen GE, Bates JHT, Simon BA. Analysis of regional mechanics

in canine lung injury used forced oscillations and 3D image registration. Ann Biomed

Eng. 2011;39:1112-24.

32. Vail DM, MacEwen EG. Spontaneously occurring tumors of companion animals as

25

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

217218219220221222223224225

models for human cancer. 2000;18:781-92.

33. Khanna C, Lindblad-Toh K, Vail D, London C, Bergman P, Barber L, Breen M, et al. The

dog as a cancer model. Nat Biotechnol. 2006;24:1065-6.

34. Paoloni M, Webb C, Mazcko C, Cherba D, Hendricks W, Lana S, et al. Prospective

molecular profiling of canine cancers provides a clinically relevant comparative model

of evaluating personalized medicine (PMed) trials. PLoS ONE 2014;9:e90028.

doi:10.1371/journal.pone.0090028

35. Modersitzki J. Numerical Methods for Image Registration. Oxford University Press.

2004.

36. Murphy K, van Ginneken B, Klein S, Staring M, de Hoop BJ, Viergever MA, et al. Semi-

automatic construction of reference standards for evaluation of image registration. Med

Image Anal. 2011;15:71-84.

37. Markelj P, Tomazevic D, Likar B, Pernus F. A review of 3D/2D registration methods for

image-guided interventions. Med Image Anal. 2012;16:642-61.

38. Livyatan H, Yaniv Z, Joskowicz L. Gradient-based 2-D/3-D rigid registration of

fluoroscopic X-ray to CT. IEEE Trans Med Imaging. 2003;22(11):1395–406.

39. Andreetto M, Cortelazzo GM, Lucchese L. Frequency domain registration of computer

tomography data. In: John A, Gabriel T, editors. Proceedings of the 2nd International

Symposium on 3D Data Processing, Visualization and Transmission; 2004 Sep 6-9;

26

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

226227228229230231232233234

Thessaloniki, Greece.

40. Kessler ML. Image registration and data fusion in radiation therapy. Br J Radiol.

2006;79:S99-S108.

41. Mattes D, Haynor DR, Vesselle H, Lewellen TK, Eubank W. PET-CT image registration in

the chest using free-form deformations. IEEE Trans Med Imaging. 2003;22(1):120-8.

42. Hellier P, Barillot C. A hierarchical parametric algorithm for deformable multimodal

image registration. Comput Methods Programs Biomed. 2004;75(2):107-15.

43. Rueckert D, Sonoda LI, Hayes C, Hill DLG, Leach MO, Hawkes DJ. Non-rigid registration

using free-form deformations: application to breast MR images. IEEE Trans Med

Imaging. 1999;18(8):712–21.

44. Meyer CR, Boes JL, Kim B, Bland PH, Zasadny KR, Kison PV, et al. Demonstration of

accuracy and clinical versatility of mutual information for automatic multimodality

image fusion using affine and thin-plate spline warped geometric deformations. Med

Image Anal. 1997;1(3):195–206.

45. Kessler ML, Pitluck S, Petti PL, Castro JR. Integration of multimodality imaging data for

radiotherapy treatment planning. Int J Radiat Oncol Biol Phys. 1991;21:1653-67.

46. Pelizzari CA, Chen GT, Spelbring DR, Weichselbaum RR. Accurate three-dimensional

registration of CT, PET, and/or MR images of the brain. J Comput Assist Tomogr.

1989;13:20-26.

27

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

235236237238239240241242243

47. Studholme C, Constable RT, Duncan JS. Accurate alignment of functional EPI data to

anatomical MRI using a physics-based distortion model. IEEE Trans Med Imaging.

2000;19(11):1115–27.

48. Schnabel JA, Tanner C, Castellano-Smith AD, Degenhard A, Martin OL, Hose DR, et al.

Validation of nonrigid image registration using finite element methods: application to

breast MR images. IEEE Trans Med Imaging. 2003;22(2):238–47.

49. Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P. Multimodality medical

image registration by maximization of mutual information. IEEE Trans Med Imaging.

1997;16(2):187–98.

50. Alterovitz R, Goldberg K, Pouliot J, Hsu IC, Kim Y, Noworolski SM, et al. Registration of

MR prostate images with biomechanical modeling and nonlinear parameter estimation.

Med Phys. 2006;33(2):446-54.

51. Vercauteren T, Pennec X, Perchant A, Ayache N. Nonparametric diffeomorphic image

registration with the demons algorithm. In: Ayache N, Ourselin S, Maeder A, editors.

Proceedings of the 10th International Conference on Medical Image Computing and

Computer-Assisted Intervention; 2007; Brisbane, Australia.

52. Keyerleber MA, McEntee MC, Farrelly J, Podgorsak M. Completeness of reporting of

radiation therapy planning, dose, and delivery in veterinary radiation oncology

manuscripts from 2005-2010. Vet Radiol Ultrasound. 2012;53:221-30.

28

534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

551

552

244245246247248249250251252

53. LaRue SM, Gordon IK. Radiation Therapy. In: Withrow SJ, Vail DM, Page RL (eds): Small

Animal Clinical Oncology, 5th ed. St Louis: Elsevier Inc; 2013:180-97.

54. Rohrer Bley C, Sumova A, Roos M, Kaser-Hotz B. Irradiation of brain tumors in dogs with

neurologic disease. J Vet Intern Med. 2005;19:849-54.

55. Kent MS, Gordon IK, Benavides I, Primas P, Young J. Assessment of the accuracy and

precision of a patient immobilization device for radiation therapy in canine head and

neck tumors. Vet Radiol Ultrasound. 2009;50:550-4.

56. McEntee MC. Portal radiography in veterinary radiation oncology: options and

considerations. Vet Radiol Ultrasound. 2009;49:S57-S61.

57. Nieset JR, Harmon JF, LaRue SM. Use of cone-beam computed tomography to

characterize daily urinary bladder variations during fractionated radiotherapy for

canine bladder cancer. Vet Radiol Ultrasound. 2011;52:580-8.

58. Eisenhauer EA, Therasse P, Bogaerts J, Schwartz LH, Sargent D, Ford R, et al. New

response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1).

Eur J Cancer. 2009;45:228-47.

59. Farrelly J, McEntee MC. A survey of veterinary radiation facilities in 2010. Vet Radiol

Ultrasound 2014;55:638-643.

60. McEntee MC. A survey of veterinary radiation facilities in the United States during 2001.

Vet Radiol Ultrasound 2004;45:476-479.

29

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

253254255256257258259260261

61. Randall E, Loeber S, Kraft S. Physiologic variants, benign processes, and artifacts from

106 canine and feline FDG-PET/computed tomography scans. Vet Radiol Ultrasound.

2014;55:213-26.

62.[60.] Hansen AE, McEvoy F, Engelholm SA, Law I, Kristensen AT. FDG PET/CT imaging in

canine cancer patients. Vet Radiol Ultrasound. 2011;52:201-6.

63.[61.] Leblanc AK, Jakoby BW, Townsend DW, Daniel GB. 18FDG-PET imaging in canine

lymphoma and cutaneous mast cell tumor. Vet Radiol Ultrasound. 2009;50:215-23.

64.[62.] Yoshikawa H, Randall EK, Kraft SL, LaRue SM. Comparison between 2-18F-fluoro-2-

deoxy-D-glucose positron emission tomography and contrast-enhanced computed

tomography for measuring gross tumor volume in cats with oral squamous cell

carcinoma. Vet Radiol Ultrasound. 2013;54:307-13.

65.[63.] Bradshaw TJ, Bowen SR, Jallow N, Forrest LJ, Jaraj R. Heterogeneity in intratumor

correlations of 18F-FDG, 18-FLT, and 61Cu-ATSM PET in canine sinonasal tumors. J Nucl

Med. 2013;54:1917-31.

66.[64.] LeBlanc AK, Peremans K. PET and SPECT imaging in veterinary medicine. Semin

Nucl Med. 2014;44:47-56.

67.[65.] LeBlanc AK, Miller AN, Galyon GD, Moyers TD, Long MJ, Stuckey AC, et al.

Preliminary evaluation of serial (18)FDG-PET/CT to assess response to toceranib

phosphate therapy in canine cancer. Vet Radiol Ultrasound. 2012;53:348-57.

30

572

573

574

575

576

577

578

579

580

581

582

583

584

585

586

587

588

589

590

262263264265266267268269270

68.[66.] Lawrence J, Vanderhoek M, Barbee D, Jeraj R, Tumas DB, Vail DM. Use of 3’-deoxy-

3’-[18F]fluorothymidine PET/CT for evaluating response to cytotoxic chemotherapy in

dogs with non-Hodgkin’s lymphoma. Vet Radiol Ultrasound. 2009;50:660-8.

69.[67.] Bradshaw TJ, Yip S, Jallow N, Forrest LJ, Jeraj R. Spatiotemporal stability of Cu-

ATSM and FLT positron emission tomography distributions during radiation therapy. Int

J Radiat Oncol Biol Phys. 2014; doi:10.1016/j.ijrobp.2014.02.016.

70.[68.] Bowen SR, Chappell RJ, Bentzen SM, Deveau MA, Forrest LJ, Jeraj R. Spatially

resolved regression analysis of pre-treatment FDG, FLT and Cu-ATSM PET from post-

treatment PDG PET: an exploratory study. Radiother Oncol. 2012;105:41-8.

71.[69.] ICRU Report 62. Prescribing, recording, and reporting photon beam therapy

(supplement to ICRU Report 50). Bethesda, USA: International Commission on

Radiation Unites and Measurements; 1999.

72.[70.] Yan D, Vicini F, Wong J, Martinez A. Adaptive radiation therapy. Phys Med Biol.

1997;42(1):123-32.

73.[71.] Li T, Zhu X, Thongphiew D, Lee WR, Vujaskovic Z, Wu Q, et al. On-line adaptive

radiation therapy: feasibility and clinical study. J Oncol. 2010;

doi:10.1155/2010/407236.

74.[72.] Birkner M, Yan D, Alber M, Liang J, Nüsslin F. Adapting inverse planning to patient

and organ geometrical variation: algorithm and implementation. 2003;30(10):2822-31.

31

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

271272273274275276277278279

75.[73.] Wu Q, Ivaldi G, Liang J, Lockman D, Yan D, Martinez A. Geometric and dosimetric

evaluations of an online image-guidance strategy for 3D-CRT of prostate cancer. Int J

Radiat Oncol Biol Phys. 2006;64(5):1596-609.

76.[74.] Jans HS, Syme AM, Rathee S, Fallone BG. 3D interfractional patient position

verification using 2D-3D registration of orthogonal images. Med Phys.

2006;33(5):1420-39.

77.[75.] Court LE, Tishler RB, Petit J, Cormack R, Chin L. Automatic online adaptive

radiation therapy techniques for targets with significant shape change: a feasibility

study. Phys Med Biol. 2006;51(10):2493-501.

78.[76.] Thongphiew D, Wu QJ, Lee WR, Chankong V, Yoo S, McMahon R, et al. Comparison of

online IGRT techniques for prostate IMRT treatment: adaptive vs repositioning

correction. Med Phys. 2009;36(5):1651-62.

79.[77.] Peroni M, Ciardo D, Spadea MF, Riboldi M, Comi S, Alterio D, et al. Automatic

segmentation and online virtualCT in head-and-neck adaptive radiation therapy. Int J

Radiat Oncol Biol Phys. 2012;84(3):e427-33. doi:10.1016/j.ijrobp.2012.04.003.

80.[78.] Sonke JJ, Belderbos J. Adaptive radiotherapy for lung cancer. Semin Radiat Oncol.

2010;20(2):94-106. doi:10.1016/j.semradonc.2009.11.003.

81.[79.] Jobse M, Davelaar J, Hendriks E, Kattevilder R, Reiber H, Stoel B. A new algorithm

for the registration of portal images to planning images in the verification of

32

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

280281282283284285286287288

radiotherapy, as validated in prostate treatments. Med Phys. 2003;30(9):2274-81.

33

629

630

289290291292293294295296297

TablesTable 1: The Classification Of Registration Method By Maintz And Viergever In 1998.5

Criteria Classification

I. Dimensionality A. Spatial dimensions only: 2D-2D; 2D-3D; 3D-3D

B. Time series (more than two images), with spatial dimensions:

2D-2D; 2D-3D; 3D-3D

II. Nature of Registration Basis A. Extrinsic: 1. Invasive (stereotactic frame, fiducials); 2. Non-

invasive (mould-frame, dental adaptor)

B. Intrinsic: 1. Landmark based; 2. Segmentation-based; 3.

Voxel-property-based

C. Non-image based

III. Nature of Transformation A. Rigid; B. Affine; C. Projective; D. Curved

IV. Domain of Transformation A. Local; B. Global

V. Interaction A. Interactive; B. Semi-automatic; C. Automatic

VI. Optimization Procedure A. Parameters computed; B. Parameters searched for

VII. Modalities Involved A. Mono-modal; B. Multimodal; C. Modality to model; D.

Patient to modality

VIII. Subject A. Intrasubject; B. Intersubject; C. Atlas

IX. Object A. Head; B. Thorax; C. Abdomen; D. Pelvis and peritoneum; E.

Limbs; F. Spine and vertebrae

34

631632

633

298299300301302303304305306

Figure Captions

Fig. 1. A typical flow chart of image-guided radiation therapy (IGRT), highlighting the need for

registration, using a canine nasal tumor as an example. Diagnostic images are obtained and

registered to the reference image for treatment-planning. Each fraction of radiation is

administered with the aid of an electronic portal imaging device (EPID) or cone-beam computed

tomography (CBCT) images that are registered to the treatment-planning reference image.

Fig. 2. A typical flow chart of image registration demonstrating the steps – transformation

function, similarity function, and optimization – that are required to obtain a registration result.

Fig. 3. An example of a B-spline control grid on the reference and target image of a dog with a

nasal carcinoma. This demonstrates a 20x20 control grid on pre-treatment and post-treatment

computed tomography (CT) (512x512) images (manually warped).

Fig. 4. The variability in the similarity function (SSD) is shown and demonstrated from canine

nasal tumor images when the transformation function (rotation) is applied on the target image

and makes it rotate around the reference image from -90° to +90°. The rotation degree was

defined as the angle between the directions of the reference (black) and the target (white) images.

35

634

635

636

637

638

639

640

641

642

643

644

645

646

647

648

649

650

651

652

307308309310311312313314315

As a result, the image in A illustrates an approximately -70° rotation while B illustrates an

approximately 30° rotation.

Fig.5. Illustration of a multi-step registration framework in a dog with a nasal tumor. Images

show registration of a pre-treatment computed tomography (CT) study with the post-treatment

CT 6 weeks after definitive intensity-modulated radiation therapy. The series demonstrates the

target image with the pre-treatment gross tumor volume (GTV) contour (blue), target GTV

contour (yellow) and an automatically generated contour using a multistep process (red). A

represents D1, registration between the original reference and the target contour; B represents

D2, registration between the rigid registered reference contour and target contour; C represents

D3, registration between non-rigid registered reference contour and target contour; D represents

D4, the final segmented result and target contour. The Dice coefficients increase following each

step in the registration framework as shown in Fig. 6: D1 = 42.65 %, D2 = 85.57%, D3 =

90.81%, D4 = 96.73%.

Fig. 65. Portal images obtained from a dog with a sinonasal tumor undergoing definitive intent

radiation therapy prior to 6 MV photon therapy with a linear accelerator (Varian 2100 CD;

Varian Medical Systems, Palo Alto, USA); orthogonal images were obtained and analyzed

immediately prior to treatment delivery. An on-board flat panel imager permits acquisition of the

dog’s position prior to treatment (image in the lower right), which can be compared with the

36

653

654

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

316317318319320321322323324

digitally reconstructed radiograph (DRR) obtained from the planning computed tomography

(CT) (image in the upper right). Images were evaluated using commercially available treatment

planning software (Eclipse v.11.0, Varian Medical Systems). The larger image on the left

demonstrates registration of the electronic portal image and the DRR. (A) Images were obtained

with the linear accelerator gantry at 0 degrees; the larger image on the left demonstrates

evaluation of the registered images using a “moving window” technique that allows the window

to move the DRR over the dog’s daily portal image. The orange graticule grid represents the

dog’s position at acquisition of the portal image while the green graticule grid represents shifts

needed in the dog’s position to match the ideal planning position. (B) Images were obtained

with the linear accelerator gantry at 90 degrees; the larger image on the left demonstrates

evaluation of the images using a “split window” technique, in which the central axis can be

moved to reveal more of the DRR image or the electronic portal imaging device (EPID) image as

needed. Manual alignment may be performed as was done here in which the daily image is

moved to match the DRR; alternatively semi-automated or point matching registration can be

performed.

Fig 76. A series of rigidly registered images acquired on a human patient treated for

prostate cancer highlighting the difficulty in interpreting critical structures in different

imaging modalities. T1- (A) and T2- (B) weighted MR images acquired on a Siemens

Symphony 1.5T scanner (Siemens, Munich, Germany) were registered to a radiotherapy 37

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

689

690

691

692

325326327328329330331332333

planning CT (C) acquired on a Philips Brilliance Big Bore CT scanner fitted with a flat

couch. The hyper-intense gold markers implanted within the prostate gland and used for

positional verification are clearly visible. The corresponding pre-treatment CBCT image

(D) acquired on a Varian Clinac and used for positional verification. Alignment is complete

and treatment begins when the hyper-intense markers are aligned on CT and CBCT. Soft

tissue matching is difficult because of the atrophy induced in the prostate by the neo-

adjuvant hormones administered between MR and CT acquisition.

Fig. 7 The proposed flow chart of the novel registration-segmentation framework applied to

serial, multi-modality images of dogs with nasal tumors. Rigid registration, non-rigid registration

(optional) and level set segmentation are built into the framework. Four Dice (D) coefficients are

used to assess the improvement in the image registration in a step-by-step fashion; D1: between

original reference and target contour; D2: between rigid registered reference contour and target

contour; D3: between non-rigid registered reference contour and target contour; D4: between

final segmented result and target contour.

Fig.8. Illustration of improved registration using a novel framework; images show registration of

a pre-treatment computed tomography (CT) study with the post-treatment CT 6 weeks after

definitive intensity-modulated radiation therapy in a dog with a nasal tumor. The series

demonstrates the target image with the pre-treatment gross tumor volume (GTV) contour (blue),

38

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

334335336337338339340341342

target GTV contour (yellow) and the automatically generated contour (shown in red). A

represents D1, registration between the original reference and the target contour; B represents

D2, registration between the rigid registered reference contour and target contour; C represents

D3, registration between non-rigid registered reference contour and target contour; D represents

D4, the final segmented result and target contour. The Dice coefficients increased following

each step in the registration framework as shown in Fig. 6: D1 = 42.65 %, D2 = 85.57%, D3 =

90.81%, D4 = 96.73%.

Fig. 9. Illustration of a novel registration framework in a dog with a nasal tumor; images show

registration of the pre-treatment magnetic resonance (MR) images with the post-treatment

computed tomography (CT) images 6 weeks after definitive intensity-modulated radiation

therapy for a dog with a nasal tumor. The target image is shown with pre-treatment gross tumor

volume (GTV) contour (blue), target GTV contour (yellow) and the automatically generated

contour (red). A represents D1, registration between the original reference and the target contour;

B represents D2, registration between the rigid registered reference contour and target contour; C

represents D3, registration between non-rigid registered reference contour and target contour; D

represents D4, the final segmented result and target contour. The Dice coefficients at each step in

the framework as shown in Fig. 6 were: D1 = 79.76 %, D2 = 86.78%, D3 = 90.07%, D4 =

93.7%.

39

713

714

715

716

717

718

719

720

721

722

723

724

725

726

727

728

729

730

731

732

343344345346347348349350351

Fig.10. The Dice coefficient calculated by comparing the (true) clinical gross tumor volume

(GTV) contour obtained from the post-treatment images to the automatically generated contour

on all images acquired on 4 dogs with nasal tumors. The overall Dice was 84.44%.

Fig. 11. Automatic topology changes determined in the gross tumor volume (GTV) based on

level set method determined in 2 dogs with nasal tumors. The image in A represents dog 2 (P2)

while the image in B represents dog 3 (P3). The target images are shown in each image with the

pre-treatment GTV contour (blue), target GTV contour (yellow) and the automatically

generated contour (red).

40

733

734

735

737

738

739

740

741

742

743

352353354355356357358359360