Model Based Tissue Differentiationin MR Brain Images

Peter H. Mowforth and Jin Zhengping

The Turing Institute,36 North Hanover Street,

Glasgow Gl 2AD.

This paper describes a technique which establishes thecorrespondence between a magnetic resonance (MR) im-age of the brain and a model anatomical image. Fol-lowing correspondence, it is demonstrated that segmen-tation of tissue types may be achieved along with theprovision of medically relevant indexes for diagnosis.

The work reported in this paper is part of a projectwhose objective is the automated, model-based interpre-tation of medical images. A suggested overall structureof this system, see Fig. 1, involves descriptions at multi-

SymboHcdescriptions

Symbolicdescriptions

Token descriptions Token descriptions

Medical images • > Imaging models * - Anotomical models

User interface

Figure 1: Suggested overall framework for project.

pie levels of abstraction. In order to interpret the med-ical images it is necessary to match them to anatomicalmodels. Although this may be possible over a rangeof abstractions, the work reported here attempts corre-spondence only at the image level. Whilst the use oflearning, symbolic reasoning and inference may be nec-essary for a complete system, this work explores howmuch can be achieved without such abstraction.

A MR machine has the ability to introduce contrastbetween tissue types on the basis of its imaging parame-ters. Given that the appearance of the resulting image isdetermined by the machine settings we require an imag-ing model for the machine. Such a model can be in the

Imagingmodel

Modelanatomical

image

MR imagefrom

patient

Instatiatedmodelimage

Discrepencymaps

Figure 2: Summary of the matching process which com-bines together a model anatomical image, an image froma patient using a MR scanner and the imaging modelspecific to that scanner.

form of a look-up table providing contrast values for dif-ferent machine settings and, if necessary, some estimateof their variation.

The first level of the anatomical model consists of im-age slices of a model brain for which the different tissuesare represented by arbitrarily chosen gray values. Thesegray-valued variables may then be instantiated via theimaging model so as to produce an instantiated modelimage. The instantiated model image thus representswhat we might expect a 'perfect' brain to look like un-der a certain set of machine protocols.

The final requirement is to match together the instan-tiated model image with the image derived from the pa-tient. The discrepancies between the two may be repre-sented via discrepancy maps. Fig. 2 shows a summaryof this process.

Similar work using CT images of the brain has beendescribed in [1]. Work which explored the usage of MRIto facilitate medical diagnosis and research has been re-ported in [9] where brain tissue MR images were classi-fied by utilizing information such as gray levels and areassupplied by a head model. In [7] a general segmentationand recognition system has been 'instantiated' by en-coding a domain-dependent subsystem of knowledge ob-tained through books and by interviewing experts to rec-ognize MR brain images and PET brain images. Otherrelevant work has been reported in [10] where multispec-tral MRI and display techniques were used to obtain

67 AVC 1989 doi:10.5244/C.3.12

Raw imagetaken from a book

[ Pixel painting I

The anatomical model uses arbitrarily chosengray values to symbolically differentiatebetween:

airscalp and bonecsfgray matterwhite matter Anatomical model

Figure 3: Anatomical model building.

higher contrast between tissue types of interest and back-ground; in [8] where multispectral MRI has been usedto provide a 'high information content' display whichwill aid in the diagnosis and analysis of the atheroscle-rotic disease process, and supervision of therapies; in [6]where MRI data have been used to reconstruct a 3Dheart model which describes the shape of each part ofthe heart in a voxel space.

ANATOMICAL MODEL

The model anatomical image used in this paper is takenfrom the Atlas of Sectional Human Anatomy by J.G.Koritke and H. Sick [5]. The image in the atlas whichcorresponded to that captured from the patient was firstdigitised. A linear geometric transform was applied soas to compensate for the distortion introduced in thedigitising process; i.e. the image was aligned verticallyand centered.

The images were then hand-painted with arbitrar-ily chosen gray levels to symbolically differentiate be-tween: background, scalp/bone, cerebro-spinal fluid(CSF), gray matter and white matter. These were cho-sen because they represent the tissue types primarily dis-tinguished under MR protocols - see Fig. 3.

The anatomical model described is clearly the simplestthat is of practical value. Work is already in progress toextend the model in two ways. First, multiple descrip-tions are necessary in order to code information such asanatomical regions (e.g. lobes) or anatomical structures(e.g. pons). Second, the model needs extending into avolumetric 3D form.

TRa

2000200010801080

TE1002010020

GM1159910789

WM8411179104

CSF1717913259

B/S18491947

Air3132

"Unit of TR and Tg is ms.

Table 1: An imaging model.

IMAGING MODEL

The basic diagnostic indexes of MRI are the longitudi-nal (Ti) and transverse (T2) proton (1H) nuclear mag-netic resonance (NMR) relaxation times of pathologicalhuman and animal tissues [2]. Whilst T\ and T2 aremachine independent, MR machines typically only allowthe user to specify two main control parameters. Theseare TR and TE- These are machine dependent and havea relation with Ti and Ti for a given tissue under somegiven condition. There -is a further relation between Tjand T2 and the resulting gray values in the images pro-duced by the machine. These gray values are machinedependent. An ideal imaging model should establish re-lations among T\ and T-j, TR and TE, image gray levels,and other conditions such as temperature, species, andin vivo versus in vitro status.

To simplify the problem, this paper describes onlythe relationship between TR and TE and the tissue-dependent image gray levels.

The MR image derived from the patient uses a givenpair of TR and TE values. The corresponding anatomi-cal model image is instantiated by mapping each of itstissues to a particular gray level according to that rela-tion. The machine dependent, instantiated model maynow be used directly for matching.

Table 1 is an imaging model specific to a particularMR machine. To generate the imaging model we took 8MR images of a human brain, using the same TR and TEvalues. The slices were 10mm thick and were separatedby 10mm increments. Fig. 4 shows examples from oneslice under four different control settings.

For each TR and TE and tissue type (including back-ground), a sample of gray values were taken from thattissue type over all images at that TR and TE- The me-dian values of each sample set were calculated and usedto construct the imaging model shown in table 1.

INSTANTIATION OFANATOMICAL MODEL

THE

We may now instantiate the anatomical model with arange of TR and TE settings. For example, from theimaging model of Table 1, we obtain GM=115, WM=84,CSF=171, BONE/SCALP=18, AIR=3 through the in-dex TR - 2000ms, TE = 100ms. These values wereassigned to the corresponding tissues of the anatomicalmodel image to instantiate it as seen in Fig. 5.

68

Geometrically transformed image

TR: 2000 TE: 100 TR: 2000 TE: 20 Figure 6: Global matching.

TR: 1080 TE: 100 TR: 1080 TE: 20

Figure 4: MR images used in building the imaging model.8 slices for each set O/TR and TE-

MATCHING

A MR image was taken from a patient using TR =2000ms and TE = 100ms for approximately the samebrain slice as shown in Fig. 5. Our goal is to matchthe image with that instantiated from the anatomicalmodel. This is achieved as a two stage process. The firstwe call global matching where a global geometric trans-form is applied manually to rotate and stretch the imageso as to bring it into approximate correspondence. Thesecond stage, elastic matching, is an automatic processwhich establishes a continuous, sub-pixel correspondencebetween the two images.

Anatomical model Anatomical modelinstantiated with

TR =2000 and TE =100

MR image Model image

Initialestimate

ofdiscrepancy

Finalestimate

ofdiscrepancy

Figure 5: Example instantiation of the anatomicalmodel.

Figure 7: Multiple scale matching.

Global Matching

The position, orientation, size and aspect ratio of a MRimage may differ from those of the anatomical model.To enable the matching process, a global transformationneeds to be done which involves translation, rotation,scaling and elongation. This was carried out manuallyand the results are shown in Fig. 6.

Note that all of these transformations except the elon-gation are irrelevant to the variations of MR images frommodel images.

Elastic Matching

The geometric shapes of tissues are different among indi-viduals. All brains are different and hence, the anatomi-cal model can be viewed as a special (average) individual.The aim of the elastic matching is to measure all the dis-crepancies between the two special cases (the model andthe patient).

The matching process used here is a multi-scale signalmatcher(MSSM) [4]. It input is the globally matchedimage pair and its output is a pixel-by-pixel, continuousmeasure of all image discrepancies. Discrepancies arerepresented by two images, one which depicts all horizon-tal discrepancies and the other which depicts all verticaldiscrepancies.

Fig. 7 shows the software architecture for the multi-scale signal matching algorithm. Each of the two inputimages is blurred using a large V2G filter whose size isdetermined by a. For each pixel in the model image,

69

Model image Discrepancy mapsabove: verticalbelow: horizontal

Figure 8: Elastic matching of patient and model images.

the matching pixel in the medical image is searched for,using a cross-correlation function. The cross-correlationsearches the neighbourhood of a pixel rather than a sin-gle pixel to provide the matching score. The searchstarts from the initial discrepancy estimate, e.g., (0,0)and ends at a local optimum around that initial discrep-ancy estimate. For the coarsest filter, the displacementof the pixel in the medical image is the initial discrep-ancy estimate for the algorithm. The output discrepancymaps (vertical and horizontal) are used as the initial dis-crepancy estimates for the input images convolved witha smaller V2G filter of <r/2. The process is repeated andso past through to finer scale filters following a coarse-to-fine regime. The output from the smallest filter is thefinal estimate of the image discrepancies.

RESULTS

The result of matching is shown in Fig. 8 with the grayvalues in the discrepancy maps providing a pixel-by-pixelmeasure of the magnitude of both the horizontal andvertical discrepancy.

Because a correspondence has now been establishedbetween the two, any knowledge available in the modelimage can be applied directly to the MR image. Forexample, we already know what tissue each pixel in themodel image represents, hence we are immediately ableto classify each pixel in the MR image. Fig. 9 showsthe segmentation of each tissue type in the matched MRimage.

Furthermore, since we knew the transformation we didin global matching, we can do the inverse transformation.Fig. 10 shows the segmented tissues following the inversetransformation which can be used directly to calculatethe areas of each tissue type. Table 2 gives a summaryof tissue type areas in pixels.

Bone/Scalp

GM

CSF

WM

Figure 9: Segmentation of the transformed MR image.

Bone/Scap

GM

CSF

WM

Figure 10: Tissue segmentation for the original MR im-age.

GM1789

WM2100

CSF967

Bone/Scalp1818

Table 2: Area measurement of tissues in pixels.

70

DISCUSSIONThis experiment has demonstrated that, for a non-pathological MR image of a normal brain it is possible toperform tissue segmentation by direct matching at theimage level. Such an approach may well prove sufficientfor providing a simple diagnostic index. For example, incalculating ventricle volume which would facilitate thediagnosis of hydrocephalus [3].

ACKNOWLEDGEMENT

The authors thank Elizabeth Robinson at Picker Inter-national Company and Brian Dalton at GEC Hirst Lab-oratories for their assistance in capturing and supplyingall the MR images necessary for this experiment. Thanksalso to Alan Colchester at Guy's Hospital, London for hisexpertise and advice in model building; Douglas Scoularfor help with providing the pixel painting software andthe links between the Sun's and the Mac's. Finally, weacknowledge the valuable support offered to us by allthe consortium members on the Alvey 134 project whichsponsored this work.

References

[1] Ruzena Bajcsy and Stane Kovacic. Multiresolutionelastic matching. Computer Vision, Graphics, andImage Processing, 46(1):1-21, 1989.

[2] P.A. Bottomley, C.J. Hardy, R.E. Argersinger, andG. Allen-Moore. A review of *H nuclear magneticresonance relaxation in pathology: are Ti and T2

diagnostic? Medical Physics, 14(l):l-37, Jan/Feb1987.

[3] B.R. Condon, J. Patterson, D. Wyper, D.M.Hadley, G. Teasdale, R. Grant, A. Jenkins, P.Macpherson, and J. Rowan. A quantitative in-dex of ventricular and extraventricular intracranialCSF volumes using MR imaging. Journal of Com-puter Assisted Tomography, 10(5):784-792, Septem-ber/October 1986.

[4] Z. Jin and P. Mowforth. A discrete approach tosignal matching. Technical Report TIRM-88-036,The Turing Institute, Glasgow, Scotland, October1988.

[5] J.G. Koritkie and II. Sick. Atlas of Sectional Hu-man Anatomy: Frontal, Sagittal, and HorizontalPlanes. Volume 1: Head, Neck, Thorax, Urban andSchwarzenberg, Baltimore-Munich, 1983.

[6] Michiyoshi Kuwahara and Shigeru Eiho. 3-Dheart image reconstructed from MRI data. In 9thInternational Conference on Pattern Recognition,pages 1198-1201, Computer Society Press, Rome,Italy, November 1988.

[7] Wei-Chung Lin, Yue-Tong Weng, and Chin-TuChen. Expert vision systems integrating image seg-mentation and recognition processes. Engineering

Applications of Artificial Intelligence, l(4):230-249,Dec. 1988.

[8] M.B. Merickel, C.S. Carmen, W.K. Watterson, J.R.Brookeman, and C.R. Ayers. Multispectral patternrecogniton of MR imagery for the noninvasive anal-ysis of atherosclerosis. In 9th International Con-ference on Pattern Recognition, pages 1192-1197,Computer Society Press, Rome, Italy, November1988.

[9] Hidetomo Suzuki and Jun-ichiroToriwaki. Knowledge-guided automatic threshold-ing for 3-dimesional display of head MRI images.In 9th International Conference on Pattern Recog-nition, pages 1210-1212, Computer Society Press,Rome, Italy, November 1988.

[10] Michael W. Vannier, Christopher M. Speidel, Dou-glas L. Rickman, Larry D. Schertz, Lynette R.Baker, Charles F. Hildebolt, Carolyn J. Of-futt, James A. Balko, Robert L. Butterfleld, andMokhtarll. Gado. Validation of magnetic resonanceimaging (MRI) multispectral tissue classification.In 9th International Conference on Pattern Recog-nition, pages 1182-1186, Computer Society Press,Rome, Italy, November 1988.

71