Visualization of Anatomic Covariance Tensor Fields

Gordon L. Kindlmann1 David M. Weinstein1 Agatha D. Lee2

Arthur W. Toga2 Paul M. Thompson2

1Scientific Computing and Imaging Institute, University of Utah, UT2Laboratory of Neuro Imaging, Brain Mapping Division, Department of Neurology,

UCLA School of Medicine, CA

Abstract—The computation, visualization, and interpretation

of brain variability remains a significant challenge incomputational neuroanatomy. Current deformableregistration methods can generate, for each vertex of apolygonal mesh modeling the cortical surface, a distri-bution of displacement vectors between the individualmodels and their average, which can be summarizedas a covariance tensor. While analysis of anatomi-cal covariance tensor fields promises insight into thestructural components of aging and disease, basic un-derstanding of the tensor field structure is hamperedby the lack of effective methods to create informativeand interactive visualizations. We describe a novelapplication of superquadric tensor glyphs to anatomiccovariance tensor fields, supplemented by colormapsof important tensor attributes. The resulting visual-izations support a more detailed characterization ofpopulation variability of brain structure than possi-ble with previous methods, while also suggesting di-rections for subsequent quantitative analysis.

Keywords— Visualization, Brain Modeling, Covari-ance Analysis, Software Tools, Computer GraphicsSoftware

I. Introduction

The quest to understand the anatomic variabil-ity of the human brain is important for three majorapplications in neuroscience. First, the functional or-ganization of the brain differs widely across subjects,and measures are required to represent and visual-ize systematic patterns. A related application is todetermine patterns of altered brain structure in dis-eases such as Alzheimer’s, dementia, and schizophre-nia, based on databases of brain scans. Finally, sta-tistical information on anatomical variance is ben-eficial for computer visions algorithms that aim toidentify and label specific brain structures automat-ically. Visualization is an essential component ofunderstanding anatomic variability. Effective visu-alizations provide feedback for verifying the variousalgorithmic steps in computing variability, and offerflexible means of displaying and interacting with thedata so as to form and refine hypothesizes about thestructure and origins of anatomic variability.

This work represents anatomic variability as a fieldof covariance tensors over the cortical surface. Ingeneral, the challenge of tensor visualization is to

This work was supported by NIH National Center forResearch Resources grants P41 RR12553, P20 HL68566,P41 RR13642, R21 RR19771, National Institute for Biomed-ical Imaging and Bioengineering grant EB 001561, and byNIMH/NIDA Human Brain Project grant P20 MH/DA52176to the International Consortium for Brain Mapping.

convey the properties of individual tensor samples,as well as the large-scale spatial structure of changesin the tensor attributes, in such a way that does notoverwhelm the viewer with an unintelligible mass ofinformation. When visualizing covariance data com-puted from deformable registration of cortical surfacemodels, we require the ability to inspect the degreeand type of population variance at particular loca-tions of interest on the cortical surface (such as thelanguage areas of the brain in Alzheimer’s disease),as well as the means of discerning overall patternsof variation that may represent novel indicators forbiologically significant subpopulation characteristics.

This paper explores the combination of su-perquadric tensor glyphs and judicious applicationof colormaps to display individual tensors and thespatial structure of the tensor field, respectively.Glyphs, or icons, depict multiple data values bymapping them onto the shape, size, orientation,and surface appearance of a base geometric primi-tive [1]. Tensor glyphs generally indicate the ten-sor eigenvalues and eigenvectors by their scaling(shape) and orientation, respectively [2], as withthe ellipsoidal glyphs commonly used to present dif-fusion tensor MRI data [3]. The limited previouswork in visualizing anatomic covariance tensor fieldsemployed nested semi-transparent ellipsoidal glyphscorresponding to a small set of confidence limits [4].

II. Methods

Fig. 1. Cortical surface models are extracted from MRI scans(a). On individual models, 3D parametric curves are man-ually traced to represent 72 sulcal landmarks (some shownin (b), defined in [7]), which in turn constrain the elasticdeformation for cortical pattern matching.

The anatomic covariance tensors are generatedfrom a non-linear pattern matching procedure pre-sented in previous work [5], [6], [7]. MRI brain scansof 40 young healthy normal subjects were alignedwith the ICBM-305 standardized brain with a 9-parameter affine transform, using established meth-

ods [5], [7]. The aligned brains were then convertedto 3D parametric models by smoothly deforming ahigh-resolution mesh to the MRI isosurface of thebrain boundary, Fig 1(a). The major fissures in thebrain surface, termed sulci and identified in Fig 1(b),have the same topology in all subjects. This enablescortical pattern matching to quantify the anatomicdifferences of interest as the minimum-energy 3Dnonlinear elastic deformation that transforms thesulcal landmarks of an average model template ontothose of each individual [7]. Anatomic variabilityis represented at each point on the average corticalmesh as the 3D covariance tensor of the displacementvectors induced by the deformations from the aver-age to all individuals (after factoring out the affinecomponents of the initial alignment).

A 3×3 covariance tensor T can be diagonalized asT = RΛΛΛR−1 where ΛΛΛ is a diagonal matrix of eigen-values and R is a rotation matrix that transforms thestandard basis onto the eigenvector basis. Glyph-based tensor visualization transforms a base glyphgeometry, G (typically a sphere), into a tensor glyph,GT (typically an ellipsoid), by GT = RΛΛΛG . Fig. 2(a)illustrates the space of tensor shapes with ellipsoidalglyphs, and shows how different tensor shapes canunfortunately present similar appearances. In con-trast, superquadric glyphs reduce visual ambiguityand enhance the depiction of tensor structure.

We will now briefly summarize the superquadricglyph method presented in previous work [8]. Su-perquadric surfaces, shown in Fig. 2(b), are a con-tinuum of shapes created by modifying the standard(θ, φ) parameterization of the sphere with two expo-nential parameters (α, β) [9]:

p(θ, φ) =

cosα θ sinβ φ

sinα θ sinβ φcosβ φ

,

0 ≤ θ ≤ 2π0 ≤ φ ≤ π

(1)

where xα = sgn(x)|x|α. The general strategy ofsuperquadric tensor glyphs is that edges indicateeigenvalue differences. Mathematically, a differencein eigenvalues implies lack of rotational symmetry,which we visually emphasize by an edge on the glyphsurface, which in turn more clearly indicates the ori-entation of the associated eigenvectors. When twoeigenvalues are equal, the numerical indeterminacy ofthe associated eigenvectors is conveyed in the glyphwith a circular cross-section. Fig. 2(c) demonstratesadvantages of superquadrics over ellipsoids for de-picting the range of tensor shapes.

The other component of our visualization methodis the use of color scales, or colormaps, to indicatevarious attributes of the covariance tensors. Thishelps present large-scale patterns in the data. Wealso show the utility of conveying two tensor at-tributes simultaneously, by applying different mapson the mesh surface and on the tensor glyphs. Im-portant attributes of the covariance tensor T include:• ‖T‖F =

√tr(TTT): The Frobenius norm

• FA(T): The fractional anisotropy, borrowed fromthe diffusion-tensor MRI literature [10]• skew(λi): For eigenvalues λ1 ≥ λ2 ≥ λ3, this variesbetween −1/

√2 when λ1 ≥ λ2 = λ3 and 1/

√2 when

λ1 = λ2 ≥ λ3, corresponding to linear (prolate) andplanar (oblate) shapes, respectively.In intuitive terms, ‖T‖F indicates the overall amountof variability, FA(T) indicates the extent to whichthe variability extends more in some directions thanothers (as opposed to being spherical or wholly ro-tationally symmetric), while skew(λi) indicates thetype or shape of the anisotropy (the precise mannerin which it differs from a sphere).

Additionally, the fraction of variability perpendic-ular to the cortical surface, which we term surface-normal variance, is an important aspect of the co-variance tensor, as it indicates regional differences inbrain shape that may be relevant for understandingdegenerative disease, cortical dysplasias, or subtleabnormalities of cortical shape. We compute surface-normal variance as nTTn/‖T‖F : the tensor contrac-tion of covariance T along surface normal n, normal-ized by Frobenius norm to vary between 0 and 1.

For visualization purposes, the field of covariancetensors was linearly downsampled by a factor of four,because individual glyphs must be large enough tosee in the overall context of the cortical mesh, andbecause the covariance tensors do not change signif-icantly within the span of a few mesh nodes. Theglyphs were drawn slightly offset from the underly-ing mesh surface to better reveal the tensor attributemapped onto the cortical surface.

We have developed the tensor visualization meth-ods described here within BioPSE, a freely availableintegrated problem solving environment for the inter-active investigation of large-scale scientific data [11].The visualization algorithms have been implementedwith modules in a reconfigurable dataflow network,thereby facilitating the exploration of different pa-rameters and techniques.

III. Results

Fig. 3 demonstrates the difference between ellip-soid and superquadric glyphs on the medial side ofthe temporal lobe. Features that are better seen withthe superquadrics include the concentration of pla-nar shapes directly above the collateral sulcus, andthe fact that glyphs with a planar orientation aregenerally tangential to the underlying surface.

Fig. 4 shows a combination of different attributesindicated on and beneath the field of superquadricglyphs. The most striking feature of the visual-izations is that cortical areas with highest variabil-ity (as indicated by FA(T)) are those that devel-oped most recently during human evolution, namelyhigher-order association areas and language cortex.Conversely, primary sensory and motor regions ofthe cortex are the least variable. The fractionalanisotropy of variance is greatest in those brain re-

(a) Ellipsoid tensor glyphs

β = 4

β = 2

β = 1

β = 12

β = 14

α = 14 α = 1

2 α = 1 α = 2 α = 4

(b) Space of superquadrics (c) Superquadric tensor glyphs

Fig. 2. Ellipsoidal glyphs (a) suffer from visual ambiguity. The range of superquadrics (b) used for tensor glyphs is highlightedwith the gray triangle. Superquadric glyphs (c) differentiate shape and convey orientation more clearly than do ellipsoids.

‖T‖F

Fig. 3. Comparison of ellipsoid (top) and superquadric (bot-tom) glyphs, colormapped with ‖T‖F .

gions that are asymmetrical (i.e. differ between leftand right hemispheres), suggesting that strongly di-rectional processes may occur during their formation.

The eigenvalue skew (b) differs markedly betweentemporal and parietal cortex on the lateral surfaceof the brain hemispheres, suggesting that most ofthe non-normal variance near the Sylvian fissure isrelatively isotropic within the cortical sheet, but thisvariance becomes strongly concentrated along one di-rection in the parietal lobe, which is formed later inembryonic development. These principal directionsof variance clearly vary by brain region, as expectedfrom the different times of emergence of the fissuresduring fetal life. This cortical subdivision may helpdefine the anatomical scope of genes that regulatebrain fissuration. Sensorimotor regions are distin-guished by their relatively low variance and their nor-

mal component of variance is relatively high [yellowcolors, (c)], suggesting that these are the only struc-tures whose tangential variability is well accommo-dated by a linear transformation to stereotaxic space.

IV. Discussion and Conclusion

The visualization method presented here is an im-portant step in characterizing anatomic variability inthe brain. Visualizing anatomic covariance tensors,both individually and as a group, illustrates the ba-sic modes of anatomic variation and motivates newstudies to better understand its origins. In light ofthe hypothesis that sulcus formation is a consequenceof tension along white matter tracts [12], future workwill seek to display and quantify the directional rela-tionships between between anatomic variability andunderlying diffusion tensor scans. The visualizationspresented here are geared towards exploration of thetensor values themselves, but additional visualiza-tions can extract or quantify higher-order tensor fieldstructure, for example hyperstreamlines [13] or topo-logical analysis [14].

References

[1] F. Post, F. Post, T. van Walsum, and D. Silver, “Iconictechniques for feature visualization,” in Proc. IEEE Vi-sualization 95, pp. 288–295, 1995.

[2] R. Haber and D. McNabb, Visualization Idioms: AConceptual Model for Scientific Visualization Systems,pp. 74–93. IEEE, 1990.

[3] P. Basser, J. Mattiello, and D. L. Bihan, “MR diffusiontensor spectroscopy and imaging,” Biophysics Journal,vol. 66, no. 1, pp. 259–267, 1994.

[4] P. Thompson and A. Toga, “Detection, visualization andanimation of abnormal anatomic structure with a de-formable probabilistic brain atlas based on random vec-tor field transformations,”Medical Image Analysis, vol. 1,no. 4, pp. 271–294, 1997.

[5] J. Mazziotta, A. Toga, and A. E. et al., “A probabilisticatlas and reference system for the human brain,” Journalof the Royal Society, vol. 356, pp. 1293–1322, Aug. 2001.

[6] P. Thompson, M. Mega, C. Vidal, J. Rapoport,and A. Toga, “Detecting disease-specific patterns ofbrain structure using cortical pattern matching and apopulation-based probabilistic brain atlas,” in Proc. 17thInternational Conference on Information Processing inMedical Imaging (IPMI2001), pp. 488–501, June 2001.

[7] P. Thompson, K. Hayashi, G. de Zubicaray, A. Janke,

FA‖T‖F

(a) Fractional anisotropy colormapped on glyphs, Frobenius norm on cortical surface

FAskew(λi)

(b) Eigenvalue skew colormapped on glyphs, fractional anisotropy on cortical surface

nTTn/‖T‖F‖T‖F

(c) Surface-normal variance colormapped on glyphs, Frobenius norm on cortical surface

Fig. 4. Sequence of quantities colormapped on superquadric glyphs, with gray-scale indication on the underlying mesh surface.

S. Rose, J. Semple, D. Herman, M. Hong, S. Dittmer,D. Doddrell, and A. Toga, “Dynamics of gray matter lossin Alzheimer’s disease,” Journal of Neuroscience, vol. 23,pp. 994–1005, Feb. 2003.

[8] G. Kindlmann, “Superquadric tensor glyphs,” in Proc.IEEE TVCG/EG Symposium on Visualization 2004,pp. 147–154, May 2004.

[9] A. Barr, “Superquadrics and angle-preserving transfor-mations,” IEEE CG & A, vol. 18, no. 1, pp. 11–23, 1981.

[10] P. Basser and C. Pierpaoli, “Microstructural and phys-iological features of tissues elucidated by quantitative-diffusion-tensor MRI,” Journal of Magnetic Resonance,Series B, vol. 111, pp. 209–219, 1996.

[11] BioPSE: Problem Solving Environment for model-ing, simulation, and visualization of bioelectric fields.Scientific Computing and Imaging Institute (SCI),http://software.sci.utah.edu/biopse.html, 2002.

[12] D. V. Essen, “A tension-based theory of morphogenesisand compact wiring in the central nervous system,” Na-ture, vol. 385, pp. 313–318, Jan. 1997.

[13] T. Delmarcelle and L. Hesselink, “Visualizing second-order tensor fields with hyper streamlines,” IEEE CG &A, vol. 13, no. 4, pp. 25–33, 1993.

[14] L. Hesselink, Y. Levy, and Y. Lavin, “The topology ofsymmetric, second-order 3D tensor fields,” IEEE TVCG,vol. 3, Jan–Mar 1997.