rad 1561 local and remote visualization tech- …...inforad 1561 local and remote visualization...

infoRAD 1561

Local and RemoteVisualization Tech-niques for InteractiveDirect Volume Render-ing in Neuroradiology1

Bernd F. Tomandl, MD ● Peter Hastreiter, PhD ● Christof Rezk-Salama,Dipl-Inf ● Klaus Engel, Dipl-Inf ● Thomas Ertl, PhD ● Walter J. Huk, MDRamin Naraghi, MD ● Oliver Ganslandt, MD ● Christopher Nimsky, MDKnut E. W. Eberhardt, MD

The increasing capabilities of magnetic resonance (MR) imaging and mul-tisection spiral computed tomography (CT) to acquire volumetric datawith near-isotropic voxels make three-dimensional (3D) postprocessing anecessity, especially in studies of complex structures like intracranial ves-sels. Since most modern CT and MR imagers provide limited postpro-cessing capabilities, 3D visualization with interactive direct volume ren-dering requires expensive graphics workstations that are not available atmany institutions. An approach has been developed that combines fastvisualization on a low-cost PC system with high-quality visualization on ahigh-end graphics workstation that is directly accessed and remotely con-trolled from the PC environment via the Internet by using a Java client.For comparison of quality, both techniques were applied to several neuro-radiologic studies: visualization of structures related to the inner ear, intra-cranial aneurysms, and the brainstem and surrounding neurovascularstructures. The results of pure PC-based visualization were comparablewith those of many commercially available volume-rendering systems. Inaddition, the high-end graphics workstation with 3D texture-mappingcapabilities provides visualization results of the highest quality. Combininglocal and remote 3D visualization allows even small radiologic institutionsto achieve low-cost but high-quality 3D visualization of volumetric data.

Abbreviations: DICOM � Digital Imaging and Communications in Medicine, MCA � middle cerebral artery, MIP � maximum-intensity projec-tion, SSD � shaded-surface display, 2D � two-dimensional, 3D � three-dimensional

Index terms: Computed tomography (CT), image processing ● Computed tomography (CT), three-dimensional, 10.12117 ● Computed tomography(CT), volume rendering ● Computers ● Magnetic resonance (MR), image processing ● Magnetic resonance (MR), three-dimensional, 10.121419

RadioGraphics 2001; 21:1561–1572

1From the Division of Neuroradiology (B.F.T., W.J.H., K.E.W.E.), Computer Graphics Group (P.H., C.R.S.), and Department of Neurosurgery (P.H.,R.N., O.G., C.N.), University of Erlangen-Nuremberg, Schwabachanlage 6, D-91054 Erlangen, Germany; and the Visualization and Interactive SystemsGroup, University of Stuttgart, Germany (K.E., T.E.). Presented as an infoRAD exhibit at the 2000 RSNA scientific assembly. Received March 12, 2001;revision requested May 23 and received June 6; accepted June 18. Address correspondence to B.F.T. (e-mail: [email protected]).

©RSNA, 2001

IntroductionInteractive three-dimensional (3D) visualizationis a prerequisite for comprehensive analysis andunderstanding of volumetric data from computedtomography (CT) and magnetic resonance (MR)imaging. When one is dealing with a large num-ber of high-resolution images, interactive multi-planar reformation is mandatory for a first inspec-tion of the data (1,2). Investigation of complex3D structures like the intracranial vasculaturerequires 3D visualization, which enables the in-vestigator to follow the vessels and to simulate thedisplay of digital subtraction angiography. Thesemethods are called CT angiography and MR an-giography and are widely used for investigation ofintra- and extracranial vessels (3). In addition, 3Dvisualization is often helpful for surgical planning(4). However, there are no defined standards forapplication of 3D visualization tools, and thesame data will give completely different informa-tion when presented with several visualizationalgorithms like maximum-intensity projection(MIP), shaded-surface display (SSD), and directvolume rendering (5,6). MIP and SSD have beenwidely used for visualization of medical data be-cause these methods provide fast “volume render-ing” by using only about 10% of the image data(7).

Owing to the rapid improvements in computerpower and memory size in recent years, MIP andSSD are being replaced by direct volume render-ing (8–10), which is now available on commercialworkstations, allowing fast and direct 3D visual-ization of the complete information within volu-metric data. There are enormous differences inthe quality of images created with direct volumerendering, which depend on both hardware andsoftware. Low-end, low-cost systems often pro-vide inferior image quality in comparison withhigh-end systems. Since the technical develop-ment of computer hardware and software is in-credibly fast, a visualization system should pro-vide the user with good image quality and be in-expensive and thus easy to replace. New technical

developments like the VolumePro board (Mitsu-bishi Electric, Tokyo, Japan) (11) or GeForcegraphics boards (nVidia, Santa Clara, Calif) willsoon enable real-time volume rendering with highquality even on personal computers.

In this article, two approaches to direct volumerendering are presented, which consist of a low-cost, PC-based direct volume-rendering systemand a high-end graphics computer with hardware-accelerated sophisticated visualization software,remotely controlled from the PC system. Thesesystems are combined into a low-cost, high-end3D visualization tool. This method was applied toseveral neuroradiologic investigations, which arepresented in this article.

Visualization Methods

General ConsiderationsThe images provided by sectional imaging tech-niques (CT, MR imaging, ultrasonography) showthe originally 3D objects in two-dimensional (2D)sections. It is up to the viewer to mentally recon-struct the structures (eg, bones, vessels, cranialnerves) from the section images. For a 3D dis-play, the data have to be transferred to a worksta-tion, where a volume is created out of the sec-tions. Multiplanar reformation is an excellent toolfor getting a better idea about the relations of 3Dobjects within a volume, especially when usedinteractively (12). Although multiplanar reforma-tion is still a 2D method, it has the advantage thatno information is lost. When an isotropic volumeis used, axial, coronal, sagittal, and all kinds ofcurved planes can be reconstructed with the samequality as the source images (2).

There are several techniques for extracting theinformation of interest from a volume. In all ofthese methods, an imaginary ray from a definedviewing position is traced through the volume to ascreen behind it, where the extracted object isdisplayed. The most commonly used methods for3D visualization are MIP, SSD, and direct vol-ume rendering. In the remainder of this section,we present the typical features of these methods,with a focus on the different ways of applying di-rect volume rendering.

1562 November-December 2001 RG f Volume 21 ● Number 6

MIP and SSDMIP and SSD are still widely used for 3D visual-ization of volumetric data (Fig 1a, 1b). A featurethey have in common is that only one layer ofvoxels within the volume is used for visualization(2,9), leading to loss of information. With MIP,only one layer of the brightest voxels parallel tothe viewing ray is used for display. All other infor-mation is ignored. Thus, from a given viewing

direction, it is possible to get some informationabout the brightness of an object (eg, differentia-tion of calcification and a contrast material–filledblood vessel), but the depth information is lost(we cannot say what is in front or behind) (Fig1b).

Figure 1. Demonstration of different results depending on the applied vi-sualization technique. Detailed analysis of an intracranial aneurysm with par-tial thrombosis of the left middle cerebral artery (MCA) was performed, andthe results were compared with intraoperative findings. SSD, MIP, and di-rect volume rendering were used to produce an anterolateral view (skull baseon top). (a) SSD image shows the aneurysm and small MCA branches (ar-rowheads), thus providing good information about the shape of the aneurysmand related vessels (depth information). It is not possible to differentiate thethrombus (arrow) from the blood-filled part of the aneurysm (loss of bright-ness information). (b) MIP image shows the thrombus (arrow), thus provid-ing some brightness information. The MCA branches are not visualized (lossof depth information). (c) High-resolution image created with direct volumerendering shows both the MCA branches (arrowheads) and the thrombus(arrow), thus providing brightness and depth information. (d) Photographshows intraoperative findings. (Fig 1d courtesy of Michael Buchfelder, MD,University of Erlangen-Nuremberg, Germany.)

RG f Volume 21 ● Number 6 Tomandl et al 1563

The opposite is true for SSD. This techniqueuses the first layer of voxels lying within definedthreshold values for display (2,9). Thus, thebrightness information is lost. For example, dif-ferentiation of calcification or thrombosis within ablood vessel is not possible (Fig 1a). However,the depth information is preserved. When an arti-ficial light source is used (shading), SSD gives agood impression of the shape of a 3D object.

Owing to these limitations, image analysisshould not be based on MIP or SSD imagesalone. Since the information provided by MIPand that provided by SSD complement eachother, the methods can be combined if direct vol-ume rendering is not available. Within the medi-cal community, there is still some confusionabout the nomenclature of “volume-rendering”techniques. From the point of view of a computerscientist, MIP and SSD are types of volume ren-dering in which only one layer of voxels is usedfor visualization. In the medical literature, theterm volume rendering is reserved for the techniquedescribed next, in which all voxels of a volume areconsidered for visualization.

Direct Volume RenderingDirect volume rendering uses all of the informa-tion contained in a volume; thus, there is theoreti-cally no unwanted loss of information (9). By as-signing a specific color and opacity value to everyattenuation or signal intensity value of the CT orMR imaging data, groups of voxels are selectedfor display. The information from all of these vox-els is collected along the viewing ray, and the in-formation from every voxel is integrated into theresulting image on the screen. Meaningful 3Drepresentations of even overlying structures likeintracranial vessels within the skull are possible,depending on the selected opacity (high opacityproduces low transparency and vice versa). Bothbrightness information and depth information arepresented (Fig 1c).

There are various rendering algorithms forboth the PC and the high-end workstation. Thepopular “shear-warp” approach (13) is often usedin commercially available workstations. In thisapproach, the sections of a volume are set parallelto the coordinate axes of the rectilinear volumedata grid (“object-aligned” sections) (Fig 2a). A2D texture-based variant of the shear-warp algo-rithm was used for our local PC-based direct vol-

Figure 2. Effect on image quality of different algorithms for direct volume rendering. (a) Top:Diagram shows an approach based on bilinear (2D) interpolation, which requires three copies ofthe original volume consisting of sections in three orthogonal directions (object aligned). The vol-ume with sections most parallel to the viewing plane (the view port) is chosen, depending on theviewing direction. Bottom: Anteromedial view of the inner ear created from spiral CT data withthis approach clearly shows line artifacts (arrows). The best image quality is achieved when theviewing direction is parallel to one of the planes. (b) Top: Diagram shows trilinear (3D) interpola-tion, which allows rendering of sections parallel to the viewing plane, independent of the viewingdirection (“view port aligned”). Bottom: Corresponding anteromedial view shows the structures ofthe inner ear without artifacts.


ume-rendering system. This approach requires usto keep three copies of the data set in the mainmemory, one set of sections for each section di-rection. Despite the high memory requirements,the major drawback of the 2D texture-basedimplementation is the missing spatial interpola-tion. As a result, the images contain visual arti-facts, especially when small objects are visualizedand zoomed closely (Fig 2a). The image appear-ance can be improved by using 2D multitextureinterpolation, in which additional sections arecalculated (14).

If 3D textures are supported by hardware, it ispossible to render sections parallel to the imageplane with respect to the current viewing direc-tion (Fig 2b). However, if the viewing matrixchanges, these view port–aligned sections must berecomputed. Since trilinear texture interpolationis supported by the hardware of the remote work-station, it can be performed at interactive framerates without distracting artifacts (15).

Materials and Methods

Local Volume RenderingFor local rendering, a desktop PC (Pentium IIIprocessor [Intel, Santa Clara, Calif], 500 MHz,128 Mbytes of random-access memory) equippedwith a GeForce256 graphics adapter (nVidia)with 32 Mbytes of double-data random-accessmemory was used as the local client. The clientapplication was implemented with Java andJava3D. Java, a programming language for theInternet (16), allows one to use the same applica-tion from any kind of machine: a PC, a Macin-tosh (Apple Computer, Cupertino, Calif), a net-work computer, or even new technologies likepersonal digital assistants (PDAs) or Internetscreen phones (17). This platform independence

allows visualization of volume data with any de-vice with minimal graphics capabilities, anywherein the world. Local rendering makes use ofJava3D, the 3D extension of the Java program-ming language. An approach similar to the shear-warp approach that makes use of locally available,low-cost 2D texture-mapping hardware was real-ized. First, the data from CT and MR imagingwere electronically retrieved from the imagers viaa local-area network and a Digital Imaging andCommunications in Medicine (DICOM) inter-face (Fig 3). The personal data of the patientswere removed from the image headers for dataprotection and privacy.

The Java client displays the data in a sectioningtool, which allows inspection of the section im-ages. A subregion of the data can be selected andvisualized in 3D with the Java3D renderer. Aframe rate of three to eight frames per second ispossible with this setup, depending on the datasize.

Since Java3D is a platform-independent graph-ics programming interface, it does not allow useof the special features of modern low-cost graph-ics adapters. The advances of these graphicsadapters, which were mainly developed for com-puter games, are enormous. To make use of thenewest rendering features, like multitextures, wealso developed a client based on the C�� pro-gramming language and the OpenGL standardthat allows us to access the graphics hardwaredirectly, without the additional abstraction layerintroduced by Java3D. This client provides evenbetter image quality and rendering speed due tothe use of multitexturing hardware for trilinearinterpolation (14).

Figure 3. Diagram of the data stream in a local network. The DICOM imagesobtained with a CT scanner and MR imager are transferred to the local PC. Then,the image data are transferred to the visualization (Vis) server, which is remotelycontrolled from the PC environment via the Java client. FTP � file transfer proto-col, GUI � graphical user interface, IRIX SGI � IRIX operating system (SiliconGraphics, Mountain View, Calif), NT � Windows NT (Microsoft, Redmond,Wash).


This approach allows use of a low-cost PCconnected to the MR imager and CT scanner andprovides the user with good interaction capabili-ties due to small latencies. However, it yields re-duced image quality when compared with special-purpose high-end graphics hardware.

To enable the user to access remotely availablehigh-end graphics hardware, the Java client alsoallows one to display and interact with remotelygenerated images (Fig 4). For this purpose, thevolume data have to be transferred from the localmachine to the remote visualization server.

Remote Volume RenderingThe remote 3D visualization was performed onan Octane workstation (R10000 processor, 250MHz) with MXE graphics hardware (SiliconGraphics), which provides 4 Mbytes of texturememory. Rendering was performed with the 3Dtexture-mapping hardware of the graphics work-station, which allows high image quality and highinteraction rates. These results are possible due tothe trilinear interpolation of the original data pro-vided by the 3D texture hardware and the high-resolution shading provided by postinterpolationlookup tables.

The local PC uploads the data to the remotegraphics workstation via the Internet. In oursetup, we used a fast Internet connection (155Mbits/sec) provided by the German research net-work. For a CT angiography data set of 150 im-ages, about 3 minutes of upload time was neces-sary. Once the data are loaded into the graphics

server, the visualization process is interactivelycontrolled from the local PC via the Java-basedclient (18) (Fig 3). Mouse interactions and key-board input are sent from the local PC to theworkstation, which immediately renders new im-ages and sends a stream of resulting images backto the PC (19). The images are compressed forthe network transfer to allow remote visualizationon low-bandwidth network connections. We testeda number of different connections, ranging fromhigh-speed research networks down to IntegratedServices Digital Network (ISDN) connections.

The data size and image size of this setup allowframe rates of up to 10 frames per second, de-pending on the data exchange rate. In our experi-mental setup, with the local PC in Erlangen, Ger-many, and the workstation located in Stuttgart,200 km away, we achieved an average frame rateof three frames per second using standard Inter-net access via the German research network.When this approach was presented at the RSNAmeeting in Chicago, Ill, in November 2000, aframe rate of about one frame per second wasachieved between the PC located in Chicago andthe workstation in Stuttgart.

Remote volume rendering allows the sharing ofexpensive remote graphics hardware via the Inter-net by using a simple client computer. By con-necting more clients to the rendering server, thecollaboration of many experts from different loca-tions is possible. However, owing to signal propa-gation delays, latencies are introduced, which arenoticeable as a delayed reaction to user input.Especially for the remote visualization setup inChicago, this was the main drawback. Combiningthe local and remote visualization approaches canprevent these latencies. During interaction, local

Figure 4. Anterolateral view of the inner ear created from CT data with direct volume rendering.Computer screen shows the results of local (left) and remote (right) rendering within a Microsoft Win-dows environment. The difference in image quality is obvious. Although the local renderer offers real-time interactivity, the performance of the remote renderer depends on the Internet connection. In ourexperimental setup, frame rates of three frames per second could be achieved.


graphics hardware is used to provide low latenciesand fast rendering. After the user stops interaction,an image with high resolution and high quality isrequested from the remote graphics workstation.

Neuroradiologic StudiesFor comparison of the local and remote directvolume-rendering systems, we used the CT an-giography data of five patients with intracranialaneurysms and the MR imaging data of two pa-tients who had been examined for suspected neu-rovascular compression syndromes. The CTstudies were performed on a multisection spiralCT scanner (Somatom Plus4 VZ; Siemens, Er-langen, Germany) with the following parameters:collimation � 4 � 1 mm, table feed � 2.7 mm perrotation, section thickness � 1.25 mm, increment �0.5 mm, field of view � 120 mm2, 100 mL ofcontrast medium, flow rate of 4 mL/sec. The MRimaging studies were performed on a 1.5-T sys-tem (Magnetom Symphony; Siemens) by using a3D constructive interference in the steady state(CISS) sequence with 0.7-mm-thick sections.

For the CT angiographic studies, additionalreconstruction of the temporal bones was per-formed (field of view � 60 mm2, high-resolutionkernel) so that these data could be used for visual-ization of the structures of the inner ear.

ResultsTo demonstrate the difference in visualizationquality between the two volume-rendering sys-tems, we applied the methods to visualize thestructures of the inner ear and intracranial aneu-rysms from spiral CT data and for 3D display ofneurovascular structures surrounding the brain-stem from MR imaging data. Three-dimensionalvisualization of the inner ear from CT data per-formed with direct volume rendering without ex-plicit segmentation requires optimal quality toobtain clear and meaningful images (20). Thus,this is a suitable application for getting an impres-sion of the visualization capabilities of a directvolume-rendering system. As mentioned earlier,we used spiral CT scans of the normal temporalbone. The small field of view of 60 mm2 not onlyleads to better in-plane resolution but also allowspreselection of the subvolume containing mainlythe required information. Figures 2 and 4 demon-strate the results achieved with both systems. Al-though the PC-based system provided a coarsedisplay of the inner-ear structures due to interpo-lation artifacts, these structures can be recognizedmuch more clearly on images created with thehigh-end graphics system by using trilinear inter-polation.

To investigate the use of the two systems underroutine conditions, the CT angiography data ofthe five patients with intracranial aneurysms weretransferred to both the local and remote worksta-tions directly after the investigation. The averagetime for the complete visualization process in ourexperimental setup, including data transfer, was15–25 minutes, depending on the complexityof the vascular structures. This is already fastenough for this method to be used in the clinicalsituation of a patient with subarachnoid hemor-rhage under emergency conditions. CT angiogra-phy is a well-known tool for therapy planning ofintracranial aneurysms. In contrast to the resultsachieved with digital subtraction angiography, notonly the shape of the aneurysms but also the rela-tion to the skull base, thromboses, and calcifica-tions are clearly demonstrated (21–24). An opti-mal quality of 3D images is mandatory for ex-tracting all available information from the data.As demonstrated in Figure 1, MIP and SSD canbe used in combination to demonstrate both thebrightness and depth information of the data.However, only direct volume rendering has thecapability of showing the complete information ina single 3D image. For therapy planning, high-quality close-up views are important.

In our setup, the PC-based system was used toget fast information about the shape and locationof an aneurysm with real-time interaction. Oncethe ideal viewing position for therapy planningwas found on the PC system, the selected areawas remotely evaluated on the high-end systemand high-resolution 3D images were created. Theperformances of both systems in visualization ofintracranial aneurysms were compared (Figs 5,6). Although the PC-based system allowed veryfast, real-time visualization of good quality, theclose-up views were still disturbed by line arti-facts. The high-end workstation produced clearerimages that showed more detail information, es-pecially on the semitransparent images. The abil-ity to create semitransparent images that showeven structures lying behind the aneurysm with-out choosing another viewing position is an ad-vantage of the remote rendering system. This ca-pability can be used to demonstrate hidden struc-tures from the viewpoint of the neurosurgeon’ssurgical microscope. Of course, the ability to in-teractively move the structures rather than look-ing at static images like those in Figure 6 supportsdetail analysis.


Figure 5. Comparison of two different direct volume-rendering systems applied to visualization of anintracranial aneurysm of the left MCA (arrow). (a) Superior opaque view created with the remote high-end system shows the vessels and skull base. (b) Corresponding view created with the PC-based systemshows more artifacts within the skull base. There is no significant difference in quality in the visualizationof vascular structures.

Figure 6. Comparison of two different direct volume-rendering systems applied to detailed analysis of an intracra-nial aneurysm of the MCA. All images are lateral views. (a) Image from the intraoperative video shows the neurosur-geon’s view. The large aneurysm hides adjacent vessels, thus necessitating surgical manipulations to obtain more in-formation. (Courtesy of Christian Strauss, MD, University of Erlangen-Nuremberg, Germany.) (b) Diagram showsthe aneurysm and adjacent vessels. (c) Opaque image created with PC-based direct volume rendering clearly showsthe shape of the aneurysm. However, the image quality is reduced by line artifacts. (d) Opaque image created withthe suggested remote direct volume-rendering approach shows the aneurysm without visible artifacts. (e) Trans-parent image created with PC-based direct volume rendering shows only a rough outline of the vasculature behindthe aneurysm due to artifacts. (f) Transparent image created with remote direct volume rendering shows a view“through” the aneurysm, thus providing additional information about the adjacent vasculature.


To evaluate the capabilities of the two visual-ization systems for MR imaging data, we used thedata of the two patients investigated for suspectedneurovascular compression syndromes (25). Fora better understanding of the 3D relationships ofthe small neurovascular structures in the area ofthe brainstem, we used direct volume renderingfollowing semiautomatic segmentation, which

created subvolumes of the brainstem and thestructures within the cerebrospinal fluid space(26). The PC-based system was able to createmeaningful 3D images of sufficient quality thatclearly demonstrated the relationships of theneurovascular structures (Fig 7). However, the

Figure 7. Three-dimensional visualization of the cranial nerves from the MR imaging data oftwo patients. Frontal (left) and frontolateral (right) views were created. Two subvolumes had beencreated prior to the visualization, which contained the brainstem and the structures within the ce-rebrospinal fluid space. (a) Image created with PC-based direct volume rendering shows the largerstructures adjacent to the brainstem, such as the oculomotor nerves (arrows), as well as the spacebetween the posterior cerebral artery and superior cerebellar artery. It is not possible to visualizevery small structures like the trochlear nerves. (b) High-resolution image created with remote di-rect volume rendering shows the brainstem and related structures. Even very small structures likethe trochlear nerve (thin arrow) and the abducent nerves (thick arrows) are visible. The remotesystem allows assignment of different colors to differentiate nerves (yellow) and vessels (red).


close-up views demonstrated the typical line arti-facts, so that very small structures like the troch-lear nerve could not be identified. The imagescreated with the high-end system demonstratedthe trochlear nerve in both cases, thus allowinghigher-quality visualization.

Both systems allowed interactive manipulationof the data, with some latency when the remotesystem was used. Nevertheless, frame rates ofthree frames per second could be achieved, whichis sufficient for fine-tuning following fast, real-time visualization of the volume on the PC-basedsystem.

DiscussionThe quality of 3D visualization depends on mul-tiple factors. Of highest importance is the qualityof the source images. Although in-plane resolu-tions of less than 0.5 mm are easily achieved withmodern CT scanners and MR imagers by usinga 5122 matrix and a narrow field of view, thethrough-plane resolution in most studies will begreater than or equal to 1 mm. Therefore, in clini-cal practice, real isotropic voxels in a volumetricdata set are an exception. When spiral CT isused, this problem can be overcome to some de-gree by using smaller reconstruction increments(eg, 0.5 mm), leading to higher visualization qual-ity at the cost of a greater number of images; how-ever, at this time, this technique is reasonableonly for small volumes to allow interactive ma-nipulation within the renderer. For small vol-umes, even a collimation of 0.5 mm is technicallypossible when multisection CT is used (1).

Use of a narrow field of view not only provideshigher in-plane resolution but also leads to pre-segmentation of the volume, reducing it to data ofdiagnostic interest and thus facilitating 3D visual-ization. It is often useful to create two data sets,with the first containing the whole examined vol-ume and the second containing mainly the infor-mation of interest. When using CT angiographyfor investigation of intracranial aneurysms, wefirst reconstruct the raw data from the wholecircle of Willis (field of view � 120 mm2) andthen create a data set containing the detected an-

eurysm within a narrow field of view (eg, 60mm2), which facilitates detail analysis. An opti-mal volume would contain nothing more than thestructures to be visualized.

It is often stated that with volume renderingthere is no information loss from the volume dataand extensive editing and segmentation are notnecessary (8,9). In fact, at this time, some of thecommercially available products are not able tohandle large numbers of high-resolution images;therefore, reduction of the image matrix to 2562 isoften used for faster manipulation of the data andthus less need of memory. Obviously, this tech-nique leads to information loss and reduced qual-ity of the 3D images, which are unfavorable fac-tors when dealing with small objects, as in neuro-radiologic studies.

The quality of the 3D images depends stronglyon the applied hardware and software. There is adifference between rendering algorithms like theshear-warp algorithm (13), which is based on 2Dtexture mapping, and the more challenging 3Dtexture mapping used in our remote workstation,which requires specific hardware acceleration(15). At this time, only 3D texture mapping re-sults in artifact-free visualization of small struc-tures due to the calculation of parallel planes ac-cording to the viewing position. The quality ofvisualization based on 2D texture mapping can beenhanced by the interpolation of intermediatesections, leading to clearer images (14).

When one is dealing with very small objects,like the examples shown in this article, some kindof segmentation will always be necessary (6). Inmost cases, segmentation can be performed suffi-ciently by using clip planes, which can be usedinteractively to eliminate parts of the volume togive a clearer view of deeper-lying structures. Forvisualization of intracranial arteries from CT an-giography data, it is always necessary to eliminatethe upper parts of the venous system to give a freeview to the basilar artery when one is lookingfrom above. Direct visualization of the cranialnerves from MR imaging data was possible tosome degree, but the results were not sufficient toprovide enough information about the relation-ships of the neurovascular structures. By semiau-tomatically creating two subvolumes of the brain-stem and the surrounding cerebrospinal fluid


space, the visualization results were of much bet-ter quality.

The presented approach offers many advan-tages. First, it is cost-effective, since expensivespecialized workstations are no longer necessaryfor every institution. Owing to the mass market ofcomputer games and entertainment software, PCgraphics accelerator boards have become moreflexible and powerful. A low-cost PC systemequipped with a high-end graphics adapter (costabout $300) is sufficient to provide satisfactorydirect volume-rendering visualization for routinestudies. In the near future, even hardware-accel-erated 3D texture mapping will be possible onstandard PC systems (14). In cases in which thehighest quality is essential, the high-end hardwareand software can be accessed from the local sys-tem via the Java client. Since many users can usethe high-end workstation, conferences about casestudies are possible. Remote rendering allows ac-cess to expensive high-end graphics workstationswith any Java-enabled device from anywhere inthe world (16,17). Therefore, it is theoreticallypossible to view and manipulate the images via amobile phone. The client has to be able only todisplay remotely generated images and send inter-action events. Remote rendering introduces ahigher latency due to the transfer of user com-mands from the client to the server and the trans-fer of rendered images back to the client. Toavoid this latency, we experimented with a combi-nation of local and remote rendering. During in-teraction with the data, local 3D graphics hard-ware is used for image generation. After the inter-action, a high-quality image from the server isrequested and displayed (18).

Remotely controlled visualization also allowsthe neurosurgeon to obtain a position-controlledview of the surgical situation (22) without theneed for expensive high-end hardware. It will alsobe possible to integrate this information into thesurgical microscope of the microsurgeon.

Although interactive use of 3D visualization isoften helpful, it has the limitation of being userdependent and thus leading to inaccurate results(9). In the future, standardized visualizationtools, with which defined video sequences areproduced that can be evaluated for sensitivity andspecificity, should replace individual interactive3D visualization. These standard tools can beprovided by remotely accessed workstations. Inaddition, specialists who could also provide thelocal institution with a report could evaluate thefindings.

ConclusionsThe approach of combining a local, low-cost, PC-based volume-rendering system with a remotelyaccessed high-end graphics workstation makesthis fascinating and promising tool for 3D visual-ization available to all institutions dealing withvolume data. In the near future, even high-qualityvisualization will be possible on standard personalcomputers equipped with modern graphics adapt-ers. The ability to access these workstations fromalmost anywhere will help radiologists developnew applications for standardized 3D visualiza-tion.

References1. Rydberg J, Buckwalter KA, Caldemeyer KS, et al.

Multisection CT: scanning techniques and clinicalapplications. RadioGraphics 2000; 20:1787–1806.

2. Blank M, Kalender WA. Medical volume explora-tion: gaining insights virtually. Eur J Radiol 2000;33:161–169.

3. Schwartz RB, Tice HM, Hooten SM, Hsu L, StiegPE. Evaluation of cerebral aneurysms with helicalCT: correlation with conventional angiographyand MR angiography. Radiology 1994; 192:717–722.

4. Kikinis R, Gleason PL, Moriarty TM, et al. Com-puter-assisted interactive three-dimensional plan-ning for neurosurgical procedures. Neurosurgery1996; 38:640–649.

5. Johnson PT, Halpern EJ, Kuszyk BS, et al. Renalartery stenosis: CT angiography—comparison ofreal-time volume-rendering and maximum inten-sity projection algorithms. Radiology 1999; 211:337–343.

6. Udupa JK. Three-dimensional visualization andanalysis methodologies: a current perspective. Ra-dioGraphics 1999; 19:783–806.

7. Heath DG, Soyer PA, Kuszyk BS, et al. Three-dimensional spiral CT during arterial portography:comparison of three rendering techniques. Radio-Graphics 1995; 15:1001–1011.

8. Rubin GD, Beaulieu CF, Argiro V, et al. Perspec-tive volume rendering of CT and MR images: ap-plications for endoscopic imaging. Radiology1996; 199:321–330.

9. Calhoun PS, Kuszyk BS, Heath DG, Carley JC,Fishman EK. Three-dimensional volume render-ing of spiral CT data: theory and method. Radio-Graphics 1999; 19:745–764.

10. Pretorius ES, Fishman EK. Spiral CT and three-dimensional CT of musculoskeletal pathology:emergency room applications. Radiol Clin NorthAm 1999; 37:953–974.

11. Pfister H, Hardenbergh J, Knittel J, Lauer H,Seiler L. The VolumePro real-time ray-castingsystem. In: Proceedings of SIGGRAPH ’99. NewYork, NY: Association for Computing Machinery,1999; 251–260.


12. Prokop M. Multislice CT angiography. Eur J Ra-diol 2000; 36:86–96.

13. Lacroute P, Levoy M. Fast volume rendering us-ing a shear-warp factorization of the viewing trans-formation. In: Proceedings of SIGGRAPH ’94.New York, NY: Association for Computing Ma-chinery, 1994; 451–458.

14. Rezk-Salama C, Engel K, Bauer M, Greiner G,Ertl T. Interactive volume rendering on standardPC graphics hardware using multi-textures andmulti-stage-rasterization. In: Proceedings of SIG-GRAPH ’00. New York, NY: Association forComputing Machinery, 2000; 109–118,147.

15. Ertl T, Hastreiter P. Visualization of scalar volumedata. In: Girod B, Greiner G, Niemann H, eds.Principles of 3D image analysis and synthesis.Boston, Mass: Kluwer Academic, 2000; 261–269.

16. Fernandez-Bayo J, Barbero O, Rubies C, SentisM, Donoso L. Distributing medical images withInternet technologies: a DICOM Web server and aDICOM Java viewer. RadioGraphics 2000; 20:581–590.

17. Lee SK, Peng CH, Wen CH, Huang SK, JiangWZ. Consulting with radiologists outside the hos-pital by using Java. RadioGraphics 1999; 19:1069–1075.

18. Engel K, Hastreiter P, Tomandl B, Eberhardt K,Ertl T. Combining local and remote visualizationtechniques for interactive volume rendering inmedical applications. In: Proceedings of IEEE Vi-sualization ’00. Piscataway, NJ: Institute of Elec-trical and Electronics Engineers, 2000; 449–452.

19. Engel K, Sommer O, Ertl T. A framework for in-teractive hardware accelerated remote 3D-visual-

ization. In: de Leeuw WC, van Liere R, eds. Datavisualization 2000. Vienna, Austria: Springer-Ver-lag Wien, 2000; 167–177,291.

20. Tomandl BF, Hastreiter P, Eberhardt KE, et al.Virtual labyrinthoscopy: visualization of the innerear with interactive direct volume rendering. Ra-dioGraphics 2000; 20:547–558.

21. Korogi Y, Takahashi M, Katada K, et al. Intracra-nial aneurysms: detection with three-dimensionalCT angiography with volume rendering—com-parison with conventional angiographic and surgi-cal findings. Radiology 1999; 211:497–506.

22. Wilkinson EP, Shahidi R, Wang B, Martin DP,Adler JR, Steinberg GK. Remote-rendered 3D CTangiography (3DCTA) as an intraoperative aid incerebrovascular neurosurgery. Comput AidedSurg 1999; 4:256–263.

23. Velthuis BK, Rinkel GJ, Ramos LM, et al. Sub-arachnoid hemorrhage: aneurysm detection andpreoperative evaluation with CT angiography. Ra-diology 1998; 208:423–430.

24. Villablanca JP, Martin N, Jahan R, et al. Volume-rendered helical computerized tomography an-giography in the detection and characterization ofintracranial aneurysms. J Neurosurg 2000; 93:254–264.

25. Johnson D, Coley SC, Brown J, Moseley IF. Therole of MRI in screening for neurogenic hyperten-sion. Neuroradiology 2000; 42:99–103.

26. Hastreiter P, Tomandl B, Eberhardt K, Ertl T.Interactive and intuitive visualization of small andcomplex vascular structures in MR and CT. In:Proceedings of the Engineering in Medicine andBiology Society (EMBS), Institute of Electricaland Electronics Engineers. Vol 20. New York,NY: Springer-Verlag, 1998; 532–535.


rad 1561 local and remote visualization tech- …...inforad 1561 local and remote visualization...

Documents