Conformal Visualization for Partially-Immersive PlatformsKaloian Petkov∗ Charilaos Papadopoulos† Min Zhang‡ Arie E. Kaufman§ Xianfeng Gu¶

Computer Science Department, Stony Brook University, Stony Brook, NY, USA

ABSTRACT

Current VR systems such as the CAVE provide an effective plat-form for the immersive exploration of large 3D data. A major lim-itation is that in most cases at least one display surface, such asa ceiling or a back wall, is missing due to space, access or costconstraints. This partially-immersive visualization results in a sub-stantial loss of visual information that may be acceptable for someapplications; however, it becomes a major obstacle for critical tasksthat need to utilize the users’ entire field of vision. We have devel-oped a conformal rendering pipeline for the visualization of datasetson partially-immersive platforms. The angle-preserving conformalmapping approach is used to render the 360◦view directly onto ar-bitrary display configurations. It has the desirable property of pre-serving shapes locally, which is important for identifying shape-based features in the data. Our conformal visualization technique isapplicable to rasterization, volume rendering and real-time raytrac-ing, and in contrast to image-based retargeting approaches, it con-structs accurate, artifact-free stereoscopic images. We demonstrateour conformal visualization pipeline in the 5-sided CAVE for Im-mersive Virtual Colonoscopy, as well as architectural and abstractdata exploration. Our user study shows that on the visual polyp de-tection task, conformal visualization leads to improved sensitivityat comparable examination times against the traditional renderingapproach.

Index Terms: I.3.3 [Computer Graphics]: Picture/ImageGeneration—Display Algorithms; I.3.7 [Computer Graphics]:Three-Dimensional Graphics and Realism—Virtual Reality

1 INTRODUCTION

A number of visualization technologies have been developed for theimmersive exploration of large-scale complex data. Prime exam-ples are the CAVE [3] and Head-Mounted Displays (HMD) whichprovide a much larger field of view into a virtual environment com-pared to traditional desktop systems, and also use stereoscopic pairsof images to improve the perception of spatial relationships. WhileHMDs allow for arbitrary views in the virtual world, they are usu-ally bulky, wired and easily lead to eye fatigue. At the same time,CAVEs provide a much more natural visualization without the needfor a virtual avatar and allow for Mixed Reality applications. Build-ing a fully enclosed CAVE, however, remains a difficult task. Al-though such installations exist [5], they present an engineering chal-lenge in terms of cost, facility access, as well as head and gesturetracking.

Immersion in the virtual data is a function of different factors,such as sensory perceptions, interaction techniques and realism ofthe visualization. We focus on visual immersion, defined by the

∗email: kpetkov@cs.stonybrook.edu†email: cpapadopoulo@cs.stonybrook.edu‡email: mzhang@cs.stonybrook.edu§email: gu@cs.stonybrook.edu¶email: ari@cs.stonybrook.edu

field of view coverage afforded by the physical visualization plat-form, as well as the coverage of the virtual scene provided by thevisualization software. The disadvantage of partially-immersive en-vironments, such as CAVEs with at least one missing display sur-face, is that important visual information may be lost. While manyapplications may tolerate one or more missing projection screens,this partial loss of visual context adversely affects the general nav-igation capabilities of the user and becomes a critical issue in theexploration of medical data.

We have developed a visualization approach that utilizes confor-mal mapping to modify scene geometries or viewing rays at run-time. As a result, the full virtual environment can be displayedon a partially-immersive visualization platform, for example a 5-sided CAVE, without the artifacts typically associated with image-based retargeting approaches. In mathematics, the conformal mapis an angle preserving function that describes a mapping betweentwo Riemannian surfaces [25]. Intuitively, it allows us to map thegeometry of a 6-sided CAVE to an arbitrary configuration of dis-play surfaces that is topologically equivalent to a disk, such as a 5-sided CAVE or a non-planar arrangement of workstation displays.This mapping is then used to transform the viewing directions dur-ing rendering. The main advantage of using a conformal map todefine the transformation is the guarantee that shapes will be pre-served locally even though distances will not be. This is particularlybeneficial for the exploration of medical data, such as in VirtualColonoscopy (VC) where potentially cancerous polyps are detectedby the radiologist based on their shape.

The following are the specific contributions of our work:

• Conformal mapping is used to obtain a shape-preservingtransformation that can map a 360◦field of view to an arbitrarydisplay configuration. In medical visualization, we demon-strate that the shape of the polyps is preserved under the dis-tortion for Immersive Virtual Colonoscopy.

• The conformal distortion is applied during rendering for ras-terization, direct volume rendering and ray-tracing. As a re-sult, we construct stereo image pairs for all three renderingmodalities using a unified transformation definition.

• The performance penalty for applying the transformation isapproximately 1% and requires minimal pre-computation.

• Our approach is applied to the visualization of medical, ar-chitectural and abstract data in a 5-sided CAVE. An informaluser study shows an improvement in the task of visual polypdetection in phantom colonoscopy datasets.

2 RELATED WORK

Immersive visualization systems allow the user to explore data innovel ways that go beyond the standard 2D images on a worksta-tion. These platforms provide a superior depiction of the informa-tion via a significantly wider field of view and enhanced depth andshape perception. The first such environment, the CAVE [3] offersan immersive experience using back-projected images on 3 wallsand front projection on the floor. Other display arrangements havebeen proposed, including the 5-sided Immersive Cabin (IC) [20]and the 6-sided CAVE [5]. Compared to Head-Mounted Displays(HMD), these environment provide a more natural visualization and

allow users to interact with visualization-augmented physical ob-jects, following the paradigm of Augmented Virtuality [15], or moregenerally, Mixed Reality. However, building fully-immersive facil-ities is expensive and introduces many challenges in terms of headtracking, sound systems and even air circulation. Our conformalvisualization system uses the discrete Ricci flow [8] to compute theRiemann mapping and maps the full 360◦spherical field of viewto partially-immersive platforms. The main advantage is that thelocal shape of the projected scene geometry and the features in vol-ume data are preserved under the distortion. The Ricci flow methodhas found a broad range of applications in the graphics and visual-ization fields, including optimal surface parameterization [30, 28],shape analysis [9] and surface matching [11].

Least square conformal maps (LSCM) is an alternative heuristictechnique for computing a conformal mapping [14]. Unlike Ricciflow, which automatically produces seam-free global conformal pa-rameterizations, LSCM and other computational approaches mayrequire special algorithms or heuristic inputs when dealing withcertain topologies [8]. The main disadvantage of LSCM is thatit cannot control the boundary of the region. In our case, we re-quire a mapping between the visibility boundary of the CAVE andthe unit circle. Therefore, LSCM is not a viable technique for ourapplication. Also, unlike the approach by Springborn et al. [23],the Ricci flow is a stable algorithm with a theoretical assurance ofconvergence.

Raskar et al. have used the LSCM to display the output of anad-hoc cluster of projectors onto an arbitrary set of surfaces [21].Our conformal visualization solves a different problem: given a setof display surfaces and a visualization system that produces the cor-rect projections (e.g., partially-immersive CAVE or an arrangementof LCD displays), we want to compute a mapping between the fullvisibility sphere and the visibility area provided by the display sys-tem. The user is able to visualize the entire data at once on thetarget platform, while the properties of the mapping guarantee thatshapes are locally preserved. Our application of conformal map-ping is fundamentally different and requires the use of advancedrendering techniques beyond texture projection.

Implementing the conformal distortion for real-time visualiza-tion requires the rendering of a non-linear, non-pinhole projection,which is not currently supported on the GPU. A number of image-based and geometry-based techniques have been proposed. Image-space approaches [29] suffer from linear sampling artifacts and pro-vide no guarantees with regard to shape preservation on the dis-torted image, which is imperative for medical screening proceduressuch as VC.

Focus and Context (F+C) techniques such as magic lenses [18,27] are designed for a single projection surface and do not trans-late directly to immersive environments with multiple and possiblynon-planar displays. Illustrative deformations for data explorationhave been proposed [2] which could be used to warp data that liesaround the CAVE volume, however they also do not provide shapepreservation. The technique presented by Lorenz and Dollner [13]handles piecewise approximation of non-planar perspective projec-tions on the GPU. Non-planar projections can be used to define aprojection surface that ”wraps” around the CAVE, including part ofthe ceiling and thus recovering the non-visualized part of the data.The technique can be applied in either image-space or geometry-space, however in the first case the sampling artifacts can impactvisual quality significantly [26], while the geometry approach doesnot scale well with mesh density [13].

Our conformal visualization is applicable to a number of visual-ization tasks, however in the user study of its effectiveness we focusprimarily on medical visualization. Virtual Colonoscopy (VC) hasbeen established as a non-invasive alternative to traditional opticalcolonoscopy (OC) for cancer screening [6, 10]. A VC session in-volves the acquisition of computed tomography (CT) scans of the

patient’s abdomen and the extraction and visualization of the colonsurface via segmentation and volume rendering techniques. Tra-ditional VC covers only about 91% of the colon surface after fullnavigation in both the antegrade and the retrograde directions [7]and the percentage is significantly lower for a single direction. Oneshortcoming of existing partially-immersive configurations for Im-mersive Virtual Colonoscopy (IVC) is that the missing projectionsmay hide a significant amount of information. While this may beacceptable in certain applications, it becomes a critical issue for theexploration of medical data. Our approach allows the radiologist toexamine the entire surface of the colon in a single pass and ensuresthat the shape of any colon polyps, which is crucial for the detectionof colon cancer, is preserved in the conformal visualization.

3 THEORETICAL BACKGROUND

According to the Riemann mapping theorem, simply connected sur-faces with a single boundary can be mapped onto the planar disk,such that the mapping is angle-preserving. Locally, the mappingis only a scaling and therefore it is also shape-preserving. This isadvantageous in our rendering technique as it maps the viewing di-rections from the 6 original projections in the fully-enclosed CAVEto the 5-sided CAVE while preserving the local shapes. Our com-putational algorithm is based on the surface Ricci flow theorem [1].

3.1 Conformal MappingLet S1 and S2 be two surfaces with Riemannian metrics g1 and g2,and let ϕ : (S1,g1)→ (S2,g2) be a homeomorphism between them.We say ϕ is conformal, if the pull back metric tensor induced by ϕon the source differs from the original metric by a scalar:

ϕ∗g2 = e2λ g1,

where λ : S1→ R, is a function. The following Riemann mappingtheorem plays a fundamental role in the current work:

Theorem 3.1 (Riemann Mapping) Suppose a surface S is simplyconnected with a Riemannian metric. There exist conformal map-pings ϕ : S→D, where D is the unit disk on the complex plane andall such mappings differ by a Mobius transformation.

A Mobius transformation of a point z on the complex plane isgiven by

z→ eiθ z− z0

1− z0z,

where θ and z0 ∈ C are constants.In order to compute the mapping ϕ , one can compute the pull

back metric first, which can be achieved by the surface Ricci flow.

3.2 Ricci FlowA surface Ricci flow is the process used to deform the Riemannianmetric of the surface. The deformation is proportional to Gaus-sian curvatures so that the curvature evolves in a manner similarto heat diffusion. In mathematics, it is a powerful tool for findinga Riemannian metric satisfying the prescribed Gaussian curvature.Chow and Luo have described the theoretic foundation for the dis-crete Ricci flow on surfaces [1], and Jin et al. have developed anefficient computational algorithm [8].

Let Σ = (V,E,F) be a triangular mesh embedded in R3, whereV , E and F are respectively the sets of vertices, edges, and faces. Adiscrete Riemannian metric on Σ is a piecewise constant metric withcone singularities at the vertices. The edge lengths are sufficient todefine a discrete Riemannian metric,

l : E→ R+, (1)

as long as, for each face fi jk = [vi,v j,vk], fi jk ∈ F , the edge lengthssatisfy the triangle inequality: li j + l jk > lki.

i

j k

Θij

Θjk

Θkiγi

γj

γk

θi

θj θk

ljk

lkilij

Figure 1: Circle Packing Metric.

For simplicity, we use ei to denote the edge against the vertex vi,namely ei = [v j,vk], and li is the edge length of ei in triangle fi jk.The cosine laws are given by:

l2k = l2

i + l2j −2lil j cosθk (2)

The discrete Gaussian curvature Ki on a vertex vi ∈ Σ can becomputed as the angle deficit,

Ki =

{2π−∑ fi jk∈F θ jk

i , vi ∈ ∂Σπ−∑ fi jk∈F θ jk

i , vi ∈ ∂Σ(3)

where θ jki represents the corner angle attached to vertex vi in all the

face fi jk ∈ F that share vertex vi, and ∂Σ represents the boundaryof the mesh Σ.

The circle packing metric (see Fig. 1) was introduced to approx-imate the conformal deformation of metrics [24, 25]. The functionΓ : V → R+ assigns a radius γi to each vertex vi ∈V . Similarly, theweight function Φ : E → [0, π

2 ] associates the acute angle Θi j witheach edge ei j ∈ E, where Θi j is defined as the angle of intersectionof the circles at vertices vi and v j. The circle packing metric for Σis the pair (Γ,Φ).

Let u : V → R be the discrete conformal factor, which measuresthe local area distortion, where ui = logγi for each vertex vi ∈ V .Then, the discrete Ricci flow is described by the following:

dui(t)dt

= (Ki−Ki), (4)

where Ki is the prescribed curvature at vertex vi. The discrete Ricciflow can also be formulated in the variational setting, namely, it isthe negative gradient flow of some special energy form:

f (u) =∫ u

u0

n

∑i=1

(Ki−Ki)dui, (5)

where u0 is an arbitrary initial metric. The integration above iswell-defined, and it is called the Ricci energy. Then, the discreteRicci flow is the negative gradient flow of the discrete Ricci energyand the discrete metric which induces k = (K1, K2, ..., Kn)

T is theminimizer of that energy.

Computing the desired metric with prescribed curvature k isequivalent to minimizing the discrete Ricci energy, which is strictly

convex (namely, its Hessian is positive definite). The global mini-mum exists uniquely, corresponding to the metric u, which inducesk. The discrete Ricci flow converges to this global minimum [1]and the global conformal parameterization for Σ can be computedin an automated and robust fashion.

4 COMPUTATIONAL ALGORITHM

The pre-computation step of our visualization approach processes apair of finely-tessellated meshes that represent the original 6-sidedCAVE and the target display configuration. The discrete Ricci flowis used to map each mesh to the unit disk and the maps are alignedusing a 3-point correspondence on the boundaries. By performingthis computation for a specific pair of closed boundaries, we obtaina one-to-one mapping between the viewing rays in the two config-urations, which is then stored in a pair of cube-map textures (onefor the forward and one for the backward correspondences). Thetechnique is general and can be applied to a number of differentdisplay configurations, however, in the following discussion we fo-cus mainly on the 5-sided CAVE as the target platform.

4.1 Discrete Ricci flowSuppose Σ is a triangle mesh embedded in R3. We associate eachvertex vi with a circle (vi,γi) where γi equals the minimal length ofany edge in the immediate neighborhood of vi. Then we computethe intersection angle Θi j such that the circle packing metric is asclose to the induced Euclidean metric as possible.

We compute the curvature at each vertex vi and adjust the con-formal factor ui in proportion to the difference between the targetcurvature Ki and the current curvature Ki. Then, we update themetric, recompute the curvature, and repeat this procedure until thedifference between the target curvature and the current curvature isless than the given threshold. Alg. 1 summarizes the computationalsteps and more details can be found in the work of Jin et al. [8].

Algorithm 1 Discrete Ricci FlowRequire: Triangular mesh Σ, target curvature for each vertex Ki,

error threshold ε .Ensure: Discrete metric (edge lengths) satisfying the target curva-

ture.1: u = [ui],v = [vi] for u,v← 02: while true do3: Compute edge length li j for edge [vi,v j]:

li j = eui + eu j +2cosϕi jeui+u j

4: Compute the corner angle θ jki in triangle [vi,v j,vk]:

θ jki = cos−1 l2

i j+l2ki−l2

jk2li j lki

5: Compute the curvature Ki at vi:

Ki =

{2π−∑ fi jk∈F θ jk

i , vi ∈ ∂Σπ−∑ fi jk∈F θ jk

i , vi ∈ ∂ΣK = [Ki]

6: if max |Ki−Ki|< ε then7: return the discrete metric li j8: end if9: Update u:

Compute the Hessian Matrix H, Hi j =∂Ki∂u j

u← u−H−1(K−K)10: end while

4.2 Riemann MappingFig. 2 illustrates the algorithm for computing the Riemann map-ping. We remove a face from the mesh Σ to convert it to a topologi-cal cylinder (Fig. 2(a)), resulting in the creation of a new boundaryγ1. The target curvature for both the interior and the boundary ver-tices is set to zero. The Ricci flow described in Alg. 1 produces

τγ1

γ2

υ1υ2

(a) Cutting the geometry

γ2

τ−

τ+

υ−

1

υ+

1

υ−

2

υ+

2

γ1

(b) Geometry flat-tening

υ2

γ1

γ2

τ

υ1

(c) Planar strip mapped to annu-lus

γ2

γ1

(d) Conformal mapping result

Figure 2: Riemann Mapping Algorithm.

a flat cylinder that is periodically embedded in the complex plane(Fig. 2(b)). Each period of the embedding is a rectangle and theoriginal boundaries γ1 along the cut face and γ2 are aligned with theimaginary axis in the complex plane. The cylinder is then mappedto the unit disk with the hole in the center of the image by the expo-nential map ez (Fig. 2(c)). Finally, the central hole is filled to obtainthe final mapping (Fig. 2(d)).

4.3 Conformal Mapping for the CAVEFig. 3 demonstrates the mapping between the 5-sided CAVE and the6-sided CAVE. We cut the closed cube (the 6-sided CAVE) along across slit at the top and map the cube to a sphere by the directionmap

p→ p− c|p− c|

,

where p is a point on the cube and c is the center of the cube. The 6-sided CAVE is mapped to a sphere with slits(Fig. 3(c)), and then itis conformally mapped to the unit disk using Ricci flow (Fig. 3(d)).Similarly, we map the 5-sided CAVE to an open sphere using thesame direction map, and then map the sphere to the unit disk.

The Mobius transformation is used to align the two disk images.We choose three corresponding boundary vertices on both the 5-sided CAVE and the 6-sided CAVE geometries, and use specialMobius transformations to map them to 1, i,−1 on the unit circle,which aligns the corresponding markers. Suppose {p,q,r} are threemarkers on the unit circle and

η1(z) =z− pz−q

r−qr− p

maps them to {0,∞,−1}. Let

η2(z) =1+ i

2z−1z− i

,

then η−12 ◦η1 is the desired Mobius transformation, which maps

{p,q,r} to {1, i,−1}. Fig. 3(b) and Fig. 3(d) show the result afterthe alignment.

γ1

(a) 5-sided CAVE mapped tothe sphere

γ1

(b) Conformal mapping of (a)to the unit disk

�1

�2

�1

�2�

γγ

γγ

(c) 6-sided CAVE mapped tothe sphere

γ

(d) Conformal mapping of (c) tothe unit disk

Figure 3: Algorithm for conformal mapping between a 5-sided CAVEand a 6-sided CAVE.

Fig. 4 illustrates the intuition for the application of the discreteRicci flow. We consider the view directions in the CAVE mappedonto a unit sphere and two curves. The first is an infinitesimal cutat the top of the sphere, which corresponds to the center of themissing projection wall (Fig. 4(b)). The second is the union of 4curves that correspond to the top edges of the side screens in theCAVE projected onto the sphere (Fig. 4(d)) and they define the vis-ibility boundary ∂Σ for the central CAVE position. Although wecan compute a conformal mapping between these curves, the re-sulting distortion would be very large near the top edges. Instead,we cut the sphere along the curves between the cusps in the visibil-ity boundary and the top curve on the sphere. The four edges shownin Fig. 4(b) are then mapped to ∂Σ, reducing the perceived distor-tion in the image. Using this mapping, the view directions are thenprojected onto the geometry of the 5-sided CAVE and encoded intoa cube map (Fig. 4(e)). Although our rendering framework can uti-lize conformal maps produced with any computational algorithms,the automation, robustness and efficiency of the Ricci flow providean advantage for quickly generating the mappings between the dif-ferent CAVE topologies.

5 IMPLEMENTING CONFORMAL VISUALIZATION

The algorithm described in Sec. 4 is implemented as a pre-processing step to the visualization. The output of that stage con-sists of the following:

• Two 8192×8192 textures in the RGB32F format, containingthe normalized viewing directions on the unit disk. These di-rections are computed directly from the conformal mappingsbetween the 5-sided and 6-sided CAVE meshes and the unitdisk.

• The tessellated meshes for the 5-sided and 6-sided CAVE con-figurations. The aligned conformal parameterizations com-puted in Sec. 4.3 are stored as UV texture coordinates, refer-encing the unit disk.

Although the mapping we have computed is defined in the spher-ical space of view directions in the CAVE, we encode it in a cube

(a) View directions in the CAVE (b) Full spherical mapping ofview directions

(c) Identity cube-map (d) Spherical mapping with visi-bility boundary

(e) Cube-map for theTray direction transfor-mation

Figure 4: All possible view directions in the CAVE (a) are mapped to a sphere (b) with the identity transformation map shown in (c). Thenormalized view directions are stored in the color channels of the texture. The cut near the top (b) is mapped to the visibility boundary on thesphere (d) and this mapping is used to compute the conformal cube-map Tray for transforming the view directions. (e).

texture, which is natively supported on the GPU and can be sam-pled efficiently in both vertex shaders and pixel shaders. The cube-map is generated by placing a virtual camera at the center of the 5-sided CAVE mesh and rendering the texture-mapped faces to 6 32-bit precision render targets, one for each CAVE wall. The buffersare then composited into a single cube-map texture in the vertical-cross format. At this point, we use the disk texture computedfrom the 6-sided mesh and we refer to the cube-map as Tray (seeFig. 4(e). Similarly, we compute the reverse mapping by renderingthe mesh for the 6-sided configuration with the correct parameter-ization for the disk texture from the 5-sided mesh, and label theresulting cube-map as Tgeom. We determined experimentally that1024×1024 cube-map resolution with 32-bit floating-point preci-sion per component is sufficient so that no banding or artifacts arevisible in the final visualization results.

The conformal maps stored in the two cube-map textures aresuitable for both forward and backward rendering pipelines. Wedemonstrate applications to mesh rendering with OpenGL, single-pass raycasting for Direct Volume Rendering (DVR) and real-timeraytracing on the GPU.

5.1 Mesh Rendering

We first apply the conformal distortion to a mesh-based renderingpipeline. We use an efficient vertex-based distortion for renderingwell-tessellated geometry with OpenGL, similar to the approachby Spindler et al. [22]. Intuitively, every vertex in the scene istransformed so that triangles that are projected on the top screenin the 6-sided CAVE configuration are instead projected on the 4side screens in the 5-sided configuration. This transformation isdefined by the following:

rc = (M−1wc )

T ·norm(

pw−M−1wc · [0,0,0,1]T

)pw → M−1

wc ·(Tgeom(rc) · |Mwc ·pw|

)(6)

where pw is the vertex position in world-space, rc is the normal-ized view direction in CAVE space and Mwc is the world-space toCAVE-space transformation matrix. In our rendering framework,the head node for the visualization cluster emits the camera infor-mation to all the rendering clients and the view matrix V associ-ated with that camera is the world-space to CAVE-space transfor-mation matrix. Each visualization node then computes the finalview and projection matrices based on the target projection surface(e.g., front, left, etc.). The geometry transformation is performed ina custom vertex shader that is bound to every primitive in the scene.

5.2 Direct Volume RenderingThe distortion is applied in a similar fashion for volume rendering.The visualization algorithm is based on single-pass ray-casting over3D textures with support for advanced lighting and shadowing, aswell as pre-integrated transfer functions [4]. Our framework inte-grates volume rendering tightly into the scene graph and we renderout the volume-space positions of the front and back faces of thevolume bounding box, modified by the depth of other scene geom-etry. One possible approach for incorporating the distortion map isto tessellate the bounding box and apply the Tgeom transformationas described in Sec. 5.1. However, our target application is the ex-ploration of the virtual colonoscopy data, in which case the camerais often within the volume and the starting positions of the rays aredefined on the near clipping plane. It is more accurate to transformthe positions on the near clipping plane and the back face of thebounding volume to world-space and then to apply the followingtransformation:

rc = (M−1wc )

T ·norm(

pw−M−1wc · [0,0,0,1]T

)pw → M−1

wc ·(Tray(rc) · |Mwc ·pw|

)(7)

where, again, pw is the vertex position in world-space, rc is the nor-malized view direction in CAVE space and Mwc is the world-spaceto CAVE-space transformation matrix. The viewing vector is thenconstructed from the modified starting and ending positions. Notethat this transformation is very similar to Eq. 6. The difference isthat since this is a backward rendering pipeline, we are transform-ing the view directions directly and the Tray cube-map should beused.

5.3 GPU RaytracingA major limitation of the geometry transformation approach is thatexisting shaders in the scene need to be modified to compute Eq. 6.This is not an issue for the virtual colonoscopy mesh dataset whichcontains a single shader, however, large architectural scenes maycontain dozens of materials and non-uniformly tessellated meshes.A number of GPU-based ray-tracing algorithms have been pro-posed that can achieve interactive frame-rates even for large geo-metric models [12, 17, 19, 31].

We integrate a ray-tracing renderer with our scene graph basedon the NVIDIA OptiX engine [17]. OptiX accelerates ray-tracingon the GPU by defining ray-generation, ray-scene intersections andshading programs in the CUDA [16] language which access tradi-tional acceleration structures that are also stored on the GPU. Theprograms are then intelligently scheduled on all CUDA-enabledGPUs in the system. Its features are comparable to other real-time

ray-tracing approaches [19, 31]. For our conformal visualization,the ray transformation is applied at the ray-generation level, whichis separate from the scene-graph and therefore much simpler to re-implement. The distortion computation is also simplified:

dw→MTwc ·

(Tray

((M−wc1)T ·dw

))(8)

Similarly to the volume rendering approach, we transform theworld-space ray direction dw to CAVE-space and fetch a new raydirection from the conformal map. CUDA (and by extension Op-tiX) does not support cube textures natively, therefore indexing theconformal map is implemented in the ray-generation program. Inaddition to simplifying the application of the conformal map, theraytracing enables a number of effects that are particularly suitablefor architectural visualization, such as dynamic shadows, global il-lumination, reflections and refractions.

6 RESULTS

We first demonstrate the visual properties of our approach on a unitsphere with a checkerboard texture. In Fig. 5, the virtual camera ispositioned at the center of the sphere so that the two poles of thetexture are shown on the left and the right displays. Fig. 5(a) showsthe results of our conformal visualization and demonstrates howthe visual information originally shown on the top screen is insteadrendered on the left, front, right and back screens. For Fig. 5(c) weapply the Tray transformation a second time rotated by 90◦ to mapthe viewing directions for the back screen to the remaining 4 walls.This particular transformation illustrates a preliminary applicationof our technique to a 4-sided CAVE, however, it results in a 8%loss in the field of view since the second pass does not account forthe missing top screen. The transformation for the 4-sided CAVEcan be generated directly with the algorithm described in Sec. 4, inwhich case no such loss will occur.

(a) 5-sided CAVE (b) 6-sided CAVE (c) 4-sided CAVE

Figure 5: Raytracing of a checkerboard sphere in (b) a simulated 6-sided CAVE, (a) 5-sided CAVE with our conformal visualization and(c) preliminary results for a 4-sided CAVE. The layout is in the stan-dard vertical cross format and the color coding is defined for the origi-nal walls in the 6-sided CAVE configuration (green for front/back, bluefor left/right, red for top/bottom).

User interactions with the virtual scenes often involve naviga-tion in 3D space. We demonstrate that aspect of our conformalvisualization with a checkerboard tunnel. Fig. 6 illustrates the cam-era panning down during the navigation. Note that compared tothe straightforward front projection, the conformal rendering pre-serves the context of the tunnel’s direction even when the camerais pointed downward. Also, the visual information correspondingto the original front projection is presented with minimal distortion,as also shown in Fig. 5(a).

Our conformal mapping is used for the real-time rendering ofstereo image pairs in the 5-sided CAVE and the distortion is com-puted and applied in our existing rendering framework. For theevaluation of the approach, all experiments are first run on a singleworkstation with 2 Intel Xeon E5430 processors and an NVIDIA

(a) (b)

(c) (d)

Figure 6: Navigation in the checkerboard tunnel with conformal vi-sualization where the camera pans down in (a)-(d). Each triplet ofimages shows the original front view (lower-left), original top view(upper-left) and front view with conformal visualization (right). Theoriginal views are provided for reference only and the conformal viewis rendered directly from the scene description with raytracing.

Quadro FX 5800 GPU with a 1024×1024 viewport. The produc-tion cluster environment consists of 5 workstations with an IntelXeon processor and dual NVIDIA Quadro FX 4600 GPUs with 101400×1050 viewports.

We first evaluate the conformal visualization for the rendering ofvolumetric data in IVC. The pair of images on the left in Fig. 7(a)are the original perspective projections for the front and top screensin a fully-enclosed CAVE. The image on the right is the result of theconformal visualization described in Sec. 5.2 for the front screenin the 5-sided CAVE. Note that unlike in image-based retargetingapproaches, the final result is rendered directly, and the original im-ages are included here only for demonstration purposes. The sourceof the volume data is a 512×512×451 16-bit CT scan of a patient’sabdomen after digital cleansing of the colon. A suspicious areais visible on the top screen, which would normally be missed inthe CAVE due to the angle of the approach and the lack of a topscreen. After the conformal distortion, this area is projected ontothe front CAVE screen and the shape of the bumps on the colonwall is preserved. Although conformal mapping is not distance-preserving and sizes are distorted, we provide tools to measure theactual distance in voxel-space. Since the conformal distortion mapis computed in a pre-processing step, the cost of the initial trans-form of the ray direction is negligible compared to the ray integra-tion. In practice, the performance drop is measurable but it is lessthan 1% on average.

As expected, the conformal distortion produces slightly unnatu-ral images in the areas of larger compressive distortions. However,the benefit is that depth perception is maintained in the distortedareas and the user can examine the entire scene even in a partially-immersive visualization platform. During the navigation of the vir-tual colonoscopy scene, the comments from the radiologist attend-ing the session indicated that this type of distortion is acceptablefor immersive exploration of the colon since the shape of potentialpolyps is preserved while allowing the entire colon surface to beexamined.

Interactive ray-tracing is supported in our visualization soft-ware as a complete replacement for the OpenGL renderer. As de-scribed in Sec. 5.3, the application of the distortion map is simpli-fied in terms of both computation and integration with the existingscenes. Fig. 7(b) compares the projection images of the front andtop screens (left pairs of images) to the conformally distorted frontview. As before, the original images are only used for illustrative

(a) Direct visualization of the raw slices from a CTscan of a patient’s abdomen.

(b) Architectural fly-through. (c) Navigation of densely connected data withconformal visualization.

Figure 7: Conformal visualization results. Each triplet of images shows the original front view (lower-left), the original top view (upper-left) andthe front view with conformal visualization (right). The conformal view presents the visual information from the original front view with minimaldistortion, while showing the missing visual information from the front 1/4 of the top screen. The lower part of the original front view is mapped tothe floor projection, as illustrated in Fig. 5(a). The shape of the suspicious area on the top view in (a) is preserved under the volumetric conformaldistortion.

purposes and the final result is rendered directly. The performanceof the renderer scales with the size of the viewport as opposed tothe scene complexity in the rasterization pipeline. Although ourcolon mesh contains geometry with uniformly-high triangle den-sity, the same is not true for the architectural scene. The ray-tracingapproach makes no assumptions regarding the tessellation of thedata and provides high-quality conformal visualization results evenin areas of low polygonal density. While the performance for thearchitectural scene is lower than with the rasterization pipeline, itallows for interactive frame-rates and special effects such as accu-rate glass reflections and shadows. Again, the cost of applying thetransformation map is negligible compared to the overall renderingand its effect on the frame-rate is below 1%.

The conformal visualization preserves shapes locally, howeverscenes with expansive straight edges present certain challenges. Inparticular, straight edges projected onto the missing ceiling surfaceare bent toward the top center of the front, side and back screens.The Ricci flow allows for direct control of the boundary during themapping with the result that the distortion is minimized for areas ofthe scenes that were projected on the original 5 surfaces. Fig. 7(b)illustrates this - the lower railings and the column are visually sim-ilar to the original projections onto the front screen. The goal ofconformal visualization in this application is to present additionalvisual context for the navigation while preserving the relationshipsbetween local structures. We provide a simple control over the pre-generated conformal maps that allows the user to specify how muchof the ceiling projection is visualized onto the screens. The result-ing choice is between visual distortions and visual coverage, and theselection can be optimized for the particular application or dataset.This control is implemented in the ray-generation kernel and can bemodified at runtime without re-computing the conformal map or thecube texture. For illustrative purposes, Fig. 7(b) presents a worst-case distortion scenario with approximately 95% ceiling coverage.

In addition to architectural scenes, we have performed visual-ization experiments with large connected data structures. The con-formal visualization is particularly suitable for scenes containinga large number of elements when the exact shape of the connec-tions is not relevant to understanding the data, but the shape of theelements may be. The expanded field of view and the resultingvisual context can be beneficial for exploration tasks in informa-tion visualization applications. Fig. 7(c) presents our test scene,which contains an abstract binary tree with 10,000 data elementsand color-coded connections.

7 EVALUATION

Our conformal visualization is evaluated informally in the 5-sidedCAVE for the visual detection of polyps in phantom colon datasets.The data is generated from a section of the colon centerline of areal patient who had undergone Virtual Colonoscopy. There are 4datasets similar to the one in Fig. 8 with between 25 and 29 hemi-spherical polyps. The control scheme in the CAVE uses an analoggamepad with the following mapping: view control with the leftanalog nub, forward/backward motion with the right analog numb,camera roll with the shoulder pads. The initial camera roll is se-lected randomly at the beginning of each experiment. The users aregiven time to study the navigation scheme for both regular and con-formal visualization, after which they are asked to find as many ofthe hemispheres as possible. Each user examines the 4 datasets witha pre-set rendering modality for each run. The examiner records thenumber of detected polyps, the number of false positives and thetime for each fly-through.

(a) Outside view

Polyp

(b) Inside view with a polyp

Figure 8: Phantom dataset for the user study. The center-line for thegenerated data is based on a section of a patient’s VC.

Our informal study had 12 participants (11 male, 1 female) andon average, the examination time was consistent between the reg-ular and conformal visualization. The conformal visualization im-proved the non-biased detection sensitivity for polyps from 91% to93% overall with individual differences in the range [−2%,+10%].On average this translates to the detection of up to 0.9 additionalpolyps over the baseline per dataset with 29 polyps. Stereoscopicvision remained effective for the majority of users in both the low-and high-distortion sections of the images. If we isolate the polypsthat were projected only to the missing ceiling, the detection rateswere close to the average for the dataset with conformal visualiza-tion, compared to 0% for the regular volume rendering. In bothcases, the false positive rate was about 0.1%.

8 CONCLUSIONS AND FUTURE WORK

We have presented a visualization technique based on conformalmapping for partially immersive visualization platforms that allowthe user to visually explore the full virtual environment. The dis-crete Ricci flow is used to compute the mapping and the theoreti-cal foundation for our approach guarantees the consistency of localshapes under the deformation.

We have demonstrated applications of our approach to rasteri-zation, volume rendering and GPU-based ray-tracing pipelines formedical, architectural and abstract data visualization in the 5-sidedCAVE. We avoid many of the challenges related to image-based re-targeting approaches by rendering the final results directly onto theoutput surfaces, which allows for the generation of accurate stereo-scopic image pairs. Our user study simulates the task of findingcancerous polyps in Virtual Colonoscopy and we demonstrate im-proved sensitivity compared to the traditional visualization at simi-lar examination times and false positive rates. In addition, the dis-crete Ricci flow can also be used to compute the conformal maps forarbitrary display configurations that are topologically equivalent toa disk and we show preliminary results for the 4-sided CAVE. Thisis a generalization of the technique that we plan to explore in detail.

Currently, the conformal mapping is defined for a specific cutin the space of viewing directions. While this provides a usefulvisualization for medical and architectural scenes, the cut is not op-timal for off-center viewing positions in the CAVE. One limitationis that the conformal mapping is currently performed on the CPU.A GPU-based implementation may provide adequate performancefor the dynamic generation of the distortion map. In addition, thegeneration of the conformal maps can be augmented with informa-tion from scene or rendering features so that the distortion can becontrolled around areas of interest. Certain distortion controls canalso be achieved by resampling the cube map at runtime without theneed to recompute the underlying conformal map.

ACKNOWLEDGEMENTS

This work has been supported by NSF grants CCF-0448399,CCF-0702699, CNS-0959979, CNS-1016829, IIS-0916235, IIS-0916286, NIH grant R01EB7530 and ONR grant N000140910228.The volumetric colon datasets have been obtained through the NIH,courtesy of Dr. Richard Choi, Walter Reed Army Medical Center.

REFERENCES

[1] B. Chow. The ricci flow on the 2-sphere. Journal of DifferentialGeometry, 2(33):325–334, 1991.

[2] C. Correa, D. Silver, and M. Chen. Illustrative deformation for dataexploration. Transactions on Visualization and Computer Graphics,13(6):1320–1327, 2007.

[3] C. Cruz-Neira, D. J. Sandin, T. A. DeFanti, R. V. Kenyon, and J. C.Hart. The cave: Audio visual experience automatic virtual environ-ment. Communications of the ACM, 35(6):64–72, 1992.

[4] M. Hadwiger, J. M. Kniss, C. Rezk-salama, and D. Weiskopf. Real-time Volume Graphics. A K Peters, 2006.

[5] A. Hogue, M. Robinson, M. R. Jenkin, and R. S. Allison. A vision-based head tracking system for fully immersive displays. In Proceed-ings of the workshop on Virtual environments, pages 179–187, 2003.

[6] L. Hong, S. Muraki, A. Kaufman, D. Bartz, and T. He. Virtual voy-age: interactive navigation in the human colon. In SIGGRAPH ’97:Proceedings of the 24th annual conference on Computer graphics andinteractive techniques, pages 27–34, 1997.

[7] W. Hong, J. Wang, F. Qiu, A. Kaufman, and J. Anderson.Colonoscopy simulation. Proc. SPIE Medical Imaging, 6511, 2007.

[8] M. Jin, J. Kim, F. Luo, and X. Gu. Discrete surface ricci flow. IEEETransactions on Visualization and Computer Graphics, 14(5):1030–1043, 2008.

[9] M. Jin, W. Zeng, F. Luo, and X. Gu. Computing Teichmuller shapespace. IEEE Transactions on Visualization and Computer Graphics,15(3):504–517, 2009.

[10] C. D. Johnson and A. H. Dachman. CT colonography: The next colonscreening examination? Radiology, 216(2):331–341, 2000.

[11] X. Li, X. Gu, and H. Qin. Surface matching using consistent pantsdecomposition. In ACM Solid and Physical Modeling Symposium,pages 125–136, 2008.

[12] B. Liu, L.-Y. Wei, X. Yang, Y.-Q. Xu, and B. Guo. Nonlinear beamtracing on a gpu. Technical Report MSR-TR-2007-34, Mcrosoft Re-search, 2007.

[13] H. Lorenz and J. Dollner. Real-time piecewise perspective projections.In GRAPP 2009 - International Conference on Computer GraphicsTheory and Applications, pages 147–155, 2009.

[14] B. Lvy, S. Petitjean, N. Ray, and J. Maillot. Least squares conformalmaps for automatic texture atlas generation. ACM Transactions onGraphics, 21(3):162–170, 2002.

[15] P. Milgram and F. Kishino. Taxonomy of mixed reality visual displays.IEICE Transactions on Information and Systems, E77-D(12):1321–1329, 1994. Introduction of the term ’Augmented Virtuality’.

[16] NVIDIA. Compute Unified Device Architec-ture Programming Guide, 2010. Version 2.3,http://developer.nvidia.com/object/gpucomputing.html.

[17] NVIDIA. NVIDIA OptiX Raytracing Engine, 2010. Version 2.2b,http://developer.nvidia.com/object/optix-beta.html.

[18] E. Pietriga, O. Bau, and C. Appert. Representation-independent in-place magnification with sigma lenses. Transactions on Visualizationand Computer Graphics, 16(3):455–467, 2010.

[19] S. Popov, J. Gnther, H.-P. Seidel, and P. Slusallek. Stackless kd-treetraversal for high performance gpu ray tracing. Computer GraphicsForum, 26(3):415–424, 2007.

[20] F. Qiu, B. Zhang, K. Petkov, L. Chong, A. Kaufman, K. Mueller, andX. D. Gu. Enclosed five-wall immersive cabin. In ISVC ’08: Pro-ceedings of the 4th International Symposium on Advances in VisualComputing, pages 891–900, 2008.

[21] R. Raskar, J. v. Baar, P. Beardsley, T. Willwacher, S. Rao, and C. For-lines. ilamps: geometrically aware and self-configuring projectors.ACM SIGGRAPH 2005 Courses, 2005.

[22] M. Spindler, M. Bubke, T. Germer, and T. Strothotte. Camera textures.In GRAPHITE ’06: Proceedings of the 4th international conferenceon Computer graphics and interactive techniques in Australasia andSoutheast Asia, pages 295–302, 2006.

[23] B. Springborn, P. Schrder, and U. Pinkall. Conformal equivalence oftriangle meshes. ACM Transactions on Graphics, 27(3):1–11, 2008.

[24] K. Stephenson. Introduction To Circle Packing. Cambridge UniversityPress, 2005.

[25] W. P. Thurston. Geometry and topology of three-manifolds. PrincetonLecture Notes, 1976.

[26] M. Trapp and J. Dllner. Generalization of single-center projectionsusing projection tile screens. In Advances in Computer Graphics andComputer Vision (VISIGRAPP), Communications in Computer andInformation Science (CCIS). Springer, 2008.

[27] L. Wang, Y. Zhao, K. Mueller, and A. Kaufman. The magic volumelens: An interactive focus + context technique for volume rendering.Proc. IEEE Visualization, pages 367–374, 2005.

[28] Y. Wang, X. Yin, J. Zhang, X. Gu, T. F. Chan, P. M. Thompson, and S.-T. Yau. Brain mapping with the ricci flow conformal parameterizationand multivariate statistics on deformation tensors. In 2nd MICCAIWorkshop on Mathematical Foundations of Computational Anatomy,pages 36–47, 2008.

[29] Y. Yang, J. X. Chen, and M. Beheshti. Nonlinear perspective projec-tions and magic lenses: 3d view deformation. Computer Graphics,25(1):76–84, 2005.

[30] Y.-L. Yang, J. Kim, F. Luo, S.-M. Hu, and X. Gu. Optimal surfaceparameterization using inverse curvature map. Transactions on Visu-alization and Computer Graphics, 14(5):1054–1066, 2008.

[31] M. Zlatuska and V. Havran. Ray tracing on a GPU with CUDA –comparative study of three algorithms. In WSCG 2010, 2010.