feature aligned volume manipulation for illustration and visualization carlos d. correa, deborah...

28
Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min Chen University of Wales, Swansea, UK

Upload: aaron-gunn

Post on 27-Mar-2015

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Feature Aligned Volume Manipulation for Illustration and Visualization

Carlos D. Correa, Deborah Silver

Rutgers, The State University of New Jersey

Min Chen

University of Wales, Swansea, UK

Page 2: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Motivation

Nucleus Inc Antonio Scrantoni and Paolo Mascagni, 1833. U.S. National Library of Medicine

Hand-drawn illustrations often include manipulating parts of an object:• They often contain cuts• They allow feature sensitive operations• They often represent virtual operations (do not necessarily conform to

reality)

Page 3: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Motivation (cont.)

We refer to such manipulation as lllustrative Deformation• Priority to interactivity, operatability and quality• As opposed to physically-based deformation, this can be thought of

as a top-down approach This type of deformations provides an intuitive depiction of internal

structure. It serves as an abstraction of different stages of a procedure, e.g. a

surgical operation. It is useful in surgery illustration/planning, education, and as a

visualization tool in general.

Page 4: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Feature Alignment

Traditional volume deformations are continuous and treat volumes as an homogeneous collection of points [Westermann et al. 2001, Rezk-Salama et al. 2001]

McGuffin [2003] introduced 3D widgets with pre-computed segmented data to allow feature sensitive manipulation of volumes. Can this approach be extended to direct volume rendering?

Recent approaches allow the definition of cuts [Correa,2006]. However, cuts appear flat as no semantics are introduced axis alignment

Cuts in general are difficult to model in computer graphics. Require costly re-tessellations. This is further complicated when cuts have to be aligned with certain features.

Page 5: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Axis Alignment

Treating volumes as homogeneous collections of voxels leads to axis alignment of cuts.

Difficult to see features of interest

Page 6: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Goal

To render deformations while preserving features of interest, by aligning cuts to a given:• Distance from surface surface alignment• Feature based on segmentation segment alignment

CT Dataset Illustrative Deformation

Illustration

Page 7: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Rendering Pipeline (axis aligned cuts)

Select operator

Sample and Deform

OPERATORS TRANSFORMATION

Adapted from Correa et al. 2006

via inverse space warping

Page 8: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Feature-Aligned Rendering Pipeline

Select operator

Apply mask

Sample and Deform

OPERATORS

MASKS

TRANSFORMATION

Adjust opacity/lighting according to alignment

Definition of features using a volumetric mask

Page 9: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Operators

Inspired by surgical tools and procedures Generic: they can be applied to any dataset Defined as a 3D texture. Iconic representations

are obtained when applied to a volumetric cube (or cylinder)OPERATORS

Page 10: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Volume Transformation

Sample Apply transformation Estimate normal Compute lighting

Subject to Alignment Mask

Sample and Deform

TRANSFORMATION

Page 11: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Modeling Deformation and Cuts

Forward transformation is simple but limited for volumes undersampling unless space between points is interpolated (for cuts, it requires re-tessellation)

Inverse transformation. Solves sampling problem, but discontinuous deformations are not a 1:1 mapping.

Forward Transformation Backward Transformation. Note introduction of special value to model discontinuity

Page 12: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Modeling Feature Alignment

Define a smooth mask M • Binary masks may cause aliasing• M(p) >= 0.5 p is non operatable• M(p) < 0.5 p is operatable

Three cases for inverse transformation

Not affected by mask: apply inverse mapping

Point inside mask: it is not transformed

Point outside mask but maps back into mask: empty space left by feature

Page 13: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Definition of Features

Distance-based vs. Segmentation based (1) Surface Alignment: Features are defined with the shape. Distance

field of surface of object defines a series of “shells”, which define features. Useful when no segmentation is available.

MASKS

M = 0.0

M = 0.5

M = 1.0

Outer shell ()

Feature’s outer surface (DT = )

Feature’s interior

Page 14: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Mask Definition

(2) Segment Alignment: Mask is already defined as a segment Usually segmentations are discrete (binary for 1 segment), For

proper rendering without aliasing, a smooth definition is required: Using a smoothing operator Using Distance Field of segmented part

M = 1M = 0.5

M = 0.0

Page 15: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Rendering and Lighting

Cuts are now along a certain feature. For surface alignment, a new surface appears. Pre-computed normals

are not necessarily perpendicular to that surface For segment alignment, gradient already defines “almost” correctly one

surface. However, it cannot define properly the “underside” of the cut

Page 16: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Normal Adjustment: Surface Alignment

Normals are oriented depending on density, not necessarily aligned with surface “shell”. This can be fixed by blending of normals

T

*T

DT

Page 17: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Normal Adjustment: Segment Alignment

Normals on the underside of a cut point in the opposite direction

T

*T

-T

Page 18: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Results (1) Peeling of Skin

AXIS SURFACE SEGMENTOriginal Dataset

Page 19: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Results (2) Frog Dissection

AXIS SURFACE SEGMENT

Original Dataset

Page 20: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Results (3) Hand Surgery

AXIS SURFACE SEGMENT

Original Dataset

Page 21: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Results (4) Forefoot Retractor

AXIS SURFACE SEGMENT

Original Dataset

Page 22: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Implementation Details

Based on discontinuous displacement mapping [Correa et al. 2006], using texture based volume rendering

Operators are stored as 3D textures (size is much smaller than size of dataset). Feature Mask is also stored as a 3D texture

Interactive results (Pentium XEON 2.8GHz Quadro FX 4400 (512 MB): d =1

Page 23: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Video

Page 24: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Applications

Medical and Biological Illustration.• Operators are metaphors of the tools used in dissection

Surgical Planning• Manipulation of operators allows the generation of deformations and

cuts in various stages of a procedure Improved Visualization

• Cutaway views with arbitrary cut geometry• Focus+Context, Distortion Lens

Page 25: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Conclusions

Volume deformation techniques often treat volumes as homogeneous collection of voxels. When modeling cuts and breaks, they appear to be axis aligned, which results in decreased realism and limited use.

It is possible to extend volume deformation to align cuts with certain features of interest. These can be defined as “shells” of the surface of the object using the distance transform, or as the product of segmentation

Feature alignment can be implemented efficiently on commodity hardware. Proper handling of cut information to reduce aliasing, and adjustment of normals near the surfaces of cuts are necessary to produce high quality rendering of cuts.

Page 26: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Future Work

Merge illustrative deformation with illustrative rendering• NPR techniques can be used to emphasize new surfaces due to

cuts or to exaggerate deformation (e.g. rendering of stress lines) Inclusion of rigid constraints for enhanced deformations, collision

avoidance. More intuitive user interface and manipulation widgets to create and

place operators.

Page 27: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Thanks

Acknowledgements• Volumetric datasets are courtesy of Lawrence Berkeley Laboratory,

UNC Chapel Hill, University of Iowa, U.S. National Library of Medicine, Viatronix Inc. and Vienna University of Technology. The illustrations are courtesy of U.S. National Library of Medicine and Nucleus Medical Art, Inc.

• We want to thank Dr. Stanley Trooskin, Dr. Sid Roychowdhury and Dr. Marsha Jessup for valuable input on surgical and medical illustration.

Further Informationhttp://www.caip.rutgers.edu/~cdcorrea

Page 28: Feature Aligned Volume Manipulation for Illustration and Visualization Carlos D. Correa, Deborah Silver Rutgers, The State University of New Jersey Min

Thanks