Versatile 3D Texture Painting using Imaging Geometry

Xinyu Zhang, Young J. Kim∗ Xiuzi YeDept. of Computer Science and Engineering Dept. of Computer Science

Ewha Womans University, Korea Zhejiang University, China{zhangxy,kimy}@ewha.ac.kr yxz@cs.zju.edu.cn

Abstract

Texture map-based traditional painting systemsare often limited by the model’s parameterizationfrom 3D model space to 2D texture space, sincefinding such a parameterization can be very diffi-cult in practice (sometimes nearly impossible) formodels with complex topology or detailed struc-ture distribution. We present a novel data repre-sentation, imaging geometry, for 3D appearancemodelling and painting. By generating 3D col-ored points for each triangle contained in a poly-hedral surface model, the imaging geometry cansupport a great variety of painting operations sim-ilar to that of the conventional 2D image editor.The key ingredient of our approach is a novel,adaptive data representation for geometry, topol-ogy and color of an arbitrary surface, which al-lows users to treat each triangle as an image. Inour experiments, we demonstrate how easily theimaging geometry is applicable to 3D painting forany arbitrary surfaces that includes a model con-sisting of only a single triangle, multi-part ge-ometry, a non-orientable model, a non-manifoldmodel, an open or a closed model.

Key words: 3D Painting, Texturing, Point geom-etry

1. Introduction

Traditional texture mapping techniques map a 2Dimage onto a 3D surface to describe visual fea-tures of the surface without increasing the com-plexity of the underlying geometry. Not only foraugmenting visual characteristics to the surface,but texturing also can be employed for augment-ing other surface attributes such as surface color,

∗Corresponding author

surface normal, peculiarity, transparency, illumi-nation and surface displacement. Most of 3Dpainting systems use texture mapping by param-eterizing the geometry and topology of a modelin 2D texture space. As a result, 3D paintingis often limited by the underlying parameteriza-tion between the model’s geometry and 2D tex-ture space, since all parameterization techniquesintroduce discontinuities, stretching, and other ar-tifacts. Moreover, finding parameterizations canbe very difficult in practice (sometimes nearly im-possible) for models with complex topology or de-tailed structure distribution.

In our 3D painting system based on our ear-lier research work [21], we apply the 2D raster-ization technique to 3D mesh geometry by point-sampling the surface of a triangular mesh withcolored 3D points until the surface is completelycovered by a suitable number of these points.We generate these sampled points with suitableresolutions and point sizes using the hardware-supported occlusion query [14]. Moreover, wealso present a parameterization-free texture map-ping method to eliminate the artifacts introducedby point-sampling.

The major advantages of our painting systemcompared to the earlier approaches are as follows:

• For 3D painting, the proposed surface repre-sentation can handle any types of polygonalmesh models and can represent both geome-try and surface properties such as color in themodel.

• No texture parameterization is required.

• Traditional texturing techniques can be inter-mixed with our representation. The transi-tion between texture mapping and imaginggeometry can be achieved easily.

1

• The geometry and topology of a given modelare preserved for further mesh editing.

The remainder of the paper is organized as fol-lows. After discussing the related work in Sec. 2,we show the data representation of imaging ge-ometry in Sec. 3. We discuss the process of 3Dpainting in our system in Sec. 4. Finally, we showthe examples and experimental results of our sys-tem in Sec. 5 and conclude the paper in Sec. 6.

2. Related Work

With the availability of a 3D painting system, de-signers are now able to paint textures directly onthe surface of a model. The seminal work pro-posed by Hanrahan and Haeberli [8] enables di-rect painting on a 3D surface and now becomesavailable in various commercial systems. Thesetools let designers paint textures directly on 3Dmodels by projecting screen space paint strokes tothe 3D surface and then into texture space, wherethey are blended with other strokes.

Many commercial 3D shape modeling pro-grams provide painting features, where designerscan freely paint a 2D texture image in a 3D view[2, 10, 13, 19]. In these systems, designers setup a UV-mapping manually or automatically, andthe system subsequently re-projects the designers’strokes in 3D to a corresponding position in a 2Dimage based on the pre-calculated UV-mapping.Manual mapping methods require repeated pro-jection operations and unwrapping of the modelinto a planar domain. This process is tedious andchallenging even for those who are familiar withsuch mathematic concepts.

A research progress has been made on au-tonomous UV-mapping [4, 7, 9, 11, 12, 20, 18].A powerful texture painting system that supportsa direct 3D interface via force-feedback deviceswas proposed by Foskey et al. [6]; Here, eventhough their technique supports automatic param-eterization, texture distortion or stretching can bestill caused by the parameterization itself.

Unparameterized painting methods exploitingan Octree have been presented recently. Debry etal. [5] present a solution for texture mapping anunparameterized model using an Octree to storetexture data. Benson and Davis [3] have also donea similar work using Octree textures. Both ofthe systems remove 2D parameterization and store

texture data as a sparse, adaptive Octree. Theresearch trends based on Octree textures are ex-pected to continue.

Another work similar to our approach is the re-search work by Zwiker et al. [22]. Their 3D paint-ing system is also based on point geometry anduses irregular 3D points as 3D image primitives.However, they point-sample an input model irreg-ularly and produce an irregularly point-sampledmodel as an output. As opposed to triangle prim-itives, point samples do not store connectivity in-formation, so they must undergo an interactive pa-rameterization step. Other painting systems usingpolynomials [2], triangles [1], implicit functions[16, 17] and images [15] have been presented forthe past decade. Among them, Agrawala et al. [1]use a Cyberware scanner device to acquire the sur-face geometry of a given a physical object. Theirsystem treats the sensor of the space tracker as apaintbrush and color the corresponding locationson the scanned mesh.

3. Data Representation

In this section we describe a flexible and adap-tive data representation for 3D painting, imaginggeometry. Our main goal is to get rid of param-eterizations required for 3D painting. In order toaccomplish this objective, we store the color in-formation inside sampled points of the model, in-stead of in parameterized 2D textures.

3.1. Regular Sampling

Given an arbitrary triangle modelM, we create aset of regular points for each triangleT in M bypoint-samplingT. Let T be a triangle inM withverticesv0, v1, v2. Initially, we createn pointsfor each edge inT and calln the resolution of theimaging triangle. Then, we connect these pointsand maintain virtual lines parallel to the edges.The intersections of these virtual lines indicate thelocations of other sampled points to be created;See Fig. 1-(a). Fig. 1-(b), -(c) and -(d) demon-strate the examples of imaging triangles. In Fig.1-(b), the resolution (R) is 11 and the rasterizeddiameter of points (D) is 10. We will explain whatwe mean by the rasterized diameter of points inSec. 3.3. Likewise, in Fig.1-(c),R is 11 andD is20 and in Fig. 1-(d),R is 251 andD is 2.

2

0

2

3

……

n (n+1)/2-1

1

4 5

n (n-1)/2

6 7 8 9

v0

v1 v2

(a)

(b) R=11,D=10 (c) R=11,D=20 (d) R=251,D=2

Figure 1. Sampling a triangle. (a) Sampling pat-tern (b)-(d) Examples of imaging triangles.

This process will createn(n+1)2 points for eachtriangle;n can be different for different triangles.If we use an index value representing the locationof a point in the triangle, we needn(n+1)2 index val-ues in total. Moreover, we can treat this imagingtriangle as a 1D array of indexed location valueswith color properties (r, g, b, a). These coloredpoints are created for all triangles. We call thesetriangles as imaging geometries and we can getthe new modelM′ from the original modelM byimaging all the geometries.

In our system, we use 4 bytes to represent aRGBA color value. As a result, the total data sizefor each triangle is 2n(n+ 1) bytes. This size ofdata can overload the rendering time especiallywhen the resolution is very high. In the follow-ing section, we explain techniques to speed up theperformance.

3.2. Parameterization-Free Texture Map

In many commercial painting systems,parameterization-based texture mapping isnormally used to perform painting operations. Inthese systems, a given mesh is parameterized in2D image space. In this section, we present aparameterization-free texture mapping method tospeed up the rendering performance.

As illustrated in Fig. 2, for each triangle in agiven mesh, we create a texture and map the tri-angle with this texture. In Fig. 2, the right lowertriangle-shaped half-image is used to texture the

v0

v2v1

v2

v1

v0

Figure 2. Texture mapping. The highlighted pix-els are the corresponding points for 4, 7 and 8 inFig 1-(a)

front of the triangle whereas the left upper is forthe back of the triangle. Initially, the resolution ofthe prepared image is the same as a user-providedmaximum resolution; however we assume that theresolutions of triangles can not be extremely high.From Fig.’s 1 and 2, we can see that there existsone-to-one mapping between the imaging geom-etry and a triangle-shaped half-image. Therefore,we can convert our imaging geometry to a regulartexture without losing any surface details. Mean-while, sampling and texturing both the front andback faces allows users to handle arbitrary sur-faces, M̈obius strips, for example.

To find the corresponding pixel for each col-ored point in imaging geometry, we use the fol-lowing transformation between imaging geometryand 2D texture space: Given a point location in-dex valueid in a triangle,id specifies the locationof a colored point. We need to compute the coef-ficients of a bi-linear interpolation. This is equiv-alent to acquiring a (u, v) coordinate value basedon the index valueid in a triangle. We can resolvethis problem by solving the equation:

id =U(U +1)

2(id ≥ 0).

We get

U =−1+

√4id +1

2.

Then, we can getu = dUe andv = id− u(u−1)2 .For the front of a given triangle, the pixel posi-

tion is [R−(u−v)−1,R−u−1], and for the backof the given triangle,[R−u−1,R−v−1]

3.3. Multi-resolution

In order to reduce the amount of memory requiredin our system, we apply a multi-resolutional con-

3

cept to imaging geometry. There are two parame-ters to control the appearance of imaging geome-try. The first one is the resolution of imaging ge-ometry and the other one is the size of a rasterizedpoint.

(a) (b)

(c) (d)

Figure 3. Multi-resolution and anti-aliasing. Therasterized diameter of a point is used to controlthe resolution of imaging geometry and the anti-aliasing should be followed to smooth out the ras-terization.

Popular graphics library such as OpenGL canspecify the rasterized diameter of a point. In ourexperiments, the maximum point size used is 63and the minimum value is 1. If the rasterized sizeof a point is constant, we can change the resolu-tion of a imaging triangle to obtain the appropriateappearance. If the resolution of imaging geome-try is constant, we can change the rasterized sizeof a point. Moreover, when zooming in the modelto see its details, we can also adjust the resolu-tion and point size to get rid of apertures. We canapproximate the resolution and the pixel size of agiven triangles by solving the following equation:

NOQ =(n)(n+1)

2× πD

2

4

whereNOQ is the number of rasterized pixels of atriangle reported by the HW-supported occlusionquery. We use the nVIDIA’s occlusion query func-tion to query the number of rasterized fragments[14].

In order to get rid of such artifacts as sharpedges and corners in imaging geometry, it is nec-essary that anti-aliasing for points should be per-

formed (see Fig. 3). Another possible aliasingeffect can be caused by the resolution differencebetween adjacent triangles. We always set the res-olution difference as one or two. Providing a userinterface to control the resolution of adjacent tri-angles is an alternative solution.

4. Painting Method

Our system can easily handle the models that arenormally used in other systems. For surface paint-ing, we move the mouse in screen space whiledrawing strokes directly on the model surface.One way to find a 3D model position from the2D mouse cursor is to find where the line of sightthrough the 2D mouse position intersects with the3D surface. If multiple intersections are found,we choose the closest one. In OpenGL, we usethe selection and feedback features to implementsuch an interactive operation. The selection op-eration provided as a current feature in OpenGLallows us to take a 2D mouse click as an inputand determine which part of the geometry in themodel is beneath it. With the OpenGL’s selec-tion feature, we can specify a viewing volume anddetermine which objects fall inside that viewingvolume. Based on the aforementioned features,we can get the vertices of a triangle,vi , v j , vk, aswell as the location information of other coloredpoints. As we mentioned in Sec. 3, for each tri-angle, we use an index value to specify the loca-tion of points. Here, we save the texture properties(i.e., the color information) in an one dimensionalarray.

5. Results

We have tested our system on a number of mod-els with a simple or complicated connectivity in-formation. In Fig. 4-(a) and -(b), we show twoimaging triangles and their painting results. Fig.’s4-(c) and -(d) show a torus with many holes. Wepaint funny patterns on the torus without requiringany additional parameterization just like we painta 2D image using well-known Painter or Photo-shop program. To produce fancy textures in Fig4-(e) and -(f), we utilize the texture features avail-able in the PointShop 3D [22]. Users can createsuch painted models very easily in ten or twenty

4

Figure 4. Painting examples

minutes. In Fig.’s 4-(e) and -(f), we show the re-sults of painting on the M̈obius strip.

6. Conclusions

We present a novel data representation to paint di-rectly on any 3D triangle surface. Imaging geom-etry allows us to treat each triangle as a 2D imageand paint points on a 3D triangle surface withoutrequiring parameterization nor texture mappingprocess. The output of our paint system is stilla triangular mesh model with colored points. Thisinformation can be further used in any renderingprogram by texturing the color pointed back to thesurface geometry.

For future work, we would like to provide an in-tuitive man/machine interface to our painting sys-tem based on haptic interfaces such as PHANToMPremium 1.5.

Acknowledgement

This work is sponsored in part by the grant R08-2004-000-10406-0 of the KRF funded by the Ko-rean government, the Ewha SMBA consortium,the ITRC program, China National Science Foun-dation (60273060; 60073026), China Ministry ofScience and Technology software special project(2003AA4Z1020).

References

[1] Maneesh Agrawala, Andrew C. Beers, andMarc Levoy. 3D painting on scanned sur-faces. InProceedings of the 1995 Sym-posium on Interactive 3D Graphics, pages145–150, 1995.

[2] Alias Wavefront. Maya.http://www.aliaswavefront.com, 2002.

[3] David Benson and Joel Davis. Octree tex-tures. InProceedings of SIGGRAPH 2002,pages 785–790, 2002.

[4] Nathan A. Carr and John C. Hart. Paintingdetail. InProceedings of SIGGRAPH 2004,pages 845–852, 2004.

[5] David Debry, Jonathan Bibbs, Devorah D.Petty, and Nate Robins. Painting and ren-dering textures on unparameterized models.In Proceedings of SIGGRAPH 2002, pages763–768, 2002.

[6] Mark Foskey, Miguel A. Otaduy, andMing C. Lin. Artnova:touch-enable 3dmodel design. InProceedings of IEEE Vir-tual Reality Conference 2002, pages 119–126, 2002.

[7] Steven Haker, Sigurd Angenent, Allen Tan-nenbaum, Ron Kikinis, Guillermo Sapiro,

5

and Michael Halle. Conformal surfaceparameterization for texture mapping. InTransactions of Visualization and ComputerGraphics, volume 6, pages 181–189, 2000.

[8] Pat Hanrahan and Paul Haeberli. DirectWYSIWYG painting and texturing on 3Dshapes. InComputer Graphics (Proceedingsof ACM SIGGRAPH 90), volume 24, pages215–223, 1990.

[9] Takeo Igarashi and Dennis Cosgrove. Adap-tive unwrapping for interactive texture paint-ing. In Proceedings of the 2001 Symposiumon Interactive 3D Graphics, pages 209–216,2001.

[10] Interactive Effects,Inc. AMAZON 3D Paint.http://www.ifx.com, 2002.

[11] Bruno Lévy. Constrained texture mappingfor polygonal meshes. InProceedings ofSIGGRAPH 2001, pages 417–424, 2001.

[12] Jerome Maillot, Hussein Yahia, and AnneVerroust. Constrained texture mapping forpolygonal meshes. InProceedings of SIG-GRAPH 93, pages 27–34, 1993.

[13] Metacreations,Inc. Paint 3D.http://www.metacreations.com, 2002.

[14] NVIDIA. http://oss.sgi.com/projects/ogl-sample/registry/NV/occlusion-query.txt.

[15] Byong Mok Oh, Max Chen, Julie Dorsey,and Frdo Durand. Image-based modelingand photo editing. InProceedings of SIG-GRAPH 2001, pages 433–442, 2001.

[16] Hans Kohling Pedersen. Decorating implicitsurfaces. InProceedings of SIGGRAPH 95,pages 291–300, 1995.

[17] Ronald N. Perry and Sarah F. Frisken.Kizamu: A system for sculpting digital char-acters. InProceedings of SIGGRAPH 2001,pages 47–56, 2001.

[18] Emil Praun, Adam Finkelstein, and HuguesHoppe. Lapped textures. InProceedings ofSIGGRAPH 2000, pages 465–470, 2000.

[19] Right Hemisphere Ltd. Deeppaint 3D (Deep UV, Deep Paint).http://www.righthemisphere.com, 2002.

[20] Pedro V. Sander, John Snyder, Steven J.Gortler, and Hugues Hoppe. Texture map-ping progressive meshes. InProceedings ofSIGGRAPH 2001, pages 409–416, 2001.

[21] Xinyu Zhang, Xiuzi Ye, and Sanyuan Zhang.3D painting based on imaged geometry.Journal of Software, 15(3):461–467, 2004.

[22] Matthias Zwicker, Mark Pauly, Oliver Knoll,and Markus Gross. Pointshop 3D: An inter-active system for point-based surface edit-ing. In Proceedings of SIGGRAPH 2002,pages 322–329, 2002.

6