A Similarity Evaluation Method for 3D Models by Using HLAC Mask Patterns

MOTOFUMI T. SUZUKI, YOSHITOMO YAGINUMA, NORITAKA OSAWA

National Institute of Multimedia Education

2-12 Wakaba, Mihama-ku, Chiba-shi 261-0014, JAPAN

Abstract: Higher order local autocorrelation (HLAC) has been used as feature descriptors for various pattern recognition applications. We have extended typical 2D HLAC mask patterns to 3D HLAC mask patterns, which enable the masks to handle extraction of shape features from 3D polygonal models. 3D HLAC features are useful for evaluating similarities of 3D models and can be applied to 3D model similarity search engines. We have examined the similarities of 3D HLAC features by applying 3D HLAC masks patterns to each 3D model in a database. An experimental search engine has been implemented, and preliminary tests have shown that the system is able to find similar shaped models efficiently from a database. Key-Words: 3D model, Mask patterns, Similarity retrieval, Search engine 1 Introduction Today, a large number of 3D models are available on the Internet and have been used for various software applications. Recently, similarity search techniques [1-8] have been investigated to retrieve 3D models from the Internet. Similarity search techniques can find similar shaped 3D models on the Internet and in databases by comparing the indices of shape descriptors for each 3D model. The shape descriptors are extracted automatically by using software programs without time consuming human indexing operation. Similarity evaluations of 3D models rely heavily on the performance of shape descriptors, and shape descriptors need to adequately describe shape characteristics of 3D models. Therefore, shape descriptors are very important, and various shape descriptor extraction techniques have been proposed recently [3][4][5]. Some examples of such shape descriptors include densities of point clouds [7], 3D Discrete Fourier Transform [1][2], Spherical Harmonics [3], and Moments. Important similarity search techniques for searching 3D models and informative shape descriptor surveys can be found in [1][3][4]. In this paper, we propose higher order local autocorrelation (HLAC) features for evaluating shape similarities of 3D models. 2D HLAC has been used as a feature descriptor for various 2D image pattern recognition applications. Although HLAC mask patterns previously were applied to 2D images, they

have not been applied to 3D models and volume data. We have extended 2D HLAC mask patterns to 3D HLAC mask patterns, and this method enables masks to extract features from 3D models. 3D HLAC mask patterns were generated by using a simulation program, and 251 patterns were found which is about 10 times the number of 2D HLAC mask patterns. By using these 3D HLAC mask patterns, search systems can perform efficient retrievals. 2 HLAC Mask Patterns In this section, popular 2D HLAC mask patterns are discussed briefly, and their extension to 3D HLAC mask patterns are discussed. 2.1 2D HLAC Mask Patterns Autocorrelation functions are translation invariants, and their unique properties have been used for various pattern recognition applications. However, in most applications, autocorrelations have been limited up to the second order because of exponential computing costs. To overcome such a problem, an important approach is presented in [9], in which higher order local autocorrelation (HLAC) is proposed which successfully reduces the computing costs. HLAC has been used as a feature descriptor for various pattern recognition methods including gesture recognition [14], image retrieval [16] and face recognition

[10][11]. For such applications, HLAC mask patterns are essential for computing features. The mask patterns can be generated by considering autocorrelation functions. The Nth order autocorrelation functions with N displacements are defined by the following equation.

drrfrfrf aaaax nn

n )()()(),,(11

++= ∫ LL

where n: Order of the autocorrelation functions r: Coordinate of reference point a: Direction of shift / translation

1

1 1 1

1 1 1 1 1

1 1 1 1

1 1 1 1 1 1 1 1

1 1 1

1 1 1

1 1 1 1 1 1 1

1 1 1 1 1

1 1 1 1

1 1 1 1 1 1 1 1

1 1 1

1 1 1 1

1 1 1 1 1 1 1

1 1 1 1

Figure 1a: 2D HLAC mask patterns

a b c

d e f

g h i

N=0 (1): e N=1 (4): ef, ec, eb, ea N=2 (20): def, ceg, beh, qei, cde, beg, qeh, dei, efg, ceh, bei, aef, bde, aeg, deh, egi, efh, cei, bef, ace

Figure 1b: Cell positions and corresponding labels.

The number of autocorrelation functions increases rapidly for the large number of N and mask sizes.

Thus, the size of N is limited to up to the second order (N=0, 1, 2), and the mask size is set to 3 x 3 for practical applications. Also, the shift equivalent displacements are eliminated from the mask. Figure 1a and 1b show typical 2D HLAC mask patterns, and there are 25 mask patterns where the order N is up to 2. 2.2 3D HLAC Mask Patterns This subsection briefly describes 3D HLAC mask patterns. Although 2D HLAC mask patterns which were described in the previous subsection have been used for various pattern recognition applications, there are no practical applications which have been tested for use with 3D HLAC mask patterns. The basic ideas of 3D HLAC mask patterns are the same for 2D HLAC mask patterns but the mask is extended to 3D. Intuitively, 3D HLAC mask patterns are a set of cubes as shown in Figure 2.

Figure 2: 3D HLAC mask patterns (N=1)

N = 0 N = 1 N = 2

1 NA NA 4

2 NA NA 36

3 NA NA 6

4 NA NA 108

6 NA NA 36

8 NA 4 32

9 NA NA 3

12 NA 6 12

18 NA 3 NA

27 1 NA NA

Total 1 13 237

Table 1: The number of 3D HLAC mask patterns

Since dimensions are extended from 2D to 3D, the number of mask patterns increases rapidly for 3D HLAC mask patterns. As shown in Table 1, the total number of 3D HLAC patterns with the order of up to the second (N=0, 1, 2) is 251 (i.e. 1 + 13 + 237 = 251). The 3D HLAC mask pattern file which was generated by the simulation program is available at the following web site: (http://www.nime.ac.jp/~motofumi/AC3D/). These 251 3D HLAC mask patterns are used for extracting 3D HLAC shape features.

a b c j k l s t u

d e f m n o v w x

g h i p q r y z A

Figure 3: Cell positions and corresponding labels.

N=0 (1): a N=1 (13): ab, ak, am, an, ad, ae, aj, km, ks, kv, ms, mt, ns N=2 (237): akl, akm, akn, ako, aks, akt, aku, akv, akw, akx, abk, abl, aln, abm, abn, abo, abc, amn, amp, amq, ams, amt, amv, amw, amy, amz, abd, ano, anp, anq, anr, ans, ant, anu, anv, anw, anx, any, anz, anA, abe, abf, abj, ack, acn, ace, adk, adm, adn, adp, adq, ade, adg, adh, adj, aek, ael, aem, aen, aeo, aep, aeq, aer, aef, aeg, aeh, aei, aej, afk, afn, agm, agn, ahm, ahn, ain, ajk, ajm, ajn, ajs, ajt, ajv, ajw, klm, kls, klv, kmn, kmo, kmp, kmq, kms, kmt, kmu, kmv, kmw, kmx, kmy, kmz, knp, kns, knv, kny, kos, kov, kpv, kpw, kqv, kst, ksu, ksv, ksw, ksx, ktv, kuv, kvw, kvx, kvy, kvz, kwy, bks, bkv, lmn, lmt, lmw, lnp, lns, lnv, lny, lpw, lst, lsw, ltv, lvw, bms, bmt, bmv, bmw, lwy, bmy, bmz, bns, bnv, bny, mns, mnt, mnu, mot, mps, mpt, mqs, mqt, mst, msv, msw, msy, msz, mtu, mtv, mtw, mtx, mty, mtz, muw, nos, nps, npt, npu, nqs, nrs, nst, nsu, nsv, nsw, nsx, nsy, nsz, nsA, ntv, nty, nuv, nuy, ost, osw, otv, psv,psw,ptv,ptw,puw, qsv, qsw, qtv, rsw, bjs, bjt, bjv, bjw, cks, ckv, cns, cnv, cny, dks, dkt, dku, dkv, dkw, dkx, dms, dmt, dns, dnt, dnu, djs, djt, djv, djw, eks, ekv, ems, emt, ens, ejs, ejt, ejv, ejw, fks, fkv, fns, gms, gmt, gns, gnt, gnu, hms, hmt, hns, ins Figure 4: 3D HLAC mask patterns (string sequence representation) Figure 4 shows all the possible 251 3D HLAC mask patterns in a string sequence representation. Each

alphabet represents the unique location of grid cells in a 3 x 3 x 3 cube. The relations of the cell positions and alphabet labels are defined in Figure 3. In the figure, the cell labeled �n� is located the center of the cube. Since there are 27 cells in the cube, each cell may be labeled by lower case letters �a� through �z� and upper case letter �A�. These letter labels can be used to indication that each mask cell position is either ON or OFF. For example, the string sequence �mno� means the three mask cell positions �m�, �n� and �o� are ON. 3 Similarity Evaluations 3D HLAC mask patterns can be applied to volume data such as 3D polygonal models, CT (Computed Tomography) and MRI (Magnetic Resonance Imaging). In this paper, we apply 3D HLAC to 3D polygonal model data. An interesting possibility for the use of 3D HLAC mask patterns is that they might be able to capture local shape similarity features rather than global shape similarity features. 3D HLAC features can be extracted against 3D polygons and volume data in a similar method in 2D HLAC, except the mask is extended to the z axis which is a 3 x 3 x 3 mask pattern. Figure 5 shows procedures needed for the similarity retrieval. As an initial step, orientations of each 3D model are normalized by using principal component analysis (PCA) by analyzing coordinates of point clouds. PCA�s standardized first, second and third axes are aligned to the X, Y, and Z axes, respectively. Once the orientations of 3D models are set, each 3D model is voxelized. 3D polygonal models or meshes may be converted to voxel volume data by using voxelization techniques [17] if the 3D polygonal models are to be treated as volume data. Although various voxelization algorithms for planar polygons have been proposed, we have voxelized 3D models by applying a variation of a 3D scan line algorithm and a triangle-box intersection algorithm. In this 3D scan line algorithm based voxcelization, inertia of 3D models is not filled with voxels, but only polygons surfaces of 3D models are filled by voxels. Since the 3D scan line algorithm checks if voxel grids intersect to each face of a 3D model, the checking requires huge computation time for a high number of grids such as a grid over 64 x 64 x 64 when typical personal computers are used. Various grid sizes may be used for voxelizing 3D models. The size of mask patterns can be enlarged instead of using 1 voxel. The uses of different mask pattern sizes enable the extraction of

multiple resolutions of 3D HLAC features. For some applications, the uses of multiple resolution features improve recognition rates significantly. In our experimental system, we have used grid sizes of 16 x 16 x 16, 32 x 32 x 32 and 64 x 64 x 64. Once the 3D models have been voxelized, 3D HLAC mask patterns are applied to voxelized 3D models to extract 3D HLAC shape features. 3D HLAC mask patterns are applied to entire grids sequentially, or applied to portions of grids randomly. Since our implementation of voxelization algorithm generates sparse voxels in which inertia of a 3D model is empty, we have applied 3D HLAC to the voxel grid randomly. The extracted 3D HLAC shape features consist of 251 vectors. Shape similarities of 3D models are compared by using a histogram of these 251 vectors for each 3D model in the database. By comparing histogram values of each model, the system can sort 3D models based on similarities.

Figure 5: Procedures for similarity retrieval by using HLAC features. 4 Results This section describes (1) search system, (2) time needed for computing HLAC features, and (3) similarity search. 4.1 Search System The experimental system is implemented by C++ language. A GNU gcc compiler and the Open

Inventor graphics libraries running on a Linux operating system are used. An Apache httpd server is used to accept users� query requests. A Pentium 4-2.66 MHz CPU with 1024 Mbytes of memory is used to compute 3D HLAC features and for running the web based search engines. Figure 6 shows the interface of the search system. To test our system, 3D models distributed as benchmark data [] from the shape retrieval and analysis group at the Princeton University are used as a database. We have eliminated some models from this database which have an extremely large number of points and faces. The eliminated model ID numbers are #303, #1377, and #1556. We used a final total of 1810 models from the original database which contained 1813 models.

Figure 6: Interface of the 3D model search engine 4.2 Time Needed for Computing HLAC

Features Table 2 shows the time needed to compute 3D HLAC features with grid sizes of 8, 16, 32 and 64 for 1810 3D models. Computation time includes (a) data normalization, (b) voxelization, and (c) 3D HLAC shape feature extraction. As shown in the table, the process (b) voxelization is a time consuming task. However, these processes, (a), (b) and (c), have to be done only once as a preprocess of similarity retrieval, thus only about 2.0 seconds are needed for listing similar shaped models by comparing and sorting 1810 3D models from the database. HLAC features can greatly decrease the number of shape features reflecting their properties which enable fast computations.

8 16 32 64 (a)Normalization 1.9 2.3 5.0 68.8(b)Voxelization 15.5 83.0 624.3 6904.0(c)3D HLAC 2.0 2.5 6.1 76.0

(Minutes) Table 2: Time needed to compute 3D HLAC features 4.3 Similarity Search Figures 7a, 7b, 7c, 7d and 7e show examples of similarity search results. The queried 3D model is in the left corner in each figure. In the examples, 64 x 64 x 64 grid voxels are used to extract 3D HLAC shape features. Unlike previously studied typical shape descriptors, 3D HLAC shape descriptors can capture local shape similarity rather than global shape similarity. Figure 7d is an example of this case, and although the search results of global shapes are not so similar, local shapes are similar in terms of partial patterns.

1 2 3 4 5 Figure 7a: Search results for �Airplane�

1

2

3

4

5

Figure 7b: Search results for �Human�

1

2

3

4

5

Figure 7c: Search results for �Dog�

1

2

3

4

5

Figure 7d: Search results for �Bookshelf�

1

2

3

4

5

Figure 7e: Search results for �Chair�

5 Conclusions and Future Work We have extended popular 2D HLAC mask patterns to 3D HLAC mask patterns, and this process enables masks to extract shape features from 3D polygonal models. In our experiment, 3D HLAC masks have been tested to extract shape features of 3D models. Our preliminary system and test shows that 3D HLAC features can classify similarities of certain 3D models. Unlike typical shape descriptors, 3D HLAC shape feature descriptors can detect the local shape similarity rather than global shape similarity. The unique properties of the 3D HLAC shape features can be useful for expand the power of 3D model search engines. For future work, we will examine not only the extraction of shape features, but also the extraction of texture features by using 3D HLAC masks. Also, we will apply extracted 3D HLAC features to statistical analysis to enhance similarity retrieval power and investigate the characteristics of 3D HLAC features. Acknowledgements The authors thank the Princeton Shape Retrieval and Analysis Group at Princeton University for providing the 3D model benchmark data from their web site: http://shape.cs.princeton.edu/search.html. This research was supported by research grants from the Okawa Foundation for Information and Telecommunication (#2002-02-29) and a grant-in-aid from the Ministry of Education, Science, Sports and Culture, Japan (#15700115)

References: [1] D. V. Vranic, D. Saupe, and J. Richter. Tools for

3D-object retrieval: Karhunen-Loeve Transform and spherical harmonics, Proceedings of the IEEE 2001 Workshop Multimedia Signal Processing, Cannes, France, pp. 293-298, October 2001

[2] D. Saupe and D. V. Vranic, 3D model retrieval with spherical harmonics and moments, Proceedings of the DAGM 2001, Munich, Germany, pp. 392-397, September 2001

[3] Thomas Funkhouser, Patrick Min, Michael Kazhdan, Joyce Chen, Alex Halderman, David

Dobkin, and David Jacobs, A Search Engine for 3D Models, ACM Transactions on Graphics, 22(1), pp. 83-105, January 2003

[4] Ryutarou Ohbuchi, Tomo Otagiri, Masatoshi Ibato, Tuyoshi Takei, Shape-Similarity Search of Three-Dimensional Models Using Parameterized Statistics, in proceedings of the Pacific Graphics 2002, pp. 265-274, October 9-11, 2002

[5] Motofumi T. Suzuki and Yuji Y. Sugimoto, A Search Method to Find Partially Similar Triangular Faces From 3D Polygonal Models, Proceedings of the International Conference Modelling and Simulation (MS2003), pp323--328, ISBN: 0-88986-337-7, Palm Springs, CA, USA, 2003

[6] Motofumi T. Suzuki and Yuji Y. Sugimoto, A Search Method to Find Partially Similar Triangular Faces From 3D Polygonal Models, Proceedings of the IASTED International Conference Modeling and Simulation (MS2003), pp323--328, ISBN:0-88986-337-7, Palm Springs, CA, USA, 02/2003

[7] Motofumi T. Suzuki, Yoshitomo Yaginuma and Yuji Y. Sugimoto, Implementation Techniques of Boolean Logic Operators in 3D Model Search Systems, Proceedings of the 7th International Conference on Internet and Multimedia Systems and Applications, (IMSA2003), 08/2003.

[8] Eric Paquet and Marc Rioux, Nefertiti: a query by content system for three-dimensional model and image databases management, Image and Vision Computing 17, 157-166, 1999 [1] T. Kurita, N. Otsu, and T. Sato, Proceedings of 11th International Conference on Pattern Recognition, Vol.II, pp.213-216, 1992.

[9] N. Otsu and T. Kurita, A new scheme for paractical flexible and intelligent vision systems, Proceedings of IAPR Workshop on Computer Vision, Tokyo, pp.431-4352, 1998.

[10] K. Hotta, T. Kurita and T. Mishima, Scale Invariant Face Detection Method using Higher-Order Local Autocorrelation Features extracted from Log-Polar Image, Third IEEE International Conference onfAutomatic Face and Gesture Recognition, pp.70-75, 1998.

[11] F. Goudail, E. Lange, T. Iwamoto, K. Kyuma and N. Otsu, Face Recognition System using Local Autocorrelations and Multiscal Integration, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 18, No. 10, pp.1024-1028, 1996.

[12] V. Popvici and J.P. Thiran, Higher Order Autocorrelations for Pattern Classification,

Proceedings of the International Conference on Image Processing, 2001.

[13] J.A. MacLaughlin and J. Raviv, Nth-Order Autocorrelations in Pattern Recognition, Information and Control, Vol.12, pp.121-142, 1968.

[14] T. Kurita and S Hayamizu, Gesture Recognition Using higher order local autocorrelation features of PARCOR, IEICE Trans. Inf. & Syst., Vol.E86-D, No.4, April, 2003.

[15] D. Chetverikov and Z. Foldvari, Affine-Invariant Texture Analysis in Machine Vision, World Scientifc, 2000, Series in Machine Perception and Artificial Intelligence.

[16] M. Kubo, Z. Aghabari, K.S Oh, and A. Makinouchi, Image Retrieval by Edge Features Using Higher Order Autocorrelation in a SOM Environment. IEICE Trans. Info and Sys., Vol. E86-D, No.8, Aug. 2003.

[17] J. Huang, R. Yagel,V. Filippov, Y. Kurzion, An Accurate Method for Voxelizing Polygon Meshes, ACM 1998 Symposium on Volume Visualization, 1998, 119 � 126.

[18] Tomas Akenine-Moller, Fast 3D Triangle-Box Overlap Testing, Journal of graphics tools 6(1):29-31, 2001

[Dec.2003e]