shapesynth: parameterizing model collections for coupled...

EUROGRAPHICS 2014 / B. Lévy and J. Kautz(Guest Editors)

Volume 33 (2014), Number 2

ShapeSynth: Parameterizing Model Collectionsfor Coupled Shape Exploration and Synthesis

Melinos Averkiou1 Vladimir G. Kim2 Youyi Zheng3 Niloy J. Mitra1

1University College London 2Stanford University 3Yale University

Abstract

Recent advances in modeling tools enable non-expert users to synthesize novel shapes by assembling parts ex-tracted from model databases. A major challenge for these tools is to provide users with relevant parts, whichis especially difficult for large repositories with significant geometric variations. In this paper we analyze un-organized collections of 3D models to facilitate explorative shape synthesis by providing high-level feedback ofpossible synthesizable shapes. By jointly analyzing arrangements and shapes of parts across models, we hierar-chically embed the models into low-dimensional spaces. The user can then use the parameterization to explore theexisting models by clicking in different areas or by selecting groups to zoom on specific shape clusters. More im-portantly, any point in the embedded space can be lifted to an arrangement of parts to provide an abstracted viewof possible shape variations. The abstraction can further be realized by appropriately deforming parts from neigh-boring models to produce synthesized geometry. Our experiments show that users can rapidly generate plausibleand diverse shapes using our system, which also performs favorably with respect to previous modeling tools.

1. Introduction

Modelers often lack a clear mental image when creating anew shape and would ideally prefer to first browse throughdifferent possible geometric realizations of the target object.For example, one may want to flip through various exist-ing designs before sketching a new chair. The ever growing3D model repositories (e.g., Trimble 3D Warehouse) providerich samplings of such design possibilities, but are typicallyunorganized and do not come with any parameterization ofthe underlying shape spaces. The challenge then is how tocharacterize, organize, and effectively navigate such shapespaces implicitly specified by model collections. An evenmore important challenge is how to provide previews of pos-sible shapes that are missing from the input collections. Inthis paper we propose how to effectively organize unorga-nized model collections, and systematically detect and pop-ulate the missing shape variations (see Figure 1).

There are a few common paradigms for exploringmodel repositories: the user can provide a query 3Dshape [FMK⇤03] or scribble a target shape in 2D [ERB⇤12],and then the system retrieves the matching shape(s). Suchapproaches rely on the user having a clear conceptual modelof the target shape and being able to effectively communi-cate the same. Further, the repositories should contain suf-ficiently similar shapes that can be retrieved, an assump-

tion that is often violated. More recently, qualitative explo-ration tools have been proposed [HSS⇤13, KFLCO13] usingdynamic embedding. Although such systems provide intu-itive local exploration of existing models, they do not sup-port synthesis of the missing shapes. Talton et al. [TGY⇤09]propose an intuitive interface that tightly couples explo-ration and synthesis for parameterized model families. How-ever, their method assumes a compact parameterizable de-

parameterized embedding

Figure 1: We analyze unorganized model collections usingtemplate-based abstractions to create a low-dimensional pa-rameterized embedding of the underlying shape space. Theuser then explores the parameterized space to create novelmodels by probing the empty regions (e.g., in red rectangles).In each case, a model was synthesized by significantly de-forming and combining parts from the input models.

c� 2014 The Author(s)Computer Graphics Forum c� 2014 The Eurographics Association and JohnWiley & Sons Ltd. Published by John Wiley & Sons Ltd.

Averkiou et al. / ShapeSynth: Coupled Exploration and Synthesis of Model Collections

input model collection extracted templates parameterized shape space synthesized geometry

offlineanalysis

embedding+ clustering

constrainedsynthesis

geometrysynthesis

synthesized abstraction

user selection

Figure 2: We analyze unorganized model collections to obtain a template-based parameterized shape space. Exploration of theabstracted shape space reveals possible shape variations, which can then be realized by appropriately mixing the input models.The combined exploration and synthesis allows users to quickly get an overview of the modeling space and subsequently createnovel shape variations via the parameterized shape space.

sign space [WP95,ACP03], which quickly becomes infeasi-ble for raw model collections with diverse shape variations.

In the context of synthesis, modeling from scratch re-quires a lot of expertise and even professional training. Asimpler interface that is suitable for novice users is to di-rectly combine parts from existing models to synthesize newmodels [FKS⇤04]. This type of direct manipulation-basedinterface still faces the challenge of finding and extractingappropriate parts from model repositories which is cum-bersome with standard retrieval techniques. Hence, smarteralternatives have been proposed, searching for parts basedon geometric [KJS07] and relational [ZCOM13] similar-ity, or by sampling suitable probabilistic graphical mod-els [CKGK11, KCKK12]. Such methods, however, do notprovide the user with a high-level preview of the space ofpossible models that can be synthesized. Further, the userneeds a clear vision of the final shape in order to effectivelyprobe such systems to retrieve suitable models or modelparts. This is often difficult, especially in the presence ofsignificant model variations (see Figure 4).

We propose ShapeSynth, a novel approach that directly in-tegrates exploration with synthesis. The key to our methodis extracting a template-based parameterizable space that ef-fectively factors out and encodes (part-) deformation acrossthe collection. Our method has an offline analysis step andan online hierarchical encoding stage to extract local embed-dings of the underlying shape spaces. The embeddings facili-tate both fast exploration of the existing shapes and preview-ing of possible shapes that are missing from the input collec-tions. The offline analysis reveals the structure of the shapesin the collection using a set of deformable templates. Theautomatically-learned templates are fitted to each shape pro-ducing co-aligned and cosegmented models with known partdeformations. The fast online embedding reveals the differ-ent shape groupings inside the collection, exposes the mainmodes of variability for each group, and lends itself to fastand intuitive synthesis of novel shapes. We performed a userstudy to test our system on several large model collectionsand compared with alternative systems (see also supplemen-tary video and demo). Figure 1 shows some of the models

created by the different users of our system. In summary,our contributions are:

• An efficient embedding technique for part-aware shapedescriptors that enables a hierarchical organization oflarge model collections and provides a parameterized ba-sis for exploring the underlying shape space;

• a novel exploration interface based on the embedding thatreveals groupings of shapes and summarizes the mainmodes of variation inside each cluster; and

• a novel tool for synthesizing new shapes using 3D modelrepositories, seamlessly tied to the exploration interface.

2. Related Work

In this work we demonstrate the synergy between twoproblems that were traditionally studied independently: ex-ploration of geometric collections and synthesis of novelshapes.

Exploration of shape collections. A large body of researchhas been devoted towards shape retrieval from collectionsof 3D models. Typically, the user either provides an ex-ample shape [FMK⇤03, FKS⇤04], a 2D sketch [ERB⇤12,LF08, SXY⇤11], or 3D scene context [FH10, XCF⇤13],while the corresponding system finds an appropriate modelfrom the repository. Topological information [STP12] canalso be used for non-rigid shape retrieval. Ovsjanikov etal. [OLGM11] observed that shape retrieval is only suitablewhen the user has prior knowledge of a database, which isitself a challenging task for unorganized datasets. Hence,they proposed an alternative tool for exploring novel col-lections. Several exploration interfaces have been proposedsince then, including navigating in the space of 3D modelsby deforming a proxy shape [OLGM11], selecting regionsof interest to define an ordering of the models [KLM⇤12],and embedding shapes into a 2D space based on their geo-metric differences [HSS⇤13,KFLCO13,ROA⇤13]. Althoughthese exploration techniques provide context for a potentialmodeler (e.g., a better understanding of variations in part ar-rangements and part geometry), none of these methods al-low synthesizing novel shapes during the exploration ses-sion, which is the key contribution of our work. In an in-

c� 2014 The Author(s)Computer Graphics Forum c� 2014 The Eurographics Association and John Wiley & Sons Ltd.


teresting earlier attempt, Talton et al. [TGY⇤09] proposedthe use of parametric design spaces to support exploratorymodeling based on human preferences. Although our workis inspired by their interface, their system is only suitablefor parametric shape spaces (e.g., human bodies, procedu-ral plants) and would not work with raw collections of 3Dmodels as its input.

Part-based model synthesis. The seminal modeling-by-example system [FKS⇤04] demonstrated that even inexpe-rienced users can synthesize interesting shapes by mixingparts from model repositories. Although several follow-upmethods have been proposed for stitching compatible partsinto a final 3D model (e.g., [SBSCO06, CKGK11]), twokey challenges remain: choosing appropriate parts and ef-fectively presenting them to the user.

In the Shuffler system, appropriate parts were chosen froma set of consistent segments [KJS07]. Subsequently, alter-native methods measured part compatibility with respectto the rest of the shape, including a probabilistic modellearned from training examples [CKGK11, KCKK12], afuzzy part correspondence based on similarity betweentheir bounding boxes [XHCOB12], part-based contextual in-formation [XXM⇤13], and functional arrangements whichencode structural relations such as symmetry and sup-port [ZCOM13]. Most existing methods present compatibleparts as sorted lists [FKS⇤04,CKGK11], which can be limit-ing if the number of parts is large. Jain et al. [JTRS12] allowchoosing pairs of models and produce intermediate shapeswith similar part arrangements. However, choosing whichshapes to blend requires prior knowledge about the collec-tion and is only appropriate for small datasets. Further, thesemethods assume compatible parts, which is easily violatedin the case of large shape variations. Recently, Chaudhuriet al. [CKGF13] proposed a browsing interface that uses1D sliders associated with adjectives, allowing to search forparts that have more or less of a certain property. Althoughthis results in a simple and effective interface, assigning nat-ural language tags to geometry requires significant manualeffort and also does not provide sufficient geometric context.

3. System Overview

Our system takes as input a collection of semantically-related man-made shapes, e.g., a set of 3D chair mod-els, which were downloaded from the web. The user startsby loading such a shape collection, which has been pre-analyzed in an off-line stage (see Section 4).

The system interface, see Figure 3, is split into three pan-els: the icon view that presents the different representativesfor quickly selecting among different style groupings; theexploration view that presents a set of embedded points, onefor each shape among the current selection of models; andthe model view that presents the current synthesized model,

model viewexploration view

icon view

Figure 3: Our system comprises of an exploration view toshow the embedding of the input models; an icon view toshow representative models for the current group(s); and amodel view for displaying synthesized models created usingour coupled exploration and synthesis algorithm.

either abstracted as a set of box proxies, or as a part-basedgeometric realization.

Input models get embedded such that models with simi-lar deformations end up as neighbors, while dissimilar mod-els are embedded to distant points (see Section 4). The em-bedded points are automatically grouped to provide the userwith a high-level overview of representative shapes in theicon view panel, with different colors indicating differentgroups. As the user selects one of the groups, the corre-sponding models are re-embedded and the process contin-ues. Essentially, the user traverses the hierarchically orga-nized models that are grouped based on their similarity, withshape variations being automatically factored out. Further-more, the user can explore variations within any single groupby studying the main variation modes across models in theselected group.

More importantly, one can use our system to previewplausible shapes that are missing in the original collections.As the user hovers over any empty region in the explorationview, at any level of the hierarchy, the system shows ab-stracted views of synthesized shapes in the model view. Thisallows users to progressively conceive and refine the miss-ing shapes that they want to create. Note that the users canwork at any level of the hierarchy and freely combine mod-els across clusters, if they desire. By providing the user anidea of the shapes she wants to combine into new shapes, wesimplify the content creation process. Essentially we providea quick glimpse of possible models by exposing the spacespanned by the input collections. Constraints such as sym-metry, contact, and size are directly preserved by our sys-tem. Finally, the user can also view plausible geometric re-alizations of the abstracted model. In the background, thesystem produces such geometric realizations by combiningdeformed parts from appropriate neighboring models (seesupplementary video and demo).



naive part-basedsynthesis

superimposed models

Figure 4: Combining a random selection of chair mod-els (top), even when they are consistently segmented, is chal-lenging. The models have different proportions of parts,that make a part selection based on visual inspection ofthe superimposed models (bottom-left) confusing and caneasily result in meaningless part ensembles (bottom-right).Instead, we expose a constrained and intuitive part-basedshape space for easy exploration and synthesis.

4. Algorithm

Starting from an unorganized model collection, our goal isto allow the user to meaningfully navigate the input mod-els, and more importantly, provide a preview of the possi-ble shapes that can be synthesized by appropriately combin-ing parts from the different input models. In order to enablesuch exploration of missing shapes, we have to overcomea few key challenges: (i) the input models typically havelarge shape variations that in turn obscure any inherent con-sistency across the collection; (ii) the models do not comewith any segmentation or consistent parameterization; and(iii) the collections typically include thousands of models,making realtime analysis non-trivial. For example, a visualinvestigation of the models in Figure 4 does not immediatelyreveal the space of possible models that can be realized bypart-level combination of the input models.

We overcome these challenges by computing intrinsic em-beddings of the models, which in turn facilitate previewingpossible part-based synthesized models. The embeddingshelp to identify which models can be combined; what re-spective parts can be combined; and finally how the partscan be deformed to produce a consistent model. Thus, theuser can directly explore the space of plausible models, withthe system factoring out the variations in configurations thatobscure novel synthesis possibilities. Intuitively, the embed-dings offer parameterizations to systematically factor outpart deformations, making subsequent synthesis simple andintuitive. We now describe the key steps of our method (seeFigure 2).

Initial analysis. First, in an offline phase [KLM⇤13], weanalyze the input collection of models {M1, . . .MN}. Start-ing with an initial template model, the method jointly op-

Figure 5: In the case of models with multiple compo-nents (left), we use the extracted part distributions ob-tained from the shape collection [KLM⇤13] to obtain aninitial point labeling (middle-bottom) and part abstrac-tion (middle-top). We refine the segments using a labelingoptimization (right).

timizes for part segmentation, point-to-point surface corre-spondence, and a compact deformation model to best explainthe input. The deformation model assumes that each shapecan be approximated with a set of box-like parts that differin position and scale.

Abstracted encoding. Based on the extracted distributionof the template parameters, we refine the segmentations ofthe individual models Mi. We assume that each model comeswith multiple disconnected components, which is true formost models in Trimble Warehouse [Tri13] that we used inour experiments. If this assumption does not hold, we simplyassign each triangle to its nearest box. Let {p1, p2, . . .} bethe set of components for model Mi. Our task is to associateeach component with a template box. This is essentially alabeling problem, where each pi can be assigned to a set oft candidate boxes {l1, . . . , lt}. We formulate the labeling as aMRF minimization:

{li}? := argmin{li}

Âi

E(pi! l j)+Âi, j

E(pi! lk, p j! ll) (1)

The unary term E(pi! l j) := vol(Bpi[l j )�vol(Bl j ), wherevol(Bpi[l j ) denotes the bounding volume of component piand template box l j, and vol(Blj ) denotes the bounding vol-ume of template box l j, measures the increase of boundingvolume of the template box when component pi is assignedto it. The pairwise term E(pi ! lk, p j ! ll) measures thepenalty when two neighboring components (based on short-est distance between them) are assigned different labels (setto 1e-5 in our tests). In the end, for each model we get a setof abstracted boxes, each enclosing a part of the input model(see Figure 5-right).

Let there be t different parts across all the extracted tem-plates. We represent each model Mi as a configuration vec-tor Xi 2 R6t , where each (axis-aligned) template box isrepresented by its centroid c and its dimensions s, i.e., itslength/breadth/height. Parameters corresponding to missing



Data: Input model collection M := {M1, . . .MN}.Result: Embedding coordinates of the selected models.1. Analyze the input collection M in an offlinestage [KLM⇤13];2. For each model Mi 2M, refine initial segmentationto obtain configuration vector Xi 2 R6t ;3. Set selection M M ;4. while new selection M available do

i. Randomly pick n models M j for j = 1, . . .n fromM as landmarks;ii. Construct distance matrix Dn⇥n with elementsdi, j := d(Mi,M j) for i, j = 1, . . .n ;iii. Compute MDS embedding of the landmarkmodels M j using D to obtain Yj 2 R2 ;iv. For all Mi 2M and Mi /2 {M j}, find its k nearestneighbor models among the landmark models anduse distance-based interpolation to obtainembedded coordinates {Yi}.v. Compute (linear) basis vectors e1 and e2 forinverse mapping of embedded coordinates toconfiguration vectors.vi. Apply mean-shift clustering on the points {Yi} ;vii. Update selection M and repeat hierarchically byreturning to step #4 ;

endAlgorithm 1: Iteratively embed a selection of models M,which is then used for exploration and synthesis.

parts are set to zero. We also detect the potential relationsamong the individual parts, i.e., symmetry and contact re-lations, and propagate the information to their associatedtemplates, which are later used in the constrained synthesisphase. Note that the relations should be unified across tem-plates within a given family. To address this in a consensusstage, we collect the relations among all the templates anduse a greedy selection strategy to filter out falsely detectedrelations. Improperly identified or conflicting relations canbe manually corrected, although advanced automated meth-ods can potentially be used [MWZ⇤13].

Efficient embedding. Both exploration and synthesis re-quire a notion of neighborhood among the models. We usethe coarse abstraction obtained above to define such a dis-similarity distance between model pairs Mi and M j as fol-lows: d(Mi,M j) := kXi�Xjk. Note that since we also havethe parameter distributions, one can instead use Mahalanobisdistance. Thus, similar models have near-zero dissimilarityscore, while dissimilar model pairs get high scores.

We use the similarity values to embed the models into alow-dimensional parameterized space. One option is to con-struct a N ⇥N matrix with all the pairwise similarity val-ues (e.g., exp(�d(Mi,M j)

2/2s2)) between the input mod-els and compute its spectral embedding [KLM⇤12]. Sucha direct computation, however, can be prohibitively expen-sive (i.e., O(N3)) for large model collections. Instead, we

propose a sampling-based approach to efficiently build anembedding of the models (see Algorithm 1). The key ob-servation is that the embedding is largely dictated by themembers from different (unknown) clusters (c.f., [dST04]).Hence, working with a random sampling of models as rep-resentatives yields an approximate embedding. Note that weuse the approximate embedding only when the number ofmodels is large, otherwise we perform the embedding withall the selected models.

We start by picking at random n landmark models,say {M j}, from the current set of models M. Using then landmark models, we compute their pairwise distancematrix Dn⇥n and embed the models to R2 using multi-dimensional scaling (MDS) based on singular value de-composition (SVD), to get {Yj}. For any other modelMi 2 M, we compute its k nearest neighbors among thelandmark models. We then interpolate the embedded co-ordinates of the landmark models to compute the embed-ding of Mi, i.e., Yi Â j=1:k w jYj/Â j=1:k w j, where w j :=exp(�d(Mi,M j)

2/2s2), with s set to the diameter of set{Xi} and M j denoting the j-th closest of the landmark mod-els .

The above embedding method has a complexity of O(n2+Nk logn). For example, for a chair dataset with 2036 models,the sparse embedding takes about 0.6 sec, which is about 12times faster than full embedding (see Figure 6).

Abstracting missing shapes. At this stage, we havemapped the initial coordinates {Xi}! {Yi}. We solve forthe dominant linear variation modes e1 and e2 such thatYi ⇡ [(Xi · e1),(Xi · e2)] for all i 2 [1,N] using a least squaresformulation. Now, given any point (a,b), we can lift up the

full embedding sparse embedding(n=200)

distance

selected cluster radius

an input modelsynthesizedabstraction

Figure 6: We embed the input models using their corre-sponding fitted template-based abstractions. We perform anefficient landmark-based embedding and analyze the pointsto obtain a parameterized template abstracting the underly-ing shape space. As the user navigates the embedded space,the extracted variation modes are used to lift the points(shown in red) to synthesize template abstractions. The dis-tribution of the pairwise distances between the embeddedpoints is used to estimate a suitable clustering radius.



startingtemplates

embedding(level 1)

embedding(level 2)

embedding(level 3)

representative models(level 1)

representative models(level 2)

main variationmodes (level 3)

Figure 7: A typical hierarchical exploration session using our interface. After the initial analysis, the system displays thetop level templates (top-left). As the user selects the green mode, its member models are embedded (level 1) and 4 dominantclusters are detected. The user selects the next representative and its member models are re-embedded. When a single cluster isdiscovered, its representative and dominant variation modes are shown (bottom-right).

point (from the empty region) to the configuration vector as(a,b)! ae1 + be2, thus providing an abstracted model Xas a preview for the empty region.

Grouping models. We cluster the embedded points usingmean-shift clustering [CM02] in order to organize the datainto a hierarchy. We automatically select the clustering ra-dius based on the histogram of the pairwise distances be-tween the embedded points. Intuitively, in the case of pointsthat can be grouped into multiple clusters, we can estimatea good clustering radius based on the first valley (if any) ofthe histogram (see Figure 6). Each of the extracted clustersis then re-embedded to extract corresponding basis vectors(i.e., e1 and e2), eventually forming a hierarchical organiza-tion (see Figure 7). As the user selects one of the clusters,she zooms into that particular cluster in order to better studythe fine scale variations.

Constrained synthesis. A direct derivation of box config-uration X = ae1 + be2 from the embedding space can eas-ily result in models that deviate from a semantically validone, e.g., symmetry being broken, part-to-part contacts be-ing lost, etc. We project the box configuration parametersto the valid shape space using a constrained optimization.We observe that many relations of interest (c.f., [MWZ⇤13])simply amount to linear constraints involving parameters ofthe configuration vector, e.g., contact, reflective symmetryabout known plane, equal dimensions of certain parts, etc.Our goal is to obtain a new configuration X such that poten-tial symmetry and contact relations among parts are restored,while X being as close to X as possible (see Figure 8). Thisamounts to solving the following minimization:

minXkX�Xk2 such that f j(X) = 0 8 j = 1, . . .c (2)

where, fi(X) is a set of c semantic constraints derived fromthe relations among the parts. We support three main typesof relations: symmetry, contact, and equal length.

Symmetry. Let two boxes, say (ci,si) and (c j,s j), have re-flective symmetry with respect to a given plane. The corre-sponding constraints take the form:

((ci + c j)/2�o) ·n = 0; (ci� c j)⇥n = 0; si� s j = 0

where n and o are the normal to the reflection plane and apoint on the plane, respectively.

X=

symmetry constraint

contact constraint

length constraint

synthesis withoutconstraints

synthesis withconstraints

Figure 8: Illustrative example of the different constraintshandled in our framework. (Left) In this 2D example, theconfiguration vector X 2 R12 represents the abstractedmodel with 3 parts. For example, the two contact constraintsinvolve the orange-green and orange-blue boxes and hencethe corresponding fi(X) involves the corresponding coor-dinates of X. (Right) Our system restores these constraintsduring the real-time exploration using a QP formulation.



Contact. When two boxes are in contact, they share a com-mon contact point. Between two boxes in contact, we as-sign the closest points as contact points. Alternatively, theuser can directly specify the contact points. Thus, betweentwo boxes, the contact constraint takes the form: ci + si/2 =c j + s j/2 (up to sign changes due to which corners get se-lected).

Equal length. In certain cases, we want a pair of boxes tohave similar dimensions (for example, the legs of a chairshould have equal height even if they are not symmetric).Since the abstracted boxes are axis-aligned, such a constrainttakes the form: sy

i = syj when equality along y-direction is

desired.

The optimization in Equation 2 amounts to solving aquadratic program with linear constraints. As the user ex-plores the configuration space extracted from the inputcollection, we perform the optimization to directly showthe constrained solution (see Figure 8 and supplementaryvideo). Note that the input templates, i.e. {Xi}, do not nec-essarily satisfy the constraints. However, we do not projectthem to the constrained space since the model parts are laterdeformed to a constrained model, as described next.

Synthesizing models. As the user moves the mouse over apoint (a,b), our system shows the corresponding abstractedbox model, ae1 +be2, which is then constrained to produceabstracted model X . Each such feature vector X 2 R6t rep-resents concatenated parameters for t boxes X = [x1, . . .xt ],where xi is a 6-dimensional vector that encodes positionand dimensions of the i-th box. When the user is satis-fied with the coarse arrangement of parts, she can click andlock the system to the current box model. This immediatelyprompts our system to fill the boxes with geometric partsthat have the most similar positions and dimensions (i.e.,argminxpart kxpart�xik), which essentially picks parts that areto be least deformed. The user can continue exploration andvisualize alternative part arrangements drawn over the se-lection, or refine the choice of selected parts by clicking ona corresponding box xi in the model view. The click promptsthe system to cycle through the candidate parts, where theparts are taken from the k nearest models M j from the se-lected model X (see Figure 9). One can use geometric simi-larity to further sort the selected box xi.

5. Evaluation

In this section, we evaluate the proposed shape synthesistool. First, we describe the data and experimental setup, andevaluate performance of our algorithm on diverse datasetsobtained from 3D Warehouse. While our coupled analysis,exploration, and synthesis framework is the first of its kind,we compare parts of our system to state-of-art shape syn-thesis methods ( [CKGK11] and [JTRS12]) and evaluate ourconstrained synthesis algorithm.

a b

c

d

e

tmodel a

model b

model c

model d

model e

deformed a

deformed b

deformed c

deformed d

deformed e

Figure 9: Our system allows to preview possible geomet-ric realizations in an empty region around the embeddedpoints (top-right). Each of the retrieved models (models a-e) is deformed to match the query configuration (indicatedas a red box). Parts from the deformed models (middle) arethen combined to create different plausible shapes (right).

Dataset. We tested our framework using Trimble Ware-house models [Tri13] obtained from [KLM⇤13]. We se-lected a subset of models such that the template fitting en-ergy is below 30, and hence our final analysis included2062 chairs, 636 planes, and 114 bikes. As an additionaldataset, to compare with the authors’ implementation of syn-thesis method [CKGK11], we use their manually-segmenteddataset of 100 airplanes from Digimation ModelBank.

Results. Several participants used our system to explore ex-isting model collections and create new shape variations.Overall the feedback was positive and they created a varietyof different models (see supplementary). Most people appre-ciated the ability to quickly obtain a high-level overview ofthe modeling space, refine the selection to a region via hi-erarchical navigation, and finally obtain immediate previewof geometric realizations. Some users complained that thefinal models were at times disconnected making them lookunrealistic. This is expected as we only enforce contact con-straints at the abstracted level of the boxes, and not on theoriginal geometry.

User experiments. We evaluated our system on 3 large anddiverse datasets from 3D Warehouse. We asked 8 volunteersfrom a computer science department to use our system toperform an open-ended task on each dataset:



T1: Create the most diverse set of 5 shapes.Next, we validated the results by comparing to a randomselection of groups of 5 models from the existing datasets.Similarly to previous work [CKGK11, CKGF13], we re-cruited a different group of volunteers to compare randompairs of results created by users of our method and modelsbelonging to the random selection. For each pair of results,we described the task T1 and asked (i) if the resulting shapeslook plausible, and (ii) if the resulting shapes look diverse.For each question, the evaluator had an option of selectingone of the results, both, or none that satisfy the plausibilityor diversity condition.

User 1

User 6

User 3

Figure 10: Sets of models created in our user experiment(please refer to supplementary for full results). Our systemenables rapid synthesis of diverse models.

In Figure 10, we present user-created models in each cat-egory (all results are provided in the supplemental material).Table 1 also shows results of the validation. Note that ourmethod is comparable to real models in terms of plausibil-ity of shapes, and allows easy creation of diverse models incomparison to random selection among the input models. Asthe user clicks on the parameterized space, new shape varia-tions are immediately shown (see supplementary demo).

Table 1: User study on Task T1 comparing our method witha random selection from a dataset. Voting indicates num-ber of times users voted for our method vs random selection(where votes for both are summed with individual votes), andtimings are in minutes.

Voting TimePlausibility Diversity (min)

Dataset Our Rnd None O R N Obikes 77 64 2 69 58 11 5chair 48 104 4 100 19 5 4

planes 53 81 2 68 48 6 3

Comparison to Chaudhuri et al. [2011]. We compareagainst this system by focusing on high-level exploratorydesign tasks. We used the authors’ implementation of themethod on their dataset of 100 consistently pre-segmentedairplanes. Our initial analysis was modified to createdeformable templates from segmented models and fit thetemplates to all models without changing the segmentation.We recruited a different group of 10 volunteers with CSbackground and asked them to perform task T1, and:

T2: Create an airplane that is best suited to win adogfight (one-on-one aerial combat) in a computer game.This is the same task as in [CKGF13].

Each user performed both tasks using both systems in aconsistent order (T1, T2), while we randomly permuted theorder in which systems were used. Figure 11 summarizesthe results produced by the same user (selected at random)in both systems. We again validated the results using anothergroup of volunteers and summarize the statistics in Table 2.Although users of our system often produced implausiblemodels when aiming at diversity in task T1, often the re-sulting sets of models were deemed comparable in diversity.Furthermore, we found that once the users became familiarwith the space of shapes, they could very rapidly synthesizeairplanes prescribed in task T2 that are comparable to re-sults produced by Chaudhuri et al. [CKGK11]. We find thevariability result particularly surprising since our parameter-ization is based on coarse box-abstractions rather than ge-ometric details. Even then, our system exposes interestinghigh-level part placement variations.

Our

Chaudhuri et al. 2011

Task 2

Task 2Task 1

Task 1

Figure 11: Example models created by User 1 (picked atrandom) for comparison to Chaudhuri et al.[2011] system.Note that both sets of models created for task T1 containdiverse and plausible shapes, as was requested in the task.



Table 2: Comparison to Chaudhuri et al. [CKGK11], tasksT1 and T2 were accomplished with two different interfaces.Voting columns indicate number of time users voted for re-sults produced with our method vs their approach (whereindividual votes are summed with votes for both methods).

Voting TimePlausibility Accomplished (min)

Task Our Ch. None O C N O CT1 67 174 17 81 131 38 6 14T2 132 150 13 158 180 12 2 4

Comparison to Jain et al. [2012]. Our method can also beused to interpolate between pairs of shapes, as in [JTRS12].In Figure 12 we demonstrate two interpolations producedby our method, and our simplified implementation of theirmethod. A visible advantage of our method is that itproduces more plausible shape variations for intermediateshapes where the deformation is high (e.g., look at the chairlegs). The quality comes from: (i) appropriate part scal-ing to facilitate model assembly; and (ii) neighboring mod-els contributing to the intermediate models allowing richervariations. Finally, unlike Jain et al. [JTRS12], our analy-sis automatically segments the input models and establishespart-level correspondence. In the future, it will be interest-ing to use their contact and relation graphs to maintain themodel structure of the synthesized variations. It should alsobe noted that as our approach is data-driven, performanceimproves as the dataset grows. The example in Figure 12is illustrated on a very small dataset and hence this is anextreme case where our method can produce visible distor-tions. In practise, we did not observe this effect when work-ing with the full datasets.

Jain et al. 2012

Our

Source Target

Figure 12: We evaluate our method in a shape interpolationscenario such as in [JTRS12]. Note that our method is morerobust to strong deformations because it uses other shapesfrom the dataset to produce intermediate results.

Constrained synthesis. We evaluate the quality of our con-strained synthesis via a leave-one-out experiment. First,starting with model collection M, we compute its embed-ding e1,e2 and then select a particular model Mi embed-ded as (ai,bi). We then remove the model Mi and re-analyze M\Mi to obtain embedding f1, f2 and re-synthesizethe model as aif1 + bif2. We then compare the effect ofleaving out Mi on the embedding . We found the variationnegligible in most cases. This is not surprising since ourlandmark-based embedding, and hence the extracted dom-inant modes are mainly dictated by the n randomly selectedmodels. If the landmarks remain unchanged, the embeddingdoes not change. Even with different landmarks the changesare small as shown in Figure 6. Furthermore, we use the newembedding to reconstruct the box structure for the model Migiven its coordinates (ai,bi), and we found that the recon-struction error is within 1% of the original box dimensions.

Restoring contacts. In a postprocessing step, it is possibleto restore contacts using a simple post-processing step asshown in Figure 13. This step however is orthogonal to thefocus of this paper, and we leave it to future work to fullyevaluate its effect.

Figure 13: The synthesized models can be further refinedusing docker-based part deformation. In these examples, theparts of the chair and bike are brought back into contactbased on nearest part dockers.

6. Conclusions

We presented an analysis approach that extracts a template-based hierarchical parameterization for input model collec-tions. The extracted parameterization factors out part de-formations to enable intuitive exploration of the modelsand provides a high-level overview of the underlying shapespace. Subsequently, the analysis results are used for inter-active creation of novel shape variations, both at an abstractbox-level and also with geometric details. We evaluated oursystem using 4 datasets ranging from 100-2000 models with18 users synthesizing 500+ novel shapes, validated by 30+(different) people, with 164 pairwise comparisons.

In this work we focused on abstracting the input modelsby axis-aligned box-templates. Given that, our system is notsuitable for datasets where the coarse structure of the shapedoes not correlate with its functionality or desired proper-ties. In the future, we would like to study the use of ori-ented box-templates. Such an abstraction can be challeng-ing to parameterize, but can potentially provide better un-



derstanding of the underlying shape space, improved cluster-ing and more meaningful exploration and synthesis of novelshapes. Furthermore, we also plan to investigate supplement-ing the shape space with part-based geometric descriptors.The challenge is to appropriately combine the high-level boxdescriptors with low-level geometric descriptors. Lastly, theextracted parameterized space provides an abstracted repre-sentation of the underlying shape space: we plan to investi-gate other problems like pose estimation, scan completion,etc., as projections to this parameterized space.

Acknowledgements. We are grateful to EvangelosKalogerakis and the reviewers for their comments; Sid-dhartha Chaudhuri for helping with the comparisonto [CKGK11]. We thank Bongjin Koo and Yanir Kleimanfor their comments, and James Hennessey for the videovoiceover. This project was supported in part by a MarieCurie CIG and ERC Starting Grant SmartGeometry.

References[ACP03] ALLEN B., CURLESS B., POPOVIC Z.: The space of

human body shapes: reconstruction and parameterization fromrange scans. ACM Trans. Graph. 22, 3 (2003), 587–594. 2

[CKGF13] CHAUDHURI S., KALOGERAKIS E., GIGUERE S.,FUNKHOUSER T.: Attribit: Content Creation with Semantic At-tributes. In Proceedings of UIST (2013), pp. 193–202. 3, 8

[CKGK11] CHAUDHURI S., KALOGERAKIS E., GUIBAS L.,KOLTUN V.: Probabilistic reasoning for assembly-based 3Dmodeling. ACM Trans. Graph. 30, 4 (2011), 35:1–35:10. 2, 3, 7,8, 9, 10

[CM02] COMANICIU D., MEER P.: Mean shift: a robust ap-proach toward feature space analysis. IEEE PAMI 24, 5 (2002),603–619. 6

[dST04] DE SILVA V., TENENBAUM J. B.: Sparse multidimen-sional scaling using landmark points. Tech. rep., Stanford Uni-versity, 2004. 5

[ERB⇤12] EITZ M., RICHTER R., BOUBEKEUR T., HILDE-BRAND K., ALEXA M.: Sketch-based shape retrieval. ACMTrans. Graph. 31, 4 (2012), 31:1–31:10. 1, 2

[FH10] FISHER M., HANRAHAN P.: Context-based search for3D models. ACM Trans. Graph. 29, 6 (2010), 182:1–182:10. 2

[FKS⇤04] FUNKHOUSER T., KAZHDAN M., SHILANE P., MINP., KIEFER W., TAL A., RUSINKIEWICZ S., DOBKIN D.: Mod-eling by example. ACM Trans. Graph. 23, 3 (2004), 652–663. 2,3

[FMK⇤03] FUNKHOUSER T., MIN P., KAZHDAN M., CHEN J.,HALDERMAN A., DOBKIN D., JACOBS D.: A search engine for3D models. ACM Trans. Graph. 22, 1 (2003), 83–105. 1, 2

[HSS⇤13] HUANG S.-S., SHAMIR A., SHEN C.-H., ZHANG H.,SHEFFER A., HU S.-M., COHEN-OR D.: Qualitative organiza-tion of collections of shapes via quartet analysis. ACM Trans.Graph. 32, 4 (2013), 71:1–71:10. 1, 2

[JTRS12] JAIN A., THORMÄHLEN T., RITSCHEL T., SEIDELH.-P.: Exploring Shape Variations by 3D-Model Decompositionand Part-based Recombination. Computer Graphics Forum 31,2pt3 (2012), 631–640. 3, 7, 9

[KCKK12] KALOGERAKIS E., CHAUDHURI S., KOLLER D.,KOLTUN V.: A Probabilistic Model for Component-Based ShapeSynthesis. ACM Trans. Graph. 31, 4 (2012), 55:1–55:11. 2, 3

[KFLCO13] KLEIMAN Y., FISH N., LANIR J., COHEN-OR D.:Dynamic Maps for Exploring and Browsing Shapes. ComputerGraphics Forum 32, 5 (2013), 187–196. 1, 2

[KJS07] KREAVOY V., JULIUS D., SHEFFER A.: Model Com-position from Interchangeable Components. In Proceedings ofthe Pacific Conference on Computer Graphics and Applications(2007), pp. 129–138. 2, 3

[KLM⇤12] KIM V. G., LI W., MITRA N. J., DIVERDI S.,FUNKHOUSER T.: Exploring Collections of 3D Models usingFuzzy Correspondences. ACM Trans. Graph. 31, 4 (2012), 54:1–54:11. 2, 5

[KLM⇤13] KIM V. G., LI W., MITRA N. J., CHAUDHURI S.,DIVERDI S., FUNKHOUSER T.: Learning part-based templatesfrom large collections of 3D shapes. ACM Trans. Graph. 32, 4(2013), 70:1–70:12. 4, 5, 7

[LF08] LEE J., FUNKHOUSER T.: Sketch-Based Search andComposition of 3D Models. In Eurographics Workshop onSketch-Based Interfaces and Modeling (2008), pp. 97–104. 2

[MWZ⇤13] MITRA N. J., WAND M., ZHANG H., COHEN-ORD., BOKELOH M.: Structure-Aware Shape Processing. In Euro-graphics State-of-the-art Report (2013), pp. 175–197. 5, 6

[OLGM11] OVSJANIKOV M., LI W., GUIBAS L., MITRA N. J.:Exploration of Continuous Variability in Collections of 3DShapes. ACM Trans. Graph. 30, 4 (2011), 33:1–33:10. 2

[ROA⇤13] RUSTAMOV R., OVSJANIKOV M., AZENCOT O.,BEN-CHEN M., CHAZAL F., GUIBAS L.: Map-Based Explo-ration of Intrinsic Shape Differences and Variability . ACM Trans.Graph. 32, 4 (2013), 72:1–72:12. 2

[SBSCO06] SHARF A., BLUMENKRANTS M., SHAMIR A.,COHEN-OR D.: SnapPaste: an interactive technique for easymesh composition. The Visual Computer 22, 9-11 (2006), 835–844. 3

[STP12] SFIKAS K., THEOHARIS T., PRATIKAKIS I.: Non-rigid3D object retrieval using topological information guided by con-formal factors. The Visual Computer 28, 9 (2012), 943–955. 2

[SXY⇤11] SHAO T., XU W., YIN K., WANG J., ZHOU K., GUOB.: Discriminative Sketch-based 3D Model Retrieval via RobustShape Matching. Computer Graphics Forum 30, 7 (2011), 2011–2020. 2

[TGY⇤09] TALTON J. O., GIBSON D., YANG L., HANRAHANP., KOLTUN V.: Exploratory Modeling with Collaborative De-sign Spaces. ACM Trans. Graph. 28, 5 (2009), 167:1–167:10. 1,3

[Tri13] TRIMBLE: Trimble 3D warehouse,http://sketchup.google.com/3dwarehouse/ 2013. 4, 7

[WP95] WEBER J., PENN J.: Creation and rendering of realistictrees. In Proceedings of SIGGRAPH (1995), pp. 119–128. 2

[XCF⇤13] XU K., CHEN K., FU H., SUN W.-L., HU S.-M.:Sketch2Scene: sketch-based co-retrieval and co-placement of 3Dmodels. ACM Trans. Graph. 32, 4 (2013), 123:1–123:15. 2

[XHCOB12] XU K., HAO Z., COHEN-OR D., BAOQUAN C.: Fitand Diverse : Set Evolution for Inspiring 3D Shape Galleries.ACM Trans. Graph. 31, 4 (2012), 57:1–57:10. 3

[XXM⇤13] XIE X., XU K., MITRA N. J., COHEN-OR D.,GONG W., SU Q., CHEN B.: Sketch-to-Design: Context-BasedPart Assembly. Computer Graphics Forum 32, 8 (2013), 233–245. 3

[ZCOM13] ZHENG Y., COHEN-OR D., MITRA N. J.: SmartVariations: Functional Substructures for Part Compatibility.Computer Graphics Forum 32, 2pt2 (2013), 195–204. 2, 3


shapesynth: parameterizing model collections for coupled...

Documents