Deep Detail Enhancement for Any Garment

MENG ZHANG, University College LondonTUANFENG WANG, miHoYo Inc.DUYGU CEYLAN, Adobe ResearchNILOY J. MITRA, University College London and Adobe Research

(a) coarse simulation (b) our result (c) failed highresolution simulation

(e) our result (d) skinning baseddeformation

Fig. 1. We present a data-driven approach to synthesize plausible wrinkle details on a coarse garment geometry. Such coarse geometry can be obtained eitherby running a physically-based simulation on a low resolution version of the garment (a), or by using a skinning-based deformation method (d). Our methodcan generate fine-scale geometry (b,e) by replacing the expensive simulation process, which, in some cases, are not even currently feasible to setup. Forexample, high-resolution simulation failed (c) for the blue dress as boundary conditions between the blue dress and the yellow belt were grossly violated. Oursgeneralizes across garment types, o�en with di�erent number of patches, associated parameterization and materials, and undergoing di�erent body motions.

Creating �ne garment details requires signi�cant e�orts and huge computa-tional resources. In contrast, a coarse shape may be easy to acquire in manyscenarios (e.g., via low-resolution physically-based simulation, linear blendskinning driven by skeletal motion, portable scanners). In this paper, weshow how to enhance, in a data-driven manner, rich yet plausible detailsstarting from a coarse garment geometry. Once the parameterization of thegarment is given, we formulate the task as a style transfer problem over thespace of associated normal maps. In order to facilitate generalization acrossgarment types and character motions, we introduce a patch-based formula-tion, that produces high-resolution details by matching a Gram matrix basedstyle loss, to hallucinate geometric details (i.e., wrinkle density and shape).We extensively evaluate our method on a variety of production scenariosand show that our method is simple, light-weight, e�cient, and generalizesacross underlying garment types, sewing pa�erns, and body motion.

CCS Concepts: •Deep Learning→ Motion Generation; Garment Mo-tion Transfer;

Additional Key Words and Phrases: garment simulation, style transfer, Grammatrix, detail enhancement, generalization

1 INTRODUCTIONHigh �delity garments are a vital component of many applicationsincluding virtual try-on and realistic 3D character design for games,VR, and movies. In real life, garments, as they undergo motion,develop folds and rich details, characteristic of the underlying mate-rials. Naturally, we desire to carry over such realistic details to thedigital world as well.

Directly capturing such �ne geometric details requires profes-sional capture setups along with computationally-heavy processingpipeline [Bradley et al. 2008; Chen et al. 2015; Pons-Moll et al. 2017],which may not be applicable for low-budget scenarios. An alternateoption involves physically-based cloth simulation [Liang et al. 2019;Narain et al. 2012; Tang et al. 2018] to virtually replace the need forphysical setups. However, despite many research advances, achiev-ing high quality and robust simulation still has many challenges.�e simulation is o�en sensitive to the choice of parameters andinitial conditions and may require a considerable amount of manualwork to setup correctly. Even when setup, it remains a computation-ally expensive process, especially as the resolution of the garmentgeometry increases (see Figure 2). Further, with increasing com-plexity of the garment geometry, e.g., garments with many layersor with non-canonical UV layout, it may be di�cult to obtain anystable result (see Figure 1(c)). Finally, for each new garment typeand/or motion sequences, majority of the setup and simulation stepshave to be repeated.

We propose a di�erent approach. As garment undergoes motion,its geometry can be analyzed in two steps. First, the correspondingmotion of the body (i.e., the body pose) changes the global shape ofthe garment, e.g., its boundary and coarse folds. We observe that theglobal shape to be consistent across di�erent resolutions of a gar-ment (see Figure 2). Such a global geometry can either be simulatede�ciently and reliably using a coarsened version of the garment, orby utilizing a skinning based deformation method. Given the coarse

arX

iv:2

008.

0436

7v1

[cs

.GR

] 1

0 A

ug 2

020

PD = 30 mm, 0.125 sec

PD = 20 mm, 0.158 sec

PD = 10 mm, 0.388 sec

PD = 5 mm, 1.812 sec

ours, (0.125+ 0.016) sec

Fig. 2. We show a garment simulated with di�erent mesh resolutions, i.e.,varying particle distances (PD), and report the computation time for simu-lating a single frame. While wrinkle details are captured well when simulat-ing high resolution meshes, i.e., lower PD, computation time significantlyincreases. Our method captures statistically similar quality details by en-hancing a coarse simulation much more e�iciently (13x speedup here).

geometry, the high frequency wrinkles can be perceived by consid-ering only local neighborhood information. As shown in Figure 2,the highlighted local patches in the coarse garment geometry pro-vide su�cient cues to hallucinate plausible high frequency wrinklesin the same region. Hence, we hypothesize that given the coarsegeometry of a garment, the dynamics of the detailed wrinkles canbe learned from many such local patches observed from di�erenttype of garments undergoing di�erent motions.

In this work, we present a deep learning based method that en-hances an input coarse garment geometry by synthesizing realisticand plausible wrinkle details. Inspired by the recent success ofimage-based artistic style transfer methods [Gatys et al. 2016], werepresent the garment geometry as a 2D normal map where weinterpret the coarse normal map as the content and �ne geomet-ric details as a style that can be well captured by the correlationbetween VGG features captured in the form of Gram matrices, char-acteristic to di�erent materials. Furthermore, in order to tacklevarious garment materials within a universal framework, we adoptthe conditional instance normalization technique [Dumoulin et al.2016] that is proposed in the context of universal style transfer.Finally, we design our detail enhancement network to operate ata patch level enabling generalization across di�erent (i) garmenttypes, (ii) sewing pa�erns, and (iii) underlying motion sequences. Infact, we show that our network trained on a dataset generated fromonly a single sheet of cloth by applying random motion sequences(e.g., applying wind force, being hit by a ball) can already reasonablygeneralize to various garment types such as a skirt, a shirt, or adress. See Figure 11 for a comparison.

We evaluate our approach on a challenging set of examplesdemonstrating generalization both across complex garment types(e.g., with layering) as well as a wide range of motion sequences.In addition to enriching low resolution garment simulations withaccurate wrinkles, we also demonstrate that our method can en-hance coarse garment geometries obtained by skinning methodsas o�en used by computer games, see Figure 1. To our knowledge,we present the �rst learning-based method for generating garmentdetails that generalizes to unseen garment types, sewing pa�erns,and body motions all at the same time.

2 RELATED WORKRealistic and physically based cloth simulation has been extensivelystudied in the graphics community [Choi and Ko 2005; Liang et al.2019; Narain et al. 2012; Nealen et al. 2006; Tang et al. 2018; Yuet al. 2019]. However, due to the associated computational costand stability concerns, as the complexity of the garments increase,several alternative avenues have been explored. For example, inthe industrial se�ings, skinning based deformation of garmentsis o�en favored for its simplicity, robustness, and e�ciency. �is,however, comes at the expense of losing geometric details. Finally,there has been various e�orts to augment coarse simulation outputwith �ne details using various approaches including optimization,data-driven, and learning-based methods.

Optimization-based methods. Muller et al. [2010] use a constraint-based optimization method to compute a high resolution mesh froma base coarse mesh. Rohmer et al. [2010] uses stretch tensor of thecoarse garment animation as a guide to place wrinkles in a post-processing stage. However, they assume the underlying garmentmotion is smooth which is not always the case with character bodymotion. Gille�e et al. [2015] extends similar ideas to real timeanimation of cloth.

Data-driven methods. Feng et al. [2010] present a deformationtransfer system that learns a non-linear mapping from a coarse gar-ment shape to per-vertex displacements that represent �ne details.Zurdo et al. [2012] share a similar idea to learn the mapping from alow-resolution simulation of a cloth to high resolution where plausi-ble wrinkles are de�ned as displacements. Given a set of coarse andmid-scale mesh pairs, Kavan et al. [2011] learn linear upsamplingoperators. Wang et al. [2010] construct a database of high reso-lution simulation examples generated by moving each body jointseparately. Given a coarse mesh, wrinkle regions for each joint aresynthesized by interpolating between database samples and blendedinto a �nal mesh. However, this method is limited to tight ��ingclothing. One of the most comprehensive system in this direction isthe DRAPE [Guan et al. 2012] system that learns a linear subspacemodel of garment deformation driven by pose variation. Hahn etal. [2014] extend this idea by using adaptive bases to generate richerdeformations. Xu et al. [2014] introduce a pose-dependent riggingsolution to extract nearby examples of a cloth and blend them in asensitive-based scheme to generate the �nal drape.

Learning-based methods. With the recent success of deep learn-ing methods in various imaging and 3D geometry tasks, a recentresearch direction is to learn garment deformations under body mo-tion using neural networks. A popular approach has been to extendparametric human body models with a per-vertex displacementmap to capture the deforming garment geometry [Alldieck et al.2019a; Bhatnagar et al. 2019; Jin et al. 2018; Pons-Moll et al. 2017].While this is a very e�cient representation, it only works well fortight clothes such as t-shirt or pants that are su�ciently close tothe body surface. Gundogdu et al. [2019] present GarNet whichlearns features of garment deformation as a function of body poseand shape at di�erent levels (point-wise, patch-wise, and globalfeatures) to reconstruct the detailed draping output. Santesteban

2

et al. [2019] learn a garment deformation model via recurrent neu-ral networks that enables real-time virtual try-on. �eir approachlearns the coarse garment shape based on the body shape and thedetailed wrinkles based on the pose dynamics. A similar two-stagestrategy is proposed by Patel et al. [2020] which decomposes thedeformation caused by body pose, shape, and garment style, intohigh frequency and low-frequency components. Wang et al. [2019]also consider garment deformations due to material properties intheir motion-driven network architecture. Others [Holden et al.2019; Wang et al. 2018; Yang et al. 2018] investigate a subspace tech-nique such as principal component analysis, to reduce the numberof variables of the high dimensional garment shape space. Whilethese learning-based methods are e�cient and fully di�erentiablecompared to standard physically-based simulation, they are hardto generalize across di�erent garment types and associated sewingpa�erns. In this work, we seek to combine the best of both worlds,physically-based simulation and learning-based approach. Speci�-cally, we rely on physically-based simulation for fast coarse garmentgeneration, we synthesize �ne geometry conditioned on the coarseshape by a neural network. By utilizing a patch-based approach,and training on synthetic data, we demonstrate that our trainedmodel generalizes across di�erent garment types as well as di�er-ent material con�gurations, and also extends to coarse geometryobtained using linear blend skin.

One of the closest learning based methods to our approach isDeepWrinkles [Lahner et al. 2018] which augments a low-resolutionnormal map of a garment captured by RGB-D sensors with high-frequency details using PatchGAN. However, their model is garment-speci�c and falls short when applied to garments unseen duringtraining (e.g., garments with a di�erent uv-parameterization ormaterial properties). In Section 5, we show a comparison of Deep-Wrinkle to contrast against ours generalization ability, wherein weuse conditional instance normalization to further adapt to mate-rial speci�cations and directly match Gram matrix based features,instead of matching style using a GAN.

Generalizing learning based methods to di�erent garment typeswith potentially di�erent number of vertices, connectivity, and ma-terial properties is an important challenge. In order to address thischallenge, the recent work of Ma et al. [2019] adopts a one-hotvector to encode the pre-de�ned garment type and trains a graphconvolutional neural network with a generative framework to de-form the body mesh into the garment surface. Zhu et al. [2020]train an adaptable template for image based garment reconstructionand introduce a large garment dataset with more than 2K real 3Dgarment scans spanning over 10 di�erent categories. �is methodgenerates plausible results if an initial template with a reasonabletopology is provided. Alldieck et al. [2019b] use normal and displace-ment maps based on a �xed body UV parameterization to representthe surface geometry of static garments. �e �xed parameteriza-tion assumption makes it di�cult to generalize the method to newgarment designs, especially more complex garments such as multi-layer dresses. We also utilize a normal map representation, however,our patch-based method focuses on local shape variations and isindependent of the underlying UV parameterization.

Image-to-image transfer. We draw inspiration from the recentadvances in image-to-image transfer methods [Fiser et al. 2016;Huang and Belongie 2017; Isola et al. 2017; Liao et al. 2017] aswe represent the garment geometry as a normal map and cast theproblem as detail synthesis in this 2D domain. Speci�cally, wefollow the framework proposed by [Gatys et al. 2015, 2016] thatcaptures the style of an image by the Gram matrices of its VGGfeatures [Simonyan and Zisserman 2014]. Similarly, we assume thata coarse normal map can be enhanced with �ne scale wrinkles bymatching the feature statistics of Gram matrices. Our work canbe loosely related to image super-resolution where, the aim is tosynthesize high-resolution images from blurry counterparts [Lediget al. 2017; Wang et al. 2015]. In Section 5, we provide comparisonwith a direct image space superresolution approach.

3 OVERVIEWOur goal is to augment a coarse garment geometry source sequence,S, with �ne details such as plausible and realistic wrinkles. Weassume that S is obtained by either a physically-based simulationrun on a low-resolution version of the garment or obtained by com-putational fast approaches such as skinning based deformation. �eoutput of our pipeline is a �ne-detailed surface sequence, S∗, whichdepicts the source garment with realistic wrinkles in a temporally-consistent manner. �e �ne wrinkles in S∗ are synthesized either bysharpening the original coarse folds in S or synthesized conditionedon nearby coarse folds.

Motivated by the recent advances in image style transfer [Gatyset al. 2016], we cast the 3D wrinkle synthesis task as a 2D detail trans-fer problem de�ned on the normal maps of the garments. Specif-ically, we present a learning-based method to learn an image-to-image transfer network Φ that synthesizes a detail-rich normal mapN∗ corresponding to S∗ given the coarse normal map N of S bymatching the feature statistics using Gram Matrices (Section 4.1).

However, unlike previous work [Lahner et al. 2018], which fo-cuses on learning garment-speci�c models using PatchGAN, ourgoal is to train a universal detail transfer network that can generalizeover di�erent types of garments (e.g., dress, shirt, skirt) made of dif-ferent materials (e.g., silk, leather, ma�e) and undergoing di�erentbody motions (e.g., dancing, physical activities). First, in order toensure generalization across di�erent materials and body motions,we adopt the conditional instance normalization (CIN) approach,introduced by Dumoulin et al. [2016] in the context of artistic styletransfer. Speci�cally, given the material type θ of a garment, weshi� and scale the learned network parameters to adapt to di�er-ent materials and motions. Given a sequence of coarse garmentgeometry, we predict its material type θ via a classi�er, and do notrequire the material speci�cation as part of the input. Second, inorder to generalize across di�erent types of garments and di�erentuv-parameterizations utilized to generate the normal maps of thegarments, we propose a novel patch-based approach. Speci�cally,our network works at a patch level by synthesizing details on over-lapping patches randomly cropped from the normal map, N, of thegarment, S. We merge the resulting detailed patches to obtain the�nal normal map, N∗, which is li�ed to 3D via a gradient-based opti-mization scheme to yield S∗ (Section 4.3). Our pipeline is illustratedin Figure 3.

3

0000`00`

θ

θ,

Detail Enhancement (Section 4.1)

Material Classi�cation (Section 4.2)

3D Recovery (Section 4.3)

coarse geometry (S)

coarse normal map (N)

�nalgeometry (S*)

enhancednormal map (N*)

silkchi�on

convolutionrelu

maxpool CIN(θ)

VGG19

fully connectedsigmoid

Fig. 3. Given a coarse garment geometry S represented as a 2D normal map N, we present a method to generate an enhanced normal map N∗ and thecorresponding high resolution geometry S∗. At the core of our method is a detail enhancement network that enhances local garment patches in N conditionedon the material type θ of the garment. We combine such enhanced local patches to generate N∗ which captures fine wrinkle details. We li� N∗ to 3D togenerate S∗ by an optimization method that avoids interpenetrations between the garment and the underlying body surface. In case the garment material isnot known in advance, we present a material classification network that operates on the local patches cropped from the coarse normal map N.

4 ALGORITHM STAGESGiven a coarse 3D garment geometry sequence S, for each frameSi ∈ S, our goal is to synthesize S∗i , which enhances the garmentgeometry with �ne wrinkle details, subject to the underlying bodymotion and the assigned garment material θ (see Section 4.2). Notethat given the nature of garments, any Si can be semantically cutinto a few developable pieces, i.e., garment pa�erns. We assume sucha parameterization is provided for the �rst frame S0 and applicableto the remaining frames in the sequence.

4.1 Detail EnhancementUsing the garment-speci�c parameterization, we represent each Siby a normal map Ni . We treat the detail enhancement problem asgenerating a normal map N∗i conditioned on Ni and the materialcharacteristics θ . While N∗i preserves the content of Ni (e.g., thedensity and the position of the wrinkles), the appearance of thedetails (e.g., shape, granularity) are learned from {Nθ }, a collectionof normal maps depicting the detailed geometry of various gar-ments of material θ undergoing di�erent body motions. �e detailenhancement step operates at each frame, hence, for simplicity, weignore the subscript i in the following.

Detail Enhancement Network. We adopt a U-net architecture forour detail enhancement network (see Figure 3). �e encoder projectsthe input N into a learned latent space. �e latent representation zis then used to construct the output N∗ via the decoder. Althoughgarments made of di�erent materials may share similar coarse ge-ometry, the �ne details can vary, conditioned on the material. Toadapt our approach to di�erent garment materials and capture �nedetails, we use θ , a one-hot vector, as input to indicate the type ofmaterial N and N∗ are associated with. We design the decoder usingconditional instance normalization (CIN) [Dumoulin et al. 2016] toadapt for di�erent material types. Speci�cally, given a material typeθ , the activation of a layer i is converted to a normalized activation

zθi speci�c to a material type θ according to,

zθi := γ θi

(xi − µ(xi )σ (xi )

)+ ϵθi ,

where µ and σ are the mean and standard deviation of xi , and γ θiand ϵθi are material-speci�c scaling and shi�ing parameters learnedfor the corresponding layer i .

Loss Function. In general, the loss function for a style transfertask is composed of two parts: (i) content signal, which in ourcase penalizes the di�erence between N∗ and N with respect to thecoarse geometry features; and (ii) style term, which in our casepenalizes the perceptual di�erence between N∗ and {Nθ }, a setof training examples that depict detailed garment geometries ofmaterial θ . As indicated by Gatys et al. [2016], content and stylelosses can be formulated using the output of a prede�ned set oflayers, LC and LS , of a pre-trained image classi�cation network, e.g.,VGG19 [Simonyan and Zisserman 2014]. Note that the de�nitionof content and style in our problem is not the same as in a classicalimage style transfer method. We performed a qualitative study to�nd the appropriate layers LC and LS . Speci�cally, we pass examplesof two sets of normal maps that depict coarse and �ne geometryof garments of a particular material through VGG19. For candidatelayers, we embed the output features in 2D space using a spectralembedding method and identify the layers where the two sets arewell separated (see Figure 4).

Given LC and LS , we de�ne content loss between N and N∗ as

Lcontent(N∗ |N) :=∑l ∈LC[Fl [N∗] − Fl [N]]2/2,

where Fl is the VGG features of the selected layer LC .We de�ne the style loss based on the correlation of the VGG

features across di�erent channels by using Gram matrices. Sincethe input to our network is a normal map that is generated basedon a UV parameterization of the garment, it consists of background

4

coarse simulation high resolution simulation

coarsehigh res

coarsehigh res

Fig. 4. We pass a given set of normal map patches cropped from coarseand high resolution simulation results through VGG19. Here, we showspectral embedding in R2 of the feature outputs of candidate layers, andselect the layers where the two sets that di�er significantly to compute thestyle loss. We show the feature plots of two such layers. This di�erence islargely responsible for the drop in visual fidelity going from high resolutionto coarse simulation.

regions which do not correspond to any UV patch. In order to avoidsuch empty regions impacting the correlation measure, we adopt aspatial control neural style transfer method by using a UV mask Mthat masks out the background regions. We capture style by a setof Gram Matrices de�ned for each LS layer of VGG:

GMl (N) := (BMl ◦ Fl (N))

T (BMl ◦ Fl (N)),

where BMl is the background mask for layer l ∈ LS downsampledaccordingly. �e style loss is then formulated as:

Lstyle(N∗ |{Nθ },M) :=∑k ∈‖ {Nθ } ‖

∑l ∈{LS }

14R2

l

[GMl (N

∗) −GMl (N

kθ )]

2,

where Rl is the number of valid pixels in the down-sampled maskBMl .

�e �nal loss function we use to train the network is a weightedsum of the content and style losses:

LossΦ := wcLcontent(N∗ |N) +wsLstyle(N∗ |{Nθ },M),where wc and ws denote the weight of each term respectively. Inour tests, we use wc = 1 and ws = 104.

Patch-based generalization. Generalizing a CNN based neural net-work to di�erent types of clothes with unknown UV parameteriza-tion is challenging. Particularly, similar 3D garments (e.g., a shortand a long skirt) might potentially have very di�erent parameter-izations due to di�erent sewing pa�erns that vary in the numberof 2D pa�erns and their shape. In order to generalize our approachacross such varying 2D parameterizations, we adopt a patch-basedapproach. Speci�cally, instead of operating with complete normalmaps, we use patches cut from the normal maps as input and output

of our network. While we use randomly cut patches during training,at test time we use overlapping patches sampled regularly on the im-age plane. We resize each input patch to ensure the absolute size ofone pixel remains unchanged. �e output patches that are enhancedby our network are then combined to generate the �nal detailednormal map N∗ by averaging the RGB values in the overlappingregions of the output patches.

4.2 Material ClassificationWe introduce a patch-based material classi�cation method to predictthe material properties in such cases. Speci�cally, we train a patch-based material-classi�cation network, Π, which is composed of4 fully-connected layers as shown in Figure 3. �e input to thisnetwork is the �a�ened feature vector of VGG calculated froma patch cropped from N. �e dimension of the last layer is thenumber of the materials captured in our dataset. �e last layeris followed with a Sigmoid function and a normalization layer tooutput the probability of the input patch belonging to each material.During training, we use Cross-Entropy loss between the predictedprobabilities and the ground truth material type.

Humans perceive the characteristics of a material when they ob-serve a garment in motion. Similarly, we observe that performingmaterial classi�cation on a single patch is not su�ciently accu-rate [Yang et al. 2017]. Hence, for a given input garment sequence,we choose randomly cropped patches and run material classi�cationon each of them sampled over time. We use a voting strategy todecide the �nal material type from all such individual predictionsas shown in Figure 5. Note that in application scenarios, e.g., if thecoarse garment geometry is obtained by deforming a template vialinear blend skinning, the material classi�cation frees the user fromhaving to guess material parameters.

4.3 3D RecoveryOnce the enhanced normal map N∗ is calculated, we �nally li� thedetails into 3D. Since coarse geometry S is known, one commonsolution in the CG industry is to bake the normalmap onto the

# of patches

probabilities

coarse normal map

probability of material 1

material 0material 1material 2material 3material 4

Fig. 5. For an input coarse normal map, we crop 42 patches as the input ofour material classification network. For each material type in our dataset,We plot the histogram of patches based on the probability of them belongingto that material. As shown, material 1 has the highest distribution amongthe five materials. We also color code the patches based on their probabilityof belonging to material 1, where darker color means a higher probability.

5

original coarse surface during rendering. �e skinning based ex-amples shown in Figure 7 and 8 are rendered using this approach.On the other hand, real 3D results are useful in many follow upapplications, such as animation, texture painting, etc. We adopt atwo-stage approach to recover the detailed 3D geometry for suchapplications.

Normal map guided deformation. We perform upsampling overthe surface of the coarse geometry S and deform the set of verticesP based on the enhanced normal map N∗. Our method uses aniterative gradient optimization scheme to �nd the deformed vertexpositions p which minimizes the following energy:

arg min{P }

∑p∈{P }

∑q∈R1

p

‖np ·q − p‖q − p‖ ‖

2+η∑

p∈{P }‖∆p‖2+ω

∑p∈{P }

‖p0−p‖2,

where np is the normal of vertex p as denoted on N∗, R1p is the

one-ring neighborhood of p, ∆ is the Laplacian operator, and p0 isthe initial position of p obtained by upsampling the coarse geometry,and η and ω are the weights of the last two regularization terms.While the Laplacian term acts as a smoothness term to avoid sharpdeformations, the last term penalizes the �nal shape from deviatingfrom the original geometry (e.g., by shi�ing or scaling).

Avoiding interpenetration. �e above deformation approach maycause interpenetrations between the garment and the underlyingbody surface. Hence, we adopt a post-processing step to resolvesuch penetrations as introduced by Guan et al. [2012]. Speci�cally,we �rst �nd the set of garment vertices PC ∈ P , which are insidethe body mesh. We project each such vertex p ∈ PC to the nearestlocation, p∗, on the body surface. We update the vertex positions psuch that the following objective function is minimized:

arg min{p }

∑p∈{PC }

‖p∗ − p‖2 + ϕ∑

p∈{P }‖Op‖2.

In our experiments, we use η = 5 × 103, ω = 8, and ϕ = 5 × 103.�e objective functions are optimized by standard gradient descentalgorithm.

4.4 Data GenerationOur goal is to generalize our method to a large variety of garmenttypes and motion sequences. To ensure such a generalization capa-bility, we generate our training data by performing physically-basedsimulation over 3 di�erent garment-motion pairs (see Figure 6).�ese are:

(1) a piece of hanging sheet interacting with 2 balls and blowingwind (901 frames at 30 fps);

(2) a long skirt over a virtual character performing Sambadance (988 frames at 30 fps);

(3) a pleated skirt over a virtual character performing moderndance (850 frames at 30 fps).

For the last two cases, the underlying virtual character is createdby Adobe Fuse1 and the motion sequences are obtained from Mix-amo2. We use Marvellous Designer3 to drape the garments over1h�ps://www.adobe.com/products/fuse.html2h�ps://www.mixamo.com/3h�ps://www.marvelousdesigner.com

single sheet of cloth long skirt pleated skirt

Fig. 6. Our base training data is simulated from a single sheet of clothinteracting with two balls and blowing wind. We also simulate a longskirt and a pleated skirt under di�erent body motions. For each trainingexample, we simulate both a coarse and a fine resolution mesh and obtainthe corresponding normal map pairs.

the 3D character body and perform the physically-based simula-tion. We simulate each garment with di�erent material samples (silkchamuse, denim lightweight, knit terry, wool melton, silk chi�on)and in two resolution se�ings. To obtain the coarse geometry, werun the simulation with a particle distance of 30mm. �is results inresulting meshes of 4440, 2287, 1756 vertices for the aforementioned3 types of garments respectively. To generate the �ne scale geome-try, we run the simulation with a particle distance of 10mm whichresults in meshes with 39330, 18950, 14433 vertices respectively. �enormal map of the 3D garment surface is then calculated. For eachexample, we generate the corresponding normal map using a givenuv parameterization for each garment type. We scale the normalmaps of di�erent examples to ensure a single pixel corresponds tothe same unit in 3D and randomly cut patches from the normal mapsand use each pair of non-empty coarse and high resolution patchesas input and target of our network. Finally, we also augment ourdataset by rotating and re�ecting the patches.

We observe that, sometimes due to instabilities in the physicalsimulation, simulation results with di�erent particle distances mayreveal geometric variations, e.g., in the position and the shape ofthe wrinkles. �is results in input and output patch pairs that arenot well aligned. To avoid this problem, we instead downsamplethe normal maps of the high resolution mesh to generate the coarsenormal maps that will be provided as input to our network. Duringtesting, the inputs to our network are obtained from the coarsesimulation results. Our experiments demonstrate that training withthe downsampled normal maps while predicting with the coarsenormal maps as input at test is a successful strategy.

Network architecture. Our network broadly consists of an encoderand a decoder. �ere are 4 layers in the encoder to project the inputpatch image to a compact latent space, by down-sampling from128 × 128 to 8 × 8. In each layer, there are blocks in the orderof 2D convolution, Relu, and maxpool. �e followed decoder isused to up-sample the patch tensor from 8 × 8 to the original size,mainly using 2D trans-convolutions, and a 2D convolution in thelast layer. Because di�erent materials share one network, we adoptCIN layer to scale and shi� the latent code to the speci�ed material

6

skin

ning

bas

edde

form

atio

nou

rre

sult

our

resu

ltco

arse

sim

ulat

ion

Fig. 7. We evaluate our method both on coarse garment simulations and garments deformed by linear blend skinning. Our method can generalize acrossdi�erent garment types and motions. The materials for the simulated garments are as follows: t-shirt (knit terry), pants (wool melton), skirt (silk chamuse),skirt laces (silk chi�on). For the skinning based example, our material classification network predicts it as silk chi�on. Please see supplementary video.

related space, a�er every 2D trans-convolution. Finally, we use skipconnection, as we regard our problem as constructing the residualfor the �ne image based on the coarse one. In our experiments, we

use the Adam optimizer, with a learning rate of 1e−4 and parametersβ1 = β2 = 0.9. Project code and data will be released.

7

skin

ning

bas

ed d

efor

mat

ion

our r

esul

tco

arse

sim

ulat

ion

our r

esul

tcoarse sim

ulationour result

Fig. 8. We evaluate our method both on coarse garment simulations and garments deformed by linear blend skinning. The materials for the simulatedgarments are as follows: hanfu (silk chamuse), hood (denim lightweight), pants (knit terry), vest (wool melton). For the skinning based example, our materialclassification network predicts it as knit terry. Please see supplementary video.

5 RESULTSWe evaluate our method on the product space of a variety of gar-ments undergoing various body motions. We present qualitativeand quantitative evaluation, comparison with baseline methods, andalso ablation study.

5.1 DatasetAs input, we test on coarse garment geometries obtained by eitherrunning a physically-based simulation on a low resolution mesh ofthe garment, as well as garments deformed via a skeleton rig usinglinear blend skinning (LBS). Similar to the dataset generation, we

8

coar

se s

imul

atio

n

coat: DL, skirt: SC

our r

esul

t

coat: SC, skirt: DL

aa

bb

c

c

d

d

e f

g h

e f

g h

coat: DL, skirt: SC coat: SC, skirt: DL

Fig. 9. We demonstrate how our method works on garments made of di�erent materials and undergoing the same motion. Note how the coarse inputs varywith material change and the synthesized details that align well with the input. The two di�erent materials used are denim light (DL) and silk chi�on (SC).

use motion sequences from Mixamo and utilize Marvelous Designeras our physical simulator. For skinning-based examples, we test ourmethod on a character from Mixamo, as well as a character fromthe Microso� Rocketbox dataset 4. We show qualitative examplesin Figures 7 and 8.

5.2 Evaluation�alitative evaluation. �e examples demonstrate the generaliza-

tion capability of our method to unseen garments and motions duringtraining time. Even for garment types that are signi�cantly di�erentfrom our training data, e.g., the multi-layer skirt in Figures 1, 7, andthe hijab in Figure 8, our network synthesizes plausible geomet-ric details an order of magnitude faster than running a physicallybased simulation with a high resolution mesh (see Figure 2). Whenevaluating our method on skinning-based rigged characters, sinceno garment material information is provided, we use our materialclassi�cation method to �rst predict plausible material parametersbased on the input normal maps. We then utilize this material infor-mation to synthesize vivid wrinkles that enhances the input garmentgeometry. Empirically, even when material speci�cation was avail-able, we found training end-to-end with the material classi�cationnetwork along with CIN resulted in faster convergence [Huang andBelongie 2017].

Material properties of a garment plays an important role in theresulting folds and wrinkles. Our method captures such deforma-tions by taking the material parameters as an additional input. InFigure 9, we show examples of garments made from di�erent ma-terials undergoing the same body motion. Our method is able togenerate plausible details in all cases that well respect the inputcoarse geometries.

�antitative evaluation. In order to evaluate our approach quanti-tatively, when ground truth data is available, we de�ne an improve-ment score given an input patch, (N), an output patch, (N∗), and the

4h�ps://github.com/microso�/Microso�-Rocketbox

corresponding ground truth patch, (N), as follows:

improvement score := 100

�����Lstyle(N) − Lstyle(N∗)Lstyle(N) − Lstyle(N)

����� .A higher improvement score, 100 being perfect, indicates that the

output patch is closer to the ground truth in terms of the styles ofthe wrinkles. Table 1 reports the mean and standard deviation ofthe improvement score for three di�erent garment types made fromtwo di�erent materials and undergoing two di�erent motion types.While one garment and motion type has been seen during training(dress A, motion 1), the remaining garment types and the motionsequence are unseen. Not surprisingly, the method performs well (∼98%) on seen garment types and motions. �e main result is the

Table 1. We report the mean and standard deviation of improvement scoreson three dresses made of two di�erent materials and undergoing two dif-ferent motion sequences. While Dress A and Motion 1 was seen duringtraining, the remaining dresses and the motion sequence are unseen. Bothmotions are sequences are composed of 200 frames.

MeanStd Dress A Dress B Dress C

Silk

cham

use Motion 1 98.41

0.8796.22

1.0191.87

3.02

Motion 2 97.551.53

95.390.92

91.254.56

Den

imlig

ht Motion 1 98.350.57

95.280.92

92.452.03

Motion 2 97.391.08

95.251.42

92.443.25

9

generalization to unseen data and achieves high improvement scoresregardless with only slight degradation in terms of performance(∼ 91 − 96%).

When ground truth is not available, we evaluate the performancefrom a statistics perspective. Given a sequence of animated gar-ments, we randomly crop patches from each frame and generatethe corresponding samples of Gram matrices of style features. Wecollect such samples from three sources: (a) patches from the coarseinput; (b) patches generated by our method; and (c) patches fromhigh resolution simulation (not for corresponding frames). We em-bed the patches using spectral embedding from R262144 → R2, andmeasure the distances in the distribution of the resultant point sets(denotes as Pa , Pb , and Pc , respectively) using Chamfer distance (CD)as C1 := CD(Pa , Pc ) and C2 := CD(Pb , Pc ), and improvement scorede�ned as DR := 100|C1 −C2 |/C1 in Table 2. As shown in the table,the Chamfer distance between our result and the ground truth is sig-ni�cantly lower than the distance between the coarse input and theground truth. �is indicates that our method not only enhances thedetails of individual frames, but also improves overall plausibilityfor a motion sequence.

Table 2. We report the Chamfer distance between (i) the coarse input andthe ground truth and (ii) our result and the ground truth. We calculate amatching score {DR := 100 |C1−C2 |/C1} to show the relative improvement.Please refer to the text for details. Note that Dress A and Motion 1 are seenduring training while the remaining dresses and the motion sequence areunseen. Dress C is tight, as indicated by the low Chamfer distance to start,and therefore leaves li�le scope to improve in terms of detail enhancement.

C1

C2 DRDress A Dress B Dress C

Silk

cham

use Motion 1

1.77

0.09 94.81

0.36

0.09 74.32

0.15

0.12 14.95

Motion 21.39

0.11 92.25

1.09

0.12 89.20

0.11

0.10 10.56

Den

imlig

ht Motion 11.31

0.01 92.54

0.89

0.13 85.86

0.13

0.12 6.36

Motion 21.50

0.12 92.33

0.93

0.11 88.23

0.10

0.09 7.70

Continuity in time. While we do use time information in the mate-rial classi�cation, the main detail enhancement step works at framelevel. In earlier versions of our network, we experimented with time-dependent networks, with GRUs and LSTMs. While the results werebe�er on training sets, we found the networks did not generalizeacross unseen data. In contrast, the proposed per-frame processingstill leverages continuity data in the coarse geometry sequence andproduces time-continuous details (please see supplementary video),with superior generalization behavior. We quantitatively evaluateour performance over continuous body motion, see Figure 6 (right).

PD30ground truth

resultground truth

input (PD30) result ground truth (PD10)

PD10 ds

PD10 downsampled

Fig. 10. Given an input normal map obtained by running a coarse simulationwith a particle distance of 30 mm (PD30), our network produces a resultthat is close to the ground truth (high resolution simulation run with aparticle distance of 10 mm) in terms of Gram matrix distribution of theoutput features of VGG19 layers that we use as style layers. We note thatwhen training our network, as input we use downsampled high resolutionnormal maps (PD10 ds) which have similar feature distribution as the coarsesimulation normal maps (PD30) used during test time.

5.3 Ablation StudyIn order to ensure coarse and high resolution patches are alignedduring training, we generate the input to our network by down-sampling the high resolution normal maps. We validate this choice,by visualizing the 2D embedding of the output features of some ofthe layers of VGG19 that we choose as style layers in Figure 10. Asshown in the �gure, the distribution of the features of the down-sampled high resolution normal maps are close to the actual coarsenormal maps. Further, given the downsampled normal maps as in-put, our method can successfully generate results that share similardistribution as the ground truth.

We also evaluate the performance of our method trained withdi�erent portions of our dataset. As shown in Figure 11, evenwhen trained with a single sheet of cloth, our method generalizes

# of

fram

espe

rcen

tage

percentage

frame indexcoarse input

trained withsingle sheet

trained withfull dataset

groundtruth

single sheet full dataset

Fig. 11. We show results of our method when trained with (i) data generatedfrom a single sheet only and (ii) all the three datasets (see Figure 6).

10

coarse simulation PS sharpeningimprov. score=60.31

super-resolutionimprov. score = 0.99

DeepWrinklesimprov. score=73.0

our resultimprov. score=95.61

ground truth

# of

fram

es

percentage

perc

enta

ge

frame index

DeepWrinkles ours

Fig. 12. Given a coarse garment geometry, we compare our results with image based methods, i.e., sharpening feature in Photoshop and image super-resolution [Yifan et al. 2018], as well as our implementation of the DeepWrinkles [Lahner et al. 2018] method. Our results are closest to the target groundtruth visually. We also outperform the alternative approaches in terms of the improvement score.

reasonably well to di�erent garment types. �e quality of the resultsimprove as the complete dataset is utilized.

5.4 ComparisonIn Figure 12, we compare our method to other alternative approaches.Speci�cally, we compare to two image-based enhancement meth-ods, namely image sharpening feature in Adobe Photoshop [Adobe2020] and a state-of-the-art image super-resolution method [Yifanet al. 2018]. We observe that image-based approaches cannot re-ally hallucinate wrinkle details and are not really suitable for ourproblem.

We further compare our method to the recent neural network-based approach of DeepWrinkles [Lahner et al. 2018]. We implementtheir method and train both their generative model and our networkwith a portion of our dataset, speci�cally with training data obtainedfrom the long skirt. While DeepWrinkles, which uses a patch-basedGAN setup, achieves similar performance during the training stage,it falls short when generalizing to unseen cases. We also reportthe improvement score across the whole motion sequence of eachmethod demonstrating the superiority of our approach quantita-tively.

our resultimprov. score=95.61

# of

fram

es

percentage

wrong material ourswrong material

improv. score=92.95

Fig. 13. We enhance the same coarse normal map respectively with right(silk chamuse) and wrong (wool melton) materials. The method cannotrecover subtle wrinkle details of the silk material when wrongly assigned adi�erent material.

Finally, given the same coarse normal map as input, we also pro-vide the result of our method when run with incorrect material

parameters and report the respective improvement scores in Fig-ure 13. Use of wrong material parameters results in less plausibleresults with around 5% drop in the improvement score.

6 CONCLUSIONS AND FUTURE WORKWe have presented a deep learning based approach to synthesizeplausible wrinkle details on coarse garment geometries. Our ap-proach draws inspiration from the recent success of image artisticstyle methods and casts the problem as detail transfer across thenormal maps of the garments. We have shown that our methodgeneralizes across di�erent garment types, sewing pa�erns, andmotion sequences. We have also trained and tested our methodwith di�erent material parameters under a universal framework.We have demonstrated results on inputs obtained from runningphysical simulation on low-resolution garment geometries as wellas garments deformed by skinning.

Limitations and Future work. While our method shows impressivegeneralization capability, it has several limitations that can be ad-dressed in future work. First of all, in order to generalize the detailenhancement network across di�erent materials, we provide thematerial parameters as input. Hence, at run time, the network canbe tested only on material types that have been seen during training.Generalizing our method to unseen materials, e.g., via learning animplicit material basis, is an interesting future direction. �e currentbo�leneck is ge�ing diverse simulations by sampling a variety ofrealistic materials due to challenges in ge�ing robust and realisticsimulations, in reasonable time, using commercial simulation frame-works. In our present implementation, our method is trained withregularly cropped 2D patches on the normal maps. Cropping geo-desic patches over 3D surface instead to minimize averaging errorsacross overlapping regions and integrating the post-processing stepinto the network is an exciting direction. Furthermore, we assumethat the coarse and high-resolution garments share the same set ofgarment pieces. In a possible garment design scenario, the designermight add some accessories to an existing garment, e.g., adding abelt or a layer to a skirt. It would be interesting to hallucinate thewrinkle details a�er such edits using the coarse simulation of thebase garment only.

11

REFERENCESAdobe. 2020. Adobe Photoshop CC. h�ps://www.adobe.com/products/photoshop.html�iemo Alldieck, Marcus Magnor, Bharat Lal Bhatnagar, Christian �eobalt, and Ger-

ard Pons-Moll. 2019a. Learning to reconstruct people in clothing from a singleRGB camera. In Proceedings of the IEEE Conference on Computer Vision and Pa�ernRecognition. 1175–1186.

�iemo Alldieck, Gerard Pons-Moll, Christian �eobalt, and Marcus Magnor. 2019b.Tex2shape: Detailed full human body geometry from a single image. In Proceedingsof the IEEE International Conference on Computer Vision. 2293–2303.

Bharat Lal Bhatnagar, Garvita Tiwari, Christian �eobalt, and Gerard Pons-Moll. 2019.Multi-garment net: Learning to dress 3d people from images. In Proceedings of theIEEE International Conference on Computer Vision. 5420–5430.

Derek Bradley, Tiberiu Popa, Alla She�er, Wolfgang Heidrich, and Tamy Boubekeur.2008. Markerless garment capture. In ACM SIGGRAPH 2008 papers. 1–9.

Xiaowu Chen, Bin Zhou, Fei-Xiang Lu, Lin Wang, Lang Bi, and Ping Tan. 2015. Garmentmodeling with a depth camera. ACM Trans. Graph. 34, 6 (2015), 203–1.

Kwang-Jin Choi and Hyeong-Seok Ko. 2005. Research Problems in Clothing Simulation.Comput. Aided Des. 37, 6 (May 2005), 585��592.

Vincent Dumoulin, Jonathon Shlens, and Manjunath Kudlur. 2016. A learned represen-tation for artistic style. arXiv preprint arXiv:1610.07629 (2016).

Wei-Wen Feng, Yizhou Yu, and Byung-Uck Kim. 2010. A Deformation Transformerfor Real-Time Cloth Animation. ACM Trans. Graph. 29, 4, Article Article 108 (July2010), 9 pages.

Jakub Fiser, Ondrej Jamriska, Michal Lukac, Eli Shechtman, Paul Asente, Jingwan Lu,and Daniel Sykora. 2016. StyLit: illumination-guided example-based stylization of3D renderings. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1–11.

Leon Gatys, Alexander S Ecker, and Ma�hias Bethge. 2015. Texture synthesis usingconvolutional neural networks. In Advances in neural information processing systems.262–270.

Leon A Gatys, Alexander S Ecker, and Ma�hias Bethge. 2016. Image style transfer usingconvolutional neural networks. In Proceedings of the IEEE conference on computervision and pa�ern recognition. 2414–2423.

Russell Gille�e, Craig Peters, Nicholas Vining, Essex Edwards, and Alla She�er. 2015.Real-Time Dynamic Wrinkling of Coarse Animated Cloth. In SCA (SCA ’15). 10.

Peng Guan, Lore�a Reiss, David A Hirshberg, Alexander Weiss, and Michael J Black.2012. Drape: Dressing any person. ACM Transactions on Graphics (TOG) 31, 4 (2012),1–10.

Erhan Gundogdu, Victor Constantin, Amrollah Seifoddini, Minh Dang, Mathieu Salz-mann, and Pascal Fua. 2019. Garnet: A two-stream network for fast and accurate3d cloth draping. In Proceedings of the IEEE International Conference on ComputerVision. 8739–8748.

Fabian Hahn, Bernhard �omaszewski, Stelian Coros, Robert W Sumner, Forrester Cole,Mark Meyer, Tony DeRose, and Markus Gross. 2014. Subspace clothing simulationusing adaptive bases. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–9.

Daniel Holden, Bang Chi Duong, Sayantan Da�a, and Derek Nowrouzezahrai. 2019.Subspace neural physics: fast data-driven interactive simulation. In Proceedings ofthe 18th annual ACM SIGGRAPH/Eurographics Symposium on Computer Animation.1–12.

Xun Huang and Serge Belongie. 2017. Arbitrary style transfer in real-time with adap-tive instance normalization. In Proceedings of the IEEE International Conference onComputer Vision. 1501–1510.

Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-imagetranslation with conditional adversarial networks. In Proceedings of the IEEE confer-ence on computer vision and pa�ern recognition. 1125–1134.

Ning Jin, Yilin Zhu, Zhenglin Geng, and Ronald Fedkiw. 2018. A pixel-based frameworkfor data-driven clothing. arXiv preprint arXiv:1812.01677 (2018).

Ladislav Kavan, Dan Gerszewski, Adam W. Bargteil, and Peter-Pike Sloan. 2011. Physics-Inspired Upsampling for Cloth Simulation in Games. ACMTrans. Graph. 30, 4, ArticleArticle 93 (July 2011), 10 pages.

Zorah Lahner, Daniel Cremers, and Tony Tung. 2018. Deepwrinkles: Accurate andrealistic clothing modeling. In Proceedings of the European Conference on ComputerVision (ECCV). 667–684.

Christian Ledig, Lucas �eis, Ferenc Huszar, Jose Caballero, Andrew Cunningham,Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang,et al. 2017. Photo-realistic single image super-resolution using a generative adver-sarial network. In Proceedings of the IEEE conference on computer vision and pa�ernrecognition. 4681–4690.

Junbang Liang, Ming Lin, and Vladlen Koltun. 2019. Di�erentiable Cloth Simulation forInverse Problems. In Advances in Neural Information Processing Systems. 771–780.

Jing Liao, Yuan Yao, Lu Yuan, Gang Hua, and Sing Bing Kang. 2017. Visual a�ributetransfer through deep image analogy. arXiv preprint arXiv:1705.01088 (2017).

Qianli Ma, Jinlong Yang, Anurag Ranjan, Sergi Pujades, Gerard Pons-Moll, Siyu Tang,and Michael J Black. 2019. Learning to Dress 3D People in Generative Clothing.arXiv preprint arXiv:1907.13615 (2019).

Ma�hias Muller and Nu�apong Chentanez. 2010. Wrinkle Meshes. In Proceedings ofthe 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA

�10). Eurographics Association, Goslar, DEU, 85��92.Rahul Narain, Armin Samii, and James F O’brien. 2012. Adaptive anisotropic remeshing

for cloth simulation. ACM transactions on graphics (TOG) 31, 6 (2012), 1–10.Andrew Nealen, Ma�hias Mller, Richard Keiser, Eddy Boxerman, and Mark Carlson.

2006. Physically Based Deformable Models in Computer Graphics. ComputerGraphics Forum 25, 4 (2006), 809–836.

Chaitanya Patel, Zhouyingcheng Liao, and Gerard Pons-Moll. 2020. �e virtual tailor:Predicting clothing in 3d as a function of human pose, shape and garment style.arXiv preprint arXiv:2003.04583 (2020).

Gerard Pons-Moll, Sergi Pujades, Sonny Hu, and Michael J Black. 2017. ClothCap:Seamless 4D clothing capture and retargeting. ACM Transactions on Graphics (TOG)36, 4 (2017), 1–15.

Damien Rohmer, Tiberiu Popa, Marie-Paule Cani, Stefanie Hahmann, and Alla She�er.2010. Animation Wrinkling: Augmenting Coarse Cloth Simulations with Realistic-Looking Wrinkles. ACM Trans. Graph. 29, 6, Article Article 157 (Dec. 2010), 8 pages.

Igor Santesteban, Miguel A Otaduy, and Dan Casas. 2019. Learning-Based Animationof Clothing for Virtual Try-On. In Computer Graphics Forum, Vol. 38. Wiley OnlineLibrary, 355–366.

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks forlarge-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

Min Tang, Tongtong Wang, Zhongyuan Liu, Ruofeng Tong, and Dinesh Manocha. 2018.I-cloth: incremental collision handling for GPU-based interactive cloth simulation.ACM Transactions on Graphics (TOG) 37, 6 (2018), 1–10.

Huamin Wang, Florian Hecht, Ravi Ramamoorthi, and James F. O�Brien. 2010. Example-Based Wrinkle Synthesis for Clothing Animation. ACM Trans. Graph. 29, 4, ArticleArticle 107 (July 2010), 8 pages.

Tuanfeng Y Wang, Duygu Ceylan, Jovan Popovic, and Niloy J Mitra. 2018. Learning ashared shape space for multimodal garment design. arXiv preprint arXiv:1806.11335(2018).

Tuanfeng Y Wang, Tianjia Shao, Kai Fu, and Niloy J Mitra. 2019. Learning an intrinsicgarment space for interactive authoring of garment animation. ACM Transactionson Graphics (TOG) 38, 6 (2019), 1–12.

Zhaowen Wang, Ding Liu, Jianchao Yang, Wei Han, and �omas Huang. 2015. Deepnetworks for image super-resolution with sparse prior. In Proceedings of the IEEEinternational conference on computer vision. 370–378.

Weiwei Xu, Nobuyuki Umetani, Qianwen Chao, Jie Mao, Xiaogang Jin, and Xin Tong.2014. Sensitivity-optimized rigging for example-based real-time clothing synthesis.ACM Trans. Graph. 33, 4 (2014), 107–1.

Jinlong Yang, Jean-Sebastien Franco, Franck Hetroy-Wheeler, and Stefanie Wuhrer.2018. Analyzing clothing layer deformation statistics of 3d human motions. InProceedings of the European Conference on Computer Vision (ECCV). 237–253.

Shan Yang, Junbang Liang, and Ming C Lin. 2017. Learning-based cloth materialrecovery from video. In Proceedings of the IEEE International Conference on ComputerVision. 4383–4393.

W. Yifan, F. Perazzi, B. McWilliams, A. Sorkine-Hornung, O Sorkine-Hornung, and C.Schroers. 2018. A Fully Progressive Approach to Single-Image Super-Resolution. InCVPR Workshops.

Tao Yu, Zerong Zheng, Yuan Zhong, Jianhui Zhao, Qionghai Dai, Gerard Pons-Moll,and Yebin Liu. 2019. Simulcap: Single-view human performance capture with clothsimulation. In 2019 IEEE/CVF Conference on Computer Vision and Pa�ern Recognition(CVPR). IEEE, 5499–5509.

Heming Zhu, Yu Cao, Hang Jin, Weikai Chen, Dong Du, Zhangye Wang, ShuguangCui, and Xiaoguang Han. 2020. Deep Fashion3D: A Dataset and Benchmark for3D Garment Reconstruction from Single Images. arXiv preprint arXiv:2003.12753(2020).

Javier S Zurdo, Juan P Brito, and Miguel A Otaduy. 2012. Animating wrinkles byexample on non-skinned cloth. IEEE Transactions on Visualization and ComputerGraphics 19, 1 (2012), 149–158.

12