photorealistic facial texture inference using deep neural ... · pca coefficients: optimization of...

input picture albedo map (LF) rendering (HF) rendering side (HF)rendering zoom (HF)albedo map (HF)

input picture albedo map (LF) rendering (HF) rendering siderendering zoomalbedo map (HF)

Photorealistic Facial Texture Inference Using Deep Neural NetworksSHUNSUKE SAITO1,2 LINGYU WEI1,2 LIWEN HU1,2 KOKI NAGANO3 HAO LI1,2,3

1Pinscreen Inc. 2University of Southern California 3USC Institute for Creative Technologies

output renderinginput image

initialface model

fitting

complete albedo map (LF)

partial albedo map (HF)

textureanalysis

texturesynthesis

complete albedo map (HF)

feature correlations

face database

00.10.20.30.40.50.60.70.80.91

real ours ours -least square

ours - closest

visio-lization PCA

Positive Rate

USER STUDYPIPELINE

Texture AnalysisOptimal Gramian matrix: convex combination of Gramian matrices from dataset of high resolution real images

Face Model FittingPCA coefficients: optimization of photo consistency, landmark distances and regularization.

Texture Synthesis

Input Output Texture