1

AAM based Face Tracking with Temporal Matching and Face

Segmentation

1Institute of Automation Chinese Academy of Sciences, Beijing, China2Microsoft Research Asia Beijing, China

Mingcai Zhou1 、 Lin Liang2 、 Jian Sun2 、 Yangsheng Wang1

CVPR 2010

2

3

Problems- AAM tracker

• Difficultly generalize to unseen images• Clutterd backgrounds

4

How to do?

• A temporal matching constraint in AAM fitting- Enforce an inter-frame local appearance constraint between

frames

• Introduce color-based face segmentation as a soft constraint

5

Temporal Matching Constraint

1. Select feature points with salient local appearances at previous frame2. I(t−1) to the Model coordinate and get the appearance A(t-1)

3. Use warping function W(x;pt) maps R(t-1) to a patch R(t) at frame t

6

Shape Initialization

Those feature points whose motion directions are inconsistent with themain direction are most likely to be outliers.

t-1 t

Face Motion Direction

Improve the stability in tracking fast face motions

7

Face Segmentation Constraint

8

Face Segmentation Constraint

9

Experiments

10

Image and Video Abstraction byAnisotropic Kuwahara Filtering

1Hasso-Plattner-Institut, Germany2University of Missouri, St. Louis

Jan Eric Kyprianidis1 、 Henry Kang2 、 Jürgen Döllner1

PG 2009

11

Features

preserving shape boundaries exhibit directional information as found in oil paintings use to video without extra processing

12

Edge-Preserving Filter

13

Edge-Preserving Filter

Kuwahara filter [KHEK76] removes detail in high-contrast regions while also protecting shape boundaries in low-contrast regions.

14

Block Artifacts

15

Anisotropic Filter

16

Method

Orientation and Anisotropy Estimation Anisotropic Kuwahara Filter

17

Experiments

18

Fast Motion Deblurring

POSTECH

Sunghyun Cho、 Seungyong Lee

SIGGRAPH Asia 2009

19

Features

fast deblurring method

20

Single Image Blind Deconvolution

21

Prediction

Image gradient mapsBlurred image

1.Suppress noise: bilateral filter2.Restore strong edge: shock filter3.Gradient magnitude threshold

22

Kernel Estimation

Image gradient maps

Minimize the energy function:

23

Kernel Estimation

CG method

A size: (5n^2) x (m^2) , L:n x n ,K:m x m

24

Kernel Estimation

25

Kernel Estimation

26

Deconvolution

latent image

27

Experiments

28

Experiments

29

Noise Brush: Interactive High Quality Image-Noise Separation

Jia Chen1 、 Chi-Keung Tang1 、 Jue Wang2

1The Hong Kong University of Science and Technology2Adobe Systems, Inc.

SIGGRAPH Asia 2009

30

Problems-denoising

• Over-smoothed image structure• Residual noise in smooth regions

31

Joint Image-Noise Filtering

Purpose:

W大小spatial

color

Image structure

32

Joint Image-Noise Filtering

33

Result

34

Noisy InputJoint Flash Nonflash ResultSingle Image Denoised by NoisewareSingle Image Our Result

35

Result

36

Inverse Texture Synthesis

Li-Yi Wei1 、 Jianwei Han2 、 Kun Zhou1,2

Hujun Bao2 、 Baining Guo1 、 Harry Shum1

1Microsoft Research Asia 2Zhejiang University

SIGGRAPH 2008

37

Flow Diagram

New

38

What is the mean of Inverse?

input

output

inverse texture synthesis

• From a large input texture– produce a small output that best summarizes input

http://www.youtube.com/watch?v=w5HY2xMCldI

39

Benefits

• Reduce storage size• Increase processing speed

890^2X11 128^2X11

40

Generate Compaction

For globally varying texture:

41

Generate Compaction

42

Generate Compaction