centre for vision, speech & signal processing university of surrey adrian hilton, john...

Centre for Vision, Speech & Signal Processing

University of Surrey

Adrian Hilton, John Collomosse

Peng Huang, Dan Casas, Ed Brookes, Chris Budd, Marco Volino(BBC icase)

4D performance modelling & animation

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and aH.264 decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

PerformanceCapture

4D ModelInteractiveAnimation

Performance Capture

QuickTime™ and a decompressor

are needed to see this picture.

Multiple view video

QuickTime™ and a decompressor

are needed to see this picture.

3D video

[Starck IEEE CGA’07]

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

SurfCap 3D Video Database [Starck et al. CGA’07]

I: Studio Capture

II: Global Non-rigid Alignment

3D video sequences Shape similarity tree 4D model

[Huang CVPR’11, Budd 3DIMPVT’11]

[Huang CVPR’11, Budd 3DIMPVT’11]

Interactive Animation

Interactive control of character animation:

editing motion

high-level parameterisation of motion

transitions between motions

4D model

QuickTime™ and a decompressor

are needed to see this picture.

InteractiveAnimation

Skeletal Character AnimationMotion Editing Motion Parameterisation

Motion Graphs

Heck & Gleicher’07Kovar&Gleicher’02

Witkin’95Brundelin’95

Rose’98

Arikan&Forsyth’02

3D video concatenation

[Huang et al. CVPR’09]

TextText

key start key end

surface motion graph representation

key-frame animation

[Starck SCA’05,CGA’07]

3D video concatenation

QuickTime™ and a decompressor

are needed to see this picture.

[Huang et al. CVPR’09]

4D motion editing

Goal: Interactive editing of 4D models

Space-time key-frame editing

Laplacian deformation framework

learnt 4D motion space

[Tejera CVMP’11]

III: 4D motion editing

[Tejera et al. CVMP’11]

III: 4D motion parameterisation

High-level real-time motion control

parameters: walk speed/direction, jump height etc

combine multiple skeletal sequences [Rose’98]

solution: mesh sequence blending ie walk/run

walk

run

4D motion parameterisation

Mesh sequence blending

(1) temporal alignment: dynamic time warp

(II) blend corresponding frames

non-linear blending (Laplacian): ~100ms/frame

linear blending: <8ms/frame but unrealistic

solution: hybrid non-linear blending 10ms/frame[Casas MIG‘11]

[Casas MIG’11,I3D’12]

4D motion parameterisation

III: Interactive Animation

interactive motion transitions

skeletal motion graph [Gleicher’02,Arikan’02]

[Casas MIG’11]

solution: 4D parametric motion graph

real-time transitions using shape similarity

SummaryPerformance capture

3D video capture indoor/outdoorjoint segmentation & reconstruction

Structured representation• Global non-rigid alignment • shape similarity tree• 4D models

Interactive Animation4D motion parametrisation 4D parametric motion graphsspace-time editing

RE@CT: Surrey

Pipeline for production of interactive 4D models

integration of skeletal control with surface motion graphs (WP3)

parametric control for general motion (WP4)

interactive control of motion constraints (foot slide, hand-contact etc.) (WP4)

interactive motion stylisation (WP4)

WP3T5

Surface motion graph extraction & development

automatic graph construction

similarity measures for dynamic shape & appearance

efficient representation

D3.1 (month 12): Initial actor model

skeleton based motion graph

WP4: Actor Animation Algorithms

Interactive animation API using surface motion graphs

T4.1 Specification (all)

T4.2 Parametric surface motion graph (Surrey,BBC)

T4.3 Integration of face and body (HHI,BBC,INRIA,ART)

T4.4 Artistic editing and control (Surrey)

• T4.5 Adaptation & manipulation of appearance

(HHI)

D4.1 (month 12): Initial RE@CT animation engine

based on skeletal surface motion graph WP3

WP4T1: Specification (All)

Data formats for parameterised motion graph

Interface for face & body animation

Interface for authoring content

Representation for parameterised 4D models

Interfaces and representation for appearance

1. input from all partners on current/proposed rep.

2. draft data/API specification (month 3, Surrey)

3. working data/API specification (month 6, Surrey)