vLab:Visual Media ResearchvLab:Visual Media Research

Prof. Adrian HiltonRoyal Society Industry Fellow

Centre for Vision, Speech and Signal ProcessingUniversity of Surrey

Visual Content Production3D shape capture

1996

1999

3D people for games

3D studio capture 2003

3D video face

2005

3D video people 2007

2009

3D video sports

3D video live-action2011

4D video

Stereo 3D

videometrics

1999

3D Studio Production

Multi-view Video Shape Reconstruction Animation

Highly realistic animated models of actors

Collaboration: BBC, Sony, The FoundryFunding: EPSRC, TSB, EU

4D Performance Capture – global alignment

4D Interactive Animation

iview: Free-viewpoint Video in Sports Production

Free-viewpoint video for production of live sports events

Editorial value - on pitch views: referee or player- offside decisions in football- goal decisions

Production requirements- broadcast video quality- action replay

Lead partner: BBCCollaborators: Hawkeye, Snell&WilcoxFunding: TSB/EPSRC

3D Sports Production

Is the player offside?

3D Sports Production IVIEW PIPELINE

Capture

2D Segmentation

Calibration

3D Reconstruction + matte refinement Rendering

OPERATOR shot selection

3D Sports Production

OFFSIDE?

3D Sports Production

IVIEW PRODUCTION TRIALSFootball: England European cup qualifiers - Wembley, Nov’07Rugby: 6 Nations - Twickenham, Feb’08

Multiple view video in post-production

Aims - Multi-view video to improve 2D video post-production- 3D video modelling of actors- 3D set modelling from images

Production requirements- Real dynamic scenes- Improved post-production workflow

Lead partner: The FoundryCollaborators: BUF, QD, Trinity, CERTH Funding: EU

i3Dpost: Film & Games Post-Production

i3Dpost: Film & Games Post-Production

Stereo production of live action from multiple view video

Aims - 3D video of ‘real’ scenes- Stereo 3D production- Video appearance editing using 3D

Production requirements- Film quality stereo- Interactive editing and 3D production

Lead partner: The FoundryCollaborators: BBC Funding: TSB/EPSRC

i3Dlive: 3D Film and TV Production

i3Dlive: 3D Film and TV Production

INPUT 3D VIDEO

STEREO

Digital Doubles

Photo-realistic doubles of real actors for film production

Funding: Royal Society Industry Fellowship (2008-12)Collaboration: Framestore

Research Directions

4D Video Animation Learning models of interaction Recognition of human movement

Videometrics Measurement of human motion Performance analysis in sports

Body Shape Measurement of body shape Clinical analysis for obesity Fashion for online retail

Visual Media Research Grouphttp://www.ee.surrey.ac.uk/CVSSP/VMRGResearch Team Jean-Yves Guillemaut, Joe Kilner, James Edge, Hansung Kim, Phil Parsonage, Evren Imre, Ash Doshi, Takeshi Takai, Nat Nadtoka, Peng Huang, Peter Williams, Chris Budd, Mykyta Fastovets, Sarim Mohammed, Dan Casas, Magara, Tejera

Collaborators BBC, Sony, Framestore, The Foundry, Snell&Wilcox, Blitz Games, Hawkeye, Rebellion, IKinema

Funding EPSRC, TSB, EU, Royal Society, Industry Portfolio: £3.5M Industry support: £400K EPSRC Visual Media Platform Grant 2003-2013