computer vision and media group: selected previous work
DESCRIPTION
Computer Vision and Media Group: Selected Previous Work. David Gibson, Neill Campbell Colin Dalton Department of Computer Science University of Bristol. Duck: The Automatic Generation of 3D Models. Generating 3D computer models is difficult Put object on turntable - PowerPoint PPT PresentationTRANSCRIPT
24/10/02 AutoArch Overview
Computer Vision and Media Group:
Selected Previous Work
David Gibson, Neill CampbellColin Dalton
Department of Computer ScienceUniversity of Bristol
24/10/02 AutoArch Overview
Duck: The AutomaticGeneration of 3D Models
• Generating 3D computer models is difficult• Put object on turntable• Take 8 pictures of it from different angles• Crank the handle…• No skilled user or expensive equipment• Make avatars by spinning person on chair
24/10/02 AutoArch Overview
24/10/02 AutoArch Overview
Cog and Stepper
• Automatically inject ‘life’ into computer animations
• 3D swathe through 4D space time• Where space is 3D computer model• Or just to make things look strange!
24/10/02 AutoArch Overview
24/10/02 AutoArch Overview
24/10/02 AutoArch Overview
Casablanca: Motion Ripper
• Computer animation driven by film• Animator labels a small number of points• System then tracks these points over all
frames• Motions are extracted and used to drive
animation
24/10/02 AutoArch Overview
24/10/02 AutoArch Overview
Laughing ManMotion Ripper Part 2
• Automatic video creation• Points are marked and tracked• System learns the motions• System generates new motions which are
different but ‘correct’• Forever!
24/10/02 AutoArch Overview
24/10/02 AutoArch Overview
AutoArch: The Automatic Archiving of Wildlife Film
Footage
David Gibson, Neill CampbellDavid Tweed, Sarah Porter
Department of Computer ScienceUniversity of Bristol
24/10/02 AutoArch Overview
Motivation
• BBC Natural History Unit• Manual archiving/meta data generation• Reuse problematic
– Inefficient/time consuming– Expensive– Limited access
• Obvious need to automate
24/10/02 AutoArch Overview
Objectives
• Generate efficient visual representations– Video segmentation– Visual browsing/summarisation– Visual searching
• Generate as much meta data automatically– Camera motions/effects– Scene structure– Scene content
24/10/02 AutoArch Overview
System Overview
ShotSegmentation
VisualSummarisation
MotionAnalysis
Colour/TextureAnalysis
Meta data extraction algorithms
Catalogue Entry
Visualisation based algorithms
Visualisation and Searching
24/10/02 AutoArch Overview
Video Segmentation
24/10/02 AutoArch Overview
Visual Summarisation
• Key frame extraction
24/10/02 AutoArch Overview
Visual Summarisation Tree
Entir
e sh
ot
Level of detail
24/10/02 AutoArch Overview
Visual Searching
• Layered 2D representationof high D clip space
24/10/02 AutoArch Overview
Motion Analysis using point tracking
•Camera Motion Estimation•Event/Area of Interest Detection•Gait Analysis•Foreground/Background Separation•Combine with Colour and Texture for Classification•See cheetah track avi
24/10/02 AutoArch Overview
Camera Pan
BCD0111.09_0085.epslines = 47, curls = 98, shorts = 5long lines = 47, mode = 95.00, mean = 95.21, std = 4.15zoom centre = (603.01, 63.65), val = -0.2356zoom residual per line = 22.92zoom residual #2 per line = 28.92Average line vector: 109.94 -8.27
pan/tilt angle: 94.30, vector: (109.94 -8.27)pan/tilt residual per line = 21.67pan/tilt residual #2 per line = 33.38percentage of lines within 5% of mode: 89.36
24/10/02 AutoArch Overview
Camera Zoom
BCD0113.15_0067.epslines = 142, curls = 1, shorts = 7long lines = 134, mode = 340.00, mean = 227.24, std = 128.76zoom centre = (182.97, 55.52), val = 0.2063zoom residual per line = 4.86zoom residual #2 per line = 6.90Average line vector: -3.81 17.28pan/tilt angle: 347.57, vector: (-3.81 17.28)pan/tilt residual per line = 13.85pan/tilt residual #2 per line = 16.13percentage of lines within 5% of mode: 17.16
24/10/02 AutoArch Overview
Tracking Failure
This could be an interestingevent in its self: flocking,herding, close up of lots ofactivity, shot grouping, etc.
24/10/02 AutoArch Overview
Event/Area of InterestDetection
24/10/02 AutoArch Overview
Frequency Analysis:Gait Detection
FFT
After trajectory segmentation
24/10/02 AutoArch Overview
Foreground/BackgroundExtraction
Feature space #1
Feat
ure
spac
e #2
Foregroundmodel
Backgroundmodel
Which pixelsare foreground?
24/10/02 AutoArch Overview
Animal IdentificationGive models a name:
= cheetah
= elephant
= zebra
= lion
24/10/02 AutoArch Overview
Some Problems
• Noise in images• Noise in measurements• Camouflage• Occlusion• Answer: Need higher level models• See next few slides
24/10/02 AutoArch Overview
Model Based Tracking
24/10/02 AutoArch Overview
Lion Tracking
• Synchronise horse model with lion points• Move and deform horse model to lion points• See avi• To do: Improve spatial deformation, especially for
legs, using colour and texture
24/10/02 AutoArch Overview
Multiple Object Tracking
24/10/02 AutoArch Overview
Conclusions
• Visualisation is very powerful• Combined with text is even better!• Assists searching and communication• Lots of meta data can be auto generated• Assists archiving• Help to prioritise manual archiving• Can be applied to any visual media