image processing
DESCRIPTION
TRANSCRIPT
PISAProduction, Indexing and Search
of Audio-visual Material
Image ProcessingTinne Tuytelaars, IBBT – PSI – K.U.Leuven
61
Computer Assisted Analysis
! Intelligent analysis = reverse engineering
! Shot cut detection
! Scene segmentation
! Video reuse detection
! Face detection
! Face recognition
! Audio classification
demo
demo
demo
demo
demo
62
Shot cut detection
= Split the video stream in atomic units, corresponding to a
continuously moving camera
! Distinguish between abrupt and smooth shotcuts
! Experimented with different methods
! Using color histograms
! Using affine motion compensation
! Using motion estimation within the compressed domain
68
Scene segmentation
69
Video reuse detection
The same video material is often reused
! can be detected automatically
! Robust to post-processing
! Efficiency
! Based on spatio-temporal local features and locality
sensitive hashing
70
Face Detection
• Face candidates selection:" Candidate regions have skin color # region-based skin
segmentation
" Personalized chrominance skin boundary
• Verification based on cues:" Shape of ellipse
" Ellipse-filling percentage
" Gray-tone smoothness
" Corners of the facial features
71
Face detection results
• 92% good face detections
"Comparable to state-of-the-art face learning of ‘Viola & Jones’ obtains 93%
"adaptation to lighting conditions and personal face looks
72
Face recognition
! Based on a 3D morphable model
! 3D model is fitted to 2D image
! Shape and texture parameters used as face descriptor
! Robust to
! viewpoint changes,
! illumination changes,
! partial occlusions.
73
Face recognition
74
Face recognition
75
Face recognition
76
Face recognition
77
Face recognition
79
Future work
! Include facial expressions in face recognition
! Multi-modal scene segmentation
! Feedback loop from Trouvaille
! Object or scene recognition
! Thesaurus-based speech recognition