detection of illicit content in video streams niall rea & rozenn dahyot
TRANSCRIPT
Detection of Illicit Content in Detection of Illicit Content in Video StreamsVideo Streams
Niall Rea & Rozenn DahyotNiall Rea & Rozenn Dahyot
http://www.pixalert.com/
What? Why?What? Why?
By illicit content, we mean By illicit content, we mean pornographic materialpornographic material
Applications: Applications: Kid protection (Parental control)Kid protection (Parental control) Company protection (company computers Company protection (company computers
scanning)scanning) Pedophilia (cyber surveillance)Pedophilia (cyber surveillance)
State of the ArtState of the Art Text Analysis (Internet Filtering) Text Analysis (Internet Filtering)
FilenamesFilenames Text surrounding imagesText surrounding images Known URLs Known URLs
Image analysisImage analysis Skin detectionSkin detection Geometrical constraints + OrientationGeometrical constraints + Orientation Face localisationFace localisation
Video analysisVideo analysis Current approach: keyframe extraction at regular time intervals and Current approach: keyframe extraction at regular time intervals and
still image analysisstill image analysis
Extension to Illicit Video Extension to Illicit Video DetectionDetection
Using Video informationUsing Video information ColorColor TextureTexture MotionMotion
Using Audio informationUsing Audio information Audio energy (loudness)Audio energy (loudness)
Video AnalysisVideo Analysis
Considered 2 approaches exploiting Considered 2 approaches exploiting features from the partially decoded MPEG features from the partially decoded MPEG video streamvideo stream Smart keyframe selection for real time Smart keyframe selection for real time
performance based on macroblock typeperformance based on macroblock type Exploiting motion vectors for periodic motion Exploiting motion vectors for periodic motion
detectiondetection Optimised open-source ffdshow decoder (extraction Optimised open-source ffdshow decoder (extraction
of compressed domain motion features from MPEG-of compressed domain motion features from MPEG-1/2/4)1/2/4)
Poesia filter for skin colour detectionPoesia filter for skin colour detection
Shot cut detection based on Shot cut detection based on ratio of ratio of macroblock types of consecutive inter-coded macroblock types of consecutive inter-coded frames in a sub-GOP frames in a sub-GOP (e.g. shot cut occurs between the first (e.g. shot cut occurs between the first
reference frame and the first B-frame)reference frame and the first B-frame) Macroblocks in both B-frames will be heavily Macroblocks in both B-frames will be heavily
backward predicted (indicated by the heavier backward predicted (indicated by the heavier arrows). A shot cut is deemed to have occurred arrows). A shot cut is deemed to have occurred if and if and
Shot Cut detectionShot Cut detection
Shot cut
n-1n-3 n-2 n
PBB I
min2 Tbfbn
min1 5.0 Tbfbn
Motion extractionMotion extraction Compressed (MPEG motion vectors)Compressed (MPEG motion vectors) Background/ global motion compensation (Coudray 2004)Background/ global motion compensation (Coudray 2004)
4 parameter motion model4 parameter motion model
Calculate zoomCalculate zoom
Calculate translationCalculate translation
Assume global motion only occurs in non-skin and Assume global motion only occurs in non-skin and reasonably high texture areasreasonably high texture areas
Compute a 2D histogram of those motion vectorsCompute a 2D histogram of those motion vectors Global translation is the mode of the histogramGlobal translation is the mode of the histogram
Motion fieldsMotion fields
When Harry met Sally Illicit video
Motion and color segmentationMotion and color segmentation
Assume a simple local homogeneous motion field and global homogeneous motion
k-means clustering (2 clusters) of motion field
Likelihoods computed from Poesia 32^3 bin RGB skin/non skin histograms
Priors set empirically
Audio stream Audio stream
Content recognition using Audio dataContent recognition using Audio data SportSport
Specificity of illicit: (pseudo) periodicity Specificity of illicit: (pseudo) periodicity Simple feature used: loudness (Audio energy)Simple feature used: loudness (Audio energy)
Does not discriminate between different sources of Does not discriminate between different sources of noise (ie voices, specific sounds, etc.)noise (ie voices, specific sounds, etc.)
Capture the dominant pattern of the audio dataCapture the dominant pattern of the audio data
Audio stream Audio stream
Illicit Audio:
Scene of When Harry met Sally
LoudnessLoudness
Harry and Sally are talking (5s) Sally is faking it (5s)
Audio energy computed over a 40ms (duration of an video frame at 25 fps)
PeriodicityPeriodicitySignal: s(t)=1+sin(t) Signal: s(t)=rand(t)
Autocorrelation: sscorr(s)
Periodicity of audio energyPeriodicity of audio energy
Harry and Sally are talking Sally is faking it
Correlation of the loudness computed over 5s
Measure of periodicity in audio Measure of periodicity in audio datadata
Harry and Sally are talking Sally is faking it
Measure of periodicity: difference between the surface defined by the maxima and the minima
Measure of periodicity in audio Measure of periodicity in audio datadata
Measure of periodicity over the sequence ‘When Harry met Sally’
Other resultsOther results
False alarm rate assessed on False alarm rate assessed on 20minutes of non-illicit material 20minutes of non-illicit material (scenes from movies and music (scenes from movies and music videos)videos)
Detection rate assessed on 10 Detection rate assessed on 10 minutes of 8 illicit materialsminutes of 8 illicit materials
FA: 2%
DR: 5 extracts (~9minutes of recordings) are flagged as illicit3 (~minutes recordings) are missed
Future work: Motion and Future work: Motion and periodicityperiodicity
Periodic Motion on P-Periodic Motion on P-framesframes Mean motion vector Mean motion vector
over skin regionsover skin regions
Correlation between Correlation between periodic motion and periodic motion and audio?audio?
Any question