efficient video browsing
DESCRIPTION
Efficient Video Browsing. Using Multiple Synchronized Views Heymo Kou. Question. What is the two main technologies applied for efficient video browsing? (one for audio, one for visual content). Table of contents. Background Current technology Advanced technology Summary Reference. - PowerPoint PPT PresentationTRANSCRIPT
Efficient Video Brows-ing
Using Multiple Synchronized ViewsHeymo Kou
What is the two main technologies applied for efficient video browsing? (one for audio, one for visual content)
Question
Background Current technology Advanced technology Summary Reference
Table of contents
Growth of digital contents data
Digital video market growth
From your◦ Smart phones◦ Notebooks◦ Webcams◦ Digital camera and camcorders◦ Security and monitoring cameras
With advanced streaming technology◦ Fast Internet access◦ MPEG-4 format
Digital video becomes ubiqui-tous
Search through categories◦ Similar to Internet shopping mall
We search for big categories Then smaller categories …and so on…
User should choose which to browse◦ Should check whether the selected data matches
what user was finding Time consuming!
Manual categorizing and annotation◦ One by one?
Current technology for finding a video data
Too complicated◦ Lack of efficient algorithm
Time consuming◦ Multimedia calculation ∝ exponential
Inaccuracy◦ Video data is increasing exponentially
Cataloging manual has a somewhat limit point◦ Manually cataloging is done by human hand that
mistakes can be happened
Problem with current video search and browsing technologies
MPEG-7 Standards Speech indexing Shot Boundary Detection Time Scale Modification of Audio Signals Storyboards, Moving Storyboards and Ani-
mation Adaptive Accelerating Fast Playback Streaming Synchronized Views
Technologies for advancedimage and video retrieval
Standardized by ISO/IEC◦ International Standard Organization◦ International Electrotechnical Commission
Not a video encoding format XML to store metadata
◦ Attached to timecode in multimedia By this tag
◦ Able to index and search efficiently Yet, improvement is needed
MPEG-7 standard
Search through speech transcripts◦ Finds familiar metaphor of free text search
Automatic speech recognition (ASR)◦ Indexed transcript → semantic information
Main advantage : Representation◦ Speech is built of words
Speech indexing
Frame
Key frame
Shot◦ Group of frames which represents similar frames
Definitions
Start key frame end key frame animation
Context◦ Meaningful information within multimedia data
3 levels of video browsing◦ Browsing a large collection of videos◦ Browsing a ranked list of videos◦ Browsing a single video to find relevant segments
Definitions
Shot Boundary Detection(SBD) algorithm◦ Completely automatic
Key frames are selected and extracted◦ Saved as JPEG files
High Accuracy and Efficiency◦ Still, fault detection problem is unsolved
Shot Boundary Detection
SBD algorithm
Similar to scene selection of dvd
Audio browsing is as important asvideo browsing◦ Except images, most digital contents are audible
Faster audio browsing is necessary Speeding up of audio signal by
◦ By deleting small audio segments◦ Especially, human speech signals are quasi-peri-
odic
Time Scale Modification ofAudio Signals
Improvement of TSMTime-Domain Harmonic Scal-
ing(TDHS) technique
Time-Domain, Pitch Synchro-nous Overlap Add
Time Scale Modification(TSM) algorithm
Waveform Synchronous Over-lap(WSOLA)
Synchronous Overlap-Add SOLA
Storyboard◦ a set of one or more pages, each consists of a two
dimensional array of key-frames, sorted in chrono-logical order.
Animation◦ a quick slide show, where each of the key-frames is
shown for a fixed short period (e.g., 0.6 seconds) Moving Storyboard (MSB)
◦ the animated key frames, fully synchronized with the original audio track. Each key-frame is shown for the entire duration of the associated shot.
Storyboards, Moving Story-boards and Animation
Very fast video playback (without audio) Ordinary fast forward depends only on
speed◦ There is a chance to miss important scene
Accelerates until new scene is met Requires less computation load
Adaptive Accelerating Fast Playback
Image for adaptive fast play-back
Example in surveillance camera
Real-use of adaptive fast play-back
Server preprocesses media◦ Keep same media, but different speed encoded
When user selects other speed◦ 1. pause current media◦ 2. open file with same content with selected
speed◦ 3. seek to the corresponding position◦ 4. play the selected view
Needs no extra computational load◦ However, requires more storage: Tradeoff
Streaming Synchronized Views
Can browse multiple videos at once
Split frames every given time◦ (i.e 10 seconds)
Strong information scent is visible◦ With aggregation of occurrences
Browsing Multiple Videos: MovieDNA
Image of movieDNA
Summary of main proper-ties
ViewVisual
AudioTypical
speedup rateStati
cDy-
namic
Full video (w/o TSM) ○ ○ 1 – 2XVideo Skim ○ ○ 2 – 20XSlide show (w/o TSM) ○ ○ 1 – 2XAdaptive Fast Playback ○ 5 – 30XAnimation ○ 10 – 40XStoryboard, mosaic ○ NA
Streaming synchronized views and movieDNA◦ Less computation, multiple videos at once
Active accelerating fast playback◦ Most useful at analyzing surveillance videos
SBD & TSM◦ Efficient for implementing above technologies
Then, what is current limitation?
Conclusion
Any questions?
Q & A
What is the two main technologies applied for efficient video browsing? (one for audio, one for visual content)
Answer : The two main technologies are Shot Boundary Detection(SBD) for visual content and Time Scale Modification(TSM) for audio signals
Answer
Shot Boundary Detection◦ http://muvis.cs.tut.fi/sbd.html
Key frame◦ http://en.wikipedia.org/wiki/Key_frame
Synchronous Overlap-Add◦ http://www.surina.net/article/time-and-pitch-scaling.html
Digital Video Market Growth◦ http://
articles.businessinsider.com/2011-12-13/research/30508929_1_fios-cable-providers-video-streaming-service/2
Amount of Digital data◦ http://
www.emc.com/collateral/analyst-reports/diverse-exploding-digital-universe.pdf
References (1/2)
Streaming synchronized view◦ http://
www.visus.uni-stuttgart.de/forschung/visualisierung-und-visual-analytics/visuelle-analyse-videostroeme/adaptive-fast-forward-for-video-surveillance.html
MovieDNA◦ http://
homepage.mac.com/juggle5/WORK/publications/HICSS34.pdf
References (2/2)