video information retrieval
DESCRIPTION
Video Information Retrieval . Mark Ruzomberka IST 497 11/07/02. Joke. Outline. What is Video Information Retrieval (VIR) ? Reasons VIR is necessary Theoretical Where we are today Examples Problems Future Work Conclusion. What is Video Information Retrieval (VIR) ?. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/1.jpg)
Video Information Retrieval
Mark RuzomberkaIST 49711/07/02
![Page 2: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/2.jpg)
Joke
![Page 3: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/3.jpg)
OutlineWhat is Video Information Retrieval (VIR) ?Reasons VIR is necessaryTheoreticalWhere we are todayExamplesProblemsFuture Work Conclusion
![Page 4: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/4.jpg)
What is Video Information Retrieval (VIR) ?
Recognition technologies Image Voice Text transcripts
Document retrieval technologies Topic segmentation Topic matching Text summarization
Presentation Technologies Combine Recognition and retrieval technologies
Result is an integrated application
![Page 5: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/5.jpg)
VIR-Need, or Why do I care?
Consider the task of trying to find a five minute video clip of interest in a library of 1000 hour long tapes.
Consider the “go to the part where” problem
![Page 6: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/6.jpg)
What do people want from IR
D-Lib Magazine’s asks:
“What do People want from Information Retrieval?”
# 8 Multimedia
![Page 7: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/7.jpg)
Specificly, Reasons for Video IR
Reading is slow compared to your potential for understanding information
Humans think in pictures not words Reading is particularly slow on a computer screen Example: Daydreaming while some one is talking Reading a page in a book and not remembering what it was about
![Page 8: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/8.jpg)
VIR makes for quicker human understanding. Palm/Grafitti 25 Hand Writing 35-40 Typing 50-70 Speaking 135-175 Reading 200 Listening 400 - 500 Thinking 500+
•Video IR allows for faster access to information
![Page 9: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/9.jpg)
Theoretical: Think of the “Jetsons mail system”
You “talk” to the computer, Computer intelligently “talks” back to you
![Page 10: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/10.jpg)
Where we are today
Two of Video Information Retrieval System are currently available:
Type One- keyword/text basedType Two- Content based
![Page 11: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/11.jpg)
Type One- keyword/text based
•DVR- basic expansion of image IR, •not as interesting
![Page 12: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/12.jpg)
Type Two- Content based
Video Mail Informedia MSR Video Skimmer
![Page 13: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/13.jpg)
Example: Video Mail University of Cambridge
1994-1996
AT&T 1999
2000-project ended
![Page 14: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/14.jpg)
Video Mail: Medusa network
Medusa multimedia environment at Olivetti Research Ltd. In Cambridge
It takes a modular approach unlike that of a pc or workstation Unified by a common interface to ATM network Devices plug directly into network and include:
Cameras Audio devices Networked frame buffers Processor farms Disk drives
![Page 15: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/15.jpg)
Video Mail: Medusa Network
“The network is the computer” metaphor is used Solves storage and network speed problems Complicates expense problem
![Page 16: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/16.jpg)
How it works-Overview
![Page 17: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/17.jpg)
The Integrated Application
“narrow” by sender,date, time
![Page 18: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/18.jpg)
Video Mail: Video Browser
Content is now being viewedKeywords are flagged
![Page 19: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/19.jpg)
Video Mail: Video BrowserIn the latest version
“thumb-nailed” pictures of key frames replace color coded line of the search keyword
![Page 20: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/20.jpg)
Informedia
The Informedia Digital Video Library Project automatically combines speech, image and natural language understanding to create a full-content searchable digital video library.
![Page 21: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/21.jpg)
Informedia
![Page 22: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/22.jpg)
Informedia: human factor issues
Interaction MotivationEffective usage modes
Commercial compressionVHS quality playback. Terabyte (1,000 gigabytes) of storage 1000 hours of video.
![Page 23: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/23.jpg)
Problems
1.Human understanding2.Spoken document retrieval3.Poor video browsers4.Expensive5.Slow access to data6.Large amounts of data
![Page 24: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/24.jpg)
Microsoft Research (MSR) Video Skimmer
![Page 25: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/25.jpg)
Microsoft Research (MSR) Video Skimmer
Enhanced Browser Controls: Time Compression Pause Removal Textual Indices:
TOC, Notes Visual Indices
Shot Boundary FramesTimeline Markers
Jump Control (Back/Next)
![Page 26: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/26.jpg)
Problem: Poor Content Based Video Browsers
Current VCR model allows for poor navigation “go the the part where they say” problem
![Page 27: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/27.jpg)
Problem: Expensive
Hard drive space expensive Video adds to problem
High bandwidth needs are also expensive
Year Drive Size Drive Cost Per MB/Cost
1956 5 megabytes 50,000.00 10,000.00
1980 26 megabytes 5,000.00 193.00
1985 10 megabytes 710.00 71.00
1989 40 megabytes 1,199.00 36.00
1995 1.2 gigabytes 680.00 68.60
2000 30.0 gigabytes 249.99 0.96
•http://www.littletechshoppe.com/ns1625/winchest.html
![Page 28: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/28.jpg)
Problem: Slow Access to Data
Broadband still not available everywhereAvailability doesn’t mean acceptanceEspecially after dot com crash 2000
![Page 29: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/29.jpg)
Problem: Large Amounts of Data
Current Systems use MPEG2Newer compression technologies
MPEG 4-DIVX -DVD QualityVideo consumes orders of magnitude
more storage than textMPEG 7 is on horizon
![Page 30: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/30.jpg)
Future Work ?
Sky the limit ?Sci-Fi the limit ?
Hard Drive Space, Bandwidth are current limitations.
![Page 31: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/31.jpg)
ConclusionNot yet ready for prime timeStorage and Network Costs decreasingSuccess is in day to day usageSlowly Becoming Mainstream E.x.TivoProblems of “real world tests”
Idiot proof ATM and Medusa aren’t mainstream
![Page 32: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/32.jpg)
Papers Video Mail Retrieval Using Voice: Report on Keyword.. - Jones, Foote, Jones.. (1994) What do people want from Information Retrieval?. Croft, Bruce W. D-Lib Magazine. (1995) Video Skimming for Quick Browsing based on Audio and Image.. - Smith, Kanade (1995) The VISION digital video library (context) - Gauch, Li et al. – (1997) Informedia: News-on-Demand Multimedia Information.. - Hauptmann, Witbrock (1997) M.G. Christel and D.J. Martin, "Information Visualization within a Digital Video Library", J.
Intelligent Info. Systems 11(3), (1998), pp. 235-257 Browsing Digital Video. Li, Gupta, Sanocki et. Al.
![Page 33: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/33.jpg)
Questions?
![Page 34: Video Information Retrieval](https://reader035.vdocument.in/reader035/viewer/2022062521/56816846550346895dde20e7/html5/thumbnails/34.jpg)
Joke?
"There are 10 types of people in the world...
those who understand binary and those who don't."