a novel framework for semantic annotation and personalized retrieval of sports video ieee...
TRANSCRIPT
![Page 1: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/1.jpg)
A Novel Framework for Semantic Annotation andPersonalized Retrieval of Sports Video
IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008
![Page 2: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/2.jpg)
Outline
Introduction Semantic Annotation of Sports Video
Text Analysis Video Analysis Text/Video Alignment Video Annotation and Indexing
Personalized Video Retrieval Experiment and Evaluation
![Page 3: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/3.jpg)
Introduction Semantic Annotation of Sports Video
Text Analysis Video Analysis Text/Video Alignment Video Annotation and Indexing
Personalized Video Retrieval Experiment and Evaluation
![Page 4: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/4.jpg)
![Page 5: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/5.jpg)
Introduction Semantic Annotation of Sports Video
Text Analysis Video Analysis Text/Video Alignment Video Annotation and Indexing
Personalized Video Retrieval Experiment and Evaluation
![Page 6: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/6.jpg)
Text Analysis
Caption text overlaid on the video The recognition of caption text overlaid on sports
video using OCR is not ideal due to the quality of the broadcast sports video.
Closed caption Closed caption is a transcript from speech to text
thus contains a lot of information irrelevant to the games and lacks of a well-defined structure.
![Page 7: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/7.jpg)
Text Analysis
Web-casting text is another text source related to sports video It is available in many sports websites such as BBC and
ESPN and can be easily accessed during or after the game
The content of web-casting text is more focused on events of sports games and has a well-defined structure
Since webcasting text is a text counterpart of broadcast sports video, it includes detailed information of an event in sports games
![Page 8: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/8.jpg)
![Page 9: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/9.jpg)
![Page 10: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/10.jpg)
The analysis of web-casting text
ROI Segmentation Keyword Identification Text Event Detection
![Page 11: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/11.jpg)
![Page 12: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/12.jpg)
ROI Segmentation
![Page 13: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/13.jpg)
Keyword Identification
![Page 14: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/14.jpg)
Text Event Detection
Example 1 (soccer):
79:19 Goal by Didier Drogba (Chelsea) drilled left-footed from right side of six-yard box (6 yards). Chelsea 4-1 Bayern Munich
Example 2 (basketball):
8:52 Kobe Bryant makes 17-foot two point shot (Smush Parker assists). LA Lakers 9-11 Denver
![Page 15: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/15.jpg)
Text Event Detection
The presentation style of the event for soccer and basketball in web-casting text is slightly different, but the event and event semantics can be easily extracted and represented using a common structure as follows.
<Event> by <Player> of <Team> at <Time>
Goal by Frank Lampard of Chelsea at 58:58 (soccer)
![Page 16: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/16.jpg)
Introduction Semantic Annotation of Sports Video
Text Analysis Video Analysis Text/Video Alignment Video Annotation and Indexing
Personalized Video Retrieval Experiment and Evaluation
![Page 17: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/17.jpg)
Video Analysis
Shot Classification Replay Detection Video Event Modeling
Event with replay
far view shot, close-up shots, replay, close-up shots, far view shot
![Page 18: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/18.jpg)
![Page 19: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/19.jpg)
Introduction Semantic Annotation of Sports Video
Text Analysis Video Analysis Text/Video Alignment Video Annotation and Indexing
Personalized Video Retrieval Experiment and Evaluation
![Page 20: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/20.jpg)
Text/Video Alignment Event Moment Detection
Clock Digits Location
Clock Digits Recognition
Event Boundary Detection Hidden Markov Model (HMM)
![Page 21: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/21.jpg)
Introduction Semantic Annotation of Sports Video
Text Analysis Video Analysis Text/Video Alignment Video Annotation and Indexing
Personalized Video Retrieval Experiment and Evaluation
![Page 22: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/22.jpg)
Video Annotation and Indexing For each game, we annotate the video in two le
vels L1 : annotation exhibits an overall game summary
including game name, date, place, teams, number of audience, scores, etc
L2 : annotates each event in the video using text semantics extracted from the text event and video boundaries obtained from text/video alignment<Event> <Priority> by <Player> of <Team> at <Time> <VideoStartFrame> <VideoEndFrame>
![Page 23: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/23.jpg)
Video Annotation and Indexing
![Page 24: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/24.jpg)
Introduction Semantic Annotation of Sports Video
Text Analysis Video Analysis Text/Video Alignment Video Annotation and Indexing
Personalized Video Retrieval Experiment and Evaluation
![Page 25: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/25.jpg)
![Page 26: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/26.jpg)
Introduction Semantic Annotation of Sports Video
Text Analysis Video Analysis Text/Video Alignment Video Annotation and Indexing
Personalized Video Retrieval Experiment and Evaluation
![Page 27: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/27.jpg)
Text Event Detection
The precisions and recalls of all the events except precision of the shot event for soccer (97.1%) achieve 100%.
![Page 28: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/28.jpg)
Shot Classification and Replay Detection
![Page 29: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/29.jpg)
![Page 30: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/30.jpg)
![Page 31: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/31.jpg)
![Page 32: A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 10, NO. 3, APRIL 2008](https://reader035.vdocument.in/reader035/viewer/2022062517/56649e955503460f94b9999c/html5/thumbnails/32.jpg)
Evaluation on Personalized Retrieval