automatic video shot detection from mpeg bit stream jianping fan department of computer science...
TRANSCRIPT
![Page 1: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/1.jpg)
Automatic Video Shot Detection
from MPEG Bit StreamJianping Fan
Department of Computer ScienceUniversity of North Carolina at CharlotteCharlotte, NC 28223
http://www.cs.uncc.edu/~jfan
![Page 2: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/2.jpg)
Why we need video shots?
a. Text Retrieval: Keyword Extraction Indexing
Document Storage
Reverse File Indexing
![Page 3: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/3.jpg)
Why we need video shots?
b. Database Query: Entity Extraction
sid name login age gpa
53666 Jones jones@cs 18 3.453688 Smith smith@eecs 18 3.253650 Smith smith@math 19 3.8
Indexing
Database Storage
B-Tree Indexing
![Page 4: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/4.jpg)
Why we need video shots?
c. Image Retrieval: Object Indexing Indexing
Database Storage
Ontology Indexing
![Page 5: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/5.jpg)
Why we need video shots?
MPEG Video Sequence
Video shot
frames
![Page 6: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/6.jpg)
Why we need video shots?
Story Unit in News Video
Each story unit may consist of multiple connected video shots!
![Page 7: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/7.jpg)
Why we need video shots?
Indexing
Video Shots in Storage
Shot Indexing
???
![Page 8: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/8.jpg)
Storage Database
Query Processing
Server
Networks
???Video shot =?= keyword in video?
Shot is used as basic unit for video indexing!
![Page 9: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/9.jpg)
1. Why we need shot detection?
Potential Search on Video Shots:
a. Give me some videos which consist of similar video shots!
b. Give me all the videos which are related to this video shot!
![Page 10: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/10.jpg)
2. Content Structure of Video Sequence
Video sequence
Video shot
frames
Shot-Based Video Content Interpretation!
![Page 11: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/11.jpg)
3. Coding Structure in MPEG Video
Video sequence
GOP
MPEG encoding
shot?
![Page 12: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/12.jpg)
Interesting Questions
If you are leader of MPEG around 1994, how can you design new MPEG standards which can be used for video indexing?
![Page 13: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/13.jpg)
New MPEG Standard with Video Shot
Video sequence
GOP GOP GOP
Video shot
frames
GOP is Video Shot!
![Page 14: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/14.jpg)
4. What’s mean of video shot?
(a) Scene Cuts;
(b) Fade in & Fade out;
© Dissolves & Wipes
Why We Have Shots: (a) Human Editing; (b) Content Change & Camera Motion
Shot Types:
![Page 15: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/15.jpg)
4. What’s mean of video shot?
time
video
Shot 1
a. Scene Cuts:
Sudden change of video content or focus
![Page 16: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/16.jpg)
b. Fade: gradual transition between a screen and a
4. What’s mean of video shot?
constant image (fade out).
![Page 17: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/17.jpg)
c. Dissolve: gradual transition from one screen to
4. What’s mean of video shot?
another, the first screen fade & the second one fade out.
![Page 18: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/18.jpg)
4. What’s mean of video shot?
Dissolve
![Page 19: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/19.jpg)
d. Wipe: a line moves across the screen & new appears behind the line
4. What’s mean of video shot?
![Page 20: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/20.jpg)
4. What’s mean of video shot?
Wipe
![Page 21: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/21.jpg)
Video Sequence
Video shot
frames
Conclusion: Why we have video shots?
Big change between two continuous video frames!
![Page 22: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/22.jpg)
How can we detect video shots? Why we can define them as cuts,
wipe, ….? Because they are changed between neighboring video frames!!!!
How can we measure the changes or similarities between the neighboring video frames?
How can make decision on whether they are change or not?
![Page 23: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/23.jpg)
5. How to detect video shot?
Major Components for Shot Detection:
a. Visual Representation of Video Frames: Color, Texture…
b. Difference Calculation for Neighboring Video Frames
c. Threshold for Decision Making: How Large is Enough?
![Page 24: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/24.jpg)
5. How to detect video shot?
How to measure statistical property of video frames?
Color Histogram
00.050.1
0.150.2
0.250.3
0.350.4
(0,0,0) (110,112,80)
3-D Column 1RGB
![Page 25: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/25.jpg)
5. How to detect video shot?
How to measure statistical property of video frames?
Color Histogram
![Page 26: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/26.jpg)
5. How to detect video shot?
How to measure statistical property of video frames?
Texture Histogram
![Page 27: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/27.jpg)
5. How to detect video shot?
![Page 28: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/28.jpg)
5. How to detect video shot?
How we can do this more efficient on MPEG videos?
MPEG videos
MPEG Decoder
Color Histogram Difference
Scene Cut Frame Output
Automatic Threshold
Determination
a. Easy but not smart
![Page 29: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/29.jpg)
Scene Cut
![Page 30: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/30.jpg)
Effects on color histogram difference by scene cuts:
5. How to detect video shot?
![Page 31: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/31.jpg)
5. How to detect video shot?
How to measure the change of video content?
Color Histogram Difference
![Page 32: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/32.jpg)
5. How to detect video shot?
How we can select the threshold automatically?
a. J. Fan, D.K.Y. Yau, W.G. Aref, A. Rezgui, ``Adaptive motion- compensated video coding scheme towards content-based bit rate allocation”, Journal of Electronic Imaging, vol.9, no.4, pp. 521-533, 2000.
This algorithm can adapt the thresholds to different videos.
b. J. Fan, et al., ``ClassView: Hierarchical video shot classification and retrieval”, IEEE Trans. on Multimedia, 2004.
This algorithm can adapt thresholds to different video shots.
![Page 33: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/33.jpg)
5. How to detect video shot?
Relationships among continuous frames can defined as:
scene cuts versus non-scene cuts
probability
0
1
1T
non-scene cut scene cut
![Page 34: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/34.jpg)
5. How to detect video shot?
How to obtain T automatically?
Entropy for non-scene cut frames:
![Page 35: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/35.jpg)
Interesting Questions
What will happen on MPEG bit stream if the current video frame is a shot boundary?
I B PHow can we use such properties for shot detection?
![Page 36: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/36.jpg)
5. How to detect video shot?
b. Complex but smart approach
GOP MPEG encoding
If scene cuts happen on I frames in MPEG Video?
How to calculate color histograms of I frames efficiently?
Only decode DC coefficient!!!
![Page 37: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/37.jpg)
5. How to detect video shot?
b. Complex but smart approach
What will happen on MPEG bit stream if there is a shot boundary?
1.5Mb/s
4Mb/s
Picture Quality high
low
![Page 38: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/38.jpg)
Reference frame( I or P)
Current P frame
Most macroblocks can not find their correspondences!!
If content change happens on P frames in MPEG video?
Only decode the block coding types in P frames!!!
![Page 39: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/39.jpg)
Previous Reference
Current B frame Future Reference
Most macroblocks in B current frame can only be predicted by backward way!!
If content change happens on B frames in MPEG video?
Only decode the coding types in B frames!!!!
![Page 40: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/40.jpg)
6. Shot Detection via Edge Extraction
Edges indicate the content structure of video frames!
![Page 41: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/41.jpg)
6. Shot Detection via Edge Extraction
MPEG videos
MPEG Decoder
Color Edge Detection
Scene Cut Frame Output
Automatic Threshold
Determination
a. Easy but not smart
![Page 42: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/42.jpg)
6. Shot Detection via Edge Extraction
a. J. Fan, W.G. Aref, M.-S. Hacid and A.K. Elmagarmid, ``An automatic isotropic color edge detection technique”, Pattern Recognition Letters, vol.22, pp.1419-1429, 2001.
1 2 1
0 0 0
-1 -2 -1
1 0 -1
2 0 -2
1 0 -1
2 1 0
1 0 -1
0 -1 -2
0 1 2
-1 0 1
-2 -1 0
![Page 43: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/43.jpg)
6. Shot Detection via Edge Extraction
b. complex but smartOnly decode only part of MPEG videos
255 255 255 255 255 255 255 255255 255 255 255 255 255 255 255255 255 255 255 255 255 255 255255 255 255 255 255 255 255 255255 255 255 255 255 255 255 255255 255 255 255 255 255 255 255255 255 255 255 255 255 255 255255 255 255 255 255 255 255 255
DCT
2550 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0
What you find from this figure?
![Page 44: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/44.jpg)
6. Shot Detection via Edge Extraction
1. Calculate the directional edge histogram
Horizontal edgesVertical edges Northwest diagonal edgesNortheast diagonal edges
2. Calculate the differences of the directional edge histogram
3. Threshold to obtain the scene cuts
![Page 45: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/45.jpg)
Effects on color histogram different by Fades & Dissolves
7. Complex Shot Detection
![Page 46: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/46.jpg)
Wipe Modeling
Dissolve Modeling
7. Complex Shot Detection
![Page 47: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/47.jpg)
8. Camera Motion Detection
a. Zoom in
![Page 48: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/48.jpg)
8. Camera Motion Detection
b. Zoom out
![Page 49: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/49.jpg)
8. Camera Motion Detection
c. Left Pan
![Page 50: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/50.jpg)
8. Camera Motion Detection
d. Right Pan
![Page 51: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/51.jpg)
Why We Need Keyframe Extraction?
![Page 52: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/52.jpg)
Why We Need Keyframe Extraction?
![Page 53: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/53.jpg)
Keyframe Extraction
What are keyframes? Different from others!
How to measure the significance of video frames?
How to make the decision?
![Page 54: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/54.jpg)
9. Background Modeling for Shot
Detection
![Page 55: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/55.jpg)
9. Background Modeling for Shot
Detection
![Page 56: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/56.jpg)
9. Background Modeling for Shot
Detection
![Page 57: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/57.jpg)
9. Background Modeling for Shot
Detection
![Page 58: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/58.jpg)
9. Background Modeling for Shot
Detection
![Page 59: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/59.jpg)
9. Background Modeling for Shot
Detection
![Page 60: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/60.jpg)
10. Shot-Based Video Content Representation
Statistical Information
Keyframe-Based Information
(a)Color histogram & variance;
(b) edge histogram & variance
© motion histogram & variance;
(a)Color histogram; (b) edge histogram
© motion histogram
![Page 61: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/61.jpg)
11. Shot-Oriented Video Semantics Interpretation
![Page 62: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/62.jpg)
Who cares?
![Page 63: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/63.jpg)
12. Who works on this topic?
1. IBM research center
2. Intel
3. Microsoft Redmond & Beijing
4. Kodak at Rochester
5. HP at Polo Alto
6. Google
7. Yahoo!
![Page 64: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/64.jpg)
12. Requirement
If you are asked to work on video shot detection from compressed video stream, what you should know (basic requirement)?a. MPEG video coding standard:
how to decode and read MPEG file
b. Knowledge on video shot analysis:
Color, motion, edge, fade, dissolve, …
c. Skills on C++
![Page 65: Automatic Video Shot Detection from MPEG Bit Stream Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC](https://reader035.vdocument.in/reader035/viewer/2022062409/56649cad5503460f94970324/html5/thumbnails/65.jpg)
Project Introduction
Design a system for automatic shot detection from MPEG video streams
Interface design: read in MPEG files and display shot boundaries, or even decision process!