hierarchical method for foreground detectionusing codebook model jing-ming guo, yun-fu liu,...
TRANSCRIPT
![Page 1: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/1.jpg)
Hierarchical Method for Foreground DetectionUsing Codebook Model
Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng HsuIEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR
VIDEO TECHNOLOGY, VOL. 21, NO. 6, JUNE 2011
![Page 2: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/2.jpg)
Outline
• Background Model Construction– Block-Based Background Subtraction– Pixel-Based Background Subtraction
• Hierarchical Foreground Detection• Background Models Updating with the
Short-Term Information Models• Experimental Results
![Page 3: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/3.jpg)
Background Model Construction
• This method involves two types of codebooks(CBs), block-based and pixel-based CBs.
• The modeling of two CBs is similar to the former CB[14]
[14] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, “Real-time foreground-background segmentation using codebook model,” Real- Time Imaging, vol. 11, no. 3, pp. 172–185, Jun. 2005.
![Page 4: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/4.jpg)
Background Model Construction
![Page 5: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/5.jpg)
Background Model Construction
• There are two different time intervals for training (xt).
• (1 ≤ t ≤ T) and (t > T) for training the background models and foreground detection.
• The updating algorithms are separated into two parts for different time zones.
![Page 6: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/6.jpg)
The Features Used in Block-Based Background Subtraction
• A frame xt of size P x Q is divied into multiple non-overlapped blocks of size M x N.
• The former block truncation coding(BTC) reduce the frame into two means,high-mean and low-mean.
• In this paper ,we have four means to represent a frame, high-top mean (μht ), high-bottom mean (μhb), low-top mean (μlt ), and low-bottom mean (μlb).
![Page 7: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/7.jpg)
The Features Used in Block-Based Background Subtraction
![Page 8: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/8.jpg)
The Features Used in Block-Based Background Subtraction
• Each means have three colors(RGB),so each codebook have 12 dimensions.
![Page 9: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/9.jpg)
Updating Block-Based Background Models (CBs) in the Training Phase
• a specific block can be represented as a vector Vb = {vb
t|1 ≤ t ≤ T }.
• A CB for a block can be represented as C = {ci|1 ≤ i ≤ L}, consisting of L codewords
• An additional weight wi is geared for indicating the importance of the ith codeword.
• Codebook size is (P/M)x(Q/N)
![Page 10: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/10.jpg)
Updating Block-Based Background Models (CBs) in the Training Phase
![Page 11: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/11.jpg)
Updating Block-Based Background Models (CBs) in the Training Phase
![Page 12: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/12.jpg)
Updating Pixel-Based Background Models (CBs) in theTraining Phase
• The same as block-based method.• Codebook size is P x Q.• Each codebook is 3 dimensions (RGB)
![Page 13: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/13.jpg)
Hierarchical Foreground Detection
• After the background models training as indicated before the time point T, the two CBs are applied to the proposed hierarchical foreground detection.
• The foreground is obtained by background subtraction.
![Page 14: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/14.jpg)
Foreground Detection with the Block-Based CB
• the input vector (vbt) extracted from a block is
compared with the ith block-based codeword (ci) to determine whether a match is found
• When a vbt is classified as background, the
corresponding block is also used to update the pixel-based CB.
![Page 15: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/15.jpg)
Foreground Detection with the Pixel-Based CB
• This subsection introduces how to classify a pixel in a block to foreground or background.
• The foregrounds are classified into one true foreground and two fake foregrounds (shadow and highlight).
![Page 16: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/16.jpg)
Foreground Detection with the Pixel-Based CB
![Page 17: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/17.jpg)
Foreground Detection with the Pixel-Based CB
![Page 18: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/18.jpg)
Background Models Updating with theShort-Term Information Models
• an additional variable timeics is involved to
store the updated time for estimating whether the corresponding ith codeword (ci
s ) has been updated for a specific period or not.
• If the duration is longer than a predefined parameter Ds
delete, the corresponding cis is
simply a temporary foreground.
![Page 19: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/19.jpg)
Background Models Updating with theShort-Term Information Models
• When cis , is favor to strong stationary (
wics ≥ Dadd), the short-term information model
can be considered as a part of the true background model.
• This additional value is employed for filtering out ci which meets the states of eventually moving as foregrounds with the predefined parameter Ddelete.
![Page 20: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/20.jpg)
Experimental Results
• λ = 5 for block-based , λ = 6 for pixel-based, η = 0.7, θcolor = 3, β = 1.15 ,γ = 0.72
,Dupdate = 3, and α = 0.05, Dadd = 100, Ds
delete = 200, and Ddelete = 200
![Page 21: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/21.jpg)
Experimental Results
• [9]MOG• [5]color model• [11][25] hierarchical MOG• [14]CB
[9] C. Stauffer and W. E. L. Grimson, “Adaptive background mixture models for real-time tracking,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2. Jun. 1999, pp. 246–252.[5] R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detection moving objects, ghosts, and shadows in video streams,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 10, pp. 1337–1342, Oct. 2003.[11] Y.-T. Chen, C.-S. Chen, C.-R. Huang, and Y.-P. Hung, “Efficient hierarchical method for background subtraction,” Pattern Recognit., vol. 40, no. 10, pp. 2706–2715, Oct. 2007.[25] C.-C. Chiu, M.-Y. Ku, and L.-W. Liang, “A robust object segmentation system using a probability-based background extraction algorithm,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 4, pp. 518–528, Apr. 2010.[14] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, “Real-time foreground-background segmentation using codebook model,” Real- Time Imaging, vol. 11, no. 3, pp. 172–185, Jun. 2005.
![Page 22: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/22.jpg)
• C)MOGd)Color modele)CBf)g) hierarchical MOG
![Page 23: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/23.jpg)
C)MOGd)Color modele)CBf)g) hierarchical MOG
![Page 24: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/24.jpg)
C)MOGd)Color modele)CBf)g) hierarchical MOG
![Page 25: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/25.jpg)
![Page 26: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/26.jpg)
Experimental Results
![Page 27: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/27.jpg)
Experimental Results
![Page 28: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/28.jpg)
Experimental Results
![Page 29: Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS](https://reader035.vdocument.in/reader035/viewer/2022062221/56649f275503460f94c3e4b7/html5/thumbnails/29.jpg)
Conclusion
• The block-based stage can enjoy high speed processing speed and detect most of the foreground without reducing TP rate.
• Pixel-based stage can further improve the precision of the detected foreground object with reducing FP rate.
• Short-term information is employed to improve background updating