video object extraction using feature matching based on nonlocal matting
TRANSCRIPT
Video Object Extraction Using Feature Matching Based on Nonlocal MattingMeidya Koeshardianto, Eko Mulyanto and Mochamad Hariadi | ITS Surabaya, Indonesia
What is video object extraction and how to make it easy implementation?
Question :
Matting Equations
• Video object extraction is used for extracting foreground and background
object from still image or video sequences.
𝐼 = 𝛼𝐹 + 1 − 𝛼 𝐵
Constraints (Scribbles and Trimap)
Trimap Interface :
Bayesian Matting (Chuang et al CVPR 01)
Poisson Matting (Sun et al SIGGRAPH 04)
Random walk (Grady et al 05)
Etc..
Scribbles Interface :
Wang&Cohen ICCV 05
Closed Form Matting TPAMI 08
Nonlocal Matting CVPR 11
Etc..
𝛼 = 1
𝛼 = 1
𝛼 = 0
𝛼 = 0
𝛼 ∈ [1,0]
Problems
1. Every frames need constraints to determine Foreground and Background
2. Accurate object extraction on each frames.
• Automatic constraints (stroke/scribbles)
• Feature matching
• Nonlocal matting for Video Object Extracting
SIFT Algorithm is used
Laplacian
TransformVideo / image sequence
Image Template
Nonlocal MattingPoint Matching Alpha Matte
Point Extraction
Procedure
Key Point Extraction
Step 1
Feature Matching
Step 2
Laplacian Transform (Nonlocal Matting)
Step 3
Why nonlocal matting is employed?
Intepretation Laplacian as a graph 𝐺 = 𝑉, 𝐸 where 𝐴 is weight of edge 𝐸(𝑖, 𝑗)
With derivation of 𝑫𝜶 = 𝑨𝜶 then 𝑫− 𝑨 𝜶 ≈ 𝟎 or 𝜶𝑻𝑳𝜶 ≈ 𝟎 where 𝑳 = 𝑫 − 𝑨 𝑻(𝑫 − 𝑨)
The quadratic form 𝜶𝑻𝑳𝜶Measure of smoothness along the
edge of 𝐺
Why nonlocal matting is employed?
If there is a subset of pixels that exactly cluster in the graph implied by Laplacian 𝐿 then the
value of the objective function 𝑞(𝛼) is minimized if the rest of the pixels in the cluster are labeled
Automatic Scribbles - SIFT
Key Detection
Keypoint Descriptor
Automatic Scribbles - SIFT
Feature Matching
𝜃 = 0,2 – 0,4 𝜃 = 0,5
𝜃 = 0,6 𝜃 = 0,7
𝜃 = 0,8 𝜃 = 0,9
Image template for automatic video object extraction
Answer :
Experiments – Samples Video
Conclusion and Road a Head
• Feature matching could be used as scribbles or stoke for object extraction,
nonlocal matting method.
• According from our sample video, it show that objects can be extracted by
most satisfactory parameter using 𝜃 = 0,05
• Future work include investigating updated or replaced image template
periodically. So that, the image template feature will be still recognized on the
frame.