fast and accurate optical flow estimation primal-dual schemes and second order priors thomas pock...
TRANSCRIPT
Fast and Accurate Optical Flow Estimation
Primal-Dual Schemes andSecond Order Priors
Thomas Pock and Daniel CremersCVPR Group, University of Bonn
Collaborators: Christopher Zach, Markus Unger, Werner Trobin, and Horst Bischof
Variational Optical Flow – Short History
Horn and Schunck
1981
Black and Anadan, Cohen
1993
Aubert
2000 2004
Brox et al.
2006
Bruhn et al.
Outline• Model of Horn and Schunck• TV-L1 Model• Fast Numerical Scheme• Parallel Implementation• 2nd order Prior
The Model of Horn and Schunck [1]
Regularization Term Data Term (OFC)
+ Convex+ Easy to solve- Does not allow for sharp edges in the solution- Sensitive to outliers violating the OFC
[1] Horn and Schunck. Determinig Optical Flow. Artificial Intelligence, 1981
Can we do better?
• Replace quadratic functions by L1 – norms• Done by Cohen, Aubert, Brox, Bruhn, ...
+Allows for discontinuities in the flow field+Robust to some extent to outliers in the OFC+Still convex- Much harder to solve
How can we minimize this functional ?
• Compute Euler-Lagrange Equations
• Non-linear, non-smooth, ...
Standard Approach
• Replace L1 – norm by regularized variants (Charbonnier function)
• Example:• Small epsilon: Nearly degenerated• Large espilon: Smears edges
Our Approach(1)
• Introduce auxiliary variables and constraints
• Quadratic penalty
Our Approach(2)• What do we gain?• We solve a sequence of simpler problems
Algorithm[3]:1. For fixed (u´,v´), solve for(u,v) using Chambolle‘s algorithm[4]2. For fixed (u,v), solve for (u´,v´) using a 1D shrinkage formula3. Goto 1. until convergence
[2] Rudin, Osher and Fatemi. Nonlinear Total Variation Based Noise Removal Algorithms, 1992[3] Zach, Pock and Bischof. A Duality Based Algorithm for Realtime TV-L1 Optical Flow, DAGM 2007 [4] Chambolle. An Algorithm for Total Variation Minimization, 2004.
1D Problem
ROF Model [2]
Implementation
• Numerical scheme can be easily parallelized• We use state-of-the-art GPUs
Performance Evaluation
Image Size Frames per Second128x128 192256x256 108512x512 36
• TV-L1 Optical Flow Implemented in CUDA 2.0• Computed on Nvidia GeForce GTX 280• 25 Overall Iterations (5 Chambolle Iterations)
Results for TV-L1
Ground Truth:
Our Results:
Input Image:
2nd order Prior• TV regularization favors piecewise constant flow
fields (frontoparallel motion)• Extension to piecewise affine flow fields?• Approach of Cremers et al. [5]– Fixed number of regions
• Approach of Nir et al. [6]– Over-parametrized optical flow
• Our approach [7]– 2nd order derivatives to regularize flow field[5] Cremers and Soatto, Motion Competition: A Variational Framework for Piecwise Parametric Motion
Segmentation.[6] Nir, Bruckstein and Kimmel, Over-Parameterized Variational Optical Flow, IJCV 2007[7] Trobin, Pock, Cremers and Bischof, An Unbiased Second-Order prPior for High-AccuracyMotion Estimation, DAGM 2008
2nd-L1 Optical Flow
• 2nd order derivatives are not orthogonal• We use a transformation due to Danielsson [8]
• Optimization– Similar strategy to TV-L1
– 4th order PDE
[8] Danielsson and Lin, Efficient Detection of Second-Degree Variations in 2D and 3D Images, 2001.
Comparison
Ground truth TV-L1 2nd -L1
Results for 2nd-L1
Ground Truth:
Our Results:
Conclusion
• TV-L1 Optical Flow– Fast Numerical Scheme
• Parallel Implementation– Realtime Performance
• 2nd order prior – Piecewise affine motion
Recent Application: Tracking