Download - Objectives
![Page 1: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/1.jpg)
Motion Detection in UAV Videos by
Cooperative Optical Flow and Parametric Analysis
Masaharu Kobashi
![Page 2: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/2.jpg)
Objectives
● Show contrast between simple research videos and UAV videos.
● Describe strengths and weaknesses of well-know conventional techniques.
● Propose a new system designed for robustness.
![Page 3: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/3.jpg)
Overview of Presentation
1. Overview of motion analysis2. UAV videos and their characteristics3. Strengths and weaknesses of conventional
techniques4. Design of our new system5. Performance of our system
![Page 4: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/4.jpg)
What is motion analysis?
● Motion is a relative concept– Relative to image plane (static camera)– Relative to objects in the image
● Factors that make it difficult– Changing extrinsic and intrinsic camera parameters
(Pan, tilt, translation, rotation, and zooming)– Illumination changes– Overlap of multiple motions with semi-transparency
(dust, fog, mist, glasses, etc.)
![Page 5: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/5.jpg)
Classes of Motion Analysis Methods
1. Detecting flow of regularly-shaped regions– Optical flow, Patch/Block flow
2. Detecting flow of irregularly-shaped regions– Segmentation-based flow analysis– Variable-window/block-based flow analysis
3. Detecting difference between two frames in terms of an objective function– Parametric motion analysis
![Page 6: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/6.jpg)
Popular Videos for Motion Analysis
![Page 7: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/7.jpg)
What are UAV videos?
![Page 8: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/8.jpg)
Characteristics of UAV videos
● Noisier and of lower quality than standard videos – Low bandwidth of UAV transmitter– Quality of camera
● Constant changes of camera parameters (airplane motion, camera's pan, tilt, zooming)– Often abrupt and quick changes
● Unrestricted natural scenes– Including dimly lit scenes, deserts, fields
![Page 9: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/9.jpg)
Contrast betweenResearch videos and UAV videos
(See following demo movies.)
![Page 10: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/10.jpg)
Strengths and Weaknesses ofTechniques Related to
Motion Analysis
● Gradient-based Search● Pyramid-Based Coarse-to-Fine Approach● Optical-Flow-Based Motion Detection● Parametric Motion Estimation● Hybrid Approaches
![Page 11: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/11.jpg)
Gradient-Based Search
● Hill-Climbing for global optimization based on local gradient information
● Examples: – Lucas-Kanade registration algorithm– Light constancy equation
● Strength:– Fast (avoiding exhaustive search)
● Weakness:– Brittle (1-pixel support; a pixel noise can derail
it)
![Page 12: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/12.jpg)
Pyramid-Based Coarse-to-Fine Approach
● Strengths:– Can summarize
information compactly
– Remedy for range-limited gradient-based methods
![Page 13: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/13.jpg)
Weaknesses of Pyramid
● Fixed coverage of child cells (Fixed partitioning)
● Same information to all child cells
● Vital information can be lost in summarizing process
● All parts of image required to use same number of levels
![Page 14: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/14.jpg)
Results from Pyramid's Weaknesses
● The more levels, the more errors.● Small motion areas suffer unnecessary errors
due to the height of the pyramid needed for areas of greater motion.
● Motion of thin/small parts tends to be detected incorrectly.
![Page 15: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/15.jpg)
Optical-Flow-Based Motion Analysis
● Steps:1. Detection of flow for all pixels in each frame.2. Accurate segmentation of the flow into
moving and stationary objects.
![Page 16: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/16.jpg)
Strengths and Weaknesses of
Optical-flow-Based Motion Analysis
● Strength: – Dense motion information (for every pixel)
available.● Weaknesses:
– Difficult to obtain accurate vectors for all pixels.– Segmenting flow into multiple moving and static
objects is difficult.
![Page 17: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/17.jpg)
Difficulties in Finding Flow Discontinuity
![Page 18: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/18.jpg)
Parametric Motion Estimation
● Steps:1. Assume a model of transformation (Affine,
Perspective).2. Compute transformation matrix of the static regions
from the point correspondences of static regions.3. Find point correspondences between two frames f1
and f2.4. Find difference between frame f1 and an image
created by warping frame f2 by the transformation matrix.
![Page 19: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/19.jpg)
Strengths and Weaknesses of Parametric Motion Estimation
● Strengths:1. Capable of precise matching between frames at all
points even at sub-pixel precision. 2. Unlike optical flow analysis, it can find different
motion layers equally well in any transformation.● Weaknesses:
1. Needs reasonably accurate point correspondences.2. Must know which points belong to static region.
![Page 20: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/20.jpg)
Hybrid Approach
● Use of multiple motion detection techniques in a single system (e.g. optical flow detection + parametric motion estimation)
● Examples:– Wang and Adelson's motion layers– Our system
![Page 21: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/21.jpg)
Overview of Our System
● Design philosophy: 1. Hybrid system based upon optical flow detector and
parametric motion estimator2. Takes advantage of strengths of both and avoid their
weaknesses3. Priority on robustness rather than efficiency
● System structure
![Page 22: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/22.jpg)
Module 1: Optical Flow Detector
● Block matching with relatively large block (16x16)● Use of Disk Balance Ratios (DBR)● Use of Sum of Squared Error (SSD)
![Page 23: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/23.jpg)
Disk Balance Ratios (DBR)
● Objective: – To describe the orientation of a region in
terms of intensity.● Definition:
![Page 24: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/24.jpg)
Limitations of SSD
![Page 25: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/25.jpg)
Use of Sum of Squared Difference (SSD)
● Steps to find the flow vector of block i:1. If the SSD distribution is too flat among candidates, make
no decision. (Near homogeneous region.)2. If there are two or more candidates which have
distinctively better SSD than the rest and flow vectors from block i to these candidates are far apart, make no decision. (Aperture problem.)
3. Otherwise, find the candidate block having the least SSD. The vector from block i to that candidate is the flow vector of block i.
![Page 26: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/26.jpg)
Module 2: Extraction of Reliable Flow
● Reliable flow is defined by the following reliability measure.
![Page 27: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/27.jpg)
Module 3:Finding Connected Reliable Flow Groups
● A variant of the single-link clustering algorithm.● Connect adjacent reliable blocks if their flow
vectors are similar in both angle and magnitude.
![Page 28: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/28.jpg)
Module 4:Finding Ground-Covering Flow Groups
● Steps:1. If there is a group whose area is more than 40% of the screen, the
group is regarded to cover the ground only.2. If available, use the grouping information of the previous frame in
the following way: • For a group “g”, if more than 70% of the area of g is covered by the
ground region in its predecessor frame, then g belongs to the ground. Otherwise, make the decision by the next step.
3. Select the set of groups that minimizes the SSD between the intensities of corresponding pixels between the template frame and the image created by warping the next frame by the inverse of the transformation matrix.
![Page 29: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/29.jpg)
Computation of Transformation Matrix
![Page 30: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/30.jpg)
![Page 31: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/31.jpg)
Singular Value Decomposition (SVD)
● Merit:– Capable of handling even singular matrix
![Page 32: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/32.jpg)
Module 5:Evaluation of Difference Data
● Steps:1. Compute the mean difference “m” of all pixels in the reliable
ground blocks.2. Exclude the reliable ground blocks from the search region for the
moving objects.3. In the remaining blocks find the pixels whose difference value is
greater than m+1.5.4. Remove isolated small regions from the list of candidate regions.5. Mark all blocks which contains a candidate region as candidate
blocks.6. Form connected components of candidate blocks. Each connected
set of blocks is a moving object.7. For each connected set of blocks, output its bounding box to
indicate the location of the moving object.
![Page 33: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/33.jpg)
Output
![Page 34: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/34.jpg)
Performance Comparison
● Lucas-Kanade plus pyramid v.s. Our system● Michael Black's Algorithm v.s. Our system● Simple research videos v.s. UAV videos
![Page 35: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/35.jpg)
Lucas-Kanade+Pyramid
![Page 36: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/36.jpg)
Lucas-Kanade + Pyramid
![Page 37: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/37.jpg)
Black's Algorithm
![Page 38: Objectives](https://reader033.vdocument.in/reader033/viewer/2022042901/56813b1f550346895da3d505/html5/thumbnails/38.jpg)
Black's Algorithm