using adaptive tracking to classify and monitor activities in a site w.e.l. grimson, c. stauffer, r....
DESCRIPTION
Motivation The system should provide statistical descriptions of typical activity patterns, e.g., normal vehicular volume or normal pedestrian traffic paths for a given time of day it should detect unusual events, by spotting activities that are very different from normal patterns, e.g., unusual volumes of traffic, or a specific movement very different from normal observation it should detect unusual interactions between objects, e.g., a person parking a car in front of a building, exiting the car, but not entering the buildingTRANSCRIPT
![Page 1: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/1.jpg)
Using Adaptive Tracking To Classify And Monitor Activities In A Site
W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee
![Page 2: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/2.jpg)
Objectives to calibrate the distributed sensors, to construct rough site models, to classify detected objects, to learn common patterns of activity for
different object classes, and to detect unusual activities
![Page 3: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/3.jpg)
Motivation The system should provide statistical descriptions of typical
activity patterns, e.g., normal vehicular volume or normal pedestrian traffic paths for a given time of day
it should detect unusual events, by spotting activities that are very different from normal patterns, e.g., unusual volumes of traffic, or a specific movement very different from normal observation
it should detect unusual interactions between objects, e.g., a person parking a car in front of a building, exiting the car, but not entering the building
![Page 4: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/4.jpg)
Hypothesis motion tracking is sufficient to support a
range of computations about site activities
![Page 5: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/5.jpg)
Hardware The system observes activities with a “forest
of sensors” distributed around the site. Each sensor unit is a compact packaging of camera, on-board computational power, local memory, communication capability and possibly locational instrumentation (e.g., GPS)
![Page 6: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/6.jpg)
Robust Adaptive Tracker
![Page 7: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/7.jpg)
Robust Adaptive Tracker contd.. Color encodes direction and intensity encodes speed
![Page 8: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/8.jpg)
Using The Tracker To CalibrateWhen using multiple cameras it is essential to coordinate the individual video streams and find a common coordinate frame.Procedure -
Each camera independently tracks an object. Correspondence is established between points in the
video frames from different cameras. For each camera pair, small but sufficient set of points is
sampled from a larger set of point correspondences and a least square solution to the homographies is found.
This solution is compared with the current solution by calculating the mean square error on all the point correspondences and the best fit is retained.
![Page 9: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/9.jpg)
Using The Tracker To Calibrate contd...
![Page 10: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/10.jpg)
Using The Tracker For Site Modeling Objective is to determine the pose of the ground plane
relative to a camera. Estimate the height of the object. By using the estimate of the ground plane and the height
the distance to the object can be estimated. The portion between the un-occluded object and the
camera can be carved out as free space. When the object is occluded, this places a lower bound
on the occluding portion in the site. Since the ground plane is known these obstacles can be placed in the world coordinates of the site.
![Page 11: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/11.jpg)
Using The Tracker For Site Modeling contd..
![Page 12: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/12.jpg)
Using The Tracker to Classify Classify objects – size and the aspect
ratio of the tracked objects can be used to classify and label the objects in the site
Classify actions – tracks of the moving objects can be used to classify activities by clustering the tracks on the basis of common attributes
![Page 13: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/13.jpg)
Using The Tracker to Classify contd..
Clustering Algorithm #1-
1) Numeric Iterative Hierarchical Clustering Algorithm
2) MDL cut
![Page 14: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/14.jpg)
Using The Tracker to Classify contd..
![Page 15: Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee](https://reader035.vdocument.in/reader035/viewer/2022062401/5a4d1aed7f8b9ab05997bc24/html5/thumbnails/15.jpg)
Using The Tracker to Classify contd..