rule-based action recognition using object trajectories
DESCRIPTION
Rule-based Action Recognition using Object Trajectories. UCF VIRAT Efforts. Recognition Process. Input: Time ordered series of 2 dimensional locations of object centroid in image coordinates (trajectory) Output: Action(s) pertaining to the given trajectory Method: - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/1.jpg)
Rule-based Action Recognition using Object Trajectories
UCF VIRAT Efforts
![Page 2: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/2.jpg)
Recognition Process Input: • Time ordered series of 2 dimensional locations
of object centroid in image coordinates (trajectory)
Output: • Action(s) pertaining to the given trajectory
Method:• Compute multiple discriminative features for
each trajectory• Classify action(s) using a comprehensive set of
rules
![Page 3: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/3.jpg)
Trajectory Features Dynamics based
• For a curve,r (t) = {(x0,y0),…,(xt,yt)}
• Instantaneous speed, v = || dr / dt ||
• Average speed,vav = ∑n vi
/ n• Acceleration,
a = || dv / dt ||• Arc length,
s = ∫ v dt
![Page 4: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/4.jpg)
Trajectory Features Shape based• Capture geometrical information
• Tangent vector, v = dr / dt
• Unit Tangent vector, T = v / ||v||• Curvature, k(t) = || dT / ds || = ||T’ / v||• Four point cross ratio,
Cr (p1,p2,p3,p4) = (p3-p1)(p4-p2) / (p4-p1)(p3-p2)
![Page 5: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/5.jpg)
Rules Rules are based on quantization of feature values
A decrease followed by an increase in unit tangent vector = Right turn, or vice versa
Two consecutive increase or decrease = U-turn
![Page 6: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/6.jpg)
Rules
Going Straight Turn Right U-turn
Trajectory
Trajectory
Trajectory
dT / dt dT / dt dT / dt
![Page 7: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/7.jpg)
Rules Accelerate / Decelerate events are detected directly from features
Deceleration to zero speed = Stopping
Maintaining speed and direction = Maintain distance
….
![Page 8: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/8.jpg)
Results Trajectories extracted from the UCF Aerial Actions dataset
Handle walking, running, turning left and right, and taking U-turn
![Page 9: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/9.jpg)
Results
Walking Forward
![Page 10: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/10.jpg)
Results
Turn Right
![Page 11: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/11.jpg)
Results
U-turn
![Page 12: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/12.jpg)
Results
U-turn
![Page 13: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/13.jpg)
Results
Turn Right
![Page 14: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/14.jpg)
Results
Turn Left
![Page 15: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/15.jpg)
Results
Running Forward
![Page 16: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/16.jpg)
VIRAT Events
Standing Walking Running Digging Gesturing Carry Object
Person Actions Accelerate
Decelerate Turning Stopping U-turn Maintain distance
Vehicle Actions
![Page 17: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/17.jpg)
Ideas & Future Work Object Classification
• Discriminate between similar trajectories of different objects• Separate person, vehicle and facility detectors• Person-Vehicle and Person-Facility events
Multiple Trajectories per object
• Track multiple points on each object• Recognize stationary trajectories using object
kinematics• Gesturing, Digging• Can also help or eliminate object detector /
classifier
![Page 18: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/18.jpg)
Ideas & Future Work Geo-Registration
• Distance, speed, and acceleration in image plane may not be useful
• Allow use of computed distances on ground• Do not have to be absolute longitude and latitude• Accelerate, Decelerate, Maintain Distance
Invariant features
• Trajectory features should be invariant to: Changes in view Changes in scale
• Projective invariant features can eliminate need for geo-registration
![Page 19: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/19.jpg)
Ideas & Future Work Learning based framework
• Learning and clustering of discriminative features• More robust compared to rule based methods• Representative of training trajectory samples• Associated confidence for each recognition• Recognition of unseen and composite events
Quality of Input Tracks
• Availability of object trajectories is assumed• Features are highly dependant on tracking accuracy• Broken, merged and split tracks severely affect
performance
![Page 20: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/20.jpg)
VIRAT Events
Standing Walking Running Digging Gesturing Carry Object
Person Actions Accelerate
Decelerate Turning Stopping U-turn Maintain distance
Vehicle Actions Loading Unload Open trunk Close trunk Getting into car Getting out of car Enter/exit building
Multi-agent Actions
• With provision of object recognition / classification, multiple trajectories and geo-registration
![Page 21: Rule-based Action Recognition using Object Trajectories](https://reader036.vdocument.in/reader036/viewer/2022062811/56815fcd550346895dceca65/html5/thumbnails/21.jpg)
Thank You!