Exploiting Simple Hierarchies for Unsupervised Human Behavior Analysis
Fabian Nater Helmut Grabner Luc Van GoolCVPR2010
Abstract
• A data-driven, hierarchical approach for the analysis of human actions in visual scenes
• Completely unsupervised• The model is suitable for coupled tracking and
abnormality detection on different hierarchical stages
Introduction
• Previous work: detect anomalies as outliers to previously trained models
• Our work: supporting autonomous living of elderly or handicapped people
• Rule-based systems: predefined dangerous cases, lacks general applicability
Introduction
• Two hierarchical representations: human appearances and sequences of appearances(actions, behavioral patterns)
• Map these images to a finite set of symbols describing what is observed
• Characterize in which order the observations occur
• learning the temporal (e.g. within a day or a week) and spatial dependencies
Appearance hierarchy
• Image stream ,arbitrary feature space
• Group similar image descriptors together using k-means to create a finite number of clusters
• Distance measuredefined in the feature space
Appearance hierarchy(H1)
• Eventually, each feature vector is mapped to a symbol
• Remove statistical outliers at every clustering step
• Distribution of distances of all the samples assigned to this cluster
Feature extraction
• Background subtraction• Rescaled to fixed size• Distance measure: chi-squared
Action hierarchy(H2)
• Basic actions to encode a state change• Only frequently occurring symbol changes are
considered• Higher level micro-actions are combination of
lower level micro-actions
• Represent image stream as a series of macri-actions of different lengths
Illustration
Anomalies
• H1 will be used for tracking and the interpretation of the appearance, H2 is used for the interpretation of actions
• To decide which cluster the extracted feature belongs to(high dimension), use data-dependent inlier:
• Threshold: 0.05 classified as outlier if its distance to the considered cluster center is larger than 95% of the data in that cluster
Update procedure
• Not all possible appearances and actions can be learnt off-line
• Include frequent appearances classified as outliers
• New leaf node clusters are established and new symbols defined
Update procedure
• Update micro-actions using the principle of exponential forgetting
• Start with empty database, everything considered abnormal at the beginning
Experiments
• Single person in-door videos• 1. Ourliers• 2. Symbols• 3. Action
length
Experiments
• 1. Frequently occurring scenes and abnormal scenes
• 2. Previously normal scenes• 3. New frequent normal scenes• 4. Anomalies
Conclusion
• Unsupervised analysis of human action scenes.
• Two automatically generated and updated hierarchies learned
• Normality and anomaly classification• Allows for model update