Exploiting Simple Hierarchies for Unsupervised Human Behavior Analysis

Fabian Nater Helmut Grabner Luc Van GoolCVPR2010

Abstract

• A data-driven, hierarchical approach for the analysis of human actions in visual scenes

• Completely unsupervised• The model is suitable for coupled tracking and

abnormality detection on different hierarchical stages

Introduction

• Previous work: detect anomalies as outliers to previously trained models

• Our work: supporting autonomous living of elderly or handicapped people

• Rule-based systems: predefined dangerous cases, lacks general applicability

Introduction

• Two hierarchical representations: human appearances and sequences of appearances(actions, behavioral patterns)

• Map these images to a finite set of symbols describing what is observed

• Characterize in which order the observations occur

• learning the temporal (e.g. within a day or a week) and spatial dependencies

Appearance hierarchy

• Image stream ,arbitrary feature space

• Group similar image descriptors together using k-means to create a finite number of clusters

• Distance measuredefined in the feature space

Appearance hierarchy(H1)

• Eventually, each feature vector is mapped to a symbol

• Remove statistical outliers at every clustering step

• Distribution of distances of all the samples assigned to this cluster

Feature extraction

• Background subtraction• Rescaled to fixed size• Distance measure: chi-squared

Action hierarchy(H2)

• Basic actions to encode a state change• Only frequently occurring symbol changes are

considered• Higher level micro-actions are combination of

lower level micro-actions

• Represent image stream as a series of macri-actions of different lengths

Illustration

Anomalies

• H1 will be used for tracking and the interpretation of the appearance, H2 is used for the interpretation of actions

• To decide which cluster the extracted feature belongs to(high dimension), use data-dependent inlier:

• Threshold: 0.05 classified as outlier if its distance to the considered cluster center is larger than 95% of the data in that cluster

Update procedure

• Not all possible appearances and actions can be learnt off-line

• Include frequent appearances classified as outliers

• New leaf node clusters are established and new symbols defined

Update procedure

• Update micro-actions using the principle of exponential forgetting

• Start with empty database, everything considered abnormal at the beginning

Experiments

• Single person in-door videos• 1. Ourliers• 2. Symbols• 3. Action

length

Experiments

• 1. Frequently occurring scenes and abnormal scenes

• 2. Previously normal scenes• 3. New frequent normal scenes• 4. Anomalies

Conclusion

• Unsupervised analysis of human action scenes.

• Two automatically generated and updated hierarchies learned

• Normality and anomaly classification• Allows for model update