Scene-independent group

profiling in crowd

Jing Shao, Chen Change Loy, and Xiaogang Wang

The Chinese University of Hong Kong

Hillsborough Disaster, England, 96 deaths, 766 injuries

The inquiry into the disaster ... the main reason …

the failure of police crowd control.

How to characterize and compare dynamics

across so many different crowded scenes?

Scene-specific Crowd flow field

R. Mehran, et al., ECCV, 2010

S. Ali, et al., CVPR, 2007

Topic model

B. Zhou, et al., CVPR, 2011

T. Hospedales, et al., ICCV, 2009

X. Wang, et al., CVPR, 2008

Social forces

B. Zhou, et al., CVPR, 2012

R. Mehran, et al., CVPR, 2009

P. Scovanner, et al., ICCV, 2009

Scene

Activity models learned from a specific-scene cannot be ap-plied to other scenes

Problem

Micro-level Multi-target tracking & individual

behavior analysis

W. Choi, et al., ECCV, 2012

S. Pellegrini, et al., ICCV, 2009

M. Andriluka, et al., CVPR, 2008

T. Zhao, et al., CVPR, 2004

Macro-level

Scale

It’s hard to detect and track in-dividuals precisely in different crowded scenes

How to characterize and compare dynamics

across so many different crowded scenes?

Micro-level

Macro-level Optical flow codewords

D. Kuettel, et al., CVPR, 2010

C. C. Loy, et al., BMVC, 2009

Spatio-temporal grids

R. Mehran, et al., CVPR, 2009

L. Kratz, et al., CVPR, 2009

S. Ali, et al., CVPR, 2007

Dynamic texture

A. B. Chan, et al., TPAMI, 2008

Finer group motion pattern cannot be captured by holistic methods

Scale

How to characterize and compare dynamics

across so many different crowded scenes?

Scene-specific

Scene-independent

Micro-level

Macro-level

Group-level

Scene Scale

Are there any common group properties to

characterize different crowded scenes?

How to characterize and compare dynamics

across so many different crowded scenes?

Biological and human crowd systems share the

similar properties

Our Goals

Collectiveness Stability Uniformity Conflict

Low

Hig

h

Group property quantification from vision view

GasSolid Pure Fluid Impure Fluid

Our Goals

Group state analysis

Gas

Solid

Fluid

Videos will be deleted after the presentation

Our Goals

Scene-independent crowd video classification

Contributions

A robust group detector

A set of scene-independent visual descriptors to

quantify group properties

Applications

Intrinsic group state identification

Scene-independent crowd video classification

474 crowd video clips

Most comprehensive crowd dataset to date

With 215 crowd scenes

Group are exhaustively labeled

Labeled with classes based on group activities

A New Crowd Dataset

Framework

Input video Group detection

11

21

2

3

3

2 Gas

Solid

1

3

Group state analysis & Crowd scene understanding

Fluid2

Crowd video classification

11

21

2

3

3

2 Gas

Solid

1

3

Group state analysis & Crowd scene understanding

Fluid2

Crowd video classification Group descriptors

Dynamic Group Detection

Collective Transition priorreveals collective motion of all

members in a groupCoherent filtering cluster Anchor tracklet

Seeding tracklets Final groups

[1]

[1] B. Zhou, et al.. In ECCV, 2012.

√×

Dynamic Group Detection

Group descriptors

Framework

Group descriptors

11

21

2

3

3

2 Gas

Solid

1

3

Group state analysis & Crowd scene understanding

Fluid2

Crowd video classification

Input video Group detection

Descriptors

Group properties

Intra-group Inter-group

Graph-driven group quantifications

Geometric

structure

Topological

structure

Collective

Transition

prior,

Collectiveness Stability Uniformity Conflict

Characterize the degree of individuals acting

as a union

A AA

Low High

Collectiveness

4 2 3 1 4 2 3 1

Stability

Characterize whether a group keeps internal

topological structure over time

Low

1 2 45

3

6

6

1 3 45 2

4

1 235

6

4

3

6

1 25

KNN KNN

High

12

3

4

4

213

1 23

4

4

21 3

KNN KNN

6 5 4 1 2 36 2 3 4 1 5

Edit distance: 5 Edit distance: 0

Cut number5

Cut number1

Uniformity

Characterize homogeneity of a group in terms

of spatial distribution

Low

High

high

low

high

low

Conflict

Characterize interaction/friction between

groups when they approach each other

Framework

Group detectionInput video

11

21

2

3

3

2 Gas

Solid

1

3

Group state analysis & Crowd scene understanding

Fluid2

Crowd video classification

11

21

2

3

3

2 Gas

Solid

1

3

Group state analysis & Crowd scene understanding

Fluid2

Crowd video classification Group descriptors

Application I: Group State Analysis

Confusion matrix Accuracy of using each

descriptor and combining them

1

0.5

0

Gas

Solid

Pure

fluid

Impure

fluid0

0.2

0.4

0.6

0.8

Collectiveness Stability Uniformity

Conflict Combination

Application I: Group State Analysis

Application II: Crowd Video Classification

(a) Holistic features (44%) (b) Our group descriptors (70%)

Confusion matrix

[3]

[3] L. Kratz, et al.. In CVPR, 2009.

1

0.8

0

0.6

0.4

0.2

1

0.8

0

0.6

0.4

0.2

Application II: Crowd Video Classification

Conclusion

Groups can

characterize and compare dynamics across different crowded scenes

be quantified from vision point of view

Scene independent crowd analysis

group state analysis

large-scale crowd video classification

Future work

crowd video retrieval

crowd scene understanding

crowd modeling, simulation, and behavior analysis.

Any Questions?Datasets and code are available. Project page is

http://www.ee.cuhk.edu.hk/~jshao/CUHKcrowd.html