vision-based analysis of small groups in pedestrian crowds weina ge, robert t. collins, r. barry...
TRANSCRIPT
![Page 1: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/1.jpg)
Vision-Based Analysis of Small Groups in Pedestrian Crowds
Weina Ge, Robert T. Collins, R. Barry Ruback
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 34, NO. 5, MAY 2012
![Page 2: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/2.jpg)
Goal
![Page 3: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/3.jpg)
Outline
• Introduction• Background and related work• Detecting and tracking individuals• Identifying small groups• Experimental evaluation• Conclusion
![Page 4: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/4.jpg)
Introduction
• There has been increasing interest in using surveillance trajectory data for human behavior analysis.
• This paper discover small groups of people who are together.
![Page 5: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/5.jpg)
Introduction
• A small group behavior suggests new strategies for police intervention during public disturbances.
• Police should look at small groups, only a few of which might merit coercion.
• This paper demonstrates that computer vision is a capable for supporting sociological analysis.(crowds data)
![Page 6: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/6.jpg)
Background and related work
• 89 percent of people with at least one other person 52 percent with 2 , 32 percent with 3, and that 94 percent left with whom they came with. [8]
• So divide a crowd of people into small pedestrain groups is useful for identification group behavior.
[8] C. McPhail, “Withs across the Life Course of Temporary Sport Gatherings,” unpublished manuscript, Univ. of Illinois, 2003.
![Page 7: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/7.jpg)
Background and related work
• Collective locomotion behavior is also studied in the traffic analysis and crowd simulation community.
• Microscopic level model is suitable for evacuation planning than macroscopic model[40].
[40] A. May, Traffic Flow Fundamental. Prentice Hall, 1990.
![Page 8: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/8.jpg)
Detecting and tracking individuals
• For videos captured from high elevation/wide angle views where people are small, detected by using (RJMCMC) to find a set of overlapping rectangles.
![Page 9: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/9.jpg)
Detecting and tracking individuals
• For higher resolution videos, We use the HoG detector, implemented from the description in Dalal and Triggs [61].
• The camera is stationary, so background subtraction is useful for detecting.
[61]N. Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 886-893, 2005
![Page 10: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/10.jpg)
![Page 11: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/11.jpg)
Detecting and tracking individuals
• Sets of tracklets extracted in overlapping sliding windows of time are combined into longer trajectories.
• Make a N(trajectories)*M(tracklets) affinity table.
• best assignment of trajectories is solved by Hungarian algorithm[64].
[64] H.W. Kuhn, “The Hungarian Method for the Assignment Problem,” Naval Research Logistics Quarterly, vol. 2, pp. 83-97, 1955.
![Page 12: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/12.jpg)
Detecting and tracking individuals
• Trajectories for which there is no matching tracklet have their “health” decremented.
![Page 13: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/13.jpg)
Identifying small groups
• Consider the trajectory of a person in the scene as a set of tuples (s,v,t)
![Page 14: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/14.jpg)
Identifying small groups
• intergroup closeness between two groups of people by a generalized, symmetric Hausdorff distance
![Page 15: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/15.jpg)
Identifying small groups
![Page 16: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/16.jpg)
Identifying small groups
• Within each temporal slice, starting from clusters with a single member, we gradually group people by agglomerative hierarchical clustering.
• Each merging step is governed by intergroup closeness(Hausdorff), and intragroup tightness.
![Page 17: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/17.jpg)
Experimental evaluation
![Page 18: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/18.jpg)
Experimental evaluation
• The ground truth data , coded by several coders .
• Coders can replay the sequence if they want to, the ground truth groups are their agreements.
• Another set of ground truth are made by interviewing the people walked in the video.
![Page 19: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/19.jpg)
Experimental evaluation
![Page 20: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/20.jpg)
Kapa test
• agreement=(20+15)/50=0.7• P(A)yes = 0.5• P(B)yes = 0.6
• Random agreement=0.5*0.6+0.5*0.4=0.5
![Page 21: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/21.jpg)
Experimental evaluation
![Page 22: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/22.jpg)
Experimental evaluation
![Page 23: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/23.jpg)
Experimental evaluation
![Page 24: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/24.jpg)
Experimental evaluation
![Page 25: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/25.jpg)
Experimental evaluation
![Page 26: Vision-Based Analysis of Small Groups in Pedestrian Crowds Weina Ge, Robert T. Collins, R. Barry Ruback IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE](https://reader030.vdocument.in/reader030/viewer/2022032703/56649d1e5503460f949f202d/html5/thumbnails/26.jpg)
Conclusion
• Results demonstrate that automated tracking is capable of real crowds faster and with similar accuracy as human observation.
• Our future work is investigation of small group configurations across different social events.