!"#$%&'()*+,-Æ/0123456789:;<=>?@ABCDEFGHIJKLMNOPQRST UVWXYou Are How You Walk: Uncooperative MoCap Gait Identificationfor Video Surveillance with Incomplete and Noisy Data
Michal Balazia · Petr SojkaMasaryk University, Faculty of Informatics, Botanická 68a, 602 00 Brno, Czech Republic
https://gait.fi.muni.cz
Identification Pipeline
Spo�ed walker
MoCap data
Gait template
Gait sample
Iden�fied walker
DEPARTURES
BANK
OPENING HOURS
BANK
Phase IV:Iden�fying walkers
Phase III:Extrac�ng gait features
Phase II:Detec�ng gait
Phase I:Acquiring MoCap data
Phase I – Acquiring Motion Capture DataMotion capture (MoCap) technology providesvideo clips of moving individuals containingan overall structure of the human body andestimated 3D joint coordinates.
MoCap data can be collected online by a readilyavailable system of multiple cameras (Vicon)or by a depth camera (Microsoft Kinect).
To visualize MoCap data, a stick figure thatrepresents the human skeleton can be recoveredfrom joint spatial coordinates in time.
Frame: 0 25 50 75 100 125 150
y
xz
Phase II – Detecting Gait CyclesPeople spotted in our tracking space do notwalk all the time; on the contrary, they performvarious activities.
Identifying people from gait requires videosegments where they are actually walking.
Clean gait cycles need to be first filtered out.
There are methods for detecting gait cyclesdirectly as well as action recognition methodsthat need a demonstrative example of a gaitcycle to query general motion sequences. Frames
0 75 225 300 450
Feet
dis
tan
ce
150 375
Gait cycles
Phase III – Extracting Gait FeaturesLet a model of the human body have J jointsand all NL learning samples of CL walkers belinearly normalized to their average length T .
Labeled learning data in a sample space havethe form GL = {(gn, `n)}NL
n=1 where
gn =[[γ1 (1) · · · γJ (1)] · · · [γ1 (T ) · · · γJ (T )]
]>is a gait sample (one gait cycle) in whichγ j (t) ∈ R3 are 3D spatial coordinates of jointj ∈ {1, . . . , J} at time t ∈ {1, . . . ,T } normalizedto the person’s position and walk direction.
Each learning sample falls into one of thelearning identity classes {Ic}
CLc=1 labeled by `n.
Feature extraction is set by matrix Φ ∈ RD×D
from D-dimensional sample space G = {gn}Nn=1
to D-dimensional feature space G ={gn
}Nn=1
with D < D.
A given gait sample gn can be transformed intogait template by gn = Φ>gn.
Templates are compared by the Mahalanobisdistance.
Evaluated methods:• 8 hand-designed geometric feature sets• Features learned by MMC• Features learned by PCA+LDA
Evaluated metrics:• Correct classification rate• Discriminativeness• Robustness to noisy and incomplete data• Clusterability• Scalability
0.55
0.60
0.65
0.70
0.75
0.80
Are
a u
nd
er t
he
RO
C c
urv
e
0.50
0.55
0 10 20 30 40 50 60 70 80 90 100
Are
a u
nd
er t
he
RO
C c
urv
e
% noise for 9 learning and 55 evaluation identities
Ahmed Ali Andersson Ball
Dikovski Kwolek Preis Sinha
MMC PCA+LDA
0.65
0.70
0.75
0.80
0.85
Are
a u
nd
er t
he
RO
C c
urv
e
0.60
(2,62)
(3,61)
(4,60)
(5,59)
(6,58)
(7,57)
(8,56)
(9,55)
(10,54)
(11,53)
(12,52)
(13,51)
(14,50)
(15,49)
(16,48)
(17,47)
(18,46)
(19,45)
(20,44)
(21,43)
(22,42)
(23,41)
(24,40)
(25,39)
(26,38)
(27,37)
(28,36)
(29,35)
(30,34)
(31,33)
(32,32)
Configurations in the form (# learning identities, # evaluation identities)
Specifications of the evaluated methods are available at web https://gait.fi.muni.cz.
Phase IV – Identifying WalkersPerson re-identification (Re-ID) focuses onspotting people of interest in multiple cameras.
But in video surveillance, labeled data for allthe people encountered is never available.
Training the model on an auxiliary databasecreates an unsupervised environment suitablefor searching for similar gait templates and forclustering them into potential walker identities.
One can retrieve information about a person’sappearances captured by the video surveillancesystem – their location trace that includestimestamp and geolocation of each appearance.
rejectedacceptedquery 39° 43' 04'' N 104° 51' 50'' W
2017/06/24 22:49:38
39° 49' 06'' N 104° 52' 10'' W
2017/06/23 13:24:19
39° 51' 05'' N 104° 40' 34'' W
2017/06/21 07:55:16
DEPARTURES
BANK
AcknowledgementsMichal Balazia and Petr Sojka: You Are How You Walk: Uncooperative MoCap Gait Identification for Video Surveillance with Incomplete and Noisy Data, International Joint Conference on Biometrics (IJCB), Denver, 2017.Michal Balazia and Petr Sojka: Gait Recognition from Motion Capture Data, ACM Transactions on Multimedia Computing (TOMM), Special Issue on Representation, Analysis and Recognition of 3D Humans, 2017.The data used in this project was created with funding from NSF EIA-0196217 and was obtained from http://mocap.cs.cmu.edu. Evaluation framework and database are available online at https://gait.fi.muni.cz.
