person re-identification, phd day 2011

Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 20111

Università degli Studi di Cagliari

Exploiting Multiple Component Representationsfor Person Re-Identification

Ph.D. candidate: Riccardo SattaAnnual report, Year I

Exploiting Multiple Component Representations for Person Re-Identification. Cagliari, 23 feb 2011 2

Outline

• Introduction to Person Re-Identification

• Problem formulation

• The proposed Multiple Component Matching (MCM) framework

• Implementation of MCM, and experimental results


Introduction

In video surveillance, it is often desirable to determine if an individualhas already been observed over a network of cameras.

This issue is called Person Re-Identification

In general, we can’t apply face recognition algorithms (low resolution!),therefore we must consider the global “appearance” of the individual

short term problem

Scenarios: Tracking of the movements of an individual in large public places

monitored by several non-overlapping cameras Use in conjunction with other common identification techniques

(RFID, biometric systems)


Main issues

- Low resolution of the frames (all figs)Less data availableMore subject to noise

- Changing lighting conditions (fig. 1)Brightness changesContrast changes

- Different sensor responses (fig. 2)Colour temperatureWhite balance

- Partial occlusions (fig. 3)

- Pose variations (fig. 1, 4)

figure 1

figure 4

figure 2

figure 3


Problem formulation

We can model the problem as follows:given a gallery set of templates T = {T1,…,Tn}

and a probe Q, find the most similar template T* T with respect to a similarity measure D(·, ·):

T*= arg min D(Ti, Q)

Descriptor generation

MATCHING SCORE(similarity)

Descriptor generation

TEMPLATE

PROBE

T=

QTi


My contribution to this field

• A general framework to for people re-identification problems– Inspired to common paradigms in Machine Learning and Computer Vision– Includes and generalizes ideas partly embedded in previous works– Aims at providing a common foundation for existing and future methods

• A method for person re-identification– Simple, yet effective, implementation of the framework

• … and some interesting ideas for further developments


Towards a Multiple Component Representation of the Human Body

Human body peculiarities– non-rigid object– complex kinematics– composed of many quasi-rigid parts

Possible approach: COMPONENT SUBDIVISION

We can take into account an arbitrary m-component subdivision by fusing matching scores of every part


Towards a Multiple Component Representation of the Human Body

How to represent every single component?

Multiple Instance Learning (MIL) paradigm:the object of interest is represented by a bag of instances

• MIL framework adapted to a matching problem• Every component is described by a series of instances

– Patches, point of interest…

{x1 , … , xp}

{x1 , … , xp}

{x1 , … , xp} {x1 , … , xp} {x1 , … , xp}

{x1 , … , xp}

{x1 , … , xp}

Object


Multiple Component Matching Framework

The proposed Multiple Component Matching framework integrates multiple instance representations with component subdivision in a matching paradigm

– every individual Xi is decomposed

into an ordered sequence of parts X j

i

– every part X j

i is represented

by a set of instances X j

i,n

– matching is performed at set level

– global matching distance is a combination of set distances

Example (4 parts)


Multiple Component Matching Framework

Multiple Component Matching (MCM) framework– Inspired to MIL statistical learning framework– Embeds two concepts (part subdivision and multiple instance representation)

that can be found, even if partly and implicitly, in the main part of the existing works

– Aims at providing a solid foundation for existing and future works

Well, nice theory… but in practice?

Implementation of MCM:– choose part subdivision– choose what to treat as an “instance”– choose an appropriate distance between sets d( ), and how to combine set

distances


MCM Implementation

Part subdivisionAnti-symmetry properties of the human silhouette [1]

Let we first define two operators:

chromatic bilateral operator

spatial covering operator

[1] M. Farenzena, L. Bazzani, A. Perina, V. Murino, and M. Cristani. Person re-identification by symmetry-driven accumulation of local features. In Proc. IEEE Conf. on Comp. Vision and Patt. Rec., 2010

[ ]∑

+−

=δδ

δiiB

ii ppdiC,

),(),( 2

[ ]( ) [ ]( )[ ]δδδδ +− −= iiii BABA

JiS ,,

1),(


MCM Implementation

Part subdivision

head-torso axis is given by

separating regions that strongly differ in area

torso-legs axis is given by

separating regions with strongly different appearance and similar area

Two parts: [torso] and [legs], discarding the head

( )( ) ( )δδ ,,1minarg iSiCii

TL +−=

( )( )δ,minarg iSii

HT −=


MCM Implementation

Instances– For each part, we extract random overlapping patches of [25% - 75%] width in

respect of the part width and [25% - 75%] height in respect of the part height– Each instance is described by an HSV colour histogram (24-12-4 bins)


MCM Implementation

Instances - Simulation– To asset lighting and contrast variations we generate synthetic patches for

templates– Each RGB pixel is multiplied by different values

• we chose [1.4 1.2 1.0 0.8 0.6]• This changes both brightness (mean) and contrast (variance) of the patches


MCM Implementation

Distance between sets d( )– k-th Haussdorff distance [2]

Metric:

distB Bhattacharyya distance between the HSV histograms of the patches

ypos,a, ypos,b mean vertical position of the patch

Set distance combination– Average distance between sets:

[2] Wang and J.-D. Zucker. Solving the multiple-instance problem: A lazy learning approach. In Proc. Int. Conf. Mach. Learn., 2000.

( ) ( ) ( )( )ABhBAhBAdH ,,,max, =

( ) ( )bakthBAhk −= min, BbAa ∈∈ ,

( ) ( ) bposaposbaB yyHSVHSVba ,,1,dist −+⋅=− β

( ) ( )∑=i

i BAdM

D ,1

, BA


Evaluation

Evaluation on the ViPER dataset– 632 individuals, 1 template and 1 probe image per individual, SvsS scenario– random selection of 316 individuals, average performance on 10 runs

SDALF refers to the state-of-the art on this dataset [1]

[1] M. Farenzena, L. Bazzani, A. Perina, V. Murino, and M. Cristani. Person re-identification by symmetry-driven accumulation of local features. In Proc. IEEE Conf. on Comp. Vision and Patt. Rec., 2010


Prototypes

AmI LAB demo applicationA prototype is being realized at the Ambient Intelligence Lab of Sardegna Ricerche


Further developments

• Research issues– Dissimilarity-based representations

– Frame quality

– Multiple frames accumulation

Prototype 1

Prototype 2

Prototype N

d1

d2

dN

Dissimilarity representation


Further developments

• New applications– Personal Photo Re-Tagging

ID 1 ID 1

ID 2

ID 2 ID 3 ID 3


Thanks!

Questions?

person re-identification, phd day 2011

Documents