visuelle perzeption für mensch- maschine schnittstellen · edgar seemann, 08.12.08 4 computer...

Interactive Systems Laboratories, Universität Karlsr uhe (TH)Edgar Seemann, 08.12.08 1

Visuelle Perzeption für Mensch-Maschine Schnittstellen

Vorlesung, WS 2008

Dr. Rainer StiefelhagenDr. Edgar Seemann

Interactive Systems LaboratoriesUniversität Karlsruhe (TH)

http://isl.ira.uka.de/msmmi/teaching/[email protected]

[email protected]


Programming

Assignments

WS 2008/09

Dr. Edgar Seemann

[email protected]


Organisatorisches

� There will be no lecture on Friday, January 23rd

Edgar Seemann, 08.12.08 4

Com

pute

r V

isio

n fo

r H

uman

-Com

pute

r In

tera

ctio

n

Res

earc

h G

roup

, Uni

vers

itätK

arls

ruhe

(TH

)cv

:hci

Assignment 1 Results

� Gruppe 1:Christian JohnerMike MorantePatrick Mehl

� Gruppe 2:Steffen Braun

� Gruppe 10:Thomas Stephan

� Gruppe 11:Igor Plotkin

� Gruppe 3:Martin WagnerHilke KieritzJan Hendrik Hammer

� Gruppe 4:Wenlei WuChengchao Qu

� Gruppe 5:Michael WeberTomas SemelaDennis Kopcan

� Gruppe 6:Johann KorndoerferDaniel KoesterDaniel Putsch

� Gruppe 8Mathias Luedtke(Florian Krupicka)

� Gruppe 9Felix ReuterElke Mueller

� Gruppe 7Benjamin BartoschThomas Lichtenstein


This Lecture

� Student presentations

� Short Intro into Assignment 3� Data Set

� Choice of Parameters

� Non-Maxiumum Suppression


Com

pute

r V

isio

n fo

r H

uman

-Com

pute

r In

tera

ctio

n

Res

earc

h G

roup

, Uni

vers

itätK

arls

ruhe

(TH

)cv

:hci

It’s your turn


Com

pute

r V

isio

n fo

r H

uman

-Com

pute

r In

tera

ctio

n

Res

earc

h G

roup

, Uni

vers

itätK

arls

ruhe

(TH

)cv

:hci

Assignment 3


Assignment 3

� People Detection� Detect for the given images, where and at which scale

the image contains people

� That is:1. We have to implement a sliding window search

2. At each location we classify the window with the SVM from assignment 2

3. We have to fuse the detections with a non-maxiumsuppression approach


The Same Training Data

� Training Set (PersonTrain.tar.bz2): � 2418 positive examples� 2436 negative examples� 96x160 pixels (64x128 + larger border)

� Idl-files:� Pos.idl� Neg.idl


Test Set

� Test set (PersonTestDetection.tar.bz2):� 41 positive images

� 0 negative images

� Ground-truth defined in testset.idl


Com

pute

r V

isio

n fo

r H

uman

-Com

pute

r In

tera

ctio

n

Res

earc

h G

roup

, Uni

vers

itätK

arls

ruhe

(TH

)cv

:hci

Quantitative Evaluation

� We use a precision-recall c++ binary (“precisionrecall”)

� A python script just sets some default command-line parameters

� ./doROC.py groundtruth.idl result.idl� Produces a text file result.txt containing the plotting

data

� plotSimple.py can be used to display the results


Your Task

� Compute an .idl file, which specifies for each test image a set of detection hypotheses

� Annotool helps to display results at different confidence levels


Sliding Window Technique

� We obtain for each position/scale a recognition score

� Parameters: scale range, scale steps, x/y-steps

� Positions with low scores can be discarded

Red: score>0.8


Sliding Window Parameters

� Use 64x128 windows

� Start at original resolution

� Shift windows� 4 pixels in x-direction

� 4 pixels in y-direction

� You free to experiment with these values

� Change scale� Shrink image with a factor of 1.2

� Other common choice is sqrt(2) as shrinking factor


Discard bad hypotheses

� We have to find a reasonable threshold for the SVM score

� Suggestion:� Accept everything with score >0� Display results in “AnnoTool”� Decrease/Increase threshold according to visual results

� Rules:� Allow enough hypotheses to have a recall of 1� We should have multiple hypotheses (shifted, scaled)

around each person


Non-Maximum Suppression

� A good detector will generally not only fire on the exact position

� Need to reduce the number of detections, since every additional detection (even on the object) will count as false positive detection


Naïve Approach

� Pick hypotheses in a greedy fashion� Accept the strongest hypothesis

� Remove all other hypotheses, which strongly overlap

� libAnnotation contains methods to compute the cover/overlap between two hypotheses


Non-Maxiumum suppression approaches

� Finding modes in a non-parametric distribution� Kernel Density Estimation

� Mean-Shift Mode Estimation (MSME) (e.g. [Dalal’05])

� Pixel-based reasoning (e.g. [Leibe et al. 2004])� Infer an object segmentation

� Use segmentation to determine, which pixels belong to which object


Mean Shift Theory


Kernel Density Estimation

� Which probability at position x should be higher?

� Single peak could result from noise, image artifacts etc.

x x


Intuitive Description

Distribution of identical billiard balls

Region ofinterest

Center ofmass

Mean Shiftvector

Objective : Find the densest region




Region ofinterest

Center ofmass

Mean Shiftvector





Region ofinterest

Center ofmass



AdaptiveGradient Ascent

Mean Shift Properties

• Automatic convergence speed – the mean shift vector size depends on the gradient itself.

• Near maxima, the steps are small and refined

• Convergence is guaranteed for infinitesimal steps only � infinitely convergent, (therefore set a lower bound)

• For Uniform Kernel ( ), convergence is achieved ina finite number of steps

• Normal Kernel ( ) exhibits a smooth trajectory, but is slower than Uniform Kernel ( ).


Real Modality Analysis

Tessellate the space with windows

Run the procedure in parallel


Real Modality AnalysisAn example

Window tracks signify the steepest ascent directions


Mean Shift Strengths & Weaknesses

Strengths :

• Application independent tool

• Suitable for real data analysis

• Does not assume any prior shape(e.g. elliptical) on data clusters

• Can handle arbitrary featurespaces

• Only ONE parameter to choose

• h (window size) has a physicalmeaning, unlike K-Means

Weaknesses :

• The window size (bandwidth selection) is not trivial

• Inappropriate window size cancause modes to be merged, or generate additional “shallow”modes � Use adaptive windowsize


Presentation

� Shortly present what exactly you have implemented (maybe a visualization of your features)

� What were the lessons learned?

� Please prepare a couple of slides

� Try to finish within the given 8 minutes

� Send me your PPT (<=PPT2003) or PDF files till February 6th, 8 am


Send me your results

� Please send me your results:� Group members (Name, MatrikelNr.)� Presentation file� Source code

� I should be able to run the code and reproduce the results

� E-Mail: [email protected]� If the e-mail is larger than 10mb, please try to split it

� I will try to give some feedback


End of Lecture

visuelle perzeption für mensch- maschine schnittstellen · edgar seemann, 08.12.08 4 computer...

Documents