13 november 2013birkbeck college, u. london1 research methods lecturer: steve maybank department of...

13 November 2013 Birkbeck College, U. London 1

Research Methods

Lecturer: Steve Maybank

Department of Computer Science and Information Systems

[email protected] 2013

Data Research Methods in Computer Vision


Digital Images

A digital image is a rectangular arrayof pixels. Each pixel has a positionand a value.

95 110

40 34

125

108

25 91

158

116

59 112

166

132

101

124

Original colour image from the EfficientContent Based Retrieval Group, Universityof Washington


Size of Images

Digital camera, 5,000x5,000 pixels, 3 bytes/pixel -> 75 MB.

Surveillance camera at 25 f/s ->1875 MB/s.

1000 surveillance cameras -> ~1.9 TB/s.

Not all of these images are useful!


Image Compression Divide the image into blocks, and

compress each block separately, e.g. JPEG uses 8x8 blocks.

Lossfree compression: the original image can be recovered exactly from the compressed image.

Lossy compression: the original image cannot be recovered.


Why is Compression Possible?

Natural image: values ofneighbouring pixels arestrongly correlated.

White noise image: values ofneighbouring pixels are notcorrelated. Compression discardsinformation.


Measurement Space

Each 8x8 block yields a vector in R64. The vectorsfrom natural images tend to lie in a low dimensionalsubspace of R64.

R64

Vectors from 8x8 blocks


Strategy for Compression

Choose a basis for R64 in which the low dimensionalsubspace is spanned by the first few coordinate vectors.Retain these coordinates and discard the rest.

R64 vectors from 8x8 blocks


Discrete Cosine Transform

small. be to tends then large, is If

. and DCT

Then .1, 0,by , vectorsDefine

norm.Euclidean

theis . where,DCTthat

Note . matrix, othogonal 6464certain a is where

DCT

Then block. 88an from obtained vector a be Let

64

1

64

1

64

2/1

64

i

i i

Tii

ij

TT

T

ci

ieUcwiecw

ieijieRie

wUwUwUww

IUUU

Uww

Rw


Basis Images for the DCT

UTe(1) UTe(2) UTe(3) UTe(4)


Example of Compression using DCT

Original image Image constructed from 3 DCTcoefficients in each 8x8 block.


Linear Classification

xx

xx

xx

y yy y

Given two sets X, Y ofmeasurement vectors fromdifferent classes, find a hyperplanethat separates X and Y.

A new vector is assigned to theclass of X or to the class of Y,depending on its position relativeto the hyperplane.


Fisher Linear Discriminant

.

maximises which 0 vector theis FLD The

.1

by covariance class within thedefine and

by covariance classbetween theDefine

.1

,1

,1

define ,,...,,,...,Given

11

1111

11

vSv

vSvvJ

v

yyyyxxxxnm

S

S

zyzynm

nzxzx

nm

mS

S

yxnm

zyn

yxm

x

RyyYRxxX

wT

bT

n

iii

m

iiiw

w

b

b

n

ii

m

ii

n

ii

m

ii

kn

km


Maximisation of J(v)

required. is maximum The . somefor

if 0/therefor

2

.0for maximise and

by Define

2

vSvS

vvJ

vSv

vSvSvvSvSv

v

vJ

vvJ

vSv

vSvvJ

vJ

wb

wT

wbT

bwT

wT

bT


Edge Regions and Regions Without a Peak

3x3 blocks with large Sobel gradients3x3 blocks such that the grey level ofthe central pixel equals the mean greylevel of the 9 pixels


Projections Onto a Random Plane

Sobel edges Regions withouta peak

Superposed plots


Projections Onto a 1-Dimensional FLD

Sobel edges Regions withouta peak

Combinedhistograms

Histogram of a DCT Coefficient


The pdf is leptokurtic, i.e. it has a peak at 0and “fat tails”.

Sparseness of the DCT Coefficients

For a given 8×8 block, only a few DCT coefficients are significantly different from 0.

Given a DCT coefficient of low to moderate frequency, there exist some blocks for which it is large.


Whitening the Data


whitened.also are

1 ,

vectors then thematrix, orthogonalan is If matrix.identity

theis theof covariance The data. whitened theare The

.1 ,

by Define

1

by covariance

theEstimate ts.coefficien DCT of vectorsbe 1 ,Let

2/1

1

niiVz

V

iziz

niiuCiz

iz

iuiun

C

C

niiu

Tn

i

Independence and Correlation


ed.uncorrelat

t thanrequiremenstronger much a is ceindependenbut ,(1)(2)

(2) .for ,,

ift independen are The

(1) .for ,0

ed,uncorrelat are The . of components thebe 1 ,Let

.

by rv thedefine and

of covariance thebe Let .in variablerandom a be Let

2/1

jizpzpzzp

z

jizzE

zzniz

uCz

z

uCRu

jiji

i

ji

ii

n

Independent Components Analysis

Let z be a rv with values in R^n such that cov(z)=I.

Find an orthogonal matrix V such that the components of s =Vz are as independent as possible.


Method

Assume a particular leptokurtic pdf p for the components si, e.g. a two sided Laplace density.

Find the orthogonal matrix V with rows v1,…,vn that maximises the product

p(s1)…p(sn) = p(v1.z)…p(vn.z)


Example Natural Image Statistics: a

probabilistic approach to early computational vision, by A. Hyvarinen, J. Hurri and P.O. Hoyer. Page 161.

ICA applied to image patches yields Gabor type filters similar to those found in the human visual system.


Stereo Pair of Images


UC Berkeley looking westhttp://arch.ced.berkeley.edu/kap/wind/?p=32

Salience A left hand image region is salient if the

correct right hand matching region can be found with a high probability.

H: hypothesis that (v(1),v(2)) is a matching pair.

B: hypothesis that (v(1),v(2)) is a background pair


Formulation using PDFs The region yielding v(1) is salient if

p(v(2)|v(1), H) differs significantly from p(v(2)|v(1), B)

Measurement of difference: the Kullback-Leibler divergence:


kR

dvBvvpHvvpHvvp 2,1|2/,1|2ln,1|2

Illustration of the Kullback-Leibler Divergence

KL divergence = 0.125


KL divergence = 0.659

Preprint on salience available from SJM

13 november 2013birkbeck college, u. london1 research methods lecturer: steve maybank department of...

Documents

birkbeck college

london22 slide

london20 slide

london23 slide

london19 slide

london25 slide

london21 slide

london18 slide