# statistical learning for image orientation · statistical learning for image orientation luke...

of 34/34
Statistical Learning for Image Orientation Luke Barrington Mentors: Nuno Vasconcelos, UCSD ECE Dept. Babak Jafarian, Cal-(IT) 2

Post on 07-Jun-2020

1 views

Category:

## Documents

Embed Size (px)

TRANSCRIPT

• Statistical Learning for Image Orientation

Luke Barrington

Mentors: Nuno Vasconcelos, UCSD ECE Dept.Babak Jafarian, Cal-(IT)2

• Agenda

Project OverviewSVM BackgroundParameter SelectionFirst ExperimentFeature Selection MethodsFuture Work / Wrap Up / Questions

• Project OverviewDevelop an image classification system for MMS-enabled cell phonesMotivation is for use by spectators at Athens 2004 Olympic gamesUse modern classification technologies to improve server-side organization and provide useful user output

Goal for my project: image orientation

• Sports Photographers

• Image OrientationCamera phones, scanners, digital cameras, online image databases, …Device output has standard dimension

At least 50% will be incorrect

Bulk, automated, corrective processing before display to human users

• System Block Diagram

MMS/Internet server

.

.

.

.

.

.

UnorderedDirectory

OrderedOutputUser Input

Classifier

• Classifier Block DiagramUser’s Parameters

+1-1…+1

KernelErrorWeights

TRAIN ModelParameters CLASSIFY+1

-1

Training DataTraining Data

Data Labels

Class Label

Unlabeled Data

• Agenda

Project OverviewSVM BackgroundParameter SelectionFirst ExperimentFeature Selection MethodsFuture Work / Wrap Up / Questions

• Decision Boundary

Margin

Training = Learn Decision Boundary

-1 training examples

+1training examples

Support Vectors

• Decision Boundary

Classification

Support Vectors

-1

+1

B

Test B = +1

A

Test A = -1

• Extension to Multi-Class Problems

SVM is a binary decision classifierOne solution is a decision tree format

n-class problem requires ½n(n-1) classifiers

A B C D

A/B C/D

A/C A/D B/C B/D

• Multi-Class: 1-vs-All ClassifierAlternative is a 1-vs-all classifierTrain each class with all training dataLabel: +1 (in the class) or

–1 (in any other class)Requires n classifiers for an n-class problemOutput may be undetermined (all classifiers return –1) Class +1 -1

a a {b, …, n}b b {a, c, …, n}

n n {a, …, n-1}M MM

• Linear Support Vector Machine for separable dataAssume the training points can be divided by a linear hyperplane:

xi●w + b = 0 w is the normal vector

is perpendicular distance to origin

With appropriate scaling, the training data satisfy:xi●w + b ≥ +1 for yi = +1xi●w + b ≤ -1 for yi = -1

Combine this into a single condition: yi(xi●w + b) – 1 ≥ 0 for all i

The width of margin is: 2/||w||Robust classifier has widest margin => minimize ||w||2

wb−

• Linear SVM, separable data contd.Primal Lagrangian formulation:

Setting the gradient of LP w.r.t. w & b = 0 gives the conditions:

Combining these equality constraints in the dual formulation gives:

SVM training now amounts to maximizing LD with respect to the weights αi, subject to the constraints above and αi ≥ 0

Support vectors are all the training points where αi > 0

For classifier’s purposes, all other points can be ignored

∑∑ ⋅−== ji

jijiji

l

iiD xxyyL

,1

ααα

( ) ∑∑==

++⋅−=l

ii

l

iiiiP bwxywL

11

2

21 αα

011

== ∑∑==

l

iii

l

iiii yandxyw αα

• Linear SVM for unseparable dataWe modify the equations from linear and separable case by the addition of a “slack” variable ξ. Now the equations become

xi●w + b ≥ +1 – ξi for yi = +1xi●w + b ≤ -1 + ξi for yi = -1 ξi ≥ 0if ξi > 1 => an error occurred Σiξi is an upper-bound on the training error

Now we want to minimize:

C is a user-specified cost parameter

This is again a constrained optimization problem where we want to maximize:

Subject to:

The data which satisfy this maximization are the Support Vectors

∑∑ ⋅−== ji

jijiji

l

iiD xxyyL

,1ααα

k

iiC

w⎟⎠

⎞⎜⎝

⎛− ∑ξ2

2

001

=≤≤ ∑=

l

iiii yandC αα

• Non-linear SVMsGeneralize to non-linear decision function for the classifierMap the data to some other (higher-dimension) Euclidean space H:

Now the problem depends only on

Find a kernel function such that

Use this (non-infinite) result to get classifier output

HRd →Φ :

)()( ji xx Φ⋅Φ

)()(),( jiji xxxxK Φ⋅Φ=

( ) ∑∑==

+=+Φ⋅Φ=SS N

iiii

N

iiii bxsKybxsyxf

11),()()( αα

• Agenda

Project OverviewSVM BackgroundParameter SelectionFirst ExperimentFeature Selection MethodsFuture Work / Wrap Up / Questions

• SVM ParametersKernel:

Linear (dot product) K(xi,xj) = (xi.xj)Polynomial: K(xi,xj) = (xi.xj + 1)p

RBF: K(xi,xj) = e-|| xi-xj ||²/2σ²

Sigmoid neural network: K(xi,xj) = tanh(κxi.xj -δ)

C (Error penalty) :Sets an upper bound on the alpha coefficients Higher C gives higher weight to outliersLow C assumes many data points are outliersAssign different C for each class based on number training data points

• Parameter Selection“Research” = correct choice of parametersRadial Basis Function generalizes bestParameter space is 2-DFind optimum set using grid search

σ

C

• Agenda

Project OverviewSVM BackgroundParameter SelectionFirst ExperimentFeature Selection MethodsFuture Work / Wrap Up / Questions

• Sample Problem: Car Orientation

= +1

= -1

• Feature Space

Grayscale pixel values (0 to 255)No input scalingDown-sample image to 128x128 and row scan

Input data is 16384-dimensional vector

• Data SetFrom MIT labs516 imagesTraining set: 400Validation set: 50RBF kernel, σ = 4 C = 25

• Car Orientation ResultsClassifiers for {0º, 30º, 60º, 90º, 120º, 150º, 180º}

In-class detection accuracy = 97.5 ± 2%False Positive < 2%

• Initial ResultsSVM solves this problem easily on a limited domainTraining time can be significant (large feature size)Classification is almost instantaneous

• Agenda

Project OverviewSVM BackgroundParameter SelectionFirst ExperimentFeature Selection MethodsFuture Work / Wrap Up / Questions

• Expansion of Training SetEfficacy of classifier depends on training data

Corel Database

Sports Images

Crawl web sites

Use subclasses to choose feature spaceTrain “final” classifier on everything

• Feature SpaceGrayscale pixel valuesSquare images

Each image provides 4 training examples

Down-sample image to 112x112Input data is 12544 dimensional vector

= +1 = -1

• ResultsBest Corel class

70% detection rate, 3% false positiveWorst Corel class

0% detection rate, 30% false positiveSports images

63% detection rate, 14% false positive

Still only working on one specific class at a time

• New Feature SpacePixel data is over-sized and under-poweredNeed to capture edge information in more succinct and compact formatEdge orientation histogram

comes from edge detection

⎟⎟⎠

⎞⎜⎜⎝

⎛=0k

⎟⎟⎠

⎞⎜⎜⎝

⎛=

k0

for vertical edges for horizontal edges

•Construct a gradient field on N x N image sub-blocks

•Sub-block size depends on image size

•Eigenvectors of H are features

•Use N = 16, 8, 4, 2, 1 to capture both local and global features

•Feature size = 2(162 + 82 + 42 + 22 + 1) = 682

• Using Color DataPixel intensity alone requires 12544 dimensional input spaceAdding color => increase by 200%HSV space mean & varianceat image borders (Wang & Zhang 2003)

• Current WorkApply new feature space to experiments reviewedExplore color feature representationsTrain general system across all classesOptimize system to work with camera phone outputDevelop web site / application for digital photos

Add external modules (e.g. face detection, scene classification) to improve performance

• Agenda

Project OverviewSVM BackgroundParameter SelectionFirst ExperimentFeature Selection MethodsFuture Work / Wrap Up / Questions

Statistical Learning for Image OrientationAgendaProject OverviewImage OrientationSystem Block DiagramClassifier Block DiagramAgendaTraining = Learn Decision BoundaryClassificationExtension to Multi-Class ProblemsMulti-Class: 1-vs-All ClassifierLinear Support Vector Machine for separable dataLinear SVM, separable data contd.Linear SVM for unseparable dataNon-linear SVMsAgendaSVM ParametersParameter SelectionAgendaSample Problem: Car OrientationFeature SpaceData SetCar Orientation ResultsInitial ResultsAgendaExpansion of Training SetFeature SpaceResultsNew Feature SpaceGradient MatrixUsing Color DataCurrent WorkAgenda