Brand recognition in real-life photos

Łukasz Czarnecki

• Machine Learning Engineer /Data Scientist (Samsung)

• Human Tech Art founder and organiser

Łukasz Czarnecki

Monthly Actives Likes Daily Average Photos Per Day

300M

Photos shared

30B+ 2.5B 70M

Instagram

Twitter

Facebook

Instagram

0.03%0.07%4.21%

Followers engagement on brand

Source: Forrester Research, Inc.

Feed noise

What’s seen by Instagram?

Brand recognition in real-life photos

Social media engagement

Deep Learning

Deep Learning

Needs big amount of examples

Great results in Image

Recognition

Hard to trainLearns

features

VS VS

heavy data augmentation

&

train your network

Use pretrained network

&

SVM

Use pretrained network

&

adjust last layers

Possible approaches

Sliding window

• 16 windows: width/3

• 9 windows: width/2

• 1 window: width

NUMERIC GALLERY SAMPLE

406 420351

340 255301

Initial dataset

Adidas

Nike

Starbucks Heineken

COSTA COFFEE Carlsberg

Random class

Network propagation time

Testing different networks

VGG_S Network

Optimizing the code

Switched for batch processing

Changing framework

Caffe

~1 sec

Initial time: 2 sec

~ 200 ms ~20 ms

Network output

224 x 224 x 3

4096 x 1Network

How do you use the network?

How do you use that with SVM?

Testing results

Getting features from Network

Training SVM on features

Testing with sliding

window

First results

68%Precission

66%Recall

Adding counterexamples

70% Precission

65%Recall

+2%

-1%

Using different layer

73%Precission

67%Recall

+3%

+1%

Adjusting probability cutoff

76%Precission

65%Recall

92%

+3%

-2%

Getting more data

82%Precission

68%Recall

12

34

5

Training with initial dataset

Getting more unlabeled data

Collecting windows with detections

Manual segregation

Training with bigger dataset

+500 per class

+6%

+3%

Data Augmentation

84%Precission

70%Recall

+2%

+2%

Adding normalisation

88%Precission

67%Recall

Getting features from Network

Training SVM on features

Testing with sliding

window

Normalisation

+4%

-3%

Examples of filtering

Final results

Adidas

Nike

Starbucks

COSTA COFFEE

Heineken

40%

47%

84%

80%

88%

80%

76%

93%

90%

97%

Precission Recall

Carlsberg 66%93%

88%Precission

67%Recall

Tough examples

Thank you