advances in computer vision / multimedia processing

Advances in Computer Vision

Advances in Computer Vision / Multimedia Processing

Petr Pulc1, Martin Holena1

1Institute of Computer Science,Czech Academy of Sciences

Prague, Czech Republic

December 20, 2018


Outline

1 Primary Methods of Computer VisionBased Directly on Pixel DataTransfer to Time SeriesBased on Time Series

2 Joint Detection, Tracking and DescriptionOn Limited ScopeWith Shot-based MetalearningWith Metalearning on Areas of Interest

3 Discussion


Primary Methods of Computer Vision

Outline



3 Discussion



Based Directly on Pixel Data

Outline

Based directly on pixel colour data (not necessarily RGB)

Individual frames as independent pictures

Very often using (D)*(C)NN




Choose Your (Deep) Weapon!YOLO

448

448

3

7

7

Conv. Layer7x7x64-s-2

Maxpool Layer2x2-s-2

3

3

112

112

192

3

3

56

56

256

Conn. Layer

4096

Conn. LayerConv. Layer3x3x192


Conv. Layers1x1x1283x3x2561x1x2563x3x512


3

3

28

28

512

Conv. Layers1x1x2563x3x5121x1x512

3x3x1024Maxpool Layer

2x2-s-2

3

3

14

14

1024

Conv. Layers1x1x512

3x3x10243x3x1024

3x3x1024-s-2

3

3

7

7

1024

7

7

1024

7

7

30

}×4 }×2

Conv. Layers3x3x10243x3x1024

Excels in object detection, good in object description.




Choose Your (Deep) Weapon!YOLO

Excels in object detection, good in object description.




Choose Your (Deep) Weapon!VGG16

Good in object detection, commonly used for face description.




Choose Your (Deep) Weapon!ResNet

Much deeper, comparable model size, used for face description.




However...

(a) Person (low) (b) Sports ball (low) (c) Untargeted (low)

(d) Person (high) (e) Sports ball (high) (f) Untargeted (high)

Chen, S. T., Cornelius, C., Martin, J., & Chau, D. H. P.: Physical Adversarial Attackon Object Detectors.




However...

Dist. Angle Target: person Target: spor ts ball Untargeted

10’ 0◦

10’ 30◦

Chen, S. T., Cornelius, C., Martin, J., & Chau, D. H. P.: Physical Adversarial Attackon Object Detectors.




Use of Networks on Statistical Data

Data(InputDim)

Linear Layer(InputDim x 100)

SigmoidLinear Layer(100 x 50)

Sigmoid

Linear Layer(50 x outputDim)

SoftmaxNegative log

likelihood loss

Linear SVM Trainer

SVM learning

Back propagation

Prediction

Legend:

Linear SVM

Prediciton

Šabata, T., Pulc, P., & Holena, M.: Semi-supervised and Active Learning in VideoScene Classification from Statistical Features. In Workshop on Interactive

Adaptive Learning.




Use of Networks on Statistical Data

(a) Room 4A (b) Room 4B

0 200 400 600 800 1000 1200

Histogram bin

0.00

0.05

0.10

0.15

0.20

Rela

tive p

ixel count

Histogram comparison

4A

4B

(c) Histogram comparison

Šabata, T., Pulc, P., & Holena, M.: Semi-supervised and Active Learning in VideoScene Classification from Statistical Features. In Workshop on Interactive

Adaptive Learning.



Transfer to Time Series

Detect, Align, Describe, Match

Marcetic, D., & Ribaric, S.: An Online Multi-Face Tracker for Unconstrained Videos.



Based on Time Series

Optical Flow

Lamba, S. & Nain N.: Oriented Tracklets Approach for Anomalous SceneDetection in High Density Crowd




Optical Flow with Camera Motion Compensation

Beumier, C. & Neyt X.: Trajectories and Camera Motion Compensation in AerialVideos





A

A'

a

b

cd

e

ΔA

ΔA'

Δa

Information on motionfrom upper octave(∆A,∆A′)





A

A'

a

b

cd

e

ΔA

ΔA'

ΔaΔa − ΔA


Feature motion relativeto upper octave(∆a −∆A)





A

A'

a

b

cd

e

ΔA

ΔA'

ΔaΔa − ΔA

Δa'


Feature motion relativeto upper octave(∆a −∆A)

Estimation of newposition of the feature(∆a′)


Joint Detection, Tracking and Description

Outline



3 Discussion



On Limited Scope

Face Detect, Track, Describe

Tapu, R., Mocanu, B. & Zaharia T.: Face recognition in video streams for mobileassistive devices dedicated to visually impaired.



With Shot-based Metalearning

Based on Type of ContentTraining

1 Split content to individual shots2 Based on statistical properties, pick set of processing methods3 Store the best-performing methods to database4 Train a method selection model based on statistical properties



With Shot-based Metalearning

Based on Type of ContentTesting

1 Split content to individual shots2 Based on statistical properties, pick set of processing methods3 Run k best methods simultaneously4 Provide result with highest support



With Metalearning on Areas of Interest

Based on Domains of InterestTraining

1 Split content to individual shots and “layers”

2 Based on statistical properties, pick set of processing methods3 Store the best-performing methods to database4 Train a method selection model based on statistical properties



With Metalearning on Areas of Interest

Based on Domains of InterestLayer Splitting


Discussion

Outline



3 Discussion


Discussion

Thank you

[email protected]

advances in computer vision / multimedia processing

Documents