neue verfahren der bildverarbeitung – auch zur erfassung ... · pdf filesince 1980s...

© Fraunhofer HHI | 13.07.2017 | 1

Peter Eisert

Neue Verfahren der Bildverarbeitung – auch zur Erfassung von Schäden in Abwasserkanälen?

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 2

Fraunhofer is Europe‘s largest organization for applied research.

69 institutes and research units in Germany

Research cooperation around the world

24 500 staff members

Budget: more than 2.1 billion €

Research highlights: MP3 audio codec, H.264, H.265 video codec, LTE mobile communications standard

Fraunhofer-Gesellschaft

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 3

Fraunhofer Institute for Telecommunications,Heinrich Hertz Institute, HHI

1928 Founding of Heinrich Hertz Institute in BerlinSince 1960s Research on fiber optical transmissionSince 1980s Research on digital video codingSince 2003 Member of Fraunhofer-GesellschaftSince 2009 Branch site in Goslar, Germany

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 4

Facts and Figures

Last updated: 2015

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 5

Photonic Network and Systems

Photonic Components

Fiber Optical Sensor Systems

Wireless Communications and Networks

Vision and Imaging Technologies

Video Coding and Analytics

Departments

Photo: fotolia.com/SeanPavonePhoto | Edit: Fraunhofer HHI

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 6

Image Processing Pipeline

geometry

motion / deformation

semantic data

inspection

Peter Eisert

scene acquisition rendering & display

Processing &

Analysis

© Fraunhofer HHI | 13.07.2017 | 7

2D Scene Capturing

Projection of 3D scene into 2D image plane

Loss of 1 dimension

Still images or video

Higher and higher resolution

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 8

Image Reprojection

Camera calibration determines parameters of projection

Image can be reprojected onto any other surface

But viewing position cannot be changed

Peter Eisert

perspective camera fisheye camera

© Fraunhofer HHI | 13.07.2017 | 9

Image Reprojection

Peter Eisert

perspective camera fisheye camera

© Fraunhofer HHI | 13.07.2017 | 10

Spectral Sensitivity

Regular camera and humans: RGB colors

Multi-spectral imaging can provide material properties

Information invisible to humans

Peter Eisert

RGB image of tumor tumor @ 500nm

NIR imaging

thermal imaging

© Fraunhofer HHI | 13.07.2017 | 11

3D Scene Capturing

3D geometry can be estimated from multiple views (photogrammetry)

Ill-posed problem

Requires determination of corresponding points between views

Epipolar geometry constrains search space

Peter Eisert

xl xr

disparity d=xl-xr

z~

triangulation

© Fraunhofer HHI | 13.07.2017 | 12

Depth Estimation from Stereo

Images captured with high resolution (10-20 MP)

Dense reconstruction from 2 or more views

Arbitrary objects

Reconstruction of entire head or persons (incl. hair, eyes, etc.)

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 13

High Detail 3D Reconstruction

Peter Eisert

[Blumenthal, Computers & Graphics 2014]

© Fraunhofer HHI | 13.07.2017 | 14

Textured 3D Model from 2 Views

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 15

3D Models in Security Applications

Peter Eisert

augmented reality visualization security gates

© Fraunhofer HHI | 13.07.2017 | 16

High Quality 3D Reconstruction

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 17

Aerial Surveillance, Object Detection

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 18

Analysis of Vegetation and Landscapes

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 19

Inspection of Laser Welds

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 20

3D Recording of Footprints

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 21

Support of Large Range of Object Sizes

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 22

Tracking and Motion Estimation

Object / camera motion can be determined from point correspondences

3D motion requires additional priors (e.g. static scene, deformation model)

Dense / sparse motion models

Peter Eisert

camera motion object motion

© Fraunhofer HHI | 13.07.2017 | 23

3D Motion Tracking of Robot

3D tracking of motion path

Corresponding points with epipolar constraint

Assumption of near cylindrical pipe shape

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 24

Enhanced Surface Textures

Projection onto 3D cylinder surface

Correction of lighting artifacts

Compared to simple unwrap

higher resolution

less stitching artifacts

Peter Eisert

Panoramounwrapping with 3D motion

© Fraunhofer HHI | 13.07.2017 | 25

Deformable Surface Tracking

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 26

Deformable Surface Tracking

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 27

3D Surface Tracking for Navigated Surgery

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 28

Image Analysis with Machine Learning

Deep neural networks highly successful for many image analysis tasks

image classification (e.g. detection of defects in pipes)

scene labeling / segmentation

image synthesis

…

Peter Eisert

[ImageNet]

© Fraunhofer HHI | 13.07.2017 | 29

Image Classification

Peter Eisert

(labeled) Training data Classifier

“Fish“

Trai

nin

gTe

stin

g

© Fraunhofer HHI | 13.07.2017 | 30

Convolutional Neural Network

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 31

Explanation of Black Box

Black box classifier

Explanation of deep neural network decisions

Visualization via relevance feedback

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 32

Detection of Morphing Attacks

Detection of morphing attacks in ABC Gates

Anomaly detection in passport images

Sematic image forensics

Machine learning

Peter Eisert

Original Morph

© Fraunhofer HHI | 13.07.2017 | 33

Creation of Face Morphs

Morphing pipeline with exchangeable components

Feature detection

Warping

Blending

Automatic alignment of face images based of feature points/feature lines

Automatic adaptive blending that prevents visible cuts

Peter Eisert

Ori

gin

al im

age

sou

rce:

pic

s.st

ir.ac

.uk

Fiel

d w

arp

ing

Piec

ewis

e af

fin

e

Alp

ha

ble

nd

ing

(α=

.5)

Ou

r ad

apti

ve

© Fraunhofer HHI | 13.07.2017 | 34

Detection of Face Morphs

Morphing attack detection using Deep Neural Networks

Creation of training and test datasets>1500 original images

Image pre-processing (filtering, noise) to increase variations in dataset

Comparison of different network architectures

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 35

VR / AR for Interaction and Assistance

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 36

Dynamic 3D Scene Capturing

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 37

TimeLab

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 38

Omnidirectional Video

High resolution 360° video

No stitching artifacts

2D and 3D

Real-time end to end streaming

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 39

Sample Productions

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 40

Summary

Image processing and analysis is a powerful tool for many applications

Significant progress in imaging hardware

High-resolution, ubiquitous and cheap cameras

VR / AR glasses

Increased computing power (mobile devices, graphics boards)

Machine learning enables novel AI technologies

New applications in water resource management ?

Peter Eisert

© Fraunhofer HHI | 13.07.2017 | 41

Fraunhofer-Institut für Nachrichtentechnik,Heinrich-Hertz-Institut, HHI

WE PUT SCIENCEINTO ACTION.

Contact:

Peter Eisertpeter.eisert@hhi.fraunhofer.de+49 30 31002 614

Einsteinufer 3710587 Berlin

Peter Eisert