presentation at ipg apply & innovate: automatic generation ......automatic generation of...

FZ

I FO

RS

CH

UN

GS

ZE

NT

RU

M

INF

OR

MA

TIK

Automatic generation of realistic simulation

scenarios out of video recordings

Jochen Kramer, Jennifer Friedmann, Dr.-Ing. Martin Hillenbrand,

Prof. Dr.-Ing. Klaus D. Müller-Glaser

apply & innovate 2014 – IPG Technology Conference

23./24. September 2014 This project is supported by

Division ESS at FZI

Embedded Systems and Sensors Engineering

• Prof. Dr.-Ing. Klaus Müller-Glaser

• Prof. Dr.-Ing. Dr. h. c. Jürgen Becker

• Prof. Dr. rer. nat. Wilhelm Stork Prof. Dr.-Ing. Michael Hübner

Directors

• 25 research associates in Karlsruhe

• Ca. 65 students as Hiwis and for theses

• Transaction volume: 2,2 Mio €

Personnel

© FZI Research Center for Information Technology | Dipl.-Ing. Jochen Kramer 2 08.09.2014

Division ESS at FZI

Embedded Systems and Sensors Engineering

Analysis, development and testing of innovative technologies, design

methods and applications for embedded systems

Model-driven design methods and tools

technology-oriented research

application-oriented research and development projects

3 08.09.2014 © FZI Research Center for Information Technology | Dipl.-Ing. Jochen Kramer

Motivation – new Testing Approaches

08.09.2014 4 © FZI Research Center for Information Technology | Dipl.-Ing. Jochen Kramer

System

Specification

& Design

Component

Specification

& Design

Implementation

Component

Integration

System

Integration

Test

Test

Vehicle

Environment

Realistic

Scenarios

against

against

How should the blue car react for a global efficient solution?

Project: EnopTraFlow

Energy optimized Traffic Flow


Image 1: Research Example in Project EnopTraFlow simulated in IPG CarMaker

Project: EnopTraFlow

Energy optimized Traffic Flow


Image 2: EnopTraFlow Project Logo

?

From real Scenario to Simulation


Recording with camera

Use of image recognition algorithms

Real scenario Simulated scenario

DashCam incl. GPS

Transformation Process Overview


D

Pre-processing

Inverse

Projective

Geometry

Vehicle

Detection

Lane

Detection

Identified Objects and Properties (e.g. reference lane, relative distance, relative speed, relative lane)

Simulation of Scenario

Transformation Process in Detail

Creation of Video Recordings

08.09.2014 9

D

Pre-processing

Inverse

Projective

Geometry

Vehicle

Detection

Lane

Detection

Identified Objects and Properties (e.g. reference lane, relative distance, relative speed, relative lane)

Simulation of Scenario

© FZI Research Center for Information Technology | Dipl.-Ing. Jochen Kramer

Creation of Video Recordings

with e.g. DashCam incl. GPS


Pre-Processing

Checking video conformity

Rectification of lens deformation

Manual selection of reference points for transformation into bird’s

eye view



Object Recognition

Transformation of recognized objects frame by frame

Creation of object lists for each frame

Filling object lists with recognized object properties

08.09.2014 11

Frame0

Ego

Obj0

Obj1

...

… FrameN

Ego

Obj0

Obj1

…




Distance and Speed Calculation

Usage of bird’s eye view

Bounding box for distance calculation between two video frames

Frame rate for time difference

08.09.2014 12

Mapping

Frame t-1 Frame t

Bottom line of

Bounding Box


Post-Processing


Results: Real Scenario (recorded)

08.09.2014

Video 1: Recorded scenario with object recognition

14 © FZI Research Center for Information Technology | Dipl.-Ing. Jochen Kramer

Results: Simulated Scenario


Video 2: Simulated scenario in IPG CarMaker

08.09.2014 16

Validation of Speed Calculation

Test setup


Setup of one driving lane with cones

Lane width: 3m

Cone difference: 5m

Driving speed: different fixed speeds ( 5 km/h, 10 km/h, … )

Video 3: Validation of speed calculation on test track

08.09.2014 17

Moving average low pass filter


Data Evaluation


08.09.2014 18

Deviation every of distance values



Data Evaluation


08.09.2014 19

Median filter on speed values




Data Evaluation


08.09.2014 20

Median filter on speed values




Data Evaluation


Summary and Facts

08.09.2014 21

Real scenario Simulated scenario

Used Dash Cam

• Resolution:

1920x1080 px

• Frames per Second:

30 fps

• Lens angle: 120°

• Focal width:

3,2 mm

Used Tools

• Visual C++

• OpenCV Library

• IPG CarMaker

Performance

• Processing time:

ca. 10 Seconds

per Frame


Actual working on

Front and Back Camera

08.09.2014

Synchronization of two camera

streams

Usage of front and back camera

to extend virtual environment

22 © FZI Research Center for Information Technology | Dipl.-Ing. Jochen Kramer

View of front camera

View of back camera

Research Project LogDaSim

08.09.2014 23

Driving Scenario

Ontology

Domain Model

Instance Model

Reactive Simulation

Environment

Tests

Adapted Log Data

Scenario Filter

Integration

Parameterized

Scenarios

Derivation use

Functional

Specification Categorization Prototypes

Data Logger

Driver Assistant System


Thank you for your attention!

This project is supported by

presentation at ipg apply & innovate: automatic generation ......automatic generation of...

Documents