infrastructure side data fusion tobias schendzielorz (tum) paul mathias (siemens)

1SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich

Infrastructure Side Data FusionInfrastructure Side Data Fusion

Tobias Schendzielorz (TUM)

Paul Mathias (Siemens)

SAFESPOTSAFESPOTSAFESPOTSAFESPOT


Munich University of TechnologyChair of Traffic Engineering and Control

Univ.-Prof. Dr.-Ing. Fritz Busch

Lecturing and Research

in Public and Private Transport

Lecturing and Research

in Public and Private Transport

Munich University of TechnologyChair of Traffic Engineering and ControlArcisstr. 21D-80333 MünchenEmail: [email protected]

www.vt.bv.tum.de


Munich University of Technology

Faculty of Civil Engineering and Geodesy

Institute of TransportationChairmen: Univ.-Prof. Dr.-Ing. F. BuschUniv.-Prof. Dr.-Ing. G. Leykauf

Institute of TransportationChairmen: Univ.-Prof. Dr.-Ing. F. BuschUniv.-Prof. Dr.-Ing. G. Leykauf

• Chair of Traffic Engineering and Control• Department of Urban Planning and Development• Institute of Road, Railway and Airfield Construction

Organization Chart


TUM-VT – Staff

Civil Engineering

Computer Sciences

MathematicsGeography

Transportation EngineeringEconomic Sciences

Disciplines

Civil Engineering

Computer Sciences

MathematicsGeography

Transportation EngineeringEconomic Sciences

Disciplines


Dinner on Monday Evening


Meeting Agenda Monday

Monday, 03rd March 2008

Time Topic Presenter

10:30 Welcome Coffee Tobias - TUM

10.45 Update on latest IP issues Angela - MIZAR

11.00Presentation of current status :Outcome of Task Force MeetingExplanation on Data Format & Messages doc

Angela - MIZAR

11:45Presentations of sensor and fusion partners and discussion (max 10min each)

concerned partners

13:00 Lunch

14:00 LDM Issues (LDM Versions / API) LDM Group Member

PARALLEL WORK GROUPS

14:30 Group 1: Sensing systems concerned partners

Group 2: Data fusion and Map providers concerned partners

17:00 Wrap up of the outcome of the WORK GROUPS

17:30 Closure for day

19.30 Dinner together in Munich Organised by TUM


Meeting Agenda Tuesday

Tuesday, 04th March 2008

Time Topic Presenter

9:00 Wrap up of the yesterdays session Tobias - TUM

9:15 Continue Discussion on Infrastructure Platform: Software Framework / Physical Architecture / Test Site Issues

ALL

11.00 Demonstrations for next Annual Review Angela - MIZAR

12.00 Planning of next deliverable Angela - MIZAR

13:00 Lunch

14.00 Launch of WP5 activities (Evaluation and Testing) LCPC

15:00 New action list and wrap the meeting Angela - MIZAR

15:30 Meeting closed


SP2 - Data Fusion

Data Receiver

LDM

Q- / T-API

20.02.2008

SP5 Application

Message Generator

VANET Messages Q- / T-APIQ-API

CCTV for Positioning (Preprocessing Unit)

Gateway to Safety Centre

Gateway to Traffic Control Centre

Thermal Camera f. Living Objects (Preprocessing Unit)

NIR Camera for Ice Detection (Sensor and Preprocessing Unit)

RFID for Ghost Driver Detection (Preprocessing Unit)

Wireless Sensor Network (Preprocessing Unit)

Gatetway to Traffic Light Controller

Message Router (VANET)

Laserscanner (Infrastructure Sensor)

Manoeuvre Estimator

Traffic Information Consolidator

Traffic Data Calculator

ECAID

Situation Refinement

EnvironmentalEvent

Recognition

EnvironmentalConsolidator

Dynamic Black Spot Recognition

Object Refinement

Sensor Level Fusion

Central Level Fusion

Object Matcher

Map Matcher

Cooperative Pre-Data Fusion

IBEO Specific

CCTV for Visibility(Preprocessing unit)

SP2 RSU System: Functional Architecture


SP2 - Data Fusion

Data Receiver

LDM

Q- / T-API

SP5 Application

Message Generator












Manoeuvre Estimator



ECAID


EnvironmentalEvent

Recognition



Object Refinement

Sensor Level Fusion


Object Matcher

Map Matcher


IBEO Specific


SIE

PTV

CIDAUT

VTT

BME

IBEO

Test Sites???

MIZAR

LCPC

SP5

PTV

VTT

TUM

CSST

SODIT

TUM

LCPC

PTV

TUM

TUM

TUM/MIZAR

under discussion

20.02.2008

SP2 RSU System: Functional Architecture

SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich

SP2 - Data Fusion

Data Receiver

LDM

Q- / T-API

20.02.2008

SP5 Application

Message Generator












Manoeuvre Estimator



ECAID


EnvironmentalEvent

Recognition



Object Refinement

Sensor Level Fusion


Object Matcher

Map Matcher


IBEO Specific

SAFESPOT RSU System - Motorway



SP2 - Data Fusion

Data Receiver

LDM

Q- / T-API

20.02.2008

SP5 Application

Message Generator












Manoeuvre Estimator



ECAID


EnvironmentalEvent

Recognition



Object Refinement

Sensor Level Fusion


Object Matcher

Map Matcher


IBEO Specific

SAFESPOT RSU System - Urban



SP2 - Data Fusion

Data Receiver

LDM

Q- / T-API

20.02.2008

SP5 Application

Message Generator












Manoeuvre Estimator



ECAID


EnvironmentalEvent

Recognition



Object Refinement

Sensor Level Fusion


Object Matcher

Map Matcher


IBEO Specific

SAFESPOT RSU System – Inter-Urban / Rural System



SP2 - Data Fusion

Data Receiver

LDM

Q- / T-API

20.02.2008

SP5 Application

Message Generator












Manoeuvre Estimator



ECAID


EnvironmentalEvent

Recognition



Object Refinement

Sensor Level Fusion


Object Matcher

Map Matcher


IBEO Specific

SAFESPOT RSU System – Core System



SP2 - Data Fusion

Data Receiver

LDM

Q- / T-API

20.02.2008

SP5 Application

Message Generator



Gateway to Motorway management system









Manoeuvre Estimator



ECAID


EnvironmentalEvent

Recognition



Object Refinement

Sensor Level Fusion


Object Matcher

Map Matcher


IBEO Specific

SAFESPOT RSU System - Motorway A16 (NL)


under discussion

Detected fog + locationDetected slippery conditions + locationDetected road obstruction + location


SP2 - Data Fusion

Data Receiver

LDM

Q- / T-API

20.02.2008

SP5 Application

Message Generator












Manoeuvre Estimator



ECAID


EnvironmentalEvent

Recognition



Object Refinement

Sensor Level Fusion


Object Matcher

Map Matcher


IBEO Specific

SAFESPOT RSU System - Rural Road N629 (NL)


under discussion

Dynamic speed advice + location


SP2 - Data Fusion

Data Receiver

LDM

Q- / T-API

20.02.2008

SP5 Application

Message Generator












Manoeuvre Estimator



ECAID


EnvironmentalEvent

Recognition



Object Refinement

Sensor Level Fusion


Object Matcher

Map Matcher


IBEO Specific

SAFESPOT RSU System - Urban Helmond


under discussion


SP2 - Data Fusion

Data Receiver

LDM

Q- / T-API

SP5 Application

Message Generator












Manoeuvre Estimator



ECAID


EnvironmentalEvent

Recognition



Object Refinement

Sensor Level Fusion


Object Matcher

Map Matcher


IBEO Specific


1

2

3

4

5

6

7

8

9

10

11

Data Items in “SF_SP7_Data_format&messages.doc”:1. LaserSensorObjects UDP2. MSGVehicleBeacon UDP3. StatusTrafficLight&PedestrianDetector,

AggregatedDetectorData SOAP4. CameraMovingObjectDetection UDP5. CameraLivingObjectDetection UDP6. CameraIceDetection UDP7. RFIDGhostDriverDetection (std.) UDP8. WirelessSensorNetworkObjects (std.) UDP9. SafetyCenterInformation (std.) UDP10. TrafficControlCenterMotorway (std.) UDP11. CameraMeteoConditionDetection (std.) UDP12. InfAppCoordinatorToDataFusion UDP13. RSU internal (not in SF_SP7_Data_format&messages.doc)

13

12

SIE

PTV

CIDAUT

VTT

BME

IBEO

Test Sites???

MIZAR

LCPC

SP5

PTV

VTT

TUM

CSST

SODIT

TUM

LCPC

PTV

TUM

TUM

TUM/MIZAR


SAFESPOT RSU System (Interface Defintions)20.02.2008

????????????UDPInfAppCoordinatorToDataFusion12

no

???

yes (?)

???

no

no

no

yes

no

yes

yes

Map Matching

CameraMeteoConditionDetection

TrafficControlCenterMotorway

SafetyCenterInformation (std.)

WirelessSensorNetworkObjects

RFIDGhostDriverDetection

CameraIceDetection

CameraLivingObjectDetection

CameraMovingObjectDetection

StatusTrafficLight&PedestrianDetector, AggregatedDetectorData

MSGVehicleBeacon

LaserSensorObjects

Data Item

Detection area is stored in LDM.

noevent

(? 1Hz)VariableUDP6

There will be no link between a certain vehicle in the LDM and the information “on this link is a ghost driver”.

noeventVariableUDP7

?????????UDP8

Reference to links/lane is stored in LDM.

no1 HzfixSOAP3

yes1 Hz (?)VariableUDP4


noevent

(? 2Hz)VariableUDP5


noevent

(? 2Hz)variableUDP11

no??????UDP10

noevent (?)VariableUDP9

yes2 HzvariableUDP2

Already matched with vehicle beacon data.

no12,5 HzvariableUDP1

RemarksObject Matching

Freq.# DataObjectsProt.No


Process Timing: Data Fusion / Object Refinement20.02.2008

0 ms 500 ms250 ms 750 ms 0 ms 500 ms250 ms

sensor 1 (1 Hz)

sensor 2 (2 Hz)

sensor 3 (6 Hz)

sensor 4 (event)

VANET (event)

• The OR-ME-block is performed twice a second.• The sensors 1-4 above are assumed to be trajectory related and provide absolute positions of objects: e.g. cameras, laser scanner, etc.• The timing of the transmission of the sensors results should be like this that periodic events lays as close as possible to the left border

of OR, i.e. in one or both of the intervals [0,250] and [500,750].• The VANET events can hardly be influenced concerning the timing of transmission. They are all used no matter when they arrive to

perform the OR. Worst case for the age of a data item at the moment of processing is 500 ms. • Situation refinement or application, that uses in some way positions of objects and/or trajectories, should start either closely after 0

msec or closely after 500 msec.

LD

M

OR ME

LD

M

LD

M

OR ME

LD

M

LD

M

OR ME

LD

M

OR = Object Refinement, ME = Manoeuvre Estimator

Any Situation Refinement Any Situation Refinement Any Situation Refinement

Any Application

described in next slide


R R

R

t1

t1+b

t1

t1-a

Data Fusion: OR Logical Description 20.02.2008

Source 1

Input at t1-a

Source 3Input at t1+b

Source 2Input at t1

Time Alignment &Tracking

Object MatchingData Receipt Map Matching Writing LDM

offset time : aoffset time : blogical reference time : t1

t1

• The alignment computes / calculates the position of an object (could be any other state / attribute of the object as well) to one specific point in time. In the example above are 3 input sources which provide data on the absolute position of objects (WGS 84). These data is sensed at different points in time (t1-a; t1; t1+b). Note: The result can be transmitted at the same point in time (technical point in time vs. logical point in time).

• The alignment “shifts” (either forward or backward) all positions to the same point in time t1. (See column 2 in the picture). This point in time should be the value of the time stamp the object position has in the LDM as well. In order to do this “shifting” the alignment function has to know more about the states of the object (e.g. speed, acceleration, position at the point in time before t 1). I.e. a tracking of the objects is necessary for doing this. And therefore a internal OR memory stack.

• After the alignment the matching of the position of objects provided by the different sources (column 3) is done. • A requirement for this is that all components have synchronized computer clocks. (NTP Server?!?)


Data Sources - TODO

1. define UDP data structures,

2. define static description of the sensor (LDM),

3. generate sample data + develop mockup,

4. consideration of NTP (ethernet connection),

5. data transmission synchronisation for trajectory related data ([0,250] and [500,750]),

6. ensure incorporation of accuracy values for the raw data,

7. ensure UDP binary transmission,

8. ensure that UDP time stamp must be the time point of the event (!!)

9. similar data might use the same data structure (classification)

infrastructure side data fusion tobias schendzielorz (tum) paul mathias (siemens)

Documents