sf d5.5.2 test and validation results annex v1.4 · 2010. 10. 26. · sf_d5.5.2_test and validation...

Deliverable N. D5.5.2 ANNEX Dissemination Level (PU) Copyright SAFESPOT

Contract N. IST-4-026963-IP

SF_D5.5.2_Test and validation results ANNEX v1.4.doc of 113 COSSIB

SAFESPOT INTEGRATED PROJECT - IST-4-026963-IP

DELIVERABLE ANNEX

SP5 – CoSSIB – Cooperative Safety Systems

Infrastructure Based

Deliverable No. (use the number indicated on technical annex)

D5.5.2 - ANNEX

SubProject No. SP5 SubProject Title CoSSIB

Workpackage No. WP5.5 Workpackage Title Test and validation

Task No. T5.5.2 Task Title Test and validation results

Authors (per company, if more than one company provide it together)

P. J. Feenstra TNO (coordinator); D. Vilain, Cofiroute; N. Etienne, Sodit; S. Glaser, LCPC; A. Possani and D. Anguita, DIBE; T. Schendzielorz, TUM; F. Visintainer, CRF; C. Torres, MIZAR

Status (F: final; D: draft; RD: revised draft): F

Version No: 1.4

File Name: SF_D5.5.2_Test and validation results ANNEX v1.4.doc

Planned Date of submission according to TA: May 2009

Issue Date: June 2010

Project start date and duration 01 February 2006, 48 Months

Test and validation results - ANNEX




Revision Log Version Date Reason Name and Company

V0.5 17/05/10 Splitting of the deliverable in a main report and this annex

P.J. Feenstra, TNO

V0.5b 21/05/10 Textual updates P.J. Feenstra, TNO

V0.6 23/06/10 Including peer review P.J. Feenstra, TNO

V1.0 25/06/10 Final version P.J. Feenstra, TNO

V1.1 26/06/10 Peer reading and approval for submission to EC

G. Vivo, CRF P.J. Feenstra, TNO

V1.2 30/06/10 Further final improvements P.J. Feenstra, TNO A. Spence, MIZAR

V1.3 06/07/10 Minor corrections F. Visintainer, CRF

V1.4 15/09/10 Reply to reviewers’ comments

A.R.A. van der Horst, TNO




Abbreviation List AEV Assistance and Emergency Vehicles AMR Anisotropic Magneto-Resistive API Application Programming Interface AV Assistance Vehicle BSCW Basic Support for Cooperative Work CoSSIB Cooperative Safety Systems Infrastructure Based CRF Centro Ricerche FIAT DFL Data Fusion Logic DFS Data Fusion Sensor Simulation ECAID Enhanced Cooperative Automatic Incident Detection ESPOSYTOR SAFESPOT SYSTEM MONITOR EV Emergency Vehicle GD Ghost Driver GPS Global Positioning System H&IW Hazard and Incident Warning HMI Human Machine Interface I2V Infrastructure-to-Vehicle ICT Information Communication Technology IRIS Intelligent Cooperative Intersection Safety - System LDM Local Dynamic Map LDP LDM Data Player LED Light Emitting Diode LIVIC Laboratoire sur les Interactions Véhicules-Infrastructure-Conducteurs MARS Multi-Agent Real-Time Simulator PC Personal Computer PND Portable Navigation Device PRM Priority Request Manager PTV-AG Planung Transport Verkehr AG Pyro Pyroelectric RDep Road Departure Prevention RFID Radio Frequency Identification RQ Requirement RSU Road Side Unit SCOVA Cooperative systems applications vehicle based SDM Safe Drive Map SiVIC Simulateur Véhicule Infrastructure Capteurs SMA Safety Margin Assistant SMAEV Safety Margin for Assistance and Emergency Vehicles SOAP Simple Object Access Protocol SP Sub-project SpA Speed Alert TBC To Be Confirmed TBD To Be Determined TLC Traffic Light Controller

TNO Nederlandse Organisatie Voor Toegepast Natuurwetenschappelijk Onderzoek

UC Use Case UDP User Datagram Protocol (communication protocol) VANET Vehicle Ad hoc Network VMS Variable Message Sign VTT Valtion Teknillinen Tutkimuskeskus WP Work-package WSN Wireless Sensor Network




Table of contents Revision Log.............................................................................................................................. 2 Abbreviation List ........................................................................................................................ 3 Table of contents ....................................................................................................................... 4 List of Figures ............................................................................................................................ 5 List of Tables ............................................................................................................................. 5 Appendix A - Test forms: Detailed test results per application.................................................. 6 Appendix B - Information about the H&IW test results .......................................................... 108




List of Figures Figure 1 Diagram of the Amsterdam Test Track ..................................................................... 66 Figure 2 Logical architecture of SMAEV application ............................................................... 82 Figure 3 Pictures of the Obstacle (traffic jam & slow vehicle) ............................................... 112 Figure 4 Technical Test Result Amsterdam – wrong way driver (Infrastructure analysis) .... 113

List of Tables Table 1 IRIS – LDM Static Content Test – Simulation .............................................................. 6 Table 2 IRIS – Functionality Test – Simulation ......................................................................... 8 Table 3 IRIS – Functionality Test – Laboratory ....................................................................... 10 Table 4 IRIS – Functionality Test – Test site........................................................................... 12 Table 5 SpA_01 Simulation ..................................................................................................... 17 Table 6 SpA_01 Test site ........................................................................................................ 21 Table 7 SpA_03 Simulation ..................................................................................................... 25 Table 8 SpA_03 Test site ........................................................................................................ 30 Table 9 H&IW_01 and H&IW_02 Simulation.......................................................................... 33 Table 10 H&IW_01 Test site 1................................................................................................. 37 Table 11 H&IW_01 Test site 2................................................................................................. 49 Table 12 H&IW_02 Test Site ................................................................................................... 57 Table 13 H&IW_02 H&IW_03 Test Site .................................................................................. 64 Table 14 Rdep SDM creator Simulation.................................................................................. 68 Table 15 Rdep Runtime Simulation......................................................................................... 71 Table 16 Rdep SDM creator Test site ..................................................................................... 74 Table 17 Rdep Runtime Test site ............................................................................................ 77 Table 18 SMAEV_01 – Functional test of components – Bench (CRF).................................. 81 Table 19 SMAEV_01 Test site (CRF)...................................................................................... 84 Table 20 SMAEV_01 Test site (SODIT)................................................................................ 103 Table 21 SMAEV_02 Test site ............................................................................................. 105 Table 22 Verification Table RFID – Ghost driver two cars in opposite direction................... 108 Table 23 Verification Table RFID – Shadowing 2 Ghost Drivers .......................................... 109 Table 24 Obstacle-Pedestrian on the CRF Test Track – Thermal imaging cameras ........... 110 Table 25 Results Wrong Way Driver – CRF – Correctness .................................................. 111 Table 26 Wrong Way Driver – CRF – Communication Test ................................................. 111




Appendix A - Test forms: Detailed test results per application. Table 1 IRIS – LDM Static Content Test – Simulation

TEST FORM

SP 5 WP 5 Prototype Version - System Configuration -

HW Release - SW Release

- DFL_v1.0.8 (incl. IRIS) - PG LDM using the C++ interface R_1_7_10 and SQLite LDM (LDM data model version 10.0.11)

Compiled by / Company TUM Date July 2009 to March 2010

Form Progressive Numb. 1 Functional Component IRIS Reference Document D5.3.3

Test Type Test Objective Test Purpose Test Environment Unitary Verification Correctness Simulation

Test Case 1: IRIS – LDM Static Content Test – Simulation Test Description

Prerequisite : - Installed LDM software and the static supply for the intersection - Installed DFL incl. IRIS Description: - Starting DFL incl. IRIS basic LDM checks are performed automatically and a report on the terminal is issued Requirements: - SP5_RQ41_19_v1.0: The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least

approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead). - SP5_RQ42_19_v1.0: The LDM shall provide a unique scheme for dynamic traffic information to refer to. Use cases: - SP5_UC22: Safe signalized intersection (crossing, turning) - SP5_UC31: Safe signalized intersection (red light violation)

Expected Values / Results




- The listed requirements are met, specifically for the LDM. Obtained Values / Results

SP5_RQ41_19_v1.0 The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead).

OK

More iterations were needed to prepare the content for all the three test sites (Helmond, Dortmund, Amsterdam)

SP5_RQ42_19_v1.0 The LDM shall provide a unique scheme for dynamic traffic information to refer to OK

Status Test Case 1 OK Link to external annexed documentation (if any) -




Table 2 IRIS – Functionality Test – Simulation

TEST FORM



- DFL_v1.0.8 (incl. IRIS) - PG LDM using the C++ interface R_1_7_10 and SQLite LDM (LDM data model version 10.0.11)

Compiled by / Company TUM Date July 2009 to March 2010

Form Progressive Numb. 1b Functional Component IRIS Reference Document D5.3.3


Test Case 2: IRIS – Functionality Test – Simulation Test Description

Prerequisite : - The LDM contains the detailed static description of the intersection such as allocations of stop lines, the reference tracks, signal groups and the assignment of the signal groups of the

appropriate lanes. In addition, the data concerning moving objects need to be provided by the SP2 data platform. - Furthermore, the intersection needs to be modelled in the simulation environment including all vehicles, cyclists and movements. The traffic is generated randomly. - The simulated data need to be fed into the LDM. Description: - Based on the current situation, the IRIS estimates the future trajectories of the vehicles at the intersection and analysis the emerging situation. If the situation is going to be safety

critical, a warning is generated. This basic function is needed for all use cases of the IRIS Application (see list below) Tools: VISSIM (micro simulation tool)

Requirements: - SP5_RQ01_36_v1.0: The system shall be able to calculate the trajectories of all vehicles approaching and passing critical points e.g. urban intersections. - SP5_RQ03_36_v1.0: The system shall be able to calculate the trajectories of all cyclists approaching and passing the intersection. - SP5_RQ04_36_v1.0: The system shall be able to predict the vehicle’s trajectories. - SP5_RQ08_19_v1.0: The system shall take into account the actual and short-term forecast of the traffic light control status. - SP5_RQ18_19_v1.0: The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously. - SP5_RQ19_36_v1.0: The system shall have nearly the same performance in case there are just two or 80 vehicles to manage. - SP5_RQ79_6_v1.1: The intended route on the intersection shall be known either in some direct manner, or indirectly through the status of vehicles indicator.




Use cases: - SP5_UC22: Safe signalized intersection (crossing, turning) - SP5_UC31: Safe signalized intersection (red light violation)

Expected Values / Results - The listed requirements are met. - The situaiton is interpreted correctly and the right warning is triggerd.

Obtained Values / Results SP5_RQ01_36_v1.0 The system shall be able to calculate the trajectories of

all vehicles approaching and passing critical points e.g. urban intersections.

OK More iterations were needed to prepare the content for all the three test sites (Helmond, Dortmund, Amsterdam)

SP5_RQ03_36_v1.0 The system shall be able to calculate the trajectories of all cyclists approaching and passing the intersection. OK

SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle’s trajectories. OK

SP5_RQ08_19_v1.0 The system shall take into account the actual and short-term forecast of the traffic light control status. not tested

This requirement was not tested hence at the real intersection only the current state of the traffic light was available.

SP5_RQ18_19_v1.0 The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously. not tested

The test scenarios only comprised the minimum number of vehicle which needed to be involved to run the test. This number could be managed without any loss in the performance.

SP5_RQ19_36_v1.0 The system shall have nearly the same performance in case there are just two or 80 vehicles to manage. not tested

SP5_RQ79_6_v1.1 The intended route on the intersection shall be known either in some direct manner, or indirectly through the status of vehicles indicator.

OK The vehicles send the status of the indicator. IRIS can interpret this information.





Table 3 IRIS – Functionality Test – Laboratory

TEST FORM


HW Release

- ebox - RSU antenna - vehicle antenna - HMI display

SW Release

- DFL_v1.0.8 (incl. IRIS) - SP5 Message Manger_v1.0.4 - PG LDM using the C++ interface R_1_7_10 and SQLite LDM (LDM data model version 10.0.11) - SNT router 0.9-985 - Translator (CVIS release 7) - CALM manager (CVIS release 7)

Compiled by / Company TUM Date July 2009 to August 2009

Form Progressive Numb. 1c Functional Component IRIS Reference Document D5.3.3

Test Type Test

Objective Test Purpose Test Environment Integration Verification Correctness Bench

Test Case 3: IRIS – Functionality Test – Laboratory Test Description

Prerequisite : - The LDM contains the detailed static description of the intersection such as allocations of stop lines, the reference tracks, signal groups and the assignment of the signal groups

of the appropriate lanes. In addition, the data concerning moving objects need to be provided by the SP2 data platform. - Furthermore, the intersection needs to be modelled in the simulation environment including all vehicles, cyclists and movements. - Complete RSU and vehicle software components need to be set up at the test bench Description: - First run to test the proper configuration of the HMIMessage by the use of the VANET player - Second run to test the whole data chain based on simulated data by the micro simulator. Tools: - VANETplayer - VISSIM (micro simulation tool)




Requirements: - SP5_RQ06_19_v1.0: The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ40_19_v1.0: Messages that are exchanged between vehicles and RSU have to be unified to insure system interoperability. - SP5_RQ50_36_v1.0: The system shall be able to transmit the warning / recommendation to equipped vehicles. - SP5_RQ86_6_v1.1: It shall be possible to give warnings to drivers. Preferably with a graphical identification of the type and location of potential conflicts. Use cases: - SP5_UC22: Safe signalized intersection (crossing, turning) - SP5_UC31: Safe signalized intersection (red light violation)

Expected Values / Results - The listed requirements are met. - IRIS application is able to transmitt the correct warning to the vehicle system.

Obtained Values / Results SP5_RQ06_19_v1.0 The system shall be able to provide a priority level for

each transmitted message. This is valid for V2I as well as for I2V communication

OK

SP5_RQ07_19_v1.0 The system shall be able to determine the period of validity of the transmitted and received messages OK

SP5_RQ40_19_v1.0 Messages that are exchanged between vehicles and RSU have to be unified to insure system interoperability OK

SP5_RQ50_36_v1.0 The system shall be able to transmit the warning / recommendation to equipped vehicles OK

SP5_RQ86_6_v1.1 It shall be possible to give warnings to drivers. Preferably with a graphical identification of the type and location of potential conflicts.

OK





Table 4 IRIS – Functionality Test – Test site

TEST FORM


HW Release

See Desrciption in D5.6.2

SW Release

- DFL_v1.0.8 (incl. IRIS) - SP5 Message Manger_v1.0.4 - PG LDM using the C++ interface R_1_7_10 and NavteqLDM (LDM data model version 10.0.11) - SNT router 0.9-985 - Translator (CVIS release 7) - CALM manager (CVIS release 7)

Compiled by / Company TUM Date 19-26 August 2009 and 25 February 2010

Form Progressive Numb. 2 Functional Component IRIS Reference Document D5.3.3

Test Type Test Objective Test Purpose Test

Environment Integration Verification Correctness Test site

Test Case 4: IRIS – Functionality Test – Test site Test Description

Prerequisite : - The LDM contains the detailed static description of the intersection such as allocations of stop lines, the reference tracks, signal groups and the assignment of the signal groups of the

appropriate lanes. In addition to that, the data concerning moving objects need to be provided by the SP2 data platform. The connection to the SINTECH router is set up and the routing software works properly. For the comprehensive description of the test site and the test procedure, check D5.6.2.

Description: - It is checked, whether in each test scenario the vehicles and vulnerable road user are present in the LDM. It will be checked, whether the traffic light status is in the LDM. In the case

IRIS generates a warning, it will be checked, whether this warning is transmitted to the router via the SP5 message manager or not.




Red Light Violation: The violator is warned

Red Light Violation: The other road users are warned

RSU TLC Critical Warning Zone

CriticalWarning(unicast)

SAFESPOT‐Vehicle

virtual stop line− Need to be marked at the road− need to changed in LDM

Critical Distance

5 ‐ 20m

Hamburgerstraße

Gerichtsstraße

West ‐ East

Critical W

arning

Zon

eCritical D

istance


SAFESPOT‐Vehicle

virtual stop line− Need to be marked at the road− need to changed in LDM

RSU TLC

SAFESPOT‐Vehicle (but does not react on warning)

CriticalWarning

(broadcast)

CriticalWarning

(broadcast)

SAFESPOT‐Vehicle

5 ‐ 20m

Right Turning: cyclist / pedestrian warning

Left Turning: oncoming vehicle warning

RSU TLC

Critical Distance


SAFESPOT‐Vehicle

Critical Warning Zone

VRU (cyclist)(no warning possible)

RSU TLC


Critical Warning Zone

SAFESPOT‐Vehicle




Requirements: - SP5_RQ06_19_v1.0: The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ21_36_v1.0: The system must be an open system that will be able to host modules from various vendors. - SP5_RQ26_36_v1.0: The system shall receive the exact position of all pedestrians at the intersection from the LDM. - SP5_RQ28_19_v1.0: The short range communication shall be available at the intersection itself and its vicinity (minimum 150 m in each direction). - SP5_RQ33_19_v1.0: All data transmitted from the vehicle to the infrastructure shall have a timestamp referring to the creation time of the data (and not to the transmission time point). - SP5_RQ34_19_v1.0: The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all vehicles in the vicinity of

the RSU. - SP5_RQ40_19_v1.0: Messages that are exchanged between vehicles and RSU have to be unified to insure system interoperability. - SP5_RQ41_19_v1.0: The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least

approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead). - SP5_RQ42_19_v1.0: The LDM shall provide a unique scheme for dynamic traffic information to refer to. - SP5_RQ44_19_v1.0: The system shall receive the position of the vehicles with an accuracy enabling to distinguish between two vehicles. - SP5_RQ45_27_v1.0: In critical points the position of the vehicles shall be determined with a minimal accuracy of +/- 1m. - SP5_RQ46_19_v1.0: The positioning of vehicles have to fulfil the accuracy up to a lane detection extend. - SP5_RQ47_19_v1.0: In case of pedestrians and cyclists the system shall take into account demand signals (push buttons) or data from according road-side sensors. - SP5_RQ49_19_v1.0: The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. - SP5_RQ50_36_v1.0: The system shall be able to transmit the warning / recommendation to equipped vehicles. - SP5_RQ53_19_v1.0: The system shall receive in the vicinity of the urban intersection the position, speed and acceleration and driving direction with a frequency of 2/sec or shorter. - SP5_RQ54_36_v1.0: The system shall receive in the vicinity of the urban intersection extended dynamic vehicle data like position of brake and acceleration pedal, angel of steering

wheel. - SP5_RQ56_19_v1.0: The system shall receive in the vicinity of the urban intersection the indicator state of the vehicles or alternatively / additionally – in case of an activated navigation

system - their turning relations with respect to this intersection. - SP5_RQ61_36_v1.0: The system must be able to receive data from the vehicles as well as the infrastructure. - SP5_RQ79_6_v1.1: The intended route on the intersection shall be known. Either in some direct manner, or indirectly through the status of vehicles indicator. - SP5_RQ80_6_v1.1: Information about emergency vehicles that need to ignore a red light shall be available. - SP5_RQ86_6_v1.1: It shall be possible to give warnings to drivers. Preferably with a graphical identification of the type and location of potential conflicts.

Use cases: - SP5_UC22: Safe signalized intersection (crossing, turning) - SP5_UC31: Safe signalized intersection (red light violation)

Expected Values / Results The requirements are met. Specifically: - successful interaction with the LDM - successful interaction with the SP2 data platform - successful reception of the IRIS message at the message manager and the SINTECH router, respectively




Obtained Values / Results SP5_RQ06_19_v1.0 The system shall be able to provide a priority level for

each transmitted message. This is valid for V2I as well as for I2V communication

OK

SP5_RQ07_19_v1.0 The system shall be able to determine the period of validity of the transmitted and received messages. OK

SP5_RQ21_36_v1.0 The system must be an open system that will be able to host modules from various vendors. OK System was tested with PEEK and Siemens Traffic

Light Controllers. SP5_RQ26_36_v1.0 The system shall receive the exact position of all

pedestrians at the intersection from the LDM. OK The laser scanners have provided positions of the pedestrians being at the crossing, which was part of the scenario at the intersection.

SP5_RQ28_19_v1.0 The short range communication shall be available at the intersection itself and its vicinity (minimum 150 m in each direction).

OK On the main road in Dortmund up to 500m could be achieved.

SP5_RQ33_19_v1.0 All data transmitted from the vehicle to the infrastructure shall have a timestamp referring to the creation time of the data (and not to the transmission time point).

OK

SP5_RQ34_19_v1.0 The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all vehicles in the vicinity of the RSU.

OK In the different scenarios, either unicast or broadcast modes of the communication were used.

SP5_RQ40_19_v1.0 Messages that are exchanged between vehicles and RSU have to be unified to insure system interoperability.

OK

SP5_RQ41_19_v1.0 The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead).

OK

SP5_RQ42_19_v1.0 The LDM shall provide a unique scheme for dynamic traffic information to refer to. OK

SP5_RQ44_19_v1.0 The system shall receive the position of the vehicles with an accuracy enabling to distinguish between two vehicles.

OK

SP5_RQ45_27_v1.0 In critical points the position of the vehicles shall be determined with a minimal accuracy of +/- 1m. OK

This strongly depends on the vehicle’s positioning system. For the test purpose, the positioning was sufficient.




SP5_RQ46_19_v1.0 The positioning of vehicles have to fulfil the accuracy up to a lane detection extend. OK

SP5_RQ47_19_v1.0 In case of pedestrians and cyclists the system shall take into account demand signals (push buttons) or data from according road-side sensors.

not tested This feature was not implemented.

SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. OK

SP5_RQ50_36_v1.0 The system shall be able to transmit the warning / recommendation to equipped vehicles. OK

SP5_RQ53_19_v1.0 The system shall receive in the vicinity of the urban intersection the position, speed and acceleration and driving direction with a frequency of 2/sec or shorter.

OK

SP5_RQ54_36_v1.0 The system shall receive in the vicinity of the urban intersection extended dynamic vehicle data like position of brake and acceleration pedal, angel of steering wheel.

Partly OK

SP5_RQ56_19_v1.0 The system shall receive in the vicinity of the urban intersection the indicator state of the vehicles or alternatively / additionally – in case of an activated navigation system - their turning relations with respect to this intersection.

Partly OK

Only the indicator status was received, but no information on the intended rout based in the navigation system’s advice.

SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure. OK

SP5_RQ79_6_v1.1 The intended route on the intersection shall be known. Either in some direct manner, or indirectly through the status of vehicles indicator.

OK The indicator status was received.

SP5_RQ80_6_v1.1 Information about emergency vehicles that need to ignore a red light shall be available. not tested

SP5_RQ86_6_v1.1 It shall be possible to give warnings to drivers. Preferably with a graphical identification of the type and location of potential conflicts.

OK





Table 5 SpA_01 Simulation

TEST FORM



LDM Schema 25, LDM server 11.9, SP1 Frame-work v10

Compiled by / Company S. Glaser / LCPC Date October 2009

Form Progressive Numb. 3 Functional Component SpA_01 Reference Document D5.3.1


Test Case 1: Speed transition at a straight road Test Description

Prerequisite : - Functional LDM and VANET modules Description: The aim of this test is to prove the basic functionality of SpA_01. It warns the driver on excessive speed, regarding the legal speed limit. The road is a simple straight line with a modification of the speed limit at a given position. - The driver enters an area that is covered by a road-side unit (RSU), the vehicle transmits its beaconing message. - The RSU detects the vehicle, and starts managing it. - If the vehicle speed is higher than the speed limit, a warning is sent from the RSU - The vehicle receives the message and displays the warning to the driver - Vehicles entering the area are chosen randomly, as is their speed Specific test conditions: - The vehicle type is identified using simple classes: passenger car or truck (which may have a specific speed limit) - The nominal road speed ranges from 90km/h; 110km/h and 130 km/h - The modification of the speed in the middle of the road section decrease to 50km/h, 90km/h and 110km/h respectively




S

V

Tools: LDM Data player Internally developed tools for the vehicle Requirements: - RQ02_36_v1.0 (Message Management) The system shall be able to decide if and to whom a message has to be send. - RQ06_19_v1.0 (Priority Level of Message) The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication - RQ12_33_v1.0 (Data exchange) The system shall be able to transmit information towards several supporters (e.g. VMS, radio, PND). - RQ16_36_v1.0 (Query from the LDM) The system shall be able to query needed data from the LDM. - RQ17_36_v1.0 (Receive data from the LDM) The system shall be able to receive and handle the LDM data. - RQ18_19_v1.0 (Simultaneity) The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously. - RQ19_36_v1. (Scalability) The system shall have nearly the same performance in case there are just two or 80 vehicles to manage. - RQ20_19_v1.0 (Time of Warning Generation) The system shall generate and transmit warning information to the drivers timely enough, so that the reaction time left to the drivers is

appropriate. - RQ24_19_v1.0 (Robustness of System) The system has to be robust enough in order to operable under different extreme weather conditions (heat, cold, snow, etc.). - RQ29_27_v1.0 (Range of Communication) The short range communication shall be available at the critical points and their vicinity (minimum 600 m). - RQ31_27_v1.0 (Simultaneous Communication) The system shall be able to communicate with all vehicles in the vicinity of the intersection (minimum radius <= 600 m) simultaneously. - RQ48_36_v1.0 (Environmental Data) The system shall receive data from environmental sensors about weather (rain, snow, fog…). - RQ49_19_v1.0 (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. - RQ50_36_v1.0 (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles. - RQ68_10_v1.0 (Road status) The system shall acquire data on the weather conditions in the 'critical area' (extent TBD) of the obstacle or accident, i.e. wet/dry road surface, rain, fog,

ice, which may affect stopping distances and visibility. - RQ69_10_v1.0 (V Warning) The system shall send appropriate warnings to equipped vehicles within the critical (e and a) zones to permit them to reduce speed and/or change lane

and avoid a (secondary) collision.




User Needs: SP5_UN38: Give to the driver a continuous access to the information of speed limit. SP5_UN39: Infrastructure can warn the driver in case of excessive speed Use cases: SP5_UC32 : Prevention of Driver Excessive Speed

Expected Values / Results The requirements are met, specifically: - The vehicle transmits its beaconing message. - The RSU detects the vehicle - In case of speeding, the vehicle is specifically warned, otherwise, the vehicle on-board system knows the legal speed limit - The legal speed limit in the first area is consistent with the speed limit in the RSU LDM - The time elapsed between the crossing of the new area and the reception of the new speed limit, is not higher than 1s - The time between the start of the over speeding and the reception of the message from the RSU is not higher than 1s - The vehicle type is identified using simple classes

Obtained Values / Results RQ02_36_v1.0 Message Management) The system shall be able to

decide if and to whom a message has to be send. OK

RQ06_19_v1.0 (Priority Level of Message) The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication

OK

RQ12_33_v1.0 (Data exchange) The system shall be able to transmit information towards several supporters (e.g. VMS, radio, PND).

OK Only VMS link was verified

RQ16_36_v1.0 (Query from the LDM) The system shall be able to query needed data from the LDM. OK

RQ17_36_v1.0 (Receive data from the LDM) The system shall be able to receive and handle the LDM data. OK

RQ18_19_v1.0 (Simultaneity) The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously.

OK

RQ19_36_v1. (Scalability) The system shall have nearly the same performance in case there are just two or 80 vehicles to manage.

Partially OK Up to 10 vehicles

RQ20_19_v1.0 to the drivers is appropriate.

(Time of Warning Generation) The system shall generate and transmit warning information to the drivers timely enough, so that the reaction time left

Not tested Verification only aims at testing the application itself not the interface with the driver

RQ24_19_v1.0 (Robustness of System) The system has to be robust Not tested The hardware was not tested under these




enough in order to operable under different extreme weather conditions (heat, cold, snow, etc.).

conditions

RQ29_27_v1.0 (Range of Communication) The short range communication shall be available at the critical points and their vicinity (minimum 600 m).

Not tested

RQ31_27_v1.0 (Simultaneous Communication) The system shall be able to communicate with all vehicles in the vicinity of the intersection (minimum radius <= 600 m) simultaneously.

Not tested

RQ48_36_v1.0 (Environmental Data) The system shall receive data from environmental sensors about weather (rain, snow, fog…).

OK

RQ49_19_v1.0 (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed.

OK

RQ50_36_v1.0 (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles.

OK

RQ68_10_v1.0 (Road status) The system shall acquire data on the weather conditions in the 'critical area' (extent TBD) of the obstacle or accident, i.e. wet/dry road surface, rain, fog, ice, which may affect stopping distances and visibility.

Partially ok

The handled situations were mainly weather related.

RQ69_10_v1.0 (V Warning) The system shall send appropriate warnings to equipped vehicles within the critical (e and a) zones to permit them to reduce speed and/or change lane and avoid a (secondary) collision.

OK

Status Test Case 1 OK Link to external annexed documentation (if any)




Table 6 SpA_01 Test site

TEST FORM


HW Release VANET 0.9.1003 SW Release

LDM Schema 25, LDM server 11.9, SP1 Frame work V10

Compiled by / Company S. Glaser / LCPC Date March 24-26, 2010

Form Progressive Numb. 3b Functional Component SpA_01 Reference Document D5.3.1

Test Type Test Objective Test Purpose Test Environment Integration Validation Performance Test Site

Test Case 1: Speed transition at a straight road Test Description

Prerequisite : - Functional LDM and VANET modules Description: The aim of this test is to prove the basic functionality of SpA_01. It warns the driver on excessive speed, regarding the legal speed limit. The road is a simple straight line with a modification of the speed limit at a given position. - The driver enters an area that is covered by a road-side unit (RSU), the vehicle transmits its beaconing message. - The RSU detects the vehicle, and starts managing it. - If the vehicle speed is higher than the speed limit, a warning is sent from the RSU - The vehicle receives the message and displays the warning to the driver - Vehicles entering the area are chosen randomly, as is their speed Specific test conditions: - The vehicle type is identified using simple classes: passenger car or truck (which may have a specific speed limit) - The nominal road speed range depends on the test site possibility Tools: Recorded data from the VANET




Requirements: - RQ02_36_v1.0 (Message Management) The system shall be able to decide if and to whom a message has to be send. - RQ06_19_v1.0 (Priority Level of Message) The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication - RQ12_33_v1.0 (Data exchange) The system shall be able to transmit information towards several supporters (e.g. VMS, radio, PND). - RQ16_36_v1.0 (Query from the LDM) The system shall be able to query needed data from the LDM. - RQ17_36_v1.0 (Receive data from the LDM) The system shall be able to receive and handle the LDM data. - RQ18_19_v1.0 (Simultaneity) The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously. - RQ19_36_v1. (Scalability) The system shall have nearly the same performance in case there are just two or 80 vehicles to manage. - RQ20_19_v1.0 (Time of Warning Generation) The system shall generate and transmit warning information to the drivers timely enough, so that the reaction time left to the drivers is

appropriate. - RQ24_19_v1.0 (Robustness of System) The system has to be robust enough in order to operable under different extreme weather conditions (heat, cold, snow, etc.). - RQ29_27_v1.0 (Range of Communication) The short range communication shall be available at the critical points and their vicinity (minimum 600 m). - RQ31_27_v1.0 (Simultaneous Communication) The system shall be able to communicate with all vehicles in the vicinity of the intersection (minimum radius <= 600 m) simultaneously. - RQ48_36_v1.0 (Environmental Data) The system shall receive data from environmental sensors about weather (rain, snow, fog…). - RQ49_19_v1.0 (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. - RQ50_36_v1.0 (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles. - RQ68_10_v1.0 (Road status) The system shall acquire data on the weather conditions in the 'critical area' (extent TBD) of the obstacle or accident, i.e. wet/dry road surface, rain, fog,

ice, which may affect stopping distances and visibility. - RQ69_10_v1.0 (V Warning) The system shall send appropriate warnings to equipped vehicles within the critical (e and a) zones to permit them to reduce speed and/or change lane

and avoid a (secondary) collision. User Needs: SP5_UN38: Give to the driver a continuous access to the information of speed limit. SP5_UN39: Infrastructure can warn the driver in case of excessive speed Use cases: SP5_UC32 : Prevention of Driver Excessive Speed

Expected Values / Results The requirements are met, specifically: - The vehicle transmits its beaconing message. - The RSU detects the vehicle - In case of speeding, the vehicle is specifically warned, otherwise, the vehicle on-board system knows the legal speed limit - The legal speed limit in the first area is consistent with the speed limit in the RSU LDM - The time elapsed between the crossing of the new area and the reception of the new speed limit, is not higher than 1s - The time between the start of the over speeding and the reception of the message from the RSU is not higher than 1s - The vehicle type is identified using simple classes




Obtained Values / Results RQ02_36_v1.0 Message Management) The system shall be able to

decide if and to whom a message has to be send. OK

RQ06_19_v1.0 (Priority Level of Message) The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V communication

OK

RQ12_33_v1.0 (Data exchange) The system shall be able to transmit information towards several supporters (e.g. VMS, radio, PND).

OK Only VMS link was evaluated

RQ16_36_v1.0 (Query from the LDM) The system shall be able to query needed data from the LDM. Partially OK

The system uses NavtTeq LDM and subscription mechanism. Closing the subscription makes the LDM crash

RQ17_36_v1.0 (Receive data from the LDM) The system shall be able to receive and handle the LDM data. OK

RQ18_19_v1.0 (Simultaneity) The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously.

OK

RQ19_36_v1. (Scalability) The system shall have nearly the same performance in case there are just two or 80 vehicles to manage.

Not OK Only three vehicles

RQ20_19_v1.0 to the drivers is appropriate.

(Time of Warning Generation) The system shall generate and transmit warning information to the drivers timely enough, so that the reaction time left

Partially OK System has sometimes huge latency (more than 1s)

RQ24_19_v1.0 (Robustness of System) The system has to be robust enough in order to operable under different extreme weather conditions (heat, cold, snow, etc.).

Not tested Hardware was not evaluated under these conditions

RQ29_27_v1.0 (Range of Communication) The short range communication shall be available at the critical points and their vicinity (minimum 600 m).

Not OK Ranges were around 300m, with a clear line of sight

RQ31_27_v1.0 (Simultaneous Communication) The system shall be able to communicate with all vehicles in the vicinity of the intersection (minimum radius <= 600 m) simultaneously.

Not OK Ranges were around 300m, with a clear line of sight


OK

RQ49_19_v1.0 (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed.

OK





OK

RQ68_10_v1.0 (Road status) The system shall acquire data on the weather conditions in the 'critical area' (extent TBD) of the obstacle or accident, i.e. wet/dry road surface, rain, fog, ice, which may affect stopping distances and visibility.

Partially ok

Situation handled were mainly weather situations

RQ69_10_v1.0 (V Warning) The system shall send appropriate warnings to equipped vehicles within the critical (e and a) zones to permit them to reduce speed and/or change lane and avoid a (secondary) collision.

OK





Table 7 SpA_03 Simulation

TEST FORM



LDM Schema 25, LDM server 11.9, SP1 Frame work v10

Compiled by / Company S. Glaser / LCPC Date 01/2010

Form Progressive Numb. 5 Functional Component SpA_03 Reference Document D5.3.1


Test Case 1: Basic Validation of SpA_03 Test Description

Prerequisite : - Functional LDM and VANET modules Description: The aim of this test is to prove the basic functionality of SpA_03: the definition of a speed profile with respect to the road geometry and the road surface condition. The application warns the driver on excessive speed, regarding the computed speed profile. The road presents a simple geometry with a straight line a clothoid and a curve. The radius of the curve can be set at different values. - The driver enters an area covered by a road side unit, the vehicle transmits its beaconing message - The RSU detects the vehicle, and starts managing it. - If the vehicle speed is higher than the speed limit, a warning is sent from the RSU - The vehicle receives the message and displays the warning to the driver




s

ρ

s

V

Requirements: - RQ34_19_v1.0 (Directly address a Vehicle) The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all

vehicles in the vicinity of the RSU. - RQ35_19_v1.0 (Time of Data Delay 2) In the roadside sub-system the delay between generation of warnings and transmission of this data to the vehicles shall be smaller than 50 ms. - RQ36_27_v1.0 (Time of Connection) The time needed to set-up a connection between an incoming vehicle and the roadside communication system should be smaller than 0.8

seconds. - RQ41_19_v1.0 (Static Map contents) The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall

comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead). - RQ46_19_v1.0 (Position of Vehicles 3) The positioning of vehicles have to fulfill the accuracy up to a lane detection extend. - RQ49_19_v1.0 (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. - RQ50_36_v1.0 (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles. User needs: SP5_UN39: Infrastructure can warn the driver in case of excessive speed. Use cases: SP5_UC41 : Prevention of lack of adherence SP5_UC45 : Sudden reduce Visibility

Expected Values / Results The requirements are met. Specifically: - In the case of over speeding, the vehicle is warned. Otherwise, the vehicle on-board system knows the legal speed limit




Obtained Values / Results RQ34_19_v1.0 (Directly address a Vehicle) The roadside

sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all vehicles in the vicinity of the RSU.

OK

RQ35_19_v1.0 (Time of Data Delay 2) In the roadside sub-system the delay between generation of warnings and transmission of this data to the vehicles shall be smaller than 50 ms.

Partially OK Mainly depends on the load of the LDM

RQ36_27_v1.0 (Time of Connection) The time needed to set-up a connection between an incoming vehicle and the roadside communication system should be smaller than 0.8 seconds.

Not tested This RQ was not evaluated because we have no exact description of the intended performance of the VANET: when is a vehicle qualified as incoming, what is exactly the range? However, we have evaluated the average range of communication, taking into account the various antenna provided.

RQ41_19_v1.0 (Static Map contents) The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead).

OK

RQ46_19_v1.0. (Position of Vehicles 3) The positioning of vehicles have to fulfill the accuracy up to a lane detection extend

OK

RQ49_19_v1.0. (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed

OK


OK

Status Test Case1 OK Link to external annexed documentation (if any)




Test Case 2: Road surface condition interaction with SpA Test Description

Prerequisite : - Functional LDM and VANET modules Description: The aim of this test is to verify that the SpA_03 application can handle the environmental variations and that it defines the associated speed. In this test case, the road is the same as defined in test case 1. Various weather conditions are injected during the simulation, as rain (various densities) or fog. Tools: LDM Data player SiVIC simulator and SP2 weather monitoring Requirements: - RQ48_36_v1.0 (Environmental Data) The system shall receive data from environmental sensors about weather (rain, snow, fog…). - RQ60_36_v1.0 (Lack of Friction) The system shall receive information about the lack of friction of a road segment. User Needs: - SP5_UN39: Infrastructure can warn the driver in case of excessive speed. - SP5_UN49: Be warned in case of low adherence road - SP5_UN51: The system transmits warning information towards the driver approaching the risky site that signals a slippery bend or a puddle road. Use cases: SP5_UC41 : Prevention of lack of adherence SP5_UC45 : Sudden reduce Visibility

Expected Values / Results The requirements are met, specifically two results are expected:

• the speed advice is varied in accordance with the weather status • a warning of over speeding is provided, which is a function of the weather condition

Obtained Values / Results





Not tested Data were directly generated with a ‘dummy’ interface

RQ60_36_v1.0. (Lack of Friction) The system shall receive information about the lack of friction of a road segment

Not tested Road friction was estimated by rain density





Table 8 SpA_03 Test site

TEST FORM


HW Release VANET 0.9.1003 SW Release

LDM Schema 25, LDM serveur 11.9, SP1 Framework V10

Compiled by / Company S. Glaser / LCPC Date 04/2010

Form Progressive Numb. 5b Functional Component SpA_03 Reference Document D5.3.1

Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Test Site

Test Case 1: Basic Validation of SpA_03 Test Description

Prerequisite : - Functional LDM and VANET modules Description: The aim of this test is to prove the basic functionality of SpA_03: the definition of a speed profile with respect to the road geometry and the road surface condition. The application warns the driver on excessive speed, regarding the computed speed profile. The road presents a simple geometry with a straight line a clothoid and a curve. The radius of the curve is set accordingly to data stored in the LDM - The driver enters an area covered by a road side unit, the vehicle transmits its beaconing message - The RSU detects the vehicle, and starts managing it. - If the vehicle speed is higher than the speed limit, a warning is sent from the RSU - The vehicle receives the message and displays the warning to the driver




s

ρ

s

V

Requirements: - RQ34_19_v1.0 (Directly address a Vehicle) The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all

vehicles in the vicinity of the RSU. - RQ35_19_v1.0 (Time of Data Delay 2) In the roadside sub-system the delay between generation of warnings and transmission of this data to the vehicles shall be smaller than 50 ms. - RQ36_27_v1.0 (Time of Connection) The time needed to set-up a connection between an incoming vehicle and the roadside communication system should be smaller than 0.8

seconds. - RQ41_19_v1.0 (Static Map contents) The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall

comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead). - RQ46_19_v1.0 (Position of Vehicles 3) The positioning of vehicles have to fulfill the accuracy up to a lane detection extend. - RQ49_19_v1.0 (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. - RQ50_36_v1.0 (Transmission of warnings) The system shall be able to transmit the warning / recommendation to equipped vehicles. User needs: SP5_UN39: Infrastructure can warn the driver in case of excessive speed. Use cases: SP5_UC41 : Prevention of lack of adherence SP5_UC45 : Sudden reduce Visibility

Expected Values / Results The requirements are met. Specifically: - In the case of over speeding, the vehicle is warned. Otherwise, the vehicle on-board system knows the legal speed limit




Obtained Values / Results RQ34_19_v1.0 (Directly address a Vehicle) The roadside

sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all vehicles in the vicinity of the RSU.

OK

RQ35_19_v1.0 (Time of Data Delay 2) In the roadside sub-system the delay between generation of warnings and transmission of this data to the vehicles shall be smaller than 50 ms.

Not OK Lot of delay with map interaction

RQ36_27_v1.0 (Time of Connection) The time needed to set-up a connection between an incoming vehicle and the roadside communication system should be smaller than 0.8 seconds.

Not tested Only beacons were available to log the communication. We have no means to estimate the time of connection

RQ41_19_v1.0 (Static Map contents) The LDM shall describe the static geometry of critical points (e.g. intersections) in a detailed, accurate and systematic way. The geometry shall comprise at least approaches, exits, lanes (also bicycle lanes), stop lines and pedestrian crossings as well as the topology of the lanes (left-turn, right-turn, straight ahead).

ok

RQ46_19_v1.0. (Position of Vehicles 3) The positioning of vehicles have to fulfill the accuracy up to a lane detection extend

Not OK We used a standard GPS mode, without any vision sensor to detect lane.

RQ49_19_v1.0. (Position and speed of vehicles) The RSU subsystem shall be able to receive at minimum the current vehicle position and speed

OK


OK





Table 9 H&IW_01 and H&IW_02 Simulation

TEST FORM

SP 5 WP 5 Prototype Version - System Configuration - HW

Release - SW Release - Compiled by / Company MIZAR Date 22/05/2009 (test case 1) 27/05/2009 (test case 2)

Form Progressive Numb. 6 Functional Component H&IW_01 H&IW_02 Reference Document D5.2.3

Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Simulation

Test Case 1: Basic functionality test for H&IW_01 and H&IW_02 #1: Event Notification to BlackSpot Monitor Test Description

Prerequisite : - Information about an event inside the “traffic event” table of LDM.

- [trafficevent.eventcause = 35 (ghost driver)] - [trafficevent.eventcause = 28 (animal)] - [trafficevent.eventcause = 30 (pedestrian)] - [trafficevent.eventcause = 35 (obstruction on road)]

- Simulation of [motorvehicle speed > 20 km/h] - Sensor source simulator: WSN (Wireless Sensor Network), RFID sensors, Thermal imaging cameras (provided by VTT). - Synchronization of the clock and Positioning:

- connect GPS receiver to Positioning PC (Application PC) - SP3 Positioning SW (responsible to acquire PPS signal from GPS and synchronize PC time) - Start NTP Server on Positioning PC - Start NTP Client on RSU PC and on ROUTER VANET PC) - In order to check synchronization it is possible to compare Beacon sending time from RSU to beacon receiving time from Vehicle using the file log vanet_msg_incoming.

- DataReceiver from SP2 - RSU with OS linux and Windows Description: The goal of this test is verify the correctness in the communication between the BlackSpotMonitor (application responsible for detecting H&IW events) and the LDM after the detection of each H&IW event. This basic function is needed for all use cases of the H&IW 01 and 02. - The H&IW sub-application is by means of a trigger launched by the LDM communicates with the BlackSpotMonitor.




- The BlackSpotMonitor creates a log that shows the state of the message (Correct message received from LDM, Error connection with LDM, Error message received from LDM)

Extended Description - The test start with the simulation of:

o Vehicle that goes in the wrong way (Ghost Driver) o Pedestrian on Motorway o Accident as Obstacle, Slow moving vehicles, or traffic jams as an obstacle

- The simulator of the relative application generates a corresponding message regarding the presence of an event o For case Ghost Driver detected by WSN and RFID o For case Pedestrian on Motorway detected by VTT o For case Obstacle by WSN

- The LDM sends the corresponding message to the BlackSpotMonitor depending on the case - The Black Spot Monitor receives this message.

Two important test were developed:

1. One using one simulator per time, (one simulator for each type of sensor) 2. Second using all use cases simultaneouly.

Test lab location: CRF Centro di Ricerche FIAT lab Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use Cases: - SP5_UC34_v1.0 (Ghost Driver) - SP5_UC17 (Pedestrian on Motorway) - SP5_UC13 (Accident as Obstacle) - SP5_UC14 (Traffic jams as obstacles (slow moving vehicles)) - SP5_UC15 (Traffic jams as an obstacle results of an accident, with poor visibility)

Expected Values / Results The listed requirements are met, specifically, - The reception of the correct message in the BlackSpotMonitor for all the UC simulated. - Detection of the obstacle with the associated position A log that shows the message reception of the corresponding UC. Note: For SP5_UC13, SP5_UC14, SP5_UC15, the message event will be the same.




Obtained Values / Results Results The BlackSpotMonitor detects correctly the H&IW events. The high frequency used to writte data in the LDM makes that the trigger is not able to detect all the messages. Some traffic events are lost. The Trigger detects one event per each second.

Conclusions (difference actual and expected requirements, system limitations) The two test were successful.. Even tough the use of high speed of sending UDP messages cause that the BlackSpotMonitor is not able to detect all the messages. Some traffic events are lost. The BlackSpotMonitor detects one event per each second. The events detected by the BlackSpotMonitor were correctly interpreted as H&IW events. The event log shows that the event cause corresponds with the parameters entered in the simulator regarding the type of Use Cases. It was important to implement a policy of cleanning the LDM because after some hours of test the LDM start to increase. The script implemented the information inside the database (each hour) was implemented.

Status Test Case1 OK Link to external annexed documentation (if any) Test Case 2: Basic functionality test for H&IW_01 and H&IW_02 #2: Send and Reception of the Message HMI

Test Description Prerequisite : In the table traffic event of LDM there is the information regarding the presence of an event corresponding to H&IW_02 or H&IW_01 - [motorvechicle speed > 20 km/h] - [trafficevent.eventcause = 35] - Sensor source: WSN and RFID sensors - Simulator of WSN, RFID and VTT messages. - DataReceiver from SP2 - RSU with OS linux and Windows - The BlackSpotMonitor have received the message from LDM. - All the test before have to be successful - Vanet unit inside RSU and Vanet in the vehicle - 1 vehicle Description: The goal of this test is to verify the correct operation of the generation of the SendHmiEventMessage and the Reception of the RSUHmiMsg in the vehicle. Extended Description: This is a IV communication test and has to be tested for all the use cases, to do this test it is necessary to do a simulation of all the cases in order to generate an event. After the reception of the corresponding notification message from the Black Spot Monitor the corresponding message has to be sent to the vehicle by means of Vanet. The Vanet has to be able to display a log of the correct receipt of this message and the correctness of it. Test lab location: - CRF Centro di Ricerche FIAT lab




Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use Cases - SP5_UC34_v1.0 (Ghost Driver) - SP5_UC17 (Pedestrian on Motorway) - SP5_UC13 (Accident as Obstacle) - SP5_UC14 (Traffic jams as obstacles (slow moving vehicles)) - SP5_UC15 (Traffic jams as an obstacle results of an accident, with poor visibility)

Expected Values / Results The requirements are met, specifically: - To Hmi Message that correspond to the use cases is received in the vehicle. - The messages arrive in the vehicle with the correct information of the obstacle and its position - The log will show a message of correct message reception.. Note: For SP5_UC14, SP5_UC15, SP5_UC13 the message event will be the same.

Obtained Values / Results The expected results were achieved. Some issues were corrected in order to have the better response of the systems. The most relevant observations are: In the system the HMI message should be sent in broadcast . The first results failed because the HMI have been defined using a unicast instead that a broadcast destination address The configuration of the correct port was important. The use of a definition of a HMI message with a relative timeValidity gives better results than the use of an absolute time validity, because the possible difference between vehicle time and rsu time can affect the results





Table 10 H&IW_01 Test site 1

TEST FORM

SP 5 WP 5 Prototype Version Cofi 1.8 System Configuration

RSU : PC / Linux PC / XP VANET / 802.11p OBU: Positioning PC PC / XP (Display) VANET / 802.11p

HW Release 1 SW Release 1 Compiled by / Company COFIROUTE Date 10/03/2009 Form Progressive Numb. 10 Functional Component H&IW_01 Reference Document D5.3.5

Test Type Test Objective Test Purpose Test Environment Integration Verification Performance Test Site

Test Case 1: Basic performance test for H&IW_01 (Obstacle) #1: Accident/Vehicle as obstacle Test Description

Prerequisite: Functional - the functional unitary correctness validation simulation tests of H&IW_01_v0.1 should have been performed and assessed on the LDM of the test site (Navteq or TeleAtlas) - Integration of other SAFESPOT component of the architecture, both in vehicle and infrastructure, and corresponding test should have been performed and validated. Deployment - One RSU and two equipped vehicles are required for this test. - One equipped vehicle (B) is stopped within the range of the RSU preferably downstream the RSU (this is the Accident/Vehicle as obstacle). The other equipped vehicle (A) passes

within the range of the RSU. It moves from upstream to downstream the RSU, in the direction of the road where the stopped vehicle VB is located (i.e. at the considered road section). - No specific sensors are required for this test. Accurate positioning and lane level details at the LDM are required for the extended version test. In addition, weather sensors at the RSU

and/or at the vehicles are required in the extended version (note that RSU gateway to a legacy system can act as a weather sensor). Finally, VMS can be connected to the RSU to inform non-equipped vehicles.




Description: The test consist on comparing a prepared table containing estimated time and position of when/where a warning from H&IW_01 should appear at a SAFESPOT equipped vehicle. This table is prepared for each test site (otherwise, it will contain too many entries). - First, the stopped vehicle (B) sends its beacons. - The H&IW_01 sub-application detects a stopped vehicle (an obstacle) through its BlackSpotMonitor function. - Then the ScenarioManager function is launched to compute the Threat level. - Then vehicle A enters the range of the RSU and sends its beacons. - The H&IW application sends the generic H&IW warning corresponding to the threat level (Comfort, Safety, Emergency) depending on its type, speed, road attributes and obstacle

position. - If the vehicle A does not adapt its speed to stay in the “comfort” situation, the different Safety and Emergency warnings will follow. Test 1: vehicle A passes trough the coverage area of the event, with the maximum legal speed authorized at the road. The vehicle A does not adapt its speed and sees the three levels of warnings. First, the Comfort warning, then the Safety warning, this is shown 3 seconds before crossing the obstacle with constant speed, plus the estimated time to stop. The emergency warning is shown one second plus time to stop before crossing the event. Obviously, the vehicle A does not drive on the lane where the obstacle is located! Test 2: vehicle A slows down (to around 20 km/h) so it stays in the comfort situation. The driver should intuitively choose the exact speed reduction. The exact speed reduction expected by the H&IW application depends on many factors and will be calculated according to the specification of H&IW prior to the test. Extended description: - the application retrieves additional information from the LDM about weather and traffic status to adapt the warning message, i.e. the warning destination area should be reduced or

extended depending on the weather and traffic status (this imply function scenarioAnalysis() of object ScenarioManager). - then it generates the appropriate RsuHMIMessage and triggers the SendHMIPeriodicMessage from SP5_MessageManager (i.e. the SP5 interface with Vanet) and it triggers the

display on possible VMS through the SP5ApplicationCoordinator. - The warning messages sent to vehicle A contains “lane merging” indications at the icon displayed onboard, if accurate positioning of the vehicles and a detailed LDM (lane level

positioning) are available. Related Test sites:

RSU

VehB

VehA




from vehicle beacons: - Netherland/A16 (TeleAtlas) - West/A86 (Navteq) - Livic TBC (Navteq) - CG22 TBC (Navteq) from rsu : - Italy/Brescia-Prodova (Navteq) Tools: The timing values can be determined by analysing H&IW log files and observing SAFESPOT UDP messages. In addition, Esposytor, can be used. Requirements: H&IW functions - SP5_RQ02_36_v1.0 / Message Management: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ05_36_v1.0 / Identify Safety Critical Situations: The system shall be able to identify safety critical situations at the surrounding of a critical point e.g. an urban intersection. - SP5_RQ06_19_v1.0 / Priority Level of Message: The system shall be able to provide a priority level for each transmitted message. This is valid for V2I as well as for I2V

communication - SP5_RQ07_19_v1.0 / Validity of Messages: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ09_19_v1.0 / Message Management 2 : The system must not send warnings concerning severe dangerous situations (that might cause drivers to extreme driving actions),

which does in fact not happen. - SP5_RQ12_33_v1.0 / Data exchange: The system shall be able to transmit information towards several supporters (e.g. VMS, radio, PND). - SP5_RQ16_36_v1.0 / Query from the LDM: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0 / Receive data from the LDM: The system shall be able to receive and handle the LDM data. LDM functionalities - SP5_RQ49_19_v1.0 / Position and speed of vehicles: The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. - SP5_RQ68_10_v1.0 / Road status: The system shall acquire data on the weather conditions in the 'critical area' (extent TBD) of the obstacle or accident, SP5_RQ65_10_v1.0 /

Obstacle description - The system shall provide other details of the obstacle where possible : type of object (accident, queue, rocks, dropped load, etc), lanes affected, speed (for moving object), precise

location etc. i.e. wet/dry road surface, rain, fog, ice, which may affect stopping distances and visibility. TIMING and Performance - SP5_RQ18_19_v1.0 / Simultaneity: The system shall be able to manage all vehicles in the vicinity of a critical point (e.g. the intersection) simultaneously. - SP5_RQ20_19_v1.0 / Time of Warning Generation: The system shall generate and transmit warning information to the drivers timely enough, so that the reaction time left to the

drivers is appropriate. - SP5_RQ22_10_v1.0 / R/S Warning: The system shall provide a roadside warning in fewer than 2 sec (TBC) in the critical zones to permit vehicles to reduce speed and/or change

lane in order to avoid a (secondary) collision. - SP5_RQ35_19_v1.0 /Time of Data Delay 2: In the roadside sub-system the delay between generation of warnings and transmission of this data to the vehicles shall be smaller than

50 ms. MESSAGES EXCHANGE and Filtering - SP5_RQ34_19_v1.0 / Directly address a Vehicle: The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to




broadcast to all vehicles in the vicinity of the RSU. - SP5_RQ50_36_v1.0 / Transmission of warnings: The system shall be able to transmit the warning / recommendation to equipped vehicles. - SP5_RQ69_10_v1.0: V Warning: The system shall send appropriate warnings to equipped vehicles within the critical (e and a) zones to permit them to reduce speed and/or

change lane and avoid a (secondary) collision. - SP5_RQ37_19_v1.0 / Filter for relevant data: The vehicle subsystem should be able to filter the relevant information to the driver. - SP5_RQ51_19_v1.0 / HMI:In vehicle assistance HMI shall present warnings to the driver in an intelligent way without distracting him. Example: Use of different actuators to smooth

signalizes hazards. - SP5_RQ48_36_v1.0 / Environmental Data: The system shall receive data from environmental sensors about weather (rain, snow, fog…). - SP5_RQ52_36_v1.0 / Static Vehicle Data: The system shall receive static vehicle data like width, length, type of vehicle, mass. - SP5_RQ60_36_v1.0 / Lack of Friction: The system shall receive information about the lack of friction of a road segment. - SP5_RQ61_36_v1.0 / Data Reception: The system must be able to receive data from the vehicles as well as the infrastructure. Use cases: SP5_UC13_v1.0 : Accident as an obstacle

Expected Values / Results The requirements are met, specifically, - The VANET component sends the RsuHMImessage and the vehicle display the text / icon - Test 1 will validate the warning sequence. The expected sequence is: Comfort, Safety, Emergency - Test 2 should validate the ability to stay in what H&IW considers as a Comfort situation. - The time shift between the specification and the implementation/deployment will be measured. The expected time measures are based on the “Safety Margin” definition of H&IW, which describe where and when a warning (Safety or Critical) message should be received. It mainly describes the time left to the driver to react for each warning types. NB: The overall SAFESPOT system, including H&IW, will introduce delays due to computation and transmission that will reduce the time left to the driver to react. Therefore, different runs should be achieved to calibrate H&IW (mainly by adding average system delay into the Safety Margin Computation of H&IW). Test1 and Test2 should be performed for different vehicle speeds corresponding to different road environments and situations (30km/h, 50km/h, 90km/h etc...) and with different speed recommendations corresponding to different obstacles (road blocked, speed limited to 30km/h or 50km/h). This lead to different expected measures on the generated warnings, mainly in terms of destination areas, transmission dates and durations. The following figure shows an example of expected warning destination areas for a vehicle driving at 90km/h, and with an obstacle speed limitation to 50km/h:




An example of expected measures for test 1 are shown in the table below.

Obtained Values / Results For validation test purpose, case 1 (vehicle/accident) and case 2 (roadwork) has been tested together in one time by writing a simulated event into the LDM; therefore, obtained values for test case 1 are displayed in hereafter test case 2.


Test Case 2: Basic performance test for H&IW_01(Obstacle) #2: Road Works as an obstacle Test Description

Prerequisite: Functional - the functional unitary correctness validation simulation tests of H&IW_01_v0.1 have been performed and assessed at the LDM of the test site (Navteq or TéléAtlas) - Integration of other SAFESPOT component of the architecture, both in vehicle and infrastructure, and corresponding test have been performed and validated. Deployment - One RSU and one equipped vehicle are required for this test. - An area at the road contains road works or something similar, downstream the RSU. - The RSU is equipped with a Gateway able to provide information about the road works location to the RSU. - The equipped vehicle is within the range of the RSU from upstream to downstream of the RSU, in the direction of the road where the “test road works” is located (i.e. in the considered

road section). - No specific sensors are required for this test (except the RSU Gateway). Precise positioning and lane level details of the LDM are required for the extended version tests. In addition,

weather sensors at the RSU and/or at the vehicles are required in extended version (note that RSU gateway to a legacy system can act as a weather sensor). Finally, VMS can be connected to the RSU to manage non-equipped vehicles.




Description: The test consist on comparing a prepared table containing estimated time and position on when/where a warning from H&IW_01 should appears at an SAFESPOT equipped vehicle. This table is be prepared for each test site (otherwise it will contains too many entries). - First the RSU Gateway transmits information about roadwork presence to the RSU. - the H&IW_01 sub-application detects the roadwork (an obstacle) through its BlackSpotMonitor function. - then the ScenarioManager function is launched to compute the Threat level. - Then the vehicle enters in the range of the RSU, sending its beacons. - The H&IW application sends the generic H&IW warning that corresponds to the threat level (Comfort, Safety, Emergency) depending on it’s type, speed, road attributes and obstacle

position. - If the vehicle does not adapt its speed to stay into the “comfort” situation, the different Safety and Emergency warnings will follow. Test 1: vehicle passes trough the coverage area of the event, with the maximum legal speed authorized at the road. The vehicle does not adapt its speed and sees the three levels of warnings. The Safety warning is shown 3 seconds plus the estimated time to stop before crossing the obstacle with constant speed. The emergency warning is shown one second plus time to stop before crossing the event. Obviously, the vehicle does not drive on the lane where the obstacle is located! Test 2: The vehicle slows down (to around 20 km/h) so it stays in the comfort situation. The exact speed reduction should be intuitively chosen by the driver. The exact speed reduction expected by the H&IW application depends on many factors and will be calculated according to the specification of H&IW prior to the test. Extended description: - The application retrieves additional information from the LDM about the weather and traffic status to adapt the warning message, i.e. the warning destination area should be reduced or

extended depending on the weather and traffic status (using the function scenarioAnalysis() of the object ScenarioManager). - Then it generates the appropriate RsuHMIMessage and triggers the SendHMIPeriodicMessage from SP5_MessageManager (i.e. the SP5 interface with Vanet) and it triggers the

display on possible VMS through the SP5 ApplicationCoordinator. - The warning messages sent to vehicle A contains “lane merging” indications at the icon displayed onboard, if accurate positioning of the vehicles and a detailed LDM (lane level

positioning) are available, Test sites: - from rsu : West/ A86 (Navteq)




Requirements: - same as “H&IW_01 Test Site : Test Case 1: Basic performance test for H&IW_01 (Obstacle) #1: Accident/Vehicle as obstacle” Use cases: - UC : SP5_UC16_v1.0: Deviations for road-works

Expected Values / Results The requirements are met, specifically: - The VANET component sends the RsuHMImessage and the vehicle displays the text / icon - In test 1 the warning sequence is: Comfort, Safety, Emergency - In Test 2 the H&IW stays in the Comfort situation. Those test should enable to measure the shift, in term of time, between the specification and the implementation/deployment. See the expected values of H&IW_01 Test Case 1 for more details.

Obtained Values / Results The LIVIC track segment used for validation test is shown in the following pictures :




Hereafter are shown the matching between the application performances and the original specifications. H&IW functions requirements

SP5_RQ02_36_v1.0 Message Management

The system shall be able to decide if and to whom a message has to be send. OK

Identify Safety Critical Situations SP5_RQ05_36_v1.0

The system shall be able to identify safety critical situations at the surrounding of a critical point. OK

SP5_RQ06_19_v1.0 / Priority Level of Message

The system shall be able to provide a priority level for each transmitted message. OK

SP5_RQ07_19_v1.0 / Validity of Messages

The system shall be able to determine the period of validity of the transmitted and received messages

OK

SP5_RQ09_19_v1.0 Message Management 2

The system must not send warnings concerning severe dangerous situations (that might cause Not Tested




drivers to extreme driving actions), which does in fact not happen

Data exchange: SP5_RQ12_33_v1.0

The system shall be able to transmit information towards several supporters (e.g. VMS, radio, PND) OK

SP5_RQ16_36_v1.0 Query from the LDM

The system shall be able to query needed data from the LDM.

OK

SP5_RQ17_36_v1.0 Receive data from the LDM

The system shall be able to receive and handle the LDM data. OK

LDM requirements

SP5_RQ49_19_v1.0 / Position and speed of vehicles

The RSU subsystem shall be able to receive at minimum the current vehicle position and speed. OK

SP5_RQ68_10_v1.0 / Road status:

The system shall acquire data on the weather conditions in the 'critical area' (extent TBD) of the obstacle or accidentnt.

Not Tested

SP5_RQ65_10_v1.0 / Obstacle description

The system shall be able to provide a priority level for each transmitted message. The system shall provide other details of the obstacle where possible : type of object (accident, queue, rocks, dropped load, etc), lanes affected, speed (for moving object), precise location etc. i.e. wet/dry road surface, rain, fog, ice, which may affect stopping distances and visibility.

OK

Timing and Performance




SP5_RQ18_19_v1.0 Simultaneity

The system shall be able to manage all vehicles in the vicinity of a critical point simultaneously. Not Tested

SP5_RQ20_19_v1.0 Time of Warning Generation

The system shall generate and transmit warning information to the drivers timely enough, so that the reaction time left to the drivers is appropriate.

OK / Partially tested

SP5_RQ22_10_v1.0 R/S Warning

The system shall provide a roadside warning in fewer than 2 sec (TBC) in the critical zones to permit vehicles to reduce speed and/or change lane in order to avoid a (secondary) collision.

OK

SP5_RQ35_19_v1.0 Time of Data Delay 2:

In the roadside sub-system the delay between generation of warnings and transmission of this data to the vehicles shall be smaller than 50 ms.

Partially OK

It mainly depends on the load of the LDM. When the number of vehicles that must be written and updated in the LDM is small, the load of the LDM is low and the access delay has the same behaviour. For a high number of vehicles, the update mechanism of the beacon message in the LDM may block the Q-API, decreasing the performance of the application.

Message Exchange and Filtering

SP5_RQ34_19_v1.0 / Directly address a Vehicle

The roadside sub-system must be able to address data directly to one vehicle distinctively (or a group of vehicles) or to broadcast to all vehicles in the vicinity of the RSU.

OK

SP5_RQ50_36_v1.0 / Transmission of warnings:

The system shall be able to transmit the warning / recommendation to equipped vehicles OK

SP5_RQ69_10_v1.0: The system shall send appropriate warnings to OK




Warning: equipped vehicles within the critical (e and a) zones to permit them to reduce speed and/or change lane and avoid a (secondary) collision.

SP5_RQ37_19_v1.0 Filter for relevant data:

The vehicle subsystem should be able to filter the relevant information to the driver. Partially Tested

SP5_RQ51_19_v1.0 HMI

In vehicle assistance HMI shall present warnings to the driver in an intelligent way without distracting him. Example: Use of different actuators to smooth signalize hazards.

Not Tested

SP5_RQ48_36_v1.0 Environmental Data:

The system shall receive data from environmental sensors about weather (rain, snow, fog...). Not Tested

SP5_RQ52_36_v1.0 / Static Vehicle Data

The system shall receive static vehicle data like width, length, type of vehicle, mass Partially Tested

SP5_RQ60_36_v1.0 Lack of Friction

The system shall receive information about the lack of friction of a road segment Not Tested

SP5_RQ61_36_v1.0 / Data Reception:

The system must be able to receive data from the vehicles as well as the infrastructure. OK




Different contributions to the total loop time

Vehicle Speed Loop time (From VANET log) Loop error in distance

Expected distance to obstacle

when warning is displayed

Measured distance to obstacle

when warning is displayed

50 km/h av.: 333ms [312, 353] ms 4.07m [3.81, 4.31] m 55 m 20 m

90 km/h av.: 522ms [458, 707] ms 12.83m [10.75, 17.86] m 120 m 80 m

120 km/h av.: 441ms [240, 751] ms 14.89m [8.09, 25.52] m 150 m 100 m

Major limitation faced during the validation tests was the VANET range which never exceeded 300 meters





Table 11 H&IW_01 Test site 2

TEST FORM


HW Release - SW Release - Compiled by / Company MIZAR Date

12/2009 (test case 1) 08/2009 – 04/2010 (test case 2) 12/2009, 03/2010 (test case 3)

Form Progressive Numb. 10b Functional Component H&IW_01 Reference Document D5.3.5


Test Case 1: Test #1: Obstacle (stopped vehicle) CRF- WSN Test Description

Prerequisite: - The successfully results of all the previous tests

- Synchronization of the clock and Positioning have to be done - Base Board with a Local Detection Algorithm installed (FW_LDA) - Main PC (ECW-281B-ATOM) - SP2 applications(Data Fusion Module) - SP5 applications installed inside Main PC - Vanet Router dedicated gateway PC connected to main PC - 2 SF FIAT (CRF) called Vehicle A and Vehicle B

Functional Description This test aims to verify the correctness of the detection of an obstacle and the communication of a warning message to approaching SAFESPOT vehicles. The components will communicate according to the following sequence: 1. Two SF-equipped vehicles drive around the test track 2. One SF-vehicle stops by the roadside 3. Both vehicles are beaconing and the messages are received by the RSU. 4. The RSU detects the stopped vehicle by reading its position inside beacon messages. 5. The communication to the Data Fusion is done and the event is written in the LDM




6. The trigger generates the notification that an obstacle is detected 7. The Black Spot Monitor receive this message 8. The SP5 application analyse the risk of the event and takes the decision of to send the message to the router Vanet (SP5_MessageManager) 9. The second vehicle (B) passes the area of risk. 10. The vehicle is able to see the message with the External Message Applicator as soon as it entered the area of area (Risk area 170 m X 8m) and switch off automatically when it left the

area

Test site: CRF Related Test: Netherlands A16 TeleAtlas France - West/A86 - Navteq Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data.

Geovalidity Area

Stopped vehicle




Use cases: - SP5_UC13 (Accident as Obstacle) - SP5_UC14 (Traffic jams as obstacles (slow moving vehicles)) - SP5_UC15 (Traffic jams as an obstacle results of an accident, with poor visibility)

Expected Values / Results The requirements are met, specifically: - The WSN detects the obstacle and its position - Communication between the components


This test was not carried out as the original planning, because the Wireless Sensor Network alone are not able to detect the presence of a stopped vehicle with its specific position. It was decided to use this opportunity to test the possibility of adopting vehicle beaconing for the same purpose. The test of the detection of an obstacle was carried out in the Test Case 2: See “Test#2 considering an obstacle as a traffic jam using the ECAID system”. IIn this test an stopped vehicle was used, the detection was made using the beaconing data from the SF-vehicle. The area of the validity of the messages was enough to alert the incoming vehicles about the risk (stopped vehicle). The test considered different speeds from 50km/h until 80 Km/h. The duration of the messages were between 10 sec until 6.8 sec (depending of speed)

Results: The communication between infrastructure and Vehicle was successful. The vehicle received the messages inside the area defined by the geovalidtity No messages out of the geovalidity area were received by the vehicle. The area of the validity of the messages was enough to alert the incomming vehicles about the risk (stopped vehicle). This test was repeated ten times. On each occasion, as shown by the RSU data log, the SP2 Platform was able to detected a stationary vehicle (vehicle speed = zero and position fixed) and the ‘event’ was written in the LDM. A trigger produced notification that an obstacle had been detected, and the H&IW_01 then generated a message using the SP5_MessageManager. It was observed that the message appeared on the HMI of the second vehicle each time it entered the ‘area of risk’ and then switched off when it left. The duration of the messages were between 6.8 and 10 sec, depending on the speed at which it was travelling. It worked for all speeds up to 80 km/h


Test Case 2: Test #2: Obstacle (traffic jam & slow vehicle ) Brescia – Padova-WSN-ECAID Test Description

Prerequisite - Successful results of all the test before - The system ECAID has the communication with the LDM - Installation of the WSN system in Brescia-Padova( test area)




o 10 Sensors (AMR and Pyro) o Base Board with a Local Detection Algorithm installed (FW_LDA)

- Main PC (ECW-281B-ATOM) - SP2 applications(Data Fusion) - SP5 applications installed inside Main PC - Vanet Router dedicated gateway PC connected to main PC - 2 FIAT BRAVO (CRF) called Vehicle A and Vehicle B (with Vanet) Description: The components will communicate in the following sequence: 1. The vehicles are driving in a normal way 2. A traffic event is detected by ECAID (analysing the information of mini-TDC) using the data from COMPANION sensors, WSN and radars 3. The communication to the Data Fusion is done and the event is written in the LDM 4. The trigger generates the notification that an obstacle has been detected 5. The SP5 application analyses the risk of the event and takes the decision to send the appropriate message to the router Vanet (SP5_MessageManager (SP5 interface with Vanet) 6. The second vehicle (B) passes the area of risk (using the simulation of a traffic jam) 7. The vehicle is able to see the message with the visualization tool Objectives of the TEST: The system will be tested also considering traffic jams as an obstacle. In this case the test will be performed in the following sub-Test: 1. Performance of WSN prototype on a 3 lane motorway (data output and writing on LDM). 2. For Brescia-Padova: integration with (TDC and ECAID) i.e. other sensor data 3. Identification of traffic patterns 4. Detection of ‘event’ (traffic jam & slow vehicle, etc) 5. Send of the HMI messages Test site - Brescia- Padova - Pila (Aosta) - Chivasso (Lab) BS-PD Installation of the sensors began at 07/2009. The sensors couldn’t work properly for the high traffic conditions of the place. After different analysis of the problems with the sensors, two important upgrades to network were developed. A first update to the network was run on 12/04/2010 developed by ISMB and MIZAR. The update consisted in modification of the communication protocol and the way of manage memory inside the sensor, these changes allow to improve the number of vehicles detected by the WSN. For other hand the network was unstable and new changes were made on it A second updated run was on 21/04/2010 developed by ISMB and MIZAR.




Pila (Aosta) This test was carried out in the highway of Pila, the sensors were installed for one month (November 2009 – December 2009). This test allows to test the sensors in a real environment Chivasso A mini-network of the WSN was installed in the laboratory using 10 sensors + gateway. The aim of this test was re-create the conditions of hardware and software as in BS-Pd in order to solve and analyze the possible issues found at BS_PD. Using this network it was possible to improve the limitations of the system installed in BS-PD Related test sites - Nether-lands A16 TeleAtlas - France - West/A86 - Navteq Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use cases: - SP5_UC13 (Accident as Obstacle) - SP5_UC14 (Traffic jams as obstacles (slow moving vehicles)) - SP5_UC15 (Traffic jams as an obstacle results of an accident, with poor visibility)

Expected Values / Results The requirements are met, specifically: - Interaction between components - Detection of the obstacle and its position - I-V communication - By comparing the detection of the WSN and ECAID systems, it is expected to obtain more precision in the results with the WSN system.


Results:

The sensors support extreme conditions, were installed At the beginning the sensor WSN were not able to work under real time traffic conditions. After modification the WSN were able to manage intense conditions of traffic. The first limitation was the high number or vehicles, this issue was solved increasing the time of request from gateway to the sensors from 5 sec to 2 sec. Also an improvement in the nformation transmited was applied, so the new configuration gives high priority to the information regarding speed and number of vehicles and low priority to the extra- information like (type




of vehicle, direction, etc). This approach allows to have a system that worked under real traffic conditions. The max number of vehicles detected by each sensor was 15 vehicles each 2 seconds

The communication between all the components was successful and the V-I and I-V communication was successful. . Experiments carried out with direct RSU-vehicle communications demonstrated correct receipt of messages at approx 570m and with ‘multi-hop’ communication via a stationary vehicle (acting as a safety vehicle) the distance increased to 640m. This latter can help to increase the safety coverage provided by the system when there are obstacles to direct communication of warning messages due the physical configuration of the ground or other obstacles. This test was repeated ten times. On each occasion, as shown by the RSU data log, the SP2 Platform was able to detected a stationary vehicle (vehicle speed = zero and position fixed) and the ‘event’ was written in the LDM. A trigger produced notification that an obstacle had been detected, and the H&IW_01 then generated a message using the SP5_MessageManager. It was observed that the message appeared on the HMI of the second vehicle each time it entered the ‘area of risk’ and then switched off when it left. The duration of the messages were between 6.8 and 10 sec, depending on the speed at which it was travelling. It worked for all speeds up to 80 km/h


Test Case 3: Test #1: Obstacle-Pedestrian on the road-CRF-Test Track- Thermal imaging cameras (provided by VTT) Test Description

Prerequisite: - Successful results of the previous tests - Synchronization between the clock and Positioning - Installation of the Thermal imaging cameras (provided by VTT) ( test area) - Main PC (ECW-281B-ATOM) - SP2 applications (DataReceiver, Map Matching, Object Refinement) - SP5 applications installed inside Main PC - Vanet Router dedicated gateway PC connected to main PC - 2 FIAT BRAVO (CRF) called Vehicle A (with Vanet) The components will communicate according the next two stages: Method First stage: Firstly detection system was firstly calibrated to the local test site in accordance with the detailed specifications prepared by VTT. The camera was installed on a metal support in a position where it could record the scene. Within the camera range, a person crossed the ‘road’ several times from one side to another. When the camera detected an object and recognized the form as a pedestrian with a confidence parameter of over >40%, the detection was considered reliable and a broadcast message was generated. It then sent to the RSU a UDP message with the detection data (broadcast message). Second stage: In the second stage, this message was simulated to test the ability of the H&IW application to recognise the message, trigger an alert and send a message to a SAFESPOT equipped




vehicle. An observer in a moving vehicle recorded whether and when the message was received. It was also possible to verify receipt of the message with the visualization tool (Esposytor). The test was repeated 50 times over a period of four hours. Test site: CRF Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use cases: SP5_UC17 (Pedestrian on Motorway)

Expected Values / Results The requirements are met, specifically: - Components provide the correct information - Messages are correctly display in vehicles - Communication between all the components

Obtained Values / Results Results

- When the UDP message was written in the LDM traffic event table: event = 30 (pedestrian), this triggered a message to the BlackSpotMonitor that an event had been found. The H&IW_01 generated and sent a message which was activated the display of the pedestrian icon on the vehicle HMI.

- In the first stage, some defects were revealed in the detection system. When a person crossed the road, the system usually registered the presence of an object, but did not always make a correct classification.

- Some problems were encountered initially with the writing of the message in the LDM, necessitating a reset of the parameters. When these were adjusted, the trigger was successfully generated and the warning message received by the vehicle. Conclusions

- With respect to the H&IW_01 functions, after some resets of the system had been carried out, the communication of the message from the infrastructure (RSU) to an equipped vehicle was successful with the display of the warning (pedestrian icon).




Status Test Case3 OK Link to external annexed documentation (if any) Table 24 Obstacle-Pedestrian on the CRF Test Track – Thermal imaging cameras




Table 12 H&IW_02 Test Site

TEST FORM

SP 5 WP 5 Prototype Version - System Configuration - HW Release - SW Release - Compiled by / Company MIZAR Date 06/2009

Form Progressive Numb. 11 Functional Component H&IW_01 Reference Document D5.3.5


Test Case 1: Test #1: Ghost Driver in CRF using RFID sensors. Communication Test Test Description

Prerequisite: - The successful results of all the test before - Synchronization of the clock and Positioning - Installation of the RFID system in CRF test area

o RFID readers (PCMCIA) o RFID antennas (both Omni directional and directed) o RIFD tags (active tags)

- Main PC (ECW-281B-ATOM) - SP5 applications installed inside Main PC - SP2 applications(DataReceiver, RFID application, Map Matching, Object Refinement) - Vanet Router dedicated gateway PC connected to main PC - 2 FIAT BRAVO (CRF) called Vehicle A (with Vanet) and Vehicle B (ghost driver) Description: The test will check the correct communication between all the components using the detection system RFID. The components will communicate in the following sequence: 1. Vehicle A drives in the correct way at the area of test 2. Vehicle B drives in the wrong way at the area of test 3. The RFID system (block of antenna, reader and elaboration system) detects the presence of a ghost driver. By means of software used for ghost driver detection correct messages are

produced and send to the main PC 4. The message arrive at the DataFusion Block and it is saved inside the traffic event [trafficevent.eventcause = 35 (ghost driver)] 5. The trigger detects the event inside LDM and communicates with the Black Spot Monitor 6. SP5 applications evaluates the Risk and generates a corresponding message 7. The message is received by the Vanet router




8. The Vanet router sends this message to the Vanet of Vehicle A 9. The Driver will see the message by means of the visualization tool (e.g. Esposytor) that shows the presence of a ghost driver in a x, y position. 10. Driver B avoids and accident by slowing down.

Antenna RFID and RFID tag

Antenna RFID omnidirectional

Ghost driver: two cars in opposite direction

Components communicated in this sequence: 1. Each Vehicle have one RFID tag 2. The vehicles moves at 50km/h. Vehicle A follows the correct direction and vehicle B the wrong direction 3. When the vehicle A passes out of the range of the antennas, no message is generated. 4. The vehicle B enters in the range of antenna B, the PC connected to antenna B, receives the information and

sends the message by Wireless Network WiFi to computer A. The time difference between sensing by antenna A and B is set by a time window. Otherwise false alarms would be generated if a car later travels in the opposite direction. The time window has to be adjusted considering the speed of traffic flow at the particular road segment. During the tests, the time window was 20 seconds.

5. Then vehicle B passes antenna A, the Pc connected to antenna A receives the information and the algorithm of ghost driver detects the (by analysing the passing orders) ghost driver

6. The message is sent to the DataReceiver, and it is saved in table motorvehicle and traffic event of LDM. 7. The RSU sent the HMI message to the vehicle vehicles. 8. The message is displayed inside the vehicle.

Shadowing 2 Ghost Drivers (2 ghost drivers traveling in Components communicated in this sequence:

Correct Direction

Wrong Direction




parallel lanes next to each other)

The test pretends to check if there is a shadowing effect in the case that two ghost

drivers are present in the track at the same time, but in different lanes.

1. Each Vehicle have one RFID tag 2. The vehicles A moves at 30km/h in the lane 1 and at the same time the vehicle B moves at 30km/h in the lane 2

(parallel lines). 3. The two vehicles pass (at the same time) in the range of antenna B, the PC connected to antenna B receives the

information and sends the 2 messages by Wireless Network to computer A. 4. Then the 2 vehicles (at the same time) pass antenna A, the Pc connected to antenna A receives the information

and the algorithm of ghost driver detects the (by analyzing the passing orders) ghost driver 5. The messages of ghost driver detection are sent to the DataReceiver, and it is saved in table motorvehicle and

traffic event of LDM. 6. The RSU sends the HMI message to the vehicle vehicles 7. The message is displayed inside the vehicle.

This test was developed in different sessions: BME arrives to Italy in May 2009, for one day, the test were developed in different sessions one with all the equipment of BME for develop the test of detection, and the rest of the test were developed using the replay of the UDP messages recorded at that time Test site: CRF: Centro di Ricerca FIAT test area Related Test sites: Italian Test Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use cases: SP5_UC34 (Ghost Driver)

Expected Values / Results The requirements are met, specifically: The expected results will be that two tables are verified: - Verification Table 1 will have the Messages sent for each component described above

Expected Send message, Real Send message, Component - Verification Table 2 will have the Message receive for each component described above

Expected Receive Message – Real Receive Message. Component The test will be considered as successful if the group of Send and Receive Messages corresponds exactly to the Expected Send and Receive Messages

Obtained Values / Results Comments: 1. The Message sent GhostDriverFrom RFID Sensor’was correctly received by DataReceiver in the RSU with the correct format described in SF_SP7_Data_formatmessages_v2 15 1

Wrong Direction




(3).doc 2. In this case the message value for ghostDriver = 1 implies that the Data Receiver writes inside the table traffic event the cause of this event (35 for Ghost Driver) and also the message

‘GhostDriverFrom RFID Sensor’ 3. The HMI messages send to the vehicle were correctly received by the vehicle, the messages were displayed correctly inside the vehicle Conclusions • The result of the test satisfied the requirements of:

a. Correct communication between elements b. Correct detection of the event Ghost Driver c. Correct communication with dataReceiver and LDM d. Correct detection of the traffic event e. Correct construction of the HMImessage f. Correct reception of the messages inside the vehicle by the External Message Applicator.

• The time difference between sensing by antenna A and B is set by a time window. This time window should be adjusted considering the speed of traffic flow at the particular road segment. During the tests, the time window was 20 seconds.

• The verification table one and two showed that the messages expected by each components were correctly interpreted • Shadowing effect might be critical at high speed • Using low speeds, aproximately 30-50 km/h the test in CRF was successful, for high speeds new test should be conducted and a calibration of the time window is needed.

The whole system is still now a prototype. it was decided that the system need to be improved in order to have a new small version of the system to be carried to the highway


Table 22 Verification Table RFID – Ghost driver two cars in opposite direction Table 23 Verification Table RFID – Shadowing 2 Ghost Drivers

Test Case 2: Test#2 Ghost Driver on CRF test track-Using WSN Test Description

Prerequisite: - The successful results of all previous tests - Synchronization of the clock and Positioning - Installation of the WSN system in Torino Caselle (test area)


- Main PC (ECW-281B-ATOM) - SP2 applications (DataReceiver, RFID application, Map Matching, Object Refinement) - SP5 applications installed inside Main PC - Vanet Router dedicated gateway PC connected to main PC - 2 FIAT BRAVO (CRF) called Vehicle A and Vehicle B (with Vanet)




Extend description SF vehicle CRF Hardware Software - Application PC: TF-AEC-6830-A1 - Main PC: eBOX638 –FL - Positioning PC: AAEON AEC-6920 - GPS 1: Ublox (only for synchronization) - GPS 2 Novatel (for DGPS). Radio system to receive the correction from the base station

- FW à 9.8.2 - OR à 9.8.2 - SR à 2.92 - DAQ à 8.4.2 - LDM à 10.0.9 - API à 1.7.10 - Positioning software à V 4.3 - Router à 0.9.985 - Application sw à 2.0

Description: This test will focus on the verification of the correct detection of a wrong way driver and the correct creation of warning for the incoming vehicles. communication between all components using the detection system of WSN. Giving the fact that is not possible to simulate a ghost driver in real traffic the test was developed under controlled conditions in CRF.In this test, a broadcast message will be send. The two vehicles provided with Vanet will receive a message from the Vanet router about the detected ghost drivers. The components will communicate in this following sequence:

1. The vehicles are driving in the legal direction 2. The sensors detects that there are ghost drivers and communicate this fact to the Main RSU PC. 3. The message arrives to the DataFusion Block and it is saved inside the traffic event [trafficevent.eventcause = 35 (ghost driver)] 4. The trigger detects the event inside the LDM and communicates with the Black Spot Monitor 5. The SP5 application evaluates the Risk and generates a corresponding message. SP5_MessageManager (SP5 interface with Vanet)) 6. Vehicle A enters the ghost driver area 7. The traffic lights turns on in order to alerts the Vehicle A (Wrong Way Driver) 8. The Driver of vehicle B will sees the message by means of the visualization tool (External Message App). 9. Additional, the driver will be alerted that there are other ghost drivers in the surrounding.

Test site: CRF




Test setting at the Orbassano test track

The 3 Sensors were distributed in the next CRF Test Track, with a distance between them of 15 meters. Then the distance between the RSU and the Gateway was the 45 meters. Also the Vanet antenna was installed in a post (3,5 meters). The original set up is described in the Figure 2. After some test the Gateway of the RSU was located in a in a straight line, in order to improve the communication between all the sensors and use the barrier of the test Track. See Figure 2. For this scenario the warning used where HMI messages, and traffic lights signaling. When the Wrong way driver was detected the trafic lights turn on and the vehicle passing by the area of risk received the message during the time that pass through the area of the geovalidity. The test considered two lanes where the wrong way driver can pass in front of the sensors. For more details see Annex: Results Wrong Way Driver – CRF. Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use cases: - SP5_UC34

Expected Values / Results The requirements are met, specifically: - All Real Sent and Received Messages correspond to the Expected Messages. - All components should communicate in a correct way.

Vanet + Positioning PC +GPS +-Antenna

Traffic lights




Obtained Values / Results All the use cases works correctly

For the case of the period of validity of the messages the system was able to send the messages with a correct period of time, and inside the area of the geovalidity.

After some tries the traffic event were not saved in the LDM, the reason was the static table in the LDM for the sensors. At the beginning the communication didn’t works because this table was empty, and the SP2 and SP5 framework needed this information in order to work properly. The problem was solved after inserted the correct information about sensors (positioning and d).

The quality of the detection depends of the position of the sensors. The angle of inclination of the sensors change the results of the detections. Changing the angle of inclination the detections change dramatically.

Using the (See Table Result Wrong Way Driver) the first system of detection was possible to see that some detections were lost when the vehicle passes by the second lane. After change the algorithm of detection inside the sensors, increase the perception of the vehicles that passes by the second lane as well as the time of detection was improved..

Messages to Esposytor were correctly sent and received.

All requirements were fulfilled

Status Test Case2 OK Link to external annexed documentation (if any) Table 25 Results Wrong Way Driver – CRF – Correctness Table 26 Wrong Way Driver – CRF – Communication Test




Table 13 H&IW_02 H&IW_03 Test Site

TEST FORM

SP 5 WP 5 Prototype Version - System Configuration - HW Release - SW Release - Compiled by / Company MIZAR Date 12-15th March 2010

Form Progressive Numb. 12b Functional Component H&IW03 Reference Document D5.3.5

Test Type Test Objective Test Purpose Test Environment Multiple Integration Verification Correctness Test Site

Test Case 1: H&IW_02 and H&IW03: Combined Wrong Way Driver and Slippery Road Amsterdam Test Description

Prerequisite: - The successful results of all previous tests - Synchronization of the clock and Positioning - Installation of the WSN system in Torino Caselle (test area)


- Main PC (ECW-281B-ATOM) - SP2 applications - SP5 applications installed inside Main PC - Vanet Router dedicated gateway PC connected to main PC - 1 SF BMW and INCA vehicle 1 Mercedes S Class - 75 m of barrier

Description: The Test combines two applications Wrong Way Driver and Slippery Road. The first objective is to identify two event traffics: ghost drivers as well as slippery road. The second objective is test the correctness of the H&IW applications, which should send the corresponding warnings to the incoming drivers. Notice that the Slippery Road is an application originally part of SP4 applications. For this test was decided to use the opportunity of the DEMO of Amsterdam “Show case” to test the sub H&IW_03 application for Abnormal road Condition using not only the V-V communication of this application but also the V-I communication.




Wrong Way Driver Method 1. The SF-vehicles (INCA or Daimler) are going around the track 2. The vehicle BMW enters the slip road from the wrong entry See Figure diagram of the Amsterdam Test Track 3. Using one of the SP2 sensing technologies (Wireless Sensor Network) and data fusion module, the SAFESPOT system detects the vehicle travelling in the wrong direction 4. The H&IW application then generates a warning message which is sent to SF-equipped vehicles in the vicinity 5. The system also activates signs on the roadside to warn all approaching vehicles (including non SAFESPOT vehicles) that a vehicle is travelling the wrong way 6. The system switches on red traffic lights to warn the wrong way driver to stop.

Slippery Road Method 1. A slippery area was created with a special sheet that was sprayed constantly with water to make the surface slippery 2. The BMW vehicle detects the slippery area using the sensors installed on it 3. The BMW generates the warning messages and send them through VANET to the incomming vehicles (INCA and S-Class) 4. The INCA vehicle display the message to the driver using a monitor and the driver feels the haptic feedback (accelerator pedal and driver seat). 5. The BMW generates the warning messages and send them through VANET to the Infrastructure (RSU) 6. The RSU analyse the message received by the BMW and turn on the VMS.

Road Side Warning messages used for the Test

Slipperty road Application

VMS warning for Slippery road

Wrong Way Driver Application

VMS warning for drivers going in the correct direction

Traffic lights for warning wrong way driver, Wireless Sensor Network and Guard-Rail

Test site: Amsterdam




Figure 1 Diagram of the Amsterdam Test Track Legend

Wireless sensor nodes

VMS panel

BMW vehicle

INCA vehicle / DAIMLER vehicle

Wrong way direction

Correct direction

Traffic light

Lane lights




Requirements: - SP5_RQ02_36_v1.0: The system shall be able to decide if and to whom a message has to be send. - SP5_RQ07_19_v1.0: The system shall be able to determine the period of validity of the transmitted and received messages. - SP5_RQ16_36_v1.0: The system shall be able to query needed data from the LDM. - SP5_RQ17_36_v1.0: The system shall be able to receive and handle the LDM data. Use cases: - SP5_UC34

Expected Values / Results The requirements are met, specifically:

• All Real Sent and Received Messages correspond to the Expected Messages as indicated in tables. Obtained Values / Results

Recommendations • Memory in Vanet. All the logs should be canceled each day Issues faced • PC positioning (laptop) was changed • Logs in RSU should be canceled • Disable of Logs was done • Major issue there was indeed the time synchronization of the different systems causing the framework to disregard information send by the other vehicle. • A system restart in the vehicles including a new time synchronization was necessary after about 2.5 to 3 hours of testing. Results The 85 % of the cases the whole system works well during the 5 days of DEMO.

During the test demo an exception coming from the SP5 application was found. This exception occurred after some hours, around after 6 hours of work. The error implied the generation of HMI messages without geovalidity information, this effect was solved after the Amsterdam Demo (going back to Italy). The problem was solved after an update of the system. The reason was a lack of memory in the application PC, and a modification in the software SP5 was implemented (a garbage collector system).

A second exception was identified, in the software of the gateway (Road Side), this exception was very rarely the reason for this effect was related with the generation of logs and the amount of memory used. An update to the gateway was installed in the gateway on 25 May after this update the problem was solved. This test demonstrate the correct definition of the geovalidity area and the correct interpretation of the External Messages application of both vehicles (INCA and Daimler) Beyond the small issues the system demonstrated to work 5 continuously days, with running applications on Infrastructure and Vehicles Total: 500 repetitions of the test

Status Test Case1 OK Link to external annexed documentation (if any) Figure 4 Technical Test Result Amsterdam – wrong way driver (Infrastructure analysis)




Table 14 Rdep SDM creator Simulation

TEST FORM

SP 5 WP 5 Prototype Version RDep_Beta System Configuration RDep_Sim HW Release 1 SW Release 1 Compiled by / Company Andre Possani, DIBE Date September 2009

Form Progressive Numb. 13 Functional Component RDep learning SDM DB LDM + player

Reference Document SF_D5.3.4_Specifications_for_RDep_v3.4


Test Case 1: Safe Drive Map creator (learning phase) for a Deviation of Road Work Test Description

Description: The Road Departure application relies on the Safe Drive Map (SDM), which is a Data Base filled with information from vehicles passing through a black spot (Deviation for Road Work). Creating the SDM is the offline part of the Road Departure application. The LDM data player (MARS) is used as input (provides vehicle information) and the output is stored in the Safe Drive Map.

The following procedure are done at least 20 times: 1. In a two lane straight road, the vehicle drives in the left lane with a speed of 15m/s ± 5m/s. 2. As soon as the driver can see that a Road Work deviation is present ahead obstructing the left lane, it should lower its speed and do the correct maneuver while changing to the right

lane. The RDep_SDM_Creator program should monitor and record the vehicle’s trajectory.

Input LDM data player

LDM

Road Departure SDM Creation

Output Safe Drive Map




After having all the sample trajectories, the RDep_SDM_Creator will analyze the obtained trajectories and will calculate the values for the Safe Drive Map. Tools: - LDM data player (MARS) Requirements: - SP5_RQ16_36_v1.0 – The system shall be able to query needed data from the LDM

Expected Values / Results The requirements are me, specifically: 1. The RDep_SDM_Creator is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The RDep_SDM_Creator is able to store the trajectories of the vehicles and analyze them. 3. The values for the Safe Drive Map are calculated by the RDep_SDM_Creator.

Obtained Values / Results SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM OK The RDep application is able to store the trajectories of the vehicles and analyze them. OK The values for the Safe Drive Map are calculated by the RDep application OK Status Test Case1 OK Link to external annexed documentation (if any) -

Test Case 2: Safe Drive Map creator (learning phase) for a Dangerous Curve Test Description

Description: The Road Departure application relies on the Safe Drive Map (SDM), which is a Data Base filled with information from vehicles passing through a black spot (Dangerous Curve). Creating the SDM is the offline part of the Road Departure application. The LDM data player (MARS) is used as input (provides vehicle information) and the output is stored in the Safe Drive Map.

The following procedure should be done at least 20 times: 1. While approaching to the dangerous bend, the vehicle enters the dangerous curve keeping the average speed of 15m/s ± 5m/s; 2. The vehicle should drive through the curve in a comfortable situation and should never leave its lane. The RDep_SDM_Creator program should monitor and record the vehicle’s

trajectory.


LDM

Road Departure SDM Creation

Output Safe Drive Map




After having all the sample trajectories, the RDep_SDM_Creator will analyze the obtained trajectories and will calculate the values for the Safe Drive Map. Tools: - LDM data player (MARS) Requirements: - SP5_RQ16_36_v1.0 – The system shall be able to query needed data from the LDM

Expected Values / Results The requirements are met, specifically: 1. The RDep_SDM_Creator is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The RDep_SDM_Creator is able to store the trajectories of the vehicles and analyze them. 3. The values for the Safe Drive Map is calculated by the RDep_SDM_Creator.

Obtained Values / Results SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM OK The RDep application is able to store the trajectories of the vehicles and analyze them. OK The values for the Safe Drive Map are calculated by the RDep application OK Status Test Case2 OK Link to external annexed documentation (if any) -




Table 15 Rdep Runtime Simulation

TEST FORM

SP 5 WP 5 Prototype Version RDep_Alfa System Configuration RDep_Sim HW Release 1 SW Release 1 Compiled by / Company Andre Possani, DIBE Date September 2009

Form Progressive Numb. 14 Functional Component

RDep_runtime SDM DB LDM HMI_simulator ShemServer



Test Case 1: Road Departure Runtime application for a Deviation of Road Work Test Description

Description: Whenever a vehicle travel along the Road Work deviation, the RSU monitors the vehicle’s trajectory and Road Departure Runtime application predicts the near future trajectory and compares it with the SDM information. Depending on the risk analysis of the application, a Comfort, Safety or Critical warning is issued and sent to the HMI through the Message Manager.

The condition of "No Warning" is summarized by the following steps: 1. In a two lane straight road, the vehicle drives in the left lane with a speed of 15m/s ± 5m/s. 2a As soon as the driver can see that a Road Work deviation is present ahead obstructing the left lane, it lowers its speed and does the correct manoeuvre while changing to the right lane.No warning is issued.


LDM

Road Departure Safe Drive Map

Output Simulated HMI




In the condition of "Warning", the step 2 above is changed as follows: 2b - The vehicle travels through the monitored area without lowering its speed making it difficult to do the correct change lane manoeuvre; 2c - The vehicle travels too near the Road Work deviation area; Tools: - LDM data player (MARS) - Simulated HMI (simple green-yellow-red icon display, included in the RDep application) - ShemServer Requirements: - SP5_RQ02_36_v1.0 – The system shall be able to decide if and to whom a warning message has to be send - SP5_RQ04_36_v1.0 – The system shall be able to predict the vehicle’s trajectory - SP5_RQ16_36_v1.0 – The system shall be able to query needed data from the LDM

Expected Values / Results The requirements are met, specifically: 1. The Road Departure Runtime application is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The Road Departure Runtime application is able to predict the trajectory of the vehicle and compare it with the values of the SDM. 4. The Road Departure Runtime application is able to decide if and to whom a warning message has to be send.


Status Test Case1 OK Link to external annexed documentation (if any) SP5_RQ02_36_v1.0 The system shall be able to decide if and to whom a warning message has to be send OK SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle’s trajectory Partly OK Difficult and time consuming SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM OK

Test Case 2: Road Departure Runtime application for a Dangerous Curve Test Description

Description: Whenever a vehicle travel along the Dangerous Curve, the RSU monitors the vehicle’s trajectory and Road Departure Runtime application predicts the near future trajectory and compares it with the SDM information. Depending on the risk analysis of the application, a Comfort, Safety or Critical warning is issued and sent to the HMI through the Message Manager.


LDM

Road Departure Safe Drive Map

Output Simulated HMI




The condition of "No Warning" is summarized by the following steps: 1. While approaching to the dangerous bend, the vehicle enters the curve keeping the average speed of 15m/s ± 5m/s. 2a The vehicle should drive through the curve in a comfortable situation and should never leave its lane. No warning should be issued. In the condition of "Warning", the step 2 above is changed as follows: 2b The vehicle travels through the monitored area without respecting the speed limits, making it difficult to drive smoothly; 2c The vehicle travels too near the lane lines or even leaves its lane; Tools: - LDM data player (MARS) - Simulated HMI (simple green-yellow-red icon display, included in the RDep application) - ShemServer Requirements: - SP5_RQ02_36_v1.0 – The system shall be able to decide if and to whom a warning message has to be send - SP5_RQ04_36_v1.0 – The system shall be able to predict the vehicle’s trajectory - SP5_RQ16_36_v1.0 – The system shall be able to query needed data from the LDM

Expected Values / Results The requirements are met, specifically: 1. The Road Departure Runtime application is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The Road Departure Runtime application is able to predict the trajectory of the vehicle and compare it with the values of the SDM. 3. The Road Departure Runtime application is able to decide if and to whom a warning message has to be send.

Obtained Values / Results SP5_RQ02_36_v1.0 The system shall be able to decide if and to whom a warning message has to be send OK SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle’s trajectory Partly OK Difficult and time consuming SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM OK Status Test Case2 OK Link to external annexed documentation (if any)




Table 16 Rdep SDM creator Test site

TEST FORM

SP 5 WP 5 Prototype Version RDep_Final System Configuration RDep_Orbassano, RDep_Satory

HW Release 1 SW Release 1 Compiled by / Company Andre Possani, DIBE Date 16 / FEB / 2010 16 / ABR / 2010


RDep_learning SDM DB LDM VANET Positioning_sys



Test Case 1: Safe Drive Map creator (learning phase) for a Deviation of Road Work Test Description

Description: The Road Departure application relies on the Safe Drive Map (SDM), which is a Data Base filled with information from vehicles passing through a black spot (Deviation for Road Work). Creating the SDM is the offline part of the Road Departure application. The following procedure is performed at least 20 times: 1. In a two lane straight road, the vehicle should drive in the left lane with a speed of 15m/s ± 5m/s. 2. As soon as the driver can see that a Road Work deviation is present ahead obstructing the left lane, it should lower its speed and do the correct manoeuvre while changing to the right

lane. The RDep_SDM_Creator program should monitor and record the vehicle’s trajectory. After having all the sample trajectories, the RDep_SDM_Creator will analyze the obtained trajectories and will calculate the values for the Safe Drive Map.




Test sites: Tthe CRF Test Track in Orbassano (Italy) will be used for the Test Case 1. Requirements: - SP5_RQ16_36_v1.0 – The system shall be able to query needed data from the LDM - SP5_RQ49_19_v1.0 – The RSU subsystem shall be able to receive at minimum the current vehicle position and speed - SP5_RQ52_36_v1.0 – The system shall receive static vehicle data like width, length, type of vehicle, mass. - SP5_RQ61_36_v1.0 – The system must be able to receive data from the vehicles as well as the infrastructure

Expected Values / Results 1. The RDep_SDM_Creator should be able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The RSU subsystem should be able to provide at minimum the current vehicle position and speed 3. The system should receive static vehicle data like width, length, type of vehicle, mass 4. The system should be able to receive data from the vehicles as well as the infrastructure 5. The RDep_SDM_Creator should be able to store the trajectories of the vehicles and analyze them. 6. The values for the Safe Drive Map should be calculated by the RDep_SDM_Creator.

Obtained Values / Results SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM OK SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed OK SP5_RQ52_36_v1.0 The system shall receive static vehicle data like width, length, type of vehicle, mass OK SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure OK The RDep application should be able to store the trajectories of the vehicles and analyze them. OK The values for the Safe Drive Map should be calculated by the RDep application OK Status Test Case1 OK Link to external annexed documentation (if any)

Test Case 2: Safe Drive Map creator (learning phase) for a Dangerous Curve Test Description

Description: The Road Departure application relies on the Safe Drive Map (SDM), which is a Data Base filled with information from vehicles passing through a black spot (Dangerous Curve). Creating the SDM is the offline part of the Road Departure application. The following procedure should be done at least 20 times: 1. While approaching to the dangerous bend, the vehicle enters the dangerous curve keeping the average speed of 15m/s ± 5m/s; 2. The vehicle should drive through the curve in a comfortable situation and should never leave its lane. The RDep_SDM_Creator program should monitor and record the vehicle’s

trajectory. After having all the sample trajectories, the RDep_SDM_Creator will analyze the obtained trajectories and will calculate the values for the Safe Drive Map.




Test sites: Most probably, the LIVIC Test Track in Satory (France) will be used for the Test Case 2. Requirements: - SP5_RQ16_36_v1.0 – The system shall be able to query needed data from the LDM - SP5_RQ49_19_v1.0 – The RSU subsystem shall be able to receive at minimum the current vehicle position and speed - SP5_RQ52_36_v1.0 – The system shall receive static vehicle data like width, length, type of vehicle, mass. - SP5_RQ61_36_v1.0 – The system must be able to receive data from the vehicles as well as the infrastructure

Expected Values / Results The requirements are met, specifically: 1. The RDep_SDM_Creator is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The RSU subsystem is able to provide at minimum the current vehicle position and speed 3. The system receives static vehicle data like width, length, type of vehicle, mass 4. The system is able to receive data from the vehicles as well as the infrastructure 5. The RDep_SDM_Creator is able to store the trajectories of the vehicles and analyze them. 6. The values for the Safe Drive Map is calculated by the RDep_SDM_Creator.

Obtained Values / Results SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM OK SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed OK SP5_RQ52_36_v1.0 The system shall receive static vehicle data like width, length, type of vehicle, mass OK SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure OK The RDep application should be able to store the trajectories of the vehicles and analyze them. OK The values for the Safe Drive Map should be calculated by the RDep application OK Status Test Case2 OK Link to external annexed documentation (if any)




Table 17 Rdep Runtime Test site

TEST FORM

SP 5 WP 5 Prototype Version RDep_Final System Configuration RDep_Orbassano, RDep_Satory

HW Release 1 SW Release 1 Compiled by / Company Andre Possani, DIBE Date 26 / FEB / 2010 19 / ABR / 2010


RDep_runtime ùSDM DB LDM VANET Positioning_sys



Test Case 1: Road Departure Runtime application for a Deviation of Road Work Test Description

Description: Whenever a vehicle travel along the Road Work deviation, the RSU monitors the vehicle’s trajectory and Road Departure Runtime application predicts the near future trajectory and compares it with the SDM information. Depending on the risk analysis of the application, a Comfort, Safety or Critical warning is issued and sent to the HMI of the vehicle through the VANET. The condition of "No Warning" is summarized by the following steps: 1. In a two lane straight road, the vehicle should drive in the left lane with a speed of 15m/s ± 5m/s. 2a As soon as the driver can see that a Road Work deviation is present ahead obstructing the left lane, it should lower its speed and do the correct manoeuvre while changing to the right lane. No warning should be issued. In the condition of "Warning", the step 2 above is changed as follows: 2b The vehicle travels through the monitored area without lowering its speed making it difficult to do the correct change lane manoeuvre; 2c The vehicle travels too near the Road Work deviation area;




Test sites: The CRF Test Track in Orbassano (Italy) will be used for the Test Case 1. Requirements: - SP5_RQ02_36_v1.0 – The system shall be able to decide if and to whom a warning message has to be send - SP5_RQ04_36_v1.0 – The system shall be able to predict the vehicle’s trajectory - SP5_RQ16_36_v1.0 – The system shall be able to query needed data from the LDM - SP5_RQ49_19_v1.0 – The RSU subsystem shall be able to receive at minimum the current vehicle position and speed - SP5_RQ50_36_v1.0 – The system shall be able to transmit the warning to equipped vehicles - SP5_RQ52_36_v1.0 – The system shall receive static vehicle data like width, length, type of vehicle, mass. - SP5_RQ61_36_v1.0 – The system must be able to receive data from the vehicles as well as the infrastructure

Expected Values / Results The requirements are met, specifically: 1. The Road Departure Runtime application is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The Road Departure Runtime application is able to predict the trajectory of the vehicle and compare it with the values of the SDM. 3. The Road Departure Runtime application is able to decide if and to whom a warning message has to be send. 4. The RSU subsystem is able to provide at minimum the current vehicle position and speed 5. The system is able to transmit the warning messages to the equipped vehicles 6. The system receives static vehicle data like width, length, type of vehicle, mass 7. The system is able to receive data from the vehicles as well as the infrastructure

Obtained Values / Results SP5_RQ02_36_v1.0 The system shall be able to decide if and to whom a warning message has to be send OK SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle’s trajectory Not tested Difficult and time consuming SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM OK SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed OK SP5_RQ50_36_v1.0 The system shall be able to transmit the warning to equipped vehicles OK SP5_RQ52_36_v1.0 The system shall receive static vehicle data like width, length, type of vehicle, mass OK SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure OK Status Test Case1 OK Link to external annexed documentation (if any)




Test Case 2: Road Departure Runtime application for a Dangerous Curve Test Description

Description: Whenever a vehicle travel along the Dangerous Curve, the RSU monitors the vehicle’s trajectory and Road Departure Runtime application predicts the near future trajectory and compares it with the SDM information. Depending on the risk analysis of the application, a Comfort, Safety or Critical warning is issued and sent to the HMI of the vehicle through the VANET. The condition of "No Warning" is summarized by the following steps: 1 – While approaching to the dangerous bend, the vehicle enters the curve keeping the average speed of 15m/s ± 5m/s. 2a – The vehicle should drive through the curve in a comfortable situation and should never leave its lane. No warning should be issued. In the condition of "Warning", the step 2 above is changed as follows: 2b - The vehicle travels through the monitored area without respecting the speed limits, making it difficult to drive smoothly; 2c - The vehicle travels too near the lane lines or even leaves its lane; Test sites: The LIVIC Test Track in Satory (France) will be used for the Test Case 2.

Requirements: - SP5_RQ02_36_v1.0 – The system shall be able to decide if and to whom a warning message has to be send - SP5_RQ04_36_v1.0 – The system shall be able to predict the vehicle’s trajectory - SP5_RQ16_36_v1.0 – The system shall be able to query needed data from the LDM - SP5_RQ49_19_v1.0 – The RSU subsystem shall be able to receive at minimum the current vehicle position and speed - SP5_RQ50_36_v1.0 – The system shall be able to transmit the warning to equipped vehicles - SP5_RQ52_36_v1.0 – The system shall receive static vehicle data like width, length, type of vehicle, mass. - SP5_RQ61_36_v1.0 – The system must be able to receive data from the vehicles as well as the infrastructure

Expected Values / Results The requirements are met, specifically: 1. The Road Departure Runtime application is able to query the LDM for all the static and dynamic information of the vehicles present in the coverage area. 2. The Road Departure Runtime application is able to predict the trajectory of the vehicle and compare it with the values of the SDM. 3. The Road Departure Runtime application is able to decide if and to whom a warning message has to be send. 4. The RSU subsystem is able to provide at minimum the current vehicle position and speed 5. The system is able to transmit the warning messages to the equipped vehicles




6. The system receives static vehicle data like width, length, type of vehicle, mass 4. The system is able to receive data from the vehicles as well as the infrastructure

Obtained Values / Results SP5_RQ02_36_v1.0 The system shall be able to decide if and to whom a warning message has to be send OK SP5_RQ04_36_v1.0 The system shall be able to predict the vehicle’s trajectory Not tested Difficult and time consuming SP5_RQ16_36_v1.0 The system shall be able to query needed data from the LDM OK SP5_RQ49_19_v1.0 The RSU subsystem shall be able to receive at minimum the current vehicle position and speed OK SP5_RQ50_36_v1.0 The system shall be able to transmit the warning to equipped vehicles OK SP5_RQ52_36_v1.0 The system shall receive static vehicle data like width, length, type of vehicle, mass OK SP5_RQ61_36_v1.0 The system must be able to receive data from the vehicles as well as the infrastructure OK Status Test Case2 OK Link to external annexed documentation (if any)




Table 18 SMAEV_01 – Functional test of components – Bench (CRF)

TEST FORM

SP 5 WP 5 Prototype Version 0.3 System Configuration - HW Release 0.1 SW Release 0.3 Compiled by / Company Filippo Visintainer/CRF Date 12/04/2010

Form Progressive Numb. 19b Functional Component SMAEV 01 Reference Document D5.3.5 Test Type Test Objective Test Purpose Test Environment

Integration Verification Correctness Bench

Test Case 1: Functional testing Test Description

Objective: The functional test aimed at verifying that all components worked according to the specifications, without failure. Description: Figure 2 shows the logical architecture of SMAEV application, taken from D5.4.1. The method of the functional testing was to highlight the most critical functionalities related to each sub-system and test each one in the laboratory. The results reporting followed a simple checklist, stating for each functionality the correctness operation or the failure.




Figure 2 Logical architecture of SMAEV application

Expected Values / Results Correct operation for each functionality

Obtained Values / Results HMI testing: By navigating through the HMI client browser, all functionalities were correctly performed by the HMI server SMAEV HMI functionalities

Functionality Correct Failure Use Case selection (tab panel switching)




Manual entering of data: all selectable options are actually valid, no fault, no conflict Manual entering of data: all selectable options are actually valid, no fault, no conflict Start/stop of application Switch from UC12 to UC51 Data correctly displayed to User (RSU message, current message) Start/stop sync (UC61)

Local Dynamic Map: Data retrieval by SMAEV application from LDM was correctly performed

Data Correct Failure Vehicle position Vehicle speed HMI messages from VANET

Variable Message Sign: A subset of the data inserted via HMI was correctly transmitted via the serial port and displayed on the VMS panel. VANET Upon reception of an UDP message the router sent it to the VANET and another vehicle received it. VANET message transmission

Message Correct Failure Assistance Vehicle (ego-) beacon HMI Message, periodic HMI Message, event

Message reception

Message Correct Failure SAFESPOT vehicle beacon RSU beacon HMI messages from VANET

Conclusions: All SMAEV functions (HMI, LDM, VMS and VANET) operated as designed, i.e. responding to the specifications of D5.3.5. The system was then ready for outdoor testing.





Table 19 SMAEV_01 Test site (CRF)

TEST FORM

SP 5 WP 5 Prototype Version Final System Configuration HW Release 0.1 SW Release 0.3 Compiled by / Company Filippo Visintainer Date 20/05/2010

Form Progressive Numb. 19 Functional Component SMAEV 1 Reference Document D5.3.5 Test Type Test Objective Test Purpose Test Environment

Integration Verification Correctness Test Site

Overview of application testing The validation aimed at verifying the correct behaviour of the application. Though the final use case evaluation is left to WP5.6, the test scenario are based on a Use Case structure, mainly because of practical reasons, since each Use Cases implied a different scenarios and thus different preparation procedures. Some of these tests were actually supposed to characterise the overall application, for instance the results of ‘event warning’ can be generalised also for other Use Cases; whereas some results are more specific, such as the ones related to ‘slow vehicle’ UC.

The test scenarios are briefly resumed:

Test Scenario Title UC Test Case

1 Slow Assistance Vehicle moving on the road UC11 1

2 Assistance Vehicle signalling an event during its trip UC12 2

3 Event warning – vehicle stopped UC51 3

4 Entering UC51 from UC12: continuity of service UC12, UC51 4

5 Assistance vehicle – RSU cooperation UC61 5

Each test scenario included multiple trials, as reported in the test forms. The trials were carried out at Centro Sicurezza Fiat test track and on Brescia-Padova Motorway. They involved the following SAFESPOT elements:

• CRF Assistance Vehicle demonstrator, Fiat Croma, called AV • CRF SAFEPROBE demonstrator, Fiat Bravo, called SF • MIZAR Roadside Unit (limited to UC 61 testing) , called RSU

Data were logged in both the AV and the SF vehicle and then re-processed offline to analyse the experiments.




Test Case 1: Slow Assistance Vehicle moving on the road Test Description

Objective: to test UC11 “Slow Vehicle” AV = Assistance Vehicle - i.e. CRF COSSIB demonstrator vehicle, integrating SP5-SMAEV01 sub-application SAFESPOT vehicle = CRF SAFEPROBE or SCOVA demonstrator vehicle, integrating SP5 client Description: 1. The AV moves at slow speed on lane 1. UC11 (Safety margin for maintenance vehicle on snow removal or salting operation) functionality is switched on 2. The operator inside the AV starts UC11 from the AV HMI module and gives corresponding warning of slow vehicle (due, for example, to road maintenance operation) 3. A SAFESPOT vehicle comes on lane 2 from behind, heading the same direction of the AV at higher speed

Trials Trials with the following characteristics have been performed: A. 1 test track loop al varying the speed values of AV (e.g. 30, 20 km/h, stopped); SF vehicle queuing behind B. 3 trials with SAFESPOTvehicle overtaking at 3 different speed values (40, 60, 80 km/h); AV at 30 km/h C. 3 trials with SAFESPOT vehicle arriving at different speed values (40, 60, 80 km/h) and slowing down as soon as the VANET message is received and interpreted by the driver; AV at

30 km/h Test Sites: - Centro Sicurezza Fiat – CRF test track




Requirements: - SP5_RQ02_36_v1.0 - SP5_RQ06_19_v1.0 - SP5_RQ07_19_v1.0 - SP5_RQ12_33_v1.0 - SP5_RQ14_33_v1.0 - SP5_RQ16_36_v1.0 - SP5_RQ17_36_v1.0 Use cases: SP5_UC51 -P5_UC11

Expected Values / Results The results should validate that the requirements are met, specifically: Results for every trial - A warning of the following kind: “Operation type Lane Speed” is displayed on the display of an incoming SAFESPOT vehicle. - A subset of this message is displayed on the VMS panel placed on top of the AV demonstrator, so that also non-SAFESPOT vehicles can see the warning. Results specific for trial A - By varying the AV speed, the attribute "Speed" in the warning should change accordingly. This is verified by comparing the message in step 3 and step 5. - When the vehicle stops the message should autonomously change into: “Operation type Lane STOPPED VEHICLE AdditionalWarning”; Results specific for trial B - VANET message is received independently of the relative speed between SAFESPOT vehicle and AV. Results specific for trials C and D: VANET versus VMS-only - Assuming a straight road, upon reception of signal the SAFESPOT vehicle should be able to slow down and - queue behind the AV (i.e. the two vehicles should proceed at the same speed). - The distance between AV and SF vehicles when they are able to queue will be considered: the higher the distance, the better the results. - The comparison between trial C and D will give an indication of the added value of VANET with respect of VMS only. The trials will be made on two different lanes for safety reasons.

Obtained Values / Results Results for every trial For single-run scenarios, out of 6 trials, for 5 times UC11 warning: “SPECIAL VEHICLE-SPEED X KM/H” was displayed on the SAFESPOT vehicle HMI, where X was the speed of the Assistance vehicle rounded to multiples of ten (20, 30, 40, 50, km/h, etc). Results specific for trial A Varying the speed the value changed accordingly. When the vehicle stopped, the message “SPECIAL VEHICLE-STOPPED” was displayed, in compliance with specifications. The wording “SPECIAL VEHICLE” was also displayed on the VMS panel. In one trial the message was received but not processed by the application.




For the queuing scenario, the SF vehicle followed the Assistance Vehicle. The latter changed its speed and was continuously sending UC11 message to the vehicle behind. The following graphs summarize the queue scenario (left image, showing that both vehicles were running at the same speed) and the slow vehicle signalling results (right image, AV speed versus signalised speed).

Queue scenario: SF vehicle speed (circles) compared to AV speed (dashes)

0

10

20

30

40

50

60

0 20 40 60 80 100 120 140 160 180

Time [s]

Spee

d C

ar [k

mh]

Speed Bravo Speed Croma

Actual AV speed (green line, smooth) compared to the speed suggested by the AV and displayed on the SF vehicle HMI (red line, step-like)

0

10

20

30

40

50

60

0 20 40 60 80 100 120 140 160 180

Time [s]

Spee

d [k

m/h

]

The results are in line with the expectations. The signalised speed, as designed in the algorithms, rounds the actual speed down to the nearest ten, e.g. an actual speed of 58 km/h yields a signalised speed of 50 km/h. Moreover, in the transitions between images, the driver experienced hardly any blinking (the latter was the primary concern in this scenario). Only some fast changes happened at the end of the trial (time > 130s in the graph) but this was also expected, as the changes were intentionally exaggerated with respect to ordinary scenarios, and overall the signalling behaviour reflects the actual speed one. As a further improvement, for instance to filter out event the fast changes experienced in the time interval 15-40 s, an hysteresis could be introduced in the algorithms. Results specific for trial B The results confirm that the message was correctly received independently from the relative speed, for relative speed values ranging between 0 and 50 km/h.




Results specific for trials C and D: VANET versus VMS-only The VANET messages were received at a distance from 55m to 100 m between the SF vehicle and the Assistance Vehicle, sufficient for the application needs, and giving an added value with respect to the Assistance vehicle VMS. The latter indeed has a readability distance of 40-50 m (see Test Case 3).


Test Case 2: Assistance Vehicle signalling an event during its trip Test Description

Objective: to test UC12 “Trip Signalling” Description: 1. A SAFESPOT vehicle is running on lane 1 2. The AV, with UC12 (Assistance vehicle patrolling or signalling a traffic event on a road) active comes from far away, heading the same direction at higher speed, and overtakes the

SAFESPOT vehicle (virtually the AV is moving to a site where an event has happened)

Trials Trials will be performed varying the relative speed between AV and SF vehicle. Test Sites: Centro Sicurezza Fiat – CRF test track Requirements: - SP5_RQ02_36_v1.0 - SP5_RQ06_19_v1.0 - SP5_RQ07_19_v1.0 - SP5_RQ12_33_v1.0




- SP5_RQ14_33_v1.0 - SP5_RQ16_36_v1.0 - SP5_RQ17_36_v1.0 Use cases: - SP5_UC51 -P5_UC12

Expected Values / Results The requirements are met, specifically: Results for every trial A warning of the following kind: - “ATTENTION Event type, Event relative position, Event direction lanes”; is displayed on the display the SAFESPOT vehicle. - A subset of this message should be correctly displayed on the VMS panel placed on top of the AV demonstrator, so that also non-SAFESPOT vehicle can see the warning. Results comparing the trials - The VANET warning should be received when the vehicle is still behind the SAFESPOT vehicle, independently from the speed. This should prove the improved effectiveness of the use of VANET with respect to VMS only. Indeed - with VMS only there is a difference in perception between the conditions “AV coming from behind” and “AV running in front”, as in the former the signal is read through the rear-view

mirror and the latter is direct in the scene in front. This difference in perception is more critical the higher the relative speed. - with the addition of VANET, the signalling is the same on both situations, and it is displayed on the vehicle HMI

Obtained Values / Results 6 trials have been performed, 2 for each value of the AV speed (40,60, 80 km/h), while the SF vehicle was proceeding at 30 km/h. Correctness of warning, including geovalidity validation The generic warning of accident was correctly received. The following graphs report some shots of the tested scenario of rear geovalidity as recorded by the data logger (just for representation sake, the shots are taken every 5 seconds ). The objective of this image is to to show the geo-validity shape and orientation: as can be seen the latter varies according to the AV heading, as designed in the algorithm. The squared dots represent the Assistance Vehicle approaching and overtaking the SAFESPOT Vehicle (circular dot), running clockwise around the track. When the AV overtook the SF vehicle, the latter entered the geovalidity area – i.e. the plotted rectangle following the AV– and HMI message reporting an accident was correctly displayed to the SF vehicle.




Geovalidity Scenario 3 Test 2 (speed AV = 60km/h, geovalidity = 150m)Map source: Google

VANET range testing and comparison to VMS The VANET warning was received when the vehicle was still behind the SAFESPOT vehicle, independently from the speed. Probably due a shielding effect of the VMS panel itself, the range of VANET was not symmetrical, being around 30-50 m in front of the AV and up to 150 m behind the AV. Therefore, the perceived range with the AV coming from the rear was limited. Nevertheless the range can potentially be considered symmetric (ca. 150 m) once the antenna placement is improved. With VMS only, the incoming AV is seen only through the rear-view mirror and the range of readability is only a few tenths of meters.





Test Case 3: Event warning – vehicle stopped Test Description

Objective: to test UC51 “Event warning” (especially focussing on spatial discriminiation) Description: 1. The AV is stopped and simulates accident signalling. 2. A SAFESPOT vehicle passes by.

Trials A. With VMS and VANET, varying the absolute speed of SF vehicle (from 50 km/h to 90 km/h of relative speed). B. With VMS only, varying the absolute speed of SF vehicle (from 50 km/h to 90 km/h of relative speed). Test Sites:Centro Sicurezza Fiat – CRF test track Requirements: - SP5_RQ02_36_v1.0 - SP5_RQ06_19_v1.0 - SP5_RQ07_19_v1.0 - SP5_RQ11_33_v1.0 - SP5_RQ12_33_v1.0 - SP5_RQ13_33_v1.0 - SP5_RQ14_33_v1.0 - SP5_RQ16_36_v1.0 - SP5_RQ17_36_v1.0




Use cases: SP5_UC51

Expected Values / Results The requirements are met, specifically: Results for every trial - A warning is correctly displayed on the display of an incoming SAFESPOT vehicle. - A subset of this message should be correctly displayed on the VMS panel placed on top of the AV demonstrator, so that also non-SAFESPOT vehicle can see the warning. Capability to stop - Upon reception of a signal the SAFESPOT vehicle should be at such a distance to be capable to slow down and stop. - The distance between the accident (or a reference point) and the SF vehicle when it is stopped: the higher the distance, the better the result. - The comparison between trial sets A and B will give an indication of the added value of VANET with respect of VMS only, assuming a straight road. The trials will be made on two

different lanes for safety reasons.

Obtained Values / Results Correctness and accuracy For each trial 3 messages were sent by the AV: A Comfort message, a Safety message and a Critical one. The geovalidity width (GW) was set such as to cover only one lane (12 to 14 m width depending on vehicle position relatively to the lane), whereas the geo-validity length (GL) was chosen according to the different scenarios (Urban, Rural, Motorway)

Geovalidity rectangle length (m) Priority Urban scenario Rural scenario Motorway scenario Comfort Message 50 150 250 3 Safety Message 35 100 170 2 Critical Message 25 65 150 1

Overall, 45 trials were performed, 16 for Urban. 18 for rural and 11 for motorway scenarios. The SF vehicle speed spanned the range 20-90 km/h with a limit of 50 km/h for the urban scenarios. In 28 cases the SF vehicle entered the geovalidity areas (positive cases) in 17 the SF vehicle, proceeding in the outer lane, did not enter the geovalidity area and thus was not supposed to display the message on the HMI (true-negative). Overall, the message “ATTENTION CAR ACCIDENT” was correctly received and displayed in the receiving car HMI. The table hereafter reports the details of the results per specific scenario.




OverallComfort Safety Critical Tot. Urban Comfort Safety Critical Tot. Rur. Comfort Safety Critical Tot. Motor.

Trials 45Total messages 135 16 16 16 48 18 18 18 54 11 11 11 33Positive cases 84 11 11 11 33 13 13 13 39 4 4 4 12Missing alarms 3 0 0 0 0 1 1 1 3 0 0 0 0Negative cases 51 5 5 5 15 5 5 5 15 7 7 7 21False alarms 2 0 0 0 0 0 0 0 0 0 0 2 2

Motorway scenarioUrban scenario Rural scenario

The table proves that, with the actual positioning system having an accuracy below 1m , message geovalidity can effectively discriminate the lanes. As can be seen, the only exceptions happened in one trial with three messages that were supposed to be displayed were not actually shown (missing alarms), and in another trial giving 2 false alarms. VANET range versus VMS: a comparison At Centro Sicurezza, for urban and extra-urban speed values the messages were received between 240 and 300 m, far beyond the distance at which the VMS panel was read (30-55 m).

0

50

100

150

200

250

300

350

400

0 10 20 30 40 50 60 70 80 90 100Speed [km/h]

Dis

tanc

e at

whi

ch th

e VA

NET

mes

sage

is re

ceiv

ed

[m]

0

50

100

150

200

250

300

350

400

0 10 20 30 40 50 60 70 80 90 100Speed (km/h)

Dis

tanc

e at

whi

ch th

e VM

S is

read

(m)




Data sampling : LDM-related time lag estimation Overall, there was between 7 ms and 180 ms time lag between message reception by the VANET router and correct LDM reading by the External message Application. The mean value was 56 ms and a standard deviation of 35 ms. This time interval is mostly due to the periodicity of LDM reading by the application (T=100 ms) which introduces an average of 50 ms time lag. Only two exceptions (3% of the total sampling) reported a time interval of 3s and 5s respectively. The results are thus in line with the expected system features and give a negligible contribution to the overall system performance. Display and driver Reaction to Comfort Area messages HMI messages were displayed just a few meters after the geovalidity area, whereas the driver reaction was between 0,5 and 1 s. The following graphs plot the driver reaction in terms of time lag and distance covered between the triggering of the message display and the driver’s reaction.

0,4

0,5

0,6

0,7

0,8

0,9

1

40 45 50 55 60 65 70 75 80 85

Speed [km/h]

Tim

e be

twee

n di

spla

y co

mm

and

and

driv

er re

actio

n [s

]

1st msg 2nd msg 3rd msg

5

10

15

20

40 45 50 55 60 65 70 75 80 85

Speed [km/h]D

ista

nce

cove

red

befo

re R

eact

ion

[m]

1st msg 2nd msg 3rd msg

It can be seen that the reaction to the first message (comfort) was slower and more speed-dependent, but this effect deserves further investigation. Moreover, the distance covered before reaction does not show a strong dependence on speed. Overall, the collected data suggest, for future deployment, to use a geovalidity setting with an addition of 15-20 m independently of the speed, for instance setting a geovalidity to 120 m if the message has to be read below 100 m).





Test Case 4: Entering UC51 from UC12: continuity of service

Test Description Objective: to test the combined scenario UC12 - UC51 (continuity of service) Description: 1. The SAFESPOT vehicle moves on lane 1, at about 50 km/h. It is heading towards an accident, without knowing it. 2. The AV vehicle approaches and overtakes the SAFESPOT vehicle, sending an UC12 message. Both vehicles proceed together on two different lanes (for safety reasons, in this testing phase) keeping at a distance of about 100-150 m from one another. 3. The AV stops 50m before the accident position and, in a single step operation on the HMI (like pressing a button) it switches to UC51. 4. The SAFESPOT vehicle is still coming behind at 50 km/h. As soon as the SAFESPOT vehicle receives the warning of UC51 it stops.

Trials Trials shall be performed with and without the shortcut UC12 => UC51. Test Sites: Centro Sicurezza Fiat – CRF test track Requirements: - SP5_RQ12_33_v1.0 - SP5_RQ14_33_v1.0 - SP5_RQ16_36_v1.0 - SP5_RQ17_36_v1.0 - Use cases: SP5_UC51, SP5_UC12




Expected Values / Results The requirements are met, specifically: Operator HMI - The operator was able to switch from UC12 to UC51 in sequence without interruption. Capability to stop - Having received the UC12 message, the vehicle should queue behind the safety car (at a distance D determined in the trial). - Upon reception of a new signal (switching from UC12 to UC51 signal), the SAFESPOT vehicle should slow down and stop. - The distance between the accident (or a reference point) and the SF vehicle when it is stopped. - The continuity of service is met if the vehicle is able to stop before the accident. - This will be tested with and without a quick link between Use Cases in the HMI, in order to quantitatively measure the effectiveness of this link.

Obtained Values / Results The test was carried out at 45 km/h.The requirements were met, the transition between UC12 to UC51 signalling in the operator HMI was correctly working, however its effectiveness was proved more in terms of comfort of the operator than in terms of signalling improvement, as in both cases (with and without the quick link on the HMI) the SF vehicle driver coming after the AV was warned on time to eventually slow down and stop. This is generally a good result in terms of effectiveness of SMAEV application: the person on the AV can easily switch between Use Cases providing continuity of service. A possible further evaluation could be with higher speeds and with actual operators on the Assistance Vehicle, a scenario that could be feasible for instance in a Field Operational Tests.


Test Case 5: Assistance Vehicle - RSU cooperation Test Description

Objective: to test UC61 (application-based multihop) Description: 1.The AV approaches a fixed RSU (100-200 m). The RSU is sending an 'RSU HMI-Message' from the application, for instance Hazard and Incident Warning. 2. As soon as the AV operator sees, through the HMI, what message of type ‘RsuHMIMessage’ the RSU is sending, the operator decides to stop the AV. Distance is expected to be 100-200 m before the RSU. 4. The operator decides to switch on amplification (UC61) and give the same warning as the RSU. 5. A SF vehicle probe passes by




Trials Trials have been performed varying the speed at which the AV approaches the RSU. Test Sites: - Centro Sicurezza Fiat – CRF test track - Brescia-Padova Motorway Requirements: - SP5_RQ02_36_v1.0 - SP5_RQ12_33_v1.0 - SP5_RQ16_36_v1.0 - SP5_RQ17_36_v1.0 Use cases: SP5_UC61

Expected Values / Results Amplification effectiveness - Let the RSU range be Dmax, assuming that the physical communication devices have an equal range on RSU and AV, the AV should ideally be capable to extend the range up to a

distance Damp= 2Dmax. Realistically this distance is expected to be less. One of the main causes is that the AV has to see the message, decide to amplify it and stop. Therefore it will not stop exactly at the border of the range, and most of the times it will not move backwards to reach the border. It makes sense to think that Damp will be higher the lower the initial speed of the AV. - These trials aim in fact at giving an estimation of Damp close not too far from the real conditions.




Obtained Values / Results Centro Sicurezza test track The following tests show how it is possible, via the UC61, to amplify the RSU signalling and cover the whole extension of the test Track. The roadside unit was sending HMI messages. The application inside the assistance vehicle, placed at 270 m from the RSU and receiving the HMI messages, created its own HMI messages with the same content as the one present in the RSU. In practice it acted as bridge for the warning message, performing a multi hop at application level. The SF vehicle ran multiple loops around the track the track, receiving the HMI messages from both sources. The image shows the points where the HMI messages were received.

HMI messages received by vehicle and RSU based on 11 laps of the test track

HMI RSU HMI AV AV postion RSU position

The picture hereafter reports only the assistance vehicle. An interesting aspect is the coverage of an area which is partially shielded by trees. The blue point on top right shows the farthest point that could be reached by an HMI message from the AV, which is 530 m away from the RSU.




Coverage of the multihop

HMI AV AV position RSU position

Brescia Padova testing Brescia Padova testing was meant to extend UC61, as implemented in the Centro Sicurezza, through a longer and rectilinear path. In the motorway Test Site the Assistance vehicle could stop in a free emergency space 350 m away from the roadside unit, close to a subway. Another aspect noted was the elevation, which was not constant along the road segment, but varied between 73,6 and 86 m as reported in the following graph.




Elevation A4from 45.430921375 , 10.639871625to 45.430607625 , 10.664430125

50

60

70

80

90

100

110

120

130

140

150

160

170

180

190

2000-65-129

-194

-259

-324

-388

-453

-518

-583

-647

-712

-777

-842

-906

-971

-1036

-1101

-1165

-1230

-1295

-1360

-1424

-1489

-1554

-1619

-1683

-1748

-1813

-1878

-1942

-2007

Position [m]

Elev

atio

n [m

]

AV position RSU position




Message plots Roadside unit messages were received up to 750 m away, as shown by the following graph. The picture represents the message received by the SF vehicle in different positions.

HMI Messages RSU

-1000 -800 -600 -400 -200 0 200 400 600

Position [m] with respect to the direction of motion:negative values indicate before the RSU and positive values indicate after RSU

(RSU position = 0m)

RSU position

test 1

test 7

test 6

test 5

test 4

test 3

test 2




HMI Messages AV, replicating RSU Messages

-800 -700 -600 -500 -400 -300 -200 -100 0

Position [m] with respect to the direction of motion:negative values indicate before the RSU and positive values indicate after RSU

(RSU position = 0m, AV position = -350m)

RSU positionAV position

test 1

test 7

test 6

test 5

test 4

test 3

test 2





Table 20 SMAEV_01 Test site (SODIT)

TEST FORM

SP 5 WP 5 Prototype Version FinalFinal System Configuration

HW Release 0.1 SW Release 1 Compiled by / Company Nicolas Etienne/ SODIT Date 3-5 March 2010

Form Progressive Numb. 20b Functional Component SMAEV 1 Reference Document D5.3.5

Test Type Test Objective Test Purpose Test Environment Integration Verification Correctness Satory Test Track

Test Case 1: Assistance vehicle provides information Test Description

Description: The aim of this test was to validate the behaviour of the system when an assistance vehicle provides information for an event on the road. - the AV (Assistance Vehicle –ie the RSU) enters an event concerning what is happening on the road (car accident simulation). - The OBU approaching the position on the event will receive a warning message describing the current event. The OBU will have a speed between 50 and 110 km/h. Test sites: Satory Test track Requirements: RQ02_36_v1.0, RQ06_19_v1.0, RQ07_19_v1.0, RQ14_33_v1.0 Use cases: SP5_UC01 case

Expected Values / Results The requirements are met, specifically: - Assistance Vehicle can enter through HMI application the status of the event, lane, user message - The OBU shall receive and display the incoming warning information. - The distance between the position of the event and the position where the OBU received the message shall be more thant 400 meters.




Obtained Values / Results The application test was successful. The obtained value were:

• We always received the warning message for a car speed in the range between 50 and 110 km/h. • The distance between the emitter (RSU) and the OBU (when the car received the event) was estimated to be about 600 meters. • No messages were lost during the test. • The message received on the OBU contained the user message that was dynamically entered by the Assistance vehicle user.





Table 21 SMAEV_02 Test site

TEST FORM

SP 5 WP 5 Prototype Version Final System Configuration Prototype HW Release 0.1 SW Release 2 Compiled by / Company Nicolas Ettiene/SODIT Date November 2009

Form Progressive Numb. 20b Functional Component SMAEV 2 Reference Document D5.3.5


Test Case 1: Emergency vehicle crosses intersection Test Description

Description: The aim of this test is to validate the behaviour of the system when an emergency vehicle wants to cross an intersection in urban area. Nominal case. - the EV (Emergency Vehicle) Driver switches its status by using the SMAEV_02 HMI. - the RSU (Road side unit) is able to detect the EV and to handle its request.

Test sites: - SATORY




Requirements: - SP5_RQ30 Use cases: SP5_UC52 case

Expected Values / Results The requirements are met, specifically: - The EV receives information telling him its request is being processed ("YOUR REQUEST IS BEING TREATED") and after receive a confirmation message: "YOU ARE HAVING

PRIORITY". - The EV Status changes to "Emergency status". - The SF vehicles are informed of the emergency situation and can take care of the EV (they receive warningHMIMessage). - The PRM (Priority Request Manager) manages the behaviour or the TLC (Traffic Light Controller). - The TLC is set in such a way that vehicles and pedestrians lights are red. Green for the EV arrival lane. When the EV has left the crossing area, the TLC must return to normal behaviour.

Obtained Values / Results The RSU received the priority request and was able to deliver the priority as wished. The OBU received a confirmation message to go through the crossing The TLC switched between normal sequence to Emergency priority sequence giving the priority to the vehicle. After crossing the intersection, the TLC went back to its normal behavior.


Test Case 2: Failure mode status Test Description

Description: The aim of this test is to validate the behaviour in warning failure mode. The EV asks for the emergency status. One of the following failures occur: - Bad data into LDM - Cannot localize the EV - TLC in security mode - VANET does not work. Test sites: - SATORY Requirements: - SP5_RQ30




Use cases: - SP5_UC52 case

Expected Values / Results The requirements are met, specifically: - No change in the EV Status - Other SF vehicle does not receive any information about the EV arriving. - PRM is not able to take any decision concerning the EV request.

Obtained Values / Results The VANET wasn’t working, so we could not really test this situation.

Status Test Case2 To Be Done Link to external annexed documentation (if any)




Appendix B - Information about the H&IW test results Table 22 Verification Table RFID – Ghost driver two cars in opposite direction Verification Table

Test N: Ghost driver: two cars in opposite direction.

No Ghost Driver

LDM-Motorvehicle Table

LDM-Traffic Event

No RFID Message Value Field Value Field Value

3 antennaPosition X:60481644 Y:360064822 geom

X:60481644 Y:360064822 event.cause 35

4 vehicleID 200142303 nodeid 200142303 message

Ghost Driver From RFID

sensor 5 typeOfGoods 0 goodstype 0 status 1

6 VehicleDescription 0 vehicledescription 0

7 vehicleWidth 0 vehiclesize_width 0 8 VehicleLength 0 vehiclesize_length 0 9 VehicleHeight 0 vehiclesize_height 0 10 vehicleMass 0 (unit 0) vehiclemass 0 11 ghostDriver 1 heading 0




Table 23 Verification Table RFID – Shadowing 2 Ghost Drivers

Verification Table

Test N: 2 Test #3: Shadowing 2 Ghost Drivers Ghost Driver 1

No Ghost Driver


LDM-Traffic Event

No RFID Message Value Field Value Field Value


X:60481644 Y:360064822 event.cause 35


Ghost Driver

From RFID sensor

5 typeOfGoods 0 goodstype 0 status 16 VehicleDescription 0 vehicledescription 0 7 vehicleWidth 0 vehiclesize_width 0 8 VehicleLength 0 vehiclesize_length 0 9 VehicleHeight 0 vehiclesize_height 0

10 vehicleMass 0 vehiclemass 0 11 ghostDriver 1 heading 0

Test N: 2 Test #3: Shadowing 2 Ghost Drivers Ghost Driver 2

No Ghost Driver


LDM-Traffic Event

No Messagio RFID Value Field Value Field Value


X:60481644 Y:360064822 event.cause 35


Ghost Driver

From RFID sensor

5 typeOfGoods 0 goodstype 0 status 16 VehicleDescription 0 vehicledescription 0 7 vehicleWidth 0 vehiclesize_width 0 8 VehicleLength 0 vehiclesize_length 0 9 VehicleHeight 0 vehiclesize_height 0

10 vehicleMass 0 vehiclemass 0 11 ghostDriver 1 heading 0




Table 24 Obstacle-Pedestrian on the CRF Test Track – Thermal imaging cameras (provided by VTT) Test 1 Parameters measured # Images.

%

total of Images analysed 597 100 Value Total Images.>40% 346 57,96

animal 1 Total image with confidence >40% 0 0 pedestrian 3 Total image with confidence >40% 64 10,72

car 5 Total image with confidence >40% 282 47,24 Real Parameters Images containing moving person 372 62,31 Images containing moving vehicles 0 0 Images containing moving animals 0 0 Test 2 Parameters measured # Images.

%


animal 1 Total image with confidence >40% 237 18,54 pedestrian 3 Total image with confidence >40% 134 10,49

car 5 Total image with confidence >40% 639 50 Real Parameters Images containing moving person 156

12.21

Images containing moving vehicles 0 0 Images containing moving animals 0 0 Test 3 Parameters measured # Images.

%


animal 1 Total image with confidence >40% 98 6,06 pedestrian 3 Total image with confidence >40% 150 9,28

car 5 Total image with confidence >40% 1076 66,58 Real Parameters Images containing moving person 1284

79,46

Images containing moving vehicles 0 0 Images containing moving animals 0 0




Table 25 Results Wrong Way Driver – CRF – Correctness

# of correct Detections for first line/No total detections

# of Correct detection second line

Detection Traffic event first

line/No Total

Detection Traffic event second line /No total of events

# of Messages

generated/# of events

# of visualization of

the message inside

the vehicle inside area

Detection System 1

9/10 6/10 8/10 8/10 52/13 13

Detection System 2

10/10 10/10 10/10 10/10 40/10 10

Table 26 Wrong Way Driver – CRF – Communication Test Distance RSU – V (m)

Vanet – Messages Received in RSU & Vehicle

Reception of HMI Message

Line of Sight

38.22 Successful Successful yes 104 Successful Successful yes 299.57 Successful Successful yes 321.81 Successful Successful yes 395.18 Successful Successful yes 478.57 Successful Successful yes 484.21 Successful Successful yes 417.71 Successful Successful yes 307.23 Successful Successful yes 209.91 Successful Successful yes 468.41 No communication No communication Not at all (in the area

there were some trees between RSU and Vehicle)




Installed Sensors in the Pila highway Hyper Lan Communication for remote control of the RSU

Installation of the elements in the BS-PD highway

Figure 3 Pictures of the Obstacle (traffic jam & slow vehicle)




Figure 4 Technical Test Result Amsterdam – wrong way driver (Infrastructure analysis)

sf d5.5.2 test and validation results annex v1.4 · 2010. 10. 26. · sf_d5.5.2_test and validation...

Documents