![Page 1: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/1.jpg)
RESULT SUMMARY OF 4TH ITS COOPERATIVE MOBILITY SERVICES PLUGTEST
© ETSI 2011. All rights reserved
Sebastian Müller 27.3.2015 ETSI Centre for Testing and Interoperability ITS Workshop, Helmond (NL)
![Page 2: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/2.jpg)
Plugtest Organization
© ETSI 2011. All rights reserved 2
Organizer Test Tools
ITS modem for ETSI Test System Supporter
![Page 3: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/3.jpg)
ETSI TC ITS Test Activities
© ETSI 2014. All rights reserved
Conformance Validation
Framework
ITS Test Framework ETSI EG 202 798
Test Architecture ETSI TR 103 099
CAM Tests ETSI TS 102 868
DENM Tests ETSI TS 102 869
GeoNetworking Tests ETSI TS 102 871
Open Source Platform
Standardized test system interfaces for
test execution, logging, and adaptation
Free tests, codec and adaptation
Tests implemented in the international
standardized testing language TTCN-3
ITS Plugtest 2015
Interop Tests
Conformance Tests
C2C CC Validation
Real Time Channel Simulation/Replay Tests
ETSI Plugtest Live Demo
65 person month of effort invested since 2010
![Page 4: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/4.jpg)
ETSI Conformance Test System
CAM/DENM/GN and Security Tests with TESTINGTECH runtime environment ITS modem support from COHDA and COMMSIGNIA
© ETSI 2011. All rights reserved 4
![Page 5: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/5.jpg)
Interop Test System
© ETSI 2011. All rights reserved 5
Attenuator matrix is controlled by the QOSMOTEC QPER software Message flow between the DUTs is supervised by a monitor device and displayed in the VECTOR CANoe software
![Page 6: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/6.jpg)
Summary of test infrastructure
More than 20 vendors ! • 3 Open Source Implementations (see Annex)
ETSI provided Wireshark dissector for all ITS protocols including Security to assist the debugging Security PKI framework provided for Conformance and Interop tests via ETSI Event WIKI • Release of ETSI Security Conformance Tests via ETSI Forge available
since Feb 2015
4 ETSI Conformance Test Systems 3 Interop areas for face-2-face 2 Forwarding setups for 2 x 3 devices Each vendor scheduled for all tests 5 test session chairs
© ETSI 2011. All rights reserved 6
![Page 7: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/7.jpg)
Interop Issues GN and DENM
GeoNetworking • Sender Position Vector vs Source Position Vector
(SePV deleted from packets but still used in algorithms)
• GBC Advanced Fwd Algorithm not often supported
DENM • Keep Alive not often supported
© ETSI 2011. All rights reserved 7
![Page 8: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/8.jpg)
Interop Issues Security
No Profile 3 tests were executed due to the open base spec issue on ITS-AID usage for messages other than CAM and DENM Pseudonym Change needs to be defined better Some ITS stations did not send long packets (694 byte signed CAM including certificate chain) • TS 102 687 (DCC Access Layer) restricts maxPacketDuration
to 0,6 msec at 6MBit/sec Sender CAM Certificate inclusion • Reaction to request and unknown digest • Timing
Exceptional behavior tests very useful for vendors
© ETSI 2011. All rights reserved 8
![Page 9: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/9.jpg)
ETSI Plugtest Live Demo
© ETSI 2011. All rights reserved 9
Demo system • 2 RSUs and 4
OBUs • HMI on Tablet • Set up under 2
days V2I use-cases: • Road Hazard
Warning • Traffic Jam
Ahead Warning V2V use-case: • Approaching
Emergency Vehicle
Vehicle 1: Vialis Vehicle 2: ITRI Vehicle 3-4: Commsignia
RSU 1: Commsignia RSU 2: Vialis
![Page 10: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/10.jpg)
Results
All vendors had ITS modem with radio interface (no more SW prototypes) Most of the vendors have implemented Security Most of the vendors have implemented ETSI Upper Tester Interface 20% of all vendors passed all conformance tests (security included) More than 2600 tests executed Only 22 potential base spec issues identified (see Annex) Devices ready for deployment! © ETSI 2011. All rights reserved 10
![Page 11: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/11.jpg)
PHY Performance Evaluation by HHI
© ETSI 2011. All rights reserved 11
Statistical Channel Emulation • Based on ETSI ITS channel models • Covers main characteristics of the
doubly-selective vehicular radio channel, e.g. multipath and Doppler
Replay Channel Emulation • Based on channel measurements
conducted by HHI • Covers exact channel properties of
selected measurement runs Interface to receive packet count
Channel Emulator
Statistical Model
Device B
Replay Data
Device A
Next step: Feedback to ETSI ITS WG4 on channel models (ETSI TS 103 257)
![Page 12: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/12.jpg)
Example Result from HHI
© ETSI 2011. All rights reserved 12
![Page 13: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/13.jpg)
Motion Emulation by octoScope/TASS/SAVARI
© ETSI 2011. All rights reserved 13
Emulate motion of vehicles and attenuation Monitoring BER performance Next Step: Demonstrate potential hardware in the loop test capabilities including application performance and behavior
Prescan Scenario
generator
Octoscope testbed
attenuator ITS G5 RF
signals position updates
octoBox
octoBox
DUT
DUT
Distances between DUTs
![Page 14: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/14.jpg)
Mobility Emulation Scenarios by Qosmotec and TASS
© ETSI 2011. All rights reserved 14
![Page 15: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/15.jpg)
Scope of next Plugtest
Test only features relevant for Day 1, e.g. no GeoUnicast, no TSB , no KeepAlive , i.e. like C2C CC Profile Develop security tests for exceptional behavior of DENM and conformance of certificates Traces / PathHistory . Test that data is interpreted correctly Include road operator in test infrastructure Robustness tests Extend demonstrated channel simulation scenarios of 3 DUTs to • dynamic channel simulations with up to 6 DUTs • emulation of real traffic scenarios
DCC tests (Sensitivity threshold correction, energy threshold test, header decoding test, DCC stability) Out of band emission + CAM Coexistence feature Drive Tests
© ETSI 2011. All rights reserved 15
![Page 16: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/16.jpg)
Concerns
No PKI deployment plan/protocols available (for infrastructure) Pseudonym Changes / Anti finger printing No Profile 3 tests were executed due to the open base spec issue on ITS-AID SPAT/MAP needs to be made available soon DCC needs to be made available soon Make C2C CC profile available to all vendors of next plugtest (use only 1 profile in Europe for C2C Communication) Create an European wide compatible C2X profile © ETSI 2011. All rights reserved 16
![Page 17: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/17.jpg)
On behalf of the Plugtest Team
© ETSI 2011. All rights reserved 17
Thank you!
![Page 18: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/18.jpg)
ANNEX
18 © ETSI 2011. All rights reserved
![Page 19: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/19.jpg)
ITS Vendors
© ETSI 2011. All rights reserved 19
Autotalks Cohda/Nxp Commsignia Ctag Denso *I-GAME (TNO + Viktoria)
Hitachi/Renesas Imtech Itri Kapsch
Marben Neavia Nec Nordsys Novero Qmic Siemens *Trialog Unex *Yogoko Vialis
* Partly Open Source
![Page 20: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/20.jpg)
Issues GN EN 302 636-4-1 v1.2.1
Typo in 9.3.12.2 Source operations – Issue6982 ambigiuous description in 9.3.8.3 Forwarder operations – Issue6983 error in 9.3.12.2 Source operations – Issue6984 DPD in 9.3.11.3 forwarder and receiver operations – Issue6985 GN protocol constant itsGnGeoAreaLineForwarding is set to TRUE – Issue6986 clock drifting – Issue6987 Restructure of FWD algorithm – Issue6988 Duplicate Packet Detection – Issue6989 MAC and GN ID – Issue6990
© ETSI 2011. All rights reserved 20
![Page 21: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/21.jpg)
Issues Security TS 103 097 V1.1.20
Uncomplete description of creating hash for signature of secured messages – Issue6950 Wrong description of considering the public key algorithm in the signature of CAMs – Issue6949 Wrong description of considering the signer info field in the signature of CAMs - Issue6948 ITS-S should accept request of unrecognized AA certificate from untrusted sender – Issue6972
© ETSI 2011. All rights reserved 21
![Page 22: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/22.jpg)
New feature for next revision of Security TS 103 097
Request of unrecognized AA certificate create high channel load when all receivers reply with a certificate chain – Issue6969 Security link between CertID and relevant IDs (Pseudonym change) – Issue6979 ITS-S should stop requesting an unrecognized AA certificate if the issuer of the AA certificate is untrusted – Issue6973 Request of unrecognized AA certificate create high channel load when all receivers reply with a certificate chain – Issue6969 Do we need certificates in CAM each second – Issue6976 add example how to define time – Issue6978 Security link between CertID and relevant IDs – Issue6979
© ETSI 2011. All rights reserved 22
![Page 23: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/23.jpg)
Issues DENM EN 302 637-3 v1.2.2
pseudonym change for active DENMs – Issue6980 StationID collisions - Issue6981
© ETSI 2011. All rights reserved 23
![Page 24: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/24.jpg)
Useful Links
ETSI Certificate Toolkit (Generator, SignatureVerifier) http://forge.etsi.org/websvn/ITS/branches/STF484_VALIDATION/tools/itscertgen/ FOKUS SecureMessage & Certificates Validator https://werkzeug.dcaiti.tu-berlin.de/etsi/ts103097 Link to Wireshark dissectors http://forge.etsi.org/websvn/WIRESHARK_ITS_PLUGINS/releases/wireshark-1.12.x/ Link to ETSI conformance framework http://forge.etsi.org/websvn/ITS/branches/STF484_VALIDATION/ ETSI GPSD Server http://forge.etsi.org/websvn/ITS_GPSD_SERVER/ Blog http://www.etsi.org/news-events/past-events/846-plugtests-2015-itscms4?tab=3
© ETSI 2011. All rights reserved 24
![Page 25: MOBILITY SERVICES PLUGTEST - ETSIdocbox.etsi.org/Workshop/2015/201503_ITSWORKSHOP/SESSION03_… · Reaction to request and unknown digest ... V2I use-cases: • Road Hazard Warning](https://reader034.vdocument.in/reader034/viewer/2022050902/5ab442837f8b9a6e1c8ba1e0/html5/thumbnails/25.jpg)
Contact
© ETSI 2011. All rights reserved 25
Sebastian Müller
Project Manager ETSI - CTI
T: +33 4 92 94 43 13 | M: +33 6 74 40 83 66 |