d4.3 final results · d4.3 final results of the tests with lessons learnt, conclusions and...
TRANSCRIPT
D4.3 Final results of the tests with lessons learnt, conclusions and
recommendations
Grant agreement no.: 325075
Pilot type A co-funded by the European Union under the Competitiveness and Innovation Programme
ICT Policy Support Programme
DG Communications Networks, Content and Technology
Version number: 1.0
Main author: NavCert
Dissemination level: PU
Lead contractor:
Due date: 31.12.2014
Delivery date: 17.12.2014
Delivery date updated document
Page left intentionally blank
CONTROL SHEET
Version history
Version Date Main author Summary of changes
0.1 2014-02-18 Monika Hentschinski template
0.2 2014-09-08 to
2014-11-13 Monika Stapelfeld
including input pilot sites
0.3 2014-11-17 Martin Grzebellus internal review
0.4 2014-11-20 to
2014-11-25 Ana Paul peer review
0.5 2014-11-20 to
2014-11-25 Kate Yeadon language review
0.6 2014-11-25 to
2014-12-02 Martin Grzebellus, Monika
Stapelfeld integrating updates
0.7 2014-12-02 to
2014-12-05 pilot sites review, flavouring
1.0 21-12-2014 Andy Rooke Authorised
Name Date
Prepared Martin Grzebellus, Monika Stapelfeld 2014-11-17
Reviewed Ana Paul, Kate Yeadon 2014-11-25
Authorized Andy Rooke 21-12-2014
Circulation
Recipient Date of submission
Project partners 21-12-2014
European Commission 21-12-2014
D4.3 Final results
17/12/2014 4 v 1.0
TABLE OF CONTENTS
Control sheet ......................................................................................... 3
Table of contents .................................................................................. 4
Figures ................................................................................................... 7
Tables .................................................................................................. 11
1 Management Summary ................................................................. 13
2 Terms and Abbreviations .............................................................. 16
3 Introduction ................................................................................... 18
3.1 Purpose of Document .................................................................................. 18
3.2 Intended audience of this document ............................................................ 18
3.3 HeERO 2 Contractual References ............................................................... 18
4 Overall Validation .......................................................................... 20
4.1 Validation Procedure ................................................................................... 21
4.2 Timings ........................................................................................................ 22
4.3 Results for KPIs ........................................................................................... 23
4.4 Results of Questionnaires ........................................................................... 26
4.4.1 Powered Two-Wheeled ...................................................................................26
4.4.2 Dangerous Goods Vehicles .............................................................................28
4.5 Recommendations on Performance Requirements ..................................... 31
4.6 General Recommendations ......................................................................... 33
4.7 Conclusions ................................................................................................. 33
5 Results of Pilot Sites ..................................................................... 35
5.1 Belgium ....................................................................................................... 35
5.1.1 General ...........................................................................................................35
5.1.2 Recommended KPIs .......................................................................................39
5.1.3 Evaluation results ............................................................................................40
5.1.4 Interoperability tests ........................................................................................42
5.1.5 Recommendations ..........................................................................................43
D4.3 Final results
17/12/2014 5 v 1.0
5.1.6 Conclusions ....................................................................................................43
5.2 Bulgaria ....................................................................................................... 43
5.2.1 General ...........................................................................................................43
5.2.2 Recommended KPIs .......................................................................................44
5.2.3 Evaluation results ............................................................................................44
5.2.4 Interoperability tests ........................................................................................54
5.2.5 Recommendations ..........................................................................................54
5.2.6 Conclusion ......................................................................................................55
5.3 Denmark ...................................................................................................... 55
5.3.1 General ...........................................................................................................55
5.3.2 Recommended KPIs .......................................................................................56
5.3.3 Evaluation results ............................................................................................56
5.3.4 Recommendations ..........................................................................................70
5.3.5 Conclusions ....................................................................................................71
5.4 Luxembourg ................................................................................................ 71
5.4.1 General ...........................................................................................................71
5.4.2 Recommended KPIs .......................................................................................71
5.4.3 Evaluation results ............................................................................................71
5.4.4 Interoperability tests ........................................................................................78
5.4.5 Recommendations ..........................................................................................79
5.4.6 Conclusion ......................................................................................................80
5.5 Spain ........................................................................................................... 80
5.5.1 General ...........................................................................................................80
5.5.2 Recommended KPIs .......................................................................................85
5.5.3 Evaluation results ............................................................................................86
5.5.4 Interoperability tests ...................................................................................... 103
5.5.5 Recommendations ........................................................................................ 108
5.5.6 Conclusions .................................................................................................. 108
5.6 Turkey ....................................................................................................... 110
5.6.1 General ......................................................................................................... 110
5.6.2 Recommended KPIs ..................................................................................... 110
5.6.3 Evaluation results .......................................................................................... 111
D4.3 Final results
17/12/2014 6 v 1.0
5.6.4 Interoperability tests ...................................................................................... 117
5.6.5 Recommendations ........................................................................................ 117
5.6.6 Conclusions .................................................................................................. 117
6 References ................................................................................... 118
ANNEX 1: eCall for P2W - User Acceptance Report ....................... 119
Abstract ............................................................................................................... 119
ANNEX 2: List of all evaluated KPIs................................................. 154
D4.3 Final results
17/12/2014 7 v 1.0
FIGURES
FIGURE 1: MEASUREMENT OF PLANNED KPIS ............................................................... 21
FIGURE 2: TIME INTERVALS DURING ECALL .................................................................... 23
FIGURE 3: RESULTS FOR KPI5: MSD PRESENTATION TIME ............................................ 24
FIGURE 4: RESULTS FOR KPI7: VOICE CHANNEL BLOCKING TIME .................................... 24
FIGURE 5: SINGLE TEST CASES FOR KPI7: VOICE CHANNEL BLOCKING TIME .................... 25
FIGURE 6: RESULTS FOR KPI8: TIME FOR CALL ESTABLISHMENT .................................... 25
FIGURE 7: SERVER ARCHITECTURE (BE) ....................................................................... 35
FIGURE 8: GRAPHICAL USER INTERFACE OF ECALL SERVER (BE) ..................................... 36
FIGURE 9: SNAPSHOT OF THE GUI ON PSAP (BE) ......................................................... 37
FIGURE 10: IVS BY NXP/S1NN (BE) ........................................................................... 37
FIGURE 11: IVS BY GENEVA MICROSYSTEMS (BE) ........................................................ 38
FIGURE 12 : TIME SERIES KPI 7 AND 8 (BE) .................................................................. 40
FIGURE 13: DISTRIBUTION KPI 7A (BE) ........................................................................ 41
FIGURE 14: DISTRIBUTION KPI 8 (BE) .......................................................................... 42
FIGURE 15: TESTING PROCESS (BG) ............................................................................. 44
FIGURE 16: KPI_007 TIME SERIES TUS IVS (BG) ......................................................... 51
FIGURE 17: KPI_007 TIME SERIES ICOM IVS (BG) ...................................................... 51
FIGURE 18: KPI_008 TIME SERIES ICOM IVS (BG) ...................................................... 52
FIGURE 19: DISTRIBUTION OF KPI_007 FOR TUS (BG) .................................................. 53
FIGURE 20: DISTRIBUTION OF KPI_007 FOR ICOM (BG) ............................................... 53
FIGURE 21: DISTRIBUTION OF KPI_008 FOR ICOM (BG) ............................................... 54
FIGURE 22: LOCATION OF PILOT TEST CALLS, AS INDICATED IN THE IVS-UNIT (DK) ........... 56
FIGURE 23 – AN IVS-UNIT INSTALLED IN A TEST VEHICLE ................................................ 57
FIGURE 24: THE IVS-UNITS USED FOR TESTING (DK) ..................................................... 58
FIGURE 25 – GEOGRAPHICAL DISTRIBUTION OF SUCCESS RATE OF CALLS MADE DURING
PHASE ONE PILOT TEST, WITH THE SATELLITE POSITION REGISTERED IN THE IVS-UNITS (DK)
................................................................................................................................. 61
D4.3 Final results
17/12/2014 8 v 1.0
FIGURE 26: TIME SERIES FOR SUCCESSFUL DURING FIRST PHASE. (DK) ........................... 63
FIGURE 27 – GEOGRAPHICAL DISTRIBUTION OF CALLS MADE DURING PHASE TWO, WITH THE
SATELLITE POSITION REGISTERED IN THE IVS-UNITS (DK) ............................................... 65
FIGURE 28: PHASE TWO CALLS (DK) ............................................................................ 67
FIGURE 29: PHASE TWO CALLS WITH FJT M2M SIM CARD (DK) ..................................... 67
FIGURE 30: PHASE TWO CALLS WITH FJT NORMAL SIM CARD (DK) ................................. 68
FIGURE 31: PHASE TWO CALLS WITH FJT NORMAL SIM CARD (DK) ................................. 68
FIGURE 32: PHASE TWO CALLS WITH GMV NORMAL SIM CARD (DK) ............................... 69
FIGURE 33: STATISTICAL DATA PHASE 2 (DK) ................................................................ 69
FIGURE 34: GEOGRAPHICAL DISTRIBUTION OF FAILED CALLS FOR PHASE 1 AND 2 (DK) ...... 70
FIGURE 35: KPI5 FUJITSU IVS (LU) .......................................................................... 75
FIGURE 36: KPI5 FUJITSU IVS (LU) .......................................................................... 75
FIGURE 37: KPI5 FICOSA 2 (LU) ............................................................................... 76
FIGURE 38: MSD PRESENTATION TIME (LU) .................................................................. 77
FIGURE 39: MSD PRESENTATION TIME (LU) .................................................................. 77
FIGURE 40: VOICE CHANNEL BLOCKING TIME (LU).......................................................... 78
FIGURE 41: CALL CHAIN FROM THE IVS TO FINAL PSAP ................................................. 80
FIGURE 42: GALICIAN SCENARIOS (ES) ......................................................................... 82
FIGURE 43: IVS CTAG IN THE REGIONAL CROSS BORDER .............................................. 83
FIGURE 44: P2W TEST ARCHITECTURE ......................................................................... 84
FIGURE 45: P2W WITH IVS INSTALLED ......................................................................... 84
FIGURE 46: TIME SERIES DIAGRAM OF IVS CTAG - KPI_05 (ES) ................................... 92
FIGURE 47: HISTOGRAM OF IVS CTAG - KPI_05 (ES) .................................................. 92
FIGURE 48: TIME SERIES DIAGRAM OF IVS CTAG - KPI_07A (ES) ................................. 93
FIGURE 49: HISTOGRAM OF IVS CTAG - KPI_07A (ES) ................................................ 93
FIGURE 50: TIME SERIES DIAGRAM OF IVS CTAG - KPI_07B (ES) ................................. 94
FIGURE 51: HISTOGRAM OF IVS CTAG - KPI_07B (ES) ................................................ 94
FIGURE 52: TIME SERIES DIAGRAM OF IVS CTAG - KPI_08 (ES) ................................... 95
FIGURE 53: HISTOGRAM OF IVS CTAG - KPI_08 (ES) .................................................. 95
D4.3 Final results
17/12/2014 9 v 1.0
FIGURE 54: TIME SERIES DIAGRAM OF IVS CTAG - KPI_23 (ES) ................................... 96
FIGURE 55: HISTOGRAM OF IVS CTAG - KPI_23 (ES) .................................................. 96
FIGURE 56: TIMES SERIES DIAGRAM OF IVS CTAG - KPI_29 (ES) ................................. 97
FIGURE 57: HISTOGRAM OF IVS CTAG - KPI_29 (ES) .................................................. 97
FIGURE 58: KPI5 FOR GMV IVS (ES) .......................................................................... 98
FIGURE 59: KPI7A FOR GMV IVS (ES) ........................................................................ 98
FIGURE 60: KPI7B FOR GMV IVS (ES) ........................................................................ 99
FIGURE 61: KPI8 FOR GMV IVS (ES) .......................................................................... 99
FIGURE 62: MSD PRESENTATION TIME (ES) ................................................................ 100
FIGURE 63: VOICE CHANNEL BLOCKING TIME (ES) ....................................................... 100
FIGURE 64: VOICE CHANNEL BLOCKING TIME (ES) ....................................................... 101
FIGURE 65: MSD PRESENTATION TIME (ES) ................................................................ 101
FIGURE 66: TESTFEST#3 ........................................................................................ 104
FIGURE 67: SUMMARY OF CTAG RESULTS IN 2013 ..................................................... 105
FIGURE 68: SUMMARY OF ALL PARTICIPANTS’ RESULTS ................................................ 107
FIGURE 69: GMV’S IVS TEST RESULTS SUMMARY ....................................................... 107
FIGURE 70: TIME SERIES PLOT OF KPI_05 (TR) .......................................................... 114
FIGURE 71: TIME SERIES PLOT OF KPI_07A (TR) ........................................................ 115
FIGURE 72: THE FREQUENCY DISTRIBUTION PLOT OF KPI_05 (TR) ............................... 116
FIGURE 73: THE FREQUENCY DISTRIBUTION PLOT OF KPI_07A (TR) ............................. 116
FIGURE 74: GENDER OF THE RESPONDENTS ................................................................ 121
FIGURE 75: PYRAMID: AGES AND GENDER ................................................................... 121
FIGURE 76: YEARS OF EXPERIENCE DRIVING A MOTORCYCLE, BY GENDER ...................... 122
FIGURE 77: FREQUENCY OF USE ................................................................................ 123
FIGURE 78: FREQUENCY OF USE BY TYPE OF MOTORCYCLE .......................................... 123
FIGURE 79: AVERAGE NUMBER OF KM. DRIVEN PER YEAR ............................................. 124
FIGURE 80: ACCIDENT STATISTICS OF RESPONDENTS ................................................... 124
FIGURE 81: ACCIDENT STATISTICS OF RESPONDENTS BY AGE GROUP ............................ 125
D4.3 Final results
17/12/2014 10 v 1.0
FIGURE 82: YEARS OF EXPERIENCE OF MOTORCYCLE DRIVERS THAT HAVE HAD AT LEAST ONE
ACCIDENT ................................................................................................................. 126
FIGURE 83: DID YOU CALL THE EMERGENCY SERVICES (YOURSELF)? ............................. 126
FIGURE 84: HOW MUCH TIME ELAPSED UNTIL THE EMERGENCY SERVICES ARRIVED? ...... 127
FIGURE 85: DISTRIBUTION OF ACCIDENTS BETWEEN URBAN AND INTERURBAN AREA ........ 127
FIGURE 86: AVERAGE TIME OF ARRIVAL OF THE EMERGENCY SERVICES TO THE ACCIDENT
SPOT DEPENDING ON THE TYPE OF AREA (URBAN / INTERURBAN) ................................... 128
FIGURE 87: ARCHITECTURE OF THE ECALL SERVICE FOR MOTORCYCLES ....................... 130
FIGURE 88: LEVEL OF ACCEPTANCE OF THE ECALL SERVICE FOR MOTORCYCLES ............ 131
FIGURE 89: RELEVANCE OF THE INFORMATION PROVIDED BY THE HELMET IN CASE OF
ACCIDENT ................................................................................................................. 132
FIGURE 90: WILLINGNESS TO PURCHASE A NEW HELMET COMPATIBLE WITH ECALL ........ 133
FIGURE 91: PRICE SENSITIVITY (FOR THE PRICE INCREASE OF AN ECALL COMPATIBLE
HELMET) ................................................................................................................... 133
FIGURE 92: IMPORTANCE RESPONDENTS GIVE TO DIFFERENT TYPES OF INFORMATION THAT
THE ECALL SERVICE FOR MOTORCYCLES COULD GENERATE .......................................... 135
FIGURE 93: SENSITIVITY TO DATA PRIVACY OF THE RESPONDENTS ................................ 141
FIGURE 94 : PRICE SENSITIVITY FOR AN AFTERMARKET ECALL SYSTEMECALL SERVICE ... 142
D4.3 Final results
17/12/2014 11 v 1.0
TABLES
TABLE 1: STATISTICAL PARAMETERS DEFINITION ........................................................... 22
TABLE 2: RECOMMENDED VALUES FOR KPIS 05 TO 08 ................................................... 32
TABLE 3: MEASURED KPIS (BE) ................................................................................... 39
TABLE 4 : RESULTS KPIS (BE) ..................................................................................... 40
TABLE 5 : STATISTICS KPIS 7 AND 8 (BE) ...................................................................... 41
TABLE 6: EVENTS AND PARAMETERS RECORDED BY DIFFERENT IVS DEVICES (BG) .......... 46
TABLE 7: SUMMARY OF RESULTS (BG) .......................................................................... 49
TABLE 8: STATISTICAL PARAMETER (BG) ....................................................................... 52
TABLE 9: KPI 2B, 3, 4, 6 AND 7A WITH FOREIGN PSAPS (BG) ........................................ 54
TABLE 10 – RESULTS DERIVED FROM PHASE ONE PILOT TEST (DK) ................................. 61
TABLE 11: STATISTICS OF MANUAL INITIATED CALLS FOR SUCCESSFUL CALLS DURING FIRST
PHASE (DK) ................................................................................................................ 63
TABLE 12 – RESULTS DERIVED FROM PHASE TWO PILOT TEST (DK) ................................. 65
TABLE 13: SUMMARY OF RESULTS (LU) ........................................................................ 74
TABLE 14: STATISTICS OF MANUAL INITIATED CALLS (LU) ............................................... 76
TABLE 15: KPI 1B WITH OWN IVS AND FOREIGN PSAPS (LU) ........................................ 78
TABLE 16: KPI 2B, 3, 4 AND 6 WITH FOREIGN PSAPS (LU) ............................................ 79
TABLE 17: KPIS MEASURED DURING PHASE 1 (ES) ........................................................ 86
TABLE 18: SUMMARY OF CALCULATED KPIS DURING PHASE 1 (ES) ................................ 87
TABLE 19: SUMMARY OF CALCULATED KPIS DURING PHASE 2 (ES) ................................ 88
TABLE 20: SUMMARY OF GALICIAN TESTS ..................................................................... 89
TABLE 21: SUMMARY OF GALICIAN KPIS ....................................................................... 90
TABLE 22: STATICS OF INITIATED ECALLS (ES) ............................................................ 102
TABLE 23: STATISTICS OF INITIATED CALLS (ES) .......................................................... 103
TABLE 24: INTEROPERABILITY TESTS PERFORMED BETWEEN CTAG AND EUROPEAN PSAPS
IN 2014 .................................................................................................................... 106
TABLE 25 CTAG’S IVS MANDATORY TEST RESULTS IN 2014 ........................................ 106
D4.3 Final results
17/12/2014 12 v 1.0
TABLE 26 : RECOMMENDED KPIS TESTED IN TURKEY ................................................... 111
TABLE 27: THE RESULTS OF OUR KPI EVALUATION (TR) ............................................... 112
TABLE 28: SUMMARY OF STATISTICAL EVALUATION RESULTS(TR) .................................. 115
D4.3 Final results
17/12/2014 13 v 1.0
1 Management Summary
This document presents the test results for the HeERO 2 project from all pilot sites in a
consistent manner to allow comparison between the results of the pilot sites involved.
Results of HeERO have been documented in detail already and therefore here only results of
HeERO 2 are presented. As HeERO 2 incorporates recommendations of HeERO 1, the
overall evaluation of the HeERO projects can be deducted with the recommendations and
conclusions presented in this document. Details of the result for HeERO 1 are documented in
the already published document HeERO_WP4_DEL_D4 5_results_v1.0.pdf.
Well defined KPIs have been agreed to measure all aspects of the eCall communication. The
consolidated evaluation is based on results of the pilot sites (Belgium, Bulgaria, Denmark,
Luxemburg, Spain and Turkey). Each pilot site provided statistical evaluations of the
measured KPIs with derived recommendations and conclusions. The quantitative analysis
was complemented by a qualitative assessment of P2W and transport of dangerous goods.
This document is structured into three parts: First a summary is given of the achievements
for HeERO 1and HeERO 2. Then the consolidated results of all pilot sites are presented and
in the end the details of the results per pilot site are described. The qualitative analysis with
questionnaires for P2W is attached as annex.
All pilot sites proposed in the preparation phase individual KPIs, so that in total more than 30
KPIs have been defined. Out of this list, a subset of KPIs has been recommended which
should be tested by all pilot sites. The most important KPIs (KPI 1 to KPI 8) were tested from
almost all pilot sites. All recommended KPIs were tested by at least four pilot sites. From a
process point of view, the most important KPI is KPI 018 (time to activate rescue forces).
This KPI measures the overall objective for introduction of eCall, the reduction of time until
rescue forces arrive at incident location. However this KPI cannot be realistically tested in
test scenarios and compared to actual data. Thus this KPI was not measured at all.
The next important KPIs are KPI 005 (duration until MSD is presented in PSAP) and KPI
007a (voice channel blocking time). ECall provides additional information compared to 112
calls and this information is transmitted in the MSD. These KPIs are measurable and
comparable and in the scope of the project as well as the standards. During the transmission
time of the MSD the passenger in the vehicle cannot communicate with the call handler. As
the vehicle occupants do not know the technical aspects and reasons for the “dead line” of
the call, every second of silence is undesirable. Passengers need to leave the car quickly, so
voice contact has to be established as swiftly as possible. The KPIs 005 and 007a have been
measured and evaluated by nearly all pilot sites. The KPIs measured by the majority of pilot
sites are the various success rates KPI 002a (success rate of completed eCalls using 112),
D4.3 Final results
17/12/2014 14 v 1.0
KPI 002b (success rate of completed eCalls using long number), KPI 003 (success rate of
received MSDs), KPI 004 (success rate of correct MSDs) and KPI 006 (success rate of
established voice transmissions). Unfortunately not all pilot sites were able to evaluate 112
call set up and initiated eCalls only via a long number. The importance for the 112 call set up
can be derived from KPI 02, as the highest success rate for call set up (KPI 02) is for 112 call
set up, as mobile networks are configured to give priority to 112 calls in case of network
congestion. The failure to test this KPI is solely due to mobile network operators in some pilot
sites still failing upgrading their network for the proper handling of the eCall flag.
The KPIs didn’t improve from HeERO 1 to HeERO 2 as expected. The modifications in the
eCall standards based on recommendations were already available in HeERO 1 phase 2
leading to an improved performance. In HeERO 2 quite some suppliers started with their
development from scratch, so that the IVSs still being on a prototypical level. The success
rates for successful MSD transmission (KPI 03 and 04) are very good. The average for KPI
005 (time until the MSD is presented in PSAP) is now 12.9 seconds compared to HeERO 1
14.1 seconds. The average of the voice channel blocking time (KPI 07) is now 9.2 seconds
compared to HeERO 1 7.9 seconds, but with more variability. This indicates that the
manufacturers were struggling with overall functionality and still far away from a mass
production with successful performance optimisation.
By implementing best practise and optimization of all components values of 4s to 6s for KPI
05 and KPI 07 are achievable.
In that case, the additional delay caused by the transmission of the MSD is in the expected
range of about 4s.
Therefore following recommendations are given:
Vehicle manufacturers should implement best practise to minimize voice channel
blocking time
The heading information does still not provide in all implementations the direction of
travel but sometimes only the direction of the vehicle when the eCall was triggered.
This is not in conformance with the underlying standard CEN 17025.
When an IVS is triggered to retransmit the MSD, some implementations provide a
clone of the already transmitted MSD; others provide a new updated content of the
MSD at the moment of request for retransmission. As the MSD with an update of the
position provides valuable information to the PSAP especially to better identify false
alarms, in case of a retransmission of the MSD, the MSD content should always be
updated
D4.3 Final results
17/12/2014 15 v 1.0
Due to limited GNSS coverage e. g. in tunnels, a GNSS position might not be
available in case of an eCall triggering. Therefore vehicle manufacturers should try to
use additional information from accelerometer, odometer, gyros etc. to determine the
position of the vehicle if feasible.
The intent of the HeERO pilot sites has been mainly to evaluate if the requested performance
of the eCall service can be met with a deployment of the approved European eCall standards
in the existing public mobile telecommunications networks and within the existing 112
system. This means that the testing has had a strong focus on the eCall standards and
capturing the key performance indicators, the KPIs. Other issues, such as the response time
of the rescue services and ambulances, use of EUCARIS and use of VIN in the operational
rescue chain, as well as non-operational issues, like legal liability, privacy issues, periodic
time inspections, change of a car ownership, etc. is not included in the work of the HeERO
pilot sites.
The outcome of the tests performed and reported in this document confirm that the pan-
European eCall is working according to expectations however there is still room for
improvement in the implementation by the suppliers.
D4.3 Final results
17/12/2014 16 v 1.0
2 Terms and Abbreviations
Term Definition
3GPP Third Generation Partnership Project
AT Attention Command
CAN Controller Area Network
CEN Comité Européen de Normalisation
CIP Competitiveness and Innovation Programme
CRF Centro Ricerche Fiat
DoW Description of Work
EC European Commission
ECR Emergency Control Room
EGNOS European Geostationary Navigation Overlay System
ENT Ericsson Nikola Tesla
ETSI European Telecommunication Standards Institute
EUCARIS European Car and Driving License Information System
ESO European Standards Organization
GDOP Geometric dilution of precision
GIS Geographic Information System
GLONASS
Globalnaja Nawigazionnaja SputnikowajaNavigazionnaja Sputnikovaja
Sistema
GNSS Global Navigation Satellite System
GPS Global Positioning System
GPRS General Packet Radio System
GSM Global System of Mobile telecommunications
GSMA GSM Association
HDOP Horizontal Dilution Of Precision
D4.3 Final results
17/12/2014 17 v 1.0
HTTP Hypertext Transfer Protocol
IMSI International Mobile Subscriber Identity
ISO International Standardization Organization
ITS Intelligent Transport Systems
IVS In-Vehicle System
KPI Key Performance Indicator
MNO Mobile Network Operator
MSC Mobile Switching Centre
MSD Minimum Set of Data
MSISDN Mobile Subscriber Integrated Services Digital Network Number
NIST National Institute of Standards and Technology
NMEA National Marine Electronics Association
PLMN
PND
Public Land Mobile Network
Personal Navigation Device
PSAP Public Safety Answering Point
RTTI Real-time traffic and travel information
SBAS Satellite Based Augmentation System
SDR Software Defined Radio
SIM Subscriber Identity Module
SOP Standard Operating Procedure
TIM Telecom Italia Mobile
TMC Traffic Management Centre
UMTS Universal Mobile Telecommunication System
USB Universal Serial Bus
UTC Universal Time Coordinated
VAS Value Added Services
VIN Vehicle Identification Number
D4.3 Final results
17/12/2014 18 v 1.0
3 Introduction
3.1 Purpose of Document
The purpose of this document is to present the test results from all pilot sites in a
comprehensive and consistent manner that gives the provision of conclusions and
recommendations as a result of the HeERO projects. The overall evaluation is based on the
results from the pilot sites. Each pilot site was asked to provide statistical evaluations of the
measured KPIs, recommendations and conclusions.
3.2 Intended audience of this document
This document is aimed at the following audiences with their respective objectives:
European Commission: for information
Member states: for information
Stakeholders: for information
3.3 HeERO 2 Contractual References
HeERO 2 is a Pilot type A of the ICT Policy Support Programme (ICT PSP), Competitiveness
and Innovation Framework Programme (CIP). It stands for Harmonised eCall European Pilot.
The Grant Agreement number is 325075 and project duration is 24 months, effective from 01
January 2013 until 31 December 2014. It is a contract with the European Commission, DG
CONNECT.
The principal EC Project Officer is:
Aude Zimmermann
EUROPEAN COMMISSION
DG CONNECT
Office: BU 31 – 6/35
B - 1049 Brussels
Tel: +32 296 2188
E-mail: [email protected]
One other Project Officer will follow the HeERO project:
Dimitrios AXIOTIS
D4.3 Final results
17/12/2014 19 v 1.0
Address to which all deliverables and reports have to be sent:
Aude Zimmermann
EUROPEAN COMMISSION
DG CONNECT
BU 31 – 6/35
B - 1049 Brussels
Tel: +32 296 2188
By mail: [email protected]
Any communication or request concerning the grant agreement shall identify the grant
agreement number, the nature and details of the request or communication and be submitted
to the following addresses:
European Commission
Communications Networks, Content and Technology
B-1049 Brussels
Belgium
By electronic mail: [email protected]
D4.3 Final results
17/12/2014 20 v 1.0
4 Overall Validation
The evaluation was performed according to the agreed test specification and methodology
(Deliverable 4.1). The definition of all KPIs in detail is also documented in this deliverable.
The figure below shows the KPIs tested by the pilot sites. The major KPIs were measured by
almost all of the pilot sites. In the lower part of the graph some deviation between planning
and realisation can be seen. Compared to HeERO1 the test phases were shorter and there
was less time to re-plan or re-adjust between the two test phases.
D4.3 Final results
17/12/2014 21 v 1.0
Figure 1: Measurement of Planned KPIs
Tested as planned
Planned, not tested
Not planned, not tested
Tested additionally
4.1 Validation Procedure
The validation considered the following criteria:
LU BE BG ES DK TR
KPI_001a planned but not evaluatedplanned but not evaluatedevaluated as plannedevaluated as plannednot plannedplanned but not evaluated
KPI_001b evaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_002a planned but not evaluatedevaluated as plannedevaluated as plannedplanned but not evaluatedplanned but not evaluatedevaluated as planned
KPI_002b evaluated as plannedevaluatedevaluated as plannedevaluated as plannedevaluatedplanned but not evaluated
KPI_003 evaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_004 evaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_005 evaluated as plannedplanned but not evaluatedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_006 evaluated as plannedplanned but not evaluatedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_007a evaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_007b not plannedplanned but not evaluatedevaluated as plannedevaluated as plannedevaluatednot planned
KPI_008 not plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluatednot planned
KPI_009 evaluatednot plannedevaluated as plannedevaluated as plannedevaluatednot planned
KPI_010 not plannednot plannedevaluated as plannedevaluated as plannednot plannednot planned
KPI_011 not plannednot plannedevaluated as plannedevaluated as plannednot plannednot planned
KPI_012 not plannednot plannedevaluated as plannedevaluated as plannednot plannednot planned
KPI_013 evaluated as plannednot plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_014 not plannednot plannedevaluated as plannedevaluated as plannednot plannednot planned
KPI_015 planned but not evaluatednot plannedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_016 planned but not evaluatednot plannedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_017 not plannedplanned but not evaluatedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_018 not plannednot plannedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_019 not plannednot plannedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_020 not plannednot plannedplanned but not evaluatedevaluated as plannednot plannednot planned
KPI_021 not plannedplanned but not evaluatedevaluated as plannedplanned but not evaluatedevaluatednot planned
KPI_022 not plannedplanned but not evaluatedevaluated as plannedplanned but not evaluatedevaluatednot planned
KPI_023 not plannednot plannedplanned but not evaluatedevaluated as plannednot plannednot planned
KPI_024 not plannedplanned but not evaluatedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_025 not plannedplanned but not evaluatedevaluated as plannedplanned but not evaluatednot plannednot planned
KPI_026 not plannednot plannedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_027 not plannednot plannedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_028 a planned but not evaluatedplanned but not evaluatedplanned but not evaluatedevaluated as plannedplanned but not evaluatednot planned
KPI_028 b planned but not evaluatednot plannedevaluated as plannedevaluated as plannedplanned but not evaluatednot planned
KPI_028 c not plannednot plannednot plannedevaluated as plannednot plannednot planned
KPI_29 not plannedplanned but not evaluatedplanned but not evaluatedevaluated as plannednot plannednot planned
KPI_30 planned but not evaluatednot plannednot plannednot plannednot plannednot planned
KPI_31 planned but not evaluatednot plannednot plannednot plannednot plannednot planned
D4.3 Final results
17/12/2014 22 v 1.0
- Time series diagrams of the values of relevant KPIs
- Fundamental KPI statistical description for every time series (mean, median,
variance, standard deviation, skew, kurtosis and histogram with normal probability)
- Discussion
The procedures for the creation of the KPI time series diagrams, and the fundamental KPI
statistical description, are described in [2], and conducted in accordance to [1] and [3].
Statistical parameter Definition Comments
Time series diagram of KPI - Graphical representation of time series values v measurement time stamps
Mean x=
1
n∑i= 1
n
xi A numerical measure of the central location of the data values
Median The value at the middle when the data is sorted in ascending order
Variance s
2=
1
n− 1∑i= 1
n
( xi− x )2
A numerical measure of data values dispersion around the mean
Standard deviation σ=
s
√n
An observation variable proportional to the square root of its variance
Skewness γ1=μ3
μ2
3 /2 A measure of the symmetry of the data distribution
Kurtosis γ2=μ4
μ2
2− 3
A measure of the peakedness of the data distribution
Histogram with normal probability diagram
- A graphical representation of the frequency distribution of a KPI value
Table 1: Statistical Parameters Definition
4.2 Timings
Due to the fact that some of the defined KPIs are based on timing issues, timers were
defined already in HeERO 1 to allow a consistent measurement of KPIs in all pilot sites.
These are summarised in Figure 2 (provided by the Czech Pilot Site during HeERO 1)
overleaf.
D4.3 Final results
17/12/2014 23 v 1.0
Figure 2: Time Intervals during eCall
4.3 Results for KPIs
In the following figures the values for the most important KPIs are shown as well as the test
cases from the HeERO1 pilot sites As the project time of HeERO1 was one year longer than
HeERO2, HeERO1 pilot sites had more time between the test phases and there was more
time to readjust the equipment.
As most of the pilot sites provided different test cases there are more columns than pilot sites
within the graphics. The test cases per pilot sites differ, either in time (phase 1 and phase 2
of testing),) or in the different PSAPs or IVSs used.
D4.3 Final results
17/12/2014 24 v 1.0
Figure 3: Results for KPI5: MSD Presentation Time
The test cases from Romania, Greece (HeERO1) and from Turkey are notable in that they
are about 10 seconds longer than the results from the other pilot sites. If these three pilot
sites are excluded the mean value of the Duration until MSD is Presented in PSAP is 17.5
with a standard deviation of 7.9 seconds.
Figure 4: Results for KPI7: Voice Channel Blocking time
The voice channel blocking time does not differ as much as the MSD presentation time. The
minimal values are reached by the Bulgarian and Dutch test sites.
D4.3 Final results
17/12/2014 25 v 1.0
Figure 5: single test cases for KPI7: Voice Channel Blocking time
Five seconds seems to be a plausible optimal minimum to be reached for a full scale
deployment across Europe.
At the moment a mean value of 9.2 seconds with a standard deviation of 2.8 seconds is
reached, giving much room for improvement.
Figure 6: Results for KPI8: time for Call Establishment
The time for call establishment differs greatly between the individual implementations. There
is unexpectedly no differentiation between eCalls with long numbers and with TS12 call set
up though in the dormant implementation the registration in the network takes place prior to
the call set up phase.
D4.3 Final results
17/12/2014 26 v 1.0
4.4 Results of Questionnaires
Questionnaires were used to complement the quantitative analysis by a qualitative
assessment of special aspects. These questionnaires are designed to help to identify issues,
concerns and areas for improvement for stakeholders. These questionnaires have been
developed for two areas: Powered Two-Wheeled (P2W) and Dangerous Goods transport.
The evaluation of the questionnaires was done by the pilot site’s representative for the
specific scopes: Spain for P2W and Luxembourg for Dangerous Goods Vehicles.
It was planned to distribute the questionnaires in all pilot sites but this failed in most pilot
sites (except Bulgaria) because of the short time allowed and the fact that stakeholders were
missing who were willing to accept responsibility for this.
Spain received more than 600 answers to their questionnaire and created an excellent report
which is attached in the Annex of this document (eCall for P2W - User Acceptance Report).
The conclusions of this report are summarised in the following chapter.
The results of the Dangerous Goods questionnaire are summarised by the Luxembourg pilot
site in the subsequent chapter.
4.4.1 Powered Two-Wheeled
The user acceptance study conducted by RACC draws some interesting conclusions:
- On the sample of surveyed motorcycle drivers: one in three respondents has suffered an
accident that required the emergency services to intervene. The results don’t provide a
statistically relevant significance between the experience or age of the motorists and the
accidents. More than half of the respondents who has experienced an accident didn’t alert
the emergency services. Not surprisingly the average arrival time of the emergency services
at the accident site is significantly higher in inter-urban areas than urban areas. However, in
both cases it was clear that the arrival time at the accident site could be clearly improved, for
example with the help of an automatic alert to the closest PSAP.
- On the level of acceptance of the eCall service for motorcycles: a large majority of
respondents motorists would like to have eCall in their motorcycles, and in slightly smaller
proportions they would be willing to change their helmet to have the full functionality of the
system, although they consider that the estimated increase in price compared to the price of
D4.3 Final results
17/12/2014 27 v 1.0
a conventional helmet is rather expensive. On the transition to a future eCall service for
motorcycles, users are receptive to paying for aftermarket devices to use eCall on their
motorcycles manufactured before the system becomes mandatory.
- On the expectations of users: surveyed motorists expect a high functionality from the eCall
service for motorcycles, and require some features that, conceptually, are outside the scope
of the eCall service. In this case, these features (sending of personal data and medical
history of the driver and passenger, etc.) might be implemented as value added services by
private service providers.
From the results of the first laboratory tests, as well as the expectations generated among
potential users, we can conclude that, although there are still many unresolved issues
(technical, standardisation, conceptual, privacy-related, ergonomics-related, costs, etc.), this
service will reduce fatalities, especially in interurban areas by providing a faster and more
efficient management of motorcycle accidents.
D4.3 Final results
17/12/2014 28 v 1.0
4.4.2 Dangerous Goods Vehicles
The HeERO2 project has extended its evaluation of eCall support to dangerous goods
transports.
An important question is: How can the PSAPs get information about which dangerous goods
are loaded into a vehicle that has just reported an accident via the eCall service?
The standard eCall message transfers a Minimum Set of Data (MSD), containing
“emergency relevant” information. To embed information about the loads of commercial
vehicles, the data part of the Optional Additional Data can be used in two ways:
It could contain all the relevant data that needs to be transferred to the emergency
services
It could contain a reference to an external source with the relevant data e.g. a web
service providing detailed information about loaded dangerous goods or a link to a pdf
with the transport documentation.
To get a feeling how the users – in this case the logistic companies – think about the different
solutions for providing dangerous goods information with eCall we created a questionnaire
that asked the users for their opinion on which option is more relevant for them.
In addition to the general questions the questionnaire had three main sections:
1. Which information do you think it is important for the emergency services to know
when your truck is involved in an incident?
( 0 = ‘not important at all’ and 5 = ‘very important’)
0 1 2 3 4 5
What type of dangerous goods are loaded (UN-number)
The danger code of the goods
What quantity of dangerous goods are loaded
The current status of the dangerous good e.g. temperature
What damage has occurred to the containers
What damage has occurred to the truck
Who is the sender
Who is the receiver
What type of truck is involved
What speed was the truck going before the accident
How the DGs should be handled after the accident
2. Which information would you consider important to provide to the emergency services
in that your truck is involved in an incident:
( 0 = ‘not important at all’ and 5 = ‘very important’)
D4.3 Final results
17/12/2014 29 v 1.0
0 1 2 3 4 5
What type of dangerous goods are loaded (UN-number)
The danger code of the goods
What quantity of dangerous goods are loaded
The current status of the dangerous good e.g. temperature
What damage has occurred to the containers
What damage has occurred to the truck
Who is the sender
Who is the receiver
What type of truck is involved
What speed was the truck going before the accident
3. In case that your truck is involved in an incident which way would you think is a
practical way to provide the information to the emergency services?:
( 0 = ‘not practical at all´ and 5 = ‘very practical’)
0 1 2 3 4 5
The information can be stored in a device in the truck and transferred to the emergency services in case of an accident
The information is stored in an electronic transport document e.g. a pdf file. In case of an accident, this document is provided to the emergency services.
The dangerous goods information is stored in your own database In case of an accident you would grant access to the needed information to the emergency service
The dangerous goods information is stored in the database of a centralised secured service. In case of an accident this service provides the needed information to the emergency service
The questionnaire was send to all HeERO2 partners for distribution to their logistic
companies and to 72 members of the Logistic Cluster Luxembourg. Unfortunately the
feedback was very low. We only received 3 answers from Luxembourg and 3 answers from
Bulgaria. However the results are still very interesting as they show a trend:
In order to maintain the anonymity of the persons who provided their feedback, we will
examine the average value of the answers. The responders were 3 large, 2 medium and 1
small logistic company from Bulgaria and from Luxembourg.
1. Which information do you think it is important for the emergency services to know
when your truck is involved in an incident?
( 0 = ‘not important at all’ and 5 = ‘very important’)
What type of dangerous goods are loaded (UN-number) 5.0
The danger code of the goods 4.2
What quantity of dangerous goods are loaded 4.0
The current status of the dangerous good e.g. temperature 3.3
D4.3 Final results
17/12/2014 30 v 1.0
What damage has occurred to the containers 4.3
What damage has occurred to the truck 3.8
Who is the sender 3.0
Who is the receiver 2.3
What type of truck is involved 2.2
What speed was the truck going before the accident 1.5
How the DGs should be handled after the accident 4.8
2. Which information would you consider important to provide to the emergency services
in that your truck is involved in an incident:
( 0 = ‘not important at all’ and 5 = ‘very important’)
What type of dangerous goods are loaded (UN-number) 5.0
The danger code of the goods 4.2
What quantity of dangerous goods are loaded 4.0
The current status of the dangerous good e.g. temperature 3.5
What damage has occurred to the containers 4.3
What damage has occurred to the truck 3.8
Who is the sender 3.0
Who is the receiver 2.3
What type of truck is involved 2.3
What speed was the truck going before the accident 1.3
3. In case that your truck is involved in an incident which way would you think is a
practical way to provide the information to the emergency services?:
( 0 = ‘not practical at all´ and 5 = ‘very practical’)
The information can be stored in a device in the truck and transferred to the emergency services in case of an accident
2.3
The information is stored in an electronic transport document e.g. a pdf file. In case of an accident, this document is provided to the emergency services.
1.8
The dangerous goods information is stored in your own database In case of an accident you would grant access to the needed information to the emergency service
0.5
The dangerous goods information is stored in the database of a centralised secured service. In case of an accident this service provides the needed information to the emergency service
3.7
All responders agreed that it is very important that the emergency services know about and
receive information about the type of the dangerous goods (the UN-number) and the danger
code. Nearly all the responders think that the quantity of the dangerous goods loaded is
important or very important information to have and that it is important to know about the
damage of the container, while the damage to the truck does not seem to be important at all.
The responders also think that it is very important that the emergency services receive
information about the handling of the dangerous goods. The information about the sender
D4.3 Final results
17/12/2014 31 v 1.0
and receiver of the dangerous goods seems to be less important. Not surprising nearly
nobody wants to see that the speed of the truck is provided to the emergency service.
The most interesting result concerned the question how the information about the dangerous
goods loaded should be provided:
Most of the responders see a central database as the best way for providing this information,
while responders from the large logistic companies say that a central database is not useful
for privacy reasons.
Nearly all responders agree that a link to an electronic transport document or access to a
database of the logistic company is not useful.
The storage of the dangerous goods in the on-board units is seen much differentiated:
3 responders coming from the bigger companies say it is very useful, the other 3 say it is
absolutely not useful.
Conclusion:
Even with the low number of responders the questionnaire shows some interesting results:
There is a common agreement about which information is needed by the emergency
services and should be provided to them: the type of the dangerous goods, the quantity and
if possible the damage of the container. The other information seems to be not as relevant.
Regarding the manner the information is provided to the emergency service, a central
database that secures full privacy of the data could be useful and could be recommended.
Storing the dangerous goods information in the IVS in the truck seems only is feasible for
large logistic companies that can afford the higher costs of the devices and the training of the
drivers.
4.5 Recommendations on Performance Requirements
The KPIs in WP4 were developed to measure the performance of the components and
processes. Within the HeERO projects the aim was to get values that measured “best
practise” and not to reach target values. Based on the results of the KPIs, in HeERO 1
recommendations for target values were developed and then updated. The KPIs were
defined only for those critical values which were measured during test drives on roads.
Many of these KPIs measure the critical spans of time, which are influenced by the
respective implementation of the IVS and/ or PSAP suppliers. Other KPIs measure the
D4.3 Final results
17/12/2014 32 v 1.0
accuracy of the GNSS or the correctness of the heading information. This has to be
complemented by thresholds that are only measurable in a laboratory such as antenna gain,
acceleration or voice quality.
The following list provides a short definition of some of the KPIs together with the thresholds.
KPI05 describes the time duration from the initiation (automatically or manually) of an
eCall to the presentation of the MSD content in the PSAP.
KPI07 represents the time the transmission of MSD blocks the voice channel. The
time the voice channel is blocked can be defined as a time between a successful call
setup (“connected” is reported by the network) and the opening of voice
communication in both directions after the MSD has been transmitted successfully or
the MSD transmission has been abandoned (after time out) and the voice
communication has been opened on both sides in both directions.
KPI07a evaluates the duration of the voice channel blocking if an automatic
retransmission of the MSD is initiated by the IVS on request by the PSAP.
KPI08 refers to the observed time difference between the time of the eCall initiation
(automatic and manual) and the time of the eCall reception at PSAP.
[seconds] mean median std. dev. minimum maximum
KPI05 12.9 13.0 3.3 8.6 21.0
KPI07 9.2 9.4 2.8 3.0 13.0
KPI07a 8.7 9.3 3.8 1.9 17.0
KPI08 7.2 6.8 4.0 2.4 17.0
Table 2: recommended values for KPIs 05 to 08
For the following KPIs the thresholds are specified directly.
KPI05, the time for MSD presentation shall be around 8 seconds based on
KPI07 plus average call establishment time of 3.5 to 4 seconds for a normal
call, as such about 3 seconds for emergency set up
KPI07, the voice blocking time should be as specified by the ETSI TS 126 269
around 4 seconds
KPI09, the accuracy of position shall reach the typical GNSS accuracy of 3 to
5m. In addition there shall be requirements for signal strength and how much
loss of signal is allowed.
For KPI13 the test criterion shall be to measure the travel direction not the
direction of the vehicle to be able to identify the right lane of the highway with
physical separation
D4.3 Final results
17/12/2014 33 v 1.0
4.6 General Recommendations
The following recommendations are provided:
The heading information is given to the PSAP as required by EN 16072. The
manufacturers have to be made aware of these requirements.
When retransmitting the MSD the IVS should update the MSD content
according to EN16072.
For type approval a performance requirement for audio should be defined and
evaluated by the technical services.
Voice channel blocking time and MSD transmission times have to be as short
as possible and be in line with the defined thresholds for these KPIs. Vehicle
manufacturers shall implement best practise to minimize voice channel
blocking time.
In order to roll-out the use of filtering instances, procedures, guidelines,
criteria and rules are to be set up to provide the long numbers of PSAPs to
filtering instances. It is however recommended to do steps forward in setting
up this certification framework together with the definition of the conformity
assessment for PSAPs
Cross border communication between adjacent member states is an important
requirement for successful deployment of eCall. Today, emergency services
between different member states in Europe are not interconnected for data
exchange. A further study should evaluate data integration concepts in more
detail
Looking to the next generation of emergency calls, there the use of in-band
modem technology is to be replaced.
4.7 Conclusions
The intention of having HeERO pilot sites has been to evaluate if the requested performance
of the eCall service can lead to a deployment of the approved eCall standards in the existing
public mobile networks and within the existing 112 system. This means that the testing has
had a strong focus on the eCall standards and on capturing the key performance indicators,
the KPIs. Other issues, such as the response time of the rescue services and ambulances,
the use of EUCARIS and the use of the VIN in the operational rescue chain, as well as non-
operational issues, such as legal liability, privacy issues, periodic time inspections, change of
a car ownership, etc were not included in the work of the HeERO pilot sites.
The outcome of the tests performed tests reported in this document confirm that pan-
European eCall works according to expectations however there is still room for
improvement, both in European standards and in its implementation by the suppliers.
D4.3 Final results
17/12/2014 34 v 1.0
One still open issue is the lack of commitment from some MNOPs to implement the
eCall flag.
D4.3 Final results
17/12/2014 35 v 1.0
5 Results of Pilot Sites
5.1 Belgium
5.1.1 General
The Belgian pilot site planned to install a filtering instance at Touring, an eCall-PSAP service
centre at Astrid and to drive around with 6 IVS from NXP. Also the eCall-flag would be
deployed by Mobistar in part of the country.
The architectural discussion was held between Astrid and Touring for interconnection of the
2 servers. Following picture shows the way the different servers were interconnected and
where/how the data was stored.
Figure 7: server architecture (BE)
5.1.1.1 Filtering instance at Touring
The filtering instance was installed on a test server of Touring. The server SW was bought
from Oecon. Also an in-band modem was bought to receive the eCalls. The EN16102
interface was provided to Astrid. The necessary access through firewalls and other security
mechanisms was provided. The picture below shows a snapshot of the graphical user
interface of the Oecon server.
GSM/PSTN
PABXModem
PABX
PCbrowser
Filtercentrale
Push X
ML
Web of XML pull
CIC/PSAP DatacenterAstrid
D4.3 Final results
17/12/2014 36 v 1.0
Figure 8: graphical user interface of eCall server (BE)
Although hardware was rather old (test-server) the server performed well and almost all
necessary tests could be executed. Only the re-transmit function was missing at the filtering-
instance side. This could be bought as an extension to the server-SW, but additional and
unplanned budget would be needed.
5.1.1.2 PSAP at Astrid
Astrid subcontracted the development of a test server to a small company. The SW was
installed on a test server of Astrid. It was properly receiving the MSDs. See below a snapshot
of the GUI.
The test-PSAP was setting up connection through the firewalls towards the test server at
Touring, using XML to extract all MSDs from the server and storing them in a local database
for access by the GUI. This concept functions quite well. It is however to be noted that, for a
real roll-out, an agreement should be made between PSAPs and filtering instance on filtered
transfers of the MSDs. As the implementation of these filtering rules is probably not really
difficult from technical point-of-view, this was not further worked out. Moreover, even if MSDs
would be sent unfiltered to the PSAP, this should not jeopardise the operation of the
services, nor impact on the operation of the operators at both the filtering instance and the
PSAP.
D4.3 Final results
17/12/2014 37 v 1.0
Figure 9: snapshot of the GUI on PSAP (BE)
5.1.1.3 IVS
NXP, partner in the project, was to provide 6 IVS. For this they co-operated with S1NN. The
IVS were delivered (see picture right), but first tests revealed several technical issues. These
were solved throughout the first part of the project, besides of one important issue: the audio-
quality was so bad that the operator at the filtering instance could not understand the person
speaking in the IVS.
Figure 10: IVS by NXP/S1NN (BE)
D4.3 Final results
17/12/2014 38 v 1.0
The cause of the technical issue was a configuration mismatch in the SW provided by NXP.
During the project, NXP transferred its activities related to HeERO2 to Telit. Although, S1NN
remained supporting the project, nor NXP, nor Telit solved the audio issues.
Figure 11: IVS by Geneva Microsystems (BE)
During the project, Geneva Microsystems was contacted to supply IVS with audio
functionality. Geneva Microsystems also provided 6 IVS. Tests with these IVS performed
well, but also here, a few technical issues needed to be resolved. Most of these issues were
solved, but unfortunately in the last SW-version, a new SW-bug slipped in which resulted in
non-functional GPS-reception. One of these technical issues was low reception and
performance in the 900 MHz band of the GSM network. To solve this, Geneva Microsystems
needed to execute a re-design. Geneva Microsystems started a new product design, but this
was unfortunately not available during the course of the project.
Summarized: when first tests were executed, none of both IVS performed good enough to be
distributed to the 6 assigned test drivers. This resulted in execution of only a limited number
of tests.
5.1.1.4 eCall flag by Mobistar
Mobistar rolled out the eCall flag in part of the Belgian country, the blue area on the picture
aside, covering about 7000 km². Mobistar also provided 12 SIM-cards for insertion in the IVS.
In the routing of the switching network, a special filter was installed for those 12 SIM-cards
that, when dialling 112 issuing an emergency call with eCall flag, the call was re-routed to the
Touring Filtering instance. The eCall-flag detection and handling functioned well. However,
some restrictions detected due to choices made:
The re-routing for these 12 SIMs was only available in the Mobistar network and not
on the 2 other networks (Belgacom and Base). At one occasion, it happened that the
IVS could not detect the Mobistar network (probably caused by a weak signal at a
D4.3 Final results
17/12/2014 39 v 1.0
specific location), so the IVS registered in another network. When the emergency call
was signalled, the call was routed to a real PSAP. As this is the fall back later on, this
is not a problem for roll-out.
It was not possible to do cross border tests with 112 eCall flag, as the areas where
the eCall flag was rolled out in Belgium and neighbouring countries (Luxemburg)
were not next to each other.
5.1.1.5 Test coordination at Testronic labs
At Testronic labs, IVS were available to execute tests. Also access to the filtering service
was provided. Access to the test-PSAP was not provided, but this link showed to give the
least problems.
5.1.2 Recommended KPIs
In the beginning of the project, it was estimated that timestamps to measure KPÏs would be
available. Unfortunately, this was not the case.
The table hereunder shows the different KPI’s we have measured.
KPI Description KPI Description
01b Number of manually initiated eCalls 06 Success rate of established voice
transmissions
02a Success rate of completed eCalls
using 112
07a Duration of voice channel blocking
02b Success rate of completed eCalls
using long number
08 Time for call establishment
03 Success rate of received MSDs 28a Number of cross-border tests
04 Success rate of correct MSDs 28b Number of interoperability tests
Table 3: measured KPIs (BE)
The other KPI’s haven’t been measured because of lack of means or because it was not
possible (e.g. KPI_01a: number of automatically initiated eCalls).
The number of eCall tests was about 532 and for the two IVS 375. There were 443 without
eCall flag and 89 with eCall flag (respectively 286 and 89 with the two IVS). Despite this
large amount of data, it was not possible to evaluate statistically more than 10 tests.
D4.3 Final results
17/12/2014 40 v 1.0
5.1.3 Evaluation results
5.1.3.1 Time series
For the time series, we have hereunder evaluated two KPI’s.: The duration of voice channel
blocking (KPI_007a) and the time for call establishment (KPI_008).
Figure 12 : time series KPI 7 and 8 (BE)
5.1.3.2 Statistical evaluation
For the KPI’s analysed we have the following results:
KPI NXP Geneva Microsystems
1b 323 52
2a 75% 79%
2b 75% 79%
3 73% 73%
4 73% 73%
7a 13 13
8 24 24
Table 4 : Results KPIs (BE)
The results for KPI’s nr 3, 4, 7a and 8 are the same for the two IVS because it was not
possible to distinguish them in our server. Furthermore, the results for KPI’s 3 and 4 are the
same because all the received MSDs were correct.
D4.3 Final results
17/12/2014 41 v 1.0
We have seen that for KPI’s 7a and 8, we had time series.
Specifically, the time before we have the audio connection (KPI nr 8) is above 24 s. It is too
high and not acceptable for road safety.
KPI_007a KPI_008
Mean 13 s 24 s
Median 14 s 25 s
Minimum 6 s 19 s
Maximum 20 s 31 s
Range 14 s 12 s
Standard deviation 5 s 4 s
Skewness -0.1464 0.02912
Kurtosis -1.4111 -1.5810
Table 5 : statistics KPIs 7 and 8 (BE)
Figure 13: Distribution KPI 7a (BE)
D4.3 Final results
17/12/2014 42 v 1.0
Figure 14: Distribution KPI 8 (BE)
For these two distributions, they aren’t similar to Gauss curves. The standard deviation is of
course high.
5.1.3.3 Discussion of results
In general, besides the technical open issues, which have not been solved due to resourcing
and priorities at the providers for the IVS and the system, the results of the tests are quite
good. However, still a few items to be noted:
The use of In-band modem technology is not state-of-the-art. Between pushbutton
and audio connection, timings of 20 seconds were often measured, mostly consumed
to transfer the MSD in the audio band. A silence of 20 seconds in case of
emergencies is not acceptable.
During the tests, retransmits could not be tested. As the combination of filtering
instances and PSAPs with retransmits poses several questions, it is a pity that this
could not be further worked out. However, all considerations have been captured in a
document and have been sent to the WP6-leader for further integration into
recommendations.
5.1.4 Interoperability tests
Cross border testing with eCall flag is not straight forward. For this, it is advised to set up a
consortium of different countries with MNOs who are capable and willing to roll out the eCall
D4.3 Final results
17/12/2014 43 v 1.0
flag in the different countries with adjacent geographical areas. Instead of Cross border
testing interoperability tests were executed in following tests:
Members of the LUX pilot site visited Belgium and executed tests of their IVS using the long
number of the filtering instance. These tests gave similar results as former tests, of course
noting the shortcomings which were known.
5.1.5 Recommendations
The use of in-band modem technology is to be replaced by out-of-band technologies
or other performant architectures. By enabling filtering instances to connect to
PSAPs, the market for private eCall will become attractive, thus replacing the public
eCall and related in-band technologies. DEL 6.7 will elaborate in more detailed on the
strategy behind filtering instances.
In order to roll-out the use of filtering instances, procedures, guidelines, criteria and
rules are to be set up to provide the long numbers of PSAPs to filtering instances. It
was expected to make steps forward and create (draft) certification guidelines for
certifying filtering instances to connect to PSAPs in Belgium. Due to the complex
situation of both the Belgian political playfield as well as the uncertain and shifting
approach for introducing eCall in the European commission, the certification
guidelines have not been drafted, yet. It is however recommended to do steps
forward in setting up this certification framework.
Per today, cross border testing across different countries was not feasible.
Emergency services between different countries in Europe are not interconnected in
a uniform way, even outside the scope of eCall. Also MNOs between different
countries are often in competing positions, making cross border tests not a
straightforward test case. To do steps forward, in cross border and cross country
eCall, an integrated project should be set up on European level.
5.1.6 Conclusions
Although there are technical issues and many of the tests could not be executed as planned,
it is not to be expected that technical issues will be the roadblock for rolling out eCall in
Belgium.
5.2 Bulgaria
5.2.1 General
The information within this chapter is based on the test results including the second
realization stage of the Bulgarian pilot - after eCall Flag implementation, Sofia’ PSAP SW
application upgrade for integration with eCall test environment, connection to national VIN
database or EUCARIS. During the test phase the incoming calls were treated as test calls
with further processing only in the test environment, incl. simulation of emergency call centre,
D4.3 Final results
17/12/2014 44 v 1.0
or were answered automatically without intervention. The test client was used to display
MSD data on its screen (incl. GIS location) and the voice channel was activated.
This chapter outlines the Bulgarian results including results of the second evaluation phase
of HeERO2.
5.2.2 Recommended KPIs
The recommended KPIs were successfully evaluated during the testing periods in both
phases of the Bulgarian pilot.
5.2.3 Evaluation results
5.2.3.1 Evaluation process
In order for all possible KPIs to be evaluated IVS and PSAP logs were made. Some KPI
parameters are based mainly on IVS logs (assuming that the IVS events are synchronized
with PSAP events with time difference <0.5 seconds) and only checked for compliance with
the PSAP logs afterwards. This is done for double check of the test results.
In order to evaluate KPIs IVS devices from two manufacturers were used – TUS’ IVS and
ICOM’s IVS.
For the KPI logging and evaluation some IVS devices were configured in special testing
mode making automatic eCalls in predefined periods of time making more tests while driving.
These tests produced logs used to measure most of the KPIs that don’t require operator to
be logged on from PSAP side. Some of the automatically performed eCall tests were made
from stationary devices (no heading information available), but most were conducted from
inside moving vehicles travelling across the country. These tests were performed in a test
scenario where the car was equipped with a device operating in automatic mode and an
interactive voice response system answering from PSAP side. A pre-recorded message was
played to the driver after MSD was sent in order to confirm the established voice connection.
These tests were performed in the manner showed in Figure 15.
IVS power ONIVS initialization
procedureUpdate the MSD &
Call 112
Send MSDDriver hears
automatic IVR response
IVS write log file Wait for timeout
Figure 15: testing process (BG)
D4.3 Final results
17/12/2014 45 v 1.0
For the KPIs involving actual voice connection to 112 operators the eCalls were made
manually including initialisation, MSD transfer, talking to test operator, confirming voice
connection and integrity of received data.
In order to be able to evaluate as many KPIs as possible the IVS devices record different
events and parameters. The following table shows different types of IVS events and
parameters and the logging capabilities of the manufacturers:
Event Short Description
IVS
Manufacturer
TUS ICOM
Dial Start IVS starts dialling the PSAP. X X
Ring Start Call has reached PSAP. X X
Call Established
Connection is established. Due to the PUSH mode
voice connection is open, but is used by the In-
Band modem.
X X
IVS sends ‘START’
signal
IVS attempts to sync with PSAP and asks for MSD
transfer. X X
IVS starts sending
MSD MSD transfer has started X X
IVS stops sending
MSD
MSD transfer has ended. At this point voice
communication is available between operator and
occupant in the car.
X X
Call end Operator or occupant hangs up. X X
Parameter
MSD successfully
sent
‘Yes’ or ‘No’ based on PSAP feedback to IVS.
PSAP feedback includes info about MSD’s
successful transfer and decoding.
X X
eCall flag Shows whether None /Auto/Manual flag is used. X X
GSM level Signal strength of GSM network. X X
Satellites in view Number of usable satellites. X X
Heading Heading in degrees according to North. X X
D4.3 Final results
17/12/2014 46 v 1.0
GDOP Shows the geometric dilution of precision. X X
GPS coordinates Accident GPS coordinates. X X
Speed Speed over ground. X X
Time between
successful positioning
fixes
Measures the time between two GPS position
fixes with accuracy of 1 second. X X
Number of
passengers
Implemented in a later version as a device
configuration parameter. X X
* True GPS coordinates used for Recent Vehicle
Location n-1 and n-2. X X
Table 6: Events and parameters recorded by different IVS devices (BG)
The PSAP is capable of logging all internal events during an eCall. The events are separated
in two categories – ‘info’ and ‘debug’. It also logs and archives all information regarding the
MSD message: its binary representation and its decoded content.
The following KPIs were measured mainly with IVS logs and then only checked for
compliance with the PSAP log: KPI_001a, KPI_001b, KPI_002a, KPI_002b, KPI_003,
KPI_006, KPI_007a, KPI_009, KPI_010, KPI_011, KPI_012 and KPI_013.
The following KPIs were measured by cross examining IVS and PSAP logs: KPI_004,
KPI_005, KPI_006, KPI_008, KPI_021, KPI_022, KPI_023, KPI_024, KPI_025 and KPI_028b
(using logs from foreign PSAPs).
5.2.3.2 Evaluation results
KPI TUS IVS/
Test PSAP Sofia
ICOM IVS/
Test PSAP Sofia
Unit Remarks and notes about
method of testing
KPI_001a 1 609 45 - Logging in IVS
KPI_001b 1 074 147 - Logging in IVS
KPI_002a 93 98 % Logging in IVS and PSAP (if
eCall flag is available)
Note: eCall flag is available in
Mtel network
KPI_002b 85 96 % Logging in PSAP (short
D4.3 Final results
17/12/2014 47 v 1.0
number is used during the
initial tests)
KPI_003 93 85 % Logging in IVS and PSAP
KPI_004 100 100 % Logging in IVS and PSAP
KPI_005 11 11 s Logging in IVS and PSAP
KPI_006 98 100 % Logging in IVS and PSAP
KPI_007a 5 3 s Logging in IVS
KPI_007b 5 3 s Logging in IVS
KPI_008 4 5 s Logging in IVS and PSAP
MNO measures once - 2.5
sec + 112 to PSAP time
KPI_009 acceptable 1.5 m Logging in IVS
KPI_010 10 >12 - Logging in IVS
KPI_011 1.37 <1.5 - Logging in IVS
KPI_012 1 1 s Logging in IVS
KPI_013 100 100 % Logging in IVS
KPI_014 100 100 % Logging in IVS and PSAP
Note: Test VIN data base is
used.
KPI_015 N/A N/A % Logging in PSAP
(if Bulgaria becomes a
member of eCall EUCARIS
agreement or after connection
with Traffic Police vehicle
registers DB)
Note: The connection to
EUCARIS is not available.
KPI_016 N/A N/A s Logging in PSAP
(if Bulgaria becomes a
member of eCall EUCARIS
agreement or after connection
D4.3 Final results
17/12/2014 48 v 1.0
with Traffic Police vehicle
registers DB)
Note: The connection to
EUCARIS is not available.
KPI_017 N/A N/A % Logging in PSAP
Note: The value is not
available as no real PSAP is
used but only "test PSAP".
KPI_018 N/A N/A s Logging in PSAP
Note: The value is not
available as no real PSAP is
used but only "test PSAP".
KPI_019 N/A N/A s No
KPI_020 N/A N/A % No
KPI_021 92 10 - Logging in PSAP
Note: Measure the cases
when an operator accepts the
eCall in test environment in
PSAP Sofia
KPI_022 100 100 % Logging in PSAP
KPI_023 <3 <3 s Logging in IVS
MNO measures once - 2,5
sec.
KPI_024 <1 s Logging in IVS
KPI_025 2.5 2.5 s Logging in PSAP
Note: The source of
information is COCON report.
KPI_026 N/A N/A s Logging in PSAP
Note: The value is not
available as no real PSAP is
used but only "test PSAP".
KPI_027 N/A N/A s No
D4.3 Final results
17/12/2014 49 v 1.0
KPI_028 a N/A N/A - Logging in PSAP
KPI_028 b 104 8 - Logging in IVS and PSAP
KPI_028 c N/A N/A - No
KPI_29 N/A N/A s Logging in PSAP
Note: The value is not
available as no real PSAP is
used but only "test PSAP".
Table 7: summary of results (BG)
5.2.3.3 Comments
KPI_001a, KPI_002a
The first phase of the Bulgarian pilot included testing before the implementation of the eCall
flag. Instead the short number 107 was used as a substitute to the eCall flag. This is why
tests performed with 107 are actually referenced and measured for both KPI_001a
(automatically initiated eCalls) and KPI_002a (Success rate of completed eCalls using long
number).
KPI_003
A link-layer ACK received by the IVS is counted as a successful delivery of the MSD to the
PSAP. The low value comes only from the automatic tests using ‘long’ number (107). The
success rate with the later manual tests using eCall flag is 100%.
KPI_007a, KPI_007b
This time is measured as the time elapsed between two events: IVS sends ‘START’ signal
and IVS stops sending MSD.
KPI_008
This time is measured as the time elapsed between an eCall is initiated (the GSM modem
starts dialling) until the PSAP picks up the call.
KPI_009
The number of available satellites is available and measured in all test sessions. However
this does not give the accuracy in meters, rather than a reference for the accuracy of the
D4.3 Final results
17/12/2014 50 v 1.0
positioning. Having in mind that the average number of usable satellites is 10, which gives
accuracy well below 100m, KPI_009 is marked as ‘acceptable’.
KPI_012
The IVS uses fixed positioning rate of 1 s.
KPI_013
Heading information uses the true direction of travel as delivered by the GPS module.
Contrary to the standard, heading is delivered using true geographical north instead of
magnetic north.
KPI_014
The decoding of the VIN number was done with a local VIN database configured for testing
purposes, but architecturally the same as a real-life solution.
KPI_021, KPI_022
The IVS logs voice calls, including call-backs. The value in KPI_022 is based on a verbal
connection between operator and driver.
KPI_024
The routing in the national 112 network is done entirely inside a software environment which
leads to very low latency times. The exact time cannot be measured, but with tests it was
confirmed that it is well below 1 second.
5.2.3.4 Time series
5.2.3.4.1 TUS IVS/ Test PSAP Sofia
D4.3 Final results
17/12/2014 51 v 1.0
Figure 16: KPI_007 time series TUS IVS (BG)
As it can be seen in Figure 16 the values for duration of voice block are very similar. In rare
cases there are extreme values, but analysis of the logs showed that these are cases where
caused by poor GSM signal strength forcing several retransmissions of the MSD.
5.2.3.4.2 ICOM IVS/ Test PSAP Sofia
Figure 17: KPI_007 time series ICOM IVS (BG)
D4.3 Final results
17/12/2014 52 v 1.0
Figure 18: KPI_008 time series ICOM IVS (BG)
5.2.3.5 Statistical evaluation
Table 8 shows different statistical parameters of KPI_007 and KPI_008.
KPI_007 KPI_008
TUS IVS ICOM IVS ICOM IVS
Minimum 4.4 2.0 1.0
Maximum 9.6 13.0 18.0
Mean 4.7 3.2 5.6
Median 4.4 3.0 5.0
Variance 2.3 2.5
Standard deviation 0.56 1.5 1.6
Skewness 3.351 5.1 4.05
Kurtosis 29.88 31.49
Table 8: statistical parameter (BG)
D4.3 Final results
17/12/2014 53 v 1.0
5.2.3.5.1 TUS/ Test PSAP Sofia
Figure 19: distribution of KPI_007 for TUS (BG)
In Figure 19 the distribution of KPI 7 shows, that the most data are between 4 and 6
seconds. Less but notable amount is between 6 and 8 seconds, which are still very
acceptable values.
5.2.3.5.2 ICOM/ Test PSAP Sofia
Figure 20: distribution of KPI_007 for ICOM (BG)
D4.3 Final results
17/12/2014 54 v 1.0
Figure 21: distribution of KPI_008 for ICOM (BG)
5.2.4 Interoperability tests
This chapter documents the results of the interoperability tests between TUS IVS, ICOM IVS
and PSAPs in Greece, Croatia and Romania.
PSAP KPI 2b KPI 3 KPI 4 KPI 6 KPI 7a
IVS
TUS
IVS
ICOM
IVS
TUS
IVS
ICOM
IVS
TUS
IVS
ICOM
IVS
TUS
IVS
ICOM
IVS
TUS
IVS
ICOM
GR 100 - 100 - 100 - 100 - 11 -
HR 100 100 100 100 100 100 100 50 7 4
RO 100 100 100 100 100 100 100 100 N/A N/A
Table 9: KPI 2b, 3, 4, 6 and 7a with foreign PSAPs (BG)
KPI_007a is not available due to lack of IVS log data and unavailability of the Romanian
PSAP for further testing after IVS logging upgrade.
5.2.5 Recommendations
The following recommendations are given:
The heading information is given to the PSAP as provided by the GPS receiver in the
IVS. This creates inaccuracies when the vehicle is not moving during eCall initiation.
D4.3 Final results
17/12/2014 55 v 1.0
Thus the IVS manufacturers should provide a better direction information by
implementing their own calculation algorithms, e.g. based on previous GPS positions.
When retransmitting the MSD not all IVS devices update the MSD content. As
updating the fields gives new information to the PSAP operator thus leading to better
judgement of the incident environment it is recommended that all IVS devices update
the content of the re-sent MSDs with the latest GPS and other available information.
5.2.6 Conclusion
The IVS devices used in all test sessions are designed and developed by two independent
manufacturers (TUS and ICOM), but perform in a very similar way and the KPIs measured
with them differ only in acceptable ranges. This means that in addition to the successful
interoperability tests during the project, PSAP implementation should not have problem
communicating with different brands of IVS equipment though both the IVS devices and the
PSAP are still in a pilot implementation stage.
The eCall flag is already implemented by one Bulgarian MNO – Mobiltel. The other two
national MNOs are already conducting tests in test environments.
5.3 Denmark
5.3.1 General
This chapter outlines the Danish results of the first and second phases of testing within
HeERO2.
The results reflected here, are solely derived from vehicles equipped with IVS driving
throughout Denmark and calling the Danish HeERO2 PSAP Test Environment. The figure
below shows the locations in Denmark where the calls have been made from.
D4.3 Final results
17/12/2014 56 v 1.0
Figure 22: Location of pilot test calls, as indicated in the IVS-unit (DK)
It shows all attempted call, where a satellite position within the frame of the map was
registered by the IVS-units. Please not, that the land mass between Zealand and Bornholm
belongs to Sweden. Please also note the four dots in the Baltic Sea, which are errors.
5.3.2 Recommended KPIs
The recommended KPI’s 1a and 2a couldn’t be tested during the project. The test-design did
not allow for testing automatically initiated calls, and none of the Danish Mobile Network
Operators supports the eCall flag yet.
5.3.3 Evaluation results
5.3.3.1 Evaluation process
Basis of the evaluation are the datasets derived in field testing with a number of eCall-
equipped vehicles driving around Denmark on other business.
D4.3 Final results
17/12/2014 57 v 1.0
Figure 23 – An IVS-unit installed in a test vehicle
5.3.3.1.1 Phase one testing
Phase one testing was conducted in June and July 2014 with drivers calling to a setup with
an Intelligent Voice Response (answering machine).
Phase one evaluation is based on: Log from IVS, Log from PSAP (eCall-router), and log from
drivers (manually entered information in a log book).
5.3.3.1.2 Phase two testing
Phase two testing was conducted Monday 22 September 2014 from 9 am to 3 pm, and
Tuesday 23 September 2014 from 9 am to 3 pm with a dedicated PSAP operator standing by
at the HeERO2 eCall test equipment.
Phase two evaluation is based on: Log from IVS, Log from PSAP (eCall-router), log from
drivers (manually entered information in a log book), log from PSAP operator (manually
entered information in a log book).
5.3.3.1.3 Time deltas
It was not possible to calibrate the timestamps in the IVS-logs with the timestamps in the
PSAP-log. In some of the IVS-logs time and date was not always set correct (showing time
since boot, instead of actual time).
At the PSAP, the clock loses seconds every day.
Therefore, some of the time deltas in this report differ from the definitions. Please note, that
the resulting time deltas have been validated in lab-testing with external clocks (stop
watches).
Please see below for calculation of KPI5, KPI 7 and KPI 8
D4.3 Final results
17/12/2014 58 v 1.0
5.3.3.1.4 IVS-units tested
Nine IVS-units participated in the test. All units were retrofit devices. For the purpose of ease
of use for the test drivers, the units were put in small cases and all the test driver had to do,
was add power from the vehicle.
Five units from Fujitsu-TEN with special profile SIM cards installed (could only call
to/be called from a limited set of telephone numbers: 112 and the long-number used
for testing). These units were part of both phase one and phase two
Two units from Fujitsu-TEN with normal Danish SIM-cards installed. These units were
part of both phase one and phase two
Two units from GMV with Spanish SIM-cards installed. These units were only part of
phase two
Figure 24: The IVS-units used for testing (DK)
5.3.3.1.5 KPI 5
Instead of calculating T2-PSAP – T0-IVS, we simply add up the results from finding KPI 7
and KPI 8.
5.3.3.1.6 KPI 7a
Fujitsu-TEN
Instead of calculating T2-PSAP – T1-IVS, we calculate T2-PSAP – T1-PSAP and add the
time indicated in the IVS-logs for the establishment of a communication channel
(EVT0_IND_VOICE_CALL_START), until the modem synchronization commences (first
“EVT0_IND_ECALL_SYNC_DETECTED” after “EVT0_IND_VOICE_CALL_START”).
GMV
D4.3 Final results
17/12/2014 59 v 1.0
Instead of calculating T2-PSAP – T1-IVS, we calculate T2-PSAP – T1-PSAP.
Please note further
T2-PSAP is the timestamp for when the MSD can be presented at the PSAP. In the test-
setup, and in the live environments, where will be an additional delay due to an obligatory
voice response telling callers in the test setup, that they have “reached the eCall test
environment, please hold.”.
5.3.3.1.7 KPI 7b
KPI7b are calculated from the eCall routers data as the difference in time between the two
following events:
event:dtmf_detected signal=5 call-ref=INTERNAL
event:call_voice_connection_established
5.3.3.1.8 KPI 8
Fujitsu-TEN
Instead of calculating T0-PSAP – T0-IVS, this KPI have been derived directly from
timestamps in the Fujitsu-TEN IVS log, as the time from initiating (T0-IVS) to the IVS has
established communication channel (EVT0_IND_VOICE_CALL_START).
GMV
Instead of calculating T0-PSAP – T0-IVS, this KPI have been derived directly from
timestamps in the GMV IVS log, as the time from initiating (T0-IVS) to the IVS has
established communication channel (MSD_Start).
5.3.3.1.9 KPI 9 and KPI 13
The evaluation of „Accuracy of position“ and „Success rate of heading information“ are based
on the opinion of trained PSAP-operators only as „Acceptable“ or „not acceptable“.
To calculate the KPI 9, the formula given in D4.1 has been used (even though the Danish
Pilot site, do find the formula misleading): “Accuracy of position”=”Acceptable accuracy”/”not-
acceptable accuracy”
5.3.3.1.10 KPI 15 and 16
As Denmark is not part of EUCARIS, these two KPI’s cannot be tested.
5.3.3.1.11 KPI 17, 18 19, 20, 26 and 27.
As the Danish eCall implementation is based on a principle of „minimum work routine
changes“, there are no changes in the communication protocols between PSAP and TMC
and Rescue forces. Therefore, these KPI’s have not been evaluated.
D4.3 Final results
17/12/2014 60 v 1.0
5.3.3.1.12 KPI 10
Only the GMV-logs had information about number of satellites.
5.3.3.1.13 Dormant testing (KPI 32)
All IVS-units had the capability of letting the GSM-radio is dormant (not registering on the
network, until a call is made). All tests were performed with this setting.
5.3.3.2 Evaluation results for phase one
KPI Name of KPI Fujitsu-TEN
7 units Unit
KPI_001a Number of automatically initiated eCalls N/A -
KPI_001b Number of manually initiated eCalls 185 -
KPI_002a Success rate of completed eCalls using 112 N/A %
KPI_002b Success rate of completed eCalls using long number 94% %
KPI_003 Success rate of received MSDs 99% %
KPI_004 Success rate of correct MSDs 100% %
KPI_005 Duration until MSD is presented in PSAP 13.6 s
KPI_006 Success rate of established voice transmissions 95% %
KPI_007a Duration of voice channel blocking 9.9 s
KPI_007b Duration of voice channel blocking: automatic retransmission of MSD Phase 2 s
KPI_008 Time for call establishment 3.7 s
KPI_009 Accuracy of position Phase 2 %
KPI_010 Number of usable satellites N/A -
KPI_011 Geometric dilution of precision N/A -
KPI_012 Time between successful positioning fixes N/A s
KPI_013 Success rate of heading information Phase 2 %
KPI_014 Success rate of VIN decoding without EUCARIS N/A %
KPI_015 Success rate of VIN decoding with EUCARIS N/A %
KPI_016 Time for VIN decoding with EUCARIS N/A s
KPI_017 Dispatch time of incident data to rescue forces N/A %
KPI_018 Mean time to activate rescue forces N/A s
KPI_019 Dispatch time of incident data to TMC N/A s
KPI_020 Success rate of presented incident data in TMC N/A %
KPI_021 Number of successful call-backs Phase 2 -
KPI_022 Success rate of call-backs Phase 2 %
D4.3 Final results
17/12/2014 61 v 1.0
KPI_023 GSM network latency N/A s
KPI_024 112 national network latency N/A s
KPI_025 112 operator reaction time N/A s
KPI_026 Time for acknowledgement of emergency services N/A s
KPI_027 Total response time N/A s
KPI_028 a Number of cross-border tests N/A -
KPI_028 b Number of interoperability tests TBD -
KPI_028 c Number of cross regional tests N/A -
KPI_29 Dispatch time of Intermediate PSAP N/A s
KPI_30 Number of calls flagged as dangerous good N/A -
KPI_31 Number of successful access of dangerous goods information N/A -
KPI_32 Number of Dormant SIM card tests 185
Table 10 – Results derived from phase one pilot test (DK)
Please note, that KPI’s noted N/A are not measured (see above). KPI’s noted Phase 2 was
derived from phase two.
Figure 25 – geographical distribution of success rate of Calls made during phase one pilot test, with the satellite position registered in the IVS-units (DK)
D4.3 Final results
17/12/2014 62 v 1.0
The few red dots in Figure 28 indicate that either the call was not successful, or that the MSD
was not received at the PSAP.
5.3.3.3 Comments on results from phase one
The majority of failed calls occurred at the beginning of the test-period. There is
therefore a risk that these calls may have failed due to the drivers handling of the
equipment.
Some timestamps in both IVS-units and at eCall-router have been reconstructed. We
believe these reconstructions have been conducted so that they represent correct
timing. The cause for the need for reconstructions are:
o If an IVS-unit calls shortly after it has been booted, it has not yet received
timing information from the GSM-network.
o The clock in the eCall-router is missing approximately a minute a day.
Some of the logs in the IVS-units were not readable. For these calls, data have been
reconstructed by extrapolating data from eCall-router-logs with the drivers manually
entered log-info. Missing data have been omitted from the statistical calculations.
For the majority of the MSDs received at the PSAP, the flag “Position cannot be
trusted” was set, even though the position given, was pretty accurate.
5.3.3.4 Time series from phase one
The figure to the right shows the time series of the KPIs 5 and 7. Note that we do not have
data for KPI 7, for the event causing the maximum for KPI 5. Gaps in the curve indicates a
successful event, with lack of data in the IVS-log
D4.3 Final results
17/12/2014 63 v 1.0
5.3.3.5 Statistical evaluation from phase one
Table 11: Statistics of manual initiated calls for successful calls during first
phase (DK)
The statistics (Table 11) and the time series
(Figure 26), shows, that KPI 5 and KPI 7
normally are pretty close to a mean time of
13,5 and 10.0 seconds, with only a few calls
(approximately 10) causing relatively long
waiting time for the driver.
KPI5 KPI7
Minimum 10.1 7.1
Maximum 27.9 24.9
Mean 13.5 9.9
Median 13.5 10.0
Std dev 2.4 2.2
Skewness 2.1 2.5
Figure 26: time series for successful during first phase. (DK)
D4.3 Final results
17/12/2014 64 v 1.0
5.3.3.6 Evaluation results phase two
KPI Name of KPI
Fujitsu-TEN DORMANT SIM-CARD
5 units
Fujitsu-TEN Normal
SIM-CARD 1 unit
GMV Roaming
SIM-CARD
2 units Unit
KPI_001a Number of automatically initiated eCalls N/A N/A N/A -
KPI_001b Number of manually initiated eCalls 43 9 21 -
KPI_002a Success rate of completed eCalls using 112 N/A N/A N/A %
KPI_002b Success rate of completed eCalls using long number
93% 67%1 95%
%
KPI_003 Success rate of received MSDs 100% 100% 95%
%
KPI_004 Success rate of correct MSDs 100% 100% 100%
%
KPI_005 Duration until MSD is presented in PSAP 13.1 11.8 18.7
s
KPI_006 Success rate of established voice transmissions 93% 67% 100%
%
KPI_007a Duration of voice channel blocking 9.6 8.5 11.6
s
KPI_007b Duration of voice channel blocking: automatic retransmission of MSD
9.1 8.9 8.8 s
KPI_008 Time for call establishment 3.6 5.2 7.1
s
KPI_009 Accuracy of position 77% 20% 13%
%
KPI_010 Number of usable satellites N/A N/A 9.1 -
KPI_011 Geometric dilution of precision N/A N/A N/A -
KPI_012 Time between successful positioning fixes N/A N/A N/A s
KPI_013 Success rate of heading information 69% 0%2 6%
3 %
KPI_014 Success rate of VIN decoding without EUCARIS N/A N/A N/A %
KPI_015 Success rate of VIN decoding with EUCARIS N/A N/A N/A %
KPI_016 Time for VIN decoding with EUCARIS N/A N/A N/A s
KPI_017 Dispatch time of incident data to rescue forces N/A N/A N/A %
KPI_018 Mean time to activate rescue forces N/A N/A N/A s
KPI_019 Dispatch time of incident data to TMC N/A N/A N/A s
KPI_020 Success rate of presented incident data in TMC N/A N/A N/A %
KPI_021 Number of successful call-backs 0 5 14 -
KPI_022 Success rate of call-backs 0%4 100% 87% %
KPI_023 GSM network latency N/A N/A N/A s
1 This poor result may be caused by one faulty IVS or that the vehicle was driving in areas with risk of
poor network-coverage (or a combination of the two). 2 This KPI is based on one single IVS. There could be a problem with this single device, and not to the
fact, that it was equipped with a normal SIM card. 3 The GMV-devices are prototypes (as are the Fujitsu devices). The poor result is based on the fact
that the GMV-device sends data on the cars current heading, and not the heading the car was driving, prior to the call (accident). 4 Call back was tried 35 times. Please see comments below for explanation for this poor result.
D4.3 Final results
17/12/2014 65 v 1.0
KPI_024 112 national network latency N/A N/A N/A s
KPI_025 112 operator reaction time N/A N/A N/A s
KPI_026 Time for acknowledgement of emergency services N/A N/A N/A s
KPI_027 Total response time N/A N/A N/A s
KPI_028 a Number of cross-border tests N/A N/A N/A -
KPI_028 b Number of interoperability tests TBD TBD TBD -
KPI_028 c Number of cross regional tests N/A N/A N/A -
KPI_29 Dispatch time of Intermediate PSAP N/A N/A N/A s
KPI_30 Number of calls flagged as dangerous good N/A N/A N/A -
KPI_31 Number of successful access of dangerous goods information N/A N/A N/A -
KPI_32 Number of Dormant SIM card tests 43 9 21 -
Table 12 – Results derived from phase two pilot test (DK)
Figure 27 – geographical distribution of calls made during phase two, with the satellite position registered in the IVS-units (DK)
In Figure 27 the red dots that indicate that either the call was not successful, or that the MSD
was not received at the PSAP. Please also note the green dot in the Baltic Sea (and the
three red dots), where no vehicle was during testing.
5.3.3.6.1 Comments on results from phase two
In lab-testing 14 May 2014, all Fujitsu-TEN equipment was tested for call-back, and it
worked. BUT, when conducting the phase two „call back did NOT work for Fujitsu-
D4.3 Final results
17/12/2014 66 v 1.0
TEN equipment with profiled SIM-card (to simulate the Dormant SIM-card
requirements). So even though call back for Fujitsu-TEN with the special profiled SIM-
cards was tried 35 times, there was no success.
One of the Fujitsu-TEN units started phase 2 with calling 112, instead of the long
number as it was configured to call with. This gave some additional information from a
real life operator, who had difficulties hearing what the driver was saying.
It took at least one reboot of the IVS for it to get back to its proper configuration, and
we no longer harassed real PSAP-operation.
GMV-units were equipped with Spanish SIM-cards. As we only tested with long-
number, the roaming may have caused problems, which would not happen, if the
roaming call could have used 112 instead. With long-number there will be some data
traffic from the Danish MNO to the Spanish MNO. With 112 (or TS12 call) the call will
be handled exclusively and prioritized within Denmark. This may have inflicted
negatively on the values for KPI_002b, KPI_003, KPI_005, KPI_007a and KPI_008.
It happened at least one time with Fujitsu-TEN equipment that the Satellite position in
the MSD was from the last call (made 15-20 minutes earlier). This was only detected
due to the manual communications between test-operator and test-driver.
Some of the test-drivers put equipment on the back-seat of the car, and they had to
turn around when talking, for the operator to properly hear them.
Some calls did not have data package with the call, but it was retrieved when a
resend of MSD was requested
Notes from interview with test-operator:
Audio quality is a real issue. Static occurs for all equipment, making it difficult for the
operator to communicate with driver.
The visual implementation on screen is perfect.
I’m pretty impressed with the accuracy of the system, this will make a difference.
Resend of MSD only takes five seconds, this is really nice, as it increases my trust in
the position
I’m a little nervous with the call-button. It would be nice, if you had a plastic guard
over the button that drivers have to switch open, before pushing for manual 112-call.
5.3.3.7 Time series from phase two
The four figures below show the time series of the KPIs 5 and 7. Gaps in the curve indicate a
successful event, with lack of data in the IVS-log.
D4.3 Final results
17/12/2014 67 v 1.0
Figure 28: Phase two calls (DK)
Figure 29: Phase two calls with FJT M2M SIM card (DK)
D4.3 Final results
17/12/2014 68 v 1.0
Figure 30: Phase two calls with FJT normal SIM card (DK)
Figure 31: Phase two calls with FJT normal SIM card (DK)
D4.3 Final results
17/12/2014 69 v 1.0
Figure 32: Phase two calls with GMV normal SIM card (DK)
5.3.3.8 Statistical evaluation from phase two
KPI5
(all calls)
KPI7
(all calls)
KPI5
(FJT
M2M)
KPI7
(FJT
M2M)
KPI5
(FJT
normal)
KPI7
(FJT
normal)
KPI5
(GMV)
KPI7
(GMV)
Minimum 9.9 6.9 9.9 6.9 13.3 7.0 15.9 9.9
Maximum 31.3 24.3 19.0 14.1 20.8 11.8 31.3 24.3
Mean 15.2 10.1 13.1 9.6 16.6 8.5 18.7 11.6
Median 14.4 10.0 13.4 10.0 15.8 8.0 18.0 10.0
Std dev 3.8 2.5 2.2 1.6 3.8 1.9 3.4 3.2
Skewness 1.4 3.0 0.7 0.1 1.0 1.1 2.9 3.3
Figure 33: statistical data phase 2 (DK)
The GMV shows poorer results than FJT. This may be due to the fact, that the GMV-units
were equipped with Roaming SIM-Cards and that they had to use a long-number for calling
instead of 112.
D4.3 Final results
17/12/2014 70 v 1.0
5.3.3.9 Discussion of results
Figure 34: geographical distribution of failed calls for phase 1 and 2 (DK)
Figure 34 shows where the IVS-units had registered to be, while attempting a call that either
failed, or where the MSD was not received at the PSAP.
The data presented above are based on non-complete datasets, as not all log-files from IVS
units were readable or accessible. This gives raise to interesting results (see footnote 2,
page 64).
The equipment used for the pilot-test were prototypes (among the first), and some of them,
arrived very late to the pilot site, because the vendor had to give the IVS-unit the capability of
letting the GSM-radio be dormant (not registering on the network, until a call is made).
The problems with obtaining, configuring and using special profiled SIM-Cards , underlines a
recommendation to put all “Dormant”-functionality on the IVS, and not in the SIM-card profile.
5.3.4 Recommendations
IVS must be in compliance with the eCall standards regarding interpretation and
understanding of what are in the MSD. We noticed, that the GMV-units sends the
D4.3 Final results
17/12/2014 71 v 1.0
“Heading”-direction the vehicle has post-crash, instead of sending the heading the
vehicle was driving pre-crash.
Focus on audio quality when installing a retrofitted IVS in vehicles. Maybe this must
be further standardised.
5.3.5 Conclusions
The system works. Even with prototypes (and all the problems that follow) our operators in
the pilot-test could recognize the value of eCall.
But we need to have better retrofit able IVS-units in the market.
5.4 Luxembourg
5.4.1 General
This chapter outlines the Luxembourg results of the evaluation phase of HeERO 2. In
addition to phase 1, KPIs 9 and 13 were evaluated.
5.4.2 Recommended KPIs
The recommended KPI 2a couldn’t be tested during the project as the eCall Flag was not
updated into POST Luxembourg’s MSCs until the end of August. Unfortunately the first tests
after that showed that the Bit setting (6 and 7) for the routing was not added in the current
SW version. A SW version implementing a workaround could only be used in a closed test
environment beginning of October. This test environment is in fact a shielded metal box, with
a size of 80cm x 80cm x 30cm that contains the antenna of a base station. This box is
needed to avoid any interference of new feature or functionalities with the commercial
network. Therefore only a couple of tests were possible and putting an IVS into this box and
pressing the eCall button. The tests showed that the updated version works well. The final
SW version including the complete eCall routing support is planned for implementation into
POST Luxembourg’s networks in Q2 2015.
5.4.3 Evaluation results
5.4.3.1 Evaluation process
The basis of the evaluation was made using the datasets that were derived from the field
tests that were carried out between March 2014 and October 2014.
Unfortunately the KPIs regarding the eCall flag could not be tested. Although the eCall Flag
has been updated in POST Luxembourg’s switches the bits setting for the routing was not
D4.3 Final results
17/12/2014 72 v 1.0
defined and this means that 112 eCalls cannot be routed as required. (An update that will
resolve this this will be available to POST in Q2 2015.)
In the following section, special cases in the evaluation are described.
5.4.3.1.1 KPI5
To compute the time until the MSD is presented in the PSAP (KPI_5) two calculation
methods were used:
1. IVS protocol available? The time from when the activation of the eCall was initiated in
the IVS (button pressed) until the ACK-MSD was received by the IVS was used.
2. The typical time for call setup of 3 seconds was added to the time when the call
reached the PSAP and the time when the MSD is presented to the operator. This
typical value was derived from the median of the delay measured by the IVS.
5.4.3.1.2 Manual and automatic tests
Only manual tests have been executed. For the manual tests all planned KPIs are evaluated.
5.4.3.1.3 KPI 13
The evaluation of the heading was made, in the manual tests, by comparing the board
instrument of the car and the heading presented to the operator when the car was stationary.
The aim of the evaluation of this KPI was to determine if it makes sense to transmit the
heading as a separate value besides the last three positions.
5.4.3.1.4 KPI 9
In the definition of KPIs there are two possibilities for specifying the result of this KPI (value
of the accuracy in meters or the success rate of the accuracy; success is defined as the
accuracy is within 100m). Here the success rate of the accuracy was measured by a
comparison of the real location and the presented location. The real position was reported to
the operator by the driver during the voice communication. The operator compared the
shown location with the real position and entered the result into the report.
The Fujitsu IVS showed a systematic error in the location. In the first eCall after powering up
the location was correct. For the second eCall the location of the first eCall was transmitted
and only when requesting a retransmission of the MSD the correct location was transmitted.
The manufacturer is working on this topic. As it is unrealistic that two eCall are issued directly
after each other as it is executed in the tests, we assumed the location as correct, when the
location was correct after the retransmission of the MSD.
D4.3 Final results
17/12/2014 73 v 1.0
5.4.3.1.5 KPI_15 Success rate of VIN decoding with EUCARIS
KPI 15 could not be tested as EUCARIS access was not available in Luxembourg as of
August 2014.
5.4.3.1.6 KPI_16 Time for VIN decoding with EUCARIS
KPI 16 could not be tested as EUCARIS access was not available in Luxembourg as of
August 2014.
5.4.3.1.7 KPI_28a Number of cross border tests
As the 112 eCall flag was not available at the border of Luxembourg to Belgium and
Germany cross border tests could not be executed.
5.4.3.1.8 KPI_28b Number of interoperability tests
Some interoperability tests have been executed. The results are shown in chapter 5.4.4.
5.4.3.1.9 KPI_30 Number of calls flagged as dangerous good
The handling of dangerous goods transports in European eCall has been defined in this
project. It was decided that instead of a flag only additional information should be provided in
the additional information field of the MSD. This standardisation was more complex than
expected and needed more time. The ESA project that should provide the implementation of
the dangerous goods tracking service was delayed until end of 2014Therefore test
implementations of IVS were not available during the project.
5.4.3.1.10 KPI_31 Number of successful access of dangerous goods information
See above
5.4.3.1.11 KPI_32 Number of Dormant IVS tests
No tests with dormant SIM cards have been executed.
D4.3 Final results
17/12/2014 74 v 1.0
5.4.3.2 Evaluation results
FUJITSU man FICOSA 1 man FICOSA 2 man Unit
KPI_001a - - -
KPI_001b 317 263 98 -
KPI_002b 62% 67% 79% %
KPI_003 62% 67% 79% %
KPI_004 100% 100% 100% %
KPI_005 11.6 s 14.4 s
KPI_006 98% % 97% %
KPI_007 8.9 s 12.1 s
KPI_009 97% 94% 100% %
KPI_013 97% 94% 100% %
Table 13: Summary of Results (LU)
5.4.3.2.1 Comments
In July FICOSA provided updated Firmware for the IVS. The new IVS (FICOSA) shows some
improvements in MSD transmission, especially in difficult environments. (Not visible in the
KPIs)
KPI 5 (MSD presentation time) is longer than KPI 7 (voice channel blocking time). This is as
expected. The FICOSA IVS has a significant higher MSD presentation time and Voice
Channel Blocking than Fujitsu. This is caused by the long synchronisation and codecs
selection time before transmitting the MSD
The accuracy of the GPS positioning is as good as expected and is beside of few outliers
between 3 and 5 meters.
The information of the heading shows the actual direction of the stationary vehicle and not
the recent direction of travel, when the vehicle was driving. So the implementation of the
heading information is not in conformance to the requirements of the standard. It does not
provide the expected additional benefit for the rescue team to know on which side of the
highway the incident happened.
D4.3 Final results
17/12/2014 75 v 1.0
5.4.3.3 Time series
Figure 35: KPI5 FUJITSU IVS (LU)
Figure 36: KPI5 FUJITSU IVS (LU)
D4.3 Final results
17/12/2014 76 v 1.0
Figure 37: KPI5 FICOSA 2 (LU)
Due to a problem with the eCall router logging only a few values could be calculated.
In Figure 36 the chronological sequence of KPI 5 during the manual test is drawn.
5.4.3.4 Statistical evaluation
The statistical evaluation was made according to the agreed procedures in Chapter 4.1.
KPI 5 KPI7
FUJITSU FICOSA1 FICOSA2 FUJITSU FICOSA1 FICOSA2
Minimum 5.0 12.0 12.0 3.0 10.0 10.0
Maximum 25.0 20.0 16.0 24.0 19.0 14.0
Mean 11.6 14.4 14.2 8.9 12.2 11.6
Median 9.0 14.0 14.0 6.0 12.0 12.0
Std dev 5.9 1.8 1.1 6.0 2.0 1.1
Skewness 1.4 1.7 -0.4 1.8 2.4 0.5
Table 14: Statistics of manual initiated calls (LU)
D4.3 Final results
17/12/2014 77 v 1.0
Figure 38: MSD presentation time (LU)
Figure 39: MSD presentation time (LU)
D4.3 Final results
17/12/2014 78 v 1.0
Figure 40: Voice channel blocking time (LU)
In Figure 39 the distribution of KPI 7 shows that most data is between 4 and 8 seconds. The
values of the FICOSA IVS are little smaller after the software update but there is a wider
scattering. In principle both IVS show a similar distribution.
5.4.4 Interoperability tests
In this chapter the results of the interoperability tests of the Luxembourg IVS are
documented. The tests done with the Luxembourg PSAP are documented in the chapters of
the home country of the testing IVS.
PSAP IVS 1 IVS2
BG 5 0
DE 10 10
BE 10 7
RO 5 0
Table 15: KPI 1b with own IVS and foreign PSAPs (LU)
PSAP KPI 2b KPI 3 KPI4 KPI6
IVS 1 IVS2 IVS1 IVS2 IVS1 IVS2 IVS1 IVS2
D4.3 Final results
17/12/2014 79 v 1.0
BG 100 0 80 0 80 0 80 0
DE 100 100 80 70 80 70 80 70
BE 100 100 80 100 80 100 80 100
RO 100 0 60 0 60 0 60 0
Table 16: KPI 2b, 3, 4 and 6 with foreign PSAPs (LU)
5.4.5 Recommendations
The following recommendations are given:
Voice channel blocking time and MSD transmission times are too high for both IVS.
The new version of the FICOSA SW reduces the time, but it is still too long. The
problem seems to be in the synchronisation of the IVS and the PSAP modem. We
recommend reviewing the synchronisation process to reduce this time.
The heading information does not provide the direction of travel as intended in all
implementations. The FUJITSU and the FICOSA units provided the heading
information from the current location of the car when the eCall was triggered. If the
car had turned around during the accident this is no longer the direction of travel. The
last three positions in the MSD should be used to determine the direction of travel
only. The PSAP SW needs to show the last three positions on the map in addition to
the last location
When an IVS is triggered to retransmit the MSD, the IVS provides updated content of
the MSD at the moment of request for retransmission. As the MSD containing the
updated position provides valuable information to PSAPs in case of a retransmission
of the MSD (especially for identifying false alarms), the content should always be
updated.
Luxembourg project partner POST Luxembourg particularly recommends that MNOs
planning on testing the eCall switch firstly check that the correct software is installed
that will allow the subsequent routing of the calls through Bits 6 and 7. After
“swapping stories” with several of the other MNO partners in theHeERO2 project this
has been shown to be a relatively common problem and the responses of the MSC
software suppliers can vary greatly as to availability and pricing.
D4.3 Final results
17/12/2014 80 v 1.0
5.4.6 Conclusion
The success rates of completed calls and received MSD could be improved in comparison
with the results of Phase 1. As such the correctness of the modifications in the standard has
been validated.
The IVS are still prototypes and so there are minor problems with stability etc. which should
be solved when the final overall introduction of eCall gets closer. The biggest problem is the
success rate of the MSD transmission and the long voice channel blocking time that needs to
be improved.
The testing proved valuable where it revealed problems that were not necessarily anticipated
to be problems; specifically the Bit routing software’s importance to the success of the call
transmission for many partners.
5.5 Spain
This chapter outlines the Spanish results of the eCalls performed in Phase 1 and Phase 2.
Sub-phase A in Phase 1 was executed in October 2013 and Sub-phase B in Phase 1
between January and March 2014. This phase was used to properly configure all the system
in order to assure the correct data logging at each stage of the e-Call chain. Phase 2 was
undertaken in October 2014 with all the logging issues solved as well as some operational
improvements learned from Phase 1,
5.5.1 General
Unlike other test sites, Spanish test site has an intermediate PSAP belonging to the Spanish
National Traffic Authority (DGT) and located in Madrid capital, in charge of managing all the
eCalls coming from Spanish territory. Once the data from the IVS gets this PSAP at DGT and
the voice channel is established, it decides what regional 112 can support the vehicle
according to its position. At that same time, the PSAP is able to communicate directly with
the IVS and retrieve the significant data of the IVS (the decoded MSD) in its PCs. Tests in
Madrid were done with Madrid and Castilla y León PSAPs and tests in Galicia with Galicia
and Castilla y León PSAPs, which allowed testing cross-regional aspects too.
Figure 41: Call chain from the IVS to final PSAP
D4.3 Final results
17/12/2014 81 v 1.0
Madrid
All KPIs have been calculated according to the information and data registered at the IVS
and intermediate PSAP at DGT. Unfortunately, some KPIs related to communication
between intermediate PSAP at DGT and regional 112 PSAPs have not been calculated due
to the absence of the necessary timestamps at the regional 112 PSAPs.
In addition, in any the tests done, the 112 number has not been used. Always a long number
was configured at the intermediate PSAP at DGT. For this reason, KPI_002a could not be
calculated.
Another KPI that could not be resolved is KPI_011. In this case, due to absence of
information from the device, this KPI is not possible to be determined.
A remark related to KPI_013 follows: No information related with the accuracy of the heading
has been recovered. Heading information provided by the IVS is considered not significant,
as no reference system is used on-board to conclude on a % of success.
In case of Call-back-related KPIs (KPI_021, KPI_022), the call-backs are considered from
intermediate PSAP at DGT to the IVS.
The protocol followed by the intermediate PSAP at DGT and by the regional 112 PSAPs was
agreed between the different participant PSAPs in the Spanish HeERO2 pilot and is
comparable to the protocol presented for the Galician Test Site.
The scenarios where the tests were conducted were similar to the ones used in the Galician
Test Site, including:
Standard Environmental Conditions (STD) with normal cellular network coverage and good
GNSS signal conditions
Weak GSM signal scenario (WEAK) where the environments chosen had low GSM coverage
High traffic load of GSM data scenario (HTRA): this environment was reproduced in areas with
high traffic load of GSM data, in urban areas.
Driving in regional cross border (REGC): it was tested with neighbouring region of Castilla y
León
High and low speed of vehicle (HIGH, LOW): the car was driving at high speed and at low
speed or stopped
Galicia
In order to get comparable results, the intermediate PSAP uses an answering protocol to set
up the basics of the communication among IVS, intermediate PSAP and final PSAP. The
Galician IVS can initiate the eCall in two modes: manual and automatic:
There is a button to push in case of manual eCall and other button to push in automatic mode
that simulates the airbag activation.
Once the eCall is initiated, the intermediate PSAP receives the data, creates the log entry and
answers the incoming call to establish voice communication with the passenger.
D4.3 Final results
17/12/2014 82 v 1.0
At this moment, the intermediate PSAP confirms some data with the IVS, such as VIN
decoding, type of call or position in GIS.
The intermediate PSAP can request a MSD retransmission if there are discrepancies between
IVS and PSAP information.
If there is a break in the eCall the intermediate PSAP can restore the contact with a call-back.
Based on the location of the IVS, the intermediate PSAP decides what PSAP is going to
attend the eCall (Galicia or Castilla y León) and creates the data for transmission
The intermediate PSAP contacts with the final PSAP and forwards the call from IVS
Final PSAP and IVS establish voice transmission
Figure 42: Galician scenarios (ES)
Five different scenarios were employed to conduct the tests, taking into account the climate
conditions of Galicia, which included rainy and cloudy days when the tests were carried out:
- Standard environmental conditions (STD): IVS drove around Porriño town, N550 road and A55 motorway. These are busy areas in south western Galicia with normal network coverage and GPS signal conditions.
- Weak GSM signal scenario (Weak): these tests were performed in environments far for population nucleus in south western Galicia
- High traffic load of GSM data scenario (HTRA): to test this scenario IVS went to Vigo city. It is a great urban area, with more than 300.000 inhabitants, where there is a high traffic load of GSM data.
- Driving in international cross border (INTC): because of the situation of the Galician region, it was possible to test the GSM coverage with the neighbouring country of Portugal.
- Driving in regional cross border (REGC): it was tested with the neighbouring region of Castilla y León.
D4.3 Final results
17/12/2014 83 v 1.0
Figure 43: IVS CTAG in the regional cross border
P2W
Also, Powered 2-wheelers (P2W) have been tested. The P2W test has been completed in 2
phases. First an emulator of PSAP has been used for validation test, and in the second
round of tests the IVS communicated with the real intermediate PSAP in DGT.
Test architecture
The test setup for the IVS of the P2W is described below:
The IVS has 3 main parts, the helmet itself, the accident detection unit and the call
unit.
o The intelligent helmet measures the impacts in the head and provides an
accident severity index, which may be useful for PSAP operators.
o The on-board accident detection unit sends the order for making a call to the
call unit. For the detection it uses several inertial sensors and an algorithm for
processing the information given by them. A manual activation button is
included in this module for a manual emergency call. It receives the severity
index from the helmet by Bluetooth.
o The call unit is the responsible for making the call and sending the MSD to the
PSAP. An audio channel is established with the helmet via Bluetooth.
D4.3 Final results
17/12/2014 84 v 1.0
The PSAP emulator is composed by an electronic board and a PC with a PSAP
software. The Software provides the main functionality for a PSAP operator: MSD
decoding, call tracking, etc.
The call between IVS and the PSAP emulator is a real call under a real Network, which
increases the cost of each test, but results more realistic and closer to the final behaviour.
The schema of the test architecture is shown in the next figure.
Figure 44: P2W Test architecture
Figure 45: P2W with IVS installed
The test results can be shown in the next table, where calls has been divided in successful
calls (OK), Failed Calls and other where the voice connection has been established, but with
some errors in the MSD, usually related to positioning.
Type of call Total Calls OK Failed Calls Connected with MSD error
Accident Detection Unit
call +MSD GPS
PSAP emulator
Accident Severity
Index
Voice
D4.3 Final results
17/12/2014 85 v 1.0
Manual 40 39 (98%) 0 (0%) 1 (3%)
Automatic 40 38 (95%) 0 (0%) 2 (5%)
Total 80 77 (96%) 0 (0%) 3 (4%)
5.5.2 Recommended KPIs
Phase 1
Data obtained may not be representative enough to be taken into account for a complete
evaluation. Table 17 includes the KPIs that could be measured during Phase 1
KPI Description
1a Number of manually initiated eCalls
1b Number of manually initiated eCalls
2b Success rate of completed eCalls using long number
3 Success rate of received MSDs
4 Success rate of correct MSDs
5 Duration until MSD is presented in PSAP
6 Success rate of established voice transmissions
8 Time for call establishment
9 Accuracy of position
10 Number of usable satellites
11 Geometric Dilution of Precision (GDOP)
12 Time between successful positioning fixes
14 Success rate of VIN decoding without EUCARIS
20 Success rate of presented incident data in TMC
21 Number of successful call-backs
22 Success rate of call-backs
28c Number of cross regional tests
D4.3 Final results
17/12/2014 86 v 1.0
Table 17: KPIs measured during Phase 1 (ES)
Phase 2
Galicia and Madrid, as Spanish test sites, planned to calculate the KPIs included in ‘D4.1
Final agreed KPIs, test specification and methodology’. The great number of eCalls
performed allowed obtaining all the committed KPIs and managing them in a proper
statistical way. KPI_19 (Dispatch time of incident data to TMC) and KPI_20 (Success rate of
presented incident data in TMC) were not obtained because of the particularities of Spanish
eCalls where TMC is actually the intermediate PSAP. In this sense, for the Spanish Test
Site, KPI_19 would be the same as KPI_07a, and KPI_20 would be the same as KPI_02a.
In the case of the Spanish pilot, KPI 2a has not been tested due to the fact that the eCall
Flag has not been deployed yet in Spain. Our device is able to do this type of eCall but only
in case of a short number configured as PSAP (for example number 112).
5.5.3 Evaluation results
Phase 1
Table 18 summarizes the KPIs calculated in Spain during Phase 1.
Galicia Madrid Unit
KPI_001a 37 115 -
KPI_001b 137 117 -
KPI_002b 72.41 62.5 %
KPI_003 83.70 64.34 %
KPI_004 83.70 62.5 %
KPI_005 -- 15 s
KPI_006 82.09 62.5 %
KPI_008 -- 17 s
KPI_009 -- 100 %
KPI_010 8.74 9 Satellites
KPI_011 1.44 -- DOP
KPI_012 1 0.1 s
KPI_014 96.77 -- %
D4.3 Final results
17/12/2014 87 v 1.0
KPI_020 83.70 62.5 %
KPI_021 93 32 -
KPI_022 98.94 84 %
KPI_028c 38 45 -
Table 18: Summary of calculated KPIs during Phase 1 (ES)
Phase 2
Table shows the KPIS calculated in Phase 2.
Galicia Madrid P2W Unit
KPI_001a 163 258 81 -
KPI_001b 194 216 79 -
KPI_002b 88 75.3 100 %
KPI_003 89 76.8 96.2 %
KPI_004 90 100 100 %
KPI_005 20.9 18.2 11.7 s
KPI_006 99 75.3 -- %
KPI_007a 10.7 10.6 -- S
KPI_007b 1.9 9.7 -- s
KPI_008 15.9 12.8 3,6 s
KPI_009 11.9 100 -- %
KPI_010 7.8 9 -- Satellites
KPI_011 1.4 -- -- DOP
KPI_012 165.,3 1 -- s
KPI_013 39 -- -- %
KPI_014 88 100 100 %
KPI_020 -- -- -- %
KPI_021 -- 54 0 -
KPI_022 -- 72 0 %
D4.3 Final results
17/12/2014 88 v 1.0
KPI_023 60 -- -- s
KPI_028a 42 -- -- -
KPI_028b 42 69 -- -
KPI_028c 68 45 -- -
KPI_029 44.7 -- -- s
Table 19: Summary of calculated KPIs during Phase 2 (ES)
5.5.3.1 Evaluation process
The evaluation process is based on the data collected during tests performed during the two
phases in which Spanish Pilot Site was divided.
In Galicia and Madrid, due to some problems with PSAP in Phase 1, the first part of the tests
done (Sub-phase A) shows a bad behaviour of the duration to connect with the PSAP or the
TMC. But in the second round of tests (Sub-phase B), the tests were OK as most of
problems were solved. All info collected takes into account both situations and the data must
be understood with this observation.
Galicia
Table 20 summarizes the tests performed in Phase 2 in October 2014. A total of 357 eCalls
and 68 call-backs were carried out from Galician IVS. Separating the tests by scenarios 123
were performed in STD scenario, 62 with weak GSM signal, 86 with a high load of GSM
network, 88 in the regional cross-border and 64 in the international cross-border. 81% of the
MSD sent by IVS were successfully received by the intermediate PSAP and 84% of the data
sent by this PSAP reached the final PSAP. The success in the voice communication is
almost 100%.
Da
te
Sc
en
ari
o
IVS PSAP INT 112
eC
all
sen
t
Ma
n.
Au
to
Ca
ll-b
ack s
en
t
Su
cc
es
s c
all-b
ack
MS
D s
en
t
MS
D s
uc
ce
ss
tra
ns.
112
ca
ll s
en
t fr
om
PS
AP
112
ca
ll s
uc
ce
ss f
rom
PS
AP
MS
D i
nfo
rma
tio
n s
en
t
fro
m P
SA
P t
o 1
12
su
ccess r
eceiv
ed
info
rma
tio
n a
t P
SA
P
07-oct STD 40 22 18 10 1 40 31 31 29 31 29
D4.3 Final results
17/12/2014 89 v 1.0
INTC 24 15 9 5 1 24 22 16 13 16 13
08-oct
STD 27 15 12 13 4 26 23 11 14 12 14
Weak 28 28 0 0 0 28 24 21 19 21 19
INTC 25 14 11 10 4 25 21 14 11 15 11
09-oct Weak 29 14 15 5 6 29 17 16 10 16 10
HTRA 50 25 25 1 0 50 50 40 40 40 40
16-oct STD 29 12 17 4 3 27 23 17 17 17 17
HTRA 35 20 15 0 0 35 35 35 35 35 35
17-oct REGC 68 28 40 20 12 59 43 61 34 60 34
Total 355 193 162 68 31 343 289 262 222 264 222
Table 20: Summary of Galician tests
Table 21 shows the Galician KPIs in the format required by ‘D4.1 Final agreed KPIs, test
specification and methodology’. It gives a general overview of the development of the
Galician tests. These KPIs were calculated for each scenario and also taking into account
the overall eCalls.
KPI_01a and KPI_01b show that was intended to conduct a comparable number of tests in
all the scenarios. KPI_02b, KPI_03, KPI_04, KPI_06, KPI_13 and KPI_14 were calculated in
the piece of chain from IVS to PSAP as it is here where the MSD transmission is produced.
KPI Name of KPI
Galician scenarios
Total Unit
STD Weak HTRA INTC REGC
01a Number of automatically initiated eCalls
46 15 40 22 40 163 -
01b Number of manually initiated eCalls
49 42 45 30 28 194 -
02b Success rate of completed eCalls using long number
94 63 100 87 85 88 %
03 Success rate of received MSDs 94 86 100 88 88 89 %
04 Success rate of correct MSDs 87 97 100 93 74 90 %
05 Duration until MSD is presented in PSAP
20.1 24.2 18.6 19.8 22.7 20.9 s
06 Success rate of established voice transmissions
99 100 99 100 97 99 %
07a Duration of voice channel blocking
10.1 13.7 8.8 10.1 12.0 10.7 s
07b Duration of voice channel 1.2 4.0 2.8 1.0 1.2 1.9 s
D4.3 Final results
17/12/2014 90 v 1.0
blocking: automatic retransmission of MSD
08 Time for call establishment 15.1 18.2 14.0 15.5 17.4 15.9 s
09 Accuracy of position 12.7 11.2 38.5 7.2 6.5 11.9 m
10 Number of usable satellites 7.8 6.4 8.6 8.4 7.7 7.8 -
11 Geometric dilution of precision 1.3 1.8 1.6 1.4 1.3 1.4 -
12 Time between successful positioning fixes
168.6 174.9 125.6 143.3 176.3 165.3 s
13 Success rate of heading information
32 57 62 35 39 39 %
14 Success rate of VIN decoding without EUCARIS
94 62 100 88 89 88 %
23 GSM network latency 61.9 67.8 49.1 71.5 50.9 60 s
28 a Number of cross-border tests 42 -
28 b Number of interoperability tests
42 -
28c Number of cross regional tests 68 -
29 Dispatch time of Intermediate PSAP
46.6 49.5 35.0 56.7 36.6 44.7 s
Table 21: Summary of Galician KPIs
Madrid
Basis of the evaluation were the datasets derived in the field tests from February to October
2014. In the following paragraphs, special cases in the evaluation are described.
All the information shown in this chapter has been calculated based on manual and
automatic eCalls.
5.5.3.1.1 Information collected
Due to some problems in PSAP, the first part of the tests done (phase 1) showed a bad
behaviour at the time to connect with the regional 112 PSAP or the TMC. However, problems
were fine-tuned and in phase 2 these problems had already been fixed and the tests were
successful most of the time. All the information collected takes in account both situations and
the data must be understood with this observation.
5.5.3.1.2 KPI5
The calculation of this KPI (Duration until MSD is presented in PSAP)) has been carried out
considering the time between the activation of the eCall in the IVS (Button pressed or
automatic eCall execution) until the MSD is presented at the intermediate PSAP at DGT.
D4.3 Final results
17/12/2014 91 v 1.0
5.5.3.1.3 KPI 9
In the definition of KPIs there are two possibilities to specify the result of this KPI (value of
the accuracy in meters or the success rate of the accuracy; success is defined as the
accuracy is within 100m). Here, the success rate of the accuracy was measured by
comparison of the real location and the presented location. The operator compared the
shown location with the real position and entered the result into the report.
Although, the position obtained in the intermediate PSAP at DGT from MSDs was
commented between the driver and the person on PSAP, and in all cases location was
completely OK.
5.5.3.1.4 Comments
All KPIs have been calculated according to information registered in IVS and data registered
at the intermediate PSAP at DGT. For some KPIs, information related to communication
between DGT PSAP and regional 112 PSAP is needed. Unfortunately, this information was
not available.
Due to this, some of the KPIs proposed have not been calculated. In any case, they are not
within the group of recommended KPIs. These KPIs are:
KPI 019, KPI 020 and KPI 023: Information from regional 112 PSAP is needed to
make these calculations, however, it is not available
KPI 011.
KPI 013
P2W
In this phase the emulated PSAP was substituted by the real intermediate PSAP in DGT. 81
automatic calls were initiated and 79 manual calls. The operator in the PSAP accessed via
VPN, so voice associated KPIs couldn’t be measured. This functionality was already tested
in the tests of the Phase 2.1.
5.5.3.2 Time series
Galicia
A time series analysis is carried out to better understand KPI_05, KPI_07a, KPI_07b,
KPI_08, KPI_23 and KPI_29. In this framework, it shows the evolution of the studied KPIs
with the temporal passing of eCalls. Figure 57 shows the plot series and histograms for these
KPIs.
D4.3 Final results
17/12/2014 92 v 1.0
Figure 46: Time series diagram of IVS CTAG - KPI_05 (ES)
Figure 47: Histogram of IVS CTAG - KPI_05 (ES)
0
5
10
15
20
25
30
35
40
1 41 81 121 161 201 241 281 321
eCalls
Dura
tion u
ntil M
SD
is p
resente
d in
iPS
AP
(s)
0
10
20
30
40
50
60
70
80
90
<10 10-12 12-14 14-16 16-18 18-20 20-22 22-24 24-26 26-28 28-30 30-32 32-34
Duration until MSD is presented in iPSAP (s)
frequency
D4.3 Final results
17/12/2014 93 v 1.0
Figure 48: Time series diagram of IVS CTAG - KPI_07a (ES)
Figure 49: Histogram of IVS CTAG - KPI_07a (ES)
0
5
10
15
20
25
30
1 41 81 121 161 201 241 281
eCalls
Dura
tion o
f voic
e c
hannel blo
ckin
g (
s)
0
20
40
60
80
100
120
140
160
180
0-2 2-4 4-6 6-8 8-10 10-12 12-14 14-16 16-18 18-20 20-22 22-24 24-26
Duration of voice channel blocking (s)
frequency
D4.3 Final results
17/12/2014 94 v 1.0
Figure 50: Time series diagram of IVS CTAG - KPI_07b (ES)
Figure 51: Histogram of IVS CTAG - KPI_07b (ES)
0
1
2
3
4
5
6
7
1 5 9 13 17 21 25 29 33 37 41
eCalls
Dura
tion o
f voic
e c
hannel blo
ckin
g:
auto
matic r
etr
ansm
issio
n o
f M
SD
(s)
0
5
10
15
20
25
30
35
0-2 2-4 4-6
Duration of voice channel blocking: automatic retransmission of MSD (s)
frequency
D4.3 Final results
17/12/2014 95 v 1.0
Figure 52: Time series diagram of IVS CTAG - KPI_08 (ES)
Figure 53: Histogram of IVS CTAG - KPI_08 (ES)
0
5
10
15
20
25
30
35
1 41 81 121 161 201 241 281 321
eCalls
Tim
e f
or
call
esta
blis
hm
ent
(s)
0
20
40
60
80
100
120
0-2 2-4 4-6 6-8 8-10 10-12 12-14 14-16 16-18 18-20 20-22 22-24 24-26 26-28 28-30
Time for call establishment (s)
frequency
D4.3 Final results
17/12/2014 96 v 1.0
Figure 54: Time series diagram of IVS CTAG - KPI_23 (ES)
Figure 55: Histogram of IVS CTAG - KPI_23 (ES)
0
50
100
150
200
250
1 41 81 121 161 201
eCalls
GS
M n
etw
ork
late
ncy (
s)
0
20
40
60
80
100
120
0-20 20-40 40-60 60-80 80-100 100-120 120-140 140-160 160-180 180-200
GSM network latency (s)
frequency
D4.3 Final results
17/12/2014 97 v 1.0
Figure 56: Times series diagram of IVS CTAG - KPI_29 (ES)
Figure 57: Histogram of IVS CTAG - KPI_29 (ES)
Madrid
0
20
40
60
80
100
120
140
160
180
200
1 41 81 121 161 201
eCalls
Dis
patc
h t
ime o
f iP
SA
P (
s)
0
10
20
30
40
50
60
70
0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180
Dispatch time of iPSAP (s)
frequency
D4.3 Final results
17/12/2014 98 v 1.0
Figure 58: KPI5 for GMV IVS (ES)
In the previous figure, the MSD presentation in time calculated as KPI_005 with all the data
obtained along all the test process.
Figure 59: KPI7a for GMV IVS (ES)
D4.3 Final results
17/12/2014 99 v 1.0
Figure 60: KPI7b for GMV IVS (ES)
Figure 61: KPI8 for GMV IVS (ES)
D4.3 Final results
17/12/2014 100 v 1.0
Figure 62: MSD presentation time (ES)
In KPI_005 distribution figure, most data are between 18 – 20 seconds. Less but notable
amount is between 16 and 18 seconds.
Figure 63: Voice channel blocking time (ES)
D4.3 Final results
17/12/2014 101 v 1.0
Figure 64: Voice channel blocking time (ES)
In the two previous figures about the KPI_007 distribution, it is possible to see that most data
are between 8 – 12 seconds (if we take in account the values obtained for KPI_007a and
KPI_007b).
Figure 65: MSD presentation time (ES)
D4.3 Final results
17/12/2014 102 v 1.0
5.5.3.3 Statistical evaluation
The results of statistical analyses by KPIs are systematically presented in this chapter. It
was done according to the agreed procedures in chapter ‘5.2 Validation Procedure’.
Galicia
Minimum Maximum Mean Median Std dev Skewness
KPI_05 10 34 20.9 21.0 4.4 -0.54
KPI_07a 3 24 10.7 10.0 3.1 1.56
KPI_07b 1 6 1.9 1.0 1.5 1.51
KPI_08 5 29 15.9 16.0 3.2 0.82
KPI_09 0.3 68.7 11.9 5.5 24.0 4.76
KPI_10 3 12 7.82 8.0 1.6 -0.02
KPI_11 0.9 4.6 1.4 1.3 0.50 2.66
KPI_12 49.0 592.0 165.3 122.0 121.3 1.95
KPI_23 26 196 60.0 52.0 28.3 1.91
KPI_29 12 178 44.7 36.0 29.6 2.08
Table 22: Statics of initiated eCalls (ES)
The low values of standard deviation and Skewness in KPI_05 indicate that there is no
dispersion around mean value (20.89s) and talk about the good precision of the measure.
KPI_09 is a measure of the accuracy of position. It was calculated with the known Haversine
formula, being one of the points the position sent by IVS to PSAP and the other one the
actual position of the vehicle. The mean value is 11.91m. (less than 100m between these two
points is considered as ‘acceptable’) but it can be seen that the standard deviation is high, as
the Skewness value.
KPI_10 shows that 7.82 satellites were used on average to estimate the position. As this
number of available satellites is high enough, the GDOP (KPI_11) ranges between 1 and 2
that denotes an excellent rating
KPI_12 is strongly affected by the duration of voice time in each eCall. The time between
eCalls is not uniform so, this KPI does not provide a reliable result. In fact, value for standard
deviation is very similar to mean or median values. Skewness value is also far from 0.
D4.3 Final results
17/12/2014 103 v 1.0
KPI_23 is also affected by the variable duration of voice phase.
Madrid
KPI_005 KPI_007a KPI7_007b
GMV GMV GMV
Minimum 7 2 4
Maximum 49 54 16
Mean 18.2 10.6 9.7
Median 23.5 15.5 9
Std dev 4.6 4.7 2.9
Skewness 2.5 4.4 0.6
Table 23: Statistics of initiated calls (ES)
5.5.3.4 Discussion of results
KPIs denote a satisfactory success of the tests carried out in Galicia and Madrid Pilot Sites.
All the proposed scenarios passed the evaluation process and there are no great differences
among KPIs calculated in different scenarios.
5.5.4 Interoperability tests
Galicia
Interoperability tests were performed in the framework of the second and third TESTFEST
events organized by ERTICO and ETSI. Those events were hosted in CETECOM in
September 2013 and CTAG test lab from 27th to 31th October 2014. TESTFESTs were
focused on checking eCall implementations confidently against other devices and eCall
standards requirements. This eCall TESTFEST event enables IVS vendors and PSAP
vendors to run interoperability test sessions using test descriptions provided in approved
guidelines
D4.3 Final results
17/12/2014 104 v 1.0
Figure 66: TESTFEST#3
So, among other tests, it was performed by CTAG IVS a set of eCalls with other European
PSAPs such as: AREU (Italy), HiTec (Luxembourg), OECON (Germany), VTT (Finland),
Belgium or Bulgaria. All of them carried out the following mandatory tests:
TD_MAN_01: Using the pull mode
TD_MAN_02: Voice communication after receipt of AL-ACK
TD_MAN_03: Retransmission of MSD on request from PSAP
TD_MAN_04: Voice communication after retransmission of MSD
TD_MAN_05: Clear down / PSAP initiated network clear down
TD_MAN_06: Clear down / PSAP initiated application layer AL-ACK
TD_MAN_07: Call Back / PSAP initiated call back to IVS
D4.3 Final results
17/12/2014 105 v 1.0
Figure 67: Summary of CTAG results in 2013
D4.3 Final results
17/12/2014 106 v 1.0
Table 24 and Table 25 summarize the satisfactory results of these tests since 74% of them
show the excellent compliance in terms of interoperability requirements.
PSAP TD_MAN
_01
TD_MAN
_02
TD_MAN
_03
TD_MAN
_04
TD_MAN
_05
TD_MAN
_06
TD_MAN
_07
Italy OK OK OK OK OK OK OK
Luxembourg OK OK OK OK OK NA OK
Germany OK OK OK OK OK OK OK
Finland OK OK OK OK OK NA OK
Belgium OK OK NA NA OK OK OK
Bulgaria NA NA NA NA NA NA NA
Table 24: Interoperability tests performed between CTAG and European PSAPs in 2014
Interoperability Not Executed Totals
OK NO NA OT Run Results
43 (97, 7%) 1 (2, 3%) 4 (8, 2%) 0 (0.0%)
44 (89,
8%) 49
Table 25 CTAG’s IVS Mandatory test results in 2014
Madrid
All the interoperability tests were done during the eCall Test Fest#2 in Essen in September
2013 and the eCall TestFest #3 in Vigo in October 2014.
The KPI only reflects the number of tests carried out with the PSAPs (either local or remote)
in 2014 and does not compute the many other tests which were carried out against
simulators and test systems.
Mandatory tests were able to be done with several PSAPs from other countries like: Bulgaria,
Italy, Luxembourg, Belgium, Finland and Germany.
GMV’s IVS interoperability results in 2013 are presented respectively in the following tables.
D4.3 Final results
17/12/2014 107 v 1.0
Figure 68: Summary of all participants’ results
Figure 69: GMV’s IVS test results summary
The comparison of GMV’s IVS results with the results of all of the participants at the eCall
Testfest shows the good interoperability and, also, good performance of GMV’s IVS.
In the eCall Testfest #3, in Vigo, October 2014, interoperability was tested between the IVS
and local and remote PSAPs from different countries and providers; concretely,
interoperability was tested with the following PSAPs: HiTec (Luxembourg), OECON
(Germany), Cestel (Spain), Telefónica (Spain), AREU (Italy), PicoSoft (Italy), VTT (Finland),
Belgium and Bulgaria. The tests used long numbers to call the PSAP. Manual and automatic
eCall tests were performed.
D4.3 Final results
17/12/2014 108 v 1.0
Tests were also carried out between IVS and test systems, including IVS testers (IOP) such
as ANRITSU, Picosoft and Rohde&Schwarz, Speech Testers (Head Acoustics) and IOP
PSAP simulators (NavCert). The tests were carried out using emergency numbers (112) in
some cases.
An improvement in the number of successful tests was observed from eCall Testfest 2013 to
eCall Testfest 2014 for GMV’s IVS.
5.5.5 Recommendations
The following recommendations are given:
GPS positioning is not accurate enough as it can be observed in values provided by
KPI_09 and KPI_13. This should be improved.
The answering protocol should establish a fixed period for the voice communication to
better assess KPI_12 and KPI_23
Voice channel blocking times should be minimized and also a sound could be
reproduced while the channel is blocked to avoid transmission noise.
In many situations the vehicle heading information does not show correctly the
direction of travel and in some cases only the direction of the vehicle when the eCall
was triggered is not enough. MSD could provide previous information about the two
or three last positions to determine the travel direction perfectly.
In case of an MSD request (once the voice channel is open and a previous MSD has
been received at the PSAP) instead of sending the same MSD Data, the best option
in this case would be updating all the MSD information with location, heading, etc.
This could be useful in case the vehicle has moved due to the impact and this is also
a way to identify false alarms.
5.5.6 Conclusions
Galicia
The number of eCalls performed in Phase 1 had not been enough to calculate some KPIs or
undertake time series or statistical analysis. Phase 1 was focused on solving various issues
and technical problems with the intermediate PSAP and final PSAPs. In Phase 2, serious
efforts have been carried out by the three actors in the eCall chain (IVS CTAG, PSAP and
Group OK NO OK NO
Mandatory 67 (98,5%) 1 (1,5%) 1232 (93,9 %) 80 (6,1%)
Optional_CFG_01 44 (100%) 0 (0,0%) 830 (93,5%) 58 (6,5%)
Optional_CFG_02 2 (100%) 0 (0,0%) 33 (89,2%) 4 (10,8%)
Total results of eCall eventCTAG IVS
D4.3 Final results
17/12/2014 109 v 1.0
PSAP) to fulfil the requirements by ‘D4.1 Final agreed KPIs, test specification and
methodology’. These efforts include:
- Improvements in the logging system
- An increased number of operators and day tests to perform a larger number of eCalls
- Development of tools to perform the statistical analysis
Special mention should be made to cross regional tests performed in Castilla y Leon. PSAP
was able to redirect properly the eCall from IVS to Galicia PSAP or Castilla y León PSAP
taking into account the GPS position of the IVS. The performance of the system in this
scenario was similar to other scenarios (Table 21).
All eCall tests in Galicia were carried out dialling to a long number (without eCall Flag) given
that the MNO has not implemented the changes necessary in the network to support it.
Nevertheless CTAG has tested in laboratory the compatibility of its IVS with eCall flag.
Additionally, it was planned to do some tests in December 2015 with Vodafone Spain to test
eCall flag implementation in a development network environment.
Madrid
The success rates of completed calls and received MSD in PSAP show an improvement with
respect to the results in the fine-tuning phase (phase 1). In phase 2 different modifications
were incorporated in order to adapt to the upgrading of standards.
The IVSs are still prototypes and for this reason, minor problems with stability, sound, etc.
have been observed. These will be solved when going to a commercial phase.
A great number of cross-regional tests were carried out between Madrid and Castilla y León
and in all the tests the intermediate PSAP at DGT was able to redirect eCalls to the correct
regional 112 PSAP based on the IVS’s reported position. The performance of the system in
this scenario was similar to other scenarios.
The eCall flag has not been implemented by the MNO; therefore, no calls to the emergency
number 112 have been tested. Laboratory tests have been performed with a test version of
the network implementing the eCall with Vodafone and results are satisfactory.
D4.3 Final results
17/12/2014 110 v 1.0
5.6 Turkey
5.6.1 General
The goal of testing and validation of eCall service in Turkey is to validate technological and
functional properties of the system. The testing environment covers the whole service chain
from the IVS unit to eCall PSAP.
The field tests are carried out with two different IVS devices. Civitronics IVS units are used
for system verification. Magneti Marelli IVS units are used for both system verification and
data collection. Civitronics IVS units are supplied by Renault and Magneti Marelli IVS units
are supplied by Tofaş. The IVS units used Turkcell’s 3G network for establishing eCall.
The tests involve only manually initiated eCalls which were activated by pressing a button on
the IVS equipment.
5.6.2 Recommended KPIs
The recommended KPIs are listed in Table 26. All the recommended KPI’s except KPI_28a
and KPI_28b are tested in the scope of our field tests. We are currently working on
interoperability tests. We will add the results of the interoperability tests under KPI_28b once
they are available. KPI_028a which defines the cross border tests is out of the scope of our
tests.
ID of KPI Name of KPI rec. Remark
KPI_01b
Number of manually
initiated eCalls
Y Describes number of “real” eCall
scenarios with vehicle not moving and
voice communication
KPI_02a
Success rate of
completed eCalls using
112
Y It is recommended to use eCall flag for call
establishment with 112
KPI_03
Success rate of received
MSDs
Y Measures exactly what differs eCall from
112 call
D4.3 Final results
17/12/2014 111 v 1.0
KPI_04
Success rate of correct
MSDs
Y Measures proper en-/de-coding of MSD
KPI_05
Duration until MSD is
presented in PSAP
Y Measures time until information is
available to operator
KPI_06
Success rate of
established voice
transmissions
Y Measures basics of eCall, MSD and voice
transmission
KPI_07a
Duration of voice channel
blocking
Y Most important to minimize, as during this
time passengers in the vehicle do not
know if eCall does work or not
KPI_13
Success rate of heading
information
Y This value is calculated by IVS and is
critical to identify right side on highways
KPI_28a
Number of cross border
tests
Y Required tests and should be specified
per member site with which cross border
was performed
KPI_28b
Number of interoperability
tests
Y Required tests and should be specified
per member site with which interoperability
was performed
Table 26 : Recommended KPIs tested in Turkey
5.6.3 Evaluation results
The field tests provide quantitative metrics about system quality and performance. The
results of our KPI evaluation are given in Table 27.
Antalya Unit
KPI_01b 969 -
KPI_02a 98 %
KPI_03 98 %
KPI_04 99 %
D4.3 Final results
17/12/2014 112 v 1.0
KPI_05 35 seconds
KPI_06 98 %
KPI_07a 7 seconds
KPI_13 97 %
Table 27: The results of our KPI evaluation (TR)
KPI_001b: Number of manually initiated eCalls: 969
KPI_002a: Success rate of completed eCalls using 112: 98 %
Number of initiated eCalls: 969
Number of failed eCalls: 16
Number of successful eCalls: Number of initiated eCalls – Number of failed eCalls = 953
eCall success rate: Number of successful eCalls / Number of initiated eCalls = 98 %
KPI_003: Success rate of received MSDs: 98 %
In all the successful test calls, MSD was presented on the operator PC. KPI_002a tells that
98 % of the calls were successful.
KPI_004: Success rate of correct MSDs: 99 %
Number of received MSDs: 953
Number of incorrect MSDs: 4
Number of correct MSDs: Number of received MSDs - Number of incorrect MSDs = 949
MSD success rate: Number of correct MSDs / Number of received MSDs = 99 %
KPI_005: Duration until MSD is presented in PSAP: 35 seconds
MSD presentation time: Point of time of presentation of MSD at operator’s desk in PSAP –
point of time for IVS initiated the eCall
KPI_006: Success rate of established voice transmissions: 98 %
Number of successful voice transmissions: 953
Number of all initiated voice transmissions: 969
Voice transmission success rate: Number of successful voice transmissions / Number of all
initiated voice transmissions = 98 %
KPI_007a: Duration of voice channel blocking: 7 seconds
Duration of voice channel blocking = Start of phase “voice transmission” – IVS starts MSD
transmission
D4.3 Final results
17/12/2014 113 v 1.0
KPI_013: Success rate of heading information: 97 %
All reported heading information: 353 (Heading information was tested in only a subset of the
total test calls)
Correct heading information: 341
Heading information success rate: correct heading information / all reported heading
information = 97 %
5.6.3.1 Evaluation process
The KPIs are evaluated according to the data collected in field tests which were performed in
September- October 2014. All the field tests took place in the province Antalya. 112 PSAP in
Antalya was used as the PSAP for handling eCalls. Manually triggered eCalls were used for
collecting test data. The location of the vehicle was available from two sources. One of them
was the GPS receiver of the IVS unit and the other is the Mobile Network Operator’s position
information. The test statistics were collected both from the logs on the IVS units and the
records in the PSAP centre.
The field tests were performed with a test vehicle exploring different locations in Antalya.
Two test operators were used to gather the call information from the IVS unit in the vehicle
and operator PC in PSAP simultaneously.
Vehicle site:
The test vehicle was equipped with an IVS unit to trigger the manual eCalls and a test PC to
read the IVS log data through serial port. During each call, the test operator in the vehicle
executed the following tasks:
Initiates the manual eCall by pressing a button on the IVS unit and records the start
time of the call.
Checks the availability of voice communication with the other test operator in PSAP
through IVS unit.
Reads the IVS log data from the serial port of the test PC and records the log for
future processing
Records the reference GPS location from a handheld GPS unit.
PSAP side:
D4.3 Final results
17/12/2014 114 v 1.0
A dedicated operator PC was used to perform the field test. This operator PC runs the eCall
operator application. This application had a statistics interface which can be accessed by
authorized personnel only. Figure 2 shows the statistics interface. This interface included the
time duration statistics related with each Call. These statistics were used in the calculation of
the KPIs. During each call, the test operator in the PSAP executed the following tasks:
Receives the eCall and records the time of reception of the call.
Checks the availability of the MSD information, records the MSD content and the time
of reception of the MSD information.
Checks the availability of voice communication with the other test operator in the test
vehicle through the operator application.
Records the timing statistics.
Compares the location information gathered from the MNO database with the one
obtained from the IVS unit and documents the integrity of the two data sources.
5.6.3.2 Time Series
The time series plots of KPI_05 and KPI_07a are given in Figures 7.6.1 and 7.6.2
respectively.
Figure 70: Time series plot of KPI_05 (TR)
D4.3 Final results
17/12/2014 115 v 1.0
Figure 71: Time series plot of KPI_07a (TR)
5.6.3.3 Statistical evaluation
The statistical evaluation is carried according to the procedure given in Section 4.2. Table 28
summarizes the statistical evaluation results.
KPI_05 KPI_07a
min 20 2
max 74 19
mean 35 7
median 35 6
std dev 4 2
skewness 3.95 4.90
Table 28: summary of statistical evaluation results(TR)
The frequency distribution plot of KPI_05 is given in Figure 72.
D4.3 Final results
17/12/2014 116 v 1.0
Figure 72: The frequency distribution plot of KPI_05 (TR)
The frequency distribution plot of KPI_07a is given in Figure 73.
Figure 73: The frequency distribution plot of KPI_07a (TR)
D4.3 Final results
17/12/2014 117 v 1.0
5.6.3.4 Discussion of results
The test results obtained in field tests in Antalya show that the eCall implementation is quite
successful. Operation tests were done in two phases. In the first phase, results were quite
satisfactory. In order to cover whole Antalya region, as a second phase, more tests were
performed than that was planned before. One another fact we discovered during the tests is
that MNO GSM coverage rate of Antalya is quite high. We noted that even in the rural areas,
most of the time the GSM signal was available. This is due to the fact that all Antalya regions
are a very popular tourism destination and the MNOs have implemented enough GSM
stations so that the GSM system is able to give service even in the rural areas.
5.6.4 Interoperability tests
Interoperability tests are planned to be executed in November with Civitronics IVS units.
Calls will be initiated from Romania and Calls will be routed to Ankara Test Bed system.
Therefore the results of these tests are not included in this document.
5.6.5 Recommendations
Vehicle manufacturers shall implement best practise to minimize voice channel blocking
time.
5.6.6 Conclusions
The field tests gave us the opportunity to test our eCall service solution in real life conditions.
The KPIs show that the implementation is quite successful. In our system solution, the eCall
PSAP and 112 PSAP was operating under the same roof and some system components
were used in common. The field tests showed that the two PSAPs can operate together
without degrading the other’s performance.
D4.3 Final results
17/12/2014 118 v 1.0
6 References
[1] Hughes, I G and T P A Hase, Measurements and their Uncertainties: A Practical
Guide to Modern Error Analysis, Oxford University Press, Inc., New York, NY, 2010,
ISBN 978-0199566334
[2] Maindonald, J and W J Brown, Data Analysis and Graphics Using R - an Example-
Based Approach (3rd edition), Cambridge University Press, Cambridge, UK, 2010,
ISBN 978-0521762939.
[3] Ott, R L and M Longnecker, An Introduction to Statistical Methods and Data Analysis
(5th ed), Duxbury, Thomson Learning, Inc., Pacific Grove, CA, 2000, ISBN 978-
0534251222.
D4.3 Final results
17/12/2014 119 v 1.0
ANNEX 1: eCall for P2W - User Acceptance Report
Abstract
The RACC has conducted a study aimed at analysing the acceptance, level of expectations
and possible deployment barriers that an eCall service for motorcycles might have among its
potential future users.
The public pan-European eCall service will only be available for cars, initially. The eCall
service for motorcycles that has been designed and tested in the Spanish pilot in HeERO2 is
partly based on the technical standards that support the eCall service for cars (encoding
information in the MSD, protocol for communication with the PSAP, etc.). However, there are
certain aspects of this system that pose technical and ergonomic/user acceptance
challenges (expectations, perceived usefulness, willingness to pay for the service, privacy).
These latter aspects justify the present study, which has been articulated through an online
survey for motorcycle drivers.
The survey was set up with the "Evalandgo" tool (http://www.evalandgo.com); it was
launched on the 3rd of February and has remained online until 7th of April 2014. The survey
has been structured into five distinct parts, which serve to establish the blocks in which the
study is structured:
1 - Characterization of motorists: age, gender, motorcycle driving experience, type of
(frequent) motorcycle and mobility habits. These data serve to further disaggregate data by
driver profile.
2 - Accident statistic: the respondents have been asked to make a very brief account of their
life as motorcycle drivers, whether they have had any accident, if so, in which environment
and how the actions of the emergency services developed. The objective is to estimate the
utility, mainly in saved time, the eCall service for motorcycles might have to improve (time of)
assistance in case of accident.
3 - Acceptance of the eCall service for motorcycles: the main features of the system have
been introduced to the user (i.e. automatic data acquisition and sending to PSAP in case of
accident; option of using a helmet connected to the system and capable of acquiring
additional data, send to the PSAP together with data from the motorcycle and establishing a
D4.3 Final results
17/12/2014 120 v 1.0
voice call hands-free) and we have asked about the usefulness and acceptability of this
solution as well as the willingness to pay for a helmet compatible with eCall and an
aftermarket device.
4 - User Expectations: we have asked what data / information is considered useful or
desirable that the system would be able to acquire and send to the PSAP, and the relative
importance of these for the emergency management. The objective is to identify what
characteristics or requirements, in terms of information, might be interesting that an eCall
service for motorcycles would consider, and that would differentiate from the standard set of
data that is sent for cars (MSD).
5 - Other issues: privacy of the data has been assessed, especially if the user would
voluntarily opt to use a helmet with sensors capable of acquiring much more data than the
standardised set of data in the MSD; finally, we have asked about the introduction of the
eCall service for motorcycles, and how you could have access to it through aftermarket
devices for those motorcycles that would not have the built-in system because were
produced before mandatory implantation.
From the analysis and further processing of the received responses, we have derived the
following statistics and conclusions.
A 1 Characterization of motorists
A 1.1 Profile of respondents
636 people have responded to the survey. The majority (83.96%) of respondents is males
and only 16.04% are female.
D4.3 Final results
17/12/2014 121 v 1.0
Figure 74: Gender of the respondents
Most respondents are concentrated in the adult age groups (27-30, 31-34, 35-38 and 39-42)
and there are few young motorcycle drivers or older than 50. However, we emphasize that
the women driving motorcycles seem to tend to be younger than men (we have not picked
any response from a woman motorist in the age range from 55 years old), a fact possibly due
to later incorporation of women to driving motorcycles.
Figure 75: Pyramid: ages and gender
This hypothesis seems to be confirmed if we look at the data referring to the years of
experience of drivers, since women report fewer years of experience than men driving
motorcycles:
83,96%
16,04%
Male
Female
14% 12% 10% 8% 6% 4% 2% 0% 2% 4%
15-18
19-22
23-26
27-30
31-34
35-38
39-42
43-46
47-50
51-54
55-58
59-62
63+
Men % Women %
D4.3 Final results
17/12/2014 122 v 1.0
Figure 76: Years of experience driving a motorcycle, by gender
A.1.1.1 Mobility patterns of the respondents
The type of motorcycle more respondents report they use is the "scooter", followed by the
model "naked". Two thirds of motorists surveyed (nearly 67%) responded that they drive a
motorcycle daily. Among these, the scooter type model is the most used (34.51% of users
driving a motorcycle on a daily basis, take a scooter).
Men
25,71%
23,44%23,25%
15,12%
12,48%
0-5
6-10
11-20
21-30
More than 30
Women
32,00%
28,00%
20,00%
18,00%
2,00%
0-5
6-10
11-20
21-30
More than 30
D4.3 Final results
17/12/2014 123 v 1.0
Figure 77: Frequency of use
Figure 78: Frequency of use by type of motorcycle
The average number of driven kilometres per year is according to the following statistic:
66,98%
33,02%
On a daily basis
Only weekends, bank
holidays or sporadically
3,76%
0,70%
23,71%
34,51%
9,39%
12,21%
2,58%
10,56%
0,23%
0,47%
1,88%
-6,67%
-3,81%
-26,19%
-13,81%
-18,10%
-11,90%
-4,76%
-12,86%
-0,48%
-1,43%
40% 30% 20% 10% 0% 10% 20% 30% 40%
Custom
Enduro
Naked
Scooter
Sport
Sport touring
Touring
Trail
Trial
Electric
Other
Daily Only weekends, bank holidays or sporadically
D4.3 Final results
17/12/2014 124 v 1.0
Figure 79: Average number of Km. driven per year
A 1.2 Accident statistics
Nearly a third of respondents have suffered an accident in which medical assistance was
needed and / or traffic police had to intervene.
“Have you suffered any accident that involved the emergency services and/or the
traffic police?”
Figure 80: Accident statistics of respondents
41,20%
32,39%
17,45%
8,96%
< 5.000 Km
5.001 - 10.000 Km
10.001 - 15.000 Km
> 15.000 Km
29,40%
70,60%
Yes
No
D4.3 Final results
17/12/2014 125 v 1.0
Figure 81: Accident statistics of respondents by age group
We have not found a clear relationship between years of experience driving a motorcycle
and the accidents statistics. Among respondents motorists that have had an accident,
statistics on years of experience is very distributed:
D4.3 Final results
17/12/2014 126 v 1.0
Figure 82: Years of experience of motorcycle drivers that have had at least one accident
More than half (55.32%) of motorists who have ever had an accident did not personally call
the emergency services, and 14.36% did not in most cases. In these cases, an automatic
warning when the accident occurs could be particularly useful.
Figure 83: Did you call the emergency services (yourself)?
Negatively surprising are the statistics on the average time of arrival of the emergency
services to the accident spot, with 16.49% of motorists who answer that it took between 30
and 50 minutes, while 4.79% said it took more 50 minutes, which can clearly be improved
(for example, with the help of the eCall service). Only 17.02% said that emergency services
arrived within 10 minutes or less.
12,97%
24,32%
28,65%
18,92%
15,14%
0-5
6-10
11-20
21-30
More than 30
10,64%
19,68%
14,36%
55,32%
Yes, in most cases
Yes, in all cases
No in most cases
No, never
D4.3 Final results
17/12/2014 127 v 1.0
Figure 84: How much time elapsed until the emergency services arrived?
Among respondents who have ever suffered a motorcycle accident, the distribution of these
between urban / interurban area (road) is the following:
Figure 85: Distribution of accidents between urban and interurban area
The average time of arrival of the emergency services depending on the accident location
(urban, city / intercity area, road) is the following:
Urban (City)
17,02%
48,40%
16,49%
4,79%
13,30%
< 10 minutes
10 - 30 minutes
30 - 50 minutes
> 50 minutes
I don't remember
61,70%
38,30%
Urban (City)
Interurban (Road)
D4.3 Final results
17/12/2014 128 v 1.0
Interurban (Road)
Figure 86: Average time of arrival of the emergency services to the accident spot depending on the type of area (urban / interurban)
Statistics shows that in urban areas the time of arrival of the emergency services to the spot
of a motorcycle accident is lower, on average, than on the road. So while in town in about 2
out of every 3 accidents (59%) the time of arrival of the emergency services is between 10
and 30 minutes, in road environment statistics is almost 1 in 3 (32 %) while statistics is the
opposite as to accidents assisted between 30 and 50 minutes after the accident, with only
20,87%
58,26%
6,96%
3,48%
10,43%
< 10 minutes
10 - 30 minutes
30 - 50 minutes
> 50 minutes
I don't remember
11,11%
31,94%
31,94%
6,94%
18,06%
< 10 minutes
10 - 30 minutes
30 - 50 minutes
> 50 minutes
I don't remember
D4.3 Final results
17/12/2014 129 v 1.0
7% in the city and 32% in road environment. Twice as many accidents are attended within 10
minutes compared with inter urban area (21 % vs. 11%).
Overall, it seems clear that the eCall service for motorcycles will be particularly useful to help
shorten assistance to a motorcycle accident especially interurban area, where time of arrival
of emergency services at an accident spot usually takes longer than in urban areas, in part
because there are greater distances to travel between accident locations and emergency
centres.
A 1.3 Acceptance of the eCall service for motorcycles
In this part of the survey was briefly introduced the characteristics of the eCall service by the
following explanation:
"In the future eCall service for motorcycles, the motorcycle will have a built-in device able to
acquire data from the accident and send it to the 112 emergency services (just like the eCall
for cars). Besides, and optionally, if you have an eCall compatible helmet it will be capable of
acquiring additional data and send it, together with the rest of data, to the 112. Moreover, the
helmet will also allow to automatically establishing a hands-free voice call with the PSAP."
D4.3 Final results
17/12/2014 130 v 1.0
Figure 87: Architecture of the eCall service for motorcycles
The following question was asked about the degree of acceptance that such a system would
have among its potential users, and the results were clearly positive, with the vast majority of
motorists who would strongly agree (60.53%) or agree (29.25%) in that the eCall service for
motorcycles would be very useful to reduce mortality caused by motorcycle accidents, and
therefore would like to have this system available on their motorcycle.
"Do you think the eCall for motorcycles would help reduce fatalities and/or injuries
caused by motorcycle accidents and, consequently, would you like that your
motorcycle or scooter would have this system?"
D4.3 Final results
17/12/2014 131 v 1.0
Figure 88: Level of acceptance of the eCall service for motorcycles
Extra details about the feature, optional, which would provide a helmet compatible with the
eCall service, were provided:
"In the prototype system we are proposing, the helmet would have sensors capable of estimating the injure caused on the driver or passenger (severity of the impact on the head). Besides, the helmet will also help know whether there was an additional passenger in the motorcycle (apart from the driver), determine the final position after the accident, etc.
All this information would also be sent to the emergency services, automatically, in case of
accident. "
We asked motorists about the relevance, in its discretion, of this type of information. Again, a
large majority considers this information as relevant (30, 03%) or very relevant (64, 31%):
60,53%
29,25%
7,08%3,14%
Strongly agree
Agree
I little agree
Disagree
D4.3 Final results
17/12/2014 132 v 1.0
Figure 89: Relevance of the information provided by the helmet in case of accident
Unlike the eCall service for cars (which will come fitted as standard equipment in all new cars
from the mandatory implementation of this system at European level) motorcycle drivers will
have to make an additional investment in order to have all the features a compatible helmet
will be able to provide.
While it is not mandatory to replace the helmet, since the built-in eCall service will be able to
send a minimum set of information about the accident (timestamp, location, direction, etc.),
as previously noted, users believe that the additional information that the helmet can send is
not only complementary, but relevant or very relevant for the emergency management, so
the willingness of respondents to buy a new helmet is very big. This is the question we
formulated:
"It is estimated that a helmet compliant with eCall will have an extra cost of around 150€
compared with a “conventional” helmet…
Would you change your helmet in order to be able to send additional information (about the
impact on the head, etc.) and to speak with the 112 after an accident?”
64,31%
30,03%
4,09%
1,57%
Very relevant
Relevant
A bit relevant
No relevant at all
D4.3 Final results
17/12/2014 133 v 1.0
Figure 90: Willingness to purchase a new helmet compatible with eCall
However, we observe that the proportion of respondents who considered the information that
the helmet can send as relevant or very relevant (94.33%) does not match the proportion of
users who would be willing, therefore, to buy a new compatible helmet (77,83%). It is quite
evident that there is a clear relationship between this result and the observed price sensitivity
(of the helmet) that some of the respondents have shown. This is the question we have
made in this regard:
"You think that the approximate extra cost of 150€ is"
And this is the result we have obtained:
Figure 91: Price sensitivity (for the price increase of an eCall compatible helmet)
77,83%
22,17%
Yes
No
14,62%
51,26%
26,42%
7,70%
Unreasonably expensive
Expensive
A fair price
Cheap, considering the
features of the system
D4.3 Final results
17/12/2014 134 v 1.0
2 out of 3 respondents (65.88 %) consider the price increase of a helmet compatible with
eCall as expensive or unacceptably expensive (however, a significant proportion of these
would change their helmet anyway).
We have not observed any noteworthy tendency in the statistics of the willingness to change
the helmet in relation with the type of motorcycle driven by the respondent, having in all
cases a very similar proportion to that in Figure 90: Willingness to purchase a new helmet
compatible with eCall We have not detected a greater willingness to purchase a new helmet
eCall support among those motorists who have had an accident, or less willingness among
those who have never had an accident. In both cases, the proportion is very similar to that of
Figure 90: Willingness to purchase a new helmet compatible with eCall. All this seems to
indicate a very homogeneous and transverse interest for the eCall service for motorcycles,
as users understand its potential to be useful in case of accident.
It is interesting how using data from figures 1-16, 1-17 and 1-18 we come to the following
conclusions:
- Among those who consider that the information that the helmet can send in case of
accident is very relevant but still would not change their helmet, 3 out of 4
respondents consider this price increase as expensive (51.06%) or unacceptably
expensive (25.53%).
- Among those who consider that the information that the helmet can send in case of
accident is relevant but still would not change their helmet, 41.27% of respondents
consider this price increase as expensive while 44.44% consider it unacceptably
expensive.
A 1.4 User Expectations
The eCall service for cars is fully standardized, that is not the case with the eCall for
motorcycles. The development of this system poses challenges at a technical level and at
user level. It is important to have a vision of what the expectations are, in terms of
requirements, manifested by the future users of the system, because this way we can design
a system that is suited to their needs.
D4.3 Final results
17/12/2014 135 v 1.0
We have tackled this topic in the following way:
"In case of accident, the eCall service will automatically send information about it to the
emergency rescue service (112). Among other data, it will send the timestamp and exact
location where the accident has occurred.
A motorcycle accident is usually different to an accident involving a car, for this reason it
might be interesting to send additional data that are currently not part of the eCall service for
cars."
The following types of information could be generated from data collected by the telematics
system embedded on the motorcycle and / or the helmet. We evaluated the importance
motorists give to these types of information on a scale of 0-5.
These are the results we obtained:
Figure 92: Importance respondents give to different types of information that the eCall service for motorcycles could generate
Overall, respondents expressed a considerable interest in the types of information suggested
by CEIT and NZI. If we arrange and analyse in further detail the assessments of the
respondents, we can draw some interesting conclusions:
D4.3 Final results
17/12/2014 136 v 1.0
Type of information Score Comments
Final location of the driver 4,67 Motorists give more importance to be able to
send the final location of the driver (4.67 rating
points, on average, over 5) or of the other
passenger (4.64 rating points, on average, over
5) versus final location of the motorcycle (3.03
rating points, on average, over 5). In fact, this
information is the most valued by respondents.
Final location of the other
passenger (if there was any) 4,64
Number of impacts (against the
ground, obstacles, etc.) suffered
by the driver
4,53
Motorists are clearly more concerned about
information relating to the severity of the impacts
on the driver (number of impacts, deceleration)
as a result of the accident that the same
information on the motorcycle, giving these types
of information a very high score, close to the
maximum of 5 points.
Deceleration caused due to the
impacts on the driver 4,45
Speed of the motorcycle when the
accident happened 3,69
The speed at which the accident occurred
together with the final location of the motorcycle
and the severity of the impacts are important
types of information, because they provide an
estimation of the severity the accident might
have, but get a lower valuation compared with
the information directly related to the severity of
the accident on the driver.
Distance between the motorcycle
and the driver 3,18
Final location of the motorcycle 3,03
Deceleration caused due to the
impacts on the motorcycle 3
Number of impacts (against the
ground, obstacles, etc.) suffered
by the motorcycle
2,92
The main conclusion we draw is the importance motorists confer to the information that a
helmet compatible with eCall can send about the location of the driver (and passenger if any)
and the severity of the impacts suffered by them in the accident, besides information directly
related to the motorcycle.
Additionally, we asked respondents to indicate, on a free text field, what other types of
information they would consider useful to automatically send in case of accident and the
D4.3 Final results
17/12/2014 137 v 1.0
features they would like the system could offer. We have grouped the responses into
thematic blocks. Following is a summary of these:
Proposals from the
respondents Comments
Personal data of the driver
and passenger (if any)
A large number of respondents have stated that it would be interesting
to send the PSAP the following personal information about the
occupants of the motorcycle:
- Name, Date of birth, Weight, Height, etc.
- Medical data: allergies, blood type, medical history, etc.
- Contact numbers of family or friends to notify in case of accident.
- Other data: if the user is a donor (in case of death in the accident).
The data would be linked to the helmet, so that a helmet would be
uniquely associated with a user. For obvious reasons, this is
information that would be sent optionally, provided the user gives
consent.
For the owner of the helmet to update these data, the helmet could
have an associated unique identifier. The user would sign in the
helmet manufacturer's website, enter this unique identifier and fill in a
form with all the data referenced above. These data would be stored in
a database. The website of the manufacturer of the helmet should fulfil
and require acceptance by the user of a privacy and data protection
policy.
In the event of an accident, the helmet could send its unique identifier
encoded in one of the optional fields of the MSD, and emergency
services could access a pan-European database where, entering this
identifier they could retrieve all relevant data of the user for a better
management of the emergency. The access protocol and design of
this database should be standardized at European level.
The user would be responsible to update the data, for example, in
case of change of ownership of the helmet.
D4.3 Final results
17/12/2014 138 v 1.0
Vital signs of the injured
users
A number of respondents indicated that they would like the helmet was
able to collect data about the user state after the accident:
- Level of consciousness
- Heartbeat
- Temperature
- Etc.
The helmet designed by NZI is unable, for the moment, to acquire this
type of data. Feasibility (at technical level) of taking vital data about
the user should be researched, and data should be standardized in the
optional data fields of the MSD. Again, the challenge of data privacy
(and probably more expensive helmet for additional instrumentation to
acquire such data) arises.
Characterization / accident
reconstruction
Some users found it interesting that the eCall service would be able to
function as a black box capable of collecting data to help latter
reconstruction of the accident. For example:
- The possibility to determine the location of the impact/s on
motorcycle (if the accident was due to a frontal impact, side impact,
rear impact ...).
- If it is an accidental fall or accident is mainly due to the impact
against an obstacle or another vehicle.
- The tilt angle of the motorcycle at the time of the accident.
- Operated controls on the motorcycle before the impact (for example,
if the motorcycle broke, which would indicate a previous reaction to the
accident).
- Video Recording / Photos: ability to store photos / video seconds
before and after an accident.
- Trace prior to the accident: store and send several lat-long
coordinates before the accident.
- Involvement of other vehicles in the accident (through correlation with
eCall sent by other vehicles in the same place and time)
Regardless of the feasibility, at technical level, to get this type of data
(and subsequent algorithms needed to process and draw conclusions
from them), the question arises whether the eCall service should, from
D4.3 Final results
17/12/2014 139 v 1.0
a conceptual point of view, serve for this purpose, as the automatic
emergency call has one goal which is to shorten the time for
assistance in case of accident and provide useful information to the
emergency services (fire brigade, ambulance, ...) for a better
emergency management.
It seems clear that this type of information would be useful not so
much for the emergency management but for subsequent claims (for
example, for the insurance company, or if any of those involved in an
accident were to give evidence in a possible lawsuit).
Again, the acquisition data other than the MSD standard poses a
challenge in terms of privacy (the user should be free to allow - or not -
that these data would be sent in case of accident, and to decide which
to which service provider these would be sent).
Finally, it is clear that the installation of a telematics device on the
motorcycle (and helmet, optionally) can serve as a basis for the
development of many applications / additional services to eCall, and
this poses a new challenge to enable service providers (such as auto
clubs, insurance companies, etc.) to develop and offer value-added
services based on the eCall service.
Environmental conditions
at the accident site
Some users have suggested that it might be interesting to transmit
data on the environmental conditions at the accident site.
- Temperature
- Humidity (if it is raining)
- etc.
The system designed by CEIT is unable, for the moment, to get this
type of data. Feasibility (at technical level) of introducing temperature
sensors, humidity, etc. should be researched, and data should be
standardized in the optional data fields of the MSD. Again, this could
increase the cost for the additional system instrumentation to acquire
such data.
Data after the accident Some users have contributed with ideas on data that could be sent
once the accident has occurred, such as:
D4.3 Final results
17/12/2014 140 v 1.0
- Position the driver / passenger after the event.
- Position of the motorcycle after the accident, if it varies, it
could mean that the driver is not unconscious or there is
someone who can provide help.
- Ability to detect if the helmet is in the head of the occupants
after the accident.
As with the cases before, it is very questionable the usefulness of this
type of information for the emergency management, as this must be as
fast as possible and there (in principle) should not exist any other
information that might distract that is not strictly necessary for prompt
assistance. However, for different applications, such as reconstruction
of an accident, these might be useful, and in this case the technical
and privacy challenges noted above would need to be considered.
Other features
- Option that the driver can manually activate or cancel the
eCall (already implemented in the system designed by CEIT).
- Enable light signal (LED or similar) in the helmet if the
accident occurs at night on unlit roads.
- The device should be easily disabled, if the user wishes so
(for example, in order to run in circuits without the eCall
activated).
A 1.5 Other issues
We have evaluated the aspect of data privacy. More than two thirds of respondents do not
care (46.38%) or care little (25.79%) that data is transmitted from the eCall service. This is
the statistic:
D4.3 Final results
17/12/2014 141 v 1.0
Figure 93: Sensitivity to data privacy of the respondents
Finally, in a scenario like that which will occur in the case of cars, where motorcycles without
the eCall service coexist with new motorcycles with the built-in eCall service, we asked about
the willingness to pay for an aftermarket eCall service, obtaining the following results.
"If your motorcycle does not have the built-in eCall service… Would you buy and install in
your motorcycle an aftermarket device that would be compliant with eCall?"
46,38%
25,79%
7,08%
14,78%
5,97%
I am not concerned
I am a bit concerned
I do not know
I am concerned
I am very concerned
D4.3 Final results
17/12/2014 142 v 1.0
Figure 94 : Price sensitivity for an aftermarket eCall systemeCall service
28,46%
47,64%
16,04%
7,86%
Yes, but only if the total cost of the device (including cost of the installation in my
motorcycle) would not exceed 50€
Yes, but only if the total cost of the device (including cost of the installation in my
motorcycle) would not exceed 100€
Yes, but only if the total cost of the device (including cost of the installation in my
motorcycle) would not exceed 200€
No
D4.3 Final results
17/12/2014 143 v 1.0
A 2 Conclusions
The Spanish pilot in HeERO2 is one of the first experiences at European level to develop,
implement and test an automatic eCall emergency call designed for two-wheeled vehicles.
The user acceptance study conducted by the RACC allows us to draw some interesting
conclusions about this system:
- On the sample of surveyed motorcycle drivers: one in three respondents has suffered an
accident that required the emergency services to participate. We have not identified a clear
link between the experience of motorists and accidents, not by age either. More than half of
respondents who have ever had an accident not personally alerted the emergency services,
while the average time of their arrival at the accident spot is significantly higher in interurban
area compared with urban. However, in both cases the time of arrival at the accident spot
could be clearly improved, for example with the help of an automatic warning to the closest
PSAP.
- On the level of acceptance of the eCall service for motorcycles: a large majority of
respondents motorists would like to have eCall in their motorcycles, and in slightly smaller
proportion they would be willing to change their helmet to have the full functionality of the
system, although they consider that the estimated increase in price compared to the price of
a conventional helmet is rather expensive. On the transition to a future eCall service for
motorcycles, users are receptive to pay for aftermarket devices to be able to have eCall on
their motorcycles manufactured before the system is implemented at European level in all
new motorcycles.
- On the expectations of users: surveyed motorists place a high potential to the eCall service
for motorcycles, and require some features that, conceptually, are outside the scope of the
eCall service. In any case, these features (sending of personal data and medical history of
the driver and passenger, etc.) will most likely be implemented as value added services by
private service providers.
From the results of the first laboratory tests, as well as the expectations generated among
future potential users, we can conclude that, although there are still many unresolved issues
(technical, standardization, conceptual, privacy-related, ergonomics-related, of costs, etc.),
D4.3 Final results
17/12/2014 144 v 1.0
this system could be useful for a faster and more efficient management of motorcycle
accidents, helping to reduce fatalities in these vehicles, especially in interurban area.
D4.3 Final results
17/12/2014 145 v 1.0
A 3 Questionnaire used
Survey eCall for motorcycles
eCall for motorcycles: the user point of view
From October 2015 all passenger cars in the EU will come with an electronic safety
system that, in case of a severe accident, will automatically call the emergency
services (112). This system will inform the 112 about the exact location of the accident
and will send additional data, such as the number of passengers in the car, the vehicle
identification number, etc. shortening the overall rescue time and contributing to save
up to 2500 lives a year in Europe.
However, this system won’t be initially available for motorcycles!
Within the HeERO2 European project a pilot system for eCall for motorcycles is being
developed, and RACC Automobile Club is coordinating a study to investigate the
requirements needed to extend the eCall to this kind of vehicles. This study is made in
cooperation with the Directorate-General of Traffic in Spain together with other
partners of the project.
In order to better understand which specific needs the eCall service for motorcycles
should take into account, we would like to ask you, only if you are a motorcycle driver,
that you answer the following questions. It will not take you more than 5 minutes and
it will be a great source of information for us.
D4.3 Final results
17/12/2014 146 v 1.0
Thank you very much for your cooperation.
[All questions are to be defined as mandatory in the Evalandgo tool]
1. Your gender
a) Male b) Female
2. Year of birth
(1900/1901/…/up to 1999)
3. Years of driving experience (motorcycle or scooter)
D4.3 Final results
17/12/2014 147 v 1.0
a) 0-5 b) 6-10 c) 10-20 d) 20-30 e) More than 30
4. How often do you drive a motorcycle or scooter?
a) Daily b) Only weekends, bank holidays or sporadically
5. What kind of motorcycle do you drive most frequently?
a) Custom
b) Enduro / MotoCross
c) Naked
d) Scooter
D4.3 Final results
17/12/2014 148 v 1.0
e) Sport
f) Sport Touring
g) Touring
h) Trail
D4.3 Final results
17/12/2014 149 v 1.0
i) Trial
j) Electric
k) Other: please specify
6. How many km do you usually drive per year by motorcycle or scooter?
a) Less than 1500 b) Between 1500 and 3000 c) Between 3001 and 5000 d) Between 5001 and 10000 e) Between 10001 and 15000 f) Between 15001 and 20000 g) More than 20000
7. Have you suffered any accident that involved the emergency services and/or the traffic police?
a) Yes b) No
[CONDITIONAL, if answer to question 7. is ‘Yes’ continue to question 8. if answer
in ‘No’ skip to question 11: be careful to configure this properly in the “Evalandgo”
tool]
D4.3 Final results
17/12/2014 150 v 1.0
8. Did you call the emergency services (yourself)?
a) Yes, always b) Yes, in the majority of cases c) In the majority of cases, No d) No, never
9. How much time elapsed until the emergency services arrived?
a) Less than 10 minutes b) Between 10 and 30 minutes c) Between 30 and 50 minutes d) More than 50 minutes e) I don’t remember
10. The accident was in…
a) Urban zone b) Interurban zone
In the future eCall service for motorcycles, the motorcycle will have a built-in device
able to acquire data from the accident and send it to the 112 emergency services (just
like the eCall for cars). Besides, and optionally, if you have an eCall compatible helmet
it will be capable of acquiring additional data and send it, together with the rest of
data, to the 112. Moreover, the helmet will also allow to automatically establishing a
hands-free voice call with the 112.
D4.3 Final results
17/12/2014 151 v 1.0
Now we will ask you some more specific questions about this system…
11. Do you think the eCall for motorcycles would help reduce fatalities and/or injuries caused by motorcycle accidents and, consequently, would you like that your motorcycle or scooter would have this system?
a) I do not agree b) I partially agree c) I agree d) I strongly agree
12. In the prototype system we are proposing, the helmet would have sensors capable of estimating the injure caused on the driver or passenger (severity of the impact on the head). Besides, the helmet will also help know whether there was an additional passenger in the motorcycle (apart from the driver), determine the final position after the accident, etc.
All this information would also be sent to the emergency services,
automatically, in case of accident. For the emergency management, you
consider this information as:
D4.3 Final results
17/12/2014 152 v 1.0
a) Not relevant at all b) A bit relevant c) Relevant d) Very relevant
It is estimated that a helmet compliant with eCall will have an extra cost of around
150€ compared with a “conventional” helmet…
13. Would you change your helmet in order to be able to send additional information (about the impact on the head, etc.) and to speak with the 112 after an accident?
a) Yes b) No
14. You think that the approximate extra cost of 150€ is:
a) Too expensive b) Expensive c) A fair price d) Cheap, considering what the system does
In case of accident, the eCall service will automatically send information about it to the
emergency rescue service (112). Among other data, it will send the timestamp and
exact location where the accident has occurred.
A motorcycle accident is usually different to an accident involving a car, for this
reason it might be interesting to send additional data that are currently not part of the
eCall service for cars.
15. Please, rate in a scale from 0 to 5 how important are, in your opinion, the following types of information, where 0 = ‘not important at all’ and 5 = ‘very important’
a) Speed of the motorcycle when the accident occurred b) Number of impacts (ground, obstacles) suffered by the motorcycle c) Number of impacts (ground, obstacles) suffered by the driver d) Deceleration suffered due to the impacts on the motorcycle e) Deceleration suffered due to the impacts on the driver f) Final location of the motorcycle
D4.3 Final results
17/12/2014 153 v 1.0
g) Final location of the driver h) Final location of the passenger (if there is one, and wears a compliant
helmet) i) Distance between the motorcycle and the driver
Please, tell us what other kind of information you would consider useful to be
sent (automatically): [free text field] [NOT mandatory]
16. Are you concerned that such a system would have access to your personal data (for example, your location, in case of accident), even if it is a public service?
a) I am not concerned b) I am a bit concerned c) I do not know d) I am concerned e) I am very concerned
17. If your motorcycle does not have the built-in eCall service… Would you buy and install in your motorcycle an aftermarket device that would be compliant with eCall?
a) No b) Yes, but only if the total cost of the device (including cost of the
installation in my motorcycle) would not exceed 50€ c) Yes, but only if the total cost of the device (including cost of the
installation in my motorcycle) would not exceed 100€ d) Yes, but only if the total cost of the device (including cost of the
installation in my motorcycle) would not exceed 200€
D4.3 Final results
17/12/2014 154 v 1.0
ANNEX 2: LIST OF ALL EVALUATED KPIS
FT-
LUX
1
FIC
-
LUX
2
FIC
-
LUX
3B
E 1
BE2
BG
1B
G2
Gal
icia
ph
1 -
ES1M
adri
d
ph
1-
Mad
rid
ph
2 -
ES
Gal
icia
ph
2 -
ES Fu
j-D
K 1Fu
j-d
orm
-D
K 2Fu
j-D
K3
GM
V
DK
4TK
IVS
1/P
SAP
1IV
S 2
/PSA
P 1IV
S 1
/PSA
P 1
IVS
2/P
SAP
17
un
its
DO
RM
AN
TN
orm
al
Ro
amin
Res
ult
Res
ult
Res
ult
Res
ult
Res
ult
Res
ult
Res
ult
5 u
nit
s1
un
its
5 u
nit
s
KP
I_0
01
aN
um
ber
of
auto
mat
ical
ly in
itia
ted
eC
alls
--
-1
60
94
53
71
15
25
81
63
N/A
KP
I_0
01
bN
um
ber
of
man
ual
ly in
itia
ted
eC
alls
31
72
63
16
35
32
10
74
14
71
37
11
72
16
19
41
85
43
92
19
69
KP
I_0
02
aSu
cces
s ra
te o
f co
mp
lete
d e
Cal
ls u
sin
g 1
12
--
-7
5.2
78
.99
2.7
79
7.7
7N
/AN
/AN
/AN
/A9
8
KP
I_0
02
bSu
cces
s ra
te o
f co
mp
lete
d e
Cal
ls u
sin
g lo
ng
nu
mb
er6
2.4
69
6.9
69
6.9
47
5.2
07
8.9
08
4.9
29
5.9
27
2.4
16
2.5
75
.38
89
4.0
59
3.0
26
6.6
79
5.2
4
KP
I_0
03
Succ
ess
rate
of
rece
ived
MSD
s6
2.4
66
7.3
07
8.5
77
3.3
07
3.3
09
2.7
78
5.4
28
3.7
64
.34
76
.88
99
9.4
31
00
10
09
5.2
49
8
KP
I_0
04
Succ
ess
rate
of
corr
ect
MSD
s1
00
10
01
00
73
.37
3.3
10
0.0
01
00
83
.76
2.5
10
09
01
00
10
01
00
10
09
9
KP
I_0
05
Du
rati
on
un
til M
SD is
pre
sen
ted
in P
SAP
9.8
81
4.1
51
4.0
01
1.0
01
1--
15
18
.22
0.8
91
3.5
51
3.0
71
1.8
31
8.7
23
5
KP
I_0
06
Succ
ess
rate
of
esta
blis
hed
vo
ice
tran
smis
sio
ns
98
.42
96
.96
96
.94
98
.06
10
08
2.0
96
2.5
75
.39
99
4.5
99
3.0
26
6.6
71
00
98
KP
I_0
07
aD
ura
tio
n o
f vo
ice
chan
nel
blo
ckin
g7
.54
11
.90
11
.29
13
13
5.0
03
10
.61
0.7
29
.88
9.5
98
.49
11
.56
7
KP
I_0
07
bD
ura
tio
n o
f vo
ice
chan
nel
blo
ckin
g: a
uto
mat
ic
--
-5
39
.71
.88
-9
.06
8.8
88
.75
KP
I_0
08
Tim
e fo
r ca
ll es
tab
lish
men
t-
--
24
45
--1
71
2.8
15
.93
3.7
3.6
5.2
-
KP
I_0
09
Acc
ura
cy o
f p
osi
tio
n9
7.1
61
93
.91
00
acce
pta
ble
1.5
--1
00
10
01
1.9
1-
3.3
80
.25
0.1
4
KP
I_0
10
Nu
mb
er o
f u
sab
le s
atel
lites
--
-1
0.5
0>1
28
.74
99
7.8
2 N
/A N
/A N
/A N
/A
KP
I_0
11
Geo
met
ric
dilu
tio
n o
f p
reci
sio
n-
--
1.3
7<1
,51
.44
--1
.41
N/A
N/A
N/A
N/A
KP
I_0
12
Tim
e b
etw
een
su
cces
sfu
l po
siti
on
ing
fixe
s-
--
11
10
.11
16
5.3
2 N
/A N
/A N
/A N
/A
KP
I_0
13
Succ
ess
rate
of
hea
din
g in
form
atio
n9
7.1
61
93
.91
00
10
01
00
39
Ph
ase
26
8.5
70
6.2
59
7
KP
I_0
14
Succ
ess
rate
of
VIN
dec
od
ing
wit
ho
ut
EUC
AR
IS-
--
10
01
00
96
.77
--1
00
88
N/A
N/A
N/A
N/A
KP
I_0
15
Succ
ess
rate
of
VIN
dec
od
ing
wit
h E
UC
AR
IS-
--
n/a
n/a
N/A
N/A
N/A
N/A
KP
I_0
16
Tim
e fo
r V
IN d
eco
din
g w
ith
EU
CA
RIS
--
-n
/an
/a N
/A N
/A N
/A N
/A
KP
I_0
17
Dis
pat
ch t
ime
of
inci
den
t d
ata
to r
escu
e fo
rces
--
-n
/an
/a N
/A N
/A N
/A N
/A
KP
I_0
18
Mea
n t
ime
to a
ctiv
ate
resc
ue
forc
es-
--
n/a
n/a
N/A
N/A
N/A
N/A
KP
I_0
19
Dis
pat
ch t
ime
of
inci
den
t d
ata
to T
MC
--
-n
/an
/a N
/A N
/A N
/A N
/A
KP
I_0
20
Succ
ess
rate
of
pre
sen
ted
inci
den
t d
ata
in T
MC
--
-n
/an
/a8
3.7
62
.5 N
/A N
/A N
/A N
/A
KP
I_0
21
Nu
mb
er o
f su
cces
sfu
l cal
l-b
acks
--
-9
21
09
33
25
4 P
has
e 2
05
14
KP
I_0
22
Succ
ess
rate
of
call-
bac
ks-
--
10
01
00
98
.94
84
72
Ph
ase
21
00
87
.5
KP
I_0
23
GSM
net
wo
rk la
ten
cy-
--
<3<3
59
.96
N/A
N/A
N/A
N/A
KP
I_0
24
11
2 n
atio
nal
net
wo
rk la
ten
cy-
--
<1 N
/A N
/A N
/A N
/A
KP
I_0
25
11
2 o
per
ato
r re
acti
on
tim
e-
--
2.5
35
2.5
35
N/A
N/A
N/A
N/A
KP
I_0
26
Tim
e fo
r ac
kno
wle
dge
men
t o
f em
erge
ncy
ser
vice
s-
--
n/a
n/a
N/A
N/A
N/A
N/A
KP
I_0
27
Tota
l res
po
nse
tim
e-
--
n/a
n/a
N/A
N/A
N/A
N/A
KP
I_0
28
aN
um
ber
of
cro
ss-b
ord
er t
ests
--
-n
/an
/a4
2 N
/A N
/A N
/A N
/A
KP
I_0
28
bN
um
ber
of
inte
rop
erab
ility
tes
ts-
--
10
48
69
42
TB
D T
BD
TB
D T
BD
KP
I_0
28
cN
um
ber
of
cro
ss r
egio
nal
tes
ts0
00
n/a
n/a
38
45
83
68
N/A
N/A
N/A
N/A
KP
I_2
9D
isp
atch
tim
e o
f In
term
edia
te P
SAP
n/a
n/a
44
.71
N/A
N/A
N/A
N/A
KP
I_3
0N
um
ber
of
calls
fla
gged
as
dan
gero
us
goo
d0
00
n/a
n/a
N/A
N/A
N/A
N/A
KP
I_3
1N
um
ber
of
succ
essf
ul a
cces
s o
f d
ange
rou
s go
od
s in
form
atio
n0
00
n/a
n/a
N/A
N/A
N/A
N/A
KP
I_3
2N
um
ber
of
Do
rman
t SI
M c
ard
tes
ts1
85
43
92
1
Nam
e o
f K
PI
FT/m
an/P
SAP
1FI
C I/
man
/PSA
P 1
FIC
II/m
an/P
SAP
1
D4.3 Final results
17/12/2014 155 v 1.0