Impact evaluation of speed regulation systems using naturalistic driving data: The EuroFOT example. Saint Pierre, Guillaume, IFSTTAR, France

Tattegrain, Hélène, IFSTTAR, France

Val, Clément, CEESAR, France

* guillaume.saintpierre@ifsttar.fr

STS N°48 TRA2014 Paris 14-17 avril 2014

Saint Pierre G., Tattegrain H., Val C.

Introduction

Increasing penetration of driving assistance systems

Needs to measure theirs impacts during real uses

Several projects launched recently (FP7 funded) Field operational tests (FOT)

FESTA methodology

Naturalistic driving data

Many challenges were adressed, and many lessons learned

Let’s come back to the french EuroFOT experience The first large scale FOT, ended in 2012

STS N°48 TRA2014 Paris 14-17 avril 2014 2

Saint Pierre G., Tattegrain H., Val C.

Saint Pierre G., Tattegrain H., Val C.

EuroFOT in France

VMC handled by CEESAR

• 35 drivers using their own car in the west of Paris, during 6 months

• Light instrumentation

5 identical cars replaced the subject ones three times

Full instrumentation (incl. Video)

545 000 km of data analysed

STS N°48 TRA2014 Paris 14-17 avril 2014 4

Low Level High Level

Vehicles used 35 drivers’ owned vehicles.

5 vehicles owned by CEESAR and loaned to participants

CTAG datalogger 2 Max 4 CAN Channels GPS GPRS data transfer

● 2 channels used ● ●

● 4 channels used ● (not used : manual transfer)

TRW AC20 radar (not part of standard vehicle equipment)

● ●

VideoLogger (custom made for CEESAR, H.264)

●

Cameras (B&W, SuperHAD Exview)

4

Mobileye AWS (added, with special firmware)

●

Smarteye Eyetracker ●

Saint Pierre G., Tattegrain H., Val C.

Experimental design

Dream

Reality

STS N°48 TRA2014 Paris 14-17 avril 2014 5

Questionnaires were administred 4 times

Rotation of fully equiped vehicules among participants

Month 1

Month 2

Month 3

Month 4

Month 5

Month 6

Month 7

Month 8

Month 9

Month 10

Month 11

Month 12

Baseline Treatment Treatment

Screening, Time 1

Time 2 Time 3 (a) Time 3 (b)

Time 4, Debriefing

Saint Pierre G., Tattegrain H., Val C.

Lessons learned (1)

Recruitment needs car owners database acces to be efficient

GPRS data transfert problematic, consider UMTS instead,

Simplify experimental plan

NDS style

« instrument and forget »

STS N°48 TRA2014 Paris 14-17 avril 2014 6

Saint Pierre G., Tattegrain H., Val C.

Some issues for ND data

Data reduction • Reduce or aggregate continuous data to a significant level

Data modelling • Avoid comparisons between heterogeneous datasets

• Control for exposure

• Take into account the intrinsic correlation present in the data (repeated measures framework)

Deal with rare events • Post processing detection of “safety related events”

Results extrapolation • Transform events based analyses into casualties reduction

STS N°48 TRA2014 Paris 14-17 avril 2014 7

Saint Pierre G., Tattegrain H., Val C.

EuroFOT « solutions »

Data reduction Identify homogeneous sections of data Split sections in identical time epochs (10-30 sec.)

Data modelling Use suitable statistical models (GEE, GLMM, instead of ANOVA) Produce Odds ratios results

Deal with rare events Automatically detect candidates events (triggers, system use

etc...) Confirm identification by video + Annotation Extract corresponding baseline and do some stats...

Results extrapolation Speed & Accidents relationships

STS N°48 TRA2014 Paris 14-17 avril 2014 8

Saint Pierre G., Tattegrain H., Val C.

Key results Behavior, acceptance, usage

CC usage does not vary significantly over time.

SL usage does not vary significantly over time.

Drivers tend to use more one of the two systems.

CC usage favorable driving conditions

SL usage adverse conditions (ex. Night)

STS N° TRA2014 Paris 14-17 avril 2014 9

Saint Pierre G., Tattegrain H., Val C.

Safety: Events based analysis

STS N° TRA2014 Paris 14-17 avril 2014 10

Safety events are rare: Odds ratios can be interpreted as relative risk

SL associated with less frequent safety related events (SRE) CC associated with less SRE, except over-speeding

Saint Pierre G., Tattegrain H., Val C.

Lessons learned (2)

Baseline selection/definition for each RQ hypothesis is crucial Needs to control for external factors (traffic, visibility)

A data aggregation method is needed It has an impact on the analysis

Scaling up proove to be very difficult Various methods tried during FOTs None is perfect

Events based analysis (EBA) applicable to any system which impact is related to

the occurrence of this event

STS N°48 TRA2014 Paris 14-17 avril 2014 11

Saint Pierre G., Tattegrain H., Val C.

Conclusion & recommendations for future FOT

Why using NDS ? To get precise estimates of safety related events frequency

(with/without system)

Identify systems usage context

Identify systems misuses and potential countermeasures

Limitations Difficult to get a representative panel

Very hard to extrapolate to casualties reductions

Further works Identify important measures for road safety

Increase panel size and representativity

Define/quantify safety critical events

STS N°48 TRA2014 Paris 14-17 avril 2014 12

Saint Pierre G., Tattegrain H., Val C.

STS N°48 TRA2014 Paris 14-17 avril 2014 13

Thank you for your attention Guillaume SAINT PIERRE Guillaume.saintpierre@ifsttar.fr COSYS/LIVIC Components & systems department Interaction vehicles/drivers/infrastructure research unit