EXAMINE DRIVER MENTAL WORKLOAD USING EXTREME LEARNING MACHINE.

Mentor: Prof. Yan YangReporter: Chengxi Li

BACKGROUND.ADVANCED DRIVER ASSISTANCE SYSTEM.

• Detectandclassifydriverdistractioninreal-time

• Adaptivein-vehiclesystemsHOWTOEVALUATE

ANDDESIGN?In-vehicle

InformationSystems(IVIS)

SafetyconcernDriverdistraction

Images: http://www.automotiveworld.com/news-releases/continental-to-showcase-digital-companion-technologies-during-ces-2013/

PublicationCX.Li and Y.Yang. Mental Workload of Young Drivers duringCurve Negotiation. IEEE International Conference on ConnectedVehicles & Expo (ICCVE 2014).

Index: IEEE Xplore®/ EI/ INSPEC/ ISTP/ ISI etc.

Previous work

BACKGROUND.ADVANCED DRIVER ASSISTANCE SYSTEM.

MOTIVATION.ADVANCED DRIVER ASSISTANCE SYSTEM.

Purpose: Advanced Driving Assistant System (ADAS)

Monitor in real-time

Fig. Driver Information Processing And Attention

VEHICLEDrive performance dataATTENTIONECG data

ATTENTION

TASK -> WORKLOADN-back [1]

Sound counting task [2]

[1] Owen, A.M., et al., N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies. Human brain mapping, 2005. 25(1): p. 46-59.[2] Healey, J.A. and R.W. Picard, Detecting stress during real-world driving tasks using physiological sensors. Intelligent Transportation Systems, IEEE Transactions on, 2005. 6(2): p. 156-166.[3] Huang, G.-B., Q.-Y. Zhu, and C.-K. Siew, Extreme learning machine: theory and applications. Neurocomputing, 2006. 70(1): p. 489-501 0925-2312.

REAL-TIME MONITORMachine learning – ELM [3]

Fig. Apparatus

• Experiment Design: 2*2*3

• Scenario: driving with secondary task- Primary task: simple highway (straight vs. curve )- Secondary task: N-back & Sound counting task (3 levels)

• Participants: 40 male drivers• Data sources: ECG & driving

performance

• Methods: Extreme Learning Machine (ELM)

Image: http://medcitynews.com/2012/05/continuous-monitoring-device-aims-for-increased-user-comfort/

Fig. Secondary Task Difficulty Fig. Secondary Task Accuracy

EXPERIMENT OVERVIEW.ADVANCED DRIVER ASSISTANCE SYSTEM.

SIMULATOR STUDY.EXTREME LEARNING MACHINE.

Extreme Learning Machine (ELM) [1]

•a training algorithm for single-hidden layer feed-forward neural networks (SLFNs)•the input weights and hidden layer bias are randomly set and need not to be tuned

Th1. If the number of hidden layer neurons was equal to training samples, SLFN can approximate the training samples for any w and b with zero training error .Th2. If the number of hidden layer neurons was less than training samples, SLFN can approximate the training samples with ε>0training error .

[1] Huang, G.-B., Q.-Y. Zhu, and C.-K. Siew, Extreme learning machine: theory and applications. Neurocomputing, 2006. 70(1): p. 489-501 0925-2312.

Fig. Extreme Learning Mach network

SIMULATOR STUDY.ESTABLISHMENT OF MODELS.Objective: To establish drivers' mental recognition model

•Featureselection•Outlierdetection

EstablishingFeatureSpace

•ELM•ELM-Kernel•SVM

EstablishingModels •Classification

accuracy•ComputationalCosts

EvaluatingPerformance

Driving performance : – Speed– Steering Wheel Angle– Accelerate– Brake

ECG measurement:– MeanIBI– SDNN– MeanHR– SDHR– RMSSD– VLF– LF– HF

Image: http://www.reddit.com/comments/28xmvphttp://research.vet.upenn.edu/smallanimalcardiology/ECGTutorial/tabid/4930/Default.aspx

SIMULATOR STUDY.FEATURE SELECTION.

Task EntropyBaseline 0.7084291-Back 0.8159842-Back 0.953102

Fig. Steering Wheel Angle Without Secondary Task

Fig. Steering Wheel Angle With Secondary Task

( ) ( 1) ( ( 1) ( 2))1/ 2(( ( 1) ( 2)) ( ( 2) ( 3)))p n n n n

n n n nθ θ θ θ

θ θ θ θ

= − + − − −

+ − − − − − − −

9log , ( 1,...,9)i iHp P P i= − =∑

FIG: Deviation Calculation Between Angle Of Steering Wheel Angle And Actual Value

1.

2.

TABLE. THE ENTROPY OF THE STEERING WHEEL ANGLE UNDER DIFFERENT SECONDARY TASKS

SIMULATOR STUDY.ENTROPY ANALYSIS.

Outliers detection: T test

1 2, ,......, nx x x maxx minx

( )m pT T n>

( ) ( )21p pnT n t nn

= −−

''

mm

x xT

s−

=

( ) ( )21p pnT n t nn

= −−

For and

If

Then,

and

SIMULATOR STUDY.OUTLIERS DETECTION.

Road type

Straight Curve

Task level

0 1 2 0 1 2

Label 1 2 3 4 5 6

Nodes Instance of

Training

Instance

of Test

Training

time

Testing

time

Training

accuracy

Testing

accuracy

ELM

Straight

18 101 53 0.0312 0.0312 0.8922 0.8654

40 101 53 0.0468 0.0312 0.9020 0.8077

60 101 53 0.0468 0.0312 0.9804 0.7500

100 101 53 0.0468 0.0312 1 0.3654

Curve

18 101 53 0.0312 0.0312 0.8356 0.7027

40 101 53 0.0312 0.0312 0.9452 0.5676

60 101 53 0.0936 0.0312 1 0.5405

70 101 53 0.0468 0.0312 1 0.4324

Fig. The Effect Of Nodes Numbers On The Accuracy On Straight Road

Fig. The Effect Of Nodes Numbers On The Accuracy On Curve Road

TABLE. DRIVING WORKLOAD CLASSIFICATION AND LABELS

TABLE. RESULTS OF ELM ALGORITHM ON STRAIGHT AND CURVE ROAD

SIMULATOR STUDY.ESTABLISHMENT AND RESULTS OF ELM.

Comparison between ELM and ELM-Kernel

ELM-Kernel：ELM：

TABLE. RESULTS OF ELM-KERNEL ALGORITHM ON STRAIGHT AND CURVE ROAD

SIMULATOR STUDY.COMPARISON OF MODELS.

Method Kerneltype

Trainingnumber

Testingnumber

TrainingTime

TestingTime

Training Accuracy

TestingAccuracy

Straight

ELM-Kernel

RBF_kernel 101 53 0.0053 0.0045 0.9109 0.8491

ELM-Kernel lin_kernel 101 53 0.0053 0.0047 0.9307 0.8679

SVM - 101 53 0.28 - - 0.75817

Curve

ELM-Kernel

RBF_kernel 101 53 0.0053 0.0046 0.9007 0.7906

ELM-Kernel lin_kernel 101 53 0.0049 0.0047 0.91 0.8021

SVM - 101 53 0.04 - - 0.66055

• SupportVectorMachines(SVM)

Advantages:• Solvingtheproblemoflinearinseparable

• Priorknowledgebeforetrainingisunnecessary

•Minimizethe upperbound on the expected generalization error

Disadvantages:Difficultyincalculation

• ExtremeLearningMachine(ELM)

Advantages:• Lowcomputationalcosts

• Real-timedetection

Disadvantages:Difficultyinparametersdetermination

SIMULATOR STUDY.COMPARISON OF MODELS.

Trend:•Internet of Things (IoT)•Connected Vehicle•Advanced driver assistance systems•Pilotless Automobile

APPLICATIONS.ADVANCED DRIVER ASSISTANCE SYSTEM.

Fig. Bosch Advanced In-Vehicle Information Systems

TIMELINE.ADVANCED DRIVER ASSISTANCE SYSTEM.

2014 Q1 Q2 Q3 Q4 2015

Essay01/05/15

Model establishment

Data processing

01/03/15

01/01/15

Simulator experiment

01/02/15

Research plan 15/11/14

01/12/14

Review 01/11/14

Model optimization

Combined ECG and driving performance data in feature selection

Collected and analyzing a large amount of physical data under complexenvironment

Introduced secondary tasks and NASA-TLX rating in study

Combined subjective rating and objective data for research

Established ELM and ELM-Kernel in classifying driving workload

Collaboration, Interdisciplinary, High-quality, Application

INNOVATIVE FEATURES.ADVANCED DRIVER ASSISTANCE SYSTEM.

THANK YOU