i quantification of human driving skill for human...

i

QUANTIFICATION OF HUMAN DRIVING SKILL FOR HUMAN ADAPTIVE

MECHATRONICS APPLICATIONS BY USING NEURAL NETWORK SYSTEM

MAZLEENDA BINTI MAZNI

UNIVERSITI TEKNOLOGI MALAYSIA

1

QUANTIFICATION OF HUMAN DRIVING SKILL FOR HUMAN ADAPTIVE

MECHATRONICS APPLICATIONS BY USING NEURAL NETWORK SYSTEM

MAZLEENDA BINTI MAZNI

A project report submitted in partial fulfilment

of the requirements for the award of the degree of

Master of Engineering (Electrical - Mechatronics & Automatic Control)

Faculty of Electrical Engineering

Universiti Teknologi Malaysia

JANUARY 2014

iii

To my beloved husband, mother, father and friends

iv

ACKNOWLEDGEMENTS

In the name of Allah, the most gracious and the most merciful.

All praises to Allah for the strengths and His blessing in completing this thesis,

Alhamdulillah. I would like to extend my deepest appreciation to my supervisor, Dr.

Mohamad Hafis Izran Bin Ishak for his excellent guidance, patience, motivation,

immense knowledge and providing me with an excellent atmosphere for doing these

project. His guidance helped me in all the time while doing a research and writing this

thesis. His support and guidance throughout the project is greatly appreciated.

Definitely, without his guidance, I would not be able complete the thesis successfully.

Besides my supervisor, my sincere thanks also go to Mr. Mustapha Muhammad, PhD

student for helping me going through into deepest research about Neural Network

System and leading me working on this project. Other than that, special thanks to my

friends Mr. Ab Wafi Ab Aziz, Miss Yeow Seuk Fei, Mr. Auwalu Muhammad, and Mr.

Mohd Fitri Allif Mohd Kasai for their supports, critics and ideas while completing this

project. Last but not least, I would like to thank my husband, Mr. Mohamad Azwan Abu

Bakar for his unconditional support, both emotionally and financially while doing this

project. Also, thanks to my parents for always supporting me and encouraging me with

their best wishes. Without them, I could not complete these thesis finished on a specific

time.

v

ABSTRACT

Human Adaptive Mechatronic (HAM) is a two way relationship between human

and machine. In HAM, not only human need to adapt the characteristics of machine but

the machine also has to learn on human characteristic. HAM is an enhance system for

Human Machine System (HMS) which consists only one way relationship between

human and machine. As a part of mechatronics system, HAM has an ability to adapt

with human skill improve the performance of machine. The example of application

where HAM can be applied is driving a car. One of the important elements in HAM is

the quantification of human skill. Thus, this project proposed a method to quantify the

driving skill by using Artificial Neural Network (ANN) system. Feedforward neural

network is used to create a multilayer neural network and five models of network were

designed and tested using MATLAB Simulink software. Then, the best model from five

models is chosen and compared with other method of quantification skill for

verification. All simulation data are taken from M. Hafis Izran’s in his PhD thesis

experiment. Based on results, the critical stage in designing the networks is to set the

number of neurons in the hidden layer that affects an accuracy of the outputs.

vi

ABSTRAK

Manusia Adaptive Mekatronik (HAM) adalah hubungan dua hala antara manusia

dan mesin. Dalam HAM, bukan sahaja manusia perlu untuk menyesuaikan diri dengan

ciri-ciri mesin tetapi mesin juga perlu belajar pada ciri-ciri manusia. HAM adalah sistem

meningkatkan Sistem Mesin Manusia (HMS) yang terdiri hubungan satu jalan di antara

manusia dan mesin. Sebagai sebahagian daripada sistem mekatronik, HAM mempunyai

keupayaan untuk menyesuaikan diri dengan kemahiran manusia meningkatkan prestasi

mesin. Contoh aplikasi dimana HAM boleh digunakan adalah memandu kereta. Salah

satu elemen penting dalam HAM adalah kuantifikasi kemahiran manusia. Oleh itu,

projek ini mencadangkan satu kaedah untuk mengukur kemahiran memandu manusia

dengan menggunakan system Rangkaian Neural Buatan (ANN). Rangkaian neural suap

depan digunakan untuk mewujudkan satu rangkaian neural multilayer dan lima model

rangkaian telah direka dan diuji dengan menggunakan perisian MATLAB Simulink.

Kemudian, satu model terbaik dipilih daripada lima model dan dibandingkan dengan

kaedah kuantifikasi yang lain untuk pengesahan. Semua data simulasi diambil daripada

M. Hafis Izran dalam eksperimen tesis PhD beliau. Berdasarkan keputusan, peringkat

kritikal dalam merekabentuk rangkaian adalah untuk menetapkan bilangan neuron pada

lapisan tersembunyi yang boleh menjejaskan ketepatan output.

vii

TABLE OF CONTENTS

CHAPTER TITLE PAGE

DECLARATION ii

DEDICATION iii

ACKNOWLEDGEMENTS iv

ABSTRACT v

ABSTRAK vi

TABLE OF CONTENTS vii

LIST OF TABLES x

LIST OF FIGURES xii

LIST OF ABBREVIATIONS xv

LIST OF SYMBOLS xvii

1 INTRODUCTION 1

1.1 The Relationship between Human and Machine 1

1.2 Problem Statement 1

1.3 Objectives 3

1.4 Scope of Work 3

1.5 Thesis Outline 4

2 LITERATURE REVIEW 5

2.1 Introduction 5

viii

2.2 Human Machine System (HMS) 5

2.3 Human Adaptive Mechatronics (HAM) 6

2.3.1 Definitions 6

2.3.2 Crucial Elements in HAM 9

2.3.3 Skills, Driving, Performance and 10

Experience in HAM

2.4 Human Index Skill 12

2.4.1 Sasaki’s Index Skill Formula 12

2.4.2 M. Hafis Izran’s Index Skill Formula 13

2.5 Intelligent Model 16

2.5.1 Neural Network Application for Decision 16

Making Problem

2.5.2 Decision Neural Network (DNN) For 20

Decision for Preference Assessment

2.6 Related Research in HAM 23

2.6.1 System Identification Analysis for Human 23

Respiratory System

2.6.2 Linear Polynomial and Hammerstein-Wiener 26

Structures of Human Dynamics

Identification to Analyze the Model Orders

2.7 Summary 29

3 RESEARCH METHODOLOGY 30

3.1 Introduction 30

3.2 Research Methodology 31

3.3 Summary 32

4 ARTIFICAL NEURAL NETWROKS (ANN) SYSTEM 33

DESIGN

4.1 Introduction 33

4.2 Collect and Prepare the Data 34

4.3 Create, Configure, and Initialize the Network 36

4.4 Train the Network 38

4.5 Validate the Network 41

ix

4.5.1 Network Performance 42

4.5.2 Network Regression 43

4.5.3 Formula Development of the New Synthetic Data 44

4.6 Classification of Index Skill 48

4.7 Summary 48

5 RESULTS AND DISCUSSIONS 49

5.1 Introduction 49

5.2 Simulation Results by Each Model 51

5.2.1 Network Performance of Each Model 52

5.2.2 Network Regression of Each Model 55

5.2.3 Compare the Simulation Results With M. 59

Hafis Izran’s Experimental Results within

5 Tracks; Straight Track, Circular Track,

Ellipse Track, Square Track, and Triangle

Track

5.2.4 Compare the Simulation Results with the 65

New Synthetic Data

5.3 Model Accuracy 67

5.4 Formula Validation and Comparison 69

5.5 Advantages and Limitation of Proposed ANN 71

Model

5.6 Summary 72

6 CONCLUSION AND RECOMMENDATIONS 73

6.1 Conclusion 73

6.2 Future Works 74

REFERENCES 75

x

LIST OF TABLES

TABLE NO. TITLE PAGE

2.1 Sasaki’s Index Skill Formula 13

2.2 Combination Of Normalized Error, En And Normalized

Time, Tn For Index Skill, J

14

2.3 M. Hafis’s Classification Formula For Human Index Skill 15

2.4 Candidate Supplier For Each Product 18

2.5 The Rating Scale of Criteria 18

2.6 Experimental Results of DNN Versus V 22

4.1 M. Hafis Izran’s Syntethic Data 44

4.2 M. Hafis Izran’s Synthetic Data With The Equivalent Value

of En, Tn And J

45

4.3 New Synthetic Data 46

4.4 New Synthetic Data With The Equivalent Values Of

En, Tn And J

47

4.5 Range For Each Level Of J 48

5.1 Comparison Of Simulation Result With M. Hafis Izran’s

Experimental Results For Straight Track

60

xi

5.2 Comparison of Simulation Result With M. Hafis Izran’s

Experimental Results For Circular Track

61


Experimental Results For Ellipse Track

62


Experimental Results For Square Track

63


Experimental Results For Triangle Track

64

5.6 Comparison of Simulation Result With The New

Synthetic Data

66

5.7 Percentage of Accuracy of J Within 5 Different Tracks 68

5.8 Percentage of Accuracy of J By Each Model Through

Synthetic Data

68

5.9 The Comparisons Using Synthetic Data 70

xii

LIST OF FIGURES

FIGURE NO. TITLE PAGE

2.1 Example of HAM concept of an electrical robot arm 7

2.2 Schematic diagram of HAM project 8

2.3 Comparison of block diagram between HMS (Upper Side)

and HAM system (Lower Side)

9

2.4 Graphic simulator design (Left Side) and the

experimental scene (Right Side)

12

2.5 En And Tn evaluation in normalized graph 13

2.6 Design of vehicle simulator (Left Side) and the

experimental setup (Right Side)

14

2.7 Artificial neural network structure 17

2.8 Hidden node mapping structure 17

2.9 The MSE versus iteration (Epoch) graph 20

2.10 The description of the desired output and the actual

network output

20

2.11 Decision Neural Network (DNN) block diagram 21

2.12 Curve variation of normal patient 24

xiii

2.13 Validation model of curve fitting 25

2.14 Curve variation of pressure 26

2.15 Trolley crane system free body diagram 27

2.16 Model order of function; SPE - sum prediction error,

SSE - sum squared error

28

3.1 Flowchart of overall methodology 31

4.1 Basic diagram of Neural Networks 34

4.2 Pre-processing and Post-processing processes of Neural

Network

35

4.3 Pre-processing and Post-processing processes of Neural

Network

40

4.4 Description of 5 neurons in the hidden layer 41

4.5 The example of the network performance 42

4.6 The example of the network performance 43

4.7 Value and range for synthetic formula developing 45

5.1 Model 1 49

5.2 Model 2 50

5.3 Model 3 50

5.4 Model 4 51

5.5 Model 5 51

5.6 Network performance of Model 1 52





5.11 Network regression of Model 1 56

xiv





xv

LIST OF ABBREVIATIONS

ANN - Artificial Neural Network

F - Fast

FLS - Fuzzy Logic System

HAM - Human Adaptive Mechatronic

HMS - Human Machine System

HS - High Skilled

L - Large

LS - Low Skilled

M - Medium

MD - Medium

MS - Medium Skilled

OOR - Out of Range

S - Small

SL - Slow

VF - Very Fast

VHS - Very Highly Skilled

VL - Very Large

xvi

VLS - Very Low Skilled

VS - Very Small

VSL - Very Slow

xvii

LIST OF SYMBOLS

E - Actual error obtained by human subject

En - Normalized Error

Es - Smallest possible error obtained by human subject

J - Skill index

Tn - Normalized Time

t - Time achieved by target

TB - The best theoretical time calculated using maximum speed

1

CHAPTER 1

INTRODUCTION

1.1 The Relationship between Human and Machine

In this new era of technology, human needs a machine to help them in

their daily lives. But most of the machine cannot operate independently without guided

and controlled by the human. Therefore, human factor is the main consideration in order

to design the motion control system (Harashima and Suzuki, 2006). Unfortunately, the

main limitation of HMS is the adaptation of machine as respect to human reaction which

the communication between human and machine is not in balance condition (Harashima

and Suzuki, 2006). Most of existing machine only required human to understand and

learn on machine characteristics which is refer to one way relationship (Harashima and

Suzuki, 2006).

1.2 Problem Statement

Based on the data source from World Health Organization (WHO 2011),

Malaysia has placed to the number of 20 in world’s rank of road traffic accident. This

ranking related with regards on the news provided by Utusan Online on 12th

October

2011, 5:25 pm, it said that 70 percent of the accident happened in Malaysia has

2

consumed by human error. Another 30 percent contributory factors consumed by road

condition and vehicle a defect which covers for 20 percent and 10 percent respectively.

From these data, it shows that human error is originates from the weaknesses of human

itself such as common mistakes by the driver, injudicious action, or inexperience driver.

Several common mistakes that normally happened like failing to observe correctly,

sudden braking, swinging over, fail to give a signal and loss of control. Injudicious

actions such as exceeding the limit of speed make an illegal turning direction,

disobeying double line, and disobeying traffic lights normally provided by several

human who did not follow the rules appropriately. Inexperience driver may affect the

accident on the road because they are not familiar with road and vehicle conditions,

reckless in driving, and careless with driving situation. These entire situations happened

in different condition, but it is still in the same type of subject which is the weaknesses

of human to make an error.

Based on statistics on accident’s report, Heinrich Domino which is pioneer in the

accident causation theories assumed that 88 percent of accidents happened due to

dangerous acts by human, 10 percent from unsafe conditions, and another 2 percent are

connected with natural disasters (Seyyed Shahab Hosseinian and Torghabeh, 2012).

Therefore, from this theory it shows that in any sudden condition, human cannot handle

it directly regards to its environment.

Generally, human needs time and effort to operate the machine. It is in the same

condition if human want to drive a car. Human should learn on how to operate the car

and to manage the car in different environments. Therefore, human needs a guidance and

training in order to increase their knowledge and understandings about driving a car.

These entire situation shows that, only human has the ability to learn the characteristics

of car, but the car does not play any role to improve the skill and performance of human.

In other hand, it shows that human still react as a main controller and the machine still

cannot identifies the changes of human features.

3

1.3 Objectives of Project

The objectives of this research in order to achieve the aims of project are:

i. To design an Artificial Neural Network (ANN) system for quantifying on human

driving skill.

ii. To obtain the best ANN system of human driving skill.

iii. To verify the results with previous experiment that consists of different method

of quantification.

1.4 Scope of Work

In HAM, there are a lot of issues to be discussed. Nevertheless, only a few

scopes will be discussed on this thesis as follows:

i. Covers only the main element in HAM which is the quantification of human

driving skills.

ii. Neural network system is used to quantify the driving skill.

iii. Simulation data is based on M.Hafis Izran’s experimental results.

iv. Simulation, analysis, and validation of the system are completed by using

MATLAB. The designed ANN system is verified through a comparison with

other methods.

v. The designed ANN system is verified through a comparison with other methods.

4

1.5 Thesis Layout

This thesis is organized as follows:

Chapter 2 introduces the literature review about Human Machine System (HMS)

and Human Adaptive Mechatronics (HAM) which are related with human and their

driving skills. The main research that related with HAM is presented.

Chapter 3 defines the methodology that used in order to complete the thesis and

its covers only for experimental works by using software applications. Detail

explanations of each stage are discussed.

Chapter 4 describes the process to design the Artificial Neural Network (ANN)

system for quantifying human driving skill in HAM. Following the system development,

data collection method, and network validation method are presented.

Chapter 5 explains the results and discussion of each ANN model. Following the

best model, model accuracy, advantages and limitation of the model are discussed.

Chapter 6 summarizes the thesis described in previous chapters. The conclusions

and future works are presented.

75

REFERENCES

ANDERSON, J. A. 1995. An Introduction to Neural Networks, MIT Press.

ANGUS, J. E. 1991. A Basic Introduction To Neural Networks; Criteria For Choosing

The Best Neural Network: Part I. [Online]. Available: Retrieved on July 8, 2013

from http://pages.cs.wisc.edu/~bolo/shipyard/neural/local.html.

ARAI, T. & FUJITA, Y. 1998. Auto Cruise System for Vehicle. Japan patent application.

23 Jun 1998.

BARON, S., KLEINMAN, D. L. & LEVISON, W. H. 1970. An optimal control model

of human response part II: Prediction of human performance in a complex task.

Automatica, 6, 371-383.

BAUCKHAGE, C., HANHEIDI, M., WREDE, S., KASTER, T., PFEIFFER, M. &

SAGERER, G. 2005. Vision Systems with the Human in the Loop. EURASIP

Journal on Applied Signal Processing 2005, 2005, 2375-2390.

BRINK, T. L. 1994. R.L. Keeney, H. Raiffa: Decisions with multiple objectives–

preferences and value tradeoffs, Cambridge University Press, Cambridge & New

York, 1993, 569 pages, ISBN 0-521-44185-4 (hardback), 0-521-43883-7

(paperback). Behavioral Science, 39, 169-170.

DUNCAN, J., WILLIAMS, P. & BROWN, I. 1991. Components of driving skill:

experience does not mean expertise. Jounal of Ergonomics, 34, 919-937.

ERTUGRUL, S. 2007. Predictive modeling of human operators using parametric and

neuro-fuzzy models by means of computer-based identification experiment.

Engineering Applications of Artificial Intelligence, 21, 259-268.

FURUTA, K. 2004. What is Human Adaptive Mechatronics? 8th International

Conference on Mechatronics Technology 2004. Hanoi, Vietnam, [CD Rom].

GOLMOHAMMADI, D. 2011. Neural network application for fuzzy multi-criteria

decision making problems. International Journal of Production Economics, 131,

490-504.

GOSSELIN, C. M. 2006. Adaptive Robotic Mechanical Systems: A Design Paradigm.

Journal of Mechanical Design, 128, 192-198.

http://pages.cs.wisc.edu/~bolo/shipyard/neural/local.html

76

HARASHIMA, F. & SUZUKI, S. 2006. Human Adaptive Mechatronics - Interaction

and Intelligence. 9th

International Workshop on Advanced Motion Control 2006.

Istanbul, Turkey.

IVANCIC, I. K. & HESKETH, B. 2000. Learning from errors in a driving simulation:

effects on driving skill and self-confidence. Ergonomics, 43, 1966-1984.

KAJARIA, A. 2013. Design of the Mathematical Model for Analysis of Human

Respiratory System. International Journal Of Engineering And Computer

Science ISSN: 2319-7242.

KULIC, D. & CROFT, E. 2007. Pre-collision Safety Strategies for Human-Robot

Interaction. Autonomous Robots, 22, 149-164.

KURIHARA, K., SUZUKI, S. & FURUTA, K. 2006. Elucidation of Skilled Human

Controller on Stabilization with Voluntary Motion. IEEE International

Conference on Control Applications 2006. Munich, Germany.

LEVISON, W. H., BARON, S. & KLEINMAN, D. L. 1969. A Model for Human

Controller Remnant. IEEE Transactions on Man Machine Systems, 10, 101-108.

LJUNG L, S. I. 1999. NJ: PRENTICE HALL.

M. HAFIS IZRAN, U. U. S., ZAHARUDDIN MOHAMED, A. B. AUJIH 2013. Skill

Index of Human Adaptive Mechatronic. pp. 110-116.

MCCALL, J. C. & TRIVEDI, M. M. 2007. Driver Behavior and Situation Aware Brake

Assistance for Intelligent Vehicles. Proceedings of the IEEE, 95, 374-387.

MILLER, R. L., JOSE, S. & HARPER, T. P. 1994. Anti-lock braking system. US patent

application. 13 Dec 1994.

SALVUCCI, D. D. 2006. Modeling Driver Behavior in a Cognitive Architecture.

Human Factors: The Journal of the Human Factors and Ergonomics Society, 48,

362-380.

SASAKI, T., TAKEYA, A., IGARASHI, H. & SUZUKI, S. Operation skill

quantification for mobile vehicle operation. SICE, 2007 Annual Conference, 17-

20 Sept. 2007 2007. 274-279.

SCHIELE, A. & VAN DER HEM, F. C. T. 2006. Kinematic Design to Improve

Ergonomics in Human Machine Interaction. IEEE Transactions on Neural

Systems and Rehabilitation Engineering, 14, 456-469.

77

SEYYED SHAHAB HOSSEINIAN & TORGHABEH, Z. J. 2012. Major Theories Of

Construction Accident Causation Models:A Literature Review. International

Journal of Advances in Engineering & Technology, Sept 2012, Vol. 4.

SUZUKI, S. 2010. Human Adaptive Mechatronics: Skill Acquisition in Machine

Manipulation. IEEE Industrial Electronics Magazine. IEEE.

SUZUKI, S., FURUTA, K. & HARASHIMA, F. Overview of Human Adaptive

Mechatronics and Assist-control to Enhance Human's Proficiency. International

Conference on Control, Automation and Systems, June 2-5, 2005 2005a

KINTEX, Gyeong Gi, Korea. 1759-1765.

SUZUKI, S. & HARASHIMA, F. 2005. Human Adaptive Mechatronics. 9th

International Conference on Intelligent Engineering System (INES) 2005.

Cruising on Mediterranean Sea.

SUZUKI, S., PAN, Y. & KURIHARA, K. 2004. Voluntary Operation Assistance Based

on Human Adaptive Mechatronics (HAM) concept. 8th

International Conference

on Mechatronics Technology 2004. Hanoi, Vietnam.

SUZUKI, S., SUZUKI, Y., BABA, Y. & FURUTA, K. 2005b. Human Skill and Brain

Activity in Machine Operation. 31st Annual Conference of IEEE Industrial

Electronics Society (IECON) 2005. North Carolina, USA.

TANAKA, Y. & IWATA, Y. 2005. Parking assist system. Japan patent application

10/226154. 24 May 2005.

TERVO, K. & MANNINEN, A. Analysis of model orders in human dynamics

identification using linear polynomial and Hammerstein-Wiener structures.

Networking, Sensing and Control (ICNSC), 2010 International Conference on,

10-12 April 2010 2010. 614-620.

WANG, J. & MALAKOOTI, B. 1992. A feedforward neural network for multiple

criteria decision making. Computers & Operations Research, 19, 151-167.

XU, Y., SONG, J., NECHYBA, M. C. & YAM, Y. 2002. Performance Evaluation and

Optimization of Human Control Strategy. Journal of Robotics and Autonomous

Systems, 39, 19-36.

i quantification of human driving skill for human...

Documents