2011 04 me1100 adas and enabling technologies

1ME1100 – ADAS class | 54

Advanced Driver Assistance Systemscurrent state of the art of systems and enabling technologies

Teacher

• David Abbink, [email protected]

• BioMechanical Engineering, Delft University of Technology

Simulation


Contents

1. Introduction• What is an Advanced Driver Support System?• What driving tasks can benefit from ADAS?

2. Enabling technologies

3. Examples of ADAS

4. Case Study: ACC – automation or support?

5. Take Home Message

Contents


Introduction


Today: serious problems!

Road Congestion

Road accidents

• EU cost: €50 billion / year

• EU cost: €160 billion / year• 1.7 million injuries / year• 40,000 deaths / year


Cell

Phones

Main cause of accidents: Driver Inattention

In-vehicle

Infotainment

Accident Causation


• A trend can be observed towards integration of active and passive safety

• This lecture will focus on active safety systems

Sheet used with permission from Olaf Gietelink

How to improve safety?


ADAS = advanced driver assistance systems

Active safety systems that use technology (sensors, computers, and actuators) to support or automate a driving task

Goal: increase safety and comfort of the driver

What are ADAS?


Sorted by Rasmussen’s Skill-Rule-Knowledge based control• Strategical: Choosing route• Tactical: Choosing speed or distance• Operational: Moving steering wheel or gas pedal

Sorted by hazard and direction• Low Hazard: Speed Control Lane-keeping• Medium Hazard: Car Following Curve Negotiation• High Hazard: Emergency Braking Swerving

What driver tasks can benefit from ADAS?


The Past


Today

Traffic jams

Near future

Advanced DriverAssistance Systems

(ADAS)

Far future?

Autonomous driving

Intelligence

PATH RWS

Current & Future Trends


Goal: High safety and reliability requirements

1) Complexity of the system and its environment

• Environmental disturbances

• Sensor characteristics

• Unpredictable traffic scenarios

2) Human factors


Main challenges for ADAS


Challenge: Complexity

• Support for interaction with road• Location / route • Lane boundaries (straight roads, curves)• Speed

• Support for interaction with other road users• Cars / trucks• Motorcycles• Pedestrians / bikes / pets (urban areas)


　 1. 　 Support for prevention of collisions

with forward obstacles

　 1. 　 Support for prevention of collisions

with forward obstacles

　　 3. 　 Support for prevention of lane

departure

　　 3. 　 Support for prevention of lane

departure

　 2. 　 Support for prevention of over

shooting on curve

　 2. 　 Support for prevention of over

shooting on curve


Examples of Road Interaction


　 4. 　 Support for prevention of crossing

collision

　 4. 　 Support for prevention of crossing

collision

　　 5. 　 Support for prevention of right

turn collision

　　 5. 　 Support for prevention of right

turn collision

　　 6. 　 Support for prevention of

collision with pedestrians

　　 6. 　 Support for prevention of

collision with pedestrians

　　 7. 　 Provision for road surface

condition information for

maintaining headway etc.

　　 7. 　 Provision for road surface

condition information for

maintaining headway etc.

Examples of Traffic Interaction


Trend towards automation

• The ‘aviation comparison’ – make driving like flying• What are the differences?

• Main obstacles for full automation?• Standardization of infrastructure, road users

• Legislation

• Preference..?

• A short trip to Transylvania shows some of these practical issues…

PATH


Different road users and hazards

Practical Issues


Different drivers

Practical Issues


Different laws, different traffic situations

Practical Issues


Different customs…

Practical Issues


Different parking possibilities…

Practical Issues


Future…?


ADAS - enabling technologies & examples

- automation vs manual control


Control• Power steer, ABS• X-by-wire (steer, brake, gas)

Sensors• For road boundaries, road users, location, speed etc…• Important Properties:

• Range & Resolution• Signal Detection Properties

• Misses, False Alarms

• Robustness to• Different weather types

• Different road conditions

• Time delay

Enabling Technologies


Pedestrians• very different (children, adults, clothing)• unpredictable (static/dynamic) • difficult to distinguish from other static

objects • a hazard at very short TTC

Solutions• Combining Infra-red with Camera’s• Seems to work well at low speeds

Specifications (e.g., accuracy) are not advertised…

Sensor Issues example

27ME1100 – ADAS class | 54Sheet used with permission from Olaf Gietelink

Trend: sensory integration


Support a (sub) task

1. Calculate Critical Thresholds2. Generate Warning Signal

• Visual (continuous)• Auditory (binary)• Haptic (binary)

Issues• Setting the threshold

• Different driver needs• Cry-wolf effect

• Nuisance

Automate a (sub) task

1. Calculate Optimal Control2. Define System Authority

Boundaries3. Define how to communicate the

boundaries4. Automate a task within authority

boundaries

Issues• Over-reliance• Complacency• Handing back control • People want to drive

themselves

And then what do we do??


And then what do we do??


Low Criticality• Navigation, Parking• Speed Control

Medium Criticality• Car Following (+ stop-and-go traffic interaction) • Lane Keeping, Lane Changing

High criticality• Pre-crash warning• Collision Avoidance• Pedestrian Protection

Target Areas for ADAS


Support

Navigatione.g., TomTom

Parkingauditory warnings

Speed ControlISA (haptic gas pedal)

Automation

Navigation -

Parking 2004 Toyota 2007 Lexus

Speed ControlCruise Control

ADAS examples: low criticality


Support

Car Following2008 Nissan – Continuous

Haptic Feedback on Gas Pedal

Lane Keeping /ChangingAuditory warningsHaptic warnings (Volvo, Nissan)

Automation

Car FollowingACC

Lane Keeping /Changing-

ADAS examples: medium criticality


• Blind Spot Monitoring (Volvo)

• Merging Support (for trucks)

Auditory/Visual Warnings

Some research, some commercialized

ADAS examples: lane change support


Auditory/Visual WarningsRemains research so far.. (movie)Haptic Lane Keeping Systems

e.g., Nissan

Provides vibrations when crossing lane



ACC (movie)

e.g., Nissan, BMW, MercedesLimited to highway situations (small decelerations)


Will be further discussed in a case study…


Own Car

Static Object

• What is critical? • small Time-to-Collision: small separation at high relative velocity• TTC < 1.2 usually results in a crash

• Human response time• perception & decision = 0.2 – 2 s• motor control (moving foot from gas to brake) = 0.2 s

• Car Dynamics• approximate maximum deceleration = 10 m/s2

ADAS examples: high criticality


Support

Collision Avoidance Support - Pre-crash Warning

Automation

Collision Avoidance Automation


ADAS examples: high criticality


Main issue

Individually, most of these systems are shown to be beneficial, but…

..how to integrate all these systems, without annoying or confusing the driver?


Case study: ACC


ACC functionality• maintain a set cruising speed unless a slower vehicle is

detected ahead• then follow at a safe distance (time headway)

• vehicle equipped with distance sensor (radar/lidar/vision) + automatic throttle & braking

• Full Speed Range ACC: ACC + Stop & Go (“IRSA”)


Adaptive Cruise Control


Technical limitations of ACC• Maximum range of sensors ~150 meter• Direct line of sight necessary• Frequent false and missed detections• Time headway usually set around 1.5 s, which degrades traffic

flow

Human Factors• How to communicate technical limitations to driver?• How to switch control authority?


Benefits• Also works during driver inattention• Speeds up deceleration response

• Human time delays don’t apply

ACC - benefits & limitations


String Stability (Cut-in)

Approaching Traffic Jam

12n

12n


ACC – unsolved technical issues


Benefits• not solely dependent on noisy environment sensors• extra vehicle information enhances ACC functionality

(e.g. feedforward of target vehicle acceleration, which is difficult to measure using a radar)

(sn , vn , an)

vehicle-vehiclecommunication

(s2 , v2 , a2) (s1 , v1 , a1)Sheet used with permission from Olaf Gietelink

Vehicle-to-vehicle communication


• Increased sensor range and redundancy• Extra vehicle information: look ahead multiple vehicles• Smaller time headway possible with higher level of safety

wireless communication


Vehicle-to-vehicle communication?


TNO Real-world experiments: approaching traffic jam

12n

Conventional ACC Cooperative ACC


Vehicle-to-vehicle communication

>> More in class 4 by Mehdi Saffarian


Unresolved Issue: Automation and Driver

Driver authority For all driving situations

Now on market: only partial automation

• Technical Reliability

• Laws & Regulations

• Acceptation (driver, society)

Partial automation -> automation boundaries

Still driver has tasks• Detect system failures

• Be aware of automation

boundaries

• Give and take authority

AutomationAuthorityNow on market

AutomationAuthority

Now technically possible

System failure

Driver does not understand

Driver authority For all driving situations


Designing for humans – remember…

Humans can adapt to system and strange situations• Very well & very quickly• Both visual control and neuromuscular control

Good, but… adaptation might cause effort or annoyance:

human adaptation is not necessarily good!

Better: engineers should understand driver to help design and evaluate systems that are - more comfortable

- better accepted by driver- safer


Human-machine interface

When driver and automation share tasks…… there is need for human-machine interfaceGood human-machine interface will allow quicker market release of automation systems

Issue 1. Does human understand automation?• Automation boundaries & Detected system failures

Current human-machine interface • Communicate through visual or auditory warning signals

Issue 2. Does automation understand human?• Desired trajectories, safety boundaries, strengths & limitations

Current human-machine interface:• Driver can only switch on/off (binary)• Driver can only provide some set-points for automation

We think: Use Haptic Shared Control (forces, stiffness) based on driver modeling and identification

We think: Use Haptic Shared Control (forces, stiffness)based on driver modeling and identification


Automation

Manual Control

Issues• Inactivity• Loss of skills• Over-reliance• Handing back control in critical situations

Alternative

Haptic Shared Control

Human Errors!

Issues• Inattention• Lack of skills• Fatigue

Human Errors! 90% accidents caused by:

Benefits• Driver remains in full control but..

.. with an additional continuous information loop • Conveys safe field of driving (Gibson 1938)• Subtle and intuitive• Also present when momentarily visually distracted• Enables use of fast reflexes• Enables probing of the environmental criticalities

Unsolved issues: human factors


Shared Control Metaphor


2002-2005 Nissan Project: Design Force Feedback Gas Pedal & Evaluation using Neuromuscular Analysis

International collaboration with universities in USA, Canada and Japan

Jan 2008 Market launch by Nissan - in Japan and USA as Distance

Control Assist (in Nissan Infiniti)

Pedal Depression

Ped

al Forc

e

Continuous Force Feedback

Own car2006-current Nissan Projects: • Neuromuscular Analysis of Steering

Example: Haptic Gas Pedal


Modeling Human-automation interaction


Manual Control

Machine perfor-mance

Environment

Slow (>200 ms) visual feedback

Fffw

goal

Xdesired

SteeringAngle

Xc

NMS

SW

IM

Tasks:Lane Keeping

Curve NegotiationEvasive Maneuver


Automation


Environment


Fffw

goal

Xdesired

SteeringAngle

NMS

SW

IM

goal Xopt

Controller

S

AutomationSystem

Xc


‘Mixing Input’ Sharing Control


Environment


goal

goal XoptAutomationSystem


Haptic Guidance


Environment


Fffw

goal

Xdesired

SteeringAngle

NMS

SW

IM

goal

Fguidance

Controller

S

SupportSystem

Xc


Haptic Shared Control


Environment


Fffw

goal

Xdesired

goalXopt

SteeringAngle

Xc

Fffw

Controller

NMS

SW

IM

IM

S

AutomationSystem

Shared HapticController


T

Xsw0

Example of “No Guidance Forces”

Steering Wheel Normally passive


T

Xsw0

Example of “Normal Guidance”

Steering Wheel Can generate feedback forces

but: driver can relax, resist or

give way

Xopt


T

Xsw0

Delft Approach to Haptic Shared ControlAbbink & Mulder (2009) – Exploring the dimensions of haptic feedback support in manual control

Joint patent with Nissan (2008)

Steering Wheel Can generate feedback forces

Can modify impedancedynamically shift

authorityin changing criticality

Xopt


Single path vs Multiple Paths

Problem: How to support multiple paths?

• How to support lane changes? • Tsoi et al. (2010) IEEE SMC Conference

• How to support multiple evasive paths?• Della Penna et al. (2010) IEEE SMC Conference

• Ideally, human should make the choice• Creative solutions may be needed

• Liability


T

Xsw0

Design Concept: Reducing Stiffness

Idea Reduce stiffness - criticality will be felt when trying

to steer- easier to steer left or right

Xopt Xopt


T

Xsw0

Design Concept: Reducing Stiffness

Xopt

Stiffness Can become negative in extreme cases

- a choosing human is supported to avoid obstacle, and is then “caught” by the support - a stubborn human needs to increase own stiffness to avoid steering left or right


Final Design of the system

T [

Nm

]

[deg]

fright

θright

fleft

STW

θleft

• Algorithm makes a smooth transition between the three lines

• Shape adapts online, depending on available time to choose (Steering Time Window)

• Also takes into account the initial heading, velocity and position of the vehicle with respect to the object

• Details are described in paper

Spring 2010 filed this idea as patent


Design Philosophy for Automation

Haptic Shared Control is a unified approach

• Continuous sharing of control authority through forces• No more binary switches (on/off), but smooth shifting

• Driver is better aware of situation and system intent

• Drivers can always overrule the system

• Can be based on any automation system that generates ‘optimal steering inputs’ (visual controller)

• Allows driver to use fast reflexes and neuromuscular adaptation (low-level neuromuscular controller)


Additional Benefits

- Allows automation system to adapt to human controller• Visual & neuromuscular control strategy• Different drivers ; different states of a driver

To design a system with these benefits, we need: • Computational driver model

• Visual & neuromuscular control• Feedback and feed-forward behaviour• Allows prototyping through models

• Time-variant evaluation/identification techniques


Take Home Message


Increasing Technology Developments• More and better sensors• Vehicle-to-Vehicle and Vehicle-to-Road communication will

increase possibilities of support and automation

Automotive trends• Current focus mainly on comfort, and safety• Bias towards automation or binary warnings• Haptics & shared control is an interesting alternative

Issues• Traffic complexity & Standardization• Human factors: how to integrate all information and support

systems?

What will be the role of the human?• What tasks should the driver be doing?• What happens to the fun in driving?

Take Home Message


Questions?

2011 04 me1100 adas and enabling technologies

Documents