autonomous vehicles and validation - t&vs · validation tries to answer the question "are...
TRANSCRIPT
![Page 1: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/1.jpg)
© Ricardo plc 2019
Autonomous Vehicles
and Validation
![Page 2: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/2.jpg)
2June 2019© Ricardo plc 2019
Validation tries to answer the question
"Are you building the right thing?”
Barry Boehm, Software Engineering Economics, 1981
But…
for an autonomous vehicle what is the right thing?
In general driving isn’t defined - it’s taught.
The First Problem
![Page 3: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/3.jpg)
3June 2019© Ricardo plc 2019
We don’t know what we are trying to validate.
Therefore…
![Page 4: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/4.jpg)
4June 2019© Ricardo plc 2019
Driving is also taught within a context
The context is
350,000 years of human evolution
16+ years of dealing with the real world
AND
The real world is complex!
The Second Problem
![Page 5: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/5.jpg)
5June 2019© Ricardo plc 2019
Example – what is this?
By Beijing Traffic Management Bureau - PDF document GIF diagrams, Public Domain, https://commons.wikimedia.org/w/index.php?curid=34836136
![Page 6: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/6.jpg)
6June 2019© Ricardo plc 2019
Example – what about this?
By Beijing Traffic Management Bureau - PDF document GIF diagrams, Public Domain, https://commons.wikimedia.org/w/index.php?curid=34836136
![Page 7: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/7.jpg)
7June 2019© Ricardo plc 2019
The ODD is large…
Operational terrain
– Road surface, curvature, banking etc.
Environmental and weather
– Rain, wind, visibility, lighting, glare etc.
Infrastructure
– Signs, traffic lights, road markings, tool booths etc.
Rules of engagement
– Road rules and differences between countries/states/cities
In addition…
People (operator, other drivers, pedestrians)
Animals – kangaroos are a real problem…
Everything else!
Operational Design Domain (ODD)
Adapted from: Koopman & Fratrik, How Many
Operational Design Domains, Objects, and Events?
![Page 8: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/8.jpg)
8June 2019© Ricardo plc 2019
Some of everything else…
By William Warby from London, England - Traffic Light Tree, CC BY 2.0,
https://commons.wikimedia.org/w/index.php?curid=5224129
Smart Australia
https://www.adsoftheworld.com/media/outdoor/specsavers_bus_back_crash
![Page 9: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/9.jpg)
9June 2019© Ricardo plc 2019
Groups yet to (fully) deploy self driving cars…
Waymo (Google)
8 million real world miles The Verge July 2018
8 million miles/day simulation The Atlantic Aug 2017
16 billion miles simulated to date
Tesla
1 Jan 2019 estimated 1 billion miles & Lex Fridman Dec 2018
31 Dec 2019 estimated 2.3 billion miles ibid
GM/Cruise
Honda joined Cruise program at $2.5 billion for a 7% shareholding Reuters Oct 2018
How BIG is the Problem?
![Page 10: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/10.jpg)
10June 2019© Ricardo plc 2019
Going to the moon might be easier...
Image: SpaceX
![Page 11: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/11.jpg)
11June 2019© Ricardo plc 2019
Limiting the ODD
Geo-fencing
Excluding conditions (rain, snow, fog)
Works reasonably well – difficult to extend
Requiring the driver to be ready to take over
Mostly works at L2
Won’t work at L3 – L4: humans are bad at monotonous tasks
External human monitoring
Even more boring than being in the car…
Can only deal with one car at a time
Partial Solutions…
![Page 12: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/12.jpg)
12June 2019© Ricardo plc 2019
Define partial safety goals (specifications)
Stay in lane unless it is unambiguously safe to manoeuvre laterally
RSS - Responsibility-Sensitive Safety
1. Do not hit someone from behind.
2. Do not cut-in recklessly.
3. Right-of-way is given, not taken.
4. Be careful of areas with limited visibility
5. If you can avoid an accident without causing another one, you
must do it.
Top Down Solutions – Rules of Engagement
Adapted from: Nilsson, Safe Self-driving Cars: Challenges and
Some Solutions SSS18
Shalev-Shwartz Shammah, Shashua , On a Formal
Model of Safe and Scalable Self-driving Cars
![Page 13: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/13.jpg)
13June 2019© Ricardo plc 2019
Stay in lane unless it is unambiguously safe to manoeuvre laterally.
What do you mean by lane?
Stay in lane unless…
![Page 14: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/14.jpg)
14June 2019© Ricardo plc 2019
What is a lane?
https://imgur.com/a/hAeQI
https://electrek.co/2019/05/20/tesla-fly-guard-rail-funny-balancing-pack/
To see what the drivers sees there is a video at
https://electrek.co/2018/04/03/tesla-autopilot-crash-barrier-
markings-fatal-model-x-accident/
![Page 15: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/15.jpg)
15June 2019© Ricardo plc 2019
Verification is difficult because we can’t define some basic
concepts.
That’s just the start…
Major Issue
![Page 16: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/16.jpg)
16June 2019© Ricardo plc 2019
Why is this so hard?
Perception Planning Execution Actuation
Planning Checker
Execution Checker
Actuation Checker
ACTUATORSSENSORS
Cross check with
safety envelope
Cross check redundant
information
Cross check Against
Feedback
Heurist planning
Algorithmic Software
Algorithmic Software
How do we know what we perceive is real?
ML Systems
Adapted from https://www.slideshare.net/PhilipKoopman1/edge-cases-and-
autonomous-vehicle-safety-sss-2019
Heuristic planning
![Page 17: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/17.jpg)
17June 2019© Ricardo plc 2019
The vehicle shall always come to a halt at a stop sign.
Consider the partial safety goal..
![Page 18: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/18.jpg)
18June 2019© Ricardo plc 2019
Humans
– When they see something they usually know what it is
• but not always…
– But they don’t always see the something
Autonomous Vehicles
- “see” everything
- have no “understanding”
- they “pattern match”
- no pattern, no match
Perception is a BIG problem
Gu, Doian-Gavit & Garg, BadNets: Identifying Vulnerabilities in
the Machine Learning Model Supply Chain arXiv 2019
![Page 19: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/19.jpg)
19June 2019© Ricardo plc 2019
More might be better…
N-version hardware
Multiple processing units homogeneous/heterogenous
M of N voting
N-version software
Value is debateable…
There is some gain, but not as much as expected – human bias
Ensemble of neural nets
Different models
Different data sets
Not always much better e.g. 94% → 96%
Probably difficult to scale…
Can we engineer a solutions?
![Page 20: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/20.jpg)
20June 2019© Ricardo plc 2019
When humans are uncertain they “often” take more care…
Can we use uncertainty to increase safety?
Can we measure uncertainty?
– Intrinsic uncertainty in a classification
– Extrinsic uncertainty over time (e.g. uncertainty in classification)
Extrinsic example - Uber crash in Phoenix Arizona, object classified as
unknown object,
then as a vehicle,
finally as a bicycle
Uncertainty
PRELIMINARY REPORT
HIGHWAY HWY18MH010
![Page 21: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/21.jpg)
21June 2019© Ricardo plc 2019
Invariants
For example - a person has certain proportions
but not on stilts
Adversarial testing
altering images
changing the focus
changing contrast
adding noise
Repeat until?
The “until” part will always be partly arbitrary.
Other ideas…
![Page 22: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/22.jpg)
22June 2019© Ricardo plc 2019
“perceiving” the stop sign is the problem
stopping at the stop sign isn’t the problem
Posted by u/ken3 on reddit
![Page 23: Autonomous Vehicles and Validation - T&VS · Validation tries to answer the question "Are you building the right thing?” Barry Boehm, Software Engineering Economics, 1981 But…](https://reader033.vdocument.in/reader033/viewer/2022042404/5f1c36ee62ad8524780dd310/html5/thumbnails/23.jpg)
23June 2019© Ricardo plc 2019
A successful test is a test that fails.
Final though…