Download - A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi
![Page 1: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/1.jpg)
A Cloud-Assisted Design for Autonomous Driving
Swarun Kumar
Shyamnath Gollakota and Dina Katabi
![Page 2: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/2.jpg)
Lot of Interest in Autonomous Vehicles
DARPA Urban Challenge (Team MIT) Google’s Autonomous Car
“Expect them on the road by 2020” – General Motors
![Page 3: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/3.jpg)
Benefits
• Lower traffic congestion
• Higher fuel efficiency
• Improved human productivity
“Estimated to save $100B annually in US alone” – WPI’08
![Page 4: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/4.jpg)
Challenge 1: Safely Detecting Hidden Objects
• Sensors on a car can only see line of sight objects
Addressed in: “CarSpeak: A Content Centric Network for Autonomous Driving”, SIGCOMM 2012
Addressed in: “CarSpeak: A Content Centric Network for Autonomous Driving”, SIGCOMM 2012
![Page 5: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/5.jpg)
• Classifying different objects
Challenge 2: Perception
![Page 6: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/6.jpg)
• Classifying different objects• Track and predict movement of objects
Challenge 2: Perception
![Page 7: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/7.jpg)
• Classifying different objects• Track and predict movement of objects• Accurate Localization– For fine localization: locate fixed known features
Challenge 2: Perception
![Page 8: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/8.jpg)
• Need to perform complex machine learning algorithms that operate on huge data sets
• Need to store huge, detailed maps and images of the world – petabytes of data
“MIT’s DARPA Challenge car had a mini-datacenter inside. An extra AC was mounted on top to cool it!”
Perception requires huge storage, computation
![Page 9: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/9.jpg)
How can we overcome this challenge?
To the Cloud!
Cloud• Cars share sensor data with cloud
![Page 10: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/10.jpg)
How can we overcome these challenges?
To the Cloud!• Cars share sensor data with cloud
• Cloud performs complex processing to track obstacles and their movements.
• Advises cars on how to navigate the environment
![Page 11: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/11.jpg)
Benefits
• Cloud has more storage & processing power– Can quickly identify pedestrians, other cars, etc.– Leverage these to provide improved paths
• Cloud has access data beyond wireless range– Has aggregate view from all vehicles & infrastructure– Can report accidents, congestion, etc.
![Page 12: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/12.jpg)
• A cloud-assisted design for autonomous driving• Assists cars to avoid obstacles beyond a their
wireless range• Suggests vehicles more efficient paths avoiding
congestion and road blocks• Preliminary prototype implemented on real
autonomous car
Contributions
![Page 13: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/13.jpg)
1. How do we design an architecture where the cloud gets data from multiple cars?
2. How do cars send sensor data over limited-bandwidth link?
3. How do we deal with high packet loss inherent to these links?
![Page 14: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/14.jpg)
1. How do we design an architecture where the cloud gets data from multiple cars?
2. How do cars send sensor data over limited-bandwidth link?
3. How do we deal with high packet loss inherent to these links?
![Page 15: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/15.jpg)
Controller
Sensor 1
sensor data
PlannerModule
path
speed,steering,gear
Sensor 2
Sensor n
Primer on Autonomous VehiclesThree modules:
![Page 16: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/16.jpg)
Controller
Sensor 1
sensor data
PlannerModule
path
speed,steering,gear
Sensor 2
Sensor n
Primer on Autonomous VehiclesThree modules: • Sensors: Provide data from onboard sensors
![Page 17: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/17.jpg)
Controller
Sensor 1
sensor data
PlannerModule
path
speed,steering,gear
Sensor 2
Sensor n
Primer on Autonomous VehiclesThree modules: • Sensors: Provide data from onboard sensors• Planner: Compute safe path to destination
![Page 18: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/18.jpg)
Controller
Sensor 1
sensor data
PlannerModule
path
speed,steering,gear
Sensor 2
Sensor n
Primer on Autonomous VehiclesThree modules: • Sensors: Provide data from onboard sensors• Planner: Compute safe path to destination• Controller: Navigate car along path
![Page 19: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/19.jpg)
Autonomous Vehicle
Cloud
Sensor n
ControllerSensor 1
PlannerSensor 2
Cloud’s Planner
Cloud-Assisted Architecture
Sensor data Safe path
Problem: Cars collect huge amount of real-time data Car should only send important data
Problem: Cars collect huge amount of real-time data Car should only send important data
![Page 20: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/20.jpg)
Cloud-Assisted Architecture
Cloud
proposedpath
Have data along path?
Is path safe?
Request for missing data
yes
no
datarequest
Find safepath
alternatepath
yes
no
![Page 21: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/21.jpg)
1. How do we design an architecture where the cloud gets data from multiple cars?
2. How do cars send sensor data over limited-bandwidth link?
3. How do we deal with high packet loss inherent to these links?
![Page 22: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/22.jpg)
What is the data that sensors gather?
• Sensor data helps cars find obstacle free paths to destination
• Tell cars which parts of environment:– Are empty and safe to pass through– Are occupied and unsafe to pass through
z
X
![Page 23: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/23.jpg)
What is the data that sensors gather?
• Divide environment recursively into 8 cubes
0
0 0
00 0
00 0
00 0
00 0
00 0
00 0
00 0
00 0
…
![Page 24: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/24.jpg)
What is the data that sensors gather?
• Divide environment recursively into 8 cubes• Each cube has one bit: Empty (0) or Occupied (1)• If cube is empty
all cubes inside are empty
0
0 0
0 0
0 00
![Page 25: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/25.jpg)
What Info does Autonomous Car Need?
• Each cube has one bit: Empty (0) or Occupied (1)• If cube is empty
all cubes inside are empty• If cube is occupied
at least one cube inside is occupied
0 1
0 01
![Page 26: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/26.jpg)
Compactly Representing Data
• Level 1 has 8 bits where 0-empty, 1-occupied
1
0 0 0 1 0 0 0 0
![Page 27: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/27.jpg)
Compactly Representing Data
• Level 1 has 8 bits where 0-empty, 1-occupied• None of 0 nodes need to be expanded
1
0 0 0 1 0 0 0 0
![Page 28: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/28.jpg)
Compactly Representing Data
• Level 1 has 8 bits where 0-empty, 1-occupied• None of 0 nodes need to be expanded• Expand 1 node to see inside at more resolution
1
0 0 0 1 0 0 0 0
0 0 1 0 0 0 0 0
![Page 29: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/29.jpg)
Compactly Representing Data
• Level 1 has 8 bits where 0-empty, 1-occupied• None of 0 nodes need to be expanded• Expand 1 node to see inside at more resolution
1
0 0 0 1 0 0 0 0
0 0 1 0 0 0 0 0
0 0 1 0 0 0 0 1
Tree representation provides a compressed representation of sensor data
Tree representation provides a compressed representation of sensor data
![Page 30: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/30.jpg)
1. How do we design an architecture where the cloud gets data from multiple cars?
2. How do cars send sensor data over limited-bandwidth link?
3. How do we deal with high packet loss inherent to these links?
![Page 31: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/31.jpg)
• Tree divided into packets and sent
How do we deal with packet loss?
1
0 0 0 1 0 0 0 0
0 0 1 0 0 0 0 0
0 0 1 0 0 0 0 1
![Page 32: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/32.jpg)
• Tree divided into packets and sent• Links to the cloud may fail packet loss
How do we deal with packet loss?
1
0 0 0 1 0 0 0 0
0 0 1 0 0 0 0 0
0 0 1 0 0 0 0 1
![Page 33: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/33.jpg)
• Tree divided into packets and sent• Links to the cloud may fail packet loss
How do we deal with packet loss?
1
0 0 0 1 0 0
0 0
0 0 1 0 0 0 0 1
? ? ? ? ? ?
? ?
![Page 34: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/34.jpg)
• Tree divided into packets and sent• Links to the cloud may fail packet loss• Loss of single packet destroys tree structure
How do we deal with packet loss?
1
0 0 0 1 0 0 0 0
0 0 1 0 0 0 10
![Page 35: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/35.jpg)
Solution: Make packets self-containedEach packet is sub-tree derived from rootCan reconstruct independent of other packets
Loss of packet Loss of resolution as opposed to complete loss of
information
Loss of packet Loss of resolution as opposed to complete loss of
information
1
0 0 0 1 0 0 0 0
0 0 1 0 0 0 0 0
0 0 1 0 0 0 0 1
![Page 36: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/36.jpg)
Empirical Results
![Page 37: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/37.jpg)
Implementation
Implemented in Robot OS (ROS)
Integrated with MIT’s Path Planner from DARPA challenge
![Page 38: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/38.jpg)
TestbedInstrumented Yamaha car with laser sensorsRoomba Robots using Kinect sensors along roadCampus-like environment with pedestrians
![Page 39: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/39.jpg)
Experiment 1: Resilience to packet loss
0 0.2 0.4 0.6 0.8 10
500
1000
1500
2000
2500
3000
Rate
at w
hich
dat
a is
rece
ived
(s-1
)
Loss Rate
![Page 40: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/40.jpg)
Experiment 1: Resilience to packet loss
0 0.2 0.4 0.6 0.8 10
500
1000
1500
2000
2500
3000
Loss Rate
Uncompressed
Rate
at w
hich
dat
a is
rece
ived
(s-1
)
![Page 41: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/41.jpg)
Experiment 1: Resilience to packet loss
0 0.2 0.4 0.6 0.8 10
500
1000
1500
2000
2500
3000
Loss Rate
UncompressedTree Compression
Rate
at w
hich
dat
a is
rece
ived
(s-1
)
![Page 42: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/42.jpg)
Rate
at w
hich
dat
a is
rece
ived
(s-1
)Experiment 1: Resilience to packet loss
0 0.2 0.4 0.6 0.8 10
500
1000
1500
2000
2500
3000
Loss Rate
Our System
UncompressedTree Compression
Our system achieves graceful degradation of resolution upon packet loss
Our system achieves graceful degradation of resolution upon packet loss
4.5 x
![Page 43: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/43.jpg)
Experiment 2: Detect Pedestrians using Cloud
Pedestrians walk into road beyond detection range of car
Information sent to cloud through wireless links – with and without our system
We measure how fast both systems detect pedestrian
![Page 44: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/44.jpg)
Outdoor ResultsCD
F
Delay in detecting pedestrian (s)0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
00.10.20.30.40.50.60.70.80.9
1
![Page 45: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/45.jpg)
Outdoor ResultsCD
F
Delay in detecting pedestrian (s)0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
00.10.20.30.40.50.60.70.80.9
1
Today’s System
![Page 46: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/46.jpg)
Outdoor ResultsCD
F
Delay in detecting pedestrian (s)0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
00.10.20.30.40.50.60.70.80.9
1
Today’s SystemOur system
4.6x
Our system enables significantly lower delays in detecting objects beyond sensor range
Our system enables significantly lower delays in detecting objects beyond sensor range
![Page 47: A Cloud-Assisted Design for Autonomous Driving Swarun Kumar Shyamnath Gollakota and Dina Katabi](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d145503460f949e98c5/html5/thumbnails/47.jpg)
Conclusion
A cloud-assisted system for autonomous driving
Enables cars to compute safer & more efficient paths by sharing data with cloud
Several ways to leverage cloud beyond what we implemented: localization, accident alerts, congestion monitoring, etc.