Augmented Vehicular Reality: Enabling Extended Visionfor Future Vehicles

Hang Qiu, Fawad Ahmad, Ramesh Govindan, Marco Gruteser, Fan Bai, Gorkem Kar

AbstractAs autonomous cars do today, future vehicles will have richsensors to map and identify objects in the environment. Forexample, many autonomous cars today come with line-of-sight depth perception sensors like 3D cameras. These cam-eras are used for improving vehicular safety in autonomousdriving, but have fundamentally limited visibility due to oc-clusion, sensing range, extreme weather, and lighting con-ditions. To improve visibility, and therefore safety, not justfor autonomous vehicles but for other Advanced DrivingAssistance Systems (ADAS), we explore a capability calledAugmented Vehicular Reality (AVR). AVR broadens the vehi-cle’s visual horizon by enabling it to share visual informationwith other nearby vehicles, but requires careful techniques toalign coordinate frames of reference, and to detect dynamicobjects. Preliminary evaluations hint at the feasibility ofAVR and highlight research challenges in achieving AVR’spotential to improve autonomous vehicles and ADAS.

1. INTRODUCTIONAutonomous cars are becoming a reality, but have

to demonstrate reliability in the face of environmentaluncertainty. A human driver can achieve, on average, ~100million miles in between fatalities, and users will expectself-driving vehicles, and other advanced driving systems,to significantly outperform human reliability. Most of thesetechnologies use advanced sensors for depth perception, andthese sensors can be used to recognize objects and otherhazards in the environment that may cause accidents.

To minimize accidents, it will be necessary to overcome thelimitations of these sensors (§2). Most autonomous vehiclesor ADAS technologies are equipped with depth perceptionsensors as well as cameras for semantic level perception.Besides limited sensing range, they all require line-of-sightvisibility. However, achieving higher reliability will likely re-quire safe handling of situations where (a) a vehicle’s sensorscannot detect other traffic participants because line-of-sightavailability does not exist, or (b) their range is impaired byextreme weather conditions, lighting conditions, sensor fail-ures, etc.. In some of these cases, even human vision cannotrecognize hazards in time.

In this paper, we explore the feasibility of communicat-ing merging visual information between nearby cars. Thiswould augment vehicular visibility into hazards, and enableautonomous vehicles to avoid accidents, or ADAS technolo-gies to guide human users in making safe driving decisions.This capability, that we call Augmented Vehicular Reality

(AVR), aims to combine emerging vision technologies thatare being developed specifically for vehicles [21], togetherwith off-the-shelf communication technologies.

Our preliminary design of AVR (§3) explores the use ofstereo cameras, which can, with their depth perception, gen-erate instantaneous 3-D views of the surroundings in a co-ordinate frame relative to the camera. Before it can sharethese instantaneous views between cars, AVR must solvethree problems: how to find a common coordinate frame ofreference between two cars; how to resolve perspective differ-ences between the communicating cars; and how to minimizethe communication bandwidth and latency for transferring 3-D views. To this end, we have developed novel techniques tolocalize a vehicle using sparse 3-D feature maps of the staticenvironment crowd-sourced from other cars, to perform 3-Dperspective transformations of the views of the cars, and todetect moving objects (as to minimize the visual informationexchanged between vehicles).

In our preliminary evaluation, we demonstrate that thesetechnologies can actually help extend the vision of neighbor-ing vehicles (§4), and we quantify some of the bandwidthand latency costs of AVR. Much work remains in makingAVR practical, including optimizing its vision processingpipeline, aggressively reducing the bandwidth and latencyby leveraging techniques such as compression and motionprediction. Our design builds on prior work in localizationand scene matching (§6), but, to our knowledge, no one hasexplored this novel, and important, capability.

2. BACKGROUND, MOTIVATION ANDCHALLENGES

Sensing Capabilities in Future Cars. A crucial componentof an autonomous car is a 3D sensing capability that providesdepth perception of a car’s surroundings.

Google self-driving car [7] mainly relies on an expensiveLiDar system including a 64-beam Velodyne LiDar, in ad-dition to radar and camera sensors, which produce a high-definition (HD) map accurate to within 1 inch. This HDmap makes the car aware of its surroundings: i.e., where isthe curb, what is the height of the traffic light, etc.. Teslahas developed a semi-autonomous system, AutoPilot [10],whose sensors include a forward radar, a forward-lookingcamera, and 12 long-range ultrasonic sensors around the car.Uber self-driving taxis [12] have a similar LiDar system asGoogle’s, and is also equipped with 20 cameras.

Abstractly, regardless of the details, this collection of sen-sors, generates successive point clouds, each which represents

1

the instantaneous 3D view of the environment. A point inthe point cloud represents a voxel in the 3D space, which isassociated with a position in 3-D and may or may not have acolor attribute depending on the sensor used. For example,the 64-beam Velodyne LiDAR can collect point clouds at 10Hz with in total 2.2 million points per second in 360 degrees.In this paper, we focus on stereo cameras, which can collectpoint clouds at faster rates (60 Hz) with over 55 million pointsper second, but with a limited field of view (110 degrees).The Problem. Most of these sensors only provide line-of-sight perception (e.g., LiDar, Radar, (stereo) camera andInfrared sensors) and obstacles can often block a vehicle’ssensing range. Moreover, the effective sensing range of thesesensors is often limited by different weather conditions (e.g.,fog, rain, snow, etc.) or lighting conditions1. In this paper, weexplore the feasibility of a simple idea: extending the visualrange of vehicles through communication. We use the termAugmented Vehicular Reality (AVR) to denote the capabilitythat embodies this idea.

AVR can be helpful in many settings. For example, con-sider a very simple platoon of two cars, a leader, and a fol-lower. The leader can provide, to the follower, those objectsin its visual range that the follower is unlikely to be able tosee, either because the leader obstructs the follower’s view,or because the objects are beyond the follower’s range. Thehuman or autonomous driving system in the follower’s carcan use this information to make decisions and safe driving.Other examples include buses that occlude children crossingat a crosswalk, or trucks that occlude a left-turning vehicle’sview of approaching cars.

AVR can also extend a vehicular vision over larger spa-tial regions, or over time. Today, for example, navigationapps like Waze warn users of hazards like parked cars on theshoulder, police vehicles, or objects on the road. An AVR ca-pability across vehicles can potentially provide more accurateinformation (today’s navigation apps rely on crowdsourcing,and their accuracy is spotty), together with precise positionsof these hazards. Finally, AVR can also be used to augmentHD maps to include temporary features in the environmentlike lane closures or other construction activity.Challenges. To provide vehicles with AVR, there are severalfundamental challenges to address, some of which we discussin this paper, and others which we defer to future work §5.

First, for AVR, each vehicle needs to transform the viewreceived from the other vehicles into its own view. To be ableto do this very accurately, this perspective transformationneeds both the exact position of the sensor (LiDar, camera,etc.) and the orientation at a high resolution, which can posenew challenges to current localization systems.

Second, the high volume of data generated by the sensorscan easily overwhelm the capabilities of most existing or

1In a recent accident [14], the radar and camera systems of Tesla’sAutopilot might have failed to detect the white truck against a brightsky, which the car crashed into. An AVR like extended visioncapability could potentially have avoided that.

Figure 1—Single Vehicle View of Point Cloud

future wireless communication systems. Fortunately, suc-cessive point clouds contain significant redundancies. Forexample, static objects in the environment may, in most cases,not need to be communicated between vehicles, because theymay already be available in precomputed HD maps storedonboard. Realizing AVR in the short term hinges on ourability to be able to isolate dynamic objects of interest forsharing between vehicles.

Finally, AVR needs to have an extremely low end to endlatency in order to achieve real-time extended vision. Thus,it requires fast object extraction, low communication latency,and fast perspective transformation and merging processing.With the advent of specialized vision systems for vehicles[21], the latency of some of these steps is approaching therealm of feasibility, but low latency communication will re-main a challenge. Finally, AVR requires tight time synchro-nization between sender and receiver in order to correctlyposition views received from other vehicles over time.

3. AVR DESIGNAVR creates an extended 3D map, updated in real-time,

of a vehicle’s surroundings regardless of any line-of-sightocclusions or sensing range limitations. With the help of sev-eral state-of-the-art enabling technologies, AVR leverages acrowd-sourced sparse HD map to enable vehicles to positionthemselves relative to each other, by positioning themselvesrelative to the same static objects in the environment. It uti-lizes wireless communication to share vehicle views amongother vehicles so that each of them is aware of not only theexact position but also the surroundings of its neighbor. Tominimize the communication overhead, AVR shares onlymoving objects by carefully analyzing the motion of the ob-jects in the environment.

Since we are exploring the feasibility, our initial explo-ration of AVR uses an inexpensive (2 orders of magnitudecheaper than high-end LiDAR devices) off-the-shelf stereocamera, together with processing software that analyzes con-current frames from the two cameras to determine the pointcloud. Specifically, by analyzing the disparity between theleft and right camera, the software can determine the 3D co-ordinate of each voxel, relative to the camera’s coordinateframe of reference. Figure 1 shows the point cloud generatedby the stereo camera while cruising on our campus. In ad-dition to resolving the depth of the surroundings, AVR alsoincorporates a state-of-the-art object recognition framework

2

Figure 2—Feature Detection and 3D Matching

[22]. Trained on a vehicular scenario, the framework detectsinteresting objects, i.e., cars, pedestrians, etc., and localizesand draws a 2D bounding box on the frame. In summary,each vehicle can continuously generate the location, and thetype of the surrounding objects.

In theory, AVR can share information at several levels ofgranularity between vehicles: the entire point cloud at eachinstant, the point cloud representing some subset of objectsin the environment, the object detected in a two-dimensionalview, or the label (type) of object. These are in decreasingorder of communication complexity, and we evaluate theselater in §4.Localization using Sparse 3-D Feature Maps. To solvethe problem of localizing one vehicle with respect to another,AVR leverages prior work in stereo-vision based simulta-neous localization and mapping (SLAM, [19]). This workgenerates sparse 3-D features of the environment, where eachfeature is associated with a precise position. AVR uses thiscapability in the following way. Suppose car A drives througha street, and computes the 3-D features of the environment us-ing its stereo vision camera. These 3-D features contribute toa static map of the environment. Another car, B, if it has thisstatic map, can use this idea in reverse: it can determine 3-Dfeatures using its stereo camera, then position those featuresin car A’s coordinate frame of reference by matching the fea-tures, thereby enabling it to track its own camera’s position.A third car, C, which also shares A’s map, can position itselfalso in A’s coordinate frame of reference. Thus, B and C caneach position themselves in a consistent frame of reference,so that B can correctly overlay C’s shared view in its own.

In AVR, this idea works as follows. As a car traverses astreet, all stable features on the street, from the buildings, thetraffic signs, the sidewalks, etc., are recorded, together withtheir coordinates, as if the camera is doing a 3D scan of thestreet. A feature is considered stable only when its absoluteworld 3D coordinates remain at the same position (within anoise threshold) across a series of consecutive frames. Fig-ure 2 shows the features detected in an example frame. Eachgreen dot represents a stable feature. Therefore, featuresfrom moving objects, such as the passing car on the left in(Figure 2), would not be matched or recorded.

Each car can then crowd-source its collected map. Wehave left to future work the mechanisms for this crowd-sourcing. Even though it is relatively sparse, the 3-D map

Figure 3—Crowdsourcing static HD map

is not amenable to real-time crowd-sourcing. However, it isrelatively straightforward to stitch together crowd-sourcedsegments from two different cars, so that a consistent 3-Dmap can be built. Suppose car A traverses one segment ofstreet X; car B can traverse that same segment of X, thenupload a 3-D map of a traversal of a perpendicular street Yby placing Y’s 3-D map in the same coordinate frame as X’s.Figure 3 shows the static HD map of our campus and thelocalized camera positions as it travels.

3.1 Extending the VisionWith the help of a common world static HD map devel-

oped above, vehicles are able to precisely localize themselves(more precisely, their cameras), both in 3D position and ori-entation, with respect to other vehicles easily. However, ifcar A wants to share its point cloud (or objects in it) with carB, AVR needs to transform the objects or point cloud in onevehicle’s local view to another vehicle’s. Vehicles have verydifferent perspectives of the world depending on differentlocation and orientation of their sensors. Note that unlikethe features, the point cloud is generated in the local cameracoordinate frame without any knowledge of the world at all.In this section, we describe how AVR achieves this.

Consider two views from two different vehicles. AVRintroduces the common world coordinate frame to bridge thegap between the two perspectives. Specifically, the camerapose in the world domain is represented by a transformationmatrix, Tcw, as shown in Equation (1), which includes a 3 x3 rotation matrix and a 3 element translation vector.

Tcw =

RotX.x RotY.x RotZ.x Translation.xRotX.y RotY.y RotZ.y Translation.yRotX.z RotY.z RotZ.z Translation.z

0 0 0 1

(1)

The translation is equivalent to the camera coordinate in theworld domain, and the rotation matrix indicates the rotationof the camera coordinate frame against the world coordinateframe. Therefore, transforming a voxel in the camera (c)domain point cloud (V = [x, y, z, 1]T ) to a voxel in the world(w) domain (V ′ = [x′, y′, z′, 1]T ) follows Equation (2).

3

Figure 4—Finding the Homography of the Current Frame in thePrevious Frame using SURF Features

x′

y′

z′

1

= Tcw ∗

xyz1

(2)

Assuming camera A has a pose of Taw, and camera Bhas a pose of Tbw, similarly, transforming a voxel Va fromcamera A to Xb in camera B follows Equation (3).

Vb = Tbw−1 ∗ Taw ∗ Va (3)

3.2 Dynamic Objects Detection and IsolationTransferring and transforming the full point cloud of a

vehicle’s surroundings is infeasible given the constraints oftoday’s wireless technologies. In this section, we explorewhether it’s possible to isolate only dynamic objects in theenvironment. A simple way to do this is to analyze the motionof each voxel in successive frames. Unfortunately, it is a non-trivial task to match two voxels among consecutive frames.Prior point cloud matching techniques [13, 18] involve heavycomputation unsuitable for real-time applications.

AVR exploits the fact that its cameras capture video, anduses 2D feature matching to match 3-D voxels (in earliersteps of our computation, we compute the correspondencebetween pixels and voxels, details omitted, and in this step,we use this information).

AVR extracts SURF features for matching and finds thehomography transformation matrix between two neighbor-ing frames. Specifically, it tries to find the position of thecurrent frame in the last frame. The intuition behind this isthat vehicles usually move forward and the last frame oftencaptures more of the scene than the current frame (Figure 4).Similar to the perspective transformation matrix discussedabove (§3.1), the homography matrix H , except which isin 2D, can transform one pixel (P = [x, y]T ) to the samepixel (P ′ = [x′y′]T ) in the last frame with a different loca-tion. Such matching among pixels indicates the matchingamong the corresponding voxels in the point cloud. Therefore,thresholding the euclidean distance between matching voxelsfrom consecutive frames can isolate the dynamic points fromthe static ones. Note that before calculating the displacement,the matching voxels should be transformed into the samecoordinate frame, either the last frame or the current frame,following Equation (2). Since the two frames are immediateneighbors to each other, AVR can approximate and acceler-ate the computation by adding the translation vector directly

Figure 5—ZED Stereo Cam-era

Figure 6—ZED Mounted ontop of the Windshield withSmartphone Attached

while omitting the rotation matrix.One optimization we have experimented with is to exploit

object detectors [22] to narrow the space for moving objects(cars, pedestrians). Once we have identified the pixels as-sociated with these moving objects, we can identify if thecorresponding voxels are moving, thereby reducing the searchspace for moving objects. We evaluate how much bandwidththis optimization can save in the following section §4.

4. PRELIMINARY EVALUATIONExperimental Setup. We experiment and evaluate AVR ontwo cars, each equipped with one ZED [24] (Figure 5) stereocamera mounted on the top of the front windshield and asmartphone attached to it (Figure 6). The stereo camerarecords the video stream and computes the 3D point cloud aswell as depth information, while the mobile phone recordsGPS and all motion sensors, i.e., gyroscope, accelerometer,magnetometer, etc.. ZED can create the real-time point cloudat a framerate of 30fps on a Titan X GPU with a resolutionof 720P (1280 x 720), and up to 100 fps with VGA (640x 480). For object recognition, AVR uses YOLO [22], astate-of-the-art object detector. For the SLAM algorithm,AVR uses ORB-SLAM2 [19], currently the highest rankedopen source visual odometry algorithm on KITTI benchmark,which is a widely accepted vehicle vision benchmark. UsingORB-SLAM2, we have collected several traces on and aroundcampus with one car following the other to both establish themap as well as realizing the extended vision.Results: Extended Vision. Figure 7 shows the point cloudview 2 of the leader, where there is FedEx truck parked on theleft and a segment of sidewalk on the right. Figure 8 showsthe perspective of the follower where a white sedan is parkedon the left. Note that neither the FedEx truck or the extensionof the sidewalk can be fully observed due to the limitedsensing range of the ZED stereo camera. Figure 9 shows theextended view of the follower car, where the point cloud ofboth the FedEx and the sidewalk, obtained from the leader,can be perfectly merged into the follower’s perspective.Results: Bandwidth and Latency. Table 1 summarizes thebandwidth requirement of transferring different represen-tations of vehicle surroundings with various granularities.Specifically, we consider the following four representations.The most fine-grained form is the full point cloud where the2In a standard openGL 3D viewer, users can rotate and zoom thepoint cloud to view different perspective. The paper only shows onevehicle perspective of the point cloud.

4

Figure 7—Point Cloud of Leader Figure 8— Point Cloud of Follower Figure 9— Extended Point Cloud

VGA(640 x 480)

720P(1280 x 720)

1080P(1920 x 1080)

Full 4.91 MB 14.75 MB 33.18 MBDynamic 0.79 MB 2.36 MB 5.30 MB

Object 0.33 MB 0.98 MB 2.21 MBLabels 0.05 MB 0.05 MB 0.05 MB

Table 1—Point Cloud Data Size Per Frame.

10-1

100

101

Point Cloud Size (MB)

0

5

10

15

20

Avera

ge

La

tency (

se

co

nd

s)

0

10

20

30

40

50A

ve

rag

e T

hro

ug

hpu

t (M

bps)

Latency

Throughput

Figure 10—Throughput and Latency of Point Cloud Transmission

car can share everything it sees to others (Full). A lighterform is the dynamic voxels only (Dynamic). A similarly lightform is the point clouds belonging to the objects detectedby YOLO (Objects). The most coarse-grained form is the3D bounding box and the label of the object (Label). Thenumbers in the table are the average point cloud data sizeevaluated over campus cruising traces with multiple movingvehicles in the scene. Depending on different wireless chan-nel states, the system adapt to choose which form to transferamong vehicles.

To explore the transmission overhead in reality, we es-tablish a peer-to-peer (P2P) WiFi direct link to measure theperformance of point cloud transfer. Figure 10 shows themean and variance of the throughput and latency for differ-ent point cloud sizes. As expected, the throughput generallyincreases as the file size increases because it allows the TCPcongestion window to open up fully and utilize more band-width. The peak throughput reaches 60Mbps after transmit-ting over 50MB. What’s interesting is that as we decreasethe point cloud size in log scale, the latency drops only lin-early. Clearly, for anything but labels, the latency of V2Vcommunication is still inadequate, and we plan to explorethis dimension in the future.

5. REMAINING CHALLENGES AND FU-TURE WORK

We see several directions of future work. The first is to tryto leverage other techniques adopted in latency sensitive ap-

plications [2, 4] like gaming and interactive VR, such as deadreckoning or motion prediction. Leveraging the trajectoryof the vehicle, information from maps, and the constraintson pedestrians, we might be able to correctly predict motionover short time scales. Receivers can use these trajectoriesto update their local views, re-synchronizing occasionally.While compensating for latency, these kinds of techniquescan also reduce bandwidth significantly. In addition, we alsoplan to explore advanced voxel compression techniques to ad-dress the bandwidth bottleneck. Finally, we have developedour AVR prototype in a modular fashion, and plan to exploreusing other sensors such as LiDAR that can generate 3-Dpoint clouds (in theory, the AVR framework should translatedirectly to these sensors), and more advanced object detectionframeworks [3].

6. RELATED WORKSeveral prior work has explored position enhancement

using differential GPS [8], inertial sensors on a smartphone[1, 9], onboard vehicle sensors [15], and WiFi and cellularsignals [23]. Visual SLAM techniques have used monocularcameras [5], stereo camera [6], and lidar [11]. Kinect [20] canalso produce high-quality 3D scan of an indoor environmentusing infrared. By contrast, AVR does relative positioningof two vehicles in a common coordinate frame of referenceusing features computed by a visual SLAM technique. Whileautonomous driving is becoming a reality [7, 10], and ADASsystems such as lane maintenance and adaptive cruise controlare already available in many cars, we are unaware of effortsto collaboratively share visual objects across vehicles in aneffort to enhance these capabilities. Existing point cloudmatching approaches [13, 18] involve heavy computation ofthree-dimensional similarity, while AVR adopts a lightweightapproach exploiting pixel-voxel correspondence. Finally,AVR can leverage complementary approaches such as [17,16] for content-centric methods to retrieve sensor informationfrom nearby vehicles, or over cloud infrastructure.

7. CONCLUSIONIn this paper, we present the concept of augmented vehicu-

lar reality and the design and implementation of AVR system.AVR is not only beneficial to human drivers as a driving as-sistant system, but also, more importantly, enables extendedvision for autonomous driving cars to make better and saferdecisions. With extended vision, we have demonstrated thatAVR can effectively remove sensing range limitations andline-of-sight occlusions. Future work on AVR will focus on

5

improving the throughput of processing objects, addressingthe bandwidth constraints, and reducing latency.

REFERENCES[1] Cheng Bo et al. “SmartLoc: Push the Limit of the

Inertial Sensor Based Metropolitan Localization Us-ing Smartphone”. In: Proceedings of the 19th AnnualInternational Conference on Mobile Computing andNetworking. MobiCom ’13. 2013 (Cited on page 5).

[2] Kevin Boos, David Chu, and Eduardo Cuervo. “Flash-Back: Immersive Virtual Reality on Mobile Devicesvia Rendering Memoization”. In: Proceedings of the14th Annual International Conference on Mobile Sys-tems, Applications, and Services. MobiSys ’16 (Citedon page 5).

[3] Zhaowei Cai et al. “A Unified Multi-scale Deep Convo-lutional Neural Network for Fast Object Detection”. In:Computer Vision - ECCV 2016 - 14th European Con-ference, Amsterdam, The Netherlands, October 11-14,2016, Proceedings, Part IV. 2016, pp. 354–370 (Citedon page 5).

[4] Tiffany Yu-Han Chen et al. “Glimpse: Continuous,Real-Time Object Recognition on Mobile Devices”.In: 13th ACM Conference on Embedded NetworkedSensor Systems (SenSys). Seoul, South Korea, 2015(Cited on page 5).

[5] Andrew J. Davison et al. “MonoSLAM: Real-TimeSingle Camera SLAM”. In: IEEE Trans. Pattern Anal.Mach. Intell. 29.6 (June 2007) (Cited on page 5).

[6] J. Engel, J. Stueckler, and D. Cremers. “Large-ScaleDirect SLAM with Stereo Cameras”. In: 2015 (Citedon page 5).

[7] Google Self-Driving Car Project Monthly ReportSeptember 2016. https://static.googleusercontent.com/media / www. google . com / en / /selfdrivingcar / files /reports/report-0916.pdf (Cited on pages 1 and 5).

[8] Mahanth Gowda et al. “Tracking Drone Orientationwith Multiple GPS Receivers”. In: Proceedings of the22nd Annual International Conference on Mobile Com-puting and Networking. MobiCom ’16. 2016 (Cited onpage 5).

[9] Santanu Guha et al. “AutoWitness: Locating and Track-ing Stolen Property While Tolerating GPS and RadioOutages”. In: Proceedings of the 8th ACM Conferenceon Embedded Networked Sensor Systems. SenSys ’10.2010 (Cited on page 5).

[10] Here’s how Tesla’s Autopilot works. http : / / www .businessinsider. com / how - teslas - autopilot - works -2016-7 (Cited on pages 1 and 5).

[11] Wolfgang Hess et al. “Real-Time Loop Closure in 2DLIDAR SLAM”. In: 2016 IEEE International Con-ference on Robotics and Automation (ICRA). 2016,pp. 1271–1278 (Cited on page 5).

[12] How Uber’s First Self-Driving Car Works. http://www.businessinsider.com/how-ubers-driverless-cars-work-2016-9 (Cited on page 1).

[13] Jing Huang and Suya You. “Point cloud matchingbased on 3D self-similarity”. In: Computer Vision andPattern Recognition Workshops (CVPRW), 2012 IEEEComputer Society Conference on. IEEE. 2012, pp. 41–48 (Cited on pages 4 and 5).

[14] Inside the Self-Driving Tesla Fatal Accident. http://www.nytimes.com/interactive/2016/07/01/business/inside-tesla-accident.html (Cited on page 2).

[15] Yurong Jiang et al. “CARLOC: Precise Positioningof Automobiles”. In: Proceedings of the 13th ACMConference on Embedded Networked Sensor Systems.SenSys ’15. 2015 (Cited on page 5).

[16] Swarun Kumar, Shyamnath Gollakota, and DinaKatabi. “A Cloud-assisted Design for AutonomousDriving”. In: Proceedings of the First Edition of theMCC Workshop on Mobile Cloud Computing. MCC’12. 2012 (Cited on page 5).

[17] Swarun Kumar et al. “CarSpeak: A Content-centricNetwork for Autonomous Driving”. In: SIGCOMMComput. Commun. Rev. 42.4 (Aug. 2012) (Cited onpage 5).

[18] Mathieu Labbé and François Michaud. “Online globalloop closure detection for large-scale multi-sessiongraph-based slam”. In: 2014 IEEE/RSJ InternationalConference on Intelligent Robots and Systems. IEEE.2014, pp. 2661–2666 (Cited on pages 4 and 5).

[19] Raul MurArtal, J. M. M. Montiel, and Juan D. Tar-dos. “ORB-SLAM: a Versatile and Accurate Monocu-lar SLAM System”. In: IEEE Transactions on Robotics(2015) (Cited on pages 3 and 4).

[20] Richard A. Newcombe et al. “KinectFusion: Real-timeDense Surface Mapping and Tracking”. In: Proceed-ings of the 2011 10th IEEE International Symposiumon Mixed and Augmented Reality. ISMAR ’11. 2011(Cited on page 5).

[21] NVidia Drive PX 2. http://www.nvidia.com/object/drive-px.html (Cited on pages 1 and 2).

[22] Joseph Redmon et al. “You Only Look Once:Unified, Real-Time Object Detection”. In: CoRRabs/1506.02640 (2015). URL: http : / /arxiv.org/abs /1506.02640 (Cited on pages 3 and 4).

[23] Arvind Thiagarajan et al. “VTrack: Accurate, Energy-aware Road Traffic Delay Estimation Using MobilePhones”. In: Proceedings of the 7th ACM Conferenceon Embedded Networked Sensor Systems. SenSys ’09.2009 (Cited on page 5).

[24] ZED Stereo Camera. https : / /www.stereolabs .com/(Cited on page 4).

6