Predictive Neural Network Models For Autonomous Vehicle Driving onMultiple Terrains

Subhransu Mishra1, Gowtham Garimella1, Christoffer Heckman2, and Marin Kobilarov1

Abstract— This work considers the problem of predictingthe dynamics of a small unmanned ground vehicle (UGV)navigating on unknown terrains. Predicting the dynamics overlong horizons accurately helps avoid obstacles by makingaccurate control decisions early on. We propose a RecurrentNeural Network (RNN) to predict the position and orientationof the vehicle driving at 8 meters per second (18 mph) upto a time horizon of 2 seconds. Our method is different fromtraditional dynamic models of a car in the sense that we donot explicitly encode the terrain dependent parameters intothe model. In contrast, the RNN model can estimate both thetype of the terrain and the terrain parameters by processingthrough a few steps of sensor data such as GPS, IMU andwheel encoders. We tested the network on a 1/5th scale RCcar driving on three terrain surfaces namely grass concreteand sand which have vastly different dynamic behaviors. Ournetwork is able to make position predictions with an accuracyof 0.5 m and orientation of the car with an accuracy of 7 degreesin the next two seconds. In addition, the network is also ableto classify the terrain accurately 74% of the time (3 terrainsconsidered) where random chance would predict at 33%. Thedynamic model created in this work can be used in predictivecontrol schemes such as Model Predictive Control (MPC) or tosimply determine the safe driving conditions of a vehicle basedon the terrain.

I. INTRODUCTION

This paper considers the autonomous navigation of smallUnmanned Ground Vehicles (UGV) on natural terrains, moti-vated by applications such as package delivery, surveillance,search and rescue operations [1], [2], [3] in environmentsthat do not necessary have regular roads that are easilytraversable. For instance, consider a ground vehicle tran-sitioning between two different surfaces or driving aggres-sively on a terrain (such as in Fig. 1). In order to safelynavigate these terrains and reach a goal state, the vehicleneeds an accurate model that can predict it’s motion for along horizon. The terrain conditions often introduce suddenchanges in the tire-to-surface interaction that, if not modeled,can lead to excessive lateral accelerations, wheel slip andpossible loss of control. In such scenarios, it is necessary toaccurately detect the terrain properties online and predict themotion of the car over a sufficiently long future horizon thatensures obstacle avoidance or safe stopping. We propose aneural network model for identifying the dynamical modelof a UGV, which can represent the vehicle motion faithfully

1Subhransu Mishra, Gowtham Garimella, and Marin Kobilarov are withthe Department of Mechanical Engineering at Johns Hopkins University,Baltimore, MD, USA. smishra1|ggarime1|marin@jhu.edu

2Christoffer Heckman is with the Computer ScienceDepartment at University of Colorado, Boulder, USA.christoffer.heckman@colorado.edu

over a long horizon by automatically recognizing the drivingterrain online based on sensor data.

Fig. 1. A small (1/5 scale) ground vehicle driving on different terrains.The top figure shows the car transitioning between grass and brick surfaces.The bottom picture shows car sliding over a grass terrain with red arrowsindicating sideways slip.

A. Related Work

Earlier work concentrated on modeling the dynamics of afour wheeled robot using first principles models [4]. Usingthe first principles models, several control approaches havebeen developed that control the vehicle on flat terrainsunder different slip conditions [5], [6], [7]. These models,although sufficient for a flat terrain with fixed terrain pa-rameters, cannot capture the full nonlinear dynamics of acar on terrains with varying conditions. Simple first prin-ciples models have also been developed for performingcontrol on rough terrains [8] at low speeds. Initial controlframeworks on these terrains were restricted to low speedswhere the dynamics was predictable using the simplifiedmodels [9]. Aggressive driving on different terrains hasbeen made possible by combining improved robot dynamicsmodeling with better control approaches. Several authorsconsidered high fidelity physics simulators to predict thecar dynamics [10], [11]. The physics-based models are usedin either a Model Predictive Control (MPC) setting [12]or in a reinforcement learning approach where a control

policy is found using a transfer learning approach [13],[14]. These methods require an accurate representation of theterrain which can be constructed from LiDAR data as shownin [15], [16], [17]. Tuning the parameters for a high fidelitysimulation model is usually hard since it is dependent onthe specific terrain and the vehicle being used. Recent workdone by Aghli and Hackerman [18] have shown ability totune these parameters by choosing selected motion segmentswith maximum information regarding the parameters.

Unlike physics-based models, neural networks have alsobeen used to learn the dynamics of robots [19], [20]. Thesemethods combine traditional first principles models with neu-ral network models to precisely predict the dynamics of therobot. The neural network models have been combined withMPC techniques to control robots under uncertainty [21],[22]. The authors were not able to find any current literaturewhere neural network methods are employed to learn thedynamics of a mobile robot on multiple terrain surfaces.

Instead of using neural networks for learning dynamics,end to end control policies which take in sensor data andproduce a control output have also been developed foraggressive driving. Sallab et al. [23] showed that deepreinforcement learning can be applied to race car drivingin simulation. Yunpeng et al. [24] further showed that deepneural networks can be used to learn control policies fordriving aggressively at 8 m/s on a 1/5th scale RC car usingonly a monocular camera and wheel encoders. Convolutionalneural networks (CNN) have been used to predict costsfor MPC optimization using a vision sensor in [25]. Ourapproach, in contrast to above methods, learns an accuratemodel over multiple terrain surfaces. The learned model isthe first step in developing a model based control schemewhich can use a neural network control policy or an MPCpolicy.

Neural networks have also been used to classify the terraintype based on cameras [26], LiDAR [27], and a subset ofcamera, Lidar and vibrational sensors [28], [29], [30]. Theseapproaches try to avoid the terrains that are hard to driveon rather than learning the dynamics on those terrains. Inthis work, we try to predict the dynamics of the car ondifferent terrains and, therefore, allow for safe navigationof the vehicle on multiple terrains.

B. Contributions

We develop a general Recurrent Neural Network (RNN)architecture that is not specific to any particular vehiclemodel or terrain type and is expected to adapt to differentenvironments. The network is applied to predicting thedynamics of the ground vehicle shown in Figure 1 traversingtwo terrain types: grass and brick. The terrains are assumedto be relatively flat and thus the dynamics of the car can becaptured using planar (as opposed to 3D) relative pose dis-placements, gyro and acceleration data, encoder odometry ofeach wheel, and control inputs. We show that with sufficienttraining data across different velocities and maneuver typesit is possible to learn a model that can predict the motionequally well irrespective of the terrain surface type, e.g. the

Measurement Source Accuracy Frequency (Hz)Position GPS 0.2 m 5Forward Velocity Motor Encoder 0.1m/s 50Wheel velocities Wheel encoders 2◦ 50Body Z angularvelocity

MEMS-Gyro 0.1rad/s 100

Body X, Y angu-lar velocity

MEMS-Accelerometer

0.1m/ss 100

Steering angle Servo-motor 3◦ 50

TABLE ITHE TABLE SHOWS THE SENSOR MEASUREMENTS USED BY THE

NETWORK, THE SOURCE SENSORS, THE MEASUREMENT ACCURACY AND

THE ACQUISITION FREQUENCY

network can predict the position of the vehicle 2 secondsin the future traveling at 7 m/s with 0.6 cm accuracy onaverage. It is shown that training the network using data fromboth terrains types is critical for achieving such performance.In addition we show that RNN can be made to encode theterrain type explicitly using controls and sensor measurementand not rely on vision based sensors. We tested with 3different terrains types and obtained an accuracy of 75%when driving with high lateral acceleration or high forwardacceleration.

.

II. NEURAL NETWORK ARCHITECTURE

In this work we use a Recurrent Neural Network (RNN) topredict the behavior of the car. We are interested in predictingthe pose of the car g which is an element of SE (2). Thepose is fully defined by the position (x ∈ R, y ∈ R) of thecenter of the rear axle of the car as well as the orientation(θ ∈ S1) of the car. The network is also designed to performother auxiliary tasks which may help in predicting the poseg better. The auxiliary tasks performed by the network are topredict the terrain its driving on and predicting a set of sensormeasurements z ∈ Rnz , where nz is the sensor dimension.The terrain prediction is performed by learning a probabilitydistribution vector ψ on a set of pre-defined terrain labelsSψ and the label with maximum probability is chosen as thepredicted terrain label.

A. Network inputs and outputs

1) Outputs: RC car is equipped with a suite of sensorswith varying accuracies and data collection frequencies asshown in Tab I. The pose measurements g are obtained byconcatenating the position measurements (x, y) from GPSand the orientation θ of the car from yaw measurement ofthe IMU. The auxiliary sensor measurements z are chosenas front left and right wheel encoders as well as z angularvelocity obtained from IMU (z ∈ R3). These sensor mea-surements have been chosen by experimenting with differentcombinations of sensors and finding a set that results in goodprediction performance.

2) Inputs: The recurrent network takes as input the cur-rent sensor measurements zi and the current controls ui.The vehicle is controlled using an Electronic Speed Control(ESC) to control the longitudinal velocity of the vehicle

v ∈ R and a servo to control the steering angle φ ∈ Rof the vehicle. The control inputs to the vehicle are given bythe desired velocity and steering angle vd ∈ R, φd ∈ R i.eu = [vd, φd]

T . The car is controlled at a frequency of 50 Hzwhile the position measurements are received at a frequencyof 5 Hz. If we supply the network with only a single controlinput, the network loses predictive capacity. To avoid theloss in control information, we fit a second order polynomial(up(t) = c0 + c1t+ c2t

2) to the controls between (ti, ti+1)where t ∈ R represents the time and c0:n ∈ R2 correspondsthe polynomial coefficients. We feed the stacked coefficientsci = [cT0,i, c

T1,i, c

T2,i]

T to the neural network at each step asshown in Fig 2. Assuming the controls between the timesare given by u0, · · · , uNu (ignoring the subscript i for clarityin notation), the coefficients for the second order polynomialare obtained by performing a least squares fit as

minc0:2

Nu∑j=0

‖c0 + c1tj + c2t2j − uj‖22 (1)

Fig. 2. Neural Network Architecture for predicting the next car pose gi+1

given the current sensor measurements zi, control coefficients ci and thehidden state hi. The transform function φ applies a relative pose δg tocurrent pose g.

B. Predicting relative pose and change in sensor measure-ments

A naive implementation of the neural network involvespredicting the pose of the RC car at the next time stepgi+1 directly using the pose measurements, controls andother auxiliary sensor measurements at the current step(g, z, ui). This approach has two main issues. First, theposition coordinates x, y have a very large output scale.When the workspace is very large, a large network sizeis required to predict the position coordinates accurately.Further, the training data is required to cover the entireposition space uniformly which increases the size of thetraining data drastically.

The second issue has to deal with angle measurementswhich lie on S1. In our example, the orientation of the carwraps around 2π, and, thus, there are jumps in the predictionswhich are not easy to model. The common solution to thisproblem is to embed the angles in a higher dimension suchas quaternions [31]. This adds additional dimensions to thestate.

In this work, we use the invariance of the car dynam-ics [32] to mitigate the issues mentioned above. The invari-ance in the dynamics allows for us to predict the changein the pose δgi ∈ R3 and change in sensor measurementsδzi which combined with gi and zi predicts gi+1, zi+1 asexplained in [33], without explicitly predicting the pose andmeasurements.

The overall architecture of the neural network is shown inFig 2. The RNN part of the network is shown in dashed lines.The recurrent network takes in the sensor measurements zi,control polynomial coefficients ci and the hidden state hiand predicts the hidden state at next step hi+1. The hiddenstate represents information accumulated over time and isequivalent to the state of the system in a traditional dynamicmodel. The size of the hidden state has been experimentedwith and is chosen as 20 dimensional state (hi ∈ R20).

The hidden state is then passed through a three fullyconnected layers with 20 nodes each to obtain the changein pose δgi and the change in sensor measurements δzi asδxi = [δgTi , δz

Ti ]T . The change in pose is then then applied

to the measured pose to obtain the predicted pose at nextstep as

gi+1 = φ(gi, δgi) = gi exp(δgi), (2)

where the exponential function maps a vector from R3

to SE(2). The predicted sensor measurements live on aEuclidean manifold (z ∈ R3) and are obtained by simplyadding the change in sensor measurements to the previoussensor measurements i.e zi+1 = zi + δzi.

C. Terrain Classification

The neural network is also trained to predict the terrainon which the vehicle is driving based on the hidden statefrom the RNN. By training on this auxiliary task, we forcethe hidden state to learn the representation of the terrain aspart of the hidden state. The hidden state hi+1 is processedthrough a couple of separate hidden layers and finally a soft-max layer to get the normalized probability vector ψi+1. Themaximum element in the vector corresponds to the predictedterrain label.

III. TRAINING

The network is trained on a series of trajectories collectedby driving the car manually over different surfaces. We usedfour different surfaces in this work: grass, turf, concreteand sand. For each of the surfaces, we varied the desiredvelocities from 0 m/s to 10 m/s and the steering commandfrom -20 degrees to 20 degrees.

The network training is performed by minimizing a train-ing cost L that consists of minimizing the error betweenpredicted pose gi+1 and measured pose gi+1, the error be-tween predicted sensor measurements zi+1 , measured sensormeasurements zi+1 and a cross entropy loss minimizing theerror in the terrain probability distribution vector ψi+1. The

loss function is given by

L =

N∑i=1

(‖Qgi(gi+1 − gi+1)‖22 + ‖Qzi(zi+1 − zi+1)‖22

+ QCE CE(ψi+1, ψi+1)),

(3)where Q(·) is the weighting matrix that scales the importanceof each of the tasks, CE represents the cross entropy loss andψi+1represents a one-hot vector with the index correspondingto label getting a value of one. The weight Qgi is furtherscaled inversely by length of the relative motion δgi tonormalize the errors across different velocities. Since thepose of the car g lives in SE (2), the subtraction ”−” betweenthe poses gi+1, gi+1 in equation (3) refers to the operator thatfinds the distance between two elements in SE (2).

The cost function (3) is minimized using the truncatedback propagation algorithm where the car trajectories aretruncated into segments of 5 steps(1 second) and the gra-dients of the network weights are computed based on backpropagation up to the segment length. Although the gradientsare truncated at the segment length, the hidden state ispropagated across continuous segments to ensure the terraininformation and vehicle velocities and accelerations are notlost.

A. Testing

During the testing phase, the network predicts the pose ofthe car for the next 2 seconds based on the current hiddenstate h0 and controls control polynomial coefficients c1:N .Since we do not know the future pose and sensor measure-ments z1:N , g1:N , we replace the measured quantities withpredicted quantities i.e z1:N , g1:N in network architectureshown in Fig 2. Accurate prediction of car dynamics basedon predicted poses and sensor values requires an accurateinitialization of hidden state. The hidden state is initializedby running the network with past sensor measurements andpast controls for a fixed period of time to ensure the hiddenstate is converged. The initialization time is determined basedon average time observed in training for the predicted sensormeasurements to catch up with observed sensor measure-ments starting from an zero initialized hidden state.

IV. RESULTS

A. Self driving Platform

The 1/5th scale electric RC car has stock mechanicallinkages, drive train and servo motors. The radio receiver,servo motor controller and electronic speed control havebeen upgraded to talk to an ATmega-2560 based controllerboard using UART interface. The front two wheels areequipped with magnetic encoders . The steering servo anddrive motor are powered by a 4000 mAh 6 cells LiPobattery. The enclosure and mounting brackets are made withlaser-cut acrylic plates and 3D printed parts. The computersub-system includes an Intel quad-core i7 processor with8 GB of RAM, an ethernet switch and wifi router. It ispowered by a 4000mAh 6cell LiPo battery. The software for

Fig. 3. Assembled car

collecting data and running MPC is setup using the RobotOperating System(ROS) framework. ROS also facilitatescommunication between computer and low level controlleralong with time synchronization. The computer sends drivemotors rpm and steering wheel angle commands to the low-level controller which is then relayed to the ESC and theservo controller for closed-loop/set-point control. The lowlevel controller sends back sensors information. Localizationis performed by 2D LiDAR based localization algorithm orthrough a combination of IMU and RTK GPS. The neuralnetwork algorithm is setup using the TensorFlow python API.

B. Data

Data from two distinct surface types was collected bymanually driving the car at various speeds and angularvelocities. While driving the vehicle, care is taken to spanthe space of controls and sensor measurements as best aspossible .The car experiences a maximum lateral accelerationof 8m/sec2, forward acceleration of 4m/sec2, an angularvelocity of 1.2rads/sec and forward speeds up to 8 m/sec.The data is divided into training, validation and test set.The training set is used to optimize the neural networkparameters. Model selection and tuning is done using thevalidation set. The plots in the results sections is obtainedby running the learned models on the test set.

C. Discussion

The network was trained separately on grass, brick andcombined training data set and their performance was evalu-ated separately on the grass and brick data set. The results aretabulated in table IV-C. The columns of the tables show theroot mean squared(RMS) error of the relative pose predictionperformed over the same dataset. Lets consider the casewhere we train the network on brick and combined dataset,separately. The prediction with combined trained networkis better than the brick trained model. This indicates thatthe network learns the terrain parameters without explicitlyspecifying the terrain, which is evident from the discussion

in the next paragraph. The network is able to make accuratepredictions over multiple steps. This is presented in fig 6.Lastly the figure 4 plots the RMS error in angle, x andy position for trajectories sampled from the set with speedgreater than 7 m/sec. That gives us an RMS error of 0.12rad, 0.5 m in x and 0.2 m in y over 2 seconds.

In the Fig 5 we show that the hidden state of the RNNcan indeed encode the explicit terrain type just based on thecontrols inputs and sensor feedback. We collect driving datawith high lateral and forward acceleration on three terrains:Turf, Sand and Concrete to train the RNN with auxiliarytask. The rows show the true labels and the columns showthe predicted labels. For example, the first row shows thatout of 286 concrete sample trajectories, 211 samples werelabeled correctly.

Fig. 4. RMS error of trajectories with minimum speed of 7 m/sec over 2secs (10 steps)

V. CONCLUSIONS

We have shown that the proposed RNN architecture isable learn the dynamics of an RC car accurately withoutany prior knowledge of the terrain. The network is alsoable to classify the terrain only based on wheel encoderand GPS data without the aid of vision based sensors. Thehigh accuracy obtained using the RNN model allows fornavigating through different terrains safely. Combining theRNN architecture with an MPC architecture to autonomouslynavigate through multiple terrains is the subject of futurework.

REFERENCES

[1] S. Karma, E. Zorba, G. Pallis, G. Statheropoulos, I. Balta, K. Mikedi,J. Vamvakari, A. Pappa, M. Chalaris, G. Xanthopoulos, et al., “Useof unmanned vehicles in search and rescue operations in forest fires:Advantages and limitations observed in a field trial,” Internationaljournal of disaster risk reduction, vol. 13, pp. 307–312, 2015.

[2] G. De Cubber, D. Doroftei, D. Serrano, K. Chintamani, R. Sabino,and S. Ourevitch, “The eu-icarus project: developing assistive robotictools for search and rescue operations,” in Safety, Security, and RescueRobotics (SSRR), 2013 IEEE international symposium on, pp. 1–4,IEEE, 2013.

Fig. 5. Confusion matrix for terrain classification over three differentterrains namely concrete, turf and sand.

[3] J. Carlson and R. R. Murphy, “How ugvs physically fail in the field,”IEEE Transactions on robotics, vol. 21, no. 3, pp. 423–437, 2005.

[4] R. Rajamani, Vehicle dynamics and control. Springer Science &Business Media, 2011.

[5] J. Funke, P. Theodosis, R. Hindiyeh, G. Stanek, K. Kritataki-rana, C. Gerdes, D. Langer, M. Hernandez, B. Muller-Bessler, andB. Huhnke, “Up to the limits: Autonomous audi tts,” in IntelligentVehicles Symposium (IV), 2012 IEEE, pp. 541–547, IEEE, 2012.

[6] R. Gonzalez, F. Rodriguez, J. L. Guzman, C. Pradalier, and R. Sieg-wart, “Control of off-road mobile robots using visual odometry andslip compensation,” Advanced Robotics, vol. 27, no. 11, pp. 893–906,2013.

[7] R. Lenain, B. Thuilot, C. Cariou, and P. Martinet, “Model predictivecontrol for vehicle guidance in presence of sliding: application to farmvehicles path tracking,” in Robotics and Automation, 2005. ICRA 2005.Proceedings of the 2005 IEEE International Conference on, pp. 885–890, IEEE, 2005.

[8] K. Iagnemma and S. Dubowsky, “Chapter 4 rough terrain control,”Mobile Robots in Rough Terrain, pp. 81–96, 2004.

[9] T. M. Howard, C. J. Green, and A. Kelly, “Receding horizon model-predictive control for mobile robot navigation of intricate paths,” inField and Service Robotics (A. Howard, K. Iagnemma, and A. Kelly,eds.), (Berlin, Heidelberg), pp. 69–78, Springer Berlin Heidelberg,2010.

[10] K. Iagnemma, F. Genot, and S. Dubowsky, “Rapid physics-basedrough-terrain rover planning with sensor and control uncertainty,” inRobotics and Automation, 1999. Proceedings. 1999 IEEE InternationalConference on, vol. 3, pp. 2286–2291, IEEE, 1999.

[11] A. S. Aquilio, J. C. Brooks, Y. Zhu, and G. S. Owen, “Real-time gpu-based simulation of dynamic terrain,” in International Symposium onVisual Computing, pp. 891–900, Springer, 2006.

[12] S. L. N. Keivan and G. Sibley, “A holistic framework for planning,real-time control and model learning for high-speed ground vehiclenavigation over rough 3d terrain,” in IROS, 2012.

[13] M. Cutler, T. J. Walsh, and J. P. How, “Real-world reinforcementlearning via multifidelity simulators,” IEEE Transactions on Robotics,vol. 31, no. 3, pp. 655–671, 2015.

[14] J. Michels, A. Saxena, and A. Y. Ng, “High speed obstacle avoidanceusing monocular vision and reinforcement learning,” in Proceedings ofthe 22nd international conference on Machine learning, pp. 593–600,ACM, 2005.

[15] K. Konolige, M. Agrawal, R. C. Bolles, C. Cowan, M. Fischler, and

Testing → Brick GrassTraining ↓ rmse(δgθ)(rad) rmse(δgx) (m) rmse(δgy) (m) rmse(δgθ)(rad) rmse(δgx) (m) rmse(δgy)(m)

Brick 0.026 0.038 0.034 0.129 0.073 0.088Grass 0.102 0.060 0.065 0.034 0.035 0.039

Brick+ Grass 0.065 0.051 0.047 0.051 0.039 0.053

TABLE IIRMS ERROR COMPARISON FOR HIGH SPEED DATASET USING RNN

Fig. 6. The top figures show the predicted and observed trajectories when driving on grass terrain using different network configurations. The bottomfigures show the RMS errors of the network configurations against longitudinal velocity. The network trained only on brick dataset does not generalizewell to grass dataset.

B. Gerkey, Outdoor Mapping and Navigation Using Stereo Vision,pp. 179–190. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008.

[16] J. Madsen, A. Seidl, and D. Negrut, “Compaction-based deformableterrain model as an interface for real-time vehicle dynamics simula-tions,” tech. rep., SAE Technical Paper, 2013.

[17] S. Putz, T. Wiemann, J. Sprickerhof, and J. Hertzberg, “3d navi-gation mesh generation for path planning in uneven terrain,” IFAC-PapersOnLine, vol. 49, no. 15, pp. 212–217, 2016.

[18] S. Aghli and C. Heckman, “Online system identification and calibra-tion of dynamic models for autonomous ground vehicles,” in Roboticsand Automation (ICRA), 2018 IEEE International Conference on,IEEE, 2018.

[19] G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou,“Aggressive driving with model predictive path integral control,” inRobotics and Automation (ICRA), 2016 IEEE International Conferenceon, pp. 1433–1440, IEEE, 2016.

[20] G. Garimella, J. Funke, C. Wang, and M. Kobilarov, “Neural networkmodeling for steering control of an autonomous vehicle,” in 2017IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS), pp. 2609–2615, Sept 2017.

[21] I. Lenz, R. A. Knepper, and A. Saxena, “Deepmpc: Learning deeplatent features for model predictive control.,” in Robotics: Science andSystems, 2015.

[22] S. Levine and P. Abbeel, “Learning neural network policies withguided policy search under unknown dynamics,” in Advances in NeuralInformation Processing Systems, pp. 1071–1079, 2014.

[23] A. E. Sallab, M. Abdou, E. Perot, and S. Yogamani, “Deep rein-forcement learning framework for autonomous driving,” ElectronicImaging, vol. 2017, no. 19, pp. 70–76, 2017.

[24] K. S. K. L. X. Y. E. A. T. B. B. Yunpeng Pan, Chign-An Cheng,“Agile autonomous driving using end-to-end deep imitation learning,”in Robotics Science and Systems, vol. 14, 2018.

[25] P. Drews, G. Williams, B. Goldfain, E. A. Theodorou, and J. M.Rehg, “Aggressive deep driving: Combining convolutional neuralnetworks and model predictive control,” in Proceedings of the 1stAnnual Conference on Robot Learning (S. Levine, V. Vanhoucke,and K. Goldberg, eds.), vol. 78 of Proceedings of Machine LearningResearch, pp. 133–142, PMLR, 13–15 Nov 2017.

[26] Y. N. Khan, P. Komma, and A. Zell, “High resolution visual terrainclassification for outdoor robots,” in 2011 IEEE International Confer-ence on Computer Vision Workshops (ICCV Workshops), pp. 1014–1021, Nov 2011.

[27] J.-F. Lalonde, N. Vandapel, D. F. Huber, and M. Hebert, “Naturalterrain classification using three-dimensional ladar data for groundrobot mobility,” Journal of field robotics, vol. 23, no. 10, pp. 839–861, 2006.

[28] F. Schilling, X. Chen, J. Folkesson, and P. Jensfelt, “Geometric andvisual terrain classification for autonomous mobile navigation,” inIntelligent Robots and Systems (IROS), 2017 IEEE/RSJ InternationalConference on, pp. 2678–2684, IEEE, 2017.

[29] K. Iagnemma and C. C. Ward, “Classification-based wheel slip de-tection and detector fusion for mobile robots on outdoor terrain,”Autonomous Robots, vol. 26, pp. 33–46, Jan 2009.

[30] C. A. Brooks and K. Iagnemma, “Self-supervised terrain classificationfor planetary surface exploration rovers,” Journal of Field Robotics,vol. 29, no. 3, pp. 445–468, 2012.

[31] A. Kendall, M. Grimes, and R. Cipolla, “Posenet: A convolutionalnetwork for real-time 6-dof camera relocalization,” in Computer Vision

(ICCV), 2015 IEEE International Conference on, pp. 2938–2946,IEEE, 2015.

[32] M. Kobilarov, Discrete geometric motion control of autonomousvehicles. University of Southern California, 2008.

[33] G. Garimella and M. Kobilarov, “Towards model-predictive controlfor aerial pick-and-place,” in Robotics and Automation (ICRA), 2015IEEE International Conference on, pp. 4692–4697, IEEE, 2015.