Terrain Adaptive Navigation for Mars RoversDaniel M. Helmick, Anelia Angelova, Matthew Livianu, and Larry H. Matthies

Jet Propulsion Laboratory4800 Oak Grove Dr.

Pasadena, CA 91106, USA818-354-3226

firstname.lastname@jpl.nasa.gov

Abstract—A navigation system for Mars rovers in very roughterrain has been designed, implemented, and tested on a re-search rover in Mars analog terrain. This navigation systemconsists of several technologies that are integrated to increasethe capabilities compared to current rover navigation algo-rithms. These technologies include: goodness maps and ter-rain triage, terrain classification, remote slip prediction, pathplanning, high-fidelity traversability analysis (HFTA), andslip-compensated path following. The focus of this paper isnot on the component technologies, but rather on the inte-gration of these components. Results from the onboard inte-gration of several of the key technologies described here areshown. Additionally, the results from independent demon-strations of several of these technologies are shown. Futurework will include the demonstration of the entire integratedsystem described here. 1 2

TABLE OF CONTENTS

1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 SYSTEM ARCHITECTURE . . . . . . . . . . . . . . . . . . . . . . 23 GOODNESS MAP/TERRAIN TRIAGE . . . . . . . . . . . . 34 TERRAIN CLASSIFICATION/SLIP PREDICTION . 45 PATH PLANNING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 HIGH-FIDELITY TRAVERSABILITY ANALYSIS . 57 SLIP-COMPENSATED PATH FOLLOWING . . . . . . . 78 RESULTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 910 FUTURE WORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 911 ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . 10

1. INTRODUCTIONIn this paper a navigation system designed for a Mars roverin rough terrain is described. The current navigation systemrunning on the Mars Exploration Rovers (MER) [6], [16], wasdesigned for relatively benign terrains and does not explicitlyaccount for terrain types or potential slip when evaluating orexecuting paths.

It is well recognized that many scientifically interesting siteson Mars are in very rough terrains with the potential for sig-nificant slippage. The “follow the water” strategy taken by

11-4244-0525-4/07/$20.00/ c©2007 IEEE2IEEEAC paper # 1668

NASA’s Mars Exploration Program inherently requires ac-cess to demanding terrains such as dry river channel sys-tems, putative shorelines, and gullies emanating from canyonwalls [5]. Therefore, it is important that the next generationof Mars rovers have the capability to autonomously navigatethrough these terrains, not only to increase the science returnefficiency, but also to enable access to previously inaccessi-ble science sites. The navigation system described here is de-signed to deal with these scenarios using more sophisticated(and thus more computationally expensive) terrain analysis;however, this system is also designed to converge to compu-tational complexity similar to that of currently deployed nav-igation systems when the terrain is benign. This navigationsystem consists of several technologies that have been devel-oped, integrated, and tested onboard research rovers in Marsanalog terrains (see Figure 1). These technologies include:goodness maps and terrain triage, terrain classification, re-mote slip prediction, path planning, high-fidelity traversabil-ity analysis (HFTA), and slip-compensated path following.

Figure 1. Rocky8 in the JPL Mars Yard

Section 2 discusses the system architecture as a whole, in-cluding design goals and operational assumptions. It explainshow the subsystems interact to create a navigation system.It also discusses the interfaces between the component tech-nologies.

Section 3 describes the generation of goodness maps fromNavcam imagery [6], [16]. Also described in this section isthe terrain triage algorithm. This is a technique to sub-divide

1

the terrain into three categories based on the goodness calcu-lation. The three categories are: definitely traversable, def-initely not traversable, and uncertain. This categorization isthen used to determine which parts of the future planned pathneed to be analyzed in more detail.

Section 4 discusses the remote slip prediction algorithm,which uses intensity and range data from stereo cameras topredict the slip of the rover on terrain at a distance. It im-plements learned, non-linear regression models that outputrover slip, using terrain geometry from stereo imagery as in-put. This rover slip is then used to augment the cost map(essentially “1 - goodness map”). This section also brieflydiscusses the role of a terrain classifier in slip prediction.

Path planning uses the D* algorithm [21] to determine an op-timal path to a goal through the cost map. This algorithm isbriefly discussed in Section 5. It is beyond the scope of thispaper to discuss this algorithm in detail, and the focus of thissection is how the path planner fits into the system.

Section 6 explains the high-fidelity traversability analysis(HFTA) algorithm, which uses a sophisticated kinematic anddynamic forward simulation [10] of the rover following apath. It is designed to calculate a more accurate and realis-tic cost to traverse that path. The simulation includes a de-tailed geometric and mass model of the rover, terrain gener-ated from onboard stereo imagery, a dynamic model of thewheel terrain interaction (with parameters based on the ter-rain classification outputs), and the same slip-compensatedpath following algorithm (see Section 7) that runs onboardthe rover. The results of the HFTA are then used to refine theplanned path of the rover through the uncertain region(s).

Once the final planned path is created, the slip-compensatedpath following algorithm (see Section 7) is invoked to en-able the rover to actually follow this path regardless of theslip [9], [7]. This algorithm compares visual odometry(VO) (a technique that measures rover motion including slip-page) [15] and vehicle kinematics (a technique that measuresrover motion minus slippage) to estimate the location and theslippage of the rover; it then compensates for this slippageand accurately follows the desired path to the goal.

Results from the integration and demonstration of several ofthe key technologies onboard a research rover (Rocky8) ina Mars analog terrain (see Figure 1) are shown. Addition-ally, the results from independent demonstrations of severalof these technologies are shown. Future work will include thedemonstration of the entire integrated system on a researchrover.

2. SYSTEM ARCHITECTUREFigure 2 is a block diagram showing the system architec-ture described in this paper. The different colors representfunctional groups of this architecture: red represents sensing,green represents mapping and terrain analysis, yellow repre-

sents path planning, and blue represents path following.

The top block is navigation camera (Navcam) imagery. TheNavcams on the research rovers [11], the MER rovers [13],and the MSL [14] rover are all stereo cameras mountedon pan/tilt masts. These cameras allow the rover to takepanoramic images from a high perspective, which decreasesobscurations, thus enabling terrain sensing at further dis-tances. Typical Navcam configurations are shown in Table 1.This configuration allows for stereo ranging at distances up to50-100 meters [13]; and with the pan/tilt capability, range in-formation spanning 360 degrees can be accurately registeredinto a single map. Alternatively, Pancams could be used inplace of the Navcams (see Table 1). Using the MSL Pan-cams at their maximum zoom (shortest focal length), errorsof less than 20 cm at 50 meters range should be possible [14].Other techniques such as wide-baseline stereo could increasethe range and the accuracy even further [17] [18].

The assumed operational scenario of this navigation archi-tecture is that a goal is designated within stereo range of therover. The maximum distance for goal designation is a func-tion of stereo range error at that distance, vehicle pose esti-mation error at that distance, and acceptable path error. Giventhe MSL Pancam range error (see Table 1) and the fact thatpose error is 1-2% of distance traveled [7], it is feasible thatgoals of up to 100 meters would be acceptable, but more con-servative goals of 20 to 50 meters away are more likely.

This goal designation could be achieved in several ways. Itcould be designated by a human operator from Navcam im-agery. It could be designated by a human operator from or-bital imagery. It could be one of many “global waypoints”all designated from orbital imagery to define extremely longtraverses (kilometers). In this case the constraint would be onthe spacing of the waypoints to be within the stereo range ofthe rover. It also could be autonomously designated onboardusing both orbital and local sensor data.

Once a goal is designated, a Navcam panorama is taken in thegeneral direction of the goal (if a Navcam panorama has notalready been taken for goal designation). Stereo is then doneon each pair of the Navcam images and registered into a mapusing the pan/tilt angles. Once the stereo point cloud data areregistered, a goodness map and a triage map are generatedusing planar statistics on the terrain geometry (as describedin Section 3). The goodness map is then augmented with slipprediction costs (Section 4), re-triaged, and then passed to thepath planner. The path planner plans an optimal path from thecurrent rover location to the designated goal (Section 5). Ifall of the planned path goes through “definitely traversable”terrain, then the path is passed to the slip-compensated pathfollower. If any part of the path goes through “uncertain” re-gions of the triage map then the HFTA is performed on thoseregions to obtain a more realistic cost of traversal and to re-fine the path in that region. The path is then passed to the pathfollower and and the rover follows the path until it reaches the

2

Terrain Characterization/

Slip Prediction

Slip-Compensated Path Follower with

Very Local Obstacle

Avoidance

High-Fidelity Kinematic/Dynamic

Traversability Analysis

Goodness Map/ Terrain Triage

D* Path Planner

Navcam Imagery

Path Found?

Goal Designation

no

yes

terrain parameters

( � , c, � )

goal (x,y)

path cost (C)path (x,y waypoints)

triage map

range and intensity data

slip augmented cost map

goodness map

triage map

range data

Re-triage

slip augmented cost map

triage map

sequence performed every 20 – 50 meters

sensing

Legend

mapping/terrain analysis

path planning

path following

primary data flow

secondary data flow

goal (x,y)

Terrain Characterization/

Slip Prediction

Slip-Compensated Path Follower with

Very Local Obstacle

Avoidance

High-Fidelity Kinematic/Dynamic

Traversability Analysis

Goodness Map/ Terrain Triage

D* Path Planner

Navcam Imagery

Path Found?

Path Found?

Goal Designation

no

yes

terrain parameters

( � , c, � )

goal (x,y)

path cost (C)path (x,y waypoints)

triage map

range and intensity data

slip augmented cost map

goodness map

triage map

range data

Re-triage

slip augmented cost map

triage map

sequence performed every 20 – 50 meters

sensing

Legend

mapping/terrain analysis

path planning

path following

primary data flow

secondary data flow

sensing

Legend

mapping/terrain analysis

path planning

path following

primary data flow

secondary data flow

goal (x,y)

Figure 2. System Architecture Block Diagram

Table 1. MER/MSL Navcam and Pancam Configurations (range error assumes 0.25 pixel stereo correlation accuracy)

MER/MSL Navcams MER Pancams MSL Pancamsbaseline (meters) 0.20 0.30 0.20camera resolution (pixels) 1024×1024 1024×1024 1200×1200camera field of view (FOV) 45.0◦ × 45.0◦ 16.0◦ × 16.0◦ 6.0◦ × 6.0◦ - 50.0◦ × 50.0◦

range error at 50 meters (meters) 2.5 0.60 0.20 (at max. zoom)

goal.

As mentioned in the introduction, one of the goals in the de-sign of this architecture was to create a rover navigation sys-tem that would be capable of navigating through rough, high-slip terrains, but which would converge to the computationalcomplexity of simpler algorithms in benign terrain. This isenabled using the terrain triage algorithm described in Sec-tion 3.

3. GOODNESS MAP/TERRAIN TRIAGEA goodness map is a regularly spaced grid representing thelocal region around the rover. The map is populated usingstereo data generated from Navcam imagery. In each cellof the map, a goodness value is calculated using the stereo

data that falls in and around that cell. A plane is then fit to arover sized patch of cells. The goodness calculation involvesmetrics such as pitch and roll of the plane, roughness of theterrain (standard deviation of the plane fit), and step heightswithin the patch [6].

Terrain triage is a simple concept that is fundamental to re-ducing the computational complexity of this navigation ar-chitecture in benign terrains. The idea is to categorize theterrain into three categories: “definitely traversable”, “defi-nitely not traversable”, and “uncertain”. This categorizationis done by simply thresholding each of the goodness valuesof the goodness map and thus binning each of the cells intoone of the three categories. This step is performed twice:once before the slip prediction and once after. The first timeis to determine whether or not slip prediction needs to be

3

performed for a particular cell in the map. If a cell is cate-gorized as “definitely not traversable” then there is no needto do slip prediction for that cell, because the slip predictionalgorithm can only increase the cost of the cell. The sec-ond time terrain triage is performed is to incorporate the newcosts from the slip prediction into the cost map. Now, whena path is planned through the cost map (as described in Sec-tion 5), if the planned path travels entirely through “definitelytraversable” terrain, then this path is deemed acceptable and itis passed to the slip-compensated path follower without anyfurther analysis. This will only happen in relatively benignterrain. If, however, the path passes through “uncertain” ter-rain, then the HFTA (see Section 6) is invoked on those sec-tions of the path.

4. TERRAIN CLASSIFICATION/SLIPPREDICTION

An independent assessment of the terrain traversability isdone in terms of rover slippage. Slip is a measure of the lackof progress or the lack of mobility of the rover on a certain ter-rain. It is defined as the difference between the commandedvelocity and the actual achieved velocity in each DOF of therover [7]. It is normalized by the commanded velocity [24]and for convenience will be expressed in percent.

Rover slippage has been recognized to be a significant lim-iting factor for the MER rovers while driving on steepslopes [4], [12]. Knowing the amount of slip beforehand andbeing able to detect areas of large slip will prevent the roverfrom getting stuck in dangerous terrain and will enable moreintelligent path planning. Slip prediction is needed in addi-tion to an obstacle detection mechanism because an area oflarge slip is a non-geometric type of obstacle and cannot bedetected with a standard obstacle avoidance algorithm suchas GESTALT [6].

Main Method and Architecture

While detecting rover slippage is relatively straightfor-ward [7], [19], the main challenge here is that the rover slipneeds to be known remotely, before the rover actually tra-verses a particular location, in order to enable safe avoidanceof areas of large slip. We have proposed an algorithm whichinfers the amount of slip on the upcoming terrain using vi-sual information and onboard sensors (e.g. a tilt sensor orthe IMU) and have shown successful slip prediction resultsfrom only these remote sensors [1], [2]. The problem is ap-proached by learning from previous examples. To tackle theproblem of slip prediction from a distance, we subdivide itinto first recognizing the soil type the rover is going to tra-verse and then predicting the amount of slip as a function ofterrain geometry, i.e. slopes [1], [2].

The main architecture of the slip prediction algorithm is givenin Figure 3. For clarity, we first describe the prediction partof the algorithm, assuming the terrain classifier and the slipmodels have already been learned. The slip prediction mod-

Figure 3. Slip Prediction Algorithm Framework

ule receives as input from the main module stereo pair im-agery and rover attitude with respect to gravity. A map ofthe environment is built using the stereo range data registeredwith the color and texture information from the input images.In particular, each cell of the map contains information aboutterrain elevation and points to an image patch which has ob-served this cell. To predict slip in a map cell, the terrain clas-sifier is applied to all the map cells in its neighborhood. A ma-jority voting among their responses is used as the final terrainclassification response at the desired rover location. Then, alocally linear fit in the cell’s neighborhood is performed to re-trieve the local slope under the potential rover footprint. Theslopes are decomposed into a longitudinal and a lateral slopewith respect to the potential orientation of the rover. The twoslope angles are used as inputs to a pre-learned nonlinear slipmodel for the particular terrain type determined by the ter-rain classification algorithm. The output of the module is thepredicted slip for a given orientation of the rover and a sliprelated cost at a given location3. In our implementation, weprovide the main module a mechanism to query slip at a de-sired map location.

During training, the rover collects appearance and geometryinformation about a particular location while it is observed bythe rover from a distance. The corresponding slip of the roveris also measured when this location is being traversed. Weuse visual odometry (VO) between two consecutive steps toestimate the actual rover velocity. The commanded velocityof the rover is computed by using the full vehicle kinemat-ics. The collected data pairs of visual information and slipmeasurements are given to the learning module which learnsa terrain type classifier and independent slip models for eachterrain type [1], [2]. More details of the two training compo-nents are given in the next two subsections. A rover positionestimation is computed within the slip prediction module byaccumulating the VO estimates. This is necessary to be ableto map the current rover location to a location previously ob-served by the rover from a distance. The slip prediction mod-

3The slip related cost is a crude estimate of rover mobility without re-gards to particular robot orientation and is intended to be used in a D* pathplanning algorithm. It selects the maximum slip within a range of rover ori-entations.

4

ule can also use a Kalman filter position estimate based onmerging multiple sensors, including VO, which can be pro-vided by the main module [8].

Terrain Classification

Our current approach to terrain classification is based on pro-cessing visual appearance information, namely texture andcolor. We apply the texton based algorithm proposed in [22]which uses both color and texture simultaneously to learn todiscriminate different terrain appearance patches. The algo-rithm proceeds as follows: initially the color R,G,B valuesin small pixel neighborhoods are collected and the most fre-quent features in the whole data are selected. Consequently,a histogram of the occurrence of any of the selected featureswithin a patch corresponding to a map cell is built and com-pared by using a Nearest Neighbor classifier to a databaseof training patches [22]. Intuitively, a patch from a bedrockclass will have a high frequency of pixels typical of previ-ously observed bedrock patches, but it might also contain asmall number of pixels which are typical of an unfamiliarto the system “rock” class which happens to be also sharedwith the soil and sand classes too (either of the terrain typespatches might have small rocks dispersed in them). That is,this representation allows for building more complex appear-ance models and taking correct decisions given the observedstatistics from the data.

Evaluation of the terrain classifier for this particular data do-main has been provided in [2]. The terrain classification re-sults are satisfactory and give initial successful slip predic-tion results [2]. The appearance-based terrain classifier canbe improved by adding more sensors, both visual, e.g. multi-spectral imagery, or mechanical, e.g. vehicle vibrations. Thisis the topic of our current work.

Apart from being an instrumental part to the slip predictionmodule, the terrain classification provides important informa-tion to the HFTA algorithm (see Section 6). After the terraintype has been recognized, a canonical set of soil parametersassociated with it are passed to the kinematic and dynamicsimulation of the rover on the part of the terrain which hasbeen deemed “uncertain”.

Learning the Slip Models

As each terrain type has a potentially different slip behav-ior [3], [24], we learn a slip model for each terrain indepen-dently. The slip models are built by learning a nonlinear ap-proximation function which maps terrain slopes to the mea-sured slip. The goal is to learn slip as a function of the terrainslopes: S = S(xlongit., xlateral). In our case we considerslip in the longitudinal direction only (parallel to the typicaldirection of travel), but the method can be trivially extendedto learning of lateral slip or slip in yaw [1]. We have applieda Receptive Field Regression technique [23], but a standardnonlinear regression technique, such as a Neural Network,can also be used.

Implementation Details

The software architecture of the algorithm is designed to pro-vide efficient slip prediction. Because terrain classificationfrom visual information is generally time consuming, the fo-cus has been on decreasing the amount of computation de-voted to image processing related to terrain classification. Inparticular, our main idea is of evaluating the terrain type permap cell, rather than evaluating the terrain type in the wholeimage. This design concept can give significant advantages.Some speed-up can be achieved, as parts of the image do notbelong to the map, e.g. the pixels above the horizon. Addi-tionally, the terrain classifier will not be invoked if slip pre-diction is not needed in a certain area, e.g. an area which themain module has already marked as populated with obstaclesor which is otherwise deemed uninteresting. Thirdly, a mapcell at a close range covers a large part of the image comparedto the ones at far ranges and can be processed selectively tospeed up the processing without hurting the overall perfor-mance. More specifically, the map cell structure we use savesonly its 3D location and pointers to images which have ob-served it (see Figure 4). When the terrain type needs to bepredicted in a particular map cell, a projection of the map cellto the image is done and an image patch corresponding to thiscell is retrieved.

Additionally, this paradigm allows for stereo imagery data tobe received asynchronously or intermittently. In other words,one rover step can use multiple images, for example: whentaking a panorama of the environment, if the rover is stalledand receives multiple identical images, if it does not receiveimagery at all, etc. In any of these cases, the map is up-dated with new information, if such is available, and when-ever a terrain classification is invoked, only the most recentterrain patch is used. The result of the terrain classification issaved with its corresponding confidence and might be com-bined with a potentially new evaluation if the confidence isinsufficient. This is in contrast to processing fully all of theincoming images, extracting visual features and saving themto the map cells.

5. PATH PLANNINGOnce the cost map is populated with information derived fromterrain geometry and the slip prediction algorithm, as de-scribed above, an optimal path can be planned through thismap from any start to any goal. We use a standard implemen-tation of the D* algorithm [21] to plan this path. It is beyondthe scope of this paper to go into the algorithmic details ofthis planner. In summary, it is a derivative of the well knownA* search algorithm, with the capability to do efficient, in-cremental replanning.

6. HIGH-FIDELITY TRAVERSABILITYANALYSIS

If any section of the path goes through “uncertain” regions ofthe triage map, then the HFTA algorithm is invoked onboardthe rover. HFTA is a full kinematic and dynamic forward

5

Figure 4. Schematic of the software design paradigm: eachmap cell keeps a pointer to an image which has observedit and the terrain classification is done only if needed. Forexample, the elevation map shown has been built from ninepanoramic stereo image pairs, but effectively the visual infor-mation from only three images will need to be processed tofully classify the terrain.

simulation of the rover following a path. It is designed to findthe lowest cost path through these “uncertain” regions.

The simulation infrastructure is provided by ROAMS [10].ROAMS is a kinematic/dynamic simulation for rovers inter-acting with terrain. A detailed geometric and mass modelis used to represent the rover in the simulation. This modelincludes all 15 degrees-of-freedom (DOFs) of the mobilitysystem of the actual rover used in the experiments, Rocky8(which is described in greater detail in Section 8). This in-cludes the 12 active DOFs (six steering and six drive) andthe 3 passive DOFs (rocker and two bogies). Each of thelinks connecting these DOFs has a mass and center of gravity(CG). Each of the active DOFs has a dynamic model repre-senting a motor, which can be commanded in the same wayas the actual motors on the rover.

The terrain is modeled geometrically using a mesh. ForHFTA, this mesh is generated from stereo data, so it repre-sents a realistic geometric model of the terrain around therover each time this algorithm is used. As with any stereodata from a single point of view, obscurations will occur thatcause “range shadowing.” This is a well-known effect andthe terrain model will simply linearly interpolate over theseshadows, essentially creating a ramp between the top of ob-ject creating the shadow and the terrain visible on the far sideof the object. This ramp is actually the worst case scenario ofthe obscured terrain so it is a conservative assumption.

Mechanical soil properties can also be associated with the ter-rain. The terrain is again gridded, with the capability of as-signing to each cell independent values for the cohesion, fric-tion angle, and density of the soil in that cell. This enablesthe ability to represent non-homogeneous terrain at arbitraryresolution. For the HFTA algorithm, the terrain classifier (de-scribed in Section 4) predicts the terrain type of each of the

cells. Then, a canonical set of soil properties (derived experi-mentally or analytically) is associated with each of the terrainclasses.

These mechanical soil properties are used in a dynamicwheel/soil interaction model used to determine the rover sink-age and slippage [20]. This model calculates and resolves the18 forces (3 at each of the 6 wheels) of the vehicle interactionwith the environment. This results in an accurate calculationof the net motion of the rover, including sinkage and slippage.

The combination of the stereo data generated terrain geome-try and the terrain classifier generated soil properties createsa realistic model of the rover traversing a realistic model ofthe terrain.

For the forward simulation, the same slip-compensated pathfollowing algorithm that is used to control the actual rover [7],is used to control the simulated rover. So a path is passed tothe simulation, the path-following algorithm follows the pathover the sensor generated terrain that models the slippage ofthe simulated rover over this terrain. While this is being sim-ulated, several metrics are being recorded that will enable theassessment of the traversability of that particular path.

The most important of these metrics is the energy requiredfor the rover to traverse the given path. The first half ofEquation 1 shows the energy calculation as the integrationof the product of wheel torque and wheel velocity. Becausethe wheel terrain interaction (and thus vehicle slip) is beingmodeled, and because the slip is being compensated for, thismetric penalizes for both high slip terrain (because the wheelsmust turn for a longer period of time to reach the goal) andfor rough terrain (because the wheel torques are higher onlocally steeper terrain, i.e. rocks, gullies, etc.). This metricalso accounts for both forward and backward slip, where for-ward slip actually decreases the cost of traversal because thewheels turn for a shorter period of time to reach the goal. Thepath cost is calculated using:

C = We(

6∑n=1

∫Tn ·ωndt)+Wp(

∫(‖ pdes−pact ‖)dt) (1)

where Tn and ωn are the torque and speed of wheel n, respec-tively; pdes and pact are the desired and actual rover positions,respectively; and We and Wp are weighting terms.

Another metric that is used is path error (the second half ofEquation 1). Because the slip-compensated path follower isbeing run in the simulation, path error is an indication of ter-rain that is more difficult to traverse, even when compensatingfor slip.

Other metrics that can be used are minimum ground clear-ance, minimum distance to extreme hazards, and maximumground interaction forces. Ground clearance is the distancefrom the ground to the underbelly of the rover. This is a po-

6

tential hazard because it can “high center” the rover whichcan be a very serious threat to mobility. This type of for-ward simulation is the only way to accurately estimate theminimum ground clearance of a path. Distance to extremehazards (such as large drop-offs, high-slip areas, wheel traps,etc.) is another metric to determine the risk of traversing apath and can be used to evaluate the relative cost of a path.Large ground interaction forces that can be caused by travers-ing over rough, stiff terrain (such as very rocky terrain) couldalso be used as a metric to increase the lifetime of the vehicleby selecting paths that place lower demands on the actuatorsand structure of the vehicle.

When the HFTA algorithm is invoked, the input is a singlepath through an “uncertain” region. The algorithm then ran-domly perturbs this single path to create multiple paths withthe same start and finish points. It then runs the forward sim-ulation of the rover through each of the paths, calculating asingle scalar value of cost for each of the paths. It then out-puts the path of the lowest cost.

7. SLIP-COMPENSATED PATH FOLLOWINGWhen the final path is created, the slip-compensated path fol-lowing algorithm is invoked to enable the rover to actuallyfollow this path regardless of the slip. This algorithm com-pares visual odometry (a technique that measures rover mo-tion including slippage) and vehicle kinematics (a techniquethat measures rover motion minus slippage) to estimate thelocation and the slippage of the rover; it then compensatesfor this slippage and accurately follows the desired path tothe goal.

This system is described in detail in [7], [9].

8. RESULTSSlip Prediction Results

We have tested the slip prediction module independently on aLAGR4 vehicle. The dataset is collected outdoors in a natu-ral park on five different terrains, including sand, soil, gravel,asphalt and woodchips. The slip models and the terrain clas-sifier have been trained on 3000 frames. The summary re-sults are presented in [2], and are the results of slip predic-tion over 2000 test frames, non-intersecting with the trainingdata, while the rover traverses any of the above mentionedterrains. The average slip prediction error achieved is about21%, which is a satisfactory result, given the amount of noisein the measured slip. Moreover, misclassification in the ter-rain type contributes to a large part of the error. In particu-lar, if the terrain type were correctly classified, the predictionerror decreases to about 11% on average. Terrain misclassi-fication errors mainly occur between visually similar terrainssuch as sand and soil, especially for the parts of the soil areaswhich were covered with dust. See [2] for detailed results.

4LAGR stands for Learning Applied to Ground Robots and is an experi-mental all-terrain vehicle program funded by DARPA.

The final slip prediction is performed on the whole forthcom-ing map and a slip related cost is calculated. The cost basedon the predicted slip is handed down to the terrain triage al-gorithm which further determines areas of the terrain whichneed more refined terrain traversability assessment and is fi-nally given to a path planner. Figure 5 shows the map gen-erated by the rover driving in deep sand and on upslope soilterrain. The corresponding terrain classification and slip pre-diction results are also shown for each map cell. As seen, ithas been predicted that driving on flat sand incurs a large costand the rover should prefer the neighboring woodchip terrain.Driving upslope on soil terrain causes about 40% slip whichis mapped to a slip cost in the mid-ranges and which willfurther invoke the HFTA algorithm to determine the safestpath. Also note that most terrain classification errors occurat ranges larger than 6m, where the image patch correspond-ing to a map cell is of very small size (for the LAGR roverconfiguration).

Integrated System Results

An integrated system including Navcam panorama capture,goal designation, goodness/triage map generation, path plan-ning, and slip-compensated path following was performedonboard a research rover (Rocky8) in the Mars yard (see Fig-ure 1). Rocky8 is a research platform that is used by manyMars Technology Program (MTP) tasks to demonstrate thesoftware and algorithms on relevant hardware in a relevantenvironment (see [7] for a more detailed description of therover). It is similar in design to the MER rovers and the fu-ture MSL rover.

For this demonstration the rover started at one side of arock field on a sandy slope (see Figure 10). It first tooka panoramic image consisting of ten stereo Navcam pairsspaced 10◦ apart, resulting in a 90◦ panorama. Then a userdesignated a goal from this panoramic imagery (see Figure 6).In this test the goal was designated at (12.0, -4.0) meters inthe initial rover frame.

The point clouds are registered using the mast pan/tilt angles,and the rover roll and pitch (from the IMU) are used to grav-ity level the data. A goodness map and a triage map weregenerated from the stereo data (see Figures 7 and 8). Thesemaps used 10 cm cell sizes. Then a path was planned throughthe goodness map using the D* algorithm.

Once the path was planned, it was passed to the slip-compensated path follower. The path follower compensatedfor slippage and successfully reached the goal. It was runningVisual Odometry at 1 Hz and vehicle kinematics at 4 Hz, andresulted in a continuous motion from the start point to thegoal. The results of the traverse can be seen in Figure 9. Ascan be seen, the rover traverses the path and arrives at the goal(see Figure 10). The actual path in this plot is the output ofVisual Odometry. For a validation of the accuracy of VisualOdometry see [7],[16]. The maximum rover roll during thistraverse was 16.3◦ and the maximum rover pitch was 7.2◦.

7

Figure 5. Example results of continuous slip prediction on the map as the rover drives (the path and the final rover positionare marked in green). An input image and the corresponding map containing average color per cell (left), the automatic terrainclassification (middle), the final slip cost based on the predicted slip (right) (brighter means larger cost). Top: the rover istraversing flat sandy terrain. Bottom: the rover is driving upslope on soil terrain.

Figure 6. Panorama Taken by the Rover with the Designated Goal

HFTA Results

The HFTA algorithm was run on an analytical (i.e. not de-rived from stereo data) terrain consisting of a large hemi-sphere. It was run offboard, so it was not integrated withthe rest of the system. The forward simulation of the roverwas run four times over four different paths (see Figure 11).Table 2 shows the energy and path error metrics for each ofthe paths. It is important to note that the absolute accuracy ofthese metrics is not important because they are only used tocompare relative path costs to determine the lowest cost pathof the set. As can be seen, going directly over the hemisphere(path 1) is the most costly path in terms of energy and travel-ing on the side of the hemisphere (paths 2 and 3) is the mostcostly in terms of path error. Traveling on the flat groundaround the hemisphere has low cost for both energy and path

error, and is clearly the most desirable path of the four. Forthis terrain, these results are supported by intuition. Whenthe terrain complexity increases this algorithm will be able todistinguish subtle differences between paths that would not bepossible using simpler methods. On average, this algorithm,takes approximately 4 seconds per meter of path analyzed.Decreasing this runtime will be a focus of future work.

Table 2. Path and Energy Costs for HFTA Simulation

Path Energy (J) Path Error (m-s)1 5101.46 1.912 3324.51 12.323 1587.44 12.794 938.52 2.21

8

x (meters)

y (m

eter

s)

Planned Path Through Goodness Map

-15 -10 -5 0 5 10 15

-15

-10

-5

0

5

10

15

good

ness

x (meters)

y (m

eter

s)

Planned Path Through Goodness Map

-15 -10 -5 0 5 10 15

-15

-10

-5

0

5

10

15

good

ness

Figure 7. Goodness Map with Planned Path

x (meters)

y (m

eter

s)

Planned Path Through Triage Map

-15 -10 -5 0 5 10 15

-15

-10

-5

0

5

10

15

definitely traversable

definitely not traversable

unsure

unknown

x (meters)

y (m

eter

s)

Planned Path Through Triage Map

-15 -10 -5 0 5 10 15

-15

-10

-5

0

5

10

15

definitely traversable

definitely not traversable

unsure

unknown

Figure 8. Triage Map with Planned Path

9. CONCLUSIONSWe show results from individual tests of various subsystemsincluding: slip prediction and HFTA. Results from the slipprediction subsystem show that we are able to successfullypredict slip at a distance using data gathered in the field froma rover. Results from the HFTA algorithm show that we cancompare traversability of different paths by measuring met-rics from a dynamic forward simulation of the rover.

We also show results from experiments of a subset of the de-scribed technologies integrated onboard a research rover inthe Mars yard. These results demonstrate the feasibility ofthis approach as an end-to-end navigation system in a realis-tic Mars analog terrain.

10. FUTURE WORKFuture work will include the integration of the terrain clas-sification, the slip prediction, and the HFTA algorithms intothe onboard research rover software, and to demonstrate theentire integrated system in the Mars yard or in the field.

0 2 4 6 8 10

-2

-1

0

1

2

3

4

5

6

x (meters)

y (m

eter

s)

desired pathactual path

Figure 9. Slip-Compensated Path Follower Results

Figure 10. Rocky8 in the JPL Mars Yard at the Goal

1

23

4

start goal1

23

4

start goal

Figure 11. Paths Evaluated with HFTA

9

11. ACKNOWLEDGMENTSThe research described in this publication was carried out atthe Jet Propulsion Laboratory, California Institute of Technol-ogy under contract from the National Aeronautics and SpaceAdministration (NASA), with funding from the Mars Tech-nology Program, NASA Science Mission Directorate.

REFERENCES

[1] A. Angelova, L. Matthies, D. Helmick, G. Sibley, P. Per-ona, “Learning to predict slip for ground robots,” Inter-national Conference on Robotics and Automation, 2006

[2] A. Angelova, L. Matthies, D. Helmick, P. Perona, “Slipprediction using visual information,” Robotics: Scienceand Systems Conference, 2006

[3] M. Bekker, Introduction to Terrain-vehicle Systems,Univ. of Michigan Press, 1969

[4] J. Biesiadecki, E. Baumgartner, R. Bonitz, B. Cooper,F. Hartman, C. Leger, M. Maimone, S. Maxwell, A.Trebi-Ollennu, E. Tunstel, J. Wright, “Mars Explo-ration Rover Surface Operations: Driving Opportunityat Meridiani Planum,” IEEE Conference on Systems,Man, and Cybernetics, 2005

[5] A. Diaz, C. Elachi, et al., “A Roadmap for theRobotic and Human Exploration of Mars,” May, 2005(http://science.hq.nasa.gov/strategy/road%5farch.html).

[6] S. Goldberg, M. Maimone, and L.Matthies, “Stereo Vi-sion and Rover Navigation Software for Planetary Ex-ploration,” IEEE Aerospace Conference, Volume 5, BigSky, Montana, March 2002.

[7] D. M. Helmick, Y. Cheng, S. Roumeliotis, D. Clouse, L.Matthies, “Path following using visual odometry for aMars rover in high-slip environments,” IEEE AerospaceConference, Big Sky, Montana, 2004

[8] D. M. Helmick, S. Roumeliotis, Y. Cheng, D. Clouse,M. Bajracharya, L. Matthies, “Slip compensation for aMars rover,” IEEE International Conference on Intelli-gent Robots and Systems, 2005

[9] D. M. Helmick, S. I. Roumeliotis, Y. Cheng, D. Clouse,M. Bajracharya, L. Matthies, “Slip-Compensated PathFollowing for Planetary Exploration Rovers,” Journalof Advanced Robotics, October, 2006.

[10] A. Jain, J. Balaram, J. Cameron, J. Guineau, C. Lim,M. Pomerantz, G. Sohl, “Recent Developments in theROAMS Planetary Rover Simulation Environment”,IEEE 2004 Aerospace Conference, Big Sky, Montana,March 6-13, 2004.

[11] W. S. Kim, A. I. Ansar, and R. D. Steele, “Rover MastCalibration, Exact Camera Pointing, and Camera Hand-off for Visual Target Tracking,” IEEE 12th Int. Conf. onAdvanced Robotics (ICAR’05), July 2005.

[12] C. Leger, A. Trebi-Ollennu, J. Wright, S. Maxwell, R.Bonitz, J. Biesiadecki, F. Hartman, B. Cooper, E. Baum-

gartner, M. Maimone, “Mars Exploration Rover SurfaceOperations: Driving Spirit at Gusev Crater”, IEEE Con-ference on Systems, Man, and Cybernetics, 2005

[13] J. N. Maki, J. F. Bell III, K. E. Herkenhoff, S. W.Squyres, A. Kiely, M. Klimesh, M. Schwochert, T.Litwin, R. Willson, A. Johnson, M. Maimone, E. Baum-gartner, A. Collins, M. Wadsworth, S. T. Elliot, A. Din-gizian, D. Brown, E. C. Hagerott, L. Scherr, R. Deen, D.Alexander, and J. Lorre, “The Mars Exploration RoverEngineering Cameras,” J. Geophys. Res., Vol 108, No.E12, 2003, 12-1 to 12-24.

[14] M. C. Malin, J. F. Bell, J. Cameron, W. E. Dietrich, K.S. Edgett, B. Hallet, K. E. Herkenhoff, M. T. Lemmon,T. J. Parker, R. J. Sullivan, D. Y. Sumner, P. C. Thomas,E. E. Wohl, M. A. Ravine, M. A. Caplinger, and J. N.Maki, “The Mast Cameras and Mars Descent Imager(MARDI) for the 2009 Mars Science Laboratory,” 36thLunar and Planetary Science Conference, 2005.

[15] L. Matthies, S. Schafer, “Error modeling in stereo navi-gation,” IEEE Journal of Robotics and Automation, Vol.RA-3, No. 3, June, 1987

[16] M. W. Maimone and J. J. Biesiadecki, “The Mars Explo-ration Rover Surface Mobility Flight Software: DrivingAmbition,” IEEE Aerospace Conference, Big Sky, Mon-tana, March 2006.

[17] Clark F. Olson, Habib Abi-Rached, Ming Ye, JonathanP. Hendrich, “Wide-Baseline Stereo Vision for MarsRovers,” In Proceedings of the IEEE/RSJ InternationalConference on Intelligent Robots and Systems, pages1302-1307, 2003.

[18] Clark F. Olson and Habib Abi-Rached, “Wide-BaselineStereo Experiments in Natural Terrain,” In Proceed-ings of the 12th International Conference on AdvancedRobotics, pages 376-383, July 2005.

[19] G. Reina, L. Ojeda, A. Milella, J. Borenstein, “Wheelslippage and sinkage detection for planetary rovers,”IEEE/ASME Transactions on Mechatronics, 2006

[20] G. Sohl and A Jain, “Wheel-Terrain Contact Model-ing in the Roams Planetary Rover Simulation,” FifthASME International Conference on Multibody Systems,Nonlinear Dynamics and Control, Long Beach, CA,September 2005.

[21] A. Stentz, “The Focussed D* Algorithm for Real-TimeReplanning”, Proceedings of IJCAI-95, August 1995.

[22] M. Varma, A. Zisserman, “Texture classification: Arefilter banks necessary?,” Conf. on Computer Vision andPattern Recognition, 2003

[23] S. Vijayakumar, A. D’Souza, S. Schaal, “Incrementalonline learning in high dimensions,” Neural Computa-tion, 12, 2602-2634, 2005

[24] J. Wong Theory of Ground Vehicles, John Wiley & SonsInc., 1993

10

Daniel Helmick received his B.S de-gree in Mechanical Engineering fromVirginia Polytechnic Institute and StateUniversity and his M.S. in Mechani-cal Engineering with a specialization incontrols from Georgia Institute of Tech-nology in 1996 and 1999 respectively.Since June 1999 he has been working at

the Jet Propulsion Laboratory on robotics research projectsinvolving vision/sensor based control of robots, state estima-tion, manipulation, and navigation and mobility algorithms.He has worked on robotic vehicles covering a wide range offunctionality, including: Mars research rovers for rough ter-rain mobility; small, tracked robots for urban mobility; a cry-obot for ice penetration; and reconfigurable wheeled robotsfor Mars exploration. His research interests include: sensor-based control of robots, sensor fusion and state estimation,and rover navigation and mobility.

Anelia Angelova has graduated with aMaster of Science degree in ComputerScience from the Department of Mathe-matics and Informatics, Sofia University,Bulgaria and a Master of Science de-gree in Computer Science from the De-partment of Computer Science, Califor-nia Institute of Technology in 2000 and

2004 respectively. She worked as a software engineer forADA Bulgaria from 1999 until 2001. She is currently a PhDstudent at California Institute of Technology. Her research in-terests are in the area of computer vision and machine learn-ing.

Matthew Livianu graduated from Har-vey Mudd College in 2004 with a B.S. inComputer Science learning AI and de-signing, building, and programming var-ious mobile robots that found fire, fol-lowed walls, and navigated using GPS.He worked at Raytheon Company SASperforming signal integrity analysis for

space hardware. Currently, he is a Master’s student at Geor-gia Tech, studying systems and controls and computer engi-neering. His research is focused on intelligent controls forautonomous mobile robotics.

Larry Matthies obtained a Ph.D. de-gree in Computer Science from CarnegieMellon University in 1989, then movedto the Jet Propulsion Laboratory, wherehe is currently a Principal Member ofTechnical Staff and supervisor of theMachine Vision Group. His research hasfocused on terrain sensing and obstacle

avoidance algorithms for autonomous navigation of roboticvehicles. At JPL, he pioneered the development of real-timealgorithms for stereo vision-based obstacle detection and hecontributed to the development of the structured light sensorused by the Sojourner Mars rover. He has also developed al-gorithms for visual motion estimation from image sequences,3-D scene reconstruction from image sequences, real-timeterrain classification using multispectral imagers, and envi-ronmental mapping using sonar and stereo vision sensors.His group currently has research projects on computer vi-sion for robotic vehicles sponsored by NASA, DARPA, and theU.S. Army; these projects include work on navigation of Marsrovers, asteroid and comet landers, and Earth-based roboticvehicles for urban and cross-country missions. He is a mem-ber of the editorial board of the Autonomous Robots journaland an adjunct member of the Computer Science Departmentat the University of Southern California. He has also been aninvited speaker at the Frontiers of Engineering Symposiumorganized by the National Academy of Engineering.

11