A Survey of the Artificial Intelligence and Autonomous Capabilities of Space ProbesWith an Emphasis on Autonomous Planner and Scheduler Functions

Filip Jagodzinski

Advisor: Dr. Frank Klassner

Completed: Fall, 2002

This Independent Study final report is submitted in partial fulfillmentof the requirements for the degree of

M.S. in Computer Science

Department of Computing SciencesVillanova University

Villanova, PennsylvaniaU.S.A.

Abstract

Artificial intelligence (AI) technologies are being integrated more and more into space exploration missions as part of a drive to produce autonomous agents. These technologies span a variety of topic areas, including machine learning algorithms, knowledge discovery tools, and data analysis systems. Various AI programs and systems that are used on board a space exploration agent can help reduce the overall cost of a mission and increase the amount of useful quality data that is gathered. An autonomous planner/scheduler, for example, circumvents the need for continued—and often lengthy—uplink and downlink communications between mission personnel on Earth and the spacecraft. An autonomous planner does not wait for command instructions from Earth, but instead the planner generates and integrates effective command sequences into the mission timeline in response to malfunctions or unforeseen science opportunities. Without the need for overhead of communicating with Earth, a spacecraft that uses an autonomous planner can quickly recover from malfunctions and can record short-lived anomalous events. The use of a variety of autonomous systems on board a spacecraft is not only useful, but in the future may be necessary. If a spacecraft is expected to travel far enough away from Earth where communication is not feasible, or if the spacecraft is expected to lose contact with Earth for some period of time, then autonomous systems can monitor and reactively respond to any changes. If an autonomous agent approaches an unstable object—for example a comet—then the autonomous planner and image analysis tools can quickly respond to dangerous events and prevent damage to the spacecraft.

This survey report explores the issues and potential solutions for supporting increased autonomous behavior in space probes. The report begins with an overview that includes a discussion on the integration of different subsystems and a discussion about the main features and components of an autonomous agent. The second section of this report explains several planned missions with autonomous features and also explains a few programs that use autonomous technologies. Included is information on an image analysis tool, the Multi-Rover Integrated Science Understanding System, The Distant Autonomous Recognizer of Events (DARE) System, the Autonomous Small Planet in Situ Reaction to Events (ASPIRE) mission, the Techsat-21 Autonomous Science Craft Constellation Demonstration mission, and the Three Corner Sat Mission (3CS). The second section concludes with a discussion about the challenges and design opportunities of autonomous agents. Machine learning concepts are explained next, as are neural networks, a feature of machine learning algorithms. The fourth part of this report takes a closer look at one component of autonomous agents, the planner. Batch planners, dynamic planners, planner logic, planner languages and hierarchical task network planners are all explained. The Automated Scheduling and Planning Environment (ASPEN) as well as the Simple Hierarchical Ordered Planner (SHOP) are discussed in detail. In the final section of this report, future work, possibilities for the continuation of this survey report are explored, and possible thesis projects are proposed.

2

Table Of Contents

Abstract.........................................................................................................................2Table Of Contents.........................................................................................................3Acknowledgements.......................................................................................................4

1. Introduction...................................................................................................................52. Autonomous Mission Designs and Different Autonomous Technologies....................8

2.1 Texture Analysis for Mars Rover Images..............................................................82.2 The Multi-Rover Integrated Science Understanding System (MISUS).................92.3 The Distant Autonomous Recognizer of Events (DARE) System.........................92.4 The Autonomous Small Planet In Situ Reaction to Events (ASPIRE) Project....112.5 The Techsat-21 Autonomous Science Craft Constellation Demonstration..........132.6 The Three Corner Sat Mission.............................................................................152.7 Challenges in the Design of Autonomous Agents and Mission Planners............16

3. Support Technology for Autonomous Agents............................................................183.1 Machine Learning.................................................................................................183.2 Neural Networks...................................................................................................20

4. Autonomous Agent Planners......................................................................................214.1 Coordination of Subsystems.................................................................................214.2 System Languages and Planner Logic..................................................................224.3 Batch Planners......................................................................................................234.4 Dynamic Planners.................................................................................................244.5 Hierarchical Task Network Planners....................................................................254.6 The Automated Scheduling and Planning Environment (ASPEN)......................264.7 The Simple Hierarchical Ordered Planner (SHOP).............................................29

5. Future Work................................................................................................................326. References...................................................................................................................33

3

Acknowledgements

I wish to thank Dr. Frank Klassner for his support and continued suggestions throughout the course of this independent study project. Also I wish to thank Dr. Don Goelman and Dr. Lillian Cassel for directing me to invaluable reference sources on data mining and information technology.

4

1. Introduction

On July 4, 1997, the Mars Pathfinder's Sojourner Rover rolled down the ramp of the Lander spacecraft and began a 90-day survey mission of the Martian surface. The trip from Earth to Mars had lasted approximately 6 months, during which time data was periodically uploaded and downloaded from the Pathfinder probe via NASA’s Deep Space Network’s 34-meter antenna at Goldstone, California [23]. A UHF antenna on board the Sojourner Rover was used to communicate with mission control on Earth; an Earth-based operator controlled the 6-wheeled Sojourner Rover. The roundtrip time for the relay of communications between Earth and Mars is approximately 20 minutes, but control commands for Sojourner were sent approximately every 12 hours because the process of generating a new command sequences was lengthy. As downlink data was received, engineers determined the state of the rover, noting the state of various system components. The rover data was then given to scientists who produced a high-level science plan, which then was given to engineers who generated a low-level command sequence that was returned to the Pathfinder Rover for execution. Although Sojourner did have a few autonomous capabilities that were tested a few times, Sojourner never encountered events that it had to respond to immediately and it did not have the ability to generate a plan for all of the subsystems. Sojourner was able to drive around objects that were it its way and could plan a path between two different points, but Sojourner was not able to continually operate in an autonomous mode. Future missions to Mars, other planets, and to regions beyond Pluto will require that spacecraft be able to function without human intervention for long periods of time and perform several autonomous functions—this is one motivation for the integration of autonomous technologies into space missions.

If many mission components of a space mission could be automated, then mission costs can be decreased and the amount of necessary communication time between Earth and a spacecraft can be greatly reduced. If a spacecraft can generate valid command sequences for itself in response to sensor inputs, can manage its own onboard resources, can monitor the condition of different spacecraft components, and can intelligently gather and analyze data, then more time could be spent on exploration and less time on waiting for new command sequences from Earth.

An autonomous spacecraft is one that performs its own science planning and scheduling, translates schedules into executable sequences, verifies that the produced executable sequences will not damage the spacecraft and executes the derived sequences without human intervention. Because the goals of each mission are unique, there does not exist a “perfect” recipe for constructing an autonomous agent. Different missions require various levels of autonomy, and depending on the sensor types and overall architecture of a spacecraft, a successful autonomous agent architecture and set of algorithms for one mission might not be effective when applied to another mission with different objectives and a different developmental history. Several common topics that reappear in different autonomous agent designs can, however, provide a general roadmap for the design of a successful autonomous agent.

5

Regardless of whether an autonomous agent is required to exhibit autonomous behavior for 5 seconds or for 100 years, several issues relevant to successful continued functioning of the spacecraft must be addressed. Resource constraints and hard deadlines require that resources be managed effectively. Non-renewable solid energy fuels must be used sparingly, and alternative, renewable energy sources and methods to acquire that energy should be used. For example, solar power can be used as a source of energy, and a calculated close flyby of a planet can produce a “slingshot” effect where the spacecraft is accelerated due to the transfer of rotational energy from the planet to the spacecraft. Resources must also be managed in reference to the number of sensors used; although an increase of sensors can increase the validity and precision of data, the addition of each new sensor necessitates the use of more energy. Effective management of resources of a spacecraft also includes the effective maintenance of concurrent activities. A planner/scheduler must be able to schedule concurrent activities in different parts of the spacecraft; hence there exists the possibility of communication overload or of an energy shortage.

The architecture of an autonomous agent must facilitate effective communication, scheduling and schedule execution functions. An effective data bus and/or communication system must allow easy access to mission goals and the mission databases, including both hard-coded data and data that has been acquired during the mission. The planner and scheduler require a host of communication capabilities to transmit goals and spacecraft status parameters; the derived schedule may be broken into various segments depending on the design of the planner. Schedule segments are sent from the planner to the executive module that executes the various schedule components. Messages are repeatedly passed between the various components of a spacecraft operating system, and the hardware and software architecture of the spacecraft must be designed so that spacecraft communications systems can function at an optimal level.

The function of the planner/scheduler of an autonomous agent is to generate a set of high-level commands that will help achieve mission goals. Planning and scheduling aspects must be tightly integrated, and any planned activities must be coordinated with the availability of global resources. Different subsystems of a spacecraft can also have their own subroutines, so a planner must be able to integrate different spacecraft functions and be aware of the different energy requirements of and working parameters of different spacecraft components.

The hybrid executive performs process and dependency synchronization and maintains hardware configurations in an effort to manage system resources. Because an autonomous agent can potentially encounter problems—both due to the incorrect functioning of the spacecraft or when introduced to an environment for which the agent is not suited—the agent must be able to enter a stable, “safe” state. Different agents use various forms and styles of a host of available abstract languages as a means of communication between different system components. Regardless of what language is used, the hybrid executive must be able to process different spacecraft modes using the available abstract language. In the case when an agent enters a “safe” mode in response to an error, the executive must be able to process the directives that are provided by the

6

planner. The ability of the executive to understand abstract language terms and a host of plan initiatives allows for the continued execution of a plan under a variety of execution environments.

An autonomous agent should also be able to identify the current operating state of each component of the spacecraft—this allows for an executive to reason about the state of the spacecraft in terms of component modes rather than in terms of low-level sensor values. This approach makes mode confirmation, anomaly detection, fault isolation and diagnosis and token tracking possible. Mode confirmation provides a confirmation that a command has been completed successfully, while anomaly detection is simply the identification of inconsistent subsystems. In response to anomaly detection, the executive also should be able to isolate and diagnose faulty components and monitor the execution of the commands that have been produced by the planner. In an effort to perform all the above functions, the agent uses algorithms that describe and monitor the current state of the spacecraft. The mode of each component part of the spacecraft can be modeled using prepositional logic; the ensuing component representations can be used by the planner and by the hybrid executive modules.

The way that knowledge is represented can have a great effect on the proper continued working status of an autonomous agent. As an example, in the case of heterogeneous knowledge representation, which is defined as “a non-uniform method used in loosely-coupled architectures for storing specialty knowledge not for general use [2]”, coverage and mode identification is efficient; at any one time there may be several representations of the same spacecraft component. This heterogeneous knowledge representation provides multiple views of the condition of various components of a spacecraft and allows for various component representations to be combined in union to form a thorough complete picture of the state of a system. As a result, task and component specialization is possible, but likewise model representations can diverge and leave an autonomous system with “too many” views of the state of the autonomous agent. The knowledge representation language must therefore be concise, yet simple enough so that quick calculations and necessary modifications to different spacecraft systems can be made.

Various components and technologies can be combined to make an “autonomous agent”, including machine learning algorithms, artificial neural networks and expert intelligence systems. The effective coordinated functioning of the different autonomous components of an autonomous agent is dependent on many factors, including, among others, the management of resources, an overall system architecture, efficient scheduling and planning techniques, and the ability to handle errors. Additionally, any one mission that requires the use of autonomous agents may need to employ a host of data mining, knowledge discovery and pattern matching capabilities, as well as be able to classify, cluster, and effectively organize sensor data. Different missions require the use of different autonomous technologies, so each autonomous component of a spacecraft must not only perform its job but fit well within an overall autonomous architecture.

7

2. Autonomous Mission Designs and Different Autonomous Technologies

Included in this section are several short reviews of a variety of space missions that employ autonomous agent technologies. In some cases, only certain aspects of an entire system exhibit autonomous behavior, while other spacecraft are designed to be fully autonomous. Included in this section are examples of an autonomous image recognition algorithm, autonomous agent architecture designs, event recognition and classification agents, autonomous systems that employ reactive planners, and multi-rover/multi-spacecraft missions.

The variety of goals, design features and methodologies inherent in each of these missions is a good indicator of how “autonomous” technology can manifest itself in many forms depending on mission objectives. Several concepts are consistent across many different autonomous projects. There is no one “perfect” recipe for constructing an autonomous agent, but instead different autonomous agent technologies are combined so that mission goals can be achieved.

2.1 Texture Analysis for Mars Rover Images

Texture analysis through the use of image pixel clustering and image classification is one example of an autonomous functionality—made possible by machine learning—that can be part of a larger autonomous system. The goal of image analysis is to enable an agent to intelligently select data targets; this is an important goal when considering that the amount of data that is acquired by a system tends to far exceed the storage capacity or the transmission capacity of an agent while on an exploration mission [7]. Different programs use different algorithms for the purpose of image analysis. Discussed here is the technique of analyzing textures as a means of image and object classification.

One way to extract texture information from an image is through the application of filters and through the use of different mathematical functions; image data, for example, can be translated into the frequency domain through the use of the Fourier Transform. Several filters have been popular in this endeavor, including Gabor filters that were first introduced by J. G. Daugman [8]. Different cells in the visual cortex can be modeled through the use of 2 dimensional functions. This allows for different filters to be able to discriminate between various visual inputs in the same way that different cells in the visual cortex are responsive to different stimuli. A bank of filters can be stored in a database so that a host of anticipated textures can be analyzed and classified. Mission resources can dictate which filters should be supplied, and different filters can be rejected or accepted depending on the amount of computational power that is required to pass a filter across an image. Additionally, various filters can be applied in tandem or in series so that a complete mathematical representation of an image can be calculated. Not only can different filters be applied, but also different filters have different parameters, many of which can be adjusted, ignored or otherwise altered in an attempt to better suit a filter to the specific data domain.

Once the set of filters is run across an image, clustering tools can be used to classify the pixels of the image into one of several categories. Statistical analysis can then be

8

performed to ensure that classification schemes have been properly developed and that the differences between separate classifications are significant enough to ensure proper separation of the pixel data into unique categories [8].

2.2 The Multi-Rover Integrated Science Understanding System (MISUS)

The Jet Propulsion Laboratories (JPL) Machine Learning Group is developing the MISUS system. The goals of this system are ideally suited for Mars exploration through the use of multiple autonomous agents working together to achieve science learning goals [1]. The MISUS system employs autonomous generation of science goals, uses learning clustering methods, and attempts to use algorithms capable of thesis formulation, testing and refinement.

The autonomous aspect of the MISUS system selects and maps out at appropriate scales those areas of the surface most likely to return information with maximum science value. The MISUS system is based on a framework capable of autonomously generating and achieving planetary science goals. Machine learning and planning techniques are used, including the use of a simulation environment where different terrains and scenarios are modeled. The system learns by clustering data that it attains through spectral mineralogical analysis of the environment [1]. The clustering method attempts to find similarities among various samples and builds similarity classes. A model of the rock type distribution is created and continually updated as more robots attain more and more sensor data of the different terrain specimens. In the goal to attain both a large number of samples and to attain a uniform collection pattern of specimens, the planning and scheduling component of the MISUS system determines and monitors the activities of the various robots.

Within the framework of the MISUS system there exists an overall goal of “science data acquisition”. In particular, the MISUS system developers aim to “test the AI algorithms on the resulting mineral distribution and reconstruct sub-surface mineral stratigraphy in candidate areas to determine possible extent (temporally and spatially) of hydrothermal activity, while considering the implications for Mars biology and the usefulness of mineral deposits [1]”. Requirements to meet the science acquisition goal are continually updated in response to the activities of the individual robots. An iterative repair algorithm continually refines goals and activity schedules, while a statistical modeling algorithm that employs stochastic parameterized grammar is used to formulate, test, and then either refine or discard a hypothesis.

2.3 The Distant Autonomous Recognizer of Events (DARE) System

The goal of the DARE system is to detect a new, potentially dangerous, event sufficiently before encounter so that there is enough time to autonomously perform mission re-planning. Autonomous agents that come within close proximity of comets or otherwise maneuver in dangerous landscapes are thought to benefit most from this type of technology. The DARE system performs autonomous mission redirection calculations in response to specific sensor events. Image segmentation, clustering methods and a host of other techniques are employed within a 6-component architecture [22].

9

The motivation to design a re-planning autonomous agent stems from the spacecraft Galileo’s discovery of a satellite of the asteroid Dactyl in 1993. At the time it was believed that Dactyl’s satellite was an anomaly, but in 1998 again such an asteroid-satellite couple was encountered. Because the spacecraft Galileo did not possess the autonomous probe capabilities widely in use today or that are currently in the development and deployment stages, mission re-planning was not possible.

In an effort to locate and identify anomalous events, image segmentation and clustering techniques are used. The tracking of regions through a sequence of images allows for noise to be effectively removed. The tracking of regions takes into account a variety of scenarios, including tracking from a stationary or moving platform, tracking a moving or stationary body, tracking “noisy” or “clean” events, tracking against stars or tracking from images alone [9]. The agent platform can be assumed to be stationary if it is moving slowly relative to the image sampling rate, but in the cases where the sampling rate is low or the movement of the agent spacecraft in reference to an event is sporadic, the projection in the image plane can be approximated by a straight line with near constant velocity. If the sampled object moves significantly from frame to frame and the sampling rate is low, then tracking of the object must be done in three dimensions. To track an object in universal coordinates, lines from sight are drawn from the imaging device through the detection object in the image plane; when lines through regions in various images intersect at approximately the same point, the intersection regions represent an object.

The architecture of the DARE is system built upon 6 main components. Images are received from a camera unit and the images are searched for new bodies. If a body is detected, then new pointing angles are calculated and passed to the scheduler. The image processing unit performs filtering and image enhancement functions, while the region segmentation processing unit performs the necessary data clustering. Regions are calculated based on background pixel intensity and are “grown” from the ratio image where each pixel gets a value that takes into account the mean and standard deviation of statistical clustering functions. A registration unit tags each object and frame based on the body of the object, the orientation of the object in reference to the background stars or based on the object’s geometric characteristics. Once an object has been tagged and identified, the matching unit finds similarities between regions in consecutive images in an effort to begin the tracking process. Because at any one time there may be a host of potential object candidates and, depending on the sampling rate, a large number of images that must be processed, the amount of possible object-image combinations becomes immense. In an effort to overcome this combinatorial explosion, tree pruning is used as a method to reduce the amount of possible states. After an object has been identified among several frames, the track-file maintenance unit follows the object as it is placed into one of three tracks, failing, reserved and object. The failing track is a registry of those objects that don’t seem to be a good object candidate because of conflicts with previous images; those objects in the failing track and subject to pruning. The reserved track maintains a registry of those objects that are more worthy than those objects in the failing track but that demonstrate ambiguous behavior over the course of tracking. The

10

object track is a registry of those items that are most certain to be objects of interest. The final unit, the detection-reporting unit, creates a record of those objects that survive over the course of object tracking and which are candidates to be returned to Earth for further study or given to a planner module in the case of autonomous re-planning.

The use of image tracking and dealing with combinatorial explosion issues places special emphasis on the proper management of system resources and the design of the overall system architecture. Aside from the combinatorial explosion due to the various objects that are being tracked over the course of several images, errors that are generated during geometric analysis of object paths and orientation, insufficient parallax computations and cosmic ray noise add further computational strain on the entire system. For example, in the case where geometric errors are prevalent, it is possible that the DARE system will track many tracks, all of which may be the same object but with only slightly different geometric and path parameters.

The DARE system has been tested and validated using an autonomous on-board system. Using the record of objects that are produced by the detection-reporting unit, a science processing unit searches for scientifically interesting events and sends discovered objects to the planner. The planner produces a sequence of mission redirection commands in response to a set of mission goals. Continued object observation can also be planned in the case that more object information is desired. The planner module can also opt to send object information to mission personnel on Earth in an effort to attain a “second opinion” or in the case of an unresolved conflict in the planner module.

2.4 The Autonomous Small Planet In Situ Reaction to Events (ASPIRE) Project

Related to the DARE system is the Autonomous Small Planet In situ Reaction to Events (ASPIRE) project. The goal of the ASPIRE system is to safely perform a close-proximity rendezvous of a low-activity comet. The target for ASPIRE is the comet duToid-Hartley, which will be at perihelion of 15 February 2003. To achieve this goal, the DARE system employs a host of autonomous activities, including close proximity navigation planning and execution, onboard pointing, onboard execution, and onboard mission planning and sequencing.

A great amount of interest lies in the study of comets due to the fact that they are composed of the “building blocks” of the solar system; their compositional data is of extremely high scientific value. Unfortunately, comets have the tendency to fracture, and hence close rendezvous can only be performed if an agent is able to intelligently detect and respond to any fracture events. A fracture event can be of varying intensities, ranging from the expulsion of gasses to a total comet breakup. Collection of comet gasses as well as collection of comet fragments requires an agent that can autonomously maneuver to within a few kilometers of an active nucleus of a comet. The autonomous agent must be able to effectively maneuver while being bombarded by high-speed particles that are being ejected from the comet rock nucleus.

11

The ASPIRE project aims to integrate the detection, analysis and investigation of short-term comet event. A science mission can have one or a series of high-order goals, and it is the goal of the ASPIRE agent to re-plan science objectives in response to events in the current environment. The agent will approach the comet from the sunward side (at the time of perihelion, the sunward side of the comet is away from the stream of gasses that form the comet tail) and will perform a series of maneuvers so that at arrival into orbit the agent will have a velocity of 0 relative to the comet [4]. Leading up to and once in orbit around the comet, the ASPIRE agent will function in one of three main modes, the parking mode, the orbiting mode, and a hovering mode. An escape mode is also available in the case that the comet exhibits break-up characteristics and endangers the spacecraft.

The parking mode involves the maneuvers that bring the agent to a safe and close proximity of the comet. From the parking stage, the agent can move into the even-closer orbiting mode or choose to remain in the parking stage depending on the condition of the comet. Within the orbiting stage, the spacecraft flies a sequence of orbits around the comet and captures data through the use of wide-field and narrow-field view cameras. The hovering state involves a close flyby of the comet surface in an effort to closely investigate various features of the comet. In both the orbiting and hovering states, the spacecraft must complete the planned routine before choosing to go to the next hovering or orbiting state—except in the case of comet break up, in which case the spacecraft enters escape mode, leaves the proximity of the comet and returns acquired data to Earth.

An example scenario includes the approach of the spacecraft to within 20km of the comet. An event is detected by the science module, and the spacecraft points the narrow-filed of view camera to the location where the event was detected. The exact coordinates of the event from the science module are used to construct a close-flyby (the hovering mode) trajectory. The close fly-by is performed, image data is collected, and the spacecraft returns to the safe 20km distance from the comet. The next event involves the ejection of two particles from the comet’s body. The ejection event is detected, analyzed and further investigated by use of the narrow-field of view camera. Further break occurs and the comet begins “total breakup”, at which time the spacecraft enters escape mode.

The science module of the ASPIRE system must be proficient in detecting events of scientific interest. There are seven components to the science module, including a mode handler component, a viewing angles calculation component, a change detection algorithm component, a confidence measure component, a clustering component, and a classification component. The mode handler component of the ASPIRE system receives the current mode of the spacecraft and appropriately determines what to do with the acquired (and stored) data images. The mode handler chooses to store images, compare images or request to acquire additional images. The viewing angles calculation component uses the spacecraft’s position in reference to various astronomical markers as well as the tilt of the camera to determine the focus of the onboard cameras. This “angles calculation” procedure allows the ASPIRE system to compare currently acquired images to those images that are stored in the database. Subsequently, images in the database can be replaced or either appended so that a time delimited record of events can be stored.

12

The change detection algorithm uses individual images of a specific region to determine any changes—events—that might be underway. The changes are detected by dividing the images into small tiles. Various correlation, interpolation and a host of statistical analysis techniques are used to detect the displacement of each tile in reference to past images of the same location. The statistical analysis techniques form a part of the confidence measure component of the science module where threshold values, upper-bound and lower-bound variables help determine confidence measures. Clustering methods are used to combine displacements of similar magnitude and direction so that large-scale events—events that span several image sets—can be detected. In the case of complete comet breakup or large particle ejection, various components of the comet will be detected through the application of the clustering component of the science module [4]. Finally, the classification component of the science module is responsible for the classification of events into one of several categories, including jet formation, surface fracture, fragment ejection or complete comet breakup. Depending on the nature of the event, the results of the statistical analysis module will help determine if the spacecraft should collect gas samples, collect particle samples or escape due to comet breakup. 2.5 The Techsat-21 Autonomous Science Craft Constellation Demonstration

The Autonomous Science Craft Constellation flight (ASC) is scheduled to fly on board Air Force’s TechSat-21 mission in October 2004. The goals of the ASC mission include onboard science analysis and algorithms, re-planning, robust execution, model-based estimation and control, and the execution of formation flying in an effort to increase science return. As is the case with the MISUS system, a series of independent agents will work together in the ASC project. Distributing functionality among the different agents will increase system robustness and maintainability, and in turn the cost of the mission will be reduced when compared to a multi-satellite mission where each spacecraft possess a complete set of autonomous routines and subprograms and where each micro satellite performs the same set of duties.

The micro satellites will at times function as one “virtual” satellite, while during other times each micro satellite will be performing special tasks. The satellites will fly as far as 5km apart or fly in formation where they are separated by as little as 100m. Radiation hardened 175 MIPS processors will be on board each micro satellite, and OSE 4.3 will be the operating system of choice because of the “message passing” nature of the program [16].

The onboard science algorithms fall into two main categories: image formation and onboard science. Image formation is responsible for creating image products, while target size, pointing parameters and radar attributes will be used to determine the resolution for an image so that the best image in the least amount of time can be created. The reduced resolution images also allow science products to be scaled and compressed, and so more data can be stored onboard the spacecraft. The onboard science algorithms will analyze the images produced by the image formation component and will output derived science products, if any. Trigger conditions across time variant images will be used to detect anomalous events, as for example backscatter properties that may be an

13

indication of recent change [24]. Trigger conditions will be checked not only on successive images but also on images that span a large amount of time; such a procedure will be able to detect slow-change events such as the movement of ice plates. Statistics will be used to determine if significant changes have occurred as measured by region size, location, boundary morphology and the image histogram. Different missions will require different computational requirements due to the nature of the content of the images, and so the statistical algorithms scale linearly to accommodate an image of varying pixel resolution. The detection of an anomalous event will trigger a series of events, including the continued monitoring and analysis of an event and, if necessary, the immediate downlink of information to the mission command center.

The Spacecraft Command Language (SCL) will handle the robust execution aspect of the ASC system where procedural programming with a real-time, forward-chaining system will be used. The communication software will be of a publish/subscribe nature, where notification and request messages will be handled appropriately. The Continuous Activity Scheduling Planning Execution and Replanning (CASPER) system will have the ability to plan and schedule various SLC scripts, and spacecraft telemetry information from the different micro satellites will be gathered by one of the satellites and used as input for an integrated expert system (For more information about the CASPER planner, see the section on the ASPEN planner, which is a forerunner of CASPER). This integrated expert system will use trigger rules for fault detection, mission recovery procedures, mission constraint checking and pre-processing initiatives [6].

The model-based monitoring and reconfiguration system will be responsible for the reduction of goals into a control sequence. To function effectively, the model-based monitoring and reconfiguration system will oversee the proper function of all components that exist along the command path so that damaged components are not initiated. Not only must the reconfiguration system ascertain the health state of various components, but also it must consider repair options and be able to produce a plan from among the components that are working. In effect, the monitoring and reconfiguration component must perform a great deal of reasoning; a model-based executive is used so that effective choices can be made.

A model-based executive will be used to track planner goals, confirm hardware modes, reconfigure hardware, generate command sequences, detect anomalies, isolate faults, diagnose various components and perform repairs. The model executive receives a stream of hardware configuration goals and sensor information that it uses to infer the state of the hardware. The model executive continually tries to transition the hardware towards a state that satisfies the current configuration goals; this allows the model-based executive to immediately react to changes in goals and failures [5]. Control actions are incrementally generated using the new observations and goals given in each state.

The model-based executive uses mode estimation, mode reconfiguration and model-based reactive planning to determine a desired control sequence. The mode estimation stage involves the setup of the planning problem where initial and target states are identified. A set of most likely state trajectories is incrementally generated. The mode

14

reconfiguration state sets up the planning problem, identifies the initial and target states, and tries to determine a reachable hardware state that satisfies the current goal configuration. The model-based reactive planning stage (which is based on an enhanced version of a Burton system [16]) reactively generates a solution plan by generating the first action in a control sequence that moves the system from the most likely current sate to the target state. The model-based executive’s link with the SCL provides execution capabilities with an expressive scripting language. This makes it possible to generate novel responses to anomalous events.

The coordination of formation flying—cluster management—will be carried out by decomposing control systems into agents consisting of multi-threaded processes. Each process consists of a message that has a content field that is used to identify the purpose of the message and its contents. Different agents will be loaded at different times, and hence can be configured at the time of deployment. As part of the coordination process among the micro satellites, agents will search out other agents that can provide needed inputs. Because of this architecture, agents are easily created while control and estimation procedures can be effectively integrated.

2.6 The Three Corner Sat Mission

The Three Corner Sat (3CS) mission—a cooperative project between The University of Colorado, Arizona State University and New Mexico State University—includes robust space command language (SCL) execution, continuous planning, onboard science validation, anomaly detection and basic spacecraft coordination. Three, 15kg nano-satellites will fly in formation during which time each nano-satellite will be tumbling, having no control mechanisms for orientation control and stabilization.

Onboard science validation will involve the analysis of images. Because each nano-satellite will be tumbling and taking photos at various time intervals, part of the photos will be of Earth, while others will be of outer space. The analysis of the images will involve the compression of the data into a series of 1 and 0s, where different pixels of different images will be assigned either a value of 1 (a threshold brightness value has been reached, hence it is most likely that the pixel is part of an image of Earth) or a value of 0 (a threshold brightness has not been reached, hence it is most likely that the pixel is part of an image of outer space). Images will receive an aggregated value based on the sum of “bright” pixels, and only those images meeting a certain brightness threshold will be used for further analysis.

The robust execution will use a standard SCL that was first developed for embedded flight environments. It is hoped that the SCL language will be reusable across different control systems. The SCL allows data from multiple sources to be processed, and includes scripts, constraints, data formats, templates and definitions. The goal of the SCL is to fully define all data and allow for the control system to organize and validate different system processes. It is hoped that the SCL will allow the operating system to take actions depending on a variety of inputs, including time dependent events, operator directives and the state of various system components [3]. The general procedure of the

15

SCL involves two main steps, the acquisition of data and the modification of the SCL database by means of an inference engine. A data IO module will acquire, filter, smooth, and convert the data to engineering units, while the Real Time Engine (RTE), the collective of an inference engine, command interpreter, scripts scheduler, and execution manager, will capture all SCL database updates and process the different rules associated with different database items. SCL scripts will perform imaging, manage communication links between the three nano-satellites, perform resource management and coordinate communication with ground control.

The 3CS mission will use the CASPER planning software as well as the Selective Monitoring System (SELMON). The SELMON system uses multiple anomaly models to identify the isolate phenomena and anomalous events. Outlier analysis will be used as an anomaly detection tool, while attention focusing will attempt to determine how much and which component of the system is being affected by an anomalous event. During the first few moments of an anomalous event, different components of a system may be bombarded with data and a host of abnormal signals, all of which should be analyzed. The goal of the attention focusing feature is to help organize and catalog the state of the various components of a system at the time of anomaly detection.

2.7 Challenges in the Design of Autonomous Agents and Mission Planners

Just as there is no one perfect autonomous agent prototype, there are no specific obstacles and challenges that are applicable to the design and implementation of all autonomous agent missions. Unique mission goals will present unique challenges. General software design and mission planning concepts are, however, widely applicable. Different components of a system must be effectively coordinated; a system language must be exact enough to allow for intricate expressions but yet simple enough for debugging and transmission purposes; image recognition and analysis tools should be effective yet not require extensive amount of computational resources.

Texture analysis algorithms must address resource availability issues. Mathematical and statistical programs, although very effective, might only be used to a limited extent when implemented in an autonomous agent with limited computational power. Tools that require less computational power must be created, or current computational-intensive tools must either be adjusted or applied in instances when absolutely necessary.

Of key concern to the MISUS project are issues related to coordination among a host of robots and the ability to determine how to form a hypothesis. Several nano-satellites will be operating in tandem in an effort to collectively increase the amount of science data that is collected. Communication and coordination must be managed, possibly through the delegation of a “master” robot. The MISUS system must also be able to effectively produce a hypothesis, and then either refute or refine the hypothesis in response to test data. How can such decisions be made effectively and “intelligently” with limited use of hard-coded parameters and without the use of preset upper or lower bounds? If hard-coded parameters and threshold values are used, how certain are we that an anomalous, yet undiscovered phenomena, will be effectively observed and analyzed?

16

Design issues in the DARE system focus on the ability to recognize and track new events. The computational complexity of detecting and tracking objects through the analysis of successive images favors the pruning of the knowledge space so that computational resources are not starved. Mathematical analysis and a host of statistical analysis procedures may be used so that object tracking is more effective. As is the case with the image texture analysis algorithms, mathematical analysis tools must be managed in light of limited onboard computational resources.

The ASPIRE project is an example of a close proximity rendezvous mission with a comet. The Deep Space 4 / Champollion (ST4) mission, in fact, proposes to land on a comet, an even more daunting task considering the unstable nature of comet surfaces. In a landing scenario, re-planning and scheduling algorithms must be very fast and efficient. Because of actual contact with a comet, a break-up activity must be detected almost instantly and likewise a plan for escape must also be initiated immediately. An autonomous agent landing on a comet will not have time for image analysis. More importantly, images that are taken while an agent is anchored to a comet surface do not provide the ability to see the comet from far away; hence there must be another way to detected comet breakup.

The Three Corner Sat (3CS) Mission is also unique because it employs a series of micro satellites that will be tumbling and taking images at spaced intervals. The micro satellites must be coordinated, and collected data must be analyzed so that only “usable” science products are returned for analysis. In the 3CS mission, there is a need to keep mission costs low while at the same time employ new technologies that can be costly to develop and perform. The tumbling feature of the 3CS mission means that the satellites cannot adjust their orientation and altitude, so spacecraft construction costs are reduced. The image analysis algorithms must be efficient and yet functional on a system with limited computation resources. The advent of new autonomous technologies will surely help advance the science of space exploration, but the extent to which the new technologies can be applied must be taken into consideration. This is similar to the image and texture analysis projects where powerful tools have been created but, because of computational or resource constraints, can be only implemented to a limited extent.

17

3. Support Technology for Autonomous Agents

There exist a variety of technologies that supports the development of autonomous agents. The products of the research on these technologies provide better and more autonomous functionalities that can be used for space missions and autonomous space probes. Machine learning, the process by which an agent organizes information and then uses that organized information to improve its future performance in some particular task, is one technology that is highly applicable to autonomous space missions. Discussed here is the general process of machine learning as well as the concept of neural networks, a technology that allows machine learning algorithms to be efficient and reliable.

3.1 Machine Learning

Machine learning is concerned with the design and deployment of algorithms that improve their performance based on past experience or data. There are various approaches to machine learning, including classification, clustering, regression (linear and non-linear), hypothesis modeling, and the use of neural networks. Classification employs feature identification tools such as template matching algorithms, clustering includes image segmentation and texture analysis techniques, regression involves the use of predictive models; while hypothesis modeling uses simulations to better define classification models. Each of the above techniques uses a host of statistical and graphical tools, and in many cases the exact design of a machine learning component of an autonomous agent is specific to a mission and is tailored to expected available resources. For example, analysis and classification can be phenomenological or hypothesis based. Phenomenological analysis includes the detection and classification of objects based on anticipated phenomena, while hypothesis based detection and classification schemes involve looking for data that either supports or refutes a hypothesis that is created based on data-specific knowledge.

The general machine learning process involves a series of steps, including observation, hypothesis generation, model formulation, testing, and refinement/rejection [10]. The first phase of machine learning involves the organization and exploration of high-dimensional data. Dimension reduction and generalization algorithms may be employed to make the data more manageable, considering that this early stage of machine learning involves a large amount of data. As data is collected, classification and anomaly detection is performed. It is the goal of classification to spatially order and categorize data, while outlier analysis—a data mining technique—is used to screen for potentially “interesting” events that can be good candidates for further investigation. During the second stage of the machine learning process, a hypothesis is formed. Clustering techniques are used to fit data into a series of probability distributions and data is made to fit into predictive models that can be used to refine classification schemes. These probability distributions, models and categorized data sets can be applied to nonlinear regression techniques that attempt to transform observed patterns into potential hypotheses.

18

The third major step in the machine learning process involves the formulation of a model in an attempt to explain the phenomena, classification categories and proposed hypotheses. Nonlinear regression again can be used, as well as a host of various mathematical analysis tools such as trainable Markov Random Fields and Bayes Networks. These analysis tools help interpret data and analyze patters [32]. The goal is to label data points as instances of interesting or non-interesting phenomena so that only relevant data points are used and analyzed when formulating or fitting a model. Labeling data as either interesting or non-interesting involves the use of generative and discriminative models. Generative models create a union of the various attributes of a dataset in an attempt to define the underlying physical phenomena, while discriminative models try to make a distinction between interesting and non-interesting data points. The generative model can be considered as a bottom-up technique while the discriminative model merely looks at the “whole picture” without concern for the underlying attributes.

At this point, classification schemes have been developed, a model or multiple models have been proposed as a response to explain the classification schemes and phenomena, and additional data sets and data points are acquired to test the validity of the model. This fourth step of the machine learning process is concerned with determining what predictions to make and what data to gather to test these predictions. Several data point selection criteria have been proposed over the years, including choosing the data point for which the current machine learning model is least certain of its classification; this is an attempt to collect and label the “most” random data point.

The final step of the machine learning method involves making the decision whether to refine, refute, or to repeat the hypotheses and model predictions. In essence, the machine learning process now attempts to answer, “Is there enough evidence to refute or confirm a hypothesis? If no, should the hypothesis production and model development protocols be repeated? If yes, should the hypothesis be included into the database of knowledge?” At this stage are several issues concerned with the use of static lower and upper bounds. A predetermined hard-coded sentinel value may be used to determine if a certain high or low-enough characteristic of a data model or dataset is significant enough to merit rejection or acceptance of a hypothesis or prediction. Sentinel values and hard-coded trigger thresholds can be used as confidence markers assuming that future models will produce results similar to previous models and model parameters.

Many programs that use machine learning protocols are being developed or are already in use within medicine, mathematics and space exploration. The 3-D Computer-Automated Threshold Amsler Grid Test performs real-time medical analysis that can aid in the diagnosis of eye diseases. The standard Amsler Grid Test is administered to a patient with the aid of a touch-sensitive computer screen. The patient’s responses are used as input for a modeling protocol that simulates successive Amsler Grid Tests at various grayscale levels. The use of additional grayscales allows scotomas (portions of the retinal field that are non-functional) to be depicted in 3-dimensions as opposed to 2-dimensions, as is the case when using the standard Amsler Grid Test. The output of the program is a depiction of a patient’s visual field [11]. The Cellerator project uses the Mathematica program package. Equations that model biological compounds are

19

generated. Ordinarily, chemical networks must be manually translated from cartoon-like diagrams to chemical equations and fitted to ordinary differential equations [13]. The manual translation process is often a tedious task that is hard to automate because of the many variations and configuration states of different biological elements. The Cellerator program automatically translates chemical networks into ordinary differential equations, which are more easily modeled. Once the Cellerator program generates and translates chemical equations into ordinary differential equations, various components of the ordinary differential equations are used to model and eventually solve for various chemical interactions [12].

The Diamond Eye project is concerned with the cataloging and identification of geological features for scientific analysis. The amount of scientific data that is acquired grows with the launch of each scientific mission because of continued improvements in data-gathering techniques. Filtering and on-board autonomous analysis capabilities allow an autonomous agent to perform some of the analysis that otherwise would have to be done by hand, a very time-consuming process [20]. The architecture of the Diamond Eye project is based on a distributed software methodology where scientists can interact with several data depositories [14,18]. An adaptive recognition algorithm mathematically processes the low-level pixels of an image and uses several training-models in an effort to construct recognizer features. Spatial structures, various object templates and temporal analysis tools are also used.

3.2 Neural Networks

The concept of artificial neural networks is inspired by biological nervous systems, is concerned with information processing, and is part of the machine learning field. Artificial neurons have one or more inputs, generally one output and include two main “modes”, training and production. Each input has an associated weight, which helps determine the “importance” of the nature of the data of the input. Training involves the giving of information to a neural network in an effort to determine input weights and is usually done in one of two ways, supervised or unsupervised. The supervised training method involves providing both the output and inputs to a neural system and then using back propagation in an effort to adjust the input weights; the amount of errors between the desired and actual output are analyzed. During unsupervised—or adaptive training—a neural network is not provided with the output, so the network must determine which features are important in the input data set and how various features can be grouped and analyzed so that effective input weights can be set. Multi-layer neural network architecture, error detection and modification, data reduction and several other issues are pertinent to effective neural networks and help further refine and enhance neural networks.

There are several advantages of using a neural network, one of which is for the purpose of pattern recognition that can lead to effective data organization methods. In the case of unsupervised learning, organization is performed without any a priori knowledge; patterns in multi-dimensional space can be transformed to lower dimensional space that can be more easily ordered [15].

20

4. Autonomous Agent Planners

The planning software for an autonomous space agent generates mission plans so that onboard science can be scheduled and performed, so that useful data can be acquired, and so that mission objectives can be met. The planner in an autonomous space agent must manage resources and must be able to resolve conflicts between different subsystems. In addition, the planner must produce commands that are valid, are not redundant, are rational and adhere to flight rules. As is the case with the overall architecture of an autonomous agent, there does not exist a recipe for a “perfect” autonomous planner.

This section includes an overview of planner functions, batch planners, dynamic planners, and Hierarchical Task Network (HTN) planners. The Automated Scheduling and Planning Environment (ASPEN) and the Simple Hierarchical Ordered Planner (SHOP) are also discussed.

4.1 Coordination of Subsystems

Different subsystems of a space agent require instructional commands over the course of a mission. Some components of an autonomous spacecraft may be turned “on” for only brief amounts of time, in which case appropriate command sequences must be sent to those components at the right moment. An example is the deployment of a heat shield when a spacecraft enters the atmosphere of a planet—the deployment of the shield must be timed just right so that that spacecraft is not damaged. Other subsystems of an autonomous agent may be functional throughout the entire mission and may require updated commands on a periodic basis. A navigation system, for example, must be operational at all times and must receive data from orientation sensors. Still other components may be turned on only in response to special events. If a spacecraft detects an anomaly, a camera can be turned on to gather information so that science data may be returned to Earth for analysis.

Because a spacecraft contains several subsystems, the planner must be able to coordinate many tasks. Some sequences of events are valid only when performed in a special order. For example, if a spacecraft approaches an anomalous object, the order in which different actions are performed is very important:

1) Turn thrusters off2) Turn camera on3) Take 30 images in 30 seconds4) Turn camera off5) Turn thrusters on6) Resume previous flight path.

If the actions are performed out of sequence, then the camera might not record the anomalous object. Even worse, if the wrong actions are performed, then the spacecraft may collide with the anomalous object and become inoperative.

21

In order for the planner to produce effective plans, the planner must take into consideration a variety of factors. Different subsystems can directly or indirectly affect other subsystems, so the planner must anticipate—and potentially resolve—possible conflicts. Each command that is generated by the planner must be valid and must conform to pre-defined rules, if any. The planner must also make sure that plans do not violate flight objectives, and the planner must make sure that plans are not redundant. The planner must also rational, meaning that the planner arrives at the right solution for the right reasons.

4.2 System Languages and Planner Logic

The ability of the planner to validate and check a command sequence is highly dependent on the underlying system language and the way that logical statements are represented. The system language must be able to easily and effectively express command sequences, flight rules, the states of different subsystems, and the contents of databases. In order for the planner to check and validate different commands, logical constructs must be allowed so that a variety of statements can be verified. If the camera on board the spacecraft can be turned on only when the spacecraft burners are turned off (the combined energy requirement of the two components may be more than is available at any one time), then there must be a way for the system language to represent that requirement. Similar axioms, corollaries, and logical facts must be represented, as well as while-loops, if-statement and other programming constructs. The ability of the system language to combine, manipulate, validate, classify, order, etc. a variety of statements is through the use of logic [27]. The sum of all logical statements can be used to define a model of a spacecraft’s subsystems, and this is how knowledge can be represented; simple facts are combined and relationships are made to form complex ideas and expressions. Most importantly, the use of logical statements and logical operators allows command sequence to be validated. Conditionals and iterations of command sequences can be represented using logical constructions and using an effective modeling language [29].

The actual way in which logical statements are represented in an autonomous agent depends on the type of logic that is used. Logic, itself, stems from the natural language constructs and generally includes two types, predicate and prepositional logic. Both predicate and prepositional logics allow for meaningful relationships to be assigned, but the two types differ in regards to the use of sentinels and predicates. Prepositional logic allows for reasoning about items only in terms of “true” and “false” values, but predicate logic allows reasoning about true and false statements and about individuals; i.e., a statement can be quantified.

In prepositional logic, a sentinel is an operator that operates on one or more complete ideas to form a new aggregate idea. Truth function operators determine the truth of a statement from the truth values of the individual components of the statement. For example, the statement “the sky is blue” can be combined with the statement, “ostriches don’t fly”, to form the sentence, “The sky is blue and ostriches don’t fly”. This sentence is a construct of the two statements and is called a conjunctive construct because the word “and” is used. The truth of the sentence can be ascertained if, and only if, each of

22

the two statements is true. Aside from the conjunctive construct, there are the disjunction and negation prepositional constructs, as well as a range of constructs that can be formed using boolean operators such as, “if, and, or, nor, implies,” etc. The ability to combine true or false statements into aggregate ideas is the underlying concept of prepositional logical—it is assumed that all propositions have a definite truth value, namely either true or false.

Predicate logic, however, is concerned not only with sentinel connectives but also with the actual structure of atomic prepositions. A predicate is whatever is said of the subject of a statement; a function that maps individuals to truth-values [30]. Atomic sentences are constructed by applying predicates to individuals, and this is how unique individuals can be quantified. For example, the two statements, “all men are mortal,” and “Socrates is a man”, can be combined to form the sentence “Socrates is mortal”. Predicate logic, then, deals with prepositions where subject and predicates are separately signified. There are several types of predicate logics, including first-order, higher-order, inclusive, monadic and polyadic predicate logics, each of which is different depending on what arguments are acquired by the predicate. First-order logic, for example, is where predicates take only individuals as arguments and quantifiers only bind individual variables; the “Socrates is mortal” example is a first-order predicate logic.

4.3 Batch Planners

A batch planner—often considered the “traditional” model—divides a mission into a series of planning horizons, where each horizon lasts for a predefined amount of time and during which a plan is set into action. At each instance when the mission timeline approaches one of these planning horizons, the planner projects what the state of the agent will be at the moment when the current plan expires. The planner generates a new plan for the next plan horizon, taking into account the expected end state of the current plan and the goals of the mission.

Figure 1: Batch Planner. Plans are joined end-to-end, and each plan must run to completion before the next plan is implemented; the plans are separated by plan horizons. Before each plan horizon, the planner users the current state values to project the state of the agent at the completion of the current plan. This projection is used as the starting point for the next phase. [16]

This traditional planning model has the advantaged of being able to determine the length of time between each event horizon, and so at the time of actual plan generation, the planner routine knows exactly how much time there is remaining, and hence continued revisions can be made to the plan routine until the event horizon occurs. The

23

traditional planning model, however, also has several limitations, including the necessity for dedicated resource, an inability to produce a quick new plan and an overall inability to produce the most effective plan.

The batch concept of the traditional planner requires that the planning phase be an off-line process, where the planning routine is invoked only when a time horizon approaches. This off-line scenario means that if system resources are limited, then the planner cannot be invoked. Alternatively, the planner may be allotted dedicated system resources, but then at the time when the planner is not running, those dedicated system resources cannot be accessed and are left unused.

If an anomalous event occurs—either positive or negative—then the response time may be significant due to the fact that a new plan can be implemented only at the next plan horizon, which may not occur for a significant amount of time. A negative event may require an immediate response, and a positive event may be a short-lived science opportunity event during which important science information could be collected. Because the next plan horizon may be several moments away, a fortuitous opportunity for data acquisition may be passed by because a new plan cannot be implemented in a short enough time.

The traditional planning model also may not be able to produce the most efficient plan if the planner is especially slow or if the planner routine must be initiated far in advance of the plan horizon. Because the batch planning method tries to project the most likely state at the end of the current batch time phase, an event or change in environmental variables that happens after the start of the planner but before the next time horizon will not be taken into consideration by the planner routine. The projection of the end of the current time phase may be grossly wrong if the planning algorithm starts well in advance of the time horizon. Alternatively, the projection of the state at the end of the current time phase cannot be calculated too close to the plan horizon because the planner must be given enough time to construct a plan that is consistent with mission goals.

4.4 Dynamic Planners

A dynamic/responsive planner, on the other hand, maintains a current goal set, a plan, a current state and a model of the expected future state. The dynamic/responsive planner can update the goals, current state or a plan horizon at any time, and the state of the current plan is altered and the planner process is invoked. An update may be a host of events or a malfunction. The dynamic/responsive planner is able to maintain a satisfactory plan because the most current sensory and goal integration data is used.

24

Figure 2: Continuous Planner. Instead of several plans joined end-to-end, there is only a current plan that is repeatedly modified in response to the current state of the agent. Goals and state representations are constantly being updated. Instead of waiting for a plan horizon, the planner makes constantly updates the plan [3].

A dynamic/responsive planner integrates a new plan by means of a simplified cycle where the goals and initial state of the current plan are appropriately updated, the effects of the changes to the initial state and goals are propagated through the entire current plan, potential conflicts are identified, and plan repair algorithms remove anticipated conflicts. Conflict resolution involves the tracking of a host of agent systems, among them the communications, science data acquisition and engineering activities as pertaining to any changes that have been made to the plan. The dynamic/responsive planner therefore has a big advantage over the batch planner because changes to the plan can be made immediately.

4.5 Hierarchical Task Network Planners

Hierarchical task network (HTN) planners aim to decrease planning time by hierarchical decomposition. HTN planners reduce a problem by recursively decomposing tasks into subtasks, stopping only when primitives have been reached. A primitive cannot be decomposed into a simpler unit. Planning operators are used on the primitives to perform various tasks. Sets of available methods that define how different tasks can be decomposed are used; each method provides a schema for decomposing a task into several subtasks. Because there are many ways to decompose a large task (which may be a conjunction of multiple tasks which themselves can be decomposed), several methods can be applied effectively [25].

The input to the planner consists of a task network, a set of operators, and a set of methods—a triplet. The task network is the entire problem the needs to be solved, where each task is one specific thing that needs to be done. A task has a name and a list with arguments that include variables, constants, and various attributes. Task and network attributes may include constraints that restrict or prevent some of the variables to be used

25

or constraints that require that a series of tasks be performed in a certain order. Tasks can be primitive, meaning that they can be performed directly; tasks can be compound, in which case the planner must decide how to decompose the tasks; or tasks can be goals, in which case they are just properties that must be made true. Available operators enumerate the effects of each of the primitive tasks. Methods indicate how to perform various non-primitive tasks and are defined as a pair (x,y), where x is a task that is performed on a network y. Because planning problems (the triplets) are define mathematically, restrictions, reductions and various comparisons can be performed effectively using the available operational semantics. A planning domain is defined as D=<Op, Me>, where OP is a set of operators and Me is a set of methods. The planning problem, therefore, is defined as P = <d, I, D>, where D is the planning domain, I is the initial state, and d is the task network that the plan needs to solve for. Using HTN planners, the overall planning process can be summarized as follows:

1. Receive the problem P2. If P is comprised of all primitives, then

a. Resolve conflicts in Pb. If Resolved, return Pc. Else, Return Failure

3. Choose a non-primitive task t in P4. Choose a decomposition for t5. Replace t with decomposition6. Use critics for find interactions and find resolutions7. Apply the resolutions8. Return to Step 2

Critics (Step 6) are functions that handle ordering constraints, resource limits and provide domain-specific guidance in the case that a planner has been designed for a specific job and tailored algorithms/methods have been developed. Step 2 of the overall process either returns a plan where all the primitives have been resolved (operators have been used successfully on all the tasks), or a failure is returned because the plan cannot be solved because an operator does not exist for a primitive task).

There is a drawback to the use of HTN planners, however. When large complex initial tasks are used and when interactions among non-primitives are complex, then the planner may not be able to find solutions, especially if subtasks in the expanded, non-decomposed, task list are interleaved. In such complex cases, HTN tasks are said to be “undecidable”.

4.6 The Automated Scheduling and Planning Environment (ASPEN)

The ASPEN planning and scheduling program is a re-configurable framework that can support a wide variety of applications. High-level goals are translated into low-level commands that help achieve the objectives of a particular mission. It is the goal of the ASPEN project to reduce mission costs and eventually permit scientists to directly issue commands to a spacecraft. Rather than be dependent on mission personnel to provide

26

command sequences, a spacecraft will receive mission goals from a scientist and in response autonomously generate a plan and schedule sequence. Under an automated scheduling and planning system, opportunistic and short-lived events can be effectively monitored.

The ASPEN software provides an expressive constraint modeling language, a management system that is used for the maintenance of spacecraft operations and resources, a host of search strategies, a reasoning system that is used for maintaining temporal constraints, a language for representing plan preferences, various graphical interfaces, and real-time re-planning capabilities [17]. Knowledge is stored as several classes, including activities, parameters, temporal constraints, reservations, resource variables, parameter dependencies and state variables. Different knowledge constructs are used to define system components that are used to produce sequences of commands.

The architecture of the ASPEN system uses iterative algorithms, heuristics, local algorithms and parameters. Iterative algorithms permit re-planning to be used at any time (in contrast to the batch planning protocol that is described above), which is one large advantage in the case that anomalous events or short-lived opportunistic science observations are expected. The use of heuristics allows for pruning of search trees and knowledge spaces and may allow for a quick discovery of a higher quality solution [18]. Local algorithms do not have a computational overhead associated with intermediate plans or previous failed plans. Consequently, local algorithms do not guarantee that unsuccessful modifications to a plan will not be retried. The use of parameters and the adhering to such parameter constraints (in contrast to least-commitment techniques) allows for resource values to be easily computed [17].

After a model is developed, ASPEN parses it into data structures that enable efficient reasoning capabilities, where seven basic components are used. Parameters are used to store simple variables and are used in parameter dependency functions. Dependencies are represented and maintained in a Parameter Dependency Newtork (PDN) which maintains all dependencies between parameters; at any given time all of the dependency relationship can be checked to ensure that relations are satisfied[18]. Temporal constraints are used to define relationships between the start and end times of two different activities. Temporal constraints allow for the derivation of complicated expressions, especially when used with conjunctive and disjunctive operators. Resources are profiles that represent actual system resources or variables and permit the use of restrictions. State variables describe the possible values of a system variable over time, as for example Busy, Idle, Corrupt. Reservations allow for activities to have resource usage constraints, and can be modified, turned on or turned off depending on the need to regulate activities. Finally, activity hierarchies allow for the breaking up of a task into a series of sub-activities. The activity hierarchies are efficient in mandating the order in which tasks must be performed, including configurations in series or parallel. The use of these seven components in a variety of combinations allows for the design of plans and the repair of plan conflicts.

27

ASPEN defines ten basic types of conflicts, including abstract activities, unassigned parameters, violated parameter dependencies, unassigned temporal constraints, violated temporal constraints, unassigned reservations, depletable resources, non-depletable resources, state requirement conflicts and state transition conflicts [19]. An abstract activity conflict occurs when a command has not been decomposed into the appropriate sub-commands or if there exist several possible command decompositions, in which case an algorithm must decide which decomposition should be performed. An unassigned parameter conflict represents the condition when a parameter has no value but has been included in a plan; a parameter can be unassigned, but once part of a plan, a parameter must have a concrete value. When two parameters violate a functional relationship, a parameter dependency violation occurs. Parameter dependencies must be constantly checked because of continued updates to parameters. When a temporal constraint exists for an activity instance that has not been selected to satisfy that constraint, then an unassigned temporal constraint conflict occurs. A violated temporal constraint conflict occurs when a temporal constraint is assigned to a relationship that does not hold; this prevents for the setting of constraints that are potentially impossible to maintain. An unassigned reservation conflict occurs when there exists a reservation in an activity that has not yet been assigned to a resource. Timeline conflicts address issues of the use of depletable and non-depletable resources. An upper or lower bound limit is set for most variables, and the use of resources is closely monitored so that any one spacecraft component does not exceed the allotted use of a resource. Timeline conflicts are generally the more difficult to recover from because the solution may require that many components of a system be adjusted. Finally, state variable conflicts result when either a reservation mandates the use of a state that is not available, in which case a state requirement conflict occurs, or when a reservation is changed to a condition that is not allowed by a state variable, in which case a state transition conflict occurs.

ASPEN performs iterative repair searches while taking into consideration different constraints. Different types of constraints are organized into classifications depending on how a constraint can be violated. The different violation types have appropriate repair methods, and the search space includes all permissible repair methods as applicable to all possible conflict types in all possible combinations. The iterative repair algorithm searches the space of schedule components and makes decisions at key instances. Choices are made when one of several elements must be selected, including a conflict, a repair method, an activity for a repair method, a start time for an activity, a duration for an activity, a timelines for a reservation, a decomposition, a change in a parameter or when selecting a value for a parameter.

The continuous planning algorithm receives a current plan, a current goal set, and a current state. The current goal is updated to reflect new goals and conflicts are detected. When a schedule contains several conflicts, the iterative repair algorithm selects a conflict to attack and chooses a repair method. Because the conflict search space contains pairs of conflicts and associated repair methods, a search for the conflict also returns an appropriate repair method(s). There are many possible classes of repair methods, including moving an activity to a different location in the plan, creating a new activity, creating a reservation, canceling a reservation or deleting an activity. During the

28

decision stage, the heuristics feature of the ASPEN system helps to pruning the tree when several solution methods exist. Several domain-independent heuristics have been developed, including a heuristic for the sorting of conflicts according to type, a heuristic for selecting the repair method when more than one repair method exists, and a heuristic for determining start and end times of activities that are being shuffled to different locations.

The iterative planning algorithm releases appropriate non-conflicting sections of a plan to an executive for execution. During the entire iterative planning process, the state of the spacecraft is represented by a series of timelines that portray the current and potential future states of the system. The algorithm continually updates the timelines in response to new events and actual plan consequences. Most importantly, the planning algorithm tries to project the state of the system only into the near future; projections for the long term are generally very abstract so that modifications can be made easily [21].

4.7 The Simple Hierarchical Ordered Planner (SHOP)

The AI Planning group at the University of Maryland is designing the Simple Hierarchical Ordered Planner (SHOP). Ordered Task Decomposition, which is a special case of Hierarchical Task Network (HTN) planning, has been developed. Using Ordered Task Decomposition, a planner generates tasks in the same order that the tasks will later be executed.

The syntax and semantics of the SHOP planning system define logical symbols, logical inferences, tasks, operators, plans, methods, domains and problems; first-order logic is used. Logical symbols include constant, function, predicate and variable symbols, as well as terms, atoms, ground atoms, conjuncts of atoms, horn clauses, substitutions and most-general unifiers. Constant symbols are “individuals”, as for example, “Susan”, and “3”. Function symbols map individuals to other individuals, as for example, “age of (Susan) = 3”. Logical inference is through the use of states, axioms and satisfiers. A state is defined as a list of ground atoms, while an axiom is a set of horn clauses. (A horn clause is a clause containing at most one positive literal; for example, “has_wings(bird)”, indicating that a bird has wings). Satisfiers are substitutions that help make a conjunction true. Tasks are divided into two types, primitive task symbols and non-primitive task symbols.

A task is of the form

(s t1 t2 … tn)

where s is the task’s name and t1, t2, …, tn are the terms which define a task’s arguments. A task list is defined as a list of tasks, either primitive or non-primitive.

Operators are expressions of the form

(: operator h D A c)

29

where h is a primitive task, D and A are lists of atoms with no variables, and c is a number, which is the cost of the primitive task h. The operator specifies that h can be accomplished if every atom in the list D is removed and placed into the list A; hence the operator is an operation that defines a “before” and “after” state. A plan is defined as a list of operator instances [31].

Methods are of the form:

(: method h C ‘T)

where h is a task, C is a conjunct (a precondition), while T is a task list.

Domains and problems are represented as a set of axioms, operators, and methods. Planning problems are triplets, (S,T,D), where S is the state, T is a task list, and D is a representation of a domain. For example, consider the following definitions for State S, operator o, the substitution u, and the plan P:

S = ((on a b) (ontable b) (clear a) (handempty));

o = (:operator (!unstack ?x ?y)((clear ?x) (on ?x ?y) (handempty))((holding ?x) (clear ?y)));

u = ((?x . a) (?y . b));

P = ((!unstack a b) (!putdown b));

Lisp is the programming language that is used in SHOP, but no in-depth knowledge of Lisp is necessary to understand the example. The state of the system is represented by 4 statements, namely that “object a is on object b”, [(on a b)]; “b is on the table”, [(ontable b)]; “there is nothing on top of a”, [(clear a)]; and “the hand is empty”, [(handempty)]. The operator o is a function that “unstacks” objects x and y [(!unstuck ?x ?y)], first making sure that object x is on top of objects y [(on ?x ?y)], that the hand is empty [(handempty)], and that there is nothing on top of object x [(clear ?x)]. The hand in this case can be a robotic hand, and hence must be empty if the robot is to pick up the object x. Likewise, there should be nothing on top of object x. Note that the use of question and explanation marks is not analogous to the use of these items when used in standard written languages; in this case the ? indicates that x and y are objects, while ! indicates that unstack is a maneuver. These checks that the operator function performs are the pre-conditions, meaning that they must be true if the operator is to be successful. The final line of the o operator is the “result” component, indicating the end state after the operator has performed the action. The substitution u specifies that objects x and y can acquire variable names of a and b; this allows the example to relate an operator to the real examples in this case, namely objects a and b as defined in the state S. The plan P is composed of two statements, “unstack objects a and b”, [(!unstack a b)] and “put object b

30

down”, [(!putdown b)]. The subgoal unstack is performed first, which would result in the following:

(o)u = (!unstack a b)

indicating that the substitution u was applied to the operator o, taking into consideration the current state S. The unstack routine would then be performed, which would first check that there is nothing on top of a, a is on top of b, and that the hand is empty. Although this is a simple example, it demonstrates how a language can be used to represent a state as well as define operations. Note also that first-order logic is evident here in the precondition states of the operator o and the state S: (clear ?x), (on ?x ?y) (handempty) must all first be true in order for the operator to function, while (on a b) (ontable b) extends the definitions of object entities x and y as refereeing to the specific examples of objects a and b.

The SHOP algorithm can be summarized as follows:

procedure SHOP(S,T,D)1. if T = nil then return nil endif2. t = the first task in T3. U = the remaining tasks in T4. if t is primitive and there is a simple plan for t then5. nondeterministically choose a simple plan p for t6. P = SHOP(result(S,p),U,D)7. if P = FAIL then return FAIL endif8. return cons(p,P)9. else if t is non-primitive and there is a simple reduction of t in S then10. nondeterministically choose any simple reduction R of t in S11. return SHOP(S,append(R,U),D)12. else13. return FAIL14. endifend SHOP

where S is the state, T is a task list, and D is a representation of a domain.

31

5. Future Work

In this survey report are included several unresolved issues and general concerns of the design and use of autonomous agents in space missions. Section 2.7 of this report lists several design and implementation issues for a few autonomous agent technologies and several autonomous agent space missions. Specific issues can be addressed with more detail, different autonomous technologies can be implemented in an actual robot, and general autonomous behavior techniques can be modified and made compatible with resources available on autonomous spacecraft. Discussed in this section are three possibilities for future work.

The image analysis tools discussed in this survey report explain several limitations of the use of object detection tools in autonomous agents. The primary obstacles in implementing image analysis tools onboard spacecraft are related to the availability of resources. Image analysis algorithms use mathematical modeling, statistical analysis functions and a variety of image acquisition and storage features—all of which require a great amount of computational power, time, and energy. An image analysis tool can be very effective when implemented on a computer system with unlimited access to a fast multi-processor, a large database, and an unlimited energy source. In comparison, a spacecraft has access to very limited resources. Space flight logistics, a limited source of energy and a limited amount of computational power are all features of spacecraft that limit the extended use of powerful image analysis tools. Future work could focus on the scalability of these powerful ground-based image analysis tools and on the development of comparable algorithms that require fewer resources. Different aspects of current image analysis tools could be optimized and new functions could be developed. Related to the development of such new tools are systems that can model space environments and provide a “virtual” test platform. Image analysis tools could be modified or developed, and then tested within an environment that mimics the conditions of space.

Several of the planner features discussed in this paper could be implemented in an autonomous robot with simple path-planning capabilities. As is the case when image analysis tools are integrated into an autonomous agent, adding path-planning capabilities to a robot would require the modification of computer code in response to the resources of the robot. Energy resources, sensors, the planning module and a command executive must all be integrated to function in collaboration. The robot could be tested in a mock alien terrestrial surface to ensure that the planner functions properly.

A detailed analysis of existing planners could ascertain whether different planning programs are scalable for use in autonomous space missions. Different features of a planner could be analyzed, including the ability of the planer to produce valid sequences, the ability of the planner to quickly generate command sequences in response to anomalous events, and the ability of the planner to recover from malfunctions. After various planners are analyzed, similarities and differences of the different benchmark studies could be used to propose a list of features and abilities that are required by a space-worthy autonomous planner.

32

6. References

1. The MISUS Multi-Rover Project, website, http://www-aig.jpl.nasa.gov/public/msl.

2. B. Pell, D.E. Bernard, S. A. Chien, E. Gat, “An Autonomous Spacecraft Agent Prototype”, ACM, pg 253-261, 1997.

3. S. Chien, B. Engelhardt, R. Knight, G. Rabideau, R. Sherwood, E. Hansen, A. Rotiviz, C. Wilklow, S. Wichman, “Onboard Autonomy on the Three Corner Sat Mission”, Proceedings of the 7 th Symposium on Artificial Intelligence, Robotics, and Automation in Space (I-SAIRAS 2001), Canadian Space Agency, Montreal, 2001.

4. Autonomous Small Planet In situ Reaction to Events (ASPIRE) Project, website, http://www-aig.jpl.nasa.gov/public/mls/aspire/aspire.html.

5. P.G. Backes, G. Rabideau, K.S. Tso, S. Chien, “Automated Planning and Scheduling for Planetary Rover Distributed Operations”, Jet Propulsion Laboratory, California Institute of Technology.

6. Casper: Space Exploration through Continuous Planning, IEEE Intelligent Systems, September/October 2001.

7. R. Castano, T. Mann, E. Mjolsness, “Texture Analysis for Mars Rover Images”, website, http://www-aig.jpl.nasa.gov/public/mls/mls_papers.html.

8. R. Congalton, “A Review of assessing the accuracy of classifications of remotely senses data”, Remote Sensing of Environment, Volume 21, Issue 1, 1991.

9. T.J. Ellis, M. Mirmehdi, G.R. Dowling "Tracking image features using a parallel computional model" ,Technical report TCU/CS/1992/27, City University, Department of Computer Science, 1992.

10. E. Mjolsness and D. Decoste, “Machine Learning for Science: State of the Art and Future Prospects”, Science, 293, pp. 2051-2055, September 2001.

11. The 3-D Computer-Automated Threshold Amsler Grid Test, website, http://www-aig.jpl.nasa.gov/public/mls/home/wfink/3DVisualFieldTest.htm.

12. B. Shapiro and E. Mjolsness. (2001) Developmental Simulations with Cellerator, Second International Conference on Systems Biology, November 2002.

13. Cellerator, Jet Propulsion Laboratory, California Institute of Technology, website, http://www-aig.jpl.nasa.gov/public/mls/cellerator/

33

14. J. Roden, M.C. Burl, C. Fowlkes, “The Diamond Eye Image Mining System”, Demo for Scientific and Statistical Database Management Conference, Cleveland, Ohio, June 1999.

15. A.P. Clare and D.R. Cohen, “An unsupervised neural network approach to the analysis of multi-element stream sediment data”, University of New South Wales, Sydney, Australia.

16. S. Chien, R. Sherwood, M. Burl, R. Knight, G. Rabideau, B. Engelhardt, A. Davies, “The Techsat-21 Autonomous Sciencecraft Constellation”, Proceeding of the 6th International Symposium on Artificial Intelligence and Robotics & Automation in Space, Canadian Space Agency, St-Hubert, Quebec, Canada, June 18-22, 2001.

17. S. Chien, G. Rabideau, R. Knight, R. Sherwood, B. Engelhardt, D. Mutz, T. Estlin, B. Smith, F. Fisher, T. Barrtt, G. Stebbins, D. Tran, “ASPEN – Automated Planning and Scheduling for Space Mission Operations”, Jet Propulsion Laboratory, California Institue of Technology.

18. A. Fukunaga, G. Rabideau, S. Chien, D. Yan, “Towards an Application Framework for Automated Planning and Scheduling”, Jet Propulsion Laboratory, California Institute of Technology.

19. G. Rabideau, R. Knight, S. Chien, A. Fukunaga, A. Govindjee, “Iterative Repair Planning for Spacecraft Operations Using the Aspen System”, International Symposium on Artificial Intelligence Robotics and Automation in Space (ISAIRAS), Noordwijk, The Netherlands, June 1999.

20. M.C. Burl, C. Fowlkes, J. Roden, A. Stechert, S. Mukhtar, “Diamond Eye: A Distributed Architecture for Image Data Mining”, SPIE AeroSense Conference on Data Mining and Knowledge Discovery, Orlando, Florida, April 1999.

21. S. Chien, R. Knight, A. Stechert, R. Sherwood, G. Rabideau, “Using Iterative Repair to Increase the Responsiveness of Planning and Scheduling for Autonomous Spacecraft”, IJCAI99 Workshop on Scheduling and Planning meet Real-time Monitoring in a Dynamic and Uncertain World, Stockholm, Sweden, August 1999.

22. V. Gor, W. Colwell, P. Stolorz, B. Merline, T. Mann, “Distant Autonomous Recognizer of Events (DARE) As An On-Board Science Processing System for Spacecraft”, website, http://www.aig.jpl.nasa.gov/public/home/gor/publications/AIAA99b/AIAA99b.html.

23. The NASA Mars Pathfinder Website, http://mars.jpl.nasa.gov/default.html.

34

24. A. G. Davies, et. al, Automating Operational And Science Analysis Processes with the Autonomous Sciencecraft Constellation, Lunar and Planetary Science, XXXIII, 2002.

25. K. Erol, J. Hendler, D. Nau, “HTN Planning: Complexity and Expressivity”, Computer Science Department, University of Maryland.

26. D. Nau, S. Smith, K. Erol, “Control Strategies in HTN Planning: Theory Versus Practice”

27. D. Nau, H. Munoz-Avila, Y. Cao, A. Lotem, S. Mitchell, “Total-Order Planning with Partially Ordered Subtasks

28. J. Funge, X, Tu, and D. Terzopoulos, “Cognitive Modeling: Knowledge, Reasoning and Planning for Intelligent Characters”, SIGGRAPH 99, Los Angeles, CA, August 11-13, 1999.

29. J. Funge, “Solving Practical Problems in Knowledge Representation with Interval Arithmetic”, Interval '98, Nanjing, China, April 20-23, 1998

30. Website for Professor Peter Suber, http://www.earlham.edu/~peters, Logical Systems course webpage.

31. D.S. Nau, Y. Cao, A. Lotem, and H. Muñoz-Avila. "SHOP and M-SHOP: Planning with Ordered Task Decomposition." Tech. Report CS TR 4157, University of Maryland, College Park, MD, June, 2000.

32. J. Webb "High performance computing in image processing and computervision", International Conference on Computer Vision and Pattern Recognition, October 1994.

35