ecmr11 proceedings

Proceedings of the 5th European Conference on Mobile Robots ECMR2011September 7-9, 2011 rebro, Sweden

Editors: Achim J. Lilienthal Tom Duckett

September 5, 2011

Achim J. Lilienthal AASS Research Centre School of Science and Technology rebro University SE-70182 rebro Sweden [email protected] Tom Duckett Lincoln School of Computer Science University of Lincoln LN6 7TS Lincoln United Kingdom [email protected] ECMR11 Proceedings of the 5th European Conference on Mobile Robots September 7-9, 2011 rebro, Sweden

5th European Conference on Mobile Robots, September 7-9, 2011, rebro, Sweden

ii

CommitteeGeneral Chair: Achim J. Lilienthal Program Chair: Tom Duckett Conference Board: Adam Borkowski Wolfram Burgard Primo Zingaretti Local Organization: Monica Wettler (Conference Coordinator) Barbro Alvin (Local Organization) Per Sporrong (Local Organization) Marcello Cirillo (Local Organization) Branko Arsov (Finance Organization) Program Committee: Kai Arras Alexander Bahr Juan Andrade Cetto Antonio Chella Marcello Cirillo Mark Cummins David Filliat Udo Frese Javier Gonzalez Horst-Michael Gross Dirk Holz Patric Jensfelt Maarja Kruusmaa Lino Marques Stefan May Ivan Petrovi c Libor Preucil Alessandro Safotti Antonio Sgorbissa Cyrill Stachniss Rudolph Triebel Jasmin Velagic

Tamim Asfour Nicola Bellotto Raja Chatila Grzegorz Cielniak Andreu Corominas Murtra Robert Cupec Thierry Fraichard Emanuele Frontoni Roderich Gross Dongbing Gu Luca Iocchi Dermot Kerr Martin Magnusson Toms Martnez-Marn Emanuele Menegatti Cedric Pradalier Dario Lodi Rizzini Thomas Schn Piotr Skrzypczy ski n Adriana Tapus Andrzej Typiak Markus Vincze


iii

Sponsors

ABB Volvo Construction Equipment Robot Dalen Atlas Copco


iv

Welcome MessageWe are honoured to welcome you to the 5th European Conference on Mobile Robots ECMR 2011, held in the city of rebro, Sweden on September 7-9, 2011. ECMR is a biannual European forum, internationally open, allowing researchers throughout Europe to become acquainted with the latest accomplishments and innovations in advanced mobile robotics and mobile human-robot systems. The rst ECMR meeting was held in September 2003 in Radziejowice, Poland, followed by ECMRs in September 2005 in Ancona, Italy; in September 2007 in Freiburg, Germany; and in September 2009 in Mlini/Dubrovnik, Croatia. A priority of ECMR is to attract young researchers to present their work to an international audience. ECMR is organized in single track mode to favour discussions. ECMR 2011 will continue this policy and will provide panel sessions and original presentations about research in progress. ECMR 2011 has continued to build on the success of the previous conferences, reaching a level of maturity that is reected in the quality of the technical program. Each of the 71 papers submitted was evaluated by three reviewers and 51 of them (from 19 different countries and 170 authors) have been accepted by the Program Committee. These papers are included in these proceedings and will be presented at the conference. They cover a wide spectrum of research topics in mobile robotics: 3D perception, navigation, path planning and tracking, SLAM, mapping and exploration, human-robot cooperation, various service applications, etc. We are especially proud to welcome our distinguished keynote speakers Professor Per-Erik Forssn from Linkping University, Sweden, who will give the talk Dynamic and Situated Visual Perception, and Professor Markus Vincze from Vienna University of Technology, Austria, who will give the talk Robot Object Classication. The Technical Program also includes a special invited talk by Professor Wolfram Burgard of Freiburg University, Germany, on Robot Control Based on System Identication, a presentation in memoriam of Professor Ulrich Nehmzow. Ulrich was a keen supporter of ECMR (as well as EUROBOT, one of the predecessor conferences to ECMR) and will be deeply missed by many colleagues in the European mobile robotics community. A further invited talk will be given by Torbjrn Martinsson of Volvo CE on Construction Equipment is a Mobile Robotic Market, reecting the strong connections between the academic community and its industrial partners at ECMR. Our sincere thanks are due to all people whose cooperation and hard work made this conference possible. First and foremost, we would like to thank the members of the Organizing Committee and Program Committee for their outstanding work and our sponsors whose support is particularly appreciated. Our special thanks go to the authors for submitting their work to ECMR 2011 and to the reviewers for their time and effort in evaluating the submissions. The results of their joint work are visible in the Program and Proceedings of ECMR 2011. It is now up to all of us to make ECMR 2011 a great success and a memorable event by participating in the technical program, by supporting our younger colleagues, especially students, as they will shape the future of mobile robotics research.

Achim J. Lilienthal Tom Duckett


v


vi

C ONFERENCE P ROGRAMDay 1 Session 1 Shared Environments 1 7 Cipriano Galindo, Javier Gonzlez, Juan-Antonio Fernndez-Madrigal, Alessandro Safotti Robots that Change Their World: Inferring Goals from Semantic Knowledge Arne Kreutzmann, Immo Colonius, Lutz Frommberger, Frank Dylla, Christian Freksa, Diedrich Wolter On Process Recognition by Logical Inference Alper Aydemir, Moritz Gbelbecker, Andrzej Pronobis, Kristoffer Sj, Patric Jensfelt Plan-based Object Search and Exploration using Semantic Spatial Knowledge in the Real World

13

Session 2 Comparative Evaluation 19 25 Todor Stoyanov, Athanasia Louloudi, Henrik Andreasson, Achim J. Lilienthal Comparative Evaluation of Range Sensor Accuracy in Indoor Environments Dirk Holz, Nicola Basilico, Francesco Amigoni, Sven Behnke A Comparative Evaluation of Exploration Strategies and Heuristics to Improve Them

Session 3 Tracking 31 37 Sre ko Juri -Kavelj, Ivan Markovi , Ivan Petrovi c c c c People Tracking with Heterogeneous Sensors using JPDAF with Entropy Based Track Management Aamir Ahmad, Pedro Lima Multi-Robot Cooperative Object Tracking Based on Particle Filters

Session 4 Navigation 43 Bernd Kitt, Jrn Rehder, Andrew Chambers, Miriam Schnbein, Henning Lategahn, Sanjiv Singh Monocular Visual Odometry using a Planar Road Model to Solve Scale Ambiguity Boris Lau, Christoph Sprunk, Wolfram Burgard Incremental Updates of Conguration Space Representations for Non-Circular Mobile Robots with 2D 2.5D or 3D Obstacle Models Maximilian Beinhofer, Jrg Mller, Wolfram Burgard Landmark Placement for Accurate Mobile Robot Navigation

49

55


vii

61

Francesco Capezio, Fulvio Mastrogiovanni, Antonello Scalmato, Antonio Sgorbissa, Paolo Vernazza, Tullio Vernazza, Renato Zaccaria Mobile Robots in Hospital Environments: an Installation Case Study

Day 2 Session 5 Visual SLAM 69 77 John McDonald, Michael Kaess, Cesar Cadena, Jos Neira, John J. Leonard 6-DOF Multi-session Visual SLAM using Anchor Nodes Gerardo Carrera, Adrien Angeli, Andrew J. Davison Lightweight SLAM and Navigation with a Multi-Camera Rig

Poster Spotlight Session 1 Shared Environments, Navigation 83 89 95 101 109 115 121 127 133 139 Abir B. Karami, Abdel-Illah Mouaddib A Decision Model of Adaptive Interaction Selection for a Robot Companion Jonas Firl, Quan Tran Probabilistic Maneuver Prediction in Trafc Scenarios Jens Kessler, Andrea Scheidig, Horst-Michael Gross Approaching a Person in a Socially Acceptable Manner using Expanding Random Trees Amir Aly, Adriana Tapus Speech to Head Gesture Mapping in Multimodal Human-Robot Interaction Hatice Kose-Bagci, Rabia Yorganci, Hatice Esra Algan Evaluation of the Robot Sign Language Tutoring Assistant using Video-based Studies Agustin Ortega, Juan Andrade-Cetto Segmentation of Dynamic Objects from Laser Data Erik Einhorn, Markus Filzhuth, Christof Schrter, Horst-Michael Gross Monocular Detection and Estimation of Moving Obstacles for Robot Navigation Robert Cupec, Emmanuel K. Nyarko, Damir Filko Fast 2.5D Mesh Segmentation to Approximately Convex Surfaces Janusz Bedkowski Data Registration Module - A Component of Semantic Simulation Engine Ernesto H. Teniente, Juan Andrade-Cetto FaMSA: Fast Multi-Scan Alignment with Partially Known Correspondences

Session 6 Perception 145 Sebastian Scherer, Daniel Dube, Philippe Komma, Andreas Masselli, Andreas Zell Robust Real-Time Number Sign Detection on a Mobile Outdoor Robot


viii

153 159 165

Marcel Hselich, Marc Arends, Dagmar Lang, Dietrich Paulus Terrain Classication with Markov Random Fields on fused Camera and 3D Laser Range Data Andrzej Pronobis, Patric Jensfelt Hierarchical Multi-Modal Place Categorization Lus Osrio, Gonalo Cabrita, Lino Marques Mobile Robot Odor Plume Tracking using Three Dimensional Information

Session 7 Planning 171 177 Jan Faigl, Vojt ch Vonsek, Libor Peu il e r c A Multi-Goal Path Planning for Goal Regions in the Polygonal Domain Jrg Stckler, Ricarda Steffens, Dirk Holz, Sven Behnke Real-Time 3D Perception and Efcient Grasp Planning for Everyday Manipulation Tasks

Poster Spotlight Session 2 Perception, Planning, Visual Mapping, SLAM, Applications 183 189 195 201 207 213 219 227 233 239 245 253 Gonalo Cabrita, Pedro Sousa, Lino Marques Odor Guided Exploration and Plume Tracking - Particle Plume Explorer Miriam Schnbein, Bernd Kitt, Martin Lauer Environmental Perception for Intelligent Vehicles Using Catadioptric Stereo Vision Systems Dominik Belter, Przemysaw abecki, Piotr Skrzypczy ski n On-Board Perception and Motion Planning for Legged Locomotion over Rough Terrain Vojt ch Vonsek, Jan Faigl, Tom Krajnk, Libor Peu il e r c A Sampling Schema for Rapidly Exploring Random Trees using a Guiding Path Rainer Palm, Abdelbaki Bouguerra Navigation of Mobile Robots by Potential Field Methods and Market-based Optimization Feras Dayoub, Grzegorz Cielniak, Tom Duckett A Sparse Hybrid Map for Vision-Guided Mobile Robots Marco A. Gutirrez, Pilar Bachiller, Luis J. Manso, Pablo Bustos, Pedro Nez An Incremental Hybrid Approach to Indoor Modeling Hemanth Korrapati, Youcef Mezouar, Philippe Martinet Efcient Topological Mapping with Image Sequence Partitioning Thomas Fraud, Roland Chapuis, Romuald Aufrre, Paul Checchin Kalman Filter Correction with Rational Non-linear Functions: Application to Visual-SLAM Paolo Raspa, Emanuele Frontoni, Adriano Mancini, Primo Zingaretti, Sauro Longhi Helicopter Safe Landing using Vision and 3D Sensing Klas Hedenberg, Bjrn strand Safety Standard for Mobile Robots - A Proposal for 3D Sensors Wajahat Kazmi, Morten Bisgaard, Francisco Garcia-Ruiz, Karl D. Hansen, Anders la CourHarbo Adaptive Surveying and Early Treatment of Crops with a Team of Autonomous Vehicles


ix

Day 3 Session 8 New Design Concepts 259 Shahriar Asta, Sanem Sariel-Talay A Differential Steering System for Humanoid Robots 265 Christian Mandel, Udo Frese Annelid - a Novel Design for Actuated Robots Inspired by Ringed Worms Locomotion Session 9 SLAM 271 Andreas Nchter, Seyedshams Feyzabadi, Deyuan Qiu, Stefan May SLAM la carte - GPGPU for Globally Consistent Scan Matching 277 Andreja Kitanov, Ivan Petrovi c Generalization of 2D SLAM Observability Condition 283 Anssi Kemppainen, Janne Haverinen, Ilari Vallivaara, Juha Rning Near-optimal Exploration in Gaussian Process SLAM: Scalable Optimality Factor and Model Quality Rating 291 Eduardo Lopez, Caleb De Bernardis, Tomas Martinez-Marin An Active SLAM Approach for Autonomous Navigation of Nonholonomic Vehicles Session 10 Localization 297 John Folkesson Robustness of the Quadratic Antiparticle Filter for Robot Localization 303 Stphane Bazeille, Emmanuel Battesti, David Filliat Qualitative Localization using Vision and Odometry for Path Following in Topo-metric Maps 309 Keisuke Matsuo, Jun Miura, Junji Satake Stereo-Based Outdoor Localization using a Line Drawing Building Map 315 Alessandro Benini, Adriano Mancini, Emanuele Frontoni, Primo Zingaretti, Sauro Longhi Adaptive Extended Kalman Filter for Indoor/Outdoor Localization using a 802.15.4a Wireless Network 321 List of Authors


x

1

Robots that change their world: Inferring Goals from Semantic KnowledgeCipriano Galindo*, Javier Gonz lez, a Juan-Antonio Fern ndez-Madrigal a Dept. of System Engineering and Automation University of M laga, Spain a Alessandro Safotti AASS Mobile Robotics Lab Orebro University, Sweden

AbstractA growing body of literature shows that endowing a mobile robot with semantic knowledge, and with the ability to reason from this knowledge, can greatly increase its capabilities. In this paper, we explore a novel use of semantic knowledge: we encode information about how things should be, or norms, to allow the robot to infer deviations from these norms and to generate goals to correct these deviations. For instance, if a robot has semantic knowledge that perishable items must be kept in a refrigerator, and it observes a bottle of milk on a table, this robot will generate the goal to bring that bottle into a refrigerator. Our approach provides a mobile robot with a limited form of goal autonomy: the ability to derive its own goals to pursue generic aims. We illustrate our approach in a full mobile robot system that integrates a semantic map, a knowledge representation and reasoning system, a task planner, as well as standard perception and navigation routines. Index TermsSemantic Maps, Mobile Robotics, Goal Generation, Goal Autonomy, Knowledge Representation, Proactivity.

I. I NTRODUCTION Mobile robots intended for service and personal use are being increasingly endowed with the ability to represent and use semantic knowledge about the environment where they operate [13]. This knowledge encodes general information about the entities in the world and their relations, for instance: that a kitchen is a type of room which typically contains a refrigerator, a stove and a sink; that milk is a type of perishable food; and that perishable food is stored in a refrigerator. Once this knowledge is available to a robot, there are many ways in which it can be exploited to better understand the environment or plan actions [10], [18], [19], [21], [23], [25], assuming of course that this knowledge is a faithful representation of the properties of the environment. There is, however, an interesting issue which has received less attention so far: what happens if this knowledge turns out to be in conict with the robots observations? Suppose for concreteness that the robot observes a milk bottle laying on a table. This observation conicts with the semantic knowledge that milk is stored in a refrigerator. The robot has three options to resolve this contradiction: (a) to update its semantic knowledge base, e.g., by creating a new subsclass of milk that is not perishable; (b) to question the*Corresponding author. System Engineering and Automation Dpt. University of M laga, Campus de Teatinos. E-29071 M laga, Spain. Email: a a [email protected]. This work has been partially supported by the Spanish Government under contract DPI2008-03527.

validity of its perceptions, e.g., by looking for clues that may indicate that the observed object is not a milk bottle; or (c) to modify the environment, e.g., by bringing the milk into a refrigerator. While some work have addressed the rst two options [6], [11], the last one has not received much attention so far. Interestingly, the last option leverages an unique capability of robots: the ability to modify the physical environment. The goal of this paper is to investigate this option. We propose a framework in which a mobile robot can exploit semantic knowledge to identify inconsistencies between the observed state of the environment and a set of general, declarative descriptions, or norms, and to generate goals to modify the state of the environment in such a way that these inconsistencies would disappear. When given to a planner, these goals lead to action plans that can be executed by the robot. This framework can be seen as a way to enable a robot to proactively generate new goals, based on the overall principle of maintaining the world consistent with the given declarative knowledge. In this light, our framework contributes to the robots goal autonomy. Although behavioral autonomy has been widely addressed in the robotic arena by developing deliberative architectures and robust algorithms for planning and executing tasks under uncertainty, goal autonomy has received less attention, being explored in the last years in the theoretical eld of multi-agents [4], [8] and implemented through motivational architectures [1], [7]. Our framework relies on a hybrid semantic map, which combines semantic knowledge based on description logics [2] with traditional robot maps [11], [18], [21]. Semantic maps have been already shown to increase the robots behavioral autonomy, by improving their basic skills (planning, navigation, localization, etc.) with deduction abilities. For instance, if a robot is commanded to fetch a milk bottle but it ignores the target location, it can deduce that milk is supposed to be in fridges which, in turn, are located at kitchens. We now extend our previous works on these issues [10], [11] to also include partial goal autonomy through the proactive generation of goals based on the robots internal semantic model. More specically, we consider a robot with the innate objective of keeping its environment in good order with respect to a given set of norms, encoded in a declarative way in its internal semantic representation. Incoherences between the sensed reality and the model, i.e., the observation of facts that violate a particular norm, will lead to the generation of the corresponding goal that, when planned and executed,


1

2

will re-align the reality to the model, as in the milk bottle example discussed above. It should be emphasized that in this work we only focus on the goal inference mechanism: the development of the required sensorial system, and the possible use of semantic knowledge in that context, are beyond the scope of this paper. Our approach to goal autonomy can be seen as a case of normative goals applied to agents which act based on beliefs and intentions [4], [8]. However, normative goals are often considered as simple if-then rules triggered when particular stimuli are given in the environment [1], [20]. Other works have used the term maintenance goals to represent innate goals that are aimed to satisfy a particular state of the world over time, e.g., the battery level should be always over a certain value [3], [12]. Our approach substantially diverges from those works, since it is not based on procedural rules, i.e., motivation-action pairs, nor if-then rules. Instead, we rely on a declarative representation of the domain, using the L OOM description logic language [17], from which the robot infers what should be done according to the current factual information in order to maintain the consistency between its environment and its representation. This manuscript is structured as follows. In the next section we present our semantic map. Section III formalizes the use of semantic knowledge for goal generation. In section IV a real experiment is described. Finally some conclusions and future work are outlined. II. A S EMANTIC M AP FOR M OBILE ROBOT O PERATION The semantic map considered in this work, derived from [10], comprises two different but tightly interconnected parts: a spatial box, or S-Box, and a terminological box, or TBox. Roughly speaking, the S-Box contains factual knowledge about the state of the environment and of the objects in it, while the T-Box contains general semantic knowledge about the domain, giving meaning to the entities in the S-Box in terms of concepts and relations. For instance, the S-Box may represent that Obj-3 is placed at Area-2, while the TBox may represent that Obj-3 is a stove which is a type of appliance. By combining the two sides, one can infer, for instance, that Area-2 is a kitchen, since it contains a stove. This structure is reminiscent of the structure of hybrid knowledge representation (KR) systems [2], which are now dominant in the KR community. Our semantic map extends the assertional component to be more than a list of facts about individuals by also associating these individuals to sensor-level information with a spatial structure hence the name S-Box. Please refer to [10] for more detail. Figure 1 shows a simple example of a semantic map of a home-like environment where both the S-Box and the TBox have a hierarchical structure. The hierarchy in the TBox reects the fact that the represented semantic knowledge forms a taxonomy. For the S-Box, the use of a hierarchical spatial representation is a convenient and common choice in the robotic literature [9], [15]. This hierarchical arrangement largely helps in reducing the computational burden in largescale scenarios when spatial information is involved, i.e. robot

Fig. 1. An example of semantic map for a home-like environment. S-Box is on the left and T-Box on the right. See explanation in the text.

localization, as well as when symbolic information is required, i.e. goal generation, or task planning [10].Of course one could also consider a at representation in the S-Box: in fact, in our framework, the S-Box can be substituted by any other spatial representation. III. I NFERRING G OALS FROM S EMANTICS The semantic map described above provides two different points of view of the robot workspace. On the one hand the spatial part (S-box) enables the robot to generate plans from basic skills, striving for behavioral autonomy. On the other hand the terminological part (T-box) provides an abstract model of the robot environment which includes general knowledge, e.g., books are located on shelves, which can be exploited for the automatic generation of robot goals. First we give an informal description of the proposed mechanism for goal generation. Then, section III-B formalizes our approach under description logic. Finally, section III-C illustrates the process with two intuitive examples. A. Informal Description In the eld of knowledge representation, semantic knowledge is usually interpreted as being descriptive of a specic domain: for example, the item of knowledge beds are located in bedrooms is used to partially describe beds. This knowledge is most useful to infer implicit properties from a few observed facts. For example, if the robot perceives a bed in a room it can infer that the room is a bedroom; conversely, if it is commanded to nd a bed it can restrict its search to bedrooms. Galindo et al. [10] offer examples of applications of these inferences in the robotic domain. Interestingly, semantic knowledge can also be interpreted as being normative: under this interpretation, the above item of knowledge is prescribing where a bed must be located. The difference becomes apparent when considering how a robot should react to an observation that contradicts this knowledge. Consider again the milk box example in the Introduction, and the three possible options to resolve the contradiction


2

3

discussed there. Options (a) (update the model) and (b) (update the perceived state) correspond to modifying the robots beliefs to recover from a contradiction, and are related to execution monitoring and uncertain state estimation. These options has been explored previously [6], [11]. The third option (c) (update the world) involves goal generation, and it is the one addressed here. Informally, our approach denes a subset of concepts and relations stored in the T-Box as normative, i.e. they are involved in norms that should be fullled, by dening a special class normative-concept and a special relation normative-relation. Items of knowledge to be treated as normative will derive from them. For instance, we can dene that the normative concept towel should be related to the concept bathroom through the normative relation place that is, towels should be located in a bathroom. When a given instance violates a norm in the T-Box, the system derives the items of knowledge involved in the norm, and hence the goal that should be posted in order to satisfy that norm. In our example, suppose that an instance of a towel is perceived in a room which is not a bathroom. Then the given denition of a towel is violated a circumstance that can be detected by most knowledge representation systems, including the L OOM [17] system used in our experiments. Since the above denition of towel is normative, the system yields a goal to satisfy the constraint, that is, to make the place of this towel be an instance of a bathroom. If the robot knows that, let say, room-3 is a bathroom, this means that the goal bring the towel to room-3 is generated. B. Description Logic Representation for Inferring Normative Goals Let I be a description logic interpretation on a particular domain D. Let dene a set of disjoint concepts = {P1 , . . . Pn }, i.e., a, a Pi j, j = i, a Pj , where x y denotes that x is subsumed by concept y. Let Nr be called a normative relation, a function dened as: Nr : NC where NC represents the so-called normative concepts, that is, concepts which ought to be properly related to those from . Nr actually denes the norms to be kept. Normative relations are dened as one-to-one function as b NC Pj , b [F ILLS : Nr Pj ] , where a [F ILLS : B C] denotes that the instance a is related to an instance derived from concept C through the relation B. The NC set is further divided into two disjoint sets: the set of all normative concepts that fulll the imposed norms, and the set of those that fail to fulll some of the norms (see gure 2). Within this structure of the domain, constraint violations are automatically inferred when instances of the dened partitions are deduced to belong to a number of disjoint concepts. Let see an example:

Normative Concepts

Normative Relations

'

'

Nr1

..

Nrn

P1

Pi

..

Pn

C(defconcept C :is (and ' (:the Nr1 Pi))) c

(defrelation Nr1 :is normative-relation :domain normative-concept :range ) x

Fig. 2. Knowledge representation for detecting inconsistencies. Boxes represent concepts while instances are represented as circles. The concept C is dened as a normative concept related to Pi through the normative relation Nr1 . See explanation in the text.

Let C a normative concept (and therefore C by denition) which is related to the Pi concept through the normative relation Nr . That is, c C, c [F ILLS : Nr x], x Pi

If in a given interpretation I, k C, k [F ILLS : Nr y], y Pj , Pj = Pi I y Pj y Pi (Incoherent y). That is, if the normative relation is not met for a particular instance of a normative concept, the ller of such an instance, in this case y, becomes incoherent. Moreover, since k is dened as k C , it is also inferred that k , which also makes k incoherent. Goal Inference. Given an incoherent instance of a normative concept, k C and the normative relation Nr , Nr (k) = x, x Pi , the inferred goal to recover the system from the incoherence is: (exists ?z (Pi z) (Nr k z)) That is, in the goal state, there should exist an instance of Pi related to k through the normative relation Nr 1 . C. Sample Scenarios In this section we describe two illustrative examples. 1) Milk should be inside fridges: Consider a home assistant robot taking care of an apartment. Among other norms, the robot might watch milk bottles so they are always kept inside the fridge (see an implementation in section IV). The semantic map for this scenario will entail information about the different rooms, i.e. kitchen, livingroom, bathroom, etc., the objects found inside, i.e. tables, chairs, fridges, books, bottles, etc, and their relations. Following the formalization given in III-B, part of the description of this scenario includes the partition of different places where bottles of milk could be found, e.g. = {f ridge, table, shelf }, being milk-bottle a normative concept, i.e. milk bottle , (see gure 3). Note that this denition implicitly provides certain restrictions that any bottle of milk should fulll. Precisely,1 It is not necessary to add the negation of (N k z) to the goal state, since r the Nr function is dened as one-to-one.


3

4

(defconcept object-places :is-primitive (and location (at-least 1 has-object) :partitions $$))

Normative Concepts (defrelation normative-relation) (defconcept normative-concept :partitions $Norms$))

Normative Relations

(defrelation normative-relation)

(defconcept fridge :is-primitive (and object-places appliance (:the temp cold) :in-partitions $$))

(defconcept table :is-primitive (and object-places furniture (:exactly 4 legs) :in-partitions $$))

(defconcept fulfilling-norm ..) (defrelation place (d f l i l :is (and normative-relation :domain object :range object-places))

(defconcept non-ulfilling-norm ..)

(defrelation has-humidity :is (and normative-relation :domain object :range humidity)..)

(defconcept plant :is ( d f lfilli i (and fulfilling-norm (:the place garden) (:the has-humidity normal-hum) ..)

(defrelation place :is (and normative-relation :domain object :range location))

Range Partitions (defconcept normative-concept :partitions $Norms$))

(defconcept room :partitions $Rooms$))

(defconcept humidity :partitions $H-levels$))

(defconcept fulfilling-norm :is (and normative-concept (:not (:some normative-relation incoherent)) :in-parititions $Norms$)

(defconcept non-fulfilling-norm :is (and normative-concept (:some normative-relation incoherent) :in-parititions $Norms$)

(defconcept kitchen :in-partition $Rooms$)) (defconcept bathroom :in-partition $Rooms$)) (defconcept garden :in-partition $Rooms$))

(defconcept dry :in-partition $H-levels$)) (defconcept normal-hum :in-partition $H-levels$)) (defconcept wet :in-partition $H-levels$))

(defconcept milk-bottle :is (and beverage fulfilling-norm (:the place fridge) (:the color white))

Fig. 4. General scheme for representing multiple norms. A particular partition has to be dened for each normative relation.2

Fig. 3. Part of the domain denition for the milk inside fridges example. For clarity sake, fullling-norm is used instead of and non-fullling-norm instead of .

the following goal is deduced:

(exist ?x (fridge ?x) (place mb ?x))

milk-bottle is assumed to be a beverage which has to meet at least one norm imposed by a normative relation, since it is subsumed by the fulfilling-norm concept. Through the denitions given in gure 3, the expression (:the place fridge) indicates that every bottle of milk ought to be located in one location that must be a fridge. Notice that in this example, the other restriction (:the color white) is not dened as normative relation, and thus, if it is not fullled in the scenario it will be simply deduced that the object is not a bottle of milk and no incoherences or robot goals will be generated. Let us now consider the following situation in which the information gathered by the robot contradicts the denitions in the domain:{(table t1)(milk-bottle mb)(fridge f1) (place mb t1)}

That is, the robot has to put the bottle of milk represented by mb inside any object ?x which is known to be a fridge. Since in the robots domain there is a single fridge f1, the above goal is instantiated as (place mb f1). 2) Plant should be properly watered: We now describe a more general example in which two norms are imposed on the same normative concept. Consider we impose that plants should be placed at the garden and have a normal humidity level. In this case we need two normative relations place and has-humidity and two partitions of concepts representing the possible, disjoint values for such relations. Figure 4 depicts part of the T-Box for this example. Let us consider the following situation:{(kitchen k)(bathroom b)(garden g) (plant p)(place p k)(humidity-value dry) (has-humidity p dry)}

Under this interpretation, L OOM infers that the instance t1 should be a fridge since there is a bottle of milk placed on it. Such an inference produces an incoherence in the model given that the instance t1 is deduced to belong to two concepts, i.e. table and fridge, which have been dened as members of a partition. In this situation t1 is marked by L OOM as incoherent. Moreover, it is also deduced that the instance mb, initially dened as mb , also belongs to since the normative relation (:the place fridge) is lled with an incoherent instance. Again the system detects that mb belongs to two concepts dened in a partition, and thus mb is also marked as incoherent. The result is that the instances involved in the violated norm are detected and marked as incoherent. By checking the domain denition of such incoherent instances,

As in the previous example, the process for detecting norm violations checks for incoherent instances. In this case instances k and dry become incoherent since they are deduced to belong to {kitchen,garden} and {dry,normal-hum} respectively. Besides, the instance p is also incoherent and therefore the following goal is generated:(and (exist ?x (garden ?x)(place p ?x)) (exist ?y (normal-hum ?y)(has-humidity p ?y)))

IV. A N I LLUSTRATIVE E XPERIMENT We now illustrate the applicability of our goal generation technique to a real robotic application by showing an illustra2 This goal is expressed in the goal language of the planner used in our experiment (see below), which is a subset of FOL.


4

5

Fig. 6. Sketch of the software architecture used in our experiments. Only the modules and connections relevant to goal generation are shown. Fig. 5. The test environment. Left: layout. Right: the robot Astrid.

tive experiment run in a home environment. The experiment is inspired by the milk scenario in Sec. III above. A. Physical setup We have used a physical test-bed facility, called the P EISHome [24], that looks like a bachelor apartment of about 25 m2 and consists of a living-room, a bedroom and a small kitchen see Fig. 5. The P EIS-Home is equipped with a communication infrastructure, and with a number of sensing and actuating devices, including a few mobile robots. Relevant to the experiments reported here are: a refrigerator equipped with a computer, some gas sensors, a motorized door, and an RFID tag reader; an RFID tag reader mounted under the kitchen table; a set of RFID tagged objects, including a milk cartoon; a set of webcams mounted on the ceiling; and Astrid, a PeopleBot mobile robot equipped with a laser scanner, a PTZ camera, and a simple gripper. A few provisions have been introduced to simplify execution. In particular, since Astrid does not have a manipulator able to pick-up an object and place it somewhere else, these operations have been performed with the assistance of a human who puts the object in and out from Astrids gripper. These simplications are acceptable here, since the purpose of our experiments is not to validate the execution system but to illustrate our goal generation algorithm in the context of a full robotic application. B. Software setup The software system used in our experiment is schematically shown in Fig. 6. The block named P EIS Ecology contains all the robotic components and devices distributed in the P EISHome. These are integrated through a specic middleware, called the P EIS-Middleware, that allows to dynamically activate and connect them in different ways in order to perform different tasks [5]. A set of activations and connections is called a conguration of the P EIS Ecology. For instance, the conguration in which the ceiling cameras are connected via an object recognition to the navigation controller onboard Astrid can be used to let the robot reach a given object. The semantic map is based on a simple metric-topological map attached to the LOOM knowledge representation system [17]. Newly observed facts are asserted in LOOM using the tell primitive. The goal generation system interacts with

LOOM as described in Sec. III above. Newly generated goals are passed to the planning system. This consists of three parts: an action planner, called PTLplan [14], that generates a sequence of actions to satisfy the goal; a sequencer, that selects those actions one by one; and a conguration planner [16], that generates the conguration needed to perform each action. When the current plan is completed, the goal generation system is re-activated. C. Execution Before the execution started, the semantic map contained a metric-topological map of the P EIS-Home, and the considered semantic knowledge in L OOM . In particular, the following statement was included in the L OOM knowledge base(defconcept MilkBox :is (:and Container FulfillingNorm (:the place Fridge) ))

This encodes the normative constraint that any instance of the normative class MilkBox must have a single ller for the place relation, and that this ller must be of the class Fridge. An RFID tag has been attached to a milk box, containing an encoding of the following information:id: mb-22 type: MilkBox color: white-green size: 1-liter

At start, the milk is put on the kitchen table, called table-1 in the map. The RFID tag reader under the table detects the new tag, and reports the information that mb-22 is a MilkBox and it is at table-1 see Fig. 7. This information is entered into L OOM by:(tell (MilkBox mb-22)) (tell (place mb-22 table-1))

As discussed in Sec. III, this information renders both the instances mb-22 and table-1 incoherent. The goal generation algorithm identies mb-22 as the normative instance. The algorithm then searches through all the relations that constrain mb-22 to nd a violated normative one, and it nds place. Since this relation should be lled by an instance of Fridge, it generates the following goal:(exists ?x (and (Fridge ?x)(place mb-22 ?x)))

PTLplan uses the knowledge in the semantic map, together with its domain knowledge about the available actions, to generate the following action plan (simplied):((MOVE astrid table-1) (PICKUP astrid mb-22) (OPEN fridge-1) (MOVE astrid fridge-1)


5

6

R EFERENCES[1] R.C. Arkin, M. Fujita, T. Takagi, and R. Hasegawa. An ethological and emotional basis for human-robot interaction. Robotics and Autonomous Systems, 42(3-4)., 42(3-4), 2003. [2] F. Baader, D. Calvanese, D.L.D McGuinness, and D. Nardi, editors. The Description Logic Handbook. Cambridge University Press, 2007. [3] C. Baral, T. Eiter, M. Bj reland, and M. Nakamura. Maintenance a goals of agents in a dynamic environment: Formulation and policy construction. Articial Intelligence, 172(12-13):14291469, 2008. [4] G. Boella and R. Damiano. An architecture for normative reactive agents. In Proc of the 5th Pacic Rim Int Workshop on Multi-Agents, pages 117, London, 2002. Springer. [5] M. Bordignon, J. Rashid, M. Broxvall, and A. Safotti. Seamless integration of robots and tiny embedded devices in a peis-ecology. In Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS). San Diego, CA, 2007. [6] A. Bouguerra, L. Karlsson, and A. Safotti. Semantic knowledge-based execution monitoring for mobile robots. In Proc. of the IEEE Int. Conf. on Robotics and Automation, Rome, Italy, pages 36933698, 2007. [7] A. M. Coddington, M. Fox, J. Gough, D. Long, and I. Serina. Madbot: A motivated and goal directed robot. In Proc The 20th Nat Conf on Articial Intelligence, Pennsylvania, USA, pages 16801681, 2005. [8] M. Dastani and L.W.N. van der Torre. What is a normative goal?: Towards goal-based normative agent architectures. In Proc of the Int Workshop on Regulated Agent-Based Social Systems (2002) Bologna, Italy, pages 210227, 2002. [9] C. Galindo, J.A. Fernandez-Madrigal, and J. Gonzalez. Multiple Abstraction Hierarchies for Mobile Robot Operation in Large Environments. Studies in Computational Intelligence, Vol. 68. Springer, 2007. [10] C. Galindo, J.A. Fernandez-Madrigal, J. Gonzalez, and A. Safotti. Robot task planning using semantic maps. Robotics and Autonomous Systems, 56(11):955966, 2008. [11] C. Galindo, A. Safotti, S. Coradeschi, P. Buschka, J.A. FernandezMadrigal, and J. Gonzalez. Multi-hierarchical semantic maps for mobile robotics. In Int. Conf. on Intelligent Robots and Systems, pages 3492 3497. IROS, Edmonton, Alberta (Canada), 2005. [12] M.V. Hindriks and M.B. van Riemsdijk. Satisfying maintenance goals. In Proc of the Int Workshop on Declarative Agent Languages and Technologies (DALT07), Honolulu, HI, USA, pages 86103, 2007. [13] J. Hertzberg and A. Safotti, editors. Special issue on semantic knowledge in robotics. Robotics and Autonomous Systems, 56(11), 2008. [14] L. Karlsson. Conditional progressive planning under uncertainty. In Proc of the Int Joint Conf on Articial Intell. (IJCAI), pages 431438, Seattle, USA, 2001. [15] B.J. Kuiper. Modeling Spatial Knowledge, chapter Advances in Spatial Reasoning, Vol. 2, pages 171198. The U. of Chicago Press, 1990. [16] R. Lundh, L. Karlsson, and A. Safotti. Autonomous functional conguration of a network robot system. Robotics and Autonomous Systems, 56(10):819830, 2008. [17] R. MacGregor and R. Bates. The loom knowledge representation language. Technical report, DTIC Research Report ADA183415, 1987. [18] D. Meger, P. Forssen, K. Lai, S. Helmer, S. McCann, T. Southey, M. Baumann, J. Little, D. Lowe, and B. Dow. Curious george: An attentive semantic robot. In IROS Workshop: From sensors to human spatial concepts, pages 390404, 2007. [19] O.M. Mozos, P. Jensfelt, H. Zender, M. Kruijff, and W. Burgard. From labels to semantics: An integrated system for conceptual spatial representations of indoor environments for mobile robots. In ICRA Workshop: Semantic Information in Robotics, 2007. [20] T.J. Norman and D. Long. Alarms: An implementation of motivated agency. In Proc of the IJCAI Workshop on Agent Theories, Architectures, and Languages (ATAL), Montreal, Canada, pages 219234, 1995. u [21] A. N chter, O. Wulf, K. Lingemann, J. Hertzberg, B. Wagner, and H. Surmann. 3D mapping with semantic knowledge. In RoboCup Int. Symp., pages 335346, 2005. [22] F. Pecora and M. Cirillo. A constraint-based approach for plan management in intelligent environments. In Scheduling and Planning Applications Workshop at ICAPS09, 2009. [23] A. Ranganathan and F. Dellaert. Semantic modeling of places using objects. In Robotics: Science and Systems Conf., 2007. [24] A. Safotti, M. Broxvall, M. Gritti, K. LeBlanc, R. Lundh, J. Rashid, B.S. Seo, and Y.J. Cho. The peis-ecology project: vision and results. In Int. Conf. on Intelligent Robots and Systems. Nice, France, 2008. [25] M. Tenorth, U. Klank, D. Pangercic, and M. Beetz. Web-enabled robots: Robots that use the web as an information resource. Robotics and Automation Magazine, 18(2), 2011.

Fig. 7. RFID tagged objects and RFID tag readers used in our experiments. Left: in the fridge. Right: under the kitchen table.

(PLACE astrid mb-22) (CLOSE fridge-1))

where the variable ?x has been instantiated by fridge-1. The sequencer passes each action in turn to the conguration planner, which connects and activates the devices in the P EIS Ecology needed to execute it. For example, the rst two actions only require devices which are on-board Astrid, while the third action requires the activation of the fridge door device. (The details of this ecological execution are not relevant here: see [16] for a comprehensive account.) After the milk is removed from the table, the RFID tag reader under the table detects its absence and it signals it to the semantic map. When the milk is placed into the fridge, it is detected by the reader in the fridge. Corresponding to these two events, the following assertions are made in L OOM :(forget (place mb-22 table-1)) (tell (place mb-22 fridge-1))

After execution is completed, the sequencer re-activates the goal generator. Since the place of mb-22 is now an instance of a fridge, no incoherence is detected and no new goal is generated.

V. D ISCUSSION AND C ONCLUSIONS This paper has explored a novel use of semantic knowledge: recognizing and correcting the situation in the world that do not comply with the given semantic model, by generating appropriate goals for the robot. A distinctive feature of our approach is that the normative model is provided in a declarative way, rather than by exhaustive violation-action rules. Simple experiments demonstrate the conceptual viability of this approach. The work reported here is a rst step in an interesting direction, and many extensions can and should be considered. For instance, in our work we assume that the robot should always enforce consistency with the semantic knowledge. However, there are cases where norm violation might be allowed. Going back to the milk example, it would be reasonable to allow that the milk bottle stays out of the fridge while the user is having breakfast. Dealing with this type of situations would require the ability to reason about both the robots plans and the users activities in an integrated way [22].


6

1

On Process Recognition by Logical InferenceArne Kreutzmann Immo Colonius Lutz Frommberger Frank Dylla Christian Freksa Diedrich Wolter SFB/TR 8 Spatial Cognition, University of Bremen, Germany

Abstract The ability to recognize and to understand processes allows a robot operating in a dynamic environment to rationally respond to dynamic changes. In this paper we demonstrate how a mobile robot can recognize storage processes in a warehouse environment, solely using perception data and an abstract specication of the processes. We specify processes symbolically in linear temporal logic (LTL) and pose process recognition as a model verication problem. The key feature of our logic based approach is its ability to infer missing pieces of information by logic-based reasoning. The evaluation demonstrates that this approach is able to reconstruct histories of good movements in a lab-simulated warehouse. Index Terms plan recognition, temporal reasoning temporal logic, spatio-

feature detection symbol grounding mapping and localization place identi cation qualitative locations

symbolic reasoning place inference and process recognition

odometry abstract process speci cation observation behavior process understanding

Fig. 1.

Conceptual overview of our software architecture

I. I NTRODUCTION Mastering dynamic environments is a demanding challenge in autonomous robotics. It involves recognition and understanding processes in the environment [7]. Recent advances in simultaneous localization and mapping (SLAM) [20, 21, 22] build the basis for sophisticated navigation in dynamic environments, but but our aim of understanding processes goes beyond navigation. In this paper we indicate how the problem of recognizing processes can be tackled on a conceptual level in the domain of warehouse logistics. In a warehouse, there is a constant ow of goods which are moved through space, establishing functional zones that are connected with certain types of storage processes (for example, admission of goods into a warehouse makes use of a buffer zone to temporarily store goods for quality assurance). Knowing about the in-warehouse processes and their whereabouts enables warehouse optimization. Hildebrandt et. al. argue for using autonomous robots as a minimally invasive means to observe in-warehouse processes [10]. However, the sensory system of the robot provides uncertain and incomplete knowledge about the environment and the observed spatio-temporal patterns. Thus the challenge is to interpret the observations sensibly. Many approaches to process recognition rely on statistical data to train probabilistic classiers such as Markov networks [6, 13], Bayesian networks [23], or supervised learning [5]. Approaches based on statistical data perform very well in terms of recognition rate, but, aside from the need for training, they do not support exible queries about processes and they have they have to be re-trained if new elements or processes are introduced in the domain. Symbolic approaches have none of these downsides, but require a model of the observable processes, which is given in our environment. Additionally, aThis paper presents work done in the project R3-[Q-Shape] of the Transregional Collaborative Research Center SFB/TR 8 Spatial Cognition. Funding by the German Research Foundation (DFG) is gratefully acknowledged.

well constructed model allows for efcient use of heuristics to speed up query processes[8]. Usually, symbolic approaches are used to tackle plan recognition, which is closely related to process recognitionsee [2, 3] for an overview. In the following we present a logic-based approach that allows us to recognize activities purely from qualitative process descriptions without prior training. By integrating and abstracting sensory information we are able to answer queries about observed spatio-temporal activities (such as How often have goods been relocated within the storage zone?) as well as about regions in space (e.g., Which areas in the warehouse have been used as a buffer zone?). Answering such queries is an important step towards logistic optimization. The contribution of this paper is to demonstrate how processes and their whereabouts can be inferred in a previously unknown environment. Referring to the decomposition of process detection by Yang [23], we propose a multi-step approach to get from low-level sensory observations to high-level symbolic representations (see Fig. 1). In our scenario, a robot performs a surveillance task in the warehouse. Object recognition is outside the scope of this paper, but in many logistics scenarios goods can easily be identied by unique labels attached to them (such as barcodes or RFID tags). Thus, we assume that the robot is able to uniquely identify goods in the warehouse. The integration of position estimates for the goods in itself presents a feature-based SLAM problem. Uncertain and incomplete position estimates of entities gathered by a probabilistic mapping procedure must be transferred into a symbolic representation in a symbol grounding process to allow for high-level descriptions of the system dynamics. What has been an uncertain position estimate in the mapping process must become a stable qualitative notion of location. Based on correspondence of features and locations in time, we are able to specify processes of interest in an abstract formal language and, in a third step, tackle the process recognition problem by model verication. The formal language we choose to formalize processes


7

2

picking zone (P)G G

buffer zone (B)G

outlet zone (O)

So when observing this kind of environment, we face the challenge that for detecting concrete storage processes we rely on the existence of certain zones, but we do not know their whereabouts. III. I N -WAREHOUSE P ROCESS D ETECTION WITH L INEAR T EMPORAL L OGIC To interpret raw sensory data such that we achieve a symbolic representation of the processes of interest, we rst introduce linear temporal logic and the axiomatization of our domain. All queries are stated as LTL formulas and can be answered by model verication. Following this, we describe the symbolic grounding. Then, we specify the in-warehouse processes in linear temporal logic and demonstrate the inference process by an example. A. Linear Temporal Logic (LTL) LTL [17] is a modal logic that extends propositional logic by a sequential notion of time. A formula in LTL is dened over a nite set of propositions with a set of the usual logic operators (, ,, ). The temporal component is established by an accessibility relation R that connects worlds (or states) and a set of modal operators, of which we use the following: next. A formula holds in in the following world always. A formula holds now and in all future worlds eventually. will hold in some world in the future ( )

G

entrance zone (E)G G

storage zone (S)

Fig. 2.

A warehouse and its functional zones.

and to state queries is linear temporal logic (LTL) [17, see Sect. III-A]. LTL was proposed earlier as a tool for mobile robotics [1], especially for robot motion planning from highlevel specications [11, 18]. Recently, this approach has also been applied to real robotic systems [12]. In the domain of smart environments, an approach to process detection by LTL model verication has been presented in [14]. LTL not only allows for queries about processes, but also about spatial relations of regions. This approach covers a wide range of reasoning tasks adequately. In particular, it allows us to query the occurrence of processes operating on spatial regions and the concrete whereabouts of those regions at the same time in one and the same reasoning process. II. T HE WAREHOUSE S CENARIO

We address the problem of understanding so-called chaotic B. Axiomatizing the Warehouse Scenario or random-storage warehouses, characterized by a lacking 1) Propositions: We dene the propositions that model the predened spatial structure, that is, there is no xed assignment desired processes in our logic with the help of the following of storage locations to specic goods. Thus, storage processes are solely in the responsibility of the warehouse operators atomic observables: and basically not predictable: goods of the same type may be a set G = {G1 , . . . , Gn } of uniquely identiable goods distributed over various locations and no data base keeps track a set L = {L1 , . . . , Lm } of locations in space at which of these locations. This makes it a hard problem for people goods have been perceived by the robot aiming at understanding the internal storage processes. a set Z = {E, B, S, P, O} of functional zones as On a conceptual level, storage processes are dened by described in Sect. II. a unique pattern [19]: On their way into and out of the The following atoms need to be dened over G, such that we warehouse, goods are (temporarily) placed into functional zones obtain a nite set of atoms, L, and Z: which serve specic purposes (see Fig. 2). All goods arrive at(G, L) holds iff a good G is known to be at location L in the entrance zone (E). From there, they are picked up in(L, Z) holds iff a location L lies within zone Z and temporarily moved to a buffer zone (B) before they are close(L1 , L2 ) holds iff two locations L1 , L2 are close nally stored in the storage zone (S). Within the storage zone to one another redistribution of goods can occur arbitrarily often. When taking 2) Axioms: Based on constraints of space and general out goods, they are rst moved to the picking zone (P) from knowledge about our domain, we axiomatize our domain. One where they are taken to the outlet zone (O), before being placed constraint is that we disregard continuous motion and therefore on a truck. A mobile robot observing such a warehouse is not able only deal with snapshots of the world. This means that all to directly perceive these zones, as they are not marked. For observed goods are temporarily xed at their positions. all zones we know that they exist (that is, that such regions A good G can only be at one location at a time. We are used within the storage operations), but not their concrete introduce the following axioms for all G G and Li , Lj spatial extents or their number of occurrences, as they appear L, i = j: as a result of dynamic in-warehouse storage processes. The A1G,Li ,Lj = at(G, Li ) at(G, Lj ) (1) robot can detect and identify goods, and estimate their position.


8

3

Any object is always located within a zone Z Z. We have for all G G and L L: A2G,L = at(G, L) in(L, Z) (2)ZZ

Locations in different zones are not close to each other, that is, zones are at least some minimum distance apart. We have for all Zk , Zl Z (k = l) and Li , Lj L (i = j): A3Li ,Lj ,Zk ,Zl = close(Li , Lj ) in(Li , Zk ) in(Lj , Zl ) (3)

the time steps t and map all goods Gi with their position estimates (xi , yi ) to corresponding observations obs(t, Gi , Lj ) that assign that Gi has been observed at Lj at time step t. This yields a series of sets of observations Ot = Gi G obs(t, G, Lj ) over time. A new world is established as soon as our observations change, that is, Ot+1 = Ot . Then, from obs(t, Gi , Lj ) follows at(Gi , Lj ), and the new world consist of B i at(Gi , Lj ). D. In-Warehouse Processes We now formalize the in-warehouse processes Admission, Take-out, and Redistribution: Admission a good G is delivered to the warehouses entrance zone E and moved to the storage zone S via the buffer zone B. For all G G and Li , Lj , Lk L: AdmissionG,Li ,Lj ,Lk = at(G, Li ) in(Li , E) (6)

Zones are static. We have for all Zk , Zl Z (k = l) and L L: A4L,Zk ,Zl = in(L, Zk ) in(L, Zl )

(4)

A set A subsumes all axioms (1) (4).

C. Grounding Symbols at(G, Lj ) in(Lj , B) at(G, Lk ) in(Lk , S) So far, we have formal descriptions of the high-level in Take-out a good G is moved from the storage zone S warehouse observables on one hand, and sensory perceptions to the outlet zone O via a picking zone P . For all G G from the robot, on the other hand. These need to be connected and Li , Lj , Lk L: to each other in order to perform reasoning on real world data. That is, we need to transform the sensory information to our TakeoutG,Li ,Lj ,Lk = at(G, Li ) in(Li , S) (7) logical propositions at(G, L), close(L1 , L2 ), and in(L, Z). at(G, Lj ) in(Lj , P ) at(G, Lk ) in(Lk , O) Mapping a perceived good to a symbol G is trivial in this task due to the unique identiers. However, for the goods Redistribution a good G is moved within the storage location we will only have an uncertain position estimate zone S. For all G G and Li , Lj L, i = j: 2 (x, y) R for the entity observed from the mapping process. These estimates are subject to noise and thus will vary over RedistributionG,Li ,Lj = at(G, Li ) in(Li , S) (8) time although the observed object remains static. A location at(G, Lj ) in(Lj , S) is a qualitative abstraction from positional measurements that abstracts from uncertainty emerging from sensory perceptions Process detection can be posed as a model checking problem: and the mapping process. Therefore, we need to transform An in-warehouse process is detected when we can nd a model position estimates to a discrete and nite set of symbols, i.e., to (based on the sensory observations from the robot) that satises subsume similar or comparable positions. This transformation the corresponding formula. The history of a good is the chain is a function f : R2 L, that is, every position estimate is of processes that the good is part of and can also be stated as mapped to a single location (see Axiom (1)). To this end, a a formula. A history for a good would be admission, zero or clustering method can be applied to map estimates to a set more redistributions and its takeout. of prototypical positionsthe locations (see Section IV-B). We ground close(L1 , L2 ) by applying a metric and checking E. Example whether the distance between L1 and L2 is below a certain A good G entered the warehouse and was stored in the threshold. To ground in(L, Z), we need to identify the functional entrance zone E at position L1 at time t0 . At t1 , it was moved zones in the warehouse. These zones are constituted by sets of to a location L2 and at t2 it was moved to L3 . All these locations. For zones Z whose extents are known a-priori by locations are not close to one another. Let us assume that we introducing the respective in-atoms the corresponding locations observe the following from this process: We perceived G to LZ L can be assigned directly. All remaining locations be at L1 at t0 , at L2 at t1 and at L3 at t4 . See Fig. 3 for a Li L\LZ are known to be not a part of Z, i.e., in(Li , Z), depiction and the logical interpretationsto ease understanding but (according to (2)) must be part of one of the other zones: the worlds are labeled just like the time points. These observations constitute a model that satises (6), such in(Li , Z ) with Z Z\Z. In addition to the axioms A, the propositions close and in that the observed process is an admission, starting in world t1 and ending in world t4 , and also deduces that location are persistent over all worlds. The set L2 is in the buffer zone and L3 is in of the storage zone. B =A close(Li , Lj ) in(L, Z) (5) Note that deduced start and end times differ from the real Li ,Lj L LL,ZZ admission times: While the admission took place from t0 to t3 , is called background knowledge. The only proposition that we detect it from observations t1 to t4 ; this is due to incomplete changes over different worlds is at(G, L). We traverse through observation of the world.


9

4

sketch

EG

BS

BS

Et1

BSG

BS

Et2

BS

BSG

Et3

BS

BSG

t0observationbackground knowledge

at(G, L1 ) in(L1 , E) in(L2 , B) in(L2 , S) in(L3 , B) in(L3 , S)


in(L1 , E) in(L2 , B) in(L2 , S) in(L3 , B) in(L3 , S)


Admission at(G, L1 ) in(L1 , E)

Fig. 3. Example: Model verication for an admission process of good G (only the relevant assertions for each world t0...3 are shown). in(L1 , E) is background knowledge, also it is known that locations L2 and L3 are either part of the buffer zone (B) or the storage zone (S) but not close to one another so that they cannot belong to the same zone. From this admission rened knowledge about the buffer and storage zone can be inferred: in(L2 , B) in(L3 , S).

at(G, L2 ) in(L2 , B) at(G, L3 ) in(L3 , S)

IV. I MPLEMENTATION A. Mapping of Positions of Goods We use visual tags to represent our observable features. To ease the evaluation, some tags are known to be static throughout the experiments. This allows the map constructed by the robot to be easily aligned with the ground-truth for evaluation. The positions of the tags relative to the camera are estimated using the tag detection routine provided by the ARToolkit software library1 , for which we determined a measurement model. Positions of detected tags with a sufcient quality as well as odometry of the robot are fed into the TreeMap SLAM algorithm [9]. In contrast to [22] we deal with dynamic objects by using only one map layer in which we handle the movement of a good by simply comparing its current position measurement with its expected position. If the positions are too far apart (in our experiments >1 meter), the good is treated as having been moved and is added as a new feature into the SLAM algorithm. This results in a sequence of maps that contain position estimates and a mapping of goods to positions at each time step. B. From Positions to Locations Measured positions are clustered after each step and the generated cluster centroids are used as qualitative locations. Therefore, the mapping of positions to clusters needs to stay xed even when new centroids are generated by added data. We implemented two clustering methods for later comparison: The rst clustering method assigns position estimates to predened locations (shown in Fig. 4(a)). We used this method for evaluation purposes. The second clustering method computes locations automatically by employing a straightforward greedy algorithm: Positions are clustered together if their surrounding circle is below a certain size; otherwise a new cluster is created (shown for one test run in Fig. 4(b)). Each observation of a good is now attributed by a location and a time step (obs(t, G, L)), which is the starting point for the symbol grounding as described in Section III-C.1 http://artoolkit.sourceforge.net/

C. From LTLWorlds to Histories As described at the end of Section III-D histories of goods are also LTL formulas and as such can also be used during model verication. It is straightforward to implement the rules as Prolog clauses and let Prolog try to prove them. The connection of the world is realized by an ordered list, i.e., two worlds Wi and Wj are connected if Wj directly follows Wi in the list. We then use Prolog to constructively prove the existence of a history for each good by model verication. The history construction includes the deduction of zones as demonstrated in the example shown in Fig. 3. In general, many histories can be veried for the same observations, e.g., moving a good from A to B to C veries the model RedistributionG,A,B RedistributionG,B,C but also RedistributionG,A,C . In the latter case the observation that the good was at location B is ignored. Therefore, in ambiguous cases we select the maximal model, i.e., the history involving the largest number of observations. V. E XPERIMENTS AND E VALUATION In our experimental setting we simulated warehouse processes in our lab in order to measure to which extend histories can be identied correctly.2 A. Experimental Setup Our robot platform is an Active Media Pioneer-3 AT controlled by a top-mounted laptop and equipped with a SONY DFW SX900 (approx. 160FOV) that delivers 7.5 frames per second. In our lab we simulate a warehouse that consists of ve dedicated zones (entrance, buffer, storage, picking, outlet) as seen in Fig. 4(a) and 4(c). 15 tags are distributed within the environment as static landmarks. Goods are represented by boxes with visual tags attached to all sides. An experiment consists of a series of movements of goods between the zones while our robot is monitoring the environment. For each of the 10 test runs, the robot was placed in the lab and driven around until each landmark had been seen at least twice to ensure a fully mapped environment. Then, we2 Video material of a test run is available at http://www.sfbtr8. uni-bremen.de/project/r3/ECMR/index.html


10

58

7Outlet

3

V

J F

Outlet

6

SStorage Picking Area

5

M4Picking Area

Storage

W Y

D

N

3

2

QBuffer EntranceBuffer

L G K 7

1Entrance

0

(a) Floor plan with zones

-1 -1

(b) Automatically3 generated 5clusters 0 1 2 4 6

7

(c) The lab (picture taken from the outlet zone)

Fig. 4. Experimental Setup: The warehouse (6.12 m 7 m) with 5 zones (Entrance, Buffer, Storage, Picking, and Outlet). In (a) the xed cluster centers are shown as red stars and the landmarks as blue dots with there corresponding letters. In (b) measurements of a single experimental run are shown as black and the automatically generated clusters as blue circles.

1

predefined centroids 0.754 0.829

automatically generated centroids Known Regions No region 0.581 Entrance, Outlet All regions

Correct Histories

0,75 0.485 0.256

0,5

0.431

0,25

0

Fig. 5. Evaluation results showing the rate of correct history detections. We evaluated 10 test runs which included a total number of 21 full histories and 18 partial histories.

steered the robot in arbitrary round courses, while we moved boxes through the lab, simulating the previously dened logistic processes (Sect. III-D). The duration of a test run was between approx. 11 and 28 minutes in which we moved 3 to 8 goods through the warehouse, resulting in 4 to 19 detectable processes per run including runs with only partial histories. Goods were moved between zones while not covered by sensor surveillance, to comply with axiom 2 in section III-B. The robot was driven by hand in the experiments. As mentioned in section IV-B, we evaluated our approach with two different clustering methods, each one with 3 different settings of background knowledge. In the rst setting all zones are previously known, in the second setting only entrance and outlet are known and in the third one the whereabouts of no zone are known. B. Evaluation We evaluate our approach based on correctly identied histories. For each good we query its history, i.e., running the model verication to generate it. A history is correctly identied if temporal order and number of processes match the ground truth.

Fig. 5 shows the result of our evaluation. In the most favorable case of knowing all zones and predened cluster centroids we achieve an average recognition rate of 83%. The experiments comprise of 21 full histories and 18 partial ones. In partial histories, a good either started in the warehouse or after its admission never left it again. Our current interpretation prefers full histories over partial histories and is biased towards an empty starting warehouse, i.e., if the observations verify both admission and take-out we prefer the admission. Especially in the case of having no prior knowledge we found that partial histories reduced detection rate. In particular, with automatically generated centroids and no prior knowledge about the zones; 37.9% of the full histories were correctly found, but only 13.3% of the partial histories were correctly found. A signicant difference can be observed between the two clustering methods, but both follow the same pattern: additional prior knowledge results in more correctly identied histories. If no zone is known, i.e. all zones needed to be inferred, the results show that the approach is still capable of correctly identifying histories. This clearly demonstrates the utility of inference in process recognition. VI. D ISCUSSION Our work targets online process detection and online queries while the robot is operating. Thus, we rely on observations of goods as soon as we detect them, even if the position estimate is still uncertain. Over time, stability of positions is achieved by clustering them into locations. Every (new) perception of a good at a different location (immediately) triggers the creation of a new world. Poor position estimates (for example, when few tags are detected due to motion blur while the robot turns) can easily be mapped to locations that incorrectly induce movement of a good or lie outside of the zone the good is in. Such cases result in incorrectly detected histories. The results in Fig. 5 conrm this: When providing stable, pre-dened cluster centers, detection rates are signicantly higher, especially when more domain knowledge is included. Thus, excluding estimates with too much uncertainty would improve the detection rate. Using uncertainty estimates for measured positions will also improve


11

6

the robustness of geometrical shape estimation for the zones. However, the current implementation of the TreeMap SLAM algorithm3 does not provide uncertainty estimates. In a real-world environment it is reasonable to assume knowledge about entrance and outlet zones (e.g., by placing tags to mark the end of the warehouse). The observable difference between knowing all zones and knowing only entrance and outlet is relatively small, especially when predened clusters are used (83% and 75% respectively). These results illustrate the feasibility of our approach. In this work, we currently restrict ourselves to use inference only on sensory observations. As stated before, the detection of correct histories improves with better clustering (e.g., by using outlier detection). To query more complex information it would be reasonable to also include knowledge gained within the mapping process. That is, information on goods we have observed before and included into the map, but that we are not able to perceive at the very moment. For these objects, we have a strong belief of their existence and position, but this belief canaccording to the actual observationnot be validated. A possibility to include reasoning on such beliefs is to use a logic that provides a modal belief operator, such as the logic for BDI agents presented in [16]. Another source of information for more complex queries could be provided by an ontology, as shown in [15]. Our logic foundation also supports multiple instances of the same type of good, e.g. splitting or merging packages for delivery. However, due to limited size in our lab, we did not include this feature in our experiments. VII. S UMMARY In this paper we propose an approach to process detection based on a specication of processes as temporal logic formulas. We show in our evaluation that our approach is applicable using real sensory data from a mobile robot. One strength of our approach is that it can ll in missing pieces of information by reasoning about processes and spatial congurations in the same formalism. It is also possible to query about previously unspecied processes as well as about spatial facts, such as functional zones. Basing our approach on the well-established linear temporal logic not only works for passive process detection but would also allow us to incorporate so-called search control knowledge and perform high-level planning [4], i.e., doing active process detection in the sense of planning where to go for more information. This is the objective of our future research. R EFERENCES[1] Marco Antoniotti and Bud Mishra. Discrete event models + temporal logic = supervisory controller: Automatic synthesis of locomotion controllers. In Proceedings of the IEEE Conference on Robotics and Automation (ICRA), volume 2, pages 14411446, 1995. [2] D. Avrahami-Zilberbrand, G.A. Kaminka, and H. Zarosim. Fast and complete symbolic plan recognition: Allowing for duration, interleaved execution, and lossy observations. In Proceedings of the AAAI Workshop on Modeling Others from Observations (MOO). Citeseer, 2005.3 as

[3] Dorit Avrahami-Zilberbrand. Efcient Hybrid Algorithms for Plan Recognition and Detection of Suspicious and Anomalous Behavior. PhD thesis, Bar Ilan University, 2009. [4] Fahiem Bacchus and Froduald Kabanza. Using temporal logics to express search control knowledge for planning. Articial Intelligence, 116(12):123 191, 2000. [5] Maria-Florina Balcan and Avrim Blum. A discriminative model for semi-supervised learning. Journal of the ACM (JACM), 57(3):146, 2010. [6] Maren Bennewitz, Wolfram Burgard, Grzegorz Cielniak, and Sebastian Thrun. Learning motion patterns of people for compliant robot motion. The International Journal of Robotics Research (IJRR), 24(1):3941, 2005. [7] Marcello Cirillo, Lars Karlsson, and Alessandro Safotti. A human-aware robot task planner. In Alfonso Gerevini, Adele E. Howe, Amedeo Cesta, and Ioannis Refanidis, editors, Proceedings of the 11th International Conference on Automated Planning and Scheduling (ICAPS). AAAI, 2009. [8] Christophe Dousson and Pierre Le Maigat. Chronicle recognition improvement using temporal focusing and hierarchization. Proceedings of the 20th International Joint Conference on Artical Intelligence (IJCAI), pages 324329, 2007. [9] Udo Frese. An O(log n) Algorithm for Simulateneous Localization and Mapping of Mobile Robots in Indoor Environments. PhD thesis, University of Erlangen-N rnberg, 2004. u [10] Torsten Hildebrandt, Lutz Frommberger, Diedrich Wolter, Christian Zabel, Christian Freksa, and Bernd Scholz-Reiter. Towards optimization of manufacturing systems using autonomous robotic observers. In Proceedings of the 7th CIRP International Conference on Intelligent Computation in Manufacturing Engineering (ICME), June 2010. [11] Marius Kloetzer and Calin Belta. LTL planning for groups of robots. In Proceedings of the IEEE International Conference on Networking, Sensing and Control (ICNSC), pages 578583, 2006. [12] Marius Kloetzer and Calin Belta. Automatic deployment of distributed teams of robots from temporal logic motion specications. IEEE Transactions on Robotics, 26(1):4861, 2010. [13] Lin Liao, Donald J. Patterson, Dieter Fox, and Henry Kautz. Learning and inferring transportation routines. Articial Intelligence, 171(56):311 331, 2007. [14] Tommaso Magherini, Guido Parente, Christopher D. Nugent, Mark P. Donnelly, Enrico Vicario, Frederico Cruciani, and Cristiano Paggetti. Temporal logic bounded model-checking for recognition of activities of daily living. In Proceedings of the 10th IEEE International Conference on Information Technology and Applications in Biomedicine (ITAB), Corfu, Greece, November 2010. [15] Fulvio Mastrogiovanni, Antonio Sgorbissa, and Renato Zaccaria. Context assessment strategies for ubiquitous robots. In IEEE International Conference on Robotics and Automation (ICRA), pages 27172722, 2009. [16] John-Jules Meyer and Frank Veltman. Intelligent agents and common sense reasoning. In Patrick Blackburn, Johan Van Benthem, and Frank Wolter, editors, Handbook of Modal Logic, volume 3 of Studies in Logic and Practical Reasoning, chapter 18, pages 991 1029. Elsevier, 2007. [17] Amir Pnueli. The temporal logic of programs. In Proceedings of the 18th Annual Symposium on Foundations of Computer Science (FOCS), pages 4657, 1977. [18] Stephen L. Smith, Jana T mov , Calin Belta, and Daniela Rus. Optimal u a path planning under temporal logic constraints. In Proceeding of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 32883293, Taipeh, Taiwan, October 2010. [19] Michael Ten Hompel and Thorsten Schmidt. Management of Warehouse Systems, chapter 2, pages 1363. Springer, Berlin Heidelberg, 2010. [20] Trung-Dung Vu, Julien Burlet, and Olivier Aycard. Grid-based localization and local mapping with moving object detection and tracking. Information Fusion, 12:5869, January 2011. [21] Chieh-Chih Wang, Charles Thorpe, Sebastian Thrun, Martial Hebert, and Hugh Durrant-Whyte. Simultaneous localization, mapping and moving object tracking. The International Journal of Robotics Research, 26:889 916, September 2007. [22] Denis F. Wolf and Gaurav S. Sukhatme. Mobile robot simultaneous localization and mapping in dynamic environments. Autonomous Robots, 19:5365, 2005. 10.1007/s10514-005-0606-4. [23] Qiang Yang. Activity recognition: Linking low-level sensors to high level intelligence. In Proceedings of the 21st International Joint Conference on Articial Intelligence (IJCAI), pages 2025, Pasadena, CA, July 2009.

provided at http://openslam.org/treemap.html


12

1

Plan-based Object Search and Exploration Using Semantic Spatial Knowledge in the Real WorldAlper Aydemir Moritz G belbecker o Andrzej Pronobis Kristoffer Sj o o Patric Jensfelt Centre for Autonomous Systems, Royal Institute of Technology, Stockholm, Sweden Institut f r Informatik, Albert-Ludwigs-Universit t Freiburg, Germany u a

Abstract In this paper we present a principled planner based approach to the active visual object search problem in unknown environments. We make use of a hierarchical planner that combines the strength of decision theory and heuristics. Furthermore, our object search approach leverages on the conceptual spatial knowledge in the form of object cooccurences and semantic place categorisation. A hierarchical model for representing object locations is presented with which the planner is able to perform indirect search. Finally we present real world experiments to show the feasibility of the approach. Index Terms Active Sensing, Object Search, Semantic Mapping, Planning

I. I NTRODUCTION Objects play an important role when building a semantic representation and an understanding of the function of space [14]. Key tasks for service robots, such as fetch-andcarry, require a robot to successfully nd objects. It is evident that such a system cannot rely on the assumption that all object relevant to the current task are already present in its sensory range. It has to actively change its sensor parameters to bring the target object in its eld of view. We call this problem active visual search (AVS). Although researchers began working on the problem of visually nding a relatively small sized object in a large environment as early as 1976 at SRI [4], the issue is often overlooked in the eld. A common stated reason for this is that the underlying problems such as reliable object recognition and mapping are posing hard enough challenges. However as the eld furthers in its aim to build robots acting in realistic environments, this assumption need to be relaxed. The main contribution of this work a method to relinquish the above mentioned assumption. A. Problem Statement We dene the active visual object search problem as an agent localizing an object in a known or unknown 3D environment by executing a series of actions with the lowest total cost. The cost function is often dened as the time it takes to complete the task or distance traveled. Let the environment be and being the search space with . Also let Po () be the probability distribution for the position of the center of the target object o dened as a function over . The agent can execute a sensing action s inThis work was supported by the SSF through its Centre for Autonomous Systems (CAS), and by the EU FP7 project CogX.

the reachable space of . In the case of a camera as the sensor, s is characterised by the camera position, (xc , yc , zc ), pan-tilt angles (p, t), focal length f and a recognition algorithm a; s = s(xc , yc , zc , p, t, f, a). The part of covered by s is called a viewcone. In practice, a has an effective region in which reliable recognition or detection is achieved. For the ith viewcone we call this region Vi . Depending on the agents level of a priori knowledge of and Po () there are three extreme cases of the AVS problem. If both and Po () is fully known then the problem is that of sensor placement and coverage maximization given limited eld of view and cost constraints. If both and Po () is unknown then the agent has an additional explore action as well. An exhaustive exploration strategy is not always optimal, i.e. the agent needs to select which parts of the environment to explore rst depending on the target objects properties. Furthermore the agent needs to trade-off between executing a sensing action and exploration at any given point. That is, should the robot search for the object o in the partially known or explore further. This is classically known as the exploration vs. exploitation problem. When Po () is unknown (i.e. uniformly distributed) but is known (i.e. acquired a priori), the agent needs to gather information about the environment similar to the above case. However in this case, the exploration is for learning about the target object specic characteristics of the environment. Knowing also means that the robot can reason whether or not to execute a costly search action at the current position, or move to another more promising region of space. The rare case where Po () is fully known but is unknown is not practically interesting to the scope of this paper. So far, we have examined the case where the target object is an instance. The implication of this is that Po () + Po ( \ ) = 1, therefore observing Vi has an effect on Po ( \ Vi ). However this is not necessarily true if instead the agent is searching for any member of an object category and the number of them is not known in advance. Therefore knowing whether the target object is a unique instance or a member of an object category is an important factor in search behavior. Recently theres an increasing amount of work on acquiring semantic maps. Semantic maps have parts of the environment labeled representing various high level concepts and functions of space. Exploring and building a semantic map while performing AVS contributes to the estimation of Po (). The semantic map provides information that can be exploited by leveraging on common-sense conceptual knowledge about


13

2

indoor environments. This knowledge describes, for example, how likely it is that plates are found in kitchens, that a mouse and a computer keyboard occur in the same scene and that corridors typically connect multiple rooms. Such information offers valuable information in limiting the search space. The sources for those can be from online common-sense databases or world wide web among others. Acknowledging the need to limit the search space and integrate various cues to guide the search, [4] proposed indirect search. Indirect search as a search strategy is a simple and powerful idea: its to nd another object rst and then use it to facilitate nding the target object, e.g. nding a table rst while looking for a landline phone. Tsotsos [13] approached the problem by analyzing the complexity of the AVS problem and showed that it is NP-hard. Therefore we must adhere to a heuristics based solution. Ye [15] formulated the problem in probabilistic framework. In this work we consider the case where and Po () are both unknown. However, the robot is given probabilistic default knowledge about the relation betweeen objects and the occurences of objects in difference room category following our previous work [1, 6]. B. Contributions The contributions of this work are four fold. First we provide the domain adaptation of a hierarchical planner to address the AVS problem. Second we show how to combine semantic cues to guide the object search process in a more complex and larger environment than found in previous work. Third, we start with an unknown map of the environment and provide an exploration strategy which takes into account the object search task. Four, we present real world experiments searching for multiple objects in a large ofce environment, and show how the planner adapts the search behavior depending of the current conditions. C. Outline The outline of this paper is as follows. First we present how the AVS problem can be formulated in a principled way using a planning approach (Section II). Section III provides the motivation for and structure of various aspects of our spatial representation. Finally we showcase the feasibility of our approach in real world experiments (Section IV). II. P LANNING For a problem like AVS which entails probabilistic action outcomes and world state, the robot needs to employ a planner to generate exible and intelligent search behavior that trade off exploitation versus exploration. In order to guarantee optimality a POMDP planner can be used in, i.e. a decision theoretic planner that can accurately trade different costs against each other and generate the optimal policy. However, this is only tractable when a complex problem like AVS is applied to very small environments. Another type of planner are the classical AI planners which requires perfect knowledge about the environment. This is not the case since both and Po () are unknown. A variation of the classical planners are the so called continual planners that interleave planning and plan monitoring in order to deal with uncertain or dynamic environments[3]. The basic

idea behind the approach is to create an plan that might reach the goal and to start executing that plan. This initial plan takes into account success probabilities and action costs however it is optimistic in nature. A monitoring component keeps track of the execution outcome and noties the planner in the event of the current plan becoming invalid (either because the preconditions of an action are no longer satised or the plan does not reach the goal anymore). In this case, a new plan is created with the updated current state as the initial state and execution starts again. This will continue until either the monitoring component detects that the goal has been reached or no plan can be found anymore. In this paper we will make use of a so cal

ecmr11 proceedings

Documents