Consistent World Models for Cooperating Robots:Separating Logical Relationships, Sensor Interpretation and Estimation

Andreas Schierl, Andreas Angerer, Alwin Hoffmann and Wolfgang ReifInstitute for Software and Systems Engineering, University of Augsburg, Augsburg, Germany

{schierl,angerer,hoffmann,reif}@isse.de

Abstract—When working with multiple independent mobilerobots, each has a different knowledge about its environment,based on its available sensors. This paper proposes an approachthat allows working with these different views by indepen-dently modeling the common logical relationships between theelements in the scene and the meaning of device-specific sensordata. Using these models, for each robot an estimation processcan automatically be derived that combines and processesthe available information to fill in the unknown geometricrelationships between the elements, while reacting to changesin the logical relationships. The proposed approach facilitatesthe consistent coordination of the robots through an applicationthat works on a common world model, while for execution eachrobot uses its available data and estimations. The approach isdemonstrated in a case study of two cooperating mobile robotsthat pick up an object and hand it over while passing eachother.

I. INTRODUCTION

Working with multiple, independent mobile robots, eachhas a different knowledge about its environment. Theseknowledge differences are based on the sensors availableto the individual robots: While for typical industrial robotsthe positions of workpieces and tasks are exactly definedand ensured through fixtures, mobile robots often work inan environment where this strict structure is not present.In conjunction with the low precision of mobile robotlocomotion, geometric uncertainty exists to an extent so thathas to be handled. Therefore, sensors are added – eitherintegrated into the robot, or mounted in the environment –to resolve the uncertainty based on measurements, and toconsistently update the world model – the representation ofthe application’s beliefs about its environment – accordingly.In popular approaches, the processing of sensor data isexplicitly implemented or configured for the given scenarioto form the robot’s world model.

In contrast, this paper proposes to independently definethe logical model (consisting of the logical relationships)and the measurement model (defining the meaning of sensormeasurements in a geometric context). Based on these def-initions and available sensor data, the proposed frameworkderives an estimation pipeline for the current situation.This estimation pipeline is created at run-time and providesconsistent data for different robots that are controlled fromthe same application, each working with its available data.

Figure 1. Handing over an object between two moving youBots.

Hence, the main contribution of this paper is a proposedseparation of logical relationships and sensor interpretationwhich allows automatic derivation an estimation pipeline. Indoing so, the estimation process no longer has to be con-figured manually, but emerges from the given relationshipsand measurements. Additionally, the estimation pipeline canautomatically adapt to changes in the environment. Forexample, when a robot grasps an object, sensor data nolonger defines the position of the object in the environment,but rather its position in the gripper.

As a further contribution, this approach allows consistentworld models between cooperating mobile robots, wherea common logical model is used, while for each robotits available sensors are used to derive information aboutgeometric aspects it knows about. This offers better reusabil-ity, because common logical models, but also definitionsof sensor interpretations can be used in other contexts,independent from the exact estimations they yield there.

As a real-world example, the interaction of two mobilerobots is analyzed. One KUKA youBot picks up a batonplaced in a predefined area of the room, and subsequentlytakes it to an area where it hands it over to a second youBot.This handover procedure takes place in motion while the

Paper accepted for 2017 First IEEE International Conference on Robotic Computing (IRC) © 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

DOI: 10.1109/IRC.2017.62

Figure 2. Software structure for distributed robots, adopted from [1].

youBots approach and pass each other. For this example, twoyouBots with laser scanners are used, along with a Viconoptical position tracking system. Both youBots are equippedwith Vicon markers, while the baton is detected using theon-board laser scanners. Fig. 1 shows the two parts of theapplication, first picking up the baton and subsequentlyhanding it over to the second youBot. The software forthis example is implemented in an object-oriented fashion,representing the robots and baton as objects and correctlytracking their logical and geometric relationships during theprocess, so that the world model remains consistent with thereal-world situation.

The remainder of the paper is structured as follows:In Sect. II, the overall approach is outlined. Then, thespecification of logical relationships (Sect. III-A) and sensormeaning (Sect. III-B) is explained. Sect. IV details howthe estimation is performed based on these specifications.In Sect. V, related approaches in theory as well as inexisting robot frameworks are compared. To conclude thepaper, Sect. VI explains the example implementation andexperimental results and Sect. VII gives a conclusion andoutlook.

II. APPROACH

When considering cooperating robots, the underlying soft-ware architecture influences how and where data is available.Fig. 2 (adopted from [1]) shows a way typical softwarearchitectures for distributed cooperating robots can be struc-tured. Each robotic device is represented and controlled bya device driver which is defined as the component that com-municates – usually in a real-time capable way – with thedevice through a vendor-specific interface. It has to forwardcontrol inputs to the device and receive feedback from thedevice. One or multiple device drivers belong to a systemwhere all knowledge is shared between the components (nolonger necessarily with real-time guarantees, e. g. ROS nodesbelonging to the same master). Hence, all components withinone system are allowed to access each other’s provided data,as well as to communicate with and send commands to eachother.

To perform a desired task, an application controls theinvolved systems to coordinate the work flow (e. g., twosystems in the example of cooperating, but independent

youBots). Within an application, data is read from andcommands are sent to controlled systems in order to havethe corresponding devices execute the overall task. Eachapplication performs its task based on its knowledge aboutcontrolled devices, systems and the environment. This in-cludes geometric information such as positions and ori-entations of the relevant objects, as well as informationabout the structure of (parts of) the environment (e. g.,topologies, maps), physical data (e. g., mass, friction) orshape data (e. g., 3D models). The world model data canbe differentiated into dynamic and static knowledge. Whilestatic knowledge (e. g., given maps, shapes or physical data)is valid and available everywhere, dynamic knowledge (e. g.,positions or sensor data) may be known in only one systemor be different among different systems. The latter can bethe case for the position estimation of mobile robots.

The environment of a robot consists of various physicalobjects, each at its respective position. The contribution ofthe presented approach is to propose how to model logicalrelationships between these objects, i. e., the topology of theknown environment, as static knowledge in order to inferdynamic knowledge automatically and in a consistent way.Inferring a consistent world model is especially importantin the case of multiple systems in one application, such asdistributed robots that have to work together to achieve atask. For example, on-board sensors a robot is equipped withare only accessible in the system the robot is part of and,thus, can only be used for commands regarding this system.The responsibility for static knowledge, as it is the commonpart of the world model, lies at the application. Hence, theapplication manages topology changes, e. g., if and how apriori unknown objects are integrated into the topology.

Depending on the amount of structure in the environment,some geometric positions of objects are constant and canthus directly be modeled as static knowledge. Other posi-tions however change over time and, thus, cannot be givenexactly for an application that should be reusable later. Still,in order to work with them the application has to know aboutthe existence of these objects and their logical relationshipsto further objects. Although these objects do have an exactposition in the physical world, the application initially hasno precise position information, which limits the amount ofinteraction that can be performed with these items.

To improve this situation, sensing can be employed to givethe robot a glance of its environment. Based on sensor data,some positions (and velocities) of objects can be recovered,contributing to the geometry and, thus, dynamic knowledgeof the application. To facilitate this, the interpretation of sen-sor measurements in a geometric context has to be defined.At application run-time, these interpretation definitions andincoming sensor data can be used to update the unknownor uncertain parts of the world model, so that it reflectsa consistent interpretation of the received sensor data. Insoftware, this process is performed through the introduction

Relation

World View

«DynamicKnowledge»

EstimatedPosition

«DynamicKnowledge»

Sensor Data

«StaticKnowledge»

Frame

«StaticKnowledge»

Sensor

«StaticKnowledge»

FixedPosition

«StaticKnowledge»

Placement

«StaticKnowledge»

StaticConnection

«StaticKnowledge»

Observation

«StaticKnowledge»

DynamicConnection

«StaticKnowledge»

Logical Relation

Estimator

GeometricRelation

«create»

«create»

*

*

1..*

1

2

Figure 3. Concept model for describing both the static and dynamicknowledge of a robotic application using different kinds of Relationsbetween spatial features of physical objects.

of logical and geometric relations, along with observationsdefining the semantics of sensor data and estimations derivedfrom this information.

Fig. 3 gives an overview of the concepts used to expressthis information. A robot’s or to be precise a system’sknowledge about itself and its surrounding world is calleda World View and consists of a set of Relations. A Relationcorrelates spatial features of objects like robots, workpiecesor obstacles in the application. Such spatial features aremodeled as Frames, which represent Cartesian coordinatesystems. One type of Relations are Logical Relations. Thosedescribe statically known logical correlations among ob-jects’ Frames. Objects that are connected to each other ina constant way, like a robot being mounted on a table,should be represented as Static Connections. Other tight,but variable connections, like the relation between a robotjoint and its associated links, should be modeled as DynamicConnections. For rather loose and changing correlations likeobjects being placed somewhere on the ground in the vicinityof another object, Placements are available.

The second type of Relations, Geometric Relations, formthe geometric model and can describe the geometric positionof an object relative to another, e. g., as a transformationmatrix. Geometric Relations correspond to (sequences of)Logical Relations. They can either be constant (Fixed Posi-tion, e. g., for the position where the robot is mounted onthe table), or be calculated from sensor values (EstimatedPosition).

However, Estimated Positions and their computation ruledo not have to be defined explicitly, but may emerge auto-matically during application runtime: The application onlyhas to define the interpretation of a sensor as Observations.Because an Observation is a special Relation, it describesa correlation between two Frames. Based on these Ob-servations and the Logical Relations, so-called Estimatorsautomatically derive ways to process sensors in order torecover geometric information about unknown positions, andprovide them as computation rules for Estimated Positions

Tracking Origin

youBot

Marker

Origin

Start Position

Sensor

Botyou

Positionrt PStar

SensorSeBaton

Static Connection

Dynamic Connection

Placement

Logical RelationL i l R l ti

Figure 4. Logical model of the youBot environment

computed from Sensor Data.

III. DEFINING THE LOGICAL MODEL ANDSENSOR INTERPRETATION

Basic ingredients of the world model are spatial fea-tures, called (coordinate) Frames. Each Frame represents a(named) position in space, including an orientation. Howevernote that a Frame per se does not know its absolute positionin space – the position is only defined relative to otherFrames through Geometric Relations. Concerning these spa-tial features, a logical structure exists that is modeled in thelogical model. Additionally, some features are part or targetof sensor measurements which have to be interpreted in theircorresponding context.

A. Defining the Logical Model

Between Frames, Relations can be established. As a basisof the environment model, Logical Relations describe thatcertain Frames are connected to each other, and includeinformation about the durability of the link. The structureof Frames and Logical Relations forms an undirected graph,which allows navigation between different Frames if a se-quence of Logical Relations exists that forms a path betweenthe two Frames.

Fig. 4 gives an example of Frames and Logical Relations.Frames are depicted as named groups of arrows, which givethe position as the intersection between the three arrows.Logical relations are shown as arrows between the Frames.The example shows Frames for a Start Position and a Track-ing Origin which are linked to an Origin frame, along witha youBot that is linked to its Start Position. Additionally, aMarker frame representing the position that can be trackedby the optical tracking system is connected to the youBot, aswell as the Sensor frame where the laserscanner is mounted.Additionally, a Baton is placed on the ground and thusconnected to the Origin.

In this scenario, the different relationships have a differentmeaning, and are thus represented as different types ofrelationships. The position of the Tracking Origin relative tothe Origin is fixed and given, and thus represented as a StaticConnection. Similarly, the position of the Marker and Sensorrelative to the youBot is fixed and constant. In contrast, theStart Position of the youBot in the world (relative to Origin)

Tracking Origin

youBot

Marker

Origin

Start Position

Bot

Positionrt PStar

SensorSe

Tracking Origin

Mark

you

Positionrt Ptar

nsorSeBaton

ObservationOb ti

Logical RelationL i l R l ti

Figure 5. Excerpt of the World model for a youBot platform tracked usingan optical tracking system.

is not fixed, but the youBot is rather placed into the world.Thus, this relationship is a Placement, as well as the one tothe Baton. The position of the youBot relative to its StartPosition is controlled by the youBot and is thus modeledas a Dynamic Connection. Consequently, Static Connectionand Dynamic Connection represent persistent relations thatwill exist for the entire run time of an application, whilePlacements are transient and can be removed. Placementsand Static Connections are assumed to model a relationshipwith constant transformation, while the transformation ofDynamic Connections varies over time.

When the baton is grasped by the youBot, the Placementfrom Origin to Baton is removed, and a new Placementfrom the Gripper to the Baton is established. This topologychange that occurs at application run time has an influenceon the geometric relations and is detailed in the followingsections.

B. Defining the Sensor Interpretation

To derive a geometric model elaborating the logical modeldefined above, Geometric Relations providing transforma-tions and velocities have to be added. For relations thatare not given statically, this transformation often has tobe derived from sensor measurements. To achieve this, thesemantics of measured sensor data have to be defined inthe form of Observations that describe that a certain aspectof the World model is measured by a given sensor, or canbe calculated from given Sensor Data. For the scope ofthis paper, we limit ourselves to measurements that providethe full transformation and velocity between two Frames,however not limited to Frames that are directly connectedby logical relations. For more complex cases, measurementscould e.g. also define partial information such as the distancebetween two Frames or the distance to a given plane (e.g.for the height of a quadcopter) or give information aboutvelocities and accelerations only.

Looking into robot environments, different types of sen-sors and correspondingly observations occur: For logicalrelations inside Actuators, most can be measured throughproprioceptive sensors. Then, the sensor semantics can bedefined following the logical structure of the object: Inthe case of the youBot base, observations can be defined

Tracking Origin

youBot

Marker

Origin

Start Position

Sensor

Tracking Origin

Mark

Bot

Positionrt P

SensorSe

Positionrt P

nsorSe

Byou

rStarrtarS

Baton

ObservationOb ti

EstimationE ti ti

Logical RelationL i l R l ti

Figure 6. Estimations established for a youBot platform when all sensorsare available

that describe the position of the wheels relative to theirwheel mount frames using sensor data provided by thewheel encoders. Similarly, the transformation between StartPosition and the youBot is provided through odometrycalculations based on wheel encoders, so the observationsdescribe transformations that follow a logical relation.

For other logical relations, however, this is not the case:Assuming the youBot platform is driving on the floor startingat a position not exactly known, application geometry isusually modeled as a Placement from the Origin frame tothe youBot Start Position (cf. Fig. 4). Its transformationhowever cannot be measured directly, because the StartPosition is an intermediate concept that does not have adirect representation in the physical world (at least afterthe youBot platform has moved). Fig. 5 as an extensionto Fig. 4 gives an overview of this situation. It shows ayouBot platform on the floor that is tracked through anoptical tracking system. In addition to the logical model,three observations are given (represented as dotted arrows).As mentioned above, the position of the youBot relative to itsStart Position is given by odometry sensors. Additionally, theMarker of the youBot is tracked using the tracking system,so the observation between Tracking Origin and Marker isgiven by the tracking system. Furthermore, the Baton istracked from the Sensor through the laserscanner (with somepost-processing). The transformations defined by the secondand third observation cannot directly be used to describethe transformation of a Logical Relation, but first have to beconverted.

Looking at the three observations given here, the first andthird are only applicable on the youBot system, becausethey use sensor data that is only available to the youBot.The second observation however can be used whereverthe tracking system’s data is available, e.g. for a secondcooperating youBot.

IV. DERIVING GEOMETRIC INFORMATIONTHROUGH ESTIMATION

To coordinate the interplay between logical relations andobservations and to provide estimations for the uncertaintiespresent, the concept of Estimators on the system level isintroduced. An Estimator listens to changes in the logical

Tracking Origin

youBot

Marker

Origin

Start Position

Bot

SensorSe

Tracking Origin

Mark

B

Positionrt PStar

Baton

ObservationOb ti

you

Positionrt Ptar

nsorSeBaton

Unavailable sensor data

EstimationE ti ti

Logical RelationL i l R l ti

Figure 7. Estimations established for a youBot platform from an outsideview if only tracking data is available

relations as well as observations, and augments its WorldView with Estimated Positions that reflect one Logical Rela-tion or shortcut multiple ones. These Estimated Positionscan be seen as estimation pipelines processing the dataprovided by sensors as defined in the observations, givingcomputation rules for how to calculate the transformationand velocity if the sensor’s dynamic knowledge is availablein the corresponding system.

Given the example in Fig. 6, the Estimator creates threeEstimated Positions: The first relation goes from the StartPosition to the youBot frame and takes the value from theodometry sensor as a transformation. The second relationfrom Origin to Start Position is a bit more complex: Thetransformation can be combined from the transformationfrom Origin to Tracking Origin, followed by the trans-formation provided by the Observation, the transformationfrom the Marker to the youBot and the transformation fromyouBot to Start Position. For the last part, the transfor-mation provided by the first estimated Relation has to beused. Similarly, the third transformation can be computedusing the observed position of the Baton, along with theestimated position of the youBot. However, when anotherrobot observes the youBot from outside, the sensor datafor the odometry Observation as well as the laserscannermeasurements are not available. In this case, the Estimatorhas to create an Estimated Position directly from the Originframe to youBot (cf. Fig. 7).

A. Establishing and Updating the Geometric Model

At application run-time, Estimator implementations workas listeners that react to changes in the Observations, Logicaland Geometric Relations. The Estimator tries to find cyclesin the graph formed by Logical Relations and Observations,and uses their information to build new known EstimatedPositions. In cycle search, it tries to minimize the numberof Logical Relations without corresponding Geometric Re-lations required to form estimations in order to keep theestimations structurally as close to the logical structure aspossible. The resulting cycle then consists of an Observation,a (maybe empty) sequence of Geometric Relations, followedby a sequence of Logical Relations (that will be estimatedand should thus be as short as possible), and a (possibly

ObservationOb ti

EstimationE ti ti

Logical Relation

(a) Adding a Logical Relation

(b) Removing a Logical Relation

Figure 8. Estimation modifications when changing Logical Relations

empty) sequence of Geometric Relations closing the loop.For a found cycle, the Estimator takes the Observation’stransformation and velocity and converts it for the Framesforming the start and end of the Logical Relation sequence,so that calculation rules describing the overall behavior(position and velocity) of the Logical Relation sequence areavailable. These calculations are then used to define theEstimated Position, and subsequently evaluated wheneverthis aspect of the World model is accessed by the application,e.g. during motion planning.

When a Logical Relation is added (cf. Fig. 8(a)), anEstimator checks if any of its known Observations canbe used to form a cycle in the World view includingthe new Logical Relation. If so, an Estimated Position isestablished based on the Logical Relation and Observation.For Geometric Relations added, the Estimator checks if thenew Geometric Relation has an effect on any of the existingestimations. This is the case when the Relation influences thefirst or last Logical Relation resolved through the estimation,causing the Estimated Position to be recreated based on thenew situation. When a Logical Relation is removed (cf.Fig. 8(b)), the corresponding estimations become invalidand are removed. Similarly, removing Geometric Relationscan invalidate estimations if they occurred in the cycleused to build the estimation, so new estimations for thecorresponding logical connections have to be built.

Adding an Observation (cf. Fig. 9(a)) may allow resolving

(a) Adding an Observation

ObservationOb ti

EstimationE ti ti

Logical Relation

(b) Removing an Observation

Figure 9. Estimation modifications when changing Observations

one of the Logical Relations by forming a cycle including thenew Observation and building a corresponding estimation.If the resulting estimation contains a subset of the LogicalRelations used in another estimation, the latter estimationis removed and recreated with the option to use the newlybuilt estimation, bringing the estimations closer to the logicalstructure (e. g. in the last step of Fig. 9(a)). When anObservation is removed that has been used by estimations(cf. Fig. 9(b)), the corresponding Estimations are removedand new cycles for the corresponding Logical Relations aresearched.

As an example, a workpiece tracked by a sensor isconsidered: The workpiece lying on the floor is connected tothe ground using a Placement. Additionally, an Observationis given that provides the position of the workpiece relativeto the sensor, and thus allows the calculation of a trans-formation for the Placement. However, once the workpieceis grasped, the Placement on the ground is removed andanother Placement to the gripper is added. In this situation,the same Observation now has to be used to provide anotherEstimated Position. This case is automatically handled byEstimators.

B. Handling Delayed and Infrequent Data

In simple cases, the Estimator can assume that all sensorsused in Observations are precise and provide values all thetime, without any time delay. Then, the estimator can usethe latest position data from all Observations to define itsEstimated Positions.

For Observations referencing Sensor Data that are onlyprovided sporadically or delayed, a more complex Estimatoris required, however still following the method outlined inSect. IV-A. It still assumes that all Observations are pre-cise, but accepts that some Observations are only providedinfrequently, but with correct time stamps (as seen by thetracking system when an object is temporarily lost or alag in wireless network communication occurs). It furtherrespects the types of Logical Relations. For Placements andStatic Connections, it assumes that they are actually constantand only have to be changed to correct measurement errors(which is true for objects placed on the ground, as wellas for the Start Position of a youBot), while for DynamicConnections extrapolating with constant velocity is an ap-propriate estimation. In this case, Estimated Positions arecreated between the same Frames as in the simple case,however, transformations are taken from a consistent timesnapshot and extrapolated if required. While supportingmore use cases, this estimator has the disadvantage ofincreased memory usage and computation time: To calculateestimations for a consistent time, it has to keep track ofprevious values and times of the sensor data, and has toselect a corresponding time to use the data from. Especiallyon resource-constrained systems that need an estimator thatruns at a high frequency with hard real-time guarantees,

using a simple estimator instead can thus be a better optionwhen no greater time delays are to be expected for the sensordata.

V. RELATED WORK

Looking at control theory, the process of integratingsensor data into the world model is similar to the concept ofstate observers used along with the state-space representationof a system model: A system model consists of a stateequation describing the behavior of the system under theinfluence of the given system inputs, and an output equationthat defines the relationship between system state and themeasured variables. Standard estimation methods such asthe Kalman Filter [2] for linear problems allow processingsystem inputs and measurements to recover the state ofthe system, tracking the uncertainty of the present statevariables. For non-linear problems (that occur in robotics,e.g. through rotations that have an non-linear effect on therobot position when the robot drives forward), extensionsof the Kalman Filter [3] are available such as the ExtendedKalman Filter linearizing the problem around the given state,or the Unscented Kalman Filter that samples chosen pointsin the probability distribution. Furthermore, Particle Filters[4] allow tackling problems with greater uncertainty, suchas an initial localization of a robot in a known map. Thesemethods can also form the basis of further Estimators foruse in the proposed approach, introducing the handling ofuncertainty and Gaussian noise.

Looking at ROS, the frame graph is typically managedthrough the tf library [5], where a tree of geometric re-lationships between frames can be established. However,no semantic information is contained there, apart froma separation between static and dynamic transformations(through the /tf static and /tf topics). Estimation and sensorintegration for robot positions is typically performed throughspecialized components. According to ROS EnhancementProposal 105 [6], mobile platforms should be modeled withthree distinguished frames: The frame map represents theorigin of the robot environment (and a corresponding map),while odom represents the starting point of the robot andbase denotes the robot itself. The transformation betweenmap and base does not have to be continuous, but maynot drift over time, while the transformation between odomand base has to be continuous, but may exhibit unboundeddrift. In the frame tree used in ROS, odom is the parentframe of base, while map is the parent of odom. Thismodeling is similar to the model used in the proposedapproach, with odom representing the Start Position and maprepresenting a fixed frame (such as the Tracking Origin orOrigin). To handle sensor data, robot pose ekf [7] and laterrobot localization as described by Moore et al. [8] providedifferent estimation algorithms as ROS Nodes, including anExtended and Unscented Kalman Filter. These Nodes canprocess various sensor data, including global positioning

(a) Situation before handover (b) Situation during handover (c) Situation after handover

ObservationOb ti

Logical RelationL i l iR l ti

(d) Model before handover

ObservationOb ti

Logical RelationL i l iR l ti

(e) Model during handover

ObservationObservationOb ti

Logical RelationL i l iR l ti

(f) Model after handover

Figure 10. Passing the baton – model and reality

systems (GPS), inertial measurement units (IMU) and odom-etry (ODOM) data, and publish the transformation betweenmap and odom or the transformation between odom andbase. However, these estimation nodes have to be configuredmanually, and cannot be used with topology changes: For anobject first tracked using multiple sensors, and then graspedby a robot, the parent frame changes, and thus the estimationnode has to be reconfigured.

In OROCOS, sensor data and estimation can e.g. behandled through the iTaSC framework [9]. There, uncertaintycan be defined for object or feature coordinates, which isthen processed during task execution using standard estima-tion techniques (as introduced above). However, this worldmodel and observation uncertainty model is tied to one giventask and is only in effect during task execution. In contrast,in the proposed framework, the logical model, uncertaintiesand observations can be defined globally and independently,which allows sensor data processing between the executionof different tasks and the derivation of uncertainty andmeasurement models for a given task from the global model.

VI. EXPERIMENTAL RESULTSThe concepts introduced in this paper have been evaluated

in multiple scenarios, one of which uses two cooperatingyouBots. On the hardware side, each youBot is equippedwith a bionic soft gripper [10] mounted at the arm, a WiFiadapter, as well as with retro-reflective markers to provideposition tracking. On the system level, both youBots arecontrolled as separate systems without implicit data transfer,based on a C++ implementation of the Robot ControlCore [11] running the onboard computer under Linux withXenomai extensions to achieve real-time control. However,they are allowed to receive Vicon tracking data throughWiFi (that sometimes exhibits time delays), as well astasks from an application. On the application level, a single

application is used, written in Java based on the RoboticsAPI [12], executed on a standalone computer. However, thisapplication could as well have been executed on any of theyouBots.

In this application, a common world model is defined,consisting of the logical model as well as the availablesensors and their observations as detailed in the previoussections. For picking up the baton, the raw distance readingsof the laser scanner are first filtered, limiting them todistance readings that are within a radius of 50cm around theexpected pick up position. Then a pole detection algorithmis used that searches for clusters of a length correspondingto the expected diameter of the baton. The detected clusterposition is used as Sensor Data for the Observation of theBaton. Based on the (dynamically estimated) position ofthe baton, the pickup process is executed. For passing thebaton, the application commands the youBot platforms topass each other, while the gripper is commanded to movetowards the center position between the two youBots. Oncethe distance between the youBots is sufficiently small, thereceiving youBot closes its gripper to grasp, followed bythe other youBot releasing the baton. When both youBotsreach a certain distance after passing each other, the armmotion is stopped, completing the task. These tasks arespecified on the full world model, while for execution foreach system the tasks are translated into a representationthat only uses available data. This way, for each system acorresponding view is automatically derived at run-time andused for command execution, using only the sensor data andestimations available at each youBot.

Fig. 10 shows pictures of an example run1 of the handoverprocedure, along with the logical relations and observationspresent at the corresponding times. During execution, the

1A video is available at http://video.isse.de/consistentworldmodel

available estimated positions differ between both youBots:While each youBot knows its start position, it does not knowthe start position of the other youBot (cf. Fig. 6 and 7).Of course, both youBots could have resorted to a modelwhere the Vicon sensor measurement is used to calculatethe position of the youBot relative to the Origin (cf. Fig. 7).However, using a more fine-grained model where available(Fig. 6) provides better performance when the trackingsystem fails: Whereas in this case no further informationabout motion is available through the Vicon system, themotion of the youBot relative to its start position can stillbe tracked using odometry, assuming that the start positionremains stationary relative to the Origin. This estimation isbetter than a simple extrapolation of the previous motionbased on a constant velocity or acceleration scheme (whichhas to be applied in the case of Fig. 7), because it takesinto account the measured wheel revolutions, and can thushandle non-uniform motions.

To show the utility of this approach and the possible re-use, the same application was used in another system set-up: There, both youBots were configured to be in a single(simulation) system. In this case and without modifying theapplication or any estimation configuration, at run time onlya single world view was created for the simulation system,including estimation models for both youBots similar toFig. 6. Also in this scenario, the baton could be handed over.Further application examples using the proposed conceptsare explained in [13].

VII. CONCLUSION

In this paper, we introduced a separation between logical,geometric and measurement information for robotic worldmodels. Logical relationships describe correlations of phys-ical objects via their spatial features which can be reusedin different use cases or even applications. Moreover, suchcorrelations are often an inherent part of the model of arobotic device that thus only have to be modeled once. Usingthe introduced concepts of Observations as measurementdefinition based on spatial features and Estimators to dy-namically integrate the sensor data as geometric informationat run-time can be seen as a powerful modeling tool for robotprogramming, allowing the specification of relationships thatcan be used by different estimation techniques. An importantcontribution of this paper is that this separation automaticallyhelps to create consistent world views when dealing withdistributed robot systems or changing environments, becausechanges to the logical model are reflected in all World views.

Apart from continuous computations at run-time, themodeled relationships can additionally be used for offline-processing, allowing parameter estimation based on recordedsensor values through non-linear optimization. Possible usecases here are to determine the exact position of a trackingmarker relative to the robot it is attached to, or the calibratea robot by performing certain motions, recording the sensor

data and optimizing the system parameters in a way tominimize the difference between observations and systemmodel.

REFERENCES

[1] A. Schierl, A. Angerer, A. Hoffmann, M. Vistein, andW. Reif, “On structure and distribution of software formobile manipulators,” in Informatics in Control, Automationand Robotics; 12th International Conference, ICINCO 2015;Colmar, France, July 21-23, 2015; Revised Selected Papers,ser. LNEE, J. Filipe, K. Madani, O. Gusikhin, and J. Sasiadek,Eds. Springer, 2016, vol. 383, pp. 209–228.

[2] R. E. Kalman, “A new approach to linear filtering andprediction problems,” Journal of Fluids Engineering, vol. 82,no. 1, pp. 35–45, 1960.

[3] S. Julier, J. Uhlmann, and H. Durrant-Whyte, “A new ap-proach for filtering nonlinear systems,” in Proceedings of the1995 American Control Conference, vol. 3, Jun 1995, pp.1628–1632.

[4] B. Arulampalam, Beyond the Kalman Filter: Particle Filtersfor Tracking Applications. Artech House, 2004.

[5] T. Foote, “tf: The transform library,” in Proceedings of the2013 IEEE International Conference on Technologies forPractical Robot Applications (TePRA 2013), Apr 2013, pp.1–6.

[6] W. Meeussen. (2010, Oct) Coordinate frames for mobileplatforms. Online, accessed Feb 2016. [Online]. Available:http://www.ros.org/reps/rep-0105.html

[7] W. Meeussen and D. V. Lu. robot pose ekf.Online, accessed Feb 2016. [Online]. Available:http://wiki.ros.org/robot pose ekf

[8] T. Moore and D. Stouch, “A generalized extended kalmanfilter implementation for the robot operating system,” in Pro-ceedings of the 13th International Conference on IntelligentAutonomous Systems (IAS-13). Springer, Jul 2014.

[9] J. De Schutter, T. De Laet, J. Rutgeerts, W. Decre, R. Smits,E. Aertbelien, K. Claes, and H. Bruyninckx, “Constraint-based task specification and estimation for sensor-based robotsystems in the presence of geometric uncertainty,” The Inter-national Journal of Robotics Research, vol. 26, no. 5, pp.433–455, 2007.

[10] BionicTripod with FinGripper – versatile movement andadaptive grasping. Online, accessed Feb 2016. [Online].Available: https://www.festo.com/cms/de corp/9779.htm

[11] M. Vistein, A. Angerer, A. Hoffmann, A. Schierl, andW. Reif, “Flexible and continuous execution of real-timecritical robotic tasks,” International Journal of Mechatronicsand Automation, vol. 4, no. 1, pp. 27–38, Jan 2014.

[12] A. Angerer, A. Hoffmann, A. Schierl, M. Vistein, and W. Reif,“Robotics API: Object-Oriented Software Development forIndustrial Robots,” Journal of Software Engineering forRobotics, vol. 4, no. 1, pp. 1–22, 2013.

[13] A. Schierl, “Object-oriented modeling and coordination ofmobile robots,” PhD thesis, Universitat Augsburg, 2017.