Lőrinc Márton, Piroska Haller, Tamás Vajda, Zoltán Szántó, Tamás Szabó, Hunor Sándor:

Distant control of robotic agents over WLAN

2

Contents Teleoperated robotic systems The effect of the communication delay

on teleoperation Data transfer rate control for teleoperation

systems Experimental measurements Conclusions

3

Teleoperated robotic systems Telerobotics is the area of robotics concerned with

the control of robots from a distance, chiefly using wireless connections (like Wi-Fi, Bluetooth, and similar), or the Internet.

A remote manipulator, is a device which, through a communication medium, allows a mechanism to be controlled by a human operator. The purpose of such a device is usually to move or manipulate hazardous materials for reasons of safety.

Distant controlled mobile robots is also an important application area of the teleoperation.

4

It consists of a Master device and a Slave robot

Unilateral teleoperation: the operator moves the slave (generally low power robot), the master follows the motion of the slave

Bilateral teleoperation: the master (when it is in contact with the environment) reflects back the contact force to the slave, which is reflected to the operator

Early teleoperation system(no network)

Teleoperated robotic systems

5

Teleoperated robotic systems

6

Telepresence When sufficient amount of

sensor information (vision, sound, force) is brought from the teleoperator site to the operator he or she feels physically present in the teleoperator site

Called also tele-existence Important information is

transferred and dangerous/noise is filtered

7

Teleoperation ower networks

Operator Haptic Device Communication Network Master robot Camera and Environment

Display

8

Teleoperation ower networks

The main goals during teleoperation system design: Maintain the stability of the teleoperation

irrespective of the behavior of the operator or the environment.

Provide a good transparency for the system - position tracking and force reflection

FIRST 9

Instability due to Communication Delay

10

Instability due to Communication Delay

11

How to deal with the instability problem?

Extend the control software of the master and slave robots with such stabilizer algorithms, that can assure the stability in the presence of delay.

However, these stabilizers always modify the received signals, compromising the transparency.

Try to reduce the communication delay and jitter.

12

Teleoperation systems over WLAN PCh – Position channel

Master Slave FCh – Force channel

Slave Master VCh – Video channel

Slave Master DCh – Other data

channels in the same WLAN

Master Slave

13

The concept of feedback control

e – controlled variable (process output) u – control signal, actuates the controlled

process d – not measurable disturbance signal r – reference signal, encodes the desired

behavior of e

14

The concept of feedback control

Let a controllable process given by a recursive model

The (optimal) control problem: find such that

)

the set of the controls which assure that the closed loop systemis stable (in some sense) for bounded .

15

Control designOur original problem is to keep the delay and its fluctuation in the communication medium as small as possible. We want to formulate it in a feedback control approach. Accordingly, we need: Measurements at Controlled Process output (e) Controlled actuation at Controlled Process input (u)

16

Measurements To measure the delay that occurs in PCh and FCh,

the clock difference between the master and slave computers is determined ().

The sent time instant is sent to the receiver side in the header of communication packet. By measuring the receive time instant , the delay () suffered by the corresponding packet is computable as:

17

Measurements The difference between two consecutive

receive time instants (Packet Delay Variation) is also be calculated. Denote the sending period in PCh and FCh by . If a packet is sent at a time instant [i − 1], the next one is sent at the time instant [i] = [i − 1] + T.

18

The control error Define the following combined errors

Here, E denotes the expected value. The value above has to be kept as small as

possible.

19

Actuation

In a teleoperation system the video stream from the slave to master is the data flow with the highest transfer rate.

By regulating the data transfer in VCh (u), the performances in the FCh and PCh can be adjusted.

The amount of video data can be modified using several approaches. the sending period over the video channels can be modified. the size of the sent video frames can be changed. the quality of the sent image can be modified.

(image compression techniques)

20

Disturbances

The communication channels of other applications, which are independent of the teleoperation application, also compete with the channels of the teleoperation. They also have the same chance of transmission.

The cumulated transfer rate of the other channels is denoted by d.

21

Disturbances In wireless networks the maximum available transfer rate

of an end node depends on the measured radio signal strength between the end node and the access point. The maximum available transfer rate of an end node automatically changes over time to maintain a reliable link between the devices in the WLAN. This is done by the dynamic rate scaling algorithm which runs on the wireless access point and it is application independent.

If the slave moves further away from the access point, its maximum available transfer rate gets reduced by the scaling algorithm. The rate decrease is done incrementally to pre-defined levels in function of the signal strength.

22

Controller Design During controller design it has to be taken into

consideration that there are multiple objectives that have to be satisfied.

Firstly, the delay and the jitter in FCh and PCh should be kept under prescribed limits.

Secondly, as good video transmission quality as possible has to be provided for all the cameras in the system.

Accordingly, the controller design can be formulated as a multi-objective optimization problem.

23

Theoretical approach Lexicographic optimization: consider a set of

N objective functions:

with (possible) constraint Arrange the objective functions in order of

importance. Formally, the problem can be defined as:

24

The objective functions Objective one: keep the delay small.

Let

The threshold value for is Objective two: assure good video

transmission quality:

25

Assumptions A1: For a fixed d value the partial derivate of

wrt. u can be approximated as:

where gr[k] results from:

A2: The expected value of the partial derivate of wrt. u satisfies

26

Minimization of Apply a gradient-like algorithm:

If the assumptions A1 and A2 hold, it can be shown that:

27

Minimization of As the obvious minimum of is . However, this control would lead to high jump in

the control signal. Consequently, the gradient descent method is applied in this case as well:

28

The video rate control law Combine the two gradient algorithm and

take into consideration the bound of the control signal

Here , where is given by

29

The developed software

30

Experimental measurements To modify the video rate in VCh, JPEG

compression was used. Influence of the video transfer rate on the

delay and jitter:

31

Experimental measurements

32

Experimental measurements Moving

robot – Video rate control not active

33

Experimental measurements Moving

robot – Video rate control active

34

Experimental measurements

Moving robot – Comparison

35

Extension for multiple cameras Teleoperation with more than one camera

PCh - Position channel Master Slave

FCh - Force channel Slave Master

VChi – Video channel Slaves Master

CChi – Channels for control signalsMaster Video slaves

36

Extension for multiple cameras The displayed video information for the human

operator received from the different cameras have different importance level.

A possible priority setting for the different video channels is:p0 - priority of the video channel from the slave robot

(VCh0).p1 < p2 - priority of those VCh’s, in the visual range of

which the tracked mobile robot is.p2 < p3 - priority of VCh’s from the other video slaves.

37

Experimental measurements The control signals (video transfer rates) for

different cameras:

38

Experimental measurements Three

cameras, video controllers not active.

39

Experimental measurements

Three cameras, video controllers active.

40

Conclusions To assure both the stability and transparency stability in teleoperation

systems reliable communication in the channels of the teleoperation system has to be assured.

We proposed a video rate control algorithm which modifies the transfer rate in the video channels of the teleoperation system based on measurements performed in the position and force channels.

The control algorithm assures the best video quality corresponding to the prescribed delay and jitter values and actual disturbance level.

All of the performed experiments show that the proposed stream controller performs well in different traffic conditions, assuring small delay and jitter in the position and force channels of the teleoperation system.