Id

ea,

Co

ncep

tio

n,

an

d R

eali

zati

on

of

Learn

ing

Ab

ilit

ies f

or R

ob

ot

Co

ntr

ol

Usin

g a

Sit

uati

on

-Op

erato

r-M

od

el

Dir

k S

öff

ker

and E

lmar

Ahle

Chair o

f D

ynam

ics a

nd C

ontr

ol

Univ

ers

ity D

uis

burg

-Essen

Le

ctu

re M

ate

ria

l g

ive

n

at

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

1

Sö

ffker,

Ahle

:

Motivation

Initia

ting I

deas

Conception o

f th

e L

earn

ing S

yste

m

Realization o

f th

e A

rchitectu

re

Conclu

sio

n a

nd F

utu

re W

ork

Ou

tlin

e

Mo

tivati

on

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

2

Sö

ffker,

Ahle

:

Mo

tivati

on

Main

go

als

Buildin

g a

n a

uto

nom

ous learn

ing s

yste

m

Learn

ing fro

m inte

raction w

ith t

he e

nvironm

ent

Goals

can b

e c

hanged

Goal-

oriente

d inte

raction in a

real w

orld e

nvironm

ent

Ap

pro

ach

/Id

ea

Str

uctu

re t

he r

eality

and m

ap t

he s

tructu

ring t

o

the m

enta

l m

odel to

enable

pla

nnin

g a

nd learn

ing

Use t

he r

ela

ted S

ituation-O

pera

tor-

Model (S

OM

) fo

r th

e c

om

mon r

epre

senta

tion

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

3

Sö

ffker,

Ahle

:

Motivation

Initia

ting I

deas

Conception o

f th

e L

earn

ing S

yste

m

Realization o

f th

e A

rchitectu

re

Conclu

sio

n a

nd F

utu

re W

ork

In

itia

tin

g I

deas

Ou

tlin

e

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

4

Sö

ffker,

Ahle

:

In

teracti

on

In t

he

min

imum

two

‚syste

ms‘ are

inte

racting.

Syste

m –

Syste

m

Hum

an –

Hum

an

Hum

an –

Syste

m

>H

um

an

-Mach

ine-I

nte

racti

on

> H

um

an

-Mach

ine-S

yste

m

What

isin

tera

ction?

How

can

the

inte

raction

be

described?

Tw

o-w

ay r

eaction b

etw

een p

layers

or

syste

ms

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

5

Sö

ffker,

Ahle

:

Hu

man

-M

ach

ine-

In

teracti

on

I

Exam

ple

:Superv

isio

n a

nd

Contr

olof

Railw

ay

Tra

ffic

,

here:

Hagen E

lectr

onic

Opera

ting

Cente

r of th

eD

euts

che B

ahn A

G

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

6

Sö

ffker,

Ahle

:

Ex

am

ple

: E

lectro

nic

Op

era

tin

g C

en

te

r H

ag

en

(G

erm

an

Ra

ilw

ay

)

Hu

man

-M

ach

ine-

In

teracti

on

II

Exam

ple

:Superv

isio

n a

nd

Contr

olof

Railw

ay

Tra

ffic

,

here:

Hagen E

lectr

onic

Opera

ting

Cente

r of th

eD

euts

che B

ahn A

G

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

7

Sö

ffker,

Ahle

:

Wh

ich

is t

he a

deq

uate

descrip

tio

n?

(techn./

phsic

. valu

es >

info

rmation)

Hig

her g

oals

:-

sta

bility /

dynam

ics

-ro

bustn

ess

-observ

ability

-contr

ollability -

> a

uto

matic c

ontr

ol

Hu

man

-Mach

ine-I

nte

racti

on

VI (

and a

ssum

ptions)

Cau

sali

tyfr

om

the

cause t

o t

he

effect

i)

fin

al

ch

ain

ii) in

ner c

on

necti

on

s

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

8

Sö

ffker,

Ahle

:

Qu

ali

tati

ve

Mo

deli

ng

Ap

pro

ach

I

(str

uctu

ral

variable

syste

ms)

(Söffker,

1998f,

2001,

2003)

Sit

uati

on

:The t

erm

situation d

escribes a

fix

ed p

roble

m c

onste

llation

and d

enote

s t

he c

onsid

ere

d s

yste

m.

The s

ituation c

onsis

ts o

f an inner

str

uctu

re,

whic

h a

lso

allow

s t

he inte

gra

tion o

f tim

e-v

ari

ant

valu

es.

The g

raphic

al re

pre

senta

tion is r

ealized b

y

chara

cte

ristic (

C)

and inner

rela

tions (

R).

Diffe

rent

deta

iled g

raphic

al re

pre

senta

tions

are

possib

le.

r 1

C1

C2

C3

C4

r 2 r 3 r 4

C1

C2

C3

C4

C2 C

3

C4

r 2 r 3

C1

r 4

r 1

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

9

Sö

ffker,

Ahle

:

Qu

ali

tati

ve

Mo

deli

ng

II

(str

uctu

ral

variable

syste

ms)

(Söff

ker,

1998f,

2001,

2003)

Op

erato

r:

Opera

tors

are

used t

o r

epre

sent

functional connections o

f re

al w

orl

d

facts

. The c

onnection c

an b

e p

assiv

e (

constitu

tional)

or

active

('ability t

o c

hange s

om

eth

ing')

. O

pera

tors

repre

sent/

model

oute

r w

orl

d facts

.

The function o

f an o

pera

tor

is d

enote

d w

ith (

F),

as 'in

put' t

he

explicit a

nd im

plicit a

ssum

ption for

realization o

f F (

eA,

iA)

are

used.

For

deta

iled m

odeling k

now

n t

echniq

ues w

ill be u

sed.

The S

OM

-techniq

ue is w

ork

ing a

s a

meta

-modeling a

ppro

ach.

O

eA

1

eA

2 eA

3

iA1

C

F1

eA

x

iAx

C

:Fu

ncti

on

/ (

Ch

arakte

ris

tic)

:A

ssu

mp

tio

n

:Exp

licit

assu

mp

tio

n

:Im

plc

it a

ssu

mp

tion

:co

mm

en

t

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

10

Sö

ffker,

Ahle

:

r 1 r 2r 2

r 3

r 4Af

Bf

Ci

Cf

Ai Bi

Ci

Initia

l situation

Fin

al situation

Opera

tor

Str

uctu

ring u

sed t

o m

odel le

arn

ing,

pla

nnin

g,

and e

xecution p

rocess

[Söff

ker

2001]

Sit

uati

on

(m

od

el

of

scen

e)

Chara

cte

ristics

,

,

,

…

Set

of re

lations ,

,

,

…

Changed b

y o

pera

tor

Assu

mp

tio

n

Changes a

nd facts

of th

e r

eal w

orld

are

unders

tood a

s a

sequence o

f scenes c

hanged b

y a

ctions.

Op

erato

r (

mo

del

of

acti

on

)

Function a

s r

esult

Explicit a

nd im

plicit

assum

ptions a

s input

AiBiCiAf

r 1r 2

r 3r 4

Sit

uati

on

-Op

erato

r-M

od

el

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

11

Sö

ffker,

Ahle

:

Sit

uati

on

-Op

erato

r-M

od

el

Learn

ing

Assu

mp

tio

ns t

o r

eali

ze l

earn

ing

Pro

ble

m-d

ependent

str

uctu

res o

f th

e r

eal w

orld s

cenes identified

as s

tructu

red s

ituation-d

ependent

chara

cte

ristics a

nd r

ela

tions

The r

eal w

orld is m

odele

d b

y S

OM

that

the r

ele

vant

str

uctu

re o

f th

e s

cenes a

nd t

he s

ituation a

re e

qual

‘Tim

e-i

ndependent' o

pera

tors

(active o

r passiv

e)

rela

ted t

o t

he

pro

ble

m s

tructu

re a

re identified/l

earn

ed

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

12

Sö

ffker,

Ahle

:

Motivation

Initia

ting I

deas

Conception o

f th

e L

earn

ing S

yste

m

Realization o

f th

e A

rchitectu

re

Conclu

sio

n a

nd F

utu

re W

ork

Ou

tlin

e

Co

ncep

tio

n o

f th

e L

earn

ing

Syste

m

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

13

Sö

ffker,

Ahle

:

Arch

itectu

re

Overvie

w

Basic

syste

m is

imple

mente

d o

n t

he r

obot

Tra

nsla

tion t

o S

OM

description b

y s

ituation

inte

rpre

tation m

odule

Hig

her

cognitiv

e c

apabilitie

s

are

pro

cessed b

y t

he

cognitiv

e functions

Experience is s

aved

in t

he k

now

ledge b

ase

Refinin

g t

he k

now

ledge

base d

ue t

o inte

raction a

nd

infe

rence

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

14

Sö

ffker,

Ahle

:

Arch

itectu

re

Fro

m p

refi

ltered

to i

nte

rp

rete

d s

itu

ati

on

Scene

Pre

filtere

dsituation

Inte

rpre

ted

situation

23 c

hara

cte

ristics

14 r

ela

tions

Sensors

Pre

filter

Situation

inte

rpre

tation

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

15

Sö

ffker,

Ahle

:

Learn

ing

Cre

ation o

f new

situation

inte

rpre

tations

Adapta

tion o

f para

mete

rs o

f re

lations

Inclu

sio

n a

nd e

xclu

sio

n

of chara

cte

ristics

Tra

nsitio

n p

robabilitie

s

betw

een s

ituations

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

16

Sö

ffker,

Ahle

:

Learn

ing

Up

dati

ng

kn

ow

led

ge b

ase

Initia

lpre

filtere

dsituation

Fin

al

pre

filtere

dsituation

Opera

tor

Know

ledge b

ase c

onta

ins e

xperiences

Syste

m m

akes a

n o

bserv

ation

Queries a

ll e

xperiences w

ith t

he s

am

e o

pera

tor

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

17

Sö

ffker,

Ahle

:

Learn

ing

Exam

ple

: Level

3 -

Retr

acti

on

Initia

l scene

Fin

al scene

Opera

tor

Initia

l situation

Fin

al situation

red

[no]

yellow

[no]

blu

e

[no]

gre

en

[no]

red

[yes]

yellow

[yes]

blu

e

[yes]

gre

en

[yes]

red

[yes]

yellow

[no]

blu

e

[no]

gre

en

[no]

red

search for

object

red

[yes]

functionsearch for object returnstrueorfalse

inputs:

channel

local variables:

blob

blob:=false

for20 steps andblob=false

tilt camera to initial position

for12 steps andblob=false

ifblob on channelexiststhenblob:=true

elsetilt camera by 6°

ifblob=falsethenrotate robot by 10°

ifblob=truethen return true

else return false

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

18

Sö

ffker,

Ahle

:

Motivation

Initia

ting I

deas

Conception o

f th

e L

earn

ing S

yste

m

Realization o

f th

e A

rchitectu

re

Conclu

sio

n a

nd F

utu

re W

ork

Ou

tlin

e

Reali

zati

on

of

the A

rch

itectu

re

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

19

Sö

ffker,

Ahle

:

Reali

zati

on

Cli

en

t/S

erver S

tru

ctu

re

Robot

(serv

er)

perc

eiv

es

info

rmation fro

m t

he

environm

ent

by s

ensors

Pre

filtere

dsituation is b

uild

and s

end t

o t

he c

lient

Goal-

dependent

sele

ction o

f an o

pera

tor

by t

he p

lannin

g

module

usin

g d

ata

base

Opera

tor

is e

xecute

d a

s

action b

y t

he r

obot

Inte

raction r

ela

ted

know

ledge is s

tore

d in

data

base a

s e

xperience

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

20

Sö

ffker,

Ahle

:

Tw

o g

iven g

oals

Giv

en inte

rpre

tation a

nd o

pera

tors

22 e

xperiences

4 a

uto

nom

ous g

enera

ted m

eta

-opera

tors

Reali

zati

on

Test

setu

p

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

21

Sö

ffker,

Ahle

:

Reali

zati

on

Test

in l

ab

en

vir

on

men

t

Sint1

1Sint1

2Sint2

3

Sint3

5

Sint4

5=Sint4

dSint3

4 Sint3

4∗

Sint4

6=Sint4

dSint2

3Sint1

1

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

22

Sö

ffker,

Ahle

:

Reali

zati

on

Test

in l

ab

en

vir

on

men

t

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

23

Sö

ffker,

Ahle

:

Motivation

Initia

ting I

deas

Conception o

f th

e L

earn

ing S

yste

m

Realization o

f th

e A

rchitectu

re

Conclu

sio

n a

nd F

utu

re W

ork

Ou

tlin

e

Co

nclu

sio

n a

nd

Fu

ture W

ork

Idea,

Conception,

and R

ealization o

f Learn

ing A

bilitie

s

for

Robot

Contr

ol U

sin

g a

Situation-O

pera

tor-

Model

© for

all f

igure

s/i

llustr

ations b

y S

RS U

DuE

24

Sö

ffker,

Ahle

:

Co

nclu

sio

n

Full inte

gra

tion o

f pla

nnin

g a

nd learn

ing

Cre

ation o

f chara

cte

ristics a

nd s

ituations b

y t

he s

yste

m its

elf

Inte

gra

ting o

f lo

calization a

nd n

avig

ation s

oft

ware

Fu

ture W

ork

Realization o

f a c

ognitiv

e-b

ased inte

ractive s

yste

m

Learn

ing,

pla

nnin

g,

and e

xecution inte

gra

ble

in o

ne

arc

hitectu

re b

ased o

n t

he r

epre

senta

tional le

vel

modele

d w

ith a

Situation-O

pera

tor-

Model

Goal can b

e c

hanged w

ithout

changin

g d

ata

base o

r arc

hitectu

re

Diffe

rent

levels

of le

arn

ing r

ealized c

onceptional

Id

ea,

Co

ncep

tio

n,

an

d R

eali

zati

on

of

Learn

ing

Ab

ilit

ies f

or R

ob

ot

Co

ntr

ol

Usin

g a

Sit

uati

on

-Op

erato

r-M

od

el

Dir

k S

öff

ker

and E

lmar

Ahle

Chair o

f D

ynam

ics a

nd C

ontr

ol

Univ

ers

ity D

uis

burg

-Essen