bio-inspired solutions for intelligent android perception and...

72
Bio-Inspired Solutions for Intelligent Android Perception and Control Emil M. Petriu University of Ottawa June 2013

Upload: others

Post on 17-Mar-2021

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Bio-Inspired Solutions for

Intelligent Android

Perception and Control

Emil M. Petriu

University of Ottawa

June 2013

Page 2: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

photo by Peter Thornton, uOttawa Gazette

Page 3: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

In order to naturally blend within human society, the new-generation robots should

not only look as humans, but should also behave as much as possible as

humans. They are expected to be, as initially imagined by Čapek in his R.U.R. Rossum's Universal Robots play, anthropomorphic artefacts, androids, enabled to

think on their own and governed by Asimov’s laws of robotics hardwired into every robot's positronic brain.

While for a long time, engineers have built upon mathematics, physics and

chemistry in order to develop an ever growing variety of industrial artefacts

and machines, this approach cannot anymore rise to the challenge of

designing these androids.

The time has now arrived to add biology and more specifically, human

anatomy, physiology and psychology to the scientific sources of knowledge to

develop a new, bio-inspired, generation of intelligent androids.

Advocating this emergent trend, this presentation will discuss a number of relevant

issues such as bio-inspired robot sensors and neural networks, human-robot

interaction techniques for symbiotic partnership, as well as moral, ethical,

theological, legal, and social challenges in a soon-to-be cyborg-society world.

Page 4: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

… HUMANS GETTING INTO

THE MATRIX AS AVATARS

Page 5: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Human

Visual

Feedback(s)

Tactile

Feedback(s)

Force

Feedback(s)

Video

Sensor(s)

Tactile

Sensor(s)

Force

Sensor(s)

Structured

Light

Audio

Feedback(s)Audio

Sensor(s)

Virtual_Environment / Real_World Interfaces

Virtual Object Manipulation

Object Shape & Behavior

ModelsObject Interaction Models

Computer Generated

Objects

Object RecognitionMotion Tracking

Sensor Data Fusion &

Interpretation

Human’s

Avatar

VIRTUAL

SCENE

AI –enabled Avatar

Animation Script The Matrix

AI-enabled

Avatar

Page 6: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

KINECTTM

Page 7: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Facial Expression Recognition using a 3D

Anthropometric Muscle-Based Active Appearance Model

• Facial Action Coding System

– 7 pairs of muscles + “Jaw Drop” = Expression Space

• Muscle “contractions” control mesh deformation in “Anthropometric-Expression (AE)” space

• Texture intensities are warped into the geometry of the shape

– Shape: apply PCA in AE space

– Appearance: apply PCA in texture space

• Model defined by rigid (rotation, translation) and non-rigid motion (AE)

• Model instances synthesized from AE space,

M.D. Cordea, E.M. Petriu, D.C. Petriu, "Three-

Dimensional Head Tracking and Facial Expression

Recovery Using an Anthropometric Muscle-Based Active

Appearance Model," IEEE Trans. Instrum. Meas., vol. 57,

no. 8, pp. 1578 – 1588, 2008.

Page 8: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Facial Expression Recognition

• Person Dependent

• Person Independent

Page 9: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

CyberForce®

CyberTouch™

CyberGrasp™

CyberGlove®

Immersionn_3D Interaction <http://www.immersion.com/>

Page 10: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

A tactile human interface placed on the operator's palm allows the human

operator to virtually feel by touch the object profile measured by the tactile sensors

placed in the jaws of the robot gripper (E.M. Petriu, W.S. McMath, "Tactile Operator

Interface for Semi-autonomous Robotic Applications," Proc.Int. Symposium on Artificial Intell. Robotics Automat. in Space, i-SAIRS'92, pp.77-82, Toulouse, France, 1992.)

Page 11: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Tactile fingertip human interface developed

at the University of Ottawa.

It consists of miniature vibrators placed on

the fingertips. The vibrators are individually

controlled using a dynamic model of the

visco-elastic tactile sensing mechanisms in

the human fingertip.

Page 12: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

… AI – ENABLED AVATARS

GETTING OFF THE MATRIX AS

INTELLIGENT AUTONOMOUS ANDROIDS

Page 13: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Matrix-trained, AI – enabled, avatar gets into the Real World as an

intelligent android able to interact and collaborate with humans

Virtual Object Manipulation

Object Shape & Behavior

ModelsObject Interaction Models

Computer Generated

Objects

Object RecognitionMotion Tracking

Sensor Data Fusion &

Interpretation

AI-enabled

Avatar

VIRTUAL

SCENE

AI-enabled Avatar

Animation Script The Matrix

Intelligent

Android

Human’s

Avatar

Real World

Human

Page 14: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

“The idea of the uncanny valley was

proposed by Masahiro Mori in 1970. His

idea was that increasing humanness in a

robot was positive only up to a certain

point …. beyond which, the not-quite-

human object strikes people as creepy.”

Crossing the uncanny valley: As computer graphics and

robots get more human, they often seem more surreal [The Economist, Nov 18th 2010 , http://www.economist.com/node/17519716 ]

Page 15: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori
Page 16: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

“InMoov”, the first Open Source life size humanoid robot you

can 3D print and animate, Gael Langevin’s project, January 2012

http://www.inmoov.fr/project/ ,

https://www.youtube.com/watch?v=vXJhgvVal7Y

“Gael Langevin is a French modelmaker and sculptor. He works for the biggest brands since

more than 25 years. InMoov is his personal project, it was initiated in January 2012

InMoov is the first Open Source 3D printed life-size robot. Replicable on any home 3D printer

with a 12x12x12cm area, it is conceived as a development platform for Universities,

Laboratories, Hobbyist, but first of all for Makers. It’s concept, based on sharing and community,

gives him the honor to be reproduced for countless projects through out the world.”

Page 17: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

For many centuries, engineers were building upon mathematics and

natural science principles from mechanics, electricity, and chemistry

in order to develop an ever growing variety of more efficient and smarter

industrial artefacts and machines.

The time has now arrived to

add biology - and more

specifically, human anatomy,

physiology and psychology –

to the scientific sources of

knowledge for engineers to

develop a new generation of

bio-inspired intelligent machines.

Page 18: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Model of the real world perceived by the human brain through sensory organs

Real/Material World

Biology-Inspired Robot Perception &

Action Mechanisms for Androids

Page 19: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

The sensory cortex: an oblique strip, on

the side of each hemisphere, receives

sensations from parts on the opposite side

of the body and head: foot (A), leg (B, C,

hip (D), trunk (E), shoulder (F), arm (G, H),

hand (I, J, K, L, M, N), neck (O), cranium

(P), eye (Q), temple (R), lips (S), cheek (T),

tongue (U), and larynx (V). Highly sensitive

parts of the body, such as the hand, lips,

and tongue have proportionally large

mapping areas, the foot, leg, hip, shoulder,

arm, eye, cheek, and larynx have

intermediate sized mapping areas, while

the trunk, neck, cranium, and temple have

smaller mapping areas.

(from [H. Chandler Elliott, The Shape of Intelligence - The Evolution of the Human Brain, Drawings by A. Ravielli, Charles

Scribner’s Sons, NY, 1969])

Bio-Inspired Sensing & Perception

Page 20: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Artificial Neural Networks

Page 21: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Biological Neurons

Neurons are rather slow (10-3 s) when compared

with the modern electronic circuits. ==> The brain is

faster than an electronic computer because of its

massively parallel structure. The brain has

approximately 1011 highly connected neurons

(approx. 104 connections per neuron).

Dendrites carry electrical signals in into the neuron

body. The neuron body integrates and thresholds the

incoming signals.The axon is a single long nerve

fiber that carries the signal from the neuron body to

other neurons. A synapse is the connection between

dendrites of two neurons.

Memories are formed by the modification of the

synaptic strengths which can change during the

entire life of the neural systems.

Body

Axon

Dendrites

Synapse

Page 22: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Looking for a model to prove that

algebraic operations with analog

variables can be performed by logic

gates, Professor J. von Neuman

advanced in 1956 the idea of representing analog variables by the mean rate of random-pulse streams [J. von Neuman, “Probabilistic logics and

the synthesis of reliable organisms

from unreliable components,” in Automata Studies, (C.E. Shannon, Ed.), Princeton, NJ, Princeton University Press, 1956].

Page 23: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

FS+VFS-V

FS FSXQ

p.d.f.

of VR

1

2 FS.

-FS

+FS

1

V

X

0

-1

VRQ1-BIT QUANTIZER

X-FS

+FS

XQ

X

0

1

-1

XQ

CLOCKCLK

VRP

ANALOG RANDOM

SIGNAL GENERATOR

-FS +FS0

R

p(R)1

2 FS

+

+VR

V

R

Analog/Random-Pulse Conversion

Page 24: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Analog/Random-Pulse and Random-Pulse/Digital Conversion

• E.M. Petriu, K. Watanabe, T. Yeap, "Applications of Random-Pulse Machine Concept to Neural Network

Design," IEEE Trans. Instrum. Meas., Vol. 45, No.2, pp.665-669, 1996,

• E. Pop, E.M. Petriu, "Influence of Reference Domain Instability Upon the Precision of Random Reference

Quantizer with Uniformly Distributed Auxiliary Source," Signal Processing (EURASIP), North Holland, Vol. 5,

pp.87-96, 1983

Page 25: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Generalized b-bit analog/random-data conversion

VR

V

RVRQ

CLOCKCLK

VRP

b -BIT

QUANTIZER

X XQ

ANALOG RANDOM

SIGNAL

GENERATOR

-D/20

R

p(R)

1/D

+D/2

+

+

.(k+0.5)D(k-0.5)D.

XQXQXQXQ

XXXX

k

k+1

k-1

0

b D.

1/ Dp.d.f.of VR

D /2D /2

b D. (1-b) D.

.V= (k-b) Dk D.

E.M. Petriu, L. Zhao, S.R. Das, V.Z. Groza, A. Cornell, “Instrumentation Applications of Multibit

Random-Data Representation,” IEEE Trans. Instrum. Meas., Vol. 52, No. 1, pp. 175- 181, 2003.

Stochastic Data Representation

Page 26: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Quantizat

ion

levels

Relative mean

square error

2 72.23

3 5.75

4 2.75

... ...

8 1.23

... ...

analog 10 10 20 30 40 50 60 70

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

Moving average window size

Mean s

quare

err

or

1-Bit

2-Bit

Page 27: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Neural Network Architectures Using

Stochastic Data Representation

ΣΣΣΣF

Y = F [ w X ]Σj=1

m.j iij

SY

NA

PS

E

SY

NA

PS

E

SY

NA

PS

E

. . .. . . X mX 1 X i

wmj

wij

w1j

Page 28: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Auto-associative memory NN architecture

P1, t1 P2, t2 P3, t3

Training setRecovery of 30%

occluded patterns

30

P

30x1

30x30

n

30x1

a

30x1W

)*Hardlim ( P Wa =

Neural Network for Pattern Recognition

Page 29: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Neural Network vs.

Analog Computer Modelling

Both the Analog Computers and Neural Networks

are continuous modelling devices.

Neural Networks don’t require a prior mathematical

models. A learning algorithm is used to adjust by trial

and error during the learning phase the synaptic

weights of the neurons.

Page 30: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Discreet vs. Continuous Modelling of

Physical Objects and Processes

x

y

CONTINUOUS MODEL

• NO sampling => NO INTEPPOLATION COST

DISCREET MODEL

• sampling => INTERPOLATION COST y(j) = y(A) +

[ x(j)-x(B)] . [ y(B)-x(A)] / [x(A)-x(B)]

x

y

x(j)

y(j) = ?

A

B

Page 31: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Compare the performance of three NN architectures used for 3D

object shape modelling:

• Multilayer Feedforward (MLFF ) • Self-Organizing Map (SOM ) • Neural Gas Network

A.-M. Cretu, E.M. Petriu, G.G. Patry, “Neural-Network-Based Models of 3-D Objects for Virtualized Reality: A Comparative Study,” IEEE Trans. Instrum. Meas.," Vol. 55, No. 1, pp.99-111, 2006.

NN Modelling of 3D Object Shapes

Page 32: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

51096 points, 20-10-1,5 extra surfaces,d=0.055, 2000 epochs,5.2 hrs.

2500 points, 12-6-1,2 extra surfaces, d=0.06,1020 epochs,45 min.

MLFF Representation - Results19000 points, 14-7-1,4 extra surfaces,d=0.055, 1100 epochs,3.3 hrs

Page 33: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori
Page 34: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Sampling points selected with

the neural gas network for the ball.Elastic ball used

for experimentation.

(from A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and

Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006 ).

Page 35: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

(a) (b)

Real and modeled deformation curves using neural network for rubber under

forces applied at different angles:

a) F=65N, α1=10° and F=65N, α2=170°,

b) F=36N, α1=25°, and F=36N, α2=155

(from .A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and

Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006).

Page 36: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Pioneered by Zadeh in the mid ‘60s fuzzy logic provides

the formalism for modeling the approximate reasoning mechanisms specific to the human brain.

“In more specific terms, what is central about fuzzy

logic is that, unlike classical logical systems, it aims at

modeling the imprecise modes of reasoning that play an

essential role in the remarkable human ability to make

rational decisions in an environment of uncertainty and

imprecision. This ability depends, in turn, on our ability

to infer an approximate answer to a question based on a

store of knowledge that is inexact, incomplete, or not

totally reliable.” [ “Fuzzy Logic,” IEEE Computer Mag,April 1988, pp. 83-93: ]

FUZZY LOGIC

Page 37: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

ANALOG (CRISP)

-TO-FUZZY

INTERFACE

FUZZIFICATION

FUZZY-TO-

ANALOG (CRISP)

INTERFACE

DEFUZZIFICATION

SENSORS ACTUATORS

INFERENCE

MECHANISM

(RULE EVALUATION)

FUZZY RULE BASE

PROCESSThe basic idea of “fuzzy logic control”

(FLC) was suggested by L.A. Zadeh, “A

rationale for fuzzy control,” J. Dynamic Syst. Meas. Control, vol.94, series G,

pp.3-4,1972.

FLC provides a non analytic alternative

to the classical analytic control theory.

==> “But what is striking is that its

most important and visible

application today is in a realm not

anticipated when fuzzy logic was

conceived, namely, the realm of fuzzy-

logic-based process control,” [L.A.

Zadeh, “Fuzzy logic,” IEEE Computer Mag., pp. 83-93, Apr. 1988].

Early FLCs were reported

by Mamdani and Assilian in

1974, and Sugeno in 1985.

CONTROLLED

SYSTEM

DESIRED SYSTEM FUNCTION

Page 38: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

INPUT

OUTPUT

x*

y*

Classic control is based on a detailed I/O

function OUTPUT= F (INPUT) which maps

each high-resolution quantization interval of

the input domain into a high-resolution

quantization interval of the output domain.

=> Finding a mathematical expression for

this detailed mapping relationship F may be

difficult, if not impossible, in many applications.

INPUT

OUTPUT

Fuzzification

y*

x*

Defu

zzific

ation

Fuzzy logic control is based on an I/O function that maps

each very low-resolution quantization interval of the input

domain into a very low-low resolution quantization interval

of the output domain. As there are only 7 or 9 fuzzy

quantization intervals covering the input and output domains

the mapping relationship can be very easily expressed using

the“if-then” formalism. (In many applications, this leads to a

simpler solution in less design time.) The overlapping of these

fuzzy domains and their linear membership functions will

eventually allow to achieve a rather high-resolution I/O

function between crisp input and output variables.

Fuzzy Logic Control

Page 39: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

The key benefit of FLC is that the desired system behavior can be

described with simple “if-then” relations based on very low-

resolution models able to incorporate empirical engineering

knowledge. FLCs have found many practical applications in the

context of complex ill-defined processes that can be controlled

by skilled human operators: water quality control, automatic train

operation control, elevator control, etc.,

Page 40: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

There is tenet of common wisdom that FLCs are meant to successfully

deal with uncertain data. According to this, FLCs are supposed to

make do with “uncertain” data coming from (cheap) low-resolution and

imprecise sensors.

However, using a truck backing-up Fuzzy Logic Controller (FLC) as test

bed, experiments show that the low resolution of the sensor data

results in rough quantization of the controller's I/O characteristic:

Experiments have shown also show that it is possible to smooth the

I/O characteristic of a digital FLC by dithering the sensor data before

quantization

“FUZZY UNCERTAINTY” –

WHAT ACTUALLY IS “FUZZY” IN A FUZZY CONTROLLER ??

E.M. Petriu, J. Mao, "Fuzzy Sensing and Control for a Truck," Proc. VIMS-2000, IEEE Workshop on Virtual and Intelligent Measurement Systems, pp. 27-32, Annapolis, MD,

April 2000.

Page 41: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

ϕθ

d

Loading Dock

( , )x y Front Wheel

Back Wheel

(0,0)x

y

The truck backing-up

Design a Fuzzy Logic

Controller (FLC) able to back

up a truck into a docking

station from any initial position

that has enough clearance

from the docking station.

Page 42: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

0 4 5 15 20-4-5-15-20-5050

x-position

900 100080060030000-90 27001200 1500 1800

truck angle

00-250-350-450 250 350 450

LE LC CE RC RI

RB RU RV VE LV LU LB

NL NM NS ZE PS PM PL

ϕ

steering angle θ

0.0

1.0

1.0

0.0

Membership

functions for the

truck backer-

upper FLC

Page 43: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

PS

NS

NM

NM

NL

NL

NL

PM PM

PM

PL PL

NL

NL

NM

NM

NS

PS

NM

NM

NS

PS

NM

NS

PS

PM

PM

PL

NS

PS

PM

PM

PL

PL

RL

RU

RV

VE

LV

LU

LL

LE LC CE RC RI

ϕ

x

ZE

1 2 3 4 5

6 7

18

31 35343332

30

The FLC is based

on the Sugeno-style

fuzzy inference.

The fuzzy rule base

consists of 35 rules.

Page 44: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

0 10 20 30 40 50 60 70-40

-30

-20

-10

0

10

20

30

Time (s)

θ [deg]

0 10 20 30 40 50 60-50

-40

-30

-20

-10

0

10

20

30

40

Time (s)

θ [deg]

Time diagram of digital FLC's output q during a docking

experiment when the input variables, j and x are analog and

respectively quantized with a 4-bit bit resolution

Page 45: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

A/D

A/D

Dither

Dither

Low-Pass

Filter

Low-Pass

Filter

Digital

FLC

Analog

Input

Analog

Input

Dithered

Analog Input

High Resolution

Digital Output

Low-Resolution

Dithered Digital

Input

High Resolution

Digital Output

Low-Resolution

Dithered Digital

Input

Dithered

Analog Input

Dithered digital FLC architecture with low-pass filters placed at the FLC's

outputs to smooth the non-linearity caused by the min-max composition

rules of the FLC.

Page 46: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

0 10 20 30 40 50 60 70-40

-30

-20

-10

0

10

20

30

Time (s)

θ [deg]

0 10 20 30 40 50 60-50

-40

-30

-20

-10

0

10

20

30

40

Time (s)

θ [deg]

Time diagram of digital FLC's output q

during a docking experiment when the

input variables, j and x are: (upper left) analog, (upper right) quantized with a

4-bit bit resolution, and (left) dithered

before being 4-bit bit quantized and

then a low-pass filter is placed at the

FLC's output

0 10 20 30 40 50 60 70-50

-40

-30

-20

-10

0

10

20

30Θ [deg]

Time (s)

Page 47: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

-50 50

0

10

20

30

40

50

X

Y

(a)

(b)

(c)

[dock]

initial position

(-30,25)

0

Truck trails for different FLC architectures: (a) analog ; (b)

digital without dithering; (c) digital with uniform dithering and

20-unit moving average filter

Page 48: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Dithered FLCDigital FLC

Analog FLC

Page 49: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

The skin of a human finger contains

four types of cutaneous sensing

elements distributed within the skin:

Meissner’s corpuscles for sensing

velocity and movement across the

skin; Merkel’s disks for sensing

sustained pressure and shapes;

Pacinian corpuscles for sensing

pressure changes and vibrations of

about 250 Hz; and Ruffini corpusclesfor sensing skin stretch and slip.

(from R. Sekuler and R. Balke,

Perception, McGraw-Hill, 1990)

BIO-INSPIRED ROBOT SENSING AND ACTUATION

Human Tactile Sensing

Page 50: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Robot arm with tendon driven

compliant joint

(E.M. Petriu, D.C. Petriu, V. Cretu, "Control

System for an Interactive Programmable Robot,"

Proc. CNETAC Nat. Conf. Electronics, Telecommunications, Control, and Computers, pp.

227-235, Bucharest, Nov. 1982, and E.M. Petriu,

D. Petriu, V. Cretu, "Multi-Microprocessor Control

System for an Experimental Robot with Elastic

Joints," Proc. Nat. Conf. Cybernetics, (in

Romanian), Bucharest, Romania, 1981).

Page 51: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

The tabs of the elastic overlay

are arranged in a 16-by-16 array

having a tab on top of each node

of Merkel’s disk-like matrix of

FSR elements sensing sustained

pressure and shapes.

This tab configuration provides a

de facto spatial sampling, which

reduces the elastic overlay's

blurring effect on the high 2D

sampling resolution of the FSR sensing matrix.

• P. Payeur, C. Pasca, A.-M.Cretu, E.M. Petriu, “Intelligent Haptic Sensor System for Robotic Manipulation,”

IEEE Trans. Instrum. Meas., Vol. 54, No. 4, pp. 1583 – 1592, 2005,

• W.S. McMath, S.K.S. Yeung, E.M. Petriu, "Tactile Sensing for Space Robotics," Proc. IMTC/89, IEEE Instrum. Meas. Technol. Conf., pp.128-131, Washington, DC., 1989.

Tactile Sensor

Page 52: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Example of GUI window (from [C. Pasca, Smart Tactile Sensor,M.A.Sc. Thesis, University of Ottawa, 2004])

Page 53: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Bio-inspired robot passive-compliant wrist allowing the tactile probe to

accommodate the constraints of the touched object surface and thus to increase

the local cutaneous information extracted during the active exploration process

under the force provided by the robot.

Page 54: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Feeling the temperature and thermal conductivityof the touched object surface. Rufini corpuscles-like

thermistors and a blood-vessel like source of heat (the white

coloured tube) distributed within the tactile sensor’s elastic skin.

Page 55: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

3D generic face deformed using muscle-based control

Avatar Face

Page 56: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Neutral Happy

Sad Surprised

Combining muscle

actions it becomes

possible to obtain a

variety of facial

expressions of

Marius’ avatar:

M.D. Cordea, E.M. Petriu, “A 3-D Anthropometric-Muscle-Based Active Appearance Model,” IEEE Trans. Instrum. Meas., Vol. 55, No. 1, pp. 91 - 98, 2006.

Page 57: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

• Plastic skull

• Latex rubber

• Proof-of-concept design

Android Face

P. Santos, E. de Castro Maia Jr., M, Goubran, E.M. Petriu, “Facial Expression

Communication for Healthcare Androids,” Proc. MeMeA2013, 8th IEEE Int. Symp. on Medical Measurement and Applications, pp. 44-48, Ottawa, ON, Canada, May 2013

Page 58: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

P. Santos, E. de Castro Maia Jr., M, Goubran, E.M. Petriu, “Facial Expression

Communication for Healthcare Androids,” Proc. MeMeA2013, 8th IEEE Int. Symp. on Medical Measurement and Applications, pp. 44-48, Ottawa, ON, Canada, May 2013

Page 59: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Avatar-Android Face Expressions Mapping

From left to right: neutral, happiness, sadness, surprise, anger, fear, disgust

P. Santos, E. de Castro Maia Jr., M, Goubran, E.M. Petriu, “Facial Expression Communication for Healthcare

Androids,” Proc. MeMeA2013, 8th IEEE Int. Symp. on Medical Measurement and Applications, pp. 44-48,

Ottawa, ON, Canada, May 2013

Page 60: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Face and Lip Animation Using Model-based Audio and Video Coding

M. D. Bondy, E. M. Petriu, M. D. Cordea, N. D. Georganas, D. C. Petriu, T. E. Whalen, “Model-based Face and Lip

Animation for Interactive Virtual Reality Applications”, Proc. ACM Multimedia 2001, pp. 559-563, Ottawa, ON, Sept. 2001

Page 61: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

The parameters of the lip contour model

xo, yo = the origin of the outside parabolas;

xi, yi = the origin of the inside parabolas; Bo= outer height; Bi = inner height; Ao = outer

width; Ai = inner width; D = depth of ‘dip’;

C = width of ‘dip’; E = offset height of

cosine function; tordero = top outside

parabola order; bordero = bottom outside

parabola order; orderi =inside parabola order

(same on both top an bottom).

The lip contur model used in the mapping:

The only parameters of the lip model that are associated to the cepstral coefficients are the

outer width Ao and the outer height Bo. Relations can be found linking the parameter values

of the inner contour of the lip model to the parameter values of the outer contour.

Therefore, estimating the inner contour values from the audio signal would be redundant.

Page 62: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Examples of the lip

model being molded

to the shape of the

speaker lips

Comparing the speech-

driven and the real lip shape

for a female speaker

saying in French the ten

digits: zero, un, deux,...neuf.

M. D. Bondy, E. M. Petriu, M. D. Cordea, N. D. Georganas, D. C. Petriu, T. E. Whalen, “Model-based Face and Lip

Animation for Interactive Virtual Reality Applications”, Proc. ACM Multimedia 2001, pp. 559-563, Ottawa, ON, Sept. 2001

Page 63: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Behaviour-Based

Android Control

Page 64: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

ANIMATION

SCRIPT

Voice

synthesizer

Face

muscle-

activation

instructions

Joint-

activation

instructions

Face Model

(Facial Action Coding )

Body Model

(Joint Control )

3-D ARTICULATED ANDROID MODEL

Android /“Machine” – level Instructions

Story-level

Instructions

INTERPRETER/COMPILER

INVERSE KINEMATIC CONTROL

Page 65: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

STORY-LEVEL INSTRUCTIONS

…..

DaneelA sits on the chair#4.

DanielA writes “Hello” on stationary.

He sees HappyCat under the white table .

DaneelA starts smiling.

HappyCat grins back.

……

BEHAVIOUR-LEVEL (“MACRO”) INSTRUCTIONS

…..

DanielA’s right hand moves the pen to follow the trace representing “H”.

DanielA’s right hand moves the pen to follow the trace representing “e”.

DanielA’s right hand moves the pen to follow the trace representing “l”.

DanielA’s right hand moves the pen to follow the trace representing “l”.

DanielA’s right hand moves the pen to follow the trace representing “o”.

……

Page 66: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

DanielA’s specific style

of moving his right arm

joints to write “H”

( NN model capturingDanielA’s writing personality )

Rotate Wrist to a i

Rotate Elbow to b j

Rotate Shoulder to g k

Wrist

Elbow

Shoulder

x

y

z

3-D Model of DanieA’s Right Hand

BEHAVIOUR-LEVEL (“MACRO”) INSTRUCTIONS

DanielA’s right hand moves the pen to follow the

trace representing “H”.

Page 67: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Human & Android & Cyborg

Hyper-Society

Page 68: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

TECHNOLOGICALLY ENHNCED HUMAN - CYBORG

eye glasses, binoculars, IR night

vision, HMD for augmented VR,...

gloves (baseball glove), hand tools

footwear, skates, bike, exoskeleton,..Knee Joint +

Artificial Knee Joint

Ear +Hearing Aid

Implant

Eye +Artificial Cornea

HUMAN

Heart +Pacemaker

Hand +Artificial Hand

Nose +Artificial Smell

Tongue +Artificial Taste

Page 69: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Brain Prosthesis

Brain Prosthesis which learns/models with an ever increasing fidelity the behaviour of

the natural brain so it can be used as behavioural-memory prosthesis (BMP) to make

up for the loss in the natural brain’s functions due to dementia, Alzheimer disease, etc.

It is quite conceivable that such a BMP could arrive in extremis to complete replace the

functions of the natural brain.

“Immortality by 2045 or bust: Russian tycoon

wants to transfer minds to machines

Russian billionaire Dmitry Itskov speaks to the Global

Future 2045 Congress, Saturday, June 15, 2013 at

Lincoln Center in New York. Some of humanity’s best

brains are gathering in New York to discuss how our

minds can outlive our bodies.” [Ottawa Citizen, June

15, 2013, http://www.ottawacitizen.com/business/Immortality+2045+bust+

Russian+tycoon+wants+transfer+minds/8531949/story.html]

Page 70: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Cyber/Machine

Society/World

{Intelligent Androids}

Human

Society/World

{Human Beings}

Asimov’s laws of the

robotics:

1st law: “A robot must notharm a human being or,through inaction allow oneto come to harm”.

2nd law: “A robot must

always obey human beings

unless that is in conflict with

the 1st law”.

3rd law: “A robot must

protect itself from harm

unless that is in conflict with

the 1st and 2nd law”.

Page 71: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Asimov’s laws of the robotics:

0th law: "A robot may not injure humanity or, through inaction, allow humanity to come to harm."

1st law- updated: “A robot must not harm a human being or, through inaction allow one to come to harm, unless this would violate the 0th law."

2nd law: “A robot must always

obey human beings unless that is

in conflict with the 1st law”.

3rd law: “A robot must protect itself

from harm unless that is in conflict

with the 1st and 2nd law”.

__________

[*] I. Asimov, Robots and Empire, Doubleday & Co., NY 1985, p.291

Cyber/Machine

Society/World

{Intelligent Androis}

Human

Society/World

{Human Beings}

Cyborg

Society/

World

{Cyborgs}

Moral, Ethical,

Theological, Legal, Biological,

Psychological Social,

Economic Challenges

Page 72: Bio-Inspired Solutions for Intelligent Android Perception and ...petriu/istas13-BioInAndroid...Avatar Real World Human “The idea of the uncanny valley was proposed by Masahiro Mori

Thank You!