the digitale transformation of human machine …...2016/04/14 · the digitale transformation of...

www.ima-zlw-ifu.rwth-aachen.de

The Digitale Transformation of Human Machine Interaction

Human-Technology-Interaction 4.0 and the future of interaction

between people and technology

5. Augsburger

Technologietransfer-Kongress

Univ.-Prof. Dr.-Ing. em. Klaus Henning

IMA/ZLW & IfU

Faculty of Mechanical Engineering

RWTH Aachen University

2

14.04.2016

Klaus Henning

Outline

I. The emerging world of interaction

� Breakthroughs

� … and a networked world

� A short history…

� … and the consequences

II. The intelligence of systems of systems

� Rediscover the Cybernetic approach

� The paradigm shift in artificial Intelligence: from Top-Down to Bottom-Up

� Approaches : multi agent systems in real-world applications

III. The new dimension of human machine interaction

� The role of the digital shadows

� It’s all about cognitive computing …

� … Addressing problems of “human-like” complexity

� … Copying human thought processes

� … Intuitive intelligent interaction with humans…

IV. Ready to go to 4.0?

3

14.04.2016

Klaus Henning

� Almost all people of this world have a mobile (7 Billions).

� Smartphones are already in charge of 30% of this world as the

first generation of „individual agents“. In 10 year the figure of

smart phones being in use will increase to the figure of people,

living on this world.

Just one line of development

From the mobile to the car robot

From this viewpoint just think about:

� In 30 years about 1 Billion cars will drive fully automatic –

they will behave as intelligent robots.

� In 30 years all of us will have „intelligent“ cloths.

� In 30 years iron of buildings has to be intelligent by law.

� ….

4

14.04.2016

Klaus Henning

4.0 will be everywhere

The digital transformation of all areas of live is just in the beginning

� The digital transformation is a global und local transformation of all areas of

live. It is unavoidable. It is a huge chance to recreate living, working and

learning in a worldwide context.

� These evolutions and revolutions are historically – if at all – comparable with

the introduction of mass printing facilities 500 years ago.

� The Internet of Things will invade into all devices of daily live.

� In the long run most of all reasonable devices technologies of this world will

be intelligent and perform a live long learning process by themselves and with

humans.

� 4.0 – Cyberphysical Systems - is the roadmap:

All areas will become „smart …“ (data, factory, logistics, home, grid,…..)

� The new 4.0 products, services and process will be the future strength of our

economy

5

14.04.2016

Klaus Henning

4.0 will be everywhere

The digital transformation of all areas of live is just in the beginning

� The smart home will be a interactive multimedia platform –

the fridge will speak, all energy consumptions will be

centrally controlled.

� The heating equipment will be intelligent (i.e. System Geniax

from WILO).

� Medical online control will be possible in the private home.

� Intelligent window glasses will absorb energy and distribute it

in a controlled way.

� The auto bumper will be intelligent and adapt it`s stiffness to

the environment.

� There will be an body embedded back up cardiovascular

pump in case of an cardiac arrest.

� The embedded computer system in my jacket will become

one of my personal agent, giving me advice and warnings.

Some Examples

6

14.04.2016

Klaus Henning

Human-Technology-Interaction 4.0

A new kind of technological intelligence

“Everything that we do now is tactile (in terms of interacting with our

environment), however, when we communicate with intelligent beings there is

very little tactile interaction. As our world becomes more intelligent, it will

require a different interactive approach.”

Bryan Brown, Human-Machine Technologies Inc. (http://www.hmtinc.org)

From a separated and interfaced interaction to an ubiquitous and inconspicuous

interaction between technology and humans.

7

14.04.2016

Klaus Henning

… leading to the 4th (industrial) (r)evolution...

Breakthroughs - A new era of artificial intelligence

Communication technologybandwidth and computational power

Embedded systemsminiaturization

Semantic technologiesinformation integration

Watson

2011

Google Car

2012

� Systems of “human-like” complexity

8

14.04.2016

Klaus Henning




… leading to the 4th (industrial) (r)evolution...

Breakthroughs - Everybody and everything is networked

Team

Robotics

Swarm

Robotics

Smart

Grid

Car2Infra-

structure

Smart

Factory

9

14.04.2016

Klaus Henning

The upcoming world

“Information Revolution”

Everybody and everything is networked. - Big Data & Cyber-Physical Systems

Power revolutionCentralized electric power

infrastructure; mass production

by division of labor

1st industrial revolutionMechanical production

systematically using the

power of water and steam

today

Digital revolutionDigital computing and

communication technology,

enhancing systems’ intelligence

Information revolutionEverybody and everything is

networked – networked

information as a “huge brain”

around 1750 around 1900 around 1970

Weidmüller, Vission 2020 - Industrial Revolution 4.0

Intelligently networked, self-controlling manufacturing systems)

“Internet of Things & Services, M2M or Cyber Physical Systems are much more

than just buzzwords for the outlook of connecting 50 billions devices by 2015.”Dr. Stefan Ferber, Bosch (2011)

Vision of Wireless Next Generation System (WiNGS) Lab at the

University of Texas at San Antonio, Dr. Kelley

„local“

to „global“

„local“

to „global“

10

14.04.2016

Klaus Henning

… towards a networked world

Not restricted to industry: cyber physical systems in all areas

Back to: The earth converted into a huge “brain”… (Tesla 1926)

Integrating complex information from multiple heterogeneous sources opens multiple possibilities of optimization:

e.g. energy consumption, security services, rescue services as well as increasing the quality of life

… and more

Building

automation

Smart grid

Room

automation

Smart

environment

Smart

metering

11

14.04.2016

Klaus Henning




… towards a world of systems of systems

And how do these systems work?

Power revolutionCentralized electric power

infrastructure; mass production

by division of labor

1st industrial revolutionMechanical production

systematically using the

power of water and steam

today

Digital revolutionDigital computing and

communication technology,

enhancing systems’ intelligence

Information revolutionEverybody and everything is

networked – networked

information as a “huge brain”

around 1750 around 1900 around 1970

Towards intelligent and (partly-)

autonomous systems AND systems of systems

?? Steering -

Controlling ??

12

14.04.2016

Klaus Henning

Outline


� Breakthroughs















13

14.04.2016

Klaus Henning

�

� Term: „governance“, to navigate

� Born around 1940

� 1948: “Cybernetics or control and

communication in the Animal and in the

machine” (Norbert Wiener)

� until 1953: Macy-Conferences

FeedbackloopCircular explanations for

systems behavior, self-

regulation

(Forrester, Ashby)

FeedbackloopCircular explanations for

systems behavior, self-

regulation

(Forrester, Ashby)

AutopoiesisSystem capacity to maintain

and stabilize itself

(Maturana, Varela)

AutopoiesisSystem capacity to maintain

and stabilize itself

(Maturana, Varela)

DecentralizationDecentralized navigation,

bottom up processes

(Stafford Beer)

DecentralizationDecentralized navigation,

bottom up processes

(Stafford Beer)

The Basis of thinking in system with high complexity und dynamics (Dynaxity)

The central elements of Cybernetics

EmergenceSpontaneous new properties,

swarm behavior

(Wolfram, Gell-Mann)

EmergenceSpontaneous new properties,

swarm behavior

(Wolfram, Gell-Mann)

Complex Systems

Multi-component systems

in complex interactions

(Stafford Beer)

Complex Systems

Multi-component systems

in complex interactions

(Stafford Beer)

14

14.04.2016

Klaus Henning

„symmetric“biological-centered

The rise of agent systems …

A paradigm shift in the definition of intelligence

‘only biological brains

are intelligent’

‘any behaviorally equivalent functional

system is intelligent’

Triple jump of

intelligent agents:

1. Sensors

2. Cognition

3. Actuators

Biological

approach!

Liberal

approach!

15

14.04.2016

Klaus Henning

The paradigm shift in artificial Intelligence

From Top-Down to Buttom-Up

� Two competing movements?

Top-Down / symbolic AI Bottom-Up / subsymbolic AI

GOFAIGood old fashioned

Artificial Intelligence;

based on high-level

"symbolic“ knowledge

representations

Connectivisminteraction as basis

of intelligence

[Szt

ipa

no

vit

s, V

an

de

rbilt,

19

97

]

[Lin

, 2

01

0]

Knowledge

storage/

knowledge

retrieval

Knowledge

on demand /

knowledge

aquisition

Top-Down / symbolic AI

GOFAIGood old fashioned

Artificial Intelligence;

based on high-level

"symbolic“ knowledge

representations

[Szt

ipa

no

vit

s, V

an

de

rbilt,

19

97

]

Knowledge

storage/

knowledge

retrieval

[Szt

ipa

no

vit

s, V

an

de

rbilt,

19

97

]

16

14.04.2016

Klaus Henning

From the history of autonomous vehicles (already old fashioned)

2009: Truck robot platoons – distributed intelligence

� The KONVOI project (several institutes from RWTH & industry partners)

�

� 2005-2009

� automated / partly autonomous

transportation e.g. by electronically

coupling trucks to convoys

� several successful tests with trucks:

Chauffeur, KONVOI, SARTRE (EU), Energy-

ITS (Japan), …

�

� expected improvements:

beyond safety, reduction of fuel

consumption and gained road space

!

� Adv. driver assistance system for trucks

� short distances between vehicles of

approx. 10m at a velocity of 80 km/h

� Energy-ITS: 4m ! (2013)

� KONVOI:

� Car2infrastrcuture components!

� Model of multi agent systems

Connectivity…

17

14.04.2016

Klaus Henning

� Organization forms on demand – individualized by client - initialized by product

!Product agitates as “super-agent”:

� Plans production and transportation steps

� Requests service from agents

� Negotiates with other products for agent-resources

�

� Heterogenous player modeled as multi agent concept

� Models from biology and social sciences

� Basis on Autopoiesis & embodiment theory

transport

unit

production

unit

virtual service

providerfab

ric

atio

n

ou

tsid

e w

orld

© D

an

iel E

we

rt 20

13

Trucks or shoes – state of the art is ….

Horizontal coupling (manufacturing/logistics) – to lot size 1

�

� Konvoi 2005-2009,

RWTH with partners

� (partly) autonomous

driving via convoys

18

14.04.2016

Klaus Henning

Outline


� Breakthroughs















19

14.04.2016

Klaus Henning


A new kind of Human-Technology-Interaction

The digital shadow (or skin) will be an dominant part of the human`s and the

technical devise`s identity. The dimensions of interaction will be:

communication between the digital shadows of humans and/or technical devices,

machine to machine communication,

human to human communication,

human to machine communication,

communication of humans / machines with its digital shadows (skins).

The ubiquitous and inconspicuous interaction between the digital shadows of

technology and of humans will dominate all aspects of communication

20

14.04.2016

Klaus Henning

Three steps

It’s all about “Cognitive Computing”

21

14.04.2016

Klaus Henning

Let’s ask Google

Definitions around “Cognitive Computing” (I)

“Cognitive computing (CC) makes a new class of problems

computable. It addresses complex situations that are

characterized by ambiguity and uncertainty; in other words it

handles human kinds of problems. …To do this, systems often

need to weigh conflicting evidence and suggest an answer that

is “best” rather than “right”.

Cognitive computing systems make context computable.”

22

14.04.2016

Klaus Henning

Dealing with complexity and dynamics

Human machine interaction and cooperative robotics

�Robots are no longer locked in work-cells but cooperate

with each other and/or with humans

human-machine-machine

interaction in the X-Cell

machine-machine cooperation Robotics entering

the scene as:

Cognitive computing is

about „solving real

problems“. Real

problems are usually

part of our real,

physical world…

The enhancement of

AI is strongly

connected to the

progress in robotics,

coupled by the

embodiment theory.

23

14.04.2016

Klaus Henning

Advantage of decentralized control structures

Intralogistics goes mobile: The Festo Logistics League

Competitions robocup:

2012: 0 points in World Cup

2013: 4th in World Cup

2014: Winner of the GermanOpen

2014: Winner of the World Cup

2015: Winner of the World Cup

�

Critical factors for success:

� Totally decentralized

� No ”hard coded components“

� Strong cooperation

� Re-planning during tasks

� Mobile transportation robots from flexible routing

!

Competencies:

� localization & navigation

� computer vision

� adaptive planning

� multi agent strategies

� sensory & hardware

24

14.04.2016

Klaus Henning

Let’s ask Google

Definitions around “Cognitive Computing” (II)








“Cognitive computing is the simulation

of human thought processes in a

computerized model…. involves self-

learning systems that use data mining,

pattern recognition and natural

language processing to mimic the way

the human brain works.”

25

14.04.2016

Klaus Henning

formerly: „neural

networks“

What has to be expected?

From organic … to cognitive computing

!Cognitive computing: the simulation of

biological thought processes !Organic computing: Following social systems and biological models

Ne

ura

ln

etw

ork

s

Deep learning (e.g. Watson)

�Relativization: It‘s not only about „human approaches“

but about approaches which have been developed along the evolution

Macro-scale

Automation

Division of labor

Micro-scale

Multi-Core

Ha

rd

wa

re

Service

oriented

Agent-

based

So

ftw

are

Genetic algorithmsDas Bild kann zurzeit nicht angezeigt werden.

26

14.04.2016

Klaus Henning

Shadow Dexterous Hand

Central terms in the field of intelligent distributed systems

From embodiment … to humanoids

�

Zykov V., Mytilinaios E., Adams B., Lipson H. (2005) "Self-reproducing machines",

Nature Vol. 435 No. 7038, pp. 163-164

Bongard J., et al., Resilient Machines Through Continuous Self-Modeling, Science

314, 2006

Lipson H. (2005) "Evolutionary Design and Evolutionary Robotics", Biomimetics, CRC

Press (Bar Cohen, Ed.) pp. 129-155

Robonaut 2- NASA

The Bongard robot – learning through

embodiment [Bongard, 2006; Lipson, 2007]

Embodiment theory:

„intelligence needs a body“

The existence of a body (incl. sensors and actuators)

are basic prerequisites to build experience and

finally the development of intelligence.

�

Embodiment theory:

„different bodies = different intelligences“

… leading to humanoids / humanoid components

�

Asimo Honda

KIT, Dillmann,

SFB 588

27

14.04.2016

Klaus Henning

Lend the robots a face

Into Service Robotics: The next step – the “Oscars”

� Transform mobile robotic experiences into the field of service robotics

!

1. Investigating “new” human machine

Interfaces and interaction schemes

� Simple, intuitive

� Schematic eyes following you

� “natural eyes behavior”: randomly

looking around, showing interest by

blinking, looking bored, …

!Performing service robot tasks

� Distribute brochures and serving drinks

� Path planning, room exploration, …

!

2. Investigating the “Uncanny Valley”: when

features look almost, but not exactly, like

natural beings, it causes a response of

revulsion among the observers (Mori 1970)

3. Investigating diversity specific reactions

(gender, age, culture) to artificial systems

and in particular robots

28

14.04.2016

Klaus Henning

Let’s ask Google

Definitions around “Cognitive Computing” (III)








“Cognitive computing systems [are] a

category of technologies that uses

natural language processing and

machine learning to enable people

and machines to interact more

naturally […]. These systems will learn

and interact to provide expert

assistance to scientists, engineers,

lawyers, and other professionals in a

fraction of the time

it now takes.”

“Cognitive computing is the simulation

of human thought processes in a

computerized model…. involves self-

learning systems that use data mining,

pattern recognition and natural

language processing to mimic the way

the human brain works.”

29

14.04.2016

Klaus Henning

New forms of human machine interactipn

About the role of emotion in human-machine-interaction

!However, it took a while before emotions were

considered important in computer science.

Plato (ca. 400 BC)“Human behavior flows from three

main sources: desire, emotion, and

knowledge.”

Rosalind Picard (since 1997; MIT)“Computers that will interact naturally

and intelligently with humans need

the ability to at least recognize and

express affect.”

Picard coined the term

“affectice computing”

�

KISMET - MIT (1990-2000; Cynthia Breazeal)

� Analysis and simulation of human-like

emotions

� Research on interaction between robots

and humans

� Part of the “organic development”

30

14.04.2016

Klaus Henning

New forms of human machine interactipn

Automatic Emotion Recognition

!

1. Emotion form an important part of communication –

� not an „add-on“ on audio, …

2. Emotions are an intrinsic part of intelligence -

� not an „add-on“ on ratio, …

Project IMOTION at IMA/ZLW & IfU:

the emotional navigation system

Using emotions as basis for behavior adaption:• Shouting at it will make it change its behavior

• Praising stabilizes behavior

• Randoms enable global instead of local optima

31

14.04.2016

Klaus Henning

New forms of human machine interaction

Automatic Emotion Recognition: Project IMOTION (1)

?Did you ever yell at your GPS,

e.g. when it told you the equivalent of “drive through that river ahead!”? ☺

Well, WE do constantly. And so the story went on like this:

! Motivation and goal: Transform the GPS into an intelligent co-driver, i.e. get it to adapt to your emotions!

� Solution: A machine-learning based system architecture. Did it work? Often! Was it fun? Hell, yeah! ☺

�

Ingredients in a nutshell

� Primary source of emotion: driver

speech

� Machine-learning algorithm:

Support Vector Machine (SVM)

� Training database: talk show

based corpus (lots of yelling)

� Test-bed: a driving simulator

� Test persons: old and young

32

14.04.2016

Klaus Henning

New forms of human machine interaction

Automatic Emotion Recognition: Project IMOTION (2)

� Impressions, findings and lessons learned during the course of the project and the trail runs

�

Trial runs and setup

� 38 test persons, from young to old. Subjects had to

pass a simulated test track, under time pressure and

had to obey the instructions of the GPS

� The GPS was intentionally mean, i.e. it gave incorrect

orders (subjects had to drive 5 times through a

roundabout)

� GPS adapted itself, e.g. it apologized when yelled at

and “updated” its map material or pacified the driver

�

Findings and lesson learned

� Speech based emotion detection worked well

� But: Some people where mute during driving (one

theoretical physicist didn't take at all, though you

could pick up his annoyance on the video recording!)

� Conclusion: We have to go multimodal in emotion

recognition, by combining audio, video with

physiological sensors (e.g. EEG, which was

researched during IMOTION)

33

14.04.2016

Klaus Henning

Outline


� Breakthroughs















34

14.04.2016

Klaus Henning

Summary

… in four steps!

4.0: The Revolution of a

distributed artificial

intelligence

4.0 is an era of highly

interdisciplinary

research

Rediscovery of

Cybernetics

4th Industrial

Revolution

The rise of cognitive

computing

35

14.04.2016

Klaus Henning


Summary

Nowadays, technology is often passive. Future HTI will see a much more active

role of everyday technology: i.e. in administration, medicine or mobility.

Is it comfortable, …

Necessary, …

Or even scary?

www.ima-zlw-ifu.rwth-aachen.de

Thank you!

Univ.-Prof. Dr.-Ing. em. Klaus Henning

Senior Advisor of Institute Cluster IMA/ZLW & IfU

phone: +49 171 860 7575

[email protected]

Co-authored by:

Univ.-Prof. Dr. rer. nat. Sabina Jeschke

Head of Institute Cluster IMA/ZLW & IfU

phone: +49 241-80-91110

[email protected]

Prof. Dr.-Ing. Tobias Meisen

Institute Cluster IMA/ZLW & IfU

[email protected]

Dipl.-Inform. Christian Kohlschein


[email protected]

aachen.de

Dipl.-Wirt.-Ing. Sebastian Reuter


[email protected]

37

14.04.2016

Klaus Henning

Univ.-Prof. em. Dr.-Ing. Klaus Henning …

… studierte Elektrotechnik und Politische Wissenschaften, promovierte über

Mensch-Maschine-Systeme und habilitierte über Entropie in der Systemtheorie. Er hat über 40 Jahre Berufserfahrung.

Er war 25 Jahre lang Leiter des größten Institutsclusters für Kybernetik

(IMA/ZLW & IfU der RWTH Aachen University). Jeweils einige Jahre war er

Mitglied im Präsidium des VDI, Prorektor für Finanzen der RWTH

und Dekan der Fakultät für Maschinenwesen der RWTH.

Heute ist er Senior-Berater und Mitgesellschafter der

OSTO Systemberatung GmbH, ein Unternehmen der P3 group.

Er ist Mitglied des Vorstands des Instituts für Unternehmenskybernetik (IfU e.V) an der RWTH Aachen und als

Senior Advisor Mitglied des Board of Management des Institutsclusters IMA/ZLW & IfU an der RWTH Aachen.

Er ist Vorsitzender des Aufsichtsrats der Xenium A.G., München, Mitglied des Universitätsrats der Universität des

Saarlands und Mitglied des Zukunftskommission des Wirtschaftsrats der CDU.

Derzeit kommen die meisten von ihm betreuten Kunden – meist auf Vorstands- und Abteilungsleiterebene aus der IT-

Branche, universitären Krankenhäusern und der Zulieferindustrie des Maschinen- und Anlagenbau, der Automotive

Branche, der Luft- und Raumfahrtindustrie sowie der Logistik.

Er hat – zusammen mit zwei Kollegen – 2011 bis 2012 den Zukunftsdialog der Bundeskanzlerin „Wie wollen wir

leben?“ wissenschaftlich koordiniert.

Seine Erfahrungen hat er in einem Buch „Die Kunst der kleinen Lösung – wie Menschen und Unternehmen die

Komplexität meistern“ zusammengefasst, das 2014 im Murrmann- Verlag erschienen ist.

the digitale transformation of human machine …...2016/04/14 · the digitale transformation of...

Documents