driven by new trends in artificial intelligence: robots in ... · pdf filedriven by new trends...

www.ima-zlw-ifu.rwth-aachen.de

Driven by New Trends in Artificial Intelligence:Robots in Medical Applications

Leonardo-Modul „Robotik in der Medizin“ RWTH Aachen University

Aachen, January 18th, 2017

Univ.-Prof. Dr. rer. nat. Sabina Jeschke

Cybernetics Lab IMA/ZLW & IfU

Faculty of Mechanical Engineering

RWTH Aachen University

2

18.01.2017

S. Jeschke

Outline

I. Introduction

Da Vinci as a starting point

towards a more general approach to robots in medicine

II. Physical robots

in different application areas

in different shapes and design principles

III. Non-physical robots

in a nutshell

from Google Flu to Watson and beyond

IV. Summary and Outlook

the question of a creative artificial mind

4.0 trends in labor and employment

3

18.01.2017

S. Jeschke

Introduction

Da Vinci is only the starting point…

!

The most prominent medical robot today: the “da Vinci® Surgical System” for laparoscopy

Designed by the American company Intuitive Surgical

Year of creation: 2000 (initial FDA approval)

most commonly for hysterectomies and prostate removals

increasingly for cardiac valve repair

Advantages: No trembling (neither surgeon nor

patient) Good visibility for laparoscopy very sharp picture of operational field

4

18.01.2017

S. Jeschke

Introduction

Classifications in robotics (selection) in general

highlow

Intelligence

(means: self-adaptivity; decision-making)

mobilestationary

Degree of mobility

high= robotdepending on human control

Low = robot fully autonomous

Cooperation level

5

18.01.2017

S. Jeschke

Introduction

Classifications in robotics for da Vinci

highlow

Intelligence

(means: self-adaptivity; decision-making)

mobilestationary

Degree of mobility

high= robotdepending on human control

Low = robot fully autonomous

Cooperation level

6

18.01.2017

S. Jeschke

Introduction

A more general approach to “medical robots”

rehabilitation / aftertreatmentprevention acute treatment

conservative

surgical

da Vinci(urology &

gynecology)

DLR MiroSurge(cardiac)

Renaissance by Mazor

(spine surgery)

7

18.01.2017

S. Jeschke

Introduction

An ever more general 4.0 approach to “medical robots”

Big Data Analytics

Hospital logistics

Patient care

Elderly care

Quantified self

Fluent transition to fitness/wellness

Humanoid robots

Robots in medical production (implants)

Robots as artificial limbs

Robots as trainers andphysiotherapists

Pet robots

8

18.01.2017

S. Jeschke

Outline

I. Introduction

da Vinci as a starting point


II. Physical robots




in a nutshell





9

18.01.2017

S. Jeschke

Applications for robots in medicine

Intralogistics goes mobile: The Festo Logistics League

Competitions robocup:

2012: 0 points in World Cup

2013: 4th in World Cup

2014: Winner of the GermanOpen

2014: Winner of the World Cup



Critical factors for success: totally decentralized no ”hard coded components“ strong cooperation re-planning during tasks

Mobile transportation robots from flexible routing

!

Competencies: localization & navigation computer vision adaptive planning multi agent strategies sensors & hardware

10

18.01.2017

S. Jeschke


Mobile intralogistics robots for medical environments

Robot by MLR, 2015, at Nye Akershus University Hospital Oslo:

automatic goods transport system

THORSTEN, IMA/ZLW & IfU @ RWTH: Based on the algorithms of the FESTO Logistics League

Hospital logistics

Car-O-Bot by Fraunhofer IPA, since about 1990

Spot, Boston Dynamics, background: military research (2015)

11

18.01.2017

S. Jeschke


Human machine interaction and cooperative robotics

Robots are no longer locked in work-cells but cooperate with each other and/or with humans

Direct interaction – object transfer

Coping with visual occlusions of the object resulting from the body/ the hands of the human

Evaluating possible grasping points Offering good grasping options for the human co-worker

12

18.01.2017

S. Jeschke


Into Service Robotics: The next step – the “Oscars”

Transform mobile robotic experiences into the field of service robotics

!

1. Investigating “new” human machine interfaces and interaction schemes Simple, intuitive Schematic eyes following you “natural eyes behavior”: randomly

looking around, showing interest by blinking, looking bored, …

!Performing service robot tasks Distribute brochures and serving drinks Path planning, room exploration, …

!

2. Investigating the “Uncanny Valley”: when features look almost, but not exactly, like natural beings, it causes a response of revulsion among the observers (Mori 1970)

3. Investigating diversity specific reactions (gender, age, culture) to artificial systems and in particular robots

13

18.01.2017

S. Jeschke

Shadow Dexterous Hand


From embodiment … to humanoids

Zykov V., Mytilinaios E., Adams B., Lipson H. (2005) "Self-reproducing machines", Nature Vol. 435 No. 7038, pp. 163-164Bongard J., et al., Resilient Machines Through Continuous Self-Modeling, Science 314, 2006Lipson H. (2005) "Evolutionary Design and Evolutionary Robotics", Biomimetics, CRC Press (Bar Cohen, Ed.) pp. 129-155

Robonaut 2- NASA

The Bongard robot – learning through embodiment [Bongard, 2006; Lipson, 2007]

Embodiment theory:„intelligence needs a body“

The existence of a body (incl. sensors and actuators)are basic prerequisites to build experience and finally the development of intelligence.

Embodiment theory:„different bodies = different intelligences“

… leading to humanoids / humanoid components

Asimo Honda

KIT, Dillmann, SFB 588

14

18.01.2017

S. Jeschke


Robots for teaching and training

Robot TOSY TOPIO

sport roboter“ – a „pal“ for leisure time but in general, the system can also be used

for rehabilitation training developed in Vietnam plays table tennis against human players

(and wins!) capable of learning, improving his style, but

also adapt to the opponent in question

Robot Sayah

Teaching roboter“ in classroom, so farfor languages

but in general, the system can also beused for all kind of instructions

developed in Japan Comprehensive mimics to illustrate

emotions like anger, surprise, happyness, fear, disgust, ....

Kids react strongly to the emotionsdisplayed

15

18.01.2017

S. Jeschke


Robots as artificial replacement of limps

DEKA Bionic Arm “LUKE”

LUKE “Life Under Kinetic Evolution” Developed by DARPA Pentagon Agency) R&D program HAPTIX “seeks to create

a prosthetic hand system that moves and provides sensation like a natural hand”

Connection of human nerves to robotby surgery (Targeted musclereinnervation)

[IEEE Spectrum, Feb 2015, DARPA.MIL Dez 2016]

A Bionic Dance Prothesis

Developed by MIT BiomechatronicsGroup

Control system improves movement ofprosthetics by muscles

Allows „expressive activity modes“ similar to non-amputees

[PLOS ONE Aug 2015]

16

18.01.2017

S. Jeschke

Applications for robots in medicin

Support for nursing by robots

ROBEAR - RIKEN and Sumitomo Riko labs in Japan, to support hospital employees and people

in their private homes (2015)

!

The bearing robot ROBEAR

Lifts people from their beds, or into a wheelchair

Stationary so far, but in the future could be extended to a “logistists tool”, transferring patients

between rooms etc.

Decreases the physical stress for hospital employees

Supports an autonomous and self-determined life for people with handicaps

17

18.01.2017

S. Jeschke


Pet robots

PARO therapeutic pet robot from Japan (movie 2011, designed since about 1993 by Takanori Shibata/AIST)

!

The therapeutic pet robot PARO

Mainly used for therapy for dementia patients

classified as a Class 2 medical device by U.S. regulators in fall 2009

responds to petting by moving its tail and opening and closing its eyes

actively seeks out eye contact, responds to touch, cuddles with people, remembers faces, …

?

Background - contact toanimals is known for itspositive impacts:

creation of meaning sozial catalyst relaxation motorical stimulation higher stress resistance stress reduction control of blood pressure

→ Enhances quality of lifeJustoCat, Robyn

Robotics AB/Sweden, Class 1 medical device

18

18.01.2017

S. Jeschke

Outline

I. Introduction

daVinci as a starting point

Towards a more general approach to robots in medicine

II. Physical robots




in a nutshell



The question of a creative artificial mind


19

18.01.2017

S. Jeschke

Applications for non-physical robots in medicine

Google Flu: predicting future (predicting the spread of diseases)

actual flu trend can be identified 7-10 days earlier by ‘Google Flu Trends’ than by official data of the Center for Disease Control (CDC)

[Helft 2008]

Analysing user behaviour

20

18.01.2017

S. Jeschke


Predicting human behavior: transparent consumers

How Target figured out a teen girl was pregnant before her father did…

Unique Target IdEach interaction with retailer is assigned to that id

Customer profilesClustering customers into groups, for example to identify disruptions in life(e. g. weddings, job changes and pregnancy)

Andrew PoleStatistician working for Target

Pole identified about 25 products that allowed him to assign each customer a “pregnancy prediction” score and the estimated due date

Coupon campaign

Group of pregnant

customers

Customer

21

18.01.2017

S. Jeschke

Linguisticpreprocessing

Generation of possiblecandidates

Evaluation of candidates


The new probabilistic engines

? Back to Watson: how is this guy running the (Jeopardy!) show??

DeepQA architecture Purely based on natural language processing (NLP) Approx. 100 different AI/linguistic methods come into play Without any specific semantic representation (“as-is”)

90 IBM-Power-750 servers For each: a 3.5 GHz POWER7

processor, with 8 cores, and 4 threads per core

In total: 2.880 POWER7 threads 16 terabytes of RAM

22

18.01.2017

S. Jeschke


Probabilistic engines “down-to-earth”

! Watson: from playing Jeopardy! towards becoming some kind of a “medical doctor”…

today, only 20% of the medical knowledge is evidence based (basis of individualized medicine)

also, amount of medical information is doubling every 5 years: physicians can’t read all the journals

Data: all types, 1. structured data from electronic medical record

databases and 2. unstructured text from physician notes and

published literature

How can we deal with these challenges?

Goal of Watson: help physicians in diagnosing and treating patients by analyzing large data

acting as a huge preprocessor for all kind of medical information

potential to transform health care into individual medicine

currently tested by several clinics, e.g. Mayo, MD Anderson, Cleveland, and Sloan-Kettering

“IBM's Watson is better at diagnosing cancer than human doctors”

Example “p53”: Watson identified possible treatments for protein p53 deficiency linked to many cancers

Example “Google Flu” (another engine): already now, doctors integrate the results of GoogleFlu (spreading and direction of contagious illnesses) as it is much faster and more precise as the results of the best medical centers in the world)

23

18.01.2017

S. Jeschke

Outline

I. Introduction

da Vinci as a starting point


II. Physical robots




in a nutshell





24

18.01.2017

S. Jeschke

Learning by doing – reinforcement learning

The next step: Using rewards to learn actions

?Remember Mario: What if the machine could learn how to solve a level? Why not use a some kind of intelligent trial-and-error?

Reinforcement learning (R-learning) is inspired by behaviorist psychology –maximizing the expected return by applying a sequence of actions at a current state.

Central part of cybernetics from its start (e.g., Minsky 1954)

[SethBling, 2015]

Neuroevolution of augmenting topologies (NEAT)

Genetic algorithms on top of neural networks

At each state the system decides what action to do

Actions are rewarded if Mario does not die in return

Level progress by evolving neural networks

[Stanley, 2002]

Now, Human factor is reduced to very general, formal

specifications of the neural network… However, human still influences the

underlying representation model

25

18.01.2017

S. Jeschke

The creative artificial mind

Where the Story Goes: AlphaGo

!

Go originated in China more than 2,500 years ago. Confucius wrote about it. As simple as the rules are, Go is a game of profound complexity. This complexity is what makes Go hard for computers to play, and an irresistible challenge to AI researchers. [adapted from Hassabis, 2016]

Bringing it all together!

The problem: 2.57×10210 possible positions – that is more than the number of atoms in the universe, and more than a googol times (10100) larger than chess.

Training set30 million moves recorded fromgames played by humans experts

Creating deep neural networks12 network layers with millions ofneuron-like connections

Predicting the human move(57% of time)

Dat

a-d

rive

n le

arn

ing

Re

info

rce

me

nt

lear

nin

g

Learning non-human strategiesAlphaGo designed by Google DeepMind, played against itself in thousands of games and evolved its neural networks; Monte Carlo tree search

! Achieving one of the grand challenges of AI

March 2016:Beating Lee Se-dol (World Champion)AlphaGo won 4 games to 1.(5 years before time)

[Has

sab

is, 2

01

6]

26

18.01.2017

S. Jeschke

!“Creativity is a phenomenon whereby something new … is formed. The created item may be intangible (such as an idea, a scientific theory, a musical composition or a joke) or a physical object (such as an invention, a literary work or a painting).” [adapted from Wikipedia, last visited 5/3/2016]


Microsoft Visual Storytelling (SIS): machines becoming creative

Visual-Storytelling by Microsoftbased on deep neural networks(convolutional neural networks)

DII (descriptions for images in isolation): Traditional storytelling software

SIS (stories for images in sequence): new approach towards storytelling, including

Based on SIND – Sequential Image Narrative Dataset: 81,743 unique photos in 20,211 sequences, aligned to both descriptive (caption) and story language.

[Margaret Mitchell / Microsoft, 04/2016, together with colleagues from Facebook]

27

18.01.2017

S. Jeschke

!“Creativity is a phenomenon whereby something new … is formed. The created item may be intangible (such as an idea, a scientific theory, a musical composition or a joke) or a physical object (such as an invention, a literary work or a painting).” [adapted from Wikipedia, last visited 5/3/2016]

Van Gogh’s Starry Nightinterpreted by Google DeepDream

based on deep neural networks

“Do Androids Dream of Electric Sheep?”

(science fiction novel by American writer Philip K. Dick, published in 1968)

Computational creativity (artificial creativity) … is a multidisciplinary endeavor that is located at the intersection of the fields of artificial intelligence, cognitive psychology, philosophy, and the arts. [adapted from Wikipedia, last visited 5/3/2016]

„Can machines be creative?“ by Iamus, a computer clustercomposing classicalmusic by genetic algorithms, concert forTurings 100th birthday [youtube]


Google DeepDream: machines becoming creative

28

18.01.2017

S. Jeschke

Changes to the job market

Industry 4.0 does not only change the “routine” jobs

The typical assumption…

… that job changes in 4.0 are mainly addressing blue collar jobs and/or routine jobs does not hold true.

White collar jobs

… are under massive change due to the enhancement in

AI, here the impact often hits “middle class jobs”

Social robots

… will become capable of taking over even complex

tasks with personal presence as in health or home care

From „blue collar – low qualified“ to „white collar – middle class“...

but probably, this is just a transition phenomenon

High qualified jobs

… as e.g. health professionals face already the taking over

through AI in certain fields by Watson, Google Flu, etc.

Decentralized platforms

… with automated consensus models (e.g. blockchain) take over complex administrative

tasks e.g. in judiciaries

Virtual and augmented environments

… allowing for new international players, even in tasks requiring humans

and presence

Autonomous systems

… as autonomous cars and more envanced production

technology will change the blue collar – low qualified as well

www.ima-zlw-ifu.rwth-aachen.de

Thank you!Univ.-Prof. Dr. rer. nat. Sabina JeschkeHead of Cybernetics Lab IMA/ZLW & [email protected]

Co-authored by:

Prof. Dr.-Ing. Tobias MeisenJunior Professor “Interoperability of Simulations”

Dipl.-Inform. Christian KohlscheinHead of Research group Cognitive Computing & eHealth

Dr. rer. nat. Katja SchneiderPersonal Assistance to the Heads of Institutes

driven by new trends in artificial intelligence: robots in ... · pdf filedriven by new trends...

Documents