lund september 2010 robotics and automation software€¦ · lund september 2010 robotics and...

Lund September 2010

Robotics and Automation software

[email protected], et.al., Lund University, Sweden

cs.LTH.se/RSS

Outline

1. Intro and projects

2. SMErobot results

3. Flexible software systems

4. Rosetta and future robots

LUCAS: Robotics and Semantic Systems, Lund, Sept. 2010 2

Robotics Observations & Motivations

1. We humans want safe and sustainable (assistance for) a good quality of life !

2. Resources are limited => Efficiency !

3. The world is changing => Flexibility !

• Sustainable quality of life:

– Transformations of natural resources

(robot-assisted manufacturing and recycling)

– Scalable solutions required

– Human aspects….

• Efficiency:

– Resource-aware implementations

– Reuse in engineering

• Flexibility:

– Interoperable and open systems? No, not sufficient in robotics!

– Applications and technologies evolve rapidly: New approach needed…..

• Safety and Security: Key issues also calling for new FLEXIBLE approaches…


New/running robot projects

• MONROE – Hyper-Modular Open Networked RObot systems with Excellent performance$: EU.FP7.ICT, within www.echord.info, with Güdel AG, Switzerland.

• COMET – Plug-and-produce COmponents and METhods for adaptive control of industrial robots enabling cost effective, high precision manufacturing in factories of the future.$: EU.FP7.NMP/FoF, coordinated by Delcam PLC Ltd., UK

• ROSETTA - RObot control for Skilled ExecuTion of Tasks in natural interaction with humans; based on Autonomy, cumulative knowledge and learning$: EU.FP7.ICT, coordinated by ABB Corporate Research, Västerås

• ENGROSS – Enabling Growing Software Systems$: SSF with LUCAS&Math partners @LTH, mobile robots

• ProFlexA – Productive Flexible Automation$: SSF ProViking, foundry applications, informally with ABB Robotics, Västerås

• INROSY – Intelligent Networked RObotics SYstems with reconfigurable exogenous system sensing. $: STINT KOSEF, with Hanyang university, Seoul, Korea


„Re-Inventing Industrial Robotics“

“established”Vision: The „SME-Robot“

Automotive industries

Changeover time < once/year

Programming offline (>90%)

Robot unit price ~ 25 T€

Workcell cost ~ 4 * robot unit price

Sensor equipped < 5% of installations

Planning simultaneous engineering

Maintenance trained staff

< once/day

• shop-floor

• 1/3 of today‘s robot price

• ~ 1 * robot unit price

• 100%

• shop floor

• by worker

Manufacturing SMEs


SMErobot video


Example industrial collaborationULUND-ABB work (F/T control)


EURON TechTransfer Award 05: Open platformfor skilled motions in productive robotics

Work

Robot

Language

Mas

ter

Contr

ol

Motor

control

Trajectory

generation

Arm

control Arm

Application ……

Task description

ABB controller

Cel

l co

ntr

oll

er

RPL/XMLPathsync

250Hz<1ms5Hz

F/T-sens.8kHz

← kernel space←

VxWorks

Linux

Windows(any OS)

userspace

The European Robot Initiative for Strengthening the Competitiveness of

SMEs in Manufacturing

Klas Nilsson [Martin Hägele]www.smerobot.org

Outlook and Lessons learned from SMErobot™

Klas Nilsson, Lund University – LTH, Sweden; [email protected]

ICRA2010, Anchorage, 3 May 20109Outlook and lessons learned in SMErobot™

Project-Partner

Project-lead

Robotics-industries

End-users

Suppliers

Transfer

Research


SMErobot: A Family of New Robots

Three Major Innovations:

1. Robot capable of understanding human-like instructions

2. Safe and productive human-aware space-sharing robot

3. Three-day-deployable integrated robot system

• FP6 Integrated Project• 17 partners, major European robot manufacturers• Project runtime March 2005 - May 2009• Coordinator: Fraunhofer IPA, Stuttgart


The SMErobot Initiative

Intuitive instruction of fettlingof castings for the foundry

Fast installation, small batchsize production change (forgery)

The SME welding robot

Automation of manualwoodworking processes

Robot capable ofunderstanding human-likeinstructions

Safe and productive human-aware space-sharing robot

Three-day-deployableintegrated robot system(install-configure-instruct)

• Training and education

• Socio-economics (new business models, LCC)

• Standardization

• Exploitation, IPR

Research & Development Demonstrations (Focus)

InnovationRelatedActivities

2010-09-15 12Outlook and lessons learned in SMErobot™

Industrial research perspective

Present Robot Applications

Future Robot Applications

Future Robot Product Development

Present Robot Product Development

Industry Segments

R&D in Robotics

New Technology

Interplay and feedback from other areas, also from present robot applications…


User Interface

Intelligent gripper

Bin picking process

What isshown:

“the objects”

What was invented forintegrating “the

objects”

Robot guide device

XIRP

Comm.

protocol

UPnP

Comm.

protocol

Config

.module

PC -Cell

contro

ller

Research

End User

Industrial

Industry linking Science and Users

• An example from P’n’P demonstration:


The SMErobot InitiativeInnovations:1. The robot capable of understanding human-like instructions2. The safe and productive human-aware space-sharing robot3. The three-day-deployable integrated robot system

Applications in SME manufacturing Research & DevelopmentImplementation

• Training and

Education

• Socio Economics

• Dissemination

• Exploitation

• Result exploitation

• Intellectual

property rights

• Management

Demonstrations

Coo

pera

tive

flexi

ble

Rob

ot W

orkc

ell

Intu

itive

Inst

ruct

ion

of F

ettli

ngC

astin

gs

Rob

ot a

ssis

tant

at

man

ualw

orkp

lace

s

Objectives

The

robo

t un

ders

tand

ing

hum

an-li

kein

stru

ctio

ns

The

safe

hum

an- w

are

spac

esh

arin

gro

bot

The

thre

e-da

yde

ploy

able

inte

grat

edro

bot s

yste

m

Innovations

Fou

ndry

Co-

oper

ativ

e sm

all

batc

has

sem

bly

Met

al p

rodu

cts

Mac

hine

ry

Woo

dwor

king

SM

E e

nd

- use

rg

rou

pS

ME

sys

tem

inte

gra

tors

Science and technology (S&T) progress beyond state of the art urgently needed!

Interplay between core sciences and SME needs, by integration project SMErobot…


Innovations – beyond state of the art

1. Robot capable of understanding human-like instructions

# New Low-Cost Human-Robot-Interaction (HRI)

→ Declarative knowledge for interaction fusion

2. Safe and productive human-aware space-sharing robot

# New Mechatronics for High-Performance Robotics

→ Safety with awareness of human intension

3. Three-day-deployable integrated robot system

# Increased modularity & Managed geometries/motions

→ Agile technology upgrading; Technologies and

SME-needs change faster than standards evolve

Legend: # =accomplished, → = results but deserves further research


Intuitive Instruction

New interaction devices and their use

Tactile guidanceGraphicsSpeech


Safe, Human-aware Space-sharing

Physically harmless and low-cost robot mechanics

Safe robot working without fences


The “3-day deployable” robot workcell

“Plug-and-Produce” Robot program generation bydistributed product-process data

Ethernetnetwork

Ethernetnetwork


Training, Socio-Economics


Conclusions

• Force control and lead-though programming

established for usage in actual products:

– Worked (for ideal cases) in labs 20 years ago,

– was difficult to fit into industrial systems,

– SMErobot enhancements and proof of concept in SMEs!

• Wide variety of competences needed for S&T

progress; SME business models needed for impact!

• Very successful barrier breakthroughs (S&T

results),

but industrial impact not completed!

• SMErobot as a model and inspiration for extended

Robotics and Automation research….


Video and Questions

• The SMErobot vision:The coffee-break video [SMErobot on YouTubeor on smerobot.org]

• Summary of results: This video……

www.smerobot.org


Robot control: R&D Situation

Problems (not just to write another algorithm/procedure):

• Reuse not even the case within a lab

• Few shared platforms, hampering scientific values

• Industrial development re-implementing controls

• Public funding goes to re-implementation

• Technologies missing for efficient reuse

q Current situation hampers the entire area of robotics….

Approach:

• Separation of concerns

• Multiple reuse perspectives (not layers)

• Middleware and frameworks in thin layers

• Bottom-up definitions and implementations

• Separation of mechanism and policy


Interoperable and Open Systems

Interoperability

• The standard driving force for standardization

• Very valuable when it works (USB, IP, SCXML, etc., etc.)

– Evolving apps and techs: “Standards playing catch-up”

Open Systems

• Public interfaces/APIs

• Modules (using those APIs) replaceable by third parties

– Fixed APIs/abstractions and data/control-flows: Not evolvable!

Standardization based on local attempts to reuse:

q We built a system X according to the architecture Y including principles Z. It worked very well, and hence we conclude that everyone should use XYZ as the standard robot control platform.


Interoperability

Interoperability by standardization [NIST.gov]:

• Interface standards and specifications that are

– Correct

– Complete

– Unambiguous

– End-user directed

– Vendor written

• Implementations that are

– Compliant

– Interoperable

– Adopted broadly and worldwide

Insight: Takes too long to agree in a global and evolving world![Compare mobile telecommunication applications]


Technical Insights / Expert comments

• Reuse: Source code, binary components, algorithms, data, definitions;

different perspectives.

• Primary separation of concerns:

– Coordination

– Configuration

– Computation

– Communication

• Loose coupling: In code and configurations, but also in assumptions,

definitions, service interaction, etc.

• Technology slicing; keep standards small and replaceable.

• Open source and proprietary systems deserve special attention.

• Service-oriented message-based interfaces to functionality.

• Compositionality; no need to know if components are composed.


Standards, modularity, and applications

• Market demands

packaged products.

• Useful components key

for time to market, or to

enable research focus,

• Price, performance,

maintainability, etc

based on tradeoffs and

optimization.

• Product development

and research need

engineering principles

in common


Robot System Architectures

Reflecting:

• Product lines

• Application areas

• User interaction

• Technologies

• Resource tradeoffs

• Variation points

• Implicit knowledge

• Business cases

• Certification

• Engineering taste

Taste/architecture is personal!

Thus, standardize components/techniques for robot architectures!

Product line architectures:

Domainunderlyingknowledge,human needs,domaincharacteristics

Business Finance,

accounting,marketing,

sales

TechnologyGeneric Tools,OTS apps,

computing/communicationsinfrastructure

CoreCompetencies

Application-Family

Architecture

Domain-IndependentInfrastructure

Idealized/context-non-specific knowledgeand architecture, not

shaped/driven/informedby business insights

An organization’sdomain-independent

technical assets

Domain expertise and knowledgethat is not captured or implemented

Domain-Specific Engineering

Product-LineArchitectures

Domainunderlyingknowledge,human needs,domaincharacteristics

Business Finance,

accounting,marketing,

sales

TechnologyGeneric Tools,OTS apps,

computing/communicationsinfrastructure

CoreCompetencies

Application-Family

Architecture

Domain-IndependentInfrastructure

Idealized/context-non-specific knowledgeand architecture, not

shaped/driven/informedby business insights

An organization’sdomain-independent

technical assets

Domain expertise and knowledgethat is not captured or implemented

Domain-Specific Engineering

Product-LineArchitectures

From http://csse.usc.edu/~neno


ENGROSS: Platform and Mobile Manipulation

• DLR, KUKA, Schunkarms depicted here.

• We will use future ABBlight-weight robot arms

• Mobile platform fromwww.care-o-bot.de(mid and right pict.)

Justin:


ENGROSS HW

Classified ABB technologies formanipulation gohere as body &arms [Rosetta]

Modules

of current

platform


ENGROSS: The safety-flexibility tradeoff

The extremes are less appropriate….


Force and vision – reaction time crucial

• Applications….

External Sensing and Control – Reconfiguration of ABB Industrial Robot Controllers, ICRA2010 32

Integrated High- and Low-level Control

Computer(Pentium)

PanelSafety

Power &UPS

PowerDriveSafety

Computer(PowerPC)

Drives(DSP)

Sensor/Process Interfaces

LAN

Control module

Drive modules

SMB

Motor

Perception, safety-watch, filtering, …

Engineering system

Skill-level ctrl

Soft/no RT

Hard RT ext_ctrl ext_ctrl

IRC5

RT ctrl

Skill exec

Task &WM

Task & WM

Overallknowledge-

base

RobAPIClient

Virtualcontrol

Sensing

YourFunc.

ROSETTA @ ISR/Robotik 2010 http://www.fp7rosetta.euA KIF for Robotics FP7 ICT 230902 ROSETTA - Slide 33

ROSETTA

RObot control for Skilled ExecuTion of Tasks in natural interaction with humans; based on

Autonomy, cumulative knowledge and learning

ROSETTA develops “human-centric” technology for industrial

robots that will not only appear more human-like, but also

cooperate with workers in ways that are safe and

perceived as natural. Such robots will be programmed in

an intuitive and efficient manner, making it easier to

adapt them to new tasks when a production line is changed to

manufacture a new product.


Research Areas

• Intuitive ways of instructing the

robot

• Task-level instruction

• Knowledge and skill representation

• System support (engineering tool)

• Robot control

• Sensor integration

• Assembly operations

• Learning

• Skill-based architectures

• Semantic acquisition and

interpretation

• Safety

• Physical human-robot interaction

• Injury criteria

• Workspace supervision

Four major efforts§ Knowledge Integration Framework

§ Human-Centric Aspects

§ Safety Criteria

§ Robot Kinematics


Rosetta

RObot control for Skilled ExecuTion of Tasks in natural interaction with

humans; based on Autonomy, cumulative knowledge and learning

Research on:

• Natural Language Processing

• Knowledge

representation

• Machine vision

• Real-time control

• Learning

• Human-cooperative

robots


Overall Architecture

Engineering ToolStation Configuration Tool

Task Definition Tool

Simulation Environment

Knowledge Integration FrameworkKnowledge about devices

Knowledge about device capabilities (Skills)

Knowledge about injury criteria

ROSETTA ControllerRuntime Controller

Safety Watcher

Task Sequencing

Robot ControllerSafety Sensors

The engineering tool gets knowledge about devices from KIF and can feed back task descriptions The ROSETTA Controller gets the

Task Description from KIF and feeds back knowledge learned.

The ROSETTA Controller acts upon sensor input. In the project new safety sensors are developed.

The ROSETTA Controller tells the Robot Controller what to do, and reads sensor values from the Robot.


Related or Prior Work in other Projects

• Baseline: The semantic web, a medium to exchange

data and knowledge

• DBPedia is a project that extracts in the form of RDF

triples and links RDF repositories

• Data sources can be accessed through a SPARQL

endpoint

• It served as main inspiration to the KIF

• The main repository of accessible knowledge in robotics

i the form of XML, databases (tables), text, etc., needs

conversion/triplification when imported.


Wikipedia: Resource_Description_Framework (RDF)

The Resource Description Framework (RDF) is a family of World Wide Web

Consortium (W3C) specifications originally designed as a metadata data

model. It has come to be used as a general method for conceptual

description or modeling of information that is implemented in web

resources, using a variety of syntax formats.

A collection of RDF statements intrinsically represents a labeled, directed

multi-graph. As such, an RDF-based data model is more naturally suited

to certain kinds of knowledge representation than the relational model

and other ontological models. However, in practice, RDF data is often

persisted in relational database or native representations also called

Triplestores, or Quad stores if context (i.e. the named graph) is also

persisted for each RDF triple. As RDFS and OWL demonstrate, additional

ontology languages can be built upon RDF.


Wikipedia: RDFS

RDFS: RDF Schema is an extensible knowledge representation language,

providing basic elements for the description of ontologies, otherwise called

Resource Description Framework (RDF) vocabularies, intended to structure

RDF resources.

rdf:type is a property used to state that a resource is an instance of a class.

rdfs:subClassOf allows to declare hierarchies of classes.

For example, the following declares that 'Every Person is an Agent':

foaf:Person rdfs:subClassOf foaf:Agent

Hierarchies of classes support inheritance of a property domain and range

(see definitions in next section) from a class to its subclasses.

rdfs:subPropertyOf is an instance of rdf:Property that is used to state that all

resources related by one property are also related by another.


Predicate-argument structures for the Open world

An open world is one in which we must assume at

any time that new information could come to light,

and we may draw no conclusions that rely on

assuming that the information available at any

one point is all the information available.

Basis for the open (human, not co-engineered) world;


Architecture


XML (left) to RDF (right) Conversion

This RDF tree is additive such that it automatically forms a graph when another tree is imported or inferred.


Client Integration


Client Visualization


Browsing the Integration server


Conclusions

XML widely used in the normative/engineered world,

suitable for tree-structured data.

Semantic web and DBPedia our inspiration sources

RDF (serialized as XML or N3 or…) basis for the open

world, with triples as graph elements.§ Data additive, even when coming from unrelated sources

§ Predicate-argument structure for natural/human language semantics

§ Dedicated graphs for specific needs (performance, algorithms, etc.)

AutomationML import and presentation, as example of

incorporation of predefined context/application data.

Next: Supporting Learning, Control, Safety, …

based on the actual scenarios (data-first manner)…


Thank you for your attention!

Questions?

The research leading to these results has received funding from the European Community’s Seventh Framework ProgrammeFP7/2007-2013 – Challenge 2 – Cognitive Systems, Interaction, Robotics – under grant agreement No 230902 - ROSETTA.

Please visit our webpage for further information and contact information:

http://www.fp7rosetta.eu/

Klas Nilsson ([email protected])

http://rss.cs.lth.se

lund september 2010 robotics and automation software€¦ · lund september 2010 robotics and...

Documents