lean intelligent assembly automation (liaa) · lean intelligent assembly automation (liaa) ... –...

Lean Intelligent Assembly Automation (LIAA)

2nd Review – Workplace Configuration

Rasmus Hasle Andersen

Danish Technological InstituteForskerparken 105230 OdenseDenmarkTel.: +45 7220 1864 / +45 7220 2000E-Mail: [email protected]

mailto:[email protected]

2LIAA 2nd Review Brussels, 2016-05-11

• Workplace Design:

– Define Bill of Materials

– Define Bill of Processes

– Define Bill of Resources

– Design Workplace Concept

– Design Workplace

– Simulate Workplace

– …

– Export Workplace Design and Documentation

• Workplace Construction (not part of LIAA)

– Purchase/manufacture components

– Integrate components

• Mechanical

• Electrical

• Pneumatic

• Hydraulic

• Workplace Configuration:

– Configure individual modules• Hardware and Software modules

– Configure dependencies between module

• e.g. Hand-eye calibration

– Finalize Skill Network

– Set Skill Parameter

– Configure configuration view

– …

– Start Runtime

• Workplace Runtime:

– Execute Skills

– Track Human

– Visualize Information

– Identify anomaly

– Recover and continue

– …

LIAA Workflow

Workplace DesignWorkplace

ConstructionWorkplace

ConfigurationWorkplaceRuntime


Workplace Configuration– Contributing WPs

LIAA Configuration Workflow

<WP5>„Symbolic Skill

Execution Engine“ by DTI

<WP7>„Situation Assessment

Module“ by DTI

<WP7>„Human Activity

Recognition“ by TECNALIA

<WP5>„Task-to-Skill Generator“

by IPA

<WP3>„Safety Zone

Configuration “ by XX

<WP6>„AR Enhanced

User Interaction” by EON

<WP5>„Generic Vision Framework “

By DTI

<WP5>„Skill Editor”

by DTI

<WP5>„Generic Vision Calibration “

by DTI & Tecnalia


• Module configuration

– Intra module, and

– Inter module

• Process Configuration

– Process instruction

• Validation of process “suggestion” from the Skill Generation Module

– Process adjustment (and final validation)

• Situation Assessment setup

• Initiate run-time

Workflow - Overview

Module Configuration

Inter & intra module

Process Instruction

Skill Generation

Module

Situation Assessment

Configuration

Inspection of process

Runtime initiation

Process Adjustment


Workflow – Mapping Steps to Pilot Case (1/2)

Mapping of Configuration Workflow Steps to Pilot Case

OP

EL

FIS

CH

ER

DD

E

SP

IN

EA

TELN

ET

Step 01: Physical setup and system start

Step 02: Safety Configuration

Step 03: Vision Setup (teach new object) O – X

Step 04: Human Activity Recognition Calibration X X – O X

Step 05: Augmented Reality Calibration – – O O

Step 06: CAD Based Skill configuration – – X –

Step 07: Hand-eye Calibration O – X

– : Not part of the Pilot Case

X : Necessary Step

O : Desired step

: Tested & Validated


Workflow – Mapping Steps to Pilot Case (2/2)

Mapping of Configuration Workflow Steps to Pilot Case

OP

EL

FIS

CH

ER

DD

E

SP

IN

EA

TELN

ET

Step 08: Process inspection and adjustment X X X X

Step 09: Process verification X X X X

Step 10: Situation Assessment setup O O O O

Step 11: Modular GUI Setup X X X

Step 12: Start runtime X X X

– : Not part of the Pilot Case

X : Necessary Step

O : Desired step

: Tested & Validated

All steps are relevant for more than two Pilot Case


• Purpose

– Ensure the physical setup is completed

– Ensure the required modules are available

• Prerequisite

– Design and construction completed

• Effect

– Modules are connected (HW) and installed for the framework to use (SW)

Step 01 – Physical Setup and System Start


Step 01 – Physical Setup and System Start – Video


Configuration of Modules


• Purpose

– Instruct the user to do the safety configuration on the Robot Controller

– A PDF is presented to the user detailing how to do the configuration.

• Or AR enhanced presentation

– Safety controller does not have interfaces which are configurable externally

• Prerequisite

– The safety must be specified in the design phase.

• Effect

– The LIAA cell is configured to operate safely in the given environment

Step 02 – Safety Configuration - Beta


Step 02 – Safety Configuration - Beta (Video)


• Purpose

– Effective and intuitive teaching of new objects using CAD models.

• Prerequisite

– CAD models of desired object must be available

• Effect

– The LIAA system is able to detect and localize the new object type

– Enables users/workers to teach a system new objects

• Benefit/Added value of the LIAA Solution

– Increases the flexibility and reusability of the LIAA framework

– Thoroughly tested and widely used calibration routine automatically included and exploited in the framework

Step 03 – Vision Configuration (teach new object)


Step 03 – Vision Configuration (Video)


Step 04 – Human Activity Recognition (HAR) Calibration

Multi-Kinects calibration

Select used Kinects and calibration marker

ResultsTransformation between Kinects and Robot

Robot-Kinects calibration

Configure Robot and Moveitparameters

Specify dimension of calibration grid Define number of Iterations


Step 04 – HAR Calibration (Video)


• Purpose

– Calibrate the virtual and the real world

• Effect

– The real and virtual worlds are aligned

– The AR enhanced Human-Robot Interaction can be used


– Enables the framework to provide intuitive operator instructions

– Enables process validation though simulation of the process but visualized in the physical world

Step 05 – Augmented Reality Calibration


Step 05 – Augmented Reality Calibration (Video)


• Configuration :

– Prepare CAD models

– Select information and define process using interface

• Execution

– CATIA

– ROS

• Results

– XML files for skill parametrization

Step 06 – Cad Based Skill Configuration


• This is a teaser for the upcoming presentation on the Telnet Pilot Case

Step 06 – Cad Based Skill Configuration (video)


• Purpose

– Extrinsic calibration between camera and robot

• Prerequisite

– A “robot” module must be connected and configured

– A camera/vision modules must be connected and configured

• Effect

– The robot can directly use the poses originating from the vision system


– Enables fast reconfiguration of the robot to/from camera transformations

– Performs a fully automated hand-eye calibration using the LIAA skill framework

Step 7 (a) – Hand-eye calibration


Step 7 (a) – Hand-eye calibration (video)


Step 7 (b) – Tecnalia Calibration

Manual Configuration (ROS .launch

file):

- Robot

- Vision sensor

- Number of images/poses

- Initial pose of the robot

- Calibration panel properties

Execution:

Results:

- Intrinsics

- Extrinsics


Step 7 (b) – Tecnalia Calibration (video)


Process Configuration


• Purpose

– Inspect and verify the process provided by the Skill generation module

– Modify/Adjust if necessary

• Prerequisite

– A skill representing the process must be available

• Effect

– The skill is verified to operate as expected


– The user can exploit the Skill Generation module to provide a good estimate of the process description.

– Adjustments can be made to tweak the final operation

Step 8 – Process inspection (and adjustment)


Step 8 – Process inspection (and adjustment) (video)


• Purpose

– Test the entire skill execution to verify the skill execution

• Prerequisite

– A Skill modelling the process must be available

• Effect

– The skill has been verified to satisfy the goal of the process (nominal flow)

– The process configuration is completed

Step 9 – Process Verification


Step 9 – Process Verification (Video)


• Purpose

– Configure the available sensors

– Record the nominal behaviour of the skill

– Connected sensors

• Prerequisite

– Skill instruction and parametrization must be finished

• Effect

– Nominal behaviour learned and ready for monitoring during execution

– The Process


– No need for anomaly detection code in skills

– Added robustness

Step 10 – Situation Assessment


Step 10 – Situation Assessment (Video)


• Purpose

– Enables the user to select which “views” to be visible

• Prerequisite

– Individual modules must register “views”

• Benefit..

– Individual modules can provide unique views

– Enables per user and per process customization of the visual interface

Step 11 – Modular GUI Setup


Step 11 – Video (Modular GUI Setup)


• Runtime activation

• Select main skill

• Click run

Step 12 – Start runtime


• Single point of access for configuration of all modules

• Intuitive configuration of individual modules

• Modular and dynamic configuration

• Simple process instruction, testing and validation

• Intra- and intermodule configuration

• Initiate run-time

• Can work as a stand-alone configuration tool-chain

• Can benefit from the integration with the LIAA design phase

Summary

Module Configuration

Inter & intra module

Process InstructionSkill Generation

Module

Situation Assessment

Configuration

Inspection of process

Runtime initiation

Process Adjustment


Summary Video


• Summarize the next development steps till 02/2017

– Integration of more modules (Individual Partners)

– Improve the configuration framework (DTI)

– Improvement of existing modules (Individual Partners)

– Tighter coupling with the Design tool-chain (DTI and IPA)

– Integration of the LIAA Skill Library (UR)

– GUI Enhancements based on partner feedback (DTI)

– Enhancing the robot control through configuration of robot path planning module(s) (DTI)

– Integrate configuration of (all) modules in the (unified) configuration view(all LIAA partners)

– Runtime visualization

Outlook for M31-M42


Thank you for your

Attention!

Questions?

For more information visit us at www.project-leanautomation.eu

Project CoordinatorMartin NaumannTel: +49 711 970-1291Fax: +49 711 970-1008E-Mail: [email protected]

Project OfficeTel: +49 711 970-1844Fax: +49 711 970-1008E-Mail: [email protected]

Contact Information

http://www.project-leanautomation.eu/




Bonus – Runtime after configuration

lean intelligent assembly automation (liaa) · lean intelligent assembly automation (liaa) ... –...

Documents