a flexible seam detection technique for robotic laser welding (shortened english version) jorg...

A flexible seam detection technique

for robotic laser welding

(Shortened English version) Jorg Entzinger

3

Seam to Weld

Laser bundel

4

Laser

Focus Lens

5

Laser diode

Dichroic mirror

Video camera

Camera lens

6

Laser

Focus Lens

7

Presentation Structure

• Introduction

• Lens & camera calibration

• Image undistortion

• Seam Detection

8

Functions of the Multifunctional Welding head

• Detect seams

• Track & learn seams

• Laser weld seams

• Process control

• Quality control

IntroductionWhy Camera Calibration?

How it is done Results PlanningIntroductionCamera &

lens calibration

Image undistortion

Seam detection & world coord.

Results Conclusies

9

Specialities of this welding head

• Multifunctionality All needed technology is integrated in one machine

• CompactnessFlexible in use for complex geometries

• LightweightFor higher accuracies with the use of robots

IntroductionCamera &

lens calibration

Image undistortion


Results Conclusies

10

Assignment

1. Develop a system that can compensate for lens distortions

2. Develop a system to determine the exact position of the workpiece with respect to the welding head from camera images


lens calibration

Image undistortion


Results Conclusies

11

Distortion Types

• Perspective distortions• Camera distortions (skew, non-squareness of pixels)

• Lens distortions (radial: barrel/pincushion)

• Noise (dust, bad focussing, CCD measurement noise)

Normal Perspective Skew Barrel Pincushion


lens calibration

Image undistortion


Results Conclusies

12

C++

Read parametersfrom file

Generate look-op table of pixel displacements

Aquire camera image

Undistort image

Program Structure

MATLAB

Make calibration pattern

Take pictures

Identify keypoints

Sort keypoints

Parameter estimation

Write Params to File


lens calibration

Image undistortion


Results Conclusies

13

Program Structure


lens calibration

Image undistortion


Results Conclusies

C++



Aquire camera image

Undistort image

MATLAB


Take pictures

Identify keypoints

Sort keypoints



14

The calibration-pattern

MATLAB


Take pictures

Identify keypoints

Sort keypoints

Estimate parameters

Write params to file


lens calibration

Image undistortion


Results Conclusies

15

Pictures of theCalibration-pattern


lens calibration

Image undistortion


Results Conclusies

MATLAB


Take pictures

Identify keypoints

Sort keypoints

Estimate parameters


16

Identified Keypoints


lens calibration

Image undistortion


Results Conclusies

MATLAB


Take pictures

Identify keypoints

Sort keypoints

Estimate parameters


17

Sorted Keypoints


lens calibration

Image undistortion


Results Conclusies

MATLAB


Take pictures

Identify keypoints

Sort keypoints

Estimate parameters


18

Parameter estimation (1)


lens calibration

Image undistortion


Results Conclusies

Perspective Camera

19

Parameter estimation (2)


lens calibration

Image undistortion


Results Conclusies

Barrel Pincushion

or

20

Estimation refinement

Homography was calculated without considering radial distortions

Distortions are calculated from an inaccurate homography

The estimations must be refined, all parameters are optimized at the same time


lens calibration

Image undistortion


Results Conclusies

21

Program Structure


lens calibration

Image undistortion


Results Conclusies

C++



Aquire camera image

Undistort image

MATLAB


Take pictures

Identify keypoints

Sort keypoints



22

Program Structure


lens calibration

Image undistortion


Results Conclusies

C++



Aquire camera image

Undistort image

MATLAB


Take pictures

Identify keypoints

Sort keypoints



23


lens calibration

Image undistortion


Results Conclusies

Undistortion

24

An Original (Distorted) Picture


lens calibration

Image undistortion


Results Conclusies

25

Test Result


lens calibration

Image undistortion


Results Conclusies

26

Test Result


lens calibration

Image undistortion


Results Conclusies

Original Undistorted

27

Simulated Distortion Test



lens calibration

Image undistortion


Results Conclusies

28

Result



lens calibration

Image undistortion


Results Conclusies

29

Amout of Distortion


lens calibration

Image undistortion


Results Conclusies

Pix

el m

ovem

ent

in %

Position on image diagonal

30

Programma Structuur


lens calibration

Image undistortion


Results Conclusies

C++



Aquire camera image

Undistort image

MATLAB


Take pictures

Identify keypoints

Sort keypoints



31

Program Structure

IntroductionLens

DistortionsCalibration Procedure

Program PlanningIntroductionCamera &

lens calibration

Image undistortion


Results Conclusies

C++



Aquire camera image

Undistort image

Determine seam location

Move Robot

MATLAB


Take pictures

Identify keypoints

Sort keypoints



32

Changes in camera image due to a change of relative position


lens calibration

Image undistortion


Results Conclusies

33

Changes in camera image due to a change of relative position


lens calibration

Image undistortion


Results Conclusies

34

World Coördinates

• How many millimeters in reality

is 10 pixels in the image?

• If the image moves to the right, in what direction did the robot move?

• Where is the camera

with respect to the welding spot?


lens calibration

Image undistortion


Results Conclusies

35

Test objects


lens calibration

Image undistortion


Results Conclusies

36

Following a seam


lens calibration

Image undistortion


Results Conclusies

Thank you

for your attention

Jorg Entzinger

39

Rotatie Matrix

40

Rodrigues Parameters

41

Rotatie Matrix Rodrigues Parameters

42

Schatten van de Parameters (1)


lens calibration

Image undistortion


Results Conclusies

met

43

Aanbevelingen

• Pixel-millimeter schaling hoogte afhankelijk maken

• Bepaling naad-positie minder afhankelijk maken van handmatige instellingen

• Zorgen voor goede afhandeling als de naad dicht bij de kruising van de lijnen komt

• Goede gebruikers-interface voor camera & lens calibration maken


lens calibration

Image undistortion


Results Conclusies

44

Camera & Lens distortions

• Perspective distortion

• Skew distortion

• Radial distortions (barrel & pincushion)

• Noise

Normal Perspective Skew Barrel Pincushion

45

Why Camera Calibration?

46

Detectie van Lijnen

47

Scheiden van Lijnen in het Kruis

48

Detectie van de Lasnaad

49

Parameters schatten

Er worden subsets gemaakt vanDatapunten uit 4 plaatjes, bijvoorbeeld:

Subset 1 Subset 2 Subset 3 ...

Dataset 1 Dataset 2 Dataset 3Dataset 2 Dataset 4 Dataset 5Dataset 3 Dataset 6 Dataset 7Dataset 4 Dataset 8 Dataset 8

Voor elke subset wordt een calibrationuitgevoerd

MATLAB

calibration patroonmaken

Foto’s nemen

Keypoints identificeren

Keypoints sorteren

Parameters schatten

Parameters naarbestand schrijven

50

Parameter Schattingen Op Basis Van Meerdere Subsets

RMS alfa f

51

Schatten van de Parameters (2)

Homografie (per plaatje): 8 DOFs

Plaatje afhankelijk: 6 DOFs(3 rotatie en 3 translatie)

-

Over voor schatting camera

parameters: 2 DOFs

Er zijn 5 camera afhankelijke parameters,

dus er is minstens 2½ plaatje nodig

52

Verstorings-gebied fotocamera

53

Experiment


lens calibration

Image undistortion


Results Conclusies

54

Experiment


lens calibration

Image undistortion


Results Conclusies

55

Meetfouten


lens calibration

Image undistortion


Results Conclusies

X-positie [mm]

X-positie [mm]

X-positie [mm]

Y-f

ou

t [m

m]

Z-f

ou

t [m

m]

To

taal

-fo

ut

[mm

]

A flexible seam detection technique

for robotic laser welding

Jorg Entzinger

60

Dagplanning

13:00 – 13:45 Presentatie

13:45 – 14:00 Vragen uit de zaal

Jorg & Examencommissie

14:00 – 14:30 Demonstratie in het lab

14:30 – 15:30 Ondervraging

Rest

14:00 – 15:30 Rondleiding door Niels en/of

Koffie/Thee in WB (Horst) kantine

Iedereen

15:30 – 16:00 Diploma-uitreiking & felicitatie (WB-Z109)

16:15 – 18:00 Borrel & Demonstraties in het Lab

(WB-Hal IV = Westhorst)

a flexible seam detection technique for robotic laser welding (shortened english version) jorg...

Documents

resultsconclusies slide

calibration pattern

file slide

image matlab

calibrationpattern matlab

laser focus lens slide

resultsconclusies matlab

image program structure