towards agricultural robotics for organic farming · arw/Öagm 12.05.2016 georg halmetschlager...

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

Towards Agricultural Robotics for Organic Farming

Wels, May 11th-13th 2016

Georg Halmetschlager, Johann Prankl, Markus Vincze Vienna University Of Technology, ACIN, V4R gh@acin.tuwien.ac.at

ÖAGM/ARW

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

FRANC v1 (2014)

2

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

FRANC v2 (Now)

3

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• Field Robot for Advanced Navigation in bio-Crops • Example: manual weed control • Modular system design

• Autonomous platform • System extensions for different applications • → Avoid re-design of existing solutions

©

BIO

-LU

TZ G

MBH

The Big Picture

4

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• Robot system • Mechanical system • Electrical system • Sensors & software modules

• Navigation • Positioning of the robot

relative to the crop rows • Crop row detection • Challenges

• Different plant species, growing stages, field/row structures, plant densities

©

BIO

-LU

TZ G

MBH

The Big Picture

5

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• Subsystems • Mechanical

• Reusability, flexibility • Electronics and Control System

• Modularity, replaceability • Row Guidance and Autonomy Software

• Robustness, replaceability, parameterization free, and GPS free

Modularity

6

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

Frame Implement Powertrain

Mechanical Subsystem (1)

7

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

Mechanical Subsystem (2)

8

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• Example 4 WS → Ackermann steering principle?

Kinematics (1)

9

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• Linear Interpolation between • steering angles → Ackermann

constraint violated

• Interpolation of ICC → Ackermann constraint fulfilled.

Kinematics (2)

10

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

Electronics

11

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• SoA • Sensors [1-7]

• Cameras, LIDAR, laser range finder • Cost efficiency → pure machine vision approach

• Segmentation [1-5] • Color Information, spectral information, 3D information • Combination of NIR and 3D information

• Crop Line Detection [2-9] • Hough Transformation, Linear regression, stripe analysis, blob

analysis • Probabilistic methods

[1] Astrand, B., Baerveldt, (2005; [2] Jiang et al., (2010); [3] Ruiz, et al., (2010); [4] Romeo et al., (2012); [5] Kise, M., et al., (2005); [6] Weiss, U., Biber, P., (2009); [7] Fontaine, V., Crowe, T., (2006)

Sensor and Navigation System

12

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• Sensorsystem • 3D Information Stereo

cameras

• Spectral Information NIR

camera

• NIRD Segmentation • Discriminate Plants from

Soil • Different growing stages • Different plant species • Combination of NIR and

3D Information

[www.theimagingsource.com/]

[www.ptgrey.com/]

Sensor System and Segmentation

13

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• Row Detection (and Tracking) • Parallel lines can be represented by 3 parameters • Basevectors |r|, q, d establish a 3D parameter space • Generate hypotheses

• Based on given data • Sequential approach

Generic Crop Row Detection

14

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• Search after parallel lines • RANSAC

• Hypotheses generation out of the given data-set • One-step algorithm

• Particle Filter • Verification of randomly generated hypotheses • Cyclical algorithm

Crop Row Detection

15

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• Parallel line search • Grouping after parallelism • Grouping after offset • Offset based group reduction

RANSAC

16

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• 3D parameter space • Initialization with N random hypotheses • Rating of N hypotheses • Redrawing of N hypotheses

• Good hypotheses are selected several times (”survival of the fittest”)

• Prediction, followed by next iteration • Hypotheses clusters after some iterations

Particle Filter- Theory

17

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• Prediction • Movement of the robot has to be modeled

Particle Filter- Prediction (1)

18

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

• Assumption→ movement of the particles within the parameter space can be modeled with Gaussian noise

Particle Filter- Prediction (2)

19

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

Particle Filter- Example

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

Results- Particle Filter

21

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

Conclusion & Contributions

22

• Mechanics • Solved problems with deadlocks during steering motion

• Electronics • Engineering • Safety remote control

• Machine Vision • Robust new segmentation method that combines strongest

features • Generic, GPS-free, probabilistic crop row detection and tracking

method that offers high detection rates • Stereo/NIR Dataset • ROS modules

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

Conclusion & Contribution

23

ARW/ÖAGM 12.05.2016

Georg Halmetschlager gh@acin.tuwien.ac.at

This work was funded by Sparkling Science a programme of the Federal Ministry of Science and Research of Austria (SPA 04/84 – FRANC).

Questions?

24