prepare for variation · iros workshop on agri-food robotics, hamburg, oct 2nd, 2015 . 2 1....
TRANSCRIPT
![Page 1: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/1.jpg)
1
Prepare for Variation – Towards Using AI/Robotics Methods for Handling
Uncertainty in Agriculture Applications
Joachim Hertzberg
Osnabrück University and
DFKI Robotics Innovation Center (RIC), Osnabrück Branch
IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015
![Page 2: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/2.jpg)
2
1. Uncertainty in AI
2. Small Example: Sensor Data Interpretation
3. Larger Example: Combine Harvester Control
4. Wrap-up
![Page 3: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/3.jpg)
3
Uncertainty in AI – Generic Methods
Part I Artificial Intelligence Part II Problem Solving Part III Knowledge and Reasoning Part IV Uncertain Knowledge and Reasoning
13 Quantifying Uncertainty 14 Probabilistic Reasoning 15 Probabilistic Reasoning over Time 16 Making Simple Decisions 17 Making Complex Decisions
Part V Learning Part VII Communicating, Perceiving, and Acting Part VIII Conclusions
3rd Ed., 2010
2005
… and in AI/Robotics
![Page 4: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/4.jpg)
4
Dealing with Uncertainty – the AI Arsenal
• Probabilistic representation & reasoning – Bayesian networks – Graphical models
• Logical r&r about defaults/prototypes and preferences – Answer set programming
• R&r about time and space – Constraint-based quantitative methods, e.g., STPs – Interval-based qualitative calculi, e.g. interval algebras
Key problem for building applications: combine • representational efficiency, learnability • inferential efficiency • integration of different reasoners & their representations
![Page 5: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/5.jpg)
5
Uncertain ≠ Chaotic!
• Literature (AI, Robotics) often mentions “unstructured environments”
• Nearly always they mean “not 150% controlled”: variety, lack of knowledge, clutter, occlusion, dynamics, … – but normally a high degree of structure! Structure is often known and/or learnable!
• R&r can exploit this structure, if part of a closed-loop control! (and so can learning, perception, action, …)
![Page 6: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/6.jpg)
6
1. Uncertainty in AI
2. Small Example: Sensor Data Interpretation
3. Larger Example: Combine Harvester Control
4. Wrap-up
![Page 7: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/7.jpg)
7
Own Contributions: Interpreting 3D Point Clouds
… but until now mostly in non-agriculture applications
![Page 8: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/8.jpg)
8
Rigor in Structure – Variation in Detail
Photos: Wikipedia/english “Grape” Image: Wikipedia/deutsch “Weinrebe”
![Page 9: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/9.jpg)
9
Model-Based Phenotyping Steinhage et al.: A Model-Based Approach to High Performance Phenotyping, 2012 Univ. Bonn, CROP.SENSe
Grammar-Based Stem-Skeleton Interpretation Relational Growth Grammar (cf. skeleton models)
Probabilistic Sampling of 3D Scan Structuring Hypotheses Monte-Carlo-based sampling in hypothesis space
![Page 10: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/10.jpg)
10
Model-Based Phenotyping, ctd.
Result Hypotheses • Length of rachis: 8.50 cm
• Av. pedicel length: 0.95 cm
• # berry groups: 18
• # nodes on rachis: 9
• # twigs with sub-twigs: 3
• # terminal pedicel twigs: 9
If this is often enough good-enough (“satisficing”) information …
![Page 11: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/11.jpg)
11
1. Uncertainty in AI
2. Small Example: Sensor Data Interpretation
3. Larger Example: Combine Harvester Control
4. Wrap-up
![Page 12: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/12.jpg)
12
Cooperative Corn Harvesting Scenario
S. Scheuren, S. Stiene, R. Hartanto, J. Hertzberg, M. Reinecke. Spatio-temporally Constrained Planning for Cooperative Vehicles in a Harvesting Scenario. KI – Künstliche Intelligenz, 27(4):341-346, 2013
Harvested area Non-harvested area Combine harvester Tractor Truck
Driving Paths
Rendezvous point
![Page 13: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/13.jpg)
13
Route Planning for the Tractor Outer field border Inner field border
Driving traces Headland trace Unloading spot
Route graph Overloading window
![Page 14: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/14.jpg)
14
Motion Planning, Structure
Tractor Harvester
Motion Planning
Approximate nominal path
SBPL lattice planner2
Graph search
Approximate nominal path
SBPL lattice planner2
![Page 15: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/15.jpg)
15
Motion Plan for the Tractor
Outer field border Inner field border
Driving traces Headland trace Unloading spot Path of tractor
Overloading window
![Page 16: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/16.jpg)
16
Variety #1: Machine Parameters
Motion Planning ?�Steering system
Path
Sensor data
max.
No AI – just engineering!
![Page 17: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/17.jpg)
17
Variety #2: Change of Yield/Zone
• Planning based on prior estimation (yield/zone in the past)
• Significant variation happens – less yield is no problem – more yield may require earlier overload
• No robust solution based on ad-hoc methods
Solution Approach
• Hierarchical hybrid (causal, temporal, spatial, resource) planning • CHIMP: Conflict-driven Hierarchical Meta-CSP Planner • S. Stock, M. Mansouri, F. Pecora, J. Hertzberg. Online Task Merging with a
Hierarchical Hybrid Task Planner for Mobile Service Robots. IROS 2015
![Page 18: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/18.jpg)
18
How to reduce this huge hybrid search space?
Reason about diffe- rent knowledge forms in an integrated way!
HTN Task Decomposition
Meta-Constraint reasoning1
1Cesta, Oddi, Smith: “A constraint-based method for project scheduling with time windows”, 2002. �
Hybrid Search Space
![Page 19: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/19.jpg)
19
• High-level requirements modelled as meta-constraints on top of a ground-CSP1�
• Hybrid planning as enforcing consistency in a family of constraint networks2�
• Meta-CSP framework3 allows to implement solvers for problems that can be cast as a meta-CSP�
1Cesta, Oddi, Smith: “A constraint-based method for project scheduling with time windows”, 2002. �2Mansouri, Pecora: “More knowledge on the table: Planning with space, time and resources for robots”, 2014 �3http://metacsp.org�
Meta-Constraint Reasoning
![Page 20: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/20.jpg)
20
• Integrates HTN-Planning into meta-CSP reasoning �• Represents initial state and resulting plan in constraint networks�• Creates partially-ordered plans �• Plans may contain actions that can be executed in parallel�• Allows online task merging of new goals into an existing plan
during plan execution �• Based on the meta-CSP Framework1�
Freely available at http://sebastianstock.github.io/chimp
1http://metacsp.org
CHIMP
![Page 21: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/21.jpg)
21
… and How Can CHIMP Go Corn Harvesting?
• Model constraints in hybrid search space – temporal (e.g., durations, duration variations), – spatial (e.g., tracks), – resource (e.g., capacity, inflow)
• Model plan change strategies as meta-constraints
• Apply plan change strategies at execution time if needed; keep to domain representation
Work in progress!
![Page 22: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/22.jpg)
22
1. Uncertainty in AI
2. Small Example: Sensor Data Interpretation
3. Larger Example: Combine Harvester Control
4. Wrap-up
![Page 23: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/23.jpg)
23
Wrap-Up
• AI has an arsenal of established generic methods for dealing with variations
• Extending them is subject of intensive current research
• Their integration may be challenging if more than one type of representation is needed
• Yet what are the alternatives? – Ad-hoc methods lack understanding/rigor – Specific methods are hard to integrate
• May be a win-win for agricultural applications and AI!
![Page 24: Prepare for Variation · IROS Workshop on Agri-Food Robotics, Hamburg, Oct 2nd, 2015 . 2 1. Uncertainty in AI 2. Small Example: Sensor Data Interpretation 3. Larger Example: Combine](https://reader034.vdocument.in/reader034/viewer/2022050203/5f56ffca2c40a144bf29cd5d/html5/thumbnails/24.jpg)
24
CHIMP paper: cf. IROS 2015 proceedings CHIMP planner: http://sebastianstock.github.io/chimp/ �
Meta-CSP framework: http://metacsp.org
Thanks to … … my Staff • Kai Lingemann • Stephan Scheuren • Stefan Stiene • Sebastian Stock • Astrid Ullrich • Thomas Wiemann
… the marion Project (2010–13) • Ronny Hartanto • CLAAS and staff • PT DLR • BMWi
… the SOILAssist Project (2015–18) • Thünen Institut • PT Jülich • BMBF
… the RACE Project (2011–14) • Masumeh Mansouri • Federico Pecora • Örebro U • Hamburg U • EU FP7