![Page 1: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/1.jpg)
10.09.2016
Physical Continuous IntegrationCI with Real Robots!
Alex Henning
![Page 2: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/2.jpg)
10.09.2016
• Using best practices - Continuous integration- Code reviews- Linters
• Releasing took a lot of time and manual effort- Verifying robots behaved appropriately- Slow feedback cycle- Real world interactions caused issues
• We want releases to be fast and robust
The Problem
![Page 3: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/3.jpg)
10.09.2016
Software Continuous Integration The process of automatically verifying each change, allowing teams to detect problems early.
The Solution
Physical Continuous Integration The process of automatically verifying each change on real robots, allowing teams to detect problems early.
![Page 4: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/4.jpg)
10.09.2016
freight: mobile platformWhat are We Testing?
fetchcore: fleet management
![Page 5: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/5.jpg)
10.09.2016
Test Facility • 7500 Sq. Ft. warehouse• Multiple robots • 16 cameras
Our SetupServers • Bag file server • Video server• fetchcore servers
![Page 6: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/6.jpg)
10.09.2016
• Pull in changes from ROS• Test locally• Start physical continuous integrationAnd… • Some robots refuse to move
Testing a Change
![Page 7: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/7.jpg)
10.09.2016
Available resources: • Robots are always bagging and logging• Cameras are always recording• Tools to get relevant bags and video
What’s Wrong?
![Page 8: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/8.jpg)
10.09.2016
Let’s investigate: • Logs:[WARNING] Global plan in collision, replanning [WARNING] Global plan in collision, replanning …
• Cameras:
• Bag files and RViz
What’s Wrong?
![Page 9: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/9.jpg)
10.09.2016
• Part of the ROS navigation stack• Used in planning paths• Helps avoid crashing
A Brief Introduction to Costmaps
Obstacle Inflated Obstacle High Cost Low Cost Free Space
![Page 10: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/10.jpg)
10.09.2016
What the Bug Looks Like
Bad
![Page 11: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/11.jpg)
10.09.2016
We know what to look for 1. Reproduce the bug2. Develop a fix3. Verify fix4. Software continuous integration5. Code review
Now What?
![Page 12: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/12.jpg)
10.09.2016
• Need a test- Automated- Catches the problem- Avoid regressions
• Run test in the warehouse• Results get posted on github
Physical Continuous Integration
>>> Preparing test of ros-planning/navigation#475 >>> freight9 is ready >>> freight10 is ready >>> freight11 is ready >>> fetchcore is ready >>> Starting test ... >>> Test is done >>> Posting results to github
![Page 13: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/13.jpg)
10.09.2016
All tests passProblem Solved
![Page 14: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/14.jpg)
10.09.2016
Ansible • Configuration• Orchestration• Manages many machines
Your Turn
Custom Software • Ties it all together• Integrates with github• Gathers data• Tools for investigating
ZoneMinder • Records video• Detects motion• 18 TB of storage
![Page 15: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/15.jpg)
10.09.2016
Need to deal with the real world • Crashing• Delocalization• Batteries• Deadlocks
To reduce problems • Code review• Software continuous integration• Early termination
Pitfalls
![Page 16: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/16.jpg)
10.09.2016
Physical Continuous Integration• Part of the a good development workflow• Not a silver bullet• Anyone shipping robots should integrate it
Final Remarks
![Page 17: Physical Continuous Integration · - Continuous integration - Code reviews - Linters • Releasing took a lot of time and manual effort - Verifying robots behaved appropriately -](https://reader033.vdocument.in/reader033/viewer/2022060318/5f0c7e8d7e708231d435af03/html5/thumbnails/17.jpg)
10.09.2016
Questions