how to place 6th in the amazon picking challenge (enb329, qut)
TRANSCRIPT
How to place 6th in theAmazon Picking Challenge
Juxi Leitner
Australian Centre for Robotic VisionQueensland University of Technology
Thanks to Adam Tow for the title, suggestions and the APC slides!
http://Juxi.net/[email protected]
Juxi
About Me
About Me
Historical: The word robot stems from a play in 1921.
Projects
http://Juxi.net/projects
Research
Cost: WALL-E: 180M USD, Mars Pathfinder mission 175M USD (+90M for launch and ops)
The context
Amazon founded 1994
more than 200 million productsmore than 2 billion products sold in 2014
more than 75 fulfillment centres in the US more than 30 000 Kiva robotsmore than 250 worldwide
Interesting Fact: Australia is serviced from … Germany!
more than 30 billion in revenue Q2/16 about 230 000 employees
.com
What would you change?!
Discuss how Amazon (or in fact any warehouse driven company) is using automation and how robotics can help to further automate!
Info: Robot is not equal to AI
What would you change?!
Discuss how Amazon (or in fact any warehouse driven company) is using automation and how robotics can help to further automate!
AI can solve chess better than humans, robots are worse at
picking up chess pieces than the average 2 year old!
But robots and AI are two sides of the same coin! Lots of interaction between these two fields of research!
The problem
Amazon Picking Challenge
Shelf Localization
Precision
The shelf is “brought” by a Kiva robot - the position is not precisely known before starting the task.
Picking Task
40 objects in the shelf
Pick the objects according to the order and place them in the shipping box (tote).
Stowing Task
Returned items
A box of objects is returned and is required to be placed back into shelf for storage.
How would you approach this problem?
Discuss with your team how YOU could solve the problem that Amazon has with getting the orders into the shipping boxes! What’s the approach?
Brainstorming!
Eye-hand coordination
Jan 2016
Application sent & Interview with Amazon
Feb 2016
Accepted as one of the 16 teams competing in the APC 2016.
Four months of crazy-ness, long hours and lots of code!
T-2d
Code until 2pm, pack and fly out of BNE to Europe at 9pm
July 2016
Competition during RoboCup 2016 in Leipzig, Germany
The teamTeam Leader
PhD students Post-Docs
& Professors
Undergrads
BEB 801/2ACRV / RAS
Running a project
Project Management
?!
Project Management
Getting everything done, that you need to get done, by the time you should be done with it!
PlanningTime available
Scope of project
Critical part
Resources
Focus
Making the best ofwhat we know now
What isyour plan?
Discuss with your team!? What is being done, what needs to be done, by when? By whom?
Interesting Fact: There are about 1.5-2 million industrial robots in operation
What isyour plan?
APC Day 1Game plan:
● Wake up at 7 am :)● Eat some food● Get to Leipziger Messe by 8 am ● Setup the robot in around 2 hours● Pick some items in the avo ● Start on the additions left on the board in the evening● Leave when we get kicked out at 9 pm
What actually happened:
● Woke up at 7 am -_-● Ate some food● Found Leipziger Messe by 9 am ● Setup the robot in around 12 hours● Pick some items in the avo ● Start on the additions left on the board in the evening● Left when we got kicked out at 11 pm
Interesting:
There were more than 3 million domestic robots sold in 2014!
Project Management
Integration
Integration
Integration
Code
Communication (for every team)
Robotics
It’s the tools, stupid!
Project Management
Getting everything done, that you need to get done, by the time you should be done with it!
It’s the tools, stupid!
It’s the tools, stupid!
Communication
Communication in one place
Works with other tools
Code snippets
AI as UI
Cross device!
Task assignment
Task follow up
Notifications
Progress
Stuff to do...
ToDo lists
Need someone that drives
Need to take responsibility
Need to delegate
Need to revisit
Next action to take
breakFun Fact: as a roboticist you will get asked about the three laws repeatedly.
breakAsimov managed to write more than 50 stories/books about the three laws,
all of them have at their core a problem with the three laws!
Fun fact: There is a “Handbook of Robotics” by Springer
Integrate
PrototypeMVP
Need someone that drives
Need to take responsibility
Need to delegate
Need to revisit
Next action to take
Scrum
Same time
Same place
Clear agenda
Current work
Road-blocks
...nobody likes meetings!
you are usingVersion Control
g i t
GitHub
Bitbuckethttps://try.github.io/levels/1/challenges/1
GitHub GitKraken
How are you managing
your team?
Discuss with your team how your project is managed? Do you have regular meetings, do you have responsible people? Do you have clear tasks to be finished by strict dates?
APC Day 1Git Server
A new pedestal
Baxter Research SDK
Camera Extrinsics Calibration
APC Day 2
SOOOOOOO MANY BUGS
MORE FEATURES NEEDED
VERY STRESSFUL THE WHOLE TIME
(except at Breakfast for some reason)
James S to the rescue though, he feature
developed while we slept that night (perks
of being an Aussie team)
APC Day 3
You knew one could be so stressed out
We trained a CNN on data annotated by Jake
the night before
Tested and seemed to work alright
Ready to stow an hour before our time slot
Feature freeze an hour before our time slot
APC Day 3 - The Stow Task
Successfully stowed 2 items
Smashed the water bottle into the shelf
and dropped it
Lost the points we’d gained with a bug
in our shelf item updater
Still haven’t searched for the bug
APC Day 3
Straight on to testing of the picking task code
More bugs to fix
Strange - we took no pictures between post-stow and pre-pick
I guess we were busy...
APC Day 4
Finished development 2 hours
before our run
Stressfully sat with the robot for 2
hours
Placed 6th - Wooo!!!
Went out
APC Day 5
Demo day - talked about our system
and approach to those interested
Pack up
Watched some robocup
Networking!
APC Tools
Team Communication (Slack)
Task Management (Asana)
Git Repositories (BitBucket)
Git GUI (GitKraken)
Shared Documents (GDrive, Dropbox)
Heaps of other tools from CADing to open source libraries,
such as, ROS, OpenCV, ...
Tools Shared References (Mendeley)
Code (Git | BitBucket)
Shared Writing (BitBucket)
DeepReaching
SolutionMore premium subscribers
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
Tools Team Communication (Slack)
Task Management (Asana, FB)
Shared Documents (GDrive)
LunaRoo
Interested?There’s cool robotics
projects at QUT!
Fact: RoboCup will be in Sydney in 2019!
ThursdayPizza and Robots
S11, QUT5pm
http://tinyurl.com/RoboticsPhD
Juxi Leitnerhttp://Juxi.net
Contact me for VRES,undergrad and PhD projects
Thank you