1 October 2015

The Next Industrial Revolution

Giving machines autonomy – and the ability to make decisions

Simon Jordan, Cambridge Consultants

Sensors and Instrumentation 2015

1 October 2015 2 Commercially Confidential

Introduction

Cambridge Consultants is a multidisciplinary

product development company.

We have been active in robotics for a while –

our largest project is a warehouse automation

system for a well known name.

This work is confidential – but today is about

another robot development and gives a view on

how robots fit an industrial society.

1 October 2015 3 Commercially Confidential

The robot worker – how does it fit in the value chain?

Aside from inventing economics, Adam

Smith is well known for the example of the

pin factory.

The idea is that people can become

specialised and efficient in one small task.

Existing robots take this to an extreme –

they typically take one small task and repeat

it to tight tolerances.

But – does this make them ‘useful idiots’?

1 October 2015 4 Commercially Confidential

But robots aren’t as good as the general public expect

Interaction with the real world is hard, and only narrow areas have been shown to be

useful.

1 October 2015 5 Commercially Confidential

‘Robots’ in a wider setting

‘Robot’ isn’t a useful word since it covers everything from this to this:

1 October 2015 6 Commercially Confidential

‘Robots’ in a wider setting

… so let’s be clear, by ‘robot’ I mean:

– A machine which can help a person be more economically productive by ‘doing

the boring stuff for them’.

And some tasks which a machine might do are:

– Driving to work

– Checking out groceries

– Assembling a camera lens

– Filing a management report

But management need to see a business case before signing a capex for a new

machine – and this means it needs to be cheaper (not necessarily faster or better)

than a human.

1 October 2015 7 Commercially Confidential

Early 21st Century Robotics

1 October 2015 8 Commercially Confidential

Factory robots

Typical usage is to plan (or teach), store

the program and repeat when triggered by

the PLC.

External sensors sometimes used to take

up tolerances or to fine tune positioning.

The performance is only as good as the

program – it’s really just a CNC machine.

The flexibility is at the capital rather than

operational level – one type of machine

can fulfil many roles.

First tool-changer CNC machine

1 October 2015 9 Commercially Confidential

Self-driving cars

Huge advances have been made in

the last few years, notably image

classification and collision

avoidance. It’s an amazing

achievement to do this.

Cars process a lot of information

(LIDAR and multiple cameras): they

have much higher ‘awareness’ than

a human driver.

But, there’s a key problem here…

1 October 2015 10 Commercially Confidential

Self-driving cars – map data

The map of the road is known in advance!

(is this really any better than the 1960’s paper tape?)

http://cdn2.alphr.com/sites/alphr/files/styles/insert_main_wide_im

age/public/4/95//hud_nvidia_px.png?itok=N8roiw-U

1 October 2015 11 Commercially Confidential

Where are the places where robots can make big gains?

An ideal area is:

– Has a high total value (can be low value but

done in massive quantity)

– Has been tricky to automate and so is

reliant on people.

– Is margin sensitive so improvements

change the bottom line.

– The businesses in the field are large

enough to invest in doing things a different

way and meet the cost of ownership1.

Look at the hierarchy of needs:

1 this probably excludes domestic robots!

1 October 2015 12 Commercially Confidential

ANSWER….

1 October 2015 13 Commercially Confidential

Getting robots out of the factories

and into the fields

1 October 2015 14 Commercially Confidential

What does a robot need to work here?

1. To be at the right point in the value chain.

Performing complex tasks like vine pruning is

possible (at an academic level) but isn’t anywhere

near a real farm yet.

2. To be able to work without any form of drawing as

it’s dealing with random objects and not

engineered parts. This means better sensing of the

environment and the ability to make decisions

without supervision.

3. To have an effector which is useful in the real world

– human hands are amazing but haven’t been

beaten by engineers yet.

1 October 2015 15 Commercially Confidential

Picking a point in the value chain

We need a point where we can make a difference – the

engineering NRE will be repaid by cost savings within a

sensible timeframe.

Planting? Not really needed as seed is cheap and we can

waste some.

Weeding? Already happening but could be limited by

ability to discriminate between plands

Harvesting? Getting closer, but much is done by machine.

Also, seasonal which makes machines hard to amortise.

Fruit and vegetable handling – yes! Lots of human labour,

can be indoors, predictable enough, massively price

sensitive. Also, some high value crops (specialist fruit,

almonds) where early adopters can see a good return.

1 October 2015 16 Commercially Confidential

Existing machine vision

1 October 2015 17 Commercially Confidential

Existing machine vision

This, again, has advanced massively. The examples

shown fulfil some well defined categories:

– Binary classification (‘is this bottle full?’)

– Data extraction (‘what serial number is this?’)

– Guidance (‘what angle are the threads at?’)

But these work on a narrow problem space – what is

expected is very well defined and so highly

programmable / teachable.

Also, 3D is new-ish to this field and could be

essential.

http://s.hswstatic.com/gif/pc-3-d-brain.jpg

1 October 2015 18 Commercially Confidential

The effector

Horses have no technology – at least

because they cannot use tools.

http://enlightenedequine.com/wp-content/uploads/2013/11/Horse-Human-Limb-Comparison.png

Humans use tools and so can control

their environment. However, our brains

had to be powerful enough to use them.

1 October 2015 19 Commercially Confidential

The fruit picking robot

1 October 2015 20 Commercially Confidential

So what are we showing?

We want to show a demo which:

– Puts robots into a new context

– Shows an innovative step such as new machine vision capability

– Has good system integration…

1 October 2015 21 Commercially Confidential

We’re showing three innovations here today

1. Novel application. Bin picking (the generic problem of getting objects out of stacks

or piles) is cutting edge robotics and is typically only attempted for engineered parts.

2. A flexible effector. Can pick up a screwdriver, but also handle a peach without

bruising.

3. Cutting edge machine vision – identification and guidance for random objects

without extensive training.

1 October 2015 22 Commercially Confidential

Gripper design decisions

1 October 2015 23 Commercially Confidential

Vacuum suckers

Suckers / grippers are well established – but there are some difficult circumstances:

http://biology.clc.uc.edu/fankhauser/Classes/Bio_103/Apple_calyx_stamen_P2210018.JPG

http://cdn4.pacifichorticulture.org/wp-content/uploads/2009/07/Barn.01.jpg http://www.teknoscienze.com/Contents/Articoli/Images/2013/-%20AF3%202013/Lourith/dragon-fruit.jpg

1 October 2015 24 Commercially Confidential

How to deal with this is a system design question

High mechanical complexity?

High per-unit costs

Likely to be fragile and therefore

expensive to own.

Tricky to adapt to other tasks

May actually have adverse software

requirements…

High software complexity?

Long development time.

Will need extensive testing if it isn’t

to ‘keep getting suck’.

Low per-unit costs once it’s written.

Flexible (if you can afford the NRE)

We’ve gone for a compromise – the gripper is soft so it can work from an

imprecise location yet wasn’t excessively complex to build.

1 October 2015 25 Commercially Confidential

Multi-finger vacuum gripper

© Cambridge Consultants

1 October 2015 26 Commercially Confidential

Vision system requirements

Aim is to pick fruit and vegetables from a pile.

Vision system has to work out:

– How the objects are connected (if at all)

– Which object is best to pick (near the top but not jammed)

– Where it is

– What’s the best angle to approach from.

This requires depth perception – 3D camera.

1 October 2015 27 Commercially Confidential

Kinect Sensor

Projects an array of dots onto the subject, their displacement from their

expected position is used to calculate depth.

Not an ideal sensor as the voxels are a bit big but it already exists.

1 October 2015 28 Commercially Confidential

Image Segmentation

Well understood technique for conventional images to detect connected regions.

Can also be applied to depth map to detect and separate objects.

We have developed this technique and applied to this problem.

1 October 2015 29 Commercially Confidential

Image segmentation

20 40 60 80 100 120 140 160 180 200

20

40

60

80

100

120

140

160

180

1 October 2015 30 Commercially Confidential

Bringing it all together

Videos from Cambridge Filmworks

1 October 2015 31 Commercially Confidential

What were the key challenges?

Machine vision – no standard algorithms to do this, needed some research and then

new code writing.

The gripper – getting the right balance between flexible but highly complex and

simple but limited in its application.

Bringing it all together – some standard components, but system integration and

getting a smooth outcome is quite time consuming.

1 October 2015 32 Commercially Confidential

Conclusions

This project shows an appropriate match between ‘what engineers can achieve’ and

‘what the customer needs’.

This is only possible by thinking at a system level, for example:

– Right compromise between software and mechanical complexity.

– Integrated vision, commercial arm and novel mechanics

New applications for robotics may not be a result of huge software and infrastructure

investments – more a result of novel algorithm and signal processing solutions.

1 October 2015

Cambridge UK

Registered No. 1036296 England

Cambridge Consultants is part of the Altran group, a global leader

in Innovation. www.Altran.com

www.CambridgeConsultants.com The contents of this presentation are commercially confidential and the

proprietary information of Cambridge Consultants © 2015 Cambridge

Consultants Ltd. All rights reserved.

Boston USA Singapore

1 October 2015 34 Commercially Confidential

Image Credits

Introduction: CC

The Robot Worker: http://www.gutenberg.org/files/38367/38367-h/images/fig319.png http://www.thwink.org/sustain/deadlock/E2_WealthOfNations.jpg http://australianroboticsreview.com/wp-

content/uploads/2015/08/Robot-Integration.jpg

But Robot’s Aren’t… https://www.youtube.com/watch?v=W7NjFGZ3tz8 http://www.cityam.com/article/how-intelligent-robots-stand-poised-change-all-our-lives-better

Robots in a wider setting https://upload.wikimedia.org/wikipedia/commons/c/c8/Sir_Killalot_spear.jpg http://www.mdpi.com/robotics/robotics-02-00054/article_deploy/html/images/robotics-02-

00054-g004-1024.png

Factory Robots: http://bobcad.com/wp-content/uploads/2014/05/Milwaukee-Matic-II-FirstToolChanger.jpg https://upload.wikimedia.org/wikipedia/commons/0/00/PaperTapes-5and8Hole.jpg

Self Driving Cars: http://cdn2.alphr.com/sites/alphr/files/styles/insert_main_wide_image/public/4/95//hud_nvidia_px.png?itok=N8roiw-U

Where are the places: http://theskooloflife.com/wordpress/wp-content/uploads/2009/05/maslows-hierarchy.gif http://spectrum.ieee.org/img/armar-kit-1387486756966.jpg

Answer: http://www.qlocal.co.uk/southport/news_list/Gangmasters_Abuse_Southport_Workers-50527792.htm http://www.earthcollective.net/wp-content/uploads/2012/03/Packhouse2.jpg

https://www.peaceriverpacking.com/harvesting.html

What does a robot need? http://abe-research.illinois.edu/faculty/grift/research/biosystemsautomation/agrobots/AgBoHiRes2.JPG

Picking a point http://extension.oregonstate.edu/umatilla/mf/sites/default/files/imagecache/preview_500/pictures/tns_packing_line.jpg

Existing machine vision http://www.microscan.com/en-us/technology/machinevisionsystems/AppsforAutomotive.aspx http://www.conceptauto.com.au/images/services/bc-2.jpg

http://www.automation.com/images/article/omron/MVWP3.jpg

Kinect Sensor: http://userpage.fu-berlin.de/~latotzky/wheelchair/wp-content/uploads/kinect1_cropped.png https://graphics.stanford.edu/~mdfisher/Images/KinectIR.png

Image Segmentation: http://cs.nyu.edu/~silberman/images/nyu_depth_v1_preprocess.jpeg

1 October 2015 35 Commercially Confidential

Notes

1 October 2015 36 Commercially Confidential

Notes #1

TITLE: “The next industrial revolution' - giving machines autonomy and the ability to make

decisions.”

S1: machines struggle with situations they haven’t been programmed to expect. Neural

networks / deep learning can cope to an extent but there’s a lengthy training time.

S2: Huge steps made (up to self driving cars) but they depend on massive infrastructure (and

who owns your data?)

C: This is about being cheaper than a human not faster / more reliable. (S.D. cars are about

overall system costs ie not owning a car)

Q? Which area spends a lot of money on people but is already ‘cost optimised’?

1 October 2015 37 Commercially Confidential

Definitions

IoT

Industry 4.0 – connecting machines, workpieces and systems: create an intelligent value chain.

Better status / health info to management. JIT maintenance, near-zero downtime.

IoS – cloud computing.

1 October 2015 38 Commercially Confidential

Why did we do Scrumping?

Robots repeatable not accurate, keeping them as CNC machines really.

Taken picking as a challenge, using new machine vision

Machine can now take decision of how to pick

– Which one

– Position and angle

– How hard to suck

– Has it got it?

Integrates maths, s/w and mechanics.