14 advanced vision

7/29/2019 14 Advanced Vision

1/47

Robotic Systems (14)


2/47

Some interesting questions.

How do ants forage

How do birds flock

Hoe do we drive a car round a track without a driver

Can we use a bees vision system to land a helicopter


3/47

Current Perception Paradigm

Moving

Object

Static

Object

Smooth

Surfaces

Vehicle

Time


4/47

Information from images

Advanced robotics needs to extract as much information aspossible from images.

Perception

Additional position information

Motion


5/47

Height from a 2D image

Distances

gives height


6/47

Motion from an images

Image differencing

Simple, but limited

Optical flow More complex

Biologically inspired


7/47

Image differencing

Background Image

Difference


8/47

Typical results

Take background

Take image

Find difference

Isolate largest object

Locate centroid

Plot


9/47

Optical FlowDistant objects appear tomove slowly

Near objects appear

to move faster


10/47

Optical Flow

Optical flow is the distribution of apparent velocities ofmovement of brightness patterns in an image.

Optical flow can arise from relative motion of objects and

the viewer

Calculate the motion between two image frames.


11/47

Optical Flow

( )

( )

tI+Vy

yI+Vx

xI=0

tt

I+y

y

I+x

x

I+t,y,xI=

)t+t,y+y,x+x(I=t,y,xI

Note this equation cannot be solved unless other constraints are imposed

on the problem, this is theAperture Problem


12/47

Optical flow


13/47

Optical Flow

Take one stepforward

http://people.csail.mit.edu/lpk/mars/temizer_2001/Optical_Flow/

Arrows indicate

direction

and magnitude

of the motion


14/47

Direct Perception

Any motion is a combination of six basic movements

The flow-field for a movement is the sum of eachmovements component.

Relatively easy to decompose a complex motion into sixbasic movements

Egomotion - estimating a camera's motion relative to

rigidly placed objects in a scene.


15/47

Bees and optical flow

How Bees Exploit Optic Flow: Behavioural Experiments and Neural Models [and Discussion],

by Mandyam V. Srinivasan and R. L. Gregory

Philosophical Transactions: Biological Sciences.
http://images.google.co.uk/imgres?imgurl=http://www.hootingyard.org/wp-content/uploads/2008/07/bee-one.jpg&imgrefurl=http://hootingyard.org/archives/category/bees&usg=__iDutYXHNB6sjPvxD3phsC1nWe-0=&h=300&w=400&sz=10&hl=en&start=10&um=1&tbnid=AfnNSy_ntJJWxM:&tbnh=93&tbnw=124&prev=/images%3Fq%3Dbee%26hl%3Den%26rlz%3D1T4GGLD_enGB310GB310%26sa%3DN%26um%3D1


16/47



17/47



18/47



19/47

The landing problem

V

D=Time

Speed = V

D

Difficult to calculate in real time

No time for slow binocular vision

No time for stopping and peering
http://images.google.co.uk/imgres?imgurl=http://www.hootingyard.org/wp-content/uploads/2008/07/bee-one.jpg&imgrefurl=http://hootingyard.org/archives/category/bees&usg=__iDutYXHNB6sjPvxD3phsC1nWe-0=&h=300&w=400&sz=10&hl=en&start=10&um=1&tbnid=AfnNSy_ntJJWxM:&tbnh=93&tbnw=124&prev=/images%3Fq%3Dbee%26hl%3Den%26rlz%3D1T4GGLD_enGB310GB310%26sa%3DN%26um%3D1http://images.google.co.uk/imgres?imgurl=http://www.hootingyard.org/wp-content/uploads/2008/07/bee-one.jpg&imgrefurl=http://hootingyard.org/archives/category/bees&usg=__iDutYXHNB6sjPvxD3phsC1nWe-0=&h=300&w=400&sz=10&hl=en&start=10&um=1&tbnid=AfnNSy_ntJJWxM:&tbnh=93&tbnw=124&prev=/images%3Fq%3Dbee%26hl%3Den%26rlz%3D1T4GGLD_enGB310GB310%26sa%3DN%26um%3D1


20/47

However

During approach:

Image grows

Speed of edges increases

Time = Image size (degrees)

Velocity of edges (degrees per second)


21/47

Diving Gannets


22/47

Optical flow from a aerial platform

Autonomous Landing for Indoor Flying Robots Using Optic Flow, Green et al, 2003


23/47

Landing Control

Forward speed, Vf

Height

Descent

Speed (Vd)

Flight path

f

d1

V

VTan=

)()( th=tV

h

V=

f

f Hold constant


24/47

Landing rules

)(

)(

)()(

)()(

)()()(

)()(

o

o

ttB

of

ttB

o

fd

f

eth=tV

eth=th

Hence

tBV=dt

tdh=tV

th=tV


25/47

Typical results


26/47

Experimental System

Optic flow-based vision system for autonomous 3D localization and control of small aerial vehiclesKendoul, Fantonia, and Nonamib, IRJJ, 2009


27/47

Experimental System


28/47

Typical application

Concept for a plane to fly

within the Martian atmosphere


29/47

Translation vs Rotation

In translation close objects move faster

In rotation Objects at all distances move at the samespeed

Translatory motion can reveal 3D structure, but rotarymotion cannot.

Motion Parallax


30/47

Biological viewpoints

Saccades

Sudden movement of the eyes or heads

Crabs can move their eyes

Insects move their heads


31/47

Fiddler Crab


32/47

Eye movements

Increase field of view

Stabilise image

Estimate distances

Track objects

Scan objects

RoboticBiological Systems


33/47

Obstacle Avoidance


34/47

Objective

Flying autonomously

Two poles 10 m apart.

The rotor span of thehelicopter is 3.4 m


35/47

Test vehicle


36/47

Lidar scans

170m


37/47

Evidence Grid

Build a 3d image of the environment as we are considering

probability, transient image are filtered out


38/47

Obstacle avoidance

Uses the principle of attraction and repulsion.

The helicopter is attracted to the objective

The helicopter is repulsed from an obstacle

Flies along the resultant valley

Flying Fast and Low Among Obstacles

Sebastian Scherer, Sanjiv Singh, Lyle Chamberlain and Srikanth Saripalli, 2007, IEEE ICRA


39/47

Example


40/47

Reported results


41/47

Final thoughts

Commercial low cost systems

Vacuum cleaners

Lawn mowers

Perimeter boundary control

Electric fence

Physical obstacles


42/47

Agriculture

Reduce labour

Smaller tractors

More hours of usage

Cabs not required

Increased safety

Fixed paths, GPS based


43/47

True autonomous Driving

Reference paper

Stanley: The Robot that Won the DARPA Grand Challenge Journal of FieldRobotics 23(9), 661692 (2006)


44/47

Sensors and computers

GPS (four separate system)

Five laser range finders looking at the terrain out to 25m

Colour camera for long range perception of the road

Two radar units out to 200m

Six Pentium processor + gigabit Ethernet

Input/Output to the vehicle

Treat autonomous

navigation as a software problem.


45/47

Architecture


46/47

Deliberative and Reactive Control

Sense

Model

Plan

Act

Modify the world

Create maps

Discover new areas

Avoid collisions

Move around

ReactiveDeliberative


47/47

Processed camera image

14 advanced vision

Documents