Introduction to RoboticsVision-based ranging and Optical Filters

CSCI 4830/7000September 27, 2010

Nikolaus Correll

Review: Sensing

• Important: sensors report data in their own coordinate frame

• Examples from the exercise– Accelerometer of Nao– Laser scanner

• Treat like forward kinematics

Laser Scanner

Today

• Perception using vision• Range information from Vision• Basic Image Processing• Why is object recognition hard?

• -> “Computer Vision” with Jane Mulligan

Range sensing

• Last week– Laser scanner (phase shift)– Infrared (path loss)– Ultrasound (time-of-flight)

• Today– Depth from focus– Depth from Stereo

Pin-Hole Camera

A. Efros

Pin-hole Model

Aperture

Thin Lens

Objects need to have the right distance to be in focus -> Depth-from-Focus method

Depth from Focus

• “in focus” + camera parameters

• = range

• How to test whether an image is “crisp” or “blurry”?

Testing for focus

Unit Step -> 2nd Derivative

Intuition: Images with high contrast have steep edges!

Convolution

• Calculate Laplacian / 2nd derivative by “convolving” image with 2D Kernel

*

Depth from Stereo

Distance between stereo pair known + distance in the image -> distance to object

Stereo Pairs

• Zero crossings of Laplacians of Gaussians– Gaussians: blurred image (suppresses noise)– Laplacians: edges

• Test how far similar edges are apart

Epipolar constraints are given by the geometry of the Stereo pair

Other example for Convolutions: Canny Edge Detector

15

1.

2.+3.

4. Trace along ridges (non-maximum suppression)

Exercise: Thresholds

1616

Screen shots by Gary Bradski, 2005

http://homepages.inf.ed.ac.uk/rbf/HIPR2/adpthrsh.htm

Exercise: Morphological Operations Examples

• Morphology - applying Min-Max. Filters and its combinations

Opening IoB= (IB)BDilatation IBErosion IBImage I

Closing I•B= (IB)B TopHat(I)= I - (IB) BlackHat(I)= (IB) - IGrad(I)= (IB)-(IB)

Why is Object Recognition Hard?The difference between seeing and perception.

Gary Bradski, 2009 19

What to do? Maybe we should try to find edges ….

Gary Bradski, 2005

20

• Depth discontinuity• Surface orientation

discontinuity• Reflectance

discontinuity (i.e., change in surface material properties)

• Illumination discontinuity (e.g., shadow)

Slide credit: Christopher Rasmussen

But, What’s an Edge?

To Deal With the Confusion, Your Brain has Rules...

That can be wrong

We even see invisible edges…

And surfaces …

We need to deal with 3D Geometry

24

Perception is ambiguous … depending on your point of view!

Graphic by Gary Bradski

And Lighting in 3D

Which square is darker?

Lighting is Ill-posed …Perception of surfaces depends on lighting assumptions

26Gary Bradski (c) 2008 26

Contrast

27

Which one is male and which one is female?

Illusion by: Richard Russell, Harvard University

Russell, R. (2009) A sex difference in facial pigmentation and its exaggeration by cosmetics. Perception, (38)1211-1219

Frequency

Color

http://briantobin.info/2009/06/lost-and-found-visual-illusion.html

Homework

• Read sections 4.2-5 (pages 145-180)• Questionnaire on CU Learn• Midterm: October 11 (during class)