aibo vision

AIBO VISION

by

Utku TatlıdedeKemal Kaplan

AIBO ROBOT

• Specifications:– ERS-210– CPU clock speed of 192MHZ – 20 degrees of freedom– Temperature,Infrared Distance,

Acceleration, Pressure, Vibration Sensors

– CMOS Image Sensor– Miniature Microphones,

Miniature Speaker, LCD Display

• Dimensions and Weight:

– Size (WxHxL) 6.06" (W) x 10.47" (H) x 10.79" (L)

– Weight 3.3 lbs. (1.5kg)

• CCD camera • 16.8 million colors

(24 bit)• 176x144 pixel image • Field of view 57.6°

wide and 47.8° tall• Up to 25 fps• Stores information in

the YUV color space

AIBO VISION

• Color Segmentation-Representations-Algorithms

• Region Building and Merging- Region Growing - Edge detection

• Object Recognition- Classification - Template matching - Sanity check

• Bounding Box Creation

PROCESSING OUTLINE

• CIE-XYZ• RGB• CMY, CMYK• HSV, HSI,

HLS • YIQ• YUV,

YCbCr• LUT

COLOR REPRESENTATION

Color can be physically described as a spectrum which is the intensity of light at each wavelength.

• Radiance: Energy (W) from light source

• Luminance: Energy perceived by observer

• Brightness: Subjective descriptor

• Additive color space

• Three primaries: red, green, and blue

• Cannot always produce a color equivalent to any wavelength

RGB (Red, Green, Blue)

• Similar to HSV (Hue, Saturation, Value)

• Represents colors similarly how the human eye senses colors.

HSI (Hue, Saturation, Intensity)

• Similar to YIQ and YCbCr

• Used for the PAL and SECAM broadcast television system

• Amount of information needed to define a color is greatly reduced

YUV (Luminance, Chrominance)

CONVERSIONS

Y =  .299R + .587G + .114B U = -.147R - .289G + .437B V =  .615R - .515G - .100B H = cos-1([(R-B)+(R-G)]/2*[(R-G)2+(R-B)(G-B)]1/2)

S = 1 – 3[min(R,G,B)]/(R+G+B)

V = (R+G+B)/3

RGB TO YUV

RGB TO HSV

• Can we reduce the color space by using unsupervised dimension reduction techniques (like PCA)?

• Can we use different domains?

• For each object, find the most accurate subspaces of the color space to represent the object.

• YUV seems the most promising color representation for our real time applicaiton.

AIM OF COLOR SEGMENTATION

First label images, then use supervised pattern recognition techniques. Most common ones:

• C4.5• MLP• KNN

FINDING THE SUBSPACES

• Forms a decision tree for classification.• Uses the concept of information gain

(effective decrease in entropy).

C4.5

• The MLP network is suited to a wide range of applications such as pattern classification and recognition, interpolation, prediction and forecasting.

MLP (Multi-Layer Perceptron)

• KNN is a simple algorithm that stores all available examples and classifies new instances of the example language based on a similarity measure.

KNN (K-Nearest Neighbor)

• Condensed KNN: We can reduce the training set by removing the samples that introduce no extra information to the system.

CONDENSED KNN

• PCA is a mathematical procedure that converts a number of possibly correlated variables into a hopefully smaller number of uncorrelated variables called principal components.

PCA (Principal Component Analysis)

REGION BUILDING AND MERGING

• RLE encodes multiple appearances of the same value.

RLE (Run Length Encoding)

REGION GROWING

• This method depends on the satisfactory selection of a number of seed pixels.

• This method may be performed before color segmentation.

• Merging algorithms: in which neighboring regions are compared and merged if they are similar enough in some features.

• Splitting Algorithms: in which large non-uniform regions are broken up into smaller regions which is hopefully uniform.

REGION MERGING AND SPLITTING

OBJECT RECOGNITION

BALLBEACON

OPPONENT GOAL

• Already done by color segmentation.– Ball: The biggest blob with “ball color”,– Beacons: Two adjecentblobs with

beacon colors, etc.

• Unclassified blobs are discarded.• Each object is classified with a

certainty.

CLASSIFICATION

• Accomplished by using convolution or correlation.

• Only works for translation of the template.

• In case of rotation or size changes, it is ineffective.

• Also fails for partial views of objects.

TEMPLATE MATCHING

• A series of sanity check inspections are performed by the AIBO vision module to ensure the object classification is logically correct. – Ball cannot be above the goal,– Goals cannot be below the field,– There cannot be two balls, etc.

SANITY CHECK

• Requires affine transformations (translation, rotation, scaling)

• Required for calculating distance and position information

• The final products of the vision module are the bounding boxes of each visible object.

BOUNDING BOX CREATION

• Cerberus RoboCup 2002 Team Report• rUNSWift RoboCup 2002 Team Report• NUBots RoboCup 2002 Team Report• CMPack RoboCup 2002 Team Report• MACHINE LEARNING, Mitchell• MACHINE VISION, Jain, Kasturi, Schunk

REFERENCES

Any Question?

REFERENCES