an omni-directional vision system for localization and object detection in middle size robocup...
TRANSCRIPT
An Omni-directional vision system for localization and object detection in
middle size RoboCup
Iranica Scientia, Dec 2007
M.Jamzad, A.Hadjkhoda Bakhshi, V.S.Mirrokni
Dept. of Computer EngineeringSharif University of Technology
1
Contents
• Background• Introduction• Navigation system of a mobile robot• Omnidirectional Vision Systems
• Omnidirectional Mirror Design Geometric model for a hyperbolic mirror Camera calibration Calculating the parameters of hyperbolic mirror Features of the image in our hyperbolic mirror
2
Contents
• Flat mirror as a collision detection sensor• Object detection• Localization
Localization using angles from fixed points Localization using fixed lines
• Conclusion and Results
3
BackgroundOmni directional view, the advantages
4
BackgroundOmni directional view, the advantages
5
BackgroundOmni directional view, the advantages
6
BackgroundOmni directional view, Panorama Eye
7
BackgroundOmni directional view, the advantages
8
BackgroundOmni directional view, the advantages
9
Industrial applications, 6.2 mm Omni-view camera
BackgroundOmni directional view, the advantages
10
Introduction
• RoboCup Competitions Two teams of real or simulated robots play soccer in a field.
• RoboCup Leagues Middle Size League Small Size League Simulation Leagues Sony AIBO League
11
Middle Size Playing Field in 2000
12
13
Sharif CE Team omni-directional middle size Robots, in 2001
14
Middle size play field since 2002
Navigation System of a Mobile Robot
• Self-Localization Computing the position (Location & Orientation) of robot itself in the field.
• Object-Localization
Computing the location of other objects interacting with robot in the environment.
15
Robot Position in the Field16
Omnidirectional Vision System• Structure
Consists of a camera installed beneath a convex mirror.
Omnidirectional Vision System17
Geometric model for a hyperbolic mirror
18
19
An omni-directional middle size RobotThe Artisti Veneti Team, Italy, 2000The Artisti Veneti Team, Italy, 2000
E. Pagello, RoboCup: Distributed Planning and Sensoring in MRS
Heterogeneous Robots
Characteristics:• Chassis shaped for omnidirectional vision• Mirror profile designed for the robot’s task
Mirror
Camera
Sharif CE omni-mirrorFabricated in Sharif University, 2000
21
22
Sharif CE Team omni-directional middle size Robots
• Advantage• Vast Field of View (more information in one image).
• Disadvantages • Inadequate and Non-uniform Resolution
• More Blur Images
• Distorted Images
23
An Omnidirectional Image Viewed by a robot from Sharif CE team
24
Omnidirectional Mirror Design
• Requirements• Having adequate resolution for boundary sections
• Covering the largest area of the field
25
• Conical Mirror • Advantage: Adequate resolution in
boundary area of the
mirror.
Larger field of view
• Disadvantage: More distorted images
• Spherical Mirror • Advantage: Less distorted images
• Disadvantage: Inadequate resolution
in boundary area of
the mirror.
Kinds Of Mirrors
26
• Conical-Spherical Mirror • Advantage:
• Adequate resolution in boundary
area of the mirror
• Less distorted images
• Larger field of view
• Hyperbolic Mirror • Advantage:
• Adequate resolution in boundary area of the mirror
• Less distorted images
• Covering the entire field27
Kinds Of Mirrors
Sphericalmirror
Conicalmirror
Conical-Sphericalmirror
Image features in different mirrors
28
Geometric model for a hyperbolic mirror
29
The equations of hyperbolic mirror is as follows
If a light beam initiated from point P on the field, hits the mirror surface insuch a way that its extension passes through Om (the 1st focus of hyperbola),then this light beam will reflect in a direction passing through the 2nd focus of the hyperbola (point Oc).
Geometric model for a hyperbolic mirror
30
The equations of hyperbolic mirror is as follows
Therefore, one has to install a CCD camera in such a way that the focal point of camera is overlapped with the 2nd focus of the hyperbola to capture the image reflected on the Hyperbolic mirror.
Geometric model for a hyperbolic mirror
31
Geometric model for a hyperbolic mirror
32
Calculating the parameters of hyperbolic mirror
33
Calculating the parameters of hyperbolic mirror
34
Since all points are on the field, their Z=0. Now if one has the coordinate of a point P(r,θ) in the image plane, Then by solving equation (10), one can find the coordinate ofits corresponding point P(R,θ) on the field in the robot’s coordinate system.
(10)
Calculating the parameters of hyperbolic mirror
35
It s important to notice that the angle (θ) of point P on the field, with respect to robot’s coordinate system origin and x coordinate,
is the same as measured for its corresponding point p on image plane with respect to image plane center and its x coordinate.
Calculating the parameters of hyperbolic mirror
36
This condition is satisfied if
• The hyperbolic mirror plane is set parallel to the field.
• The CCD camera axis is installed perpendicular to the robot chassis.
• Center of the hyperbolic mirror is set along the camera axis.
Camera calibration
37
Camera Calibration
Camera Calibration Algorithm
1- Set four points P1, P2, P3, P4 in the field and their corresponding points on the image p1, p2, p3, p4. For these points we know their corresponding R and r.
2- Substitute these points in the mapping function assuming a1,a2, a3 and f are unknowns
38
Camera Calibration
Camera Calibration Algorithm
3- Solve the resulted set of 4 equations and find values for parameter a1, a2, a3 and f.
39
Camera Calibration
Camera Calibration Algorithm (Cont):
4- Use these values as an initial kernel to find more accurate values for the parameters as follows:
4-1 Locate some points Pi in the field and their corresponding points on the image pi. This process in done manually.
4-2 Pass each pair of these sample points to the mapping function and compute the error ei for each of them.
4-3 Find a set of values for the parameters in the neighborhood of initial values such that sum of the squared errors of the mapping function for sample points will be minimized.
40
Features of the Image using this Configuration
2. Independent of the robot position, the upper edges of the wall are projected on a constant circle in the image.
1. Independent of robot position, the image of entire field is projected on the mirror.
3. The upper part of the goals is projected in the image plane.
41
Surronding Circle
42
Why are the upper edges of the wall projected on a constant circle in the image?
43
h = wall height
Object detection algorithm
• Minimizing the search space by introducing the idea of jump points.
• Applying a fast object detection algorithm to find the
estimated boundary of an object placed under a jump point.
44
Jump points distribution
45
Boundary detection process
The process of finding surrounding sector of an object
If the jump point J is red, then it is located on the ball.
The ball will be estimated by a circle. As shown in above Fig.46
Initial guessCircle forthe ball boundary
Boundary detection process
How to find points A,B,C, D:Consider line JO as L1 and L2 as a line passing through J and perpendicular to L1.
Move from J along both lines L1 and L2 and check pixels for their colors.As long as the tested pixels are red, we are on the ball. As soon as a non red pixel is determined, we should have reached one of A,B,C or D points.
47
Initial guessCircle forthe ball boundary
Boundary correction process
48
J is a jump point showing a red point on the ball.A,B,C,D are points on the border of ball. The circle shows an estimate for the ball.K is the center of the sector.If d is larger than a threshold, then K is taken as a new jumppoint and the procedure is repeated.
How can a robot localize itself
49
How can a robot localize itself
• There are many methods for mobile robot localization.
• All methods need to know some information about some fixed positions on the environment.
• Using the coordinate of these fixed points a robot can find its own position with respect to these fixed points.
50
1- Localization using the angle from 3 fixed landmark points (P1, P2, P3).
Finding the location
Localization Algorithms
51
These 3 landmarks are selected from 4 intersection points of the two goals with the green field.
Localization Algorithms
52
با توجه به اینکه از هر سه نقطه فقط یک دایره میگذرد ، روبات در
تالقی دو دایره ، یعنی در محل قرار خواهد داشت.Pنقطه
بنابراین الزم است که این دو دایره پیدا شوند.
Localization Algorithms
53
با توجه به اینکه از هر سه نقطه فقط یک دایره میگذرد ، روبات در
تالقی دو دایره یعنی در نقطه محل P.قرار خواهد داشت
بنابراین الزم است که این دو دایره پیدا شوند.
DisadvantageWhen the robot is placed on the circle passing through four post goals, localization using angle from four is impossible.
Exceptional Circle
54
پیدا کردن زاویه روبات نسبت به مختصات جهانی
زمین بازی
Image processing algorithm to find relative angles of the post goals to the robot
دو نكته مهم در مورد تصاوير همه جهته اين روبات• وجود دارد كه پيدا كردن تير هاي دروازه ها آسان ميكند:
پرو جكشن يك خط عمود بر زمين ، بر روي صفحه تصوير 1.ازمركز تصوير عبور ميكند.
لبه باالي ديوار دور زمين ، همواره بر روي محيط يك دايره 1.ثابت پروجكت ميشود و اين دايره همواره دو دروازه را
بطور كامل در عرض قطع ميكند.
55
Finding the goal intervals in the image
56
Finding the goal intervals
1) For each point on the surrounding circle, determine if it is located on blue goal, yellow goal or the wall.
2) Find the longest consecutive blue points and the longest consecutive yellow points.
3) The start and the end points of the intervals are four points on the goal posts and their relative angles to the image center are the relative angles of goal posts to the robot.
57
2- Localization using fixed lines in the field Assuming we can detect the vertical and horizontal field lines that intersect each other.
58
Vertical field line
Algorithm for finding the equation of the wall lines in the robot coordination system
.پيدا كردن نقاط مرز زمين )نقاط واقع بر روي خط سفيد(•
مربوط به آينه هذلولوي بكار Mappingبا استفاده از تابع •برده شده بر روي روبات ، نقطه روي مرز )خط هاي سفيد(
ميكنيم.Mapرا بر روي سيستم مختصات رويات
با استفاده از الگوريتم تبديل هاف معادالت خطوط مرزي •زمين را بدست مي آوريم.
59
The Awards of Sharif CE Robotic team
1999-2001
60
The Awards of Sharif CE Robotic team 1999-2001
• 1st place of the 1999 robot soccer world cup in Stockholm,
Sweden
اولین تیم روبوکاپ ایران که برای اولین بار در مسابقات جهانی روبوکاپ شرکت کرد
20 دانشجو و اینجانب( برای کسب این موفقیت تقریبا 7تیم روبوکاپ ما )هزار نفر ساعت در یکسال کار کرد.
• 1st place of the 2000 robot soccer European world cup in Amsterdam, Netherland
اولین تیم روبوکاپ ایران که برای دومین بار، در مسابقات اروپائی روبوکاپ شرکت کرد
•
61
The Awards of Sharif CE Robotic team 1999-2001
• 3rd place of the 2000 robot soccer world cup in Melbourne, Australia
• The Best Engineering Challenge Award in RoboCup Symposium (A paper on robot vision system), Seattle, USA, 2001
62
Computer hardware and the processing speed
Platform:
AMD K6, 500MHz processorand 64MB of RAM.
O.S : MS DOS.
Speed:Object Detection : 41 frames / secondSelf Localization : 48 frames / secondTotal : 22 frames / second
63
64
65
66
67
Border Points and their corresponding points on the field
68
Algorithm for finding the equation of the wall lines in the robot coordination system
1) Finding the border points
2) Mapping the border points using the mapping function computed for hyperbolic mirror
3) Finding the equation of the lines by applying Hough transform to this set of points.
69
Image processing algorithm to find relative angles of thepost goals to the robot
2) The upper edges of the wall are always projected on a constant circle in the image (this circle always passes through both goals).
There are two interesting points in images that simplifies the process of finding the post goals in the image:
1) The reflection of a perpendicular line in the field is a straight line that passes through the image center.
70