object detection with superquadrics
DESCRIPTION
Object Detection with Superquadrics. Facoltà di Ingegneria. Presenter: Ilya Afanasyev. Trento, 2010. Superquadrics (SQ). - PowerPoint PPT PresentationTRANSCRIPT
Object Detection with Superquadrics
Presenter: Ilya Afanasyev
Facoltà di Ingegneria
Trento, 2010
14/12/2010 1/20
Superquadrics (SQ)
A superquadric surface can be defined by the 3D vector:
r ( )
a1cos ( )1cos ( )
2
a2cos ( )1sin ( )
2
a3sin ( )1
2
2
- are parameters of shape squareness;a1 a2 a3 - define the superquadric sizes.
1 2
14/12/2010 2/20
Superquadrics are a family of parametric shapes, which are used as primitives for shape representation and play the role of prototypical parts and can be further deformed and glued together into realistic looking models.
Vector r(η,ω) sweeps out a closed surface in space when η,ω change in the given intervals:
η
ω
x
y
z
r(η,ω)
Creation of SQ surface
η,ω – independent parameters (latitude and longitude angles) of vector r(η,ω) expressed in spherical coordinates.
2
2
r ( )
a1cos ( )1cos ( )
2
a2cos ( )1sin ( )
2
a3sin ( )1
a1
a2
a3
14/12/2010 3/20
Parametric equation of SQ
The parametric equation of a superquadric surface:
- are parameters of shape squareness;a1 a2 a3 - define the superquadric sizes.
1 2
f x y z( )xa1
22 y
a2
22
21
za3
21
14/12/2010 4/20
Superquadrics can model a large set of standard building blocks, like spheres, cylinders, parallelepipeds and shapes in between. Modeling capabilities of SQ can be enhanced by tapering, bending and making cavities.
x’
x
y
z
2nd way for creation of SQ surface
x,y – independent parameters (Cartesian coordinates of SQ), which are used to obtain z coordinate.
f x y z( )xa1
22 y
a2
22
21
za3
21
y’
z’
Use the parametric equation in Cartesian coordinates, considering
f(x,y,z) = 1
-a1 ≤ x ≤ a1
-a2 ≤ y ≤ a2
z=NaN
y=a2
x=a1
14/12/2010 5/20
Warning: complex numbers in SQ equation1. If ε1 or ε2 < 1 and cos or sin of angles ω or η < 0, then vector r(η,ω) has complex values. To escape them, it should be used absolute values of the vector components, taking into account the sign of sin or cos for further SQ surface reconstruction.
2
2
2. Analogically if x or y < 1 and ε1 > 1, the function f(x,y,z) willl have the complex values of z. To overcome it, use the f(x,y,z) in power of exponent ε1.
f x y z( )xa1
22 y
a2
22
21
za3
21
f(x,y,z)ε1 = 1
6/2014/12/2010
ε1 = 0.1 ε1 = 1 ε1 = 2
ε2 = 0.1
ε2 = 1
ε2 = 2
Samples of SQ, depending on ε1 or ε2
14/12/2010 7/20[1]
8
Reconstruction of complex object [2]
Applications with SQ
14/12/2010
Reconstruction of complex object [4]Reconstruction of complex object [3]
9
Reconstruction of multiple objects [5]
Applications with SQ
14/12/2010Reconstruction of multiple objects [5]
What are we trying to achieve? To build the robust algorithm of recovery of pallets surface by Superquadrics with the real 3D data from a rangefinder (possibly, without segmentation).
14/12/2010 10
1. Develop a robust algorithm of recovery of simulated SQ objects (excluding noise).
2. Test it with real 3D data.3. Build a superquadric model
of the europallet.4. Take 3D data of pallet from
rangefinder.5. Fit the superquadric model
of the europallet to real 3D data by the algorithm.
Idea of work with SQ simulation1. Create SQ with known parameters a1, a2, a3, ε1, ε2.2. Get representation of SQ in points (like “Cloud of points”).3. Rotate and translate SQ according to “viewpoint of interest”.4. Get the silhouette of SQ (visible part of SQ from viewpoint).5. Add the noise to SQ silhouette to obtain more realistic model.6. Detect the orientation and location of SQ, using the known model of SQ and detection
algorithm (based on solving a system of nonlinear equation).
Azimuth, Elevation, Distance ???
1. 2. 3.
4.
5.6.
11/2014/12/2010
Rotation and translation of SQ
Elevation
Azimuth
x
y
z
zW
xW
yW
T – transformation matrix.n – amounts of points in SQ surface.
SQ – coordinates of points of SQ surface. Pw – coordinates of points of rotated SQ surface.
xW, yW, zW – world system of coordinates (with center in viewpoint).
Warning: Standard MATLAB function: viewmtx is useless for rotation and translation purposes.
12/2014/12/2010
13
Rotation and translation of SQ
Elevation
Azimuth
x
y
z
zW
xW
yW
px
az,el,px,py,pz,x,y,z – are given; xW,yW,zW – should be found.14/12/2010
T SQ Pw
Getting the silhouette of SQ 1) Calculate x’,y’,z’ (R’,Az’,El’) coordinates of every point of line laid between center of world coordinate system and point of interest from rotated SQ. yw
xw
p2
p’
p”
Az’
El’
zw
p’ – lays outside SQ;p’’ – lays inside SQ.
(x’; y’)
p1
14/2014/12/2010
15
Getting the silhouette of SQ 2) To recover the real x,y,z coordinates of every point of rotated SQ (rotate and translate it back):
T – transformation matrix;SQ – non-rotated point of SQ silhouette;P’w – rotated point of SQ silhouette.
Elevation
Azimuth
x
y
z
zW
xW
yW
Silhouette(visible part of SQ from viewpoint)
P(x;y;z)
14/12/2010
Getting the silhouette of SQ
- the point is on the surface of SQ.
- the point is outside SQ.
- the point is inside SQ.
3) Calculation of inside-outside function f(x,y,z)ε1.
4) If all points of a line from the “SQ point of interest” to the world coordinate center lay outside – this point of SQ is visible, otherwise – invisible.
x
y
z
zW
xW
yW
Silhouette
p1
p2
p1 – is the point of SQ silhouette; p2 – is not.
16/2014/12/2010
Adding the noise to SQ silhouette
2) Also the algorithm should process the SQ surface points, which cab be slightly deviated from real positions of SQ surface. We can create the deviation of mean values of z-coordinate for points of SQ surface according to law of normal distribution for small standard deviation (for example, σ=0.5).
To build the robust algorithm of recovery of SQ surface it is necessary to take into account the noise, which can exist in the real data (from a rangefinder).
1) The algorithm should be robust to outlets. To simulate the outlets, we can add to SQ silhouette some points obtained by uniform distribution.
silhouette
Noised silhouette
outliers
17/2014/12/2010
Algorithm of Recovery of SQ surface from silhouetteTo detect az,el from silhouette of rotated SQ, the given points should rotated and translated back to SQ coordinate system.
T – transformation matrix.SQ – non-rotated points of SQ silhouette.P’w – rotated points of SQ silhouette.
Elevation
Azimuth
x
y
zzW
xW
yW
Silhouette
P(x’;y’;z’)
T SQ P’w
ε1, ε2,a1-a3, xW,yW,zW – are given; Az,El,ω,η,px,py,pz – should be found. 1814/12/2010
Algorithm of Recovery of SQ surface from silhouetteTo solve this system of nonlinear equations, when silhouette consists n ≥10 points, use MATLAB fsolve function (from Optimization toolbox) with Levenverg-Marquardt method.
Future improvement of the algorithm deal with adding the new variable: px, using the parameter dist for initial estimates x0.
x
y
zzW
xWyW
Silhouette
T SQ P’w
py=0;pz=0.
ε1, ε2,a1-a3,py,pz,xW,yW,zW – are given; Az,El,ω,η, px – should be found.
px
dist
Variables
1914/12/2010
Algorithm of Recovery of SQ surface from silhouette
2014/12/2010
ElevationAzimuthxs
z1s
P1
xw’
yw’
zw’
ys
zs
ω
η
xw
yw
zw
zcw
ycw
xcw
x1w
y1w
z1w
Links1. Solina F. and Bajcsy R. Recovery of parametric models from
range images: The case for superquadrics with global deformations. // IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-12(2):131--147, 1990.
2. Chella A. and Pirrone R. A Neural Architecture for Segmentation and Modeling of Range Data. // 10 pages.
3. Leonardis A., Jaklic A., and Solina F. Superquadrics for Segmenting and Modeling Range Data. // IEEE Transactions On Pattern Analysis And Machine Intelligence, vol. 19, no. 11, 1997.
4. Bhabhrawala T., Krovi V., Mendel F. and Govindaraju V. Extended Superquadrics. // Technical Report. New York, 2007. 93 pages.
5. Jaklic Ales, Leonardis Ales, Solina Franc. Segmentation and Recovery of Superquadrics. // Computational imaging and vision 20, Kluwer, Dordrecht, 2000.
14/12/2010 21/20
Grazie per attenzione!!