optimization and search in discrete spaces - own...

163
Optimization and search in discrete spaces Own investigation Dr. Arno Formella Departamento de Informática Universidad de Vigo 10/05/07 Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 1 / 163

Upload: phungkhanh

Post on 29-Apr-2018

224 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Optimization and search in discrete spacesOwn investigation

Dr. Arno Formella

Departamento de InformáticaUniversidad de Vigo

10/05/07

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 1 / 163

Page 2: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Three applications

approximate point set match in 2D and 3D

selection of node B sites in UMTS wireless networks

approximation of point sets with shapes

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 2 / 163

Page 3: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

First applicationpsm

approximate Point Set Match in 2D and 3D

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 3 / 163

Page 4: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

¿Dónde está Wally?

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 4 / 163

Page 5: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Motivationpsm

searching of patterns(relatively small sets of two– or three–dimensional points),within search spaces(relatively large point sets)

comparing point sets

key wordsgeometric pattern matching, structure comparison, point setmatching, structural alignment, object recognition

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 5 / 163

Page 6: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Joint work with

Thorsten Pöschel

some ideas from: Kristian Rother, Stefan Günther

Humboldt Universität—Charité Berlinhttp://www.charite.de/bioinf/people.html

Current project: superimpose (Raphael Bauer)

there will be other algorithms to compare with

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 6 / 163

Page 7: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Search of a substructure in a proteinsearch space

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 7 / 163

Page 8: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Search of a substructure in a proteinsearch pattern

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 8 / 163

Page 9: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

¿Dónde está Wally?

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 9 / 163

Page 10: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Informal problem description

given a search space and

a search pattern,

find the location within the spacewhich represents best the pattern

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 10 / 163

Page 11: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Extensions

find the best part of the patternwhich can be represented within the search space

allow certain types of deformation of the pattern

find similar parts within the same point set

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 11 / 163

Page 12: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Formal problem description

search space:S = {s0, s1, . . . , sn−1} ⊂ IRd , |S| = n

search pattern:P = {p0, p1, . . . , pk−1} ⊂ IRd , |P| = k ≤ n

dimension d = 2 or d = 3

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 12 / 163

Page 13: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Search and alignment

the aligning process can be separated in two partsfind the matching points in the pattern and the search spacefind the necessary transformation to move the pattern to its location

an approximate alignment must be qualified

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 13 / 163

Page 14: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Matching

a matching is a function that assigns to each point of the searchpattern a different point of the search space

µ : P −→ S injective, i.e.,

if pi 6= pj then µ(pi) 6= µ(pj)

let’s write: µ(pi) = s′i and µ(P) = S′

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 14 / 163

Page 15: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Transformations

transformations which maintain distances:translation, rotation and reflection

transformations which maintain angles:translation, rotation, reflection and scaling

deforming transformations:shearing, projection, and others (local deformations)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 15 / 163

Page 16: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Congruent and similar transformations

rigid motion transformation(euclidean transformation or congruent transformation)only translation and rotation

similar transformationrigid motion transformation with scaling

we may allow reflections as well (L–matches)

let T be a transformation (normally congruent)

we transform the pattern

let’s write: T (pi) = p′i and T (P) = P ′

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 16 / 163

Page 17: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Alignment

a matching µ together with a transformation T is an alignment(µ, T )

rigid motion transformation: congruent alignment

with scaling: similar alignment

with reflection: L–alignment

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 17 / 163

Page 18: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

One–dimensional example

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 18 / 163

Page 19: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Distance of an alignment

let (µ, T ) be an alignment of P in S

we can measure the distances between transformed points of thepattern and their partners in the search space

i.e., the distances

di = d(T (pi), µ(pi)) = d(p′i , s′i )

obviously, if di = 0 for all ithen the alignment is perfect

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 19 / 163

Page 20: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Examples of different distances of an alignment

root mean square distance (RMS)

d =

√1n

∑i

(p′i − s′i )2

average distance (AVG)

d =1n

∑i

|p′i − s′i |

maximum distance (MAX)

d = maxi|p′i − s′i |

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 20 / 163

Page 21: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Quality of an alignment

there are many interesting distance measures

a distance d has its value in [0,∞[

we use the quality Q of an alignment Q = 1/(1 + d)

(other possibility: Q = exp(−d))

hence: Q = 1 perfect alignment, Q ∈]0, 1]

and: Q < 1 approximate alignment

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 21 / 163

Page 22: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Formal problem description

given a search space S, and

given a search pattern P

given a distance measure

find an alignment (µ, T ) of P in Swith minimum distance d (or maximum quality Q)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 22 / 163

Page 23: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Perfect congruent alignments

congruent alignments in IR3 (Boxer 1999):

O(n2.5 4√

log∗ n + kn1.8(log∗ n)O(1)︸ ︷︷ ︸output

log n)

for small k the first term is dominant

log∗ n is smallest l such that

22...2}

l−veces≥ n log∗ n = 5 =⇒ n ≈ 265000

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 23 / 163

Page 24: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Perfect similar alignments

similar alignments in IR3 (Boxer 1999):

O(n3 + kn2.2︸ ︷︷ ︸output

log n)

searching approximate alignments and/or partial alignmentsis a much more complex problem

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 24 / 163

Page 25: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Perfect alignmentsideas

choose one triangle, e.g. (p0, p1, p2), of P

search for all congruent triangles in S(and their corresponding transformations)

verify the rest of the points of P(after having applied the transformation)

the run time is not proportional to n3 (in case of congruence)because we can enumerate the triangles of S in a sophisticatedmanner and there are not as many possibilities

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 25 / 163

Page 26: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Approximate alignments

as stated, we work in two stepswe search for adequate matchings µ(according to a certain tolerance)we calculate the optimal transformations T(according to a certain distance measure)

we select the best alignment(s)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 26 / 163

Page 27: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Optimal Transformation

let S′ = µ(P) be a matching

let d be a distance measure

we look for the optimal rigid motion transformation T ,(only translation and rotation), such that

d(T (P), µ(P)) = d(P ′, S′) is minimal

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 27 / 163

Page 28: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Root mean square distance

d =

√1n

∑i

d(p′i , s′i )2

=

√1n

∑i

(U · pi + t − s′i )2

U 3x3 rotation matrix, i.e., orthonormal

t translation vector

Objective: find U and t such that d is minimal

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 28 / 163

Page 29: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

A little bit of math

we observe: t and U are independent

with the partial derivative of d according t

∂d∂t

= 2 ·∑

i

(U · pi + t − s′i ) = 2U∑

i

pi + 2nt − 2∑

i

s′i

we obtain

t = −U1n

∑i

pi +1n

∑i

s′i

= −U · pc + s′c

where pc and s′c are the centroids of both sets

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 29 / 163

Page 30: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Optimal Rotation

with the above, d can be written as

d =

√1n

∑i

(U · pi + t − s′i )2

=

√1n

∑i

(U · (pi − pc)− (s′i − s′c))2

where U is a matrix with restrictions (has to be orthonormal)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 30 / 163

Page 31: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Extremal points of functions with restrictions

one converts the problem with restrictions

with the help of LAGRANGE multiplies into

a problem without restrictions

which exhibits the same extremal points

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 31 / 163

Page 32: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Let’s skip the details

basically, we calculate first and second derivative according to theentries uij of U

we search for the extremal points

KABSCH algorithm 1976, 1978

open source code at my home page

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 32 / 163

Page 33: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

KABSCH algorithm

let S be the matrix of rows containing the s′ilet P be the matrix of rows containing the pi

we compute R = S · P>

we set A = [a0 a1 a2] with ak being the eigenvectors of R>R

we compute B = [‖Ra0‖ ‖Ra1‖ ‖Ra2‖]and finally, we get U = B · A>

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 33 / 163

Page 34: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Introduction of scaling

let us introduce a scaling value σ ∈ IR

d =

√1n

∑i

(σU · (pi − pc)− (s′i − s′c))2

let p′′i = U · (pi − pc) be the translated and rotated point pi

let s′′i = s′i − s′c be the centralized point s′i

the solution for the optimal σ: σ =

∑i

〈s′′i , p′′i 〉∑i

〈p′′i , p′′i 〉

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 34 / 163

Page 35: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Different distance measures—different alignments

AVG

MAX

RMS

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 35 / 163

Page 36: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Optimal transformations for non–derivable distancemeasures

if the function for d is not derivable, e.g., the average

we use a gradient free optimization method(only with evaluations of the function)

recently developed iterative method that is guaranteed toconverge towards a local minimum

algorithm of RODRÍGUEZ/GARCÍA–PALOMARES (2002)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 36 / 163

Page 37: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Idea behind RODRIGUEZ/GARCIA–PALOMARES

let f (x) be the function to be minimized

we iterate contracting and expanding adequately parametershk > 0 and τ > 0 such that

f (x i+1) = f (x i ± hkdk ) ≤ f (x i)− τ2

where dk is a direction taken from a finite set of directions(which depends on the point x i )

with τ −→ 0, x i converges to local optimum(while there are no constraints)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 37 / 163

Page 38: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Rotation in quaternion space

a rotation U · p of the point p with the matrix U can be expressedas

q ? p̄ ? q−1 in quaternion space IH(HAMILTON formula, C∼ IR2, IH ∼ IR4)

where p̄ = (0, p) is the canonical quaternion of the point p

and q = (sin(ϕ/2), cos(ϕ/2)u) is the rotation quaternion(with u ∈ IR3 being the axis and ϕ the angle of rotation)

instead of U with 9 constraint variableswe have u and ϕ, i.e., 4 unconstraint variables

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 38 / 163

Page 39: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Matching Algorithms

1 maximal clique detection within the graph of compatible distances2 geometric hashing of the pattern3 distance geometry

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 39 / 163

Page 40: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Maximal clique detection

we generate a graph G = (V , E) (graph of compatible distances)

vertices vij ∈ V all pairs (pi , sj)

edges e = (vij , vkl) ∈ E , if d(pi , pk ) ≈ d(sj , sl)

search for maximum cliques in G

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 40 / 163

Page 41: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Properties of Clique Detection

the problem is NP–complete(however, we search only for cliques of size ≤ k )

fast algorithms need adjacency matrices

if n = |S| = 5000 and k = |P| = 100 we need 30 GByte(counting only one bit per edge)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 41 / 163

Page 42: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Geometric hashing

preprocessing of the search spacelet’s describe the two–dimensional case

we align each pair (si , sj)with si at the origin and sj in direction xwe insert some information for each other point sk ∈ Sin a hashtable defined on a grid over S

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 42 / 163

Page 43: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Example: geometric hashing

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 43 / 163

Page 44: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Searching with geometric hashing

we simulate an insertion of the points of P into the hashtable

but we count only the non–empty entries

many votes reveal candidates for partial alignments

e.g., if we encounter a pair (pi , pj) such thatfor each other point of the pattern there is a non–empty cell in thehashtablewe have found a perfect candidate

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 44 / 163

Page 45: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Properties of geometric hashing

grid size must be selected beforehand

preprocessing time O(nd+1)

searching time O(kd+1)

works only for rigid motion transformations

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 45 / 163

Page 46: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Distance geometry

we represent both sets S and P as distance graphs

the vertices of the graphs are the points of the sets

the edges of the graphs hold the distances between thecorresponding vertices

e.g., GP = (P, P × P) complete graph

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 46 / 163

Page 47: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

The ideas behind psm

we define adequate distance graphs GP and GS

we search for subgraphs G′S of GS that are congruent to the graph

GP

(allowing certain tolerances)

we optimally align GP with the subgraphs of G′S

we select the best one among all hits

we extend the search to work with subgraphs of GP as well

we select a best subgraph as final solution

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 47 / 163

Page 48: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

The four main steps of psm

construction of the graphswith: exploitation of locallity properties

search of subgraphswith: sophisticated backtracking

alignmentwith: minimization of cost functions

search of partial patternswith: reactive tabu search

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 48 / 163

Page 49: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Construction of the graphs

let us assume that the pattern P is small

we construct GP as the complete graph

we generate a dictionary D(ordered data structure)that contains all distances (intervals) between points in P

we consider an edge between two vertices in GSif the distance is present in the dictionary D

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 49 / 163

Page 50: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Construction of the dictionary

let dij = d(pi , pj) be the distance between two points of P

the dictionary will contain the interval

[ (1− ε) · dij , (1 + ε/(1− ε) · dij ] ∈ D

where 0 ≤ ε < 1 is an appropriate tolerance

the upper limit can be simplified to (1 + ε)(but we loose the symmetry)

we can join intervals in the dictionary D if they intersect

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 50 / 163

Page 51: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Construction of the graphs that way

GP GS

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 51 / 163

Page 52: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Fast construction of the graphs

we construct GP as a connected (and rigid) graphmantaining only the short edges

we order the points of S previously in a grid of size similar to thelargest of the intervals

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 52 / 163

Page 53: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Construction of the graphs that way

GP GS

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 53 / 163

Page 54: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Matching

let us assume (at the beginning) that GP is a complete graph

we order the points of GP according to any order e.g.(p0, . . . , pk−1)

we apply a backtracking algorithmthat tries to encounter for earch pi a partner si

following the established ordering

hence:

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 54 / 163

Page 55: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Backtracking for the search

let us assume that we already found a subgraph Gs0,...,si

where the graph Gp0,...,pi can be matchedwe look for candidates si+1 for the next point pi+1

that must be neighbors of the point si within GS

that must not be matched already andthat have similar distances to the sj (j ≤ i) as the pi+1 to the pj

(j ≤ i)

while there is a candidate we advance with i

if there are no more candidates for si+1, si cannot be a partner forpi neither (i.e.: backtracking)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 55 / 163

Page 56: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Termination of the algorithm

the algorithm informs each timea candidate for for pk−1 has been foundthe algorithm terminates when

there are no more candidates for p0 orthe first solution has been found

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 56 / 163

Page 57: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Optimizations of the basic algorithm

reduction of the edges in GP

implies: reduction of the edges in GS

good ordering of the pi

implies: reduction of the number of candidates

consideration of the type of point (e.g. element type of the atom)implies: reduction of the number of candidates

all heuristics imply:the backtracking advances faster

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 57 / 163

Page 58: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Search for partial alignments

find the subset of the points of the pattern that can be matchedbest to some points in the search space

NP–complete

there are |P(P)| = 2k possibilities to choose a subsetwe apply:

genetic algorithmreactive tabu search

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 58 / 163

Page 59: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Genetic Algorithm

maintain graph GS as complete graph

genome: sequence of bits indicating si a point belongs to theactual pattern or not

crossover: two point crossover

mutation: flip

selection: roulette wheel

cost function: distance and size of alignment

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 59 / 163

Page 60: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Termination of the GA

it is not that easy

once the first solution has been found

once a sufficently good solution has been found

after a certain number of iterations

once diversity of population is too low

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 60 / 163

Page 61: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Problems with GA

GS must be a complete graph¿You know a crossover operation for non–complete graphs?

more precisely:we need a crossover (and mutation) operation that maintains aspecific property of the graphs (e.g., connectivity, rigidness)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 61 / 163

Page 62: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Reactive Tabu Search

we start with an admissible solution

we search for possibilities to improve the current solution

if we can: we choose one randomlyif we cannot:

we search for possibilities to reduce the current solutionif we can: we again try improvementsif we cannot: we jump to another admissible solution

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 62 / 163

Page 63: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

The Tabu criterion

we avoid repetitive movements taking advantage of a memory thatstores intermediate solutions

i.e.: we mark certain movements as tabu for a certain number ofiterations

reactive means: we adapt the tabu period dynamically

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 63 / 163

Page 64: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Quality of a partial alignment

evaluation of the cost of a solution:number of aligned points plus quality of the alignment

remember: quality Q ∈]0, 1], but we will use Q ≥ threshold

hence, maximal quality: |P|+ 1

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 64 / 163

Page 65: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Reactive Tabu Search for psm

representation of the problem:sets of indices of the matched points

search for candidates to improve (add):(rigidly) connected neighbors within graph GS

search for candidates to reduce (drop):any point of the current solution that mantains the graph GSconnected (and rigid)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 65 / 163

Page 66: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

¿When do we terminate?

not that simple

once we found a sufficiently good solution

once we have run a certain number of iterations

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 66 / 163

Page 67: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Search for the largest common pattern

P S

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 67 / 163

Page 68: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Search for the largest common pattern

p/P p/S

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 68 / 163

Page 69: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Search for the largest common pattern

s/P s/S

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 69 / 163

Page 70: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Search for pattern with deformation

instead of the complete graph use a connected sparse graph

parts of the graph could be rigid

the graph may specify hinges or torsion axis

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 70 / 163

Page 71: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

psm

command line tool with configuration file

GUI

web–site to perform searches

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 71 / 163

Page 72: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Softwareold version

almost 13.000 specific lines of C++uses the libraries:

Leda (library of efficient data structures and algorithms)gsl (Gnu scientific library)GAlib (genetic algorithms, MIT)Gtkmm (graphical user interface)own libraries

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 72 / 163

Page 73: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Softwarenewer version

almost 11.000 specific lines of C++uses the libraries:

mtl (matrix template library)Gtkmm (graphical user interface)own libraries

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 73 / 163

Page 74: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

psm

psm

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 74 / 163

Page 75: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Possible extensions

enumerate more rigorously all locations(up to now we have concentrated on the best solution)

extend the properties of the graphs defining deformations of thepattern (e.g. torsion of parts, restriction of angles)

allow local tolerances (e.g. per edge)

improve the user interface

more applications

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 75 / 163

Page 76: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

optimUMTS

optimUMTS

Optimization of wireless UMTS networks

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 76 / 163

Page 77: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Joint work with

Fernando AguadoDepartamento de Teoría de la SeñalUniversidad de Vigo

Luis MendoUniversidad Politécnica de Madrid

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 77 / 163

Page 78: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Objectives

given a set of possible nodes B (base stations)

find optimal subset

to guarantee certain services (bandwidth)

to an estimated user distribution

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 78 / 163

Page 79: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Principal algorithm

LOAD

GENERATE

COMPUTEASSIGNMENT

DownlinkUplink

static user distributionpath lossUMTS settings

terrain datanode B locations

traffic description

best set B−nodes best set B−nodesassignment

GA HBB

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 79 / 163

Page 80: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Cartography of Madrid

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 80 / 163

Page 81: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Calculation of attenuation matrix α

simple logarithmic decay

L(m, k) = 100.1·(32.2+35.1·log(d(m,k)))

α(m, k) =Geff(m) ·Geff(k)

L(m, k)

simplied Xia model

mixed model: close—Xia, far—simple

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 81 / 163

Page 82: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Coverage around a node B

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 82 / 163

Page 83: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Traffic distribution

static distribution

grid of estimated user with activation percentage

polygons with Poisson process per service

uniform distibution everywhere or only on streets

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 83 / 163

Page 84: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Calculation of SIR

uplink SIR γUL:

γUL(m, k) = (Eb/N0)UL · b0(k)/B0

downlink SIR γDL:

γDL(m, k) = (Eb/N0)DL · b0(k)/B0

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 84 / 163

Page 85: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Calculation of noise

uplink noise NT (m) at transmitter m:

F (m) = 100.1·NF (m)

N(m) = kB · Tamb · B0 · F (m)

downlink noise NR(k) at receiver k :

F (k) = 100.1·NF (k)

N(k) = kB · Tamb · B0 · F (k)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 85 / 163

Page 86: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Calculation of power (Hanly and Mendo)

initial power P0(k) for receiver k :

t0(m, k) =γUL(m, k)

α(m, k)· N(m) P0(k) = min

m{t(m, k)}

power Pi(k) for receiver k in iteration i :

si = Pi−1 · α(m)

ti(m, k) =γUL(m, k)

α(m, k)· ((si − Pi−1(k) · α(m, k)) · a(k) + N(m))

Pi(k) = minm{ti(m, k)}

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 86 / 163

Page 87: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Termination criteria for iteration

∃k with P(k) > Pmax(k) =⇒ no assignment

i > Imax =⇒ no assignment

maxk

{Pi(k)

Pi−1(k),Pi−1(k)

Pi(k)

}≤ ∆ =⇒ assignment possible

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 87 / 163

Page 88: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Calculation of assignment

A(k) = m with t(m, k) = minm{t(m, k)}

P(k) = minm{t(m, k)}

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 88 / 163

Page 89: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Validation of assignment

uplink:

Pmin(k) ≤ P(k) ≤ Pmax(k)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 89 / 163

Page 90: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

preliminaries

downlink:

β(n, k , m) =

{ρ(m, k) if m = n1 otherwise

calculation of SSIR γ̃:

γ̃(m, k) =γDL(m, k)

1 + ρ(m, k) · γDL(m, k)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 90 / 163

Page 91: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

System to solve for transmitter power calculation

H(m, n) = δm,n −∑

k∈A−1(m)

α(n, k) · β(n, k , m) · γ̃(m, k)

α(m, k)

ν(m) = Pplt(m) +∑

k∈A−1(m)

γ̃(m, k) · N(k)

α(m, k)

H · T = ν

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 91 / 163

Page 92: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Validation of power restrictions

validation of maximum power of transmitter m:

0 < T (m) ≤ Tmax(m)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 92 / 163

Page 93: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Validation of power restrictions

calculation of downlink power of receiver k :

P(k) =γ̃(m, k)

α(A(k), k)

(M∑

m=1

α(m, k) · β(m, k , A(k)) · T (m) + N(k)

)

validation of transmitter maximum channel power:

P(k) ≤ Pchn

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 93 / 163

Page 94: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Optimization with heuristic backtracking

Observation:

if there is no assignment with certain m nodes B

then there is no assignment with less nodes B

hence

start with all nodes B

eliminate nodes B til optimum found with branch&bound

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 94 / 163

Page 95: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Termination with heuristic backtracking

stop if stagnation occurs(if within a subtree all minimum solutions are at the same depth)

finds optimum solution

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 95 / 163

Page 96: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Heuristics

ordering of nodes B plays an important role in finding fast goodsolutions

reorder nodes B for backtracking according to heuristics

for instance: eccentricity, random, number of initial connections,etc.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 96 / 163

Page 97: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Optimization with genetic algorithm (GA)

steady-state incremental evolution(in each iteration two new decendents are generated)

selection: roulette wheel

mutation: flip

crossover: two–point–cyclic

quality: number of nodes B plus power as tiebreak

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 97 / 163

Page 98: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Genom generation I

matrix representation of nodes B

exploiting locality properties

using allele: usable, unusable, used, fixed

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 98 / 163

Page 99: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Genom generation II

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 99 / 163

Page 100: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Genom generation III

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 100 / 163

Page 101: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Crossover I

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 101 / 163

Page 102: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Crossover II

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 102 / 163

Page 103: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Evolution

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 103 / 163

Page 104: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Assignment and heuristic branch result

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 104 / 163

Page 105: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

GA results with and without downlink

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 105 / 163

Page 106: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Software

almost 22.000 specific lines of C++uses the libreries:

Leda (library of efficient data structures and algorithms)GAlib (genetic algorithms, MIT)own libraries

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 106 / 163

Page 107: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Further research and implementations

use of MonteCarlo method

to find best subset of nodes B

for several user distributions

i.e., find best subset to satisfy different scenarios

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 107 / 163

Page 108: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaprox

Approximation of point sets with shapes

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 108 / 163

Page 109: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Informal problem description

given a set of points in the plane

construct a geometric figure interpolating the sample points

that reasonably capture the shape of the point set

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 109 / 163

Page 110: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Applications

pattern recognition

object definition in geographic information systems

CAD/CAM services

vectorization tasks

curve reconstruction in image analysis

single–computation pose estimation

geometric indexing into pictorial databases

shape tracking etc.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 110 / 163

Page 111: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Examplepoint set

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 111 / 163

Page 112: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Exampleinitial shapes

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 112 / 163

Page 113: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Exampleadapted shapes

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 113 / 163

Page 114: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Approximation task

Three steps:

clustering of the points to identify the individual parts of a set ofshapes,

generating of an initial guess of the individual shapes,

adapting the individual shapes to the underlying point setaccording to some distance metric.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 114 / 163

Page 115: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Brain storming

distances, metrics, optimization, local and globalminima, discrete–continuous, partially plain functions,multi-objetive optimization, local decisions,multi–scale, simplification, VORONOI–diagram,DELAUNAY–diagram, graph analysis, similaritydetection, clasification (with and without supervision),filtering

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 115 / 163

Page 116: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Polygonal approximationtype of clustering

Given a set of points of a plane curve,

construct a polygonal structure

interpolating the sample points

that reasonably captures the shape of the point set.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 116 / 163

Page 117: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Three approaches for polygonal approximation

α–shapes

crust

curve approximation

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 117 / 163

Page 118: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Alpha–shapesapplication

The algorithm is able to detect

the outer boundary of a set of points

which covers more or less evenly distributed the interior of ashape.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 118 / 163

Page 119: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Alpha–shapesalgorithm

Compute the DELAUNAY triangulation of the point set.

Eliminate all triangles of the resulting graph which have a radiuslarger than α times the minimum radius.

The final shape is given by the outer edges of the remaininggraph.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 119 / 163

Page 120: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

VORONOI–diagram

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 120 / 163

Page 121: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

VORONOI–diagram with more points

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 121 / 163

Page 122: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

DELAUNAY–diagram

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 122 / 163

Page 123: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Alpha–shapesExample

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 123 / 163

Page 124: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Alpha–shapesshort comings

Need of suitable alpha.

Alpha is constant over the entire point set.

Interior points needed.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 124 / 163

Page 125: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

CRUSTapplication

The algorithm is able to reconstruct

a curve

that is sampled sufficiently dense

especially smooth curves, i.e., possibly many component curveswithout branches, endpoints, or self–intersections.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 125 / 163

Page 126: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

CRUSTalgorithm

The crust

is the set of edges

selected from the Delaunay triangulation of the initial point set

extended by its Voronoi points

where both endpoints of the edges belong to the initial set.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 126 / 163

Page 127: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

CRUSTExample

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 127 / 163

Page 128: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

CRUSTExample

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 128 / 163

Page 129: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Curve reconstructionalgorithm

Computes the GABRIEL graph as a subgraph of the Delaunaygraph

(an edge between two input points belongs to the Gabriel graph ifa disk with this edge as diameter does not contain any other inputpoint).

Eliminate the edges which do not fulfill the local granularityproperty,

i.e., at each point only the two shortest edges are maintained.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 129 / 163

Page 130: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Curve reconstructionapplication

The method works well if there exists a regular interpolant,

that is, a polygonal closed curve such that the local granularity,

defined as minimum distance to an input point,

at each point of the curve is strictly smaller

than the local thickness at that point,

defined as the distance to the medial axis of the shape.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 130 / 163

Page 131: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Skeleton

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 131 / 163

Page 132: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Skeleton

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 132 / 163

Page 133: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Simplification problemobjective

Given polygonal chain (or polygon) P (with n vertices),

approximate P by another one Q whose vertices are a subset of kvertices of P.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 133 / 163

Page 134: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Simplification problemtwo variants

min–# problem: minimize the number of vertices of anapproximating polygonal chain (or closed polygon) with the errorwithin a given bound;

min–ε problem: minimize the error of an approximating polygonalchain (or closed polygon) consisting of a given number of vertices.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 134 / 163

Page 135: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Simplification problemsolutions

Both problems can be solved in optimal time O(n2).

There exist near–optimal algorithms for solving the min–#problem for the Euclidean distance which for practical problemsoutperform the optimal algorithms.

There exist a genetic algorithm to cope with the min–# problemwhich found near optimal solutions in the presented experiments.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 135 / 163

Page 136: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxshape description

A shape S is defined by a number of points and certain parameters:

line: a point L ∈ IR2 and an angle ϕ ∈ IR;

circumference: a center point C ∈ IR2 and a radius % ∈ IR;

set of circumferences: set of pairs of center points andcorresponding radii, i.e., (C,R) ⊂ IR2 × IR;

polyline or polygon: an ordered set of corner pointsQ = {Q1, Q2, . . . , Qk}, Qj ∈ IR2, j = 1, . . . , k , where the onlydifference between the two shapes is that for a polygon the lastand first corner are connected;

rounded box: a line segment defined by two points Q1, Q2 ∈ IR2,an aspect ratio α ∈ IR, and a corner radius % ∈ IR.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 136 / 163

Page 137: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxdistance functions: point—shape I

lineδL2(Pi ,S) =

∣∣∣Det(Pi − L, (cos ϕ, sin ϕ)T )∣∣∣

circumferenceδL2(Pi ,S) = |‖Pi − C‖ − %|

set of circumferences

δL2(Pi ,S) = min(Cj ,%j )∈(C,R)

∣∣‖Pi − Cj‖ − %j∣∣

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 137 / 163

Page 138: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxdistance functions: point—shape II

polyline or polygon, first we need the distance for a segmentQQj = Qj+1 −Qj :

δL2(Pi , QQj) =‖Pi −Qj‖ if (Pi −Qj)

T QQj < 0‖Pi −Qj+1‖ if (Pi −Qj+1)

T QQj > 0∣∣∣∣Det(QQj ,Pi−Qj )

‖QQj‖

∣∣∣∣ otherwise

and obtain for a polyline or polygon

δL2(Pi ,S) = minQj∈Q

δL2(Pi , QQj)

where for a polyline the index j runs from 1, . . . , k − 1 and for apolygon from 1, . . . , k with Qk+1 = Q1.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 138 / 163

Page 139: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxdistance functions: point—shape III

rounded box

δL2(Pi ,S) =

∣∣∣∣ minj=1,...,4

δL2(Pi , QQj)± %

∣∣∣∣where Q3 = Q2 + α Q>

12 and Q4 = Q1 + α Q>12 being Q>

12 the leftturned perpendicular vector to Q2 −Q1 of same length. +% istaken when the point Pi lies inside the rectangular box throughQ1, . . . , Q4, and −% when Pi lies outside.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 139 / 163

Page 140: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxdistance functions: non–euclidean

vertical distance to a line:

δV (Pi ,S) =∣∣∣P2

i − L2 − (P1i − L1) · sin ϕ/ cos ϕ

∣∣∣length of the segments could have an influence as a weight

δwL2(Pi , QQj) =∣∣∣Det(QQj , Pi −Qj)

∣∣∣

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 140 / 163

Page 141: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxdistance functions: point set—shape

RMS

δRMS,L2(P,S) =

√√√√1l

∑Pi∈P

δL2(Pi ,S)2

AVG

δAVG,L2(P,S) =1l

∑Pi∈P

δL2(Pi ,S)

MAXδMAX ,L2(P,S) = max

Pi∈PδL2(Pi ,S)

or δRMS,V (P,S) or δAVG,L1(P,S), etc

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 141 / 163

Page 142: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxalgorithm for local search

Algorithm of RODRÍGUEZ/GARCÍA–PALOMARES

derivative–free minimization method

proved convergence for either locally strictly differentiable ornon–smooth locally convex functions.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 142 / 163

Page 143: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxSubsidiary objective

Til now concentrated on the minimization of a single function.

However, in our optimization problem, it can happen that there isno change in the value of the distance function although the shapeis modified.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 143 / 163

Page 144: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxExample: Subsidiary objective

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 144 / 163

Page 145: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxSubsidiary objective: idea

Minimize perimeter as well.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 145 / 163

Page 146: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxSubsidiary objective: possible solution

Formulate a convex combination of all objectives,

i.e., optimize the single–objective function

f (x) = β1f1(x) + β2f2(x) + · · ·+ βl fl(x)

where the βj > 0, for j = 1, . . . , l , are strictly positive weights,

fj(·) : IRn −→ IR are the individual objective functions.

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 146 / 163

Page 147: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxSubsidiary objective: short coming of possible solution

Explores the Pareto front defined by the weights βj ,

it might be difficult to find weights such that the encounteredminimum is sufficiently close to the minimum considering only theprincipal objective f1(·).

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 147 / 163

Page 148: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxExample: convex combination as objective function

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 148 / 163

Page 149: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxSubsidiary objective: better solution

modify the comparison f (z) ≤ f (x)− τ2 (in the local searchalgorithm)

with the following iteratively defined comparison function for lobjectives:

[f1(z), . . . , fl(z)] ≤τ2 [f1(x), . . . , fl(x)] ⇐⇒∀j = 1, . . . , l : fj(z) ≤ fj(x)− τ2 or

(j < l and fj(z) ≤ fj(x) and fj+1(z) ≤ fj+1(x)− τ2)

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 149 / 163

Page 150: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxExample

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 150 / 163

Page 151: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

NP–Completeness and non–convexity of the objectivefunction

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 151 / 163

Page 152: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxExamples: wedge

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 152 / 163

Page 153: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxExamples: simple polygon

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 153 / 163

Page 154: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxExamples: convex hull

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 154 / 163

Page 155: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxExamples: circumferences

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 155 / 163

Page 156: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxExamples: influence of metrics

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 156 / 163

Page 157: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxExamples: completing shapes

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 157 / 163

Page 158: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxExamples: rounded rectangle

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 158 / 163

Page 159: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaproxExamples: complex polygon

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 159 / 163

Page 160: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

Software

more than 7.000 specific lines of C++uses the libraries:

ImageMagickGtkmm (graphical user interface)own libraries

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 160 / 163

Page 161: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

shaprox

shaprox

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 161 / 163

Page 162: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

¿Questions?

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 162 / 163

Page 163: Optimization and search in discrete spaces - Own investigationformella.webs.uvigo.es/doc/tc06/tc_oed.pdf · Optimization and search in discrete spaces ... we use the quality Q of

¡Many thanks!

Dr. Arno Formella (Universidad de Vigo) OED 10/05/07 163 / 163