spatial indexing for nn retrieval r-tree. r-trees a multi-way external memory tree index nodes and...

48
Spatial Indexing for NN retrieval R-tree

Post on 15-Jan-2016

217 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Spatial Indexing for NN retrieval

R-tree

Page 2: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same level Every node contains between m and M

entries The root node has at least 2 entries

(children) Extension of B+-tree to multiple

dimensions

Page 3: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Example

eg., w/ fanout 4: group nearby rectangles to parent MBRs; each group -> disk page

A

B

C

DE

FG

H

J

I

Page 4: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Example F=4

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4

F GD E

H I JA B C

Page 5: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Example F=4

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4

P1 P2 P3 P4

F GD E

H I JA B C

Page 6: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees - format of nodes

{(MBR; obj_ptr)} for leaf nodes

P1 P2 P3 P4

A B C

x-low; x-highy-low; y-high

...

objptr ...

Page 7: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees - format of nodes

{(MBR; node_ptr)} for non-leaf nodes

P1 P2 P3 P4

A B C

x-low; x-highy-low; y-high

...nodeptr ...

Page 8: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Search

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4

P1 P2 P3 P4

F GD E

H I JA B C

Page 9: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Search

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4

P1 P2 P3 P4

F GD E

H I JA B C

Page 10: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Search

Main points: every parent node completely covers its

‘children’ a child MBR may be covered by more than

one parent - it is stored under ONLY ONE of them. (ie., no need for dup. elim.)

a point query may follow multiple branches. everything works for any(?) dimensionality

Page 11: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Insertion

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4

P1 P2 P3 P4

F GD E

H I JA B CX

X

Insert X

Page 12: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Insertion

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4

P1 P2 P3 P4

F GD E

H I JA B CY

Insert Y

Page 13: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Insertion

Extend the parent MBR

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4

P1 P2 P3 P4

F GD E

H I JA B CY

Y

Page 14: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Insertion How to find the next node to insert

the new object? Using ChooseLeaf: Find the entry that

needs the least enlargement to include Y. Resolve ties using the area (smallest)

Other methods (later)

Page 15: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Insertion

If node is full then Split : ex. Insert w

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4

P1 P2 P3 P4

F GD E

H I JA B C

W

K

K

Page 16: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Insertion

If node is full then Split : ex. Insert w

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4

Q1 Q2

F G

D E

H I J

A BW

K

C K W

P5 P1 P5 P2 P3 P4

Q1Q2

Page 17: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Split

Split node P1: partition the MBRs into two groups.

A

B

CW

KP1

• (A1: plane sweep,

until 50% of rectangles)

• A2: ‘linear’ split

• A3: quadratic split

• A4: exponential split:

2M-1 choices

Page 18: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Split pick two rectangles as ‘seeds’; assign each rectangle ‘R’ to the ‘closest’ ‘seed’

seed1

seed2

R

Page 19: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Split pick two rectangles as ‘seeds’; assign each rectangle ‘R’ to the ‘closest’

‘seed’: ‘closest’: the smallest increase in area

seed1

seed2

R

Page 20: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Split How to pick Seeds:

Linear:Find the highest and lowest side in each dimension, normalize the separations, choose the pair with the greatest normalized separation

Quadratic: For each pair E1 and E2, calculate the rectangle J=MBR(E1, E2) and d= J-E1-E2. Choose the pair with the largest d

Page 21: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees:Insertion Use the ChooseLeaf to find the

leaf node to insert an entry E If leaf node is full, then Split,

otherwise insert there Propagate the split upwards, if

necessary Adjust parent nodes

Page 22: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-Trees:Deletion Find the leaf node that contains the

entry E Remove E from this node If underflow:

Eliminate the node by removing the node entries and the parent entry

Reinsert the orphaned (other entries) into the tree using Insert

Page 23: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees: Variations R+-tree: DO not allow overlapping, so

split the objects (similar to z-values) R*-tree: change the insertion, deletion

algorithms (minimize not only area but also perimeter, forced re-insertion )

Hilbert R-tree: use the Hilbert values to insert objects into the tree

Page 24: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-tree

1

2

3

4

5

6

78

910

11

12

13

1

2 3

Page 25: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees - Range search

pseudocode: check the root for each branch, if its MBR intersects the query rectangle apply range-search (or print out, if this is a leaf)

Page 26: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees - NN search

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4q

Page 27: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees - NN search Q: How? (find near neighbor; refine...)

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4q

Page 28: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees - NN search A1: depth-first search; then range query

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4q

Page 29: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees - NN search A1: depth-first search; then range query

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4q

Page 30: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees - NN search A1: depth-first search; then range query

A

B

C

DE

FG

H

I

J

P1

P2

P3

P4q

Page 31: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

R-trees - NN search: Branch and Bound

A2: [Roussopoulos+, sigmod95]: At each node, priority queue, with promising

MBRs, and their best and worst-case distance

main idea: Every face of any MBR contains at least one point of an actual spatial object!

Page 32: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

MBR face property MBR is a d-dimensional rectangle, which is the

minimal rectangle that fully encloses (bounds) an object (or a set of objects)

MBR f.p.: Every face of the MBR contains at least one point of some object in the database

Page 33: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Search improvement

Visit an MBR (node) only when necessary

How to do pruning? Using MINDIST and MINMAXDIST

Page 34: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

MINDIST MINDIST(P, R) is the minimum distance

between a point P and a rectangle R If the point is inside R, then MINDIST=0 If P is outside of R, MINDIST is the distance of P

to the closest point of R (one point of the perimeter)

Page 35: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

MINDIST computation

MINDIST(p,R) is the minimum distance between p and R with corner points l and u

the closest point in R is at least this distance away

ri = li if pi < li

= ui if pi > ui

= pi otherwise

pp

p

R

l

u

MINDIST = 0

d

iii rpRPMINDIST

1

2)(),(

l=(l1, l2, …, ld)

u=(u1, u2, …, ud)

),(),(, oPRPMINDISTRo

Page 36: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

MINMAXDIST MINMAXDIST(P,R): for each dimension, find the

closest face, compute the distance to the furthest point on this face and take the minimum of all these (d) distances

MINMAXDIST(P,R) is the smallest possible upper bound of distances from P to R

MINMAXDIST guarantees that there is at least one object in R with a distance to P smaller or equal to it. ),(),(, RPMINMAXDISToPRo

Page 37: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

MINDIST and MINMAXDIST

MINDIST(P, R) <= NN(P) <=MINMAXDIST(P,R)

R1

R2

R3 R4

MINDIST

MINMAXDIST

MINDISTMINMAXDIST

MINMAXDIST

MINDIST

Page 38: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Pruning in NN search Downward pruning: An MBR R is discarded if there

exists another R’ s.t. MINDIST(P,R)>MINMAXDIST(P,R’)

Downward pruning: An object O is discarded if there exists an R s.t. the Actual-Dist(P,O) > MINMAXDIST(P,R)

Upward pruning: An MBR R is discarded if an object O is found s.t. the MINDIST(P,R) > Actual-Dist(P,O)

Page 39: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Pruning 1 example

Downward pruning: An MBR R is discarded if there exists another R’ s.t. MINDIST(P,R)>MINMAXDIST(P,R’)

MINDIST

MINMAXDIST

R

R’

Page 40: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Pruning 2 example

Downward pruning: An object O is discarded if there exists an R s.t. the Actual-Dist(P,O) > MINMAXDIST(P,R)

Actual-Dist

MINMAXDIST

O

R

Page 41: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Pruning 3 example

Upward pruning: An MBR R is discarded if an object O is found s.t. the MINDIST(P,R) > Actual-Dist(P,O)

MINDIST

Actual-Dist

R

O

Page 42: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Ordering Distance MINDIST is an optimistic distance where

MINMAXDIST is a pessimistic one.

P

MINDIST

MINMAXDIST

Page 43: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

NN-search Algorithm1. Initialize the nearest distance as infinite distance2. Traverse the tree depth-first starting from the root. At

each Index node, sort all MBRs using an ordering metric and put them in an Active Branch List (ABL).

3. Apply pruning rules 1 and 2 to ABL4. Visit the MBRs from the ABL following the order until it is

empty5. If Leaf node, compute actual distances, compare with the

best NN so far, update if necessary.6. At the return from the recursion, use pruning rule 37. When the ABL is empty, the NN search returns.

Page 44: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

K-NN search Keep the sorted buffer of at most k current

nearest neighbors

Pruning is done using the k-th distance

Page 45: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Another NN search: Best-First

Global order [HS99] Maintain distance to all entries in a common

Priority Queue Use only MINDIST Repeat

Inspect the next MBR in the list Add the children to the list and reorder

Until all remaining MBRs can be pruned

Page 46: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Nearest Neighbor Search (NN) with R-Trees

Best-first (BF) algorihm:

E

20 4 6 8 10

2

4

6

8

10

x axis

y axis

b

E f

query point

omitted

1 E2e

d

c

a

h

g

E3

E5

E6

E4

E7

8

searchregion

contents

E9i

E 11 E 22Visit Root

E 137

follow E1 E 22

E 54E 55

E 83 E 96

E 83

Action Heap

follow E2 E 28

E 54E 55

E 83 E 96

follow E8

Report h and terminate

E 179

E 137E 54E 55

E 83 E 96E 179

Result

{empty}

{empty}

{empty}

{(h, 2 )}

a5

b13

c

18

d

13

e

13

f

10

h

2

g

13

E11

E22

E38

E45

E55

E69

E713

E82

Root

E917

i

10

E1E2

E4 E5E8

i 10E 54E 55

E 83 E 96E 137

g13

Page 47: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

HS algorithm

Initialize PQ (priority queue)InesrtQueue(PQ, Root)While not IsEmpty(PQ)

R= Dequeue(PQ)If R is an object

Report R and exit (done!)If R is a leaf page node

For each O in R, compute the Actual-Dists, InsertQueue(PQ, O)If R is an index node

For each MBR C, compute MINDIST, insert into PQ

Page 48: Spatial Indexing for NN retrieval R-tree. R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same

Best-First vs Branch and Bound

Best-First is the “optimal” algorithm in the sense that it visits all the necessary nodes and nothing more!

But needs to store a large Priority Queue in main memory. If PQ becomes large, we have thrashing…

BB uses small Lists for each node. Also uses MINMAXDIST to prune some entries