2003/11/051 the temporal and topological characteristics of bgp path changes di-fa chang ramesh...

2003/11/05 1

The Temporal and Topological Characteristics of BGP Path Changes

Di-Fa Chang Ramesh Govindan John Heidemann

USC/Information Sciences Institute

Presented by Ram Sudhir Nandyala

February 3, 2005

2003/11/05 2

Goal: To understand the changesof inter-domain routing

?

2003/11/05 3

How to Know? Specifically,

How frequently routing events occur? Where do they take place? How many routes do they change?

However, we have only sparse monitoring infrastructure to answer these questions.

2003/11/05 4

Existing Public Monitoring Infrastructure

Looking glasses (provided by ISPs) Active probing per user request. Provide router-level routing information. The monitored space is small.

BGP vantage points (connected to RouteViews and RIPE RRC) Passive monitoring. Provide AS-level routing information. The monitored space is relatively broad.

We start with the analysis of RouteViews datasets.

2003/11/05 5

Challenges To establish the relation between

Path changes observed by vantage points and Routing events taking place in some AS peerings.

Constraints: The routing events are unknown. The routing messages don’t contain information

about the events. The propagation timing of routing messages is

unknown.

2003/11/05 6

Example: Network Topology

Origin ASfor prefix p1

Vantage Points

x

e

a b c

o

yd

2003/11/05 7

Example: Original Paths

Path x -> o: (x, a, b, c, o)Path y -> o: (y, b, c, o)

Vantage Points

x

e

a b c

o

yd Origin AS

for prefix p1

2003/11/05 8

Example: Routing Event

Path x -> o: (x, a, d, e, c, o)Path y -> o: (y, d, e, c, o)

Vantage Points

x

e

a b c

o

yd Origin AS

for prefix p1

2003/11/05 9

Example: Observed Path Changes

Vantage Points

x

e

a b c

o

yd

VP in AS-x observes Time: 0 Path: (x, a, b, c, o) initial path


2003/11/05 10


Vantage Points

x

e

a b c

o

yd

VP in AS-x observes Time: 0 Path: (x, a, b, c, o) Time: 10 Path:

withdrawn

initial path


2003/11/05 11


Vantage Points

x

e

a b c

o

yd


withdrawn Time: 35 Path: (x, a, d,

e, c, o)

initial path

converged path


2003/11/05 12

Vantage Points

x

e

a b c

o

yd


VP in AS-y observes Time: 0 Path: (y, b, c, o)initial path


2003/11/05 13


Vantage Points

x

e

a b c

o

yd

VP in AS-y observes Time: 0 Path: (y, b, c, o) Time: 11 Path: (y, d, b, c,

o)

initial path transient path


2003/11/05 14


Vantage Points

x

e

a b c

o

yd


o) Time: 30 Path: (y, d, e, c,

o)

initial path transient path

converged path


2003/11/05 15


Vantage Points

x

e

a b c

o

yd


withdrawn Time: 35 Path: (x, a, d,

e, c, o)


o) Time: 30 Path: (y, d, e, c,

o)


2003/11/05 16

Example: Original Paths

Path x -> o: (x, a, b, c)Path y -> o: (y, b, c)

Vantage Points

x

e

a b c

yd


2003/11/05 17

Example: Routing Event

Path x -> o: (x, a, d, e, c)Path y -> o: (y, d, e, c)

Vantage Points

x

e

a b c

yd


2003/11/05 18


Vantage Points

x

e

a b c

yd

VP in AS-x observes Time: 0 Path: (x, a, b, c) Time: 30 Path: (x, a, d,

e, c)

VP in AS-y observes Time: 0 Path: (y, b, c) Time: 11 Path: (y, d, b,

c) Time: 29 Path:

withdrawn Time: 50 Path: (y, d, e, c)


2003/11/05 19

Example: Observed Path Changes VP in AS-x observes

Time Prefix Path

10 p1 withdrawn

30 p2 (x, a, d, e, c)

35 p1 (x, a, d, e, c, o)

VP in AS-y observesTime Prefix Path

11 p1 (y, d, b, c, o)

11 p2 (y, d, b, c)

29 p2 withdrawn

30 p1 (y, d, e, c, o)

50 p2 (y, d, e, c)

2003/11/05 20

Example: Observed Path Changes VP in AS-x observes

Time Prefix Path

4 p3 (x, a)

10 p1 withdrawn

25 p3 withdrawn

25 p4 (x, a, d)

26 p3 (x, a)

27 p5 withdrawn

28 p6 (x, a, d)

30 p2 (x, a, d, e, c)

32 p7 (x, a, d, e)

33 p6 withdrawn

35 p1 (x, a, d, e, c, o)

42 p5 (x, a, b)

42 p8 (x, a, d)

45 p7 withdrawn

47 p8 (x, a, b, d)

50 p5 withdrawn


6 p8 withdrawn

8 p3 (y, b, a)

10 p4 withdrawn

11 p1 (y, d, b, c, o)

11 p2 (y, d, b, c)

25 p5 withdrawn

28 p6 (y, d)

29 p2 withdrawn

30 p1 (y, d, e, c, o)

31 p8 (y, b, d)

32 p3 (y, d, a)

48 p4 (y, d)

48 p7 withdrawn

49 p6 withdrawn

49 p5 (y, b)

50 p2 (y, d, e, c)

2003/11/05 21

Message Correlation Problem

How to find the routing messages triggered by the same events in the stream of BGP updates?

Difficulties: Number of transient path changes is unknown. Number of prefixes affected is unknown. Number of events happened is unknown. Number of observing vantage points is unknown.

2003/11/05 22

Peering Location Problem

How can we locate the candidate peerings where the event takes place?

2003/11/05 23



VP in AS-x observesTime Prefix Path

0 p1 (x, a, b, c, o)

10 p1 withdrawn

35 p1 (x, a, d, e, c, o)


0 p1 (y, b, c, o)

11 p1 (y, d, b, c, o)

30 p1 (y, d, e, c, o)

2003/11/05 24




0 p1 (x, a, b, c, o)

10 p1 withdrawn

35 p1 (x, a, d, e, c, o)


0 p1 (y, b, c, o)

11 p1 (y, d, b, c, o)

30 p1 (y, d, e, c, o)

x

e

a b c

o

yd

Example: (d, e) High local_pref Recover from previous

failure

2003/11/05 25

Which Peerings Can Cause The Path Change?

x

e

a b c

o

y d

x

e

a b c

o

y d

x

e

a b c

o

y d

x

e

a b c

o

y d

x

e

a b c

o

y d

x

e

a b c

o

y d

x

e

a b c

o

yd

x

e

a c

o

y d

b

2003/11/05 26

Peering Locating Problem (cont.)

Hint: The candidate peerings include those peerings that appears only in the original paths or the new paths.

2003/11/05 27

Peering Locating Problem (cont.)

Hint: The candidate peerings include those peerings that appears only in the original paths or the new paths.


0 p1 (x, a, b, c, o)

10 p1 withdrawn

35 p1 (x, a, d, e, c, o)


0 p1 (y, b, c, o)

11 p1 (y, d, b, c, o)

30 p1 (y, d, e, c, o)

x a b c

e

o

y d

2003/11/05 28

Methodology

Cluster the BGP updates triggered by the same routing event into one group based on: Temporal Relation: the receiving time. Topological Relation: prefix, peerings of the path.

Two-step clustering: Prefix-based: for message correlation problem. Peering-based: for peering location problem.

2003/11/05 29

Prefix-based Clusters A sequence of path advertisements for the same

prefix that are sent by the same vantage point and closely spaced in time.

2003/11/05 30



VP in AS-x observesTime Prefix Path4 p3 (x, a)10 p1

withdrawn25 p3

withdrawn25 p4 (x, a,

d)26 p3 (x, a)27 p5


d)30 p2 (x, a,

d, e, c)32 p7 (x, a,

d, e)33 p6


d, e, c, o)42 p5 (x, a,

b)42 p8 (x, a,

d)45 p7


b, d)50 p5

withdrawn

VP in AS-y observesTime Prefix Path6 p8

withdrawn8 p3 (y, b,

a)10 p4

withdrawn11 p1 (y, d,

b, c, o)11 p2 (y, d,

b, c)25 p5

withdrawn28 p6 (y, d)29 p2


e, c, o)31 p8 (y, b,

d)32 p3 (y, d,

a)48 p4 (y, d)48 p7

withdrawn49 p6

withdrawn49 p5 (y, b)50 p2 (y, d,

e, c)

2003/11/05 31




withdrawn25 p3


d)26 p3 (x, a)27 p5


d)30 p2 (x, a,

d, e, c)32 p7 (x, a,

d, e)33 p6


d, e, c, o)42 p5 (x, a,

b)42 p8 (x, a,

d)45 p7


b, d)50 p5

withdrawn



a)10 p4


b, c, o)11 p2 (y, d,

b, c)25 p5



e, c, o)31 p8 (y, b,

d)32 p3 (y, d,

a)48 p4 (y, d)48 p7

withdrawn49 p6


e, c)

<= T1

> T1

2003/11/05 32




withdrawn25 p3


d)26 p3 (x, a)27 p5


d)30 p2 (x, a,

d, e, c)32 p7 (x, a,

d, e)33 p6


d, e, c, o)42 p5 (x, a,

b)42 p8 (x, a,

d)45 p7


b, d)50 p5

withdrawn



a)10 p4


b, c, o)11 p2 (y, d,

b, c)25 p5



e, c, o)31 p8 (y, b,

d)32 p3 (y, d,

a)48 p4 (y, d)48 p7

withdrawn49 p6


e, c)

<= T1

> T1

converged path

2003/11/05 33




withdrawn25 p3


d)26 p3 (x, a)27 p5


d)30 p2 (x, a,

d, e, c)32 p7 (x, a,

d, e)33 p6


d, e, c, o)42 p5 (x, a,

b)42 p8 (x, a,

d)45 p7


b, d)50 p5

withdrawn



a)10 p4


b, c, o)11 p2 (y, d,

b, c)25 p5



e, c, o)31 p8 (y, b,

d)32 p3 (y, d,

a)48 p4 (y, d)48 p7

withdrawn49 p6


e, c)

10 (ab,bc,ad,de,ec)

cluster

2003/11/05 34




withdrawn25 p3


d)26 p3 (x, a)27 p5


d)30 p2 (x, a,

d, e, c)32 p7 (x, a,

d, e)33 p6


d, e, c, o)42 p5 (x, a,

b)42 p8 (x, a,

d)45 p7


b, d)50 p5

withdrawn



a)10 p4


b, c, o)11 p2 (y, d,

b, c)25 p5



e, c, o)31 p8 (y, b,

d)32 p3 (y, d,

a)48 p4 (y, d)48 p7

withdrawn49 p6


e, c)

10 (ab,bc,ad,de,ec)

cluster

2003/11/05 35




withdrawn25 p3


d)26 p3 (x, a)27 p5


d)30 p2 (x, a,

d, e, c)32 p7 (x, a,

d, e)33 p6


d, e, c, o)42 p5 (x, a,

b)42 p8 (x, a,

d)45 p7


b, d)50 p5

withdrawn



a)10 p4


b, c, o)11 p2 (y, d,

b, c)25 p5



e, c, o)31 p8 (y, b,

d)32 p3 (y, d,

a)48 p4 (y, d)48 p7

withdrawn49 p6


e, c)

4 (xa)10 (ab,bc,ad,

de,ec)25 (xa)27 (xa)28 (xa)30 (ab,bc,ad,

de,ec)32 (xa)42 (xa)

6 (bd)8 (ba)10 (yd)11 (yb,bc,yd,

de,ec)11 (yb,bc,yd,

de,ec)25 (yb)28 (yd)48 (yd)

2003/11/05 36

Peering-based Clusters A sequence of prefix-based clusters that are closely

spaced in time and have at least one candidate peerings.

4 (xa)10 (ab,bc,ad,

de,ec)25 (xa)27 (xa)28 (xa,ad)30 (ab,bc,ad,


6 (bd)8 (ba)10 (yd)11 (yb,bc,yd,

de,ec)11 (yb,bc,yd,

de,ec)25 (yb)28 (yd)48 (yd)

2003/11/05 37



4 (xa)10 (ab,bc,ad,

de,ec)25 (xa)27 (xa)28 (xa,ad)30 (ab,bc,ad,


6 (bd)8 (ba)10 (yd)11 (yb,bc,yd,

de,ec)11 (yb,bc,yd,

de,ec)25 (yb)28 (yd)48 (yd)

4 (xa)6 (bd)8 (ba)10 (ab,bc,ad,de,ec)11 (yb,bc,yd,de,ec)11 (yb,bc,yd.de,ec)25 (yb)25 (xa)27 (xa)28 (yd)28 (xa,ad)30 (ab,bc,ad,de,ec)32 (xa)42 (xa)48 (yd)

2003/11/05 38




<= T2

> T2

2003/11/05 39




10 (bc,de,ec)

2003/11/05 40




10 (bc,de,ec)

Event occurs at time-10.

3 candidate peerings

2003/11/05 41

Experimental Data

Dataset: BGP updates from RouteViews from July 2002 to June 2003. 31 vantage points in 24 ASes. 1.68 billion path advertisements. 62%: AS_PATH changes.

Experiments Prefix-based clustering: T1 = 60, 120, 240 seconds Peering-based clustering: T2 = 30, 60, 90 secondsResults are similar for T1 , T2 we studied.

2003/11/05 42

How Many Path Changes Per Event?

Prefix-based clustering: ~50% of clusters have only one path advertisement.MinRouteAdver rating limiting is effective.

Peering-based clustering: median number of path advertisements is 8-9.Not all vantage points see each event.

2003/11/05 43

What Is The Impact of Event?

There are ~4,000 peering-based clusters per hour. Median number of prefixes in a cluster is 2.

Most events have only local impact.But a few events can affect tens of thousands of

prefixes.

2003/11/05 44

Where Events Happen?

47% of clusters are caused by routing events in transit peerings. These clusters consist of at least 50% of all path advertisements.Question: Is Internet core more unstable than edges?

Near-far problem: vantage points receives more routing messages when events occur in near peerings than far peerings.

2003/11/05 45

Summary

Summary MinRouteAdver rate limiting mechanism is

effective. Most events have only local impact. Open question: does Internet core change more

frequently than edges?

Future work Parameter evaluation Validation

2003/11/05 46

Question?

2003/11/051 the temporal and topological characteristics of bgp path changes di-fa chang ramesh...

Documents

o path y o

c path y o

o time

routing event path x

o initial path origin

path changes vp

original paths path

oinitial path origin