Routing Policy

CS 6250Nick Feamster

Fall 2011

BGP Policies in ISP Networks

• Introduced as fairly simple path vector protocol

• Many incremental modifications added over time

• Many policies used by operators; can’t enumerate them all– Nevertheless, some patterns emerge

2

How is Policy Implemented?

• Preference: Which route will be chosen for each destination?– Adding/deleting/modifying route attributes

• Filtering: Eliminates certain routes from consideration– Can be done on inbound or outbound

• Tagging: Adding state to a route– Tagging with community attribute

3

4

Filtering and RankingsRanking: route selectionFiltering: route advertisement

Customer

Competitor

Primary

Backup

5

Internet Business Relationships(Simplified)

• Customer/Provider: One AS pays another for reachability to some set of destinations

• “Settlement-free” Peering: Bartering. Two ASes exchange routes with one another.

Provider

Peer

Customer

Preferences implemented with local preference manipulation

Destination

Pay to use

Get paid to use

Free to use

6

Rankings

• Routes from customers over routes from peers• Routes from peers over routes from providers

provider

peer

customer

Traffic Engineering

• Outbound traffic control: Control the way traffic leaves the network– Via local preference and IGP costs

• Inbound traffic control: Control the way traffic enters the network– Via AS prepending and MED

• Remote control: Control the preferences of a remote AS– Through the community attribute

7

8

Policy Interactions

1

2 3

1 3 0 1 0

3 2 0 3 0

2 1 0 2 0

0

Varadhan, Govindan, & Estrin, “Persistent Route Oscillations in Interdomain Routing”, 1996

9

Strawman: Global Policy Check

• Require each AS to publish its policies• Detect and resolve conflicts

Problems:

• ASes typically unwilling to reveal policies• Checking for convergence is NP-complete• Failures may still cause oscillations

10

Think Globally, Act Locally

• Key features of a good solution– Safety: guaranteed convergence– Expressiveness: allow diverse policies for each AS– Autonomy: do not require revelation/coordination– Backwards-compatibility: no changes to BGP

• Local restrictions on configuration semantics– Ranking– Filtering

11

Can BGP Be Made Stable?

• Permit only two business arrangements– Customer-provider– Peering

• Constrain both filtering and ranking based on these arrangements to guarantee safety

• Surprising result: these arrangements correspond to today’s (common) behavior

Gao & Rexford, “Stable Internet Routing without Global Coordination”, IEEE/ACM ToN, 2001

12

Relationship #1: Customer-ProviderFiltering

– Routes from customer: to everyone– Routes from provider: only to customers

providers

customer

From the customerTo other destinations

advertisements

traffic

From other destinationsTo the customer

customer

providers

13

Relationship #2: Peering

Filtering – Routes from peer: only to customers– No routes from other peers or providers

advertisements

traffic

customer customer

peer peer

How Do Business Relationships Appear in Policy?

• Local preference: influence the decision process to prefer customer routes

• Controlling route export: prevent neighbors from sending traffic to some destinations

• Defensive programming: prevent neighboring AS from influencing its choice of routes

14

Physical Interconnections

• Public peering (IXP): Across a layer-2 access technology (typically, port on “shared fabric”). – Larger number of smaller peers– Trial peering

• Private peering: Direct interconnection between only two networks.– Previously: Circuit between two facilities– Today: Typically occur at “carrier hotels”

15

16

The Art of Peering: The Peering Playbook

William B. Norton

Co-Founder & Chief Technical Liaison

Equinix, Inc.

NaMeX Member MeetingOctober 7, 2005 Rome, Italy<wbn@equinix.com>

Research “The Art of Peering”

• Follow up to the first three white papers.Q: When e-mail to peering@<ispdomain>.net generates no response,

what do Seasoned Peering Coordinators do?

• Smartest Peering Coordinator: “Tricks of the Trade”• 20 Tactics successfully used to obtain Peering where you otherwise

might not be able to.

Disclaimer: These are NOT recommended tactics…I am simply documenting what has been successfully used in the field to obtain peering.

P?

Graphical Notation of Tactics

BA

ISP Initiator ISP Target

A B

ISP A Customers

ISP BCustomers

Larger Circle=More Customer Prefixes Thicker Lines=More Traffic

P?=Peering Request w/Peering Coordinator

Peering Negotiation

T?=Transit RequestTo Sales Person

Transit Negotiations

APC BPC

P?

APC BS

T?

APC BPC

APC BS

T?

$

$

To Portray Peering Plays Pictorially…

Transit and Peering Sessions

A B

T

A B

P

T=Established Transit Session(Selling Access to entire Internet)Size indicates effective size of transportSupporting the session

P=Established Peering Session(Reciprocal Access to each others customers) Size indicates effective size of transportSupporting the session

A B

P

A B

T

$

$

Graphical Display of Routing Announcements

Represents “the rest of the Internet”

Traffic over Transit and Peering Sessions

A B

T

A B

P

Traffic showed as directed lines

Thickness of line indicates amount of Traffic in relevant direction

A B

P

A B

T $

$

Other VariationsP->T = Transition of RelationshipP | T = Either Peering or Transit apply = Traffic destined anywhere = Fictitious Traffic = Packet Loss ridden Traffic = Traffic destined to green network = Traffic destined to brown network

Other Graphical Symbols

PeeringPoint

Exchange Point, Telco Hotel

Tiedwith

Indicates two or moreElements tied withrelationship

Indicates a ordering:a sequence to be followedin the Peering Tactic

or

1)    The Direct Approach uses peering@<ispdomain>.net , phone calls, face to face meetings, or otherwise direct interactions with Peering Coordinators to establish peering.

P?

P?=Peering RequestTo Peering Coordinator(s)

Peering Negotiation

Leading to

Peering Session

APC BPC

P?

APC BPC

A B

P

{null}

-or-

{“No”,null}

2)    The Transit with Peering Migration tactic leverages an internal advocate to buy transit with a contractual migration to peering at a later time.

APC BS

T?

APC BS

T?

APC BS+BPC

T->P?

A B

T->P $

A B

PTransit Negotiations with Salesleads to Peering

(…if peering prerequisites be met…)

$

$

$

3)    The End Run Tactic minimizes the need for transit by enticing a direct relationship with the target ISP’s largest traffic volume customers.

APC B

T?/P?

T?/P?

$

6)    Paid Peering as a maneuver is positioned by some as a stepping stone to peering for those who don’t immediately meet the peering prerequisites.

A B

P

A B

P

$

PeeringPoint

7)    In the Partial Transit tactic, the routes learned at an exchange point are exchanged with the peer for a price slightly higher than transport costs.

$

A B

EG

DC F

H

I

J

KLM

Routing AnnouncementsForwarding all customer & Peering Pt Routes(almost peering – maybe costs less)Geographically

Remote Router

pt

8)    The Chicken tactic involves de-peering in order to make the other peer adjust the relationship.

A B

P

A B

P

Who will blink first?A<->B Traffic has to go somewhere

YX

T T$$

A B

T

or$

31

The Business Game and Depeering• Cooperative competition (brinksmanship)• Much more desirable to have your peer’s customers

– Much nicer to get paid for transit

• Peering “tiffs” are relatively common

31 Jul 2005: Level 3 Notifies Cogent of intent to disconnect.16 Aug 2005: Cogent begins massive sales effort andmentions a 15 Sept. expected depeering date.31 Aug 2005: Level 3 Notifies Cogent again of intent todisconnect (according to Level 3)5 Oct 2005 9:50 UTC: Level 3 disconnects Cogent. Masshysteria ensues up to, and including policymakers inWashington, D.C.7 Oct 2005: Level 3 reconnects Cogent

During the “outage”, Level 3 and Cogent’s singly homed customers could not reach each other. (~ 4% of the Internet’s prefixes were isolated from each other)

32

Depeering ContinuedResolution…

…but not before an attempt to steal customers!As of 5:30 am EDT, October 5th, Level(3) terminated peering withCogent without cause (as permitted under its peering agreement withCogent) even though both Cogent and Level(3) remained in fullcompliance with the previously existing interconnection agreement.Cogent has left the peering circuits open in the hope that Level(3)will change its mind and allow traffic to be exchanged between ournetworks. We are extending a special offering to single homed Level 3 customers.

Cogent will offer any Level 3 customer, who is single homed to theLevel 3 network on the date of this notice, one year of full Internettransit free of charge at the same bandwidth currently being suppliedby Level 3. Cogent will provide this connectivity in over 1,000locations throughout North America and Europe.

http://66.249.93.104/search?q=cache:NOS3HJhX9jcJ:www.merit.edu/mail.archives/nanog/msg11606.html+level3+network+status+cogent&hl=it

What tactic is this?

36

A B

L

G

P

T

T

T $

$

$

A forces trafficOver B’s transit

9)    In the Traffic Manipulation tactic, ISPs or content players force traffic along the network path that makes peering appear most cost effective.

APC BPC

L

G

P

T

T

T $

$

$

P?

1 MONTH LATERContact PC-We should Peer!

B hears A’s route‘for free’ through Peer L

CH ISPi.e.Yahoo!

A B

L

G

P

T

T

T $

$

$

$ $$

APC BPC

L

G

P

T

T

T

A B

L

G

P

T

T

T

A B

L

G

P

T

T

T

9b)    For Access Heavy Guys…In the Traffic Manipulation tactic, Access ISP a) stop announcing routes, orb) insert Target AS# into announcement to trigger BGP Loop Suppression to force traffic along the network path that makes peering appear most cost effective.

$

$

$

$

$

$

P?

1 MONTH LATERContact PC-We should Peer!

B hears A’s route‘for free’ through Peer L

A forces trafficOver B’s transit

Access ISPi.e. Verizon

10) The Bluff maneuver is simply overstating future traffic volumes or performance issues to make peering appear more attractive.

A B

P?

A B

L G

T

P

T$ $

FictitiousPerformanceProblems

Overstating Traffic Futures “You better peer with me now cause…Lots of transit fees coming otherwise!”

11) The Wide Scale Open Peering Policy as a tactic signals to the Peering Coordinator Community the willingness to peer and therefore increases the likelihood of being contacted for peering by other ISPs.

APC

P?

From the highest mountain“We will Peer with Anyone!”

To anyone who will listen!

12) The Massive Colo Build tactic seeks to meet the collocation prerequisites of as many ISPs as possible by building POPs into as many exchange points as possible.

A

IX

A

IX A

IX

A

IX

A

IX

A

IX

A

IX A

IX

A

IX

A

IX

A

IX

A

IX A

IX

A

IX

A

IX

“Meet us in 3 Time Zones”

PacificTimeZone

EasternTimeZone

M C

14) Friendship-based Peering leverages contacts in the industry to speed along and obtain peering where the process may not be in place for a peering.

APC

P? BPC

Forums to meet Peering CoordinatorsGPFNANOGAPRICOTRIPEIETF:

15) The Spam Peering Requests tactic is a specific case of the Wide Scale Open Peering tactic using the exchange point contact lists to initiate peering.

APC

P?IX Participants List

IX Participants List

IX Participants List

::

17) Purchasing Legacy Peering provides an immediate set of peering partners.

GA U

P

PA B

P

A

G U

P

P B

PA

A PurchasesG and P

A

Legacy (early Internet day) Peering

19) The False Peering Outage tactic involves deceiving an ill-equipped NOC into believing a non-existing peering session is down.

ANOC

BNOCPeering

Point

X

ANOC: Hey – Emergency! ANOC: Our Peering Session with you Went Down!BNOC: Strange. <looks on router> I don’t see it configured.ANOC: It was. Don’t make me escalate to <famous person>BNOC: Ah – I bet is was that last config run that trashed it.BNOC: Give me a few minutes to fix it on both ends.

20) The Leverage Broader Business Arrangement takes advantage of other aspects of the relationship between two companies to obtain peering in exchange for something else.

APC BS

P?

A B

P

A B

Other

APC BS

P?PeeringTied with“Other”+Fiber deal+Dial-in deal+Racks+Transport+Strategic deal:

Tiedwith

50

Additional Assumption: Hierarchy

Disallowed!

51

Safety: Proof Sketch

• System state: the current route at each AS

• Activation sequence: revisit some router’s selection based on those of neighboring ASes

52

Activation Sequence: Intuition

• Activation: emulates a message ordering– Activated router has received and processed all

messages corresponding to the system state

• “Fair” activation: all routers receive and process outstanding messages

53

Safety: Proof Sketch• State: the current route at each AS

• Activation sequence: revisit some router’s selection based on those of neighboring ASes

• Goal: find an activation sequence that leads to a stable state

• Safety: satisfied if that activation sequence is contained within any “fair” activation sequence

54

Proof, Step 1: Customer Routes

• Activate ASes from customer to provider– AS picks a customer route if one exists– Decision of one AS cannot cause an earlier AS to

change its mind

An AS picks a customer route when one exists

55

Proof, Step 2: Peer & Provider Routes

• Activate remaining ASes from provider to customer– Decision of one Step-2 AS cannot cause an earlier Step-

2 AS to change its mind– Decision of Step-2 AS cannot affect a Step-1 AS

AS picks a peer or provider route when no customer route is available