csci5221: inter-domain routing and bgp 1 inter-domain routing: bgp, routing policies, etc. overview...

CSci5221: Inter-Domain Routing and BGP

1

Inter-Domain Routing: BGP, Routing Policies, etc.

• Overview of BGP – Network Domains and Autonomous Systems (ASes)– Internet Interconnection Structure and BGP– Basic BGP Features– BGP Path Selection Criteria

• Internet Settlement Models– AS Relationships– BGP Policies

Readings: Do the required readings


2

Routing in the Internet• The Global Internet consists of Autonomous

Systems (AS) interconnected with each other:– Stub AS: small corporation: one connection to other AS’s– Multihomed AS: large corporation (no transit): multiple

connections to other AS’s– Transit AS: provider, hooking many AS’s together

• Two-level routing: – Intra-AS: administrator responsible for choice of

routing algorithm within network– Inter-AS: unique standard for inter-AS routing: BGP


3

Internet Architecture

LANs

International lines

ISP ISPcompany university

national network

regionalnetwork

NAPInternic

on-line services

companyaccess via

modem

Internet: “networks of networks”!


4

: Routing session

routes Control plane:exchange routes

Internet routing

rusty.cs.berkeley.eduIP=169.229.62.116

Prefix=169.229.0.0/16

www.cnn.comIP=64.236.16.52

Prefix=64.236.16.0/20

Internet

IP traffic

Data plane:forward traffic

Fail over to alternate route


5

More than 20,000 ASes today

Berkeley

Internet

CNN

Calren

Level3

GNN

IP traffic

Qwest Sprint UUnet

University company

AT&T

business

ISP ISP ISP

ISP ISP ISP ISP

ISP

AutonomousSystem

Berkeley

Calren

Level3 Qwest Sprint UUnet

University company

AT&T

business

ISP ISP ISP

ISP ISP ISP ISP

ISP

Berkeley

Calren

Level3 Qwest Sprint UUnet

University company

AT&T

business

ISP ISP ISP

ISP ISP ISP ISP

ISP


6

ASs & AS Numbers (ASNs)

64512 through 65535 are “private”

• Genuity: 1 • MIT: 3• Harvard: 11• Yale: 29 • UCLA: 52• AT&T: 7018, 5075, …, 6341, … • UUNET: 701, 702, 284, 12199, …• Sprint: 1239, 1240, 6211, 6242, …• …

ASNs represent units of routing policy

Currently over 14,000 in use.


7

Having Internet Connectivity• To have complete Internet connectivity

you must be able to reach all destinations on the net.

• Your packets have to get delivered to every destination. This is easy (default routes).

• Packets from everywhere else have to “find you”. This is done by having your ISP(s) advertise routes for you.


8

• To allow networks (ASs) to tell other networks (ASs) about “routes” (parts of the IP address space) that they are “responsible” for and how to reach them– Using “route advertisements”, or “promises” - also called

“NLRI” or “network-layer reachability information”– “Path-vector” routing protocol

• Policy-based: allow ISPs to richly express their routing policy, both in selecting outbound paths and in announcing internal routes – keep this in mind as we progress!

• Relatively “simple” protocol, but configuration is complex and the entire world can see, and be impacted by, your mistakes

BGP: The Glue of Internet


9

Network Interconnections

• Exchange Point– Layer 2 or Layer 3

• Private Circuit– May be provided by a third

party


10

U of Minnesota Neighborhood

AS 1Genuity

AS 57 UMN GigaPoP

AS 7018 AT&T

AS 3908SuperNet (Qwest)

AS 217 UMN

AS 1998 State of Minnesota

128.101.0.0/16


11

BGP: Some BasicsBGP: Some Basics• BGP exchanges routes between ASs.• When routes are exchanged, ASNs are stamped on

the routes “on the way out” – adding one “AS hop” per network traversed -> AS path

– no concept of pipe size, internal router hop-count, congestion -> in some sense BGP treats all ASs the same

• Routes are exchanged over “peering sessions”, which run on top of TCP– The routes are “objects”, or “bags” of “attributes”

• BGP is actually two protocols – iBGP, designed for “internal” route exchange– eBGP, designed for “external” route exchange

• 1995: BGP-4 [RFC 1771] – Support for Classless Interdomain Routing (CIDR)


12

BGP (Peering) SessionsBGP (Peering) Sessions • BGP session set up over TCP

– When session set up, both sides flood the other end with all of their best BGP routes

– Over time, only incremental updates are exchanged– If session dies, all associated routes must be withdrawn

• BGP peers (neighbors) must be specified explicitly

• BGP session set-up: Cisco ExampleRouter A in AS 1 router bgp 1 neighbor 129.213.1.1 remote-as 2Router B in AS 2 router bgp 1 neighbor 129.213.1.2 remote-as 1


13

BGP messages

• OPEN: set up a peering session• UPDATE: announce new routes or

withdraw previously announced routes • NOTIFICATION: shut down a peering

session• KEEPALIVE: confirm active connection at

regular interval


14

BGP Operations (Simplified)BGP Operations (Simplified)

Establish session on TCP port 179

Exchange all

active routes

Exchange incremental updates

AS1

AS2

While connection is ALIVE exchangeroute UPDATE messages

BGP session

router A129.213.1.2

router B129.213.1.1


15

Establish BGP session

12.10.0.1 12.10.0.2

Establish neighboring session between 12.10.0.1 and 12.10.0.2

Prefix Next hop12.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1

Prefix Next hop135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.1

TCP 179


16

Exchange all candidate routes

12.10.0.1 12.10.0.2

Prefix Next hop12.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.1

Prefix Next hop135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.112.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1

12.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1

135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.1


17

Send incremental updates

12.10.0.1 12.10.0.2

Prefix Next hop12.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.1

Prefix Next hop135.120.0.0/24 10.128.0.168.35.0.0/16 10.192.1.112.70.0.0/24 10.20.0.112.9.0.0/16 10.20.1.1

Withdraw 12.9.0.0/16


18

BGP: Net Prefixes, ASNs and Route BGP: Net Prefixes, ASNs and Route AdvertisementsAdvertisements

AS 4969

AS 5000

AS 6461

AS 701

AS 12001BGP route advertisement: Net prefix: 207.8.128.0/17 AS path: 4969 6461


19

BGP Route AdvertisementBGP Route Advertisement • Think of a BGP route as a “promise”

– If I advertise 207.8.128.0/17, I promise that if you deliver traffic destined to any IP address within 207.8.128.0/17 to me, I know how to deliver it (at least as well as anyone else)

• By making sure these routes, or “promises”, are heard by all ASes, your provider ensures a return path for all of your packets– Sending packets out is easier than getting them

back.– Sending routes out causes IP traffic to come in


20

Internal vs. external BGP

Internet I-BGP

E-BGP

AS A

AS B

AS C

E-BGPupdate

I-BGPupdate

I-BG

Pup

date


21

EBGP vs. IBGP SessionsEBGP vs. IBGP Sessions • EBGP: between (usually directly-connected) routers in

different ASs• IBGP: between (BGP-speaking) routers in same AS• Different (operational) rules and polices apply!

AS 7007XP

AS 1239

AS 6079

AS 701

AS 4006


22

iBGPiBGP

AS 3847

• IBGP speakers are (usually) fully meshed: why?• IBGP session set up: Router A in AS 3847

router bgp 3847 neighbor 129.213.1.1 remote-as 3847 neighbor 128.28.10.2 remote-as 3847

Router B in AS 3947 router bgp 3847 neigbhor 129.213.1.2 remote-as 3847 neighbor 127.101.1.1 remote-as 3847

Router C in AS 3947 router bgp 3847 neigbhor 128.28.10.1 remote-as 3847 neigbhor 127.101.1.2 remote-as 3847

B

A

c


23

eBGP vs. iBGP eBGP vs. iBGP eBGP Rules:eBGP Rules:• By default, only talks to directly-connected

router.• Sends the one best BGP route for each

destination.• Sends all of the important “attributes”; omits

the “local preference” attribute.• Adds (prepends) the speaker’s ASN to the “AS-

Path” attribute.• Usually rewrites the “next-hop” attribute.


24

eBGP vs. iBGP eBGP vs. iBGP

iBGP Rules:iBGP Rules:• Can talk to routers many hops away by default.• Can only send routes it “injects”, or routes heard directly

from an external peer.• Thus, requires a full mesh.• Sends all attributes.• Leaves the “as-path” attribute alone.• Doesn’t touch the “next hop” attribute. • With iBGP, next-hop is not a router directly connected.

– So a “recursive lookup” is needed.– After the next-hop is found, a second lookup is made to

figure out how to send the packet “in the direction” of the next-hop.


25

iBGP Route Distribution iBGP Route Distribution RestrictionRestriction

AS 1239

AS 2828

A

B

C

170.10.0.0/16170.10.0.0/16

D E

• Assume AS1239 sends route 170.10.0.0/16 to AS2828. Router A will send that route to Routers B and C

• When Router B receives 170.10.0.0/16, it will not propagate that route to Router C because it was learned from an iBGP neighbor. Router C will behave similarly


26

Making BGP Scalable• Address and route aggregation• iBGP fully meshed, not scalable for large AS• Two mechanisms:

– BGP route reflector (RR)• Client: used to identify “client” of the RR(s). • Non-client: identifies standard BGP peers.• Cluster: a group of clients under same RR(s).• Cluster-id: unique identifier for a cluster.• Originator-id: router-id of the originator of the route.

– BGP confederation, e.g.,– Fully-mesh all BGP speakers at a POP– Use fake ASNs at each POP– Between POPs, use eBGP rules (send everything)– Within POPs, use iBGP rules– Preserve local_prefs between POPs


27

Scaling I-BGP for large AS

• Route reflectors • Confederations

E-BGP update

RR RR

Only best paths being sent by RR

AS 1000

EBGP

EB

GP

EBGPIBGP IBGP

AS 65010 AS 65020


28

Establish connectivity

135.120.0.0/16

12.10.0.1

12.10.0.2

Prefix Next hop AS path135.120.0.0/16 12.10.0.1 1

EBGP

IBGPIBGP

IBGPEBGP

12.10.0.5

12.10.0.6

AS 1 AS 2

AS 3Prefix Next hop AS path135.120.0.0/16 12.10.0.5 2 1



29

IGP and BGP working together

135.120.0.0/1612.10.0.1

12.10.0.2


EBGP

IBGPIBGP

IBGPEBGP

12.10.0.5

12.10.0.6

AS 1 AS 2

AS 3Prefix Next hop AS path135.120.0.0/16 12.10.0.1 1

10.10.0.1

Prefix Next hop12.10.0.0/30 10.10.0.1135.120.0.0/16 10.10.0.1

12.10.0.0/30


30

BGP Messages: Four TypesBGP Messages: Four Types

• Open : Establish a peering session.

• Keep Alive : Handshake at regular intervals.

• Notification : Shuts down a peering session.

• Update : Announcing new routes or

withdrawing previously announced routes.

route announcement = prefix + attributes values


31

What Is an Attribute?What Is an Attribute?

• Attribute encoded in a TLV (type-length-value) format.

• Attribute length is 4 bytes long• Attributes can be transitive (across ASs) or non-

transitive (between AS neighbors only) • Some are mandatory: e.g., AS Path, Next-Hop, etc.

Next Next HopHop

AS AS PathPath

............MEDMED......


32

BGP Attributes

Value Code Reference----- --------------------------------- --------- 1 ORIGIN [RFC1771] 2 AS_PATH [RFC1771] 3 NEXT_HOP [RFC1771] 4 MULTI_EXIT_DISC [RFC1771] 5 LOCAL_PREF [RFC1771] 6 ATOMIC_AGGREGATE [RFC1771] 7 AGGREGATOR [RFC1771] 8 COMMUNITY [RFC1997] 9 ORIGINATOR_ID [RFC2796] 10 CLUSTER_LIST [RFC2796] 11 DPA [Chen] 12 ADVERTISER [RFC1863] 13 RCID_PATH / CLUSTER_ID [RFC1863] 14 MP_REACH_NLRI [RFC2283] 15 MP_UNREACH_NLRI [RFC2283] 16 EXTENDED COMMUNITIES [Rosen] ... 255 reserved for development

From IANA: http://www.iana.org/assignments/bgp-parameters

Mostimportantattributes

Not all attributesneed to be present inevery announcement


33

AS Path Attribute

AS7018135.207.0.0/16AS Path = 6341

AS 1239Sprint

AS 1755Ebone

AT&T

AS 3549Global Crossing

135.207.0.0/16AS Path = 7018 6341

135.207.0.0/16AS Path = 3549 7018 6341

AS 6341

135.207.0.0/16

AT&T Research

Prefix Originated

AS 12654RIPE NCCRIS project

AS 1129Global Access

135.207.0.0/16AS Path = 7018 6341

135.207.0.0/16AS Path = 1755 1239 7018 6341

135.207.0.0/16AS Path = 1129 1755 1239 7018 6341

How to detect loop using AS path?


34

Origin AttributeOrigin Attribute

• One of the mandatory, but minor, attributes of a BGP route is the origin. It is one of (in order of preference):– IGP (i) (from a network statement)– EGP (e) (from an external peer)– Unknown (?) (from IGP redistribution)

• It can be re-set, but that is not often done.• It is almost-last in the path selection algorithm.


35

Inserting Routes into BGPInserting Routes into BGP • How do routes get into BGP?

– You have to insert routes into BGP, and someone had to insert external routes that you get into BGP somewhere else in the first place.

• Two main ways:– network statements: “network x.y.z.q [mask a.b.c.d]”

• MUST have an EXACTLY matching IGP route, e.g., router bgp 1 network 192.213.0.0 mask 255.255.0.0 …… ip route 192.213.0.0 255.255.0.0 null 0

– redistributing from OSPF, static, etc...• redistribute <igp-protocol> <protocol-id>

– need to be careful, must filter externally learned routes!

• redistribute static


36

Next Hop AttributeNext Hop Attribute

• Next-hop IP address to reach a network.• Router A will advertise

198.3.97.0/24 to router B with a next-hop of 207.240.24.202.

• With IBGP, the next-hop does not change.

• IGPs should carry route to next-hops, using intelligent forwarding decision (i.e., via IGP).

AS 6201

AS 3847

198.3.97.0/24A

B

207.240.24.200/30

.201

A

B

.202

C


37

iBGP and Next-Hop: ExampleiBGP and Next-Hop: ExampleIn AS 2828: • Router A: “next hop” for 170.10.0.0/16 will be

the serial interface on Router D in AS1239 router• This is true even in Router B’s and Router C’s

forwarding table.

AS 1239

AS 2828

A

B

C

170.10.0.0/16170.10.0.0/16

D E


38

Local Preference AttributeLocal Preference Attribute• Local to AS

– transitive throughout your network. It is never advertised to an eBGP peer.

• Used to influence BGP path selection

AS 6201

208.1.1.0/24

A B

208.1.1.0/24 100Preferred by all AS3847 routers

208.1.1.0/24 80

AS 3847

GF E

C D

• Default 100– Highest local-pref preferred

• For example, you can express the policy “prefer private connect” by making the “local_pref” be 150 and leaving all other peers at 100.


39

• Indication to external peers of preferred path into an AS

– Advertised to external neighbors– Neighbors are not obliged to heed it

• Affects routes with same AS path* Lowest MED preferred• A commonly used attribute by ISPs

– Usually based on IGP metric– For example, big ISPs with multiple connections with each other use MED to indicate which PoP is “closest” to an advertised route, thus more preferred

• It comes after AS_PATH in evaluation, and thus isn’t quite as much of a “hammer” as local-pref

Multi-Exit Discriminator (MEDMulti-Exit Discriminator (MED)


40

MED Attribute: ExampleMED Attribute: Example

+20+20

+5+5

128.11.10/24128.11.10/24

AS 1

AS 701


41

Community AttributeCommunity Attribute• Defined in RFC 1997• 32-bit integer:

– represented as two 16-bit integer x:y

• Used to group routes (“net prefixes”)– Each route could be member of multiple communities

• Transitive: carried across ASs• Very useful in applying policies• Well-known communities

– No-Export: do not advertise to eBGP peers– No-advertise: do not advertise to any peer– Local-AS: do not advertise outside local AS (only used with

“confederations”)


42

Community Attribute: ExampleCommunity Attribute: Example

200.10.0.0/16200.10.0.0/16

AS 500

AS 100

160.10.0.0/16 300:1160.10.0.0/16 300:1

160.10.0.0/16160.10.0.0/16

AS 400

AS 300

AS 200

170.10.0.0/16170.10.0.0/16

170.10.0.0/16 300:1170.10.0.0/16 300:1

160.10.0.0/16 300:1160.10.0.0/16 300:1170.10.0.0/16 300:1 170.10.0.0/16 300:1

200.10.0.0/16 300:9200.10.0.0/16 300:9


43

Attributes are Used to Select Best Routes

192.0.2.0/24pick me!

192.0.2.0/24pick me!

192.0.2.0/24pick me!

192.0.2.0/24pick me!

Given multipleroutes to the sameprefix, a BGP speakermust pick at mostone best route

(Note: it could reject them all!)


44

Policy routing

ISP1

ISP4ISP3

Cust1 Cust2

ISP2

traffic

traffic

Connectivity DOES NOT imply reachability!

Policy determines how traffic can flow on the Internet

45

BGP Route Processing

Best Route Selection

Apply Import Policies

Best Route Table

Apply Export Policies

Install forwardingEntries for bestRoutes.

ReceiveBGPUpdates

BestRoutes

TransmitBGP Updates

Apply Policy =filter routes & tweak attributes

Based onAttributeValues

IP Forwarding Table

Apply Policy =filter routes & tweak attributes

Open ended programming.Constrained only by vendor configuration language


46

Best Route Selection Rule

Highest Local Preference

Shortest ASPATH

Lowest MED

iBGP < eBGP (i.e. prefer eBGP)

Lowest IGP cost to BGP egress

Lowest router ID

Traffic Engineering

Enforce relationships(provider-customer, peer)

Throw up hands andbreak ties

Origin: prefer IGP over BGP over INCOMPLETE


47

Typical AS relationships

• Provider-customer – customer pay money for transit

• Peer-peer– typically exchange respective customers’ traffic for free

• Siblings– Mutual transit agreement– Provide connectivity to the rest of the Internet for each

other


48

Internet Business Model (Simplified)

• Customer/Provider: One AS pays another for reachability to some set of destinations

• “Settlement-free” Peering: Bartering. Two ASes exchange routes with one another.

Provider

Peer

Customer

Preferences implemented with local preference manipulation

Destination

Pay to use

Get paid to use

Free to use


49

Customers and Providers

Customer pays provider for access to the Internet

provider

customer

IP trafficprovider customer


50

The Peering Relationship

peer peer

customerprovider

Peers provide transit between their respective customers

Peers do not provide transit between peers

Peers (often) do not exchange $$$

trafficallowed

traffic NOTallowed


51

Peering Provides Shortcuts

Peering also allows connectivity betweenthe customers of “Tier 1” providers.

peer peer

customerprovider


52

Which AS paths are legal?

• Valley-free:– After traversing a provider-customer or peer-peer

edge, cannot traverse a customer-provider or peer-peer edge

– Invalid path: >= 2 peer links, downhill-uphill, downhill-peer, peer-uphill


53

Example of valley-free paths

XX

[1 2 3], [1 2 6 3] are valley-free

[1 4 3], [1 4 5 3] are not valley free


54

AS relationships translate into BGP export rules

• Export to a provider or a peer– Allowed: its routes and routes of its customers and

siblings– Disallowed: routes learned from other providers or

peers

• Export to a customer or a sibling– Allowed: its routes, the routes of its customers and

siblings, and routes learned from its providers and peers


55

Filtering and RankingsRanking: route selectionFiltering: route advertisement

Customer

Competitor

Primary

Backup


56

BGP Router - Processing RoutesBGP Router - Processing Routes

• For each route received:– If it’s a valid route AND passes any filters, it must be

put into the BGP routing table.– Then, unless it is replacing a duplicate, a best-path

computation must be run on all candidate BGP routes of the same prefix.

– Then, if the best route changed, the RIB and/or FIB must be updated.

– This process is done for ALL incoming BGP routes.


57

BGP Policy ConfigurationBGP Policy Configuration• Some Cisco BGP filtering mechanisms:

– To decide what routes can and can’t go to various other routers, you can “filter” using:

• “distribute lists” (“prefix filters”) - lists of routes• “filter lists” (“as-path filters”) - lists of regular

expressions matching or denying ASs• “route maps” (“BGP Basic programs”) that allow

you to match and change most BGP attributes


58

What policy looks like in Cisco IOS

Inbound “Route Map”(import policy)

eBGP Session


59

Tweak Tweak Tweak• For inbound traffic

– Filter outbound routes– Tweak attributes on

outbound routes in the hope of influencing your neighbor’s best route selection

• For outbound traffic– Filter inbound routes– Tweak attributes on

inbound routes to influence best route selection

outboundroutes

inboundroutes

inboundtraffic

outboundtraffic

In general, an AS has morecontrol over outbound traffic


60

Tuning Inbound BGP Routes(i.e., outbound traffic)

• Inbound BGP routes make traffic go out – Having a route means that an outbound packet can

use it as basis for a forwarding decision (well, the router can)

– It is far easier to control outbound traffic than inbound

• Goal is generally to provide fastest, lowest-loss, path for all destinations,

i.e., to optimize connectivity “quality”, whatever that is– E.g., to optimize throughput and latency– to reduce transit costs, say,

• squash traffic via a certain provider,• prefer customer than peer/provider paths, and prefer

peer than provider paths– to load balance, or to ensure reliability with back-up

routes


61

Controlling Outbound BGP Routes

(i.e., inbound traffic)• Outbound BGP routes make traffic come in

– It’s a lot harder to control inbound traffic as other ASs’ policies complicate your life!

• If you are a stub AS with a single connection– Not much you need to do except to filter out routes not in your

AS

• If you are a multi-homed stub AS,– Want to control through which link/provider that traffic to certain

destinations in your AS may take, to load balance or for back-up

• If you are an ISP, you want to minimize transit cost,– carry transit traffic from customers only ! – use “hot-potato” routing to hand off traffic to

peers/providers as soon as possible– to load balance, or to ensure reliability with back-up routes

62

Shedding Inbound Traffic with ASPATH Padding Hack

Padding will (usually) force inbound traffic from AS 1to take primary link

AS 1

192.0.2.0/24ASPATH = 2 2 2

customerAS 2

provider

192.0.2.0/24

backupprimary

192.0.2.0/24ASPATH = 2

63

Padding May Not Shut Off All Traffic

AS 1

192.0.2.0/24ASPATH = 2 2 2 2 2 2 2 2 2 2 2 2 2 2

customerAS 2

provider

192.0.2.0/24

192.0.2.0/24ASPATH = 2

AS 3provider

AS 3 will sendtraffic on “backup”link because it prefers customer routes and localpreference is considered before ASPATH length!

Padding in this way is oftenused as a form of loadbalancing

backupprimary

64

Hot Potato Routing: Go for the Closest Egress Point

192.44.78.0/24

15 56 IGP distances

egress 1 egress 2

This Router has two BGP routes to 192.44.78.0/24.

Hot potato: get traffic off of your network as soon as possible. Go for egress 1!

65

Getting Burned by the Hot Potato

15 56

172865High bandwidth

Provider backbone

Low bandwidthcustomer backbone

Heavy Content Web Farm

Many customers want their provider to carry the bits!

tiny http request

huge http reply

SFF NYC

San Diego


66

Inter-Domain Traffic Engineering

• Better to cooperate?– Negotiate where to send– Inbound and outbound– Mutual benefits

• But, how to do it?– What info to exchange?– How to prioritize the

many choices?– How prevent cheating?

• Open research territory

Customer A

Customer B

multiplepeeringpoints

Provider A

Provider B

Early-exit routing

csci5221: inter-domain routing and bgp 1 inter-domain routing: bgp, routing policies, etc. overview...

Documents

routing policies

routing policy

choice of routing algorithm

ass togethertwolevel

networks ass

networks of networks

routes parts

internal routes