stateless multicast withd2zmdbbm9feqrf.cloudfront.net/2016/usa/pdf/brkipm-2239.pdf · stateless...

Stateless Multicast withBit Indexed Explicit Replication

IJsbrand Wijnands – Distinguished Engineer

BRKIPM-2239

• Introduction

• Solution Overview

• Encapsulation

• Sets and Areas

• Forwarding

• ECMP

• BIER Overlay

• Deployment Scenarios

• Conclusion

Agenda

Introduction

© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public

BIER history

• A team was formed to investigate solutions for multicast in the context of Segment Routing.

• Encoding a Sourced routed Multicast tree path using MPLS labels is difficult.

• The packet header would get very large, and its very hard to parse such header.

BRKIPM-2239 5


The BIER Epiphany

• Only encode the end-receivers (as a Bit Position) in the packet header.

• Not the intermediate nodes.

• Encode the Bit String in the packet header.

• Using some sort of encapsulation.

• Create a Bit Forwarding Table on all BIER nodes to allow multicast packet forwarding using the Bit String in the packet.

• Derived from the RIB, SPF based.

• We call it, Bit Indexed Explicit Replication (BIER).

BRKIPM-2239 7


Tech Fund

• Chambers Tech Fund to prototype the idea Jan 2014.

• Core TF team: Greg Shepherd, Neale Ranns, IJsbrand Wijnands.

• Goal: build a prototype on a real router platform.

• We have a working prototype code on XRVR, VPP, CRS, ASR1K and ASR9K.

• Planned Release

• ASR9K - IOS-XR 6.4.1

• ASR1K - IOS-XE 16.6S (July 2017 FCS)

BRKIPM-2239 8


IETF

• The BIER idea was presented in a BOF at the IETF in Hawaii.

• November 2014.

• A new BIER Working Group has been formed ([email protected])

• Vendors collaborating (co-authoring) with us;

BRKIPM-2239 9


IETF drafts

• draft-ietf-bier-problem-statement

• draft-ietf-bier-architecture

• draft-ietf-bier-encapsulation-mpls

• draft-ietf-bier-use-cases

• draft-ietf-l3vpn-mvpn-bier

• draft-ietf-ospf-bier-extensions

• draft-ietf-bier-isis-extensions

• draft-xu-idr-bier-extensions

BRKIPM-2239 10

Solution Overview


Basic Idea BIER

BitString

BIER Domain

LSA

1 - A/32

LSA

5 – E/32

LSA

4 – D/32

LSA

3 – C/32

LSA

2 – B/32

1. Assign a unique Bit Position from a BitString to each Edge router in the BIER domain.

2. Each Edge floods their Bit Position - Prefix mapping using the IGP (OSPF, ISIS)

12345

B/32

A/32

C/32D/32

E/32 12345

BRKIPM-2239 12


Basic Idea BIER

1. Assign a unique Bit Position from a BitString to each BFER in the BIER domain.

2. Each BFER floods their Bit Position to BFR-prefix mapping using the IGP (OSPF, ISIS)

B/32

A/32

C/32D/32

E/32

BitMask Nbr

0011 A

0100 B

1000 C

BRKIPM-2239 13


Bit Index Forwarding Table

• Based on shortest path route to RID, the Bit Mask Forwarding Table is created

CA B

D

FE

BM Nbr

0111 B

BM Nbr

0011 C

0100 E

BM Nbr

0001 D

0010 F

BFR-ID 1

BS:0001

BM Nbr

0011 C

B

BM-ER

BM-ERBM-ER

BFR-ID 2

BS:0010BFR-ID 3

BS:0100

BRKIPM-2239 14


Forwarding Packets

CA B

D

FE

BM Nbr

0111 B

BM Nbr

0011 C

0100 E

BM Nbr

0001 D

0010 F

00010001

AND ANDAND

0001

&0011&0111 &0001

BFR-ID 1

BS:0001

BFR-ID 2

BS:0010BFR-ID 3

BS:0100

Nbr

0011 C

B

Suppose A leans about D’s interest,

in the blue multicast flow.

(via BGP, SDN, STATIC, etc…)

BRKIPM-2239 15


Forwarding Packets

A CA B

D

FE

Nbr

0111 B

Nbr

0011 C

0100 E

Nbr

0001 D

0010 F

00010101

AND ANDAND

0101

&0011&0111 &0001

&0100

AND

BFR-ID 1

BS:0001

BFR-ID 2

BS:0010BFR-ID 3

BS:0100

Nbr

0011 C

B

Suppose A leans about D and E’s interest,



BRKIPM-2239 16


Forwarding Packets

A CA B

D

FE

Nbr

0111 B

Nbr

0011 C

0100 E

Nbr

0001 D

0010 F

00110111

AND ANDAND

0111

Nbr

0011 C

B

&0011&0111 &0001

&0100 &0010

AND

BFR-ID 1

BS:0001

BFR-ID 2

BS:0010BFR-ID 3

BS:0100

Suppose A leans about D, E and F’s interest,



BRKIPM-2239 17


Forwarding Packets

• As you can see from the previous slides, the result from the bitwise AND (&) between the Bit Mask in the packet and the Forwarding table is copied in the packet for each neighbor.

• This is the key mechanism to prevent duplication.

• Look at the next slide to see what happens if the bits are not reset

• If the previous bits would not have been reset, E would forward the packet to C and vice versa.

BRKIPM-2239 18


Forwarding Packets

A CA B

D

FE

Nbr

0111 B

Nbr

0011 C

0100 E

Nbr

0001 D

0010 F

01110111

AND ANDAND

0111

Nbr

0011 C

B

&0011&0111

&0100

AND

0111

BFR-ID 1

BS:0001

BFR-ID 2

BS:0010BFR-ID 3

BS:0100

BRKIPM-2239 19

Encapsulation


How many Bits and Where?

• The number of multicast egress routers that can be addressed is depending on the number of Bits that can be included in the BitString

• The BitString length is dependent on the encapsulation type and router platform.

• We’ve analyzed the MPLS option, CRS, ASR1K and ASR9K platform.

• Both these platforms can do 256 bits.

• Other vendors confirmed they can do 256.

• We identified 5 different encoding options, most attractive below;

1. MPLS, below the bottom label and before IP header.

2. IPv6, extensions header.

BRKIPM-2239 21


MPLS encapsulation

• The Top Label is allocated by BIER from the downstream platform label space.

• The BIER Header follows directly below the BIER label.

• There is a single BIER label on top, unless the packet is re-encapsulated into a unicast MPLS tunnel.

• The VPN label is allocated from the upstream context label space (optional).

BIER Label BIER Header VPN Label Payload

MPLS Label IPv4/IPv6/L2Upstream Label

(optional)

BIER header

EO

S

EO

S

BRKIPM-2239 22


BIER Header

• http://www.ietf.org/id/draft-ietf-bier-mpls-encapsulation-01.txt

0 1 2 3

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

|0 1 0 1| Ver | Len | Entropy |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| BitString (first 32 bits) ~

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

~ ~

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

~ BitString (last 32 bits) |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

|OAM| Reserved | Proto | BFIR-id |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

256bits = 32bytes

BRKIPM-2239 23

Sets and Areas


BIER Sets

• To increase the scale we group the egress routers in Sets.

I

A

G B

1:0001

C

D

E

F

H

1:0010

1:0100

2:0001

2:0010

2:0100

Set 1

Set 2

1:0111

2:0111

Set BM Nbr

1 0111 I

2 0111 I

Note, Bit Positions 1,2,3

appear in both Sets, and

do not overlap due to Sets.

Note, we create different

forwarding entries for each Set

J

BRKIPM-2239 25


BIER Sets

• There is no topological restriction which set an egress belongs to

I

A

G B

1:0001

C

D

E

F

H

1:0010

2:0001

1:0100

2:0010

2:0100

Set 1

Set 2

1:0111

2:0111

Note, we create different

forwarding entries for each Set

J

Set 2

Set 1Set BM Nbr

1 0111 I

2 0111 I

BRKIPM-2239 26


BIER Sets

• If a multicast flow has multiple receivers in different Sets, the packet needs to be replicated multiple times by the ingress router, for each set once.

• Is that a problem? We don’t think so…

• The Set identifier is part of the packet.

• Can be implemented as MPLS label.

BRKIPM-2239 27


BIER Areas

• BIER area’s are like IGP area’s, you only need reachability to the ABR

I

A

G B

1:001

C

D

E

F

H

1:010

1:100

2:001

2:010

2:100

Area 1

Area 2J

BIER Forwarding

0:011

0:001

0:010

Area BM Nbr

0 0011 I

1:010

2:010

ABR

ABR

Area 0

BRKIPM-2239 28


BIER Areas

• The ABR removes the BIER header from Area 0, and imposes a new BIER header for Area 1 and 2.

• The new BIER header can be determined by a Group/Label lookup.• Look for the inner IPv4/6 packet group address, do a lookup in the MFIB

• Requires flow state on the ABR.

• Similar to Segmented Inter-AS MVPN

BRKIPM-2239 29

Sub-domains


BIER Sub-domains

• Nodes are configured to be in a BIER sub-domain.

• BIER MPLS labels are not installed between sub-domains

A B C D

E F G H

PIM/

IGMP

PIM/

IGMP

Domain 1

Domain 2

BRKIPM-2239 31


BIER Sub-domains

• When IGP changes SPF for B to F, traffic is dropped.

• BIER traffic is never leaked between sub-domains.

• Routers can be part of multiple sub-domains.

• This is a very easy mechanism to force dual plane.

A B C D

E F G H

PIM/

IGMP

PIM/

IGMP

Domain 1

Domain 2

X

BRKIPM-2239 32

The BFR-id


The BFR-id

• A BFER is uniquely identified by a two tuple {Set ID, Bit Position}

• The number of Bit Positions is depending on the length of the support BitString in the network.

• To make the BFER identifier independent of the BitString length we defined the BFR-id as a number between [1,65535]

• We auto-generate the BFR-id into a {Set ID, Bit Position}, based on the BitString Length.

BRKIPM-2239 34


The BFR-id

• Formula:SI = (BFR-id -1) / BitStringLength

BP = ((BFR-id -1) modulo BitStringLength) + 1

• Example BFR-id = 129.BitString Length 128 -> {SI = 1, BP = 1}

BitString Length 256 -> {SI = 0, BP = 129}

BRKIPM-2239 35


The BFR-id

• By decoupling the BFR-id from the SetID and Bit Position tuple, the BFR identifier is agnostic to the supported BitString length in the network.

• This is very useful during migration.

• If a network supports multiple BitString lengths, an egress router only needs oneBFR-id, and is reachable via each BitString length.

BRKIPM-2239 36

BIER Forwarding


BIER Forwarding

We define two different forwarding methods for BIER.

1. Neighbor based forwarding.

2. Bit Indexed forwarding

BRKIPM-2239 38


BIER Forwarding, neighbor based.

• A packet BitString is matched against each Neighbor in the BFT.

• This model works well for systems that have multi-cores.

• Its works less well for serialized processing as it requires a neighbor walk for each packet

BIER Forwarding Table

5 4 3 2 1 Nbr

1 A

1 B

1 C

1 1 D

Packet IN Packet OUT 00100=10101 10101&

1

2

3

4

Packet OUT 10001=10101&

&

BRKIPM-2239 39


BIER Forwarding, Bit Indexed

• We translate the BFT neighbor table in to a table sorting on Bit Position

(so not by neighbor)

• We walk the Bit Mask in the packet and Index into the FIB table.

BFT Neighbors

5 4 3 2 1 Nbr

1 A

1 B

1 C

1 1 D

BFT Indexed

54321 Nbr

1 10001 D

2 00010 A

3 00100 B

4 01000 C

5 10001 D

BRKIPM-2239 40



• We walk the Bits in the packet, as soon as we hit a ‘1’, we copy the packet, index into the FIB table with the position of the Bit.

• The Bit Mask entry is reverse ‘&’ with the Bit Mask in the packet.

• This resets the Bits that where processed.

BIER FIB Label

Forwarding Table

54321 Nbr

1 10001 D

2 00010 A

3 00100 B

4 01000 C

5 10001 D

10101Packet IN

1

Walk Bit Mask

Packet OUT 10001=

01110

&

10101

Copy

&

BRKIPM-2239 41



BIER FIB Label

Forwarding Table

54321 Nbr

1 10001 D

2 00010 A

3 00100 B

4 01000 C

5 10001 D

00100Packet IN Packet OUT

5 4 3 2

Walk Bit Mask

1000110101 =

Copy

&

00100= Packet OUT 00100 &

Copy

00000

&

11011

• We walk the Bits in the packet, as soon as we hit a ‘1’, we copy the packet, index into the FIB table with the position of the Bit.

• The Bit Mask entry is reverse ‘&’ with the Bit Mask in the packet.

• This resets the Bits that where processed.

BRKIPM-2239 42



• Walking the bits in a Bit String takes less clock cycles compared to walking a list of neighbors.

• For that reason its faster to walk the Bit String and index into the neighbor table.

• The table is a NxN bit matrix, where N is the Bit String length.

• Bits that where already processed are reset so we don’t processes them if they appears later in a Bit String. This way we avoid multiple copies being forwarded.

BRKIPM-2239 43



• Walking the Bit String in the packet is basically a repeating of ‘Find First Set’ (FFS) Bit operation.

• Could be optimized to record the last position.

• http://en.wikipedia.org/wiki/Find_first_set

• http://graphics.stanford.edu/~seander/bithacks.html

• Checkout Bruijn sequence (Count the consecutive zero bits)

• Is supported in compilers FFS and can be done in HW fairly easily.

BRKIPM-2239 44


ECMP

• It is possible the same Bit Position is reachable over different interfaces if there are ECMP paths.

• We distinguish two different ECMP behaviors

• ECMP via parallel interfaces to a single neighbor.

• ECMP via different neighbors.

• The handling is different for both cases.

• If the ECMP path is going to a single neighbor, the Bit Mask is the same for each candidate path, no special processing needed.

• If the ECMP path is to different neighbors, the Bit Mask will be different for each neighbor because the Bit String in the packet is the end result of the lookup in BIER, which is different for each nbr.

BRKIPM-2239 46


ECMP

CA B

D

FE

Nbr

0111 B

Nbr

0011 C

0110 E

Nbr

0001 D

0010 F

00110111

AND ANDAND

0111

0001

0010

0100Nbr

0011 C

B

0010 F

&0011&0111 &0001

&0110 &0010

AND

Duplicate bit positions need to be resolved,

ECMP logic needs to select based on Hash.

In the example we selected C

BRKIPM-2239 47


ECMP

• We distribute the Bit Positions over multiple tables.

• Each Bit Position only appears once in each table.

• Table selection is based on entropy, done before Bit Index lookup

ECMP Table 1

5 4 3 2 1

1 1 1

1

1

ECMP Table 2

5 4 3 2 1

1 1 1

1 1

ECMP Table 3

5 4 3 2 1

1 1

1

1 1

ECMP Table 4

5 4 3 2 1

A

1 1 B

C

1 1 1 D

BFT Neighbor

5 4 3 2 1 Nbr

1 1 1 A

1 1 1 B

1 1 C

1 1 1 D

BRKIPM-2239 48


ECMP

• The number of tables is depending on the number of ECMP paths to different neighbors.

• If the max number of ECMP paths is 4, but there is a Bit Position reachable via 3 paths, this will cause unequal distribution.

BRKIPM-2239 49

BIER Overlay


BIER Overlay

• One of the deployment challenges of BIER is HW support.

• Some routers may be delayed, others may never support it.

• We can run BIER as an overlay using the existing network.

BRKIPM-2239 51

The Next BFR in the path


The Next BFR in the path

• When sending a BIER packet from ingress to egress, some nodes in the path may not support BIER.

• If a node is just pass-through and not doing any replication, its ok to skip this node and go to the next BIER node in the path.

• We identified how to modify the SPF algorithm in order to find the next BIER node in the path and tunnel to it (easy with MPLS).

• This solution is beneficial if some routers don’t support BIER, we do expect ingress and egress routers to support BIER.

BRKIPM-2239 53


The next BFR in the path

• With enhanced IGP calculation, A can calculate next BIER node in the path.

• C is installed as an alternate next-hop in the RIB at A.

E

F

A

BM A Nbr

0011 C

0001

0010

B C

RIB-C Nbr

E E

F F

RIB-A Nbr

E B

C

F B

C

BM C Nbr

0001 E

0010 F0011Alternate

Alternate

BRKIPM-2239 54

BIER on a stick


BIER on a stick

• With BIER on a stick, a Fullstack BIER capable router is connected to a non-BIER capable router.

• The BIER router advertises it self through the IGP as being connected to the non-BIER router.

• When calculating the path towards the destination, the SPF is modified such that the BIER router appears to be on the SPF (instead of the non-BIER router)

BRKIPM-2239 56


BIER on a stick

• The SPF on A for E is through B, the modified SPF logic knows G is the BIER router for B.

• G will deal with BIER traffic on behalf of B. Same as D for C

• G and G are Fullstack BIER capable, as control plane and forwarding plane.

• It looks like G is connected to B through a zero cost Tunnel.

E

F

A

0001

0010

B C

DG

1 1

1

1

0

RIB-A Nbr

E B

G

F B

G

RIB-G Nbr

E B

D

F B

D

BIERBIER

BIER

BIER

IP

IP

IP

0

RIB-D Nbr

E C

E

F C

F

Alternate

Alternate

Alternate

Alternate

Alternate

Alternate

BRKIPM-2239 57


BIER on a stick

• A BIER on a stick router can also be used to replace the PE.

• IP traffic tunneled into the BIER router, BIER traffic comes out.

• Same on the egress, BIER traffic is tunneled in, IP traffic comes out.

• This will remove Multicast from the core completely.

BRKIPM-2239 58


BIER on a stick

• The IP multicast traffic on Ingress A is pushed to H. This requires changes on the IP Multicast side (either config or code).

• On the Egress side the RPF instance has to point to the BIER node in order to accept the IP traffic coming down.

0001

0010

B C

D

1 1

1

1

BIER BIER

BIER

BIER

A

H

I

J

E

F

IP

IP

IP

0 0

0

0

RIB-H Nbr

E A

D

F A

D

RIB-D Nbr

E C

I

F C

J

Alternate

Alternate

Alternate

Alternate

BRKIPM-2239 59

Deployment Scenarios

Native BIER


Native BIER

• With Native BIER there is NO PIM involved, just IGMP and BIER.

• The Source and Receiver(s) are connected to BIER router.

• There are no RP’s.

• There is no equivalent of PIM modes, like sparse, ssm, bidir etc..

• We speak of ‘single’ sender and ‘multi’ sender, which is basically the same solution.

• The overlay can be BGP or SDN based.

BRKIPM-2239 62


Native BIER

• E and F announce their Group membership via overlay to all other routers.

• A BIER router connected to the Source can immediately start sending.

DC

E

F

A

B

(S1,G)(*,G):0:000

1 IGMP

(*,G)

IGMP

(*,G)

0 Nbr

0001 E

0010 F

1100 C

0 Nbr

0011 D

0100 A

1000 B

0 Nbr

1011 C

0 Nbr

0111 C

0 Nbr

1110 D

0 Nbr

1101 D

(*,G):0:000

1

(*,G):0:000

1

0100

1000

0001

0010

(*,G):0:001

0

(*,G):0:001

0

(*,G):0:001

0

BRKIPM-2239 63


Native BIER

• When B leans about a new source, it can immediately start sending.

DC

E

F

A

B

(S1,G)

(*,G):0:000

1

IGMP

(*,G)

IGMP

(*,G)

0 Nbr

0001 E

0010 F

1100 C

0 Nbr

0011 D

0100 A

1000 B

0 Nbr

1011 C

0 Nbr

0111 C

0 Nbr

1110 D

0 Nbr

1101 D

(*,G):0:000

1

(*,G):0:000

1

0100

1000

0001

0010

(*,G):0:001

0

(*,G):0:001

0

(*,G):0:001

0

(S2,G)

BRKIPM-2239 64

MVPN over BIER


MVPN over BIER

• BIER replaces PIM, mLDP, RSVP-TE or IR in the core.

• BIER represents a full mesh (P2MP) connectivity between all the PE’s in the network.

• There is no need to explicitly signal any MDT’s (or PMSI’s).

• With MVPN there are many profiles,

• This is partly due to the tradeoff between ‘State’ and ‘Flooding’.

• Different C-multicast signaling options.

• MVPN over BIER, there is one profile.

• BGP for C-multicast signaling.

• No need for Data-MDTs.

BRKIPM-2239 66


MVPN over BIER

• The BGP control plane defined for MVPN can be re-used.

• Big difference, there is no Tree per VPN…!!!

• The BIER packets needs to carry Source ID and upstream VPN context label

C

D

A

B

0100

1000

0001

0010

BIER

PIM

PIM

PIM

PIM

PIM

PIM

RR(*,G):0:0001

(*,G)

(*,G)

(*,G)

(S,G)

(S1,G)

(S2,G)

(*,G):0:0010

(*,G):0:0001

(*,G):0:0001

BRKIPM-2239 67

Conclusion


Stateless

• There is no Multicast receiver or flow state in the core network (only edge).

• Imposition of the BIER Header may be done by application, removes state from ingress.

• There is no tree state in the network.

• There is no tree building protocol or logic in the network.

• There is only topology state for the BFER’s, derived from unicast routing.

BRKIPM-2239 69


Scale

• Since there is no flow and tree state, converges as fast as unicast.

• Compared to Ingress Replication, saves 256x (minimum)

BRKIPM-2239 70


Simplicity

• No Reverse Path Forwarding (RPF)

• No Rendezvous Points

• No shared tree / source tree switchover

• No receiver driven tree building

• BIER is like unicast

• State is in the packet (like Segment Routing)

BRKIPM-2239 71

Order your BIER online at:

[email protected]


Complete Your Online Session Evaluation

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online

• Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card.

• Complete your session surveys through the Cisco Live mobile app or from the Session Catalog on CiscoLive.com/us.

BRKIPM-2239 76

CiscoLive.com/Online

http://ciscolive.com/Online

http://ciscolive.com/Online

http://ciscolive.com/us

http://ciscolive.com/us


Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Lunch & Learn

• Meet the Engineer 1:1 meetings

• Related sessions

BRKIPM-2239 77

Thank you

stateless multicast withd2zmdbbm9feqrf.cloudfront.net/2016/usa/pdf/brkipm-2239.pdf · stateless...

Documents