1 © 2007 cisco systems, inc. all rights reserved. edcs-412962 advanced crs/ios-xr training crs-1...

1© 2007 Cisco Systems, Inc. All rights reserved.EDCS-412962

Advanced CRS/IOS-XR Training CRS-1 ‘Deep-dive’


Agenda

• Line Card Chassis's

• Power distribution

• Route Processors, Fan Controllers, and Alarm Modules

• Internal System Control Plane

• PLIMs and MSCs


CRS-1 Line Card Chassis's


CRS-1 16-slot Line Card Chassis

• Midplane design with front & rear accessFront

16 PLIM slots

2 RP slots + 2 Fan Controllers

Back 16 LC Slots

8 Fabric cards

• Dimensions:23.6” W x 41*” D x 84” H

(60 W x 104.2 D x 213.36H (cm))

• Power: 13.9 KW DC or 14.6 KW AC

• Weight: ~1600 lbs/723kg

• NEBS ~1780 lbs/753kg

• Heat Dis.: 41000 BTUs



• Midplane design:Front

8 PLIM slots2 RP slots

Back 8 LC Slots4 Fabric cards

• Dimensions:17.5” W x 36.6” D x 38.5” H(44.5 W x 93 D x 97.8 H cm)

• Power: 8 KW DC or 8.75 KW AC

• Weight: ~ 600 lbs/275kg• Heat Dis.: 27350 BTU• Rack mountable



• Midplane design:Front

4 PLIM slots4 LC Slots2 RP slots (same as CRS8)

Back 4 Fabric cards

• Dimensions:18.5” W x 30.2” D x 30” H(47.1 W x 76.9 D x 76.2 H)

• Power: 4.7KW DC 4.75KW AC

• Weight: 380 lbs/ 172.4kg• Heat Dis.: 14570 BTU• Rack mountable


Chassis differences

16-slot 8-slot 4-slot

Separate cards for fan controllers and alarm modules

Integrated alarm modules on RP card – fan controller present in Fan Tray

Integrated alarm modules on RP card – fan controller present in Fan Tray – Same card as CRS8

RP/A - 2 CPUs in SMP complex @ 800Mhz

RP/B – 2 CPUs in SMP complex @ 1.2Ghz

1 CPU running @ 1.2Ghz 1 CPU running @ 1.2Ghz

8 fabric cards one per plane

4 fabric cards with 2 planes per card

4 fabric cards with 1 plane per card

6 PEMs on 2 power shelves with load zones

2 PEMs with notional load zones

4 PEMs for 2N redundancy. No load zones

Up to 8 DRPs Up to 8 DRPs Only 1 DRP, no SDR support


Power distribution


Power shelves

• The CRS-1 16-slot chassis power distribution system divides the chassis into physical ‘load-zones’ rather than a ‘bus’ system

• While the power system is designed to provide resilience in the event of a failure, a double failure may result in critical linecards loosing power and hence may result in a CRS being cut off from the rest of the network


Power shelves

• The CRS-1 16 slot can operate with a single power shelf in the event of a single shelf failure

• Full system redundancy is not supported until full power is restored

• A PEM / Rectifier in Shelf A will be backed up by a counterpart PEM / Rectifier in Shelf B and vice versa should one or other device fail

RP

Imp

edan

ce

Car

rier

Fan

Co

ntr

olle

r Im

ped

ance

Car

rier

PL

IM Im

ped

ance

C

arri

erP

LIM

Imp

edan

ce

Car

rier

PL

IM Im

ped

ance

C

arri

erP

LIM

Imp

edan

ce

Car

rier

PL

IM Im

ped

ance

C

arri

erP

LIM

Imp

edan

ce

Car

rier

PL

IM Im

ped

ance

C

arri

erP

LIM

Imp

edan

ce

Car

rier

PL

IM Im

ped

ance

C

arri

erP

LIM

Imp

edan

ce

Car

rier

PL

IM Im

ped

ance

C

arri

erP

LIM

Imp

edan

ce

Car

rier

PL

IM Im

ped

ance

C

arri

erP

LIM

Imp

edan

ce

Car

rier

PL

IM Im

ped

ance

C

arri

erP

LIM

Imp

edan

ce

Car

rier

RP

0R

P1

FC

0F

C1

PL

IM 0

PL

IM 1

PL

IM 2


Each Fan Controller draw power from Zone 2 for Upper Fan Tray and Zone 5 for Lower Fan Tray

System will continue to operate with one FC or one Fan Tray removed

Removal of both FCs or Fan Trays will result in the system shutting down!

Load Zones

Switch Fabric cards 0-3 serviced by Zone 2 PEMS. Cards 4-7 services By Zone 5

Loss of Zone 2 or Zone 5 will reduce Fabric throughput capacity!


Slot Placement

• To avoid connectivity loss in a ‘double-fault’ scenario, connectivity functionality should be distributed across the load-zones to avoid a single-point of failure

• Note – the configuration outlined below is purely an example. Any MSC/PLIM may be inserted into any linecard slot position

Load Zone Slot number Card type Functionality

1 0 4*OC192 Core / Intra-POP

1 1 16*OC48 Edge

1 2 OC768 Intra-POP

1 3 8*10GE Edge

3 4 OC768 Core

3 5 8*10GE Edge

3 6 4*OC192 Edge

3 7 16*OC48 Edge

4 8 OC768 Core

4 9 8*10GE Edge

4 10 4*OC192 Edge

4 11 16*OC48 Edge

6 12 4*OC192 Core / Intra-POP

6 13 16*OC48 Edge

6 14 OC768 Intra-POP

6 15 8*10GE Edge


8-slot power distribution

• The CRS-1 8-slot also has the concept of ‘load-zones’

• Three zones span the chassis

• All zones are served by both PEM A and PEM B

• Failure of A or B does not result in loss of a zone



Plim

0

Plim

1

Plim

2

Plim

5

Plim

6

Plim

7

MS

C 2

MS

C 1

MS

C 0

MS

C 7

MS

C 6

MS

C 5

Plim

3

Plim

4

RP

0R

P 1

MS

C 4

MS

C 3

SF

C 0

SF

C 1

SF

C 3

SF

C 2

Load zone 1

Load zone 2

Load zone 3



• The CRS-1 4-slot has no ‘load-zones’

• 4 PEMs provide resilient power

PEM 0 & 1 back each other up

PEM 2 & 3 back each other up

PEM 0 & 1 will back up PEM 2 & 3



AC/DC PSU 2Kw

AC/DC PSU 2Kw

AC/DC PSU 2Kw

AC/DC PSU 2Kw

-54V

-54V

MSC 0

RP0

RP 1

MSC 1

AC0220V/11A

AC1220V/11A

AC2220V/11A

AC3220V/11A


Route Processors, Fan Controllers, and Alarm Modules


MIDPLANE

FE/GESwitch

LC FELinks

CPUMEMCTL

CTL GE linkCTL GE link

PCI

Mgmt 10/100/1GE link

IDE

Aux & ConsoleFabric Connection

FabricQ

CPUCTRL

RP/A = 2x800 Mhz PPC (7455) in SMP configuration

RP/B = 2x1.Ghz PPC (7457) in SMP configuration

2G/4GB DDR DRAM via memory controller

2 MB external cache

64 MB boot flash

2 MB NVRAM

2 PCMCIA slots

slot 0 is sealed with 1 GB Flash disk for OS

Console AND aux serial ports for debug

40 GB IDE hard drive

CRS-1 16-slot RP Architecture

IngressQ


MIDPLANE

FE/GESwitch

LC FELinks

CPUMEMCTL

CTL GE links

PCI

Mgmt 10/100/1GE link

IDE

Aux & Console

Fabric Connection

MPC7457 (1.2Ghz), uni-processor

2G/4GB DDR DRAM via memory controller

2 MB external cache

64 MB boot flash

2 MB NVRAM

2 PCMCIA slots

slot 0 is sealed with 1 GB Flash disk for OS

Console AND aux serial ports for debug

40 GB IDE hard drive

Alarm card integrated into RP card

CRS-1 8-slot / 4-slot RP Architecture

Squirt

IngressQ

CPU Ctrl

FabricQ


Distributed Route Processor

• 2 Components – DRP PLIM and DRP CPU• Occupies 1 slot position in the chassis• Provides 2 additional ‘routers’

Each has independent route memory, flash, disk, pair of CPUs

• Used for process placement and Secure Domain Routers

DRP PLIM DRP CPU


DRP Architecture

I2C1:4

MUX

CPUCTRL

CPUCTRL

SMP

SMP

MIDPLANE

FabricQ

FabricQ

IngressQ

SPModule

IDEMgmt 10/100/1GE link

Aux & Console

IDEMgmt 10/100/1GE link

Aux & Console

48V

48V

QNET FEControlPower

AndIPComm

FPGA

DRP DC-DC3.3V Power

SP DC-DC3.3V Power


DRP/DRP PLIM

ASMP

CpuCtrl

IngressQ

FabricQCon, Aux & Ethernet


CRS-1 RP Hardware Summary

4/8 Slot RP 16 Slot RP DRP

CPUs 1x 1.2GHz PPCRP/A = 2x 800MHz PPC

RP/B = 2x 1.2Ghz PPC2x (2x1.2 GHz) PPC

Memory 2-4 GB 2-4 GB 2 x 4 GB

Mgmt Eth

1x 10/100/1000 1x 10/100/1000 2x 10/100/1000

FlashPCMCIA

1 GB Internal

1 GB External

1 GB Internal

1 GB External

2x 1 GB Internal

2x 1 GB External

Disk 40 GB 40 GB 2x 40 GB


Fan Controllers

• Fan controllers are present in both the 16-slot and 8-slot systems – provides monitoring and control for Fan trays

• CRS16 - two Fan Controller boards which are situated on the front of the chassis above the RP slots – boards also provide BITS signal input interfaces

• Boards operate in tandem – no primary/backup relationship

• At least one Fan Controller must be operational for the system to operate, otherwise system will be shut down

• CRS8 - Fan controller functionality is integrated on the Fan Tray. BITS input is present on the RP

• Based on the reported state, fan speeds can be increased or decreased as needed


Fan Controller for 16 slot system

• Fan controllers board consist of a Service Processor (SP) device, a series of Fan Controller ASICs as well as BITS signal distribution circuitry

• SP polls the Fan Controller ASICs for RPM and Voltage status information

• Thermal sensors present on the various boards in the chassis, indicate state to the SPs on the Fan Controller boards

• Internal Fast-Ethernet connection is used for the Service Processor to communicate to the RP


16 slot Fan Controller Block Diagram

Service Processor

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

BrickC

on

tro

ller

AS

IC Fan PowerBrick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

Brick

Co

ntr

oll

erA

SIC Fan Power

BrickL1

L2

L3

L6

L5

L4

L7

L8

L9

U1

U2

U3

U7

U8

U9

U4

U5

U6


Resilience

• In the 16-slot chassis, each Fan Controller board communicates and can control both the Upper and Lower Fan Trays

• If a Fan tray fails, remaining Fans increase speed to maintain required airflow

Note – for this function to work, both Fan Controllers must be operational due to the power draw required for 16 slot system only


Chassis Air Flow Path

8-slot 16-slot

Front

PLIMs LCs & FCs

Fan Trays

Power System

PLIMs LCs & FCs

Power System

Fan Trays

Front

Power System

Power System

PLIMs, MSCs, RPs

FCs

4-slot


Alarm Modules on CRS-1 16 slot

• On right side of power shelf

• Draws power from the Power shelf directly

• Standard DA-15S alarm relay connector

• Major, Minor, and Critical LED

• 16 character LED

• Monitors power shelf status

• Indicates any system alarms

• Has its own Service Processor


System Control Plane


• Independent system control plane network

NOT used for network control plane data transfer – i.e. BGP/OSPF/etc. updates

Enables multi-chassis system components to communicate independent of data plane

Used for:

• System Discovery/inventory

• Image transfers/downloads

• Heartbeats/alarms

• Configuration transfers

Overview


• Two parts to the CRS-1 system control plane

Intra-chassis => internal FE network per chassis

Uses Broadcom switches located on Primary/Standby RP and each node’s SP

Inter-chassis => external GE network between RPs and SCs

Uses external 6509 prior to release of Integrated Switch on SC in Fabric chassis (rls 3.4.1 onwards)

Overview


High-level diagram

Broadcom switch

modules on the RP

Broadcom switch

modules on the LC

Intra-chassis FE

links

Inter-chassis GE

links

5618_0 5618_1 5618_0

5325 5325 MSC_0 MSC_n

RP0 RP1

CPU SP CPU SP

CPU CPU

5618_1

External links

External links5618_0 5618_1 5618_0

5325 5325 MSC_0 MSC_n

RP0 RP1

CPU SP CPU SP

CPU CPU

5618_1

External links

External links5618_0 5618_1 5618_0

5325 5325 MSC_0 MSC_n

RP0 RP1

CPU SP CPU SP

CPU CPU

5618_1

External links

External links5618_0 5618_1 5618_0

5325 5325 MSC_0 MSC_n

RP0 RP1

CPU SP CPU SP

CPU CPU

5618_1

External links

External links


SP (Service Processor) Broadcom 5325

• 6 port 100Mb Fast Ethernet Switch

Port 0 MII port connects to SP CPU

Port 1 connects to RP0 Broadcom switch

Port 2 connects to RP1 Broadcom switch

Port 3 connects to node CPU0

Port 4 connects to node CPU1 (for DRP)

Port 5 is unused today


16 slot RP Broadcom 5618 switch 0

show controllers switch 0 stats location <loc> Port Tx Frames Tx Errors Rx Frames Rx Errors Connects ---------------------------------------------------------------- 1 : 732969 0 845275 11 0/RP0 2 : 600074 0 935824 3 0/RP1 3 : 92628 0 279172 5 0/FC0 4 : 92756 0 278427 4 0/FC1 5 : 98560 0 282167 5 0/AM0 6 : 99080 0 282157 5 0/AM1 7 : 2 0 0 0 <<< unused 8 : 2 0 0 0 <<< unused 9 : 168844 0 327210 5 0/SM010 : 168833 0 327561 5 0/SM111 : 169411 0 328545 5 0/SM212 : 168675 0 326775 5 0/SM313 : 168817 0 326859 5 0/SM414 : 169669 0 327919 5 0/SM515 : 169470 0 327666 5 0/SM616 : 169160 0 327636 5 0/SM725 : 166958 0 0 0 GE_0 <<< external GE port 0 26 : 3809516 0 3328905 0 Switch 1 <<< GE stacking link to switch 1



show controllers switch 1 stats location <loc>

Port Tx Frames Tx Errors Rx Frames Rx Errors Connects

----------------------------------------------------------------

1 : 447070 0 642441 3 0/LC0

2 : 0 0 0 0 0/LC1

3 : 449393 0 645699 3 0/LC2

4 : 0 0 0 0 0/LC3

5 : 460650 0 655972 3 0/LC4

6 : 0 0 0 0 0/LC5

7 : 0 0 0 0 0/LC6

8 : 0 0 0 0 0/LC7

9 : 0 0 0 0 0/LC8

10 : 0 0 0 0 0/LC9

11 : 0 0 0 0 0/LC10

12 : 0 0 0 0 0/LC11

13 : 0 0 0 0 0/LC12

14 : 0 0 0 0 0/LC13

15 : 1016475 0 335516 3 0/LC14

16 : 879538 0 280385 5 0/LC15

25 : 166958 0 0 0 GE_1 <<< external GE port 1 26 : 3399368 0 3874254 0 Switch 0 <<< GE stacking link to switch 0



show controllers switch 0 stats

Ports Active on Switch 0

FE Port 1 STP State : FORWARDING (Connected to - 0/RP0)

FE Port 2 STP State : FORWARDING (Connected to - 0/RP1)

FE Port 3 STP State : FORWARDING (Connected to - 0/SM0)




FE Port 9 STP State : FORWARDING (Connected to - 0/LC0)





Infrastructure IP interfacessh controllers backplane ethernet detail location 0/0/SP

FastEthernet0_0_SP is up Hardware is 10/100 Ethernet, H/W address is 5246.4800.0211 Internet address is 10.0.2.17 MTU 1514 bytes Encapsulation HFRIES (HFR Internal Ethernet Server) Mode : Full Duplex, Rate : 100Mb/s 120428671 packets input, 2586226503 bytes, 0 total input drops 0 packets discarded (0 bytes) in garbage collection 1 packets discarded (198 bytes) in recv processing 0 incomplete frames discarded 0 packets discarded due to bad headers 2 packets waiting for clients 0 packets waiting on Rx Received 21 broadcast packets, 18818982 multicast packets Input errors: 0 CRC, 0 overrun, 0 alignment, 0 length, 0 collision 122253704 packets output, 3854643928 bytes, 0 total output drops Output 19160687 broadcast packets, 19160687 multicast packets Output errors: 0 underruns, 0 aborts, 0 loss of carrier Write rejects : 0

sh controllers backplane ethernet detail location 0/0/CPU0FastEthernet0_0_CPU0 is up Hardware is 10/100 Ethernet, H/W address is 5246.4800.0001 Internet address is 10.0.0.1 MTU 1514 bytes Encapsulation HFRIES (HFR Internal Ethernet Server) Mode : Full Duplex, Rate : 100Mb/s 23439652 packets input, 1666743902 bytes, 0 total input drops 205985 packets discarded (13595010 bytes) in garbage collection 0 packets discarded (0 bytes) in recv processing 0 incomplete frames discarded 0 packets discarded due to bad headers 4 packets waiting for clients 0 packets waiting on Rx Received 19160257 broadcast packets, 1591253 multicast packets Input errors: 0 CRC, 0 overrun, 0 alignment, 0 length, 0 collision 3729907 packets output, 363479959 bytes, 0 total output drops Output 1 broadcast packets, 1 multicast packets Output errors: 0 underruns, 0 aborts, 0 loss of carrier Write rejects : 0


PLIMs and MSCs


PLIM• 4-Port OC-192c/STM-64c POS/DPT

• 16-Port OC-48c/STM-16c POS/DPT

• OC768c/STM256 POS

• OC768c/STM256 POS tunable WDMPOS

• 8x10GE (XENPACK optics inc DWDM options)

• 4x10GE tunable WDMPHY

• SPA jacket

Line Card• Adaptive Packet processor

80Mpps Forwarding capability (IPv4)

Hardware based prefix lookup for (IPv4/IPv6/MPLS)

Flexible microcoded massively parallel forwarding engine with extensive queuing capability

8K Queues ingress/egress

Configurable queue mapping

• 800 Mhz PPC

• 2GB CPU memory/2GB packet memory

CRS-1 PLIM and Line Card Overview


CRS-1 PLIM and Line Card Overview

FromFabric

FabricQReass.

MIDPLANE

IngressQIn Q/

Segmenter

RX PSEL3 Engine

CPUCTRLGW

CPU

FabricQReass.

TX PSEL3 Engine

EgressQOut Q

OC192Framer

& Optics

ToFabric

FromPLIM

ToPLIM

PLA OC192Framer

& Optics

OC192Framer

& Optics

OC192Framer

& Optics

Ingress Packet Flow

Egress Packet Flow

123

5 6 7

8

4


CRS-1 Line Card Picture

Line Card CPU

Egress PSE

Ingress PSE

IngressQ

Power Regulators

SP

2*FabricQ

EgressQ


MIDPLANE

PLAPLIM I/F

OC192Framer

OC192Optics

OC192Optics

OC192Framer

OC192Framer

OC192Optics

OC192Optics

OC192Framer

PLA – L2 ASIC• Some L2 Statistics Gathering• 10GE MAC

• Congestion Control• Resolves 10GE over subscription• Consolidation of port streams for RX

PSE• Stream separation on TX• Ingress monitoring RX PSE - Buffers

for congestion • Exact PLA variant and number of

PLAs varies from PLIM to PLIM

CRS-1 Packet FlowPacket enter and exit thru PLIM Asic


4xOC192 PLIM POS/DPT HW Architecture

Line card 4xoc192 PLIM

EgressQ

RX PSE

PLA 0

PLA 1

RAC0

RAC1

Framer 0

Framer 1

RAC2

RAC2

Framer 2

Framer 3


16xOC48 PLIM POS/DPT HW Architecture

Line card 16xoc48 PLIM

EgressQ

RX PSE

PLA 0

PLA 3

RAC0

RAC1

Framer 0

Framer 1

RAC14

RAC15

Framer 14

Framer 15

RAC2 Framer 2

RAC3 Framer 3

RAC12 Framer 12

RAC13 Framer 13


8x10GE PLIM HW Architecture

Line card 8x10GE PLIM

EgressQ

RX PSE

PLA 0

PLA 1

PHY0

PHY1

Optics 0

Optics 1

PHY6

PHY7

Optics 6

Optics 7

PHY2 Optics 2

PHY3 Optics 3

PHY4 Optics 4

PHY5 Optics 5


1xOC768 PLIM HW Architecture

EgressQ

RX PSE

PLA768

Line card 1xOC768 PLIM

Framer


1xOC768 PLIM


CRS-1 Line Card OverviewWDMPOS/WDMPHY PLIMs

• WANPOS OC768 bandwidth-compressed signal on 50GHz grid, C-Band tunable, 50GHz spacing

40G over existing 10G DWDM systems

• WANPHY PLIM uses ONS technology providing four 10GE C-band tunable, 50GHz spacing interfaces


SIPs and SPAs

• A SIP is a physical layer interface module (PLIM) that inserts into a line card chassis slot like any other PLIM. It provides no network connectivity on its own

• MSC is connected to the SIP in the same manner as the current ‘Fixed’ PLIMs

• CRS-1 SIP will house up to 6 Shared Port Adapters (SPA) which are installed in ‘subslots’. The SPA provides the interface ports and media type

• SPAs may be single height or double-height depending on the required accommodation


SIPs and SPAs

SPA Bay 3 SPA Bay 4 SPA Bay 5

SPA Bay 0 SPA Bay 1 SPA Bay 2

Double HeightSPA Bay 3

SPA Bay 4

SPA Bay 1

SPA Bay 5

SPA Bay 2





SIPs and SPAs

• Release 3.2.x supports:1-Port OC-192c/STM-64c POS XFP SPA

8-Port Gigabit Ethernet SPA

4-Port OC-3c/STM-1 POS SPA

• Release 3.3.x supports:8-Port OC-3c/STM-1 OC-12c/STM-4 switchable POS SPA

• Release 3.4.x supports:5-Port Gigabit Ethernet Shared Port Adapter, Version 2

8-Port Gigabit Ethernet Shared Port Adapter, Version 2

10-Port Gigabit Ethernet Shared Port Adapter, Version 2

1-Port Ten Gigabit Ethernet Shared Port Adapter, Version 2

2-Port / 4-Port OC48c/STM-16c POS/RPR Shared Port Adapter

2-Port / 4-Port T3/E3 Shared Port Adapter (3.4.1)

1-Port OC-192c/STM-64c POS VSR (fixed optics) SPA (3.4.1)

• MSC ‘Oversubscription’ is supported for Ethernet SPAs onlyOversubscription is NOT supported if there are ANY POS SPAs present in the SIP

• More SPA variants to be supported in future releases


SPA ‘Jacket Card’ PLIM and SPA Modules


SPA Jacket card Architecture

Line card SPA Jacket Card

EgressQ

RX PSE

PLA 0

PLA 1

SPA0

SPA1

SPA5

SPA3

SPA2

SPA4


MIDPLANE

RX PSEL3 Engine

PLAPLIM I/F

OC192Framer

OC192Optics

OC192Optics

OC192Framer

OC192Framer

OC192Optics

OC192Optics

OC192Framer

FromPLIM

RX PSE (Metro)• Flexible microcoded massively

parallel forwarding engine• Prefix Lookup

(V4/V6/Unicast/Multicast/MPLS)• L2/L3 Statistics/Traffic Monitoring• ACL• Netflow• QoS (Policing, WRED)

CRS-1 Packet FlowPacket sent into Ingress PSE


PLU TLU STATS

DISTRIB MUX

PPE1

PPE2

PPE188

TCAM

• 188 Packet Processing Engines (PPE)

Packets evenly distributed across PPEs

Each PPE is a 32-bit RISC processor @250MHz

• PSE is implemented on both RX and TX sides

CRS-1 Packet FlowPSE (PSE) Architecture

Prefix lookup Table lookup


MIDPLANE

IngressQIn Q/

SegmenterRX PSE

L3 Engine

ToFabric

FromPLIM

IngressQ (Sprayer)• 1 IngressQ/Linecard• Ingress Queuing and Shaping• 8K queues mapped to

• 1K ports (VLANs, DS3)• Ingress rate shaping• Packet segmentation into cells

• 136 byte cell• 120 byte payload

• Classification into hi/lo fabric priorities• Cells evenly (pseudo RR) distributed to

S1 devices in the chassis

Line Card PLIM

CRS-1 Packet FlowPackets sent from ingress PSE to IngressQ to fabric


CRS-1 Packet FlowInput queuing/shaping on the IngressQ

ConfigurableDynamicMapping ofQueues to ports

To fabric

MDRR/AggregateShaper

ShapingMax BW

Max BW

8192 Queues 1K Ports (Port/VLAN)HPLP

LP

MDRR/Shaper P1

MDRR/Shaper

MDRR/Shaper

P2

P1023

WRED

FromRX PSE

PSE IngressQ


CRS-1 Packet FlowCell Packing

• Packets are inserted into cells on ‘quadoct’ boundaries• Each packet has an Buffer Header (bhdr) appended prior to

segmentation. IPv4 header is typically 8 bytes. IPv6 header is typically 10 bytes.

• Multiple packets can be ‘packed’ into a cell IF they share the same fabric destination & priority

• Multicast packets cannot be ‘packed’

Fields (Data Cells) Bits

Cast (Mcast/Ucast) 1

Fabric Priority 1

Vital 1

Source Address 10

Packet 1 Sequence Number 13

Packet 2 Payload Offset/FGID 2

Trailing Offest 2

Packet Cell Count 7

Packet 1 Cell Sequence Number 7

Destination Address (12 bits)/FGID 18

Snoop/CTB 1

Internal Fabric control bits 32


S1

S1

S2

S2 S3

S3

S1

S1

S2

S2 S3

S3

50 Gb/LC(1) -->

100 Gb/LC(2) -> (2.5X Speedup )

S1

S1

S2

S2 S3

S3

1 of 8

2 of 8

8 of 8Line Card

2

1

8

136 Bytes cells

Multi-stage Interconnect – 3 Stage Benes topology - buffered non-blocking switch

Line Card2

1

16

Fabric access

2 Levels of priorityHP Low latency path

LP Best effort traffic

Multicast support1M multicast groups

40 Gb

(1) Actually 64Gb total ~ 50 Gb remain for data (2) Actually 128Gb total ~ 100 Gb remain for data


FabricQComplex

MIDPLANE

FabricQComplex

TX PSEL3 Engine

EgressQOut Q

FromFabric

ToPLIM

FabricQ (Sponge) Complex• Consists of 2 FabricQ ASICs• Cell -> Packet conversion• Packet Reassembly• Cell and Packet Resequencing

CRS-1 Packet FlowPackets received from fabric into FabricQ on Egress LC


CRS-1 Packet FlowFabricQ Re-assembly overview

• FabricQ uses several cell header fields to re-assembly

– Source Address

– Cast (MC/UC)

– Packet Sequence #

– Cell Sequence #

– Total Cell Count

• Basic algorithm:

– Re-assemble the packet utilizing cell count and cell sequence #.

– Re-sequence flow based on source address and packet sequence# and MC/UC bit


FabricQComplex

MIDPLANE

FabricQComplex

TX PSEL3 Engine

EgressQOut Q

FromFabric

ToPLIM

TX PSE (Metro)• Prefix Lookup and treatment

(V4/V6/Unicast/Multicast/MPLS)• L2/L3 Statistics/Traffic Monitoring• ACL• Netflow• QoS

CRS-1 Packet FlowPackets sent from FabricQ to Egress PSE


FabricQComplex

MIDPLANE

FabricComplex

TX PSEL3 Engine

EgressQOut Q

FromFabric

ToPLIM

EgressQ (Sharq)• 1 EgressQ/Line Card• Egress Queuing and Shaping• 8K queues mapped to

• 2K groups (VLAN, VPN, tunnels), mapped to 768 ports

• Egress rate shaping• 1GB Packet Memory

CRS-1 Packet FlowPackets sent from Egress PSE to EgressQ


2K Groups(VLAN/Tunnel)

CRS-1 Packet FlowOutput queuing/shaping on the EgressQ

To PLIM

Configurable DynamicMapping ofQueues to groups and ports

ShapingMin/max BW

8192 Queues 768 Ports (Port/VLAN)

HPLP

LP P1

PX

WRED

MDRR/Shaper G1

MDRR/Shaper

MDRR/Shaper

G2

GX

MDRR/Group

MDRR/Group

MDRR/AggregateGroup

Max BW

fromTX PSE


FabricQComplex

MIDPLANE

IngressQIn Q/

Segmenter

RX PSEL3 Engine

CPUCTRLGW

CPU

FabricQComplex

TX PSEL3 Engine

EgressQOut Q

ToFabric

FromFabric

FromPLIM

ToPLIM

CPUCtrl (Squid)• 1 CPUCtrl/Linecard• Mediates all traffic needing to go

to the LC CPU• Connects to all devices on the LC

and the PLIM• Provides memory interfaces,

queueing, DMA, etc for control plane traffic

• Assists CPU in FIB updates and other maintenance items

CRS-1 Packet FlowNetwork Control traffic sent to LC CPU

1 © 2007 cisco systems, inc. all rights reserved. edcs-412962 advanced crs/ios-xr training crs-1...

Documents