paris academic network

Juniper R&E Summit Zagreb 19/05/2003

- Paris Academic Network -High speed Metropolitan Area Network for

research and educationalJean-Paul GAUTIER – Laurent GYDE

www.rap.prd.fr


AgendaAgenda

�� Network designNetwork design

�� JuniperJuniper M5 , M5 , knownknown as as gwgw--raprap : a : a leadingleading rolerole


FortyForty partnerspartners

��TheThe eighteight UniversitiesUniversities in Paris.in Paris.–– lawlaw, , economicseconomics, arts, , arts, medecinemedecine, sciences, social sciences ..., sciences, social sciences ...

��CNRS.CNRS.–– National National CenterCenter for for ScientificScientific ResearchResearch ((governmentgovernment agencyagency).).

��INSERM.INSERM.–– National National InstituteInstitute for for HealthHealth andand MedecinalMedecinal ResearchResearch ((governmentgovernment agencyagency).).

��HighHigh educationeducation..–– engineersengineers schoolsschools, , economicseconomics schoolsschools, , includingincluding somesome sciences sciences museumsmuseums..

��MinistryMinistry of Education of Education andand ResearchResearch..

todaytoday network network usersusersstudentsstudents : 300.000.: 300.000.researchersresearchers, , teachersteachers, , technicaltechnical andand administrative support : 40.000.administrative support : 40.000.


TheThe aimsaims

�� ReduceReduce the global network cost.the global network cost.

�Deploy a new network architecture providing high speed connections to a hundred sites.

�Satisfy the new needs.–multimedia, video-*, imagery, telephony, metacomputing.

�Integrate :–into the high speed regional network which will interconnect MANs. –into Renater 2

»the national research network which offers IP services, ATM services with QoS, VPN ...

–Into international projects using the international links of Renater3»GEANT in Europe.»to STARTAP in US.

�� A network «A network « for for andand withwith thethe communitycommunity »»


Global ArchitectureGlobal Architecture

DWDM

TransportInfrastructure

ServiceInfrastructure

Services

ATM

SDH/SONET

Ethernet

IP

Data, voice, vidéo, multimédia, VPN …


Infrastructure Infrastructure andand Technologies: Technologies: DWDM, ATM, GigabitDWDM, ATM, Gigabit……

��InternetworkingInternetworking services services providedprovided to the sites.to the sites.–– IP, Ethernet, ATM on IP, Ethernet, ATM on opticaloptical links.links.

OpticalNetworking

Service LayerNetworking(IP, ATM)

optical networking element(OADM, OXC)

��TheThe corecore of RAP : an of RAP : an opticaloptical DWDM ring for DWDM ring for thethePoPs network.PoPs network.

»»Dense Dense WavelengthWavelength Division Division MultiplexingMultiplexing»»OpticalOptical protection by protection by channelchannel (Och)(Och)»»cohabitation cohabitation

��of multiple networks (technologies, of multiple networks (technologies, protocolsprotocols ...)....).��of of dedicateddedicated networks (QoS, networks (QoS, securitysecurity, , thematicthematic).).��withoutwithout interactions interactions betweenbetween operationaloperational andand experimentalexperimentalnetworks.networks.

»»flexibilyflexibily andand evolutivityevolutivity��wavelenghtswavelenghts for new for new projectsprojects, new , new protocolsprotocols..��minimizeminimize service disruption service disruption whenwhen addingadding new new featuresfeatures..

»»readyready for future technologies (IP on for future technologies (IP on fiberfiber…)…) ..

��FiveFive Points of Points of PresencePresence ((PoPsPoPs).).

��A single mode A single mode opticaloptical fiberfiber infrastructure.infrastructure.


�� FiveFive PoPPoP : 99 sites : 99 sites »» Jussieu : 27 sitesJussieu : 27 sites»» OdOdééon : 34 siteson : 34 sites»» Auteuil : 15 sitesAuteuil : 15 sites»» Malesherbes : 10 sitesMalesherbes : 10 sites»» CNAM : 13 sitesCNAM : 13 sites

�� WirelessWireless local local looploop–– 22 22 smallsmall sites sites àà 2 2

Mbs.Mbs.

�� DarkDark fiberfiber : Single Mode G652: Single Mode G652»» 69 sites (100 69 sites (100 MbsMbs or 155 or 155 MbsMbs))»» Total Total lengthlength 346.1 km (links) 346.1 km (links)

�� SubwaySubway : 312.3 km : 312.3 km �� SewersSewers : 33.5 km: 33.5 km�� Civil Civil engeeneringengeenering: 0.3 km.: 0.3 km.

»» ShortestShortest linklink : 1 km.: 1 km.»» LongestLongest linklink : 9.6 km.: 9.6 km.»» No No redundancyredundancy for for thethe fiberfiber itselfitself

�� HertzianHertzian beamsbeams–– 3 sites 3 sites nearnear ParisParis–– 34 Mbit/s 34 Mbit/s andand 155 Mbit/s155 Mbit/s

PhysicalPhysical characteristicscharacteristics


WW

W

WW

E

E

EE

dwdmdwdm

E

s-atm

crcr

dwdmdwdm

dwdmdwdm

dwdmdwdm

dwdmdwdm

crcr

crcr

crcr

crcrss--atmatm

ss--atmatm

ss--atmatmss--atmatm

Site Site AA

Site Site FF

Site Site BB

Site Site DD

Site Site CC

Site Site GG

c

c

NRD

ATM

IP

Eth 100

Gigabit Ethernet

ATM OC12

ATM OC12

GigaEthernet

2 2 fiberfiber G6552G6552

5082 m

8672 m

8410 m

3100 m

5120 m

10 10 wavelenght20 20 opticaloptical virtualvirtual networksnetworks

SPSP

ATM OC3

Eth 100

TheThe physicalphysical architecturearchitecture


JuniperJuniper M5 configurationM5 configuration

�� SinceSince octoberoctober20012001

PIC ATM OC-12Renater

PIC GigaEthernet MMBackbone RAP

PIC TUNNELPIM/SM

Junos 5.6IPv6


��IPv4 routingIPv4 routing–– UnicastUnicast–– MulticastMulticast

��Traffic handlingTraffic handling–– rate limitingrate limiting–– filteringfiltering–– accountingaccounting

��MetrologyMetrology–– SNMPSNMP–– CFLOWCFLOW

��a few words about 6RAP projecta few words about 6RAP project

JuniperJuniper M5 , M5 , knownknown as as gwgw--raprap : a : a leadingleading rolerole


5 Extreme NetworksBlackDiamond 6808 Renater/

Internet

Juniper NetworksM5

Cisco 12008

ATM over0C12 links

GigaEthernet

Sites

GigaEth

FastEth/OC3

Lower rates(2 or 4 Mbit/s)

Main way to RAP ring

Backup way to RAP ring

gwgw--rap positioning : IPv4 perspectiverap positioning : IPv4 perspective


��Static routing between sites and RAPStatic routing between sites and RAP

��OSPF routing as RAP IGPOSPF routing as RAP IGP–– only one area (area 0)only one area (area 0)–– IP level redundancy at a low latencyIP level redundancy at a low latency–– default route generated by default route generated by gwgw--raprap

��BGP routing with BGP routing with RenaterRenater/Internet/Internet–– full routing (more than 125000 routes)full routing (more than 125000 routes)–– sites routes announcement (approx. 250 routes)sites routes announcement (approx. 250 routes)

��BGP with some dual homed sitesBGP with some dual homed sites–– EBGP (EBGP (multihopmultihop if necessary)if necessary)–– routes announcements depending on sites requirements (by a routes announcements depending on sites requirements (by a

selection using the ASselection using the AS--PATH)PATH)–– making redundancy available between RAP and some other ISPmaking redundancy available between RAP and some other ISP

gwgw--rap : unicast IPv4 routingrap : unicast IPv4 routing



Internet

Juniper NetworksM5

Cisco 12008

Static routing

OSPF area 0

Main way to RAP ring

Backup way to RAP ring

BGP full routing

E-BGP multihopfor specific sites

requirements (multi-homing)

gwgw--rap : unicast IPv4 routingrap : unicast IPv4 routing


bgpbgp {{advertiseadvertise--inactiveinactive;;familyfamily inetinet {{

anyany;;}}group RENATER {group RENATER {

type type externalexternal;;authenticationauthentication--keykey «« xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx »»; # SECRET; # SECRET--DATADATA

export BGPexport BGP--FILTERFILTER--RENATERRENATER--OUTOUT ;;

peerpeer--as 2200;as 2200;neighborneighbor 193.51.182.202;193.51.182.202;

}}group BGPgroup BGP--GROUPGROUP--G06 {G06 {

type type externalexternal;;passive;passive;import BGPimport BGP--FILTERFILTER--G06G06--IN;IN;export BGPexport BGP--FILTERFILTER--G06G06--OUT;OUT;peerpeer--asas 1712;1712;neighborneighbor 195.221.126.102 {195.221.126.102 {

multihopmultihop;;locallocal--addressaddress 193.50.20.79;193.50.20.79;

}}}}

}}

gwgw--rap : unicast IPv4 routing, BGP conf.rap : unicast IPv4 routing, BGP conf.


policypolicy--optionsoptions {{……

prefixprefix--listlist reseauxreseaux--3.03 {3.03 {193.55.108.0/24;193.55.108.0/24;194.199.121.0/24;194.199.121.0/24;194.199.122.0/24;194.199.122.0/24;195.221.77.0/24;195.221.77.0/24;195.221.79.0/24;195.221.79.0/24;195.221.80.0/24;195.221.80.0/24;

}}……

policypolicy--statementstatement BGPBGP--FILTERFILTER--RENATERRENATER--OUT {OUT {term 1 {term 1 {

fromfrom {{......

prefixprefix--listlist reseauxreseaux--3.03;3.03;......

}}thenthen acceptaccept;;

}}termterm 2 {2 {

thenthen rejectreject;;}}

}}



policypolicy--statementstatement BGPBGP--FILTERFILTER--G06G06--OUT {OUT {termterm 1 {1 {

fromfrom asas--pathpath BGPBGP--ASAS--PATHPATH--G06G06--OUT;OUT;thenthen rejectreject;;}}

termterm 2 {2 {thenthen acceptaccept;;}}

}}policypolicy--statementstatement BGPBGP--FILTERFILTER--G06G06--IN {IN {

termterm 1 {1 {fromfrom asas--pathpath BGPBGP--ASAS--PATHPATH--G06G06--IN;IN;thenthen nextnext termterm;;}}

termterm 2 {2 {fromfrom {{prefixprefix--listlist reseauxreseaux--G06;G06;}}thenthen acceptaccept;;

}}termterm 3 {3 {

thenthen rejectreject;;}}

}}

asas--pathpath BGPBGP--ASAS--PATHPATH--G06G06--OUT 2200.*;OUT 2200.*;asas--pathpath BGPBGP--ASAS--PATHPATH--G06G06--IN 1712.*;IN 1712.*;



��One unique RAP PIMOne unique RAP PIM--SM domain available for transit and simplified SM domain available for transit and simplified multicast site connectionsmulticast site connections

��BSR mechanisms used in RAP PIM domain for RP selectionBSR mechanisms used in RAP PIM domain for RP selection

��Multicast routing with Multicast routing with RenaterRenater/Internet uses :/Internet uses :–– BGP for multicast routes announcements (for MSDP and PIM RPF cheBGP for multicast routes announcements (for MSDP and PIM RPF check)ck)–– MSDP for multicast sources announcementsMSDP for multicast sources announcements–– PIM for the multicast routing itself (as a border router)PIM for the multicast routing itself (as a border router)

��Separate PIM domains for sites are allowed withSeparate PIM domains for sites are allowed with–– PIM border router configuration on the site router facing RAPPIM border router configuration on the site router facing RAP–– MSDP between the site and RAP PIM domain (MSDP between the site and RAP PIM domain (gwgw--rap)rap)–– MultiMulti--sites organisms may build independent PIM infrastructures by depsites organisms may build independent PIM infrastructures by deployingloying

PIM + MSDP over interPIM + MSDP over inter--sites layer 2 VPN (usually VLANs over RAP backbone)sites layer 2 VPN (usually VLANs over RAP backbone)

gwgw--rap : multicast IPv4 routingrap : multicast IPv4 routing



Internet

Juniper NetworksM5

Cisco 12008

BSR+RP

(except for 239.0.0.0/8)

PIM-SM

M-BGP + MSDPwith Renater

gwgw--rap : multicast IPv4 routingrap : multicast IPv4 routingRAP PIM domainRAP PIM domain



Internet

Juniper NetworksM5

Cisco 12008

PIM-SM withPIM border-routeron the site router

PIM-SM

M-BGP + MSDPwith Renater

MSDP withsites deploying a

separate PIM domain

BSR+RP

(except for 239.0.0.0/8)

gwgw--rap : multicast IPv4 routingrap : multicast IPv4 routingmultiple PIM domainsmultiple PIM domains


��general traffic filters general traffic filters –– antianti--spoofingspoofing

»» exhaustive list of inside RAP IP networksexhaustive list of inside RAP IP networks»» unicastunicast RPF filters planned soonRPF filters planned soon

–– RAP backbone protectionRAP backbone protection

��rate limitingrate limiting–– to respect maximum rate agreed with to respect maximum rate agreed with RenaterRenater (400 (400 Mbit/sMbit/s at this time)at this time)–– to enforce DOS protectionto enforce DOS protection–– to limit some specific traffics (ICMP, multicast, to limit some specific traffics (ICMP, multicast, ……))–– as a service for allowing organism to respect their own as a service for allowing organism to respect their own RenaterRenater rate agreement.rate agreement.

gwgw--rap : traffic handling and metrologyrap : traffic handling and metrology


��general bandwidth usage measurement (also done on the other general bandwidth usage measurement (also done on the other equipments)equipments)

–– Input/Output traffic on each interface (backbone and site)Input/Output traffic on each interface (backbone and site)–– SNMP + MRTGSNMP + MRTG

��rates volume with rates volume with RenaterRenater/Internet for each organism on RAP/Internet for each organism on RAP–– used by used by RenaterRenater for organisms invoicingfor organisms invoicing–– traffic is measured for each IP network to and from traffic is measured for each IP network to and from RenaterRenater/Internet/Internet–– the whole is shared out among the organisms (remember : there arthe whole is shared out among the organisms (remember : there are some multie some multi--sites sites

organisms and some multiorganisms and some multi--organisms sites)organisms sites)»» CFLOWD + CFLOWD + NetMetNetMet

��specific traffics observationspecific traffics observation–– configuration of meters on firewall filtersconfiguration of meters on firewall filters–– viewable in the CLI (for debugging) or via SNMP (for metrology)viewable in the CLI (for debugging) or via SNMP (for metrology)–– SCU/DCU (Source Class Usage/Destination Class Usage) planned sooSCU/DCU (Source Class Usage/Destination Class Usage) planned soonn

»» gives the ability to know bandwidth usage from/to remarkable netgives the ability to know bandwidth usage from/to remarkable networks works (universities, research networks, countries, (universities, research networks, countries, ……) )



Renater/Internet

CFLOW

CFLOWD collector« NETMET »

(see www.netmet-solutions.org)

MRTG(see www.mrtg.com)

SNMP

Traffic handling

- filtering- rate-limiting- measuring- accounting

RAP network: 40 organisms / 99 sites / 340000 users

120 Mbit/s bidir.average

bandwidth usage



…prefix-list reseaux-3.03 {

193.55.108.0/24;194.199.121.0/24;194.199.122.0/24;195.221.77.0/24;195.221.79.0/24;195.221.80.0/24;

}…filter renater-out {

policer rate-limit-Renater {if-exceeding {

bandwidth-limit 362m;burst-size-limit 16m;

}then discard;

}term count-multicast-renater-out {

from {destination-prefix-list {

reseaux-multicast;}

}then {

count multicast-renater-out;next term;

}

term accept-prefix-list-renater-out {from {

source-prefix-list {...

reseaux-3.03;...

}}then {

policer rate-limit-Renater;accept;

}}

*****************************************************************

admin@gw-rap> show firewall filter renater-out Filter: renater-out

multicast-renater-out148173 packets 206083162

bytescompteur-icmp-renater-out

50400996 packets 2987115159 bytes

rate-limit-Renater0 packets



Renater/Internet

Cisco 12008

BGP

3 IPv6 routers(evaluations in progress)

gwgw--rap acts for IPv6 rap acts for IPv6 unicastunicast routing as it does for IPv4routing as it does for IPv4IPv6 backbone has its own gigabit/s lambdas over the optical rinIPv6 backbone has its own gigabit/s lambdas over the optical ringg

RIPng

Traffic handling

- filtering- rate-limiting- measuring- accounting

MRTG(see www.mrtg.com)

SNM

P

6RAP project : gw6RAP project : gw--rap positioningrap positioning


��gwgw--rap is well controlled, thanks to JUNOS configuration rap is well controlled, thanks to JUNOS configuration modemode–– the NOC prepares the configuration (e.g. new subthe NOC prepares the configuration (e.g. new sub--interface + routing + interface + routing +

filtering)filtering)–– the whole is activated at one time by the COMMIT orderthe whole is activated at one time by the COMMIT order–– if necessary, man can return to (one of) the previous if necessary, man can return to (one of) the previous state(sstate(s) manually ) manually

((““rollbackrollback””) or automatically () or automatically (““commit confirmedcommit confirmed””))

��gwgw--rap is finally very adaptablerap is finally very adaptable–– accept any connection from 2 accept any connection from 2 Mbit/sMbit/s to 1 to 1 Gbit/sGbit/s with the same functionalitieswith the same functionalities

(routing, filtering, rate (routing, filtering, rate limiting, limiting, ……))–– allow nevertheless easy punctual configurations if neededallow nevertheless easy punctual configurations if needed

ConclusionConclusion


-Thank you for your attention

Jean-Paul GAUTIER ([email protected])Laurent GYDE ([email protected])

www.rap.prd.fr only in french

paris academic network

Documents