Conference HighlightsConference Highlights

David WilliamsDavid Williams

TERENA Networking ConferenceTERENA Networking Conference

Thursday 8 October 1998Thursday 8 October 1998

David.O.Williams@cern.chDavid.O.Williams@cern.ch

Slides: http://nicewww.cern.ch/~davidw/public/TERENA.pptSlides: http://nicewww.cern.ch/~davidw/public/TERENA.ppt

Top level outlineTop level outline

Report on ICFA-NTFReport on ICFA-NTF Personal view of futurePersonal view of future SummarySummary

AcknowledgementsAcknowledgements

The ICFA-NTF report describes the work of The ICFA-NTF report describes the work of 31 people from 21 different institutions (of 31 people from 21 different institutions (of which 8 universities) in 11 countries which 8 universities) in 11 countries spread over three continentsspread over three continents

The people who provide the networks The people who provide the networks which we use (inside institutions, which we use (inside institutions, regionally, nationally, internationally etc.)regionally, nationally, internationally etc.)

The “real users” who took time to answer The “real users” who took time to answer our questions and provide commentsour questions and provide comments

MonitoringMonitoring

Packet Loss (1/2)Packet Loss (1/2)

Packets are discarded when the router buffers fill up Packets are discarded when the router buffers fill up and overflowand overflow

When packet flow from input link(s) exceeds the When packet flow from input link(s) exceeds the capacity of the output link(s)capacity of the output link(s)

Especially at ends of (expensive) transoceanic linksEspecially at ends of (expensive) transoceanic links Packet loss rates (PLRs) are about the best approx-Packet loss rates (PLRs) are about the best approx-

imation to what the end-user feels is the overall imation to what the end-user feels is the overall “quality” being provided by a link“quality” being provided by a link

But note that PLR measurements don’t tell the But note that PLR measurements don’t tell the network engineer much about the nature and the network engineer much about the nature and the location of any congestion that is present on a link location of any congestion that is present on a link (so not widely used by them).(so not widely used by them).

Packet Loss (2/2)Packet Loss (2/2)

Different applications stop working at Different applications stop working at different PLRs. E-mail “always” works. The different PLRs. E-mail “always” works. The more a session is interactive (telnet, X) or more a session is interactive (telnet, X) or depends on delivery of many packets without depends on delivery of many packets without much jitter (audio, video) the lower the much jitter (audio, video) the lower the acceptable PLR.acceptable PLR.

As a general rule of thumb:-As a general rule of thumb:-<1% is excellent 1-2.5% is good 2.5-5% is ~OK

5-12% is poor >12% is unusable

The intercontinental issueThe intercontinental issue

Quite detailed history is availableby clicking on each entry

SatSat SunSunSunSun

MonMon

TueTue

WedWed

ThursThurs

FriFri

Daily packet loss structure on a congested route

50% peaks 8 quiet hours at nightFrom ~01.00 to 09.00 CET

Close to (national) perfectionClose to (national) perfection

Most of the following data is for the seven day Most of the following data is for the seven day period 13-19 August 1998period 13-19 August 1998

Because most students are on holiday and there Because most students are on holiday and there have been some recent capacity upgrades, have been some recent capacity upgrades, there is less “daily” congestion than “normal” at there is less “daily” congestion than “normal” at the moment, especially on the transatlantic the moment, especially on the transatlantic links. Things will probably start to get worse in links. Things will probably start to get worse in September.September.

Fermi to SLAC

No packet loss~55 msec RTT~2000 miles?

DESY to Dresden

No packet loss25-30 msec RTT, with bumps, a few to 100 msec

~500 km

Close to (international) Close to (international) perfectionperfection

CERN to NBI Copenhagen

13/3360 packets lost60-90 msec RTT, with regular jitter

~1000 km

SLAC to KEK

1/3360 packets lost180-250 msec, some jitter

~10,000 km

CERN to Osaka, standard dates

23/3360 packets lost330-400 msec RTT.

~20,000 km (via USA)

Some problem casesSome problem cases

NationalNational

CMU (Pittsburgh) to Cincinnati

Sat Sun

10.37% packet loss over month of July50-100 msec RTT, with heavy daily congestion

400 miles??

No data available for standard dates

More problem casesMore problem cases

InternationalInternational

Fermi to Madrid

Classic congestionBroken

Monday

Longer term plotsLonger term plots

Some longer term plots of Some longer term plots of difficult casesdifficult cases

Period is normally 3 months up to 20 Period is normally 3 months up to 20 AugustAugust

NationalNational

Fermi to Brown (15 April to 25 August)

Packet loss 16.15% in April, 0.05% in July. They changed their ISP!!

??

CNAF Bologna to Perugia

Packet loss only 3-4%, but RTT goes wild

Some longer term plots of Some longer term plots of difficult casesdifficult cases

InternationalInternational

CMU to DESY

Packet loss ~15%

CERN to ITEP Moscow (month of July)

Packet loss only 3.44%, but RTT degrading

SLAC to IHEP Beijing

Packet loss 15-20%

Monitoring - conclusions?Monitoring - conclusions?

Monitoring conclusionsMonitoring conclusions

ESnet is an example to us all. It is well-ESnet is an example to us all. It is well-configured and provides very good service configured and provides very good service between ESnet sites.between ESnet sites.

Many of the national nets in Europe and Many of the national nets in Europe and Japan also do a very good job. They are, Japan also do a very good job. They are, however, not normally as well-configured however, not normally as well-configured as ESnet.as ESnet.

The problems come when you hit The problems come when you hit congestion. Normally caused by congestion. Normally caused by saturation, often on cross-ocean links.saturation, often on cross-ocean links.

You need “headroom” You need “headroom”

RecommendationsRecommendations

Inter-continental links (1/3)Inter-continental links (1/3) ICFA should encourage the provision of some considerable ICFA should encourage the provision of some considerable

extra bandwidth for ICFA traffic, especially over the Atlanticextra bandwidth for ICFA traffic, especially over the Atlantic A workshop should be held a.s.a.p., preferably in October A workshop should be held a.s.a.p., preferably in October

1998. Would be best if someone “more global” takes the 1998. Would be best if someone “more global” takes the lead, but we cannot wait for ever...lead, but we cannot wait for ever...– Participants:- ICFA labs and universities; NRNs, TEN-155 and EU in Europe; I2 Participants:- ICFA labs and universities; NRNs, TEN-155 and EU in Europe; I2

+ ESnet + ?? in USA; bandwidth and service suppliers; other disciplines+ ESnet + ?? in USA; bandwidth and service suppliers; other disciplines– Broad objective:-Broad objective:-

try to simplify complex arrangements over Atlantictry to simplify complex arrangements over Atlantic see how QoS/DS etc. could improve situation for general traffic and see how QoS/DS etc. could improve situation for general traffic and

disciplinary trafficdisciplinary traffic– More specifically:-More specifically:-

review first tests with QoS/DS over Atlanticreview first tests with QoS/DS over Atlantic review Europe/US gateways and look for improvementsreview Europe/US gateways and look for improvements coordinate arrangements for “labelling” trafficcoordinate arrangements for “labelling” traffic better coordination of transit and global cost sharing (first discussions??). better coordination of transit and global cost sharing (first discussions??).

Maybe better after first QoS/DS pilot projects. Maybe better after first QoS/DS pilot projects.

Submarine cablesSubmarine cables

TM

The next set of slides are selected from presentations by the Project Oxygen management at a recent Data Gathering Meeting

They are publicly available on the Web.

What is Project OXYGEN?What is Project OXYGEN?

A global optical fiber cable NetworkA global optical fiber cable Network 158,000 km of cable: 150,000 submarine, 158,000 km of cable: 150,000 submarine,

8,000 terrestrial8,000 terrestrial 100 landing points in 73 countries and 100 landing points in 73 countries and

locationslocations A flexible routed networkA flexible routed network 3 Network Management Centers3 Network Management Centers Unprecedented international bandwidthUnprecedented international bandwidth

How is Project OXYGEN How is Project OXYGEN different?different?

Pricing independent of distance and Pricing independent of distance and destinationdestination

Network rather than point-to-point systemNetwork rather than point-to-point system True bandwidth on demand with flexible True bandwidth on demand with flexible

routing rather than fixed circuitsrouting rather than fixed circuits Planned global infrastructurePlanned global infrastructure

World Class SponsorsWorld Class Sponsors

Alcatel Network SystemsAlcatel Network Systems TYCO (ex-AT&T SSI)TYCO (ex-AT&T SSI) NECNEC Mitsui & Co.Mitsui & Co. Sumitomo Corp.Sumitomo Corp.

NTT InternationalNTT International CorningCorning Lucent Technologies.Lucent Technologies.

JP MorganJP Morgan (financial advisors)(financial advisors)

CostsCosts

July overviewJuly overview(to countries where we would normally expect “good” (to countries where we would normally expect “good”

performance)performance)

Jul-98Monitoring from CERN DESY Italy KEK UK 1 US univ 3 US labs

(on ESnet)Monitoring to Belgium 1.40 9.58Czech Rep. 0.80 0.64 3.30 1.62 5.34 0.94Denmark 0.05 1.88 0.26 0.70 3.52 1.36France 0.08 0.30 0.18 1.62 8.87 1.68Germany 0.23 0.01 0.39 1.23 0.31 18.82 1.25Hungary 2.32 3.01 1.97 2.90 6.43 3.02Ireland 0.73 1.13Israel 2.96 1.68Italy 1.58 0.69 0.81 1.18 6.93 1.61Japan 1.47 0.82 0.37 0.08 8.38 0.52Netherlands 0.39 0.97 0.78 1.40Norway 1.21Poland 1.02 2.96Spain 3.99 3.18 4.46 1.94 19.48 17.45Sweden 0.47 1.23 0.40 1.67 3.08 1.23Switzerland 0.19 0.35 0.38 0.54 0.13 6.04 0.36UK 0.64 0.75 0.73 1.23 0.40 5.58 1.76US universities 3.51 11.68 4.75 3.53 2.61 2.04 0.98US labs (on ESnet) 0.50 2.76 1.84 0.43 0.87 1.84 0.42

July 1998 overviewJuly 1998 overviewto countries where we might expect problemsto countries where we might expect problems

Jul-98Monitoring from CERN DESY Italy Japan UK 1 US univ 3 US labs

(on ESnet)Monitoring to Argentina 22.17 15.07Australia 1.67 11.95 5.03 9.26 2.55 1.23Colombia 15.08 15.67New Zealand 12.00 8.55 1.52 8.41 7.98Russia 2.78 6.10 3.57 3.75 14.26 2.60

Personal commentsPersonal comments

Networking, telecoms and the EU are all complex topicsNetworking, telecoms and the EU are all complex topics Since many people don’t want to disagree openly, they Since many people don’t want to disagree openly, they

often try to hide their disagreements in impenetrable often try to hide their disagreements in impenetrable arguments and complex language. arguments and complex language.

Working in that environment can be very Working in that environment can be very tiring/frustrating. As scientists and/or engineers we (I) tiring/frustrating. As scientists and/or engineers we (I) prefer to say what we (I) think, and disagree, if prefer to say what we (I) think, and disagree, if necessary.necessary.

I believe that providing excellent Internet service I believe that providing excellent Internet service during the whole period 1999-2002 is a basic during the whole period 1999-2002 is a basic requirement for European Research and Technical requirement for European Research and Technical Development, and that this clear message should come Development, and that this clear message should come through in the FP5 paperwork. This is not yet the case. through in the FP5 paperwork. This is not yet the case. Keep up the pressure!!Keep up the pressure!!

Can we (the people in the Can we (the people in the room) do anything to help? room) do anything to help?

(1)(1) Keep lobbying that a good pan-European Keep lobbying that a good pan-European

research network needs to be openly accepted research network needs to be openly accepted as a fundamental component of FP5. We need as a fundamental component of FP5. We need to do a better selling job on:-to do a better selling job on:-– University headsUniversity heads– Fellow researchers (scientists and non-scientists)Fellow researchers (scientists and non-scientists)– Politicians & civil servantsPoliticians & civil servants– Telecoms and Internet suppliersTelecoms and Internet suppliers– Industry and commerceIndustry and commerce

Note that we have still not managed to create Note that we have still not managed to create the US synergy between all of these groups.the US synergy between all of these groups.

Can we (the people in the Can we (the people in the room) do anything to help? room) do anything to help?

(2)(2) Personally encourage competition for service provision in the Personally encourage competition for service provision in the

liberalised era after January 1998 (or whenever it arrives liberalised era after January 1998 (or whenever it arrives chez vouschez vous). ). Especially competition for European infrastructure.Especially competition for European infrastructure.

Greece, Ireland, Luxembourg, Portugal and Spain have a variety of Greece, Ireland, Luxembourg, Portugal and Spain have a variety of waivers concerning the dates by which they must implement full waivers concerning the dates by which they must implement full competition.competition.

For the provision of alternative infrastructures, where competition For the provision of alternative infrastructures, where competition across the bulk of the EU started already in July 1996, the across the bulk of the EU started already in July 1996, the extensions authorised were only for a few months, with Greece being extensions authorised were only for a few months, with Greece being the last to liberalise, at the start of October 1997.the last to liberalise, at the start of October 1997.

For voice telephony and networks, where the normal starting date For voice telephony and networks, where the normal starting date was January 1998, the extensions are somewhat longer, with was January 1998, the extensions are somewhat longer, with liberalisation arriving in Spain at the end of November 1998, in liberalisation arriving in Spain at the end of November 1998, in Ireland at the start of January 2000, and in Greece at the end of Ireland at the start of January 2000, and in Greece at the end of December 2000. December 2000.

Can we (the people in the room) Can we (the people in the room) do anything to help? (3)do anything to help? (3)

Lobby for a pan-European telecoms (and Lobby for a pan-European telecoms (and Internet) regulator. In the future this will Internet) regulator. In the future this will be the only way for Europe to act quickly be the only way for Europe to act quickly enough to protect its economy in the enough to protect its economy in the Internet era.Internet era.

Start to think about the whole European Start to think about the whole European A&R community buying in greater bulk. A&R community buying in greater bulk. For national and international lines. For For national and international lines. For services?? This will be very difficult, but services?? This will be very difficult, but could have a huge influence on prices.could have a huge influence on prices.