Performance MeasurementCANARIE/DANTE/Internet2 Rome Meeting

(Jan 05)

Jeff Boote and Eric Boyd - Internet2

Nicolas Simar - Dante

2

Agenda

Update of Action Items Internet2/JRA1 Interaction Update GÉANT2-JRA1 Activities Internet2 performance activities High level framework description Summary: Internet2/JRA1 Next Steps

3

Action Item Update

A5 - Light Path (intermediate measurements)

A6 - Joint White Paper A7 - Joint SW Development Feasibility A8 - Regularly Scheduled Tests

4

A5: Light Path Intermediate Measurements

• Goal: Figure out how to do partial path analysis of a lightpath.

• Real technical challenge.• No real progress yet

• Internet2 HOPI project will need to address this.

• JRA3 will be following this topic.

5

A6: Joint White Paper• First draft largely complete• Current development from participants is

focus on JRA1 General Framework Document• (JRA1 General Framework Document acting as

the technical description of the architecture to be included in the Joint White Paper, the joint white paper being broader as it will also include use-case)

• Will continue to iterate until JRA1 GFD deadline (due Mid February)

6

A7: Joint SW Development• Open Source Development Plan

• http://people.internet2.edu/~eboyd/Joint_Open_Source_Development_Environment.pdf

• BSD Style License (GN2 to determine the exact GN2 contract requirements and their impacts on the license)

• Shared authority structure• Sourceforge (most likely) development environment• Disengagement non-punitive (a carrot for participation)• Both projects have similar structure involving several

partners working on the same issues.

7

A8: Test Links between GEANT and Abilene

• On-demand is available between Abilene and GÉANT• http://e2epi.internet2.edu/pipes/pmp/pmp-dir.html

• Preparing for regular measurements Los Angeles <-> CERN lightpath

• OWAMP and BWCTL monitoring constantlyhttp://ndb1-blmt.abilene.ucaid.edu/lightpath/

piPEs Software Evaluation• PSNC (Poland) reviewed Internet2 efforts in a whitepaper• PSNC deploying BWCTL, OWAMP, piPEs Measurement

Framework v0.1 alpha prototype• Ongoing discussions as part of Architecture discussions

8

Agenda

Update of Action Items Internet2/JRA1 Interaction Update GÉANT2-JRA1 Activities Internet2 performance activities High level framework description Summary: Internet2/JRA1 Next Steps

9

Internet2/JRA1 Joint Activities UCL E2E Monitoring Workshop 2003

• http://people.internet2.edu/~eboyd/ucl_workshop.html Internet2, DANTE, CANARIE biannual meetings (12/03, 07/04,

01/05) Transatlantic Performance Monitoring Workshop 2004

• http://people.internet2.edu/~eboyd/transatlantic_workshop.html Caltech <-> CERN Demo

• March ’04• November, December ‘04

Haystack, USA <-> Onsala, Sweden• In use by eVLBI community• Added SUnet node to the available mix through outreach to that group

10

Internet2/JRA1 Joint Activities Contribution to the GGF NM-WG

• both Internet2 and Dante provided a significant contribution to the effort (four to five people contributing regularly - 2 from Europe, 2-3 from US)

• Contribute to design, early adoption and prototyping, feedback General Framework Design

• Workshop on the General Framework design in Brussels• Weekly conf calls, joint mailing list

Installation of tools : Internet2 OWAMPs, Internet2 BWCTLs, Internet2 piPEs framework, DFN IPPMs and their evaluation.

Use-cases.

11

Internet2/JRA1General Framework Design

Metcalf’s Law Our version: The value of a performance measurement

framework scales with the square of the deployment footprint

One organization cannot create a successful measurement framework in a vacuum

GGF NMWG: Enable multiple measurement frameworks to work together• piPEs, MonALISA, Advisor, and AMP• Demonstrate interoperability of NMWG schema• Working to build demo with EGEE JRA4 (PMP) for GGF13 in March

involving piPEs, AMP, and Asian PMPs Shared goal of building a next generation measurement

framework

12

Agenda

Update of Action Items Internet2/JRA1 Interaction Update GÉANT2-JRA1 Activities Internet2 performance activities High level framework description Summary: Internet2/JRA1 Next Steps

13

GÉANT2-JRA1 Activities Requirements

• Three questionnaires were written targeting: the NRENs, the projects and the end-users.

• Goal: get an overview of the existing monitoring infrastructure (metric, tools used) the visualisation of the data the need to access monitoring information from other networks.

• 45 answers were received in total (respectively 16, 14, 15) Strong interest to access monitoring information form

multiple network.• NRENs: less than 5-10% of the problems they are encountering

involves several domains ( => times 30 NRENs). They want to see improved the capability of localising the problems.

• International projects want to have a view on what’s happening between their sites (uses: troubleshooting, SLA and internal decision making).

• End-user: less important than for NRENs or projects (uses: troubleshooting, service verification)

14

GÉANT2-JRA1 Activities Readiness to open access to measurement data

• Some ready to show everything (or nearly so)• Some want to apply restriction (about what and to who) • Some don’t want to

Monitoring Information:• RTT and OWD• bandwidth utilisation and achievable TCP throughput• RTT and OWD packet loss• Delay variation• Interfaces error and drops• Routing/path information

On-demand capability (to and from other domains)

15

GÉANT2-JRA1 Activities Be able to monitor the services deployed

• IPv4/IPv6• Multicast/unicast• IP QoS• VPN/point-to-point connections• Emulate behavior close from the one from the

application used Different tools used amongst the networks,

need to abstract the data provided from the type of measurement tools used.• Provide data through a well define interface.• Inter-operability between tools.

16

GÉANT2-JRA1 Activities

Keep in mind: installation and maintenance! Had a look at existing tools and went more in depth

for the most interesting ones. We have chosen so far the following tools:

• OWD: DFN IPPM• Throughput: iperf based• Flow monitoring: flowtool• Visualisation: CNM

• Pending: Packet capture tool (SW: scampi - tbc, HW:

Endace or scampi - further work needed), other visualisation

17

GÉANT2-JRA1 Activities

Current actions• General Framework Design v1 (mid-February)• Prototype (June-July 05)• Work on measurement concatenation (now -> September)• Buy equipment and install it.

Next steps• AA (discussion with JRA5)

Which model to follow? Authorisation based on groups (NOC, PERT, projectA, user).

How to have easy agreement between domains? (don’t want to negotiate an agreement with all the US universities or with all the European NRENs)

• Detailed design of the modules v1 (September 05)• Trial phase (November 05-December05)

18

Agenda

Update of Action Items Internet2/JRA1 Interaction Update GÉANT2-JRA1 Activities Internet2 performance activities High level framework description Summary: Internet2/JRA1 Next Steps

19

piPEs

BWCTL• Stable - fair amount of interest

OWAMP• Significant changes to specification.

IETF working group last call completed

• New version of implementation forthcoming to reflect the changes

20

piPEs

NDT• Redirection to closest NDT server within a

group of servers• Funded to significantly improve

understanding and detection of duplex mismatch problems (NIH/NLM Grant)

PMP registry

http://e2epi.internet2.edu/pipes/pmp/pmp-dir.html

21

Bridging the Gap Workshop (NSF)

Explore network performance solutions across scientific application communities• Network experts• Researchers (network users)• Network application developers• Campus network engineers

22

Internet2 Detective

Evaluating future development using SURFnet Detective platform

Strategic investment: Gateway for naïve entrance to advanced services like Shibboleth and Pipes

23

Internet2 Transport Effort Congestion control researchers/high-end

users (led by Stanislav Shalunov) Goal: user-space transport tool

• High performance: Suitable for both bulk file transfer and interactive multimedia

• Tolerance for minor non-congestive packet loss• Completely end-to-end: no router modifications• Portable, easy to install and use (no kernel

modifications)• Advanced congestion control using existing

research https://mail.internet2.edu/wws/arc/transport

24

Agenda

Update of Action Items Internet2/JRA1 Interaction Update GÉANT2-JRA1 Activities Internet2 performance activities High level framework description Summary: Internet2/JRA1 Next Steps

25

General Framework Overview

Architecture refinement Proposal High-level description of components Interaction description

26

Architecture Refinement

Review of existing systems• Insights based upon Abilene prototype framework,

DANTE’s perfmonit and IPPM experiences New insights gained from inter-domain

framework test experience (lightpath measurements, Abilene/ESnet, etc)

Additional use cases and experience of collaborators• Internet2, GÉANT2 JRA1, GGF NMWG

27

Architecture Proposal

Services Oriented Architecture• In a simple scenario, each domain consists

of a set of services. All services are well defined and independent

• Services within a domain represent the domain with the help of Authentication and Authorization – they respond to requests only if the Authentication service of the domain has authenticated the user and the policy of the given service authorizes it

28

Basic Services

Lookup Authentication Measurement Point Measurement Archive Resource Protector (Authorization) Aggregation

• Topology

29

Measurement Point

Service to wrap measurement tools Interacts with resource protectors to protect

shared resources Registers with lookup service and specifies

the authentication credentials required to interact

Registers with lookup service to indicate types of tests it can perform

Accepts requests for tests

30

Test Request (Initialization)

Test Request Client

Lookup

Test Executor

Test Executor

Authentication

1) Register

1) Register4) Find Test Peers

5) Present credentials, receive authtoken for Test Executors

Initialization Phase: Registration/Lookup

1) Register

Lookup will be P2P “Bootstrapping” can use some combination of: Well known hosts Broadcast Multicast Previously detected

31

Lookup Service

Initial discovery• Multicast / Anycast• Well known servers• Required servers (by administrative

configuration)• Previously detected servers (organized in a

P2P network – lookup services find out about other lookup services…

32

Lookup Service (II)

Lookup is not simply by name• Type (type of measurement, type of service)• Community• Network path (proximity information from Topology)• Organization• Type of authentication required• Other…

Response contains• Contact information• Available services• Authentication required• Other…

33

AuthenticationRegisters with lookupClient requests “kind” of authentication token based lookup results

Authentication grants time-limited token used to request service

Protocol for determining “role/identity” for request. (Shib: federated trust)

• Allow new measurement points to be created as easily as possible

• Allow new data consumers access as easily as possible

34

Process Flow (Client)

Discovery.• Find lookup servers.• Use lookup servers to find tool beacons for a given problem.

(On correct path, with acceptable authentication requirements, with acceptable tools/measurements.).

Authentication.• Authenticate to correct auth servers that are needed for

desired test executors.Test execution.

• Implement subscriber to accept results.• Make test requests presenting credentials and reference to

subscriber interface for returned data.

35

Full Test

Test RequestClient

Lookup

Test ExecutorTest Executor

Authentication

1) Find Test Peers (including auth reqs)

2) P

rese

nt c

rede

ntia

ls,

rece

ive

auth

toke

n fo

r

Tes

t Exe

cuto

rs

Test Request

3) T

est R

eque

st (

para

m/s

ched

ule

req)

3) Test Request (param/schedule req)

4) Measurement

4) M

easu

rem

ent R

esul

ts

5) Measurem

ent Results

36

Request Phase (Scheduling)

Test RequestClient

Test Executor“B”Test Executor

“A”

Request Phase

Client repeats steps 1-4 until the timereturned from “A” matches the timereturned from “B”.

Each TestExecutor will have amaximum time into the future it iswilling to schedule a test - after thattime it will return “denied”

The client will have a maximum timeinto the future it is willing to schedulea test - after that time it will notaccept a slot offered by aTestExecutor.

37

Resource Protector

Enables centralizing of resource allocation (not globally - this is within spheres of administrative control)

Multiple measurement points interact with a given resource protector to limit the shared resources

Resource protectors can be chained hierarchically to control aggregations of shared resources across larger frameworks.

38

Resources Protectors

Test RequestClient

Brokering - In depth (Scheduling shared resources)

Resource Broker

HostResources?

Yes - requestLink

return result

No - returndeny

Resource Broker

LinkResources?

Yes - retunaccept

No - returndeny

Test Executor

ParametersValid?

Yes - requestHost

return result

No - returndeny

39

Measurement Archive

Subscribes to some set of data – either from a measurement point or from an aggregation service

May publish the derived data sets

40

Topology

Network topology information is necessary for measurement system optimization

Creates overviews/”maps” to illustrate networkLayered approach (domain level through to wavelengths and physical level)

Specific type of aggregation (translation)• Collects raw data from measurement points and pushes topology information into the lookup service (allows topologically based queries to lookup service)

41

Topology (Initialization)

MP1

MP2

Tests

Lookup

Topology

Archive Current Topology

Historical/FullTopology

42

Aggregation (Translation)

Data translation service (pipelines data between other components in the framework)

Subscribes and Publishes data Provides:

• Aggregation• Correlation• Caching• Duplication• Translation

Event generation Data analysis

43

Agenda

Update of Action Items Internet2/JRA1 Interaction Update GÉANT2-JRA1 Activities Internet2 performance activities High level framework description Summary: Internet2/JRA1 Next Steps

44

Summary: Internet2/JRA1 Collaboration Next Steps

Open Source Shared Development• Sourceforge-based Sub-Projects• Modified Berkeley Licensing

Common Service-based Architecture Architecture spans superset of deployment use cases ~Quarterly face-to-face meetings ~Weekly phone conferences Split development according to interest, resources

45

General Framework Next Steps

Architecture continuing to be refined Architecture validation

• Detailed use-case flow descriptions• Interfaces• Prototypes

New Action Item: Jointly developed, services-based, measurement framework prototype by Summer ‘05

46

Internet2/JRA1 Interaction Update

Agreement from the management on the way to proceed for the joint development, license, and open-source

Clarification from GN2 on the impact of the GN2 contract on the license

Agreement on AA as it covers several groups (Possibly should be deferred until AA discussion)

Openness to share measured information Will have to set-up measurement peering

agreements (who can do what, and up to what extent)

47

Questions?