performance update jeff boote senior network software engineer internet2 martin swany assistant...

Performance UpdateJeff Boote

Senior Network Software Engineer

Internet2

Martin Swany

Assistant Professor

University of Delaware

The Distributed CI Computer

Instrumentation

Security

Control

DataGeneration

Computation

Analysis

Simulation

Program

Security

ManagementSecurity and

AccessAuthentication

AccessControl

Authorization

Researcher

ControlProgram

ViewingSecurity

3DImaging

Display andVisualization

.

DisplayTools Security

DataInput

CollabTools Publishing

HumanSupportHelp

Desk

Policy andFunding

ResourceProviders

FundingAgencies

Campuses

SearchData SetsStorage

Security

RetrievalInput

SchemaMetadata

DataDirectories

Ontologies

Archive

EducationAnd

Outreach

Network

Training

The Network is the Backplane for the Distributed CI Computer

CollabTools Publishing

HumanSupportHelp

Desk EducationAnd

Outreach

Network

Training

CI Components

Network

Performance Infrastructure / Tools

MiddlewareControl Plane

….

Bulk Transport

2-Way Interactive

Video

Real-Time Communications

Applications

Applications call on Network Cyberinfrastructure

….

…. ….Phoebus

Netw

ork C

yberinfrastructure

Measurement Nodes

Control Plane Nodes

Internet2 Performance CI Software

• Performance Monitoring Infrastructure• perfSONAR (with ESnet, GEANT2 JRA1, RNP,

many others)• BWCTL, NDT, OWAMP, Thrulay

• Transport Infrastructure• Phoebus

• Distributed System Infrastructure• Topology Service (with University of Delaware)• Distributed Lookup Service (with University of

Delaware, PSNC)

Overview

• Performance Measurement Requirements• Measurement Tools• perfSONAR• perfSONAR-PS Services• Transport Middleware• Distributed System Infrastructure

• Schema for sharing• Services for sharing (discovery/topology/authentication)

Performance Measurement Requirements (Internet2 network support)

• Deployed on Internet2 Network• Software: perfSONAR, BWCTL, NDT, OWAMP• Goals: IU NOC Support *, proof of performance for Application Community

• Meet the needs of the LHC Application Community (T0-T3)• Partnership with GEANT2 JRA1, ESnet, RNP to meet that need

• Packaged perfSONAR solution for US connectors and universities• Ease of deployment is key *• Veneer on existing NOC solutions rather than a new solution *

• Integrated into a Cyberinfrastructure Solution• Support DRAGON• Rely on common AA infrastructure• Common elements (AA, Discovery, Topology)

• * Drives the need for a perl-based packaged solution, perfSONAR-PS

OWAMP (One way latency data)

• 3.0c (RFC 4645 version) available now

• Maintenance mode

• Diagnostic test points available at all Internet2 IP Network router

locations:

• owamp.POP.net.internet2.edu

• POP = losa,salt,hous,kans,chic,atla,newy,wash - soon seat

• Full mesh of Internet2 IP network tests collected and available from:

http://owamp.net.internet2.edu/owamp_grid.cgi

BWCTL (Throughput tests)

• 1.2a is current version

• 2.0 due out soon with additional testers

• Diagnostic test points available at all Internet2 IP Network router locations:

• bwctl.POP.net.internet2.edu

• POP = losa,salt,hous,kans,chic,atla,newy,wash - soon seat

• Full mesh of Internet2 IP network tests collected and available soon

NDT updates

• Administrator can allow clients to run repetitive tests

• MCNC modifications rolled into NDT code

• New post-processing tools to improve analysis of log data

• New version available by end of this week

Post-processing Tasks

• Allow site administrators to find problems effecting multiple clients

• Allow site administrators to verify that problems have been resolved

• Allow developers to experiment with new detection algorithms

• Allow developers to quickly test new features

NDT Post-processing Tools

• Genplot– Simply program to display Web100 data

• JAnalyze– Java based version of Analyze program

Genplot – Textual Display

• ‘genplot -tmCurCwnd,SlowStart,CongAvoid,CongestionSignals fn | less’

Genplot – Graphical Display

• ‘genplot –C fn; xplot fn.xpl’

JAnalyze

NPToolKit

• Recent versions of Measurement tools installed and pre-configured

• Knoppix Live-CD bootable system

• Current Version: 1.7• http://e2epi.internet2.

edu/network-performance-toolkit.html

perfSONAR Motivation

• Most organizations can do monitoring and diagnostics of their own network

• Networking is becoming an increasingly cross-domain effort

• Monitoring and diagnostics must also become a cross-domain effort

perfSONAR

• A set of protocols and schemas for implementing a SOA for sharing and controlling network performance tools

• A community of users and developers• Multiple sets of interoperable software

perfSONAR-PS Motivation

• Create separate implementation of perfSONAR standard

• Use same protocol/standards

• Proof of interoperability (strengthens the standard)

• Targeted for University NOC deployments

• Lightweight

• Easy to deploy/manage

• (We were unable to convince our primary users to deploy Java

services due to the complexity of dependencies)

perfSONAR-PS Services

• Focus on development of major perfSONAR components• LS• Topology• Link Status• Circuit Status• SNMP Based MP/MA• PingER (SLAC)

• New additions• OWAMP/BWCTL• Traceroute

• Contact Jason or Jeff if you are interested in testing pre-releases

Lookup Service

• 'Oracle' of perfSONAR deployments• Accept service registrations• Handle complex data/service queries• Manage the lifetimes of data and services to keep

framework relevant

• Web Service interface to XML Database• Berkeley DB XML• Service Info/Data kept in native formats

• Draw away the complex query tasks from otherwise 'busy' services

Lookup Service

• Also XML based configuration/protocol• Native storage/query mechanisms [Xpath/XQuery]• Message format to exchange the data

• Targeted at single domain deployment• Single instance to manage multiple services• Mutli-domain 'distributed' LS in development

• Client applications aware of LS to find services• perfSONAR-UI• perfAdmin

Lookup Service

• Current Deployment:• Internet2 (Ann Arbor)• University of Delaware

• Planned Deployment:• IU for Internet2 network and regionals• International Partners

Topology Service

• Provides a queryable repository for obtaining topology information about a domain• Can obtain the entire network• Xquery interface allows the construction of

arbitrarily complex queries about the network

• Topology is specified according to the schema in development for OGF

Topology Service

• Current Deployments• Internet2

• Planned Deployments• Internet2 DCN• SLAC (PingER Topology Information)

Link Status Measurement Archive

• Provide access to up/down status information about layer2 links• Data stored in a SQL database• Database schema allows for storing time ranges during

which a link had a certain status• Minimizes storage costs for rarely changing links

• Communication/Configuration via XML• Target audience is network operators and users

interested in obtaining the status of the links over which their data flows

Link Status Measurement Archive

• Collector• Allows for the periodic collection of the status of

one or more links• Can use SNMP, Scripts or simply Constants• Can store results directly into a database or into

a remote Measurement Archive

Link Status Measurement Archive

• Visualization• A perfSONAR-UI Plugin is available that can

display a network and the status of its links

• Current Deployment• Internet2 Network• HOPI (in2p3 circuit)

• Planned Deployment• SLAC

Circuit Status Measurement Archive

• An e2emon-compatible service• Integrates with the Link Status MA to provide the

information stored in MAs • Can work with local MAs directly or with remote MAs

• Can use the Topology MA to obtain necessary information about nodes

• Can use a Lookup Service to lookup the MA containing information on each link

• Target audience is administrators who want to publish circuit status information to e2emon clients

Circuit Status Measurement Archive

• Visualization• Any tool that is compatible with e2emon will

work with this service

• Current Deployment• Internet2 Network• HOPI (in2p3 circuit)

• Planned Deployment• SLAC

SNMP Measurement Archive

• Provide access to network performance data• Utilization• Errors• Discards

• Numerous tools exist to collect passive measurements (via SNMP):• MRTG• Cacti• Cricket

• Expose archives from RRD files

SNMP Measurement Archive

• XML based configuration/protocol• Configuration file to expose the data• Message format to exchange the data

• Targeted at researchers, NOC staff debugging intra/inter domain network problems

• Many client applications available to graphically display the results:• perfSONAR-UI• perfAdmin• perfOMeter (General Session Demo)

SNMP Measurement Archive

• Current Deployment:• Internet2 Network• ESnet• Georgia Tech/SOX• Fermilab

• Planned Deployment:• Regional Networks• International Partners

Visualization

• Utilizing the plugin architecture of perfSONAR-UI• Data visualization beyond network utilization

• Google Maps• Utilization by physical location• 'Weather Map' of Internet2 Network

• Web based speedometer to interact directly with MA code

Other Services (in development)

• Traceroute MP/MA• Joint effort between UD and SLAC• Integrate into visual traceroute tools• Currently in testing

• OWAMP MA• Make regular results available

• BWCTL MA• Make regular results available

perfSONAR-PS Releases

• Currently exploring testing and packaging strategies• Releases will ultimately be CPAN based• Standing issues include configuration/automation

help.

• Versions of SNMP MA and LS will be available this fall

• Pre-CPAN versions will be available for LHC commitments in August (using CPAN conventions)

Transport Middleware (Phoebus)

• The Phoebus project aims to help bridge the performance gap by bringing dynamic networks to end-users• Phoebus is another name for the mythical Apollo in his role as the

“sun god”

• Phoebus is based on the concept of a “session” that enables multiple adaptation points in the network• Improves end-to-end throughput for users

• Depots on the ingress/egress points of backbone networks mitigate the effects of edge networks

• Allows transparent use of network features like cross-domain dynamic circuits

Session Layer

• A session is the end-to-end composition of segment-specific transports and signaling• More responsive control loop via reduction of signaling

latency• Adapt to local conditions with greater specificity• Buffering in the network means retransmissions need not

come from the source

PhysicalData LinkNetwork

TransportSession

PhysicalData LinkNetwork

TransportSession

PhysicalData LinkNetwork

TransportUser Space

Optical Net

End-to-EndSession

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Optical Net

End-to-EndSession

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Phoebus - Upcoming SC Demo

• Allocate dynamic circuits using DCN Control Plane

• UDT for inter-depot communication

• Transparent operation (library, iptables)

• Legacy applications can use the DCN with no modification

Phoebus - Current Work

• Separating front-end (control plane) from back-end (data plane)• Allows creation of new front-end services

• Constructing a web services based front-end• Andy Lake for Google Summer of Code

• Construction of high-performance forwarding devices with myricom NICs

Phoebus - Future

• Simple file transfer tool • Transparently use Phoebus/Dynamic

Circuits• Utilize Measurement Infrastructure

• Help find best routes, provide information about paths and achievable bandwidth

• Extension of Path Finding / Routing• Authentication and Authorization

Schema/Protocol Developments

• The perfSONAR Topology schema is also used in the DCN control plane• We’ve spent quite a bit of effort harmonizing these

• The obvious win is that we have the measurement system have immediate access to dynamic circuits

• The broader impact is that we’re approaching a unified network interaction model (UNIM)

Schema Developments

• The perfSONAR schema is being standardized in the Open Grid Forum (OGF) Network Measurement Working Group (NM-WG)

• The topology efforts have spawned a new WG called the Network Markup Language WG (NML-WG)

Schema - Network Element Identifiers

• A scheme for identifying network elements

• Each network element gets a unique identifier

• This identifier will be included with any measurement associated with that element.

Network Element Identifiers

• Use Cases:• A topology service can be used to find the

identifier for a network element• An LS could then be queried to find all

measurements associated with that element• Dynamic service path-finding can be

integrated with ongoing measurements

Network Element Identifiers

• Identifiers use URN notation• Prefixed with “urn:ogf:network:”

• Consists of name/value pairs separated by colons

• Possible field names: domain, node, port, link, path, network

• Set of rules defined for each field to keep identifiers compact and finite

Network Element Identifiers

• Examples• urn:ogf:network:domain=Internet2.edu• urn:ogf:network:domain=internet2.edu:node=packrat• urn:ogf:network:domain=internet2.edu:node=rtr.seat:port=so-2%2F1%2F0.16• urn:ogf:network:domain=internet2.edu:node=rtr.seat:port=198.32.8.200• urn:ogf:network:domain=Internet2.edu:node=packrat:port=eth0:link=1• urn:ogf:network:domain=internet2.edu:link=WASH to ATLA OC192• urn:ogf:network:path=anna-11537-176

Distributed Systems Infrastructure

• perfSONAR, DCN and Phoebus have similar system requirements• Lookup and Topology Services comprise a generic

Information Service that is useful to all these Network Services

• Authentication and Policy services are cross-cutting as well

• Rather than have silos of mission-specific functionality, we envision pervasive system components

Distributed Systems Infrastructure

• Synergies of information bases are obvious• Multi-layer path-finding including current

network state, available resources on a variety of layers

• It is a compelling vision to imagine a dynamic, reactive, visible service-rich network

Summary

• A rich set of tools are being developed• To federate network monitoring and diagnostics• To enable dynamic network resource allocations• To leverage new network capabilities from an ‘end-user’

application (phoebus)

• A longer view toward an evolution of “in the network” services

Questions?

• Jeff Boote• boote@internet2.edu

• Martin Swany• swany@cis.udel.edu