introduction to perfsonar · 1/40 introduction to perfsonar perfsonar workshop: implementing the...
TRANSCRIPT
![Page 1: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/1.jpg)
1/40
Introduction to perfSONAR
perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring
Szymon Trocha, Poznań Supercomputing and Networking Center, Poland
26th APAN Meeting, 4-8 August 2008, Queenstown, New Zealand
![Page 2: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/2.jpg)
2/40
• „Internet measurements is fun”
• „We need to share our data”
N.Brownlee, „Measuring Internet Evolution - If We Don't Measure, We Don't Know What's Happening!”, APAN26
![Page 3: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/3.jpg)
3/39
Performance Measurement Goals
• Increase network awareness• Set user expectations accurately
• Reduce diagnostic costs• Performance problems noticed early • Performance problems addressed efficiently• Network engineers can see & act outside their
domains
• Transform application design• Incorporate network intuition into application
behavior
![Page 4: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/4.jpg)
4/40
Why perfSONAR ?
![Page 5: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/5.jpg)
5/39
Network Boundaries
• Users un-interrupted communication • Network’s administrative domain
• perfSONAR offers• Standard access to the information• Access to already deployed tool instances • GUIs to present and analyse the information
• Benefits• Better understanding, better ownership• Reduces users downtime, increase user
satisfaction
![Page 6: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/6.jpg)
6/39
Quick answers and solutions
• Diagnosing takes time
• After some time, the user moves on• Lost opportunities• Problem disappear
• perfSONAR offers• The same view to the operators• Pre-authorised access
• Benefits• Retention/creation of opportunities
![Page 7: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/7.jpg)
7/39
Resources Discovery
• Data is there
• Difficult to find measurement capability
• perfSONAR offers• Discovery mechanism
• Benefits• Save staff time
![Page 8: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/8.jpg)
8/39
Bringing Evidence
• Difficulty to bring evidences about the existence or non-existence of a problem
• perfSONAR offers• Access to on-demand and historical information
• Delay• Packet loss
• Interface errors and discards
• Throughput transfers
• Benefits• Reduce the realm of uncertainty for the user• Improve staff and user satisfaction
![Page 9: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/9.jpg)
9/39
Applications
• User interest in what they use the network for
• Network Operators for the network itself
• perfSONAR offers• Access to historical TCP throughput transfers and
delay information
• Benefits• Provide a better understanding about what the
user experiences
![Page 10: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/10.jpg)
10/39
Vision: Performance Information is …
• Available• People can find it• “Community of trust” allows access across administrative
domain boundaries
• Ubiquitous• Widely deployed
• Paths of interest covered• Reliable
• Consistently configured correctly and maintained
• Valuable• Actionable
• Analysis suggests course of action• Automatable
• Applications act on data
![Page 11: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/11.jpg)
11/40
Use Cases
![Page 12: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/12.jpg)
12/39
Use Cases
• Backbone• Collected from Internet2, GÈANT2
• State and regional networks• Collected from MCNC (US)
• Application community• Collected from LHC
![Page 13: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/13.jpg)
13/39
Backbone Use Case (Internet2 network)
• Traditional requirements• Collect multiple performance metrics over a wide geographic area
(passive/active)
• Alarms (real-time analysis for ‘exceptional’ conditions)
• Ad-hoc measurements by ‘peers’• Data requests by ‘peers’
• Contemporary additions• Topological correlations
• Spatial – node and link connections, including inter-domain ones• Layer – Map L3 paths to lower-level infrastructure
• Federated administration• Need ability to share information and even control (with adequate authorization)• This ability must extend beyond direct peers
• Automation• User applications require more direct access to topology/performance information
![Page 14: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/14.jpg)
14/39
Backbone Use Case (GÈANT2 network)
• NOC requirements• Bring evidence about where the problem is
located• Give indication if the problem is rather on the
network or on the end-site• Monitor e2e links for specific projects
• PERT requirements• Multi-domain access to already deployed and well
known tool instances that can be started on demand
• Have a trustable baseline against which data results can be compared• TCP throughput
![Page 15: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/15.jpg)
15/39
State and regional Use Case (MCNC)
• Simple deployable measurement systems in edge networks• Some local administration should be possible• Remote (centralized) administration and data
collection is required• Performance and capacity planning
• Challenges• K12 network admin skill levels/time availability• Middleboxes (firewalls/NATs)• Outdated hardware
See: http://www.internet2.edu/presentations/spring08/20080423-netperf-hunt.pdf
![Page 16: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/16.jpg)
16/39
Application CommunityUse Case (LHC)
• Autonomous system per-site is required• Sites must be able to monitor from their site to
other sites of interest (Tier-2 want to actively probe upstream Tier-1s and downstream Tier-3s)
• Must allow for multiple levels of participation from sites
• Must have ability to determine what diagnostics or tools are available from remote sites
• A formal plan is being developed within the US-LHC community and the Internet2 HENP SIG
See: https://wiki.internet2.edu/confluence/display/PSPS/Tier-2-BCP *draft*
![Page 17: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/17.jpg)
17/40
perfSONAR Overview
![Page 18: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/18.jpg)
18/39
What is perfSONAR
• A collaboration• Production network operators focused on designing and
building tools that they will deploy and use on their networks to provide monitoring and diagnostic capabilities to themselves and their user communities
• An architecture & a set of protocols• Web Services Architecture• Protocols based on the Open Grid Forum (OGF) Network
Measurement Working Group (NM-WG) Schemas
• Several interoperable software implementations• Java, Perl, Python…
• A deployed measurement infrastructure
![Page 19: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/19.jpg)
19/39
Proud of Our Collaborators
![Page 20: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/20.jpg)
20/39
perfSONAR Protocols
• Get perfSONAR protocols standardized within the Open Grid Forum -NMC Working Group
![Page 21: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/21.jpg)
21/39
perfSONAR Architecture
• Interoperable network measurement middleware (SOA)• Modular• Web services-based• Decentralized• Locally controlled
• Integrates• Network measurement tools and archives• Data manipulation• Information Services
• Discovery• Topology• Authentication and authorization
• Based on• Open Grid Forum Network Measurement Working Group
schema
![Page 22: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/22.jpg)
22/39
perfSONAR Client Developments
• Most tuned to specific services currently
• Different user focus (micro vs macro view)• These represent what is possible
• Data is available• Location is known (due to LS)• It is expected that individual application
communities and NOCs will want something more tuned to what they care about
![Page 23: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/23.jpg)
23/39
Available in production
• MDM 3.0 release• Bundle of perfSONAR MDM installers comprising
Java and Perl web services as well as an admin guide
• Originated in GÈANT2
• PS 0.9 release• Perl implementation including CPAN distribution• Originated in Internet2 and UoD
• Visualization toolsSee: http://www.perfsonar.net/download.html
![Page 24: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/24.jpg)
24/39
perfSONAR is active
• Developers meetings• Regular calls• Service desk (EU)
• User support
• Monitoring
• Publications• http://wiki.perfsonar.net/jra1-
wiki/index.php/PerfSONAR_and_JRA1_Publications
![Page 25: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/25.jpg)
25/40
Example Tools
![Page 26: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/26.jpg)
26/39
perfSONAR Tools Examples
• Client applications• perfSONAR UI (ISTF)• CNM (DFN)
• Web Based• GMAPS (SLAC)• DCN Administrative Tool (Internet2)• perfAdmin (Internet2)• pingER Analysis (FNAL)• perfSONAR-BUOY (Internet2)• E2E Link Monitoring (DFN)
• Different ways of using infrastructure and presenting data
![Page 27: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/27.jpg)
27/39
perfSONAR UIhttp://perfsonar.acad.bg/perfsonar.jnlp
• Utilization / errors / discards
• Throughput• Looking Glass• Flows• Circuits
• Developed by ISTF
![Page 28: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/28.jpg)
28/39
Gmaps (1/3)http://packrat.internet2.edu:8008/
• Utilization
• Developedby SLAC
![Page 29: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/29.jpg)
29/39
Gmaps (2/3)http://packrat.internet2.edu:8008/
• ping
![Page 30: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/30.jpg)
30/39
Gmaps (3/3)http://packrat.internet2.edu:8008/
• BWCTL
![Page 31: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/31.jpg)
31/39
DCN Administrative Toolhttps://dc211.internet2.edu/cgi-bin/dcn2.cgi
• TopologyService
• DevelopedbyInternet2
![Page 32: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/32.jpg)
32/39
perfAdmin (1/2)https://dc211.internet2.edu/cgi-bin/perfAdmin.cgi
• BWCTL
• DevelopedbyInternet2
![Page 33: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/33.jpg)
33/39
perfAdmin (2/2)https://dc211.internet2.edu/cgi-bin/perfAdmin.cgi
• PingER
![Page 34: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/34.jpg)
34/39
pingER (1/2)http://tukki.fnal.gov/pinger/pingerUI.pl
• Ping
• Developedby FNAL
![Page 35: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/35.jpg)
35/39
pingER (2/2)http://tukki.fnal.gov/pinger/pingerUI.pl
• Delay• Loss
• IPD
![Page 36: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/36.jpg)
36/39
perfSONAR-BUOY (1/2)http://owamp.net.internet2.edu/owamp_grid.cgi
• OWAMP• BWCTL
• Developed by Internet2
![Page 37: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/37.jpg)
37/39
perfSONAR-BUOY (2/2)http://owamp.net.internet2.edu/owamp_path.cgi/LAT4/HOUS/CHIC/now
![Page 38: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/38.jpg)
38/39
CNMhttp://wiki.perfsonar.net/jra1-wiki/index.php/CNM
Access restricted to project participants
• Utilization
• Developedby DFN
![Page 39: Introduction to perfSONAR · 1/40 Introduction to perfSONAR perfSONAR Workshop: Implementing the perfSONAR platform for measurement and monitoring Szymon Trocha, Pozna ńSupercomputing](https://reader034.vdocument.in/reader034/viewer/2022043008/5f95fa1abe90845b8f705fca/html5/thumbnails/39.jpg)
39/39
e2e Link Monitoring
Access restricted to project participants
• Path status
• Developed by DFN
• Multiple organizations providing data