Sponsored by the National Science Foundation

GEC16 Plenary Session: GENI Solicitation 4 Tool Context

Marshall Brinn, GPO

March 20, 2013

Sponsored by the National Science Foundation 2March 20, 2013

Overview

• GENI Solicitation 4 seeks tools to help researchers use GENI to perform their experiments

• This presentation outlines the essentials of the software context in which such tools will operate– Resources– Services– Interfaces

Sponsored by the National Science Foundation 3March 20, 2013

GENI Software Context Review

Experimenter: A researcher seeking to perform network experiments on customized data plane.

Federation: A collection of people and institutions who agree to share resources and abide by common procedures in order to share resources in a reliable, mutually beneficial manner.

Resources: Physical resources (compute, network, storage) made available to the federation by means of a participating aggregate.

Aggregates: Software entities that represent federated resources in transactions with experimenter tools.

Tools: Software capabilities that interact with federation resources on behalf of experimenters

Clearinghouse: Set of services establishing federation-level authentication, authorization and accountability of experimenter use of federation resources.

Operations Center: Processes and tools monitoring activity on GENI resources for adherence to policies.

Grey boxes are real-world entities, represented in software by Purple boxes.

Sponsored by the National Science Foundation 4March 20, 2013

Tools: Adding Value on Top of GENI Services

• We expect tools will help operators work with GENI by providing value-added services such as:– Visualization: Allowing for visualizing the state of the network,

resources or experiment– Aggregation: Combining information that is held in separate

resources or services to provide convenient “macro” views.– Streamlining: Making it easier to formulate and submit complex

resource or orchestration requests– Monitoring: Performing background checks and generating regular

reports and alerts (as needed)– Uniformity: Presenting different underlying services or API’s in

uniform manner

Such value-added services are built on top of the basis of existing GENI services

Sponsored by the National Science Foundation 5March 20, 2013

Tools for Experimenter Support

• Topology: Construct a custom data plane on which to build and run an experiment– Browse: Determine what resources are available and what their

capabilities are– Allocate: Construct a data plane topology from available resources– Program: Install/configure software to support my experiment

• Experiment: Deploy and run an experiment and capture and analyze results.– Instrument: Establish points at which to collect different metrics of

system performance– Orchestrate: Manage the distributed start/stop/pause steps of

experiment operations– Monitor: Collect measurements on the running experiment– Analyze: Gather and analyze data collected from experiment

Tools to help experimenters will tend to contribute to one or more of these categories.

Sponsored by the National Science Foundation 6March 20, 2013

GENI Services: Overview

• Authentication: GENI uses SSL certificates signed by a GENI trust root. – Anything trusted by GENI (directly or indirectly) may

have such a certificate

• GENI Tools (notably the GPO Clearinghouse Portal) support single-sign-on via credentials supplied by Shibboleth InCommon – Tools may want to be constructed to fit into such a

single-sign-on environment.– We also support SSO using OpenID credentials.

Sponsored by the National Science Foundation 7March 20, 2013

Topology Services: Details

• GENI Resources are presented by Aggregates which speak the AM (Aggregate Manager) API– http://groups.geni.net/geni/wiki/GAPI_AM_API_V3– Client/Resource negotiation represented by three kinds

of RSpecs (Resource Specifications):• Advertisement: What do you provide?• Request: What does the client need/want?• Manifest: What was actually allocated to client?

– Include non-compute aggregates such as ION, FOAM and WiMAX

– Including GMOC, Slice Authorities

GENI services are advertised in the federation Clearinghouse Service Registry

Sponsored by the National Science Foundation 8March 20, 2013

Experiment Services: Details

• There are two families of Experiment Support tools:– GIMI: http://groups.geni.net/geni/wiki/GIMI

• Based on OML (ORBIT Measurement Library) and iRODS data grid repository

– GEMINI: http://groups.geni.net/geni/wiki/GEMINI• Based on PerfSONAR, integrating other key GENI I&M

projects including LAMP and INSTOOLS

• There are archiving services (e.g. iRODS), topology services (e.g. UNIS)

• Each provides mechanisms for instrumenting, capturing and archiving experiment measurements

Sponsored by the National Science Foundation 9March 20, 2013

Challenges and Caveats

• There are variations among implementations of AM API: – Different versions (V1, V2, V3)– Different RSpec extensions– Different compute environments

• There are two independent families of experiment support tools:– Different orchestration mechanisms– Different instrumentation and collection protocols

One theme you will hear at this GEC is “Unified Experimenter Experience” which attempts bridge these gaps

Sponsored by the National Science Foundation 10March 20, 2013

Summary

• There is a significant need and opportunity to develop tools to help experimenters

• There are a lot of services that exist from which useful tools can be constructed

• These services are not as uniform as we'd like: we're trying to unify them as much as possible, but it may fall on the tool developer to bridge some of these gaps, at least in the short term.

Consult the GENI WIKI http://groups.geni.net/geni for details on the essential GENI services

Sponsored by the National Science Foundation 11March 20, 2013

NOTES / BACKUP

Sponsored by the National Science Foundation 12March 20, 2013

NOTES

• GMOC• XMLRPC and Signed Docs• Service Registry and other CH functions

– Show CH diagram

• AM API– Advertisement Rspec (images, VM’s, H/W)

• Slice Authorities• IDP’s• Single-Sign-On and Shib• Resource Allocation• Resource Measurement Repositories/Directories

Sponsored by the National Science Foundation 13March 20, 2013

Notes

• Credentials for common authorization• InCommon/OpenID for SSO• GMOC• OpenFlow switches• Non-compute aggregates: FOAM, ION, WiMAX• L2 vs L3• Topology information available on WIKI pages• What services are there? What are their attributes? How

do you talk to them?• Tools are wrappers around services that make them easier

to use (wrapping, caching, joining, pre-computing)• What do the services provide? What does the experiment

want? Tools can bridge the gap.