Sponsored by the National Science Foundation

GEC16 Service Developers Roundtable:Strawman Unified I&M Tools and Services

Marshall Brinn, GPO

March 19, 2013

Sponsored by the National Science Foundation 2

Overview

• Goal: Provide researchers with an intuitive, powerful set of tools to support experimentation

• Problem: We have two different families of tools (GIMI, GEMINI) that have different architectures, interfaces and are (currently) focused on different compute platforms

• Solutions:– Make all services available on all platforms– Provide common ‘base’ interface that allows

experimenters to manage experiments in a platform-agnostic manner

Sponsored by the National Science Foundation 3

I&M Proposed Simple Common API# Let's keep it very simple.# Something like:

# Open a session for storing metricssession_url <= open_session(session_name)

# Close a session for storing metricsstatus <= close_session(session_url)

# What are the metrics I can ask for on this platform?metric_names <= available_metrics()

# What are the attributes for a given metricattributes <= metric_attrributes(metric_name)

# Start metric capture# Specify attributes tailoring the nature of capture (frequency, detaile.g)metric_capture_id <= start_metric_capture(session_url, metric_name, attribs)

# Stop metric capturesuccess <= stop_metric_capture(session_url, metric_capture_id)

# Gather all data for a given session# Optionally filtered by attributes of matching metric typesdata <= grab_session_data(session_url, attributes=None)

Sponsored by the National Science Foundation 4

I&M Proposed Simple common API

• Clearly this is a simple API, and one can do much more with either GIMI and GEMINI than is available through this API.– Haven’t discussed common analysis or orchestration,

e.g.– Nor have we touched on common resource topology

allocation• There is considerable non-uniformity in this area as well

– But it is a starting point and we can add to it as we go

• Just getting this basis established will be a major win for the experimenter community– Supporting common scripts that can run on ANY GENI

topology provided from ANY GENI aggregates.

Sponsored by the National Science Foundation 5

BACKUP / NOTES

Sponsored by the National Science Foundation 6

• Strawman Combined Tool– 2 slides: MSB

• Roundtable discussion– 15 minutes: Ilia, Rob, Tom, Niky, MSB– Discuss common graphical resource assignment tool,

and relationship to portals– Bring in discussion of Solicitation 4

Sponsored by the National Science Foundation 7

• Orchestration• Instrumentation• Monitoring• Archiving• Analysis

Sponsored by the National Science Foundation 8

I&M

• We know that there are two different I&M tool suites being developed and deployed: GIMI and GEMI– Each is developed on one platform, and is slated to be

ported to the other platform this year (though personally I think this is a stretch).

• I would like to define a common set of API’s that will be available on any GENI image supplied by GENI racks.

Sponsored by the National Science Foundation 9

I&M : Proposal

• I suggest we write a simple API for generic I&M support– And then write implementations of this API that run on

any given image. – The image may support this implementation with GIMI

or with GEMINI (or perhaps a hybrid)

Sponsored by the National Science Foundation 10

I&M [3]

• There’s lots more to do in this area (Orchestration, data analytics, etc.)

• But I think this is a very good start and if we only do this much we’ll have done a lot.

This effort is likely to be less in the hands of the Architects than in the members of the different I&M teams. But I want us to monitor and encourage this kind of common functionality.

Sponsored by the National Science Foundation 11

Notes

• Common API for writing, requesting, tagging• Different underlying implementations• From PORTAL• PORTAL : SSO integration of separate tools• Allocation time : AM API (initialization scripts)• Execution time : Scripts, services within slivers• LCD + Value Added differentiators