software defined network exchanges(sdxs) for data ... · ng optical networks and testbeds, public...
TRANSCRIPT
Software Defined Network Exchanges(SDXs)For Data Intensive Sciences: BioInformatics SDX
Jim ChenAssociate Director
International Center for Advanced Internet Research(iCAIR)Northwestern University
APAN41Manila, PhilippinesJanuary 25-29, 2016
• Creation and Early Implementation of Advanced Networking Technologies - The Next Generation Internet All Optical Networks, Terascale Networks, Networks for Petascale Science
• Advanced Applications, Middleware, Large-Scale Infrastructure, NG Optical Networks and Testbeds, Public Policy Studies and Forums Related to NG Networks
• Three Major Areas of Activity: a) Basic Research b) Design and Implementation of Prototypes c) Operations of Specialized Communication Facilities (e.g., StarLight)
Accelerating Leading Edge Innovation
and Enhanced Global Communications
through Advanced Internet Technologies,
in Partnership with the Global Community
Introduction to iCAIR:
Open Innovation Platforms
• GENI: Global Environment for Networking Innovations, “a virtual laboratory for exploring future internet technologies and protocols at scale”.
• NSF13602 NSFCloud: “enable the academic research community to develop and experiment with novel cloud and cloud-like architectures that can support a diversity of innovative applications”.
• NSF14554 IRNC RXP: “platforms of network innovation supporting the research and education needs of global scientific collaborations”.
www. chameleoncloud.org
CHAMELEON HARDWARE
SCUs connect to core and fully connected to each other
Heterogeneous Cloud Units
Alternate Processors and Networks
SwitchStandard
Cloud Unit42 compute
4 storage
x10
Chicago
To UTSA, GENI, Future Partners
AustinChameleon Core Network
100Gbps uplink public network(each site)
Core Services3.6 PB Central File Systems, Front End and Data Movers
Core ServicesFront End and Data
Mover Nodes 504 x86 Compute Servers48 Dist. Storage Servers102 Heterogeneous Servers16 Mgt and Storage Nodes
SwitchStandard
Cloud Unit42 compute
4 storage
x2
www. chameleoncloud.org
CAPABILITIES AND SUPPORTED RESEARCH
Virtualization technology (e.g., SR-IOV, accelerators), systems, networking, infrastructure-level resource management, etc.
Repeatable experiments in new models, algorithms, platforms, auto-scaling, high-availability, cloud federation, etc.
Development of new models, algorithms, platforms, auto-scaling HA, etc., innovative application and educational uses
Isolated partition, bare metal reconfiguration: OpenStack and Grid’5000
Isolated partition, Chameleon Appliances
Persistent, reliable, shared clouds
Software Defined Networking Exchanges (SDXs)
• With the Increasing Deployment of SDN In Production Networks, the Need for an SDN Exchange (SDX) Has Been Recognized.
• Current SDN Architecture Is Single Domain Centralized Controller Oriented
• Required Capabilities for Multi-Domain Distributed SDN Resource Discovery, Signaling Provisioning, Operations, and Fault Detection and Recovery Are Fairly Challenging.
• Nonetheless – Many Motivations Exist for SDXs
StarLight International Software Defined Networking Exchange (SDX)
• The National Science Foundation-Funded StarLight International SDX Will Provide The Services, Architecture, and Technologies Designed To Provide Scientists, Engineers, and Educators With Highly Advanced, Diverse, Reliable, Persistent, and Secure Networking Services, Enabling Them to Optimally Access Resources in North America, South America, Asia, South Asia (including India), Australia, New Zealand, Europe, the Middle East, North Africa, And Other Sites Around the World.
• The StarLight SDX Initiative Undertakes Continued Innovation and Development of Advanced Networking Services and Technologies.
Sloan Digital Sky
Survey
www.sdss.org
Globus Alliance
www.globus.org
LIGO
www.ligo.org TeraGrid
www.teragrid.org
ALMA: Atacama
Large Millimeter
Array
www.alma.nrao.edu
CAMERA
metagenomics
camera.calit2.net
Comprehensive
Large-Array
Stewardship System
www.class.noaa.gov
DØ (DZero)
www-d0.fnal.gov
ISS: International
Space Station
www.nasa.gov/statio
n
IVOA:
International
Virtual
Observatory
www.ivoa.net
BIRN: Biomedical
Informatics Research
Network
www.nbirn.net
GEON: Geosciences
Network
www.geongrid.org
ANDRILL:
Antarctic
Geological
Drilling
www.andrill.org
GLEON: Global Lake
Ecological
Observatory
Network
www.gleon.orgPacific Rim
Applications and
Grid Middleware
Assembly
www.pragma-
grid.net
CineGrid
www.cinegrid.orgCarbon Tracker
www.esrl.noaa.gov/
gmd/ccgg/carbontrack
er
XSEDE
www.xsede.org
LHCONE
www.lhcone.net
WLCG
lcg.web.cern.ch/LCG/publi
c/
OOI-CI
ci.oceanobservatories.org
OSG
www.opensciencegrid.org
SKA
www.skatelescope.o
rg
NG Digital
Sky Survey
ATLAS
Compilation By Maxine Brown
Motivations for SDXs• White House Office of Science and Technology Policy –
Large Scale Science Instrumentation
• Large Scale Ultra High Resolution Digital Media Services
• Multi-Domain Networks Interconnecting Data Centers (SDN Is Already in Production Within Large Scale Data Centers)
• Multi-Domain SDN Services
• Providing Capabilities for Edge Control
• Democratization Of Exchange Facilities
• Many New Types of Services and Capabilities
• Etc.
Software Defined Networking Exchanges (SDXs)
• With Support From the GENI Project, the International Center for Advanced Internet Research (iCAIR) and Its Research Partners Have Designed and Implemented a Prototype SDX at the StarLight International/National Communications Exchange Facility
• This SDX Is a Multi-Domain Service Enabling Federated Controllers To Exchange Signaling and Provisioning Information.
Selected SDX Architectural Attributes
• Control and Network Resource APIs
• Multi Domain Integrated Path Controller
• Controller Signaling, Including Edge Signaling
• SDN/OF Multi Layer Traffic Exchange
• Multi Domain Resource Advertisement/Discovery
• Topology Exchange
• Multiple Service Levels At All Layers
• Granulated Resource Access (Policy Based), Including Through Edge Processes
• Foundation Resource Programmability
• Various Types of Gateways To Other Network Environments
• Integration of OF and Non-OF Paths, Including 3rd Party Integration
• Programmability for Large Scale Large Capacity Streams
Architectural Components• Hybrid Networking Services (Multi-Service, Multi-
Layer, Multi-Domain)
• Network Programming Languages (e.g., P4, Frenetic)
• Abstraction Definitions
• APIs
• Service Signaling and Policy Bundling
• Policy Bundle Distribution
• Primitives
• BGP Extensions and Substitutes
• NDL Schema
• Orchestration Processes
Other Architectural Components 2• Northbound Interfaces
• Network OSs
• Network Hypervisors
• State Information Data Bases
• Data Modeling Languages (e.g., YANG)
• Controller Federation Processes
• Hybrid Services/Services Federation/Services Chaining
• Southbound Interfaces
• East-west bound Interfaces
Other Architectural Components 3
• Data Plane acceleration
• Network Function Virtualization (NFV)
• Measurements
• Real Time Analytics
• Distributed Virtual NOC Operations
GLIF is a consortium of institutions, organizations, consortia and country National Research & Education Networks who voluntarily share optical networking resources and expertise to develop the Global LambdaGrid for the advancement of scientific collaboration and discovery –a Federation!.
StarLight – “By Researchers For Researchers”
Abbott Hall, Northwestern University’s
Chicago CampusView from StarLight
StarLight is an experimental optical infrastructure andproving ground for network services optimized forhigh-performance applicationsMultiple10GE+100 GbpsStarWaveMultiple 10GEsOver Optics –World’s “Largest”10G/100G ExchangeFirst of a KindEnabling InteroperabilityAt L1, L2, L3
Multiple HPC Cloud Computing Testbeds
Specifically Designed for Science Research
At Scale Experimentation
Integerated With High Performance Networks
=> Open Commons Consortium
The OCC And Next Generation Networking
• The OCC Has Formed a Partnership With The Advanced Networking Community To Created Large Scale (Global) Distributed Environments For Data Intensive Scientific Research – Based On a Foundation of Programmable, Deterministic, Edge Customizable High Performance Large Capacity Networks (e.g., 100 G and 100 G +)
Creating A Data Commons
• Clouds and Data Commons For Global Data Intensive Science
• Vast Amounts Of Data Are Being Created By Many New Powerful Sophisticated Instruments – Zettabytes!
• Few Individual Scientists Have the Resources To Manage, Store, Analyze, and Transport This Data
• A Data Commons Is A Facility Designed To Co-locate Data, Storage, Computing, and Networking Infrastructure, Along With Commonly Used Tools For Analyzing and Sharing Data As A Research Resource.
Core Data Commons Services
• Policy Based Access Procedures, Digital IDs
• Metadata Services
• Storage Services
• High Performance Transport, Including Specialized Transport Services For Large Scale Streams
• Policy Based Data Export Tools/Services
• Compute Services with Images/Containers Integrating Commonly Used Tools, Applications and Services, Specialized for Individual Science Domain Research Communities
GENI Engineering Conference (GEC 19) SDX Demonstration Atlanta March 18-20, 2014
• Initial SDX Capability Between GENI Sites (StarLight and SOX) Was Demonstrated
• Motivation: To Share a Vision of Interconnected US Nationwide SDN Infrastructure, With Multiple SDN Capable Networks and Domains
• SDX Benefits Was Showcased Through a Compelling Application –Nowcast – Developed By Mike Zink and His Colleagues at University of Massachusetts, Amherst
• (GEC 19 Was Co-Located With the GLIF Tech Workshop, March 19-20)
GENI SDX Demo Scenario 1
Starlight SDX
ATLSDX
RENCI
GT(GEC)
R2 R3R1 R4
MiddleBox
Radars
MergeGrid
Nowcast
Publish on web
Starlight SDX
ATLSDX
OSF
GT(GEC)
R2 R3R1 R4
MiddleBox
Radars
MergeGrid
Nowcast
Publish on web
Slide by Mike Zink, UMass Amherst
ESnet
ORNL
AL2S
Georgia
Tech SDX
StarLight
SDX
R4R3R2R1
GENI SDX Demo Scenario 2
Simulated
Radar (4)
GEC 19
Mid
Box
Global LambdaGrid Workshop (GLIF)
• International SDX (iSDX) Demonstrations Showcase a World-wide Prototype Environment That Could Be Used for Modeling Major Weather Systems, Including The Depiction of Severe Weather Patterns.
• Application Based on Nowcast System Being Developed by the NSF Engineering Research Center for Collaborative Adaptive Sensing of the Atmosphere (Led By University of Massachusetts at Amherst), Which Is Being Designed for Next Generation Weather Prediction and Visualization Systems.
Global LambdaGrid Workshop Sept 30-Oct 1 New Zealand
• Demonstrations of A Prototype Implemented Across the Globe Using the GLIF To Interconnect Sites World-Wide, Supported By Interoperable International SDXs, Including Prototype SDXs at – a) the StarLight International/National Communications Exchange Facility, Designed by iCAIR
– b) Auckland, New Zealand, Designed by REANNZ and Google,
– c) Taiwan, Designed by High Performance Computing Center/TWAREN
– d) SOX in Atlanta, Designed by Georgia Tech
– d) NetherLight in Amsterdam, Designed by SURFnet,
– e) Ottawa, Designed by CANARIE and Cybera,
f) Tokyo, Designed by the University of Tokyo
• Each Site Had Its Own SDN/OpenFlow Controllers, Which Were Federated To Enable Cross-Domain Interoperability.
• Controllers Used Distributed Control Plane To Directly Address and Dynamically Manage Multiple Paths Among Sites Via Distributed Data Plane - Transporting
Nowcast Instrumentation Traffic Among Sites.
Global Research Platform
• Currently – A Concept
• A Specialized Globally Distributed Platform For Science Discovery and Innovation
• Based On State-Of-the-Art-Clouds
• Interconnected With Computational Grids, Supercomputing Centers, Specialized Instruments, et al
• Also, Based On World-Wide 100 Gbps Networks
• Leveraging Advanced Architectural Concepts, e.g., SDN/SDX/SDI – Science DMZs
• Ref: Demonstrations @ SC15, Austin Texas November 2015
Pipe-
lines
Proc-
essing
100G
Crypto
SCInet
Oakland, CA
NERSC
Chicago StarLight
Austin, TX
SC15
Controller
Pipelines
Manager
Software Defined Exchange
Selective Data
Display
& User Interaction
100G
Crypto X
X
NERSC-
TB1
AOFA-
CR5
Source
Data
N x 4K
Video
Dynamic Remote I/O Network
Remote Data Source
(Gigapixels/sec)
Network on
demandProcessing on
demand
Pipe-
lines
Proc-
essing
Washington, DC
NRL
CTL
STAR-
TB1
40
G40
G40
G
Live Video
4K x 60 fps
Pipe-
lines
Proc-
essing
Source
Data
N x 4K
Video
Pipe-
lines
Proc-
essing
>95% BW
Efficiency
100G Encryption
(MACSEC)
Processing on
demand
Network/cry
pto on
demand
Dynamic Remote I/O Demonstration of large-scale remote data access, a dynamic pipelined distributed processing framework and software defined networking enabled automation between distant operating locations. Live production quality 4K video workflows across a nationally distributed set of storage and computing resources - relevant to emerging data processing challenges.
CTL
ExoGENI @ UvA
SAGE2 VMServer
Tiled Display@ iCAIR
SAGE2 Display
Tiled Display@ Ottawa
SAGE2 Display
PublicInternet
Google Maps, Weather Data, Etc
Starlight
10G10G
10G
1G1G
1G
SAGE2 10GImagesVideo
Desktop SharingApplications
Desktop Sharing
ONOS framework for resource discovery, allocation and reservation
On demand allocation of multi-domain (SDX) network resources for collaborative sessions
On demand exploration of large remote data setsLeverage high speed network for high quality
video streaming (QoS) Secure access through ScienceDMZSupport for VM deployments
Network connectivity/performance visualization
:Remote Resource Plans
Automated GOLE Fabric
JGN-X
NetherLight
CERNCaltech
UvACzechLight
KRLight
AIST
KDDI Labs
StarLightGLORIAD
GÉANT
ANA & EEX
NORDUnetANA
ESnet
Pionier
Caltech
AMPATH
SouthernLight
MAN LAN
Arista
Controller Controller
Controller Controller
BDDP
BDDP
StarLight
NCHC
iCAIR
KISTI
CANARIE
Juniper
Arista(vlan Translation)
vlan2782
vlan2782
vlan2780
vlan2780
vlan2780
vlan2780
SC14-SC15 NSI-SDN-Topology
NSA
NSA
NSA
NSA
NSA
A Peer-to-Peer Distribution Protocol
• Once an NSA comes up, it contacts its neighbors to request the topology information
iCAIR
NCHCKISTI
CANARIE
StarLight
1. Request
2. Response
controller
NSA
update
NSA ANSA B
NSA C
NSA D
Topology Exchange (from NSA A)
NSA A Update topology & link cost computation (to Domain B) from Controller A
Contacts neighbor NSA peer B with AA
By policy, NSA B replies info of its peer NSA C and D with link costs
NSA A Contacts remote NSA peer C and D with AA and obtain the link between C and D by policy
A DB
C
Ctrl ACtrl B
Ctrl C
Ctrl D
NBI
PetaTrans: Petascale Sciences Data Transfer
SDXs100G
Switches
PetaTrans@100G
ExoGENI@40G
100GSwitch
100GSwitch
PacificWave100G
Switches
100GSwitch
ExoGENI@100G
ExoGENI@40G
HECN 100G Node
Seattle/California
Washington D.C.
Chicago
Ottawa
Amsterdam
PRP SitesESnet/ANA200
@100GPacificWave
/GRPNet@ 2 X 100G
ESnet/MAX @ 100G
@ 100G
Persistence 100G Services Beyond SC15
100GE End to End Services based on Open Architecture (E2SOA) for Peta
Scale Sciences: Prototype Set Up
StarLightPetaTrans1
StarLightPetaTrans
2
InventecOCP 100G
Switch
Ciena8700
MREN8700
ANA200Nether Light8700
InventecOCP 100G
Switch
SURFNet100GENode
StarLight SDXService Hub
OpenFlow NSI/openNSACANARIE
CANARIE
6500CIENA 8700
ExoGENI40G
Nodes
Hospitals,
Doctors
Cloud Computation
Genomic Data Commons
Patients
Output: Data-
Aware,
Analytics-Informed
Diagnosis,
Prognosis,
Optimal Treatment
Future Vision:
A Nationwide Virtual Comprehensive Cancer Center
Biomedical Data Commons:Flow Orchestration: Control Plane + Data Plane
Data Plane
Control Plane
Data Repository A (West Coast)
Data Repository C (Asia) Data Repository D (Europe)
Data Repository B
(South)
Visualization Engines
North AmericaCompute Engines
(Midwest)
DiCOS
Distributed Cloud Operating System
Based on Grid Computing’s middleware
(Large Hadron Collider experiment, CERN)
The middleware links more than 260,000 processing cores
and 180 PB disk storage
Got a successful experience from high energy Physics field
Promote the system to biological/genetic field
Provide Web-UI and CLI
Other field’s applications are also welcome
Academia Sinica Grid Computing Center,
Taiwan
RNA Seq Data Queue
Software Repository
DJM
Initiate aProcess Job
DDM Supported
http, xml
Pilots
WNWN
SESESE
DiCOS DDM
Pilot Factory
Cloud Storage
ASGC
Platform and Software
Configuration Management
LocalCache
NCU, NCHC
Pull
BDCVDB
Parcel/UDPOver SDN
VisualizationR/ROOT
CERN
DiCOS for BDC-TP
Xrootd/GridFTP
U of Chicago/StarLight FASTQ/
BAM
Data Bundle
Slide source: Eric Yen/ASGC
www.startap.net/starlight
Thanks to the NSF, DOE, NIH, USGS, DARPA
NOAA, Universities, National Labs,
International Partners,
and Other Supporters