© 2009 – rnp large-scale testbeds for network r&d planetlab everywhere rio de janeiro, april...

© 2009 – RNP

Large-scale testbeds for network R&D

PlanetLab EverywhereRio de Janeiro, April 2009

Michael Stanton Rede Nacional de Ensino e Pesquisa - RNP

[email protected]

Summary

• We discuss R&D testbeds for networking and distributed systems, seen from the point of view of RNP

• In addition to activities carried out in Brazil, we also examine some testbeds in use or planned in other countries, especially in relation to future Internet development

• In conclusion suggestions are made about future steps to be followed in Brazil.

Large-scale testbeds for R&D in Brazil 2

Large-scale testbeds for R&D in Brazil

Introduction to Brazil• In 1494 Spain and Portugal had

divided between themselves undiscovered lands by the Treaty of Tordesillas

– The Tordesillas Line was to be the frontier between the dominions of Spain (W) and Portugal (E)

• Brazil is the successor country to the Portuguese dominions in South America

– Rather over one half of present Brazil lies to the WEST of the Tordesillas Line

• Brazil is a BIG place!– diameter of about 4,200 km– 42 x 42 = 1764 ms2

• Current population of about 180 millions, unevenly distributed

– most of the population and communications infrastructure concentrated to the EAST of the Tordesillas Line

Tordesillas Line

to Spain to Portugal

3


RNP – Rede Nacional de Ensino e Pesquisa

• RNP is the Brazilian NREN– maintained by the Brazilian government (since 1989) to enable

network access to the national research and education community

– provides national (inter-state) and international R&E connectivity for more than 300 public and private universities and research centers through the provision of advanced networking infrastructure

• also provides commodity access – one-stop shopping

– promotes the development of advanced networking and applications

• Since 2000, RNP is managed for the federal government by a non-profit private company, RNP-OS, legally recognised as an “Organização Social”, which allows the government to contract its services without competitive tender.

4


RNP’s service networksRNP includes the following funded connectivity:

• National backbone network – Rede IPÊ– 1 PoP (Point of Presence) in each state – usually a federal

university– Link capacity depends on the available telco infrastructure– Currently from 2 Mbps to 10 Gbps

• Direct intercity connections between state PoP and non-local federal instituions (education, science and technology)– Currently from 2 to 155 Mbps (depends on the institution)

• Community-based optical metro networks connected to PoPs – Currently being built out – 9 out of 27 already in operation

No service charges are made to end user institutions• Non-federal institutions are normally required to fund their own

access links5

National integration

RedClara, beyond 2008• Extend RedClara to all

LA&C countries

• Promote applications in education and health

• Start of a new project (ALICE2), with the support of the EC, in 2009

Source: www.redclara.net

7Large-scale testbeds for R&D in Brazil

A world-class network – RNP in GLIF

Source: www.glif.is


A world-class network – worldwide GLIF

Source: www.glif.is9Large-scale testbeds for R&D in Brazil


Rede IPÊ – national backbone networkLast big reform in 2005 (5th

phase)

Capacity reflects available telco infrastructure

Currently composed of:• Multigigabit core network

– 4 PoPs at 10 Gbps, and 6 PoPs at 2.5 Gbps

– IP over lambdas (12.000 km)

• Terrestrial SDH connections to 15 PoPs– Most links are 34 Mbps– Some at 2 Mbps– Some upgrades in 2007 to

102, 155 and 622 Mbps• 2 PoPs connected by satellite

at 2 Mbps

10


Evolution of academic networks in Brazil

RNPPhase

Year Technology Link capacities Comment

1988 BITNET up to 9.6 kbps first national network

1 1992 Internet 9.6 and 64 kbps first national IP network (RNP)

2 1995 up to 2 Mbps also: commercial IP deployed

3 1999 IP/ATM, IP/FR

VC up to 45 Mbps, access up to 155

Mbps

RNP2 national backbone;

testbed metro networks in 14 cities (using ATM/dark fiber)

4 2003 IP/SDH 34, 155, 622 Mbps also: IP/WDM interstate testbed network (Project GIGA)

5 2005 IP/WDM 2.5 and 10 Gbps IPÊ national backbone;

metro networks in 27 capitals

11


Evolution of academic networks in Brazil

Capacidade dos enlaces

1

10

100

1.000

10.000

100.000

1.000.000

10.000.000

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Ano

kbp

s

Phase 0BITNET

Phase 1Internet

Phase 2comercial Internet

Phase 3RNP2

Phase 4RNP2+

Phase 5Ipê

(Link capacity)

12

RNP and its innovating networks

• The first version of RNP’s network was deployed in 1992, and pioneered in using TCP/IP technology nationwide

• Since then, the different generations of network deployed and operated by RNP have innovated technically, at least within Brazil

• With the development of the network industry here, since the beginning of commercial IP networks starting in 1995, RNP is no longer the only IP network operator in Brazil, but continues to lead technologically, in the pursuit of new models of infrastructure and applications


Technology changes in the network

• The beginning of each new technology phase was a step in the dark– new circuits (or service, in the case of ATM/FR) were

ordered and delivered, and equipment configured– after configuration by the network engineers, the new

network would begin to work, and would be put into operation as soon as possible.

– users could then begin to use the resources of the new network

• Problems:– lack of familiarity with the new technologies before

operational deployment– the technology transition became a singularity


New application services

• Internet technology is very accommodating of new application services– any user can develop a new service, impelemnted as a

distributed application using the sockets API– this permits and encourages experimentation with

applications that can be built initially in the laboratory (in a LAN environment) and then migrated to the wide area network

• Some problems which arise:– some distributed services require componentes “within the

network” – security problems – performance monitoring may be a problem


Alternative solutions

• Both for

– preparing an upcoming network technology change

– developing a a new large-scale distributed application

we really need a testbed facility, isolated from the production network, and which reproduces its characteristics of scale and performance

• This isolation can be real or virtual

– Real – the testbed is based on separate physical infrastructure, independent of the production network

• example: Project GIGA

– Virtual – the testbed shares the same infrastructure used by the production network

• example: PlanetLab



Project GIGA – optical networking testbed

• Partnership between – RNP , CPqD (telco industry R&D centre in Campinas, SP), R&D

community in networks and distributed systems– Financed by FUNTTEL between 2002 and 2007 – US$20M– telcos – provide optical fiber at no cost

• Objectives:– build an advanced networking testbed for development and

demonstration purposes– support R&D subprojects in optical and IP networking technology and

advanced applications and services• Network support (since May 2004)

– R&D subproject consortia provided with internal connectivity using VLANs – 20 institutions connected

– however: the testbed (with some exceptions) did not provide exteral connectivity, limiting its usefulness

FUNTTELFUNTTEL


GIGA testbed network - location

• dark fibre-based 700-km inter-city backbone in states of São Paulo and Rio de Janeiro (south-east Brazil)

• Initially 20 universities and R&D centers in 7 cities

• 2.5G DWDM in the inter-city backbone

• 2.5G CWDM used in the metropolitan area

testbed network


GIGA testbed network - location

UniversitiesIMEPUC-RioPUC-CampinasUERJUFFUFRJMackenzieUNICAMPUSP

R&D Centers CBPF CPqDCPTECINCORCTA FIOCRUZ IMPA INPE LNCC LNLS


Testbed network design

• 2.5G DWDM inter-city network between Campinas and Rio de Janeiro (some upgrades to 10G)

– up to 6 waves per link (can use 8)• 2.5G CWDM metro networks in São Paulo, Campinas

and Rio de Janeiro• all links currently 1 Gigabit Ethernet

– optical equipment from the Brazilian firm, Padtec (www.padtec.com.br)

– IP equipment from Extreme Networks S.J. dos Campos

São Paulo

Campinas

Rio de Janeiro

Campinas

São Paulo S. José dosCampos

Rio de Janeiro

CachoeiraPaulista

MANCP

MANSP

MANRJ

Petrópolis

Niterói

2λ

2λ

2λ

1λ3λ

1λ

2λ1λ3λ

What is PlanetLab?(base d on slides by Marc Fiuczynski, Sept. 2007)

• Facility: Planetary-scale “overlay” & “underlay” network

– 900+ Linux-based servers at 400+ sites in 40+ countries

– Currently there exist a handful of PL sites in Brazil operated by RNP and a few universities and research institutions

PlanetLab Facility Today

1000+ servers at 450+ sites in 40+ countriesCo-located throughout the world @ Uni. & Companies

Co-located at network crossroads (Internet2, RNP, CERNET, …)


Virtualization in PlanetLab?

• Research Community: Distributed Sys. & Networking

– Researchers can get a set of “virtual machines” across these servers (SLICE)

– In a SLICE researchers can deploy & evaluate …

– … distributed systems services and applications

“The next Internet will be created as an overlay in the current one”

– … network architectures and protocols

“The new Internet will be created in parallel next to the current one”(see later)

Example Network Services

• Scalable Large-File Transfer: CoBlitz—Princeton, LoCI—Tennessee • Content Distribution: Coral—NYU, CoDeeN—Princeton, CobWeb—Cornell• Distributed Hash Tables: OpenDHT—UC Berkeley; Chord-MIT• Routing Overlays: I3 Internet Indirection Infrastructure—UC Berkeley• Multicast Delivery Nets: End System Multicast—CMU, Tmesh-U. Michigan• Serverless Email: ePOST—Rice University• Publish-Subscribe News Access: CorONA—Cornell• Robust DNS Resolution: CoDNS—Princeton, CoDoNs—Cornell• Mobile Access: DHARMA—U. of Pennsylvania• Location/Anycast Services: OASIS—NYU, Meridian—Cornell• Internet Measurement: ScriptRoute—U. of Maryland

• Above services communicate with >1M real users and transmit ~4TB of data per day


PlanetLab Node Software Architecture

SliceManager

(SM)

Virtualization Software

x86 Server Hardware

Sli

ce

Sli

ce

Sli

ce

Sli

ce25Large-scale testbeds for R&D in Brazil

Slices


Thoughts about the future Internet

• The success of the Internet has been so enormous that it is tempting to imagine its future by extrapolating from the present

• However, there are consequences of its design, based on decisions taken in the 1970s, which severely limit its security, availability, flexibility and manageability

• These limitations can not be removed through small incremental adjustments of the existing network, and, if they are not removed, they will create huge impediments to the ability to use and exploit the Internet in the future


Removing these limitations

• For many years, combatting Internet limitations has been carried out through a series of “patches”, introduced to solve specific problems.

• Unfortunately, these patches result in increased complexity, resulting in a less robust system, whose operation has become more difficult and costly

• A growing consensus exists in the network research community that we have already reached the point where patches are inadequate, and a fundamental rethink of the Internet is required

(from the GENI Research Plan, 2007)


GENI - http://www.geni.net

(Global Environment for Network Innovations)• An initiative of NSF (USA) to create a shared testbed

environment to allow the validation of new network architectures– Initial phase: 2005 to 2007 – design of the GENI

environment– Present phase: since 2008, deployment and use

• GENI will support research which can lead to a future Internet with improved chacteristics– more comprehensive security– greater generality– better integration of optical and wireless technologies– integration of the world of sensors and embedded

processors– improved options for the economic health of ISPs


The GENI testbed environment

GENI will:• allow experiments with alternative large-scale network

architectures, services and applications under realistic conditions

• use vitualizations to permit carrying out multiple independent and simultaneous experiments

• permit long-duration experiments, allowing mature prototypes to serve “living” user communities

• facilitate experimental research through the use of extensive tools for measurement and data collection

In summary, GENI will provide support for the taking ideas on large-scale ideas from their conception to their deployment, by means of experimental validation


How the GENI environment will be built

• The GENI environment is inspired on PlanetLab, and especially the “Meta-Testbed” VINI - http://www.vini-veritas.net

• VINI extends the scope of PlanetLab to allow– “slicing” of links between the nodes (link virtualization) – substitution of level 3 protocols (IP)


Next Step: Meta Testbed(base d on slides by Marc Fiuczynski, Sept. 2007)

• Goals– support experimental validation of new architectures

• simultaneously support real users and clean slate designs

• allow a thousand flowers to bloom

– provide plausible deployment path• Key ideas

– virtualization• multiple architectures on a shared infrastructure

• shared management costs

– opt-in on a per-user / per-application basis• attract real users

• demand drives deployment / adoption


VINI: Our Meta Testbed approach

• Infrastructure– PlanetLab provides “access network” with global reach

• user desktops run proxy that allows them to opt-in

• treat nearby PlanetLab node as ingress router

– NLR/I2 provides high-speed backbone in the United States• populate with programmable routers

• extend slice abstraction to these routers

• Usage model– each architecture (service) runs in its own slice– two modes of use

• short-term experiments

• long-running stable architectures and services


Extending Slices to a VINI testbed


User Opt-in

Client

ServerNAT

wireless


Internet in a Slice (IIAS)

XORP(routing protocols)

vif1 vif2vif0

IPv4Fwd table

User

Kernel

Filters, shapers

PlanetLab VM

E-GRE tunnels

XORP in Network Container– Adds routes to copy of kernel IPv4

forwarding table– Kernel forwards packets between

virtual interfaces

Filters and shapers– Add delay and loss, constrain

bandwidth

Virtual interfaces– Appear as Ethernet devices in a

slice– Reduce MTU for tunneling

E-GRE tunnels– Hack standard GRE tunnels to

preserve MAC headers


GENI

• Extenssion of VINI – key ideas:– virtualization

• multiple network architectures sharing a common infrastructure

– user opt-in: per user / per application• intended to attract real users

• Infrastructure– NLR/Internet2 provide high-capacity backbone in the US

• populate with programmable resources (processors, storage)

• populate with programmable routers– more sophisticated than the PCs used in VINI

• extend the “slice” abstraction to these routers

• include “extensions” to wireless and sensor networks


GENI: The Physical Network(slides by C. Qiao, 2008)

• Large-Scale Facility of Networked Systems– Reasonable Representation of the Internet’s Complexity– A Nationwide Optical Network ~ 200 Universities– Clusters for Processing/Storage– Wireless Access Networks

• Mobility, Location Awareness

– Sensor Networks– Connected to a large number of User Communities

• Partnerships to Extend GENI within the US– Add Technologies and Users

• Federation to Extend GENI on a Global Scale


Core Nodes

Schematic GENI Network

Mobile Wireless Network

Edge Site

Sensor Network

Edge Nodes

FederatedInternational Facility


Programmability

All network elements programmable via open interfaces and/or downloadable user code

ProgrammableSensor Node

Open APIRadio platform

ProgrammableEdge NodeProgrammable

Core Node

GENI Control & Management Plane


Slicing and Virtualization

Mobile Wireless Network Edge Site

Sensor Network

- share resources to support many simultaneous experiments45Large-scale testbeds for R&D in Brazil

GENI Design Principles

• Physical network ‘substrate’– building block components– elements / nodes / links / subnets

• Software control & management framework– knits building blocks together– allows many parallel experiments (slices)– creates arbitrary logical topologies (virtualization)

• Programmable for ‘Clean Slate’ research• Instrumented for accurate analysis• Flexible and Phased Design

– Support Technology Introduction during GENI Lifetime


GENI Phase 1(2008-9)(based on slides by Chip Elliott – GENI program director)

• Provides the very first, national-scale prototype of an interoperable infrastructure suite for Network Science and Engineering experiments

• Creates an end-to-end GENI prototype in 6-12 months with broad academic and industrial participation, while encouraging strong competition in the design and implementation of GENI’s control framework and clearinghouse

• Includes multiple national backbones and regional optical networks, campuses, compute and storage clusters, metropolitan wireless and sensor networks, instrumentation and measurement, and user opt-in

• Because the GENI control framework software presents very high technical and programmatic risk, the GPO has funded multiple, competing teams to integrate and demonstrate competing versions of the control software in Phase 1


GENI phase 1: integration:5 competing control schemes


Research using GENI - FIND

• Prior to the building of the GENI testbed, NSF launched the initiative FIND – Future Internet Design, with the aim of identifying and financing research activities

• http://www.nets-find.net/ • One product of this initiative is the GENI Research Plan, which

details the research motivation for GENI, and some of the research goals which will become possible:

• http://www.geni.net/GDD/GDD-06-28.pdf


Experimental activities in Europe

FIRE - Future Internet Research & Experimentation

http://cordis.europa.eu/fp7/ict/fire/home_en.html• European initiative directed towards the design of the future Internet,

similar to the FIND and GENI initiatives of the NSF• promotes the concept of experimental research, combining visionary

academic research with validation and experimentation typical of industry

• aims to create a multidisciplinary research environment to investigate and validate experimentally innovative ideas for new paradigms of networks and services

• plans to create a “European Experimental Facility” (EEF), formed by the interconnection and federation of both existing and future testbeds, for emerging and future Internet technologies

• First projects selected in 2008


FIRE/Panlab (FOKUS/DE)– www.panlab.net


FIRE/OneLab (UPMC, FR) – www.onelab.eu

• History:– Mar/04: based on ENEXT (Identification of critical testbeds

for networking research)– Sep/06: OneLab1 project funded as an IST project under the

FP6 funding program:10 partners, 2 years– Sep/08: proposal funded as an IST project under

the FP7 funding program: 26 partners, 2 years• OneLab1 – Goals

– extend PlanetLab into new environments, beyond the traditional wired internet

– improve PlanetLab’s monitoring capabilities– provide a European administration for PlanetLab nodes in

Europe


FIRE/Federica – www.fp7-federica.eu


Comparison of FIRE prototypes


Future Internet activities in Japan

• AKARI - http://akari-project.nict.go.jp/eng/overview.htm – plans to deploy a new generation network by 2015, based

on a new architecture– supposes the use of an experimental testbed incorporating

virtualization techniques, as in GENI


Perspectives in Brazil

• Phase 2 of Project GIGA (RNP- CPqD)– original Project GIGA funding ended in Dec/2007, although

the experimental network continues to operate– since 2007 RNP and CPqD have been seeking to maintain

an experimental facility for their research communities– RNP’s proposal is to place greater emphasis on research

into architectures for a future Internet


Perspectives in Brazil

• INCT/Web Science (consortium led by UFRJ) – more than 100 researchers – approved in 2008 (3 to 5 years)

• A group of 8 researchers from (RNP, UFF, UFPA, UNIFACS, USP) included a research proposal in “Future Internet Architectures”

– main emphasis on experimental research, using an environment based on PlanetLab / VINI, with extensions for wireless access networks

– RNP network will be used for long-distance communications (within Brazil and externally)

• Note:

– VINI requires access to a “lower than level 3” network – this depends on the next phase of the RNP network (FuturaRNP), expected in 2010


FuturaRNP and 2010

• RNP is engaged in several initiatives, which are changing the face of its infrastructure:– adoption of a layer 2 national backbone, and introduction of

static and dynamic end-to-end circuits as well as routed IP (Hybrid Network)

– extension to all 27 capitals and 10 other cities of optical metro networks based on Ethernet technology (layer 2)

– huge increase in capacity, when possible through multiple lambdas in DWDM systems

• A possible desirable consequence would be the permanent reservation of capacity for experimental activities, segregated from production traffic (à la GENI)


Federation with other initiatives

• The deployment of an experimental facility in Brazil to support research into new architectures and applications will simplify international collaboration with similar initiatives abroad

• This would be brought about by the interconnection (federation) of the national facility with similar facilities in other countries

• It should be noted that this style of federation is one of the characteristics of the projects we have described from the USA and Europe.


The Future Internet will be polymorphic


Michael Stanton ([email protected])

www.rnp.br

Thank you!

Yellow ipê in blossom

© 2009 – rnp large-scale testbeds for network r&d planetlab everywhere rio de janeiro, april...

Documents

rd testbeds

brazillargescale testbeds

rnplargescale testbeds

network rnp

network access

federal government

state pop

national research