1

Re-Thinking Internet Architecture• Today’s Internet

– Original Design Goal, Philosophy and Principles– End-to-End Principle and “Hourglass” Architecture of

Internet– Pros and Cons; Challenging Issues– What have changed? What may have yet to come?

• Overlay Networks

• Future Internet Architectures?– What are key challenges/issues?

• E.g., mobility, security, “services-oriented” …• Diversity of “end systems”: laptops, cell phones,

sensors, …

2

Network Architecture

What is (Network) Architecture?– not the implementation itself– “design blueprint” on how to “organize”

implementations• what interfaces are supported• where functionality is implemented

• Two Basic Architectural Principles – Modularity (e.g., layering)

• how to break network functionality into modules– End-to-End Argument

• where to implement functionality

3

Architectural Principles (not unique to networks!)

Zhi-Li’s version (synthesized from various sources)• End-to-end argument

– functionality placement• Modularity

– Increase inter-operability and manage complexity• vertical partition -> layered architecture• horizontal partition?

• Keep it simple, stupid (KISS principle)– Occam’s Razor: choose simplest among many solutions!

• complicated design increases system coupling (inter-dependence), amplifies errors, ..

• don’t over-optimize!• Separating policies from mechanisms

– decouple control from data– “semantics-free”

• Design for scale – hierarchy, aggregation, …

4

Some Design/Implementation Principles

• virtualization• indirection• soft state vs. hard state• fate sharing• randomization• expose faults, don’t suppress/ignore• caching• ……

5

Original Internet Design Goals[Clark’88]

0 Connect existing networks– initially ARPANET and ARPA packet radio network

1. Survivability- ensure communication service even with network and

router failures 2. Support multiple types of services3. Must accommodate a variety of networks4. Allow distributed management5. Allow host attachment with a low level of effort6. Be cost effective

7. Allow resource accountability

In order of importance:

6

Priorities• The effects of the order of items in that list are

still felt today– E.g., resource accounting is a hard, current research

topic• Different ordering of priorities would make a

different architecture!• How well has today’s Internet satisfied these

goals? • Let’s look at them in detail

7

Summary: Internet Architecture

• Packet-switched datagram network

• IP is the “compatibility layer” – Hourglass architecture– All hosts and routers run

IP• Stateless architecture

– No per flow state inside network

IP

TCP UDP

ATM

Satellite

Ethernet

8

Summary: Minimalist Approach

• Dumb network– IP provide minimal functionalities to support

connectivity• Addressing, forwarding, routing

• Smart end system– Transport layer or application performs more

sophisticated functionalities• Flow control, error control, congestion control

• Advantages– Accommodate heterogeneous technologies

(Ethernet, modem, satellite, wireless)– Support diverse applications (telnet, ftp, Web, X

windows)– Decentralized network administration

• Beginning to show age– Unclear what the solution will be probably IPv6?

9

Questions• What priority order would a commercial

design have?• What would a commercially invented

Internet look like?• What goals are missing from this list?• Which goals led to the success of the

Internet?

10

Requirements for Today’s Internet

Some key requirements (“-ities”)• Availability and reliability

– “Always on”, fault-tolerant, fast recovery from failures, …• Quality-of-service (QoS) for applications

– fast response time, adequate quality for VoIP, IPTV, etc.• Scalability

– millions or more of users, devices, …• Mobility

– untethered access, mobile users, devices, … • Security (and Privacy?)

– protect against malicious attacks, accountability of user actions?• Manageability

– configure, operate and manage networks– trouble-shooting network problems

• Flexibility, Extensibility, Evolvability, ……? – ease of new service creation and deployment?– evolvable to meet future needs?

11

End System Based Overlay/P2P Services/Solutions

• General Concept of Overlays • Some Examples• End-System Multicast

– Rationale– How to construct “self-organizing” overlay– Performance in support conferencing applications

• Internet Indirection Infrastructure (i3)– Motivation and Basic ideas– Implementation Overview– Applications

12

Overlay Networks

13

Overlay NetworksFocus at the application level

14

Overlay Networks• A logical network built on top of a physical

network– Overlay links are tunnels through the underlying

network• Many logical networks may coexist at once

– Over the same underlying network– And providing its own particular service

• Nodes are often end hosts– Acting as intermediate nodes that forward traffic– Providing a service, such as access to files

• Who controls the nodes providing service?– The party providing the service (e.g., Akamai)– Distributed collection of end users (e.g., peer-to-

peer)

15

Routing Overlays• Alternative routing strategies

– No application-level processing at the overlay nodes– Packet-delivery service with new routing strategies

• Incremental enhancements to IP– IPv6– Multicast– Mobility– Security

• Revisiting where a function belongs– End-system multicast: multicast distribution by end

hosts

• Customized path selection– Resilient Overlay Networks: robust packet delivery

16

IP Tunneling• IP tunnel is a virtual point-to-point link

– Illusion of a direct link between two separated nodes

• Encapsulation of the packet inside an IP datagram– Node B sends a packet to node E– … containing another packet as the payload

A B E FtunnelLogical view:

Physical view:A B E F

17

6Bone: Deploying IPv6 over IP4

A B E F

IPv6 IPv6 IPv6 IPv6

tunnelLogical view:

Physical view:A B E F

IPv6 IPv6 IPv6 IPv6

C D

IPv4 IPv4

Flow: XSrc: ADest: F

data

Flow: XSrc: ADest: F

data

Flow: XSrc: ADest: F

data

Src:BDest: E

Flow: XSrc: ADest: F

data

Src:BDest: E

A-to-B:IPv6

E-to-F:IPv6

B-to-C:IPv6 inside

IPv4

B-to-C:IPv6 inside

IPv4

18

MBone: IP Multicast• Multicast

– Delivering the same data to many receivers– Avoiding sending the same data many times

• IP multicast– Special addressing, forwarding, and routing schemes– Not widely deployed, so MBone tunneled between

nodes

unicast multicast

19

End-System Multicast• IP multicast still is not widely deployed

– Technical and business challenges– Should multicast be a network-layer service?

• Multicast tree of end hosts– Allow end hosts to form their own multicast tree– Hosts receiving the data help forward to others

20

RON: Resilient Overlay NetworksPremise: by building application overlay network,

can increase performance and reliability of routing

Two-hop (application-level)Berkeley-to-Princeton route

application-layer router

Princeton Yale

Berkeley

21

RON Can Outperform IP Routing

• IP routing does not adapt to congestion– But RON can reroute when the direct path is

congested

• IP routing is sometimes slow to converge– But RON can quickly direct traffic through

intermediary

• IP routing depends on AS routing policies– But RON may pick paths that circumvent policies

• Then again, RON has its own overheads– Packets go in and out at intermediate nodes

• Performance degradation, load on hosts, and financial cost

– Probing overhead to monitor the virtual links• Limits RON to deployments with a small number of

nodes

22

Secure Communication Over Insecure Links

• Encrypt packets at entry and decrypt at exit• Eavesdropper cannot snoop the data• … or determine the real source and destination

23

Communicating With Mobile Users

• A mobile user changes locations frequently– So, the IP address of the machine changes often

• The user wants applications to continue running– So, the change in IP address needs to be hidden

• Solution: fixed gateway forwards packets– Gateway has a fixed IP address– … and keeps track of the mobile’s address changes

gatewaywww.cnn.com

24

Unicast Emulation of Multicast

End Systems

Routers

Gatech

CMU

Stanford

Berkeley

25

IP Multicast

• No duplicate packets• Highly efficient bandwidth usage

Key Architectural Decision: Add support for multicast in IP layer

Berkeley

Gatech Stanford

CMU

Routers with multicast support

26

Key Concerns with IP Multicast

• Scalability with number of groups– Routers maintain per-group state– Analogous to per-flow state for QoS guarantees– Aggregation of multicast addresses is complicated

• Supporting higher level functionality is difficult– IP Multicast: best-effort multi-point delivery service– End systems responsible for handling higher level

functionality – Reliability and congestion control for IP Multicast complicated

• Deployment is difficult and slow– ISP’s reluctant to turn on IP Multicast

27

End System MulticastStanford

Gatech Stan1

Stan2

Berk1

CMU

Stan1

Stan2

Berk2

Overlay TreeGatech

Berk1

Berkeley Berk2

CMU

28

• Scalability– Routers do not maintain per-group state– End systems do, but they participate in very few groups

• Easier to deploy• Potentially simplifies support for higher level

functionality– Leverage computation and storage of end systems– For example, for buffering packets, transcoding, ACK

aggregation– Leverage solutions for unicast congestion control and reliability

Potential Benefits

29

Design Questions• Is End System Multicast Feasible?• Target applications with small and

sparse groups • How to Build Efficient Application-Layer

Multicast “Tree” or Overlay Network? – Narada: A distributed protocol for constructing

efficient overlay trees among end systems– Simulation and Internet evaluation results to

demonstrate that Narada can achieve good performance

30

Performance Concerns

CMU

Gatech Stan1

Stan2

Berk1

Berk2

Duplicate Packets:

Bandwidth Wastage

CMU

Stan1

Stan2

Berk2

Gatech

Berk1

Delay from CMU to

Berk1 increases

31

What is an efficient overlay tree?• The delay between the source and receivers is

small• Ideally,

– The number of redundant packets on any physical link is low

Heuristic used:– Every member in the tree has a small degree – Degree chosen to reflect bandwidth of connection to

Internet

Gatech

“Efficient” overlay

CMU

Berk2

Stan1

Stan2

Berk1Berk1

High degree (unicast)Berk2

Gatech

Stan2CMU

Stan1

Stan2

High latency

CMU

Berk2

Gatech

Stan1

Berk1

32

Why is self-organization hard?

• Dynamic changes in group membership – Members may join and leave dynamically– Members may die

• Limited knowledge of network conditions– Members do not know delay to each other when they join– Members probe each other to learn network related

information – Overlay must self-improve as more information available

• Dynamic changes in network conditions – Delay between members may vary over time due to

congestion

33

Performance Metrics• Delay between members using Narada• Stress, defined as the number of identical copies

of a packet that traverse a physical link

Berk2

GatechStan1

Stress = 2CMU

Stan2

Berk1

Berk2CMU

Stan1

Stan2Gatech

Berk1

Delay from CMU to

Berk1 increases

34

ESM Conclusions• Proposed in 1989, IP Multicast is not yet widely

deployed– Per-group state, control state complexity and scaling

concerns– Difficult to support higher layer functionality– Difficult to deploy, and get ISP’s to turn on IP

Multicast• Is IP the right layer for supporting multicast

functionality?• For small-sized groups, an end-system overlay approach

– is feasible– has a low performance penalty compared to IP

Multicast– has the potential to simplify support for higher layer

functionality– allows for application-specific customizations

35

Internet Indirection Infrastructure (i3)

Motivations• Today’s Internet is built around a unicast point-

to-point communication abstraction:– Send packet “p” from host “A” to host “B”

• This abstraction allows Internet to be highly scalable and efficient, but…

• … not appropriate for applications that require other communications primitives:– Multicast – Anycast – Mobility– …

36

Why?• Point-to-point communication implicitly

assumes there is one sender and one receiver, and that they are placed at fixed and well-known locations– E.g., a host identified by the IP address 128.32.xxx.xxx is

located in Berkeley

37

IP Solutions• Extend IP to support new communication

primitives, e.g., – Mobile IP – IP multicast– IP anycast

• Disadvantages:– Difficult to implement while maintaining Internet’s

scalability (e.g., multicast)– Require community wide consensus -- hard to achieve in

practice

38

Application Level Solutions• Implement the required functionality at

the application level, e.g., – Application level multicast (e.g., Narada, Overcast,

Scattercast…)– Application level mobility

• Disadvantages:– Efficiency hard to achieve– Redundancy: each application implements the same

functionality over and over again– No synergy: each application implements usually

only one service; services hard to combine

39

Key Observation• Virtually all previous proposals use

indirection, e.g., – Physical indirection point mobile IP– Logical indirection point IP multicast

“Any problem in computer science can be solved by adding a layer of indirection”

40

i3 Solution

• Use an overlay network to implement this layer– Incrementally deployable; don’t need to change IP

Build an efficient indirection layer

on top of IP

IP

TCP/UDP

Application

Indir.layer

41

Internet Indirection Infrastructure (i3): Basic Ideas

• Each packet is associated an identifier id• To receive a packet with identifier id, receiver R

maintains a trigger (id, R) into the overlay network

Sender

id Rtrigger

iddata

Receiver (R)

iddata

Rdata

42

Service Model

• API– sendPacket(p);– insertTrigger(t);– removeTrigger(t) // optional

• Best-effort service model (like IP)• Triggers periodically refreshed by end-

hosts• ID length: 256 bits

43

Mobility

• Host just needs to update its trigger as it moves from one subnet to another

Sender

Receiver(R1)

Receiver(R2)

id R1id R2

44

iddata

Multicast

• Receivers insert triggers with same identifier• Can dynamically switch between multicast

and unicast

Receiver (R1)id R1

Receiver (R2)

id R2

Sender

R1data

R2data

iddata

45

Anycast• Use longest prefix matching instead of exact

matching– Prefix p: anycast group identifier – Suffix si: encode application semantics, e.g., location

Sender

Receiver (R1)p|s1 R1

Receiver (R2)p|s2 R2

p|s3 R3

Receiver (R3)

R1datap|adata p|adata

46

Service Composition: Sender Initiated

• Use a stack of IDs to encode sequence of operations to be performed on data path

• Advantages– Don’t need to configure path– Load balancing and robustness easy to achieve

SenderReceiver (R)

idT Tid R

Transcoder (T)

T,iddata

iddata

Rdata

idT,iddata idT,iddata

47

Service Composition: Receiver Initiated

• Receiver can also specify the operations to be performed on data

Receiver (R)

id idF,R

Firewall (F)

Sender idF F

idF,Rdata

Rdata

F,Rdata

iddata iddata

48

Quick Implementation Overview

• i3 is implemented on top of Chord– But can easily use CAN, Pastry, Tapestry, etc

• Each trigger t = (id, R) is stored on the node responsible for id

• Use Chord recursive routing to find best matching trigger for packet p = (id, data)

49

Routing Example• R inserts trigger t = (37, R); S sends packet p = (37, data)• An end-host needs to know only one i3 node to use i3

– E.g., S knows node 3, R knows node 35

3

7

20

35

41

37 R

37

20

35

41

37 R

S

R

trigger(37,R)

send(37, data)

send(R, data)

Chord circle

S

R

02m-1

[8..20]

[4..7]

[21..35]

[36..41]

[40..3]

50

Sender (S)

Optimization #1: Path Length• Sender/receiver caches i3 node mapping a

specific ID• Subsequent packets are sent via one i3

node

[42..3]

[4..7]

[8..20]

[21..35][36..41]

37 R

37data

Rdatacache node Receiver (R)

51

Optimization #2: Triangular Routing

• Use well-known trigger for initial rendezvous• Exchange a pair of (private) triggers well-located• Use private triggers to send data traffic

[42..3]

[4..7]

[8..20]

[21..35][36..41]

37 RR[2]

2 S37[2]

2 [30]30 R

S [30]30data

Rdata

Receiver (R)

Sender (S)

52

Example 1: Heterogeneous Multicast

• Sender not aware of transformations

Receiver R1(JPEG)

id_MPEG/JPEG S_MPEG/JPEG

id (id_MPEG/JPEG, R1)

send(id, data)

S_MPEG/JPEG

Sender(MPEG)

send((id_MPEG/JPEG, R1), data)

send(R1, data)

id R2

Receiver R2(MPEG)

send(R2, data)

53

Example 2: Scalable Multicast

• i3 doesn’t provide direct support for scalable multicast– Triggers with same identifier are mapped onto the same i3 node

• Solution: have end-hosts build an hierarchy of trigger of bounded degree

R2

R1

R4R3

g R2

g R1

gx

x R4

x R3

(g, data)

(x, data)

54

Example 2: Scalable Multicast (Discussion)

Unlike IP multicast, i31. Implement only small scale replication

allow infrastructure to remain simple, robust, and scalable

2. Gives end-hosts control on routing enable end-hosts to – Achieve scalability, and– Optimize tree construction to match their needs, e.g.,

delay, bandwidth

55

Example 3: Load Balancing• Servers insert triggers with IDs that have random

suffixes• Clients send packets with IDs that have random

suffixes

S1

1010 0101 S2

1010 1010 S3

1010 1101 S4

S1

S2

S3

S4

A

B

send(1010 0110,data)

send(1010 1110,data)

1010 0010

56

Example 4: Proximity• Suffixes of trigger and packet IDs encode

the server and client locations

1000 0010 S1

1000 1010 S21000 1101 S3

S1

S2S3

send(1000 0011,data)

57

Outline

• Implementation• Examples• SecurityApplications

Protection against DoS attacks– Routing as a service– Service composition platform

58

Applications: Protecting Against DoS

• Problem scenario: attacker floods the incoming link of the victim

• Solution: stop attacking traffic before it arrives at the incoming link– Today: call the ISP to stop the traffic, and hope for the

best!

• Our approach: give end-host control on what packets to receive– Enable end-hosts to stop the attacks in the network

59

Why End-Hosts (and not Network)?

• End-hosts can better react to an attack– Aware of semantics of traffic they receive– Know what traffic they want to protect

• End-hosts may be in a better position to detect an attack– Flash-crowd vs. DoS

60

Some Useful Defenses1. White-listing: avoid receiving packets on

arbitrary ports2. Traffic isolation:

– Contain the traffic of an application under attack– Protect the traffic of established connections

3. Throttling new connections: control the rate at which new connections are opened (per sender)

61

1. White-listing• Packets not addressed to open ports are dropped

in the network– Create a public trigger for each port in the white list– Allocate a private trigger for each new connection

IDS S

Sender (S)

Receiver (R)

S [IDR]

IDS [IDR]IDR

RRdata

IDPR

R[IDS]

IDP[IDS]IDR data

IDP – public trigger IDS, IDR – private triggersIDP – public trigger IDS, IDR – private triggers

62

2. Traffic Isolation• Drop triggers being flooded without affecting

other triggers– Protect ongoing connections from new connection

requests– Protect a service from an attack on another services

Victim (V)

Attacker(A)

Legitimate client(C)

ID2V

ID1 V

Transaction server

Web server

63

2. Traffic Isolation (cont’d)• Drop triggers being flooded without affecting

other triggers– Protect ongoing connections from new connection

requests– Protect a service from an attack on another services

Victim (V)

Attacker(A)

Legitimate client(C)

ID1 V

Transaction server

Web server

Traffic of transaction serverprotected from attack on web server

Traffic of transaction serverprotected from attack on web server

64

3. Throttling New Connections• Redirect new connection requests to a

gatekeeper – Gatekeeper has more resources than victim – Can be provided as a 3rd party service

Server (S)Client (C)IDC C

X S

puzzle

puzzle’s solution

Gatekeeper (A)

IDPA

65

Service Composition Platform• Goal: allow third-parties and end-hosts to

easily insert new functionality on data path– E.g., firewalls, NATs, caching, transcoding, spam filtering,

intrusion detection, etc..

• Why i3? – Make middle-boxes part of the architecture– Allow end-hosts/third-parties to explicitly route through

middle-boxes

66

Example• Use Bro system to provide intrusion

detection for end-hosts that desire so

M

client Aserver B

i3

Bro (middle-box)

idM MidBA B

idAB A

(idM:idBA, data)(idBA, data)

(idM:idAB, data)(idAB, data)

67

Design Principles

1) Give hosts control on routing– A trigger is like an entry in a routing table!– Flexibility, customization– End-hosts can

• Source route• Set-up acyclic communication graphs • Route packets through desired service points• Stop flows in infrastructure• …

2) Implement data forwarding in infrastructure– Efficiency, scalability

68

Design Principles (cont’d)

Host Infrastructure

Internet &Infrastructure overlays

Data plane

Control plane

p2p & End-host overlays

Data plane

Control planei3 Data planeControl plane

69

Conclusions• Indirection – key technique to implement

basic communication abstractions– Multicast, Anycast, Mobility, …

• I3– Advocates for building an efficient Indirection Layer on

top of IP – Explore the implications of changing the communication

abstraction; already done in other fields• Direct addressable vs. associative memories• Point-to-point communication vs. Tuple space (in

Distributed systems)

70

Requirements for Today/Tomorrow’s Internet?

Some key requirements (“-ities”)• Availability and reliability

– “Always on”, fault-tolerant, fast recovery from failures, …• Quality-of-service (QoS) for applications

– fast response time, adequate quality for VoIP, IPTV, etc.• Scalability

– millions or more of users, devices, …• Mobility

– untethered access, mobile users, devices, … • Security (and Privacy?)

– protect against malicious attacks, accountability of user actions?• Manageability

– configure, operate and manage networks– trouble-shooting network problems

• Flexibility, Extensibility, Evolvability, ……? – ease of new service creation and deployment?– evolvable to meet future needs?

71

Key Issues, Challenges, Solutions …A More Network-Centric View• New Naming/Addressing?

– Separating “identifiers” and “locators” to better support mobility– “semantic-free” flat id space ?– Data centric? – Role of “search” on naming, etc.

• Scalable and Robust Routing– Better and more adaptive to failures, and other network events– Also better support for network management, security, …– how to perform routing on “flat id” space?– Or shall we decouple routing from “naming” or “addressing” ?

• Manageability– “Centralized” approach – …?

• Security (and Privacy?)– More “accountable” networks, e.g., through “naming,” or id

management?– …?

72

Key Issues, Challenges, Solutions …Applications and Technology are Dual Drivers !• More devices are connected, novel technologies,

disruptive new applications/services– Google, and its impact of how we access Internet today– social networking: Facebook, MySpace, … – iPod/iTune, Skype, BitTorrent, P2P video streaming,

YouTube, Hulu.com, Kindle and Amazon, Ebay, …– smart phones, etc., “third screen”– “Cloud computing”, data centers, and “software as

services”–

• Flexibility, Evolvability, and Economic Viability of Network Architectures! – It’s “service”, stupid!

• But is network a (shared) “utility”, “commodity”, or “service” ?

– “Networks” as services (e.g., VPNs), network security as services, …

– Network Virtualization and Virtualized Network Architectures– User/application “customize-able” network services?

Ultimately, networks should be “invisible” !