centre for applied internet research cair-uk

Centre for AppliedInternet Research

Centre for AppliedInternet Researchwww.cair-uk.org


The Internet:A difficult beast to control?

Professor Vic GroutDirector of the Centre for Applied Internet Research (CAIR)Glyndŵr University, North Wales

[email protected]

MIC 2011 Keynote, 14/02/2011, Innsbruck


The Internet:A difficult beast to control?


A rambling – and probably confused – collection of thoughts from 25 years’ research into network algorithms and optimization!


Control? Optimization?optimize or optimise verb (optimized, optimizing) 1 to make the most or best of (a particular situation or opportunity, etc). 2 to make the most efficient use of something, especially by analysing and planning. 3 intrans to be optimistic or act optimistically. 4 intrans to become optimal. 5 computing to prepare or modify (a computer system or program) so as to achieve the greatest possible efficiency. optimization noun. ETYMOLOGY: 19c.

So what’s ‘Optimizing the Internet’?

Making the Internet perfect?

Having a look at something somewhere and consider tinkering with it?


Internet Optimization?There you are … I’ve

optimized it!


Internet Optimization? There you are … I’ve

optimized it!


We don’t always agree what optimization is!

Thought #1


Conventionally, two different types of model/problem/solution:

Design

Topologies

Dimensioning

Off-line/Centralized

Control/Management

Traffic handling

Routing

Filtering

Real-time/Distributed

Internet/Network Optimization


Actually, there’s a much more interesting (and relevant) way of classifying models/problems/solutions!

Thought #2


An alternative taxonomy:

Internet/Network Optimization

Things that have to be done (because finding any solution

is a form of optimization).

eg, routing

Things that don’t have to be done

(because there’s an existing valid

solution already).eg, compression

Things that have an obvious

default/initial solution (but it’s

probably distinctly sub-optimal).

eg, physical design

access-list 101 permit tcp 192.168.212.0 0.0.0.255 10.0.0.0 0.255.255.255 eq telnetaccess-list 101 permit tcp 192.168.212.0 0.0.0.255 10.0.0.0 0.255.255.255 eq ftpaccess-list 101 permit tcp 192.168.212.0 0.0.0.255 10.0.0.0 0.255.255.255 eq httpaccess-list 101 deny ip 192.168.212.0 0.0.0.255 10.0.0.0 0.255.255.255access-list 101 permit icmp any 10.0.0.0 0.255.255.255 administratively-prohibitedaccess-list 101 permit icmp any 10.0.0.0 0.255.255.255 echo-replyaccess-list 101 permit icmp any 10.0.0.0 0.255.255.255 packet-too-bigaccess-list 101 permit icmp any 10.0.0.0 0.255.255.255 time-exceededaccess-list 101 permit icmp any 10.0.0.0 0.255.255.255 unreachableaccess-list 101 permit icmp 172.16.20.0 0.0.255.255access-list 101 deny icmp any anyaccess-list 101 permit ip 202.33.42.0 0.0.0.255 anyaccess-list 101 permit ip 202.33.73.0 0.0.0.255 anyaccess-list 101 permit ip 202.33.48.0 0.0.0.255 anyaccess-list 101 permit ip 202.33.75.0 0.0.0.255 anyaccess-list 101 deny ip 202.33.0.0 0.0.255.255 anyaccess-list 101 deny tcp 210.120.122.0 0.0.0.255 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 deny tcp 210.120.183.0 0.0.0.255 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 deny tcp 210.120.114.0 0.0.0.255 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 deny tcp 210.120.175.0 0.0.0.255 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 deny tcp 210.120.136.0 0.0.0.255 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 deny tcp 210.120.177.0 0.0.0.255 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 permit tcp any 10.2.2.0 0.255.255.255 eq wwwaccess-list 101 deny tcp any any eq wwwaccess-list 101 permit tcp any anyaccess-list 101 deny ip 195.10.45.0 0.0.0.255 anyaccess-list 101 permit ip any any{access-list 101 deny all} {implicit}

Essential Optional


A Cautionary TaleStart with one of the (conceptually) simplest optimization problems in graph theory: Minimum Spanning Tree (MST)

“The EMST problem is a common component in applications involving networks. If one desires to set up a communications system among N nodes requiring interconnection cables, using the EMST will result in a network of minimal cost”, Michael Shamos, PhD Thesis, Yale University, 1978

Not practical!


Network Topology

Complex!

CoreAccess

Distribution


A Further Complication

ciji

j

Difficult to assign knowncosts as inputs ‘Double-drop’ and ‘triple-

drop’ heuristics typical


There’s often a big difference between the textbook theory and real-world practice!

Thought #3


Wireless Networks

Fibre backbone

Subscriberlocations

Minimum Connected Dominating Set

(MCDS)


Sometimes the textbook works!

Thought #4


Wireless Networks

Initial network

(Feasible links)


Wireless Networks

MST

(Inappropriate)


Wireless Networks

MCDS


Optical Networks

Network topology with Impairment Feasible Paths

Actual link

Feasible path

Regenerators needed to

maintain signal integrity

Very expensive!


Optical Networks

Transformed graph of the network

Effective link


Optical Networks

Graph transformation and CDS

Core network


Optical Networks

a 2-CDS of the transformed graph

Constraints:

k-connectivity (core)

k-domination (edge)

Mk-CDS


Wireless Networks

Real-timeoptimisation?

Distributed optimisation?


Many problems are just too hard!

Thought #5


Classes of Internet ProblemReal-time

Runs repetitively/frequently within the network (not part of initial off-line planning)

Line-speedHas to complete processing one packet/frame before the next arrives (at least, on average)

DistributedRuns independently on each network device (switch, router, etc.)

CooperativeNeeds input from other network devices prior to solution (eg, topology status)

ResponsiveNeeds input from other network devices during solution (eg, control negotiation)


Classes of Internet Problem

RT: Real-time, LS: Line-speed, D: Distributed, C: Cooperative, R: Responsive

RT

‘Conventional’

LS

R

C

D

SpanningTree Protocol

STP

eg, MST

Algorithms and Algorithmics!


Routing

Routers exchangelink-state

Information when topology changes

Network must converge before too many

packets are lost orpoorly routed


Shortest Paths


Shortest Paths

Dijkstra’s Shortest Path Algorithm (DSPA) finds all shortest paths (and places them in the routing table)DSPA is polynomial

complexity. Is that OK?


Sometimes, even the easy problems are hard!

Thought #6


Routing

cij

j

i

c = 1 / bandwidth


Routing

cij

j

i

c = 108 / bandwidth


Routing

cij

j

i

c = 108 / bandwidth

?

P


Routing

cij

j

i

c = 108 / bandwidth

C = ΣijP cij =ΣijP 1/bij ?

C = minijP bij ?

P

4

53

21 256 kr

kdk

l

bkbkC

Bandwidth (b)Delay (d)Load (l)

Reliability (r)

When we try to optimize something in the Internet,

what’s our objective function?

What are we trying to maximise or minimise?

throughput?delay?

reliability?customer satisfaction

bank balance?

P = f(b)


No, seriously, we really don’t know what optimization means!

Thought #7


Traffic Filtering• “Access Control Lists (ACLs)”

• Interfaces: in and out (permit/deny)• Also selecting packets for traffic policies• Across an internet• Can add considerable packet latency


Access Control Lists access-list 173 permit icmp any any access-list 173 permit tcp any any established access-list 173 deny ip RANGE MASK any access-list 173 deny ip 10.77.23.0 0.255.255.255 any access-list 173 deny ip 172.16.2.0 0.15.255.255 any access-list 173 deny ip 192.168.1.0 0.0.255.255 any access-list 173 deny ip 169.254.1.0 0.0.255.255 any access-list 173 deny ip 192.168.2.0 0.0.0.255 any access-list 173 permit tcp any host MAILSERVER eq smtp access-list 173 permit tcp any host NAMESERVER eq domain access-list 173 permit udp any host NAMESERVER eq domain access-list 173 permit udp any eq 53 host NAMESERVER gt 1024 access-list 173 permit tcp host MANAGER host SUN eq telnet access-list 173 permit tcp host MANAGER host SERIAL0 eq telnet access-list 173 permit tcp host MANAGER host ETHERNET0 eq telnet access-list 173 permit udp host MANAGER host SERIAL0 eq snmp access-list 173 permit tcp any host FTPSERVER eq ftp access-list 173 permit tcp any eq ftp-data host FTPSERVER access-list 173 permit tcp any eq ftp-data any gt 1024 access-list 173 permit tcp any host WWWSERVER eq www access-list 173 permit tcp any host SWWWSERVER eq 443 access-list 173 permit udp EXT-NTPSERVER any eq 123 access-list 173 permit udp any range 6970 7170 any access-list 173 deny ip any any

Sequence of ‘permit’and ‘deny’ rules

Each rules tries to matchsome feature of the packet

being processed

Rules processedsequentially …

… until a rule matchesthe packet (stop) …

… or the last ruleis reached

Various possibleimplementations:

Hardware (TCAMs)Trees/Tries, etc.


Linear ACL Optimizationn rules in list LHit-rate hi(L)

probability that packets match rule i in list LLatency i(L)

time taken to process rule i in list LCumulative latency i(L)

time taken to process list up to and including i in list L

Expected latency E(L)average time to process List L

i

jii LL

1

)()(

)()()(1

LLhLEn

iii


Linear ACL Optimization

)()()(1

LLhLEn

iii

:0

:1ijd

Rules i and j are dependent

otherwiseA major problem, even with approximations, is having to re-evaluatethe objective function

for each potential reordering of the list


Simplified ACL OptimizationIn fact, in comparing rule order for a list L, the significance of rule hit-rates is only relative. It is not necessary for them to be normalised probabilities. This implies that the hit-rate of a newly hit rule, i, can increase without changing the hit-rates of the other rules.

Following an increase in a rule i’s hit-rate, the only possible change in rule order (to reduce E(L)) is to promote i up the list. The most likely candidate with which to exchange it is rule i-1, immediately above it. The potential saving in expected latency in swapping rules i-1 and i is given by

a simple, local calculation.iiii

iiiiiiiiii

n

ikkkiiii

i

kkk

n

ikkkiiii

i

kkk

hh

hhhh

hhhhhhhh

11

11111

111

2

1111

2

1

)()(

access-list 173 permit icmp any any access-list 173 permit tcp any any established access-list 173 deny ip RANGE MASK any access-list 173 deny ip 10.77.23.0 0.255.255.255 any access-list 173 deny ip 172.16.2.0 0.15.255.255 any access-list 173 deny ip 192.168.1.0 0.0.255.255 any access-list 173 deny ip 169.254.1.0 0.0.255.255 any access-list 173 deny ip 192.168.2.0 0.0.0.255 any access-list 173 permit tcp any host MAILSERVER eq smtp access-list 173 permit tcp any host NAMESERVER eq domain access-list 173 permit udp any host NAMESERVER eq domain access-list 173 permit udp any eq 53 host NAMESERVER gt 1024 access-list 173 permit tcp host MANAGER host SUN eq telnet access-list 173 permit tcp host MANAGER host SERIAL0 eq telnet access-list 173 permit tcp host MANAGER host ETHERNET0 eq telnet access-list 173 permit udp host MANAGER host SERIAL0 eq snmp access-list 173 permit tcp any host FTPSERVER eq ftp access-list 173 permit tcp any eq ftp-data host FTPSERVER access-list 173 permit tcp any eq ftp-data any gt 1024 access-list 173 permit tcp any host WWWSERVER eq www access-list 173 permit tcp any host SWWWSERVER eq 443 access-list 173 permit udp EXT-NTPSERVER any eq 123 access-list 173 permit udp any range 6970 7170 any access-list 173 deny ip any any


Simplified ACL OptimizationThree-part heuristic (-opt ):

Step 1: Initialisation (following manual ACL configuration)for i := 1 to n do

hi := 1 \ hit rates equal at start

Step 2: Promotion (on a packet matching rule i)hi := 2hi \ exponentially increase matched hit-rateif di-1 i=0 and hiλi-1 > hi-1λi then

Swap(i-1, i) \ promote if E(L) reduced

Step 3: Reduction (periodically to prevent overflow)for i := 1 to n do

hi := hi / max j hj


ACL Optimization EffectivenessACL characteristics: DI (dependency index)

probability of two rules being dependentTraffic self-similarity: SI (self-similarity index)

Probability that a packet matches the same rule as the previous packet

Minimum number of rules (n*) for -opt to work:

DI = 0.00 0.25 0.50 0.75 1.00

SI = 0.00 19 21 23 33

0.25 16 19 21 29

0.50 13 15 19 26

0.75 9 10 13 21

1.00 8 9 12 17


Sometimes, just sometimes, we get a break!

Thought #8


The Spanning Tree Protocol


Complexity can be complex!

Thought #9


RecapWe don’t always agree what optimization is!

There are different ways of classifying problems!

There’s often a big difference between theory and practice!

Sometimes the textbook works!

Many problems are too hard!

Sometimes even the easy problems are hard!

We really don’t know what optimization means!

Sometimes we get a break!

Complexity can be complex!


Some ConclusionsMatching textbook problems to Internet applications requires care to make potential solutions realistic and appropriate

Real-time optimization within the Internet places severe restrictions on time (and space) complexity and often needs to be distributed

Often a lot of the elegance of the original model is lost in practical application

However, a use for standard methods can sometimes still be found – but not necessarily in the obvious applications

A successful network algorithmist or algorithmatist probably needs a foot in both camps!


Thank you … … Any

questions?Professor Vic GroutDirector of the Centre for Applied Internet Research (CAIR)Glyndŵr University, North Wales

[email protected]



Centre for AppliedInternet Researchwww.cair-uk.org

centre for applied internet research cair-uk

Documents

eq

permit tcp

eq telnetaccesslist

eq ftpaccesslist

eq httpaccesslist

permit ip

permit icmp

appliedinternet researchtheres