real-time switched networks - comp.polyu.edu.hkcsqwang/research/realtimeswitchednet… · real-time...
TRANSCRIPT
Real-Time Switched Networks
Qixin WangDepartment of Computing
The Hong Kong Polytechnic University2012
Robotic Surgery: each task is a continuous loop of sensing (or actuating) jobs
Each job:
1. Must catch deadline
2. Does not have to be fast
Aviation and Industrial Control: each task is a continuous loop of sensing (or actuating) jobs
Each job:
1. Must catch deadline
2. Does not have to be fast
A typical real-time task is a continuous loop of periodic jobs.
TimePeriod
Exe. Time Exe. Time Exe. Time
Job: (Period, Exe. Time, Deadline)A Job
Deadline
A typical real-time task is a continuous loop of periodic jobs.
TimePeriod
Exe. Time Exe. Time Exe. Time
Job: (Period, Exe. Time, Deadline)
Real-time = each job catches deadline
Real-time ≠ running fast
A Job
Deadline
Real-Time Switch is the key component for many infrastructures
Real-Time WAN, Real-Time Internet Subnet
Real-Time Fieldbus
RT High Performance Computing Architecture, e.g., InfiniBand
Real-Time Switch is the key component for many infrastructures
Real-Time WAN, Real-Time Internet Subnet
Real-Time Fieldbus
RT High Performance Computing Architecture, e.g., InfiniBand
Real-Time WAN, Real-Time Internet Subnet
Tele-robotic underground mining saves lives
2000ft (609.6m)
2000ft (609.6m)
> 5000 annual death toll
Real-Time Switch is the key component for many infrastructures
Real-Time WAN, Real-Time Internet Subnet
Real-Time Fieldbus
RT High Performance Computing Architecture, e.g., InfiniBand
Real-Time Switch is the key component for many infrastructures
Real-Time WAN, Real-Time Internet Subnet
Real-Time Fieldbus
RT High Performance Computing Architecture, e.g., InfiniBand
RT High Performance Computing Architecture (e.g., InfiniBand): Scaling from Shared Medium to Multi-Hop
Concord
1976
RT High Performance Computing Architecture (e.g., InfiniBand): Scaling from Shared Medium to Multi-Hop
A380
2007,
Several Hundreds of computing nodes
Real-Time Switch is the key component for many infrastructures
Real-Time WAN, Real-Time Internet Subnet
Real-Time Fieldbus
RT High Performance Computing Architecture, e.g., InfiniBand
Designing real-time switch is not easy1.
A. Demers, S. Keshav, and S. Shenker, “Analysis and simulation of a fair queueing
algorithm,”
in SIGCOMM’89, 1989. pp. 1-12.
2.
D. Ferari, and D. Verma, “A scheme for real-time channel establishment in wide-area networks,”
in IEEE JSAC, v8, n3, April, 1990
3.
D. C. Verma, H. Zhang, and D. Ferrari, “Delay jitter control for real-time communication in packet switching network,”
in Proceedings of Tricomm’91, pp. 35-46, April 1991.
4.
S. Golestani, “A framing strategy for congestion management,”
in IEEE JSAC, v9, n7, pp. 1064-1077, Sep., 1991
5.
L. Zhang, "VirtualClock: a new traffic control algorithm for packet-switched networks," ACM Trans. on Computer Systems, Vol. 9, No. 2, May 1991
6.
A. K. Parekh, R. G. Gallager, “A Generalized Processor Sharing Approach to Flow Control in Integrated Services Networks: The Single-Node Case,”
in IEEE/ACM ToN, v1, n3, June, 1993.
7.
A. K. Parekh, R. G. Gallager, “A Generalized Processor Sharing Approach to Flow Control in Integrated Services Networks: The Multiple Node Case,”
in IEEE/ACM ToN, v2, n2, April, 1994.
8.
S. Jamaloddin
Golestani, ``A self-clocked fair queuing scheme for broadband applications,'' Proc. IEEE INFOCOM'94, pages 636--646, IEEE, 1994.
9.
H. Zhang, "Service Disciplines For Guaranteed Performance Service in Packet-Switching Networks," Proceedings of the IEEE, 83(10), Oct 1995
10.
Jon C. R. Bennett and H. Zhang, ``WF2Q: worst-case fair weighted fair queueing,'' IEEE INFOCOM'96, March, 1996
11.
Pawan
Goyal, Harrick
M. Vin, and Haichen
Cheng, "Start-time fair queueing: a scheduling algorithm for integrated services packet switching networks," ACM SIGCOMM'96, pages 157--168, ACM press, August, 1996
12.
I. R. Philp, “Scheduling real-time messages in packet-switched networks,”
Ph.D. Thesis, Technical Report No. UIUCDCS-R-96-1977, CS Dept., UIUC, Oct., 1996.
13.
Subhash
Suri, George Varghese, and Girish
Chandranmenon, ``Leap forward virtual clock: A new fair queuing scheme with guaranteed delays and throughput fairness,'' IEEE Proc. INFOCOM'97, 1997
14.
I. Stoica, S. Shenker, and H. Zhang, ``Core-stateless fair queuing: achieving approximation fair bandwidth allocations in high speed networks,'' Proc. of ACM SIGCOMM'98, October 1998, pp. 118-130.
15.
C. R. Kalmanek, H. Kanakia, and S. Keshav, “Rate controlled servers for very high speed networks,”
in IEEE Global Telecommunications Conference, Dec., 1999.
16.
W. Sun, and K. G. Shin, “End-to-end delay bounds for traffic aggregates under guaranteed-rate scheduling algorithms,”
in IEEE ToN, 13(5): 1188-1201, Oct., 2005.
Huge Volume of Literature since 1989
Industry acceptance has the final say. Two things rules: simplicity and switch hardware features
Output
Queueing
+
QoS
Crossbar
+
Input
Queueing
Crossbar
+
VOQ
+
iSLIP
Combined input-output queueing, or
more complicated switch hardware,
to emulate output queueing
+
QoS
(e.g. WFQ)
N Speed-Up
Problem
HOL
Problem
Past lessons tell us to design the real-time switch compatible to “crossbar + VOQ + iSLIP”
Output
Queueing
+
QoS
Crossbar
+
Input
Queueing
Crossbar
+
VOQ
+
iSLIP
Combined input-output queueing, or
more complicated switch hardware,
to emulate output queueing+
QoS
(e.g., WFQ)
N Speed-Up
Problem
HOL
Problem
What is “Crossbar + VOQ + iSLIP”?
A widely implemented switch architecture
Simple
High switch utilization
Fast adaptation to random traffic (Internet traffic)
Advantages
2012/8/2Qixin
Wang, UIUC,
I1
I2
I3
O1
O2
O3
What is “Crossbar + VOQ + iSLIP”?
Synchronous periodic cell forwarding Cell-Time
I1
I2
I3
O1
O2
O3
What is “Crossbar + VOQ + iSLIP”?
Why Matching? An input/output can only send/receive one cell per cell-time
I1
I2
I3
O1
O2
O3I1
I2
I3
O1
O2
O3
Request (apply job)
Grant (offer job)
What is “Crossbar + VOQ + iSLIP”? iSLIP
is like job hunting
I1
I2
I3
O1
O2
O3I1
I2
I3
O1
O2
O3
I1
I2
I3
O1
O2
O3
Request (apply job)
Grant (offer job)
Accept (take offer)
What is “Crossbar + VOQ + iSLIP”? iSLIP
is like job hunting
But we cannot directly adopt “crossbar + VOQ + iSLIP”
Huge per hop delay bound because of poor isolation;
E2E delay bound is an open problem [Gopalakrishnan06].
But we cannot directly adopt “crossbar + VOQ + iSLIP”
Huge per hop delay bound because of poor isolation;
E2E delay bound is an open problem [Gopalakrishnan06].
What to do?
But we cannot directly adopt “crossbar + VOQ + iSLIP”
Huge per hop delay bound because of poor isolation;
E2E delay bound is an open problem [Gopalakrishnan06].
What to do?
Look at changed assumptions and
new features.
Changed assumption: traffic predictability
iSLIP
is for non-real-time traffic: UnpredictableFlows change rapidlyUnpredictable message size and arrival
Unpredictable traffic forces
iSLIP
to negotiate for each cell-time
I1
I2
I3
O1
O2
O3I1
I2
I3
O1
O2
O3
I1
I2
I3
O1
O2
O3
Request (apply job)
Grant (offer job)
Accept (take offer)
Changed assumption: traffic predictability
iSLIP
is for non-real-time traffic: UnpredictableFlows change rapidlyUnpredictable message size and arrival
RT-Switch for real-time traffic: Deterministic
Changed assumption: traffic predictability
iSLIP
is for non-real-time traffic: UnpredictableFlows change rapidlyUnpredictable message size and arrival
RT-Switch for real-time traffic: DeterministicFlows rarely changeRegular message size and arrivalNon-real-time traffic real-time traffic
Deterministic traffic allows RT-Switch to forward cells like clockwork, no need to negotiate!
I1
I2
I3
O1
O2
O3
Deterministic Grant is enough
No need for Request and Accept
Simplifies instead of extends iSLIP
New feature: Large message arrival period allows clock-driven time slicing
Real-time trafficSmall msg
size (sensing: 1cell/msg, TV quality video: 480cell/msg)
New feature: Large message arrival period allows clock-driven time slicing
Real-time trafficSmall msg
size (sensing: 1cell/msg, TV quality video: 480cell/msg)Short per msg
transmission time (s: 0.5us, v: 240us)
New feature: Large message arrival period allows clock-driven time slicing
Real-time trafficSmall msg
size (sensing: 1cell/msg, TV quality video: 480cell/msg)Short per msg
transmission time (s: 0.5us, v: 240us) Large msg
arrival period (s: 10ms, v: 30ms)
New feature: Large message arrival period allows clock-driven time slicing
Real-time trafficSmall msg
size (sensing: 1cell/msg, TV quality video: 480cell/msg)Short per msg
transmission time (s: 0.5us, v: 240us) Large msg
arrival period (s: 10ms, v: 30ms)
Reminds us of clock-driven time slicing OS
New feature: Large message arrival period allows clock-driven time slicing
Real-time trafficSmall msg
size (sensing: 1cell/msg, TV quality video: 480cell/msg)Short per msg
transmission time (s: 0.5us, v: 240us) Large msg
arrival period (s: 10ms, v: 30ms)
Reminds us of clock-driven time slicing OS
RT-Switch runs fine time grain clock period (1ms), serving time slices (cell-time) to coarse time grain periodic RT flows
Solution: crossbar RT-Switch runs deterministic clock-driven time slicing
Clock period of M cell-time, e.g., M = 5
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
Solution: crossbar RT-Switch runs deterministic clock-driven time slicing
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
Fit original real-time flow-forwarding tasks into clock period, e.g., (11, 3) (5, 2), i.e., (10, 4)
Clock period of M cell-time, e.g., M = 5
Solution: crossbar RT-Switch runs deterministic clock-driven time slicing
Fit original real-time tasks into clock period, e.g., (11, 3) (5, 2), i.e., (10, 4)
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
Demand
a cell to send to
O1
a cell to send to
O2
a cell to send to
O3
a cell to send to
O4
Clock period of M cell-time, e.g., M = 5
Demand
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
Clock period of M cell-time
Solution: crossbar RT-Switch runs deterministic clock-driven time slicing
Schedule
Scheduling
Algorithm
Fit original RT tasks into clk
period
Config. time scheduling (O(N4))
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
a cell to send to
O1
a cell to send to
O2
a cell to send to
O3
a cell to send to
O4
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:Forward cells according to the schedule during runtime
Solution: crossbar RT-Switch runs deterministic clock-driven time slicing
Fit original RT tasks into clk
period
Config. time scheduling (O(N4))
Clock period of M cell-time
Schedule
a cell to send to
O1
a cell to send to
O2
a cell to send to
O3
a cell to send to
O4
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
Solution: crossbar RT-Switch runs deterministic clock-driven time slicing
Forward cells according to the schedule during runtime
Fit original RT tasks into clk
period
Config. time scheduling (O(N4))
Clock period of M cell-time
Schedule
a cell to send to
O1
a cell to send to
O2
a cell to send to
O3
a cell to send to
O4
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
Solution: crossbar RT-Switch runs deterministic clock-driven time slicing
Forward cells according to the schedule during runtime
Fit original RT tasks into clk
period
Config. time scheduling (O(N4))
Clock period of M cell-time
Schedule
a cell to send to
O1
a cell to send to
O2
a cell to send to
O3
a cell to send to
O4
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
Solution: crossbar RT-Switch runs deterministic clock-driven time slicing
Forward cells according to the schedule during runtime
Fit original RT tasks into clk
period
Config. time scheduling (O(N4))
Clock period of M cell-time
Schedule
a cell to send to
O1
a cell to send to
O2
a cell to send to
O3
a cell to send to
O4
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
Solution: crossbar RT-Switch runs deterministic clock-driven time slicing
Forward cells according to the schedule during runtime
Fit original RT tasks into clk
period
Config. time scheduling (O(N4))
Clock period of M cell-time
Schedule
a cell to send to
O1
a cell to send to
O2
a cell to send to
O3
a cell to send to
O4
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
Solution: crossbar RT-Switch runs deterministic clock-driven time slicing
Forward cells according to the schedule during runtime
Fit original RT tasks into clk
period
Config. time scheduling (O(N4))
Clock period of M cell-time
Schedule
a cell to send to
O1
a cell to send to
O2
a cell to send to
O3
a cell to send to
O4
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
Satisfies demands:
•
Real-Time Guarantee•
Backward Compatibility
•
Simplicity•
Performance
•
Isolation
Solution: crossbar RT-Switch runs deterministic clock-driven time slicing
Schedule
a cell to send to
O1
a cell to send to
O2
a cell to send to
O3
a cell to send to
O4
Evaluation: Schedulabililty
Randomized traffic demand based on typical real-time industrial/medical application models
Total demanded switch utilization is U
Given U, what is the schedulable ratio?
Evaluation: E2E Delay Bound
Randomized traffic demand based on typical real-time industrial/medical application models
Max hop count is 15
Compare E2E Delay Bound provided by Real-Time Switch and iSLIP
Conclusion
Short E2E delay guarantee
Good schedulability
Compatible to, and simpler than the widely implemented iSLIP
crossbar switch.
O(1) runtime computation
Polynomial configuration time computation
References[Bharghavan94] Vaduvur
Bharghavan, et al., “MACAW: A Media Access Protocol for Wireless LANs.”
In SIGCOMM 1994: 212-225.[Gopalakrishnan06] S. Gopalakrishnan, M. Caccamo, and L. Sha, Switch Scheduling and Network Design for Real-Time Systems. In Proc. Of IEEE Real-Time and Embedded Technology and Applications (RTAS), Apr. 2006.[March06] Marsh J., Glencross
M., Pettiffer
S., Hubbold
R. J., “A robust network architecture supporting rich behaviour
in collaborative interactive applications.”
In IEEE Transactions on Visualization and Computer Graphics (TVCG), 12, 3 (May 2006), 405=416.[Mckeown99] Nick McKeown, “The iSLIP
Scheduling Algorithm for Input-Queued Switches”, in IEEE/ACM Transactions on Networking, vol. 7, no. 2, April 1999. [Ploplys04] N.J. Ploplys, et al., “Closed-Loop Control over Wireless Networks.”
In IEEE Control Systems Magazine, vol. 24, June, 2004.[wang08] Qixin
Wang, Sathish
Gopalakrishnan, Xue
Liu, and Lui
Sha, “A switch design for real-time industrial networks,”
in Proc. of IEEE RTAS 2008, Apr. 2008, pp. 367–376.[wang10] Q. Wang and S. Gopalakrishnan, Adapting a Main-Steam Internet Switch Architecture for Multi-Hop Real-Time Industrial Networks, in IEEE Transactions on Industrial Informatics, v6, n3, May, 2010. [Willig02] Willig
A., et al., “Measurements of a wireless link in an industrial environment using an IEEE 802.11-compliant physical layer.”
In IEEE Transactions on Industrial Electronics, v43, n6, pp1265-1282, Dec., 2002.
Contents of Part II are published in [wang08] and [wang10].
ContentDemand
Background
Problem Definition and Complexity
Heuristic Algorithm
Evaluation
Related Work
Real-Time Switched Networks (RTSN)
Specialized networks used in industrial, mining, medical, vehicular, avionic environments
Specialized networks used in industrial, mining, medical, vehicular, avionic environments
RTSN is fundamentally different from the Internet
Typically serves continuous sensing/actuating feedback control traffic: (Period, Workload, Deadline)
Specialized networks used in industrial, mining, medical, vehicular, avionic environments
RTSN is fundamentally different from the Internet
RTSN: The Internet:
Typically serves continuous sensing/actuating feedback control traffic: (Period, Workload, Deadline)
Specialized networks used in industrial, mining, medical, vehicular, avionic environments
RTSN is fundamentally different from the Internet
RTSN:(Hard) Real-Time
The Internet:Best Effort
Typically serves continuous sensing/actuating feedback control traffic: (Period, Workload, Deadline)
Specialized networks used in industrial, mining, medical, vehicular, avionic environments
RTSN is fundamentally different from the Internet
RTSN:(Hard) Real-Time
Periodic Stable Traffic
The Internet:Best Effort
Random Bursty Traffic
Typically serves continuous sensing/actuating feedback control traffic: (Period, Workload, Deadline)
Specialized networks used in industrial, mining, medical, vehicular, avionic environments
RTSN is fundamentally different from the Internet
RTSN:(Hard) Real-Time
Periodic Stable Traffic
Bounded w/ Global Info
The Internet:Best Effort
Random Bursty Traffic
Unbounded w/o Global Info
Typically serves continuous sensing/actuating feedback control traffic: (Period, Workload, Deadline)
Specialized networks used in industrial, mining, medical, vehicular, avionic environments
RTSN is fundamentally different from the Internet, hence requires different solutions.
RTSN:(Hard) Real-Time
Periodic Stable Traffic
Bounded w/ Global Info
The Internet:Best Effort
Random Bursty Traffic
Unbounded w/o Global Info
Shared medium multi-hop switched: real-time multicast becomes a problem.
11
111
)()()()()(
nnmm
mmnnnnn
txKtutuBtxAtx
Modern control assumes MIMO Real-Time Multicast btw
Sensors-Controllers-Actuators/Observers
Shared medium multi-hop switched: real-time multicast becomes a problem.
11
111
)()()()()(
nnmm
mmnnnnn
txKtutuBtxAtx
Shared Medium Real-Time Multicast: Easy
Multi-Hop Switched Real-Time Multicast: ?
I1
I2
I3
O1
O2
O3
Per-Flow-Queueing
Recap of our real-time switch architecture: crossbar per-flow-q TDMA
I1 O1
O2
O3
I2
I3
cell cell cell
cell cell
cell
Recap of our real-time switch architecture: crossbar per-flow-q TDMA
I1
I2
I3
O1
O2
O3
Synchronous periodic cell forwarding Cell-Time
Recap of our real-time switch architecture: crossbar per-flow-q TDMA
I1
I2
I3
O1
O2
O3
Why Matching? An input/output can only send/receive one cell per cell-time
Recap of our real-time switch architecture: crossbar per-flow-q TDMA
I1
I2
I3
O1
O2
O3
Internal Matching: if an input has multiple per-flow-q for the same output, only one is picked every cell-time.
Recap of our real-time switch architecture: crossbar per-flow-q TDMA
Fit all real-time flows’
periods into frame, e.g., (11, 3) (5, 2), i.e., (10, 4)
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
Demand
a cell to send to
O1
a cell to send to
O2
a cell to send to
O3
a cell to send to
O4
TDMA scheduling frame
of M cell-time, e.g., M = 5
Recap of our real-time switch architecture: crossbar per-flow-q TDMA
Demand
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
Schedule
Scheduling
Algorithm
Theorem 1: If demand matrix’ every color ≤ M cell, then
have config. time scheduler with O(N4) time cost [wang10].
Cell time: 1 2 3 4 5
I1:
I2:
I3:
I4:
a cell to send to
O1
a cell to send to
O2
a cell to send to
O3
a cell to send to
O4
Recap of our real-time switch architecture: crossbar per-flow-q TDMA
Simple extension to support multicast: only need to change network layer.
I1
I2
I3
O1
O2
O3
Support for Multicast
c
I1
I2
I3
O1
O2
O3c
cc
Support for Multicast
Simple extension to support multicast: only need to change network layer.
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
Graph of the RTSN
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
Switches (nodes) of
the network
Graph of the RTSN
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
Switches (nodes) of
the network
Edges of the network
Graph of the RTSN
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
A (real-time)
multicast group
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
A (real-time)
multicast groupSource
End
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
A (real-time)
multicast groupSource
End
Destination Ends
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
A (real-time)
multicast groupSource
End
Destination Ends
Cells to multicast
every period
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
A (real-time)
multicast groupSource
End
Destination Ends
Cells to multicast
every period
Period (unit: cell-time)
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
A (real-time)
multicast groupSource
End
Destination Ends
Cells to multicast
every period
Period (unit: cell-time)
Deadline (relative, unit: cell-time)
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
The set of all real-time multicast groups
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
The set of all real-time multicast groups
The ith
real-time multicast group
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
An instance of RTMS problem
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
An instance of RTMS problem
The RTSN topology
Real-Time Multicast Scheduling (RTMS) problem in RTSN
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
An instance of RTMS problem
The RTSN topology
The set of RT multicast groups
that user demands
RTMS Problem: Given a q, how to schedule each switch s.t. every mi meets its needs?
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
RTMS Problem: Given a q, how to schedule each switch s.t. every mi meets its needs?
),( EVG
),,,,( HTwDsm
}{ imM
)),,(( MEVGq
Theorem: RTMS Problem is NP-Hard.
M-slot Periodic RTMS: a subset of RTMS problems.
}|{ instance RTMS an is qqQ
},),,,,,(},{
,),('|'{
MTiHTwDsmm
Gqq'
i
iiiiiii
andM
QMQ
M-slot Periodic RTMS: a subset of RTMS problems.
}|{ instance RTMS an is qqQ
},),,,,,(},{
,),('|'{
MTiHTwDsmm
Gqq'
i
iiiiiii
andM
QMQProposition: M-slot Periodic
RTMS is NP-Hard.
M-slot Periodic RTMS: a subset of RTMS problems.
}|{ instance RTMS an is qqQ
},),,,,,(},{
,),('|'{
MTiHTwDsmm
Gqq'
i
iiiiiii
andM
QMQProposition: M-slot Periodic
RTMS is NP-Hard.
Search for Heuristic Solutions.
Transform an M-slot Periodic RTMS instance into a Real- Time Multicast Routing (RTMR) instance.
.0,1
max~
)},~),(,,,{(}~{~
),~
,(~
)},),(,,,{(}{
,'),('
MMHH
HMTwDsm
Gq
HMTwDsm
Gq
ii
iiiiii
iiiiii
and where
define where
Given
M
M
M
QM
Transform an M-slot Periodic RTMS instance into a Real- Time Multicast Routing (RTMR) instance.
.0,1
max~
)},~),(,,,{(}~{~
),~
,(~
)},),(,,,{(}{
,'),('
MMHH
HMTwDsm
Gq
HMTwDsm
Gq
ii
iiiiii
iiiiii
and where
define where
Given
M
M
M
QM
M-slot Periodic RTMS instance
Transform an M-slot Periodic RTMS instance into a Real- Time Multicast Routing (RTMR) instance.
.0,1
max~
)},~),(,,,{(}~{~
),~
,(~
)},),(,,,{(}{
,'),('
MMHH
HMTwDsm
Gq
HMTwDsm
Gq
ii
iiiiii
iiiiii
and where
define where
Given
M
M
M
QM
M-slot Periodic RTMS instance
RTMR instance
Transform an M-slot Periodic RTMS instance into a Real- Time Multicast Routing (RTMR) instance.
.0,1
max~
)},~),(,,,{(}~{~
),~
,(~
)},),(,,,{(}{
,'),('
MMHH
HMTwDsm
Gq
HMTwDsm
Gq
ii
iiiiii
iiiiii
and where
define where
Given
M
M
M
QM
M-slot Periodic RTMS instance
Max Multicast Tree HeightRTMR
instance
Transform an M-slot Periodic RTMS instance into a Real- Time Multicast Routing (RTMR) instance.
.0,1
max~
)},~),(,,,{(}~{~
),~
,(~
)},),(,,,{(}{
,'),('
MMHH
HMTwDsm
Gq
HMTwDsm
Gq
ii
iiiiii
iiiiii
and where
define where
Given
M
M
M
QM
).,('
)~
,(~
MM
Gq
Gq
to solution a
is to solutionA
Transform an M-slot Periodic RTMS instance into a Real- Time Multicast Routing (RTMR) instance.
.0,1
max~
)},~),(,,,{(}~{~
),~
,(~
)},),(,,,{(}{
,'),('
MMHH
HMTwDsm
Gq
HMTwDsm
Gq
ii
iiiiii
iiiiii
and where
define where
Given
M
M
M
QM
).,('
)~
,(~
MM
Gq
Gq
to solution a
is to solutionA
RTMScheduling
problem becomes a Routing problem.
Heuristic Routing Algorithm
Existing mainstream internet multicast routing algorithms become Dijkstra
when network is static
and global info is available
Dijkstra’s
short-coming: only cares about # of hops, ignores congestion.
s
s d
Heuristic Routing Algorithm
Existing mainstream internet multicast routing algorithms become Dijkstra
when network is static
and global info is available
Dijkstra’s
short-coming: only cares about # of hops, ignores congestion.
s
s d
Heuristic Routing Algorithm
Existing mainstream internet multicast routing algorithms become Dijkstra
when network is static
and global info is available
We want a heuristic routing algorithm that considers both (hops and congestion).
s
s d
Heuristic Routing Algorithm
Existing mainstream internet multicast routing algorithms become Dijkstra
when network is static
and global info is available
We want a heuristic routing algorithm that considers both (hops and congestion).
s
s d
Heuristic Routing Algorithm
Grow the multiple trees simultaneously with multiple iterations.
In each iteration, ≤1
link is added to each tree.
Heuristic Routing Algorithm
Grow the multiple trees simultaneously with multiple iterations.
In each iteration, ≤1
link is added to each tree.
When multiple trees contend in a same switch, we carry out a job-hunting-like negotiation to let only ≤1 contending tree grow through an output in this iteration.
The job-hunting-like contention negotiation is itself an iterative algorithm.
I1
I2
I3
O1
O2
O3
different colors for different trees
The job-hunting-like contention negotiation is itself an iterative algorithm.
I1
I2
I3
O1
O2
O3Each tree ranks all outputs and only apply to
its favorite
output
different colors for different trees
The job-hunting-like contention negotiation is itself an iterative algorithm.
I1
I2
I3
O1
O2
O3I1
I2
I3
O1
O2
O3
Each tree ranks all outputs and only apply to
its favorite
output
Each output offers job
to the most loyal
applicant
different colors for different trees
The job-hunting-like contention negotiation is itself an iterative algorithm.
I1
I2
I3
O1
O2
O3I1
I2
I3
O1
O2
O3
I1
I2
I3
O1
O2
O3Each tree ranks all outputs and only apply to
its favorite
output
Each output offers job
to the most loyal
applicant
Accept job
by reserving corresponding frame slots.
different colors for different trees
updated demand matrix
Ranking function design: considers both E2E delay (hops) and congestion.
rank of o to tree t
routing flexibility: slack to reaching max hop (max e2e delay) bound
Ranking function design: considers both E2E delay (hops) and congestion.
rank of o to tree t
routing flexibility: slack to reaching max hop (max e2e delay) bound
shortest distance to target end
Ranking function design: considers both E2E delay (hops) and congestion.
rank of o to tree t
congestion
routing flexibility: slack to reaching max hop (max e2e delay) bound
shortest distance to target end
The proposed heuristic algorithm terminates within polynomial time.
Grow the multiple trees simultaneously with multiple iterations.
In each iteration, ≤1
link is added to each tree.
When multiple trees contend in a same switch, we carry out a job-hunting-like negotiation to let only ≤1 contending tree grow through an output in this iteration.
Evaluation Setup.
4x4 port real-time switches
15x15 square grid network topology
Per port capacity: 1Gbps,
M
= 2000 (cell/frame), 1 cell = 500 bit
10000 Trials, in each trial:
Random number of multicast groups,
wi
= 1~20 cell/frame (500K~10Mbps)
Evaluation Setup.
Network Demanded Utilization (Application Layer E2E Utilization)
Allowed Tree
Height
Minimum Tree
Height
Mainstream Internet multicast routing algorithms mainly concern about dynamic distributed group management.
Reverse Path Broadcasting/Multicasting (RPB/RPM) [semeria97]
Truncated Reverse Path Broadcasting (TRPB) [semeria97]
Distance Vector Multicast Routing Protocol (DVMRP) [waitzman88]
Multicast Extension to Open Shortest Path First (MOSPF) [moy94a][moy94b]
Protocol Independent Multicast (PIM) [fenner06][adams05]
Core-Based Tree Multicast Routing (CBT) [ballardie97]
Mainstream Internet multicast routing algorithms become Dijkstra
for static network with global info.
Reverse Path Broadcasting/Multicasting (RPB/RPM) [semeria97]
Truncated Reverse Path Broadcasting (TRPB) [semeria97]
Distance Vector Multicast Routing Protocol (DVMRP) [waitzman88]
Multicast Extension to Open Shortest Path First (MOSPF) [moy94a][moy94b]
Protocol Independent Multicast (PIM) [fenner06][adams05]
Core-Based Tree Multicast Routing (CBT) [ballardie97]
OthersP2P and Overlay Network Multicast are concerned with statistical performance instead of hard real-time E2E delay bound.
De facto real-time switch architecture [wang10][wang08] [dopatka07][leung97]
Conclusion
Multicast routing in RTSN is NP-hard.
Heuristic algorithm exists that performs much better than shortest path (Dijkstra).
References[adams05] A. Adams, J. Nicholas, and W. Siadak, Protocol Independent Multicast–
Dense Mode (PIM-
DM): Protocol Specification (Revised), RFC 3973, Jan., 2005.[ballardie97] A. Ballardie, Core Based Tees (CBT) Multicast Routing Architectre, RF 2201, Spe., 1997.[chen11] Lixiong
Chen, Xue
Liu, Qixin
Wang, Yufei
Wang, "A Real-Time Multicast Routing Scheme for Multi-Hop Switched Fieldbuses," in Proc. of the 30th IEEE International Conference on Computer Communications (IEEE INFOCOM 2011), Shanghai, China, April, 2011. pp.3209-3217. [dopatka07] F. Dopatka
and R. Wismuller, “Design of a realtime
industrial ethernet
network including hot-pluggable asynchronous devices,”
IEEE International Symposium on Industrial Electronics (ISIE 2007), 2007.[fenner06] B. Fenner, M. handley, H. Holbrook, and I. Kouvelas, Protocol Independent Multicast –
Sparse Mode (PIM-SM): Protocol Specification (Revised), RFC 4601, Aug., 2006.[leung97] Yiu-Wing Leung and Tak-Shing
Yum, “A TDM-based multibus
packet switch,”
IEEE Trans. on Communications, vol. 45, no. 7, pp. 859–866, Jul. 1997.[moy94a] J. Moy, Multicast Extensions to OSPF, RFC 1584, Mar., 1994.[moy94b] J. Moy, MOSPF: Analysis and Experience, RFC 1585, Mar., 1994.[semeria97] C. Semeria
and T. Maufer, Introduction to IP Multicast Routing, IETF draft-ietf-mboned-
intro-multicast-03.txt, Jul., 1997.[waitzman88] D. Waitzman, C. Partridge, and S. Deering, Distance Vector Multicast Routing Protocol, RFC 1075, Nov., 1988.[wang08] Qixin Wang, Sathish
Gopalakrishnan, Xue
Liu, and Lui
Sha, “A switch design for real-time industrial networks,”
in Proc. of IEEE RTAS 2008, Apr. 2008, pp. 367–376.[wang10] Q. Wang and S. Gopalakrishnan, Adapting a Main-Steam Internet Switch Architecture for Multi-Hop Real-Time Industrial Networks, in IEEE Transactions on Industrial Informatics, v6, n3, May, 2010.
Contents of Part III are published in [chen11].
AFDX is a data network for safety-critical applications that utilizes dedicated bandwidth while providing deterministic
Quality of Service (QoS). –
from wikipedia
AFDX Network
10/100Mbit switched Ethernet
Based upon IEEE 802.3 and ARINC 664
Bridges the gap on reliability of guaranteed bandwidth
in ARINC 664
Adopted by Airbus A380, Boeing 787 Dreamliner
etc
Background: Avionics Full DupleX
(AFDX) Switched Ethernet
AFDX Concepts
Elements in an AFDX network:AFDX End-systemAFDX SwitchAFDX Links
AFDX FeaturesReal-time end-to-end delay guarantee through traffic shapingOpen switch architecture: AFDX standard leaves the switch
architecture design open, so that vendors can reuse their legacy
switch architectures.
AFDX compliant switch: as long as the switch can support virtual link (VL) traffic shaping.
Each VL conducts one unicast
flow from a source-end to a destination-end (e.g. E1 to E5) ;Along the VL’s
route, before entering each AFDX switch, the VL flow must behave as if policed by a token bucket;With the per hop token bucket policing and proper switch architecture design, we can guarantee end-to-end real-time for each VL.
Demand
AFDX standard leaves the switch architecture design open, so that vendors
can reuse their legacy switch architectures.
Demand: build AFDX networks using our real-time switch architecture, the TDMA crossbar real-time switch
AFDX Compliance
AFDX complianceAlong the VL’s
route, before entering each AFDX switch, the VL flow must behave as if policed by a token bucket;With the per hop token bucket policing and proper switch architecture design, we can guarantee end-to-end real-time for each VL
Analysis: AFDX Compliance
•
Theorem 2 (AFDX Compliance)–
Per flow analysis with network calculus–
Giving end to end delay
src
end: source a: arrival curve for each flow at a switchs: service curve for each flow at a switchv: TDMA crossbar real-time switchH: total number of hops for a flowdes end: destination Lf
: flow f’s
in-network maximum packet lengthPf
: flow f’s
in-network periodCf
: per-frame allocated slots
Resource Planning Problem
P(G(V,E),F): TDMA crossbar real-time switch AFDX network resource planning problem
•
AFDX network G(V,E), where V
is the set of all switches and E
is the set of links between the switches• F
is the set of flow in the AFDX network
Objectivenetwork utility maximization
Constraintsswitch schedulability
Constraintsend-to-end delay guarantee
Decision variablesCf
: per-frame allocated slots
Resource Planning Problem
P(G(V,E),F): TDMA crossbar real-time switch AFDX network resource planning problem
•
AFDX network G(V,E), where V
is the set of all switches and E
is the set of links between the switches• F
is the set of flow in the AFDX network
Objectivenetwork utility maximization
Constraintsswitch schedulability
Constraintsend-to-end delay guarantee
Decision variablesCf
: per-frame allocated slots
Alternatives for flow f
Resource Planning Problem: NP-HardObjectivenetwork utility maximization
Constraintsswitch schedulability
Constraintsend-to-end delay guarantee
Decision variablesCf
: per-frame allocated slots
•
Knapsack problem has been known as NP-Hard•
An instance of knapsack problem ҡ(Ξ, size, value, Өs ,Өv
) can be reduced to an instance of TDMA crossbar real-time switched AFDX network resource planning problem:–
Construct an AFDX network of three nodes: one source-end, connected by one TDMA crossbar switch to one destination end.
–
becomes equivalent to asking ‘is the constructed resource planning problem results in a maximum ≥ Өv
’
Why NP-Hard
Approximation Algorithm
To address the challenge that resource planning problem P(G(V,E),F) is NP-Hard, we propose a re-modeling approach, with which, we propose an approximation algorithm for P(G(V,E),F)
Let U~
and U*
be the total utility achieved by approximation algorithm and the optimum total utility respectively. We have
Related Work
•
Analysis of real-time behavior of AFDX networks upon switches [scharbarg09][charara06][chen11]
•
Industrial fieldbus
designs [santos09]•
TDMA Crossbar Switch Design [wang10]
•
Knapsack problem approximation algorithm
Conclusion
We proved that TDMA crossbar real-time switched network is AFDX compliant.
We proved the corresponding AFDX network’s resource planning problem is NP-Hard.
We proposed an approximation algorithm for the resource planning problem.
References[charara06] H. Charara, J. luc
Scharbarg, J. Ermont, and C. Fraboul, “Methods for bounding end-to-end delays on an afdx
network,”
in Euromicro
Conference on Real-Time Systems, 2006.[chen11] Lixiong
Chen, Xue
Liu, Qixin
Wang, Yufei
Wang, "A Real-Time Multicast Routing Scheme for Multi-Hop Switched Fieldbuses," in Proc. of the 30th IEEE International Conference on Computer Communications (IEEE INFOCOM 2011), Shanghai, China, April, 2011. pp.3209-3217. [rao12] Lei Rao, Qixin
Wang, Xue
Liu, Yufei
Wang, "Analysis of TDMA Crossbar Real-Time Switch Design for AFDX Networks," in Proc. of IEEE INFOCOM'12, March, 2012. pp.2462-2470. [santos09] R. Santos, R. Marau, A. Vieira, P. Pedreiras, A. Oliveira, and L. Almeida, “A synthesizable ethernet
switch with enhanced real-time features,”
in Proc. of IECON’09, pp. 2817–2824, 2009.[scharbarg09] J.-L. Scharbarg, F. Ridouard, and C. Fraboul, “A probabilistic analysis of end-to-end delays on an afdx
avionic network,”
in IEEE Transactions on Industrial Informatics, v5, n1, pp.38-49, 2009.[wang10] Q. Wang and S. Gopalakrishnan, Adapting a Main-Steam Internet Switch Architecture for Multi-Hop Real-Time Industrial Networks, in IEEE Transactions on Industrial Informatics, v6, n3, May, 2010.
Contents of Part IV are published in [rao12].
Background: Token Bucket
A token bucket flow is defined by (ρ,σ)
ρ
denotes the bucket refilling rate at which tokens(credits) are accumulated
ρf
= Lmaxf
(1+Jf
/ BAGf
)
σ
is the bucket size
σf
= Lmaxf
/ BAGf
Lmax
f
: the maximum packet bit length
BAGf
:
the bandwidth allocation gap
Jf
: the maximum admissible jitter