load sharing for cluster-based network service
DESCRIPTION
Load Sharing for Cluster-Based Network Service. Jiani Guo and Laxmi Bhuyan Architecture Lab Department of Computer Science and Engineering University of California, Riverside. - PowerPoint PPT PresentationTRANSCRIPT
Load Sharing for Cluster-Based Network Service
Jiani Guo and Laxmi Bhuyan
Architecture LabDepartment of Computer Science and Engineering
University of California, Riverside
2
Courtesy: “Cluster-Based Scalable Network Services”, Armando Fox, Steven D. Gribble, Yatin Chawathe, Eric A. Brewer and Paul Gauthier.
3
Video on Demand System
Multimedia Grid
Media Server
Camera
Transcoding Service
Internet
a large number of clientsheterogeneity in clients’ inbound network bandwidth, CPU/MEM capacity or display resolutionStoring multiple copies in server will give rise to server overload and scalability problem
4
Multimedia Grid
Media Server
Camera
Cluster-Based Transcoding Service
process the stream on-the-fly according to the client’s requirements => make some money
wide range of needs in video rates, sizes, and bandwidths can be met by real-time transcoding service – Need parallel processing
Transcoding Service
5
Existing Load Balancing Schemes
Plethora of research in the field of load balancing, but most of them only did simulations
Random or Round-robin implemented in practice
Adaptive load balancing is desirable, but the overhead in collecting statistics is very high – we found no real implementation
How does one maintain QoS while doing load balancing? Ex: To reduce out-of-order departures of multimedia units, the GOPs must be assigned to one processor, when a good load balancing needs distribution of the workload
6
EX: Round Robin
Worker 1
Dispatcher
Worker N
Worker 2
Unit Buffer
Receiver
.
.
.
Manager
fetch a unit Find an available
Worker
Send the unit
High communication protocol (UDP) overhead
7
Round Robin – A Multithreaded Model to Reduce Communication Cost
Worker 1
Worker N
Worker 2
Unit Buffer
Receiver
.
.
.
Manager
fetch a unit Find an available
Worker
Send the unit
Dispatcher 1
Dispatcher M
.
.
.
8
Load Sharing SchemesRound Robin - First Fit
Methods Searches for an available Worker in round robin way The first available Worker is chosen to be dispatched a GOP How a manager detects if a Worker is available is implementation-
dependent.
Properties Load is naturally balanced among all the Workers. Fast processing rate because no extra load analyzer is needed to
guide scheduling. May incur severe delay jitter for each stream because the GOPs of
the same stream are most likely to be distributed to different Workers.
9
Round Robin – First Fit
Worker 1
Worker N
Worker 2
dispatch queues
Receiver .
.
.
Manager Node
.
.
.
scheduler
Dispatcher 1
Dispatcher 2
Dispatcher N
.
.
.
GOP Queue
Is the Worker available? = Is there a vacancy in the dispatch queue? Depends on power of the worker!
10
Stream-based MappingMethods The media unit is mapped to a Worker according to the following
function: f ( C ) = C mod N
where C is the stream number to which the unit belongs; N is the total number of Workers in the cluster.
All media units belonging to one stream are sent to the same Worker.
Properties Preserves the order of computation among media units. Simple algorithm. Most efficient for some specific input patterns in a homogeneous cluster.
Specific patterns : M is multiple of N, where M is the total number of streams What if M < N?
11
Adaptive Load Balancing - Least Load First
Feedback-based Scheme — Least Load First Efficient load test mechanism is needed for the Manager to monitor
load distribution in the cluster. Workers periodically report their load statistics information to the Manager.
The Worker with the least load is chosen to dispatch the job. May incur substantial overhead to implement the feedback
mechanism.
Each Worker reports to the Manager its load information during each epoch ∆t. Load information reported by each Worker
CPU utilization AUi(t)
Maximal possible throughput Ai(t)
Actual throughput: Ai(t) – N N is the number of outstanding requests, i.e., the number of GOPs
already dispatched to it but not yet completed
Manager chooses the least loaded Worker: Worker with the maximal actual throughput
12
Adaptive Load SharingUnit-to-ComputingPC Mapping ( Done by the dispatcher)
Robust Hashing Mapping — The unit identifier ( such as the stream number of the unit in our experiment) and the Computing PC number together are used to assign a random value to each Computing PC. The unit is mapped to the Computing PC with the highest random value. If the Computing PCs have unequal capacity, the random value assigned to each Computing PC may be scaled by a weight which guarantees that the Computing PC with higher capacity can receive a proportionately higher portion of the load.
Thus, the mapping is calculated base on three values: the stream number of the unit C (1,2,…S), the Computing PC number J (1,2,…, N) and the weight vector (x1 , x2 , x3 , …, xN).
Minimize the probability of units belonging to the same stream being dispatched to different nodes. And this goal is achieved without keeping state information per stream.
Dynamic Weight Adaptation (Done by the manager) The workload on the Computing PCs (ρ1(t), ρ2(t), …, ρN(t)) is collected periodically
and the weight vector (x1 , x2 , x3 , …, xN) is adapted in a specific way such that the amount of stream re-mappings is minimized as well as load balancing is achieved.
The adapted weight vector is fed to the dispatchers.
13
Adaptive Load Sharing
Manager
(x1 , x2 , x3 , …, xN)
Computing PC 1
Dispatcher 1
Computing PC N
Computing PC 2
Dispatcher M
Unit Buffe
rReceiver
.
.
.
.
.
.
.
Routing PC
ρ1(t)
ρN(t)
ρ2(t)
(x1 , x2 , x3 , …, xN)
Fetch a unit
F( C ) = J Send to node J
J available?
Start EndYes
No
14
Experimental Set-Up
Computing PCComputing
PC
Computing PC
Gigabit Ethernet
Manager
Processed packets
Un-nrocessed packets
Media Server
100M Ethernet 100M Ethernet
Computing PC
15
Transcoding Service
What is transoding? Transforming video/audio streams such as changing the bit-rate,
resizing video frames, and adjusting the frame resolution and so on.
How to transcode?
MPEG
Stream
Raw
Stream
Manipulated
Stream
MPEG
Stream
MPEG
Decoder
Video/Audio Frame
Manipulator
MPEG
Encoder
16
Transcoding Workload
A media unit is a Group Of Pictures(GOP) of MPEG streamA media unit can be transcoded independently by any Worker in the cluster. Transcoding one media unit is considered an independent job.No communication is required among jobs.Each job consumes similar amount of processing time.Consecutive media units in a stream are preferred to be processed in order.
17
Design Goals of the Load Sharing Schemes
Balance the transcoding workload among all Workers
High system throughput Low overhead taken by the load balancing algorithm itself Good tradeoff between computation and communication
Provide good Quality of Service - NEW In-order departure of media units Even output time interval among successive media units of a
media stream
18
Computation Model of the Transcoding Cluster
19
Manager NodeReceiver Thread
Accepts incoming media units into the GOP Queue
Scheduler Thread Fetches GOPs from the GOP queue and puts them into an appropriate
dispatch queue according to the specific load sharing scheme
Dispatcher Thread per Worker Each Dispatcher maintains a dispatch queue Once requested by the corresponding Worker, dispatches one GOP to the
Worker
Manager Thread — Only for Least Load First Scheme Collects the load statistics information from the Workers during each epoch Feeds the load information to the scheduler
Collector Thread Collects processed video units from Workers and sends them out
20
Worker NodeReciever Thread
Receives packets from the Manager Node and assembles them into a complete GOP.
Once a complete GOP is received, gives it to the Transcoder thread, and then requests for another GOP from the Manager Node.
Transcoder Thread Transcode a GOP.
Sender Thread Delivers the transcoded GOPs to the clients.
Monitor Thread Collects the load statistics information on the Worker node and
reports to the Manager Node periodically.
21
Scalability of the System
5 media streams
22
Scalability of the System Throughput
System throughput scales well with First Fit and Least Load First.
Load test overhead in Least Load First scheme doesn’t affect the system throughput a lot, because the overhead is relatively small compared with the time taken to transcode one GOP.
Stream-based Mapping cannot disperse media units of the same stream among different Workers even if a Worker is free.
Waste of resources. Occasional imbalance in load distribution. Reduced throughput.
23
Out-of-Order Rate per Stream
24
Out-of-Order Rate per Stream
Out-of-order departure of media units Occurs when consecutive GOPs of a stream are transcoded on different Workers
The worklaod on different Workes is different Different media units consume different amount of computation
time
Stream-based Mapping eliminates out-of-order departure of media units.
Largest OFO rate for First Fit.
Least Load First improves 50% over First Fit.
25
Output Time Interval (OTI) per Stream
26
Output Time Interval(OTI) per Stream
Experiment setting 4 homogeneous Workers, 5 media streams
First Fit achieves the best performance.
Least Load First approaches First Fit.
Longer delay for Stream-based Mapping because of the limitation that one stream can only be processed by one Worker.
27
Load Sharing Overhead with LLF
Load Test Overhead Average time consumed by the Manager node to poll through all
Workers to collect the load statistics information.
Load Remapping Overhead Time used to set the current loads for each Worker.
Cluster Size 1 2 3 4
Load Test Overhead ( msecs)
0.87 1.6 2.3 3.0
Cluster Size 2 3 4
Load Remapping Overhead (usecs)
4.2 4.5 4.8
28
Load Sharing Overhead with LLF
Load test overhead increases roughly proportional to the cluster size.
Load re-mapping overhead is much smaller than the load test overhead, almost negligible.
The operation overhead involved in load re-mapping is much less than the network communication overhead involved in load test.
29
Evolution of Average Workload on Each Node
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
10 20 30 40 50 60 70
Time (sec)
Lo
ad
(%
)
node1
node2
node3
30
Load Sharing Schemes
How to take QoS into consideration?
Transcoding PC 1
Scheduler
Transcoding PC N
Transcoding PC 2
Unit Buffer
Receiver
.
.
.
fetch a unit Find an available
Computing PC
Send the unit
SchedulePC
31
Differentiated Service(Fair Scheduling)
A system is said to be capable of affording differentiated service among service classes if
The system permits its resources to be proportioned among the service classes
Given sufficient request load, a service class receives at least as much resources as were assigned to it irrespective of the load on other service classes
Resources not used by some service class may be distributed among other service classes.
32
Framework of Fair Scheduling
33
Fair Scheduleing Fairly distribute resource among streams
Streams make reservations Received service is proportional to the reservations
UnitScheduler - Weighted Round Robin ( WRR ) Provide differentiated service rate to multiple streams Weights in each round-robin cycle are dynamically adapted to achieve
the best performance Weight of stream i Wi(t): the number of GOPs scheduled for stream i
during one round robin cycle