cross-layer scheduling in cloud computing systems
DESCRIPTION
Cross-Layer Scheduling in Cloud Computing Systems. Authors: Hilfi Alkaff , Indranil Gupta. Motivation. Many cloud computing frameworks out there Batch Processing Framework: Hadoop Stream Processing Framework: Storm Current applications are not aware of underlying network topology - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/1.jpg)
Cross-Layer Scheduling in Cloud Computing Systems
Authors: Hilfi Alkaff, Indranil Gupta
![Page 2: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/2.jpg)
Motivation
• Many cloud computing frameworks out there– Batch Processing Framework: Hadoop– Stream Processing Framework: Storm
• Current applications are not aware of underlying network topology– Might schedule tasks on machines with low
bandwidth.
![Page 3: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/3.jpg)
Challenges
• Need to expose underlying network topology efficiently to applications
• Huge state space to search– Thousands of machines in a cluster– Users demand more interactive jobs
• Multiple possible data-path representation– Want to have generic schedulers
![Page 4: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/4.jpg)
Data-Path: Map-Reduce
![Page 5: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/5.jpg)
Data-Path: Stream
![Page 6: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/6.jpg)
Proposed Solution
• Cross-Layer Scheduling Framework– First-level scheduler in application Level– Second-level scheduler in routing level
• Use Simulated Annealing at each level– Probabilistic framework– Idea: If neighboring state is better, always move
there but if it is not, move there with probability P(T) that decreases with time
![Page 7: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/7.jpg)
Proposed ArchitectureApplication
Master
SDN Controller
Cross-Layer Scheduling
![Page 8: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/8.jpg)
Algorithm: Pre-computation
• Pre-compute all-pairs (, k-shortest paths– Stored in Topology-Map hash-table with key=(, ,
value=array of k-shortest paths• Too many duplicates– Intelligently merge similar sub-paths– Hash-Table’s value is now a tree instead of array
![Page 9: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/9.jpg)
Algorithm: Main
![Page 10: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/10.jpg)
Algorithm: genState() Heuristic
• Too many neighboring states– Not possible to traverse all of them
• Application Level– Prefer node that has higher # of sink vertices– Prefer node that has higher # of source vertices
• Routing Level– Prefer paths that have lower number of hops– Prefer paths that have higher amount of available
bandwidth
![Page 11: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/11.jpg)
Emulab Result: Throughput
![Page 12: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/12.jpg)
Simulation Result: Computation Time
![Page 13: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/13.jpg)
Simulation Results: CDF
![Page 14: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/14.jpg)
Le Questions?
![Page 15: Cross-Layer Scheduling in Cloud Computing Systems](https://reader035.vdocument.in/reader035/viewer/2022062410/56816063550346895dcf8d65/html5/thumbnails/15.jpg)
Algorithm: Failures
• Link-Failures– Need to re-allocate flows using that link– Keep a separate hash-table where key=edge,
value=flows– Get another path from Topology-Map.
• Machine-failures– Re-run main algorithm on