consistent resource scheduling and qos management
TRANSCRIPT
Applied Research Center for Computer Networks
Moscow State University
Ruslan SmelyanskiyARCCN Director, Professor at Moscow State University
Consistent Resource Scheduling and QoS management
15.04.2023
Content
Brief ARCCN introduction and main
research direction overview
Two research problems examples:
The Consistent Resource Scheduling in DC
FDM-TCP: Flow DeMultiplexing TCP for QoS management
Huawei Shenzhen R.Smelyanskiy 2
15.04.2023
ARCCN
Research
Ideas
Education
Adoption
Cooperation
Development
Integration
Applied Research Center for Computer Networks (ARCCN) is a Russian
non-profit organization initially funded by Skolkovo intended to:
world-class competency center for Computer Networking in Russia;
collaboration between Russian and international research, scientific, educational and commercial experts and institutions;
demonstrate innovations in Networking for National Industry;
promote commercialization of a new networking technologies and services.
ARCCN is the only R&D competency center for SDN&NFV
in Russia today
Partners: Skolkovo, The Ministry of Education and Science of Russia, Rostelecom, Rostech, Sberbank, EMC, Intel, etc.
Huawei Shenzhen R.Smelyanskiy 3
15.04.2023
Project Ecosystem
7%
20%
12%54%
7%Ph.D.
Assos. Prof
Postgraduate
Engineeres
Vendor's cer-tificated
ARCCN Team
Huawei Shenzhen R.Smelyanskiy 4
15.04.2023
Key R&D directions:
Distributed SDN Controller and Applications for it;
Safety and Security in SDN;
QoS management in SDN;
NFV platform and VN functions;
Architecture for Open Flow-switch;
WAN operation prototyping and analysis;
Self Organizing Cloud for Carriers (Cloud Conductor);
Data Center Resource Management;
Forwarding policy validation and troubleshooting automation;
Software for integration with legacy transport on the Internet.
Huawei Shenzhen R.Smelyanskiy 5
Data center resources model
Physical resource model: ,where
• Р - set of computational nodes: – vh(p) - number of CPU cores on node p– qh(p) - amount of RAM on node p
• М - set of data storages: – uh(m) – size of data storage m– type(m) - m data storage type
• К - set of network switches: – - bandwidth of the switch k
• L - set of network channels: – rh(l) – bandwidth of the channel l
),( LKMPH
( )h k
Huawei Shenzhen R.Smelyanskiy 815.04.2023
Tenant model
Tenant model:
• W - set of virtual machines: – v(w) – requested number of cores for VM w– q(w) – requested amount of RAM for VM w
• S - set of storages: – u(s) – size of storage s– type(s) – s storage type
• E - set of virtual channels:
– r(e) – bandwidth of the virtual channel e
Tenant types:• Loosely coupled:
• Tightly coupled:
),( ESWG
)]),}{,}[({ 11 ESW Kjj
Nii
)}]}{,}{{(),}{,}[({ 1'
1'
11Kjj
Nii
Kjj
Nii SSWWESW
Huawei Shenzhen R.Smelyanskiy 915.04.2023
Tenant Mapping on DC Resources
where
Mapping A is correct if the following constraints are fulfilled:
a)
b)
c)
d)
)()(),()( pqhwqpvhwvpp WwWw
: { , , { , }},A G H W P S M E K L
)()( lrherlEe
)()( kherkEe
),()( muhsumSs
)()(: mtypestypeSs m
, , ; , , , W S E G P M K L H
Huawei Shenzhen R.Smelyanskiy 1015.04.2023
Replication and migration
Replication: • Duplicates storage m to m’• Channel to support consistency between m and m’:
• VM connects to m’
Migration: • Moves virtual machine or storage to another physical resource
': ii AAR
( ', , , , , , ); ; ; ',1 1 1
m l k k l m k K l L m m Mn n i i
Huawei Shenzhen R.Smelyanskiy 11
': ii AAT
15.04.2023
Tenant mapping problem
The problem:
• For every tenant Gi from set of tenants Z = {Gi} and
given DC resources Hres define correct mapping Ai
that set A = {Ai} covers the maximum number of
tenants from Z
Huawei Shenzhen R.Smelyanskiy 1215.04.2023
Residual graph
Renew Z –
Residual graph Hres:
• The following functions are recalculated: vh(p), qh(p), rh(l), τh(k), uh(m)
pWw
res wvpvhpvh )()()( ( ) ( ) ( )p
resw W
qh p qh p q w
( ) ( ) ( )
l
rese E
rh l rh l r e
( ) ( ) ( )
k
rese E
h k h k r e
( ) ( ) ( )
m
ress S
uh m uh m u s
Huawei Shenzhen R.Smelyanskiy 1315.04.2023
Scheduling Round
The scheduling algorithm runs at the begging of the scheduling rounds
On a scheduling round:– Add new tenants to Z– Delete the tenants with TTL=0– Recalculate Residual graph as a new H
Huawei Shenzhen R.Smelyanskiy 1415.04.2023
Scheduling algorithms
• A1: greedy and limited exhaustive search strategies– maps tenants as a whole (minimal common subgraph isomorphism algorithm)– suitable when critical resource is a data center physical network
• A2: greedy and limited exhaustive search strategies– constructs mappings element by element (bin-packing algorithm)– suitable when critical resources are computational nodes or data storages
• A3: Ant colony algorithm:– universal, but greater computational complexity
Huawei Shenzhen R.Smelyanskiy 1515.04.2023
Initial data for test
Test # DC model (graph H) Tenants models (graph G)
Test 1450 computational nodes with 21000MB RAM and 16 cores
per node
450 data storages with 21000GB disk space per
storage
1350 virtual machines and 1350 storages total in 135 tenants.
Test 1 – loosely coupled tenantsTest 2 – tightly coupled tenants,
network load = 70%Test 2
Test 3500 computational nodes with 10000MB RAM and 16 cores
per node
500 data storages with 10000GB disk space per
storage
600 virtual machines and 600 storages per 100 tenants.
Test 3 – loosely coupled tenantsTest 4 – tightly coupled tenants,
network load = 70%Test 4
Fattree topology with 5 TOR switches, 10 aggregation switches, 40 edge switches
Huawei Shenzhen R.Smelyanskiy1615.04.2023
Comparison with OpenStack scheduling algorithms
Test #
Open stack CRM
FF RF
Test 1 50% 87.5% 100%
Test 2 0% 1% 100%
Test 3 100% 70% 100%
Test 4 0% 0% 100%
- FF – First Fit- RF – Random Fit- CRM – Consistent Resource Mapping
Huawei Shenzhen R.Smelyanskiy 1715.04.2023
Conclusions
Open Stack algorithms are:– Ineffective for some cases of loosely coupled tenants– Not applicable for tightly coupled tenants
Consistent Resource Mapping algorithms:– Effective as for loosely as for toughly coupled tenants– Suitable for IaaS with SLAs
Huawei Shenzhen R.Smelyanskiy 1915.04.2023
Hosts do affect connection quality!
• TCP congestion avoidance algorithms:– Goal: get connection with maximal bandwidth– Strategy: cut-and-try to detect the maximum currently available
amount of resources to utilize all of them
– Primary heuristic: AIMD (pessimistic)
– Recovering action: decries CWND
– Parameter: congestion window size (CWND), timeout
– Primary modes: slow start and congestion avoidance
– Triggering criteria: duplicate ACKs&timeouts, threshold
• There is no way to control routes intersections in traditional networks
Huawei Shenzhen R.Smelyanskiy 2115.04.2023
FDM-SDN provides a new leverage
• Flow demultiplexing along several non-xing routes:– Goal: get connection with required bandwidth– Strategy: use cut-and-try to detect the amount of
provided resources and utilize all of them on several routes
– Primary heuristic: decries the number of routes (optimistic)
– Parameter: cumulative bandwidth of all routes– Primary mode: keep busy all routes– Triggering criteria: bandwidth deficiency– Recovering action: open new non-xing route
Huawei Shenzhen R.Smelyanskiy 2215.04.2023
Flow DeMultiplexing Protocol
Standard socket API
Activity Monitor (Bandwidth Scarcity Detection)
TCP subflow (extra options)
TCP subflow
(extra options)
TCP subflow (extra options)
Application Layer
Transport Layer
Network Layer
FDM TCP (Packet Scheduling & Reordering)
Subflow Manager (Split Degree Adjustment)
Huawei Shenzhen R.Smelyanskiy 2315.04.2023
Routing FDMP flows with SDN
Host A Host B
SDN Controller
Has MP_CAPABLE option?Install new FDMP connection!
SYNMP CAP
Key A
Set up new FDMP connection
Huawei Shenzhen R.Smelyanskiy 2515.04.2023
Host A Host B
SDN Controller
Has MP_CAPABLE option?Complete partial FDMP connection!
SYN, ACKMP CAP
Key B
Routing FDMP flows with SDNSet up new FDMP connection
Huawei Shenzhen R.Smelyanskiy 2615.04.2023
Host A Host B
SDN Controller
Has MP_JOIN option?Install new FDMP subflow for a
known connection!
SYNMP JOINToken B
Routing FDMP flows with SDNSet up new FDMP subflow
Huawei Shenzhen R.Smelyanskiy 2715.04.2023
Host A Host B
SDN Controller
Subflow is not active any more!Remove the path!
Routing FDMP flows with SDN FDMP subflow manipulation
Actually, we store metadata and allow some subflows to resume.
We use flow eviction to remove this data.
Huawei Shenzhen R.Smelyanskiy 2915.04.2023
Actually, we store metadata and allow some subflows to resume.
We use flow eviction to remove this data.
Host A Host B
SDN Controller
Get FDMP packet of a expired subflow!Either reroute the remembered subflow,
or force hosts to close it
RSTRST
Close violet subflow! Reschedule the packet to
red subflow!Close violet subflow!
Routing FDMP flows with SDN FDMP subflow manipulation
Huawei Shenzhen R.Smelyanskiy 3015.04.2023
h2h1lower = 50 Mbps
middle = 50 Mbps
upper = 50 Mbps
Experiments
FDMP in a hammock
TCP MP TCP FDMP
49 Mbps 48 Mbps 81 Mbps
Required h1 to h2 bandwidth: 80 Mbps
Huawei Shenzhen R.Smelyanskiy 3115.04.2023
upper = 50 Mbps
h2h1
h4h3
lower = 50 Mbps
middle = 50 Mbps
Adjustment to congestion
TCP MP TCP FDMP
49 Mbps 48 Mbps 81 Mbps
Required h1 to h2 bandwidth: 80 Mbps
Huawei Shenzhen R.Smelyanskiy 3215.04.2023
Conclusion
1. In context of bandwidth, the party of FDMP & our routing application outperform both single-flow TCP and MP TCP without regards to ECMP
2. FDMP improves efficiency of infrastructure:It can increase utilization under the same load and allows network to process more traffic
3. Under a heavy load FDMP will support faster communication and/or transmit more data within the same period of time
Huawei Shenzhen R.Smelyanskiy 3315.04.2023
Challenges we are working on
Use TCP-FDM to reduce delay and packet loss:– Which packet scheduling algorithm to use?– What routing algorithms are appropriate?
Enforce resource allocation & QoS policies:– How to avoid flow competition on congestion?– Do we need to control request frequency?– Is it efficient enough to rate-limit subflows?– How to combine TCP-FDM and Diff Serv model?
Develop FDMP as VNF within network:– How to integrate TCP-FDM routing application?– How to build an efficient TCP to TCP-FDM proxy?
Huawei Shenzhen R.Smelyanskiy 3415.04.2023
15.04.2023 Huawei Shenzhen R.Smelyanskiy 35
http://arccn.ru/
+7 (495) 984 27 64
@ArccnNews