neutron cascading to address scalability issue
TRANSCRIPT
HUAWEI TECHNOLOGIES CO., LTD. Page 1Huawei Confidential
Neutron cascading to address scalability issue
Chaoyi Huang ( [email protected] )
Last edited May 18, 2015
HUAWEI TECHNOLOGIES CO., LTD. Page 2Huawei Confidential
Compute node
Compute node
Compute node
Compute node
Compute node
Compute node
Compute node
Neutron - like a “multi-cast”/“burst” domain
• Security-group: project-level, neutron-levelsecurity_groups_rule_updatedsecurity_groups_member_updated…
• DVR: router level:add_arp_entrydel_arp_entryrouter_deletedrouters_updateddvr_mac_address_updatesync_router…
• L2-population: network-leveladd_fdb_entriesremove_fdb_entries
…
Neutron ServerNeutron ServerNeutron Server
DBDBDB
Msg. Bus
VM
VM
VM
VM
VM
VM
VM
VM
VM
VM
Compute node
Lots of nodes involved in Neutron API request processing
HUAWEI TECHNOLOGIES CO., LTD. Page 3Huawei Confidential
Because Neutron has to touch regarding nodes where the VMs (in network level, router level…) resides for networking purpose
( It’s quite different from Nova/Cinder which mostly manipulate single VM/volume entity !!! )
Neutron - Why challenge?
HUAWEI TECHNOLOGIES CO., LTD. Page 4Huawei Confidential
Neutron ServerNeutron ServerNeutron Server
DBDBDB
Msg. Bus
VM
Compute node
VM
Compute node
Compute node
Compute node
Compute node
VM
Compute node
Compute node
Compute node
VM
Compute node
VM
Compute node
VM VM VM VM VM
As VMs/nodes increased, “multi-cast”/”burst” heavily impacts the Neutron scalability
HUAWEI TECHNOLOGIES CO., LTD. Page 5Huawei Confidential
Neutron ServerDB
Msg. Bus
VM
Compute node
VM
Compute node
Compute node
Compute node
Compute node
VM VM VM
Then the large “multi-cast”/”burst” domain should be partitioned into smaller workable ones
Neutron ServerDB
Msg. Bus
VM
Compute node
VM
Compute node
Compute node
Compute node
Compute node
VM VM VM
Portioning according to availability zone(AZ) in Nova, one AZ one small “multi-cast”/”burst” domain
AZ1 AZ2
Neutron ServerNeutron ServerNeutron Server
DBDBDB
Msg. Bus
HUAWEI TECHNOLOGIES CO., LTD. Page 6Huawei Confidential
Why partitioning in AZs ?
1. Availability zone is also often meaning fault isolation domain. Crash of one AZ should not affect other AZ’s running
2. Application often independently (or weak relationship) deployed in multiple availability zones for redundancy purpose
3. Virtual machines in one network often located inside one availability zone. (Amazon, for example ).
HUAWEI TECHNOLOGIES CO., LTD. Page 7Huawei Confidential
Neutron ServerDB
Msg. Bus
VM
Compute node
VM
Compute node
Compute node
Compute node
Compute node
VM VM VM
Neutron ServerDB
Msg. Bus
VM
Compute node
VM
Compute node
Compute node
Compute node
Compute node
VM VM VM
AZ1 AZ2
After partition, the Neutron server API should still be reserved for API compatibility.
Neutron ServerDB
Neutron API
HUAWEI TECHNOLOGIES CO., LTD. Page 8Huawei Confidential
ProxyProxy
Neutron ServerDB
Msg. Bus
VM
Compute node
VM
Compute node
Compute node
Compute node
Compute node
VM VM VM
Neutron ServerDB
Msg. Bus
VM
Compute node
VM
Compute node
Compute node
Compute node
Compute node
VM VM VM
AZ1 AZ2
And the API request should be proxy to regarding partitioned Neutron...
Neutron ServerDB
Neutron API
HUAWEI TECHNOLOGIES CO., LTD. Page 9Huawei Confidential
ProxyProxy
Neutron ServerDB
Neutron ServerDB
Forward the network/subnet/port/SG creation to regarding partitioned Neutron where VM resides
Neutron ServerDB
Create Port ( device_owner: AZ2)
if device_owner belongs to AZ1Create Network/Subnet/Port/SG here
AZ1
VM
HUAWEI TECHNOLOGIES CO., LTD. Page 10Huawei Confidential
ProxyProxy
Neutron ServerDB
Neutron ServerDB
Forward the network/subnet/port creation to regarding partitioned Neutron where VM resides
Neutron ServerDB
Create Port ( device_owner: AZ2)
if device_owner belongs to AZ2Create Network/Subnet/Port/SG here
VM
AZ2
HUAWEI TECHNOLOGIES CO., LTD. Page 11Huawei Confidential
ProxyProxy
Neutron ServerDB
Neutron ServerDB
Router will be created where network/VM resides
Neutron ServerDB
Router1 . add_router_interface( Network1 )
AZ1
VM R1
Network1
Create Router1Router1 . add_router_interface( Network1 )
HUAWEI TECHNOLOGIES CO., LTD. Page 12Huawei Confidential
ProxyProxy
Neutron ServerDB
Neutron ServerDB
Router will be created where network/VM resides
Neutron ServerDB
Router1 . add_router_interface( Network1 )Router1 . add_router_interface( Network2 )
Create Router1Router1 . add_router_interface( Network2 )
AZ2
VM R1
Network1
VMR1
Network2
Create Router1Router1 . add_router_interface( Network1 )
AZ1
HUAWEI TECHNOLOGIES CO., LTD. Page 13Huawei Confidential
ProxyProxy
Neutron ServerDB
Neutron ServerDB
Connect distributed router in different AZs…. in many ways, and automatically by the proxy
Neutron ServerDB
AZ2
VM R1
Network1
VMR1
Network2
Provider L2 network
HUAWEI TECHNOLOGIES CO., LTD. Page 14Huawei Confidential
ProxyProxy
Neutron ServerDB
Neutron ServerDB
connect distributed router in different AZs…. in many ways, and automatically by the proxy
Neutron ServerDB
AZ2
VM R1
Network1
VMR1
Network2
GRE/VPN… tunneling
HUAWEI TECHNOLOGIES CO., LTD. Page 15Huawei Confidential
ProxyProxy
Neutron ServerDB
Neutron ServerDB
Different Neutron backend ( SDN controller ) co-existence
Neutron ServerDB
AZ2
VM R1
Network1
VMR1
Network2
AZ1
OpenDayLight OVN
GRE /VPN..tunneling
HUAWEI TECHNOLOGIES CO., LTD. Page 16Huawei Confidential
ProxyProxy
Neutron ServerDB
Neutron ServerDB
Even in different data centers (geographically)
Neutron ServerDB
AZ2
VM R1
Network1
VMR1
Network2
AZ1
OpenDayLight OVN
DC1 DC2GRE /VPN…tunneling
HUAWEI TECHNOLOGIES CO., LTD. Page 17Huawei Confidential
ProxyProxy
Neutron ServerDB
Neutron ServerDB
Even inter-connection network between AZs can be dynamically created by DCI controller
Neutron ServerDB
AZ2
VM R1
Network1
VMR1
Network2
AZ1
OpenDayLight OVN
Data Center Interconnection
Controller
creation dynamically
Provider network
HUAWEI TECHNOLOGIES CO., LTD. Page 18Huawei Confidential
ProxyProxy
Neutron ServerDB
Neutron ServerDB
Neutron ServerDB
AZ2AZ1
Proxy
Neutron ServerDB
AZ3
Router_updateRouter_update
Router_update
No challenge for “multi-cast/burst” in the cascading Neutron
For one tenant’s resource will be allocated in limited AZs ( 3 AZs by default in Amazon ), the “multi-cast/burst” will only happen in limited proxy nodes, which reduce load greatly in cascading Neutron layer
HUAWEI TECHNOLOGIES CO., LTD. Page 19Huawei Confidential
1 million ports can be supported by Neutron cascading,
verified by semi-simulation test
http://www.slideshare.net/JoeHuang7/test-report-for-open-stack-cascading-solution-to-support-1-million-v-ms-in-100-data-centers
HUAWEI TECHNOLOGIES CO., LTD. Page 20Huawei Confidential
More information:
Wiki: https://wiki.openstack.org/wiki/OpenStack_cascading_solutionPoC Source Code: https://github.com/stackforge/tricircle