hyper-scaling on openstack with open source tooling...open source: interoperability with most open...
TRANSCRIPT
Hyper-scaling on Openstack with Open Source toolingA use case in deploying hyper-scale grid computing on Open Telekom Cloud
Why Open Source?
Community LeadContributor ModelCommunity Tested
Open Standards
Standards Organizations:IEEE, ISOC, W3C, IETF, IAB, ETSI, OASIS, NIST, CSCC, ANSI
Open Design
Cloud Design PatternsApplication Design PatternsDistributed Systems Design Patterns
Open Choice
No-Lock inInterchangeable
Components
Interoperability
Reusability
Invest in People not Software
Why Openstack ?
Community: The OpenStack community
is nearing 10,000 members in nearly 100
countries, with foundation support by more than
200 companies.
Mixed Hypervisor: OpenStack is the only solution
that allows for mixed hypervisor IT environments
Open Source: Open source solutions are now
widely accepted and adopted after 20 years in the
enterprise, paving the way for OpenStack success.
Standards: provides a common, open standard
through its API and allows portability between
cloud environments. T-Systems Hybrid Cloud and
Public Cloud based on Openstack (Same
Architecture) providing interoperability and ease of
migration or Burst scenarios.
Community: Sharing ideas with the community
allows our Developers to learn fast and deliver
quicker. Contributing advances the entire
ecosystem
Research & Development: Lowers our R&D cost
and speeds time to market for new PaaS services.
Open Source: Interoperability with most open
source projects such a Kubernetes, Spark, Etc
providing us the ability to deliver standardized
services.
Standards: Standardization allows us to delivery
Public Cloud and Private Cloud solutions
architected exactly the same.
We can deliver new PaaS services within 90-120
days of requirements from customers.
User Benefits Benefits to Cloud Provider
Openstack Contribution
Source: stackalytics.com
Adaptability and access to the source code
The open architecture design gives in-house IT more scope to tailor the technology to specific needs
Participating in the community enables us to influence developments
Community: many providers and solutions that enhance functionality
Establishment of multi-cloud scenarios
Hardware neutrality: connectors are available for almost all storage, server and network components on the market
High degree of standardization (open APIs)
Resources can be provisioned and marketed across provider boundaries
Data and applications can be transferred from provider to provider
How do we contribute to the Open Source community
Zaqar
Kuryr
Tacker
Rally
Ceilometer
Search Light
Karbor
Manila
Neutron
Zun
Heat
Tricircle
We developed the Tricircle project which allows
Openstack to be a viable Public Cloud architecture.
We contribute regularly to the project listed to the right as
leader or team member.
We are on the board of governors for Openstack
The Openstack and Open Source community benefit
from much of the Research and Development work we
do for the public cloud and private cloud services.
We have circa 5,000 engineers dedicated to the
Openstack projects for Public and Private Cloud
Solutions who are leading the way on AI, ML, IOT,
Kubernetes Container services.
Corporate member of the Linux Foundation and on
governing board of CNCF, Member LF Deep Learning
Foundation.
Hybrid Cloud Solution based on OpenStack
Management Link (OPS – public OTC)
Management Link (OPS – Hybrid/Private Region)
Data link (Hybrid/Private Region to public OTC)E.g. Ethernet Connect or MPLS
TSI Datacenter
Datacenter of choice
Small scale
O&M Team
Budapest
Public Cloud Architecture on-premise at
customer site or hosted location
Agility and scalability built in
Cloud Native capability
• Public Cloud based on Openstack
Deployment Tooling with Terraforms and Ansible
What is Terraform ? Terraform is a multi-cloud DevOps tool for
building, changing, and versioning infrastructure safely and efficiently. Available as Open Source or Enterprise License.
Enables Infrastructure-As-Code deployments
Open Telekom Provider developed for Terraform
What is Ansible ?
Application Deployment Tooling
Agentless Architecture
Integrates seamlessly with Terraforms
Grid Computing in Finance Industry -Use Case for OTC
What is Grid Computing Grid computing is a distributed architecture of large numbers of computers
connected to solve a complex problem. In the grid computing model, servers or
personal computers run independent tasks and are loosely linked by the Internet or
low-speed networks. Computers may connect directly or via scheduling systems.
Finance Use Case (Grid Computing)
• Bank was looking to cloud computing to carry out their complex calculations, marking a shift from investing heavily in in-house systems that are expensive to develop, maintain and upgrade.
• Regulatory and Business demands such as
Fundamental review of the trading book (FRTB)
• Derivatives Valuation Adjustments (XVA)
• End-Of-Day & Monte Carlo
• Transforming IT Operations to DevOPS methodology.• Standardized on Open Source tooling and systems such as Ansible, Terraforms, Kubernettes, Kafka, etc)• Need for massively scalable cloud services for Grid Computing, Big Data, and Devops( Kubernetes, ML, Analytics,
Open Banking)• Opex payment model.• Do not want to invest in on premise servers that are not used much of the time or ever have enough capacity.• Must speed up time to results to the business (Currently 36 to 48 hours) on FRTB and XVA calculations.
Business Requirements
IT Requirements
Grid Computing Requirements and Challenges
Technical RequirementsCapacity requirements
26,000 vCPUs minimum
50,000 vCPUs Burst
Grid Software: Tibco Gridserver
Infrastructure deployment with Terraform
Server Configuration with Ansible
Elastic Compute (ECS) and Elastic Volume Service (EVS)
Challenges to DeliveryAPI Gateway overloaded – Rearchitectected Openstack API services to handle up to 10,000 API calls a second.
Server provisioning to slow due to delay in image services deployment of images. Improved storage network capacity and latency as well image distribution across pools).
Required lots of very large hosts which were not already available. Deployed 2000 additional host to meet demand.
Improvements in Processing time
Primary Benefit to Bank – Time is money
Deploying the Grid using Terraforms
resource "opentelekomcloud_compute_instance_v2"
"engine_node1" {
count = "${var.vm_engine_count}"
availability_zone ="eu-de-01"
name = "${var.project}-node${format("%02d", count.index+1)}-az1"
image_name = "Enterprise_Windows_STD_2012R2_XEN"
flavor_name = "s2.medium.4"
key_pair = "${openstack_compute_keypair_v2.keypair.name}"
security_groups = [ "AD" ]
#ANSIBLE scripts
user_data = "#ps1\nGet-NetAdapter\nnet user ansible Test1234+
/add\nnet localgroup administrators ansible /add\n#ps1\nGet-
NetAdapter -name 'Local Area Connection' | Set-DnsClient -
ConnectionSpecificSuffix cloudgrid.bank.com -
RegisterThisConnectionsAddress:$true -
UseSuffixWhenRegistering:$true\nSet-NetFirewallProfile -Profile
Domain,Public,Private -Enabled False "
OTC API
Full Grid deployed to OTC
Terraforms Plan
Circa 2 Hours to deploy
15,000 vCPUs