Download - Scaling Azure for Outgoing HTTP Requests
Scaling Azure for Outgoing HTTP RequestsScaling to thousands of requests / sec
Gergely Orosz@GergelyOrosz
The Problem
Implementation
• Windows Server 2012• C#• HttpClient
GET & POST
Implementation
NetworkErrors
>~1000 req / sec
4 DCs,12 VMs each
Network Errors• System.Net.WebException:
The underlying connection was closed: An unexpected error occurred on a send.
• Sympthom of Port Exhaustion• Let’s scale up!
Number of Deployments
VMs per Deployment
Treshold for Errors (req/sec)
4 12 10004 24 10008 12 10008 24 2000
Network Errors After some more research…
Two LimitsPer deployment: ~350 req / secPer VM: ~20 req / sec
Number of Deployments
VMs per Deployment
Treshold for Errors (req/sec)
1 1 171 2 341 100 3502 100 700
What’s Going On?1. The VM Limit of ~17 / sec netstat
lots of TIME_WAIT connections TCP
TCP close sequence Windows default TCP configuration
TIME_WAIT = 4 minutesPorts to use: 1024 – 50004000 / (4 * 60) = 16.66
What’s going on?1. The VM Limit of ~17 / sec
Port exhaustionChange default Windows configurations
Decrease TIME_WAITOr increase number of ports to use
Increase number of portsnetsh int ipv4 set dynamicport tcp start=1025 num=64511New limit: 265 req / sec / VM
What’s going on?2. The Deployment Limit of ~350 req / sec
NATTIME_WAIT in Azure is 180 seconds65K ports65,000 / 180 = 360
Need to add more deployments to scale beyond this limit
Some other findings Choice of the .NET client library
Success rate under heavy loadHttpClient – 78%Shared HttpClient – 98.5%WebRequest – 99.5%
Use WebRequest when possibleOr share HttpClient instances
Some other findings Tuning of IIS
A configuration that worked better than the default onedynamic port allocation: 64KappConcurrencyLimit: 750KqueueLength: 65KminWorkerThreads: 5K, maxWorkerThreads: 10KminIoThreads: 500, maxIoThreads: 1000requestQueueLimit: 750KconnectionTimeout: 3 minute
Increased successful responses by 30%
Summary Know how TCP & NAT works
Increase number of dynamic ports on VMs to scale beyond 17 req / secIncrease number of deployments to scale beyond 350 req / sec
Use WebRequest or shared HttpClient
Tune IIS for performance
You’re now ready to scale
Thank You@GergelyOrosz
Visualtini