I’m going to document this once and for all.
You want to an endpoint multiple times in parallel. Or maybe you want to call multiple endpoints under the same domain name. For instance, you might want to drill an API with multiple requests because the API doesn’t support batch mode.
The problem is that .NET, by default, supports only 2 TCP connections to the same IP address in parallel. If you async a bunch of web-calls, it’s going to simply queue them and run them 2-by-2. So you won’t scale much.
Sure this limit of two isn’t hardcoded in .NET and you can change it, can’t you!?
Yes we can override that number. It is driven by what is called the Service Point Manager in System.Net.
There are two ways to override it: by configuration or by code. I would suggest to use the configuration route if your needs are static and by code if you need to change it given an input.
Here’s an example on how to override in in configuration:
<configuration> <system.net> <connectionManagement> <add address="myapi.com" maxconnection="12"/> </connectionManagement> </system.net> </configuration>
Here I specify to have a maximum number of connections of 12 instead of 2 on the domain myapi.com only.
I could specify different rules for different domains:
<configuration> <system.net> <connectionManagement> <add address="myapi.com" maxconnection="12"/> <add address="yourapi.com" maxconnection="8"/> <add address="hisapi.com" maxconnection="4"/> </connectionManagement> </system.net> </configuration>
Or I could do a blanket statement:
<configuration> <system.net> <connectionManagement> <add address="*" maxconnection="15"/> </connectionManagement> </system.net> </configuration>
In code, the easiest way is to do a blanket statement on all domains, using the static ServicePointManager.DefaultConnectionLimit property:
ServicePointManager.DefaultConnectionLimit = 15;
In order to go by domain, I would go through ServicePoint objects, BUT I NEVER TRIED IT:
var myApiServicePoint = ServicePointManager.FindServicePoint("myapi.com"); myApiServicePoint.ConnectionLimit = 12; var yourApiServicePoint = ServicePointManager.FindServicePoint("yourapi.com"); yourApiServicePoint.ConnectionLimit = 8; var hisApiServicePoint = ServicePointManager.FindServicePoint("hisapi.com"); hisApiServicePoint.ConnectionLimit = 4;
The Silver Bullet
Now that we have this solution, we feel all empowered, right?
I mean, we can now bombard our favorite APIs limitlessly. Typically the heavy lifting is done on the API side so if we invoke APIs asynchronously, we can stream a lot of activity from a low-compute server, right?
The caveat is… will the API let you? If you hammer an API, there are two typical outcomes:
- You’re gona crash it
- It’s going to throttle you, or worse, it’s going to black list you for a while a actively refuse your connections
Basically the second outcome is from an API owner that didn’t want the first outcome to happened 😉
And I know it, because I’ve done it! I’ve tried that trick on Azure Active Directory Graph API ; performance climbed for a few seconds then dropped drastically: I got throttled. Worse: I got throttled for an hour. For an entire hour the performance sucked because my IP got black listed.
So be mindful about that and wield the connection limit sword carefully 😉