Beyond 2 concurrent connections in .NET


I’m going to document this once and for all.

The Problem

Ethernet-Cable-icon[1]You want to an endpoint multiple times in parallel. Or maybe you want to call multiple endpoints under the same domain name. For instance, you might want to drill an API with multiple requests because the API doesn’t support batch mode.

The problem is that .NET, by default, supports only 2 TCP connections to the same IP address in parallel. If you async a bunch of web-calls, it’s going to simply queue them and run them 2-by-2. So you won’t scale much.

Sure this limit of two isn’t hardcoded in .NET and you can change it, can’t you!?

The Solution

Business-Parallel-Tasks-icon[1]Yes we can override that number.  It is driven by what is called the Service Point Manager in System.Net.

There are two ways to override it:  by configuration or by code.  I would suggest to use the configuration route if your needs are static and by code if you need to change it given an input.

Configuration

Here’s an example on how to override in in configuration:

<configuration>
 <system.net>
  <connectionManagement>
   <add address="myapi.com" maxconnection="12"/>
  </connectionManagement>
 </system.net>
</configuration>

Here I specify to have a maximum number of connections of 12 instead of 2 on the domain myapi.com only.

I could specify different rules for different domains:

<configuration>
 <system.net>
  <connectionManagement>
   <add address="myapi.com" maxconnection="12"/>
   <add address="yourapi.com" maxconnection="8"/>
   <add address="hisapi.com" maxconnection="4"/>
  </connectionManagement>
 </system.net>
</configuration>

Or I could do a blanket statement:

<configuration>
 <system.net>
  <connectionManagement>
   <add address="*" maxconnection="15"/>
  </connectionManagement>
 </system.net>
</configuration>

Configuration

In code, the easiest way is to do a blanket statement on all domains, using the static ServicePointManager.DefaultConnectionLimit property:

ServicePointManager.DefaultConnectionLimit = 15;

In order to go by domain, I would go through ServicePoint objects, BUT I NEVER TRIED IT:

var myApiServicePoint =
  ServicePointManager.FindServicePoint("myapi.com");

myApiServicePoint.ConnectionLimit = 12;

var yourApiServicePoint =
  ServicePointManager.FindServicePoint("yourapi.com");

yourApiServicePoint.ConnectionLimit = 8;

var hisApiServicePoint =
  ServicePointManager.FindServicePoint("hisapi.com");

hisApiServicePoint.ConnectionLimit = 4;

The Silver Bullet

Now that we have this solution, we feel all empowered, right?

I mean, we can now bombard our favorite APIs limitlessly.  Typically the heavy lifting is done on the API side so if we invoke APIs asynchronously, we can stream a lot of activity from a low-compute server, right?

The caveat is…  will the API let you?  If you hammer an API, there are two typical outcomes:

  • You’re gona crash it
  • It’s going to throttle you, or worse, it’s going to black list you for a while a actively refuse your connections

Basically the second outcome is from an API owner that didn’t want the first outcome to happened 😉

And I know it, because I’ve done it!  I’ve tried that trick on Azure Active Directory Graph API ; performance climbed for a few seconds then dropped drastically:  I got throttled.  Worse:  I got throttled for an hour.  For an entire hour the performance sucked because my IP got black listed.

I’ve done it with IMDB last week.  I was trying to download its entire catalog by hitting every movie page using Azure Batch and I got black listed (again!).

So be mindful about that and wield the connection limit sword carefully 😉

0D9BF61E08[1]

 

One thought on “Beyond 2 concurrent connections in .NET

  1. Pingback: Beyond 2 concurrent connections in .NET | Dinesh Ram Kali.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s