VNET Service Endpoints for Azure SQL & Storage


internet-1676139_640It’s finally here, it has arrived:  Azure Virtual Network Service Endpoints.

This was a long requested “Enterprise feature”.

Let’s look at what this is and how to use it.

Please note that at the time of this writing (end-of-September 2017) this feature is available only in a few region in Public Preview:

  • Azure Storage: WestCentralUS, WestUS2, EastUS, WestUS, AustraliaEast, and AustraliaSouthEast
  • Azure SQL Database: WestCentralUS, WestUS2, and EastUS

Online Resources

Here is a bit of online documentation about the topic:

The problem

The first (historically) Azure Services, e.g. Azure Storage & Azure SQL, were built with a public cloud philosophy:

  • They are accessible through public IPs
  • They are multi-tenant, e.g. public IPs are shared between many domain names
  • They live on shared infrastructures (e.g. VMs)
  • etc.

Many more recent services share many of those characteristics, for instance Data Factory, Event Hub, Cosmos DB, etc.  .

Those are all Platform as a Service (PaaS) services.

Then came the IaaS wave, offering more control and being less opinionated about how we should expose & manage cloud assets.  With it we could replicate in large parts an on premise environment.  First came Virtual Machines, then Virtual Networks (akin to on premise VLANs), then Network Security Groups (akin to on premise Firewall rules), then Virtual Network Appliances (literally a software version of an on premise Firewall), etc.  .

Enterprises love this IaaS as it allows to more quickly migrate assets to the cloud since they can more easily adapt their governance model.

But Enterprises, like all Cloud users, realize that the best TCO is in PaaS services.

This is where the two models collided.

After we spent all this effort stonewalling our VMs within a Virtual Network, implementing access rules, e.g. inbound PORT 80 connections can only come from on premise users through the VPN Gateway, we were going to access the Azure SQL Database through a public endpoint?

That didn’t go down easily.

Azure SQL DB specifically has an integrated firewall.  We can block all access, leave only IP ranges (again good for connection over the internet) or leave “Azure connections”.  The last one look more secure as no one from an hotel room (or a bed in New Jersey) could access the database.  But anyone within Azure, anyone, could still access it.

The kind of default architecture was something like this:

image

This did put a lot of friction to the adoption of PaaS services by Enterprise customers.

The solution until now

The previous diagram is a somewhat naïve deployment and we could do better.  A lot of production deployments are like this though.

We could do better by controlling the access via incoming IP addresses.  Outbound connections from a VM come through a Public IP.  We could filter access given that IP within the PaaS Service integrated Firewall.

In order to do that, we needed a static public IP though.  Dynamic IP preserves their domain name but they aren’t guaranteed to preserve their underlying IP value.

image

This solution had several disadvantages:

  • It requires a different paradigm (public IP filtering vs VNET / NSGs) to secure access
  • It requires static IPs
  • If the VMs were not meant to be exposed on the internet, it adds on configuration and triggers some security questions during reviews
  • A lot of deployment included a “force tunneling” to the on premise firewall for internet access ; since the Azure SQL DB technically is on the internet, traffic was routed on premise, increasing latency substantially

And this is where we were at until this week when VNET Service Endpoints were announced at Microsoft Ignite.

The solution

The ideal solution would be to instantiate the PaaS service within a VNET.  For a lot of PaaS services, given their multi-tenant nature, it is impossible to do.

That is the approach taken by a lot of single-tenant PaaS services though, e.g. HD Insights, Application Gateway, Redis Cache, etc.  .

For multi-tenant PaaS where the communication is always outbound to the Azure service (i.e. the service doesn’t initiate a connection to our VMs), the solution going forward is VNET Service Endpoints.

At the time of this writing, only Azure Storage, Azure SQL DB & Azure SQL Data Warehouse do support that mechanisms.  Other PaaS services are planned to support it in the future.

VNET Service Endpoints does the next best thing to instantiating the PaaS service in our VNET.  It allows us to filter connections according to the VNET / Subnet of the source.

This is made possible by a fundamental change in the Azure Network Stack.  VNETs now have identities that can be carried with a connection.

So we are back to where we wanted to be:

image

The solution isn’t perfect.  For instance, it doesn’t allow to filter for connections coming from on premise computers via a VPN Gateway:  the connection needs to be initiated from the VNET itself for VNET Service Endpoints to work.

Also, the access to PaaS resources is still done via the PaaS public IP so the VNET must allow connection to the internet.  This is mitigated by new tags allowing to target some specific PaaS ; for instance, we could allow traffic going only to Azure SQL DBs (although not our Azure SQL DB instance only).

The connection does bypass “force tunneling” though and therefore the traffic stays on the Microsoft Network thus improving latency.

Summary

VNET Service Endpoints allow to secure access to PaaS services such as Azure SQL DB, Azure SQL Data Warehouse & Azure Storage (and soon more).

It offers something close to bringing the PaaS services to our VNET.

Advertisements

2 thoughts on “VNET Service Endpoints for Azure SQL & Storage

  1. Pingback: Virtual Network Service Endpoint – Hello World | Vincent-Philippe Lauzon's blog

  2. Pingback: Azure Weekly: Oct 9, 2017 – Build Azure

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s