This was a long requested “Enterprise feature”.
Let’s look at what this is and how to use it.
Please note that at the time of this writing (end-of-September 2017) this feature is available only in a few region in Public Preview:
- Azure Storage: WestCentralUS, WestUS2, EastUS, WestUS, AustraliaEast, and AustraliaSouthEast
- Azure SQL Database: WestCentralUS, WestUS2, and EastUS
Here is a bit of online documentation about the topic:
- Service Launch blog
- Virtual Network Service Endpoints
- How to Configure Virtual Network Service Endpoints
- Virtual Network Service Endpoints with Azure SQL
- Virtual Network Service Endpoints with Azure Storage
- Service Tags
The first (historically) Azure Services, e.g. Azure Storage & Azure SQL, were built with a public cloud philosophy:
- They are accessible through public IPs
- They are multi-tenant, e.g. public IPs are shared between many domain names
- They live on shared infrastructures (e.g. VMs)
Many more recent services share many of those characteristics, for instance Data Factory, Event Hub, Cosmos DB, etc. .
Those are all Platform as a Service (PaaS) services.
Then came the IaaS wave, offering more control and being less opinionated about how we should expose & manage cloud assets. With it we could replicate in large parts an on premise environment. First came Virtual Machines, then Virtual Networks (akin to on premise VLANs), then Network Security Groups (akin to on premise Firewall rules), then Virtual Network Appliances (literally a software version of an on premise Firewall), etc. .
Enterprises love this IaaS as it allows to more quickly migrate assets to the cloud since they can more easily adapt their governance model.
But Enterprises, like all Cloud users, realize that the best TCO is in PaaS services.
This is where the two models collided.
After we spent all this effort stonewalling our VMs within a Virtual Network, implementing access rules, e.g. inbound PORT 80 connections can only come from on premise users through the VPN Gateway, we were going to access the Azure SQL Database through a public endpoint?
That didn’t go down easily.
Azure SQL DB specifically has an integrated firewall. We can block all access, leave only IP ranges (again good for connection over the internet) or leave “Azure connections”. The last one look more secure as no one from an hotel room (or a bed in New Jersey) could access the database. But anyone within Azure, anyone, could still access it.
The kind of default architecture was something like this:
This did put a lot of friction to the adoption of PaaS services by Enterprise customers.
The solution until now
The previous diagram is a somewhat naïve deployment and we could do better. A lot of production deployments are like this though.
We could do better by controlling the access via incoming IP addresses. Outbound connections from a VM come through a Public IP. We could filter access given that IP within the PaaS Service integrated Firewall.
In order to do that, we needed a static public IP though. Dynamic IP preserves their domain name but they aren’t guaranteed to preserve their underlying IP value.
This solution had several disadvantages:
- It requires a different paradigm (public IP filtering vs VNET / NSGs) to secure access
- It requires static IPs
- If the VMs were not meant to be exposed on the internet, it adds on configuration and triggers some security questions during reviews
- A lot of deployment included a “force tunneling” to the on premise firewall for internet access ; since the Azure SQL DB technically is on the internet, traffic was routed on premise, increasing latency substantially
And this is where we were at until this week when VNET Service Endpoints were announced at Microsoft Ignite.
The ideal solution would be to instantiate the PaaS service within a VNET. For a lot of PaaS services, given their multi-tenant nature, it is impossible to do.
For multi-tenant PaaS where the communication is always outbound to the Azure service (i.e. the service doesn’t initiate a connection to our VMs), the solution going forward is VNET Service Endpoints.
At the time of this writing, only Azure Storage, Azure SQL DB & Azure SQL Data Warehouse do support that mechanisms. Other PaaS services are planned to support it in the future.
VNET Service Endpoints does the next best thing to instantiating the PaaS service in our VNET. It allows us to filter connections according to the VNET / Subnet of the source.
This is made possible by a fundamental change in the Azure Network Stack. VNETs now have identities that can be carried with a connection.
So we are back to where we wanted to be:
The solution isn’t perfect. For instance, it doesn’t allow to filter for connections coming from on premise computers via a VPN Gateway: the connection needs to be initiated from the VNET itself for VNET Service Endpoints to work.
Also, the access to PaaS resources is still done via the PaaS public IP so the VNET must allow connection to the internet. This is mitigated by new tags allowing to target some specific PaaS ; for instance, we could allow traffic going only to Azure SQL DBs (although not our Azure SQL DB instance only).
The connection does bypass “force tunneling” though and therefore the traffic stays on the Microsoft Network thus improving latency.
VNET Service Endpoints allow to secure access to PaaS services such as Azure SQL DB, Azure SQL Data Warehouse & Azure Storage (and soon more).
It offers something close to bringing the PaaS services to our VNET.