Kubernetes Services in Azure AKS - Network integrationSolution ·
Azure Kubernetes Services (AKS) is a managed Kubernetes service in Azure. Not only does it provides a managed cluster, it also integrates with Azure services.
In this article we’ll explain the integration between Kubernetes’ services and Azure Networking. We won’t give code sample though as we’ll keep that for a future article.
Update (30-08-2018): See the following article in order to deploy and test the concepts established here using an ARM Template.
Anatomy of Services
Let’s dive right in. I like to think about a Kubernetes service in the following way:
From the ground up:
|Container Image||Contains all the files enabling a container runtime. It also holds the ports configuration.|
|Container||Based on an image, it runs within the image context. It also maps VM ports to inside-container ports.|
|Pod||A pod can run one or many containers. A pod has its own IP. A pod maps the VM ports used by the container the IP of the pod.|
|Replica Set||Pods are ephemeral and can die. A Replica set ensures a number of pods following the same template (e.g. container images) are up and running. A selector allows the replica set to aggregate its pods.|
|Service||A service exposes a bunch of pods as a single... service. A service also has its own IP. It maps the pods port to that IP. It aggregates multiple pods via a selector.|
A few observations:
- A service isn't dependant on a replica set directly. The replica set ensures multiple pods are running. The service exposes those pods.
- Ports are mapped multiple times (from image to container to pod to service)
- IPs are mapped a few times (from VM to pod to service)
- Kubernetes' DNS plugin allows IP virtualization ; this is why some IPs are accessible from within the cluster but not from outside of it.
Services in AKS
Kubernetes is designed to integrate to cloud environments. It has different plugins implemented differently on different cloud platforms.
There are two network modes for AKS: Basic & Advanced.
Under basic networking, AKS manages its own Virtual Network. Pods IPs are accessible from within the cluster. Only publicly exposed services are accessible outside the cluster.
Under advanced networking, AKS’s nodes are deployed within a specified Azure Virtual Network we control. This allows us to keep the services running on AKS on private IPs. It also allows us to make them accessible from on premise network.
Advanced Networking scenario
There are many ways to leverage Kubernetes in AKS. To simplify the discussion, let’s take one scenario:
- One private service (i.e. no public IP)
- 3 pods based on one container
- AKS Cluster with 3 nodes
- Service is exposed in a separate subnet
- We use Azure Load Balancer
- Ports are different
- Container exposes port 80 internally
- Pod remaps it on its port 85
- Service is exposing the pods on port 90
This is a simple scenario for Advanced Networking. The last point, i.e. port mapping, is somewhat complicated. We chose that configuration to show port mapping.
Let’s see how this scenario lands in AKS.
First, we have our Virtual Network before we deploy AKS in it.
We configure it with two subnets:
- AKS: this is the subnet for AKS
- Services: this is where we are going to take private IPs for services
Now we deploy AKS in Advanced Networking mode. We specify the AKS subnet.
Under the hood, AKS deploys 3 VMs with a NIC each. Each VM has a private IP. In Kubernetes, those are the Nodes IPs.
Deploying a replica set
Now let’s deploy our replica set. This deploys 3 pods on our cluster.
Each pod is given a private IP within the same subnet as the VM. The pod expose port 85. It remaps the container port 80 to port 85.
Under the hood, those IPs belong to the VMs and are assigned at creation. They are mapped to pods as pods are created.
Deploying a service
Finally, we deploy our service using Azure Load Balancer. We specify Services Subnet as the target subnet.
The Azure Load Balancer has a private IP and forwards traffic to pods. It exposes the port 90 and forwards traffic from that port to the pods’ port 85.
In Kubernetes lingo this is called the External IP.
The service also has an Internal IP. That internal IP doesn’t belong to the Virtual Network. It is a virtual address routable only from within the cluster. That IP simply routes to the load balance private IP.
We hope this overview of Services in Kubernetes and AKS clarified some of the mechanisms at play.
In a future article, we will give explore to implement what we’ve explore in this article.