Understanding identities in Azure AKS / Kubernetes


https://pixabay.com/en/face-faces-dialogue-talk-psyche-3189811/We’ve recently looked at Azure AKS (Kubernetes Cluster Managed Services).  We’ve looked at how to create a Kubernetes Cluster with 3 lines of Azure CLI.

With this we are able to interact with the cluster & deploy containers.

There is a bit of “auto magic” happening in there.  In this article we will focus on that magic.  Specifically, we’ll focus on identity.

The first bit of magic is how do we connect to the cluster?  At no point have we logged in on the cluster.  When we interact with the cluster, we use kubectl.  That is a Kubernetes tool, not an Azure tool.  Can that tool use our Azure AD credentials?

The second bit of magic might not be obvious on first trial.  Let’s consider using Kubernetes for tasks such as:

Those tasks do create Azure resources:  they create Public IPs and Azure Disks.

A question we could ask is “who is creating those resources”?  Which Azure AD user is creating the Azure resources?  Is it our own personal account?  We’re the one behind the keyboard and we’re logged into Azure CLI.  But this isn’t Azure CLI, it’s the Kubernetes AKS Cluster, from within Azure.

Let’s unpack that magic.

Identity Future

A caveat from the get go, Azure RBAC integration is coming to AKSJuan Manuel Rey give a sneak peak of how it might look here.

As of this writing (end of April 2018), it isn’t implemented.

Workflow with identities

Here are the three CLI lines to create and connect to an AKS Kubernetes Cluster:

(code)

az aks create –resource-group myResourceGroup –name myAKSCluster –node-count 1 –generate-ssh-keys

az aks install-cli

az aks get-credentials –resource-group myResourceGroup –name myAKSCluster

(code)

The first line creates the cluster.  The second installs kubectl (Kubernetes client controller).  The third one configures kubectl to connect to the cluster.

Only step 1 & step 3 involves identity.  Step 2 litterally is only an install of kubectl tool.

Let’s add the following steps:

  1. Creating a replica set (through a Kubernetes deployment)
  2. Creating an Azure Disk via Persistant Volume Claim

Step #4 deals with the cluster only.  Step #5 creates objects on the cluster.  But it also integrates with Azure to create an Azure Disk and attaching it to a VM in the cluster.

So let’s look at the entire workflow focussing on the identity.

Step 1 – az aks create

Step1

Let’s walk through the sub steps, focussing on how identities are handled:

  1. End-user launch the Azure CLI command “az aks create…
  2. The Azure CLI uses cached user’s token (i.e. Azure AD user account) to do an Azure REST API call to create the AKS service
  3. Azure Resource Manager provisions an AKS Service.  That service will orchestrate different sub resources creation.
  4. AKS creates a Service Principal in the same Azure AD tenant as the user
  5. AKS provisions a set of worker nodes VMs and install Kubernetes on them
  6. When Kubernetes is created (on the master nodes not shown on this diagram), it creates a cluster admin.  That cluster admin is a Kubernetes user and doesn’t belong to Azure AD.
  7. Service Principal credentials are copied on master and worker nodes at /etc/kubernetes/azure.json
  8. REST API returns the Service Principal credentials.  They are written locally on ~/.azure/aksServicePrincipal.json.  That file gets overwritten every time, i.e. it doesn’t accumulate credentials.

Already, on the first step, on this innocuous CLI command, we see three identities:

  1. The Azure AD User identity:  this is the user running Azure REST API commands.  In automation context, that could also be a service principal.
  2. A Service principal:  this identity will later interact with Azure (e.g. creating Azure disks) on behalf of the cluster
  3. The Kubernetes admin user:  a non-Azure AD user.  This is the identity that will later run kubectl commands.

An important point here.  The Azure AD user must have the right to create service principals.  If not, creation will fail.

Step 2 – az aks install-cli

To our knowledge, nothing happens at the identity level with that command.  It installs a version of kubectl CLI compatible with Azure.

Step 3 – az aks get-credentials

Step3

Here are the sub steps:

  1. End-user launch the Azure CLI command “az get credentials…
  2. The Azure CLI uses cached user’s token (i.e. Azure AD user account) to do an
    Azure REST API call to get credentials from the AKS service
  3. Azure Resource Manager queries AKS Service to get the Kubernetes Admin credentials
  4. The REST API returns the Kubernetes Admin credentials.  They are written
    locally (e.g. on end user laptop).  Path is ~/.kube/config.

Subsequent command will then be able to simply lookup the ~/.kube/config file to authenticate against the cluster.  That YAML file accumulates information about different clusters and keeps track of the “current” one.  Here is an example with 2 clusters:

image

Step 4 – Create replicate set

Let’s assume we deploy some containers on a replica set without creating Azure resources.  This step illustrate the typical interaction with Kubernetes cluster.  It could be a simple read operations (e.g. kubectl get pods).

image

Sub steps:

  1. End-user launch the kubectl CLI command “kubectl create…
  2. kubectl CLI uses cached credentials to do a Kubernetes API call
  3. AKS service relays the call to the underlying cluster

We see kubectl commands use credentials gathered in previous step.

Step 5 – Creating Azure Disks

Let’s assume we do another deployment with kubectl create.  This time, one of the underlying pod will provision an Azure disk.

This shows how Kubernetes integrates back to Azure.

image

Sub steps:

  1. End-user launch the kubectl CLI command “kubectl create…
  2. kubectl CLI uses cached credentials to do a Kubernetes API call
  3. AKS service relays the call to the underlying cluster
  4. Kubernetes calls Azure Resource Manager to provision an Azure Disk.  It uses Service Principal credentials.

Alternatives

We unveiled the magic behind the scenes.

That path is the quickest to get value out of Azure AKS but in production scenarios, we might want more control.

For instance, we might want to create the Azure AD Service Principal beforehand.  The procedure to do that is well explained in online documentation.

There might be different reasons to do that:

  • Nomenclature / standard:  we might want to control the name of the service principal
  • Rights:  maybe a user doesn’t have rights to create service principals
  • Reuse service principal

Summary

We did drill down in the different identities involved in different AKS scenarios.

That knowledge isn’t required to start using AKS.  It does become useful when considering operationalizing the service though.

Advertisements

One thought on “Understanding identities in Azure AKS / Kubernetes

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s