Flex Volume in AKS


I wanted to start looking at a few modules helping integrate AKS with the rest of Azure.

In a past article, we looked at Pod Identity.

This time around, we will look at Flex Volume.

Flex Volume is an integration between AKS and Key Vault. It allows keys, secrets & certificates (the three main objects of Key Vault) to be exposed as files in a mounted volume in a pod.

As usual, code used here is in GitHub.

Some details and constraints

We need an Azure Identity to access Azure Key Vault. Flex Volume allows for Service Principal and AAD Pod Identity. In this article we’ll use that latter.

Technically, a version of each object (key, secret & certificate) is copied to the mounted volume at pod creation. For this reason, if an object value changes, i.e. a new version is created, those changes aren’t reflected in the mounted volume. Pods need to be restarted for that.

The mounted volume is of type Secret. Kubernetes secret volumes are based on tmpfs. This a RAM based and hence can’t be access by other pods. The volume is deleted when the pod is deleted.

Demo

Here, we’ll simply go through the online documentation but choosing only one path:

  • AKS
  • Azure AD Pod Identity

Install

Let’s install Flex Volume. This is done by applying a configuration file. We can see the file contains a namespace, kv, and a simple daemon set:

$ kubectl create -f https://raw.githubusercontent.com/Azure/kubernetes-keyvault-flexvol/master/deployment/kv-flexvol-installer.yaml
$ kubectl get ds -n kv

NAMESPACE     NAME                  DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                 AGE
kv            keyvault-flexvolume   4         4         4       4            4           beta.kubernetes.io/os=linux   71s

If we dig in the daemon set configuration, we see it simply installs a driver on each node at /etc/kubernetes/volumeplugins.

Managed Identity access

For the Azure AD Pod Identity configuration, we refer to a previous article.

Basically, we have an Azure Managed Identity integrated in AKS. We need to give that identity two separate access:

  1. Reader access on the Key Vault resource
  2. Get permission on each object (key and/or secret and/or certificate)

The first access is Azure Control plane. The second is part of the access policies of Azure Key Vault itself.

Deploying the pod identity

We then need to deploy an AzureIdentity and AzureIdentityBinding as usual with AAD Pod Identity.

Deploying a pod

We then deploy a pod with the usual kubectl apply -f pod.yaml.

Here is the pod.yaml file:

apiVersion: v1
kind: Pod
metadata:
  name: access-kv
  labels:
    aadpodidbinding:  little-pod-binding
spec:
  containers:
  - name: main-container
    image: vplauzon/get-started:part2-no-redis
    volumeMounts:
    - name: secrets
      mountPath: /kvmnt
      readOnly: true
  volumes:
  - name: secrets
    flexVolume:
      driver: "azure/kv"
      options:
        usepodidentity: "true"
        keyvaultname: # Azure Resource name for the Azure Key Vault
        keyvaultobjectnames: myname;myage
        keyvaultobjecttypes: secret;secret # OPTIONS: secret, key, cert
        #keyvaultobjectversions: #  If not provided, take latest
        resourcegroup: # Resource Group where the Key Vault is
        subscriptionid: # ID of the subscription where the Key Vault is
        tenantid: # AAD Tenant ID of the subscription

First, we notice the aadpodidbinding label that must match the one defined in the AzureIdentityBinding we defined in previous step.

Second, we see the volume is mounted at /kvmnt within the pod.

Third, we see the volume definition includes the ‘coordinates’ of the Key Vault.

Finally, we see that we fetch two secrets: myname and myage.

Test

We can then test the secrets materialized inside the pod. For instance, for myname secret:

$ kubectl exec -it access-kv cat /kvmnt/myname

Summary

Flex Volume integrates Azure Key Vault inside AKS.

It is quite easy to setup and use.

A difficulty we encountered was forgetting some permission for the managed identity. Make sure you assigned both Azure RBAC and Key Vault access policies.

Advertisements

7 thoughts on “Flex Volume in AKS

  1. Do you know, if it’s possible to store files (entire files like appsettings.config for example) in Azure Key Vault, and mount them in the pods?

      1. Yeah! configmap is the first way we’ve tried. Some cases, application config contains lots of “secrets” within, so.. we’re evaluating to include it as a whole secret to avoid lots of secrets replacements.
        The size limit for a secret is 25Kb, so.. in most cases it should fit.
        Not sure if it’s the best way to proceed, but it’s a way 🙂

          1. Another way is to insert your secret in your Continuous Delivery (CD) process. This way you do not put secrets in source control, you put them in KeyVault and you interrogate Key Vault during the CD process. Typically app settings files are XML, JSON or YAML which can all easily be manipulated in CD process to insert secrets in them.

            1. Yeah! that’s the way we’ve been done until i realize that i can insert whole file as a secret..
              Why? Because, there is a .json with next to twenty secrets to replace.. (lazy mode on)
              i think it should work, but, i haven’t tried yet…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s