Accessing Azure Key Vault from within Azure API Management


Today we look at a common although slightly advanced scenario with API Management: accessing Azure Key Vault from Azure API Management.

In an Enterprise, API Management service are often shared between teams. This means a lot of people might open it in the Portal and look at it. It also means that putting secrets in the properties / named values isn’t a great idea.

Therefore, it is best practice to put secrets in an Azure Key Vault.

Azure API Management can then use its Managed Service Identity to access the secrets from Azure Key Vault.

This is what we’re going to look at concretely here.

This article is heavily inspired by a code snippet from Azure API Management. In general, all their code snippets are worth looking at.

We’re going to add a little twist with caching. Looking for a static secret every time an API is requested would add latency to the requests and incur costs. So, we’re going to show how to cache the secret. You can have a refresh on custom caching in Azure API Management here.

As usual, code is in GitHub.

Deploying demo

First, let’s deploy the demo solution:

Deploy button

There is only one parameter: the secret value we want to store. It has a default value.

We need to make sure to deploy to a region where both Azure API Management and Azure Key Vault are available.

This ARM template deploys an API Management service and a Key Vault. The API Management service is Developer sku and hence incur little cost.

The template typically takes over 30 minutes to deploy…

Key points

Let’s look at the key points of the solution through the Azure Portal.

The resource group should look like this:

Resource Group

Managed Identity

Let’s first open the API Management service and look at the Managed Identities pane (under Settings).

Managed Identity

The first thing to notice is the status is On. This means the service has a Managed Service Identity. We are even given its Object ID within the Azure AD tenant.

Using a managed identity means we do not have to separately create a Service Principal and store its secret somewhere. API Management Service has its own identity and as we’ll see later on, we do not need to see its credentials to acquire tokens with it.

Key Vault Access Policies

Let’s go to the Access Policies pane of Azure Key Vault (under Settings section):

Key Vault Access Policies

We can see a policy attributed to the actual API Management Service identity. That policy grants get actions on secrets.

We need this so the API Management can read the secret.

We do not have an access policy of our own so we can’t look at the secret. If we create one, we’ll see one secret with the value we passed in parameter.

Named Values

Let’s go back to the API Management Service and let’s look at the Named values pane (under API Management section):

Named Values

We see two values. Those are not secrets.

It is good practice in automation to store variables in named values and reference them in the policies. This way it is easier to vary the values of those variables in different deployments (e.g. different environments).

get-secret

Now, let’s look at the APIs:

apis

Beside the default Echo API, we have a secret-manager API with two operations. Let’s look at the get-secret operation. Here are its policies:

<!-- This operation fetches a secret from Key Vault and returns it as payload -->
<policies>
    <inbound>
        <base />
        <!-- Retrieve secret from Key Vault -->
        <send-request
            mode="new"
            response-variable-name="vault-secret"
            timeout="20"
            ignore-error="false">
            <set-url>https://{{vault-name}}.vault.azure.net/secrets/{{secret-name}}/?api-version=7.0</set-url>
            <set-method>GET</set-method>
            <authentication-managed-identity
                resource="https://vault.azure.net" />
        </send-request>
    </inbound>
    <backend>
        <!-- Return secret (no back-end service call) -->
        <return-response
            response-variable-name="existing context variable">
            <set-status code="200" />
            <set-body>@(((IResponse)context.Variables["vault-secret"]).Body.As<string>())</set-body>
        </return-response>
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

This API doesn’t call a back-end API (as a typical API). Nor is it a Mock API. Instead, it calls an external service (Key Vault) and returns its value.

The Azure Key Vault call is done with the send-request policy. We notice the configuration:

<authentication-managed-identity resource="https://vault.azure.net" />

This tells the policy to used API Management MSI to acquire a token on the resource / audience https://vault.azure.net.

The url points to Azure Key Vault REST API.

The response-variable-name configuration specifies in which context variable to store the response.

In the backend policies we found a return-response policy:

<return-response>
    <set-status code="200" />
    <set-body>@(((IResponse)context.Variables["vault-secret"]).Body.As<string>())</set-body>
</return-response>

It uses the value stored in a context variable.

We can test that API. We should get the following response:

{
  "value": "The secret is 42",
  "contentType": "string",
  "id": "https://kv-mtjlwqhrbyyio-demo.vault.azure.net/secrets/my-secret/bcd494e3c9ab4eff9a265e03e79e1a97",
  "attributes": {
    "enabled": true,
    "created": 1573774008,
    "updated": 1573774008,
    "recoveryLevel": "Purgeable"
  },
  "tags": {}
}

get-cached-secret

Typically, we don’t want to return a secret but use it within a policy. For instance, we might want to retrieve a username / password to authenticate to a back-end API.

For that reason, we don’t want to cache the entire response but the secret itself. API Management allows both response caching and variable caching.

This is what we have done in get-cached-secret

<!-- This operation caches the secret -->
<policies>
    <inbound>
        <base />
        <!--Look for secret in the cache -->
        <cache-lookup-value key="cached-secret" variable-name="cached-secret" />
        <!-- If API Management doesn’t find it in the cache, fetch it from Key Vault -->
        <choose>
            <when condition="@(!context.Variables.ContainsKey("cached-secret"))">
                <!-- Retrieve secret from Key Vault -->
                <send-request mode="new" response-variable-name="cached-secret" timeout="20" ignore-error="false">
                    <set-url>https://{{vault-name}}.vault.azure.net/secrets/{{secret-name}}/?api-version=7.0</set-url>
                    <set-method>GET</set-method>
                    <authentication-managed-identity resource="https://vault.azure.net" />
                </send-request>
                <!-- Store response body in context variable as a string -->
                <set-variable name="cached-secret" value="@(((IResponse)context.Variables["cached-secret"]).Body.As<string>())" />
                <!-- Store result in cache -->
                <cache-store-value key="cached-secret" value="@((string)context.Variables["cached-secret"])" duration="5" />
            </when>
        </choose>
    </inbound>
    <backend>
        <!-- Return secret (no back-end service call) -->
        <return-response response-variable-name="existing context variable">
            <set-status code="200" />
            <set-body>@((string)context.Variables["cached-secret"])</set-body>
        </return-response>
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

The flow of the logic is:

  • Lookup for a cached item (cache-lookup-value policy)
  • Test if the cached item exists (choose policy)
    • If it does, keep going
    • If it doesn’t
      • Retrieve it from Key Vault (send-request policy, similar to previous operation)
      • Set a variable with its value (set-variable policy)
      • Cache the value (cache-store-value policy)
  • Return the cached value (return-response policy)

Although it is a little verbose, the actual logic is quite simple.

We can test the API. The value is cached for 5 seconds. We can test that by looking at the tracing and see something like this when the cache is empty (i.e. after 5 seconds of inactivity):

choose (0.017 ms)
{
    "message": "Expression was successfully evaluated.",
    "expression": "!context.Variables.ContainsKey(\"cached-secret\")",
    "value": true
}

And something like this when the cache contains the value:

choose (0.013 ms)
{
    "message": "Expression was successfully evaluated.",
    "expression": "!context.Variables.ContainsKey(\"cached-secret\")",
    "value": false
}

Summary

We’ve seen how we can easily access a secret from Azure Key Vault within a policy in Azure API Management.

A realistic scenario would be using the secret on a back-end API.

In our case, we simply saw how to return the secret or cache it.

This pattern can help us make our Azure API Management solution more secure by hiding secrets from operators.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s