Accessing Key Vault Secrets in Pulumi

Pulumi’s Azure Native Provider allows you to create infrastructure as code for Azure using real programming languages such as C#, Go, Python, Typescript and more. One of the benefits of the Azure Native provider is that it is generated directly from the Azure ARM REST APIs, so it always has the latest resources available, and it’s not waiting on manual updates as you see with Terraforms Azure provider. However, one of the downsides is that it is affected by any of the not so optimal design decisions made in the Azure API. One of these is retrieving Key Vault secrets which is a common hurdle I see people new to Pulumi hitting.

You can create secrets without issue; however, when you come to retrieve a secret, you hit a problem, and you can’t get the value of the secret. There is an Azure Native Secret component that can be used to retrieve a secret from the Azure API. When they see this the first time, most people assume you can create one of these with the correct settings and get the value property. However, if you look at the API docs or the Pulumi docs and read the description of the properties.value section, you will see this:

The value of the secret. NOTE: ‘value’ will never be returned from the service, as APIs using this model are is intended for internal use in ARM deployments. Users should use the data-plane REST service for interaction with vault secrets.

So this value property is essentially useless to us. The value of a secret is treated as a data-plane resource, for which there isn’t a way to autogenerate the API, so Pulumi doesn’t support this. Tools like Terraform solve this problem because their resources are handcrafted and have a layer of abstraction on top of them, where they can solve something like this. This is one of the occasions where the benefit of an autogenerated, 100% coverage solution is balanced by the fact that there is no further layer of abstraction to deal with these issues. If it’s not in the Azure API, it’s not in the Pulumi component.

Hopefully, this is something Pulumi will address in the future. However, we have a couple of options to deal with this and get hold of the secrets we need in the meantime.

Option 1 - Use the Azure Classic Provider

Before the Azure Native provider existed, there was the Azure Classic (or just Azure) provider, which was based on top of the Terraform Azure provider. This can do whatever the Terraform provider can do, including additional functionality to pull secrets out of Key Vault.

The classic provider still exists, is still supported and has not been deprecated. However, it is dependent on the Terraform provider, so updates to add new services can be a lot slower. It is perfectly possible to run the Native and Classic providers together in the same project. So one solution is to use the classic provider to retrieve secrets from Key Vault. You can still do everything else in the native provider, but grab the secret with the classic provider, put it in a variable and use it elsewhere.

You’ll need to import the Pulumi.Azure package using your package manager, and you can then use it in your project. Here is a C# example that grabs a secret from Key Vault using the classic provider and then passes it to a resource created by the native provider.

using Pulumi;
using Pulumi.AzureNative.Resources;

class MyStack : Stack
{
    public MyStack()
    {
        // Create an Azure Resource Group
        var resourceGroup = new ResourceGroup("resourceGroup");

        var config = new Pulumi.Config("lambda");

        var secret = Pulumi.Azure.KeyVault.GetSecret.InvokeAsync(new Pulumi.Azure.KeyVault.GetSecretArgs()
        {
            KeyVaultId = config.Require("keyVaultId"),
            Name = "connectionString"
        }).Result;

        var container = new Pulumi.AzureNative.ContainerInstance.ContainerGroup("container", new Pulumi.AzureNative.ContainerInstance.ContainerGroupArgs()
        {
            ContainerGroupName = "containerGroup",
            Containers =
            {
                new Pulumi.AzureNative.ContainerInstance.Inputs.ContainerArgs
                {
                    Command = {},
                    EnvironmentVariables = {
                    new Pulumi.AzureNative.ContainerInstance.Inputs.EnvironmentVariableArgs()
                    {
                        Name = "connectionString",
                        SecureValue = secret.Value

                    }
                    },
                    Image = "nginx",
                    Name = "demo1",
                    Ports =
                    {
                        new Pulumi.AzureNative.ContainerInstance.Inputs.ContainerPortArgs

                        {
                            Port = 80,
                        },
                    },

                }
             }
        });
   
    }  
}

Option 2 - Use the REST API

If you don’t want to use the classic provider, the other option is to call the Azure Data plane Rest API directly. Because we’re using actual code to write our Infrastructure as Code, we can use all the functionalities of that language, not just the IaC parts. I’ve discussed calling the REST API in a Pulumi project before; check out this article for full details on how this works. The critical point is that we can use the authentication we have already done through Pulumi to also talk directly to the Azure API.

We can either use the following code (c# example) to get the authentication token:

var token = await GetClientToken.InvokeAsync();

We can then use this token to call the REST API directly or use the appropriate MS SDK to do some of the work.

Alternatively, if your using the .net SDK you can also use the “DefaultAzureCredential” library which will try all possible options for retrieving an existing credential and do this work for you. Using this and the Azure SDK, retriving a secret can be compress to two lines:

var client = new SecretClient(vaultUri: new Uri(config.Require("keyVaultUrl")), credential: new DefaultAzureCredential());
var secret = client.GetSecret("connectionString").Value;

Here’s the same program we used above, switching out the classic provider for the SDK.

using Azure.Identity;
using Azure.Security.KeyVault.Secrets;
using Pulumi;
using Pulumi.AzureNative.Resources;
using System;

class MyStack : Stack
{
    public MyStack()
    {
        // Create an Azure Resource Group
        var resourceGroup = new ResourceGroup("resourceGroup");

        var config = new Pulumi.Config("lambda");

        var client = new SecretClient(vaultUri: new Uri(config.Require("keyVaultUrl")), credential: new DefaultAzureCredential());
        var secret = client.GetSecret("connectionString").Value;

        var container = new Pulumi.AzureNative.ContainerInstance.ContainerGroup("container", new Pulumi.AzureNative.ContainerInstance.ContainerGroupArgs()
        {
            ContainerGroupName = "containerGroup",
            Containers =
            {
                new Pulumi.AzureNative.ContainerInstance.Inputs.ContainerArgs
                {
                    Command = {},
                    EnvironmentVariables = {
                    new Pulumi.AzureNative.ContainerInstance.Inputs.EnvironmentVariableArgs()
                    {
                        Name = "connectionString",
                        SecureValue = secret.Value

                    }
                    },
                    Image = "nginx",
                    Name = "demo1",
                    Ports =
                    {
                        new Pulumi.AzureNative.ContainerInstance.Inputs.ContainerPortArgs

                        {
                            Port = 80,
                        },
                    },

                }
             }
        });

    }
}