Using the Azure SDK with Pulumi

Azure, Pulumi

I’ve been using Pulumi for a while now and have found the ability to use real programming languages to define my Infrastructure as Code to be a really nice experience. Recently I encountered an issue where following the standard declarative Infrastructure as Code process didn’t work for what I was doing, and I had to look at alternative ways to do this. Because Puluimi code is just code, in my case C#, I could use this to directly call the Azure SDK to help solve my issue.

This is something I would only look to do if I really needed it. The power of Pulumi is in using the declarative approach to creating resources, but sometimes, like in the scenario below, you need a non-declarative approach.

The Problem

The issue I encountered was when looking to create an App Service Certificate. To do this, you need to create a DNS TXT record to validate you own the domain; this needs to be created on the “@” TXT record on the root domain, not a subdomain, even if the certificate is being created for a subdomain. It is perfectly possible to create a TXT record in Azure DNS with Pulumi; however, Pulumi (and Terraform) would assume it owns that DNS record and is the only one to update it. Because this TXT record needs to be used by anyone creating an SSL cert in that domain, someone else will likely need to change this record outside of Pulumi, and that is going to break things.

So, I needed to update the TXT record without Pulumi importing it into its state. If I was doing this in Terraform, I would probably use it to call a PowerShell script using Azure PowerShell, but because Pulumi is actual code, I can utilise any functionality that I can write in C# (or other languages Pulumi support) and so I can use the Azure SDK.

Using the Azure SDK

Before we get started, we need to make sure we are clear on how using non-Pulumi code will work with Pulumi. When you execute your Pulumi stack with “Pulumi Up”, any additional code you have added will get executed; however, to ensure it gets executed correctly and at the right time, we need to bear a couple of things in mind:

  1. Pulumi will call your code every time it runs, so you need to ensure it is idempotent, so each time it is run, it does the same thing and does not break if resources already exist
  2. Use Apply to add dependencies. If you need your non-Pulumi code to run after a specific resource is created, make sure you use Apply to use a value from that resource so that a dependency is created.

The rest of the code examples here will use C#, as this is the language I use, but the process can be used with any other Pulumi language that supports using the Azure SDK.

Nuget Packages

To use the Azure SDK, we need to import the SDK Nuget Packages relevant to what we are doing:

  1. The Azure.Identity package which we will use to authenticate to Azure
  2. The appropriate Nuget package for the part of the Azure SDK you want to use. In my case for DNS, it was Microsoft.Azure.Management.Dns

We will import them and then add the appropriate using definitions to our Pulumi stack.

using Microsoft.Azure.Management.Dns;
using Microsoft.Azure.Management.Dns.Models;
using Azure.Identity;
using Microsoft.Rest;

Inputs from Configuration

I need some input values to run my code, mainly the DNS Zone name and Resource Group. I am storing these in my Pulumi stack configuration file, so pull these in with my code.

var config = new Config("sc");

var DNSZoneName = config.Require("DNSZoneName");
var DNSZoneResourceGroup = config.Require("DNSZoneResourceGroup");

Check for Preview

At the start of our code, we need to check whether we are running Pulumi Preview or Pulumi Up. If we don’t do this, then your code will run at both times, and since you always do a preview before an up, it will get executed twice. Assuming you don’t want that, you can prevent this by checking for the isDryRun property.

if (!Pulumi.Deployment.Instance.IsDryRun)
{
    //Code
}

This means our code only runs on the “up” step.

Dependency Using Apply

I need this code to run after creating the AppServiceCertificateOrder object, a standard Pulumi resource.

     var order = new Pulumi.AzureNextGen.CertificateRegistration.Latest.AppServiceCertificateOrder("cert-order", new Pulumi.AzureNextGen.CertificateRegistration.Latest.AppServiceCertificateOrderArgs
        {
            AutoRenew = true,
            CertificateOrderName = "sc-dev-cert",
            DistinguishedName = "CN=dev.samcogan.com",
            Location = "Global",
            ValidityInYears = 1,
            ProductType = CertificateProductType.StandardDomainValidatedSsl
            
        });

Not only do I need to run after this, but I need an output from this step, the “DomainVerificationToken”, which is the actual value I need to put in the DNS record. I need to wrap the code I am going to use to create the DNS record in an “Apply” statement, which ensures that the previous step has run and I have the value for the token.

if (!Pulumi.Deployment.Instance.IsDryRun)
{
    var record = order.DomainVerificationToken.Apply(dnsVerificationToken =>
    {
        //code
    });
}

I’m also putting the result of my code into a variable which I will use later.

Authentication

To be able to hit the Azure Rest API’s using the SDK, we need to authenticate. We really want to use the same authentication token that the rest of our Pulumi code is using, whether that is CLI Creds, Service Principal or Managed Identity. Fortunately, there is an easy way to do this using the DefaultAzureCredential method. DefaultAzureCredential follows a flow to try and find a credential to use, so you don’t need to provide any if you are already using one (which we are).

Default Credential Flow

Most of the Azure SDK clients need a “ServiceClientCredentials” object to auth, so we can use DefaultAzureCredential to get the token and then create a ServiceClientCredentials object. We also need to set the subscription we want to use in the SDK client.

var deafultClient = new DefaultAzureCredential();
            var token = deafultClient.GetToken(new Azure.Core.TokenRequestContext(new[] { $"https://management.azure.com/.default" }));
            ServiceClientCredentials serviceClientCreds = new TokenCredentials(token.Token);
            var dnsClient = new DnsManagementClient(serviceClientCreds);
            dnsClient.SubscriptionId = "<subscriptionId>";

Create Azure Resources

Now we’re all set up and authenticated, we need to write whatever Azure SDK code we want to create our resources. This is where we need to concentrate on making this idempotent to not break things when it is run multiple times. In my scenario for the DNS TXT records, I follow this process:

  • If there is no “@” TXT record in the domain, create one and add my value
  • If the “@” TXT record does exist, then check the values to see if my required value already exists
  • If my value does exist, then do nothing; if it does not add my required value

By following this process, I can run this multiple times with no issue. Bear in mind I am not a C# developer, so there may be better ways to do this!


RecordSet? recordSet = null;
try
{
    recordSet = dnsClient.RecordSets.Get(DNSZoneResourceGroup, DNSZoneName, "@", RecordType.TXT);
}
catch { }

if (recordSet != null)
{

    if (!(recordSet.TxtRecords.Any(txt => txt.Value.Any(val => val == dnsVerificationToken))))
    {
        recordSet.TxtRecords.Add(new TxtRecord(new List<string>() { dnsVerificationToken }));
        var result = dnsClient.RecordSets.Update(DNSZoneResourceGroup, DNSZoneName, "@", RecordType.TXT, recordSet);
    }
}
else
{
    recordSet = new RecordSet();
    recordSet.TTL = 3600;
    recordSet.TxtRecords = new List<TxtRecord> { new TxtRecord(new List<string>() { dnsVerificationToken }) };
    var result = dnsClient.RecordSets.CreateOrUpdate(DNSZoneResourceGroup, DNSZoneName, "@", RecordType.TXT, recordSet);
}

Add An Output

The code above should now run and create the required DNS record. However, I have seen that there can be some inconsistency on whether this runs if there is nothing else being run in the code, so Pulumi sees no changes to be applied. To ensure this will run every time, you can return a value from your “Apply” code and then pass this as an Output from Pulumi. This ensures it will always run. To keep this simple, I am outputting the number of values in my TXT record.

At the top of my Pulumi code I create an Output I want to populate:

public Output<string> TxtRecordCount { get; set; }

At the end of my Apply section, I output the record set details, which I then assign the count to the Output variable.

var record = order.DomainVerificationToken.Apply(dnsVerificationToken =>
{
    //SDK Code
    return recordSet.TxtRecords;
});

TxtRecordCount = record.Apply(x => x.Count.ToString());

Full Code

That’s all the code we need to use the Azure SDK. To create an SSL cert, I would then have another step to create a certificate verification resource, which is just more Pulumi code. The complete code for our SDK access is below.

using Azure.Identity;
using Microsoft.Azure.Management.Dns;
using Microsoft.Azure.Management.Dns.Models;
using Microsoft.Rest;
using Pulumi;
using Pulumi.AzureNextGen.CertificateRegistration.Latest;
using System.Collections.Generic;
using System.Linq;

class MyStack : Stack
{
    [Output]
    public Output<string> TxtRecordCount { get; set; }
    public MyStack()
    {


        var config = new Config("sc");

        var DNSZoneName = config.Require("DNSZoneName");
        var DNSZoneResourceGroup = config.Require("DNSZoneResourceGroup");

        var order = new Pulumi.AzureNextGen.CertificateRegistration.Latest.AppServiceCertificateOrder("cert-order", new Pulumi.AzureNextGen.CertificateRegistration.Latest.AppServiceCertificateOrderArgs
        {
            AutoRenew = true,
            CertificateOrderName = "sc-dev-cert",
            DistinguishedName = "CN=dev.samcogan.com",
            Location = "Global",
            ValidityInYears = 1,
            ProductType = CertificateProductType.StandardDomainValidatedSsl
            
        });

        if (!Pulumi.Deployment.Instance.IsDryRun)
        {
            var record = order.DomainVerificationToken.Apply(dnsVerificationToken =>
            {
            var deafultClient = new DefaultAzureCredential();
            var token = deafultClient.GetToken(new Azure.Core.TokenRequestContext(new[] { $"https://management.azure.com/.default" }));
            ServiceClientCredentials serviceClientCreds = new TokenCredentials(token.Token);
            var dnsClient = new DnsManagementClient(serviceClientCreds);
            dnsClient.SubscriptionId = "<subscriptionId>";

            RecordSet? recordSet = null;
            try
            {
                recordSet = dnsClient.RecordSets.Get(DNSZoneResourceGroup, DNSZoneName, "@", RecordType.TXT);
            }
            catch { }

            if (recordSet != null)
            {

                if (!(recordSet.TxtRecords.Any(txt => txt.Value.Any(val => val == dnsVerificationToken))))
                {
                    recordSet.TxtRecords.Add(new TxtRecord(new List<string>() { dnsVerificationToken }));
                    var result = dnsClient.RecordSets.Update(DNSZoneResourceGroup, DNSZoneName, "@", RecordType.TXT, recordSet);
                }
            }
            else
            {
                recordSet = new RecordSet();
                    recordSet.TTL = 3600;
                    recordSet.TxtRecords = new List<TxtRecord> { new TxtRecord(new List<string>() { dnsVerificationToken }) };
                    var result = dnsClient.RecordSets.CreateOrUpdate(DNSZoneResourceGroup, DNSZoneName, "@", RecordType.TXT, recordSet);
                }

                return recordSet.TxtRecords;
            });

            TxtRecordCount = record.Apply(x => x.Count.ToString());
        }
    }
}