Completing your automated VM deployments with the DSC VM extension

ARM, Configuration Management, DSC, PowerShell, Powershel

Azure Resource Manager (ARM) templates are a great resource for deploying Azure infrastructure, including virtual machines, in a declarative manner. However, using an ARM template to deploy a VM will only get you as far as having a VM deployed and the operating system installed and running. The next step is to get any applications and supporting software installed on those machines. Of course you can RDP to machine and do this manually, but this breaks down quickly if you have lots of machines or need to regularly deploy VM’s. This is where PowerShell Desired State Configuration (DSC) can come in. DSC is another declarative configuration method, but this time it is for configuring Windows and the applications that run on it, by combining ARM templates and DSC we are able to fully automate the provisioning of a VM from creating the Azure object all teh way up to installing your applications. In this post we will look at how you can use the ARM DSC Extension to trigger your DSC configurations on a VM as part of your automated deployment .

[mks_icon icon=”fa-question-circle” color=”#1e73be” type=”fa”] I’m not going to go into great detail on DSC in this post, if you are interested in learning about DSC then you can start here. It should also be noted that DSC is not the only method of deploying VM configuration, other tools like Chef, Puppet, Salt etc. can all be used in a similar way and many also have ARM extensions that can be deployed in a similar manner. 

VM Extensions

Before we delve into the DSC extension, it is worth mentioning VM extensions in general. VM extensions are a way for application vendors (including Microsoft) to provide a pre-packaged way to deploy their software to Azure VM’s. Extension are configured to deploy to a VM, usually with some parameters set by the user and when deployed install an application in the desired configuration. Extensions are available for things like antivirus, monitoring and in this case, configuration management. Extensions can be deployed through the portal, where you can see a full catalog of available extensions, or through PowerShell, CLI and ARM templates. Some extensions do require either bringing your own licence or have an additional cost (things like third party AV).

Publishing DSC Files

The first thing we need to do to work with the DSC extension is to put our DSC files somewhere the extension can reach them. At the time of deployment your ARM template will run the VM deployment and then the DSC extensions, the first thing it will do is reach out to a location you specify as a parameter and download your DSC files, so we need to put them somewhere it can access them. Unless you have VPN or ExpressRoute connectivity at deployment time this will usually need to be a public URL. The easiest way I have found to deploy my DSC resources is to an Azure storage account, that way I can have a public URL but I can also protect it with a SaS key if the data is sensitive.

The extension expects to find your DSC files in a zip file at the URL specified. One of the really good things about the extension is that it allows you to bundle your DSC file with any DSC resources it requires. That way you don’t have to ensure any require resources are already on your VM, you just bundle them in your zip and the extension will take care of putting them in the right place. The structure of your zip file should look something like this:

In this Zip file I have two DSC resources I need in my configuration, the extension will take care of placing these in C:\Program Files\WindowsPowerShell\Modules so I can use them. DSCConfiguration.ps1 is my configuration file which we will call to configure the VM.

Configuration File

The content of the DSC configuration file does not need to be updated especially for use with the DSC extension, your existing configuration files should work fine. The only change you may need to make is if you wish to pass parameters from your ARM template to the DSC file, in which case your DSC file needs to be configured to accept these parameters. Again these are nothing special, just standard PowerShell parameters.

Configuring the Extension

Now we have our zip file uploaded somewhere accessible we can amend our ARM template to install the extension on the VM. The full specification for the extension ARM template is available here. The resource can sit either underneath the VM you wish to deploy it to, or as a top level resource with a dependency on the  VM it is to be deployed to. The example below is provided as a top level resource, with a dependency.

{ “type”: “Microsoft.Compute/virtualMachines/extensions”, “name”: “[concat(parameters(‘vmName’),'/’, variables(‘vmExtensionName’))]”, “apiVersion”: “2015-05-01-preview”, “location”: “[resourceGroup().location]”, “dependsOn”: [ “[concat(‘Microsoft.Compute/virtualMachines/’, parameters(‘vmName’))]” ], “properties”: { “publisher”: “Microsoft.Powershell”, “type”: “DSC”, “typeHandlerVersion”: “2.19”, “autoUpgradeMinorVersion”: true, “settings”: { “configuration”: { “url”: “[parameters(‘modulesUrl’)]”, “script”: “dscConfiguration.ps1”, “function”: “Configuration” }, “Properties”: { “SQLVersion”: “[parameters(‘sqlVersion’)]”, “SQLAdminUserName”: “[parameters(‘SQLAdminUserName’)]” } }, “protectedSettings”: { “configurationArguments”: { “AdminPassword”: “[parameters(‘adminPassword’)]” }, “configurationUrlSasToken”: “[parameters(‘sasToken’)]” } } }

There are a number of different options with this extension, the configuration above is one that should work for most, for full details on all the options see this page. If we take a look at the options in the properties section:

  • Published and Type – This defines which extension we want to install
  • typeHandlerVersion – This is the version of the extension you wish to install, you can find the latest version via the portal or Powershell. You can also use wildcards, such as “2.*”. If you use the next setting to autoupgrade you should always get the latest version regardless of which you choose here.
  • autoUpgradeMinorVersion – This determines whether the extension should automatically get upgrade on release of a new version. I would recommend enabling this.
  • configuration – This section contains all the details on how to download and run your DSC configuration - url – This is the URL where we uploaded the zip file with your configuration earlier. Note if you used a SaS token with this URL, do not include it here, we will add that later
  • script – the name of the configuration file to be run
  • function – the name of the function inside your configuration file to be run
  • Properties – Here we can list any arguments we want to pass through to the extension. These must exist as arguments in your DSC configuration. All proprties here are passed unencrypted to the VM, which will then be stored in your configuration file unencrypted on that VM, so do not pass any sensitive data in this section
  • Protected Settings – Here is where sensitive data can be passed, there are a couple of sections - configurationArguments – This is any data you wish to be passed as arguments to your DSC configuration, but you wish to be encrypted. The DSC file arguments should expect to receive a secure string
  • configurationURLSaSToken – If your uploaded DSC zip file needs a SaS token to access, this is where it can be passed

These are the basic values that will exist in nearly all DSC extensions and should be enough to get started. There are a couple of more advanced features that could be of use to some:

ForceUpdateTag

With the configuration above, the first time you deploy your ARM template it will deploy and run your DSC configuration as expected. If you run your deployment again however it will not re-run your DSC extension, it will just report the previous status. This includes if you have actually changed your DSC configuraiton and re-uploaded it, so you do want it to be run again. This is even the case when the installed extension resulted in an error. This is because as far as the ARM deployment is concerned, things have not changed, none of the values in your template ARM extension are different. Yes your DSC zip file has different content, but at the time of deployment it does not know this. To get around this and ensure that you are able to re-run your DSC configuration each time you can use the ForceUpdateTag value, as below:

“publisher”: “Microsoft.Powershell”, “type”: “DSC”, “typeHandlerVersion”: “2.20”, “autoUpgradeMinorVersion”: true, “forceUpdateTag”: “[parameters(‘DscExtensionUpdateTagVersion’)]”,

The value of the forceUpdateTag section can be anything you want, it just needs to differ from the previous version. In my settings, I am using Octopus Deploy to push my ARM templates, so I pass the release number here, which will differ each time.

ConfigurationData

Instead of passing any variable data through to the DSC configuration through arguments, you can instead pass a configuration data file. This is a PSD1 file that contains a hashtable of your data to be passed. You can see more details on the layout of this here. To pass configuration data you again need to upload this to an accessible location and supply a URL, this should be a URL directly to the PSD1 file. This is configured in the settings section:

“settings”: { “configuration”: { “url”: “[parameters(‘modulesUrl’)]”, “script”: “dscConfiguration.ps1”, “function”: “Configuration” }, “configurationData”: { “url”: “[parameters(‘configurationUrl’)]” }, “Properties”: { “SQLVersion”: “[parameters(‘sqlVersion’)]”, “SQLAdminUserName”: “[parameters(‘SQLAdminUserName’)]” } },

If you wish to protect this file with a SaS toke, you can again specify this in the protected settings section under the configurationDataUrlSasToken section:

“protectedSettings”: { “configurationArguments”: { “AdminPassword”: “[parameters(‘adminPassword’)]” }, “configurationUrlSasToken”: “[parameters(‘sasToken’)]”, “configurationDataUrlSasToken”:"[parameters(‘dataSasToken’)]” }

Deployment

Now you’ve setup your extensions in your ARM template your ready to deploy. At this point a deployment is no different from your standard ARM template deployment. Depending on how complex your DSC configurations are your deployment time may increase.

Debugging

Once you test your deployment there will invariably be issues and errors. If your DSC configuration fails, this will be reported in the Azure portal and will stop your ARM deployment from continuing. The portal will give you a brief description of the error but often this is just something along the lines of “configuration failed to apply” rather than a useful error. To investigate the issue you will need to login to the VM and check a few locations:

  1. C:\WindowsAzure\Logs\Plugins\Microsoft.Powershell.DSC<DSC Extension Version Number> – This is where your verbose DSC logs will be, you can look at these and find where the error occured
  2. C:\Packages\Plugins\Microsoft.Powershell.DSC<DSC Extension Version Number> – This is where the extensions data is stored and there are a few places in here to look for information 1. Status folder – In here is the latest status of the DSC extension, it’s useful to see what the date and time on this is to see if your error is actually from your recent configurationDSC or if you are just getting a cached response
  3. DSCWork – This is where your DSC file is actually extracted to and your MOF files are generated, you can look in here to see that you are actually running the right file or even manually alter and compile your DSC file if you wan to test it on that machine

Tips

  • With all extensions, including the DSC one, I have found it less error prone if you install one extension at a time. So if you are installing multiple extensions on a VM, use dependencies in your template to ensure only one runs at a time
  • I have seen times where even though I am using the ForceUpdateTag option, I still get a cached response rather than a re-run of my script. In this instance you can force it to re-deploy by deleting the DSC extension from the VM and then re-running your deployment. Removing the extension does not remove any of the existing configuration changes you have deployed
  • If your passing sensitive data as arguments, make sure you use the protected settings section to encrypt. You can check that is working by looking at your compiled MOF files in the Packages folder and checking that you can’t find the sensitive data in plain text
  • If your following Devops processes and checking your DSC files into source control then using a build/release tool (VSTS, Jenkins, Octopus etc.) to bundle up your DSC files and push them to your blob store whenever you make changes is a great way to avoid having to do this manually (and forgetting)