Building Azure Scale Set Images with Packer and DSC

2017-09-05

ARM, Deployment, Devops, images, scalesets, Virtual Machines

Virtual Machines in a Scale Set provide a great way to spin up many identical machines in parralell. Thes VMs are all based on a common VM image and so unless all you need on your VMs is Windows, you are going to need to load applicaitons and data into these VMs. Getting your applications into that image can be done one of two ways:

Use a Gallery image for your scale set and apply your applications on each VM when it is deployed using something like the PowerShell DSC extension, or Chef/Puppet etc.
Create a custom image that already includes your application and build your scale set from this

Option 1. is somewhat simpler, in that it doesn’t involve managing images. Future changes, upgrades etc. only require you to amend your deployment scripts, not bake a new image. However, this comes with a downside, you are taking a hit on deployment time, as each VM needs to install the application when it is created. If your scale set is fairly static that’s not a big deal, but if you are changing the size regularly this delay can be a problem. This was the case with a scale set I was using, which was being scaled on demand, with a fairly complex application and VMs needed to be available quickly.

With run time deployment not being feasible, we need to create a custom image. I wanted to create this image in an automated way and leverage the PowerShell DSC files we had already created to install the applications into our image. This is where Packer comes in.

Packer

Packer is a tool for automating the building of VM images, this includes building the machine to be imaged, apply configurations and applications, creating the required image and cleaning up. Packer is not Azure specific, but it does have capabilities to be able to use Azure virtual machines as the base for the image and to create images suitable for Azure. Packer also allows us to utilize DSC as the method of configuring our VM before creating the image. In this way, if we need to update our image all we need to do is update the DSC files (and any resources they use) and then run our image build script with Packer.

Installing Packer

The installer for Packer can be found here or can be installed via tools like Chocolaty. It’s a fairly simple install that just requires extracting the files and placing them where you want, then adding that location to the Windows Path. Full instructions can be found here.

I’m not going to go deep into the workings of Packer here, if you want to investigate that take a look at the Packer Getting Started, but Packer configurations are created using a JSON file which makes use of 3 types of resources:

Builders – These are the components that deal with the creation of the machine from which to take the image. There are builders for Azure, AWS, VMWare, VirtualBox etc.
Provisioners – These are used to configure the machine prior to creating the image, this includes installing applications, Windows Features etc. There are various provisioners using different technologies like DSC, Chef, Puppet or just plain old PowerShell or CMD.
Post-Processors – These run after the Builder and Provisioner to conduct various operations. These are optional and we won’t make use of them here.

The default installation of Packer includes the Azure builder, so nothing to install there, howeve, we do need to install the DSC provisioner. This is a community project that can be downloaded here. Once downloaded this needs to be extracted to a location where Packer will find it, this can either be alongside your Packer JSON file, or in the %APPDATA%/packer.d/plugins folder.

There is a bug in the current release (at the time of writing) of the DSC Provisioner, where it expects there to be a tmp folder in root of the drive where your packer file is, so make sure you create this.

DSC File

We are going to use PowerShell DSC as the method of configuring our VM once it is deployed, before the image is created, so we need a DSC configuration file to use. There is nothing Packer specific about this file, it’s plain old DSC. So if you already have DSC you are using to configure your VM’s then you can re-use this here and you can continue to use DSC for configuration of both VMs and images. All you need to do is place this somewhere accessible from your packer file. If you want to learn more about DSC you can start here.

You can of course use any of the other provisioning methods as an alternative to DSC.

Azure Resources

Packer needs to be able to authenticate to Azure and have the rights to create VMs and Images in Azure. It does this using a service principal, effectively a service account in Azure. This Service Principal needs the rights to create resource groups, VMs and Images, the easiest way to achieve this is to create a SP and grant it contributor rights on your subscription. Here is some PowerShell that will create this:

$sp = New-AzureRmADServicePrincipal -DisplayName “svcPackerBuilder” -Password “password” Sleep 20 New-AzureRmRoleAssignment -RoleDefinitionName Contributor -ServicePrincipalName $sp.ApplicationId write-output “Application ID: $($sp.ApplicationId) write-output “ObjectID: $($sp.ObjectID)

Make a note of the App ID, Object ID and Password as you will need this later.

Packer also needs a storage account to place your images in, so go ahead and create a resource group, storage account and container to keep these.

Creating Packer File

The Packer file is what defines your Builders, Provisioners and Post-Processors. It’s a JSON file, so we will go ahead and create this file. The basic syntax of this file looks like this

{
    "builders": [
        {}
    ],
    "provisioners": [
        {}
    ]
}

Builder

The first thing we need is to add a builder that will create the Azure VM. This builder has a number of parameters which are documented here. These parameters tell the build all the information it needs to know to be able to connect to your Azure subscription. Below is a sample Builder

"builders": [
    {
        "type": "azure-arm",
        "client_id": "<this is the ApplicationID from your service principle>",
        "client_secret": "<this is the password from your service principle>",
        "object_id": "<object ID for the service principle>",
        "tenant_id": "<ID of the Azure AD tenant your SP is in>",
        "subscription_id": "<subscription ID you want the VM and images created in>",
        "resource_group_name": "<name of the resource group you created to store images>",
        "storage_account": "<name of the storage account you created to store images>",
        "capture_container_name": "<name of the storage container, in your storage account, you created to store images>",
        "capture_name_prefix": "<prefix to use for your images, up to you>",
        "os_type": "Windows",
        "image_publisher": "MicrosoftWindowsServer",
        "image_offer": "WindowsServer",
        "image_sku": "2016-Datacenter",
        "communicator": "winrm",
        "winrm_use_ssl": "true",
        "winrm_insecure": "true",
        "winrm_timeout": "3m",
        "winrm_username": "packer",
        "location": "West Europe",
        "vm_size": "Standard_DS2"
    }
],

Replace the appropriate sections with your information, and change the OS as required. Once you have all the information added you have a builder setup to create an Azure VM. You could even run this now and it would create a VM with the gallery image, then taken an image of this, but it would be a bit pointless.

Provisioners

So now we have a Builder in place to create the VM, we need to configure it ready for imaging. For this we are going to use 3 provisioners

A PowerShell provisioner which will enable WINRM, this is not on by default in the gallery image but is required for DSC to run
The DSC provisioner to run our DSC file and configure the VM
A second PowerShell provisioner to clean the VM and run SysPrep so it is ready for imaging (this is a requirement for creating Azure Windows Images).

Our provisioner section is going to look like this:

"provisioners": [
    {
        "type": "powershell",
        "inline": [
            "WINRM QuickConfig -q"
        ]
    },
    {
        "type": "dsc",
        "manifest_file": "DSCConfiguration.ps1",
        "configuration_name": "WebServer",
        "install_package_management": true,
        "install_modules": {
            "xPSDesiredStateConfiguration": "6.4.0.0"
        },
        "configuration_params": {
            "-PackageLocation": "https://urlformyfiles.blob.core.windows.net/files",
            "-AppliationName": "App1"
        }
    },
    {
        "type": "powershell",
        "inline": [
            "if( Test-Path $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml ){ rm $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml -Force}",
            "& $Env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /shutdown /quiet"
        ]
    }
]

The two PowerShell provisioners are pretty straight foward, they just contain an “inline” section which contains the lines of PowerShell I want to run.

The DSC provisioner takes a few more parameters:

The Manifest_File – the path to my DSC file, in this example it is sat in the same folder as the packer file
Configuration_name – the name of the configuration inside my DSC file
Install_package_mangement – installs OneGet on the server making it much easier to download any DSC modules you need for your configuration
Install_Modules – an array of DSC modules I need in my DSC file, these will get installed for me
configuration_params – these are any parameters you need to send to your DSC file

There are some other parameters that may be useful with the DSC provisioner, you can find them listed here .

Full Packer File

Now we have created a builder and provisioners, we have a working Packer file. We could add in some post-processors, but we don’t need them here. This is what the final file looks like.

{
    "builders": [
        {
            "type": "azure-arm",
            "client_id": "<this is the ApplicationID from your service principle>",
            "client_secret": "<this is the password from your service principle>",
            "object_id": "<object ID for the service principle>",
            "tenant_id": "<ID of the Azure AD tenant your SP is in>",
            "subscription_id": "<subscription ID you want the VM and images created in>",
            "resource_group_name": "<name of the resource group you created to store images>",
            "storage_account": "<name of the storage account you created to store images>",
            "capture_container_name": "<name of the storage container, in your storage account, you created to store images>",
            "capture_name_prefix": "<prefix to use for your images, up to you>",
            "os_type": "Windows",
            "image_publisher": "MicrosoftWindowsServer",
            "image_offer": "WindowsServer",
            "image_sku": "2016-Datacenter",
            "communicator": "winrm",
            "winrm_use_ssl": "true",
            "winrm_insecure": "true",
            "winrm_timeout": "3m",
            "winrm_username": "packer",
            "location": "West Europe",
            "vm_size": "Standard_DS2"
        }
    ],
    "provisioners": [
        {
            "type": "powershell",
            "inline": [
                "WINRM QuickConfig -q"
            ]
        },
        {
            "type": "dsc",
            "manifest_file": "DSCConfiguration.ps1",
            "configuration_name": "WebServer",
            "install_package_management": true,
            "install_modules": {
                "xPSDesiredStateConfiguration": "6.4.0.0"
            },
            "configuration_params": {
                "-PackageLocation": "https://urlformyfiles.blob.core.windows.net/files",
                "-AppliationName": "App1"
            }
        },
        {
            "type": "powershell",
            "inline": [
                "if( Test-Path $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml ){ rm $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml -Force}",
                "& $Env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /shutdown /quiet"
            ]
        }
    ]
}

Running Packer

Now we have our Packer file we are ready to run this and create an image. Running this is a simple command from PowerShell or the command line.

Packer Build buildFile.json

This will start the build process and you should see logs showing you the process of your build. The build process will create a resource group, and the required resources inside this (VM, storage, Keyvault) once it is done it will clean all of these up.

Once it completes successfully you will see output that give you the URLs for you completed images, and you should see these in your designated storage account. You can then use images to create your scale sets, or just use as a standard VM.

If you need to update your images all you need to do is update your DSC file as required, then re-run your build. Packer will create a new image, with a different name.

Debugging

Creating images doesn’t always work as you expect and debugging can be quite hard when the VM you created gets deleted as soon as your build finishes. There are a couple of command line switches that may help.

Packer Build buildFile.json -on-error=ask

What this will do is pause when an error occurs, rather than immediately cleaning up the VM. You can then connect to the VM if require and debug any issues with DSC etc. You will find the IP and Username and Password for the VM in the build log.

Packer Build buildFile.json -debug

This will output additional debugging information, it will also pause after each section to allow you to review that information.

These commands can be used together if required.

Packer Build buildFile.json -on-error=ask -debug

Next Steps

Now that we have a working Packer build my next step will be to hook this into my Continuous Integration system (in this case Visual Studio Team Services) so that any time a new version of my application is created an image is built. We’ll cover this in the next part of this article.