Secure Credential Access with Azure Batch and KeyVault

Azure, Batch, KeyVault, PowerShell

Following on from my post on joining Azure batch pools to a vNet, this leads on to a requirement to access resources on the vNet and this means credentials are needed. Rather than hard-coding these credentials in scripts, we want to obtain these from a secure storage location on demand and this is where Azure KeyVault comes in, providing a secure, encrypted storage location for our credentials.
Obviously there is no point putting your admin credentials in KeyVault, then hard-coding credentials to access KeyVault in your script, so the solution is to use a certificate to give your batch VM’s access to KeyVault. Fortunately batch already has functionality to associate a certificate with your batch account, so with a few steps we can implement secure key storage for batch.

The steps below detail how to set this up, and assume that you already have some knowledge of Azure Batch and running Batch jobs. If you don’t, take a look at the Batch Documentation first. It aslo assumes you are familiar with KeyVault, and in particular storing secrets in KeyVault.

Obtain a certificate

Obviously to grant access using a certificate, you need to actually get a certificate. The easiest way is to generate a self signed certificate using makecert

makecert -sv batchcertificate.pvk -n "cn=batch.cert.mydomain.org batchcertificate.cer -b 01/17/2017 -e 01/17/2018 -r -pe

This generates a self signed cer file called batchcertificate.cer and batchcertificate.pvk, the CN used doesn’t really matter for what we are doing here, but ideally make it something that tells you what this cert is used for.

We need a PFX file to import into batch, so we can use pvk2pfx to convert the created files to a pfx file.

pvk2pfx -pvk batchcertificate.pvk -spc batchcertificate.cer -pfx batchcertificate.pfx -po

Obviously if you prefer you can purchase a commercial certificate to do this.

Create a Service Principal and associate the certificate

Access to KeyVault it granted to either a user or a service principal, as we are going to access this programmatically, we want to create a service principal. If you’re not up to speed on Azure Service Principals have a look at this article.

Note: the Service Principal must be in the same AAD tenant as the KeyVault.

$now = [System.DateTime]::Parse("2017-01-17") 
#Set this to the expiration date of the certificate 
$expirationDate = [System.DateTime]::Parse("2018-01-17") 
#Point the script at the cer file you created $cerCertificateFilePath = 'c:\temp\batchcertificate.cer' 
$cer = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2 
$cer.Import($cerCertificateFilePath) 
#Load the certificate into memory 
$credValue = [System.Convert]::ToBase64String($cer.GetRawCertData()) 
#Create a new AAD application that uses this certifcate 
$newADApplication = New-AzureRmADApplication -DisplayName "Batch KeyVault Access" -HomePage "https://batch.mydomain.com" -IdentifierUris "https://batch.mydomain.com" -certValue $credValue -StartDate $now -EndDate $expirationDate 
#Create new AAD service principle that uses this application 
$newAzureAdPrincipal = New-AzureRmADServicePrincipal -ApplicationId $newADApplication.ApplicationId

The URL’s for the application don’t really matter as we will only use it for KV access.

Grant Rights to KeyVault

The service principal you just created now needs right to retrieve the secrets from KeyVault, you can do this through the portal or with the PowerShell below.

Set-AzureRmKeyVaultAccessPolicy -VaultName 'BatchVault' -ServicePrincipalName '"https://batch.mydomain.com' -PermissionsToSecrets 'Get'

Assign Certificate to Batch Account

We need to assign the certificate to the batch account, this will in turn allow us to assign it to the pools and then to the VM’s. The easiest way to do this is a one off task through the portal, go to your batch account, then the “certificates” blade and click add. Upload the PFX file we generated earlier and supply the password. Once complete this should show in the list of certificates and you can verify the thumbprint.

You can now create a batch pool, and you should fined that when this is created you can go to the certificate blade in the pool and then assign the certificate you created to the pool. When you do so, make sure you select “LocalMachine” for the store location. This cert will now be loaded on all the batch nodes.

Install Azure PowerShell

If you plan on accessing KeyVault using PowerShell scripts on your nodes then you need the Azure PowerShell library installed. There are a few ways to do this, if you nodes have WMF 5 installed then you can use the install-module command to download it, but in my case we were using Server 2012 R2 nodes that didn’t have WMF 5, so the easiest way is to bundle up the Azure PowerShell msi file with your batch files, and then call the installer as the first part of your batch startup script, similar to this:

 

$psModuleCheck=Get-Module -ListAvailable -Name Azure -Refresh 
if($psModuleCheck.count -eq 0){
    $psInstallerPath = Join-Path $downloadPath "azure-powershell.3.4.0.msi" Start-Process msiexec.exe -ArgumentList /i, $psInstallerPath, /quiet -wait 
}

 

Access KeyVault

We’re now all setup to access KeyVault in our scripts running on the batch VM. To do this all you need is for your script to authenticate against AAD using the certificate. To do this in PowerShell use the following Powershell, specifying the appropriate GUID for Thumbprint, App ID (the ID of your service principal) and Tenant ID (the tenant where your service principal exists).

Add-AzureRmAccount -ServicePrincipal -CertificateThumbprint -ApplicationId

Once authenticated we can then access KeyVault as normal.

$adminPassword=Get-AzureKeyVaultSecret -VaultName BatchVault -Name batchAdminPass

and we can then use these credentials in our script.