Azure Security Audits with Pester

Azure, Pester, Testing

Pester is a versatile testing framework built using PowerShell. In this article we are going to look at using Pester to audit our Azure infrastructure for security compliance.

We’ve previously discussed using Pester to test Azure resources in our infrastructure pipeline. In that article we used Pester to run against a deployed Azure environment to check what we thought we had deployed had actually been deployed. In  this article we’re going to take this a bit further and look at using Pester to validate that we have setup our environment in a secure manner and look for areas of concern, essentially a security audit.

Concept

Security is obviously a significant concern for those deploying into the cloud. Azure provides a wide array of tools and techniques to help with this, but these are only useful if they are used. We are going to write a set of tests that can be run against an Azure Resource Group and check that the resources in there are following our security standards.

There are obviously many ways to achieve this, you could  use Azure Security Centre, but this restricts you only to the rulesets they provide, you could use resource policies, but these are still in preview and take a lot of work to setup. What we are producing is a simple set of tests that we can run against any resource group we like, with any number of resources and quickly see results.

If your just looking for the full script to run you can find it here on Github – https://github.com/sam-cogan/AzureAudit

Requirements

So the first thing we need to do is decide what we want to test for. Obviously we can test pretty much anything that is available through Azure PowerShell, but for this we are will focus on some of the most common security configurations. This is a good starter that you can then expand however you see fit.

We will test that:

  • All VM’s are configured with Antivirus/Antimalware protection
  • All VM’s are behind an NSG
  • All VM’s have Bitlocker encryption enabled on all drives
  • No NSG’s have ports open for “All”
  • All storage accounts have storage service encryption enabled
  • All Azure SQL databases are using transparent encryption
  • All Azure SQL databases have threat detection turned on
  • All resource groups have Azure Security Centre enabled.

Tests

Setup

We want to make these tests fairly generic, so that we can run them against any resource group and it will test all the resources in that resource group no matter how many there are. So, the only information we need to feed into our script is the resource group name.

param( [Parameter(Mandatory)] [ValidateNotNullOrEmpty()] [string]$ResourceGroupName )

 

Virtual Machine Tests

We’ll start by implementing the virtual machine tests. We’re going to use the Pester Describe Section to delineate the different type of resource tests, and inside the Describe block we will setup any resources that are going to be used by all tests, so in this section we will get all of the VM’s in the resource group, so we can use these in our tests without calling get-azurermvm every time.

Describe “Virtual Machine Tests” { $VMs = Get-AzureRmVM -ResourceGroupName $ResourceGroupName

Then we’re going to group our tests into context blocks, for different types of VM tests. So the first VM test is going to be for Antivirus, and we are going to check two things here:

  1. That the Microsoft Antimalware extension is installed
  2. That real time protection is enabled

We are just going to use regular Azure RM PowerShell commands to get the information about the availible VM extensions and how they are configured then apply this data against our expected criteria to test. We’re also going to loop through all teh VM’s in the resource group to make sure we test them all.

Context "Antivirus" { 
    foreach ($vm in $vms) { 
        $avExtension = Get-AzureRmVMExtension -ResourceGroupName $ResourceGroupName -VMName $vms[0].name -Name IaaSAntimalware 
        $publicSettings = ConvertFrom-Json $avExtension.PublicSettings 
        
        It "$($vm.name) Should Have Micrsoft Antimalware Extension Installed" { 
            $avExtension | Should Not Be $null 
            $avExtension.ProvisioningState | Should Be "Succeeded" 
            $publicSettings.AntimalwareEnabled | Should Be "True" 
        } 
        
        It "$($vm.name) Should Have Real Time Protection Enabled" { 
            $publicSettings.RealtimeProtectionEnabled | Should Be "true" } 
    } 
}

 

As you can see, in some of the “IT” sections we are actually testing multiple things, and if one of those fails the whole test will fail. We’re checking the extension is installed and enabled, and then in our second test checking the properties to see that real time protection is enabled.

In our next set of tests, we want to make sure that every VM has a Network Security Group applied to it, so that we can be sure no VMs are sat out on the internet with no security filtering of traffic. This test is a little bit more complicated as a VM can get an NSG from 2 places, either applied to it’s network card, or applied at the subnet level, so we need to test both of these and only fail if both are missing.

Context "VM Network Security Groups" { 
    foreach ($vm in $vms) { 
        foreach ($nicID in $vm.NetworkProfile.NetworkInterfaces) { 
            $Nic = Get-AzureRMNetworkInterface -ResourceGroupName $ResourceGroupName | where { $_.Id -eq $nicID.id } 
            $nicNSG = $nic.NetworkSecurityGroup 
            $subnet = $nic.IpConfigurations.subnet 
            $VirtualNetwork = Get-AzureRMVirtualNetwork -ResourceGroupName $ResourceGroupName | Where { $_.Subnets.ID -match $subnet.id } 
            $subnetNSG = $($VirtualNetwork.Subnets | Where { $_.ID -match $subnet.id }).NetworkSecurityGroup 
            
            It "$($vm.name) NIC $($nicID.name) Should Have an NSG Enabled" { 
                ($nicNSG -eq $null) -and ($subnetNSG -eq $null)| Should Not Be $true } 
        } 
    } 
}

 

The thing to remember here is that the actual assertion we are testing here is just comparing the two values either side of the should statement, so we can do what ever PowerShell conversion we need to do on the left to get the aggregated result to then pass to the comparison. So here we are testing both NIC and Subnet level NSGs and combing the result to give an overall true/false to compare.

The final VM based test we are undertaking is to check all VM’s have bitlocker encryption enabled. Fortuantely the encryption status of the VM is exposed through the storage profile so we can get this data easily. We undertake two tests, one for the OS disk and one for the data disks, checking they all show as “Encrypted”


Context "VM Bitlocker Encryption" { 
    foreach ($vm in $vms) { 
        $encryptionStatus = Get-AzureRmVMDiskEncryptionStatus -ResourceGroupName $resourcegroupname -VMName $vm.name 

        It "$($vm.name) Should have an encrypted OS disk" { 
            $encryptionStatus.OsVolumeEncrypted | should be "Encrypted" 
        } 
        
        It "$($vm.name) Should have encrypted Data disks" { 
            $encryptionStatus.DataVolumesEncrypted | should be "Encrypted" 
        } 
    } 
}

 

NSG Tests

We’ve already tested that all VMs have an NSG, but now we want to make sure that none of these NSG’s have rules that allow inbound access to the “All” group, in our environments we want to always scope NSG rules to specific IP ranges. This might not be applicable to everyone, obviously in some environments you do need to open ports to the whole internet, but you can tailor this as required. Again, we are going to use a Describe and Context block to group things, even though we only actually have one test at the moment.

Describe "Network Security Group Tests" { 
    $NSGS = Get-AzureRmNetworkSecurityGroup -ResourceGroupName $ResourceGroupName 
    
    Context "Ports Open to All" { 
        foreach ($NSG in $NSGS) { 
            $openAllCount = 0 
            foreach ($rule in $NSG.SecurityRules) { 
                if ($rule.Direction -eq "Inbound" -and $rule.SourceAddressPrefix -eq "*") { 
                    $openAllCount ++ 
                } 
            } 
            
            It "$($NSG.name) Should Have no inbound rules open to all" { 
                $openAllCount| Should Be 0 
            } 
        }
    } 
}

 

As you can see, we are getting all the NSG’s in a resource group and looping through them, and then looping through the rules in each NSG. We count how many rules allow inbound “all” and if it’s more than 0 fail the test.

Storage Account Test

For storage accounts we want to check that storage account encryption is enabled. This is actually configured at the storage resource level, and we are interested in blob encryption, so we test that. Again, looping through all storage accounts in the resource group.

Describe "Storage Account Tests" { 
    $storageAccounts = Get-AzureRmStorageAccount -ResourceGroupName $ResourceGroupName 
    foreach ($storageAccount in $storageAccounts) { 

        It "$($storageAccount.StorageAccountName ) Should have encrypted blob storage" { 
            $storageAccount.Encryption.Services.Blob.enabled | should be $true
        } 
    } 
}

Azure SQL Tests

Azure SQL has a couple of security measures we want to check for. The first is transparent data encryption, this is where the entire DB is encrypted and the decryption of this is handle by Azure as required, meaning no application changes are required. Because of this it is really recommended it is used where ever possible. The second is threat detection, this enables additional auditing on the SQL database to try and detect threats (and can be fed into Azure Security Centre). There is a cost to this, so may not be appropriate for everyone, but we are going to check that is enabled. Again, there are convenient Azure RM PowerShell commands that will check this for us.

Because the top level unit for SQL is the server what we need to do here is get all Azure SQL servers in a resource group, loop through these, and then loop through the databases in each server. We do also have to do a check on the database name, as using get-AzureRMSqlDatabase does retrieve the master db, but you can’t enable threat detection or encryption on this, so we only run these test when the DB is not “master”.

Describe "Azure SQL Tests" { 
    $sqlServers = Get-AzureRmSqlServer -ResourceGroupName $ResourceGroupName 
    foreach ($sqlserver in $sqlServers) { 
        $sqlDatabases = Get-AzureRmSqlDatabase -ServerName $sqlServer.ServerName -ResourceGroupName $ResourceGroupName 
        foreach ($sqlDatabase in $sqlDatabases) { 
            if ($sqlDatabase.databaseName -ne "Master") { 
                $tdeStatus = Get-AzureRmSqlDatabaseTransparentDataEncryption -ServerName $sqlserver.ServerName -DatabaseName $sqlDatabase.databaseName -ResourceGroupName $ResourceGroupName 
                $threatDetectionStatus = Get-AzureRmSqlDatabaseThreatDetectionPolicy -ServerName $sqlserver.ServerName -DatabaseName $sqlDatabase.databaseName -ResourceGroupName $ResourceGroupName 
                
                It "$($sqlDatabase.DatabaseName) on server $($sqlServer.serverName) Should have TDE Enabled" { 
                    $tdeStatus.State| should be "Enabled" 
                } 
                
                It "$($sqlDatabase.DatabaseName) on server $($sqlServer.serverName) Should have Threat Detection Enabled" { 
                    $threatDetectionStatus.ThreatDetectionState| should be "Enabled" 
                } 
            } 
        } 
    } 
}

 

Azure Security Centre Tests

The final test we want to do is check that Azure Security Centre data collection is enabled for this resource group. ASC will automatically on-board new resources to have their data collected, if you tell it to. This is set in an ASC resource policy, this can be set at the subscription or resource group level. If set at the subscription level it will apply to the resource groups unless an exception is added.

The standard Azure RM PowerShell cmdlets don’t include commands to interact with ASC, fortunately there is a module you can download form here that will do this for you, so you will need to install that to run this example. Using this module we collect all the policies for this subscription, then locate the one for this resource group and check that automatic log collection is enabled

Describe "Azure Security Center Test" {
    Import-Module "Azure-Security-Center" -WarningAction SilentlyContinue 
    $ascPolicies = Get-ASCPolicy 
    $rgPolicy = $ascPolicies | Where-Object { $_.name -eq "$resourcegroupname"} 
     
    It "$($resourcegroupname) should have Azure Security Center Enabled" { 
        $rgPolicy.properties.logCollection| should be "On" 
    } 
}

 

Running Tests

Now we have written all the tests all we have to do to run them is execute this ps1 file and supply a resource group to test. This assumes you have already logged in through Login-AzureRmAccount and selected the appropriate subscription. Once we execute the file it will run through all the tests, looping as required and output the results, pass or fail. In the example below I’m running this on an arbitrary demo resource group, and everything is passing except the Bitlocker encryption, which I haven’t yet enabled.

It’s very clear what is failing and what is good and where I need to take action. Running these tests takes a couple of minutes and you can immediately see where the issues are.

Once you have this running you could very easily have this audit run whenever you deploy an environment, as part of your overnight testing or even on a regular interval to check for non compliance that may have crept in (although if this is something you want to do a lot of then this may be the point to start looking at configuration management systems which can incorporate these tests).

Next Steps

Hopefully this has given you a good introduction into using Pester for Azure auditing. In no way have we covered everything you would need in a complex audit, but it should give you a starting point.

The full script is available here – https://github.com/sam-cogan/AzureAudit. I will be looking to evolve this to add more tests and make it a useful boilerplate for people to start auditing the Azure Infrastructure. If you have tests you want to add in please do submit a pull request and I will be happy to add them.