Enable AKS Flux Extension with Infrastructure as Code
Flux is a solution for doing GitOps on your AKS cluster. You can configure your cluster configuration in Git and have it sync with your cluster and set up all your resources. If you want an intro to GitOps this is a great primer.
Microsoft recently announced support for Flux v2 on AKS using the new Flux GitOps extension. Using this extension, you can enable Flux, deploy the required Flux resources and configure your GitOps sources and get your cluster syncing with Git. The documentation for this is pretty good if you want to enable the extension using the Azure CLI, but if you’re creating your AKS cluster using Infrastructure as Code (which you should be!) then it’s not very well documented at all, and if you don’t know where to look it can be challenging to figure out how to do it. This article will show you how to get this set up in your IaC tool of choice.
Using IaC is a bit more complicated than with the CLI, as there are two components you need to deploy:
- The extension, which setups up the Flux controllers and deploys the required containers
- The flux configuration, which hooks up Flux to your Git repository
The CLI commands bundle these all up together, but for IaC, we need to define them separately.
Pre-Requisites
The pre-requisites for this are similar to those defined in the docs.
Register the AKS extension feature on your subscription:
az feature register --namespace Microsoft.ContainerService --name AKS-ExtensionManager
Register the required resource providers:
az provider register --namespace Microsoft.Kubernetes
az provider register --namespace Microsoft.ContainerService
az provider register --namespace Microsoft.Kubernetes configuration
Enable the Flux Extension
We’re now ready to add the Flux extension to the cluster, setting up the flux controllers. Extensions are a relatively new feature for AKS, coming from Azure ARC. They look to be replacing AKS add-ons and are separate sub-resource types rather than being a part of the top-level AKS resource. The code below will enable the extension on your cluster. I’m using an existing cluster to apply the extension in the examples below. You can, of course, deploy the AKS cluster as part of the same code.
The Flux components will get deployed to the “flux-system” namespace in this example. The scope must be set to cluster; currently, namespace scoped Flux is not supported.
Once the extension is installed, you should see a Flux-System namespace created and the Flux pods running inside it.
Bicep
resource aks 'Microsoft.ContainerService/managedClusters@2021-10-01' existing = {
name: 'akscluster001'
}
resource flux 'Microsoft.KubernetesConfiguration/extensions@2021-09-01' = {
name: 'flux'
scope: aks
properties: {
extensionType: 'microsoft.flux'
scope: {
cluster: {
releaseNamespace: 'flux-system'
}
}
autoUpgradeMinorVersion: true
}
}
Pulumi
The example below is in C#. This can be transformed to other languages using the documentation here.
var fluxExtension = new Pulumi.AzureNative.KubernetesConfiguration.V20210901.Extension("fluxExtension", new Pulumi.AzureNative.KubernetesConfiguration.V20210901.ExtensionArgs
{
ClusterName = aks.Name,
ClusterRp = "Microsoft.ContainerService",
ExtensionType = "microsoft.flux",
Scope = new Pulumi.AzureNative.KubernetesConfiguration.V20210901.Inputs.ScopeArgs()
{
Cluster = new Pulumi.AzureNative.KubernetesConfiguration.V20210901.Inputs.ScopeClusterArgs()
{
ReleaseNamespace = "flux-system"
}
},
AutoUpgradeMinorVersion = true,
ClusterResourceName = "managedClusters",
ExtensionName = "flux",
ResourceGroupName = rgName
});
Terraform
There was no Terraform resource for defining AKS extensions at the time of writing.
Flux Configuration
Once the extension is deployed, we can now create a Flux Configuration. The Flux configuration links Flux to your Git repository and defines:
- The git repo you want to use
- The branch you want to use
- The root Customization objects you want to run, which will then deploy the rest of your workloads
You can define multiple Flux Configurations, as each one will be tied to a specific Git repo, so if you have multiple repositories, you will have multiple configurations.
The examples below connect to public Git repositories. We will look at authenticated repositories in a moment. Once deployed, you should see the resources being deployed. If you install the Flux CLI, you can view the status of these resources with the command.
flux get all -n flux-system
Bicep
resource fluxConfig 'Microsoft.KubernetesConfiguration/fluxConfigurations@2021-11-01-preview' = {
name: 'gitops-demo'
scope: aks
dependsOn: [
flux
]
properties: {
scope: 'cluster'
namespace: 'gitops-demo'
sourceKind: 'GitRepository'
suspend: false
gitRepository: {
url: 'https://github.com/fluxcd/flux2-kustomize-helm-example'
timeoutInSeconds: 600
syncIntervalInSeconds: 600
repositoryRef: {
branch: 'main'
}
}
kustomizations: {
infra: {
path: './infrastructure'
dependsOn: []
timeoutInSeconds: 600
syncIntervalInSeconds: 600
validation: 'none'
prune: true
}
apps: {
path: './apps/staging'
dependsOn: [
{
kustomizationName: 'infra'
}
]
timeoutInSeconds: 600
syncIntervalInSeconds: 600
retryIntervalInSeconds: 600
prune: true
}
}
}
}
Pulumi
var flux = new Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.FluxConfiguration("flux", new Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.FluxConfigurationArgs
{
ClusterName = aks.Name,
ClusterResourceName = "managedClusters",
ClusterRp = "Microsoft.ContainerService",
ResourceGroupName = rgName,
Namespace = "flux-system",
Scope = Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.ScopeType.Cluster,
FluxConfigurationName = "gitops-demo",
GitRepository = new Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.Inputs.GitRepositoryDefinitionArgs()
{
Url = "https://github.com/fluxcd/flux2-kustomize-helm-example",
RepositoryRef = new Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.Inputs.RepositoryRefDefinitionArgs
{
Branch = "main",
},
SyncIntervalInSeconds = 200,
TimeoutInSeconds = 600,
},
Customizations =
{
{ "infra", new Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.Inputs.KustomizationDefinitionArgs
{
DependsOn = {}},
Path = "./infra",
SyncIntervalInSeconds = 600,
TimeoutInSeconds = 600,
Prune = true
} },
{ "apps", new Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.Inputs.KustomizationDefinitionArgs
{
DependsOn = {new Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.Inputs.DependsOnDefinitionArgs(){
KustomizationName = "infra"
}},
Path = "./apps/staging",
SyncIntervalInSeconds = 600,
TimeoutInSeconds = 600,
Prune = true
} }
}
}, new CustomResourceOptions() { DependsOn = {fluxExtension}});
Authentication with Git
If you are using a private Git repo, you will need to authenticate with it. Using the CLI, there are some HTTP auth options; these don’t exist in the IaC option. Instead, we can use the ConfigurationProtectedSettings option to pass in some settings that will be converted into a Kubernetes secret called <configuration name>-protected-settings so, in our example, the secret is called gitops-demo-protected-parameters. We can then use the LocalAuthRef option to refer to that secret for auth.
For GitHub or Azure DevOps, you can generate a PAT token and specify this as the password in your secret, the username is not important, but I usually use my account name. The values you define in the template need to be bas64 encoded.
Bicep
resource fluxConfig 'Microsoft.KubernetesConfiguration/fluxConfigurations@2021-11-01-preview' = {
name: 'gitops-demo'
scope: aks
dependsOn: [
flux
]
properties: {
scope: 'cluster'
namespace: 'srs-namespace'
sourceKind: 'GitRepository'
suspend: false
gitRepository: {
url: 'https://github.com/fluxcd/flux2-kustomize-helm-example'
timeoutInSeconds: 600
syncIntervalInSeconds: 600
LocalAuthRef: 'gitops-demo-protected-parameters'
repositoryRef: {
branch: 'main'
}
}
ConfigurationProtectedSetting: {
username: '<Base 64 encoded username>'
password: '<Base 64 encoded PAT token'
}
customizations: {
...
Pulumi
var flux = new Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.FluxConfiguration("flux", new Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.FluxConfigurationArgs
{
ClusterName = aks.Name,
ClusterResourceName = "managedClusters",
ClusterRp = "Microsoft.ContainerService",
ResourceGroupName = rgName,
Namespace = "flux-system",
Scope = Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.ScopeType.Cluster,
FluxConfigurationName = "gitops-demo",
GitRepository = new Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.Inputs.GitRepositoryDefinitionArgs()
{
Url = "https://github.com/fluxcd/flux2-kustomize-helm-example",
RepositoryRef = new Pulumi.AzureNative.KubernetesConfiguration.V20220101Preview.Inputs.RepositoryRefDefinitionArgs
{
Branch = "main",
},
SyncIntervalInSeconds = 200,
TimeoutInSeconds = 600,
LocalAuthRef = "gitops-demo-protected-parameters"
},
ConfigurationProtectedSettings =
{
{"username", "<Base 64 encoded username>" },
{"password", "Base 64 encoded PAT token>" }
},
Customizations =
{
...
