Creating a cross-zone AKS cluster

This article describes the process of creating a cross-zone AKS cluster.

Before you begin...

• These steps are best done through the console in the Azure Portal. Alternatively, it may also be done through Azure Cloud Shell.
• AKS clusters can currently be created using availability zones in the following regions (updated list):
  • Central US (centralus)
  • East US 2 (eastus2)
  • East US (eastus)
  • France Central (francecentral)
  • Japan East (japaneast)
  • North Europe (northeurope)
  • Southeast Asia (southeastasia)
  • UK South (uksouth)
  • West Europe (westeurope)
  • West US 2 (westus2)
• You have to deploy 'Standard' type Load Balancer SKU (included in 'Creating a Cluster' section below). Standard Load Balancer SKU is only available for Kubernetes version 1.13.5 and above.
• You can get the list of available Kubernetes versions using the following command. Substitute the region name for <region-name>:

  az aks get-versions --output table --location <region-name>

• You can check if the feature required to have nodes in separate zones is enabled by executing the following command:

  az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AvailabilityZonePreview')].{Name:name,State:properties.state}"

  It should say 'Registered'.
  
  If it says 'NotRegistered', execute the following commands to enable the feature:

  az extension add --name aks-preview

  az extension update --name aks-preview

  az feature register --name AvailabilityZonePreview --namespace Microsoft.ContainerService

  az provider register --namespace Microsoft.ContainerService

  
  To verify, execute:

  az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AvailabilityZonePreview')].{Name:name,State:properties.state}"

Creating a Resource Group

All Azure devices have to be created in a resource group. Execute the following command to create a new resource group. You have to provide a <resource-group-name> and a <region-name>.

az group create --name <resource-group-name> --location <region-name>

Sample command:

az group create --name myResourceGroup --location eastus

Creating a Cluster 

The command is az aks create. The important flags are:

• --name <cluster-name>
• --resource-group <resource-group-name>
• --kubernetes-version <k8s-version>
• --load-balancer-sku standard
• --node-count <no-of-nodes>
• --node-zones <zones>
• --node-vm-size <vm-size>

• Read more here.
• Get the list of VM sizes available in <region-name> using the following command: az vm list-sizes --location <region-name>

Next, execute the az aks get-credentials command to get the credentials for the managed Kubernetes cluster:

az aks get-credentials --name <cluster-name> --resource-group <resource-group-name>

Sample command:

az aks create \
	--resource-group myResourceGroup \
	--name myAKSCluster \
	--kubernetes-version 1.15.4 \
	--load-balancer-sku standard \
	--node-count 3 \
	--node-zones 1 2 3 \
	--node-vm-size Standard_B2ms \
	--node-osdisk-size 40 \
	--vm-set-type VirtualMachineScaleSets

az aks get-credentials --name myAKSCluster --resource-group myResourceGroup

Creating and Attaching disks

A separate resource group is automatically created for the cluster and its components. We have to use the name of this resource group to execute commands. Also the VMs are a part of a Virtual Machine Scale Set. The name of the Scale Set is also needed to execute commands directed towards the cluster.

STEP 1

We create the CLUSTER_RESOURCE_GROUP and SCALE_SET_NAME variables using the following commands:

CLUSTER_RESOURCE_GROUP=$(az aks show --resource-group <resource-group-name> --name <cluster-name> --query nodeResourceGroup -o tsv)

SCALE_SET_NAME=$(az vmss list --resource-group $CLUSTER_RESOURCE_GROUP --query [0].name -o tsv)

STEP 2

Next, we create disks using the az disk create command. The required flags are:

• --name <disk-name>
• --resource-group $CLUSTER_RESOURCE_GROUP
• --location <region-name>
• --zone <zone>
• --os-type linux
• --size-gb <size>
• --sku <disk-type>

There are 4 types of disk SKUs at the moment:

• Ultra SSD (UltraSSD_LRS)
• Premium SSD (Premium_LRS)
• Standard SSD (StandardSSD_LRS)
• Standard HDD (Standard_LRS)

Read more about managed Azure disks here.

Sample Command for creating 3 disks with the names disk1, disk2 and disk3 in the 1st, 2nd and 3rd availability zone of East US region respectively, using a shell script. In this case, we are using Standard HDD disks.

for x in 1 2 3
do
az disk create --name disk$x --resource-group $CLUSTER_RESOURCE_GROUP --location eastus --zone $x --os-type linux --size-gb 32 --sku Standard_LRS
done

Use the following command to verify the creation of the disks. This command lists the disks in the same resource group as the cluster:

az disk list --resource-group $CLUSTER_RESOURCE_GROUP -o table

STEP 3

Attach disks using the following command. Only disks in the cluster's resource group can be attached to the cluster. You can see the disks available for attachment using the command given in the previous step.

az vmss disk attach --vmss-name $SCALE_SET_NAME --resource-group $CLUSTER_RESOURCE_GROUP --instance-id <instance-id> --disk <disk-name>
##Disks can only be attached to instances in the same region and availability zone.

Sample command to attach the disks created in the previous step to the 3 nodes, while making sure the disks and nodes are in the same zone:

for x in 0 1 2
do
y=$(($x+1))
az vmss disk attach --vmss-name $SCALE_SET_NAME --resource-group $CLUSTER_RESOURCE_GROUP --instance-id $x --disk disk$y
done

To verify, use the following command on each vm-instance to get the lsblk output from them:

az vmss run-command invoke --resource-group $CLUSTER_RESOURCE_GROUP --name $SCALE_SET_NAME --command-id RunShellScript --instance-id <instance-id> --scripts "lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL" --query value -o table;