At our office, we’ve been using Docker containers deployed to Azure App Service for Containers for all of our microservices, but after a few incidents in the past couple of weeks, we’ve decided to look into managing our own Kubernetes cluster, but we quickly found out that integrating Azure Kubernetes Service and Azure Container Registry took a little bit of tweaking.
The main issue is that having AKS authenticate against ACR requires setting up a service principal and then instructing AKS to use that SP. It’s not hard to set up, but there are a few steps and I wanted to bring them all to one place to make things easier for you in the future.
I’m going to describe this process from the perspective of someone who already has a container registry and Kubernetes cluster stood up and just need to tie the two together. There are plenty of tutorials on how to stand each of those services up themselves and so I’ll leave it as an exercise for the reader to get to this point.
You should also be aware that the AKS cluster and the ACR must be in the same subscription for this process to work.
Step 1: Create the service principal using the following BASH script
echo -n "Ensure you are logged into Azure CLI before continuing and then press [ENTER]"
echo -n ""
echo -n "Enter the name of the container registry (*without* the .azurecr.io) and press [ENTER]: "
echo -n "Enter the name of the service prinicpal you would like to create (e.g. test-sp-dev) and press [ENTER]: "
# Populate the ACR login server and resource id.
ACR_LOGIN_SERVER=$(az acr show --name $ACR_NAME --query loginServer --output tsv)
ACR_REGISTRY_ID=$(az acr show --name $ACR_NAME --query id --output tsv)
# Create acrpull role assignment with a scope of the ACR resource.
SP_PASSWD=$(az ad sp create-for-rbac --name http://$SERVICE_PRINCIPAL_NAME --role acrpull --scopes $ACR_REGISTRY_ID --query password --output tsv)
# Get the service principal client id.
CLIENT_ID=$(az ad sp show --id http://$SERVICE_PRINCIPAL_NAME --query appId --output tsv)
# Output used when creating Kubernetes secret.
echo "Service principal ID: $CLIENT_ID"
echo "Service principal password: $SP_PASSWD"
You will want to make note of this ID and password combination for future troubleshooting as well as in the instances where you cannot proceed through step 4 of this post in the same terminal instance.
Step 2: Install the aks cli from az cli
az aks install-cli
NOTE: You may need to run this command using
sudo if you are on a Linux/MacOS/BSD computer
Step 3: Authenticate to your AKS cluster
Let’s assume that I have a Kubernetes cluster called
nexxai-k8s-dev in a resource group also called
nexxai-k8s-dev. I would run:
az aks get-credentials --resource-group nexxai-k8s-dev --name nexxai-k8s-dev
kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard
kubectl command just solves a permissions issue that appears if you try to access the Kubernetes dashboard using
az aks browse --resource-group nexxai-k8s-dev --name nexxai-k8s-dev. If you plan on doing things only through the CLI, I’m not sure if this step is required and you may be able to skip it.
Step 4: Tell Kubernetes to use that Service Principal
Let’s assume that I have a container registry called
nexxai-acr-dev and I want to name the secret
acr-auth. I would run:
kubectl create secret docker-registry acr-auth --docker-server nexxai-acr-dev.azurecr.io --docker-username $CLIENT_ID --docker-password $SP_PASSWD --docker-email [email protected]
This command assumes you are working in the same terminal as when you executed the BASH script in step 1, and uses their variables. If you are not working in the same terminal session as in step 1, simply substitute
$SP_PASSWD with their actual values.
Step 5: Profit!
At this point your new Kubernetes cluster is ready to talk to your container registry!
Now all you need to remember is that when you go to create a deployment, you need to provide the entire ACR address (including the
.azurecr.io part of the domain) for the
image setting, and you’ll need to define the
imagePullSecrets block and set its
name property to
acr-auth in your YAML file like this:
- name: demo-app
- containerPort: 4000
- name: acr-auth
And finally, by exposing the deployment using the command below, you’ll have a fully functional application that lives in Azure Kubernetes Service and uses Azure Container Registry to host its code.
kubectl expose deployment demo-app-dev-deploy --type=LoadBalancer --name=demo-app-dev