Integrating Azure Kubernetes Service and Azure Container Registry

At our office, we’ve been using Docker containers deployed to Azure App Service for Containers for all of our microservices, but after a few incidents in the past couple of weeks, we’ve decided to look into managing our own Kubernetes cluster, but we quickly found out that integrating Azure Kubernetes Service and Azure Container Registry took a little bit of tweaking.

The main issue is that having AKS authenticate against ACR requires setting up a service principal and then instructing AKS to use that SP. It’s not hard to set up, but there are a few steps and I wanted to bring them all to one place to make things easier for you in the future.

I’m going to describe this process from the perspective of someone who already has a container registry and Kubernetes cluster stood up and just need to tie the two together. There are plenty of tutorials on how to stand each of those services up themselves and so I’ll leave it as an exercise for the reader to get to this point.

You should also be aware that the AKS cluster and the ACR must be in the same subscription for this process to work.

Step 1: Create the service principal using the following BASH script

#!/bin/bash

echo -n "Ensure you are logged into Azure CLI before continuing and then press [ENTER]"
read UNUSEDVARIABLE
echo -n ""
echo -n "Enter the name of the container registry (*without* the .azurecr.io) and press [ENTER]: "
read ACR_NAME
echo -n "Enter the name of the service prinicpal you would like to create (e.g. test-sp-dev) and press [ENTER]: "
read SERVICE_PRINCIPAL_NAME

# Populate the ACR login server and resource id.

ACR_LOGIN_SERVER=$(az acr show --name $ACR_NAME --query loginServer --output tsv)
ACR_REGISTRY_ID=$(az acr show --name $ACR_NAME --query id --output tsv)

# Create acrpull role assignment with a scope of the ACR resource.

SP_PASSWD=$(az ad sp create-for-rbac --name http://$SERVICE_PRINCIPAL_NAME --role acrpull --scopes $ACR_REGISTRY_ID --query password --output tsv)

# Get the service principal client id.

CLIENT_ID=$(az ad sp show --id http://$SERVICE_PRINCIPAL_NAME --query appId --output tsv)

# Output used when creating Kubernetes secret.

echo "Service principal ID: $CLIENT_ID"
echo "Service principal password: $SP_PASSWD"

You will want to make note of this ID and password combination for future troubleshooting as well as in the instances where you cannot proceed through step 4 of this post in the same terminal instance.

Step 2: Install the aks cli from az cli

az aks install-cli

NOTE: You may need to run this command using sudo if you are on a Linux/MacOS/BSD computer

Step 3: Authenticate to your AKS cluster

Let’s assume that I have a Kubernetes cluster called nexxai-k8s-dev in a resource group also called nexxai-k8s-dev. I would run:

az aks get-credentials --resource-group nexxai-k8s-dev --name nexxai-k8s-dev
kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard

NOTE: The kubectl command just solves a permissions issue that appears if you try to access the Kubernetes dashboard using az aks browse --resource-group nexxai-k8s-dev --name nexxai-k8s-dev. If you plan on doing things only through the CLI, I’m not sure if this step is required and you may be able to skip it.

Step 4: Tell Kubernetes to use that Service Principal

Let’s assume that I have a container registry called nexxai-acr-dev and I want to name the secret acr-auth. I would run:

kubectl create secret docker-registry acr-auth --docker-server nexxai-acr-dev.azurecr.io --docker-username $CLIENT_ID --docker-password $SP_PASSWD --docker-email [email protected]

This command assumes you are working in the same terminal as when you executed the BASH script in step 1, and uses their variables. If you are not working in the same terminal session as in step 1, simply substitute $CLIENT_ID and $SP_PASSWD with their actual values.

Step 5: Profit!

At this point your new Kubernetes cluster is ready to talk to your container registry!

Now all you need to remember is that when you go to create a deployment, you need to provide the entire ACR address (including the .azurecr.io part of the domain) for the image setting, and you’ll need to define the imagePullSecrets block and set its name property to acr-auth in your YAML file like this:

apiVersion: apps/v1
kind: Deployment
metadata:
name: demo-app-dev-deploy
labels:
app: demo-app
spec:
replicas: 3
selector:
matchLabels:
app: demo-app
template:
metadata:
labels:
app: demo-app
spec:
containers:
- name: demo-app
image: nexxai-acr-dev.azurecr.io/demo-app:latest
ports:
- containerPort: 4000
imagePullSecrets:
- name: acr-auth

And finally, by exposing the deployment using the command below, you’ll have a fully functional application that lives in Azure Kubernetes Service and uses Azure Container Registry to host its code.

kubectl expose deployment demo-app-dev-deploy --type=LoadBalancer --name=demo-app-dev

Congratulations!