Accessing GCR repos from Kubernetes

February 7, 2020 - 2 minutes read - 363 words

Until today, I’d not accessed a Google Container Registry repo from a non-GKE Kubernetes deployment.

It turns out that it’s pretty well-documented (link) but, here’s an end-end example.

Assuming:

BILLING=[[YOUR-BILLING]]
PROJECT=[[YOUR-PROJECT]]
SERVER="us.gcr.io"

If not already:

gcloud projects create {$PROJECT}

gcloud beta billing projects link ${PROJECT} \
--billing-account=${BILLING}

gcloud services enable containerregistry.googleapis.com \
--project=${PROJECT}

Container Registry

IMAGE="busybox" # Or ...

docker pull ${IMAGE}

docker tag \
  ${IMAGE} \
  ${SERVER}/${PROJECT}/${IMAGE}

docker push ${SERVER}/${PROJECT}/${IMAGE}

gcloud container images list-tags ${SERVER}/${PROJECT}/${IMAGE}

Service Account

Create a service account that’s permitted to download (read-only) images from this project’s registry

ACCOUNT=[[YOUR-ACCOUNT]] # Must be >=6 alphanumeric
FILE="${ACCOUNT}.json"

gcloud iam service-accounts create ${ACCOUNT} \
--display-name=${ACCOUNT} \
--project=${PROJECT}

gcloud iam service-accounts keys create ${FILE} \
--iam-account=${ACCOUNT}@${PROJECT}.iam.gserviceaccount.com \
--project=${PROJECT}

gcloud projects add-iam-policy-binding ${PROJECT} \
--member=serviceAccount:${ACCOUNT}@${PROJECT}.iam.gserviceaccount.com \
--role=roles/storage.objectViewer

Secret

Here’s the trick, create a Kubernetes secret for this service account (key):

SECRET="gcr" # Or ...
EMAIL="any@valid.email"
NAMESPACE="default" # Or ...

kubectl create secret docker-registry ${SECRET} \
--docker-server=${SERVER} \
--docker-username=_json_key \
--docker-password="$(cat ${FILE})" \
--docker-email=${EMAIL} \
--namespace=${NAMESPACE}

We can introspect the secret:

kubectl get secret ${SECRET} \
--namespace=${NAMESPACE} \
--output=yaml

yields:

apiVersion: v1
data:
  .dockerconfigjson: ...
kind: Secret
metadata:
  creationTimestamp: "2020-02-07T00:00:00Z"
  name: "${SECRET}"
  namespace: "${NAMESPACE}"
type: kubernetes.io/dockerconfigjson

NB The data|type is dockerconfigjson this is a pre-req for this configuration (documentation)

and we can decode out the secret:

kubectl get secret ${SECRET} \
--namespace=${NAMESPACE} \
--output="jsonpath={.data.\.dockerconfigjson}" \
| base64 --decode \
| jq -r ".auths[\"${SERVER}\"].auth" \
| base64 --decode

yielding the original key details:

_json_key:{
  "type": "service_account",
  "project_id": "${PROJECT}",
  "private_key_id": ...,
  "private_key": ...,
  "client_email": "${ACCOUNT}@${PROJECT}.iam.gserviceaccount.com",
  "client_id": ...,
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/k8s-gcr%40pypi-transparency.iam.gserviceaccount.com"
}

NB This should match the content of ${FILE}.

Deployment

echo "
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
      app: ${IMAGE}
  name: ${IMAGE}
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ${IMAGE}
  template:
    metadata:
      labels:
        app: ${IMAGE}
    spec:
      imagePullSecrets:
      - name: ${SECRET}
      containers:
      - name: ${IMAGE}
        image: ${SERVER}/${PROJECT}/${IMAGE}
        args:
        - /bin/sh
        - -c
        - while true; do echo 'Hello Freddie'; sleep 2s; done
" | kubectl apply --filename=- --namespace=${NAMESPACE}

And:

kubectl get deployment/busybox \
--namespace=${NAMESPACE}

yields:

NAME      READY   UP-TO-DATE   AVAILABLE   AGE
busybox   1/1     1            1           5s

And, checking the logs to confirm:

kubectl logs deployment/busybox \
--namespace=${NAMESPACE}

yields:

Hello Freddie
Hello Freddie
Hello Freddie

HTH!