Private Docker Registries and the Canonical Distribution of Kubernetes

This blog post refers to an earlier version of Charmed Kubernetes. For the current methods of dealing with registries, please see the official documentation.

This originally appeared on Tim Van Steenburgh’s blog

How do I use a private image registry with my Kubernetes cluster? How do I set up my own registry? Let’s look at how to perform these tasks on the Canonical Distribution of Kubernetes (CDK).

Using an Existing Insecure Registry

In order to connect to an insecure registry, the Docker daemon must be reconfigured and an --insecure-registry option must be added.

This can be done directly via Juju, using the command:

juju config kubernetes-worker docker-config=”--insecure-registry registry.domain.com:5000"

Creating a Secure CDK Registry

CDK provides an option to deploy a secure Docker registry within the cluster, and expose it via an ingress.

Note: The registry provided is not a production grade registry, and should not be used in a production context.

Requirements

To deploy and use the provided registry, you will need:

  • A DNS entry (registry.acme.com) pointing at the ingress of the cluster (directly, via DNS round robin or with a load balancer)
  • A valid TLS certificate and key for registry.acme.com (registry.crt and registry.key)
  • A set of usernames and passwords stored in a file for htpasswd authentication (format: username:password, one user per line)

Considering a htpasswd.cleartxt file filled with users, the following loop will generate an encoded version of it:

while read line
do
  USER=$(echo ${line} | cut -f1 -d':')
  PASS=$(echo ${line} | cut -f2 -d':')
  docker run \
    --rm \
    xmartlabs/htpasswd \
    ${USER} ${PASS} \
    | tee -a htpasswd.enc
done < htpasswd.cleartxt
sed -i "/^$/d" htpasswd.enc

Deployment

To deploy the registry, run:

juju run-action kubernetes-worker/0 registry \
  domain=registry.acme.com \
  htpasswd=”$(base64 -w0 htpasswd.enc)” \
  htpasswd-plain=”$(base64 -w0 htpasswd.cleartxt)” \
  tlscert=”$(base64 -w0 registry.crt)” \
  tlskey=”$(base64 -w0 registry.key)” \
  ingress=true

Tear down

To tear down the registry, run

juju run-action kubernetes-worker/0 registry \
  delete=true \
  ingress=true

Storage

The registry provided by CDK will use a /srv/registry hostPath to store the images. This means that in case of a rescheduling of the registry (failure, overload…), if the new pod is scheduled on a different host, you will lose your images.

Alternatively, you can use a network mount such as NFS on all workers to benefit from a single point of storage for the images.

Ingress Configuration

The CDK registry action makes the assumption that the ingress running is nginx and will enforce a change of the configuration to increase the client_max_body_size from 1MB to 1GB. This is done via a patch, hence will not overwrite other configuration keys.

If you are using another ingress, deploy with ingress=false and make sure your ingress will support image upload (typical images are ~300MB, and typical CUDA images are 1 to 4GB)

Alternatives

If you want a similar setup but with flexibility on the storage and management via native Kubernetes tools you will find a derived work delivered via a Helm chart on https://github.com/madeden/charts

If you’d like to follow along more closely with CDK development, you can do so in the following places:

Until next time!

ubuntu logo

What’s the risk of unsolved vulnerabilities in Docker images?

Recent surveys found that many popular containers had known vulnerabilities. Container images provenance is critical for a secure software supply chain in production. Benefit from Canonical’s security expertise with the LTS Docker images portfolio, a curated set of application images, free of vulnerabilities, with a 24/7 commitment.

Integrate with hardened LTS images ›

Newsletter signup

Select topics you're
interested in

In submitting this form, I confirm that I have read and agree to Canonical's Privacy Notice and Privacy Policy.

Related posts

Ubuntu in the wild – 22nd of June

The Ubuntu in the wild blog post ropes in the latest highlights about Ubuntu and Canonical around the world on a bi-weekly basis.

Operator Day returns for KubeCon NA 2021

Register for free What is Operator Day? Operators simplify everyday application management on Kubernetes. Learn how to use them, how to create them in Python,...

Atresmedia modernises Spain’s most popular streaming platform with Canonical Charmed Kubernetes

The streaming service for the leading communication group in Spain leaves behind outdated applications and adopts a microservices architecture enabled by...