1.17+ck2 Bugfix release

March 2, 2020 - charmed-kubernetes-410

Fixes

CephFS is now supported in Charmed Kubernetes. This allows for ReadWriteMany volumes which can be attached to multiple pods. More information can be found in the storage documentation.

Additional bug fixes included in this release can be found at https://launchpad.net/charmed-kubernetes/+milestone/1.17+ck2.

1.17+ck1 Bugfix release

January 15, 2020 - charmed-kubernetes-372

Fixes

We fixed an issue where pod-to-pod network traffic was being unnecessarily masqueraded when using Flannel or Canal. More details can be found at https://launchpad.net/charmed-kubernetes/+milestone/1.17+ck1.

1.17

December 17, 2019 - charmed-kubernetes-335

Before upgrading, please read the upgrade notes.

What's new

• CIS Benchmark

The Center for Internet Security (CIS) maintains a Kubernetes benchmark that is helpful to ensure clusters are deployed in accordance with security best practices. See the CIS Compliance documentation for instructions on how to run this compliance benchmark.

• Snap Coherence

Beginning with Charmed Kubernetes 1.17, updates to Kubernetes snap packages used by kubernetes-master and kubernetes-worker charms will be applied in a controlled fashion. Known as Snap Coherence, this feature ensures snap updates are first applied to individual master units, followed by workers. If an update fails, the process is aborted before affecting the entire cluster. This feature also allows snap revisions to be controlled by a snap store proxy. See snap coherence documentation for details.

• Nagios checks

Additional Nagios checks have been added for the kubernetes-master and kubernetes-worker charms. These checks enhance the monitoring and reporting available via Nagios by collecting data on node registration and API server connectivity.

• Improved metrics

kube-state-metrics is now added by default to the cluster when monitoring is enabled. New default dashboards are also included to highlight these metrics with Prometheus/Grafana.

• Storage Classes created by default

Storage classes will now be created if the kubernetes-master charm is related to an integrator charm. These classes are for AWS, GCE, Openstack, and Azure and are named cdk-ebs, cdk-gce-pd, cdk-cinder, and cdk-azure-disk, respectively.

• Support for etcd 3.3 and 3.4

Whilst Charmed Kubernetes 1.17 ships with etcd 3.3 by default, it also brings support for running etcd 3.4. To do so, you can simply run the followiung Juju command:

juju config etcd channel=3.4/stable

Component Upgrades

Many of the components in Charmed Kubernetes 1.17 have been upgraded. The following list highlights some of the more notable version changes:

• calico 3.10.1
• coredns 1.6.5
• etcd 3.3
• nfs-provisioner 3.1.0
• nginx-ingress-controller 0.26.1

Fixes

A list of bug fixes and other minor feature updates in this release can be found at https://launchpad.net/charmed-kubernetes/+milestone/1.17.

Notes / Known Issues

• The registry action for the kubernetes-worker charm has been deprecated and will be removed in a future release. To enable a custom container registry, please see the registry documentation.

1.16+ck2 Bugfix release

November 22, 2019 - charmed-kubernetes-316

Fixes

A list of bug fixes and other minor feature updates in this release can be found at https://launchpad.net/charmed-kubernetes/+milestone/1.16+ck2.

1.16+ck1 Bugfix release

October 4, 2019 - charmed-kubernetes-270

Fixes

A list of bug fixes and other minor feature updates in this release can be found at https://launchpad.net/charmed-kubernetes/+milestone/1.16+ck1.

1.16

September 27, 2019 - charmed-kubernetes-252

Before upgrading, please read the upgrade notes.

What's new

• Kata Containers support

Beginning with Charmed Kubernetes 1.16, the Kata Containers runtime can be used with containerd to safely run insecure or untrusted pods. When enabled, Kata provides hypervisor isolation for pods that request it, while trusted pods can continue to run on a shared kernel via runc. For details on using Kata Containers with Charmed Kubernetes, consult the documentation.

• AWS IAM support

Amazon AWS IAM authentication and authorisation is now supported via a subordinate charm. See AWS-IAM documentation for details on how to use AWS credentials to log in to your Charmed Kubernetes cluster.

• SSL passthrough support

A new configuration parameter was added to the kubernetes-worker charm to enable ssl passthrough. This allows TLS termination to happen on the workload. Refer to the upstream documentation for more information.

• Improved LXD support

LXD containers used for hosting Kubernetes components require some specific profile settings. These profiles are now embedded in the charms themselves and applied when deployed, dramatically simplifying the process of installing Charmed Kubernetes on a single machine. See the Local install documentation for the updated instructions.

• Improved Prometheus/Grafana integration

The setup and configuration of Prometheus and Grafana has been significantly streamlined with new relations to allow the charms to manage the scraper job and dashboards. This means that monitoring can now be added by specifying a single overlay when deploying Charmed Kubernetes. Refer to the updated documentation for more information.

• Improved OpenStack integration

The OpenStack Integrator charm can now replace the Kube API Load Balancer by providing a native OpenStack load balancer (Octavia or Neutron) to provide HA load balancing for the Kubernetes control plane. Refer to the updated documentation for more information.

• Docker Registry with Containerd

The Docker registry charm can now be related directly to the Containerd runtime charm. Refer to the documentation for instructions on how to deploy the charm.

• Renamed default container registry

The Canonical container image registry has a new, firewall-friendly name: image-registry.canonical.com:5000 is now rocks.canonical.com. The old URL is an alias for rocks and will continue to work. However, the default configuration for current charms has changed to the new URL.

Fixes

A list of bug fixes and other minor feature updates in this release can be found at https://launchpad.net/charmed-kubernetes/+milestone/1.16.

Special thanks to pierrop for contributing a fix to issue 1841965!

Known Issues

The Kubernetes Dashboard shipped with Charmed Kubernetes 1.16 is version 2.0.0-beta4. While unusual to ship a beta component with a stable release, in this case it was necessary, since the latest stable dashboard (v1.10.1) does not work with Kubernetes 1.16.

1.15+ck1 Bugfix release

August 15, 2019 - charmed-kubernetes-209

Fixes

A list of bug fixes and other minor feature updates in this release can be found at https://launchpad.net/charmed-kubernetes/+milestone/1.15+ck1.

1.15

June 28, 2019 - charmed-kubernetes-142

What's new

• Containerd support

Although Docker is still supported, containerd is now the default container runtime in Charmed Kubernetes. Containerd brings significant peformance improvements and prepares the way for Charmed Kubernetes integration with Kata in the future.

Container runtime code has been moved out of the kubernetes-worker charm, and into subordinate charms (one for Docker and one for containerd). This allows the operator to swap the container runtime as desired, and even mix container runtimes within a cluster. It also allows for additional container runtimes to be supported in the future. Because this is a significant change, you are advised to read the upgrade notes before upgrading from a previous version.

• Calico 3.x support

The Calico and Canal charms have been updated to install Calico 3.6.1 by default. For users currently running Calico 2.x, the next time you upgrade your Calico or Canal charm, the charm will automatically upgrade to Calico 3.6.1 with no user intervention required.

The Calico charm's ipip config option has been changed from a boolean to a string to allow for the addition of a new mode. This change is illustrated in the table below:

New value	Old value	Description
"Never"	false	Never use IPIP encapsulation. (The default)
"Always"	true	Always use IPIP encapsulation.
"CrossSubnet"	\	Only use IPIP encapsulation for cross-subnet traffic.

• Calico BGP support

Several new config options have been added to the Calico charm to support BGP functionality within Calico. These additions make it possible to configure external BGP peers, route reflectors, and multiple IP pools. For instructions on how to use the new config options, see the CNI with Calico documentation.

• Custom load balancer addresses

Support has been added to specify the IP address of an external load balancer. This support is in the kubeapi-load-balancer and the kubernetes-master charms. This allows a virtual IP address on the kubeapi-load-balancer charm or the IP address of an external load balancer. See the custom load balancer page for more information.

• Container image registry

By default, all container images required by the deployment come from the Canonical image registry. This includes images used by the cdk-addons snap, ingress, dns, storage providers, etc. The registry can be configured with the new image-registry config option on the kubernetes-master charm.

The addons-registry config option is now deprecated. If set, this will take precedence over the new image-registry option when deploying images from the cdk-addons snap. However, the addons-registry option will be removed in 1.17. Users are encouraged to migrate to the new image-registry option as soon as possible.

Fixes

A list of bug fixes and other minor feature updates in this release can be found at https://launchpad.net/charmed-kubernetes/+milestone/1.15.

Known Issues

• Docker-registry interface does not support containerd (bug 1833579)

When a docker-registry charm is related, kubernetes-worker units will attempt to configure the Docker daemon.json configuration file and may also attempt to use docker login to authenticate with the connected registry. This will not work in a containerd environment, as there is no daemon.json file nor docker command available to invoke.

Users relying on docker-registry to serve container images to Kubernetes deployments should continue to use the Docker subordinate runtime as outlined in the upgrade notes, under the heading "To keep Docker as the container runtime".

• Containerd charm does not work on LXD (bug 1834524)

We intend to fix this shortly after release. For now, if you want to deploy Charmed Kubernetes on LXD, we recommend using the Docker subordinate charm instead. Instructions for this can be found in the Container runtimes section of our documentation.

1.14 Bugfix release

June 19th, 2019 - charmed-kubernetes-124

Fixes

• Fixed leader_set being called by kubernetes-master followers (Issue)

1.14 Bugfix release

June 6th, 2019 - charmed-kubernetes-96

Fixes

• Fixed leader_get import error in .reactive/kubernetes_master_worker_base.py (Issue)
• Fixed kernel network tunables need better defaults and to be configurable (Issue)
• Fixed proxy-extra-args config missing from kubernetes-master (Issue)

1.14 Bugfix release

May 23rd, 2019 - charmed-kubernetes-74

Fixes

• Fixed missing core snap resource for etcd, kubernetes-master, kubernetes-worker, and kubernetes-e2e charms (Issue)
• Fixed kubernetes-master charm resetting user changes to basic_auth.csv (Issue)
• Fixed charm upgrades removing /srv/kubernetes directory (Issue)
• Fixed docker-opts charm config being ignored on kubernetes-worker (Issue)
• Fixed master services constantly restarting due to cert change (Issue)
• Fixed kubernetes-worker tag error on GCP (Issue)

1.14 Bugfix release

April 23rd, 2019 - charmed-kubernetes-31

Fixes

• Added automatic and manual cleanup for subnet tags (Issue)
• Added action apply-manifest (Issue)
• Added label to inform Juju of cloud (Issue)
• Added support for loadbalancer-ips (Issue)
• Fixed handling "not found" error message (Issue)
• Fixed snapd_refresh smashed by subordinate charm (Issue)
• Fixed making sure cert has proper IP as well as DNS (Issue)
• Fixed etcd charm stuck on "Requesting tls certificates" (Issue)
• Fixed cert relation thrashing due to random SAN order (Issue)
• Fixed contact point for keystone to be public address (Issue)
• Fixed cluster tag not being sent to new worker applications (Issue)
• Fixed removal of ceph relations causing trouble (Issue)
• Fixed pause/resume actions (Issue)
• Fixed ingress address selection to avoid fan IPs (Issue)
• Fixed snapd_refresh handler (Issue)
• Fixed credentials fields to allow for fallback and override (Issue)

1.14 Bugfix release

April 4th, 2019 - canonical-kubernetes-471

Fixes

• Fixed Ceph PV fails to mount in pod (Issue)
• Fixed Problems switching from kube-dns to CoreDNS (Issue)
• Fixed defaultbackend-s390x image (Issue)
• Fixed keystone-ssl-ca config description (Issue)
• Partial fix for using custom CA with Keystone (Issue)

1.14

March 27, 2019 - canonical-kubernetes-466

What's new

• Tigera Secure EE support

CDK extends its support for CNI solutions by adding the option of using Tigera Secure EE, the enterprise-ready alternative to Calico. Users are now able to deploy CDK with Tigera Secure EE installed and subsequently configure additional features such as ElasticSearch and the CNX secure connectivity manager. For further details, please see the CDK CNI documentation

• Additional options for High Availability

Version 1.13 of CDK introduced support for keepalived to provide HA for the api-loadbalancer. This new release adds support for both HAcluster and MetalLB. See the relevant HAcluster and MetalLB pages in the documentation, as well as the HA overview for more information.

• Added CoreDNS support

All new deployments of CDK 1.14 will install CoreDNS 1.4.0 by default instead of KubeDNS.

Existing deployments that are upgraded to CDK 1.14 will continue to use KubeDNS until the operator chooses to upgrade to CoreDNS. See the upgrade notes for details.

• Docker upgrades: Docker 18.09.2 is the new default in Ubuntu. CDK now includes a charm action to simplify upgrading Docker across a set of worker nodes.

• Registry enhancements: Read-only mode, frontend support, and additional TLS configuration options have been added to the Docker registry charm.

• Cloud integrations: New configuration options have been added to the vSphere (folder and respool_path) and OpenStack (ignore-volume-az, bs-version, trust-device-path) integrator charms.

Fixes

• Added an action to upgrade Docker (Issue)
• Added better multi-client support to EasyRSA (Issue)
• Added block storage options for OpenStack (Issue)
• Added dashboard-auth config option to master (Issue)
• Added docker registry handling to master (Issue)
• Added more TLS options to Docker registry (Issue)
• Added new folder/respool_path config for vSphere (Issue)
• Added proxy support to Docker registry (Issue)
• Added read-only mode for Docker registry (Issue)
• Fixed allow-privileged not enabled when Ceph relation joins (Issue)
• Fixed apt install source for VaultLocker (Issue)
• Fixed Ceph relation join not creating necessary pools (Issue)
• Fixed Ceph volume provisioning fails with "No such file or directory" (Issue)
• Fixed detecting of changed AppKV values (Issue)
• Fixed docker-ce-version config not working for non-NVIDIA configuration (Issue)
• Fixed Docker registry behavior with multiple frontends (Issue)
• Fixed Docker registry not cleaning up old relation data (Issue)
• Fixed Docker registry to correctly handle frontend removal (Issue)
• Fixed Docker registry to work behind a TLS-terminating frontend (Issue)
• Fixed error: snap "etcd" is not compatible with --classic (Issue)
• Fixed file descriptor limit on api server (Issue)
• Fixed GCP NetworkUnavailable hack when only some pods pending (Issue)
• Fixed handle_requests being called when no clients are related (Issue)
• Fixed handling of nameless and SANless server certificates (Issue)
• Fixed inconsistent cert flags (Issue)
• Fixed ingress=false not allowing custom ingress to be used (Issue)
• Fixed installing from outdated docker APT respository (Issue)
• Fixed IPv6 disabled on kubeapi-loadbalancer machines leads to error during installation (Issue)
• Fixed Keystone not working with multiple masters (Issue)
• Fixed kubeconfig should contain the VIP when keepalived used with kubeapi-load-balancer (Issue)
• Fixed metrics server for k8s 1.11 (Issue)
• Fixed proxy var to apply when adding an apt-key (Issue)
• Fixed RBAC enabled results in error: unable to upgrade connection (Issue)
• Fixed registry action creating configmap in the wrong namespace (Issue)
• Fixed rules for metrics-server (Issue)
• Fixed status when writing kubeconfig file (Issue)
• Fixed "subnet not found" to be non-fatal (Issue)
• Fixed vSphere integrator charm not updating cloud-config when setting new charm defaults (Issue)
• Removed deprecated allow-privileged config from worker (Issue)
• Removed use of global / shared client certificate (Issue)
• Updated default nginx-ingress controller to 0.22.0 for amd64 and arm64 (Issue)

Previous releases

Please see this page for release notes of earlier versions.