Installing Charmed Kubernetes offline
There are many reasons why it may be desirable to install Charmed Kubernetes on a system which does not have unfettered access to the internet. To make this possible, it is necessary to prepare the required resources, and configure Charmed Kubernetes to make use of them.
As user needs may vary, this documentation does not present a prescriptive recipe, but outlines the types of resources which are required and some recommendations on how to provide them. If you are already installing services in a restricted environment you may already have some 'air-gap' resources available, and may only need to configure Charmed Kubernetes to make use of them.
APT package repository
Access to a repository is required for installing software which is not yet available
as snap packages, as well as receiving updates for the underlying operating system.
In normal use this requires network access to
http://archive.ubuntu.com/ or one
of its localised mirrors.
In order to access the APT package repository, it is common to set up a local mirror or allow traffic through a proxy to the main archive.
There are many ways of setting up a local mirror. The repository is essentially just a directory of files and some means of serving them (http, ftp, etc). It is common to use tools such as rsync, apt-mirror or aptly to create the mirror.
Series and architectures
Note that the mirror should contain packages for the required series (e.g. focal
(Ubuntu 20.04), bionic) and architectures (e.g.
i386) expected to be used
in the deployment. The core Charmed Kubernetes components all use the
but some additional charms may be based on other series.
Setting up a proxy
- Tutorial on setting up a mirror with Rsync from ubuntu community docs.
- Tutorial on setting up a mirror with apt-mirror from HowtoForge.
- Using aptly to mirror APT repositories from the aptly documentation.
The majority of charms, including all the core Charmed Kubernetes charms, rely on snap packages to deliver applications. Snaps are packages for desktop, cloud and IoT that are easy to install, secure, cross‐platform and dependency‐free.
The list of snaps required by Charmed Kubernetes is detailed in the "components" page for each release. For example, for 1.23, the snaps are listed here.
While it is possible to download a snap package from the store, each snap will then need to be authenticated, and subsequent updates, even in the case of security updates, will require manual intervention. To avoid this, the recommended solution is to use the Snap Store Proxy software.
The Snap Store Proxy can also be configured to run in an "air-gap" mode, which disconnects it from the upstream store and allows snaps to be "sideloaded" into the local store. Information on how to do this is in the Snap Store Proxy documentation.
Note: Running the Snap Store Proxy also requires access to a PostgreSQL database, and an Ubuntu SSO account.
Charmed Kubernetes requires Juju, the Juju environment will also
need to be deployed in an offline-mode. Details of how to install and run Juju
are in the relevant section of the Juju documentation.
Charmed Kubernetes relies on container images for many of its components. To
run an air-gap or offline installation, it will be necessary to make these
images through Juju configuration or a Juju deployed registry.
Creating a private registry
The registry is simply a store for managing and serving up the requested images. Many public clouds (Azure, AWS, Google etc) also have registry components which could be used, but for the small number of images required for Charmed Kubernetes it is sufficient to run a local repository using Docker.
The recommended method is to use Juju to deploy a Docker registry and use that to serve the required images. See the Docker registry documentation for more details.
Note that if you wish to deploy the registry in the same Juju model (recommended) as Charmed Kubernetes, you should populate the registry with the images before deploying the rest of Charmed Kubernetes.
Fetching the required images
A list of the required images for each supported release is made available as part of the Charmed Kubernetes bundle repository on github. You can inspect or download the lists from the container images directory.
Using this list, it is possible to fetch the desired images locally on a system which has access to public repositories. The shrinkwrap tool also gathers all the necessary images for a specific release into one "containers" folder of the resulting tar.gz ready for installation.
When using the Juju docker-registry charm, the image archives can be copied to the
running unit added to the registry. Note that if the
docker-registry charm itself
has been deployed offline, you will also need to fetch the registry image:
docker pull registry docker save registry | gzip > registry.tgz
The local image files can then be copied to the unit running the docker-registry and loaded:
juju scp *.tgz docker-registry/0: juju ssh docker-registry/0 ls -1 *.tgz | xargs --no-run-if-empty -L 1 docker load -i rm -rf *.tgz exit
You can confirm the images are present by running the action:
juju run docker-registry/0 images
Juju will require access to OS images to install on machines. This is usually handled by Juju in conjunction with the underlying cloud, and has no need of any user interaction. If you are using a private cloud which has not yet been configured for use with Juju or Ubuntu images, the following documentation may be useful.
- OpenStack image metadata from Juju docs
- Image metadata for MAAS from the MAAS docs
LXD is a special case, as not only can Charmed Kubernetes be deployed entirely on LXD containers (See the localhost documentation), but LXD is also used in other clouds to co-locate applications on a single machine. In both cases, the OS images need to be fetched from somewhere.
This can be configured in various ways using LXD, either by pre-caching image files or pointing to an accessible repository. This is covered in detail in the LXD image documentation.
Python packages and PyPI
Charmed Kubernetes base charms all come with the necessary pip wheels.
Other charms (e.g. those used to monitor or provide metric data) may require
additional packages which aren't bundled as wheels, and expect to install those
dependencies from PyPI. Any charm attempting to do so, will need to handle pip installing
from a different pypi-server using the
extra-index-url argument and charm configuration.
Check the documentation of any additional charms.
The Linux Kernel supports realtime updates to the running kernel without restarting the existing kernel. In normal use this requires network access to pull the kernel patches and apply to the running kernel. However, with On Prem Livepatch, patches can be published to a locally available livepatch hosting server.
Bundle and charms
The specific bundle and charms which are required by those bundles must first be retrieved, then locally installed with Juju. The bundles, overlays, and charms to install can be retrieved using the Shrinkwrap tool.
from an internet connected machine:
- Download the installable bundles, charms, snaps, and containers using Shrinkwrap
Pull the archive for the deployment
git clone https://github.com/charmed-kubernetes/cdk-shrinkwrap.git /tmp/.shrinkwrap cd /tmp/.shrinkwrap BUNDLE='charmed-kubernetes --channel=1.26/stable' # Choose a deployment bundle (example is 1.26.x) ./shrinkwrap-lxc.sh $BUNDLE ls /tmp/.shrinkwrap/build/
The shrinkwrap script will create a tar archive named after the channel and the date, and stored in a directory named after the bundle and version (e.g in this case 'kcharmed-kubernetes-1.26).
In the air-gapped environment with access to the Juju controller:
- Extract the tar.gz file
- Print Available instructions from the deploy.sh
- Push the snaps to the snap-store-proxy
- Push the container images to your offline container registry
- Ensure the Juju environment is configured to pull from the snap-store-proxy and container registry
- This will require configuration changes on the
kubernetes-control-planeapplication in the
- This will require configuration changes on the
Finally, deploy the Juju charms and resources from the provided local bundle:
tar -xvf charmed-kubernetes-*/*.tar.gz --force-local cd charmed-kubernetes-*/ ./deploy.sh
Examine the provided instructions and ensure necessary modifications are considered and made. Then...
juju deploy ...
Configuring Charmed Kubernetes to work with proxies
Whether you decide to proxy any or all of the above services, the only extra configuration required is for Juju to route the traffic to the relevant proxies. Proxy configuration for Charmed Kubernetes is covered in the proxy documentation.