How to build a Raspberry Pi Kubernetes cluster using MicroK8s

This tutorial will be a brief walk through the process of getting MicroK8s up and running on Raspberry Pi, and joining multiple Pis to form a production-grade Kubernetes cluster.

MicroK8s is a lightweight, fast, enterprise-grade Kubernetes. Whether you’re new to K8s or a power user, MicroK8s will help you save time and space on any embedded device or IoT projects.

This setup can be fully headless or using an HDMI screen and USB keyboard to control nodes of your cluster.

What you’ll learn

• Deploying Kubernetes on Raspberry Pi using MicroK8s
• Joining multiple deployments to form a cluster
• Managing the cluster: adding and removing nodes

What you’ll need

• A 16.04 LTS (or later) Ubuntu desktop
• A minimum of 2 Raspberry Pis, they need to be Pi 3B or later. (You can use as many as you like, here we use 3)
• A micro-USB power cable for each Pi (USB-C for Pi4)
• A USB power-supply with as many ports as you have boards
• A microSD card per Pi (8GB recommended), flashed with an Ubuntu Server image
• Some basic command-line knowledge

Optional:

• A monitor with an HDMI interface
• An HDMI cable if you’re using Pis 2 or 3 or a MicroHDMI cable for the Pi 4
• A Cat5/6 network cable for each board is preferred, but Wi-FI setup is possible as well
• A USB keyboard
• A cluster rack (Here we are using the Cluster Case from the PiHut)

You will also need to have all of the boards on the same network, with a terminal window ready to connect to each Pi through SSH.

If you have gone ahead and purchased a rack for your Pis now is the time to set it up. The time to build will vary depending on which rack you bought and how nimble your fingers are, but it shouldn’t take you longer than 45 minutes. We recommend you do this at the start to have everything nicely organised before you get going.

If you have the PiHut Cluster Case that we used here, the assembly instructions are very straight forward. Here are a couple of in-progress shots for reference:

Once you’re done it should look something like this:

If you don’t have a rack, just ensure that the Pis can be connected to a power source and be in a location allowing them to connect to the same network (through WiFi or ethernet).

Each Pi is going to need an Ubuntu server image and you’ll need to be able to SSH into them.

This tutorial will teach you how to get to this stage. Follow it all the way until the install a desktop section. Go ahead and do that in another tab. We’ll wait.

Afterwards you should be able to log in to your Pis on your network using their IP addresses.

Follow this section for each of your Pis. Once completed you will have MicroK8s installed and running everywhere.

SSH into your first Pi and there is one thing we need to do before we get cracking. We need to enable c-groups so the kubelet will work out of the box. To do this you need to modify the configuration file /boot/firmware/cmdline.txt:

sudo nano /boot/firmware/cmdline.txt

And add the following options:

cgroup_enable=memory cgroup_memory=1

The full line for this particular raspberry pi looks like this:

cgroup_enable=memory cgroup_memory=1 net.ifnames=0 dwc_otg.lpm_enable=0 console=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait

Now save the file in your editor and reboot:

sudo reboot

Once that’s done we can now Install the MicroK8s snap:

sudo snap install microk8s --classic

What Kubernetes version is this installing?

MicroK8s is a snap and as such it will be automatically updated to newer releases of the package, which is following closely upstream Kubernetes releases.

To follow a specific Kubernetes upstream series it’s possible to select a channel during installation. For example, to follow the v1.17 series:

sudo snap install microk8s --classic --channel=1.17/stable

Channels are made up of a track (or series) and an expected level of stability, based on MicroK8s releases (Stable, Candidate, Beta, Edge). For more information about which releases are available, run:

snap info microk8s

Discovering MicroK8s

Before going further here is a quick intro to the MicroK8s command line:

• The start command will start all enabled Kubernetes services: microk8s.start
• The inspect command will give you the status of services: microk8s.inspect
• The stop command will stop all Kubernetes services: microk8s.stop
• You can easily enable Kubernetes add-ons, eg. to enable “kubedns”: microk8s.enable dns
• To get the status of the cluster: microk8s.kubectl cluster-info

MicroK8s is easy to use and comes with plenty of Kubernetes add-ons you can enable or disable.

Now that you have MicroK8s installed on all boards, pick one is to be the master node of your cluster.

On the chosen one, run the following command:

sudo microk8s.add-node

This command will generate a connection string in the form of <master_ip>:<port>/<token>.

Adding a node

Now, you need to run the join command from another Pi you want to add to the cluster:

microk8s.join <master_ip>:<port>/<token>

For example:

microk8s.join 10.55.60.14:25000/JHpbBYMIevZSAMnmjMHmFwanrOYCWZLu

You should be able to see the new node in a few seconds on the master with the following command:

microk8s.kubectl get node

For each new node, you need to run the microk8s.add-node command on the master, copy the output, then run microk8s.join <master node output> on the leaf.

Removing nodes

Duration: 1:00

To remove a node, run the following command on the master:

sudo microk8s remove-node <node name>

The name of nodes are available on the master by running the microk8s.kubectl get node command.

Alternatively, you can leave the cluster from a leaf node by running:

sudo microk8s.leave

You are now in control of your Kubernetes cluster: once Pis are setup with MicroK8s, adding and removing nodes is easy and you can scale up or down as you go.

What’s next?

The opportunities from here onwards are endless, we can’t wait to see what you come up with with your Pi cluster. For feedback, bug reports or contributing, reach out on GitHub, chat with us on the Kubernetes Slack, in the microk8s channel, Kubernetes forums or tag us @canonical or @ubuntu, on Twitter (microk8s).

And we of course recommend reviewing the microk8s documentation to get better acquainted with MicroK8s.