CNI with Cilium
Cilium CNI is a powerful networking plugin for Kubernetes which provides enhanced security and networking capabilities for containerised applications. It leverages the power of eBPF (extended Berkeley Packet Filter), a highly efficient and programmable kernel-level technology, to deliver transparent network security and traffic monitoring features.
Deploying Charmed Kubernetes with Cilium
To deploy a cluster with Cilium, deploy the
charmed-kubernetes bundle with
the Cilium overlay:
juju deploy charmed-kubernetes --overlay cilium-overlay.yaml
You can apply any additional customisation overlays to this deployment as well.
Cilium configuration options
A full list of Cilium configuration options and their descriptions can be found in the Cilium charm page on Charmhub.
Checking the current configuration
To check the current configuration settings for Cilium, run the command:
juju config cilium
Setting a config option
To set an option, simply run the config command with an additional
<key>=<value> argument. For example, to configure the port-forward
service for Hubble:
juju config cilium port-forward-hubble=true
The configuration settings that require further explanation are provided below.
Configuring the default Cluster Pool CIDR
The Cilium charm uses the Cluster Scope IPAM mode as its default
method for IP allocation. To define the IP address range that can be
assigned to Kubernetes Pods within the cluster, two configuration
settings can be adjusted:
cluster-pool-ipv4-cidr configuration specifies the IPv4 CIDR
block for the cluster pool. This determines the range of IP addresses
that the Kubernetes Pods can use.
On the other hand, the
setting specifies the number of bits in the subnet mask to be used
for the IP addresses in the cluster pool. It defines the number of
IP addresses that are available in the node from the cluster pool.
For instance, if
cluster-pool-ipv4-cidr is set to
cluster-pool-ipv4-mask-size is set to
24, then the available
IP addresses for the Kubernetes Pods in the one of the nodes will fall
in the range of
If you need to configure Cilium with a different CIDR pool during
deployment, simply create a file named
and include the following contents:
applications: cilium: options: cluster-pool-ipv4-cidr: 10.1.0.0/16 cluster-pool-ipv4-mask-size: 24
After creating the
cluster-pool-overlay.yaml file, add it as an
overlay during the deployment of Charmed Kubernetes. Keep in mind
that the order of the overlays is important:
juju deploy charmed-kubernetes --overlay cilium-overlay.yaml --overlay cluster-pool-overlay.yaml
Hubble is a fully distributed networking and security observability platform that complements the Cilium networking solution. It provides a comprehensive view of the Kubernetes networking infrastructure, allowing users to monitor and troubleshoot network traffic, microservices, and policies.
Since Hubble is developed as a separate daemon that operates alongside the Cilium agent, it must be enabled manually. To enable this component in the Cilium charm, execute the following command:
juju config cilium enable-hubble=true
This command will deploy Hubble in your cluster, allowing you to take advantage of its networking and security observability features.
The Cilium charm includes the
hubble CLI tools which enable
you to manage, troubleshoot, and visualise insights about your Kubernetes
cluster running Cilium. These CLI tools are accessible in each Kubernetes
Control Plane and Worker units. The upstream documentation
provides guidance on how to use these CLI tools. Please refer to it for further details.
Hubble Port Forward
To access the Hubble API, you need to port-forward the Hubble service, which enables you to connect the Hubble client to the service and access the Hubble Relay in your Kubernetes cluster. You can do this by running the following command:
juju config port-forward-hubble=true
Once you have done this, you should be able to reach Hubble from your units. Run the following command to verify the connection and check the status:
...which will return the current stats, for example:
Healthcheck (via localhost:4245): Ok Current/Max Flows: 20,475/20,475 (100.00%) Flows/s: 17.18 Connected Nodes: 5/5
For further information and details on how to use the Hubble client, please consult the relevant documentation.
Cilium and Hubble provide valuable metrics about the cluster and its workloads. However, by default, the Cilium charm disables metric exposure. Once you have enabled metric exposure for Cilium and Hubble, you can use the Canonical Observability Stack or another observability stack of your choosing to scrape and visualize the metrics.
To enable metrics for Cilium and Hubble, follow the instructions below:
To enable metrics for the Cilium CNI components, execute the following command:
juju config cilium enable-cilium-metrics=true
By executing this command, you will be able to access metrics
for Cilium CNI components such as
For more detailed information, please refer to the
Hubble provides several sets of metrics that can be enabled independently. To activate specific metrics, you can pass a list of metrics to enable as a space-separated string. Here is an example:
juju config cilium enable-hubble-metrics="dns icmp"
This command will enable both the
icmp metrics. For
additional information about the available metrics that
can be enabled, refer to the Hubble metrics
section in the upstream documentation.
COS Lite Integration
The Cilium charm integrates seamlessly with the Canonical Observability Stack (COS). You can refer to the COS observability stack documentation for detailed instructions on how to deploy the required components: Deploy the COS Lite observability stack on MicroK8s
Prometheus, from the COS Lite stack, can automatically scrape metrics from the Cilium and Hubble components. To set this up, you can integrate both Prometheus and Cilium via a Cross-Model Integration (CMI). The COS Lite documentation provides detailed instructions on how to create cross-model integrations using overlays. You can also refer to the Juju documentation for more information on how to set up and manage cross-model integrations.
The Cilium charm includes the necessary dashboards to visualise the metrics exported by the different Cilium components. For more information about the dashboards bundled in the Cilium charm, please refer to the Cilium repository.
To retrieve the dashboards, the Cilium charm needs to be related via a Cross-Model Integration to Grafana from the COS Lite stack.
If there is an issue with connectivity, it can be useful to inspect the Juju logs. To see a complete set of logs for Cilium, use the following command:
juju debug-log --replay --include=cilium
This will provide a detailed log output that can be used to identify and resolve issues. For additional troubleshooting pointers, please see the dedicated troubleshooting page.
- Cilium Documentation
- Cilium Network Policies
- Hubble Introduction
- Cilium on GitHub
- Cilium Charm on Charmhub