MetalLB is a Kubernetes-aware solution that will monitor for services with
LoadBalancer and assign them an IP address from a virtual pool.
MetalLB has support for local traffic, meaning that the machine that receives the data will be the machine that services the request. It is not suggested to use a virtual IP with high traffic workloads because only one machine will receive the traffic for a service - the other machines are solely used for failover.
BGP does not have this limitation but does see nodes as the atomic unit. This means if the service is running on two of five nodes then only those two nodes will receive traffic, but they will each receive 50% of the traffic even if one of the nodes has three pods and the other only has one pod running on it. It is recommended to use node anti-affinity to prevent Kubernetes pods from stacking on a single node.
Layer 2 mode
The best way to deploy MetalLB in layer 2 mode on Charmed Kubernetes is with the MetalLB bundle, which includes Kubernetes operator charms both the controller and speaker components.
To deploy the operator, you will first need a Kubernetes model in Juju. Add your Kubernetes as a cloud to your Juju controller:
juju add-k8s k8s-cloud --controller $(juju switch | cut -d: -f1)
Next, create a new Kubernetes model:
juju add-model metallb-system k8s-cloud
Then you can deploy MetalLB:
juju deploy cs:~containers/metallb
You will likely want to change the IP addresses allocated to MetalLB to suit your environment. The IP addresses can be specified as a range, such as "192.168.1.88-192.168.1.89", or as a comma-separated list of pools in CIDR notation, such as "192.168.1.240/28, 10.0.0.0/28".
Configuring the IP addresses can be done either at time of deployment via a bundle overlay, or later by changing the charm config via Juju.
An example bundle overlay might look like:
applications: metallb-controller: options: iprange: "192.168.1.88-192.168.1.89"
You would then specify this when deploying the bundle:
juju deploy cs:~containers/metallb --overlay ./overlay.yaml
Alternatively, you can change the config directly on the metallb-controller charm at any time:
juju config metallb-controller iprange="192.168.1.240/28, 10.0.0.0/28"
Note: Using RBAC
If RBAC is enabled in the Kubernetes cluster, an extra deployment step is
required: before deploying MetalLB, you must apply the RBAC permissions
manifest. This manifest gives permissions to the operator pods
to use the Kubernetes API to create the necessary resources to make MetalLB
work. You can apply the manifest using
wget https://raw.githubusercontent.com/containers/metallb-operator/master/docs/rbac-permissions-operators.yaml kubectl apply -f rbac-permissions-operators.yaml
Be aware that the manifest has to refer to the namespace in which MetalLB is
deployed. This namespace will be the same as the Juju model you deployed it
into, above. If you used a model name other than
metallb-system, you will
need to edit the manifest before applying it.
If you forgot to apply this manifest before deploying MetalLB, the units will fail in the start hook. But don't worry! You can apply the manifest afterwards, and then resolve the units that are in error to get them back into a working state:
juju resolve metallb-controller/0 juju resolve metallb-speaker/0
Since the Kubernetes operator charms for MetalLB do not yet support BGP mode, for now the recommended way to deploy MetalLB in BGP mode is to use the upstream manifests:
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/namespace.yaml kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/metallb.yaml # On first install only kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
The BGP configuration can then be performed by using a MetalLB ConfigMap.
Once deployed, MetalLB will automatically assign IPs from its pools to any
service of type
LoadBalancer. When the services are deleted, the IPs are
To test your deployment of MetalLB, you can use the microbot manifest to
deploy a simple webapp with a service type of
wget https://raw.githubusercontent.com/containers/metallb-operator/master/docs/example-microbot-lb.yaml kubectl apply -f example-microbot-lb.yaml kubectl get service microbot-lb
The EXTERNAL-IP is the IP assigned to the microbot service by the MetalLB controller. If you reach this IP with a browser, you should see the image of a microbot. If you cannot, most probably the IP range is not correctly chosen; the IP range needs to be a pool reserved solely for MetalLB to avoid IP conflicts.
To remove the test webapp and service, simply delete the manifest with kubectl:
kubectl delete -f example-microbot-lb.yaml