HA for kubeapi-load-balancer
The standard deployment of Charmed Kubernetes includes a single instance of the kube-api-loadbalancer. For many use cases this is perfectly adequate, but in a production environment you should be keen to eliminate any single point of failure.
The recommended way to provide a failover for the kube-api-loadbalancer on bare metal or MAAS is by using keepalived. This is available as a Juju charm and can be deployed into your Charmed Kubernetes model and configured as follows:
juju deploy keepalived
juju integrate keepalived:juju-info kubeapi-load-balancer:juju-info
Configure the keepalived application. You should substitute a suitable IP address and FQDN in the example below:
export VIP=10.10.74.250 export VIP_HOSTNAME=test.example.com juju config keepalived virtual_ip=$VIP juju config keepalived vip_hostname=$VIP_HOSTNAME
Allocating a VIP and ensuring that it can route to all of the instances is a manual process which depends on your infrastructure. It does require that the VIP be able to route to each instance, and that the VRRP protocol is allowed on the network. While this should be the case on bare metal and MAAS, and can be made to work on OpenStack, it will generally not be possible on public clouds. Thus, it is generally better to in those cases to replace kubeapi-load-balancer with a cloud-provided load balancer with health checks, such as Octavia or ELB.
Add both the new hostname and VIP to the API server certificate. This is done by specifying additional SANs:
juju config kubeapi-load-balancer extra_sans="$VIP $VIP_HOSTNAME" juju config kubernetes-control-plane extra_sans="$VIP $VIP_HOSTNAME"
Configure kubernetes-control-plane to use the VIP as the advertised Kubernetes API endpoint:
juju config kubernetes-control-plane loadbalancer-ips="$VIP"
Wait for the new service to settle. You can check the status of the
keepalivedapplication by running:
juju status keepalived
Once the application reports a 'ready' status, continue to the next step.
Scale up the
kubeapi-load-balancer. You can specify as many units as your situation requires. In this example, we add two additional units for a total of three:
juju add-unit kubeapi-load-balancer -n 2
Check for correct functionality by using kubectl and verifying it returns results. You can also check the SANs listed in the certificate returned by the VIP.
kubectl get pods --all-namespaces openssl s_client -connect $VIP:443 | openssl x509 -noout -text
Note that the
keepalived application is a
subordinate charm - it does not require a machine of its
own to run on, but rather runs alongside the
kubeapi-load-balancer charm. If
for any reason you need to view logs or troubleshoot this
application, it will be found co-located on the machines running the load