Skip to main content

K3s Kubernetes

Network setting

K3s will come with pretty much everything pre-configured like traefik.

More info about traefik: https://doc.traefik.io/traefik/ or small guide from me Traefik.

However, I would like to have LoadBalancer, and in essence to be able to give services (pods) an external IP. Just like my Kubernetes nodes and not from internal Kubernetes ranges. Normally, this is an external component, and your cloud provider should somehow magically give that to you, but since we are our own cloud provider, and we are trying to keep everything in one cluster... in short MetalLB is the answer.

What is MetalLB

https://metallb.universe.tf/

Deployment

💡
Do this on your controll node.

In previous chapter, we have installed helm and arkade, so now its helms time to shine !

# First add metallb repository to your helm
helm repo add metallb https://metallb.github.io/metallb
# Check if it was found
helm search repo metallb
# Install metallb
helm upgrade --install metallb metallb/metallb --create-namespace \
--namespace metallb-system --wait

This should install MetalLB into your cluster and return something like this:

Release "metallb" does not exist. Installing it now.
NAME: metallb
LAST DEPLOYED: Sat Apr 1 10:49:30 2023
NAMESPACE: metallb-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
MetalLB is now running in the cluster.

Now you can configure it via its CRs. Please refer to the metallb official docs
on how to use the CRs

That is it for the installation. Now we need to configure it.

Configuration

MetalLB needs to know what IP range to use for external IPs. This is done via a Custom Resource. You can find the CR in the official documentation. So lets create and apply a Custom Resource with the following content:

cat << 'EOF' | kubectl apply -f -
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: default-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.0.200-192.168.0.250
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: default
  namespace: metallb-system
spec:
  ipAddressPools:
  - default-pool
EOF

It should return something like this:

ipaddresspool.metallb.io/default-pool created
l2advertisement.metallb.io/default created

As you can see, I specified a range from 192.168.0.200 to 192.168.0.250. That will give me 50 "external" IPs to work with for now.

Check

Check if everything deployed OK

root@control01:~/metallb# kubectl get pods -n metallb-system
NAME                         READY   STATUS    RESTARTS   AGE
controller-57fd9c5bb-rdl7v   1/1     Running   0          5m42s
speaker-h7chj                1/1     Running   0          5m42s
speaker-pg7kp                1/1     Running   0          5m42s
speaker-78pdz                1/1     Running   0          5m42s
speaker-ghpxz                1/1     Running   0          5m42s
speaker-8cf7k                1/1     Running   0          5m42s
speaker-2t6jp                1/1     Running   0          5m42s
speaker-cjcpn                1/1     Running   0          5m41s
speaker-mv7v4                1/1     Running   0          5m42s

You should have as many speaker-xxxx as you have nodes in the cluster, since they run one per node.

Now services that use LoadBalancer should have an external IP assigned to them.

For example:

root@control01:~/metallb# kubectl get svc -n kube-system
NAME                   TYPE              CLUSTER-IP    EXTERNAL-IP       PORT(S)                                 AGE
kube-dns           ClusterIP        10.43.0.10           <none>           53/UDP,53/TCP,9153/TCP       3h45m
metrics-server   ClusterIP         10.43.254.144     <none>           443/TCP                                  3h45m
traefik                LoadBalancer 10.43.159.145      192.168.0.200        80:31771/TCP,443:30673/TCP 3h44m

You can also check the events for the service:

root@control01:~/metallb# kubectl get events -n kube-system --field-selector involvedObject.name=traefik
LAST SEEN TYPE REASON OBJECT MESSAGE
61s Normal IPAllocated service/traefik Assigned IP ["192.168.0.200"]
60s Normal nodeAssigned service/traefik announcing from node "cube02" with protocol "layer2"

Look how traefik automatically got an IP from the external range. In the end, this is what you would want. Not to point to a single node IP and be redirected based on DNS, which would stop working the moment the node with that IP died. This way, we make the external IP node independent. Now, you can point DNS to MatalLB IP and be sure it will be routed correctly.

👍
This is how I prefer my network settings, and makes most sense to me when creating external services. I'm sure there are like a hundred different methods using external load balancers, Nginx ingress (basically reverse proxy) and who knows what in production, but hey, there is no official The one standardized setting for Kubernetes (which can be such a pain sometimes) so who’s to say this is not OK? 🙂