Introduction:
Rancher Labs’ K3s, a simplified Kubernetes distribution, is being embraced for its simplicity and resource efficiency. K3s users may face known issues when deploying it in production settings, as with any program. The following section outlines the upgrade method for upgrading clusters from v1.24.x to v1.25.x in K3s, which also emphasises the major issues affecting the project.
Known Issues:
Snap Docker:
Using K3s with Docker necessitates the avoidance of installing Docker using a snap package. When using Docker from snap, it is known that difficulties can occur when running K3s. Flawless execution requires Docker downloaded from other reliable sources.
Iptables and nftables Mode:
Using K3s may lead to complications while employing iptables in nftables mode rather than legacy mode. Using the newest iptables is recommended because it is less likely to disrupt K3s.
Rootless Mode:
Running K3s in Rootless mode should be attempted with care because it could potentially have several bugs and problems. Users must validate that this mode is dependable and trustworthy when used in a production environment.
Upgrading Hardened Clusters from v1.24.x to v1.25.x:
Kubernetes has omitted the PodSecurityPolicy (PSP) feature from v1.25 to utilize Pod Security Standards (PSS). K3s users with PSP configurations must perform specific manual steps during the upgrade process. Here’s a step-by-step guide:
Update kube-apiserver-arg Value:
The kube-apiserver-arg value on every node needs to be updated to eliminate the PodSecurityPolicy admission-plugin. Instead, add the following arg value:
arduino
'admission-control-config-file=/var/lib/rancher/k3s/server/psa.yaml'Do not turn on or update K3s instantly.
Create psa.yaml File:
Prepare the /var/lib/rancher/k3s/server/psa.yaml file with the text presented below. Customize exemptions as per your requirements:
yaml
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: PodSecurity
configuration:
apiVersion: pod-security.admission.config.k8s.io/v1beta1
kind: PodSecurityConfiguration
defaults:
enforce: "restricted"
enforce-version: "latest"
audit: "restricted"
audit-version: "latest"
warn: "restricted"
warn-version: "latest"
exemptions:
usernames: []
runtimeClasses: []
namespaces: [kube-system, cis-operator-system, system-upgrade]These names represent the cluster system, Cisco Operator system, and system-upgrade
Perform the Upgrade:
Once the kube-apiserver-arg is updated and the psa.yaml file is generated, the K3s upgrade should be done in the ordinary fashion.
Remove Remaining PSP Resources:
Upon completion of the upgrade process, eliminate all PodSecurityPolicies and the connected RBAC resources from the cluster. You can use the following commands to identify and delete the resources associated with PSPs:
shell
Get the resources associated with PSPs
kubectl get roles,clusterroles,rolebindings,clusterrolebindings -A | grep -i psp
Delete those resources:
kubectl delete clusterrole.rbac.authorization.k8s.io/psp:restricted-psp clusteConclusion:
A smooth K3s deployment is achieved by learning about known issues and taking the necessary steps to harden clusters. To guarantee that their K3s clusters remain stable, safe, and up to date with the most recent Pod Security Standards, users must adhere to these guidelines.
