Kubernetes Distributions — Build Them Yourself
I edited k0s.yaml and even restarted, and nothing happened
This lab runs on a real k0s
There is one k0s v1.36.4+k0s.0 (controller and worker in one) running in the VM with static configuration. The configuration file is /etc/k0s/k0s.yaml and the working directory is /root/k0scfg. It takes about 3 minutes to come up the first time. During the lab you restart k0s three times.
Goal
You check, by mode, where k0s configuration comes from. You show with a worker profile ConfigMap that under static configuration a file change takes effect only after a restart, and that under dynamic configuration the ClusterConfig object, not the file, takes effect. You declare and delete a Helm extension and record how the Chart object and the release follow.
Why it matters
k0s is known for being configured with a single file, so even on a cluster with dynamic configuration turned on, people edit the file. Then nothing changes even after a restart, and no error appears. Conversely, under static configuration, the settings suddenly change on the day the node whose file was edited is restarted. In both cases, you can tell only by looking at what was actually created, not at "the command succeeded."
Steps
- Compare the defaults with this VM's settings and save them to
/root/k0scfg/defaults.yamland/root/k0scfg/defaults.json. - Leave the result of
k0s sysinfoin/root/k0scfg/sysinfo.jsonand/root/k0scfg/sysinfo-summary.txt. - Under static configuration, put a worker profile
edge-smallin the file, and record in/root/k0scfg/static-edit.jsonthat it does not take effect without a restart. - Restart k0s and record in
/root/k0scfg/static-restart.jsonthat the profile ConfigMap appears. - Switch to dynamic configuration and record the ClusterConfig object in
/root/k0scfg/dynamic.json. - Under dynamic configuration, put a profile
from-filein the file, restart, and record the result in/root/k0scfg/file-ignored.json. - Add a profile
from-apito ClusterConfig and record in/root/k0scfg/api-patch.jsonthat it takes effect without a restart. - Declare the podinfo chart in ClusterConfig and write
/root/k0scfg/helm.json. - Write the result of deleting the chart declaration to
/root/k0scfg/removal.jsonand the summary to/root/k0scfg/report.md.
Notes
- The worker profile ConfigMap name is
worker-config-<프로필>-1.<마이너>(the placeholders are the profile and the minor version; this VM is 1.36). The profile values go into thekubeletConfigurationdata as JSON. - A restart is
k0s stopfollowed byk0s start. After that, wait until the API responds. The service name isk0scontroller. - To change the execution arguments of an already installed service, run
k0s install controlleragain with the same arguments plus the new flag, with--force. If you leave out--enable-worker --no-taints -c /etc/k0s/k0s.yaml, the worker disappears. - The chart is
podinfo6.14.1 from thehttps://stefanprodan.github.io/podinforepository. The image is on ghcr.io, so you do not hit pull limits. - Common mistake 1: restarting first in step 3. The grader checks whether the record was written before the ConfigMap appeared.
- Common mistake 2: in step 7, overwriting
workerProfileswholesale with a merge patch and deletingedge-small. A merge patch replaces list fields. The re-grading of step 4 fails.
What happens to values you did not write
Save the output of k0s config create to /root/k0scfg/defaults.yaml, and write to /root/k0scfg/defaults.json the keys default_pod_cidr, default_service_cidr, default_storage_type, default_telemetry, file_pod_cidr, file_service_cidr, file_telemetry, apiserver_service_cidr, and reason (the reason the ranges were changed, at least 30 characters).
The installed binary tells you the defaults. The file values are in /etc/k0s/k0s.yaml, and the Service CIDR the actual API server uses is in service-cluster-ip-range of the kube-apiserver process arguments. If you think about where this VM runs, the reason for changing the ranges comes out.
Read the pre-installation check
Save the result of k0s sysinfo -o json to /root/k0scfg/sysinfo.json, and in /root/k0scfg/sysinfo-summary.txt write a total=<항목 수> line (the placeholder is the number of items), a <분류>=<수> line for each result category (for example pass=; the placeholders are the category and the count), a cgroups=<Control Groups 항목의 값> line (the placeholder is the value of the Control Groups item), and, for each item that is not pass, a not_pass=<displayName> line.
The JSON is an array of items, and each item has category, displayName, and prop. Count by category and copy the names of the ones that are not pass. The grader runs sysinfo again and checks whether the same numbers come out.
You edited the file but nothing happened
Put a worker profile edge-small (maxPods: 40) in the spec of /etc/k0s/k0s.yaml, check it with k0s config validate, then without restarting, wait at least 30 seconds and write to /root/k0scfg/static-edit.json the keys profile, configmap (the name of the ConfigMap that should appear), configmap_present, and k0s_pid (the MainPID of the k0scontroller service).
A worker profile is the name and values in the spec.workerProfiles list. If k0s watched the file, a ConfigMap should appear in kube-system while you wait. The PID is a value you use later to compare 'whether a restart happened after that', so read it from systemctl.
It appeared after the restart
Restart k0s, and once the edge-small ConfigMap appears, write to /root/k0scfg/static-restart.json the keys configmap, configmap_uid, max_pods (the value read from the kubeletConfiguration of the ConfigMap), k0s_pid_before, and k0s_pid_after.
The restart is k0s stop and start. After the API comes back, wait briefly until the ConfigMap appears. The kubeletConfiguration is a JSON string inside the ConfigMap data. The PID from before the restart is in the record from step 3.
Move the source of truth to the API
Reinstall and start the k0scontroller service with --enable-dynamic-config added, and then write to /root/k0scfg/dynamic.json the keys clusterconfig_uid, object_has_edge_small, object_service_cidr (the object's spec.network.serviceCIDR), and apiserver_service_cidr (the actual API server argument).
For an already installed service, simply running k0s install controller again is rejected. You must keep all the original arguments so that the worker does not disappear. When dynamic configuration is turned on, a clusterconfig resource appears in kube-system. Read the object's Service CIDR and the actual value separately and compare them.
This time it is ignored even after a restart
Under dynamic configuration, add from-file (maxPods: 30) to the worker profile list in /etc/k0s/k0s.yaml, restart k0s, wait at least 20 seconds, and write to /root/k0scfg/file-ignored.json the keys profile, generation_before, generation_after (the metadata.generation of ClusterConfig), object_has_profile, and configmap_present.
This is a change that would have taken effect with a restart had it been static configuration. For what the file is used for under dynamic configuration, see the section 'Cluster settings and node settings' in the reading. The generation goes up every time the object's spec changes.
If you edit the object, it takes effect right away
Without restarting, add from-api (maxPods: 60) to the worker profile list of ClusterConfig without deleting the existing profile, and once the ConfigMap appears, write to /root/k0scfg/api-patch.json the keys profile, configmap_uid, max_pods, and controller_started_at (the time the k0scontroller service last started, in Unix seconds).
A merge patch replaces a list field wholesale. The way to add one at the end of a list is in JSON patch. You can get the service start time by converting ActiveEnterTimestamp from systemctl with date. The grader checks whether the ConfigMap appeared after that time.
A declared chart becomes a Chart object
In spec.extensions.helm of ClusterConfig, declare the repository podinfo (https://stefanprodan.github.io/podinfo) and the chart podinfo (chartname podinfo/podinfo, version 6.14.1, namespace podinfo), and once the deployment is ready, write to /root/k0scfg/helm.json the keys chart_object (the name of the Chart that appeared), chart_uid, chart_version (the version in the Chart status), and namespace_uid (the UID of the podinfo namespace).
k0s converts the declaration into a Chart object in kube-system. Check its naming rule with kubectl get charts. When the chart installation finishes, the version is filled in the Chart's status. The namespace UID is a value you use in the next step to compare 'what remains.'
What remains if you delete the declaration
Delete only the podinfo chart declaration from ClusterConfig, and write to /root/k0scfg/removal.json the keys removed_chart_uid, chart_object_present, release_secrets (the number of Helm release Secrets in the podinfo namespace), and namespace_kept. Then in /root/k0scfg/report.md write the five lines static_restart_needed=, dynamic_file_applied=, dynamic_object_applied= (all three yes/no), object_service_cidr=, and apiserver_service_cidr=, along with an explanation of at least 150 characters.
You may delete only the chart list and leave the repository declaration. It takes a few seconds for the Chart object to disappear. You can find Helm release Secrets by the owner=helm label. In the explanation, include which was the source of truth in which mode and whether you can trust the object's Service CIDR.