enable and start Are Different Questions — systemd, nginx, iptables, bond, chrony, podman

In one line

Half of Linux administration is asking "does it work now?" and "will it work next time?" separately. systemctl start and enable, sysctl -w and /etc/sysctl.d, iptables -A and iptables-save, ip link and netplan — in each pair, the first is for now and the second is for the next boot. The LFCS requires both, and the labs in this module grade both.

Why this exists

The earlier modules (lfcs-operating-systems, lfcs-networking) ran in a Pod, so all you could do was write unit files and verify them with systemd-analyze verify. You could not see whether a timer really runs, whether nginx really passes requests to the backend, or whether an iptables rule drops packets. This module runs as root on an Ubuntu 24.04 VM. The grader looks at the system as it is now with systemctl is-active, curl, iptables -S, /sys/class/net, sysctl -n, and podman inspect.

There is one boundary. The VM has only DNS and public internet 80/443 open to the outside, and UDP 123 is blocked. So chrony cannot synchronize even if it knows a server, and that step is graded by the configuration, the service state, and the output files. And because the grading agent comes in on 8899/tcp, if you change the INPUT default policy of iptables to DROP or block 8899 or 22, grading cannot reach the VM from that moment.

How it works

A timer is a unit that calls a service. According to systemd.timer(5), if you omit Unit=, a .timer unit activates the .service with the same name. OnBootSec= is when it first runs after boot, and OnUnitActiveSec= is how often after that service was last activated. The one being called must be Type=oneshot from systemd.service(5) to be treated as "a job that runs once and ends". And enable looks at WantedBy=timers.target in the [Install] section and creates a symbolic link, while start turns it on now. --now does both at once. That is-enabled and is-active can give different answers is the whole point of this step.

A reverse proxy receives at the front and passes to the back. proxy_pass of ngx_http_proxy_module is that one line. The Ubuntu package puts configuration in sites-available and turns it on with a symbolic link in sites-enabled. The default site also holds 80 as default_server, so if you do not remove it, nginx -t rejects the duplicate. For the backend, python3 -m http.server is enough, but if you start it by hand with &, it dies along with the session when the session drops. Make it a unit and let systemd hold on to it.

Netfilter is tables and chains. In iptables(8), the nat table changes addresses and the filter table decides pass or block. PREROUTING is the moment a packet comes in (before routing), INPUT is what is addressed to this host, and OUTPUT is what this host creates. So a packet you send to yourself does not pass through PREROUTING — you cannot see DNAT with curl localhost:8080. DNAT in iptables-extensions(8) changes the destination with --to-destination, and REDIRECT is a special form of it. DROP discards without a response, so the client sees not a refusal but a timeout. Rules disappear on reboot, and the iptables-save output is the material for the next boot. Also be aware that iptables on Ubuntu 24.04 runs on top of the nf_tables backend.

Bridges and bonds. A bridge is a software switch and a bond ties several links into one. With ip-link(8) you create type dummy|bridge|bond and put members in with master. active-backup in the kernel bonding documentation is a mode that uses only one and switches to another if it dies, so it needs no switch configuration, and a slave must be down before it is added. On a VM with only one real NIC, you make the members from dummy interfaces — to the kernel they are real interfaces. Writing the same configuration declaratively is the bridges and bonds sections of netplan YAML, and this lab keeps it outside /etc/netplan and does not apply it. If you lost the real NIC, there would be no way back.

Time and time zone are different things. The kernel clock is always UTC and the time zone is a display rule. set-timezone of timedatectl changes the latter, and server … iburst of chrony.conf sets the former. iburst is an option that sends packets in a burst right after start to speed up the first synchronization. If the first column of sources in chronyc is ?, no response has been received yet, and on this VM that is normal.

A container image is a tar. podman-import(1) turns a tarball into a one-layer image. When you see by hand that a single /bin/busybox becomes an image with no Dockerfile and no registry, what an image is stays with you. Podman in this lab runs as root and runs with no network configuration using --network none of podman-run(1).

You do sysctl twice. -w of sysctl(8) changes the kernel now, and files of sysctl.d(5) are read at boot. sysctl --system applies the standard directories in order, so if you run it after writing the file, both now and the next boot are right. As proc_loadavg(5) explains, the load average is the average number of tasks that are running or waiting to run (and waiting on disk I/O), and it means something only when you compare it with the number of CPUs.

What it looks like in the field

The systemd version of "I put it in cron but it doesn't run" is "I only ran start on the timer". If it is not in systemctl list-timers after a reboot, look at is-enabled first. Conversely, if you only enable and do not start, nothing runs until the next boot.

"I added a firewall rule and it disappeared after a reboot" is the classic iptables problem. The rules are in kernel memory, and restoring a saved file at boot is the job of a separate service (on Ubuntu, rules.v4 of iptables-persistent). That is why the lab ends by saving iptables-save to a file.

When you get a report that a node is "slow", you divide the load average in uptime by nproc. 4.0 on 4 cores is full, and 4.0 on 1 core means three are waiting. If wa in top is large, it is disk, and if st is large, a hypervisor neighbor is eating the CPU.

What you will do in the next lab

A oneshot service + a 1-minute timer (enable, start, logs) → a Python backend unit + an nginx proxy (default site disabled) → DNAT 8080→80 and DROP on 9999 (leaving 8899 and 22 alone) + iptables-save → br0 and bond0 on dummies and a netplan document → chrony server, time zone, and output files → git init, branch, and --no-ff merge → podman import a busybox tar and run it → a sysctl file + --system → uptime, top, and NPROC. At every step, take care of both "now" and "the next boot". The grader looks at both.