Narrowing the Cause From Logs

This lab runs on a real VM

This box is not a Pod but a virtual machine launched by KubeVirt. A separate Linux kernel runs in it, systemd actually manages services, and docker is a real Docker engine, not an imitation. A container started with docker run becomes a real process, and both docker exec and docker logs work as usual.

This lab used to run inside a Pod, and that box had dropped every kernel capability, so all the steps that start a container were blocked. Now no step is blocked.

There are two things to know.

• The first start takes a little over a minute. The VM boots and installs Docker, so it is slower than a Pod lab (usually 40 seconds).
• There is no browser preview. Only one grading port is open for connections into the VM. If you start a web server, check it with curl from inside the VM.

Goal

You diagnose a live container and a dead container, and get used to the order of narrowing down the cause with three things: logs, exit code and state fields.

Why it matters

Container diagnosis is hard not because there is no information but because there is no order in which to look. With an order, most cases are finished within 3 minutes. The reason to look at the logs first is that they are the last words the app left on its own, and the reason to look at the exit code next is that when the log is empty (that is, it died instantly from a signal), it is the only clue. You will use this same order later in Kubernetes. Only the tool names change.

Steps

1. Create /root/dk3 and make the dk-noisy container print line-1 through line-50, then print done-50 at the end and exit.
2. Save only the last 5 lines of the dk-noisy log to /root/dk3/tail5.txt.
3. Create a dk-err container that sends to-stdout to standard output and to-stderr to standard error, then save only standard output to /root/dk3/out.txt and only standard error to /root/dk3/err.txt, separately.
4. Add timestamps to the dk-noisy log and save only the first line to /root/dk3/ts.txt.
5. Using nginx:1.27-alpine, start dk-web2, check the nginx version inside it, and save it to /root/dk3/nginx-version.txt.
6. Create a dk-crash container that prints starting and then fails by reading a nonexistent path. Then, in /root/dk3/crash.txt, write one line: exit=<실제 종료코드> (with the actual exit code in place of the placeholder).
7. In the dk-noisy log, count how many times line- appears and write only the number to /root/dk3/count.txt.
8. In /root/dk3/triage.md, write one line each for the three containers dk-noisy, dk-crash and dk-web2, putting the container name and its current state in each line, and also the exit code in the dk-crash line.

Notes

• Use docker logs --tail N and docker logs -t.
• The nginx version comes out on standard error. You have to capture it with 2>&1 for it to land in the file.
• Common mistake 1: if you add --rm in step 1, no container is left to look at logs from.
• Common mistake 2: in step 7, check whether done-50 also contains line-. Grading recounts the actual log and compares.

Create a container that floods the log

Create /root/dk3 and make the dk-noisy container print line-1 through line-50, then print done-50 at the end and exit.

Print many lines with a loop and leave a completion marker at the end. The last line appears only if the container runs to the end.

Look at only the last few lines

Save only the last 5 lines of the dk-noisy log to /root/dk3/tail5.txt.

When a log is long, you need to look at the end, not the beginning. There is an option that limits the number of output lines.

Separate standard output and standard error

Create a dk-err container that sends to-stdout to standard output and to-stderr to standard error, then save only standard output to /root/dk3/out.txt and only standard error to /root/dk3/err.txt, separately.

docker logs passes each of the two streams through as it is. You can filter out one at a time with shell redirection.

Find out when a log line was written

Add timestamps to the dk-noisy log and save only the first line to /root/dk3/ts.txt.

There is an option that adds timestamps. Save only the first line, which is prefixed with an RFC3339 time.

Check inside a running container

Using nginx:1.27-alpine, start dk-web2, check the nginx version inside it, and save it to /root/dk3/nginx-version.txt.

nginx prints its version to standard error. If you forget the redirection, the file may look empty.

Diagnose a failed container

Create a dk-crash container that prints starting and then fails by reading a nonexistent path. Then, in /root/dk3/crash.txt, write one line: exit=<실제 종료코드> (with the actual exit code in place of the placeholder).

If you look at the last line of the log together with the exit code, the cause can be pinned down. Check the exit code with inspect.

Count a pattern in the log

In the dk-noisy log, count how many times line- appears and write only the number to /root/dk3/count.txt.

You may count directly through a pipe without moving the log to a file. Leave only the exact number in the file.

Summarize the state of the three containers

In /root/dk3/triage.md, write one line each for the three containers dk-noisy, dk-crash and dk-web2, putting the container name and its current state in each line, and also the exit code in the dk-crash line.

Check the actual state and exit code of each container with inspect and write one line each. A container that died must also have its exit code.