Loop With Globs, Return With Exit Codes

In a nutshell

Shell functions do not return values. They report true or false through the exit code and send values out through standard output. And if you parse ls to get a list of files, it will surely break someday.

Why this was needed

for f in $(ls *.log) is easy to learn, and so it is widely used. But this one line has three traps stacked together.

If you take the output of ls through command substitution, it is split into words at spaces. access 2026.log becomes two.
If no files match, ls prints an error, and that error message becomes the input of the loop.
If a file name contains a newline or glob characters, the behavior is unpredictable.

The right answer is to let the shell do it directly.

for f in /var/log/*.log; do
  [ -f "$f" ] || continue
  ...
done

The key here is [ -f "$f" ] || continue. If nothing matches, the glob pattern remains as a literal string, so without this line of defense the loop tries to process a file that does not exist, called /var/log/*.log. This trap also appears in the actual lab.

How it works

Use the three forms of iteration as the situation calls for.

for x in a b c - a fixed list
for f in 경로/*.log - iterating over files, which the shell expands directly
while IFS=, read -r name role; do ... done < users.csv - line-by-line input

In the third form, putting IFS=, in front of read applies the delimiter to that command only. -r means not to interpret backslashes as escapes, and in practice you always add it.

A function's interface consists of the following three things.

What	Where
A computed value	Standard output (`echo`)
Success/failure, true/false	Exit code (`return 0` / `return 1`)
A message for a person	Standard error

If a predicate function like is_number 123 prints its result with echo yes, you cannot use it as in if is_number "$x". A judgment must always be reported through the exit code.

The most important thing in handling arguments is the difference between "$@" and $*. "$@" expands each argument as if each were wrapped in quotes, while $* joins everything into one string. In places like a retry function that re-runs a command exactly as given, it must be "$@".

Variables inside a function are global by default. If you do not add local, it silently overwrites a variable of the same name in the caller. It is good to form the habit of declaring every variable created inside a function as local, without exception.

What it looks like in the field

Splitting out a library. If several scripts copy and paste the same function, there are several places to fix. If you gather the functions in lib.sh and load them with . /root/bin/lib.sh, you only fix one place. source and . are the same command, and because they run in the current shell without creating a new process, the function definitions remain.

External commands inside a loop eat away performance. If you call $(echo "$v" | sed ...) inside a loop over 1000 lines, 2000 processes are created. Doing the same thing with ${v//old/new} creates 0 processes. Half of the impression that shell scripts are slow comes from here.

When you need parallelism, use xargs -P or & + wait. However, to avoid missing failures in the parallel jobs, you must collect each PID and check the status one by one with wait "$pid".

Where scripts are silently wrong

The shell moves on to the next line even when it meets an error. So it is common for a script to be in a state where it ends as if it succeeded while doing nothing. There are several mechanisms to prevent this, and you need to use each one knowing what it prevents.

set -e stops when a command fails. But there are many exceptions. It does not stop on failure inside a conditional, on the left of && or ||, or after !. And if something that failed inside a function is called as part of a conditional, the whole function does not stop. So you must not rely on set -e alone, and you should add explicit checks in important places.

set -u stops when an undefined variable is used. It prevents the accident in which a single typo makes an empty string so that rm -rf "$PREFX/" becomes rm -rf /. However, in places where there may be no argument, you need to write a default such as ${1:-}.

set -o pipefail changes the exit code of a pipeline to the last one among those that failed. Without it, in curl ... | jq ..., even if curl fails, the whole pipeline succeeds if jq succeeds. This is the trap you saw in the previous course.

Turning on all three together is the convention, but after turning them on, run the script once and check that there are no places where it stops unintentionally. In particular, if you add set -u to an existing script, it suddenly dies in branches that are not normally used.

And finally, guarantee cleanup. If you set trap 'rm -rf "$TMP"' EXIT, no temporary files are left even if it fails midway. If you want to keep them only on failure, it is better to delete explicitly on the success path, and either way, deciding in advance what to keep for debugging and what to delete is how you avoid filling the disk later.

What you will do in the next lab

Starting with a loop that prints numbers, you will go through glob iteration, reading a CSV, splitting out functions and source, judging by exit code, safely handling arguments that contain spaces, and summing variable arguments, and finally build a script that takes a directory and produces a summary.