Variable Expansion Is Not String Substitution

In a nutshell

The shell splits words after it replaces variables with their values. So if you leave out a single pair of double quotes, one argument breaks into several, and that accident first shows up the day you meet a file name that contains a space.

Why this was needed

A deployment script ran fine for six months. One day someone uploaded a file named My Report.pdf, and the script tried to delete two things, My and Report.pdf, and failed. Not one character of the code had changed.

The reason lies in the order in which the shell processes things. When it meets rm $FILE, the shell first substitutes $FILE with its value and then splits that result into words again at spaces. If you wrap it in double quotes, as in rm "$FILE", this splitting does not happen. There is one rule: always use double quotes when you use a variable. It is faster to just add them than to spend time thinking about exceptions.

How it works

Parameter expansion finishes default-value handling and required-value validation in a single line. You need to tell the three forms apart.

Syntax	What it does	Does it keep the value in the variable?
${VAR:-기본값}	If empty, uses the default instead	No
${VAR:=기본값}	If empty, assigns the default	Yes
${VAR:?메시지}	If empty, prints the message and exits	-

You use the same forms to make function arguments mandatory. One line, local name="${1:?사용법: deploy.sh <앱이름>}", does both the argument validation and the usage message.

String manipulation also works without external commands. Instead of starting two processes with echo | sed, using the expansion syntax is much faster.

file="/var/log/nginx/access.log"
${file##*/}      # access.log      (앞에서 최장 매칭 제거 = 경로 떼기)
${file%.*}       # /var/log/nginx/access   (뒤에서 최단 매칭 제거 = 확장자 떼기)
${file%.log}.bak # /var/log/nginx/access.bak
${VER^^}         # 대문자로
${var//old/new}  # 전부 치환

There are three syntaxes for condition tests, and each has a different purpose.

• [ ... ] - the POSIX standard. In scripts that run under sh, this is the only one you can use.
• [[ ... ]] - a bash extension. Word splitting and globbing do not happen, so it is safer, and you can also use regular expressions with =~.
• (( ... )) - arithmetic only. Numeric comparison is natural, as in (( retries > 3 )).

For file tests, use -f (regular file), -d (directory), -s (size greater than 0), -r (readable), -x (executable), and -e (exists) according to what you need. -e is also true for a directory, so if you want only files you should use -f.

What it looks like in the field

Environment variables are the standard channel for configuration. In 12-factor style deployments, configuration is passed through environment variables rather than files. So a script takes the form "use the environment variable if it exists, and otherwise run with a reasonable default". That one line is : "${LOG_LEVEL:=info}".

Send error messages to standard error. Successful results go to stdout, and errors and usage messages go to stderr. Only when you keep this convention does a usage such as capturing only the result with result=$(script.sh) work. If the usage message comes out on stdout, it immediately becomes the result value.

Exit codes have conventions too. 0 is success, 1 is a general failure, 2 is a usage error, and if you follow the sysexits convention, 64 is a usage error and 66 is a missing input file. More important than the values themselves is using them consistently within a project and writing them down in documentation.

Where quotes are unnecessary, and where they are useless

"Always double quotes" is the right rule, but if you know why, you will not get lost in the exceptional places.

Where quotes are not needed. Inside [[ ... ]], word splitting and globbing do not happen, so [[ -n $VAR ]] is safe. In VAR=$OTHER, which puts a value in a variable, splitting also does not happen on the right side of an assignment. The value of knowing these two is not that you may leave the quotes out, but that it becomes clear why they are needed in other places.

Where quotes are useless. Double quotes only prevent word splitting, and variable expansion and command substitution still happen inside them. So if you pass an untrusted string to eval or bash -c, the $(...) inside it is executed no matter how many quotes you add. For such places, you must change to a form in which that string is not interpreted as a command in the first place, rather than relying on quotes.

Single quotes expand nothing. So things the shell must not touch, such as regular expressions or awk programs, are wrapped in single quotes. Conversely, to use a variable inside them you end up closing and reopening the single quotes briefly, and when it gets that complicated it is usually better to pass the value as an environment variable or an argument.

Finally, a word on arrays. It is common to keep an argument list in one variable and expand it later, but if you store it as a string it always breaks on an argument that contains a space. If you store it in a bash array and expand it with "${ARGS[@]}", you get the same property as the "$@" seen earlier. It is also a form that is safe when the array is empty, which is especially valuable when assembling a command that may have no arguments depending on conditions.

What you will do in the next lab

Starting with a greeting script, you will handle argument defaults and the priority of environment variables, a branch that identifies the file type, boundary values for score grades, and finally a deployment script that validates environment variables and reports its result through the exit code.