TT Lab
Blog Get started
Learn Learning paths Courses

Shell Scripting

Scripts That Survive Hostile Filenames

Continue in TT Lab

Goal

You will write scripts that withstand file names that contain spaces, newlines, or start with a dash, and even options and injection strings. For each rule, you check the broken version and the fixed version side by side, and at the end you clean up after an incident in which a cleanup script deleted the wrong files last night.

Why it matters

A script that mishandles arguments usually runs fine for months and then, on the day a name containing a space or newline first appears or one configuration name changes, deletes what it must not delete. The investigation that day is reconstructing "what arguments that rm actually received", and the cause may differ for each damaged file. After fixing it, check again with hostile names you have never seen, not with this time's names, so that the same incident does not come back under a different name.

Environment

You work under /root/args. You create the materials yourself.

mkdir -p /root/args/data && cd /root/args/data
: > "my report.tmp"
: > "$(printf 'two\nlines').txt"
: > "-rf"
: > "normal.txt"
ls -b

The third one is the key. You can create a file named -rf. And, as with the second one, names can also contain newlines. The only character that cannot appear in a file name is NUL.

What you will build

/root/args/split.txt    따옴표 유무에 따른 인자 수
/root/args/nul.txt      줄 단위와 NUL 단위로 센 개수
/root/args/clean.sh     파일 하나를 이름이 무엇이든 안전하게 지운다
/root/args/wrap.sh      받은 인자를 쪼개지 않고 그대로 넘긴다
/root/args/opts.sh      getopts 로 옵션을 판다
/root/args/search.sh    입력을 값으로만 쓴다
/srv/media/bin/purge.sh (사건) 어떤 이름·설정에서도 엉뚱한 것을 지우지 않게 고친다

How grading works

From step 4 on, the grader calls your scripts directly with hostile input.

clean.sh   "-rf" 라는 이름을 인자로 직접 넘겨 지워지는지
wrap.sh    "a b" "c" "d  e" 를 넘겨 3개로 도착하는지
opts.sh    -v -o 값 / 붙여 쓴 -vo 값 / 값 빠진 -o / 모르는 -Z
search.sh  "alpha; touch /tmp/…" 를 넣고 그 파일이 생기는지
purge.sh   사건에 없던 이름(공백·줄바꿈·대시)이 든 임시 트리 / 보관함 이름이 빠진 설정

Steps

  1. Create four kinds of hostile names.
  2. Count the number of arguments when quotes are added to the same value and when they are left out, and write it down.
  3. Write down the count when you count the same directory line by line and when you count it with -print0. The two numbers differ.
  4. Write clean.sh. It takes one file name as an argument and deletes it. If the file does not exist, it must end with a nonzero code.
  5. Write wrap.sh. For each argument it receives, it prints ARG:<값> on one line (the value goes in place of the placeholder).
  6. Write opts.sh. It accepts -v and -o <값> (a value) and prints the number of remaining arguments. The output has the form verbose=0 out= rest=0.
  7. Write search.sh. It takes a pattern and a file and searches. Do not use eval.
  8. Incident incident sh-wrong-delete: last night the cleanup script deleted three files it must not delete. For each file, identify the cause, restore it from the snapshot, and fix the script so it never deletes the wrong thing under any name or configuration, then run it once to clean up today's files.

Notes

Create hostile names

Create four kinds of hostile names.

Names with a space, a newline, or starting with a dash, plus one ordinary name. If you look with ls -b, special characters appear escaped.

One quote changes the number of arguments

Count the number of arguments when quotes are added to the same value and when they are left out, and write it down.

Pass the value with and without quotes to the built-in command that resets the positional arguments, then look at the special variable that holds the number of arguments, and you get a number. Also write one line on why it turns out that way: the shell splits into words after expanding.

Counting by lines is wrong, counting by NUL is right

Write down the count when you count the same directory line by line and when you count it with -print0. The two numbers differ.

A file whose name contains a newline is counted as two when read line by line. find -print0 splits on NUL, the only character that cannot appear in a file name, so it is safe.

When a name starts with a dash

Write clean.sh. It takes one file name as an argument and deletes it. If the file does not exist, it must end with a nonzero code.

rm -f "$f" reads the name as an option if it is -rf. -- is the marker meaning "from here on, arguments". For a file that does not exist, it must end with a nonzero code, because rm -f succeeds even when the file is absent.

Pass on the received arguments as they are

Write wrap.sh. For each argument it receives, it prints ARG:<값> on one line (the value goes in place of the placeholder).

"$@" preserves each argument separately, and $* joins them into one. An unquoted $@ is also split again. The grader passes "a b" and "d e" to check that the spaces survive.

Do not build an option parser yourself

Write opts.sh. It accepts -v and -o <값> (a value) and prints the number of remaining arguments. The output has the form verbose=0 out= rest=0.

A hand-rolled parser falls apart on a joined form such as -vo 값 (the Korean word means "value"). In the reading, check what the colon after an option that takes a value and the leading colon in getopts' option string each change (quiet error mode and the two branches you get then). If you do not shift the positional arguments by the number of options processed after the loop ends, the count of remaining arguments is wrong; getopts has a variable that holds the next position it will look at.

Input is a value, not code

Write search.sh. It takes a pattern and a file and searches. Do not use eval.

eval executes a string as shell code. The grader passes alpha; touch <파일> as the pattern (with a file path in place of the Korean word) and checks whether that file gets created. If you only block that and the normal search dies, that is also a failure.

Incident: the cleanup script deleted three wrong files

Running incident sh-wrong-delete raises the incident. Last night /srv/media/bin/purge.sh, which runs daily (it deletes *.tmp under upload and *.old in the archive), deleted /srv/media/upload/budget, /srv/media/upload/logo.png, and /srv/media/nginx.conf.old, and left some of the .tmp files it should have deleted. (1) Restore the three from /srv/media/.snapshot/ with the same content (leave the snapshot as it is), (2) fix purge.sh so it is safe for any name, and so that if the archive name ARCHIVE_SUB is missing from the configuration (/srv/media/purge.conf) it deletes nothing and ends with a nonzero code, and (3) correct the configuration and run it once to clean up today's *.tmp and archive/*.old. The grader also runs your fixed purge.sh on a temporary tree containing names that were not in this incident. In /root/incident/sh-wrong-delete/report.txt, write three lines, budget: <코드>, logo.png: <코드>, and nginx.conf.old: <코드>, plus evidence: <관찰한 출력 한 줄> (one line of output you observed). Each file's code is one of the following: word-split (an unquoted expansion was split into words), newline-name (a newline inside a name was read line by line), empty-var (an empty variable shortened the path), glob (*, ?, or [ inside a name was expanded), dash-option (a name starting with - was read as an option).

Direction: for each lost file, recreate "the arguments that the rm that deleted it actually received". A value being split into words, names being read line by line, and an empty variable shortening a path are different defects, and the places to fix them are different too. Tools: ls -b, cat -A, bash -x, NUL separation (-print0). Concretely: look at the two stages of purge.sh separately: the loop that reads find's output and the rm arguments inside it, and which line of purge.conf the variable that builds the second rm's path comes from. ls -b makes spaces and newlines visible. After fixing, run it yourself with new names containing spaces, newlines, and a leading dash, and with a configuration missing the archive name.