vim Search and Substitute

Goal

You filter and substitute in a log file inside vim with ex commands, and save those commands to files so the work is reproducible. The material is /opt/lab/data/app.log (480 lines, in the format IP 날짜 메서드 경로 상태코드 [ERROR]: IP, date, method, path, status code, then an optional ERROR marker).

Why it matters

Every ex command has a range + action structure. :%s/.../.../g, :5,20s/... and :'<,'>s/... are three cases of the same rule. Add the line filters :g (lines that match the pattern) and :v (lines that don't), and most log processing is done. Two things deserve particular care: without the g flag only the first match on each line is replaced, and :g and :v are exact opposites. These two are the typical causes of results that are wrong without any error.

Steps

Work in the /root/vimp directory. Put only editing commands in the .ex files; do not include save or quit commands.

1. Copy /opt/lab/data/app.log to /root/vimp/work.log.
2. Count how many lines contain /api/pay and write just the number to /root/vimp/count.txt.
3. Write the line number of the last line whose status code is 500 to /root/vimp/last500.txt.
4. Save the command that replaces the last octet of the IP with xxx to /root/vimp/mask.ex, and its result to /root/vimp/masked.log. (10.1.2.3 becomes 10.1.2.xxx)
5. Save the command that deletes the lines that do not contain ERROR to /root/vimp/errors.ex, and the result to /root/vimp/errors_only.log.
6. Save the command that deletes only the lines ending in 200 to /root/vimp/drop200.ex, and the result to /root/vimp/no200.log.
7. Save the command that deletes the second field (the date) to /root/vimp/nodate.ex, and the result to /root/vimp/nodate.log.
8. Put the commands that apply all three of the above into /root/vimp/report.ex, and save the result to /root/vimp/report.log. The final form is 10.1.2.xxx GET /api/item 500 ERROR.

Notes

• :%s/패턴//gn changes nothing and only reports the number of matches. Use it to check the reach before you substitute.
• A back-reference is the syntax for recalling a parenthesized part; the whole match is &.
• The grader applies the .ex file to a copy of the original and checks that it matches the result file.
• Common mistake 1: using :g in step 5 gives the exact opposite result.
• Common mistake 2: writing something like :g/200/d in step 6 also removes lines that have 200 in the IP or in another field.

Make a working copy

Copy /opt/lab/data/app.log to /root/vimp/work.log.

The basic practice is to leave the original read-only and work on a copy.

Count matches only

Count how many lines contain /api/pay and write just the number to /root/vimp/count.txt.

If you add the n flag to a substitution command, it reports the count without changing anything. A pattern that contains a slash needs escaping.

Find the last 500 response

Write the line number of the last line whose status code is 500 to /root/vimp/last500.txt.

Go to the end of the file with G, then search backward with ?. You can see the current line number with Ctrl-G or :set number.

Mask the IP with a substitution

Save the command that replaces the last octet of the IP with xxx to /root/vimp/mask.ex, and its result to /root/vimp/masked.log. (10.1.2.3 becomes 10.1.2.xxx)

Wrap the part to keep in parentheses and recall it with a back-reference. Only the last octet should change.

Keep only the ERROR lines

Save the command that deletes the lines that do not contain ERROR to /root/vimp/errors.ex, and the result to /root/vimp/errors_only.log.

:g and :v are opposites. Think about which one deletes the lines that don't match.

Remove the normal responses

Save the command that deletes only the lines ending in 200 to /root/vimp/drop200.ex, and the result to /root/vimp/no200.log.

If you forget to anchor to the end of the line, it also catches a 200 in another field.

Delete the date column

Save the command that deletes the second field (the date) to /root/vimp/nodate.ex, and the result to /root/vimp/nodate.log.

If you capture the first two fields together and bring back only the first, the second disappears.

Three transformations in one script

Put the commands that apply all three of the above into /root/vimp/report.ex, and save the result to /root/vimp/report.log. The final form is 10.1.2.xxx GET /api/item 500 ERROR.

Just put the three commands you wrote earlier in order. The result can change with the order, so check it.