Every apply restarted the service

Goal

Control through triggers when the side effects attached inside a declarative tool (restarts and one-time jobs) occur, check the tainted state that a failed operation leaves and the order of replacement, and then prove idempotency with a twice-applied test.

Why it matters

Writing a file gives the same result however many times you do it, but a service restart or a data migration leaves a trace every time it runs. So the condition "run only when something has changed" is the core of idempotency, and in a declarative tool that condition is expressed as a trigger value. If you put a value that changes every time into a trigger, the plan is never empty and the CI twice-applied test breaks, and because a failed operation is marked tainted and runs again on the next apply, the operation itself must be safe to retry. That a replacement happens in the order of stop and then start is also the starting point of zero-downtime design.

Steps

In /root/iac-prov/noisy/main.tf, put a port variable (default 8080); a resource that writes to out/app.conf the line port=<값> (the placeholder is the value), namely local_file.conf; and a resource with triggers_replace = timestamp(), namely terraform_data.restart. When it is created, restart uses local-exec to append to out/restarts.log the single line restart. After init, run apply twice. The grader builds a new plan in this directory and checks whether a change appears every time.
Keep /root/iac-prov/app/main.tf the same as step 1, but change restart's trigger to the hash of the configuration file content (on local_file.conf, its content_sha256). After init, run apply twice and confirm that the plan is clean. The grader builds, in a copy, a plan with port given a different value to check that restart is replaced, and that nothing changes if the value is the same.
Count the lines of /root/iac-prov/app/out/restarts.log, run apply two more times, count again, and write it to /root/iac-prov/app/proof.txt as the single line before=<수> after=<수> (the placeholders are counts). The grader also runs apply twice in a copy to check that the record really does not grow.
In /root/iac-prov/app/terraform.tfvars, put port = 8081 and run apply. out/app.conf must become port=8081, and restarts.log must grow by exactly one line over the after of proof.txt.
To app/main.tf, add terraform_data.migrate. When created, it uses local-exec to check whether ${path.module}/ready.flag exists, and if so, appends to out/migrate.log the single line migrated (if not, the command fails). Apply with ready.flag missing and save the output to /root/iac-prov/app/taint.txt, then save the output of tofu plan to /root/iac-prov/app/taint-plan.txt. The grader also recreates migrate in a copy without ready.flag to see whether it becomes tainted.
Create /root/iac-prov/app/ready.flag and apply. migrate is replaced and this time must succeed, and the plan after that must be clean. restart must not run again in this process (the number of lines in restarts.log is the same as after step 4).
To app/main.tf, add a worker_version variable (default v1) and terraform_data.worker. Both input and triggers_replace are that variable; on creation it appends to out/worker.log the line start <버전>, and with a when = destroy provisioner it appends stop <버전> (the placeholder is the version). After applying with v1, add worker_version = "v2" to terraform.tfvars and apply again. worker.log must be in the order start v1, stop v1, start v2.
Create /root/iac-prov/twice.sh <작업디렉터리> (the placeholder is the working directory). After apply, it checks the second plan with plan -detailed-exitcode, and finishes with idempotent and 0 if it is empty, a line starting with not-idempotent and 2 if changes remain, and a line starting with error and 1 if apply or plan fails. The grader checks with copies of /root/iac-prov/app and /root/iac-prov/noisy, and a temporary directory with a broken configuration.

Notes

The Pod has OpenTofu 1.9.0 and a local provider mirror, so it runs without internet. terraform_data is a built-in resource, so it needs no provider.
Provisioners come in two kinds, creation time (the default) and when = destroy, and a destroy provisioner can reference only self.
Common mistake: deleting the state or mixing the copy with the original so that the restarts.log line-count comparison goes wrong. The grader makes its own copies for grading.
The official documentation also recommends provisioners only as a last resort. Here they are used in order to see with your own eyes when a side effect runs.
Provisioners · lifecycle · tofu plan · Resource Behavior

A restart runs every time you apply

In /root/iac-prov/noisy/main.tf, put a port variable (default 8080); a resource that writes to out/app.conf the line port=<값> (the placeholder is the value), namely local_file.conf; and a resource with triggers_replace = timestamp(), namely terraform_data.restart. When it is created, restart uses local-exec to append to out/restarts.log the single line restart. After init, run apply twice. The grader builds a new plan in this directory and checks whether a change appears every time.

timestamp() is a different value every time it runs, so the trigger always changes. When triggers_replace changes, terraform_data is replaced, and a replacement is a creation, so the creation-time provisioner runs again.

Set the trigger so it restarts only when the configuration changes

Keep /root/iac-prov/app/main.tf the same as step 1, but change restart's trigger to the hash of the configuration file content (on local_file.conf, its content_sha256). After init, run apply twice and confirm that the plan is clean. The grader builds, in a copy, a plan with port given a different value to check that restart is replaced, and that nothing changes if the value is the same.

local_file exports computed attributes such as content_sha256. The value you use for the trigger is the definition of "what changes require a side effect."

Record that the restart log does not grow after two more applies

Count the lines of /root/iac-prov/app/out/restarts.log, run apply two more times, count again, and write it to /root/iac-prov/app/proof.txt as the single line before=<수> after=<수> (the placeholders are counts). The grader also runs apply twice in a copy to check that the record really does not grow.

In an apply with no changes, no resource is created, so the creation-time provisioner does not run either. You can get just the number of lines with wc -l < 파일 (the placeholder is the file).

If the configuration really changes, it restarts exactly once

In /root/iac-prov/app/terraform.tfvars, put port = 8081 and run apply. out/app.conf must become port=8081, and restarts.log must grow by exactly one line over the after of proof.txt.

terraform.tfvars is read automatically. Confirm in the plan that local_file.conf and terraform_data.restart are replaced together.

A failed provisioner leaves the resource tainted

To app/main.tf, add terraform_data.migrate. When created, it uses local-exec to check whether ${path.module}/ready.flag exists, and if so, appends to out/migrate.log the single line migrated (if not, the command fails). Apply with ready.flag missing and save the output to /root/iac-prov/app/taint.txt, then save the output of tofu plan to /root/iac-prov/app/taint-plan.txt. The grader also recreates migrate in a copy without ready.flag to see whether it becomes tainted.

If a creation-time provisioner fails, the resource itself is marked as "not properly finished" even though it was created, and the next plan tries to replace it. You can also see it with tofu state show or in instances[].status of the state JSON.

Fix the cause and apply again, and it is replaced only once

Create /root/iac-prov/app/ready.flag and apply. migrate is replaced and this time must succeed, and the plan after that must be clean. restart must not run again in this process (the number of lines in restarts.log is the same as after step 4).

A tainted resource is replaced on the next apply. For a retry to be safe, the operation must be fine running several times — check in migrate.log what remains if this migrate runs twice.

A replacement is the old one's stop followed by the new one's start

To app/main.tf, add a worker_version variable (default v1) and terraform_data.worker. Both input and triggers_replace are that variable; on creation it appends to out/worker.log the line start <버전>, and with a when = destroy provisioner it appends stop <버전> (the placeholder is the version). After applying with v1, add worker_version = "v2" to terraform.tfvars and apply again. worker.log must be in the order start v1, stop v1, start v2.

A destroy provisioner can reference only itself (self) — using a variable directly is an error. With the default create_before_destroy=false, the old object is deleted first and then the new object is created.

A twice-applied test for CI

Create /root/iac-prov/twice.sh <작업디렉터리> (the placeholder is the working directory). After apply, it checks the second plan with plan -detailed-exitcode, and finishes with idempotent and 0 if it is empty, a line starting with not-idempotent and 2 if changes remain, and a line starting with error and 1 if apply or plan fails. The grader checks with copies of /root/iac-prov/app and /root/iac-prov/noisy, and a temporary directory with a broken configuration.

This is the "team that runs the playbook twice in CI" from the reading, moved to a declarative tool. If you use set -e, it ends early at 2, so capture the exit code and decide from it.