Marked sensitive, yet the password sat in the state file

Goal

Find for yourself where a password created by the tool remains in plain text in the state, the JSON output, and the saved plan; migrate OpenTofu's state and plan encryption from an existing plaintext state and enforce it; and then build a setup that keeps the secret outside the state.

Why it matters

Sensitive is only a marker that hides the value on the screen a person sees, and the value goes as it is into the records the tool stores. The state file and the plan files that CI uploads effectively become a secret store, and if you do not know this, you leave the state anywhere, under the mistaken belief that "it's hidden, so it's safe." There are three layers of defense: encryption at rest (where the key becomes the key to the state), forcing it so that it cannot fall back to plain text, and a design that does not hand the value to the tool in the first place, passing only a reference.

Steps

In /root/iac-secret/main.tf, put random_password.db (length 20, no special characters); a resource that writes that value to out/db.conf as password=<값> (the placeholder is the value), namely local_sensitive_file.dbconf; and an output with sensitive = true, namely db_password. After init and apply, confirm that tofu output shows the value masked.
Find every place in the state file that holds the password value and write the JSON paths to /root/iac-secret/leak.txt, one per line. A path is in the form of keys and array numbers joined by dots (for example, resources.0.instances.0.attributes.x). The grader does the same calculation on the state obtained with tofu state pull and compares.
Save the output of tofu output to /root/iac-secret/masked.txt. Then save a plan with tofu plan -out=/root/iac-secret/leak.tfplan, count how many times the password appears in the tfstate entry inside the plan file (a zip), and write it to /root/iac-secret/planleak.txt as planleak=<수> (the placeholder is the count).
In /root/iac-secret/encryption.tf, declare a sensitive variable state_passphrase and state and plan encryption: a pbkdf2 key provider (the passphrase is that variable), an aes_gcm method, and, for reading the existing plaintext, an unencrypted method set as the fallback for state and plan. The passphrase is labhub-state-passphrase-01, and you pass it with the environment variable TF_VAR_state_passphrase. After apply, terraform.tfstate must be encrypted (the file has no password and has encrypted_data).
In encryption.tf, delete the unencrypted method and the two fallbacks and put enforced = true on state and plan. With the correct passphrase the plan must be clean. With a wrong passphrase (TF_VAR_state_passphrase=wrong-passphrase-000000), run tofu plan and save the output to /root/iac-secret/wrongkey.txt. The grader checks in a copy that reading actually fails with the wrong key.
With the correct passphrase, save tofu plan -out=/root/iac-secret/safe.tfplan. This file must not contain the password in plain text, and it must not be a zip either. Confirm that with the key, tofu show -json safe.tfplan works, and without the key, it fails.
In /root/iac-secret/ref/secret/db.key, create a secret with openssl rand -hex 16 and set its permission to 600 (a secret created outside the tool). In /root/iac-secret/ref/main.tf, put, without random_password, only a resource that writes to out/db.conf the single line password_file=<그 경로> (the placeholder is that path), namely local_file.dbconf, and run init and apply. The grader checks that the value of db.key is nowhere in ref's state and only the path is there.

Notes

The Pod has OpenTofu 1.9.0 and local and random provider mirrors, so it runs without internet. From step 4 on, first run export TF_VAR_state_passphrase='labhub-state-passphrase-01' in each shell.
After you turn on encryption, tofu state pull shows the decrypted state. If you open the file directly, you see only encrypted_data.
Ephemeral resources and write-only attributes, which leave no value in the state or plan at all, arrived in OpenTofu 1.11. They are not in 1.9.0 in this Pod, so the reference approach is used instead.
Common mistake: turning on encryption without a fallback in step 4 so that the existing plaintext state cannot be read. Common mistake: writing the passphrase as a default in main.tf and committing it to the repository.
Sensitive Data in State · State and Plan Encryption · Ephemerality · Terraform: Sensitive data

Create a password and mask the output

In /root/iac-secret/main.tf, put random_password.db (length 20, no special characters); a resource that writes that value to out/db.conf as password=<값> (the placeholder is the value), namely local_sensitive_file.dbconf; and an output with sensitive = true, namely db_password. After init and apply, confirm that tofu output shows the value masked.

The result of random_password is marked as a sensitive value by the provider. To expose that value as an output, you must also attach sensitive to the output for the plan to pass. local_sensitive_file only keeps the content from being shown on screen; it writes the file the same.

Where does the masked secret sit in the state file

Find every place in the state file that holds the password value and write the JSON paths to /root/iac-secret/leak.txt, one per line. A path is in the form of keys and array numbers joined by dots (for example, resources.0.instances.0.attributes.x). The grader does the same calculation on the state obtained with tofu state pull and compares.

You can get the value with tofu output -raw db_password (which is also evidence that the masking is only a display method). With jq's paths(scalars) and getpath you can find every place with the same value. There is more than one place.

The masking is only on screen — JSON output and saved plans

Save the output of tofu output to /root/iac-secret/masked.txt. Then save a plan with tofu plan -out=/root/iac-secret/leak.tfplan, count how many times the password appears in the tfstate entry inside the plan file (a zip), and write it to /root/iac-secret/planleak.txt as planleak=<수> (the placeholder is the count).

A saved plan file is a zip, and inside it is a copy of the state from when the plan was made. Look at the entries with unzip -l and extract and count with unzip -p <파일> tfstate (the placeholder is the file). If CI uploads the plan file as an artifact, that artifact becomes a secret too.

Move a plaintext state to an encrypted state

In /root/iac-secret/encryption.tf, declare a sensitive variable state_passphrase and state and plan encryption: a pbkdf2 key provider (the passphrase is that variable), an aes_gcm method, and, for reading the existing plaintext, an unencrypted method set as the fallback for state and plan. The passphrase is labhub-state-passphrase-01, and you pass it with the environment variable TF_VAR_state_passphrase. After apply, terraform.tfstate must be encrypted (the file has no password and has encrypted_data).

If you merely turn on encryption, the tool refuses to read the existing plaintext state (because it could be tampered plaintext). So for one round, put in a fallback that lets it read the plaintext, apply, and have it rewritten with the new method. The passphrase must be at least 16 characters.

When the migration is done, remove the plaintext fallback and enforce

In encryption.tf, delete the unencrypted method and the two fallbacks and put enforced = true on state and plan. With the correct passphrase the plan must be clean. With a wrong passphrase (TF_VAR_state_passphrase=wrong-passphrase-000000), run tofu plan and save the output to /root/iac-secret/wrongkey.txt. The grader checks in a copy that reading actually fails with the wrong key.

If you leave the fallback in place, a plaintext state that someone slips in will still be read. enforced blocks writing in plain text itself. If you lose the key you can never read the state again, so keeping the key is keeping the state.

An encrypted plan file cannot be read without the key

With the correct passphrase, save tofu plan -out=/root/iac-secret/safe.tfplan. This file must not contain the password in plain text, and it must not be a zip either. Confirm that with the key, tofu show -json safe.tfplan works, and without the key, it fails.

Compare the first two bytes with head -c 2, side by side with the leak.tfplan from step 3. A zip starts with PK. Plan encryption is decided by the plan block, separately from state.

Keep the secret outside the state and put only a reference in the state

In /root/iac-secret/ref/secret/db.key, create a secret with openssl rand -hex 16 and set its permission to 600 (a secret created outside the tool). In /root/iac-secret/ref/main.tf, put, without random_password, only a resource that writes to out/db.conf the single line password_file=<그 경로> (the placeholder is that path), namely local_file.dbconf, and run init and apply. The grader checks that the value of db.key is nowhere in ref's state and only the path is there.

If the tool creates or reads a value, that value goes into the state. The simplest way around it is a setup where the side that uses the value (the application) reads it directly from a file or a secret store, and the tool only tells it the location. Note that even reading a secret file through a data source leaves it in the state.