I Ran Destroy Without Realizing prod Was Selected
Goal
When you split environments with workspaces, you open up where the state is actually created, cause a wrong-selection incident yourself and then block it with a guard, and build the same thing with directory separation to compare the two approaches in numbers.
Why it matters
The reason you reach for workspaces as the first tool for splitting environments is clear — there is one copy of the code, one provider cache, and you switch with one line of command. But those advantages are the risk itself. One backend means the person using dev and the person using prod open the same store with the same credentials, and 'which environment am I in now' is written not on the screen but only in a single file inside the working directory. This is why the official documentation says not to use workspaces for system decomposition or for deployments where credentials split. So the practical choice is not 'which of the two is right' but 'what do you split with what' — temporary copies used by the same team go into workspaces, and boundaries where permissions split go into directories.
Steps
- In
/root/tfa-ws/main.tf, put the local provider andlocal_file.env. The file name isout/<workspace 이름>.conf, and the contents areworkspace=<workspace 이름>on the first line andreplicas=<local.replicas>on the second line (where the placeholders stand for the workspace name). In/root/tfa-ws/sizes.tf, put onlyreplicas = 1inlocals. Init and apply. - Create the
devandprodworkspaces and apply in each. When you finish, there must be three files in/root/tfa-ws/out/and two state files under/root/tfa-ws/terraform.tfstate.d/. - In
/root/tfa-ws/where.tsv, write the three workspaces, one per line, in name order (default, dev, prod). The three tab-separated columns are<workspace 이름>,/root/tfa-ws 기준 상태 파일 상대경로, and그 상태의 lineage(in order: the workspace name, the relative path of the state file from /root/tfa-ws, and the lineage of that state). - Edit
/root/tfa-ws/sizes.tfso that it takes the current workspace's value from thelocal.sizesmap (default = 1,dev = 2,prod = 6) withlookupand uses it aslocal.replicas. If you apply again in all three workspaces, the three files in out/ have different replicas values. - With
prodselected, runtofu destroy -auto-approveto cause the incident yourself. Leave the output in/root/tfa-ws/incident/prod-destroy.log, and the name of the workspace that was selected at that time in/root/tfa-ws/incident/selected.txt(copy.terraform/environmentas is). Then write the four linesselected:,command:,lost:, andrecovery:in/root/tfa-ws/incident.md, and recover by applying prod again. - Create
/root/tfa-ws/guard.sh. It must exit with 0 and print a confirmation message if the name received as an argument equals the currently selected workspace, exit with 1 showing the currently selected name if they differ, and exit with 2 if there is no argument. It does not change the workspace. - In
/root/tfa-ws/modules/app/main.tf, put a module that takesvar.envandvar.replicasand writes/root/tfa-ws/out-dir/<env>.conf(where the placeholder stands for the environment name), and call that module from the two directories/root/tfa-ws/envs/devand/root/tfa-ws/envs/prodwithenv="dev" replicas=2andenv="prod" replicas=6respectively. Init and apply the two directories separately. Do not use workspaces. - In
/root/tfa-ws/compare.tsv, write the five lines below with two tab-separated columns. Do not make up the values; count them on the disk now.workspace_states= the number of state files the workspace approach has,directory_states= the number of state files under envs/,workspace_plugin_dirs= the number of.terraformdirectories of the workspace approach,directory_plugin_dirs= the number of.terraformdirectories under envs/,selected_marker= the relative path of the file that remembers the currently selected workspace.
Notes
- The Pod has OpenTofu 1.9.0 and a local provider mirror, so it runs without the internet. All backends in this lab are local.
- In the local backend, the default workspace's state is terraform.tfstate in the working directory, and other workspaces go under terraform.tfstate.d. The path rules differ by backend.
- Common mistake: calling workspace new again for a name that already exists in step 2. You must switch with select.
- Common mistake: fixing only sizes.tf in step 4 and running apply in only one workspace. The state is separate per workspace, so you must apply to each.
- Workspaces · Managing Workspaces (CLI) · Command: workspace · Backend: local
Where is the default workspace's state?
In /root/tfa-ws/main.tf, put the local provider and local_file.env. The file name is out/<workspace 이름>.conf, and the contents are workspace=<workspace 이름> on the first line and replicas=<local.replicas> on the second line (where the placeholders stand for the workspace name). In /root/tfa-ws/sizes.tf, put only replicas = 1 in locals. Init and apply.
Within the configuration, you read the currently selected workspace name with terraform.workspace (in OpenTofu too, the name is still terraform). The default workspace's state does not yet go to a special directory — just run ls on the working directory after apply.
If you create two more workspaces, where does the state go?
Create the dev and prod workspaces and apply in each. When you finish, there must be three files in /root/tfa-ws/out/ and two state files under /root/tfa-ws/terraform.tfstate.d/.
tofu workspace new both creates and selects. If the name already exists it errors, so switch with select. Check with find at which path the state of a non-default workspace is created, and also see whether the lineages of the three states differ from each other.
Write down which workspace's state is which file
In /root/tfa-ws/where.tsv, write the three workspaces, one per line, in name order (default, dev, prod). The three tab-separated columns are <workspace 이름>, /root/tfa-ws 기준 상태 파일 상대경로, and 그 상태의 lineage (in order: the workspace name, the relative path of the state file from /root/tfa-ws, and the lineage of that state).
The state file is JSON, so read the lineage with jq. The key point of this step is that the default workspace's path and the other two's paths are different. Do not put ./ in front of the relative path.
Make the same code have different values per environment
Edit /root/tfa-ws/sizes.tf so that it takes the current workspace's value from the local.sizes map (default = 1, dev = 2, prod = 6) with lookup and uses it as local.replicas. If you apply again in all three workspaces, the three files in out/ have different replicas values.
The third argument of lookup decides what should happen when a name that is not in the map is selected. You may reference another local inside locals. After changing the value, you must run apply once in each workspace for the files to be updated.
Issue a command with the wrong selection
With prod selected, run tofu destroy -auto-approve to cause the incident yourself. Leave the output in /root/tfa-ws/incident/prod-destroy.log, and the name of the workspace that was selected at that time in /root/tfa-ws/incident/selected.txt (copy .terraform/environment as is). Then write the four lines selected:, command:, lost:, and recovery: in /root/tfa-ws/incident.md, and recover by applying prod again.
A workspace remembers the current selection in one file inside the working directory — open .terraform/environment. The incident is not 'typing the command wrong' but 'nothing on the screen shows where it is being issued.' In the lost line, write the path of the file that disappeared, and in the recovery line, the command you used to bring it back.
Put a guard in front of write commands
Create /root/tfa-ws/guard.sh. It must exit with 0 and print a confirmation message if the name received as an argument equals the currently selected workspace, exit with 1 showing the currently selected name if they differ, and exit with 2 if there is no argument. It does not change the workspace.
The currently selected name comes out with the single line tofu workspace show. The reason to split exit codes into three is that in CI you need to distinguish 'it was wrong' from 'it was called wrongly.' The grader calls this script directly in three ways.
Build the same thing with directory separation
In /root/tfa-ws/modules/app/main.tf, put a module that takes var.env and var.replicas and writes /root/tfa-ws/out-dir/<env>.conf (where the placeholder stands for the environment name), and call that module from the two directories /root/tfa-ws/envs/dev and /root/tfa-ws/envs/prod with env="dev" replicas=2 and env="prod" replicas=6 respectively. Init and apply the two directories separately. Do not use workspaces.
The content format of the module file is the same two lines as in step 1 (workspace= and replicas=). The goal is to produce the same result without workspaces. After apply, check that each of the two directories has its own state and its own provider cache.
Compare the two approaches in numbers
In /root/tfa-ws/compare.tsv, write the five lines below with two tab-separated columns. Do not make up the values; count them on the disk now.
workspace_states = the number of state files the workspace approach has, directory_states = the number of state files under envs/, workspace_plugin_dirs = the number of .terraform directories of the workspace approach, directory_plugin_dirs = the number of .terraform directories under envs/, selected_marker = the relative path of the file that remembers the currently selected workspace.
Count the four numbers with find (do not include those under envs/ in the workspace side's numbers). The last line is the very file you copied in step 5. These five lines are the answer to 'why can't a workspace create a credential boundary' — because the backend and the cache are both one copy.