Declarative Infrastructure and the State File as a Ledger

In one line

Terraform is a tool in which you write not "run this command" but "when it is done, this infrastructure should exist." To keep that promise, the tool carries a ledger called the state file.

Why this was needed

When you create servers by clicking in the console, three things vanish at once. A way to recreate the same environment, a record of who changed what and when, and confidence that development and production are the same. Up to ten servers people get by from memory, but a hundred is impossible. The configuration drift in which dev, stage, and prod subtly diverge begins at exactly this point.

So if you move to shell scripts, a new problem appears. At first it was the single line "create it," but on the second run an "already exists" error appears and you attach a conditional, and since attributes may have changed, you attach a comparison. In the end half of the script becomes code that checks the current state. A declarative tool takes that half away entirely. People write only the desired shape, and calculating the difference between the current and desired is the tool's job.

Here the real question arises. How does the tool know the "current state"? Scanning the whole cloud every time is slow, and above all, among that many resources there is no way to tell which ones I created. The answer to this problem is the state file.

How it works

plan compares three values.

Value	Where it is	Meaning
Desired state	.tf code	What a person declared
Last known state	terraform.tfstate	The result of the last apply
Actual state	Provider API	What really exists now

It first queries the actual (refresh) to update the state, then compares the updated state with the code to make the plan. So if there is no state file, the tool judges it to be "infrastructure I have never seen" and tries to create everything anew. This is why the state file is a more dangerous asset than the infrastructure itself.

The state file is JSON, and only a few fields are actually read in practice.

{
  "version": 4,
  "serial": 42,
  "lineage": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "resources": [
    { "mode": "managed", "type": "local_file", "name": "hello",
      "instances": [ { "attributes": { "filename": "/root/out/hello.txt" },
                       "dependencies": ["random_pet.suffix"] } ] }
  ]
}

What matters here is that attributes is a record of the last applied result. If the value written in the state diverges from the value of the actual resource, that itself is the definition of drift. So when reading the state, you need the habit of also checking "is this value the same as the actual?"

version is the state file format version, currently 4. serial is a number that goes up every time the state changes and is used in a remote backend to catch concurrent modification conflicts. lineage is a unique identifier that represents the lineage of this state file and prevents the accident of accidentally overwriting another project's state. Inside resources, the type and name joined together form local_file.hello, the address we work with.

Meanwhile, init reads required_providers in the code to download providers and writes the confirmed versions and hashes into .terraform.lock.hcl. This lock file must be committed to the repository. That is how my laptop and CI use the same providers.

What you see in the field

First, the accident of committing the state file to Git. The state contains database passwords and tokens in plain text. Moreover, if two people each apply, merge conflicts arise, and a wrongly merged state can make you lose resources entirely. So in practice you keep it in a remote backend such as S3 with versioning turned on, and block concurrent applies with a lock. On AWS, the S3 + DynamoDB combination was the standard for a long time.

Second, the day a state with a different lineage was overwritten. If a team writes the backend key wrongly and puts its own state on top of another stack's state, the next plan says "I will delete hundreds of things." What saves you then is the earlier version in S3 versioning. lineage is a field that exists so that such accidents are noticed in advance.

Third, secrets in the state. OpenTofu supports state and plan file encryption in the tool itself. To get the same level in Terraform you need a paid service. The command names and subcommand structure are almost identical, so what you learn with tofu in this lab carries over to terraform as it is.

What you will do in the next lab

In /root/tf/first, you download the provider, initialize, and declare and apply one file. Then you add a resource that makes a random name, change the declaration and apply again, and see with your own eyes serial go up. You create a temporary resource and then delete only that one, and extract the list of addresses remaining in the state. Finally, instead of copying a value, you reference it so that the dependency is recorded in the state.