Mutable vs. immutable infrastructure, and lifecycle in Terraform

Note: This post was written in Spanish and translated into English with AI assistance. Read the original.

The previous post covered the CLI commands. This one goes back to configuration blocks, and explains something that has shown up in every lab without us spelling it out: why Terraform, almost always, destroys a resource and creates a new one instead of updating it in place.

Mutable infrastructure

A server running PostgreSQL 14 that you upgrade to 15 one day, and later to 16, without replacing the machine, is mutable infrastructure: the same server keeps changing state over time. The upgrade can be done by hand during a maintenance window or with a configuration management tool like Ansible, but either way the server is the same one before and after.

If you have several servers behind a load balancer and upgrade each one separately, the usual risk is that one of them gets stuck halfway: a network issue, a disk space problem, or a dependency that isn’t where the script expects it. That server stays on the old version while the others get updated. Over time, after several rounds of updates, you end up with a group of servers that should be identical but aren’t. This is called configuration drift, and the longer it takes to notice, the harder it gets to figure out which server is running which version of what.

Immutable infrastructure

The alternative is to never touch a server once it’s deployed. To go from PostgreSQL 14 to 15 you create new servers running 15, verify they work, and only then retire the old ones. If something goes wrong, the old servers are still there, untouched.

This doesn’t eliminate drift, it makes it harder to produce: there’s no window where a half-upgraded server sits alongside the rest. Either the new server was created correctly, with the right version, or it wasn’t created and the old one keeps serving traffic. It also makes rolling back simpler: if 15 causes problems, you point traffic back at the servers running 14 instead of trying to undo a half-finished upgrade.

Terraform replaces by default

Terraform follows the immutable model unless the provider can update the resource without recreating it. This post’s lab starts from a random_pet and a local_file whose content references the name generated by the first one (the same implicit dependency from the attributes and outputs post):

resource "random_pet" "office" {
  length    = 2
  separator = "_"
}

resource "local_file" "notice" {
  filename = "${path.module}/notice.txt"
  content  = "Office pet of the month: ${random_pet.office.id}"
}

separator can’t be changed on an existing random_pet, so a change there forces a replacement. And since local_file.notice depends on the pet’s name, the replacement cascades:

$ terraform apply
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # local_file.notice must be replaced
-/+ resource "local_file" "notice" {
      ~ content              = "Office pet of the month: optimum_colt" -> (known after apply) # forces replacement
      ~ id                   = "8b716eaf9955c51228fcc9acecce0f57b50c3c0c" -> (known after apply)
        # (3 unchanged attributes hidden)
    }

  # random_pet.office must be replaced
-/+ resource "random_pet" "office" {
      ~ id        = "optimum_colt" -> (known after apply)
      ~ separator = "_" -> "-" # forces replacement
        # (1 unchanged attribute hidden)
    }

Plan: 2 to add, 0 to change, 2 to destroy.
local_file.notice: Destroying... [id=8b716eaf9955c51228fcc9acecce0f57b50c3c0c]
local_file.notice: Destruction complete after 0s
random_pet.office: Destroying... [id=optimum_colt]
random_pet.office: Destruction complete after 0s
random_pet.office: Creating...
random_pet.office: Creation complete after 0s [id=pet-tomcat]
local_file.notice: Creating...
local_file.notice: Creation complete after 0s [id=38cb0f5caf4afdec009e54b9e05902601e10aa87]

Apply complete! Resources: 2 added, 0 changed, 2 destroyed.

The plan flags # forces replacement next to each responsible attribute. Not every change forces a replacement: most attributes update without recreating the resource (~ without -/+), and which ones do depends on what the provider supports, not on a general Terraform rule. That’s documented attribute by attribute in each provider.

When the default behavior doesn’t work

Replacing by default is fine most of the time, but there are resources where that’s exactly what you don’t want: a database that can’t disappear even for an instant, or a resource that shouldn’t be deletable by accident. The lifecycle block changes that default behavior for a specific resource.

create_before_destroy

With the default order (destroy, then create) there’s a window without the resource. create_before_destroy reverses the order: it creates the replacement first, and only destroys the original once it exists.

resource "local_file" "notice" {
  filename        = "${path.module}/notice.txt"
  content         = "Office pet of the month: ${random_pet.office.id}"
  file_permission = "0600"

  lifecycle {
    create_before_destroy = true
  }
}

With this rule active, a permission change produces a different plan: +/- create replacement and then destroy, instead of -/+. The order of actions in the output changes too, Creating... first, then Destroying... of the old object, which at this point Terraform calls deposed because it’s no longer the active resource, just the old copy pending removal:

$ terraform apply
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+/- create replacement and then destroy

Terraform will perform the following actions:

  # local_file.notice must be replaced
+/- resource "local_file" "notice" {
      ~ file_permission      = "0644" -> "0600" # forces replacement
      ~ id                   = "38cb0f5caf4afdec009e54b9e05902601e10aa87" -> (known after apply)
        # (3 unchanged attributes hidden)
    }

Plan: 1 to add, 0 to change, 1 to destroy.
local_file.notice: Creating...
local_file.notice: Creation complete after 0s [id=38cb0f5caf4afdec009e54b9e05902601e10aa87]
local_file.notice (deposed object e81c538f): Destroying... [id=38cb0f5caf4afdec009e54b9e05902601e10aa87]
local_file.notice: Destruction complete after 0s

Apply complete! Resources: 1 added, 0 changed, 1 destroyed.

This works without any extra effort here because a local file can just be overwritten at the same path. With many cloud resources the problem is different: if the name has to be unique (a GCS bucket, for example), you can’t create the new one under the same name while the old one is still alive. The usual fix is to have the name include something that changes on every replacement, a random suffix or a hash of the content, so both can coexist for as long as the apply takes.

prevent_destroy

Blocks any plan that would destroy the resource, whether it comes from a terraform destroy or from a change that forces a replacement.

resource "random_pet" "office" {
  length    = 2
  separator = "-"

  lifecycle {
    prevent_destroy = true
  }
}

$ terraform destroy
random_pet.office: Refreshing state... [id=pet-tomcat]

Terraform planned the following actions, but then encountered a problem:

  # random_pet.office will be destroyed
  - resource "random_pet" "office" {
      - id        = "pet-tomcat" -> null
      - length    = 2 -> null
      - separator = "-" -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

Error: Instance cannot be destroyed

  on main.tf line 1:
   1: resource "random_pet" "office" {

Resource random_pet.office has lifecycle.prevent_destroy set, but the plan
calls for this resource to be destroyed. To avoid this error and continue
with the plan, either disable lifecycle.prevent_destroy or reduce the scope
of the plan using the -target option.

The protection is Terraform’s, not the provider’s: it’s Terraform refusing to generate the plan. If someone deletes the resource by hand from the console or the provider’s API, prevent_destroy doesn’t stop it and won’t notice until the next plan. To actually remove the protection you have to edit the .tf file and delete the block, and that ends up in the git history: whoever removes protection from a resource leaves a trace of having done it.

ignore_changes

Tells Terraform to ignore changes in certain attributes when comparing the plan against reality. It’s for attributes managed by something else outside Terraform, that you don’t want Terraform trying to revert on every apply; the typical case is an autoscaling group whose instance count goes up and down with load, while Terraform keeps managing the rest of the configuration:

resource "google_compute_instance_group_manager" "app" {
  # ...
  target_size = 3

  lifecycle {
    ignore_changes = [target_size]
  }
}

Without this rule, every apply would reset target_size back to 3 and fight with the autoscaler. With ignore_changes = [target_size], Terraform still sees and applies the rest of the resource’s changes, but leaves that attribute as it is in the actual infrastructure.

Other pieces of the lifecycle block

create_before_destroy, prevent_destroy and ignore_changes cover most cases, but the block has more arguments: replace_triggered_by forces a resource to be replaced when another resource or attribute it depends on changes, even if its own configuration hasn’t; precondition and postcondition check conditions before and after creating the resource and fail the apply with a custom message if they aren’t met. These are out of scope for this post; they’re documented in the official lifecycle block reference.

Summary

• Mutable infrastructure: the same server changes version over time, with a risk of drift if an update fails halfway.
• Immutable infrastructure: a deployed resource is never modified, it’s replaced by a new one and the old one is retired only once the new one works.
• Terraform replaces by default when the provider can’t update the resource without recreating it; the plan flags the responsible attribute with # forces replacement.
• create_before_destroy reverses the order to create-then-destroy, to avoid a window without the resource; with unique names, the name needs to change on every replacement.
• prevent_destroy fails any plan that would destroy the resource; it doesn’t protect against changes made outside Terraform.
• ignore_changes excludes specific attributes from the comparison between plan and reality, to coexist with something that manages them externally.

The full lab is in the terraform-zero-to-hero repository, under labs/07-immutable-infra.