Managing Terraform State for Small Teams: A Practical Guide

Terraform state is one of those things that works fine until it doesn’t. When you are the only person running terraform apply, keeping a local terraform.tfstate file is not a big deal. But once a second person joins and both of you need to run Terraform against the same project, things start breaking quickly — state drift, overwritten resources, or worse, someone running destroy against infrastructure someone else just created.

In this article, we will go through how to set up remote state backends for small teams, when to split your state into multiple files, and the practical gotchas that documentation usually skips over. We will use GCP as our cloud provider, but the concepts apply the same to AWS, Azure, or any other provider.

Why Local State Fails for Teams

When you first start with Terraform, you run terraform apply and Terraform creates a terraform.tfstate file in your working directory. This file is the single source of truth — it maps your Terraform resources to what actually exists in the cloud.

Here is what goes wrong when a team shares code but not state:

1. You run terraform apply on your machine and create a Cloud Storage bucket. Your local state file now says “bucket X exists.”
2. Your teammate pulls the same code, runs terraform apply on their machine. Their local state file is empty, so Terraform tries to create the bucket again — and fails because the bucket already exists.
3. Worse: your teammate makes a change to the bucket’s configuration and applies it. Terraform creates a new state file on their machine that knows about the updated bucket. Your machine still has the old state file. Now neither of you has a complete picture of what exists.

This is the fundamental problem: Terraform state needs to be shared and it needs to be locked so only one person can change it at a time.

Setting Up a Remote Backend with GCS

The most common fix is to store the state file in a shared location. For GCP, that means using a GCS bucket. Terraform has built-in support for this through its gcs backend.

First, create the bucket that will hold your state files. You can do this once manually or through Terraform itself with a local backend for just this bootstrap step:

# bootstrap/main.tf — run once with local backend to create the state bucket
provider "google" {
  project = "my-project"
  region  = "us-central1"
}

resource "google_storage_bucket" "tf_state" {
  name          = "my-team-tf-state"
  location      = "US"
  force_destroy = false
  versioning {
    enabled = true
  }
}

Apply this once with a local backend, and you have your state bucket. Now update your main Terraform configuration to use it:

# main/backend.tf
terraform {
  backend "gcs" {
    bucket = "my-team-tf-state"
    prefix = "terraform/state"
  }
}

Now whenever anyone runs terraform init, Terraform will detect the backend configuration and pull the latest state from GCS. Any plan or apply reads from and writes to this shared location. This solves the sharing problem, but not the locking problem yet.

Why You Need State Locking

Even with a shared remote backend, two people running terraform apply at the same time can still corrupt your state. The state file is not a database — it does not handle concurrent writes gracefully. If two applies happen simultaneously, the second one to finish can overwrite the first one’s changes, or worse, both writes can partially complete and leave you with a broken state file.

State locking ensures that only one person can run apply at a time. For GCS backends, Terraform handles locking automatically — GCS supports object versioning and optimistic concurrency, and Terraform uses conditional requests on the state object to detect conflicts. If someone else has a lock, the second person gets an error telling them to wait.

For AWS, you need a DynamoDB table for locking alongside your S3 backend:

terraform {
  backend "s3" {
    bucket         = "my-team-tf-state"
    key            = "terraform/state"
    region         = "us-east-1"
    dynamodb_table = "terraform-state-lock"
  }
}

Here is a quick comparison of remote backends across clouds:

Backend	State Storage	Locking Mechanism	Setup Effort
GCS	GCS Bucket	Built-in (conditional requests)	Low — just a bucket
S3	S3 Bucket	DynamoDB table (required)	Medium — bucket + table
AzureRM	Storage Account	Built-in (blob leases)	Low
Terraform Cloud	Managed	Built-in	Low (but costs beyond free tier)
Local	Filesystem	None	Trivial (and risky)

When to Split Your State

A single state file works well for small projects. But as you add more resources, two problems appear:

1. Plan times get slower. Terraform has to refresh the state of every resource on every plan. With a hundred resources, this adds up.
2. Blast radius increases. One bad apply on a shared state file can affect infrastructure that multiple people depend on.

A practical rule of thumb: split state when a single terraform plan takes more than 2–3 minutes, or when you find yourself saying “I hope this change doesn’t break X” more than once a week.

Common ways to split state for small teams:

• By environment. Keep dev, staging, and production in separate state files. This is the bare minimum — you should never share state between environments.
• By service or component. Networking (VPC, subnets, firewall rules) in one state, compute (VM instances, GKE clusters) in another, and storage (Cloud SQL, GCS) in a third.
• By lifecycle. Resources that change frequently (application layer) separate from those that rarely change (networking, IAM).

You can split state by using different prefix values or different backend configurations per module. In practice, most teams end up with a directory structure like:

infra/
├── bootstrap/     # creates state bucket, IAM, service accounts
│   └── main.tf
├── networking/    # VPC, subnets, firewall rules
│   ├── backend.tf
│   └── main.tf
├── compute/       # GKE, Cloud Run, VM instances
│   ├── backend.tf
│   └── main.tf
└── data/          # Cloud SQL, GCS, BigQuery
    ├── backend.tf
    └── main.tf

Each subdirectory has its own backend.tf pointing to the same GCS bucket but with a different prefix:

# networking/backend.tf
terraform {
  backend "gcs" {
    bucket = "my-team-tf-state"
    prefix = "networking"
  }
}

# compute/backend.tf
terraform {
  backend "gcs" {
    bucket = "my-team-tf-state"
    prefix = "compute"
  }
}

This way, a change to your database tier does not touch the networking or compute state at all.

Practical Gotchas

Here are things that caught me out in production that are worth knowing upfront:

State file versioning. Enable versioning on your state bucket. If someone corrupts the state file (it happens), you can restore a previous version instead of losing everything. On GCS, this is a checkbox when creating the bucket. On S3, it is a setting on the bucket itself.

Don’t edit state files manually. It is tempting to fix a small issue by opening the JSON and tweaking something. Do not do this. The state file has a serial number and a lineage UUID. If you break either, Terraform will reject the state. Use terraform state mv, terraform state rm, or terraform import instead.

Secrets in state files. Terraform stores everything in state in plaintext — including database passwords, API keys, and any sensitive values passed to resources. This means your state file becomes a secrets management problem. Anyone with read access to your GCS bucket can see every secret you have ever put in Terraform. For production, either encrypt sensitive values at rest (GCS does this by default but access control still matters), or better, use a secrets manager and reference secrets through data sources rather than storing them in config.

CI/CD and state access. When running Terraform from a CI pipeline (GitHub Actions, Cloud Build), the pipeline needs read/write access to the state bucket. Use workload identity federation on GCP or OIDC on AWS instead of storing long-lived service account keys. We covered the basics of this in a previous article about Terraform with GitHub Actions.

State drift detection. Terraform only knows about resources it manages. If someone creates something manually in the console, your state file will not know about it. Run terraform plan regularly (even daily in CI) to catch drift early. For larger setups, some teams use tools like driftctl or built-in drift detection from Terraform Cloud.

Production vs. Development

In a development environment, relaxed state management is sometimes fine. You might use a single state file with a shared GCS backend and no locking. If something breaks, you can always tear everything down and rebuild.

In production, you want:

• Remote backend with locking enabled. Non-negotiable.
• Separate state files per concern. At minimum, separate state per environment.
• Versioned state storage. Easy rollback when things go wrong.
• Read access restricted. Not everyone on the team needs to see the state file directly. Use IAM to scope access.
• Automated drift detection. Even a simple scheduled CI job that runs terraform plan and fails if anything is out of sync can save you from surprises during an outage.

Wrapping Up

Managing Terraform state well is not complicated, but it is one of those things that is easy to skip until it causes a real problem. The three things that matter most for small teams are: use a remote backend (GCS, S3, or equivalent), make sure locking works, and split your state before plan times or blast radius become painful.

Start simple — one state file per environment with a remote backend is already a huge step up from local state. You can always split further when the project grows.