https://en.bioerrorlog.work/entry/terraform-backend-config

This guide explains how to dynamically configure backend settings for storing Terraform state files.

Introduction

I encountered a use case where I needed to dynamically configure backend settings (S3/DynamoDB, etc.) for managing Terraform state files at runtime, rather than hardcoding them in the Terraform configuration.

However, due to Terraform's design, you cannot use any kind of variables—including input variables—in backend configuration blocks.

A backend block cannot refer to named values (like input variables, locals, or data source attributes).

From the Terraform documentation

I found a workaround and am documenting it here for future reference.

# Test environment
$ terraform -version
Terraform v1.5.3
on darwin_amd64

Note: This article was translated from my original post.

How to Dynamically Configure the Terraform Backend

You can specify backend configuration using the -backend-config option during terraform init.

First, prepare your Terraform code with an empty backend block as shown below:

terraform {
  required_version = ">= 1.2.7"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.28"
    }
  }

  # Leave the backend configuration empty
  backend "s3" {}
}

With this configuration in place, you can specify where the state file should be stored remotely (in S3) by providing backend information through the -backend-config option:

terraform init \
  -backend-config="bucket=mybucket" \
  -backend-config="key=backend-demo/terraform.state" \
  -backend-config="region=ap-northeast-1"

Example execution result:

$ terraform init \
  -backend-config="bucket=mybucket" \
  -backend-config="key=backend-demo/terraform.state" \
  -backend-config="region=ap-northeast-1"

Initializing the backend...

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding hashicorp/aws versions matching ">= 4.28.0"...
- Installing hashicorp/aws v5.75.0...
- Installed hashicorp/aws v5.75.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

The remote state was successfully initialized with S3, and terraform apply worked as expected afterward.

Note that the values passed to the -backend-config option can be specified either as inline strings in the command or via a configuration file.

terraform init -backend-config=backend.hcl

# backend.hcl
bucket = "mybucket"
key    = "backend-demo/terraform.state"
region = "ap-northeast-1"

Conclusion

I've documented a method for specifying Terraform backend configuration via command-line options.

I hope this helps someone facing a similar challenge.

[Related Articles]

en.bioerrorlog.work

Dynamically Configuring Terraform Backend Settings

Introduction

How to Dynamically Configure the Terraform Backend

Conclusion

References