Migrating from remote-state to backend in Terraform 0.9 - amazon-s3

Question

I downloaded terraform 0.9 and tried to follow the migration guide to move from remote-state to backend
But it doesn't seem to work. I replaced:
data "terraform_remote_state" "state" {
backend = "s3"
config {
bucket = "terraform-state-${var.environment}"
key = "network/terraform.tfstate"
region = "${var.aws_region}"
}
}
with
terraform {
backend "s3" {
bucket = "terraform-backend"
key = "network/terraform.tfstate"
region = "us-west-2"
}
}
yet when I run terraform init in one of my environment folders, I get:
Deprecation warning: This environment is configured to use legacy
remote state. Remote state changed significantly in Terraform 0.9.
Please update your remote state configuration to use the new 'backend'
settings. For now, Terraform will continue to use your existing
settings. Legacy remote state support will be removed in Terraform
0.11.
You can find a guide for upgrading here:
https://www.terraform.io/docs/backends/legacy-0-8.html
I also had to drop the variable interpolation since this is not allowed anymore. Does that mean that one S3 Bucket is used for multiple environments? What have I missed here?

Answer 1

Per upgrade guide (https://www.terraform.io/docs/backends/legacy-0-8.html) after terraform init you also have to run terraform plan to finalize the migration, which will update the remote state file at s3.
As for configuring for multiple environments we ended up using a wrapper shell script with passing in parameters for ${application_name}/${env}/${project}, and using partial configuration.
For a project structure like this:
├── projects
│   └── application-name
│   ├── dev
│   │   ├── bastion
│   │   ├── db
│   │   ├── vpc
│   │   └── web-cluster
│   ├── prod
│   │   ├── bastion
│   │   ├── db
│   │   ├── vpc
│   │   └── web-cluster
│   └── backend.config
└── run-tf.sh
for each application_name/env/component = folder (i.e. dev/vpc) we added a placeholder backend configuration file like this:
backend.tf:
terraform {
backend "s3" {
}
}
Where the folder content for each component will look like this:
│   ├── prod
│   │   ├── vpc
│   │   │   ├── backend.tf
│   │   │   ├── main.tf
│   │   │   ├── outputs.tf
│   │   │   └── variables.tf
At "application_name/" or "application_name/env" level we added a backend.config file, like this:
bucket = "BUCKET_NAME"
region = "region_name"
lock = true
lock_table = "lock_table_name"
encrypt = true
Our wrapper shell script expects parameters application-name, environment, component, and the actual terraform cmd to run.
The content of run-tf.sh script (simplified):
#!/bin/bash
application=$1
envir=$2
component=$3
cmd=$4
tf_backend_config="root_path/$application/$envir/$component/backend.config"
terraform init -backend=true -backend-config="$tf_backend_config" -backend-config="key=tfstate/${application}/${envir}/${component}.json"
terraform get
terraform $cmd
Here is how a typical run-tf.sh invocation looks like:
$ run-tf.sh application_name dev vpc plan
$ run-tf.sh application_name prod bastion apply

Answer 2

You got confused with terraform remote command with remote-state. You dont have to change any remote-state things you have in your tf files.
Instead of configuring your remote state with terraform remote command and use backend config file mentioned in the migration link.
See the second github comment in this link. It has nice step by step procedure on what he did to migrate.
https://github.com/hashicorp/terraform/issues/12792