Michal Checinski

Saving Terraform state file on Azure Storage

October 5, 2020 , posted under Terraform Azure
Saving Terraform state file on Azure Storage

Terraform has its state. This state contains a definition for the current state of the resources managed by Terraform. Terraform uses this ‘state’ for tracking infrastructure and its changes. When you create a resource using Terraform, it will save the connection between this resource and appropriate Terraform config describing this resource. If you then delete or change this resource, Terraform will check the state of the resource in its state and modify or delete the resource in Azure accordingly.

You can read more about the state in the Terraform documentation.

Why remote state

You can use the state file stored on your machine. And if you followed the steps in my previous Terraform with Azure article you did. It’s okay for learning or when you work on the infrastructure by yourself. But if you work on a team, all other members will need access to this file. Or you are using a CI/CD automations. They will also read the remote state file. Another benefit of the remote state file is that when one person/worker is doing changes in the infrastructure, Terraform will lock the state, and that way not allow modifying the infrastructure by other people/workers.

Why the Terraform need the state file to be stored in the central location? Cannot it just pull the info that it needs when doing operations on the infrastructure?

You could ask. And you’re right. I also don’t like the one centralized tfstate file. You can read why it’s needed in the docs: Purpose of Terraform State.

Storing Terraform state file on Azure Storage

If we use Azure as our cloud provider which is managed by Terraform, the simplest way is to use Azure Blob Storage to store Terraform’s state file. But it’s not only option. You can use the other cloud-provider’s storage solution (eg. AWS S3). You can use dedicated service from Terraform, called Terraform Cloud Remote State Management.

But as we are managing Azure resources let’s stick to the Azure Storage for keeping Terraform state file.

1. Create Azure Storage for Terraform State

There are two ways of creating Azure Storage and blob container in it to keep state file:

  • Using script (Az Powershell module or Azure CLI)
  • Using Terraform

Let’s go them one by one.

Using script: Azure CLI

Firstly take a look on the script and then let’s describe it.

The script takes 3 variables: RESOURCE_GROUP_NAME for a resource group name that will hold Azure Storage, STORAGE_ACCOUNT_NAME actual Azure Storage account name (it uses $RANDOM bash constant/function), CONTAINER_NAME for an Azure Blob Storage container name. Then the script creates a resource group and storage account. After creating them, the script queries for the Storage account key. Then using this key, Azure Blob storage container is created. Scripts output the three values that we will need: storage account name, container name, and storage account access key.

Using Terraform

You need to create new main.tf file with the content:

provider "azurerm" {
  version = "~>2.5.0"
  features {}
}

# Resource group name variable
variable "rg_name" {
  type = string
  description = "Resource group name"
  default="tfstate2" # Default value for resource group name
}

# Random integer
resource "random_integer" "tfstate" {
  min     = 1
  max     = 50000
}

# Storage account name variable
variable "st_name" {
  type = string
  description = "Storage account name"
  default="tfstate" # Default value for resource group name using random integer
}

# Blob Storage container name variable
variable "container_name" {
  type = string
  description = "Blob Storage container name"
  default="tfstate" # Default value for Blob Storage container name
}

# Create Azure resource group
resource "azurerm_resource_group" "tfstate" {
  name     = var.rg_name
  location = "eastus"
}

# Create Azure Storage account
resource "azurerm_storage_account" "tfstate" {
  name                     = "${var.st_name}${random_integer.tfstate.result}"
  resource_group_name      = azurerm_resource_group.tfstate.name
  location                 = azurerm_resource_group.tfstate.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

# Create Blob storage container
resource "azurerm_storage_container" "tfstate" {
  name                  = var.container_name
  storage_account_name  = azurerm_storage_account.tfstate.name
  container_access_type = "private"
}

output "storage_account_name" {
  value       = "${var.st_name}${random_integer.tfstate.result}"
  description = "Storage account name"

  depends_on = [
    azurerm_storage_account.tfstate
  ]
}

output "container_name" {
  value       = var.container_name
  description = "Blob Storage container name"

  depends_on = [
    azurerm_storage_container.tfstate
  ]
}

output "access_key" {
  value       = azurerm_storage_account.tfstate.primary_access_key
  description = "Storage access key"

  depends_on = [
    azurerm_storage_account.tfstate
  ]
}

Then you need to run those commannds that you should be familiar to after reading my previous Terraform with Azure article:

terraform init
terraform apply -auto-approve

If after running approve the output will be similar to this:

Error by Terraform after creating Azure storage with name that already exists

it will mean that the Azure Storage with that name already exists. What you need to do is to remove Terraform state from your computer using rm terraform.tfstate and re-run the terraform apply -auto-approve. Deleting a Terraform state file terraform.tfstate will cause the random number to be regenerated, and will fix the problem.

Either you choose to create the Azure Storage you will end up with three values: storage account name, container name, and storage account access key.

You will need those values, so copy them to notepad, or remember another way.

Set Terraform config to use created Azure storage as a place for Terraform state file

Now when we have Azure storage created we need to set it as a place to keep Terraform state file. To do this you need to insert this portion of Terraform code in your main.tf Terraform config file, in the project that you want to use Azure storage as the state file storage:

terraform {
  backend "azurerm" {
    resource_group_name  = "tfstate"
    storage_account_name = "tfstate24216"
    container_name       = "tfstate"
    key                  = "terraform.tfstate"
  }
}

Here you can see the parameters populated with my values. You need to change resource_group_name, storage_account_name and container_name to reflect your config. If you used my script/terraform file to create Azure storage, you need to change only the storage_account_name parameter. The last param named key value is the name of the blob that will hold Terraform state. You can change accordingly to keep many states in one storage and container, eg. test.terraform.tfstate or projectName.terraform.tfstate etc.

The last thing we need to set is Storage access key. This key will allow Terraform to access the state file saved on Azure Blob storage and modify it. The storage access key was in the output of both: script and Terraform. There are two options to give Terraform access to storage:

  • set the key in the Terraform file
  • set it in the environment variable

There is another option to access the storage account from terraform using Managed Identity (formerly Managed Service Identity (MSI)). But I won’t cover it in this post.

I prefer the second method, but I will show you both.

Storage account access key in the Terraform file

To provide storage account key in the terraform file you need to provide additional parameter named access_key. So the provider config will look like that:

terraform {
  backend "azurerm" {
    resource_group_name  = "tfstate"
    storage_account_name = "tfstate24216"
    container_name       = "tfstate"
    key                  = "terraform.tfstate"

    access_key           = "qwertyuiop7486879hdtct6c7wchw=="
  }
}

Storage account access key in the environment variable

The second option (preferred) is to set the environment variable. The variable name is ARM_ACCESS_KEY. So setting it in bash would look like that:

export ARM_ACCESS_KEY="qwertyuiop7486879hdtct6c7wchw=="

And that’s it. Now you can use the terraform apply command and check if the terraform.tfstate file will be saved in the Azure storage in tfstate container.

In todays post I showed you how to:

  • create Azure Storage account and blob storage container using Azure CLI and Terraform
  • add config to Terraform file to tell it to use Azure storage as a place for keeping state file
  • Give Terraform access (using the storage key) to access Azure Storage account to write/modify Terraform state file.

I hope you enjoyed my post. If you know someone who will benefit from knowledge in this post, please share it to them.

Featured image by JOSHUA COLEMAN on Unsplash