Terraform Cloud dynamic workspaces

Dynamic Workspaces with Hashicorp Terraform Cloud and Terraform Enterprise

At the time of writing this article ( terraform 0.14 ) providers cannot have dependencies. So we gonna assume the following use case for this article:

We want to create a gitlab instanace with a cloud provider. The desired state for this has to include the following requirements

• Create a Gitlab instance based on a VM Template
• Disable Signup
• Disable AutoDevOps
• Set project default visibility
• Set a root password

As second requirement we want a desired state (customization) of our gitlab instance after some basic configuration stuff by

• Create projects used as templates on forks
• Create a number of users
  • with a certain prefix ( e.g. "git_user_XX" )
• Set default values on user settings
  • password
  • email
  • project limit

Project Setup

We can seperate those desired state parts into 2 modules - or better said we are forced to seperate these into 2 terraform projects because providers cannot have dependencies. If those two desired state parts are within a single terraform projects, the plan phase will already fail because the terraform provider will try to interact with the gitlab instance which is not available at this time.

When running this tf-project on your local machine you are able to workaround this problem by using the -target option of terraform. The target option allows you to run parts of terraform code to create a certain resource with all its dependencies.

The target option is representing a workaround which can cause unexpected side-effects. Be aware that this option must be used carefully!!

In our current example we assume that each desired state is represented with a distinct module.

 1resource "random_password" "gitlab_root_password" {
 2  length           = 16
 3  special          = true
 4  override_special = "_%@"
 5}
 6
 7module "gitlab_instance" {
 8    source = "modules/instance"
 9    hostname           = "gitlab"
10    gitlab_version     = "13.1.4"
11    domain             = "infralovers.com"
12    auto_devops        = false
13    signup             = false
14    root_password      = random_password.gitlab_root_password.result
15}
16
17module "gitlab_customize" {
18    source             = "modules/customize"
19    hostname           = module.gitlab_instance.hostname
20    domain             = module.gitlab_instance.domain
21    users              = 20
22    user_prefix        = "git_user"
23    project_limit      = 5
24}

So on your local machine you are able to run following commands:

1terraform apply -auto-approve -target="module.gitlab_instance"

This will create the terraform instance and not interact with gitlab. And follwoing command will then run the customize module:

1terraform apply -auto-approve

Terraform Cloud/Terraform Enterprise

But most likely switch to run your apply in Terraform Cloud or Terraform Enterprise to allow multiple users/administrators to interact with the terraform code and also have a verified handling of your terraform state.

In this case the -target Option is not available anymore and you need to use another solution to overcome this problem:

• Trigger a run in a second workspace by run triggers
• Dynamically create a workspace by terraform

Both solutions use the modules as seperate workspace. We gonna asume that these are in two subdirectories within an git repository.

Terraform Cloud Run Triggers

Terraform Cloud and Terraform Enterprise have the option of run triggers available. With these triggers you can trigger a run in another workspace or even workspaces. The option can be very handy in setups where resources are always available, but not in our exapmle.

In our example this option can create an invalid terraform state in the customize workspace which can only be fixed by deleting the workspace and recreate it. This terraform state can be created, when you destroy the gitlab instance before destroying the customize projects. In this case the customize state is still representing that n users were created in a gitlab instance, which is already not available anymore.

Dynamic Workspace Creation

To overcome this problem you can also use the Terraform Enterprise provider which is able to handle Terraform Cloud resources. In our use case we gonna create a workspace which calls the module from the example code above.

 1resource "tfe_workspace" "gitlab_customize" {
 2  name                  = "GitLab_customize"
 3  organization          = "Infralovers"
 4  auto_apply            = true
 5  file_triggers_enabled = true
 6  terraform_version     = "0.14.2"
 7  working_directory     = "customize"
 8  vcs_repo {
 9    identifier     = "infralovers/gitlab-setup"
10    oauth_token_id = var.gitlab_oauth_token
11  }
12}

The customize terraform code is now updated to:

 1data "terraform_remote_state" "gitlab" {
 2  backend = "remote"
 3
 4  config = {
 5    organization = "Infralovers"
 6    workspaces = {
 7      name = "GitLab"
 8    }
 9  }
10}
11module "gitlab_customize" {
12    source             = "modules/customize"
13    hostname           = data.terraform_remote_state.gitlab.outputs.hostname
14    domain             = data.terraform_remote_state.gitlab.outputs.domain
15    users              = 20
16    user_prefix        = "git_user"
17    project_limit      = 5
18}

In our example it is also ok to destroy the workspace which is used for customization because it is not necessary to delete users from a gitlab instance, which will be deleted a few seconds later.