HashiCorp Vault GitHub Secrets Sync with Terraform

Secrets Distribution

The problem domain of secrets distribution is a common one. You have a secret, you want to distribute to a service, and you want to implement this action in a secure way. The secret could be a password, an API key, a TLS certificate, or anything else that you want to keep secret.

Also a common problem within secrets distribution is the need to rotate secrets. This is a security best practice, and it's often required by compliance standards. Rotating secrets can be a complex process, especially when you have to do it across multiple services.

And finally, you might also be aware of the problem that distributed secrets can be hard to manage. You might have secrets stored in multiple places, and it can be hard to keep track of where they are and who is allowed to access them.

HashiCorp Vault GitHub Secrets Sync

HashiCorp Vault is a secrets management tool that can help you solve these problems. Vault can store secrets, distribute them to services, and rotate them. It can also help you manage access to secrets, so you can manage access permissions effectively.

In this article, we'll look at how you can use HashiCorp Vault to sync secrets with GitHub. This can be useful if you want to distribute secrets to a GitHub repository - or multiple repositories - and keep them in sync.

Prerequisites

On the documentantion you can find the prerequisites for this tutorial. You need to have a GitHub account and a HashiCorp Vault instance. You also need to have the Vault CLI installed on your machine to follow the documnetation instructions.

We want to do that with Terraform as Code - so let's get started:

HashiCorp Vault GitHub Secrets Sync with Terraform

First, we have to select the target repository where we want to sync the secrets. We can do this by creating a new repository or using an existing one.

Because there is no specific resource available within the Terraform Vault provider, we have to use the vault_generic_endpoint resource to interact with the Vault API.

 1resource "vault_generic_endpoint" "vault_github_sync" {
 2  path                 = "/sys/sync/destinations/gh/${var.github_repository}"
 3  disable_read         = true
 4  disable_delete       = true
 5  ignore_absent_fields = true
 6  data_json = jsonencode({
 7    "access_token"     = var.github_token,
 8    "repository_owner" = var.github_repository_owner,
 9    "repository_name"  = var.github_repository
10  })
11  write_fields = ["request_id", "data"]
12}

Within this code snippet, we are creating a new vault_generic_endpoint resource that will interact with the Vault API. We are using the path attribute to define the endpoint we want to interact with. In this case, we are using the /sys/sync/destinations/gh/${var.github_repository} endpoint to sync the secrets with the GitHub repository. The actual values for the github_repository, github_token, and github_repository_owner variables are passed in from the Terraform configuration. At this endpoint the ${var.github_repository} is used as key within this structure to define the syncronization, it is just a string and can be anything you want it to be.

We are using the data_json attribute to pass in the data that we want to send to the API. In this case, we are passing in the access_token, repository_owner, and repository_name values. These values are used by the Vault API to authenticate with the GitHub API and sync the secrets with the repository.

Finally, we are using the write_fields attribute to specify the fields that we want to write to the API. In this case, we are writing the request_id and data fields.

After this definition we can create the associations for the secrets we want to sync with the GitHub repository. We can do this by creating a new vault_generic_endpoint resource for each secret that we want to sync.

The actual endpoint requires as parameter the path of the secret within Vault, and also the actual secret engine path where the secret is stored.

 1resource "vault_generic_endpoint" "vault_github_sync_items" {
 2  depends_on           = [vault_generic_endpoint.vault_github_sync]
 3  path                 = "/sys/sync/destinations/gh/${var.github_repository}/associations/set"
 4  disable_read         = true
 5  disable_delete       = true
 6  ignore_absent_fields = true
 7  data_json = jsonencode({
 8    mount       = "kv",
 9    secret_name = "team_devops/dockerhub"
10  })
11  write_fields = ["request_id", "data"]
12}

In this code snippet, we are creating a new vault_generic_endpoint resource that will interact with the Vault API. We are using the path attribute to define the endpoint we want to interact with. In this case, we are using the /sys/sync/destinations/gh/${var.github_repository}/associations/set endpoint to create an association between the secret and the GitHub repository secret.

The secret from HashiCorp Vault is synced as json object to the GitHub repository.

1{
2  "username": "team_devops",
3  "password": "dockerhub",
4  "api_token": "1234567890"
5}

Within your Github Actions you need to use tools like jq or fromJson() to parse the json object and use the secrets within your workflow.

Have fun trying it yourself!