Automating DNS Updates from Kubernetes - HashiCorp Style

In dynamic Kubernetes environments, managing DNS records can be a challenge. As services scale, redeploy, or change IP addresses, ensuring that DNS records remain up-to-date is crucial for service availability. Automating this process with Consul-Terraform-Sync, Kubernetes-based Consul Sync, and Cloudflare, while leveraging Kubernetes service annotations to define hostnames, is an efficient way to manage this complexity.

This article will guide you through automating DNS updates using Kubernetes service annotations for hostname definitions, combined with Consul-Terraform-Sync and Cloudflare.

Tools Overview

1. Consul-Terraform-Sync (CTS): Automates infrastructure updates based on changes in Consul’s service catalog, using Terraform to manage external services like DNS.
2. Consul Service Sync: Monitors Kubernetes services and syncs them with Consul, ensuring that Consul’s catalog reflects the current state of the cluster.
3. Cloudflare: A globally distributed DNS provider offering APIs to manage DNS records.
4. Kubernetes Service Annotations: Allows defining custom hostnames within service metadata, which we will use to automate DNS record updates.

Using Kubernetes service annotations for hostname definitions offers a simple way to dynamically map services to DNS records. The annotations will serve as the source for service names in DNS updates, ensuring that DNS records remain current as services scale or redeploy.

Architecture Overview

1. Kubernetes Services: Applications run in Kubernetes, where service annotations define custom hostnames for DNS records.
2. Consul Sync: Consul Sync monitors the Kubernetes services and registers their metadata (including annotations) in Consul.
3. Consul-Terraform-Sync (CTS): Watches for changes in the Consul catalog and triggers Terraform runs to update DNS records in Cloudflare.
4. Cloudflare DNS: Terraform updates DNS records in Cloudflare whenever a service IP or hostname changes, ensuring that DNS stays in sync with the actual state of the Kubernetes cluster.

Using Kubernetes Service Annotations for Host Definitions

Kubernetes services allow you to attach custom annotations to describe additional metadata. We can define DNS hostnames within these annotations, which will later be used by CTS to update the DNS records in Cloudflare.

Example service with custom annotations:

 1apiVersion: v1
 2kind: Service
 3metadata:
 4  name: web-app
 5  annotations:
 6    consul.hashicorp.com/service-sync: "true"
 7    consul.hashicorp.com/service-tags: dns.hostname=`web.example.com`
 8spec:
 9  selector:
10    app: web-app
11  ports:
12    - protocol: TCP
13      port: 8080

In this example, the annotation dns.hostname is used to define the DNS name (web.example.com) that should be registered and kept updated in Cloudflare. The consul.hashicorp.com/service-sync annotation tells Consul to sync this service with the Consul catalog. And consul.hashicorp.com/service-tags specifies the metadata tags to be registered in Consul, that can be consumed within the Consul-Terraform-Sync configuration.

Set Up Consul-Terraform-Sync (CTS)

Now, configure Consul-Terraform-Sync (CTS) to automate DNS updates in Cloudflare whenever a service IP or hostname changes. CTS will monitor the Consul catalog for changes and trigger Terraform runs to keep DNS records updated.

Example CTS configuration:

 1consul {
 2  address = "http://localhost:8500"
 3}
 4
 5vault {
 6    address = "http://localhost:8200"
 7}
 8
 9driver "terraform" {
10  log         = false
11  persist_log = false
12
13  backend "consul" {
14    gzip = true
15  }
16  required_providers {
17    cloudflare = {
18      source  = "cloudflare/cloudflare"
19      version = "~> 4.0"
20    }
21  }
22}
23
24terraform_provider "cloudflare" {
25    api_token = "{{ with secret \"kv/network/cloudflare\" }}{{ .Data.data.api_token }}{{ end }}"
26}
27
28task {
29  name       = "cloudflare-dns-update"
30  description = "Automatically update Cloudflare DNS records based on service annotations"
31  source     = "./k8s-cloudflare"
32  providers  = ["cloudflare"]
33
34  condition "services" {
35    regexp = ".*"
36    filter = "Service.Meta[\"external-source\"] == \"kubernetes\""
37  }
38}

This configuration tells CTS to monitor all services that are related to the Kubernetes Consul Sync within the Service Catalog of Consul. When the IP or hostname changes, CTS will trigger Terraform to update the corresponding DNS record in Cloudflare. The actual configuration of the provider needs to be applied within the CTS configuration, because we donot want to expose the API token in the terraform code. And we can use Vault to store the API token securely and pull it during the CTS run. CTS will also keep the applied token alive and renew it when needed.

Configure Cloudflare Provider in Terraform

To manage DNS records in Cloudflare, you’ll need to configure the Cloudflare provider in Terraform. Here’s a Terraform configuration to create or update DNS records:

 1locals {
 2
 3    dns_tagged = {
 4        for name, service in var.services : name => [
 5            for tag in service.tags : tag if can(regex("dns.hostname=.*", tag))
 6        ]
 7    }
 8    dns_hosts = {
 9        for name, tag in local.k8s_tagged : name => regex("dns.hostname=`\\((?P<hostname>.*)`\\)", element(tag,0)).hostname
10    }
11
12}
13
14resource "cloudflare_record" "web" {
15
16  for_each = local.dns_hosts
17
18  zone_id = "YOUR_ZONE_ID"
19  name    = each.value
20  type    = "A"
21  value   = var.services[each.key].address
22  ttl     = 300
23}

This configuration dynamically pulls the DNS name from the dns.hostname annotation and updates the record to point to the current IP of the service.

6. Automate DNS Record Updates

With everything configured, your system is now set up to automatically update DNS records in Cloudflare. As services in Kubernetes are scaled or redeployed, their IP addresses may change, and Consul will capture those changes in real-time. Consul-Terraform-Sync will then trigger Terraform runs to update DNS records in Cloudflare accordingly.

Benefits of Using Service Annotations for DNS Automation

• Simplified DNS Management: Service annotations provide a simple way to define DNS hostnames within Kubernetes service metadata, which CTS uses to manage DNS records dynamically.
• Real-Time DNS Updates: Consul and CTS ensure that DNS records in Cloudflare are updated as soon as changes occur in Kubernetes, reducing downtime and improving service availability.
• Seamless Integration: Using service annotations allows you to manage DNS names directly within Kubernetes, streamlining your workflow.
• Scalability: As Kubernetes services scale up or down, the system automatically adjusts DNS records without manual intervention.

Conclusion

By using Consul-Terraform-Sync, Kubernetes Consul Sync, Cloudflare, and service annotations in Kubernetes, you can fully automate DNS updates in dynamic environments. This ensures that DNS records are always current and accurate, reducing manual overhead and improving service resilience. A very similar approach can also be used for HashiCorp Nomad.

Further Reading

• Consul-Terraform-Sync Documentation
• Consul Kubernetes Sync
• Cloudflare Terraform Provider

This approach provides a scalable, reliable solution for automating DNS management in Kubernetes using simple service annotations and Cloudflare’s DNS capabilities. And of course Cloudflare can be replaced with any other DNS provider that offers an API and terraform provider to manage DNS records.