Consul
Requirements
The following components are required to run Consul-Terraform-Sync (CTS):
- A Terraform provider
- A Terraform module
- A Consul cluster running outside of the
consul-terraform-sync
daemon
You can add support for your network infrastructure through Terraform providers so that you can apply Terraform modules to implement network integrations.
The following guidance is for running CTS using the Terraform driver. The HCP Terraform driverEnterprise has additional prerequisites.
Run a Consul cluster
Below are several steps towards a minimum Consul setup required for running CTS.
Install Consul
CTS is a daemon that runs alongside Consul, similar to other Consul ecosystem tools like Consul Template. CTS is not included with the Consul binary and needs to be installed separately.
To install a local Consul agent, refer to the Getting Started: Install Consul Tutorial.
For information on compatible Consul versions, refer to the Consul compatibility matrix.
Run an agent
The Consul agent must be running in order to dynamically update network devices. Refer to the Consul agent documentation for information about configuring and starting a Consul agent.
When running a Consul agent with CTS in production, consider that CTS uses blocking queries to monitor task dependencies, such as changes to registered services. This results in multiple long-running TCP connections between CTS and the agent to poll changes for each dependency. Consul may quickly reach the agent connection limits if CTS is monitoring a high number of services.
To avoid reaching the limit prematurely, we recommend using HTTP/2 (requires HTTPS) to communicate between CTS and the Consul agent. When using HTTP/2, CTS establishes a single connection and reuses it for all communication. Refer to the Consul Configuration section for details.
Alternatively, you can configure the limits.http_max_conns_per_client
option to set a maximum number of connections to meet your needs.
Register services
CTS monitors the Consul catalog for service changes that lead to downstream changes to your network devices. Without services, your CTS daemon is operational but idle. You can register services with your Consul agent by either loading a service definition or by sending an HTTP API request.
The following HTTP API request example registers a service named web
with your Consul agent:
$ echo '{
"ID": "web",
"Name": "web",
"Address": "10.10.10.10",
"Port": 8000
}' > payload.json
$ curl --request PUT --data @payload.json http://localhost:8500/v1/agent/service/register
The example represents a non-existent web service running at 10.10.10.10:8000
that is now available for CTS to consume.
You can configure CTS to monitor the web service, execute a task, and update network device(s) by configuring web
in the condition "services"
task block. If the web service has any non-default values, it can also be configured in condition "services"
.
For more details on registering a service using the HTTP API endpoint, refer to the register service API docs.
For hands-on instructions on registering a service by loading a service definition, refer to the Getting Started: Register a Service with Consul Service Discovery Tutorial.
Run a cluster
For production environments, we recommend operating a Consul cluster rather than a single agent. Refer to Getting Started: Deploy a Consul Datacenter Tutorial for instructions on starting multiple Consul agents and joining them into a cluster.
Network infrastructure using a Terraform provider
CTS integrations for the Terraform driver use Terraform providers as plugins to interface with specific network infrastructure platforms. The Terraform driver for CTS inherits the expansive collection of Terraform providers to integrate with. You can also specify a provider source
in the required_providers
configuration to use providers written by the community (requires Terraform 0.13 or later).
Finding Terraform providers
To find providers for the infrastructure platforms you use, browse the providers section of the Terraform Registry.
How to create a provider
If a Terraform provider does not exist for your environment, you can create a new Terraform provider and publish it to the registry so that you can use it within a network integration task or create a compatible Terraform module. Refer to the following Terraform tutorial and documentation for additional information on creating and publishing providers:
Network integration using a Terraform module
The Terraform module for a task in CTS is the core component of the integration. It declares which resources to use and how your infrastructure is dynamically updated. The module, along with how it is configured within a task, determines the conditions under which your infrastructure is updated.
Working with a Terraform provider, you can write an integration task for CTS by creating a Terraform module that is compatible with the Terraform driver. You can also use a module built by partners.
Refer to Configuration for information about configuring CTS and how to use Terraform providers and modules for tasks.
Partner Terraform Modules
The modules listed below are available to use and are compatible with CTS.
A10 Networks
- Dynamic Load Balancing with Group Member Updates: Terraform Registry / GitHub
Avi Networks
- Scale Up and Scale Down Pool and Pool Members (Servers): GitHub
AWS Application Load Balancer (ALB)
- Create Listener Rule and Target Group for an AWS ALB, Forward Traffic to Consul Ingress Gateway: Terraform Registry / GitHub
Checkpoint
- Dynamic Firewalling with Address Object Updates: Terraform Registry / GitHub
Cisco ACI
- Policy Based Redirection: Terraform Registry / GitHub
- Create and Update Cisco ACI Endpoint Security Groups: Terraform Registry / GitHub
Citrix ADC
- Create, Update, and Delete Service Groups in Citrix ADC: Terraform Registry / GitHub
F5
- Dynamic Load Balancing with Pool Member Updates: Terraform Registry / GitHub
NS1
- Create, Delete, and Update DNS Records and Zones: Terraform Registry / GitHub
Palo Alto Networks
- Dynamic Address Group (DAG) Tags: Terraform Registry / GitHub
- Address Group and Dynamic Address Group (DAG) Tags: Terraform Registry / GitHub