Deploy HCP Consul Dedicated with Terraform

In Deploy HCP Consul Dedicated, you learned the basics of HashiCorp Cloud Platform (HCP) Consul and learned how to manually deploy a cluster and peer the HashiCorp Virtual Network (HVN) with your AWS Virtual Private Cloud (VPC).

Once you learned the basic components and steps required to deploy a Consul cluster, the recommended approach is to automate your deployment using HashiCorp Terraform.

For this reason we provide a Terraform provider for HCP Consul Dedicated that exposes resources used to interact with HCP and create a new Consul datacenter using Terraform.

This enables you to accomplish a number of tasks, including, but not limited to:

• Create a new HashiCorp Virtual Network (HVN) to deploy your datacenter
• Deploy your Consul datacenter
• Create an AWS VPC and peer it with your HashiCorp Virtual Network (HVN)
• Peer your HVN with an already existing AWS VPC

Prerequisites

To complete the steps listed in this tutorial you will need:

• Terraform 0.14.+ installed on your local machine. You can follow the Install Terraform tutorial in case you do not have it yet.

• A HashiCorp Cloud Platform (HCP) account.

• An AWS account. The tutorial will also provide you instructions to automatically deploy an AWS VPC and peer it with your HashiCorp Virtual Network (HVN). To do so you will need valid credentials for AWS.

• Your AWS credentials configured locally.

You can configure the AWS credentials using environment variables.

export AWS_ACCESS_KEY_ID=<your AWS access key ID>
export AWS_SECRET_ACCESS_KEY=<your AWS secret access key>
export AWS_SESSION_TOKEN=<your AWS session token>

Create service principal and key

To leverage the Terraform integration and be able to deploy your HCP Consul Dedicated using Terraform, you have to create a Service Principal and a key associated to it.

From the left menu select Access Control (IAM) in the Settings section.

In the Access Control (IAM) page click on the Service Principals tab, and on the page click on the create link.

Use the name you prefer to create the Service Principal (we used learn-hcp for this tutorial) but remember to use Contributor as the role.

Once the Service Principal is created, click on the service principal name on the page to view its details. From the detail page, click on + Generate key. A popup like the following will appear:

Set environment variables

Once the key for the service principal has been created you can use the client ID and the secret in Terraform to authenticate with HCP.

We suggest you pass the values to terraform using environment variables.

export HCP_CLIENT_ID=<your client id>
export HCP_CLIENT_SECRET=<the key generated>

Clone the code repository

The tutorial provides you with a git repository that contains two examples that show how to deploy a Consul cluster in HCP. They are divided into the following folders:

• hcp_consul_base: a basic example that deploys an HVN and Consul cluster on HCP. This example only creates the HVN and deploys a Consul cluster in it. You will still have to create an AWS VPC manually and peer it with your HVN.
• hcp_consul_vpc: a more complete example that creates an AWS VPC and peers it automatically with your HVN. This scenario requires that you have a valid AWS account.

Clone the repository to get the example files locally:

$ git clone git@github.com:hashicorp/learn-hcp-consul.git

Move into the hcp_consul_vpc folder.

$ cd learn-hcp-consul/hcp_consul_vpc

Ensure that you are in the correct directory by listing the contents.

$ ls -1

 consul.tf
 variables.tf
 vpc_peering.tf

The folders contains the following files

• consul.tf: describes the HPC Consul cluster you are going to create.
• vpc_peering.tf: describes the AWS VPC and the peering with the HVN.
• variables.tf: sets the variables for your deployment.

Define your HVN

Before you can deploy an HCP Consul Dedicated cluster, you will first need to configure a HashiCorp Virtual Network (HVN). HVNs enable you to deploy HashiCorp Cloud products without having to manage the networking details.

You can define an HVN using the hcp_hvn resource.

resource "hcp_hvn" "example_hvn" {
  hvn_id         = var.hvn_id
  cloud_provider = var.cloud_provider
  region         = var.region
}

resource "hcp_consul_cluster" "example_hcp" {
  hvn_id          = hcp_hvn.example_hvn.hvn_id
  cluster_id      = var.cluster_id
  tier            = "development"
#   public_endpoint = true
}

Using the variables.tf file you can tune the attributes for the HVN.

Define your HCP Consul Dedicated cluster

HCP Consul Dedicated enables you to quickly deploy Consul servers in AWS while offloading the operations burden to the SRE experts at HashiCorp.

You can define an HCP Consul Dedicated cluster using the hcp_consul_cluster resource.

resource "hcp_hvn" "example_hvn" {
  hvn_id         = var.hvn_id
  cloud_provider = var.cloud_provider
  region         = var.region
}

resource "hcp_consul_cluster" "example_hcp" {
  hvn_id          = hcp_hvn.example_hvn.hvn_id
  cluster_id      = var.cluster_id
  tier            = "development"
#   public_endpoint = true
}

Using the variables.tf file, you can tune the attributes for the Consul cluster.

For this tutorial you are going to deploy a development environment that only deploys one Consul server.

Define VPC peering

Creating peering connections from your HVN will allow you to connect and launch AWS resources from your HCP account. Each peering connection requires an HVN route to set its CIDR block.

The Amazon VPC can be managed with the AWS provider.

provider "aws" {
  region = var.region
}

resource "aws_vpc" "peer" {
  cidr_block = "172.31.0.0/16"
}

data "aws_arn" "peer" {
  arn = aws_vpc.peer.arn
}

resource "hcp_aws_network_peering" "peer" {
  hvn_id              = hcp_hvn.example_hvn.hvn_id
  peering_id          = var.peering_id
  peer_vpc_id         = aws_vpc.peer.id
  peer_account_id     = aws_vpc.peer.owner_id
  peer_vpc_region     = data.aws_arn.peer.region
}

resource "hcp_hvn_route" "peer_route" {
  hvn_link         = hcp_hvn.example_hvn.self_link
  hvn_route_id     = var.route_id
  destination_cidr = aws_vpc.peer.cidr_block
  target_link      = hcp_aws_network_peering.peer.self_link
}

resource "aws_vpc_peering_connection_accepter" "peer" {
  vpc_peering_connection_id = hcp_aws_network_peering.peer.provider_peering_id
  auto_accept               = true
}

Once the VPC is peered with the HVN the resources you deploy inside the VPC will be able to reach the Consul servers.

Deploy your infrastructure

Once you have cloned the repository and defined the environment variables for HCP and AWS, you are now ready to deploy you infrastructure.

First, initialize terraform. This will download the necessary providers and initialize the backend.

$ terraform init

Initializing the backend...

Initializing provider plugins...
...

Terraform has been successfully initialized!
...

Once Terraform has been initialized, you can verify that the resources will be created using the plan command.

$ terraform plan

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create
 <= read (data resources)

Terraform will perform the following actions:

  # data.aws_arn.peer will be read during apply
  # (config refers to values not yet known)
 <= data "aws_arn" "peer"  {
      ...
    }

  # aws_vpc.peer will be created
  + resource "aws_vpc" "peer" {
      ...
    }

  # aws_vpc_peering_connection_accepter.peer will be created
  + resource "aws_vpc_peering_connection_accepter" "peer" {
      ...
    }

  # hcp_aws_network_peering.peer will be created
  + resource "hcp_aws_network_peering" "peer" {
      ...
    }

  # hcp_hvn_route.peer_route will be created
  + resource "hcp_hvn_route" "peer_route" {
      ...
    }

  # hcp_consul_cluster.example_hcp will be created
  + resource "hcp_consul_cluster" "example_hcp" {
      ...
    }

  # hcp_hvn.example_hvn will be created
  + resource "hcp_hvn" "example_hvn" {
      ...
    }

Plan: 6 to add, 0 to change, 0 to destroy.

Finally you can deploy the resources using the apply command.

$ terraform apply

...
Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value:

Remember to confirm the run by entering yes.

Once you confirm, it will take a few minutes to complete the deploy. If the deploy was successful you should get the following output.

Apply complete! Resources: 6 added, 0 changed, 0 destroyed.

Access Consul

Once the deploy is complete, you can verify that the resources have been created in the HCP portal. Open the https://portal.cloud.hashicorp.com page in your browser.

You should be able to confirm the HVN is created in the HashiCorp Virtual Network page.

As the screenshot confirms, you will have a HVN created with:

• A peering connection to a VPC
• An HVN route targeting that peering connection
• A Consul cluster showed in the associated resources

On the Consul page, you'll find the details for your newly deployed Consul cluster.

From here you can use the Generate Token button to generate an ACL token for your Consul datacenter or click the Download client config to retrieve the configuration files and CA certificate needed to connect clients to your datacenter.

Destroy resources

Use the terraform destroy command to clean up the resources you created.

$ terraform destroy

...
Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value:

Remember to confirm by entering yes.

Once you confirm, it will take a few minutes to complete the removal. If the command was successful you should get the following output.

Destroy complete! Resources: 6 destroyed.

Next steps

In this tutorial you learned how to automate the deployment of an HCP Consul Dedicated cluster using Terraform. You used the newly released Terraform HCP provider to achieve that, and you learned the what resources are made available by the provider.

You can find the full documentation for the HashiCorp Cloud Platform Consul Terraform provider in the Terraform registry documentation

To get hands-on experience with Consul's service discovery and service mesh features, you will need to connect a Consul client deployed in a virtual machine or on Elastic Kubernetes Service (EKS).

If you encounter any issues, please contact the HCP team at support.hashicorp.com.