Deploy HCP Consul Dedicated with EC2 using Terraform

11min
|
HCP
Consul
Terraform
Nomad

In the previous tutorial, you learned how to deploy a new HCP Consul Dedicated cluster and to deploy your workload in an EKS run time created in the same operation with Terraform.

In this tutorial, you will learn how to deploy a new HCP Consul Dedicated cluster and deploy your workload, a demo application, in an EC2 instance, created in the same operation with Terraform. The Terraform module will use Nomad to automate the application deployment.

HCP Consul Dedicated deployment workflow EC2

Prerequisites

To complete this tutorial you will need the following.

Basic command line access
Terraform v1.0.0+ CLI installed
Git installed
Admin access to the HashiCorp Cloud Platform (HCP) Consul portal
Note
HCP Admin access is necessary to create the Service Principal credentials used by Terraform to interact with HCP. If you already have a Service Principal key and client id provided by your admin, you only require Contributor access. If you are an Admin and would like to create a Service Principal, check Deploy HCP Consul Dedicated with Terraform tutorial for instructions on how to create a Service Principal.

An AWS account and AWS Access Credentials configured locally.

You can configure the AWS credentials using environment variables.

export AWS_ACCESS_KEY_ID=<your AWS access key ID>
export AWS_SECRET_ACCESS_KEY=<your AWS secret access key>
export AWS_SESSION_TOKEN=<your AWS session token>

Generate Terraform template

You can generate a Terraform template for this example directly from the Overview page in your HCP organization.

HCP UI Consul - Deploy with Terraform

HCP UI Consul - Deploy with Terraform - EC2 with existing VPC

Authenticate Terraform to HCP

To authenticate Terraform to HCP you need a Service Principal with Contributor permissions. If you are logged with an Admin account you can create one during this step.

In the Authenticate Terraform to HCP section click on the Generate Service Principal and Key.

HCP UI Consul - Deploy with Terraform - Download TF code for EKS

HCP will generate a new set of credentials for you and you can copy them using the Copy code button and export them in your terminal.

export HCP_CLIENT_ID=<your client id>
export HCP_CLIENT_SECRET=<the key generated>

Note

If you are not an Admin on HCP you should contact your administrator and obtain valid Service Principal credentials before proceeding with the tutorial.

Get Terraform code

Once you have filled in all the options in the bottom side of the page you will find the generated Terraform code.

HCP UI Consul - Deploy with Terraform - Download TF code for EC2

Click on Copy code to copy it to your clipboard and save it in a file named main.tf.

Click here for an example configuration

Note

Content should resemble the example below. This example is not guaranteed to be up to date. Always refer to the template file provided by HCP UI after the configuration.

locals {
  vpc_region            = "us-west-2"
  hvn_region            = "us-west-2"
  cluster_id            = "consul-quickstart-1642461904605"
  hvn_id                = "consul-quickstart-1642461904605-hvn"
  vpc_id                = "{{ .VPCID }}"
  public_route_table_id = "{{ .PublicRouteTableID }}"
  public_subnet1        = "{{ .PublicSubnet1 }}"
}

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.43"
    }
    hcp = {
      source  = "hashicorp/hcp"
      version = ">= 0.18.0"
    }
  }

}

provider "aws" {
  region = local.vpc_region
}


resource "hcp_hvn" "main" {
  hvn_id         = local.hvn_id
  cloud_provider = "aws"
  region         = local.hvn_region
  cidr_block     = "172.25.32.0/20"
}

module "aws_hcp_consul" {
  source  = "hashicorp/hcp-consul/aws"
  version = "~> 0.5.0"

  hvn             = hcp_hvn.main
  vpc_id          = local.vpc_id
  subnet_ids      = [local.public_subnet1]
  route_table_ids = [local.public_route_table_id]
}

resource "hcp_consul_cluster" "main" {
  cluster_id      = local.cluster_id
  hvn_id          = hcp_hvn.main.hvn_id
  public_endpoint = true
  tier            = "development"
}

resource "hcp_consul_cluster_root_token" "token" {
  cluster_id = hcp_consul_cluster.main.id
}

module "aws_ec2_consul_client" {
  source  = "hashicorp/hcp-consul/aws//modules/hcp-ec2-client"
  version = "~> 0.5.0"

  subnet_id                = local.public_subnet1
  security_group_id        = module.aws_hcp_consul.security_group_id
  allowed_ssh_cidr_blocks  = ["0.0.0.0/0"]
  allowed_http_cidr_blocks = ["0.0.0.0/0"]
  client_config_file       = hcp_consul_cluster.main.consul_config_file
  client_ca_file           = hcp_consul_cluster.main.consul_ca_file
  root_token               = hcp_consul_cluster_root_token.token.secret_id
  consul_version           = hcp_consul_cluster.main.consul_version
}

output "consul_root_token" {
  value     = hcp_consul_cluster_root_token.token.secret_id
  sensitive = true
}

output "consul_url" {
  value = hcp_consul_cluster.main.public_endpoint ? (
    hcp_consul_cluster.main.consul_public_endpoint_url
    ) : (
    hcp_consul_cluster.main.consul_private_endpoint_url
  )
}

output "nomad_url" {
  value = "http://${module.aws_ec2_consul_client.host_dns}:8081"
}

output "hashicups_url" {
  value = "http://${module.aws_ec2_consul_client.host_dns}"
}

Locals

The values you provided in the UI during the creation are used as local variables in the generated Terraform code.

vpc_region - This is the region where you deployed your VPC.
hvn_region - The HashiCorp Virtual Network (HVN) region.
cluster_id - The HCP Consul Dedicated cluster ID. Use a unique name to identify your HCP Consul Dedicated cluster. HCP will pre-populate it with a name following the pattern consul-quickstart-<unique-ID>.
vpc_id - Since you are using an existing VPC you need to instruct Terraform on the VPC ID to be able to use it.
public_route_table_id - A route table contains a set of rules, called routes, that are used to determine where network traffic from your subnet or gateway is directed.
public_subnet1 - A subnet is a range of IP addresses in your VPC. You can launch AWS resources into a specific subnet.

Note

The HCP Consul Dedicated UI helps you fill in these fields and locate the values you need from your AWS console. Click on the Where can I find this? links in the UI to get help in locating the right values.

HCP UI Consul - Deploy with Terraform - EC2 with new VPC

Authenticate Terraform to HCP

To authenticate Terraform to HCP you need a Service Principal with Contributor permissions. If you are logged with an Admin account you can create one during this step.

In the Authenticate Terraform to HCP section click on the Generate Service Principal and Key.

HCP UI Consul - Deploy with Terraform - Download TF code for EKS

HCP will generate a new set of credentials for you and you can copy them using the Copy code button and export them in your terminal.

export HCP_CLIENT_ID=<your client id>
export HCP_CLIENT_SECRET=<the key generated>

Note

If you are not an Admin on HCP you should contact your administrator and obtain valid Service Principal credentials before proceeding with the tutorial.

Get Terraform code

Once you have filled in all the options in the bottom side of the page you will find the generated Terraform code.

HCP UI Consul - Deploy with Terraform - Download TF code for EC2

Click on Copy code to copy it to your clipboard and save it in a file named main.tf.

Click here for an example configuration

Note

Content should resemble the example below. This example is not guaranteed to be up to date. Always refer to the template file provided by HCP UI after the configuration.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.43"
    }
    hcp = {
      source  = "hashicorp/hcp"
      version = "~> 0.19"
    }
  }
}

locals {
  vpc_region = "us-west-2"
  hvn_region = "us-west-2"
  cluster_id = "consul-quickstart-1637696214864"
}

provider "aws" {
  region = local.vpc_region
}

data "aws_availability_zones" "available" {}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "3.10.0"

  name                 = "${local.cluster_id}-vpc"
  cidr                 = "10.0.0.0/16"
  azs                  = data.aws_availability_zones.available.names
  private_subnets      = []
  public_subnets       = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  enable_dns_hostnames = true
}

resource "hcp_hvn" "main" {
  hvn_id         = "${local.cluster_id}-hvn"
  cloud_provider = "aws"
  region         = local.hvn_region
  cidr_block     = "172.25.32.0/20"
}

module "aws_hcp_consul" {
  source  = "hashicorp/hcp-consul/aws"
  version = "0.3.0"

  hvn             = hcp_hvn.main
  vpc_id          = module.vpc.vpc_id
  subnet_ids      = module.vpc.public_subnets
  route_table_ids = module.vpc.public_route_table_ids
}

resource "hcp_consul_cluster" "main" {
  cluster_id      = local.cluster_id
  hvn_id          = hcp_hvn.main.hvn_id
  public_endpoint = true
  tier            = "development"
}

resource "hcp_consul_cluster_root_token" "token" {
  cluster_id = hcp_consul_cluster.main.id
}

module "aws_ec2_consul_client" {
  source  = "hashicorp/hcp-consul/aws//modules/hcp-ec2-client"
  version = "0.3.0"

  subnet_id                = module.vpc.public_subnets[0]
  security_group_id        = module.aws_hcp_consul.security_group_id
  allowed_ssh_cidr_blocks  = ["0.0.0.0/0"]
  allowed_http_cidr_blocks = ["0.0.0.0/0"]
  client_config_file       = hcp_consul_cluster.main.consul_config_file
  client_ca_file           = hcp_consul_cluster.main.consul_ca_file
  root_token               = hcp_consul_cluster_root_token.token.secret_id
}

output "consul_root_token" {
  value     = hcp_consul_cluster_root_token.token.secret_id
  sensitive = true
}

output "consul_url" {
  value = hcp_consul_cluster.main.consul_public_endpoint_url
}

output "nomad_ec2_id" {
  value = module.aws_ec2_consul_client.host_id
}

output "hashicups_url" {
  value = "http://${module.aws_ec2_consul_client.host_dns}:8080"
}

Locals

The HCP Consul Dedicated UI will guide you into picking the right values for the locals. If you want to, you can edit the cluster_id but make sure it does not conflict with other deployments you have in your organization.

vpc_region - This is the region where you deployed your VPC.
hvn_region - The HashiCorp Virtual Network (HVN) region.
cluster_id - The HCP Consul Dedicated cluster ID. Use a unique name to identify your HCP Consul Dedicated cluster. HCP will pre-populate it with a name following the pattern consul-quickstart-<unique-ID>.

Run terraform

With the Terraform manifest files and your custom credentials file, you are now ready to deploy your infrastructure.

Check that the following setup is complete before executing the terraform init step:

Your AWS credentials are populated as environment variables and Terraform install is complete (refer to prerequisites)
You have exported the HCP credentials from the UI as environment variables
If you are deploying into an existing VPC, ensure the subnet has internet connectivity.

Issue the terraform init command from your working directory to download the necessary providers and initialize the backend.

$ terraform init

Initializing the backend...

Initializing provider plugins...
...

Terraform has been successfully initialized!
...

Once Terraform has been initialized, you can verify the resources that will be created using the plan command.

$ terraform plan

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:
...

Finally, you can deploy the resources using the apply command.

$ terraform apply

...
Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value:

Remember to confirm the run by entering yes.

Once you confirm, it will take a few minutes to complete the deploy. Terraform will print the following output if the deployment is successful.

Apply complete! Resources: 37 added, 0 changed, 0 destroyed.

Examine Terraform output

At the end of the execution Terraform will output the following lines:

Outputs:

consul_root_token = <sensitive>
consul_url = "https://consul-quickstart-learn.consul.11eb5071-85f5-1eb2-992c-0242ac110003.aws.hashicorp.cloud"
hashicups_url = "http://ec2-3-9-13-130.eu-west-2.compute.amazonaws.com:8080"
nomad_ec2_id = "i-0983b0561f7799b4b"

As you can notice the consul_root_token is not showed since is a sensitive value.

You can retrieve it using:

$ terraform output consul_root_token

Verify created resources

Consul UI

Visit the Consul UI using the consul_url link in the output values.

Consul UI services HashiCups login request

Login in Consul using the token retrieved in the previous step and verify the services are all present in your Consul datacenter UI

Consul UI services HashiCups

Consul CLI configuration

Using the Terraform output values you can setup your Consul CLI to connect to the datacenter you created.

Setup environment variables:

$ export CONSUL_HTTP_TOKEN=$(terraform output -raw consul_root_token)

$ export CONSUL_HTTP_ADDR=$(terraform output -raw consul_url)

Verify Consul can connect to the datacenter:

$ consul members

Example output:

Node             Address            Status  Type    Build       Protocol  DC                       Segment
ip-172-25-41-89  172.25.41.89:8301  alive   server  1.10.4+ent  2         consul-quickstart-learn  <all>
ip-10-0-1-65     10.0.1.65:8301     alive   client  1.10.4+ent  2         consul-quickstart-learn  <default>

HashiCups application

The Terraform code deployed an application that exposes a web UI accessible using the hashicups_url URL.

You can access the configurations of the deployed Hashicups app services here.

HashiCups UI

Nomad

From the output you can notice the presence of a nomad_ec2_id element. This is because the deploy of the application is performed using Nomad on an EC2 node with Docker installed.

The usage of Nomad is also evident on the Services tab of your Consul UI where the services show 3 instances of service nomad, representing the Nomad servers, and one instance of service nomad-client.

You can access the Nomad UI using the hashicups_url and changing the port to 8081.

So in the example output provided you will use http://ec2-3-9-13-130.eu-west-2.compute.amazonaws.com:8081 as URL.

To login into the Nomad instance use nomad as username and the consul_root_token as the password.

Nomad UI - Jobs

Cleanup environment

Use the terraform destroy command to clean up the resources you created.

$ terraform destroy

...
Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value:

Remember to confirm by entering yes.

Once you confirm, it will take a few minutes to complete the removal. Terraform will print the following output if the command is successful.

Destroy complete! Resources: xx destroyed.

Next steps

In this tutorial you learned how to use Terraform to deploy a demo application on AWS EC2 instances using HCP Consul Dedicated as your service mesh.

In the next tutorial you will use Terraform to deploy a demo application on AWS ECS instances using HCP Consul Dedicated as your service mesh.

If you encounter any issues, please contact the HCP team at support.hashicorp.com.

Deploy HCP with EKS

Deploy HCP with ECS