Deploy Consul on VMs

28min
|
Consul
Interactive

Workflow

Consul is a service networking solution that enables you to manage secure network connectivity between services and across on-premise and multi-cloud environments and runtimes. Consul offers service discovery, service mesh, traffic management, and automated updates to network infrastructure device. Check out the What is Consul? page to learn more.

In this tutorial, you will configure, deploy, and bootstrap a Consul server on
a virtual machine (VM). After deploying Consul, you will interact with Consul using the UI, CLI, and API.

In the following tutorials, you will deploy a demo application, configure it to use Consul service discovery, secure it with service mesh, allow external traffic into the service mesh, and enhance observability into your service mesh. During the process, you will learn how to leverage Consul to securely connect your services running on any environment.

In this tutorial, you will:

Deploy your VM environment on AWS EC2 using Terraform
Configure a Consul server
Start a Consul server instance
Configure your terminal to communicate with the Consul datacenter
Bootstrap Consul ACL system and create tokens for Consul management
Interact with Consul API, KV store and UI

Note

This tutorial is part of the Get Started collection, for this reason all the steps used to configure Consul agents and services are shown and require to be executed manually. If you are setting up a production environment you should codify and automate the installation and deployment process. Refer to the VM production patterns tutorial collection for Consul production deployment best practices.

Tutorial scenario

This tutorial uses HashiCups, a demo coffee shop application made up of several microservices running on VMs.

At the beginning of this tutorial, you have an instance of HashiCups running on four VMs and one empty VM that you will use to deploy Consul server.

Architectural diagram 00

By the end of this tutorial, you will have deployed a Consul server agent running on the extra virtual machine.

Architectural diagram 01

Prerequisites

For this tutorial, you will need:

An AWS account configured for use with Terraform
aws-cli >= 2.0
terraform >= 1.0
consul >= 1.15.0

Clone GitHub repository

Clone the GitHub repository containing the configuration files and resources.

$ git clone https://github.com/hashicorp-education/learn-consul-get-started-vms

Change into the directory that contains the complete configuration files for this tutorial.

$ cd learn-consul-get-started-vms/self-managed/infrastructure/aws

Create infrastructure

With these Terraform configuration files, you are ready to deploy your infrastructure.

Issue the terraform init command from your working directory to download the necessary providers and initialize the backend.

$ terraform init
Initializing the backend...
Initializing provider plugins...
## ...
Terraform has been successfully initialized!
## ...

Then, deploy the resources. Confirm the run by entering yes. The Terraform deployment could take up to 15 minutes to complete. Feel free to explore the next sections of this tutorial while waiting for the environment to complete initialization.

$ terraform apply -var-file=./conf/00_hashicups.tfvars 
## ...
Do you want to perform these actions?
 Terraform will perform the actions described above.
 Only 'yes' will be accepted to approve.
 Enter a value: yes
## ...
Apply complete! Resources: 49 added, 0 changed, 0 destroyed.

Once the deployment is complete, Terraform will return a list of outputs you can use to interact with the newly created environment.

Outputs:

connection_string = "ssh -i certs/id_rsa.pem admin@`terraform output -raw ip_bastion`"
ip_bastion = "<redacted-output>"
remote_ops = "export BASTION_HOST=<redacted-output>"
retry_join = "provider=aws tag_key=ConsulJoinTag tag_value=auto-join-hcoc"
ui_consul = "https://<redacted-output>:8443"
ui_grafana = "http://<redacted-output>:3000/d/hashicups/hashicups"
ui_hashicups = "http://<redacted-output>"

The Terraform outputs provide useful information, including the bastion host IP address. The following is a brief description of the Terraform outputs:

The ip_bastion provides IP address of the bastion host you will use to run the rest of the commands in this tutorial.
The remote_ops lists the bastion host IP, which you can use access the bastion host.
The retry_join output lists Consul's retry_join configuration parameter. The next tutorial will use this to generate Consul server and client configuration.
The ui_consul output lists the Consul UI address. The Consul UI is not currently running. You will use this in a later tutorial to verify Consul started correctly.
The ui_grafana output lists the Grafana UI address. You will use this in a future tutorial.
The ui_hashicups output lists the HashiCups UI address. You can use it to verify the HashiCups demo application is running properly.

$ ssh -i certs/id_rsa.pem admin@`terraform output -raw ip_bastion`

Verify Consul binary

Verify that the VM you want to deploy the Consul server on has the Consul binary.

$ consul version
Consul v1.16.1
Revision e0ab4d29
Build Date 2023-08-05T21:56:29Z
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

Generate Consul server configuration

To securely configure Consul by ensuring all communications between the Consul nodes are inaccessible to unintended agents, you need to provide Consul with:

A gossip encryption key.
A root certificate authority (CA) certificate from a private CA. Consul will use this CA certificate to sign all certificates in the Consul datacenter.
A certificate key-pair for each server you intend to deploy signed by the above CA.

In addition, you want to enable ACLs to ensure every request to your Consul datacenter is authorized.

This tutorial and interactive lab environment uses scripts in the tutorial's GitHub repository to generate these secrets and the Consul configuration files. You will use these scripts to generate your Consul server configuration.

On the bastion host, verify that the scripts are correctly present in the home directory of the admin user.

$ tree ops/scenarios/99_supporting_scripts/
ops/scenarios/99_supporting_scripts/
├── generate_consul_client_config.sh
├── generate_consul_monitoring_config.sh
├── generate_consul_server_config.sh
├── generate_consul_server_tokens.sh
├── generate_consul_service_config.sh
└── generate_consul_service_intentions.sh

0 directories, 6 files

The scripts rely on default parameters to generate the configuration files. Set the following default values. Ensure you have permission to write in the specified paths.

Tip

We suggest using an unprivileged user to run Consul for security reasons.

$ export CONSUL_DATACENTER="dc1"; \
export CONSUL_DOMAIN="consul"; \
export OUTPUT_FOLDER="./assets/scenario/conf/"; \

A Consul datacenter is composed by multiple nodes that, at startup, connect with each other using the Gossip protocol. To be able to automatically locate other nodes at startup, Consul configuration needs a retry_join parameter.

In this scenario we use the Cloud auto join functionality for AWS and the value to include in the Consul configuration is generated by Terraform during the infrastructure provisioning.

Terraform output includes a retry_join value. Use it to populate the following environment variable.

$ export CONSUL_RETRY_JOIN="<use value of Terraform output retry_join here>"

With all environment variables set, generate all necessary files to configure and run the Consul server agent.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_server_config.sh
 [generate_consul_server_config.sh] - Generate Consul servers configuration
 -- [WARN] CONSUL_DATACENTER = dc1
 -- [WARN] CONSUL_DOMAIN = consul
 -- [WARN] CONSUL_SERVER_NUMBER =
 -- Cleaning Scenario before apply.
 -- [WARN] Removing pre-existing configuration in ./assets/scenario/conf/
 -- Generate scenario config folders.
 -- Generate secrets.
 -- Generating Gossip Encryption Key.
 -- Generate CA
==> Saved consul-agent-ca.pem
==> Saved consul-agent-ca-key.pem
 -- Generate Server Certificates
==> WARNING: Server Certificates grants authority to become a
    server and access all state in the cluster including root keys
    and all ACL tokens. Do not distribute them to production hosts
    that are not server nodes. Store them as securely as CA keys.
==> Using consul-agent-ca.pem and consul-agent-ca-key.pem
==> Saved dc1-server-consul-0.pem
==> Saved dc1-server-consul-0-key.pem
 -- Generating Configuration for consul-server-0
 -- Validate configuration for consul-server-0

When the script completes, list the generated files.

$ tree $OUTPUT_FOLDER
./assets/scenario/conf/
├── consul-server-0
│   ├── agent-gossip-encryption.hcl
│   ├── agent-server-acl.hcl
│   ├── agent-server-networking.hcl
│   ├── agent-server-specific-ui.hcl
│   ├── agent-server-specific.hcl
│   ├── agent-server-telemetry.hcl
│   ├── agent-server-tls.hcl
│   ├── consul-agent-ca.pem
│   ├── consul-agent-key.pem
│   ├── consul-agent.pem
│   └── consul.hcl
└── secrets
    ├── agent-gossip-encryption.hcl
    ├── consul-agent-ca-key.pem
    ├── consul-agent-ca.pem
    ├── dc1-server-consul-0-key.pem
    └── dc1-server-consul-0.pem

2 directories, 16 files

The scripts generated multiple configuration files so it is easier to read and tune them for your environment. The following are the generated files and a description of what they do:

The agent-gossip-encryption.hcl file configures gossip encryption.
The agent-server-acl.hcl file configures the ACL system.
The agent-server-networking.hcl file configures networking for the server agent.
The agent-server-specific-ui.hcl file contains specific server UI configuration.
The agent-server-specific.hcl file contains specific server parameters.
The agent-server-telemetry.hcl file configures agent telemetry parameters.
The agent-server-tls.hcl file configures specific TLS encryption.
The *.pem files are certificate key-pair Consul uses to enforce mTLS for datacenter communications.
The consul.hcl file contains node specific configuration and it is needed, with this specific name, if you want to configure Consul as a systemd daemon.

Visit the agent configuration documentation to interpret the files or to tune them when applying them to your scenario.

Test configuration

Verify the configuration generated is valid. Despite the INFO messages, the Consul configuration files are valid.

$ consul validate ${OUTPUT_FOLDER}consul-server-0
## ...
Configuration is valid!

Copy configuration on Consul server node

Once the configuration is created, copy it to the Consul server VM.

Tip

In the AWS lab environment, all the VM nodes have a running SSH server. As a result, you can use ssh and scp commands to perform the following operations. If the nodes in your personal environment do not have an SSH server, you may need to use a different approach to create the configuration directories and copy the files.

First, configure the Consul configuration directory.

$ export CONSUL_REMOTE_CONFIG_DIR=/etc/consul.d/

Then, remove existing configuration from the server.

$ ssh -i certs/id_rsa consul-server-0 "sudo rm -rf ${CONSUL_REMOTE_CONFIG_DIR}*"

Finally, use scp to copy the configuration into the server node.

$ scp -i certs/id_rsa ${OUTPUT_FOLDER}consul-server-0/* consul-server-0:${CONSUL_REMOTE_CONFIG_DIR}
agent-gossip-encryption.hcl
agent-server-acl.hcl
agent-server-networking.hcl
agent-server-specific-ui.hcl
agent-server-specific.hcl
agent-server-telemetry.hcl
agent-server-tls.hcl
consul-agent-ca.pem
consul-agent-key.pem
consul-agent.pem
consul.hcl

Start Consul server

Now that you have created the Consul configuration files, you can start the Consul server.

$ ssh -i certs/id_rsa consul-server-0

Configure the Consul configuration and data directories.

$ export CONSUL_CONFIG_DIR=/etc/consul.d/ \
export CONSUL_DATA_DIR=/opt/consul/

Make sure your user has write permissions in the Consul data directory.

$ sudo chmod g+w ${CONSUL_DATA_DIR}

Finally, start the Consul server process.

$ consul agent -config-dir=${CONSUL_CONFIG_DIR} > /tmp/consul-server.log 2>&1 &

The command starts the Consul server in the background to not lock the terminal. You can access the Consul server log through the /tmp/consul-server.log file.

$ cat /tmp/consul-server.log
==> Starting Consul agent...
               Version: '1.16.1'
            Build Date: '2023-08-05 21:56:29 +0000 UTC'
               Node ID: '253954d5-a05f-464b-d7f4-23195b3c516e'
             Node name: 'consul-server-0'
            Datacenter: 'dc1' (Segment: '<all>')
                Server: true (Bootstrap: true)
           Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: 8443, gRPC: -1, gRPC-TLS: 8503, DNS: 8600)
          Cluster Addr: 10.0.4.241 (LAN: 8301, WAN: 8302)
     Gossip Encryption: true
      Auto-Encrypt-TLS: true
           ACL Enabled: true
     Reporting Enabled: false
    ACL Default Policy: deny
             HTTPS TLS: Verify Incoming: false, Verify Outgoing: true, Min Version: TLSv1_2
              gRPC TLS: Verify Incoming: false, Min Version: TLSv1_2
      Internal RPC TLS: Verify Incoming: true, Verify Outgoing: true (Verify Hostname: true), Min Version: TLSv1_2

==> Log data will now stream in as it occurs:

...
[INFO]  agent: Started DNS server: address=0.0.0.0:8600 network=tcp
[INFO]  agent: Starting server: address=127.0.0.1:8500 network=tcp protocol=http
[INFO]  agent: Starting server: address=[::]:8443 network=tcp protocol=https
[INFO]  agent: Started gRPC listeners: port_name=grpc_tls address=127.0.0.1:8503 network=tcp
[INFO]  agent: Consul agent running!
...
[WARN]  agent: Node info update blocked by ACLs: node=bcff042e-614b-e20c-4eda-bcf2fb1233ab accessorID="anonymous token"

Once the Consul server is started, exit the SSH session to return to the bastion host.

$ exit

The output shows the Consul server successfully starting, but some checks are blocked by ACLs. This is because the datacenter is configured to have ACL enabled by default, denying any request that does not present a valid token.

You must bootstrap the ACL system to finish setting up your Consul server.

Configure Consul CLI to interact with Consul server

In order to interact with the Consul server, you need to setup your terminal.

Make sure scenario environment variables are still defined.

$ export CONSUL_DATACENTER="dc1"; \
export CONSUL_DOMAIN="consul"; \
export OUTPUT_FOLDER="./assets/scenario/conf/"; \
export CONSUL_CONFIG_DIR="${OUTPUT_FOLDER}secrets"

Configure the Consul CLI to interact with the Consul server.

$ export CONSUL_HTTP_ADDR="https://consul-server-0:8443" \
export CONSUL_HTTP_SSL=true \
export CONSUL_CACERT="${OUTPUT_FOLDER}secrets/consul-agent-ca.pem" \
export CONSUL_TLS_SERVER_NAME="server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}"

Bootstrap ACLs

Verify that the Consul CLI can reach your Consul server.

$ consul info
Error querying agent: Unexpected response code: 403 (Permission denied: anonymous token lacks permission 'agent:read' on "consul-server-0". The anonymous token is used implicitly when a request does not specify a token.)

The output informs you that while the Consul CLI can reach your Consul server, Consul's ACLs are blocking the request.

Bootstrap the Consul ACL system and save the output in a file named acl-token-bootstrap.json.

$ consul acl bootstrap --format json > ./acl-token-bootstrap.json

The command generates a management token with full permissions over your datacenter. The output will look similar to the following. The management token is the value associated with the SecretID key.

{
  "CreateIndex": 24,
  "ModifyIndex": 24,
  "AccessorID": "a09ee518-f4bf-f1a0-7bdd-2124ef1157af",
  "SecretID": "4becaa33-8fa3-a4c4-6ac5-fed2e20ee3a3",
  "Description": "Bootstrap Token (Global Management)",
  "Policies": [
    {
      "ID": "00000000-0000-0000-0000-000000000001",
      "Name": "global-management"
    }
  ],
  "Local": false,
  "CreateTime": "2022-08-23T14:17:57.686753287Z",
  "Hash": "X2AgaFhnQGRhSSF/h0m6qpX1wj/HJWbyXcxkEM/5GrY="
}

Extract the management token from the file and set it to the CONSUL_HTTP_TOKEN environment variable.

$ export CONSUL_HTTP_TOKEN=`cat ./acl-token-bootstrap.json | jq -r ".SecretID"`

Warning

A management token is a very sensitive information for the security of your Consul datacenter. In these tutorials, we use a management token in the scenarios to simplify the get started experience. We do not recommended for production environments.

After you set the management token, the Consul server logs will signal that you have successfully initialized the ACL system.

[WARN]  agent.server.acl: failed to remove bootstrap file: error="remove /opt/consul/acl-bootstrap-reset: no such file or directory"
[INFO]  agent.server.acl: ACL bootstrap completed

Now that you have bootstrapped the ACL system, you can interact with the Consul server.

$ consul info
agent:
    check_monitors = 0
    check_ttls = 0
    checks = 0
    services = 0
build:
    prerelease =
    revision = 5e08e229
    version = 1.15.2
    version_metadata =
consul:
    acl = enabled
    bootstrap = true
    known_datacenters = 1
    leader = true
    leader_addr = 10.0.4.137:8300
    server = true
## ...

Create server tokens

The Consul datacenter is now fully bootstrapped and is ready to receive requests.

The Consul server logs still contain warnings related to ACL permissions. This is because the server tries to update the node information but the ACL system blocks the requests.

[WARN]  agent: Node info update blocked by ACLs: node=bcff042e-614b-e20c-4eda-bcf2fb1233ab accessorID="anonymous token"
[WARN]  agent: Coordinate update blocked by ACLs: accessorID="anonymous token"

In order to complete configuring the Consul server, you need to create the tokens for the server agents and assign them to the server.

The generate_consul_sever_tokens.sh script automates the process of creating policies and tokens for your Consul server. This script generates ACL policies for Consul DNS service and for the server agent and applies them to the Consul server.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_server_tokens.sh
 [generate_consul_server_tokens.sh] - - Generate Consul server tokens
 -- Cleaning Scenario before apply.
 -- Create policies
 -- Setting environment variables to communicate with Consul
 -- Generate server tokens
ACL token "agent" set successfully
ACL token "default" set successfully

You will create ACL tokens for the Consul DNS service and your server agent. For each token, you will:

define the policy configuration file,
create the Consul policy,
create the ACLs token for the policy,
assign the token to the server agent.

Warning

Manual steps are currently only fully compatible with the interactive environment. While the steps provided are correct, paths and filenames present in the AWS environment might differ. Please double check commands before applying them.

Define the policy configuration files

First, define the DNS policy file.

$ tee ./acl-policy-dns.hcl > /dev/null << EOF
## dns-request-policy.hcl
node_prefix "" {
  policy = "read"
}
service_prefix "" {
  policy = "read"
}
# Required if you use prepared queries
query_prefix "" {
  policy = "read"
}
EOF

Then, define the server policy file.

$ tee ./acl-policy-server-node.hcl > /dev/null << EOF
## consul-server-one-policy.hcl
node_prefix "consul" {
  policy = "write"
}
EOF

Create the Consul policy

First, create the DNS policy using the previously created policy definition.

$ consul acl policy create \
      -name 'acl-policy-dns' \
      -description 'Policy for DNS endpoints' \
      -rules @./acl-policy-dns.hcl

Then, create the server policy using the previously created policy definition.

$ consul acl policy create \
      -name 'acl-policy-server-node' \
      -description 'Policy for Server nodes' \
      -rules @./acl-policy-server-node.hcl

Create ACL tokens for each policy

First, create the DNS token.

$ consul acl token create \
      -description 'DNS - Default token' \
      -policy-name acl-policy-dns \
      --format json > ./acl-token-dns.json

Find the ACL token specific to DNS in the generated file.

{
  "CreateIndex": 44,
  "ModifyIndex": 44,
  "AccessorID": "270eeab9-0d0e-a0c7-8dca-4cde065dc2ea",
  "SecretID": "5e785a4d-035f-4834-82fc-e71a61e7046f",
  "Description": "DNS - Default token",
  "Policies": [
    {
      "ID": "3f6d39c8-844d-0412-cb06-c8f6c0cb4a3e",
      "Name": "acl-policy-dns"
    }
  ],
  "Local": false,
  "CreateTime": "2022-08-23T14:34:43.223378819Z",
  "Hash": "iqlU4q6wel2SC7PtAwzvgb8mYjyU8NnmT9C+kS1hsNM="
}

Then, create the server node token.

$ consul acl token create -description "server agent token" -policy-name acl-policy-server-node  --format json > ./server-acl-token.json

Find the ACL token specific to your server node in the generated file.

{
  "CreateIndex": 47,
  "ModifyIndex": 47,
  "AccessorID": "0a6f1b1b-fc38-1979-4b44-c7cee91966ed",
  "SecretID": "47dc9322-78a2-51d1-c1ac-bcc16959b635",
  "Description": "server agent token",
  "Policies": [
    {
      "ID": "d01854bc-6298-6bce-76e1-0c08bebd9dc9",
      "Name": "acl-policy-server-node"
    }
  ],
  "Local": false,
  "CreateTime": "2022-08-23T14:36:29.932621394Z",
  "Hash": "H3wfyn0nqViLmerl32uM3uRCAcY3JkEb8VX22NRnvBs="
}

Assign tokens to the server agent

After you created the tokens, you will now assign them to your Consul agent.

First, define two environment variables containing the tokens.

$ export DNS_TOKEN=`cat ./acl-token-dns.json | jq -r ".SecretID"`; \
export SERVER_TOKEN=`cat ./server-acl-token.json | jq -r ".SecretID"`

Then, assign the DNS token to the server.

$ consul acl set-agent-token default ${DNS_TOKEN}
ACL token "default" set successfully

Next, assign the server token to the server.

$ consul acl set-agent-token agent ${SERVER_TOKEN}
ACL token "agent" set successfully

After you create the server tokens, your Consul logs will show the updated ACL tokens.

## ...
[INFO]  agent: Updated agent's ACL token: token=agent
## ...
[INFO]  agent: Updated agent's ACL token: token=default
## ...

Interact with Consul server

Now that you have completed configuring and deploying your Consul server, you will interact with it. Consul provides different ways to retrieve information about the datacenter — select the tab(s) for your preferred method.

Use the Consul CLI to retrieve members in your Consul datacenter.

$ consul members
Node             Address         Status  Type    Build   Protocol  DC   Partition  Segment
consul-server-0  10.0.4.52:8301  alive   server  1.16.1  2         dc1  default    <all>

Check the Consul CLI commands reference for the full list of available commands.

Query the Consul API to retrieve members in your Consul datacenter.

$  curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --connect-to server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443:consul-server-0:8443 \
  --cacert assets/scenario/conf/secrets/consul-agent-ca.pem \
  https://server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443/v1/agent/members | jq

This will respond with something similar to the following:

[
  {
    "Name": "consul-server-0",
    "Addr": "10.0.4.241",
    "Port": 8301,
    "Tags": {
      "acls": "1",
      "bootstrap": "1",
      "build": "1.16.1:e0ab4d29",
      "dc": "dc1",
      "ft_fs": "1",
      "ft_si": "1",
      "grpc_tls_port": "8503",
      "id": "253954d5-a05f-464b-d7f4-23195b3c516e",
      "port": "8300",
      "raft_vsn": "3",
      "role": "consul",
      "segment": "",
      "use_tls": "1",
      "vsn": "2",
      "vsn_max": "3",
      "vsn_min": "2",
      "wan_join_port": "8302"
    },
    "Status": 1,
    "ProtocolMin": 1,
    "ProtocolMax": 5,
    "ProtocolCur": 2,
    "DelegateMin": 2,
    "DelegateMax": 5,
    "DelegateCur": 4
  }
]

Check the Consul API reference for the full list of available endpoints.

Retrieve the Consul UI address from Terraform.

$ terraform output -raw ui_consul

Open the address in a browser.

After accepting the certificate presented by the Consul server, you will land on the Services page.

Services page - Default token

Consul UI permissions are the same of the ones associated with the default token configured for the VM where the UI is running. In this scenario, on the Consul server VM, we configured that the token is able to see all nodes and services using the acl-dns-policy.

Security Note

Consider tuning the policy for your own security requirements when planning your production configuration.

If you try to access an unauthorized resource, you will get redirected to a 403 page saying you are not authorized to get information from your Consul datacenter unless you successfully authenticate using a token.

403 Unauthorized

To authenticate, click on Login and enter the management token you created while bootstrapping the Consul server.

Once authenticated, Consul will redirected to the Services page where you will find a healthy and available consul service.

Service Page

Interact with Consul KV

Consul includes a key/value (KV) store that you can use to manage your service's configuration. Even though you can use the KV store using the CLI, API, and UI, this tutorial only covers the CLI and API methods. Select the tab(s) for your preferred method.

Create a key named db_port with a value of 5432.

$ consul kv put consul/configuration/db_port 5432
Success! Data written to: consul/configuration/db_port

Then, retrieve the value.

$ consul kv get consul/configuration/db_port  
5432

Create a key named db_port with a value of 5432.

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --connect-to server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443:consul-server-0:8443 \
  --cacert assets/scenario/conf/secrets/consul-agent-ca.pem \
  --request PUT \
  --data '5432' \
  https://server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443/v1/kv/consul/configuration/db_port

This will respond with something similar to the following:

true

Then, retrieve the value.

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --connect-to server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443:consul-server-0:8443 \
  --cacert assets/scenario/conf/secrets/consul-agent-ca.pem \
  https://server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443/v1/kv/consul/configuration/db_port | jq

This will respond with something similar to the following:

[
  {
    "LockIndex": 0,
    "Key": "consul/configuration/server_number",
    "Flags": 0,
    "Value": "MQ==",
    "CreateIndex": 129,
    "ModifyIndex": 129
  }
]

Notice the response returns the base64 encoded value.

To retrieve the raw value, extract the value and then base64 decode it.

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --connect-to server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443:consul-server-0:8443 \
  --cacert assets/scenario/conf/secrets/consul-agent-ca.pem \
  https://server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443/v1/kv/consul/configuration/db_port | jq -r ".[].Value" | base64 --decode

This will respond with something similar to the following:

Interact with Consul DNS

Consul also provides you with a fully featured DNS server that you can use to resolve the IPs for your services. By default, Consul DNS service is configured to listen on port 8600.

$ dig @consul-server-0 -p 8600 consul.service.consul
; <<>> DiG 9.16.42-Debian <<>> @consul-server-0 -p 8600 consul.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56210
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;consul.service.consul.         IN      A

;; ANSWER SECTION:
consul.service.consul.  0       IN      A       10.0.4.241

;; Query time: 4 msec
;; SERVER: 10.0.4.241#8600(10.0.4.241)
;; WHEN: Thu Aug 31 06:58:26 UTC 2023
;; MSG SIZE  rcvd: 66

Tip

If you want to test Consul DNS interface from the Consul server node, use the following command.

$ dig @127.0.0.1 -p 8600 consul.service.consul

Next steps

In this tutorial, you deployed a Consul server on a VM. After deploying Consul, you interacted with Consul using the CLI, UI, and API.

Even when deployed without Consul clients, you can:

use Consul's KV store as a centralized configuration management tool. You can use it with consul-template to configure your services automatically.
use Consul server as a DNS server. You can use it to register and resolve external services in your network.

(Optional) Destroy infrastructure

If you want to stop at this tutorial, you can destroy the infrastructure now.

From the ./self-managed/infrastruture/aws folder of the repository, use terraform to destroy the infrastructure.

$ terraform destroy --auto-approve

In the next tutorial, you will deploy Consul clients on the VMs hosting your application. Then, you will register the services running on each server and set up health checks for each service. This enables service discovery using Consul's distributed health check system and DNS.

For more information about the topics covered in this tutorial, refer to the following resources:

Collection Overview

Get Started on VMs

Tutorial scenario

Prerequisites

Clone GitHub repository

Create infrastructure

Login into the bastion host VM

Verify Consul binary

Generate Consul server configuration

Test configuration

Copy configuration on Consul server node

Start Consul server

Configure Consul CLI to interact with Consul server

Bootstrap ACLs

Create server tokens

Interact with Consul server

Interact with Consul KV

Interact with Consul DNS

Next steps