Register your services to Consul

27min
|
Consul
Interactive

Workflow

In the previous tutorial, you deployed a Consul server with all security features enabled and explored how to use Consul as a KV store and DNS server.∂ç

Consul's value shines when you use them with Consul clients to serve as a distributed health monitoring platform for your services and as a centralized service catalog.

In this tutorial, you will deploy Consul client agents to your virtual machine (VM) workloads. Then, you will register the services to the Consul catalog and setup a distributed monitoring system using Consul health checks.

In this tutorial, you will use a test environment to:

Configure Consul client agents for the different VMs
Start Consul client instances on your workload VMs
Configure your terminal to communicate with the Consul datacenter
Verify Consul datacenter members
Query Consul catalog using CLI, API and DNS interfaces
Modify a service definition and update the service in Consul catalog

Note

This tutorial is part of the Get Started collection, for this reason all the steps used to configure Consul agents and services are shown and require to be executed manually. If you are setting up a production environment you should codify and automate the installation and deployment process. Refer to the VM production patterns tutorial collection for Consul production deployment best practices.

Tutorial scenario

This tutorial uses HashiCups, a demo coffee shop application made up of several microservices running on VMs.

At the beginning of the tutorial, you have an instance of HashiCups running on four VMs and one Consul server (you deployed this in the previous tutorial).

Architectural diagram 01

By the end of this tutorial, you will have deployed and started a Consul client agent on each VMs that hosts HashiCups. In addition, you will have registered the HashiCups services in the Consul service catalog and setup health checks for each service.

Architectural diagram 02

Prerequisites

This tutorial uses an interactive lab to guide you through how to deploy Consul client agents on the VMs hosting your services. The lab environment includes all required binaries and sample configurations.

Launch Terminal

This tutorial includes a free interactive command-line lab that lets you follow along on actual cloud infrastructure.

Verify Consul binary in service servers

Verify that the VMs you want to deploy the Consul agents on have the Consul binary.

$ consul version
Consul v1.16.0
Revision 192df66a
Build Date 2023-06-26T20:07:11Z
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

Configure environment

This tutorial and interactive lab environment uses scripts in the tutorial's GitHub repository to generate the Consul configuration files for your client agents.

The interactive lab environment includes these scripts and the pre-configured Consul server. In **Bastion Host**, list the files in the `ops` directory.

$ tree ./ops
./ops
`-- scenarios
    `-- 99_supporting_scripts
        |-- generate_consul_client_config.sh
        |-- generate_consul_monitoring_config.sh
        |-- generate_consul_server_config.sh
        |-- generate_consul_server_tokens.sh
        |-- generate_consul_service_config.sh
        `-- generate_consul_service_intentions.sh

3 directories, 6 files

The scripts rely on default parameters to generate the configuration files. Set the following default values. Ensure you have permission to write in the specified paths.

Tip

We suggest using an unprivileged user to run Consul for security reasons.

The lab contains two environment files that will help you configure the terminal and generate the required configuration files.

$ ls -1 assets/scenario/*.env
assets/scenario/env-consul.env
assets/scenario/env-scenario.env

Import the two files in your environment.

$ source assets/scenario/env-scenario.env; \
source assets/scenario/env-consul.env

The script creates all the files in a destination folder. Export the path where you wish to create the configuration files for the scenario.

$ export OUTPUT_FOLDER="./assets/scenario/conf/";

Make sure the folder exists.

$ mkdir -p ${OUTPUT_FOLDER}

Verify your Consul CLI can interact with your Consul server.

$ consul members
Node             Address           Status  Type    Build   Protocol  DC   Partition  Segment
consul-server-0  10.2.79.127:8301  alive   server  1.16.0  2         dc1  default    <all>

Generate Consul clients configuration

The interactive lab can generate the configuration using the info provided via the environment files previously loaded.

Since the Consul datacenter is configured with ACL enabled by default, you will need to define the ACL tokens you want to pass to the Consul clients when the configuration gets created.

First, export the token you generated for DNS so you can use it as default token for clients.

$ export CONSUL_DNS_TOKEN=`cat ${OUTPUT_FOLDER}secrets/acl-token-dns.json | jq -r ".SecretID"`

Then, for Consul service definition to be created properly, define a Consul token to be included in the service definition file.

In this example, you will use the bootstrap token.

$ export CONSUL_AGENT_TOKEN="${CONSUL_HTTP_TOKEN}"

Generate configuration for Database agent

First, define the Consul node name.

$ export NODE_NAME="hashicups-db"

Then, generate the Consul configuration.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_client_config.sh
[generate_consul_client_config.sh] - - [hashicups-db]
-- Parameter Check
-- Cleaning Scenario before apply.
-- [WARN] Removing pre-existing configuration in ./assets/scenario/conf/
-- Generate folder structure
-- Copy available configuration
-- Generating configuration for Consul agent hashicups-db

To complete Consul agent configuration, you need to setup tokens for the client. For this tutorial, you are using the bootstrap token. We recommend you create more restrictive tokens for the client agents in production.

$ tee ${OUTPUT_FOLDER}${NODE_NAME}/agent-acl-tokens.hcl > /dev/null << EOF
acl {
  tokens {
    agent  = "${CONSUL_HTTP_TOKEN}"
    default  = "${CONSUL_DNS_TOKEN}"
    config_file_service_registration = "${CONSUL_HTTP_TOKEN}"
  }
}
EOF

Once Consul agent configuration is generated, you can add the service configuration to the output folder. This will make sure your services are configured automatically at Consul startup.

First, generate the service configuration for the agent.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_service_config.sh
[generate_consul_service_config.sh] - [hashicups-db]
-- Parameter Check
-- Generating service definition for service discovery
-- Generating service definition for service mesh

Then, copy the service definition file into the general configuration directory.

$ cp ${OUTPUT_FOLDER}${NODE_NAME}/svc/service_discovery/*.hcl ${OUTPUT_FOLDER}${NODE_NAME}

Generate configuration for API agent

First, define the Consul node name.

$ export NODE_NAME="hashicups-api"

Then, generate the Consul configuration.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_client_config.sh
[generate_consul_client_config.sh] - - [hashicups-api]
-- Parameter Check
-- Cleaning Scenario before apply.
-- [WARN] Removing pre-existing configuration in ./assets/scenario/conf/
-- Generate folder structure
-- Copy available configuration
-- Generating configuration for Consul agent hashicups-api

$ tee ${OUTPUT_FOLDER}${NODE_NAME}/agent-acl-tokens.hcl > /dev/null << EOF
acl {
  tokens {
    agent  = "${CONSUL_HTTP_TOKEN}"
    default  = "${CONSUL_DNS_TOKEN}"
    config_file_service_registration = "${CONSUL_HTTP_TOKEN}"
  }
}
EOF

Once Consul agent configuration is generated, you can add the service configuration to the output folder. This will make sure your services are configured automatically at Consul startup.

First, generate the service configuration for the agent.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_service_config.sh
[generate_consul_service_config.sh] - [hashicups-api]
-- Parameter Check
-- Generating service definition for service discovery
-- Generating service definition for service mesh

Then, copy the service definition file into the general configuration directory.

$ cp ${OUTPUT_FOLDER}${NODE_NAME}/svc/service_discovery/*.hcl ${OUTPUT_FOLDER}${NODE_NAME}

Generate configuration for Frontend agent

First, define the Consul node name.

$ export NODE_NAME="hashicups-frontend"

Then, generate the Consul configuration.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_client_config.sh
[generate_consul_client_config.sh] - - [hashicups-frontend]
-- Parameter Check
-- Cleaning Scenario before apply.
-- [WARN] Removing pre-existing configuration in ./assets/scenario/conf/
-- Generate folder structure
-- Copy available configuration
-- Generating configuration for Consul agent hashicups-frontend

$ tee ${OUTPUT_FOLDER}${NODE_NAME}/agent-acl-tokens.hcl > /dev/null << EOF
acl {
  tokens {
    agent  = "${CONSUL_HTTP_TOKEN}"
    default  = "${CONSUL_DNS_TOKEN}"
    config_file_service_registration = "${CONSUL_HTTP_TOKEN}"
  }
}
EOF

Once Consul agent configuration is generated, you can add the service configuration to the output folder. This will make sure your services are configured automatically at Consul startup.

First, generate the service configuration for the agent.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_service_config.sh
[generate_consul_service_config.sh] - [hashicups-frontend]
-- Parameter Check
-- Generating service definition for service discovery
-- Generating service definition for service mesh

Then, copy the service definition file into the general configuration directory.

$ cp ${OUTPUT_FOLDER}${NODE_NAME}/svc/service_discovery/*.hcl ${OUTPUT_FOLDER}${NODE_NAME}

Generate configuration for NGINX agent

First, define the Consul node name.

$ export NODE_NAME="hashicups-nginx"

Then, generate the Consul configuration.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_client_config.sh
[generate_consul_client_config.sh] - - [hashicups-nginx]
-- Parameter Check
-- Cleaning Scenario before apply.
-- [WARN] Removing pre-existing configuration in ./assets/scenario/conf/
-- Generate folder structure
-- Copy available configuration
-- Generating configuration for Consul agent hashicups-nginx

$ tee ${OUTPUT_FOLDER}${NODE_NAME}/agent-acl-tokens.hcl > /dev/null << EOF
acl {
  tokens {
    agent  = "${CONSUL_HTTP_TOKEN}"
    default  = "${CONSUL_DNS_TOKEN}"
    config_file_service_registration = "${CONSUL_HTTP_TOKEN}"
  }
}
EOF

Once Consul agent configuration is generated, you can add the service configuration to the output folder. This will make sure your services are configured automatically at Consul startup.

First, generate the service configuration for the agent.

$ bash ./ops/scenarios/99_supporting_scripts/generate_consul_service_config.sh
[generate_consul_service_config.sh] - [hashicups-nginx]
-- Parameter Check
-- Generating service definition for service discovery
-- Generating service definition for service mesh

Then, copy the service definition file into the general configuration directory.

$ cp ${OUTPUT_FOLDER}${NODE_NAME}/svc/service_discovery/*.hcl ${OUTPUT_FOLDER}${NODE_NAME}

Check generated files

Once you have generated the configuration files, your directory should look like the following:

$ tree ${OUTPUT_FOLDER}hashicups*
./assets/scenario/conf/hashicups-api
├── agent-acl-tokens.hcl
├── agent-gossip-encryption.hcl
├── consul-agent-ca.pem
├── consul.hcl
├── svc
│   ├── service_discovery
│   │   └── svc-hashicups-api.hcl
│   └── service_mesh
│       └── svc-hashicups-api.hcl
└── svc-hashicups-api.hcl
./assets/scenario/conf/hashicups-db
├── agent-acl-tokens.hcl
├── agent-gossip-encryption.hcl
├── consul-agent-ca.pem
├── consul.hcl
├── svc
│   ├── service_discovery
│   │   └── svc-hashicups-db.hcl
│   └── service_mesh
│       └── svc-hashicups-db.hcl
└── svc-hashicups-db.hcl
./assets/scenario/conf/hashicups-frontend
├── agent-acl-tokens.hcl
├── agent-gossip-encryption.hcl
├── consul-agent-ca.pem
├── consul.hcl
├── svc
│   ├── service_discovery
│   │   └── svc-hashicups-frontend.hcl
│   └── service_mesh
│       └── svc-hashicups-frontend.hcl
└── svc-hashicups-frontend.hcl
./assets/scenario/conf/hashicups-nginx
├── agent-acl-tokens.hcl
├── agent-gossip-encryption.hcl
├── consul-agent-ca.pem
├── consul.hcl
├── svc
│   ├── service_discovery
│   │   └── svc-hashicups-nginx.hcl
│   └── service_mesh
│       └── svc-hashicups-nginx.hcl
└── svc-hashicups-nginx.hcl

12 directories, 28 files

Each directory corresponds to a Consul client configuration for the respective node.

The scripts generated multiple configuration files so it is easier to read and tune them for your environment. The following are the generated files and a description of what they do:

The agent-acl-tokens.hcl file contains tokens for the Consul agent.
The agent-gossip-encryption.hcl file configures gossip encryption.
The consul-agent-ca.pem file is the public certificate for Consul CA.
The consul.hcl file contains node specific configuration and it is needed, with this specific name, if you want to configure Consul as a systemd daemon.
The svc-hashicups-*.hcl files contains the service definition for the services that need to be registered on each node.

Visit the agent configuration and service configuration documentation to interpret the files or to modify them when applying them to your environment.

Copy configuration on client VMs

After the script generates the client configuration, you will copy these files into the respective Consul configuration directories in each client node.

Tip

In the lab environment, the HashiCups application nodes have a running SSH server. As a result, you can use ssh and scp commands to perform the following operations. If the nodes in your personal environment does not have an SSH server, you may need to use a different approach to create the configuration directories and copy the files.

First, define the Consul configuration directory.

$ export CONSUL_REMOTE_CONFIG_DIR=/etc/consul.d/

Then, remove existing configuration from the VMs.

$ ssh -i certs/id_rsa hashicups-db "sudo rm -rf ${CONSUL_REMOTE_CONFIG_DIR}*"; \
  ssh -i certs/id_rsa hashicups-api "sudo rm -rf ${CONSUL_REMOTE_CONFIG_DIR}*"; \
  ssh -i certs/id_rsa hashicups-frontend "sudo rm -rf ${CONSUL_REMOTE_CONFIG_DIR}*"; \
  ssh -i certs/id_rsa hashicups-nginx "sudo rm -rf ${CONSUL_REMOTE_CONFIG_DIR}*";

Finally, copy the configuration files into the different VMs.

$ scp -O -i certs/id_rsa ${OUTPUT_FOLDER}/hashicups-db/* hashicups-db:${CONSUL_REMOTE_CONFIG_DIR}; \
  scp -O -i certs/id_rsa ${OUTPUT_FOLDER}/hashicups-api/* hashicups-api:${CONSUL_REMOTE_CONFIG_DIR}; \
  scp -O -i certs/id_rsa ${OUTPUT_FOLDER}/hashicups-frontend/* hashicups-frontend:${CONSUL_REMOTE_CONFIG_DIR}; \
  scp -O -i certs/id_rsa ${OUTPUT_FOLDER}/hashicups-nginx/* hashicups-nginx:${CONSUL_REMOTE_CONFIG_DIR}

Start Consul on client VMs

Now that you have copied the configuration files to each client VMs, you will start the Consul client agent on each VM.

Setup Database Consul client agent

Select the tab that corresponds with the service — in this case, Database.

Define the Consul configuration and data directories.

$ export CONSUL_CONFIG_DIR=/etc/consul.d/ \
export CONSUL_DATA_DIR=/opt/consul/

Ensure your user has write permission to the Consul data directory.

$ sudo chmod g+w ${CONSUL_DATA_DIR}

Finally, start the Consul server process.

$ consul agent -config-dir=${CONSUL_CONFIG_DIR} > /tmp/consul-client.log 2>&1 &

The command starts the Consul server in the background to not lock the terminal. You can access the Consul server log through the /tmp/consul-server.log file.

Setup API Consul client agent

Select the tab that corresponds with the service — in this case, API.

Define the Consul configuration and data directories.

$ export CONSUL_CONFIG_DIR=/etc/consul.d/ \
export CONSUL_DATA_DIR=/opt/consul/

Ensure your user has write permission to the Consul data directory.

$ sudo chmod g+w ${CONSUL_DATA_DIR}

Finally, start the Consul server process.

$ consul agent -config-dir=${CONSUL_CONFIG_DIR} > /tmp/consul-client.log 2>&1 &

The process is started in background to not lock the terminal. Consul server log can be accessed in the /tmp/consul-client.log file.

Setup Frontend Consul client agent

Select the tab that corresponds with the service — in this case, Frontend.

Define the Consul configuration and data directories.

$ export CONSUL_CONFIG_DIR=/etc/consul.d/ \
export CONSUL_DATA_DIR=/opt/consul/

Ensure your user has write permission to the Consul data directory.

$ sudo chmod g+w ${CONSUL_DATA_DIR}

Finally, start the Consul server process.

$ consul agent -config-dir=${CONSUL_CONFIG_DIR} > /tmp/consul-client.log 2>&1 &

The process is started in background to not lock the terminal. Consul server log can be accessed in the /tmp/consul-client.log file.

Setup NGINX Consul client agent

Select the tab that corresponds with the service — in this case, NGINX.

Define the Consul configuration and data directories.

$ export CONSUL_CONFIG_DIR=/etc/consul.d/ \
export CONSUL_DATA_DIR=/opt/consul/

Ensure your user has write permission to the Consul data directory.

$ sudo chmod g+w ${CONSUL_DATA_DIR}

Finally, start the Consul server process.

$ consul agent -config-dir=${CONSUL_CONFIG_DIR} > /tmp/consul-client.log 2>&1 &

The process is started in background to not lock the terminal. Consul server log can be accessed in the /tmp/consul-client.log file.

Verify Consul datacenter members

After you started all Consul agents, verify they successfully joined the Consul datacenter. If you are using the interactive lab environment, go to the Bastion Host tab.

Retrieve the agents in the Consul datacenter.

$ consul members
Node                Address          Status  Type    Build   Protocol  DC   Partition  Segment
consul-server-0     10.0.4.241:8301  alive   server  1.16.1  2         dc1  default    <all>
hashicups-api       10.0.4.179:8301  alive   client  1.16.1  2         dc1  default    <default>
hashicups-db        10.0.4.7:8301    alive   client  1.16.1  2         dc1  default    <default>
hashicups-frontend  10.0.4.59:8301   alive   client  1.16.1  2         dc1  default    <default>
hashicups-nginx     10.0.4.146:8301  alive   client  1.16.1  2         dc1  default    <default>

Use the /v1/agent/members API endpoint to retrieve the agents in the Consul datacenter.

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --connect-to server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443:consul-server-0:8443 \
  --cacert ${CONSUL_CACERT} \
  https://server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443/v1/agent/members | jq

This will respond with something similar to the following:

[
  {
    "Name": "consul-server-0",
    "Addr": "10.0.4.241",
    "Port": 8301,
    ## ...
  },
  {
    "Name": "hashicups-db",
    "Addr": "10.0.4.7",
    "Port": 8301,
    ## ...
  },
  {
    "Name": "hashicups-api",
    "Addr": "10.0.4.179",
    "Port": 8301,
    ## ...
  },
  {
    "Name": "hashicups-frontend",
    "Addr": "10.0.4.59",
    "Port": 8301,
    ## ...
  },
  {
    "Name": "hashicups-nginx",
    "Addr": "10.0.4.146",
    "Port": 8301,
    ## ...
  }
]

Query services in Consul catalog

When you started the Consul client agents, they registered the service running on their node into the Consul catalog. Each service definition also contained the service's health check. You can find the service definition files in each node's respective Consul configuration file you defined earlier.

Query the healthy services using the Consul CLI, API, or DNS.

Use the Consul CLI to query the service catalog.

$ consul catalog services -tags
consul
hashicups-api           v1
hashicups-db            v1
hashicups-frontend      v1
hashicups-nginx         v1

Use the /v1/catalog/services API endpoint to get a list of the services in the Consul catalog.

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --connect-to server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443:consul-server-0:8443 \
  --cacert ${CONSUL_CACERT} \
  https://server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443/v1/catalog/services | jq

This will respond with something similar to the following:

{
  "consul": [],
  "hashicups-api": [
    "v1"
  ],
  "hashicups-db": [
    "v1"
  ],
  "hashicups-frontend": [
    "v1"
  ],
  "hashicups-nginx": [
    "v1"
  ]
}

The output shows the tags (v1) associated with each service instance.

If you want more information on a specific service, use the /v1/catalog/service/:service API endpoint.

Retrieve more information about the database service.

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --connect-to server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443:consul-server-0:8443 \
  --cacert ${CONSUL_CACERT} \
  https://server.${CONSUL_DATACENTER}.${CONSUL_DOMAIN}:8443/v1/catalog/service/hashicups-db | jq

This will respond with something similar to the following:

[
  {
    "ID": "873442d2-f63c-8203-874d-d7f6bb90f277",
    "Node": "hashicups-db",
    "Address": "10.0.4.217",
    "Datacenter": "dc1",
    "TaggedAddresses": {
      "lan": "10.0.4.217",
      "lan_ipv4": "10.0.4.217",
      "wan": "10.0.4.217",
      "wan_ipv4": "10.0.4.217"
    },
    "NodeMeta": {
      "consul-network-segment": ""
    },
    "ServiceKind": "",
    "ServiceID": "hashicups-db-1",
    "ServiceName": "hashicups-db",
    "ServiceTags": [
      "v1"
    ],
    "ServiceAddress": "",
    "ServiceWeights": {
      "Passing": 1,
      "Warning": 1
    },
    "ServiceMeta": {},
    "ServicePort": 5432,
    "ServiceSocketPath": "",
    "ServiceEnableTagOverride": false,
    "ServiceProxy": {
      "Mode": "",
      "MeshGateway": {},
      "Expose": {}
    },
    "ServiceConnect": {},
    "CreateIndex": 50,
    "ModifyIndex": 50
  }
]

Use the Consul DNS service to resolve the service name into an IP address. Consul uses SERVICE_NAME.service.DATACENTER.DOMAIN format to query services.

Retrieve the database service's IP address.

$ dig @consul-server-0 -p 8600 hashicups-db.service.dc1.consul ANY
; <<>> DiG 9.16.42-Debian <<>> @consul-server-0 -p 8600 hashicups-db.service.dc1.consul ANY
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 50824
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;hashicups-db.service.dc1.consul. IN    ANY

;; AUTHORITY SECTION:
consul.                 0       IN      SOA     ns.consul. hostmaster.consul. 1693466220 3600 600 86400 0

;; Query time: 4 msec
;; SERVER: 10.0.4.241#8600(10.0.4.241)
;; WHEN: Thu Aug 31 07:17:00 UTC 2023
;; MSG SIZE  rcvd: 110

Tip

If you want to test Consul DNS interface from the Consul server node, use the following command.

$ dig @127.0.0.1 -p 8600 consul.service.consul

Modify service definition tags

When using Consul CLI or the API endpoints, Consul will also show you the metadata associated with the services. In this tutorial, you registered each service with the v1 tag.

In this section, you will update the database service definition to learn how to update Consul service definitions. You must run these commands on the virtual machine that hosts the services.

Select the tab that corresponds with the service — in this case, Database.

Create the new configuration file with the following contents. Notice that this configuration adds a v2 tag to the database service.

## -----------------------------
## svc-hashicups-db.hcl
## -----------------------------
service {
  name = "hashicups-db"
  id = "hashicups-db-1"
  tags = ["v1", "v2"]
  port = 5432


  check
  {
    id =  "check-hashicups-db",
    name = "hashicups-db status check",
    service_id = "hashicups-db-1",
    tcp  = "localhost:5432",
    interval = "5s",
    timeout = "5s"
  }
}

{
  "service": {
    "id": "hashicups-db-1",
    "name": "hashicups-db",
    "tags": ["v1", "v2"],
    "port": 5432,
    "checks": [
      {
        "ID" : "check-hashicups-db",
        "Name" : "hashicups-db status check",
        "ServiceID" : "hashicups-db-1",
        "TCP": "localhost:5432",
        "interval": "1s",
        "Timeout": "1s"
      }
    ]
  }
}

{
  "ID": "hashicups-db-1",
  "Name": "hashicups-db",
  "Tags": ["v1", "v2"],
  "Port": 5432,
  "Checks": [
    {
      "CheckID" : "check-hashicups-db",
      "Name" : "hashicups-db status check",
      "ServiceID" : "hashicups-db-1",
      "TCP": "localhost:5432",
      "Interval": "1s",
      "Timeout": "1s"
    }
  ]
}

Once you have created the new service definition file, update the service in the Consul catalog.

Consul can automatically update some of its configuration by reloading the content of the configuration folder.

To use this feature, move the svc-hashicups-db.hcl file into the Consul configuration directory (/etc/consul.d).

$ mv svc-hashicups-db.hcl /etc/consul.d/

Setup the environment variables to connect with Consul.

$ export CONSUL_CONFIG_DIR="/etc/consul.d"; \
export CONSUL_HTTP_ADDR="localhost:8500"; \
export CONSUL_HTTP_TOKEN=`cat ${CONSUL_CONFIG_DIR}/agent-acl-tokens.hcl | grep agent | awk '{print $3}'| sed 's/"//g'`

Then, use the reload command to update the service definition.

$ consul reload
Configuration reload triggered

Setup the environment variables to connect with Consul.

$ export CONSUL_CONFIG_DIR="/etc/consul.d"; \
export CONSUL_HTTP_ADDR="localhost:8500"; \
export CONSUL_HTTP_TOKEN=`cat ${CONSUL_CONFIG_DIR}/agent-acl-tokens.hcl | grep agent | awk '{print $3}'| sed 's/"//g'`

Then, update the service in Consul catalog. You may need to update the file extension if you created a JSON service definition.

$ consul services register svc-hashicups-db.hcl
Registered service: db

When the Consul client restarts, it will load the service definitions in the Consul configuration directory. As a result, we recommend using the automatic (reload) method for a more persistent solution.

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
  --data @svc-hashicups-db-api.json \
  --request PUT \
  http://127.0.0.1:8500/v1/agent/service/register

Query services by tags

After you have updated the database service definition, query it to verify the new tag.

Retrieve the tags associated with each service and verify the new v2 tag for the database service.

$ consul catalog services -tags
consul
hashicups-api           v1
hashicups-db            v1,v2
hashicups-frontend      v1
hashicups-nginx         v1

Use the /v1/catalog/services API endpoint to get a list of the services in the Consul catalog and verify the new v2 tag for the database service.

$ curl --silent \
  --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
   http://127.0.0.1:8500/v1/catalog/services | jq

This will respond with something similar to the following:

{
  "consul": [],
  "hashicups-api": [
    "v1"
  ],
  "hashicups-db": [
    "v2",
    "v1"
  ],
  "hashicups-frontend": [
    "v1"
  ],
  "hashicups-nginx": [
    "v1"
  ]
}

The DNS service does not expose the tags information, since tags are an internal Consul metadata. Consul exposes the tags as an extra domain associated with the service in the form: TAG.SERVICE_NAME.service.DATACENTER.DOMAIN.

Retrieve the database service's IP address using the v2 tag.

$ dig @127.0.0.1 -p 8600 v2.hashicups-db.service.dc1.consul
; <<>> DiG 9.16.42-Debian <<>> @127.0.0.1 -p 8600 v2.hashicups-db.service.dc1.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 37270
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;v2.hashicups-db.service.dc1.consul. IN A

;; AUTHORITY SECTION:
consul.                 0       IN      SOA     ns.consul. hostmaster.consul. 1693466350 3600 600 86400 0

;; Query time: 3 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Thu Aug 31 07:19:10 UTC 2023
;; MSG SIZE  rcvd: 113

Tip

The previous command is expected to be run a Consul client node. If you want to test Consul DNS interface from the bastion host use the following command.

$ dig @consul-server-0 -p 8600 v2.hashicups-db.service.dc1.consul

Next steps

In this tutorial, you deployed Consul clients on each HashiCups' nodes VMs. In addition, you configured Consul to perform health checks on the registered services and updated a service definition.

You now have a distributed system to monitor and resolve your services all without changing your services' configuration or implementation. At this stage, you can use Consul to automatically configure and monitor your services. However, they have the same security they had before introducing Consul.

In the next tutorial, you will learn how to introduce zero trust security in your network by implementing Consul service mesh.

For more information about the topics covered in this tutorial, refer to the following resources:

Deploy Consul on VMs

Securely connect your services