Vault Agent Templates

10min
|
Terraform
Vault
Interactive

The adoption of Vault is an incremental journey. First, you move your secrets into Vault so that they are securely encrypted and stored. The next step is to update your applications' behavior to use secrets from Vault.

Clients must authenticate with Vault and get a token before accessing authenticated endpoints. Vault Agent was introduced to reduce the credential lifecycle management burden for distributed applications.

The following tutorials demonstrate the Vault Agent Auto-Auth method:

In addition, the Vault Agent Caching tutorial walked through the Caching feature introduced in Vault 1.1.

Launch Terminal

This tutorial includes a free interactive command-line lab that lets you follow along on actual cloud infrastructure.

Challenge and solution

After the successful authentication, Vault clients can start interacting with the Vault. Many Vault users adopted the Consul Template tool to minimize the level of changes introduced to their existing applications.

In this case, Consul Template is the client directly interacting with Vault; therefore, you had to configure Consul Template to read the token from the agent's sink location in order to interact with Vault. This requires you to operate two distinct tools to provide secrets to applications.

In Vault 1.3, Vault Agent introduced Vault Agent Templates allowing Vault secrets to be rendered to files using the Consul Template markup language. This significantly simplifies the workflow when you are integrating your applications with Vault.

Note

This tutorial focuses on the templates feature of Vault Agent assuming that you are already familiar with Vault Agent Auto-Auth. If you are new to Vault Agent, try the Vault Agent with AWS tutorial or the Vault Agent with Kubernetes tutorial before continuing with this tutorial.

Prerequisites

To complete this section of the tutorial, you will need:

AWS account and associated credentials that allow for the creation of resources
Vault version 1.3 or later
Terraform installed

Provision the cloud resources

Clone the demo assets from the hashicorp-education/learn-vault-agent-templates GitHub repository to perform the steps described in this tutorial.
```
$ git clone https://github.com/hashicorp-education/learn-vault-agent-templates.git
```
This repository contains supporting content for all of the tutorial.

Be sure to set your working directory to where the terraform-aws directory is located.

$ cd learn-vault-agent-templates

The following assets are located under this directory:

$ tree

Example output:

.
├── LICENSE
├── README.md
└── terraform-aws
   ├── aws.tf
   ├── iam.tf
   ├── kms.tf
   ├── network.tf
   ├── outputs.tf
   ├── security-groups.tf
   ├── templates
   │   ├── userdata-vault-client.tpl
   │   └── userdata-vault-server.tpl
   ├── terraform.tfvars.example
   ├── variables.tf
   ├── vault-client.tf
   ├── vault-server.tf
   └── versions.tf

Note

The example Terraform configuration in this repository was designed just for demonstration purposes.

Set an AWS_ACCESS_KEY_ID environment variable to hold your AWS access key ID.
```
$ export AWS_ACCESS_KEY_ID = "<YOUR_AWS_ACCESS_KEY_ID>"
```
Set an AWS_SECRET_ACCESS_KEY environment variable to hold your AWS secret access key.
```
$ export AWS_SECRET_ACCESS_KEY = "<YOUR_AWS_SECRET_ACCESS_KEY>"
```
Tip
The above example uses IAM user authentication. You can use any authentication method described in the AWS provider documentation.
Create a file named terraform.tfvars and specify the key_name. (You can copy the terraform.tfvars.example file and rename it as terraform.tfvars.)
Example terraform.tfvars:
```
# SSH key name to access EC2 instances (should already exist) on the AWS region
key_name = "my-key-pair"
```
Tip
If you don't have an EC2 Key Pair, refer to the AWS documentation and create one.
Run terraform init to initialized the Terraform workspace, and download the necessary provider resources.
```
$ terraform init
```
Run terraform plan to verify your changes, and the resources that Terraform will create when the plan gets applied.
```
$ terraform plan
```
If all looks good, run terraform apply to provision the resources; in this example, approval to apply the plan is skipped for convenience with the -auto-approve flag.

Note

This step creates actual real-world resources in AWS. Be mindful of their presence going forward, and be sure to follow the suggestion clean up steps at the conclusion of this tutorial.

$ terraform apply -auto-approve

Example expected output:

...
Apply complete! Resources: 20 added, 0 changed, 0 destroyed.

Outputs:

endpoints =
Vault Server IP (public):  192.0.2.224
Vault Server IP (private): 10.0.101.62

For example:
  ssh -i vault-test.pem ubuntu@192.0.2.224

Vault Client IP (public):  198.51.100.24
Vault Client IP (private): 10.0.101.10

For example:
  ssh -i vault-test.pem ubuntu@198.51.100.24

The Terraform output will display the public IP address to SSH into your Vault server and client instances.

Configure AWS IAM auth method

Note

This step should be performed on the Vault server instance.

In this step, you configure Vault to allow AWS IAM authentication from specific IAM roles.

SSH into the Vault server instance.
When you are prompted, enter "yes" to continue.
```
$ ssh -i <path_to_key> ubuntu@<public_ip_of_server>
```
Example expected output:
```
...
Are you sure you want to continue connecting (yes/no)? yes
```
Note
If you get a Permissions 0664 for '<key_name>.pem' are too open error, be sure to appropriately set the file permission.
```
$ chmod 600 <key_name>.pem
```
Run the vault operator init command to initialize Vault. For the convenience of this tutorial, save the generated recovery keys and the initial root token in a file named key.txt.
```
$ vault operator init > key.txt
```
The Vault server is configured to auto-unseal with AWS Key Management Service (KMS); therefore, once the server is initialized, it gets automatically unsealed. To learn how to auto-unseal Vault using AWS KMS, refer to the Auto-unseal using AWS KMS tutorial.

Log into Vault using the generated initial root token save in the key.txt

$ vault login $(grep 'Initial Root Token:' key.txt | awk '{print $NF}')

Examine the /home/ubuntu/aws_auth.sh script.
```
$ cat aws_auth.sh
```

Execute the /home/ubuntu/aws_auth.sh script.

$ ./aws_auth.sh

Success! Enabled the kv-v2 secrets engine at: secret/
Key              Value
---              -----
created_time     2019-12-04T00:16:07.107076447Z
deletion_time    n/a
destroyed        false
version          1
Success! Uploaded policy: app-pol
Success! Enabled aws auth method at: aws/
Success! Data written to: auth/aws/config/client
Success! Data written to: auth/aws/role/app-role

The script enabled key/value v2 secrets engine at secret and wrote some data at secret/customers/acme. It also created an app-pol policy, enabled and configured aws auth method, and created a role named app-role.

Read the app-pol policy.

$ vault policy read app-pol

path "secret/data/*" {
  capabilities = ["read"]
}
path "auth/token/*" {
  capabilities = ["create", "update"]
}

Get the secrets written at secret/customers/acme

$ vault kv get secret/customers/acme
====== Metadata ======
Key              Value
---              -----
created_time     2019-12-04T00:16:07.107076447Z
deletion_time    n/a
destroyed        false
version          1

======== Data ========
Key              Value
---              -----
contact_email    james@acme.com
customer_id      ABXX2398YZPIE7391
organization     ACME Inc.
region           US-West
status           active
type             premium
zip_code         94105

Start Vault Agent

Note

This step should be performed on the Vault Client instance.

In the client node, Vault Agent authenticates with Vault via aws auth method you configured in the configure AWS IAM auth method step. Using the acquired token, the agent pulls secrets from Vault defined by the template file.

SSH into the Vault Client instance.

Output

When you are prompted, enter "yes" to continue.

$ ssh -i <path_to_key> ubuntu@<public_ip_of_client>

Example output:

...
Are you sure you want to continue connecting (yes/no)? yes

Explore the Vault Agent configuration file, /home/ubuntu/vault-agent.hcl.

$ cat vault-agent.hcl

The contents of the file resembles this example:

pid_file = "./pidfile"

auto_auth {
  method "aws" {
    mount_path = "auth/aws"
    config = {
      type = "iam"
      role = "app-role"
    }
  }

  sink "file" {
    config = {
      path = "/home/ubuntu/vault-token-via-agent"
    }
  }
}

vault {
  address = "http://10.0.101.62:8200"
}

template {
  source      = "/home/ubuntu/customer.tmpl"
  destination = "/home/ubuntu/customer.txt"
}

The Vault Agent uses the aws auth method to authenticate with the Vault server running on the server instance as app-role. Also notice that there is template block which sets /home/ubuntu/customer.tmpl as the source template file and the output will be written to the destination, /home/ubuntu/customer.txt.

Execute the following command to start the Vault Agent with log level set to debug.

$ vault agent -config=/home/ubuntu/vault-agent.hcl -log-level=debug

You should get output similar to the following:

[INFO]  sink.file: creating file sink
[INFO]  sink.file: file sink configured: path=/home/ubuntu/vault-token-via-agent mode=-rw-r-----
[INFO]  auth.handler: starting auth handler
[INFO]  auth.handler: authenticating
[INFO]  template.server: starting template server
...
[INFO]  auth.handler: authentication successful, sending token to sinks
...
[DEBUG] (runner) running initial templates
[DEBUG] (runner) initiating run
[DEBUG] (runner) checking template 56ba7f3e29857b85850ab5e0eb1151a3
...
[DEBUG] (runner) receiving dependency vault.read(secret/data/customers/acme)
[DEBUG] (runner) initiating run
[DEBUG] (runner) checking template 56ba7f3e29857b85850ab5e0eb1151a3
[DEBUG] (runner) rendering "/home/ubuntu/customer.tmpl" => "/home/ubuntu/customer.txt"

Open a second SSH terminal into the client machine.
Verify the client token stored in /home/ubuntu/vault-token-via-agent.
```
$ more vault-token-via-agent
```
Example output:
```
s.4G9iuH8MtZqrvYtOmWeaGqnJ
```

Check the details about the token (e.g. attached policies, TTL, etc.).

$ VAULT_TOKEN="$(cat vault-token-via-agent)" vault token lookup

Example output:

Key                  Value
---                  -----
...
orphan               true
path                 auth/aws/login
policies             [app-pol default]
renewable            true
ttl                  22h22m17s
type                 service

The app-pol policy is attached to the token. Remember that the app-pol policy grants read operation against the secret/data/* path.

Review the customer.tmpl file which is written using the Consul Templates markup language.

$ cat customer.tmpl

Example output:

{{ with secret "secret/data/customers/acme" }}
Organization: {{ .Data.data.organization }}
ID: {{ .Data.data.customer_id }}
Contact: {{ .Data.data.contact_email }}
{{ end }}

Notice that the secret path is set to secret/data/customers/acme.

Examine the resulting customer.txt file.
```
$ cat customer.txt
```
```
Organization: ACME Inc.
ID: ABXX2398YZPIE7391
Contact: james@acme.com
```
Based on the customer.tmpl file, Vault Agent Templates read the secrets at secret/data/customers/acme and then generated the customer.txt file.

Challenge

What happens if the data at secret/data/customers/acme was updated?

On the server instance terminal, update the contact_email to jenn@acme.com.

$ vault kv patch secret/customers/acme contact_email="jenn@acme.com"

Verify updates to the data.
```
$ vault kv get secret/customers/acme
```
Return to the client instance SSH session where the Vault Agent is running. Wait for a few minutes (about 4 minutes). The agent pulls the secrets again.
```
...
[DEBUG] (runner) checking template 494ea5cbe4765bfe2e5eca2363eff06b
[DEBUG] (runner) rendering "./customer.tmpl" => "./customer.txt"
[INFO] (runner) rendered "./customer.tmpl" => "./customer.txt"
[DEBUG] (runner) diffing and updating dependencies
[DEBUG] (runner) vault.read(secret/data/customers/acme) is still needed
[DEBUG] (runner) watching 1 dependencies
[DEBUG] (runner) all templates rendered
...
```
When reading secrets from Key/Value v2 secrets engine, omitting the ?version parameter reads the latest version of the secrets at the path. If you want to always pull a specific version of the secret, specify the desired version number. For example, {{ with secret "secret/data/customers/acme?version=3" }} will always read the version 3 of the secret/data/customers/acme values.

Clean up

Execute the following commands to destroy cloud resources.

$ terraform apply -destroy -auto-approve

At this point, you can delete the Terraform state files from the directory.

$ rm -rf .terraform terraform.tfstate*

Summary and reference

Vault Agent is a client daemon that solves the secret-zero problem by authenticating with Vault and manage the client tokens on behalf of the client applications. The Consul Template tool is widely adopted by the Vault users since it allowed applications to be "Vault-unaware".

Vault Agent Templates combines the best of the two tools to make the end-to-end workflow even simpler.

Vault Agent Templates

Challenge and solution

Prerequisites

Provision the cloud resources

Configure AWS IAM auth method

Start Vault Agent

Challenge

Clean up

Summary and reference

Help and reference

Other Vault Agent tutorials: