HashiConf 2024 Now streaming live from Boston! Attend for free
Consul
Archive

Migrate Legacy ACL Tokens

  • 15min
  • |
  • Consul

Compatibility notice

Consul 1.11.x introduced a new denomination for the default tokens. The global management token created by the ACL bootstrap process, master token , has been replaced with Bootstrap Token. Additionally, the management token included in the consul configuration files, master token , has been replaced with initial_management token . This tutorial still uses the legacy naming conventions to reflect the naming you will find in the command output.

Consul 1.4.0 introduced a new ACL system with improvements for the security and management of ACL tokens and policies. Since the policy syntax changed to be more precise and flexible to manage, it's necessary to manually translate old tokens (now called "legacy") into new ones to take advantage of the new ACL system features.

This tutorial describes the process for migrating the tokens already present in your pre-1.4.0 Consul deployment after an upgrade to 1.4.0 or later. You will use three different methods for migrating tokens; single token, simple policy mapping, and combine policy updates. Single token updates are a manual process for sensitive tokens. The simple policy mapping and combine policy updates can be used to automate the update of many tokens.

To successfully run the commands in this tutorial, you will need to configure CONSUL_HTTP_TOKEN with a token that has management privileges. You can set the environment variable temporarily with export, so that it will not persist once you've closed the session.

$ export CONSUL_HTTP_TOKEN=<your_token_here>

Post upgrade ACL status

This process assumes that the 1.4.0 datacenter upgrade is complete, including all legacy ACLs having their accessor IDs populated. Accessor IDs are new to the 1.4 ACL system. This process can take up to several minutes after the servers upgrade. Ensure your tokens have an AccessorID before starting the migration. If the command consul acl token list shows any blank AccessorIDs, wait until Consul has finished the upgrade.

$ consul acl token list

...
AccessorID:       00000000-0000-0000-0000-000000000002
Description:      Anonymous Token
Local:            false
Create Time:      0001-01-01 00:00:00 +0000 UTC
Legacy:           false

AccessorID:       dd40ea2f-ff15-e7cb-8249-5d5e95b7ee75
Description:      foo-service-token_1
Local:            false
Create Time:      0001-01-01 00:00:00 +0000 UTC
Legacy:           true

AccessorID:       905755a7-0872-746a-8aee-e9bff2936c11
Description:      Agent Token
Local:            false
Create Time:      0001-01-01 00:00:00 +0000 UTC
Legacy:           true

AccessorID:       5c72a103-d94e-a379-797c-2b418adec88b
Description:      Master Token
Local:            false
Create Time:      0001-01-01 00:00:00 +0000 UTC
Legacy:           false
Policies:
   00000000-0000-0000-0000-000000000001 - global-management
...

From the output, you can notice that two tokens already exist in the new system, the master token and the anonymous token. All the other tokens present in the list are supported as "legacy" as you can verify by the Legacy: true setting.

Consul 1.4.0 retains full support for "legacy" ACL tokens so upgrades from Consul 1.3.0 are safe. Existing tokens will continue to work in the same way for at least two "major" releases (1.5.x, 1.6.x, etc; note HashiCorp does not use SemVer for our products).

Note

Before starting the token migration process, all Consul agents, servers, and clients must be running at least version 1.4.0. Additionally, you must ensure the datacenter is in a healthy state including a functioning leader. Once the leader has determined that all servers in the datacenter are capable of using the new ACL system, the leader will transition itself. Then, the other servers will transition themselves to the new system, followed by the client agents. You can use consul info to investigate the datacenter health.

Single token upgrade

While "legacy" tokens will continue to work for several major releases, it's advisable to migrate existing tokens as soon as possible. Migrating also enables using the new policy management improvements, stricter policy syntax rules, and other features of the new system without re-issuing all the secrets in use.

You can upgrade a legacy token in three steps:

  1. Identify the token that needs to be upgraded
  2. Create a new policy or policies that grant the required access
  3. Update the existing token to use those policies

Identify token

Pick a legacy token from the output of consul acl token list command.

$ consul acl token list

...
AccessorID:       905755a7-0872-746a-8aee-e9bff2936c11
Description:      Agent Token
Local:            false
Create Time:      0001-01-01 00:00:00 +0000 UTC
Legacy:           true
...

You can recognize the legacy tokens by the Legacy: true setting in the output.

For this tutorial,the legacy token will be the "Agent Token".

Create a new policy

For the selected token, you can check what the rules will be once the policy is migrated using the acl translate-rules command.

$ consul acl translate-rules -token-accessor "905755a7-0872-746a-8aee-e9bff2936c11"

node_prefix "" {
  policy = "write"
}

service_prefix "" {
  policy = "read"
}

Notice the new policy has a service_prefix rule and not the original service. Since the old ACL syntax had an implicit prefix match, the _prefix was applied to all rules without needing to be specified. Now that _prefix is no longer implicit, you will need to add it to ensure any clients relying on prefix matching behavior will still work. You can use the Consul CLI to update the policy, without needing to edit the policy file.

It might be beneficial for security to translate prefix matches into exact matches. This however requires the operator knowing that clients using the token really don't rely on the prefix matching semantics of the old ACL system.

Once you have verified the set of rules for the new policy, you will generate the new one.

$ consul acl policy create -name "migrated-Agent-policy" -from-token 905755a7-0872-746a-8aee-e9bff2936c11 -description "Migrated from legacy ACL agent token";

ID:           dcd37998-a564-b26a-dd8a-89414926d015
Name:         migrated-Agent-policy
Description:  Migrated from legacy ACL agent token
Datacenters:
Rules:
node_prefix "" {
  policy = "write"
}
service_prefix "" {
  policy = "read"
}

Note, the use of the -from-token option to specify the rules for the new policy will be based on the rules embedded in the legacy token.

To verify the new policy is in place use the consul acl policy list command again.

$ consul acl policy list

migrated-Agent-policy:
   ID:           dcd37998-a564-b26a-dd8a-89414926d015
   Description:  Migrated from legacy ACL agent token
   Datacenters:
...

Update the token

Finally, you will need to update the token to use the new policy.

$ consul acl token update -id 905755a7-0872-746a-8aee-e9bff2936c11 -policy-name "migrated-Agent-policy" -upgrade-legacy

Token updated successfully.
AccessorID:       905755a7-0872-746a-8aee-e9bff2936c11
SecretID:         dbb88e67-817d-7eb7-f03b-af570462ae6f
Description:      Agent Token
Local:            false
Create Time:      0001-01-01 00:00:00 +0000 UTC
Policies:
   dcd37998-a564-b26a-dd8a-89414926d015 - migrated-Agent-policy

For the upgrade to succeed you need to use the -upgrade-legacy option, which will ensure that legacy rules are removed as well as the new policies added.

If not specified the migration will result in an error:

$ consul acl token update -id 905755a7-0872-746a-8aee-e9bff2936c11 -policy-name "migrated-Agent-policy"

Failed to update token 905755a7-0872-746a-8aee-e9bff2936c11: Unexpected response code: 500 (Rules cannot be specified for this token)

After updating, the token is no longer considered "legacy" and will have all the properties of a new token, however it keeps its SecretID (the secret part of the token used in API calls) so clients already using that token will continue to work. It is assumed that the policies you attach continue to grant the necessary access for existing clients; this is up to the operator to ensure.

You can also perform the upgrade using the PUT /v1/acl/token/:AccessorID endpoint. Specifically, ensure that the Rules field is omitted or empty. Empty Rules indicates that this is now treated as a new token.

Simple policy mapping

If you have multiple tokens migrating them manually might not be a viable option.

One fast alternative to manual migration is to write an automation script to create a different policy with the same rules set, per legacy token and then upgrade them. This approach is easy to automate, but may result in a lot of policies with exactly the same rules and with non-human-readable names which will make managing policies more difficult.

Identify existing legacy tokens

You can get the AccessorID of every legacy token from the API. For example, using curl and jq in bash.

$ LEGACY_IDS=$(curl -sH "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
   'localhost:8500/v1/acl/tokens' | jq -r '.[] | select (.Legacy) | .AccessorID')

$ echo "$LEGACY_IDS"

7dff8907-eace-b403-1966-51ba92189ac7
3d288b1c-ff11-47fa-e010-42d15a117534
dd40ea2f-ff15-e7cb-8249-5d5e95b7ee75

Create policies from legacy tokens

To create a policy for each token you can use a loop to iterate the AccessorID list and migrate the policy.

$ for id in $LEGACY_IDS; do \
  consul acl policy create -name "migrated-$id" -from-token $id \
    -description "Migrated from legacy ACL token"; \
done

ID:           95acbf71-9795-174d-d8fb-fef338c780ca
Name:         migrated-7dff8907-eace-b403-1966-51ba92189ac7
Description:  Migrated from legacy ACL token
Datacenters:
Rules:
service_prefix "bar" {
  policy = "write"
}
ID:           31189810-9503-eae7-9c09-01eb7c1346af
Name:         migrated-3d288b1c-ff11-47fa-e010-42d15a117534
Description:  Migrated from legacy ACL token
Datacenters:
Rules:
service_prefix "bar" {
  policy = "write"
}
ID:           06435a7b-59c2-585f-ae8e-b49a576b9c71
Name:         migrated-dd40ea2f-ff15-e7cb-8249-5d5e95b7ee75
Description:  Migrated from legacy ACL token
Datacenters:
Rules:
service_prefix "foo" {
  policy = "write"
}
## ...

Each policy now has an equivalent set of rules to the original token. This means that the newly created policy grants the same effective permissions as the legacy token rules did.

Update tokens

With the policies created as above, you can automatically upgrade all legacy tokens.

$ for id in $LEGACY_IDS; do \
  consul acl token update -id $id -policy-name "migrated-$id" -upgrade-legacy; \
done

Token updated successfully.
AccessorID:       7dff8907-eace-b403-1966-51ba92189ac7
SecretID:         1532ab31-f7aa-41f7-4989-5da3d537456a
Description:      bar-service-token_1
Local:            false
Create Time:      0001-01-01 00:00:00 +0000 UTC
Policies:
   95acbf71-9795-174d-d8fb-fef338c780ca - migrated-7dff8907-eace-b403-1966-51ba92189ac7
Token updated successfully.
AccessorID:       3d288b1c-ff11-47fa-e010-42d15a117534
SecretID:         87a185c6-f42a-be80-e72e-d5489d68ddc4
Description:      bar-service-token_2
Local:            false
Create Time:      0001-01-01 00:00:00 +0000 UTC
Policies:
   31189810-9503-eae7-9c09-01eb7c1346af - migrated-3d288b1c-ff11-47fa-e010-42d15a117534
Token updated successfully.
AccessorID:       dd40ea2f-ff15-e7cb-8249-5d5e95b7ee75
SecretID:         c6aedb37-9600-a3b4-1ce1-d70aee04abc1
Description:      foo-service-token_1
Local:            false
Create Time:      0001-01-01 00:00:00 +0000 UTC
Policies:
   06435a7b-59c2-585f-ae8e-b49a576b9c71 - migrated-dd40ea2f-ff15-e7cb-8249-5d5e95b7ee75
...

The update is now complete, all legacy tokens are now new tokens with identical secrets and enforcement rules.

Combine policies

An automatic batch upgrade for legacy tokens is the suggested approach when policies are relatively simple and specific for each token. With the combined policies method, you will use the same policy for multiple tokens.

If policies are complex or you have overlapping rules across multiple tokens, it is worth it to leverage the migration process to re-engineer the ACL rules in your Consul deployment. This permits you to end up with a minimal set of policies and to take advantage of the flexibility of the new system.

There are a variety of options for how to do this, however you will need to consider the trade-offs. Mainly, increased operator involvement will result in better clarity and re-usability of the resulting policies.

In the tutorial examples, you have two tokens associated to identical policies.

$ for id in $LEGACY_IDS; do \
   echo "Policy for $id:";  \
   consul acl translate-rules -token-accessor "$id"; \
done

...
Policy for 7dff8907-eace-b403-1966-51ba92189ac7:
service_prefix "bar" {
  policy = "write"
}
Policy for 3d288b1c-ff11-47fa-e010-42d15a117534:
service_prefix "bar" {
  policy = "write"
}
Policy for dd40ea2f-ff15-e7cb-8249-5d5e95b7ee75:
service_prefix "foo" {
  policy = "write"
}

One possibility, in these cases, would be to generate only one policy and then upgrade both the legacy tokens to use the same policy.

Enforce the use of new ACLs in your environment

In the tutorial it is assumed that all clients that need to create ACL tokens (e.g. Vault's Consul secrets engine) have been updated to use the new ACL APIs.

Specifically if you are using Vault's Consul secrets engine you need to be running Vault 1.0.0 or higher and you must update all roles defined in Vault to specify a list of policy names rather than an inline policy (which causes Vault to use the legacy API).

Note

if you have systems still creating "legacy" tokens with the old APIs, the migration steps below will still work, however you'll have to keep re-running them until nothing is creating legacy tokens to ensure all tokens are migrated.

Next steps

In this tutorial, you learned the different strategies and commands available in Consul 1.4+ to migrate legacy ACL tokens.

You followed those strategies to:

  • manually migrate an agent token and get familiar with the CLI commands for ACL;
  • automate the token migration for tokens and policies that don’t require tuning;
  • identify opportunities for improvement in your ACL configuration thanks to the new ACL system improvements.
Collection Overview

Archive

 Next

Admin Partitions with HCP Consul and Amazon Elastic Container Service