AWS dynamic secrets

Plus tier

This feature is available in HCP Vault Secrets Plus tier.

Public beta available!

Dynamic secrets are available for public beta by upgrading to the plus tier.

HCP Vault Secrets can create short-lived AWS credentials on demand.

Prerequisites

Ability to create AWS IAM identity providers, users, and roles
- Your AWS principal may use the managed IAMFullAccess policy or a custom policy for authorization
Ability to create HCP Vault Secrets integrations, apps, and secrets

Set up AWS

HCP Vault Secrets is able to authenticate with your AWS account using two different methods. Each method requires AWS resources to provision dynamic credentials:

OpenID Connect (OIDC) Federation (Recommended)
- IAM OIDC identity provider
- IAM role that HCP can assume through its web identity
- IAM role with the permissions to grant to the generated dynamic credentials
Access Keys
- IAM user with an access key pair
- IAM role with the permissions to grant to the generated dynamic credentials

Configure your AWS account using either the AWS console or Terraform.

Add identity provider

Navigate to the Add an Identity provider section in the AWS IAM service.
Select the OpenID connect provider type.
Use https://idp.hashicorp.com/oidc/organization/<org-id> as the provider URL, replacing the placeholder with your HCP organization ID.
Note
You can navigate to the AWS integration creation page on the HCP portal and select AWS STS credentials from the list to easily find the appropriate Provider URL and org ID.
Use arn:aws:iam::<account-id>:oidc-provider/idp.hashicorp.com/oidc/organization/<org-id> as the audience, replacing the placeholders with your AWS account ID and HCP organization ID.
Click Add provider.
Select the identity provider you just created and note its ARN and Audience for the next steps.

Create IAM role for integration

Navigate to the Create Role section in the AWS IAM service.
Select the Web Identity trust entity type.
Select the Identity provider and Audience created in the previous step from the dropdown and click Next.
This role does not need permissions, click Next.
Give the role a name and optionally a description and tags, then click Create role.
Select the role you just created and note its ARN for the next steps.

Create IAM role for dynamic secret

Navigate again to the Create Role section in the AWS IAM service.
Select the Custom trust policy trust entity type.

Paste the following into the trust policy editor, replacing the highlighted placeholder text with the AWS IAM role ARN created earlier, and click Next.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
      "AWS": "<iam-role-arn-from-step-2.6>"
    },
    "Action": "sts:AssumeRole",
    "Condition": {}
    }
  ]
}

Attach or create a new policy with the permissions to grant to the dynamic credentials HCP Vault Secrets will provision.
Give the role a name and optionally a description and tags, then click Create role.
Select the role you just created and note its ARN for the next steps.

Create IAM user for integration

Navigate to the Create User section in the AWS IAM service.
Give the user a name, click Next.
This role does not need permissions, click Next.
Add optional tags and click Create user.
Select the user you just created and note its ARN for the next steps.
Click the Security credentials for your new user
Click Create access key
Select the Third-party service option, check the confirmation box, and click Next
Add optional tags and click Create access key.
Take note of the Access key and Secret access key for the next steps.

Warning

Be cautious with the AWS access key and secret access key while they are in transit to HCP Vault Secrets. These credentials provide access to your AWS account and and do not automatically expire.

Create IAM role for dynamic secret

Navigate again to the Create Role section in the AWS IAM service.
Select the Custom trust policy trust entity type.

Paste the following into the trust policy editor, replacing the highlighted placeholder text with the AWS IAM user ARN created earlier, and click Next.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
      "AWS": "<iam-user-arn-from-step-1.6>"
    },
    "Action": "sts:AssumeRole",
    "Condition": {}
    }
  ]
}

Attach or create a new policy with the permissions to grant to the dynamic credentials HCP Vault Secrets will provision.
Give the role a name and optionally a description and tags, then click Create role.
Select the role you just created and note its ARN for the next steps.

The Terraform AWS provider can be used to provision the necessary resources.

Follow the Terraform documentation to authenticate the AWS provider.

Use your preferred text editor and create a Terraform configuration file with the following snippet:

provider "aws" {}

variable "organization_id" {
  type        = string
  description = "Your HCP organization ID."
}
variable "project_id" {
  type        = string
  description = "Your HCP project ID."
}
variable "integration_name" {
  type        = string
  description = "Your AWS integration name. Must match the name of the AWS integration you'll eventually create in HCP Vault Secrets."
}

data "aws_caller_identity" "current" {}

locals {
  account_id = data.aws_caller_identity.current.account_id
  issuer     = "idp.hashicorp.com/oidc/organization/${var.organization_id}"
  audience   = "arn:aws:iam::${local.account_id}:oidc-provider/${local.issuer}"
  subject    = "project:${var.project_id}:geo:us:service:vault-secrets:type:integration:name:${var.integration_name}"
}

resource "aws_iam_openid_connect_provider" "hcp_vault_secrets" {
  url             = "https://${local.issuer}"
  thumbprint_list = ["9e99a48a9960b14926bb7f3b02e22da2b0ab7280"]
  client_id_list  = [local.audience]
}

resource "aws_iam_role" "integration_role" {
  name = "hcp-vault-secrets-integration"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRoleWithWebIdentity"
        Effect = "Allow"
        Principal = {
          Federated = aws_iam_openid_connect_provider.hcp_vault_secrets.arn
        }
        Condition = {
          StringEquals = {
            "${local.issuer}:sub" = local.subject
          }
        }
      }
    ]
  })
}

resource "aws_iam_role" "dynamic_secret_role" {
  name = "hcp-vault-secrets-dynamic-secret"
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Effect" : "Allow",
        "Action" : "sts:AssumeRole",
        "Principal" : {
          "AWS" : aws_iam_role.integration_role.arn
        },
        "Condition" : {}
      }
    ]
  })

  inline_policy {
    name   = "hcp-vault-secrets-s3-lister"
    policy = data.aws_iam_policy_document.dynamic_secret_policy.json
  }
}

data "aws_iam_policy_document" "dynamic_secret_policy" {
  # Replace with the permissions to grant to your dynamic secret
  statement {
    effect = "Allow"
    actions = [
      "s3:List*"
    ]
    resources = ["*"]
  }
}

output "integration_name" {
  value = var.integration_name
}

output "audience" {
  value = local.audience
}

output "integration_role_arn" {
  value = aws_iam_role.integration_role.arn
}

output "dynamic_secret_role_arn" {
  value = aws_iam_role.dynamic_secret_role.arn
}

Initialize Terraform.

$ terraform init

This will prepare a working directory:

...snip...

Terraform has been successfully initialized!

Generate your AWS IAM configuration with the apply command:

$ terraform apply

Review the actions Terraform is about to perform and say yes to accept the plan:

...snip...

Plan: 3 to add, 0 to change, 0 to destroy.

Changes to Outputs:
+ dynamic_secret_role_arn = (known after apply)
+ integration_role_arn    = (known after apply)

Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.

Enter a value: yes

Keep the Terraform outputs on hand for the next steps:

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Outputs:

audience = "arn:aws:iam::<account-id>:oidc-provider/idp.hashicorp.com/oidc/organization/<org-id>"
dynamic_secret_role_arn = "arn:aws:iam::<account-id>:role/hcp-vault-secrets-dynamic-secret"
integration_name = "<integration-name>"
integration_role_arn = "arn:aws:iam::<account-id>:role/hcp-vault-secrets-integration"

The Terraform AWS provider can be used to provision the necessary resources.

Follow the Terraform documentation to authenticate the AWS provider.

Use your preferred text editor and create a Terraform configuration file with the following snippet:

provider "aws" {}

resource "aws_iam_user" "integration_user" {
  name = "hcp-vault-secrets-integration"
}

resource "aws_iam_access_key" "integration_user_access_key" {
  user = aws_iam_user.integration_user.name
}

resource "aws_iam_role" "dynamic_secret_role" {
  name = "hcp-vault-secrets-dynamic-secret"
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Effect" : "Allow",
        "Action" : "sts:AssumeRole",
        "Principal" : {
          "AWS" : aws_iam_user.integration_user.arn
        },
        "Condition" : {}
      }
    ]
  })

  inline_policy {
    name   = "hcp-vault-secrets-s3-lister"
    policy = data.aws_iam_policy_document.dynamic_secret_policy.json
  }
}

data "aws_iam_policy_document" "dynamic_secret_policy" {
  # Replace with the permissions to grant to your dynamic secret
  statement {
    effect = "Allow"
    actions = [
      "s3:List*"
    ]
    resources = ["*"]
  }
}

output "integration_access_key_id" {
  value = aws_iam_access_key.integration_user_access_key.id
}

// Note that this will be written to the state file. 
// If you use this, please protect your backend state file.
// https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key#secret
output "integration_secret_access_key" {
  value = aws_iam_access_key.integration_user_access_key.secret
  sensitive = true
}

output "dynamic_secret_role_arn" {
  value = aws_iam_role.dynamic_secret_role.arn
}

Initialize Terraform.

$ terraform init

This will prepare a working directory:

...snip...

Terraform has been successfully initialized!

Generate your AWS IAM configuration with the apply command:

$ terraform apply

Review the actions Terraform is about to perform and say yes to accept the plan:

...snip...

  Plan: 3 to add, 0 to change, 0 to destroy.
  
  Changes to Outputs:
  + dynamic_secret_role_arn       = (known after apply)
  + integration_access_key_id     = (known after apply)
  + integration_secret_access_key = (sensitive value)

Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.

Enter a value: yes

Keep the Terraform outputs on hand for the next steps:

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Outputs:

dynamic_secret_role_arn = "arn:aws:iam::<account-id>:role/hcp-vault-secrets-dynamic-secret"
integration_access_key_id = "AKIA..."
integration_secret_access_key = <sensitive>

Configure dynamic secrets

Navigate to the Vault Secrets app panel and select an app where you want to create a dynamic secret.
Click Create new secret and select Dynamic secret.
Select the AWS option from the pull down menu.
Select an existing integration or select Add new integration.
Select an Authentication method and follow the appropriate steps below.

Provide a unique Integration Name for this integration.
Use the AWS IAM identity provider audience configured during the previous steps. The format is arn:aws:iam::<account-id>:oidc-provider/idp.hashicorp.com/oidc/organization/<org-id>.
Use the Integration AWS IAM role ARN configured during the previous steps. This role is used by HCP to manage credentials.
Click Add new integration to return to the new secret form.
Note
If you encounter an error, make sure the audience matches between HCP and AWS. Also verify the condition on the trust relationship for the AWS IAM role corresponds to your HCP organization ID, project ID and integration name.

Add new dynamic secret

Provide a unique Secret Name for this secret.
Use the dynamic secret AWS IAM role ARN configured during the previous steps.
Select a Time to Live (TTL) for the generated dynamic credentials between 15 minutes and 12 hours. Note that the upper limit may vary based on your AWS IAM’s role’s maximum session duration, which is typically 1 hour.

Accessing dynamic credentials

A dynamic secret is a template to generate dynamic credentials on demand. Each time a dynamic secret is accessed, a new credentials set is generated and shared exclusively with the requesting client. Dynamic credentials can be generated using:

HCP portal

HCP API

curl \
--location "https://api.cloud.hashicorp.com/secrets/2023-11-28/organizations/${HCP_ORG_ID}/projects/${HCP_PROJ_ID}/apps/${APP_NAME}/secrets/${SECRET_NAME}:open" \
--request GET \
--header "Authorization: Bearer ${HCP_API_TOKEN}" | jq

HCP CLI

hcp vault-secrets secrets open ${SECRET_NAME}

HCP Terraform provider

data "hcp_vault_secrets_dynamic_secret" "my_dynamic_secret" {
  app_name    = "my-app"
  secret_name = "my_dynamic_secret"
}