GCP dynamic secrets

Plus tier

This feature is available in HCP Vault Secrets Plus tier.

Public beta available!

Dynamic secrets are available for public beta by upgrading to the plus tier.

HCP Vault Secrets can create short-lived GCP credentials on demand.

Prerequisites

Ability to create GCP IAM identity pools, identity providers, and service accounts
- Your GCP principal may use the predefined IAM Workload Identity Pool Admin, and Service Account Admin, and Service Account Key Admin roles for authorization
Ability to create HCP Vault Secrets integrations, apps, and secrets

Set up GCP

HCP Vault Secrets is able to authenticate with your GCP project using two different methods. Each method requires GCP resources to provision dynamic credentials:

Workload Identity Federation (Recommended)
- Workload identity pool
- Workload identity provider
- IAM service account that HCP can impersonate through its web identity
- IAM service account with the permissions to grant to the generated dynamic credentials
Service Account Keys
- IAM service account with a key
- IAM service account with the permissions to grant to the generated dynamic credentials

Configure your GCP project using either the Google Cloud console or Terraform.

Add identity provider

Navigate to the New workload provider and pool section in the Google Cloud IAM & Admin service.
Provide a name and optionally a description then click Continue.
Select the OpenID connect (OIDC) provider type.
Provide a descriptive name such as hcp-dynamic-secrets-<project-name>.
Use https://idp.hashicorp.com/oidc/organization/<org-id> as the issuer URL, replacing the placeholder with your HCP organization ID.
Note
You can navigate to the GCP integration creation page on the HCP portal and select GCP service account credentials from the list to easily find the appropriate Issuer URL.
Note the default audience for the next steps then click Continue.
Enter assertion.sub in the OIDC 1 input field to map the provider attributes then click Save.

Create Service Account for integration

Click + Grant Access by the top navigation bar.
Select Grant access using Service Account Impersonation.
Click the drop-down menu for Select service account, then New Service Account.
Provide a descriptive name such as hvs-dynamic-secrets-integration and optionally a description, then click Create and Continue.
Select the Service Account Token Creator role, then click Continue.
Do not grant users access to this service account, simply click Done.
Navigate back to the Grant Access panel and select the service account we just created.
Use project:<project-id>:geo:us:service:vault-secrets:type:integration:name:<integration-name> as the attribute value, replacing the placeholders with your HCP project ID and HCP integration name.
Note
You can navigate to the GCP integration creation page on the HCP portal and select GCP service account credentials from the list to easily find the appropriate Service Account subject.
Dismiss the Configure your application modal.
Click to Service Accounts in the left navigation bar, and note the email for the service account you just created for the next steps.

Create Service Account for dynamic secret

Navigate to the Create Service Account section in the Google Cloud IAM & Admin service.
Give the service account a name and optionally a description, then click Create and Continue.
Attach or create a new role with the permissions to grant to the dynamic credentials HCP Vault Secrets will provision, then click Done.
Click on Service Accounts in the left navigation bar, and note the service account email you just created for the next steps.

Create Service Account for integration

Navigate to the Create Service Account section in the Google Cloud IAM & Admin service.
Give the service account a name and optionally a description, click Create and Continue
Select the Service Account Token Creator role, click Continue
Do not grant users access to this service account, click Done
Select the service account you just created from the Service Accounts table
From the Keys tab, click Add Key, then Create new key
Select the JSON key type, click Create
A JSON file will be downloaded by your browser. You'll need this later when configuring the integration.

Warning

Be cautious with the service account key JSON file while it is in transit to HCP Vault Secrets. These credentials provide access to your GCP project and do not automatically expire.

Create Service Account for dynamic secret

Navigate to the Create Service Account section in the Google Cloud IAM & Admin service.
Give the service account a name and optionally a description, then click Create and Continue.
Attach or create a new role with the permissions to grant to the dynamic credentials HCP Vault Secrets will provision, then click Done.
Click on Service Accounts in the left navigation bar, and note the service account email you just created for the next steps.

The Terraform Google provider can be used to provision the necessary resources.

Follow the Terraform documentation to authenticate the Google provider.

Use your preferred text editor and create a Terraform configuration file with the following snippet:

provider "google" {
  project = var.gcp_project_id
}

variable "gcp_project_id" {
  type        = string
  description = "Your GCP project ID"
}
variable "organization_id" {
  type        = string
  description = "Your HCP organization ID"
}
variable "project_id" {
  type        = string
  description = "Your HCP project ID"
}
variable "integration_name" {
  type        = string
  description = "Your GCP integration name. Must match the name of the GCP integration you'll eventually create in HCP Vault Secrets."
}

data "google_project" "gcp_project" {}

locals {
  pool_id            = "hcp-vault-secrets"
  pool_provider_id   = "hcp-vault-secrets"
  gcp_project_number = data.google_project.gcp_project.number
  audience           = "https://iam.googleapis.com/projects/${local.gcp_project_number}/locations/global/workloadIdentityPools/${local.pool_id}/providers/${local.pool_provider_id}"
  subject            = "project:${var.project_id}:geo:us:service:vault-secrets:type:integration:name:${var.integration_name}"
}

resource "google_iam_workload_identity_pool" "hcp_vault_secrets" {
  workload_identity_pool_id = local.pool_id
}
resource "google_iam_workload_identity_pool_provider" "hcp_vault_secrets" {
  workload_identity_pool_id          = google_iam_workload_identity_pool.hcp_vault_secrets.workload_identity_pool_id
  workload_identity_pool_provider_id = local.pool_provider_id
  display_name                       = "HCP Vault Secrets"
  description                        = "OIDC identity pool provider for HCP Vault Secrets"
  attribute_mapping = {
    "google.subject" = "assertion.sub"
  }
  oidc {
    allowed_audiences = [local.audience]
    issuer_uri        = "https://idp.hashicorp.com/oidc/organization/${var.organization_id}"
  }
}

resource "google_service_account" "integration_service_account" {
  account_id   = "integration-service-account"
  display_name = "Integration service account"
}
resource "google_service_account_iam_member" "integration_binding" {
  service_account_id = google_service_account.integration_service_account.id
  role               = "roles/iam.workloadIdentityUser"
  member             = "principal://iam.googleapis.com/projects/${local.gcp_project_number}/locations/global/workloadIdentityPools/${google_iam_workload_identity_pool.hcp_vault_secrets.workload_identity_pool_id}/subject/${local.subject}"
}
resource "google_project_iam_member" "integration_binding" {
  project = var.gcp_project_id
  role    = "roles/iam.serviceAccountTokenCreator"
  member  = "serviceAccount:${google_service_account.integration_service_account.email}"
}

resource "google_service_account" "dynamic_secret_service_account" {
  account_id   = "secret-service-account"
  display_name = "Dynamic secret service account"
}
resource "google_project_iam_member" "secret_binding" {
  project = var.gcp_project_id
  member  = "serviceAccount:${google_service_account.dynamic_secret_service_account.email}"
  
  # Replace with the role/permissions to grant to your dynamic secret
  role    = "roles/iam.serviceAccountViewer"
}

output "integration_name" {
  value = var.integration_name
}

output "audience" {
  value = local.audience
}

output "integration_service_account_email" {
  value = google_service_account.integration_service_account.email
}

output "dynamic_secret_service_account_email" {
  value = google_service_account.dynamic_secret_service_account.email
}

Initialize Terraform.

$ terraform init

This will prepare a working directory:

...snip...

Terraform has been successfully initialized!

Generate your Google Cloud configuration with the apply command:

$ terraform apply

Review the actions Terraform is about to perform and say yes to accept the plan:

...snip...

Plan: 7 to add, 0 to change, 0 to destroy.

Changes to Outputs:
+ audience                             = "https://iam.googleapis.com/projects/<proj-id>/locations/global/workloadIdentityPools/hcp-vault-secrets/providers/hcp-vault-secrets"
+ dynamic_secret_service_account_email = (known after apply)
+ integration_name                     = "<integration-name>"
+ integration_service_account_email    = (known after apply)

Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.

Enter a value: yes

Keep the Terraform outputs on hand for the next steps:

Apply complete! Resources: 7 added, 0 changed, 0 destroyed.

Outputs:

audience = "https://iam.googleapis.com/projects/<proj-id>/locations/global/workloadIdentityPools/hcp-vault-secrets/providers/hcp-vault-secrets"
dynamic_secret_service_account_email = "secret-service-account@hvs-test-<id>.iam.gserviceaccount.com"
integration_name = "<integration-name>"
integration_service_account_email = "integration-service-account@hvs-test-<id>.iam.gserviceaccount.com"

The Terraform Google provider can be used to provision the necessary resources.

Follow the Terraform documentation to authenticate the Google provider.

Use your preferred text editor and create a Terraform configuration file with the following snippet:

provider "google" {
  project = var.gcp_project_id
}

variable "gcp_project_id" {
  type        = string
  description = "Your GCP project ID"
}

resource "google_service_account" "integration_service_account" {
  account_id   = "integration-service-account"
  display_name = "Integration service account"
}
resource "google_project_iam_member" "integration_binding" {
  project = var.gcp_project_id
  role    = "roles/iam.serviceAccountTokenCreator"
  member  = "serviceAccount:${google_service_account.integration_service_account.email}"
}
resource "google_service_account_key" "integration_service_account_key" {
  service_account_id = google_service_account.integration_service_account.name
  public_key_type    = "TYPE_X509_PEM_FILE"
}

resource "google_service_account" "dynamic_secret_service_account" {
  account_id   = "secret-service-account"
  display_name = "Dynamic secret service account"
}
resource "google_project_iam_member" "secret_binding" {
  project = var.gcp_project_id
  member  = "serviceAccount:${google_service_account.dynamic_secret_service_account.email}"

  # Replace with the role/permissions to grant to your dynamic secret
  role    = "roles/iam.serviceAccountViewer"
}

// Note that this will be written to the state file. 
// If you use this, please protect your backend state file.
// https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_service_account_key#private_key
output "integration_service_account_key" {
  value     = base64decode(google_service_account_key.integration_service_account_key.private_key)
  sensitive = true
}

output "dynamic_secret_service_account_email" {
  value = google_service_account.dynamic_secret_service_account.email
}

Initialize Terraform.

$ terraform init

This will prepare a working directory:

...snip...

Terraform has been successfully initialized!

Generate your Google Cloud configuration with the apply command:

$ terraform apply

Review the actions Terraform is about to perform and say yes to accept the plan:

...snip...

Plan: 5 to add, 0 to change, 0 to destroy.

Changes to Outputs:
+ dynamic_secret_service_account_email = (known after apply)
+ integration_service_account_key      = (sensitive value)

Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.

Enter a value: yes

Keep the Terraform outputs on hand for the next steps:

Apply complete! Resources: 5 added, 0 changed, 0 destroyed.

Outputs:

dynamic_secret_service_account_email = "secret-service-account@<project-id>.iam.gserviceaccount.com"
integration_service_account_key = <sensitive>

Configure dynamic secrets

Navigate to the Vault Secrets app panel and select an app where you want to create a dynamic secret.
Click Create new secret and select Dynamic secret.
Select the GCP option from the pull down menu.
Select an existing integration or select Add new integration.
Select an Authentication method and follow the appropriate steps below.

Provide a unique Integration Name for this integration.
Use the integration service account email configured during the previous steps.
Use the GCP workload identity provider audience configured during the previous steps. The format is https://iam.googleapis.com/projects/<project>/locations/global/workloadIdentityPools/<pool-name>/providers/<provider-name>.
Click Add new integration to return to the new secret form.
Note
If you encounter an error, make sure the service account subject and audience matches between HCP and GCP.

Add new dynamic secret

Provide a unique Secret Name for this secret.
Use the dynamic secret service account email configured during the previous steps.
Select a Time to live (TTL) for the generated dynamic credentials between 1 minute and 12 hours. The upper bound varies depending on your organization's policy constraints, for most use cases the TTL should not be above 1 hour.

Accessing dynamic credentials

A dynamic secret is a template to generate dynamic credentials on demand. Each time a dynamic secret is accessed, a new credentials set is generated and shared exclusively with the requesting client. Dynamic credentials can be generated using:

HCP portal

HCP API

curl \
--location "https://api.cloud.hashicorp.com/secrets/2023-11-28/organizations/${HCP_ORG_ID}/projects/${HCP_PROJ_ID}/apps/${APP_NAME}/secrets/${SECRET_NAME}:open" \
--request GET \
--header "Authorization: Bearer ${HCP_API_TOKEN}" | jq

HCP CLI

hcp vault-secrets secrets open ${SECRET_NAME}

HCP Terraform provider

data "hcp_vault_secrets_dynamic_secret" "my_dynamic_secret" {
  app_name    = "my-app"
  secret_name = "my_dynamic_secret"
}