Google Cloud auth method

The gcp auth method allows Google Cloud Platform entities to authenticate to Vault. Vault treats Google Cloud as a trusted third party and verifies authenticating entities against the Google Cloud APIs. This backend allows for authentication of:

Google Cloud IAM service accounts
Google Compute Engine (GCE) instances

This backend focuses on identities specific to Google Cloud and does not support authenticating arbitrary Google or Google Workspace users or generic OAuth against Google.

This plugin is developed in a separate GitHub repository at hashicorp/vault-plugin-auth-gcp, but is automatically bundled in Vault releases. Please file all feature requests, bugs, and pull requests specific to the GCP plugin under that repository.

Authentication

Via the CLI helper

Vault includes a CLI helper that obtains a signed JWT locally and sends the request to Vault.

# Authentication to vault outside of Google Cloud
$ vault login -method=gcp \
    role="my-role" \
    service_account="authenticating-account@my-project.iam.gserviceaccount.com" \
    jwt_exp="15m" \
    credentials=@path/to/signer/credentials.json

# Authentication to vault inside of Google Cloud
$ vault login -method=gcp role="my-role"

For more usage information, run vault auth help gcp.

Note: The project parameter has been removed in Vault 1.5.9+, 1.6.5+, and 1.7.2+. It is no longer needed for configuration and will be ignored if provided.

Via the CLI

$ vault write -field=token auth/gcp/login \
    role="my-role" \
    jwt="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."

See Generating JWTs for ways to obtain the JWT token.

Via the API

$ curl \
    --request POST \
    --data '{"role":"my-role", "jwt":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."}' \
    http://127.0.0.1:8200/v1/auth/gcp/login

See API docs for expected response.

Configuration

Auth methods must be configured in advance before users or machines can authenticate. These steps are usually completed by an operator or configuration management tool.

Enable the Google Cloud auth method:
```
$ vault auth enable gcp
```
Configure the auth method credentials if Vault is not running on Google Cloud:
```
$ vault write auth/gcp/config \
    credentials=@/path/to/credentials.json
```
If you are using instance credentials or want to specify credentials via an environment variable, you can skip this step. To learn more, see the Google Cloud Credentials section below.
Note: If you're using a Private Google Access environment, you will additionally need to configure your environment’s custom endpoints via the custom_endpoint configuration parameter.
In some cases, you cannot set sensitive IAM security credentials in your Vault configuration. For example, your organization may require that all security credentials are short-lived or explicitly tied to a machine identity.
To provide IAM security credentials to Vault, we recommend using Vault plugin workload identity federation (WIF) as shown below.

Alternatively, configure the audience claim value and the service account email to assume for plugin workload identity federation:

 $ vault write auth/gcp/config \
     identity_token_audience="<TOKEN AUDIENCE>" \
     service_account_email="<SERVICE ACCOUNT EMAIL>"

Vault's identity token provider signs the plugin identity token JWT internally. If a trust relationship exists between Vault and GCP through WIF, the auth method can exchange the Vault identity token for a federated access token.

To configure a trusted relationship between Vault and GCP:

- You must configure the [identity token issuer backend](/vault/api-docs/secret/identity/tokens#configure-the-identity-tokens-backend)
  for Vault.
- GCP must have a
  [workload identity pool and provider](https://cloud.google.com/iam/docs/manage-workload-identity-pools-providers)
  configured with information about the fully qualified and network-reachable
  issuer URL for the Vault plugin's
  [identity token provider](/vault/api-docs/secret/identity/tokens#read-plugin-identity-well-known-configurations).

Establishing a trusted relationship between Vault and GCP ensures that GCP can fetch JWKS public keys and verify the plugin identity token signature.

Create a named role:

For an iam-type role:

$ vault write auth/gcp/role/my-iam-role \
    type="iam" \
    policies="dev,prod" \
    bound_service_accounts="my-service@my-project.iam.gserviceaccount.com"

For a gce-type role:

$ vault write auth/gcp/role/my-gce-role \
    type="gce" \
    policies="dev,prod" \
    bound_projects="my-project1,my-project2" \
    bound_zones="us-east1-b" \
    bound_labels="foo:bar,zip:zap" \
    bound_service_accounts="my-service@my-project.iam.gserviceaccount.com"

Note that bound_service_accounts is only required for iam-type roles.

For the complete list of configuration options for each type, please see the API documentation.

GCP credentials

The Google Cloud Vault auth method uses the official Google Cloud Golang SDK. This means it supports the common ways of providing credentials to Google Cloud.

The environment variable GOOGLE_APPLICATION_CREDENTIALS. This is specified as the path to a Google Cloud credentials file, typically for a service account. If this environment variable is present, the resulting credentials are used. If the credentials are invalid, an error is returned.
Default instance credentials. When no environment variable is present, the default service account credentials are used.

For more information on service accounts, please see the Google Cloud Service Accounts documentation.

To use this auth method, the service account must have the following minimum scope:

https://www.googleapis.com/auth/cloud-platform

Required GCP permissions

Enabled GCP APIs

The GCP project must have the following APIs enabled:

iam.googleapis.com for iam and gce type roles.
compute.googleapis.com for gce type roles.
cloudresourcemanager.googleapis.com for iam and gce type roles that set add_group_aliases to true.

Vault server permissions

For iam-type Vault roles, the service account credentials given to Vault can have the following role:

roles/iam.serviceAccountKeyAdmin

For gce-type Vault roles, the service account credentials given to Vault can have the following role:

roles/compute.viewer

If you instead wish to create a custom role with only the exact GCP permissions required, use the following list of permissions:

iam.serviceAccounts.get
iam.serviceAccountKeys.get
compute.instances.get
compute.instanceGroups.list

These allow Vault to:

verify the service account, either directly authenticating or associated with authenticating GCE instance, exists
get the corresponding public keys for verifying JWTs signed by service account private keys.
verify authenticating GCE instances exist
compare bound fields for GCE roles (zone/region, labels, or membership in given instance groups)

If you are using Group Aliases as described below, you will also need to add the resourcemanager.projects.get permission.

Permissions for authenticating against Vault

If you are authenticating to Vault from Google Cloud, you can skip the following step as Vault will generate and present the identity token of the service account configured on the instance or the pod.

Note that the previously mentioned permissions are given to the Vault servers. The IAM service account or GCE instance that is authenticating against Vault must have the following role:

roles/iam.serviceAccountTokenCreator

WARNING: Make sure this role is only applied so your service account can impersonate itself. If this role is applied GCP project-wide, this will allow the service account to impersonate any service account in the GCP project where it resides. See Managing service account impersonation for more information.

Plugin Workload Identity Federation (WIF)

This feature requires Vault Enterprise(opens in new tab).

The GCP auth method supports the plugin WIF workflow and has a source of identity called a plugin identity token. A plugin identity token is a JWT that is signed internally by the Vault plugin identity token issuer.

If there is a trust relationship configured between Vault and GCP through workload identity federation, the auth method can exchange its identity token for short-lived access tokens needed to perform its actions.

Exchanging identity tokens for access tokens lets the GCP auth method operate without configuring explicit access to sensitive IAM security credentials.

To configure the auth method to use plugin WIF:

Ensure that Vault openid-configuration and public JWKS APIs are network-reachable by GCP. We recommend using an API proxy or gateway if you need to limit Vault API exposure.
Create a workload identity pool and provider in GCP.
1. The provider URL must point at your Vault plugin identity token issuer with the /.well-known/openid-configuration suffix removed. For example: https://host:port/v1/identity/oidc/plugins.
2. Uniquely identify the recipient of the plugin identity token as the audience. You can use the default audience for the identity pool or a custom value less than 256 characters.
Authenticate a workload in GCP by granting the identity pool access to a dedicated service account using service account impersonation. Filter requests using the unique sub claim issued by plugin identity tokens so the GCP Auth method can impersonate the service account. sub claims have the form: plugin-identity:<NAMESPACE>:auth:<GCP_AUTH_MOUNT_ACCESSOR>.

Configure the GCP auth method with the OIDC audience value and service account email.

$ vault write auth/gcp/config \
  identity_token_audience="//iam.googleapis.com/projects/410449834127/locations/global/workloadIdentityPools/vault-gcp-auth-43777a63/providers/vault-gcp-auth-wif-provider" \
  service_account_email="vault-plugin-wif-auth@hc-b712f250b4e04cacbadd258a90b.iam.gserviceaccount.com"

Your auth method can now use plugin WIF for its configuration credentials. By default, WIF credentials have a time-to-live of 1 hour and automatically refresh when they expire.

Please see the API documentation for more details on the fields associated with plugin WIF.

Group aliases

As of Vault 1.0, roles can specify an add_group_aliases boolean parameter that adds group aliases to the auth response. These aliases can aid in building reusable policies since they are available as interpolated values in Vault's policy engine. Once enabled, the auth response will include the following aliases:

[
  "project-$PROJECT_ID",
  "folder-$SUBFOLDER_ID",
  "folder-$FOLDER_ID",
  "organization-$ORG_ID"
]

If you are using a custom role for Vault server, you will need to add the resourcemanager.projects.get permission to your custom role.

Implementation details

This section describes the implementation details for how Vault communicates with Google Cloud to authenticate and authorize JWT tokens. This information is provided for those who are curious, but these details are not required knowledge for using the auth method.

IAM login applies only to roles of type iam. The Vault authentication workflow for IAM service accounts looks like this:

The client generates a signed JWT using the Service Account Credentials projects.serviceAccounts.signJwt API method. For examples of how to do this, see the Generating JWTs section.
The client sends this signed JWT to Vault along with a role name.
Vault extracts the kid header value, which contains the ID of the key-pair used to generate the JWT, and the sub ID/email to find the service account key. If the service account does not exist or the key is not linked to the service account, Vault denies authentication.
Vault authorizes the confirmed service account against the given role. If that is successful, a Vault token with the proper policies is returned.

GCE login only applies to roles of type gce and must be completed on an infrastructure running on Google Cloud. These steps will not work from your local laptop or another cloud provider.

The client obtains an instance identity metadata token on a GCE instance.
The client sends this JWT to Vault along with a role name.
Vault extracts the kid header value, which contains the ID of the key-pair used to generate the JWT, to find the OAuth2 public cert to verify this JWT.
Vault authorizes the confirmed instance against the given role, ensuring the instance matches the bound zones, regions, or instance groups. If that is successful, a Vault token with the proper policies is returned.

Generating JWTs

This section details the various methods and examples for obtaining JWT tokens.

Service account credentials API

This describes how to use the GCP Service Account Credentials API method directly to generate the signed JWT with the claims that Vault expects. Note the CLI does this process for you and is much easier, and that there is very little reason to do this yourself.

curl example

Vault requires the following minimum claim set:

{
  "sub": "$SERVICE_ACCOUNT_EMAIL_OR_ID",
  "aud": "vault/$ROLE",
  "exp": "$EXPIRATION"
}

For the API method, providing the expiration claim exp is required. If it is omitted, it will not be added automatically and Vault will deny authentication. Expiration must be specified as a NumericDate value (seconds from Epoch). This value must be before the max JWT expiration allowed for a role. This defaults to 15 minutes and cannot be more than 1 hour.

If a user generates a token that expires after 15 minutes, and the gcp role has max_jwt_exp set to the default, Vault will return the following error: Expiration date must be set to no more that 15 mins in JWT_CLAIM, otherwise the login request returns error "role requires that service account JWTs expire within 900 seconds. In this case, the user must create a new signed JWT with a shorter expiration, or set max_jwt_exp to a higher value in the gcp role.

One you have all this information, the JWT token can be signed using curl and oauth2l:

ROLE="my-role"
SERVICE_ACCOUNT="service-account@my-project.iam.gserviceaccount.com"
OAUTH_TOKEN="$(oauth2l header cloud-platform)"
EXPIRATION="<your_token_expiration>"
JWT_CLAIM="{\\\"aud\\\":\\\"vault/${ROLE}\\\", \\\"sub\\\": \\\"${SERVICE_ACCOUNT}\\\", \\\"exp\\\": ${EXPIRATION}}"

$ curl \
  --header "${OAUTH_TOKEN}" \
  --header "Content-Type: application/json" \
  --request POST \
  --data "{\"payload\": \"${JWT_CLAIM}\"}" \
  "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/${SERVICE_ACCOUNT}:signJwt"

gcloud example

You can also do this through the (currently beta) gcloud command. Note that you will be required to provide the expiration claim exp as a part of the JWT input to the command.

$ gcloud beta iam service-accounts sign-jwt $INPUT_JWT_CLAIMS $OUTPUT_JWT_FILE \
    --iam-account=service-account@my-project.iam.gserviceaccount.com \
    --project=my-project

Golang example

Read more on the Google Open Source blog.

GCE

You can autogenerate this token in Vault versions 1.8.2 or higher.

GCE tokens can only be generated from a GCE instance.

Vault can automatically discover the identity token on a GCE/GKE instance. This simplifies authenticating to Vault like so:
```
$ vault login \
  -method=gcp \
  role="my-gce-role"
```

The JWT token can also be obtained from the "service-accounts/default/identity" endpoint for a instance's metadata server.

Curl example

ROLE="my-gce-role"

$ curl \
  --header "Metadata-Flavor: Google" \
  --get \
  --data-urlencode "audience=http://vault/${ROLE}" \
  --data-urlencode "format=full" \
  "http://metadata/computeMetadata/v1/instance/service-accounts/default/identity"

API

The GCP Auth Plugin has a full HTTP API. Please see the API docs for more details.

Code example

The following example demonstrates the Google Cloud auth method to authenticate with Vault.

package main

import (
    "context"
    "fmt"
    "os"

    vault "github.com/hashicorp/vault/api"
    auth "github.com/hashicorp/vault/api/auth/gcp"
)

// Fetches a key-value secret (kv-v2) after authenticating to Vault
// via GCP IAM, one of two auth methods used to authenticate with
// GCP (the other is GCE auth).
func getSecretWithGCPAuthIAM() (string, error) {
    config := vault.DefaultConfig() // modify for more granular configuration

    client, err := vault.NewClient(config)
    if err != nil {
        return "", fmt.Errorf("unable to initialize Vault client: %w", err)
    }

    // For IAM-style auth, the environment variable GOOGLE_APPLICATION_CREDENTIALS
    // must be set with the path to a valid credentials JSON file, otherwise
    // Vault will fall back to Google's default instance credentials.
    // Learn about authenticating to GCS with service account credentials at https://cloud.google.com/docs/authentication/production
    if pathToCreds := os.Getenv("GOOGLE_APPLICATION_CREDENTIALS"); pathToCreds == "" {
        fmt.Printf("WARNING: Environment variable GOOGLE_APPLICATION_CREDENTIALS was not set. IAM client for JWT signing and Vault server IAM client will both fall back to default instance credentials.\n")
    }

    svcAccountEmail := fmt.Sprintf("%s@%s.iam.gserviceaccount.com", os.Getenv("GCP_SERVICE_ACCOUNT_NAME"), os.Getenv("GOOGLE_CLOUD_PROJECT"))

    // We pass the auth.WithIAMAuth option to use the IAM-style authentication
    // of the GCP auth backend. Otherwise, we default to using GCE-style
    // authentication, which gets its credentials from the metadata server.
    gcpAuth, err := auth.NewGCPAuth(
        "dev-role-iam",
        auth.WithIAMAuth(svcAccountEmail),
    )
    if err != nil {
        return "", fmt.Errorf("unable to initialize GCP auth method: %w", err)
    }

    authInfo, err := client.Auth().Login(context.TODO(), gcpAuth)
    if err != nil {
        return "", fmt.Errorf("unable to login to GCP auth method: %w", err)
    }
    if authInfo == nil {
        return "", fmt.Errorf("login response did not return client token")
    }

    // get secret from the default mount path for KV v2 in dev mode, "secret"
    secret, err := client.KVv2("secret").Get(context.Background(), "creds")
    if err != nil {
        return "", fmt.Errorf("unable to read secret: %w", err)
    }

    // data map can contain more than one key-value pair,
    // in this case we're just grabbing one of them
    value, ok := secret.Data["password"].(string)
    if !ok {
        return "", fmt.Errorf("value type assertion failed: %T %#v", secret.Data["password"], secret.Data["password"])
    }

    return value, nil
}

using System;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using Google.Apis.Auth.OAuth2;
using Google.Apis.Services;
using Google.Apis.Iam.v1;
using Newtonsoft.Json;
using VaultSharp;
using VaultSharp.V1.AuthMethods;
using VaultSharp.V1.AuthMethods.GoogleCloud;
using VaultSharp.V1.Commons;

using Data = Google.Apis.Iam.v1.Data;

namespace Examples
{
    public class GCPAuthExample
    {
        /// <summary>
        /// Fetches a key-value secret (kv-v2) after authenticating to Vault via GCP IAM,
        /// one of two auth methods used to authenticate with GCP (the other is GCE auth).
        /// </summary>
        public string GetSecretGcp()
        {
            var vaultAddr = Environment.GetEnvironmentVariable("VAULT_ADDR");
            if(String.IsNullOrEmpty(vaultAddr))
            {
                throw new System.ArgumentNullException("Vault Address");
            }

            var roleName = Environment.GetEnvironmentVariable("VAULT_ROLE");
            if(String.IsNullOrEmpty(roleName))
            {
                throw new System.ArgumentNullException("Vault Role Name");
            }

            // Learn about authenticating to GCS with service account credentials at https://cloud.google.com/docs/authentication/production
            if(String.IsNullOrEmpty(Environment.GetEnvironmentVariable("GOOGLE_APPLICATION_CREDENTIALS")))
            {
                Console.WriteLine("WARNING: Environment variable GOOGLE_APPLICATION_CREDENTIALS was not set. IAM client for JWT signing will fall back to default instance credentials.");
            }

            var jwt = SignJWT();

            IAuthMethodInfo authMethod = new GoogleCloudAuthMethodInfo(roleName, jwt);
            var vaultClientSettings = new VaultClientSettings(vaultAddr, authMethod);

            IVaultClient vaultClient = new VaultClient(vaultClientSettings);

            // We can retrieve the secret after creating our VaultClient object
            Secret<SecretData> kv2Secret = null;
            kv2Secret = vaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(path: "/creds").Result;

            var password = kv2Secret.Data.Data["password"];

            return password.ToString();
        }

        /// <summary>
        /// Generate signed JWT from GCP IAM
        /// </summary>
        private string SignJWT()
        {
            var roleName = Environment.GetEnvironmentVariable("GCP_ROLE");
            var svcAcctName = Environment.GetEnvironmentVariable("GCP_SERVICE_ACCOUNT_NAME");
            var gcpProjName = Environment.GetEnvironmentVariable("GOOGLE_CLOUD_PROJECT");

            IamService iamService = new IamService(new BaseClientService.Initializer
            {
                HttpClientInitializer = GetCredential(),
                ApplicationName = "Google-iamSample/0.1",
            });

            string svcEmail = $"{svcAcctName}@{gcpProjName}.iam.gserviceaccount.com";
            string name = $"projects/-/serviceAccounts/{svcEmail}";

            TimeSpan currentTime = (DateTime.UtcNow - new DateTime(1970, 1, 1));
            int expiration = (int)(currentTime.TotalSeconds) + 900;

            Data.SignJwtRequest requestBody = new Data.SignJwtRequest();
            requestBody.Payload = JsonConvert.SerializeObject(new Dictionary<string, object> ()
            {
                { "aud", $"vault/{roleName}" } ,
                { "sub", svcEmail } ,
                { "exp", expiration }
            });

            ProjectsResource.ServiceAccountsResource.SignJwtRequest request = iamService.Projects.ServiceAccounts.SignJwt(requestBody, name);

            Data.SignJwtResponse response = request.Execute();

            return JsonConvert.SerializeObject(response.SignedJwt).Replace("\"", "");
        }

        public static GoogleCredential GetCredential()
        {
            GoogleCredential credential = Task.Run(() => GoogleCredential.GetApplicationDefaultAsync()).Result;
            if (credential.IsCreateScopedRequired)
            {
                credential = credential.CreateScoped("https://www.googleapis.com/auth/cloud-platform");
            }
           return credential;
        }
    }
}