HashiConf 2024 Now streaming live from Boston! Attend for free
Vault
Vault Home

S3 storage backend

The S3 storage backend is used to persist Vault's data in an Amazon S3 bucket.

  • No High Availability – the S3 storage backend does not support high availability.

  • Community Supported – the S3 storage backend is supported by the community. While it has undergone review by HashiCorp employees, they may not be as knowledgeable about the technology. If you encounter problems with them, you may be referred to the original author.

storage "s3" {
  access_key = "abcd1234"
  secret_key = "defg5678"
  bucket     = "my-bucket"
}

s3 parameters

  • bucket (string: <required>) – Specifies the name of the S3 bucket. This can also be provided via the environment variable AWS_S3_BUCKET.

  • endpoint (string: "") – Specifies an alternative, AWS compatible, S3 endpoint. This can also be provided via the environment variable AWS_S3_ENDPOINT.

  • region (string "us-east-1") – Specifies the AWS region. This can also be provided via the environment variable AWS_REGION or AWS_DEFAULT_REGION, in that order of preference.

The following settings are used for authenticating to AWS. If you are running your Vault server on an EC2 instance, you can also make use of the EC2 instance profile service to provide the credentials Vault will use to make S3 API calls. Leaving the access_key and secret_key fields empty will cause Vault to attempt to retrieve credentials from the AWS metadata service.

  • access_key – Specifies the AWS access key. This can also be provided via the environment variable AWS_ACCESS_KEY_ID, AWS credential files, or by IAM role.

  • secret_key – Specifies the AWS secret key. This can also be provided via the environment variable AWS_SECRET_ACCESS_KEY, AWS credential files, or by IAM role.

  • session_token (string: "") – Specifies the AWS session token. This can also be provided via the environment variable AWS_SESSION_TOKEN.

  • max_parallel (string: "128") – Specifies the maximum number of concurrent requests to S3.

  • s3_force_path_style (string: "false") - Specifies whether to use host bucket style domains with the configured endpoint.

  • disable_ssl (string: "false") - Specifies if SSL should be used for the endpoint connection (highly recommended not to disable for production).

  • kms_key_id (string: "") - Specifies the ID or Alias of the KMS key used to encrypt data in the S3 backend. Vault must have kms:Encrypt, kms:Decrypt and kms:GenerateDataKey permissions for this KMS key. You can use alias/aws/s3 to specify the default key for the account.

  • path (string: "") - Specifies the path in the S3 Bucket where Vault data will be stored.

s3 examples

Default example

This example shows using Amazon S3 as a storage backend.

storage "s3" {
  access_key = "abcd1234"
  secret_key = "defg5678"
  bucket     = "my-bucket"
}

S3 KMS encryption with default key

This example shows using Amazon S3 as a storage backend using KMS encryption with the default S3 KMS key for the account.

storage "s3" {
  access_key = "abcd1234"
  secret_key = "defg5678"
  bucket     = "my-bucket"
  kms_key_id = "alias/aws/s3"
}

S3 KMS encryption with custom key

This example shows using Amazon S3 as a storage backend using KMS encryption with a customer managed KMS key.

storage "s3" {
  access_key = "abcd1234"
  secret_key = "defg5678"
  bucket     = "my-bucket"
  kms_key_id = "001234ac-72d3-9902-a3fc-0123456789ab"
}

AWS instance metadata timeouts

Affects Vault 1.4 and later

Anytime Vault uses the instance metadata service on an EC2 instance, such as for getting credentials from the instance profile, there may be a delay with the introduction of v2 of the instance metadata service (IMDSv2). The AWS SDK used by Vault first attempts to connect to IMDSv2, and if that times out, it falls back to v1. In Vault 1.4, this timeout can take up to 2 minutes. In Vault 1.5.5 and later, it can take up to 2 seconds with this fix: #10133.

The timeout occurs in situations where there is a proxy between Vault and IMDSv2, and the instance hop limit is set to less than the number of "hops" between Vault and IMDSv2. For example, if Vault is running in docker on an EC2 instance with the instance hop limit set to 1, the AWS SDK client will attempt to connect to IMDSv2, timeout, and fall back to IMDSv1 because of the extra network hop between docker and IMDS.

To avoid the timeout behavior, the hop limit may be adjusted on the underlying EC2 instances. With the docker example, setting the hop limit to 2 will allow the AWS SDK in Vault to connect to IMDSv2 without delay.

Edit this page on GitHub