Kubernetes requirements

We recommend that developers have a deep understanding of Kubernetes before deploying Terraform Enterprise to a production Kubernetes environment.

Example of Kubernetes Architecture

Kubernetes deployments have different operational and observability considerations than traditional deployments, and external service dependencies should be deployed outside the cluster and scale reliably to accommodate Terraform Enterprise workloads.

External services

Terraform Enterprise requires the following external services to install on Kubernetes:

PostgreSQL
Blob Storage (AWS S3, Azure Cloud Storage, Google Cloud Storage, or any S3-compatible storage service)
Redis version 6 or 7 (Redis Cluster is not currently supported)

Runtime

Terraform Enterprise requires the following to deploy in a Kubernetes runtime:

A hostname for Terraform Enterprise
A valid TLS certificate and private key provisioned and matching the hostname selected in pem format
License as the password for Terraform Enterprise FDO container registry: images.releases.hashicorp.com
Install the Helm CLI version 3.0 or above. Learn more about Helm.

Network

Refer to the Network requirements

Configuration

You must create a custom values file (e.g., /tmp/overrides.yaml) to override the default values in the terraform-enterprise helm chart. Refer to Application configuration for a full list of customizable settings.

Example configurations

The below examples for each cloud-platform are based on cloud native hosted PostgreSQL, storage, or Redis cache services. Please customize the values in angle brackets before using these examples for you configuration.

The following is true for all of the below YAML examples:

Values under .env.variables are set as a ConfigMap and mounted as Terraform Enterprise environment variables.
Values under .env.secrets are set as Kubernetes secrets and mounted as Terraform Enterprise environment variables.
Extend the env.configMapRefs[] or env.secretRefs[] with your own resources to add additional ConfigMap or Secret resources within your environment configuration.

Note: In the below examples, any values marked BASE_64_ENCODED* indicates that the value given must be base 64 encoded. If you are using this certificate configuration to host Terraform Enterprise web traffic, this value must be valid with the env.TFE_HOSTNAME, or match the wildcard pattern.

AWS Elastic Kubernetes Service (EKS)

Below is an example configuration for AWS Elastic Kubernetes Services.

replicaCount: <Number of replicas e.g 3>
tls:
  certData: <BASE_64_ENCODED_CERTIFICATE_PEM_FILE>
  keyData: <BASE_64_ENCODED_CERTIFICATE_PRIVATE_KEY_PEM_FILE>
  caCertData: <BASE_64_ENCODED_CERTIFICATE_CA_CERTIFICATE_PEM_FILE>
image:
  repository: images.releases.hashicorp.com
  name: hashicorp/terraform-enterprise
  tag: <vYYYYMM-#>
env:
  variables:
    TFE_HOSTNAME: <TFE hostname>
    TFE_IACT_SUBNETS: <IACT subnet, eg. 10.0.0.0/8,192.168.0.0/24>

    # Database settings.
    TFE_DATABASE_HOST: <Database hostname with port e.g "xxx.us-west-2.rds.amazonaws.com:5432">
    TFE_DATABASE_NAME: <Database name>
    TFE_DATABASE_PARAMETERS: <Database extra params e.g "sslmode=disable">
    TFE_DATABASE_USER: <Database user>

    # Redis settings.
    TFE_REDIS_HOST: <Redis host, eg. 10.101.0.4>
    TFE_REDIS_USE_TLS: <To use tls? eg. "false">
    TFE_REDIS_USE_AUTH: <To use customized credential to authenticate? eg. "true">
    TFE_REDIS_USER: <Redis username>

    # S3 settings. For Server Side Encryption settings, see to the configuration reference.
    TFE_OBJECT_STORAGE_TYPE: s3
    TFE_OBJECT_STORAGE_S3_BUCKET: <S3 bucket name>
    TFE_OBJECT_STORAGE_S3_REGION: <S3 region>
    TFE_OBJECT_STORAGE_S3_USE_INSTANCE_PROFILE: <To use the pod's credentials to authenticate with AWS? e.g. false>
  secrets:
    TFE_DATABASE_PASSWORD: <Database password>
    TFE_OBJECT_STORAGE_S3_ACCESS_KEY_ID: <Required if TFE_OBJECT_STORAGE_S3_USE_INSTANCE_PROFILE is false>
    TFE_OBJECT_STORAGE_S3_SECRET_ACCESS_KEY: <Required if TFE_OBJECT_STORAGE_S3_USE_INSTANCE_PROFILE is false>
    TFE_REDIS_PASSWORD: <Redis password>
    TFE_LICENSE: <Hashicorp license>
    TFE_ENCRYPTION_PASSWORD: <Encryption password>

Google Kubernetes Engine (GKE)

Below is an example configuration for Google Kubernetes Engine.

replicaCount: <Number of replicas e.g 3>
tls:
  certData: <BASE_64_ENCODED_CERTIFICATE_PEM_FILE>
  keyData: <BASE_64_ENCODED_CERTIFICATE_PRIVATE_KEY_PEM_FILE>
  caCertData: <BASE_64_ENCODED_CERTIFICATE_CA_CERTIFICATE_PEM_FILE>
image:
 repository: images.releases.hashicorp.com
 name: hashicorp/terraform-enterprise
 tag: <vYYYYMM-#>
env:
  variables:
    TFE_HOSTNAME: <TFE hostname>
    TFE_IACT_SUBNETS: <IACT subnet, eg. 10.0.0.0/8,192.168.0.0/24>

    # Database settings.
    TFE_DATABASE_HOST: <Database hostname with port e.g 11.22.33.44:5432>
    TFE_DATABASE_NAME: <Database name>
    TFE_DATABASE_PARAMETERS: <Database extra params e.g "sslmode=require">
    TFE_DATABASE_USER: <Database user>

    # Redis settings.
    TFE_REDIS_HOST: <Redis host, eg. 10.101.0.4>
    TFE_REDIS_USE_TLS: <To use tls? eg. "false">
    TFE_REDIS_USE_AUTH: <To use customized credential to authenticate? eg. "true">
    TFE_REDIS_USER: <Redis username>

    # Google Cloud Storage settings.
    TFE_OBJECT_STORAGE_TYPE: google
    TFE_OBJECT_STORAGE_GOOGLE_BUCKET: <Bucket name>
    TFE_OBJECT_STORAGE_GOOGLE_PROJECT: <GCP project ID>
  secrets:
    TFE_DATABASE_PASSWORD: <Database password>
    TFE_OBJECT_STORAGE_GOOGLE_CREDENTIALS: <BASE_64_ENCODED_SERVICE_ACCOUNT_CREDENTIALS>
    TFE_REDIS_PASSWORD: <Redis password>
    TFE_LICENSE: <Hashicorp license>
    TFE_ENCRYPTION_PASSWORD: <Encryption password>

Azure Kubernetes Service (AKS)

Below is an example configuration for Azure Kubernetes Service.

replicaCount: <Number of replicas e.g 3>
tls:
  certData: <BASE_64_ENCODED_CERTIFICATE_PEM_FILE>
  keyData: <BASE_64_ENCODED_CERTIFICATE_PRIVATE_KEY_PEM_FILE>
  caCertData: <BASE_64_ENCODED_CERTIFICATE_CA_CERTIFICATE_PEM_FILE>
image:
 repository: images.releases.hashicorp.com
 name: hashicorp/terraform-enterprise
 tag: <vYYYYMM-#>
env:
  variables:
    TFE_HOSTNAME: <TFE hostname (DNS) e.g. terraform.example.com>
    TFE_IACT_SUBNETS: <IACT subnet, eg. 10.0.0.0/8,192.168.0.0/24>

    # Database settings.
    TFE_DATABASE_HOST: <Database hostname with port e.g "xxx.postgres.database.azure.com:5432">
    TFE_DATABASE_NAME: <Database name>
    TFE_DATABASE_PARAMETERS: <Database extra params e.g "sslmode=disable">
    TFE_DATABASE_USER: <Database user>

    # Redis settings.
    TFE_REDIS_HOST: <Redis host, eg. 10.101.0.4>
    TFE_REDIS_USE_TLS: <To use tls? eg. "false">
    TFE_REDIS_USE_AUTH: <To use customized credential to authenticate? eg. "true">
    TFE_REDIS_USER: <Redis username>

    # Azure container storage settings.
    TFE_OBJECT_STORAGE_TYPE: azure
    TFE_OBJECT_STORAGE_AZURE_ACCOUNT_NAME: <Azure storage account name>
    TFE_OBJECT_STORAGE_AZURE_CONTAINER: <Azure storage container name>
    TFE_OBJECT_STORAGE_AZURE_ENDPOINT: <Azure storage endpoint>
  secrets:
    TFE_DATABASE_PASSWORD: <Database password>
    TFE_OBJECT_STORAGE_AZURE_ACCOUNT_KEY: <Azure storage account key>
    TFE_REDIS_PASSWORD: <Redis password>
    TFE_LICENSE: <Hashicorp license>
    TFE_ENCRYPTION_PASSWORD: <Encryption password>

Below are additional reference materials for setting up these value files:

Terraform Enterprise Helm repository
Tag (release version)
Generic reference for values file to override the default values in the helm chart.

Follow the Kubernetes installation guide to install Terraform Enterprise application using helm.

Security context configuration

Modify the .securityContext helm chart value to set pod security configuration for Terraform Enterprise Flexible Deployment Options. Modify the .container.securityContext helm chart value to set the container security configuration. The allowPrivilegeEscalation container security context option must be omitted or set to true in order for Terraform Enterprise to function properly.