HashiConf 2024 Now streaming live from Boston! Attend for free
Terraform
Terraform Home

You are viewing documentation for version v202309-1. View latest version.

Kubernetes requirements

We recommend that developers have a deep understanding of Kubernetes before deploying Terraform Enterprise to a production Kubernetes environment.

Example of Kubernetes Architecture

Kubernetes deployments have different operational and observability considerations than traditional deployments, and external service dependencies should be deployed outside the cluster and scale reliably to accommodate Terraform Enterprise workloads.

External services

Terraform Enterprise requires the following external services to install on Kubernetes:

  1. PostgreSQL
  2. Blob Storage (AWS S3, Azure Cloud Storage, Google Cloud Storage, or any S3-compatible storage service)
  3. Redis version 6 or 7 (Redis Cluster is not currently supported)

Runtime

Terraform Enterprise requires the following to deploy in a Kubernetes runtime:

  1. A hostname for Terraform Enterprise
  2. A valid TLS certificate and private key provisioned and matching the hostname selected in pem format
  3. License as the password for TFE FDO container registry: images.releases.hashicorp.com
  4. Install the Helm CLI version 3.0 or above. Learn more about Helm.

Network

Refer to the Network requirements

Configuration

You must create a custom values file (e.g., /tmp/overrides.yaml) to override the default values in the terraform-enterprise helm chart. Refer to Application configuration for a full list of customizable settings.

Example configurations

The below examples for each cloud-platform are based on cloud native hosted PostgreSQL, storage, or Redis cache services. Please customize the values in angle brackets before using these examples for you configuration.

The following is true for all of the below YAML examples:

  • Values under .env.variables are set as a ConfigMap and mounted as Terraform Enterprise environment variables.
  • Values under .env.secrets are set as Kubernetes secrets and mounted as Terraform Enterprise environment variables.
  • Extend the env.configMapRefs[] or env.secretRefs[] with your own resources to add additional ConfigMap or Secret resources within your environment configuration.

Note: In the below examples, any values marked BASE_64_ENCODED* indicates that the value given must be base 64 encoded. If you are using this certificate configuration to host Terraform Enterprise web traffic, this value must be valid with the env.TFE_HOSTNAME, or match the wildcard pattern.

AWS Elastic Kubernetes Service (EKS)

Below is an example configuration for AWS Elastic Kubernetes Services.

replicaCount: <Number of replicas e.g 3>
tls:
  certData: <BASE_64_ENCODED_CERTIFICATE_PEM_FILE>
  keyData: <BASE_64_ENCODED_CERTIFICATE_PRIVATE_KEY_PEM_FILE>
  caCertData: <BASE_64_ENCODED_CERTIFICATE_CA_CERTIFICATE_PEM_FILE>
image:
  repository: images.releases.hashicorp.com
  name: hashicorp/terraform-enterprise
  tag: <vYYYYMM-#>  
env:
  variables:
    TFE_HOSTNAME: <TFE hostname>
    TFE_DATABASE_HOST: <Database hostname with port e.g "xxx.us-west-2.rds.amazonaws.com:5432">
    TFE_DATABASE_NAME: <Database name>
    TFE_DATABASE_PARAMETERS: <Database extra params e.g "sslmode=disable">
    TFE_DATABASE_USER: <Database user>
    TFE_REDIS_HOST: <Redis host, eg. 10.101.0.4>
    TFE_REDIS_USE_TLS: <To use tls? eg. "false">
    TFE_REDIS_USE_AUTH: <To use customized credential to authenticate? eg. "true">
    TFE_REDIS_USER: <Redis username>
    TFE_OBJECT_STORAGE_TYPE: s3
    TFE_OBJECT_STORAGE_S3_BUCKET: <s3 bucket name>
    TFE_OBJECT_STORAGE_S3_SERVER_SIDE_ENCRYPTION: aws:kms
    TFE_OBJECT_STORAGE_S3_USE_INSTANCE_PROFILE: "true"
    TFE_OBJECT_STORAGE_S3_ENDPOINT: <s3 endpoint>
    TFE_OBJECT_STORAGE_S3_REGION: <s3 region>
    TFE_IACT_SUBNETS: <IACT subnet, eg. 10.0.0.0/8,192.168.0.0/24>
  secrets:
    TFE_DATABASE_PASSWORD: <Database password>
    TFE_OBJECT_STORAGE_S3_ACCESS_KEY_ID: <s3 access key id>
    TFE_OBJECT_STORAGE_S3_SECRET_ACCESS_KEY: <s3 secret access key>
    TFE_OBJECT_STORAGE_S3_SERVER_SIDE_ENCRYPTION_KMS_KEY_ID: <s3 server side encryption KMS key id>      
    TFE_REDIS_PASSWORD: <Redis password>
    TFE_LICENSE: <HASHICORP_LICENSE>
    TFE_ENCRYPTION_PASSWORD: <Encryption password>
Google Kubernetes Engine (GKE)

Below is an example configuration for Google Kubernetes Engine.

replicaCount: <Number of replicas e.g 3>
tls:
  certData: BASE_64_ENCODED_CERTIFICATE_PEM_FILE
  keyData: BASE_64_ENCODED_CERTIFICATE_PRIVATE_KEY_PEM_FILE
  caCertData: <BASE_64_ENCODED_CERTIFICATE_CA_CERTIFICATE_PEM_FILE>
image:
 repository: images.releases.hashicorp.com
 name: hashicorp/terraform-enterprise
 tag: <vYYYYMM-#>
env:
  variables:
    TFE_HOSTNAME: <TFE hostname>
    TFE_DATABASE_HOST: <Database hostname with port e.g 11.22.33.44:5432>
    TFE_DATABASE_NAME: <Database name>
    TFE_DATABASE_PARAMETERS: <Database extra params e.g "sslmode=require">
    TFE_DATABASE_USER: <Database user>
    TFE_REDIS_HOST: <Redis host, eg. 10.101.0.4>
    TFE_REDIS_USE_TLS: <To use tls? eg. "false">
    TFE_REDIS_USE_AUTH: <To use customized credential to authenticate? eg. "true">
    TFE_REDIS_USER: <Redis username>
    TFE_OBJECT_STORAGE_TYPE: google
    TFE_OBJECT_STORAGE_GOOGLE_BUCKET: <bucket name>
    TFE_OBJECT_STORAGE_GOOGLE_PROJECT: <GCP project ID>
    TFE_IACT_SUBNETS: 10.0.0.0/8,192.168.0.0/24
  secrets:
    TFE_DATABASE_PASSWORD: <database password>
    TFE_OBJECT_STORAGE_GOOGLE_CREDENTIALS: <BASE_64_ENCODED_SERVICE_ACCOUNT_CREDENTIALS>
    TFE_REDIS_PASSWORD: <Redis password>
    TFE_LICENSE: <HASHICORP_LICENSE>
    TFE_ENCRYPTION_PASSWORD: <encryption password>
Azure Kubernetes Service (AKS)

Below is an example configuration for Azure Kubernetes Service.

replicaCount: <Number of replicas e.g 3>
tls:
  certData: BASE_64_ENCODED_CERTIFICATE_PEM_FILE
  keyData: BASE_64_ENCODED_CERTIFICATE_PRIVATE_KEY_PEM_FILE
  caCertData: <BASE_64_ENCODED_CERTIFICATE_CA_CERTIFICATE_PEM_FILE>
image:
 repository: images.releases.hashicorp.com
 name: hashicorp/terraform-enterprise
 tag: <vYYYYMM-#>
env:
  variables:
    TFE_HOSTNAME: <TFE hostname>
    TFE_DATABASE_HOST: <Database hostname with port e.g "xxx.postgres.database.azure.com:5432">
    TFE_DATABASE_NAME: <Database name>
    TFE_DATABASE_PARAMETERS: <Database extra params e.g "sslmode=disable">
    TFE_DATABASE_USER: <Database user>
    TFE_REDIS_HOST: <Redis host, eg. 10.101.0.4>
    TFE_REDIS_USE_TLS: <To use tls? eg. "false">
    TFE_REDIS_USE_AUTH: <To use customized credential to authenticate? eg. "true">
    TFE_REDIS_USER: <Redis username>
    TFE_OBJECT_STORAGE_TYPE: azure
    TFE_OBJECT_STORAGE_AZURE_ACCOUNT_NAME: <Azure storage account name>
    TFE_OBJECT_STORAGE_AZURE_CONTAINER: <Azure storage container name>
    TFE_OBJECT_STORAGE_AZURE_ENDPOINT: <Azure storage endpoint>
    TFE_IACT_SUBNETS: 10.0.0.0/8,192.168.0.0/24
  secrets:
    TFE_DATABASE_PASSWORD: <database password>
    TFE_OBJECT_STORAGE_AZURE_ACCOUNT_KEY: <Azure storage account key>
    TFE_REDIS_PASSWORD: <Redis password>
    TFE_LICENSE: <HASHICORP_LICENSE>
    TFE_ENCRYPTION_PASSWORD: <Encryption password>

Below are additional reference materials for setting up these value files:

Follow the Kubernetes installation guide to install Terraform Enterprise application using helm.